Pelle Wessman

2016 is in full swing and it’s long overdue to take a look back at the past year and what happened then. This is part two of four in a series of posts about 2015.

The summer

For the remainder of the summer, after I had left Bloglovin, I decided to take a long vacation, or as Chris Messina puts it: a ”funemployment”. Reading books in the sun during the day and coding IndieWeb in the nights.

Lots of interesting IndieWeb experiments therefore happened during the summer, that pushed me closer to being able to have a more complete IndieWeb experience at this site.

Micropub – the missing piece

First and largest of those experiments was my decision to finally tackle the main challenge I have had in participating in the Indieweb: The fact that this blog is a static Jekyll based GitHub Pages site.

The static nature of course means that I can’t do anything dynamic on the server side and with the ethos in the IndieWeb world (and beyond) being that websites should be curlable it has been a challenge to add anything but non-original content such as WebMentions to it.

I therefore decided to look into the feasibility of making a separate service which could accept instructions to add or modify original content on my site and perform the commits to my site that it needed to carry out those instructions.

With Micropub becoming the standard API within the IndieWeb community for such type of instructions I went on to create a bridge between that and the GitHub CRUD API. I created three different Node.js modules through which I got an interface for Micropub requests, a transformation of it into the files I need to commit and a simple module for commiting those to my repository.

By connecting those I got a service with which I could finally accept posts to my blog from tools like OwnYourGram and Quill. I even got support for uploading images and videos that are part of those posts.

My Jekyll theme also got extended so that it could present the now much more rich posts in an appropriate way, through some more complex templating, and I also added two new lists on my site, Links and Social, so any non-blog content had somewhere to go.

I’ve yet to polish the glue that binds the three modules into a service and the Jekyll templates that renders the posts. When I have they will be released as their own projects as well.

iOS – the dogfeeding

With the publishing working I went on to look for useful ways of using it. Since I had been intrigued by the Instapaper annotations flow that Federico of MacStories had described for the Editorial app in some articles I couldn’t resist trying to adapt it to my new Micropub endpoint so that I could start publishing my own annotations in an IndieWeb selfdogfeeding way.

That meant I had to create a Micropub client for the Editorial app, but thanks to the amazing Python integration that Editorial.app has, I pretty soon had a rough version of such a client up an running.

I recorded some screencasts showing the Editorial flows in practise. Here’s one for publishing said Instapaper annotations to my Links section on my blog (sorry for the silent sound):

It uses the three Editorial workflows: Micropub, IndieAuth and optionally, if one wants to re-authenticate, IndieAuth Deauthorize.

I later extended my main flow with more metadata + added a flow for replying to posts from within a browser through a new Workflow.app workflow and Editorial workflow. Here’s an updated screencast showing that in practice:

I also created a Workflow.app workflow for publishing a bookmark (with an optional quote from the selection on a page in a browser), reusing the same IndieAction Editorial workflow as the reply flow did.

So – from at the start of the summer not being able to programmatically publish anything to my blog I could at the end of the summer both publish posts with images and video and publish posts from iOS. I could even reply to posts from within a browser. That’s pretty cool. Happy that I had time to finally overcome the obstacle and imagining that it can open up some new possibilities going forward.

My blog – nearly IndieWeb ready

With Micropub support done my blog now nearly supports all the most important aspects of an IndieWeb:ish site. All that’s missing before I feel that I can say that it is fully IndieWeb ready is automatic WebMention sending. That has proved a bit problematic for static sites as there’s often a delay between publishing something and it being published, as the site has to be built. One therefore has to wait with the Webmention pinging until one knows that the update has actually been published., which requires some special mechanisms. I have some ideas for that as well and some tools have also started to emerge within the IndieWeb community so I hope that in 2016 I will have that little last piece of the puzzle solved.

Webmentions – a status update

Micropub wasn’t the only IndieWeb project I worked on in 2015. I of course also worked on my Webmentions project which for long has been my main IndieWeb project.

In 2015 I released a bunch of new features and tweaks to my Webmention project, such as:

• At the very beginning of the year a new experimental embed, ”cutting-edge”, was introduced to allow for new UI-additions without breaking existing embeds. With the initial release a facepiles was added and all interactions such as like and repost was moved there.
• Basic support for updates as well as some deletes of mentions was finally introduced – fully realtime for some embeds. So now any awkward misspellings etc. can be quickly and swiftly fixed. Full delete support is being tracked in issue #5.
• I extracted the PG Pubsub from the realtime solution and launched it as a separate module so it can help fulfill Postgres Pubsub needs in other projects as well. As part of that I also launched Promised Retry as a generic Promised based wrapper and retry mechanism that ensures that something like a working database connection is always returned to the caller, even if the original connection has been lost and a retry has to be made.
• I also extracted the code responsible for ensuring that all web pages fetched are fetched ”politely” (rate limited) and combined that with an optional robots.txt mechanism extracted from my relspider project and launched that as a combined module called Fetch Politely so that ultimately both my projects as well as those of others can utilize the same basic polite fetching mechanics. I also added a new optional database backend to the module, which I began using in the Webmentions project to ensure that any throttled fetches was persisted and shared beyond the life of a single instance, which in turn made me relax the mechanisms around eventually rejecting source URL:s from a domain that’s flooded with requests.
• As with most of my projects I moved to iojs and then later Node.js 4.x and 5.x to get newer features like arrow functions and native Promises. I still don’t do eg. Babel, especially not server side.
• In the later part of the year yet I added yet another new embed option. This one solved the curlability problem of the original two embeds by moving to a progressively enhanced curlable link (identified by experimental microformat class ”u-responses”). This enabled eg. evenntually supporting Salmentions. As a consequence of rendering more things server side due to this I also decided to move the project to my Tema theme engine to get an improved server side templating solution that’s also shared with my other projects and can eg. enable new themes to be created for the project, which opens up some new possibilities.
• A full Salmentions flow was also added, but not released. It receives comments from downstream, presents them and repings them upstream. As part of this I again stumbled upon the amazingness of Postgres by being able to recursively fetch the comment tree, no matter how deep it was, through the use of ”WITH RECURSIVE”. The current implementation of Salmentions is yet a bit too optimistic to be released. It repings upstream no matter if a mention has changed or not, which means quite some excessive pings and maube even in the worst case scenario an infinite sequence of circular pings. I therefore plan to add some kind of check whether a post has been updated or not before launch. That and other Salmentions issues is being tracked in issue #21.
• As part of the Salmentions feature I also moved the parsing of pages to an extendable fork of the Metadataparser module that I had made at Bloglovin previously in the year. But with added in Microformats parsing for this project of coruse.

All changes in my Webmention project in 2015 can be found in its changelog and one can also subscribe to releases to get notified about changes as they are released.

The future

In 2016 I plan to wrap up my Micropub setup and release all of the code related to it as reusable compontents. I also hope to finalize Salmentions support in my Webmention project and to get my blog to actually start sending Webmentions itself so that the full circle of Reply -> Notification -> Reply starts to work for me.

In the nearest future I will do a presentation on the future of the social web in general and IndieWeb in particular on March 23 at Foo Café here in my hometown of Malmö.

I also hope to be able to attend some of the IndieWebCamps that are coming up, such as the Nuremburg and Düsseldorf ones.

Next part

I’ll soon publish the remaining parts of this serious of four posts about my 2015.