Mapping out my IndieWeb functionality

To keep on top of this process, I have created a permanent page to record everything - Review the notes so far.

This post is part of a series where you can follow along as I redesign the blog and my IndieWeb Server.

Mastr Cntrl is my Indie Web server. It is hosted on Github and Heroku and is open source. It provides the extra layers of functionality and triggers to rebuild my Jekyll website when new content is added, received, or sent to external sources. It’s the biggest and most complicated part of my architecture so this article is a bit more complex than previous posts.

Separation of Concerns

Upon deciding an approach to building IndieWeb functionality there was one thing I wanted to ensure. If I break my IndieWeb server, I don’t want the blog to go down.

My blog is a static website and should function as such, layering the additional functionality over the blog and not depending or expecting it to be there. If I remove the IndieWeb server at a later date the content should remain unaffected. I want to own my data and not become a slave to it. Furthermore tying my data to any service, even of my own creation is something to be avoided at all costs.

Here is what I scribbled in my notebook a year ago.

• If the IndieWeb server stops working the blog should function as normal.
• The IndieWeb server should be plug and play. Modules should be hot swappable.
• Content on the blog should work without any dynamic elements.
• The IndieWeb server is extra not essential.

I managed 3 out of 4. Mastr Cntrl functions are not hot swappable. If I pull a piece of code or functionality there is a high chance it will crash the server. This was a conscious decision when I began building my prototype to try things out and see what sticks. Now my prototype testing is coming to an end it is time to address this.

Architecture

There is much to cover so I am going to split this up in to logical sections to try and make sure I cover all the angles.

I started by attempting to sketch the overall architecture, which took some time, but I eventually arrived at this.

This is an accurate reflection of what I have built so far. It doesn’t look that bad. But there is plenty of opportunity to improve.

I like that I have a single point of entry from Mastr Cntrl to the code source (unless I manually write an article or code). I like that it also has a single point out to Netlify. This keeps tight control over my publishing of content and minimises the complexity in getting content on to the site and publishing it out.

What I don’t like is that Mastr Cntrl is a bit of a black box monolith. “Things” go in and “Something” comes out. That feels like there is a good case to break it down in to separate smaller functions or applications.

I have been mulling over the last 6 months if it would be better to translate most or all of Mastr Cntrl’s functionality in to Netlify/Lambda functions. Looking at the architecture, I think I need to explore this option. I prefer smaller moving parts that have a fixed input and output. I am not a software developer so I favour small, simple modules similar to a MicroServices approach. It would be my hope that by doing this I could reduce the amount of reoccurring issues I keep fixing and simplify my technical overheads. I spotted Max Bock wrote about syndicating using Netlify functions a while back, so I need to read through this again and understand what he did and what I can learn from this.

IndieWeb Use Cases

I wanted to dig in to what is happening in Mastr Cntrl and get an unbiased view of what data I am receiving and translating. I thought about all the Use Cases. The goal here is to map all the inputs and outputs alongside the context. If I am going to design a better blog, first I need to get a view of:

• All the touchpoints human and machine
• The interaction types
• Pain points
• Possible failures

Creating this table helped me to understand what I was doing and the flow on effects I was producing, with the added bonus of verifying my thinking in the above diagram. To help make this clearer I added an Intended Audience row with the view to understand where content should be when I create it. Presently I dump everything in the timeline which isn’t the most elegant user experience.

When I thought through the processes, I began to consider the current issues with the website such as Check-ins dominating the timeline. I thought if I consider what I am doing for me and what is for everyone then when I come to looking at the design later, this list will be really handy to identify all my state types and refresh my thinking over its context. To extrapolate that thought further, I added another row that stated my intent over why I am capturing this information. Again this is me thinking ahead to the design stage for a moment and adding context to what is happening here and why. I am sure it will be useful later.

Hosting on Github, Netlify and Heroku

While I consider the architecture it is worth noting that at currently my content lives across 3 eco-systems. I have no idea on the geography over where this lives except most probably “America”, and I live in New Zealand.

Apart from the problems in maintaining 3 eco-systems there are also running costs associated. The costs are pretty minimal in all honesty but it is a point nonetheless.

Reoccurring Issues

It is worth taking a moment here to also cover reoccurring issues that need specific attention.

• Attaching images to notes via 3rd party Micropub clients yields an unpredictable error. Sometimes it works, sometimes it does not. Mastr Cntrl returns no error and I have been unable to track down the specific trigger. I suspect a race condition or flaw in my logic. I should probably re-write from scratch with better testing and error checking.
• Punctuation is my enemy, especially commas. I have been able to clean up the encoding for most of this but commas in particular are a sticking point. I’m sure someone else has already solved this!
• Sending multiple Webmentions is broken. I should probably fix this.
• Auto sending to IndieNews broke, a minor issue to address.

Code Quality

The quality of the code is variable, I coded quickly to get up and running so I could understand how and what the IndieWeb was and if I wanted to be a part of it. Now I am comfortable in this space it is the time to rewrite to a higher standard, including some testing!

What I learned

Things that feel right

• The static site model on Netlify
• Gihub as my singular code source
• Flow on effects from Mastr Cntrl
• The architecture I built organically

Things that need investigation

• Heroku feels like it could be an unnecessary complication.
• Can Lambda functions replace Mastr Cntrl or key functions?
• Can I reduce 3 eco-systems?

Future Actions

• Explore the possibility of Lambda functions instead of Mastr Cntrl. Is this even feasable?
• Explore the possibility of splitting Mastr Cntrl in to discreet modules with Mastr Cntrl acting as the gatekeeper.
• Add testing coverage to Mastr Cntrl
• Investigate the recurring issues