Single Points of Failure

As I understand it when DARPA were first looking at what eventually became the Internet back in the early 60s making sure it was resilient was high on the list. It was still the cold war and having bits of the Net taken out by those pesky Russians was not on.

If you look at the way email works, or Usenet, they are examples of how that same concept worked at a higher level. If one email or NNTP server goes down, either temporarily or permanently, all is not lost.

We seem to have lost sight of that in recent times as we've become dependent on single companies to provide services who can then pull the rug out from under our feet if they decide it's no longer in their interest to do so.

The example that triggered me into writing this if Google's announcement yesterday that Google Reader is going offline on 1st July. If you've not come across it before Google Reader is an RSS agregator. I think people liked using it rather than running their own agregator because it meant Google kept track on what you'd read, so you could move from home laptop to mobile phone to work desktop seamlessly. Because it was good and free it drove out of the market a lot of other solutions. Now, having implicitly destroyed those other solutions, Google is stepping out of the market.

There's been something of a shitstorm across the Net about this today. I've seen people complaining on both Twitter¹ and Facebook².

But this isn't the first example. By a long way. Take Twitter. They have a product called TweetDeck which is a Twitter and Facebook agregator. It has both a Web and a mobile app version³ and the mobile version uses v1.0 of Twitter's API to log on. They announced recently⁴ that they're dropping that version of the API and not upgrading TweetDeck for the mobile apps as they intend to concentrate solely on the Web version. So we're now all scratching about looking for alternatives⁵.

This is the second time Twitter have done this recently. I've written here before about Twitter's dropping of Posterous which I used as a secondary blog which means everyone has been scrambling about to move their content off there before it goes offline. There's a grim irony in the fact that Twitter announced it was effectively taking TweetDeck down using Posterous.

You'd think we'd learn. Or perhaps not. Most of us are still treating Facebook as our primary posting forum, indeed for many it's the only place they post content. Either that or Twitter. So other solutions wither on the vine.

So do I have any conclusions? Well some. I've already gone back to posting more here where I control the horizontal and the vertical⁶, I already use identi.ca as a secondary "Twitter" account (also as paulatthehug unsurprisingly).

I've also long since solved the RSS agregator thing across multiple platforms in another way so which I thought I'd share as it's simple and effective. There are two prerequisites:

1. You use IMAP for mail rather than POP3
2. You have access to a Linux box

You then need to download and install feed2imap on your Linux box. It's a ruby script so there's some dependencies but those should be easy to resolve especially as feed2imap is a package on some distribution. Then you tell it which RSS feeds you want to read and set it to run periodically via cron. What it does when it finds new content in any feed is convert the entry into an email and posts it onto your IMAP server in the folder you specify. You can then read the content either directly via your mail client (if it's a full feed like my new one) or click on a link in the email to go to the original post in your web browser.

As the articles are effectively emails on an IMAP server then when you read it on whichever device it's marked read on all your devices.

This seems like a resilient solution to me.

In my config file I have currently have 26 feeds and they're dropped into various sub-folders of a folder of my inbox called "0-rss" (so it appears at the top of my folder list). I poll the feeds every fifteen minutes so I'm always up to date.

It works. Give it a go.

1. Oh the irony.
2. Ditto.
3. There's also an app called TweakDeck which is a sort of better TweetDeck using the same code base.
4. On Posterous, oh the irony #3.
5. I think I will be going for Seesmic but don't quote me on that: I'm still trying out various products on my phone and none are as good as TweakDeck.
6. New RSS feed? Sure, no problem.