In my last post I wrote up a scheme to share structured information with friends that doesn't require a central service like Facebook or Twitter. If you didn't read that post this post will make very little sense to you. In this post I will explain how losing central control might not mean losing everything you are used to and I will revise a couple of the implementation details.

Some Questions

So, you might be thinking, if there is no central control can we find each other? Can we have a feature similar to Twitter's hashtags? Great questions. The answer is "definitely!" Currently, websites are completely federated and there are many services that allow you to quickly get publically shared information: search engines. The protocol will make it so you mark whatever information you want as public. If you don't want to be discovered you don't have to be. Search companies can write bots that crawl the nodes to gather up public information just like they do with websites today. Private will be the default setting so you will have to explicitly mark content as public. I think this is a huge step forward from centralized services because no middle man ever has to see your private information but we can still have the benefits of the centralized services.

What about backups? What if my hard disk fails? Will I lose all the photos and status updates I've posted over the years? Another great question. Let's add a feature so every post is signed digitally by your private key. In the event that something does happen you can authenticate with your private key to your friends' nodes and request all the personal information that you have shared with them and then verify the integrity of the information that you received. You will definitely want to back up your non-public assets. We could add another feature to make it so you can export encrypted copies of your content and you can back it up however you please.

Implementation Changes

Requiring a VPS or a host that is directly accessible via the internet is probably going to limit who can use a system like this. I'm starting to think that the client should run on your machine and connect with other clients via NAT hole punching. NAT punching is used to share information among peers on P2P networks so it is perfectly suited for this project. There would need to be a service that connects clients in this scenario. Perhaps the lookup could be based on some user UUID or public key signature. This is the point where, if you are paranoid, outsiders could potentially see who is connecting with whom. There would have to be a way for people to connect directly to one another as well so you can avoid the matching services if you are paranoid. I looked at a Google project called libjingle which implements TCP on UDP for telephony. Also, Skype's protocol was partially reverse engineered very recently. Some existing library will make this functionality possible.

Git is starting to look like a pretty bad choice for synchronizing data. Since you have to synchronize all or none of the data in a repository it makes it impossible to share only some of the data with certain peers. I'm going to replace Git with a much simpler protocol that offers more flexibility. Since it knows which friend is making a particular request the system can limit what data is shared with them based on your settings. The synchronization protocol will be very similar to the protocol that CouchDB uses to show what updates have been made to a database. This is what the CouchDB update feed spits out:

{"seq":12,"id":"couchid","changes":[{"rev":"1-beef2479643c2b380f99507a7767f3d5"}]}

Similarly, in the new synchronization protocol after a client authenticates to another client with their key (all clients will run SSH servers) the requesting client would make an HTTP request for changes since their last successful synchronization. The response would be a list of all the ids that have changed or been added which are visible to the peer making the request:

f572d396fae9206628714fb2ce00f72e94f2258f
7269918432597df3ec42b62acd81643d79134cf8
...

I don't want to make too grand a statement about the importance of having a decentralized replacement for services like Facebook and Twitter. I will say that I think email would have failed spectacularly if it had been centralized instead of federated and my guess is that it will be better for everyone except the investors and owners of the centralized social networks if we move to more secure distributed systems.