Yen: You Can’t Have Both… At Least Not Yet.

Some of the larger (and very exciting) challenges that we’re bumping up against are how to best present the data that we all will be creating every single day on this new social network.

And, our technical goal (singular) is very simple and straight-forward: We want to make our experience fast as !@#%.

To do this well we need fast and reliable architecture and a lot of this comes down to some very important choices early on. For instance, one of the very classic technical issues that social networks have to work through is what is called “The Bieber Problem”. 

Essentially, in the early days of Twitter, Facebook, and others, if a celebrity like Justin Bieber updated a handful of times in quick succession, it could take down the entire network.

This is because he had millions of followers and each person that was following him was immediately presented with a new data set (e.g. update or post). Do this a few times and you get a massive data issue.

The architecting of feeds, consequently, is of paramount importance (and this is partly why decentralized protocols can’t manage this at scale… at least for the time being).

More specifically, do we display data on “write” or “read“?

If Charlie Lee buys some Litecoin (or, *gasp*… Litecoin Classic… … …) do we present that information instantaneously on all of his followers feeds (on “write”) or do we wait for the user to refresh their feed and then pull the update from the data structures (on “read”)?

Twitter, at least in the beginning, was a “write” type of system and architecture. Facebook, in contrast, has a more algorithmic and weighted feed and does “read” (although both are now pretty heterogeneous). 

If you’re curious about these types of things like “fan out on read” vs “fanout on write” then you can read this seminal piece by Yahoo!:

In any case, these are the things that Peter and I are thinking through every single day as we build and it’s important to note that, for us, it’s more important that we have an amazing user-experience first and foremost before we begin to apply technically-heavy decentralization protocols that could easily reduce a successful encounter with the app.

An aside… I get a good laugh when I hear of other “crypto social networks” tout their decentralized infrastructures because there is no way that they’ve even thought through the user experience vs scaling debate, at least practically-speaking.

There just isn’t a good solution… at least not yet. You compromise. You trade one for the other. Either you wait a few days to see your update post or you see it immediately. I prefer the latter.

– john