Federation: The future of open online services, and the war against it

To clarify, by “federation”, I mean federation in contrast to a client-server or P2P models. Specifically, a collection of independent servers, each serving a number of users, all using a standardized, federated protocol.

Let’s go over the basics real quick. At first, there was the client-server model.

client-server-modelThere is a server that communicates directly with a bunch of clients. For instance, when you open up Facebook, you’re using the client-server model (yes, Facebook has multiple servers, but they’re all owned and controlled by Facebook, so in this context, it’s essentially a single server).

Then, around the time of Napster, we realized that it might be a good idea to take servers out of the equation. This introduced the peer-to-peer model.

peer-to-peer-modelThis distributed model offered us decentralization — a network that can’t be destroyed by removing a single server out of the middle. It’s a network that’s a collection of peers with no central authority. Protocols like BitTorrent use the P2P model. However, there is a downside to the P2P model: each of the peers have a fairly high processing cost, and are usually expected to be constantly connected to the network.

So what’s the in-between? Federation: a collection of independent servers, each serving a number of clients.

federation-model“Independent” is the important word here: the idea is that anyone can host their own server, and can join it to the network of servers by using an agreed-upon, or “federated”, protocol. This allows us to have an open network (unlike, say, Facebook’s servers) while not burdening the clients with all the processing. An excellent example of the federation model is email: an email server can be run by your ISP, your company, an online ad-supported service, or you can run one yourself. Multiple clients connect to each server (ie all of your ISP’s customers), and the servers can talk to each other via an established protocol (SMTP). There is no central authority in the email system: your little home server has, by design, the same “say” in the network as Gmail’s servers.

The federated model, while being old tech, is still the best compromise between client-server and P2P models. It enforces an open network, gives us the option to completely own our data, while still leaving room for our corporate peers. However, there’s been an increasing trend away from federated models by several large service providers.

XMPP is an instant messaging protocol (it’s actually a lot more than just an IM protocol, but that’s not important here) which uses the federated model. Users connect to the server for their domain, and they can chat with users on different domains via server-to-server communication. Google Talk (aka Hangouts) implemented XMPP support in 2006. The idea was that a user on Google Talk, say, “user1@gmail.com” could chat with a remote user, like “user2@jabber.com” without user2 having to create a Gmail account. This was a Good Thing, because it let users have choice of IM providers, while still letting users on different networks chat with each other. In 2010, Facebook Chat added XMPP support. This was also a good thing, for an additional 400 million accounts could be reached via XMPP. It looked like XMPP was going to get as popular as email.

But then… it all fell apart. Both Google and Facebook dropped XMPP support in late 2014 / early 2015. There was never much of an explanation from either corporation, just something along the lines of “We’re switching to X new API and we didn’t bother adding XMPP support” and “We promise we might eventually one day look at maybe adding something resembling XMPP support. Maybe.”


So what actually happened? They realized the business value of vendor lock-in. Effectively, “we know that user1 wants to chat with user2. Why make it easy for user2 to chat externally? We can just force them to join to chat with user1, giving us more product a new client!” You want to chat with grandma over Facebook Chat? Too bad, you’ll have to make an FB account now… even though perfectly good tech exists to let you chat with her from wherever.

But that’s just XMPP. We have slightly bigger issues to worry about.

It’s becoming increasingly more difficult to run your own email server. If you set up a new email server on a dedi/VPS somewhere, and follow all the usual recommended practices (PTR, SPF records, DKIM), your emails will be put in the Spam bin on both Gmail and Microsoft inboxes. Gmail will direct you to read their Bulk Sender Guidelines (even if you only sent a single email), and Microsoft will give you a place to “register” your server for a chance to avoid the spam bin. In order to avoid the spam bin on Gmail, however, you’ll need to build up a “reputation”, by having conversations with numerous Gmail accounts, and having them mark you as “Not Spam”. Here are some HN threads griping about this issue. To make matters worse, having some reputation with Gmail doesn’t guarantee that your email will get delivered: the Gmail servers will accept your email, but they may still end up in the user’s spam bin or disappear completely, especially if this is the first time you’ve emailed this particular user.

This is a ridiculous amount of effort for a user who just wants to run their own email server because, oh I dunno, maybe they want to actually own their emails? This is also extremely concerning, because email is the original example of an open and decentralized system. I suspect that, within the next few years, it’ll gradually become impossible for anyone to run a small email server. Eventually, you’ll use your work email to talk to your work colleageues, your Gmail to talk to your friends on Gmail, and your Outlook to talk to your friends on Outlook. Here’s vendor lock-in again — why let you run your own server when you can maintain three accounts (and, of course, see ads from all three providers).

Ad-blocking: The new “stealing”

Since the dawn of filesharing, our corporate overlords have been shouting about how media piracy is “stealing“. Thankfully, the idea of making a digital copy being equivalent to theft has been beaten down by countless arguments and dissertations, to the point where courts have, in certain cases, prohibited copyright holders from using the words “piracy”, “theft”, and “stealing” in jury trials. However, there is a new group of people complaining about “stealing” — websites that earn their revenue through ads.

nobody pirates games anymore though so lol

Blocking ads is a little different than piracy. Viewing a web page actively consumes resources: bandwidth, which the site pays for. This amount is fairly negligible, however. Consider that my $30/month Hetzner server is allotted 20TB of upload, and that the homepage of this blog is roughly a megabyte, each page load costs me about $0.0000015 . Not really an amount worth crying about.

The anti-ad-blockers’ primary argument is usually something like, “We are content creators! Ads are the only way we make money! If you block our ads, the internet will never have any new content ever again!” Let’s pretend for a second that 75% of the “creative content” blogs and websites aren’t just regurgitated bullshit served for the sole purpose of getting ad views. Bad news: I’ve seen better content generated by the internet hate machine and even in Reddit comments than any ad-supported blog. We’re talking about unpaid, pseudo-anonymous users of these communities creating better content than anyone paid to do so.

But LG, those communities wouldn’t exist without ads!
I can run a Reddit clone on a $2 VPS. Good creative content spreads between all of the communities. In other words, the content would’ve still been created even if these communities didn’t exist, or were more fragmented.

So why block ads? It’s fairly simple: they’re annoying. Ads have evolved from being simple text ads, to flashing and jumping, to literally screaming the name of a product in a background window. Fuck you and your content, I don’t want to put up with that shit. If you included a few affiliate links or something in your blog posts, I wouldn’t mind, but because of the behaviour of the ad industry, I have no choice but to go full nuclear and block everything resembling an ad.

even billy blocks ads, despite his profession

The idea of “acceptable advertisements” has been a topic of debate recently, but I’m not interested. I simply cannot trust a company to accept money to decide which ads to show me: it’s basically bribery. Nor am I going to trust a some democratic process: Reddit is a prime example of democracy failing. No, fuck advertisements alltogether, you people will have to find better ways to fund your activities.

So let’s talk alternatives. “Donation” mechanisms have been around approximately forever, and are the sole source of revenue for all sorts of sites (private trackers are a notable example). A somewhat more complicated idea is Flattr: basically, users pay a monthly donation of their choosing that’s automatically distributed between the sites they clicked a “flattr” button on. But that’s the essential idea: users contributing directly to your site, rather than some evil network in the middle. I bet if I contributed a single penny to every website I’ve ever visited, I would’ve given 90% of those websites more revenue than they ever earned by showing me ads.

pls gib mony

Anyways, back to the original point: is blocking ads “stealing”? Kinda, like copyright infringement is kinda stealing, but it doesn’t matter. In short, advertising is basically getting #rekt by ad blockers, there’s going to be an all out war soon, and advertisers will lose. If you run a site supported by ads, now’s the time to get out and find a donation-based alternative. And if you can’t sustain your site on donations… I’m sorry, but your site is probably clickbait shit that nobody cares about, and the net would be better off if you just left.