Centralization Strikes Again - July 2021 Edition

“We demonstrated that the Web had failed instead of served humanity, as it was supposed to have done, and failed in many places. The increasing centralization of the Web has ended up producing—with no deliberate action of the people who designed the platform—a large-scale emergent phenomenon which is anti-human.”—Tim Berners-Lee


If any newsworthy bulletins about centralization have been missed this month, please reach out via email or Twitter - we will get it added! There’s just too much to keep up with, and we’d deeply appreciate your contributions (credits will be attributed!) as we continue to track the failures of centralized models.


Why the Web 3.0 stack needs real-time data

It’s hard to say something original when it comes to criticizing the current state of the centralized web. What seems more important is to get started on building a better, decentralized internet. A new internet where, on the one hand, the power to pull a kill switch is never in the hands of just a few, but also a new internet infrastructure in which one cannot easily get DDoS’d, similar to how it is in the current, rather centralized set-up.

The threats of centralization will only grow over time as more people and companies become more dependent on being online. This problem can only be solved in small pieces.

One of them being real-time data streaming infrastructure. 

When it comes to static data, the decentralized web might not have quite reached maturity yet, but we’ve already come a long way since the early peer-to-peer file sharing days, the Bitcoin whitepaper, and even Ethereum, which has now been around for more than half a decade. This is unlike real-time data which is really just coming into the forefront and on the horizon.

To understand why the decentralized web needs real-time data, we should first understand what real-time data is because not all data is created equally. Most of today’s data eventually ends up in a centralized database where, by now, decentralized infrastructures like IPFS or Skynet provide great alternatives to Google Cloud or AWS. But often, data needs to first travel from point A to point B in order to end up in some database. More than that, there can be many point “Bs”, i.e. the data needs to be broadcasted to many destinations for consumption.

Databases can be thought to serve real-time data in the sense that when you query the database, you get the latest result. But whenever the data changes, you will not be notified about it. By contrast in streaming data, you receive all data in real-time whenever there is new data or any changes to the database.

That’s where real-time data and streaming data transport enter the conversation. Some examples that call for real-time data in action are: push notifications, online gaming, an air probe taken by an IoT device, or oracle price feeds. For that data to be processed, sold, or consumed, it needs to be pushed from publisher to (any number of) subscribers as soon as the data is created or changed.

Unlike blockchains, forming global consensus is not needed for data transport, allowing for performance on par with centralized services – and scalability well beyond! In part, the Bitcoin blockchain is secure because it is so slow. Blockchains can be used to replace trust or a state’s authority. In the case of real-time data transport, we’re looking at replacing centralized services and infrastructure such as AWS or Google Cloud (which can be compared to nation states), and therefore, the speed a decentralized service offers should be the same as the centralized one. Otherwise, politics aside, there’ll never be an incentive to give the decentralized web a try. 

Luckily, several projects have been working over the past few years on solving the issue of real-time data transport in a decentralized fashion. Projects such as Ceramic, Matrix, NKN and Streamr (the project I co-founded) all offer solutions to a new form of data transportation without intermediaries. 

In a centralized system, data travels from publisher through a central server to subscribers, whereas the Streamr network is composed of several nodes which relay streams peer-to-peer. Each stream or pub-sub topic has its own peer-to-peer overlay network that is built and maintained by a set of BitTorrent-like trackers. 

Applications that publish or subscribe to data streams interact with the network via Broker nodes. Apps interface with nodes using simple standard protocols like MQTT, HTTP, and websockets, making integration easy from any programming language. The node also runs natively inside JS-based applications.

The network itself is run by Broker node operators. This summer, Streamr will be launching a series of testnets to onboard the first node operators. During these testnets, we want to trial the Network’s capabilities, before opening up the Brubeck milestone mainnet to our community and beyond. If you’re interested, you can sign up here.

Last year, we ran a series of testnets in which we researched the scaling of the network and found that its speed and latency is on par with centralized solutions. Which is a good first step away from today’s centralized internet infrastructure.

After a hopefully successful testnet launch, we’ll continue with the Brubeck mainnet, which can then be used for a myriad of cases. You can think of the Streamr Network as a middleware solution to be leveraged by any application, blockchain network, or distributed protocol. For instance, Nfty DM uses the Streamr Network to add a decentralized chat to the NFT space, where collectors can message creators and vice versa.

Think about what happens when all instant messaging moves to Web 3.0. In such a scenario, we can entirely re-think how our personal data is handled, stored and monetized, whilst playing with the various lego pieces the decentralized stack offers us already, like DeFi. I’m quite excited about what’s to come once the potential of real-time data has caught fire in the Web 3.0 space.

Thank you to Henri Pihkala and Marlene Ronstedt (from Streamr) for their contributions to this month’s edition of our newsletter.

If you would like to contribute to the Op-Ed in next month's edition of Centralization Strikes Again, then please email tim.ho@chainsafe.io. We would love to hear your thoughts on how we can leverage decentralized technologies as a solution to the many problems centralized systems present for humanity!

Like all great things, our newsletter starts humble and, well... unpolished. If you have any feedback, please don't hesitate to send them my way.

Learn more about ChainSafe at https://chainsafe.io/, through our Medium, via Twitter, or by visiting the ChainSafe GitHub.

❤️ Thank you for reading! ❤️