Facebook and the BGP: How the Network Disappeared from the Internet

Enlarge image

Facebook logo: Technically disconnected from the rest of the internet for a few hours

Photo: Dominic Lipinski / DPA

Politically, Facebook is currently under pressure: The whistleblower Frances Haugen has provided the US Congress with numerous internal company documents and will testify in the Capitol this Tuesday.

According to her personal assessment, the company is well aware of many of the societal problems that its services such as Facebook and Instagram are causing worldwide and could do more about them.

For many, the assumption was therefore that the several hours of failure of Facebook, WhatsApp and Instagram on Monday was related to revelations about those internal documents or the political discussion about them. But it was probably just a mundane technical problem that led to the failure. The cause of the failure was apparently a misconfiguration in a system that connects the Internet at the top level.

The so-called Border Gateway Protocol, or BGP for short, ensures that data can find its way on the Internet.

The Internet is ultimately made up of many smaller networks that are connected to one another via BGP.

Big players like Google or Facebook have their own networks and are directly connected to BGP.

There they have to announce routes on a regular basis.

These are, so to speak, technical directions that tell the other BGP participants how they can reach a certain network.

This is exactly where the problem arose on Monday: The Facebook network had disappeared from the BGP network, as the company Cloudflare explained in a blog post.

There were no longer any technical directions telling the other nodes in the BGP network how to connect to Facebook.

Facebook wasn't connected to the rest of the internet

The reason for this was probably a mistake by Facebook itself. The company announced that an incorrect configuration change had been carried out on its own router systems, which led to this effect.

Facebook writes that there is no evidence that personal data has been compromised.

In this case, that's quite believable: technically, Facebook was simply not connected to the rest of the internet.

The Facebook failure also led to indirect effects.

Countless computer systems and smartphones tried again and again to access Facebook after incorrect connections.

On so-called name servers, which are responsible for translating domain names such as Facebook.com into Internet addresses, this led to significantly more inquiries and thus a higher utilization.

Facebook also struggled with some follow-up problems internally.

The entire internal communication and work structure of Facebook uses systems connected to the Internet that were apparently inaccessible.

The system is fragile and has security problems

The BGP system is an important element of the infrastructure that holds the Internet together.

But it is known to be fragile and has systemic security problems.

In the past it happened time and again that data traffic was forwarded to the wrong address in the BGP network.

Sometimes something like this is simply a mistake, sometimes data espionage.

In 2019, a science experiment caused widespread BGP failures, especially in Asia and Australia.

An international team of scientists wanted to see if they could introduce a new role in BGP.

However, a bug in software that was used on many BGP nodes led to system crashes on the relevant servers.

The experiment was then terminated.

Google data shows that very few users know about the BGP system, although it essentially holds the Internet together.

In any case, the number of searches for his abbreviation suddenly rose sharply on Monday around 6 p.m.

For Facebook, the failure of its services should ultimately only have short-term consequences.

For the Internet, however, the question arises of how to deal with the centralization of infrastructure and on what shaky basis the core infrastructure of the network is in part.