This week saw Facebook experience its most severe outage in over two years, lasting six hours and taking down some of the most highly used sites on the internet. Sites affected by the outage include Facebook, Messenger, Instagram, Whatsapp, and even Oculus Virtual Reality. Employees at Facebook were also unable to enter Facebook, Inc. buildings due to key card denial. The outage had a major impact internationally and resulted in large-scale losses to the company and its founder, Mark Zuckerberg.
What Happened?
The answer to what exactly caused the outage is multifaceted. According to Santosh Janardan, Facebook’s vice president of infrastructure, the outage was caused by an engineering error during routine maintenance.
“This outage was triggered by the system that manages our global backbone network capacity. The backbone is the network Facebook has built to connect all our computing facilities together, which consists of tens of thousands of miles of fiber-optic cables crossing the globe and linking all our data centers,” said Santosh Janardan on behalf of Facebook.
So, what exactly does that mean? Let’s break it down.
Facebook’s outage has been traced back to a withdrawal of the site’s border gateway protocol routes, also referred to as BGP. BGP helps networks determine the best path through which to deliver information from one party to another. If Facebook is a train, BGP helps decide which tracks to follow.
When routine maintenance set off a bug that withdrew Facebook’s BGP, the site’s DNS systems also went offline. DNS is a shorthand for Domain Name System: the phonebook of the internet. DNS translates domain names (for example, www.facebook.com) into IP addresses. This is how websites find and communicate with each other. Following on from the train analogy, DNS would be the stations at which the train stops. For six grueling hours, Facebook was a train with no tracks and no stations: completely lost.
To learn more details about the outage, read Cloudflare’s article.
Outcomes
The consequences of the six-hour outage were enormous. Estimates for just how much Facebook lost during their outage vary widely but range in the tens of millions. Forbes predicted Facebook lost $65 million during their outage period, while Fortune put their estimates at $100 million.
Facebook founder and CEO Mark Zuckerberg’s personal wealth also took a hit of approximately $6 billion. This also resulted in Zuckerberg losing his rank as fifth wealthiest person in the world—he is now sixth after Larry Page.
As Facebook continues its recovery process following its most dramatic server outage in years, the site will have plenty of work to do to get the train back on its tracks.
