How An Actual Cloud Got Into Facebook’s Data Center

Posted Jun 10, 2013

Data centers are known for making it possible for companies to set up a cloud computing experience.  But what happens when an actual cloud is created a data center?  This is what happened to Facebook’s first data center in Prineville, Oregon.

A cloud was formed in the IT room and started raining in the servers.  Facebook hinted that there was a “humidity event” at their first data center in the past.  However, Facebook VP of Infrastructure Engineering Jay Parikh told The Register UK that for a few minutes in the summer of 2011, the cloud was formed and started pouring water on the servers.

“I got a call, ‘Jay, there’s a cloud in the data center’,” Parikh says. “‘What do you mean, outside?’. ‘No, inside’,” said Parikh in the interview.

The cloud formed because the data center has a chiller-less air conditioning system.  Other traditional data centers use electricity-intensive, direct-expansion cooling units to maintain a cooler temperature.  Facebook’s data center uses the outside air to keep the inside cool.

In the first summer that the data center was open, there was a problem with the building’s temperature system.  This led to high temperature and low humidity air from the hot aisles being endlessly recirculated through an evaporative cooling system.  The air came back into the cold aisle for the servers.

“This resulted in cold aisle supply temperature exceeding 80°F and relative humidity exceeding 95%. The Open Compute servers that are deployed within the data center reacted to these extreme changes. Numerous servers were rebooted and few were automatically shut down due to power supply unit failure,” said Facebook in a blog post from back then.  Many of Facebook’s servers broke because it had front-facing power supplies that shorted.  Now Facebook puts a seal around their power supplies to act as a raincoat.