Facebook has announced on their blog today that live commenting is now available on the walls and news feeds for every user. That means you don’t have to refresh the page to see what people have written. Live commenting began as a Facebook Hackathon project where employees are essentially “locked in” to come up with new and interesting ideas.
Every minute Facebook serves over 100 million pieces of content that tends to receive a good amount of comments. Users submit about 650,000 comments every minute.
Initially Facebook was considering handling the load by using a polling system. The polling system would periodically send a request to check whether new comments have arrived or not. This was not “real-time” enough since it would still take 5 seconds or less to update the articles, status messages, etc.
Facebook decided to use a push-based approach instead. “Write locally, read globally” is the idea behind their push-based approach. Below is an excerpt from the Facebook blog about how the new system works:
Because of our unique situation, we settled on the completely opposite approach: “write locally, read globally.” This meant deploying distributed storage tiers that only handled writes locally, then less frequently collecting information from across all of our data centers to produce the final result. For example, when a user loads his News Feed through a request to our data center in Virginia, the system writes to a storage tier in the same data center, recording the fact that the user is now viewing certain pieces of content so that we can push them new comments. When someone enters a comment, we fetch the viewership information from all of our data centers across the country, combine the information, then push the updates out. In practice, this means we have to perform multiple cross-country reads for every comment produced. But it works because our commenting rate is significantly lower than our viewing rate. Reading globally saves us from having to replicate a high volume of writes across data centers, saving expensive, long-distance bandwidth.