Andy Baio and Joshua Schachter Create Greasemonkey Script To Determine Political Bias On Memeorandum Stories
Memeorandum.com is a political news aggregation website that is similar to Techmeme.com. One of the biggest problems with reading political blogs and news is that it is sometimes unknown whether they have a certain political bias. For example, if you go to TheHuffingtonPost.com for the first time, then you won’t know that it has a liberal bias until actually reading a few articles. If you go to Michelle Malkin’s blog, then you won’t realize that it has a conservative bias until after reading a few articles. Andy Baio, founder of Upcoming.org and Joshua Schachter, founder of Del.icio.us have put their brains together to solve this problem using Greasemonkey and Memeorandum.
Check out the screenshot below of how it works. The dark blue represents an extremely liberal bias. The dark red represents an extremely conversative bias. The lighter colors represents more of a moderate bias. Josh and Andy used a recommendation algorithm to score all of the blogs tracked by Memeorandum based on the sites that they linked to in the last 3 months. Then Greasemonkey pulls in the color assignment on anyone’s Firefox browser.
Andy pointed out that the colors don’t represent each blogger’s personal political view, but gives a score reflected upon their linking activity. And there is a tendency for bloggers to link to sites that they feel reflects similar views as them. To install it, click here.
How Did They Do It?
Andy e-mailed Gabe Rivera, the founder of Memeorandum to gather the list of every blog being tracked by the crawler. Josh and Andy then crawled all of these blogs to find out what other sites they linked to in the last 3 months. Since Memeorandum archives all of the old posts on the homepage, Josh and Andy took a snapshot of the Memeorandum homepage that represented every 6 hours in the past 3 months. They basically collected 360 snapshots. Then using a Python script, Josh was able to scrape the links from the HTML of Memeorandum’s past homepage stories.
Using all of the backlink information, Josh and Andy created a matrix of all of the data. Then they created a score of -1 to 1 in a specific column of a Google Spreadsheet. Based on a website’s score, they were given a color. I think this is a great way of making publication companies more transparent of their views.
Put the founders of two companies that were acquired by Yahoo! on the same project and there is no telling what genius of an idea can come out of it. Upcoming.org was acquired by Yahoo! in October 2005. Del.icio.us was acquired by Yahoo! in December 2005.This article was written by Amit Chowdhry. You can follow me at @amitchowdhry or on Google+ at +AmitChowdhry