Friday, May 1, 2009


With the growing popularity of Twitter, I decided to make a Canadian political twitter focussed site. The idea to have a site display only Canadian political tweets proved to be much more complex and troublesome that I had imaged and I spent weeks developing the site.

The site indexes tweets based on a large growing list of political tweeters comprising of MPs, and other notable political tweeters. The site also tracks hash tags and some keywords. It then combines these sources and presents them in many different views. A user can view by category, province, MP only and more. The site started with a federal politics focus, but I quickly expanded to support provincial politics.

A lot of the problems arose from the face twitters servers go down or bog down to really slow speeds very often. And since my site relied on the twitter API to get the tweets, it was a problem. So I had to develop complex caching systems and instead of doing processing on pageload, I have a bunch of background processes on the server constantly running to update the local cache. A visitor loads data from this local cache 90% of the time, so even if twitter.com is down, politwitter will continue to load fast.

I also used a lot of javascript and asynchronous calls on this site, more so that any site I’ve made before.

The site is also an almost complete twitter client, allowing users to post to twitter and do many of the twitter functions directly from politwitter. With many features not on politwitter like posting to twitpic, tinyurl and hash tag suggests.

I also use track many statistics on tweeters, hash tags and political parties. These stats are store in the database, allowing for trends to be created.