@Ben,
It could be a problem with Twitter’s new Oauth rules that went into effect on August 31st. Alex updated the plugin to work with Oauth but I am hearing a lot of people are having problems with it.
Are you running the latest version of the plugin? Have you set up your Twitter app and api key?
Also to note that php5 is required.