In Defence of Polling

There was a recent discussion on Slashdot about the problem of too many RSS aggregators fetching their feeds on the hour every hour and overloading the server. The common accusation levelled at syndication technology is that it depends on polling – the client checks periodically for new content even when there is none. The obvious solution, Slashdotters felt, was to notify the client when the feed changes. They are dead wrong. RSS is a success precisely because it doesn’t work this way.

I think this is a common misconception about syndication, and it should be put to bed. Concerns about polling are narrowly focused on technical efficiency. In the end, the technology has to do something for people – and not just Slashdot people. RSS is a success for the same reason that HTML is a success: both hit the 80/20 point of simplicity. Creating an RSS feed requires nothing more than writing a file far simpler than an HTML web page. There is no special software required on the server – or, heaven forbid, a special protocol1. If syndication had required special software, we wouldn’t be worrying about its efficiency because no-one would be using it.

Many people (Tim Bray, Eric Raymond) have said this before, but it bears repeating: the best technologies are simple, with just enough complexity to do something useful. Their ranks include HTML, XML, HTTP, UNIX pipes, even poor pathetic DOS meets this criterion. Worse is Better. Polling is fine.2


1 Update: I originally said “or, heaven forbid, on the client” – which is of course incorrect.

2 Gabriel’s brilliant Worse is Better argument is in section 2.1