Today I spent more time--by several orders of magnitude--reading LiveJournal entries than I have in the last five years put together. And all this without actually even going to the LiveJournal site. I've been working on a Ruby module to parse Six Apart's update stream service for use in an upcoming site feature, and spent a good chunk of the day watching as entries scrolled across the terminal.
I was a bit worried about how I was going to parse the never ending stream of RSS entries. Most examples of XML parsing assume a (relatively short) document that is parsed into a DOM tree. This clearly isn't an option. Looking through the REXML source files I found a couple attempts at stream parsers. A search on Google found an article demonstrating the use of REXML::Parsers::StreamParser.
The StreamParser example looked good, but it still suffered from the problem that it assumed you would be passing in a string or a file. What I had, however, was a socket. I didn't think it would work, but I just handed the parser my socket and it happily started firing events. With a little more fussing around with code, I was able to set up a simple state machine to decipher the data. Now all I have to do is figure out what to do with it....
