Phil Gyford

Writing

Friday 31 December 2004

PreviousIndexNext Combining RSS feeds into one feed and page

I’ve finally got round to finishing my script for combinging all my various fragments of web output into one beautiful stream. At Gyford.com I have a weblog of my writing, which this entry is part of, and a weblog of notes. I also have my links at del.icio.us which I now automatically aggregate daily and post to a local weblog. Finally, I have all the photos I post at Flickr.

I didn’t like having a different RSS feed for each source, and separating them all out on my home page — it all seemed unnecessarily clunky and too much work for the few people who want to see what I’ve been doing recently. So I wrote a script that grabs four feeds and combines them into one, also creating a combined chronological front page for the site.

The trickiest bit was pulling all the Flickr photos from one day into a single, compact, daily item but making sure I didn’t flood the front page or the feed if I post post dozens of dull photos in a single day. Instead only the most recent are displayed, with a link to see the rest. Deciding how much information to display generally is a tricky balancing act: I chose to show the photo descriptions in the feed, but not on the page; and the feed gets the full text of my writing and notes while the page just gets the first paragraph.

Anyway, it’s not quite beautiful (I’d dearly love to redesign my entire site) but it does the job. I’ve put up a version of the perl script I’ve written in case it’s of use to anyone else (or in case anyone can tell me anything I’ve done really badly).

If you really find this interesting, you might also like to read Ben Hammersley’s post from yesterday about posting entries from various feeds into a single Movable Type daily entry. And, of course, Feedburner can do all the combining of Flickr photos and del.icio.us links into your feed for you if you don’t want to get your hands too dirty.

Comments

There is also Reblog, a project we've been working on with the fine folks at Eyebeam. It wears a lot of hats, one of them being a feed aggregator and republisher. Check it out: http://reblog.org and http://eyebeam.org/reblog

Posted by Michal Migurski on 9 January 2005, 8:27 pm | Link

I've been using rawdog http://offog.org/code/rawdog.html to do the same thing on my homepage.

Posted by J. Lasser on 16 January 2005, 8:08 pm | Link

I recently had the same problem and created a tool for this myself: http://feedjumbler.com/

Posted by Marcel Marchon on 27 April 2005, 10:16 pm | Link
Some sites linking to this entry (Trackbacks)

Experiments in Aggregation
I've been describing OrthodoxBlogs.com as "an experiment in aggregation." Right now, this site is based on Drupal and uses its default aggregator to manage feeds. Beta #1 is based on Gregarius and Beta #2 uses Lilina. We've also tried to implement reFeed
At 'orthodoxblogs.com' on Monday 15 August 2005, 9:04 PM

Blogs are not the only fruit
Blogs may have been a word of the year for 2004, but a wider variety of social software tools and group structures will start to gain widespread adoption in 2005, which will present both challenges and opportunities for those of us who are implemen...
At 'Headshift' on Thursday 6 January 2005, 4:22 PM