Respect for HTTP 304: Positive Feedback Loop?
The FeedLounge crew are considering rewarding feeds that publish feeds that are capable of sending HTTP 304 responses. Here’s how this is a win for most everyone:
- I, as a FeedLounge user, get fresher feed data. This is where the rubber really meets the road!
- Content publishers who send 304 get fresher visits from FeedLounge users. Don’t think that other Web aggregators won’t mimic this idea as they expand, either.
- Aggregators and 304-aware publishers both get huge bandwidth savings from not having to send data back and forth. [See Anne van Kesteren's discussion of 304 for a funny personification of the exchange between aggregator and server.]
- Content publishers who aren’t 304-aware will have positive incentive to get that way. [They already have bandwidth-savings as an incentive, but the blogosphere is notorious for getting secondary effects like this to get CTOs to reconsider problems they're presently ignoring.]
Kudos for the idea, Alex and Scott. I hope Bloglines and the rest of the folks in this space will adopt this approach.
Posted March 23rd, 2006 in FeedLounge.
I saw this on the FL blog this morning, too, and thought it was a brilliant idea. Good use of positive feedback.
March 23rd, 2006 at 2:04 pmFast feed refresh and “conditional GET” support
When a feed has not changed since the last fetch, a server can answer the request with a “304 HTTP code”. Not all servers have this feature enabled, and the result is bandwidth loss.
March 24th, 2006 at 3:27 amHere’s a good idea from Scott (Feedlounge). Server…
However ironically, it seems that http://gfmorris.org/feed/rss2/ is not 304 friendly
March 24th, 2006 at 7:32 pmHoly hell!? Is this a characteristic of RSS2, or have I borked a server setting?
March 24th, 2006 at 7:33 pm[...] After I lauded software that publishes HTTP 304-aware feeds, Scott tells me that IJSM.org’s feeds are no longer 304-aware. UHOH! [...]
March 25th, 2006 at 8:40 pmBloglines crawls every feed every 30 minutes. Why would they ever want to delay that for feeds that don’t support 304s?
March 26th, 2006 at 10:50 pmPaul: It may not ever make sense for Bloglines to do that; if they consider the 30-minute guarantee to be a key component of their service, that’s their choice; there’ll just be a resource penalty for doing so.
Where it might make sense for them, though is in situations where a large feed [say, 30kB] is being loaded every half-hour when the feed averages an update once-a-week. In such a case, you’re looking at 48 polls/day * 7 days/week * 30 kB / pull = 9.84375 MB. A penny here, a penny there …
March 26th, 2006 at 11:05 pmBandwidtth is only getting cheaper, the harder part is writing your crawler and backend infrastructure to handle millions of sites, every 30 minutes
March 26th, 2006 at 11:47 pmWhile it is true that bandwidth is generally getting cheaper, at some point, the laws of commodity pricing seem to indicate that it will bottom out.
March 27th, 2006 at 6:06 amI noticed this independently and found your site via the trac.wordpress.org. I tracked down what I thought was
March 27th, 2006 at 10:11 pmthe problem, and a hack to fix it for now. I don’t think my fix has security implications since the results are only being used for string comparison. Here’s my writeup: http://www.emilsit.net/blog/archives/wordpress-etag-bug/
[...] Update: Someone else noticed and was able to file a bug. Their comments led me to realize it was a more recent change, part of the 2.0.2 upgrade, that probably caused the problem. Where are the regression tests? [...]
April 3rd, 2006 at 12:07 pm