6blog Part of 6bot

October Woe

We've been unable to add new track information for a few days because the BBC have changed the now playing part of the 6 music page that we scrape the data from. Normally, we can deal with this pretty quickly, but they've changed it to a two stage process:

  1. The standard web page loads as normal, but the now playing data can be up to 30 minutes out of date.
  2. The up to date now playing data is then injected (probably by javascript) into the web page afterwards.

The scraping mechanism that we use ignores javascript, so we were left with using data that was out of date, and no way of accurately working out when the track was actually played.

So, we've now moved to a different mechanism entirely - we're scraping the now playing data from the BBC6MusicBot twitter feed, and then blending in any missing data from scraping the web page as before. Hopefully, this will be a more stable way of getting the data.

The change may also have broken how we backfill any missing data, we're just looking into that now. EDIT we've updated one of the backfill routines, so that should kick in and start working properly now.

Easter woe

The BBC appear to have retired their API that 6bot uses to populate the track information in 6bot. The alternative page that we were using as a backup also seems to have been retired too. They might just be down for Easter, but please bear with us whilst we work on a workaround.