A map for every day

Eighteen months ago I wrote about redesigning my site’s front page and mentioned in passing that I’d also created a page for every day which aggregated many things:

I’ve created a page for every single day that pulls together the day’s bits and pieces. It doesn’t aggregate everything I’ve done — yet — but it’s worth it for me just to have a daily archive of my Twitters. Most dates on the site link to the relevant page; here’s an example of one.

I’ve now taken this a step further and added a map for every day which aggregates various pieces of location-based information about me. Here’s an example of that.

Here’s a screenshot for posterity:

Daily map

I realised that I was generating a few bits of location data every day and so, without actively doing anything more, I could keep a rough track of where I’d been. There are three main sources of data, all of which are aggregated on the same maps:

  1. Located tweets. If I tweet from my iPhone, and I’m out and about, I’ll usually have it send the current location. So, seeing as I already grab all my tweets to display in the sidebars of date-based pages on this site, it was easy enough to also grab the latitude and longitude, where present.

  2. Foursquare check-ins. I’ve been using Foursquare for a while, without being entirely sure why. It rarely, if ever, results in serendipitously meeting friends. But I liked the graphs it generates which made me think the data might be useful someday. A while back I started having it send check-ins as tweets to a private Twitter account that I’m using for such automated things. (Thanks to Russell for that idea.) So now I’m grabbing and archiving this Twitter feed and using the Foursquare tweets for map locations.

  3. Flickr photos. Whether I take a photo with my iPhone and use its location, or manually position a photo on the map after uploading to Flickr, all those photos make for a lot of location data. So they’re on the maps too.

All of which means that for most recent days there are one or more points that show where I’ve been. Here are some examples, showing good and problematic things:

  • 14 March 2010, while I was in Austin, TX, for SXSW. A mixture of photos and located tweets, also showing how useless the iPhone’s positioning can be: point 4 should have been from the convention centre, rather than a park I’ve never been to. In case you can’t guess, the numbers indicate the order in which the events occurred.

  • 15 March 2010, from the same trip. Point 5 (and 9) aggregates several tweets that were at the same location — at the moment I assume consecutive tweets/check-ins/photos within 0.00025 of a degree in both directions are the same point. This page also shows some confusion over days and time zones: Points 1 and 2 were actually the previous day in Texas. I’ve tried to ignore time zone differences from all these sources in the hope of preserving my sanity.

  • 18 March 2010. After Austin I went to New York and, although I didn’t record and transmit any location information while there (data roaming charges being expensive, and wi-fi more scarce), the photos I took and subsequently positioned in Flickr provide a nice map of some of the places I went that day. Similarly for days like this in Morocco in May, although some of those positions were guesses a few weeks later.

  • 7 August 2010. A Saturday in London, combining Foursquare check-ins and tweets. Point 4 aggregates a Foursquare check-in and a tweet that were from the same location but had very slightly different lat/long data. The rest of the page features plenty of other stuff I did online that day.

  • 24 August 2010. When most of my London-based weekdays look pretty much the same, a few dots around Shoreditch, a map view makes unusual days stand out. This one shows me going to a few places (as a non-located tweet says in the sidebar, “a pleasant afternoon of flaneuring”) before popping to Heathrow Airport later. That’ll be nice to see and remember in the future.

This idea isn’t to highlight content that’s fascinating to other people — these date pages aren’t even particularly obvious on the site — but are a manifestation of aggregating whatever data I can about myself. Save it now as you don’t know what or when it will be interesting. Given that so many of us are constantly leaving footprints across so many loosely-joined online places I can’t help feeling it should be easier for people to (a) archive and save such data and (b) view it all in one place.

One further thought… I was surprised that although much of this data is available online already — my located tweets and photos are public, although my Foursquare check-ins are only visible to friends on the service — putting it all on daily maps made me feel uneasy. It suddenly felt much more transparent and “stalkerish.” Seeing a single point on a map, for a tweet or a photo, is one thing. Looking through my history and seeing where I’ve been regularly, and recently, feels odd.

As a result of this the site currently doesn’t show maps for the past week. This isn’t a huge change, and it’s not even particularly logical, but it does feel slightly better. As Tom put it, this emphasises the archival nature of the concept, rather than the “where am I right now?” idea. I’ll see how it goes.

Which is a demonstration of why privacy is such a really, really difficult thing to deal with. If I can’t even work out for myself when I’m invading my own privacy, deciding on policies for thousands or millions of very different people using a service is a big problem.

Anyway, there we go: a map for every day.


  • Interesting stuff Phil. Do you have a music data? I'm thinking where you were when you were listening to a specific piece of music.

  • The only music data I have is from Last.fm, which (last time I looked) only gave me charts for a specific week. I show that on the relevant page, but yes, it would be great to have something more specific.

  • user.getRecentTracks (www.last.fm/api/show?s…) has all the times of every track you've ever scrobbled.

    Unfortunately it has no way of providing a start and end date for the tracks you're interested in (unlike user.getArtistTracks - go figure...). The end date is less troublesome obviously because once you have the start date you can just keep looping through the results until you go beyond the end date.

    The best I can think of for the start date is to grab the first results with a limit of 1, see how many tracks there are and just zero in from there.

    Of course, if you're grabbing all the data and stashing it server-side (and looking at how you're generating your twitter list and map, I'm guessing you are) then all that is a non-problem.

  • Ah, interesting, thanks Alex. I'll have to look into that.

  • If you want to make it easier for people to archive their social geodata, any chance you'll release the source for what you're doing so we can do it (easily) too? :)

  • I would love to but this site is an annoying mixture of Movable Type and custom PHP. Not easy to untangle it in any way that would be at all useful to anyone. I'm thinking about how best to release something though.

  • Brilliant.

  • Last.fm have now updated user.getRecentTracks (www.last.fm/api/show?s…) so that you can pass to and from parameters.

    So that should make things a lot easier.

Commenting is disabled on posts once they’re 30 days old.

2 Sep 2010 in Photos

2 Sep 2010 at Twitter

  • 10:00pm: @tobybarnes Unfortunately it's a complicated mix of Movable Type and custom PHP. I am thinking about how to abstract it, somehow...
  • 9:28pm: @tobybarnes Thanks.Yes, I have a local database that stores my tweets and caches photo info.
  • 9:26pm: What a lovely evening. And a lovely day ahead tomorrow, especially if I get up early enough...
  • 5:03pm: Also Ping, "Matt wants to follow you on iTunes Ping" is not very useful. I know more Matts than you know genres. But that's not hard.
  • 5:00pm: @moleitau I've always been surprised that Twitter clients don't make it more/at all obvious which tweets are private.
  • 3:03pm: Ping's a social music recommendation network without the recommendations. Or knowledge of music. Or much sociability.
  • 3:00pm: @smagdali It's exactly the same list of artists for everyone. Which makes no sense whatsoever, but don't take it personally. Very odd.
  • 1:31pm: In more positive Apple news, the Multiple Speakers window in iTunes 10 is better, with individual volume controls per output.
  • 12:54pm: @megp You should draw a vicious cartoon satirising such profligacy.
  • 12:11pm: @danhon My boss is awful. I may have to sleep with him to get some time off. Right, I have to go and buy his lunch now!
  • 10:01am: Really, Ping? You could know more about my listening habits than anyone but Last.fm and you just recommend me artists who paid to be there?
  • 9:51am: The world's biggest, and most useless, pigeonholes: "Rock", "Pop", "Alternative", etc.
  • 9:41am: Apple TV: It doesn't actually show or record TV? It's a thing you buy so you can rent videos? Or am I missing something? #highcourtjudge
  • 7:07am: After a couple of weeks off yoga I was like the Tin Man.

2 Sep 2010 in Links

Music listened to most that week

  1. Emmy the Great (15)
  2. The Weakerthans (14)
  3. Lady Gaga (13)
  4. Psapp (11)
  5. First Words (7)
  6. Earl Hines (5)
  7. Boards of Canada (5)
  8. Isan (4)
  9. Brian Eno (2)
  10. DJ Shadow (2)

More at Last.fm...