FFFFOUND! export script

FFFFOUND! is shutting down on 8th May. I haven’t used it for ages but I saved over 500 images in there a few years ago. I couldn’t find a backup tool that worked for me so I adapted one to make my own.

Andy Baio wrote a good post about how poorly the shutdown of FFFFOUND! has been handled, so let’s take that as read.

Screenshot of the webpage

FFFFOUND! has no API so getting images off it involves scraping the site’s pages. I tried a couple of scripts, like the three-year-old ffffexport which stopped abruptly after a few pages.

I looked at some other old scripts, some of which seemed unfinished and others only fetched the images. Only fetching the images doesn’t seem like enough — I want to know where they were originally, when they were saved, that kind of thing. Metadata!

This script by Aaron Scott Hildebrandt mostly worked for me but only fetched the images. So I started adapting that to do more. I also wanted:

  • To make HTML pages, a bit like those on FFFFOUND! itself, for browsing the images, including the information about the images.

  • All the data saved in a machine-readable format. I thought I might want to write code to upload the images to Pinboard (or wherever) at some point, and this would save having to scrape my new HTML pages all over again.

Screenshot of the webpage

This was just going to be a really quick hack but I’ve ended up spending a whole Saturday on it. The script seems to works now. I’ve downloaded a couple of entire archives successfully. It creates pages that look like this second screenshot, plus a single JSON file including the URLs, page titles, and the local filenames of all the images.

It’s been lovely to look through it all again. I have no memory of most of these images, from only a few years ago. It’s like finding an old shoebox of photos and cuttings.

Getting this to work I spent a lot of time struggling with character encoding and it’s not perfect but enough was enough. It would probably have been quicker to start from scratch. But that’s the wisdom-of-having-done-the-work talking. Hopefully this vaguely embarrassing code will be useful to someone else.

ffffound-export on GitHub

In Projects on 22 April 2017. Permalink

Add a Google map to Django admin to get latitude and longitude

While writing a Django app that contains a model with latitude and longitude fields I wanted to use a map in the Django Admin to set those values. I couldn’t find anything quite like what I wanted so, inevitably, I wrote my own.

There are alternatives, like:

  • django-geoposition. I haven’t tried it but this module provides a new field type (GeopositionField) which has a map in the admin. This sounds like the simplest solution unless you love extra work, like I do.

  • GeoDjango has maps in the admin, using OpenLayers maps which can be replaced, e.g., with django-map-widgets. In my app the locations aren’t very important and GeoDjango seemed like overkill.

The below is all from this app on GitHub in case you prefer to find the code there; I’ll link to the specific files below. NOTE: The example on GitHub also uses Google Geocoder to make an address string (like “Ripley, Derbyshire, England”) and add that and a country code (“GB”, “US”, etc.) to additional fields on the model.


1. Make or update your model

For our example we have a basic model called Venue, in models.py (on GitHub):

from django.db import models

class Venue(models.Model):

    name = models.CharField(max_length=255)

    latitude = models.DecimalField(
                max_digits=9, decimal_places=6, null=True, blank=True)

    longitude = models.DecimalField(
                max_digits=9, decimal_places=6, null=True, blank=True)

2. Get an API key

I’m using Google Maps because it’s relatively easy.

Get a Google Maps Javascript API key.

Put it in your Django project’s settings.py file like this:

GOOGLE_MAPS_API_KEY = 'your-key-goes-here'

3. Add the admin CSS

In your app’s static files add this CSS file. Ours is at appname/static/css/admin/location_picker.css (on GitHub).

#setloc-map {
    max-width: 100%;
    height: 400px;
    margin-top: 1em;
    border: 1px solid #eee;

4. Add the admin JavaScript

There are some variables near the top of this that you might want to change:

  • prev_el_selector — By default our map will be positioned below the input for our model’s longitude field. If you want it elsewhere, or you’ve changed the name of your longitude field, you’ll want to change this.

  • lat_input_selector and lon_input_selector — If your model’s latitude and longitude fields are called something other than latitude and longitude, you wierdo, you’ll have to change the values of these variables to reflect those.

  • initial_lat and initial_lon — The default centres the map in London, if the model has no location set. You’ll want to change these values if you live elsewhere or it’ll get really annoying.

  • initial_zoom and initial_with_loc_zoom — These are the map zoom levels we start with if the model has no location, or if it has a location, respectively. Feel free to tweak.

We put this file at appname/static/js/admin/location_picker.js (on GitHub, with additional fields and geocoding).

 * Create a map with a marker.
 * Creating or dragging the marker sets the latitude and longitude values
 * in the input fields.
;(function($) {

  // We'll insert the map after this element:
  var prev_el_selector = '.form-row.field-longitude';

  // The input elements we'll put lat/lon into and use
  // to set the map's initial lat/lon.
  var lat_input_selector = '#id_latitude',
      lon_input_selector = '#id_longitude';

  // If we don't have a lat/lon in the input fields,
  // this is where the map will be centered initially.
  var initial_lat = 51.516448,
      initial_lon = -0.130463;

  // Initial zoom level for the map.
  var initial_zoom = 6;

  // Initial zoom level if input fields have a location.
  var initial_with_loc_zoom = 12;

  // Global variables. Nice.
  var map, marker, $lat, $lon;

   * Create HTML elements, display map, set up event listenerss.
  function initMap() {
    $prevEl = $(prev_el_selector);
    if ($prevEl.length == 0) {
      // Can't find where to put the map.

    $lat = $(lat_input_selector);
    $lon = $(lon_input_selector);

    var has_initial_loc = ($lat.val() && $lon.val());

    if (has_initial_loc) {
      // There is lat/lon in the fields, so centre the map on that.
      initial_lat = parseFloat($lat.val());
      initial_lon = parseFloat($lon.val());
      initial_zoom = initial_with_loc_zoom;

    $prevEl.after( $('<div id="setloc-map"></div>') );

    map = new google.maps.Map(document.getElementById('setloc-map'), {
      zoom: initial_zoom,
      center: {lat: initial_lat, lng: initial_lon}

    // Create but don't position the marker:
    marker = new google.maps.Marker({
      map: map,
      draggable: true,

    if (has_initial_loc) {
      // There is lat/lon in the fields, so centre the marker on that.
      setMarkerPosition(initial_lat, initial_lon);

    google.maps.event.addListener(map, 'click', function(ev) {
      setMarkerPosition(ev.latLng.lat(), ev.latLng.lng());

    google.maps.event.addListener(marker, 'dragend', function() {
      setInputValues(marker.getPosition().lat(), marker.getPosition().lng());

   * Re-position marker and set input values.
  function setMarkerPosition(lat, lon) {
    marker.setPosition({lat: lat, lng: lon});
    setInputValues(lat, lon);

   * Set both lat and lon input values.
  function setInputValues(lat, lon) {
    setInputValue($lat, lat);
    setInputValue($lon, lon);

   * Set the value of $input to val, with the correct decimal places.
   * We work out decimal places using the <input>'s step value, if any.
  function setInputValue($input, val) {
    // step should be like "0.000001".
    var step = $input.prop('step');
    var dec_places = 0;

    if (step) {
      if (step.split('.').length == 2) {
        dec_places = step.split('.')[1].length;

      val = val.toFixed(dec_places);




5. Load the CSS and JavaScript

In your model’s admin you’ll need to include the CSS and JavaScript. So, in our app’s admin.py (on GitHub) we have:

from django.conf import settings
from django.contrib import admin

class VenueAdmin(admin.ModelAdmin):
    list_display = ('name', 'latitude', 'longitude',)
    search_fields = ('name',)

    fieldsets = (
        (None, {
            'fields': ( 'name', 'latitude', 'longitude',)

    class Media:
        if hasattr(settings, 'GOOGLE_MAPS_API_KEY') and settings.GOOGLE_MAPS_API_KEY:
            css = {
                'all': ('css/admin/location_picker.css',),
            js = (

It’s only really the class Media bit that’s new/unusual there.

6. Use it

Click on the map to create the marker. Click somewhere else to move the marker. Or drag it. The latitude and longitude values will update with the marker’s position.

If you want to remove the latitude and longitude from the model, delete the values from the fields as normal; don’t worry about there still being a marker on the map.

This solution was inspired by this blog post by Random Sequence which is a little out of date now. As mine will be one day.

In Web Development on 16 March 2017. Permalink

Two blogs, one WordPress

We recently moved Mary’s website from Movable Type to a self-hosted WordPress install. The site is made up of two blogs and it took a while to figure out how to get WordPress to handle that. This is how I did it.

By default WordPress is a single blog, featuring blog Posts, which can be grouped using Categories and Tags. We wanted to add a second blog to the same WordPress-powered site, with its own, separate, Posts, Categories and Tags.

I saw a few blog posts about how to manage this but none of them seemed like the real deal, relying on faking multiple blogs by using categories. I wanted two “real” separate blogs, but on one site. I thought this would be fairly easy, as I knew creating custom post types is simple. But fulfilling all the requirements was a pain.

They are:

  • A “main Blog” with its own, standard, Posts, Categories and Tags.
  • A separate “Reading” blog, featuring book reviews, with its own, entirely separate, Posts, Categories and Tags (which we’ll call “Reviews”, “Genres” and “Authors”).
  • Separate URL structures for each blog.
  • A home page for each blog featuring the most recent posts on each one, and are both paged.
  • A front page that displays the most recent posts from both blogs combined, and is paged.
  • Separate sections in the WordPress Admin for editing each blog’s Posts.
  • All of this in one site, using one theme.

I eventually pieced all this together from disparate bits of documentation, blog posts and Stack Exchange answers, and thought it worth bringing together here. The full theme is on GitHub, where the code might be more readable, but I’ll include the relevant code as I step through it below.

Our theme is a child theme of the standard WordPress Twenty Sixteen theme. Some of the later steps are specific to this situation but hopefully they’re easy to adapt to yours, if different.

You might find that using directory-based WordPress Multisite is the better solution. I’ve never tried it and ended up here, after initially thinking this would just be a quick hack. This is what I pieced together and it seems to work.

1. Assumptions

You should have your own theme, which might be a child theme. Ours, as mentioned, is a child of Twenty Sixteen theme. Your child theme should have a functions.php file, where we’ll put most of this code.

All the functions and item names are prefixed with sparkly_ in our theme — you can replace all those with whatever’s appropriate to you.

WP’s standard Posts, Categories and Tags will be part of our “main Blog”. We’ll create a separate “Reading” blog with its own Posts (called Reviews), Categories (Genres) and Tags (Authors). You can name your second blog and its entities whatever you like.

We want the URLs for the parts of our “Reading” blog to be like this:

/reading/2017/02/19/post-name/  # A single Review.
/reading/2017/02/19/            # Reviews from one day.
/reading/2017/02/               # Reviews from one month.
/reading/2017/                  # Reviews from one year
/reading/                       # The most recent Reviews.
/reading/genre/genre-name/      # Reviews in a Genre.
/reading/author/author-name/    # Reviews by an Author.

The URLs for the “main Blog” will be similar, but start with /blog/ instead of /reading/.

2. Preparation

Some things we need to do in the WordPress admin first:

  • Create two new, empty, Pages: “Home” and “Blog”.
  • In ‘Settings’ go to the “Reading” section.
    • For “Front page displays”, choose ‘A static page’.
    • Set the front page to be “Home” and the Posts page to be “Blog”.
    • Save changes.
  • In “Settings” go to “Permalinks” and set the link structure you want for the “main Blog”. I used a Custom Structure of /blog/%year%/%monthnum%/%day%/%postname%/. Save changes.

3. Creating the custom post type

The basics of creating a custom post type are documented here. To create our Review post type (known in the code as sparkly_review) we add this to our functions.php:

function sparkly_create_review_post_type() {
            'public' => true,
            'has_archive' => true,
            'hierarchical' => false,
            'rewrite' => array(
                // So the front page of recent Reviews will be at /reading/
                'slug' => 'reading',
                'with_front' => false,
            'capability_type' => 'post',
            'menu_position' => 5, // places menu item directly below Posts
            'menu_icon' => 'dashicons-book-alt',
            'labels' => array(
                'name' => __( 'Reviews' ),
                'singular_name' => __( 'Review' )
            'supports' => array( 'title', 'editor', 'author', 'thumbnail', 'excerpt', 'trackbacks', 'custom-fields', 'comments', 'revisions',),
add_action( 'init', 'sparkly_create_review_post_type', 10);

This should create what acts much like a standard WordPress Post, only it’s called a Review. We use the rewrite array to specify that the front page of this new blog will be at /reading/.

While this creates the new Review posts it doesn’t specify the URLs of those posts. Onward…

4. Custom post type URLs

We need to add some mappings to translate from the nice URLs we described earlier into the URLs that WordPress uses internally to display things.

Inside the function we just created in step 3 we add more after the register_post_type() call:

function sparkly_create_review_post_type() {
    // Add the custom post type itself.

    // Add the permalink structure we want:

    // Matching nice URLs to the URLs WP will use to get review(s)...

    // All reviews from one day:
    // An individual review:
    // All reviews from one month:
    // All reviews from one year:
add_action( 'init', 'sparkly_create_review_post_type', 10);

The add_permastruct() describes the URL structure, like we would in WordPress’s Permalinks settings. And the add_rewrite_rules() maps from different URLs to the internal WordPress URL that it uses to find the correct posts, templates, etc to display.

NOTE: After changing anything to do with the rewrite rules, you’ll need to flush the rules WordPress has stored. Do this by going to “Settings” and “Permalinks” and clicking “Save Changes”.

We also need to add another function:

function sparkly_review_permalinks( $url, $post ) {
    if ( 'sparkly_review' == get_post_type( $post ) ) {
        $url = str_replace( "%year%", get_the_date('Y'), $url );
        $url = str_replace( "%monthnum%", get_the_date('m'), $url );
        $url = str_replace( "%day%", get_the_date('d'), $url );
        $url = str_replace( "%name%", $post->post_name, $url );
    return $url;
add_filter( 'post_type_link', 'sparkly_review_permalinks', 10, 2 );

I think this works with the add_permalinks() call we added and means that when WordPress wants to generate the URL of a Review post, it makes it using our nice permalink structure.

Thanks to “Milo” on the WordPress Stack Exchange for help with the URLs.

5. Custom categories

Our Reviews are going to be categorised by genre. We don’t want to use the standard Categories — which are used by the “main Blog” — so we create a new hierarchical taxonomy:

function sparkly_genres_init() {
    $labels = array(
        'name' => _x( 'Genres', 'taxonomy general name' ),
        'singular_name' => _x( 'Genre', 'taxonomy singular name' ),
        'search_items' =>  __( 'Search Genres' ),
        'all_items' => __( 'All Genres' ),
        'parent_item' => __( 'Parent Genre' ),
        'parent_item_colon' => __( 'Parent Genre:' ),
        'edit_item' => __( 'Edit Genre' ), 
        'update_item' => __( 'Update Genre' ),
        'add_new_item' => __( 'Add New Genre' ),
        'new_item_name' => __( 'New Genre Name' ),
        'menu_name' => __( 'Genres' ),
        'sparkly_genres', // The taxonomy name.
        'sparkly_review', // Associate it with our custom post type.
            'hierarchical' => true,
            'labels' => $labels,
            'show_ui' => true,
            'show_admin_column' => true,
            'query_var' => true,
            'rewrite' => array(
                // So Reviews in one Genre will be found at
                // /reading/genre/science-fiction/
                'slug' => 'reading/genre',
                'with_front' => false
add_action( 'init', 'sparkly_genres_init', 0 );

This creates the new taxonomy, known in the code as sparkly_genres, and associates it with the Review custom post type we created first.

We specify that this is hierarchical, like the standard Categories.

We also specify what the start of the genres’ URLs will be using the rewrite array.

6. Custom tags

We also want to tag each Review with the names of authors. We’ll do this with a new taxonomy that will behave like standard Tags but be entirely separate:

function sparkly_authors_init() {
    $labels = array(
        'name' => _x( 'Authors', 'taxonomy general name' ),
        'singular_name' => _x( 'Author', 'taxonomy singular name' ),
        'search_items' =>  __( 'Search Authors' ),
        'popular_items' => __( 'Popular Authors' ),
        'all_items' => __( 'All Authors' ),
        'parent_item' => null,
        'parent_item_colon' => null,
        'edit_item' => __( 'Edit Author' ), 
        'update_item' => __( 'Update Author' ),
        'add_new_item' => __( 'Add New Author' ),
        'new_item_name' => __( 'New Author Name' ),
        'separate_items_with_commas' => __( 'Separate authors with commas' ),
        'add_or_remove_items' => __( 'Add or remove authors' ),
        'choose_from_most_used' => __( 'Choose from the most used authors' ),
        'menu_name' => __( 'Authors' ),
        'menu_icon' => 'dashicons-admin-users',
        'sparkly_authors', // The taxonomy name.
        'sparkly_review',  // Associate it with our custom post type.
            'hierarchical' => false,
            'labels' => $labels,
            'show_ui' => true,
            'show_tagcloud' => true,
            'show_admin_column' => true,
            'update_count_callback' => '_update_post_term_count',
            'query_var' => true,
            'rewrite' => array(
                // So Reviews by an Author will be found at
                // /reading/author/author-slug/
                'slug' => 'reading/author',
                'with_front' => false
            'capabilities' => array()
add_action( 'init', 'sparkly_authors_init' );

This creates the new Tag-like taxonomy, called Authors (or sparkly_authors in the code), and associates it with our Review custom post type.

Again, we use the rewrite array to specify what the start of its URLs will be like.

7. Tweaking the admin

There are a couple of things we need to add to make the admin experience nicer.

The default icons used for our Reviews and Authors, as shown in the WP admin sidebar, didn’t quite fit. So when we created them you may have noticed we specified menu_icons:

        'menu_icon' => 'dashicons-book-alt',


        'menu_icon' => 'dashicons-admin-users',

To find a new icon, look at the Dashicons and select an icon to have it appear at the top of the page, along with its name.

We also wanted the number of Reviews to appear in the Dashboard’s “At a Glance” widget, alongside Posts, Pages and Comments:

Screenshot of the WordPress Dashboard

To do that add this to functions.php:

function add_custom_post_counts() {
    // array of custom post types to add to 'At A Glance' widget
    $post_types = array('sparkly_review');
    foreach ($post_types as $pt) {
        $pt_info = get_post_type_object($pt); // get a specific CPT's details
        $num_posts = wp_count_posts($pt); // retrieve number of posts associated with this CPT
        $num = number_format_i18n($num_posts->publish); // number of published posts for this CPT
        $text = _n( $pt_info->labels->singular_name, $pt_info->labels->name, intval($num_posts->publish) ); // singular/plural text label for CPT
        echo '<li class="'.$pt_info->name.'-count"><a href="edit.php?post_type='.$pt.'">'.$num.' '.$text.'</a></li>';
add_action('dashboard_glance_items', 'add_custom_post_counts');

No, that’s not as simple as I imagined it would be. And that’s not all. We also need to specify the icon to use for our Reviews, which requires another function:

function sparkly_admin_head() {
  echo '<style>
    #dashboard_right_now .sparkly_review-count a:before {
        content: "\f331";
add_action('admin_head', 'sparkly_admin_head');

This adds some CSS to our admin pages that puts the icon in before our new count of Reviews.

This time, rather than use the name of one of the Dashicons we use its “f” number, as listed on that page, preceded by a \\.

8. The front page

Remember, we want a front page that combines recent Posts, from the “main Blog”, and Reviews. Exactly how you do this depends a bit on your theme, or your parent theme. This is how we did it.

We need to create a front-page.php to combine the two post types. We copied Twenty Sixteen’s index.php into our child theme, renamed it front-page.php, and customised it. The template is on GitHub but, briefly, we added this before get_header(); ?>:

$paged = (get_query_var('page')) ? get_query_var('page') : 1;

// Get both Posts and Reviews:
$args = array(
    'post_type' => array('post', 'sparkly_review'),
    'orderby' => 'date',
    'order' => 'DESC',
    'posts_per_page' => 5,
    'paged' => $paged
$the_query = new WP_Query( $args );

// Pagination fix - to make the pagination appear.
$temp_query = $wp_query;
$wp_query   = NULL;
$wp_query   = $the_query;

And then we changed all mentions of:

  • have_posts()
  • is_home()
  • is_front_page()
  • the_post()


  • $the_query->have_posts()
  • $the_query->is_home()
  • $the_query->is_front_page()
  • $the_query->the_post()

Finally, after the call to the_posts_pagination() we had to add:


in order to make the pagination appear. I’ve no idea.

9. Blog home page

We wanted the /blog/ page, the home page of the “main Blog”, to look the same as /reading/, the home page of the new “Reading” blog. To do this we added an index.php into our child theme (on GitHub) that contains only this:

include( get_template_directory() . '/archive.php' );

That ensures we use the same template, archive.php, for the “main Blog” as is used for the Reading blog home page.

We had to add something to have the title on both blog home pages appear exactly how we want though. This is in our functions.php:

function sparkly_modify_archive_title( $title ) {
    if ($title == 'Archives: Reviews') {
        return 'Reading';
    } elseif ($title == 'Archives') {
        return 'Blog';
    } else {
        return $title;
add_filter( 'get_the_archive_title', 'sparkly_modify_archive_title', 10, 1 );

This just changes the page titles to be “Blog” or “Reading”.

10. Other tidying up

There were a few other little tweaks we had to make to get things just right.

First, back in functions.php we overrode the parent theme’s twentysixteen_entry_meta() method (in twentysixteen/inc/template-tags.php) so that it worked for our new Review post type as well as standard Posts. You can see this on GitHub. In brief, it meant copying the method into our functions.php and then wherever it had something like:

if ( in_array( get_post_type(), array( 'post' ) ) )

adding 'sparkly_review' to it:

if ( in_array( get_post_type(), array( 'post', 'sparkly_review' ) ) )

Second, and similarly, we had to override the parent theme’s twentysixteen_entry_taxonomies() method so that it would use our two new taxonomies instead of Categories and Tags. This is on GitHub.

In our version, in our functions.php, we replaced:

    $categories_list = get_the_category_list( _x( ', ', 'Used between list items, there is a space after the comma.', 'twentysixteen' ) );


    if ( 'sparkly_review' === get_post_type() ) {
        $categories_list = get_the_term_list(get_the_id(), 'sparkly_genres',  '', ', ');
    } else {
        $categories_list = get_the_category_list( _x( ', ', 'Used between list items, there is a space after the comma.', 'twentysixteen' ) );

And replaced:

    $tags_list = get_the_tag_list( '', _x( ', ', 'Used between list items, there is a space after the comma.', 'twentysixteen' ) );


    if ( 'sparkly_review' == get_post_type() ) {
        $tags_list = get_the_term_list(get_the_id(), 'sparkly_authors',  '', ', ');
    } else {
        $tags_list = get_the_tag_list( '', _x( ', ', 'Used between list items, there is a space after the comma.', 'twentysixteen' ) );

Third, the next/previous links at the bottom of individual posts didn’t appear for Reviews. To get that working we had to copy the single.php file from the parent theme into our child theme. And then change this line:

    } elseif ( is_singular( 'post' ) or ( is_singular( 'sparkly_review' )) ) {

To this:

    } elseif ( is_singular( 'post' ) or ( is_singular( 'sparkly_review' )) ) {

A bit of a pain copying the whole file for that one tweak, but there we go.

Fourth and, yes, finally, we found that the footer that appears below posts in the search results didn’t work correctly for our new Reviews. So we had to copy the twentysixteen/template-parts/content-search.php file into our own child theme’s template-parts/content-search.php and then change this single line:

    <?php if ( in_array(get_post_type(), array('post')) ) : ?>

To this:

    <?php if ( in_array(get_post_type(), array('post', 'sparkly_review')) ) : ?>

11. Menus

You’ll probably want a menu on your site that links to the front page and to each of the two blogs’ home pages.

Using WordPress’s Menus, you can add the “Home” Page and the “Blog” Page (that we created way back at the start).

To link to the Reading blog’s home page I added a Custom Link with the URL of /reading/.

12. Widgets

If you want to use the standard widgets to list your new posts and taxonomies you’re out of luck, as they only work with the standard ones. However, I found the Custom Post Type Widgets plugin works really well as an alternative — standard-looking widgets but for any and all post types and taxonomies.


That was more complicated than I expected.

It feels like wanting to do this is going against the grain of WordPress, despite “custom post types” existing. Although the name suggests creating a new post type will create a parallel to a standard Post there’s a lot more to it if you want to create an entirely separate blog.

In Web Development on 14 March 2017. Permalink

Twelescreen updated for 2017

Keeping several projects up-to-date is like spinning plates (is that still a thing people do?). Sometimes things get out of date / crash to the floor. I just updated Twelescreen / picked up a plate.

That’s enough metaphor. I wrote about Twelescreen in 2013 but to recap, it is (a) a scary-looking broadcast system for our rulers’ announcements via Twitter and (b) a Node.js app that is a fullscreen, one-Tweet-at-a-time display with customisable themes, Twitter lists, and so forth.

Thing (a) was so out of date that the UK Twelescreen was still displaying David Cameron’s tweets, never mind the US one featuring Barack Obama’s. Now both are fully updated for the new order of 2017 in which the humour of displaying these terse announcements feels a little less humorous.

Thing (b), the code itself, needed some polish. Given I last (and first) used Node when I wrote this code over three years ago, I thought it might be a huge pain to update it from Node v0.10 up to the current v6.9. Six versions!

Thankfully it was pleasantly easy, the main difference being that a lot of things bundled into Node back then have been broken out into separate packages. But they seem to operate much the same as before, and it took me less than half a day to update that, and all the packages, and get it working again. Finished!

Except, of course, it took about another whole day to fix a bunch of bugs and otherwise tweak things. The other 20% of the work.

As far as I know, no one uses the site or the code but it still amuses me occasionally, so I may as well keep it ticking along.

Screenshot: 'President Trump: JOIN US LIVE! - DJT'

It’s funny right? Right?!

In Projects on 15 February 2017. Permalink

US maps from 1963

In 1963 my mum, Janet Gyford, travelled across America, and dipped into Mexico, and collected a load of stuff along the way. I’ve scanned in the covers of the 66 road maps she collected.

You can see them all in this Flickr album. While many are quite nice individually, they do look great all together on that page. Many of them were collected at gas stations, presumably why so many are branded by oil companies.

I wasn’t sure which to choose as examples here, so here are the only four whose covers are in landscape format (that Alaska one is probably the classiest of the lot, to my modernist eyes):

Alaska Highway Map 1961

Missouri Official Highway Map 1962

Nevada Highways 1962-1962 map

Esso Western United States map

It might be interesting to scan some of the actual maps themselves but (a) time and (b) I wasn’t sure where to start. If you’re eager to see any particular parts of any of those 54-year-old maps, then let me know.

In Misc on 15 February 2017. Permalink

Last.fm added to Django Ditto

Last year I wrote about Django Ditto, a collection of Django apps for copying your Flickr, Pinboard and Twitter “stuff”. I’ve now added the ability to copy your Last.fm listening data.

You can see the default public templates on the demo site, see the code on GitHub or read the documentation.

Screenshot of the website

Using Last.fm’s API Django Ditto can store records of your listening and produce the usual charts of your top artists, tracks and albums, along with a page per artist, track and album. As with Flickr, Pinboard and Twitter, it can do this for multiple accounts. And, as with the other services, you can see everything from a single day (for example).

I initially thought I might pull in more data, given Last.fm often supplies MusicBrainz IDs, but this was a bit unreliable, especially when it comes to artists sharing the same name. This isn’t Last.fm’s fault, given their data comes solely from the names of tracks played, but I ended up walking away from that idea and doing nothing more than link to MusicBrainz if an ID is present.

Using Django Ditto don’t have to use the templates (which use Bootstrap v4), or the Django views. You could use the app to store copies of your listening (or Tweets, or photos, or bookmarks) and use them however you like on your own Django website. There are some template tags to make it easier to output common things in your own templates.

Thankfully, the Last.fm data was simpler than dealing with Tweets or Flickr photos, the last parts I added, as this has been taking, inevitably, longer than I planned. I hope to add Foursquare/Swarm check-ins next but I might write some different code first, to have a change.

As ever, if you or your organisation need some web stuff making, particularly this kind of thing — APIs, wrangling data, archiving, etc — do email me.

In Web Development on 9 February 2017. Permalink

MoMA Exhibition Seplunker

I recently worked on another site with Good, Form & Spectacle, the MoMA Exhibition Spelunker, exploring sixty years of exhibitions at the Museum of Modern Art in New York.

MoMA released a lot of data about their exhibitions from the museum’s opening in 1929 to 1989 and commissioned GF&S to make something from it.

Screenshot of the home page

A spelunker, according to Chambers, is “a person who explores caves as a hobby” and we aimed to explore MoMA’s raw data and make it more visible and penetrable by everyone else. It’s hard to get a decent sense of the shape of lists of data so we set off to explore.

The fundamentals of this are to make the huge amount of rows and columns of data easily browsable. So you can view the list of exhibitions in date order. You can see all the artists who have been featured or, for example, all the curators involved. Or see all the people, no matter what they’ve done. Or you can get a sense of the museum’s directors and departments over time.

Visualising the data

Such lists are much more friendly than raw data but they’re still hard to take in at a glance. We wanted to be able to get a better overview of those sixty years, and so we created various graphs to depict activity over time. These, made using D3.js, sometimes act both as sort-of sparklines and also as forms of navigation.

For example, a simple chart of the number of exhibitions per year can be clicked/tapped to jump to a particular year:

Screenshot of the exhibitions-per-year chart

Or you can compare how the number of people performing different non-artist roles has changed over time:

Screenshot of the roles charts

Or, for one of those roles, see which people performed them when. Here are the people who were curators most often, and you can see when their work overlapped:

Screenshot of the curators list

A bigger chart was designed to show MoMA’s different departments, to give a sense of when they began and also who was in charge of them. Here we can see that William S. Lieberman was director of three departments:

Screenshot of the departments chart

More media

It’s always nice when you can use raw data to fetch more related data or media.

MoMA had already done some work finding reviews of exhibitions in the New York Times, so we were able to tie that in to the existing data to show summaries of, and links to, the articles. For example, here’s a review of the first exhibition, C�zanne, Gauguin, Seurat, Van Gogh from 1929:

Screenshot of a New York Times review

MoMA’s data also included different kinds of identifiers for the people involved, which meant we could use the Wikidata ID to fetch an image for many of them. Seeing faces makes the whole thing much more alive… and makes clear how the most involved people over six decades are white men:

Screenshot of the people list

Because this was a project for MoMA we also had access to their images — brilliantly they have a large collection of photographs of the exhibitions themselves. It’s great to see artworks in place. For example, here’s an exhibition about the Bauhaus in 1938-9:

Screenshot of the Bauhaus exhibition page

I like it when you occasionally catch sight of a person in the otherwise empty galleries.

Another nice project, making a large amount of data easy to take in and easy to explore.

This was built using Django as the back end. Read more about the project on the Good, Form & Spectacle work diary.

In Projects on 6 February 2017. Permalink

Recent comments on writing

Writing archives by category