snorfalorpagus dot net

Sorted Open Data (Part 2) with D3

Posted on

This post is Part 2 of a series. See Part 1: Sorted Open Data with Shapely and SVG.

D3 is a JavaScript library for manipulating documents based on data. Support for SVG in all modern browsers allows you to create beautiful, interactive maps all though manipulation of the DOM using D3.

D3 has quite a steep learning curve and every time I use it I feel like I’m fumbling around in the dark. Despite this I thought I’d try to implement the previous sorted lakes graphic with D3 and GeoJSON.

Provided you have JavaScript in your browser enabled, the result should be shown below. It looks more or less identical to the previous post, but is created in the browser from a GeoJSON file: http://snorfalorpagus.net/static/lakes.json.

The first step is to convert our source data into GeoJSON. This could be achieved very easily using the ogr2ogr command line tool that ships with GDAL, but I also wanted to simplify the data to reduce the file size. The Python script below does just this: simplifies the geometries with a threshold of 25 metres, reprojects the coordinates from OSGB to WGS84 (required by GeoJSON), and rounds the coordinates to 5 decimal places.

import fiona
from shapely.geometry import shape, mapping, Polygon, MultiPolygon
from shapely.ops import transform
from functools import partial
import pyproj
import os
import json

# reproject from OSGB to WGS84
project = partial(
    pyproj.transform,
    pyproj.Proj(init="epsg:27700"),
    pyproj.Proj(init="epsg:4326"))

# round coordinates
def round_coord(x, y, z=None):
    x = round(x, 5)
    y = round(y, 5)
    return tuple(filter(None, [x, y, z]))

# reduce a geometry
def minify(geometry):
    geometry = geometry.simplify(25)
    geometry = transform(project, geometry)
    geometry = transform(round_coord, geometry)
    return geometry

features = []
with fiona.open("WFD - Lake Waterbodies.shp") as src:    
    for feature in src:
        geometry = shape(feature["geometry"])
        geometry = minify(geometry)
        feature["geometry"] = mapping(geometry)
        feature["properties"] = {
            "name": feature["properties"]["name"],
            "shape_area": int(float(feature["properties"]["shape_area"]))
        }
        features.append(feature)

fc = {"type": "FeatureCollection", "features": features}

data = json.dumps(fc)
print(data)

Now the data is prepared we’re ready to start using D3. The approach to positioning and scaling is the same as in the previous post. The main difference is the use of a pre-calculated area, as we want the area in square-kilometers not square-degrees. We’re also using the mercator projection for display.

<style>
.lakes svg {
    margin: 0 2px;
}
.feature {
    fill: #75CFF0;
    stroke: #000;
    stroke-opacity: 0.4;
}
</style>
<script src="https://d3js.org/d3.v3.min.js"></script>
<script>
var scale = undefined;

var base_height = 200;

var container = d3.select("body").append("div").attr("class", "lakes");

var projection = d3.geo.mercator()
    .scale(100000);

var path = d3.geo.path()
    .projection(projection);

d3.json("lakes.json", function(error, json) {
    if (error) throw error;
    var features = json.features;

    features.sort(function(a,b) { return b.properties.shape_area - a.properties.shape_area});

    svg = container.selectAll("svg")
        .data(features);

    // add paths
    svg.enter().append("svg")
        .append("g")
        .each(function(d, i) {

            var bounds = path.bounds(d),
                dx = bounds[1][0] - bounds[0][0],
                dy = bounds[1][1] - bounds[0][1],
                x = (bounds[0][0] + bounds[1][0]) / 2,
                y = (bounds[0][1] + bounds[1][1]) / 2;

            dx *= 1.04;
            dy *= 1.04;

            if (scale == undefined) {
                height = base_height;
                width = (dx / dy) * base_height;
                scale = height / dy;
            } else {
                height = dy * scale;
                width = dx * scale;
            }
            d3.select(this.parentNode)
                .attr("width", width)
                .attr("height", height)

            var translate = [width / 2 - scale * x, height / 2 - scale * y];

            d3.select(this).selectAll("path")
                .data([d])
                .enter().append("path")
                    .attr("class", "feature")
                    .style("stroke-width", 1 / scale)
                    .attr('d', path);
            d3.select(this).attr("transform", "translate(" + translate + ")scale(" + scale + ")");
        });

    svg.append("title").text(function(d,i ) {
        return d.properties.name + " " + Math.round(d.properties.shape_area / Math.pow(1000, 2) * 100) / 100 + "km2";
    });

    container.append("p").html("Contains public sector information licensed under the <a href=\"https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/\">Open Government Licence v3.0</a>.");
});
</script>