snorfalorpagus dot net

I’m a big fan of the Anaconda Python distribution. It makes managing multiple Python environments on different operating systems easy (at least in theory). I recently came across an issue trying to import a Cython extension I’d built for Linux on a different machine. We’d be testing the module on Travis-CI for months without any issues so this came as a surprise. When I tried to import the module the following exception was raised:

Automatic pickle support in Cython is still a pending feature. In order to support pickling of cdef classes you must implement the pickle protocol. This is done by implementing the __getstate__ and __setstate__ methods. Although the official documentation is quite clear, it lacks a simple example and also instruction on handling objects that can’t be directly pickled. A minimal example is given below for the Person class which stores a name (string) and age (integer).

A few years ago I came across Umbrella Today. The site had a simple function, providing a weather forecast with only one variable: is it going to rain today? Enter your zip code and you get a Yes-No answer to the question. The simplicity of this appealed to me. Although it was featured on LifeHacker in 2008 the site has since gone offline. Anyway, I don’t live in the USA and therefore didn’t have a zip code.

Large polygons are a problem for geoprocessing operations. Intersections involving large polygons are slow because the entire geometry needs to be evaluated. In addition to this, spatial indexing doesn’t help (much) because the bounding box of the polygon is itself large. This is a common problem with flood extent data, which is exported from hydraulic models without much thought as to how it will be used. An example of this is the Risk of Flooding from Rivers and Sea (RoFRS) layer available from the Environment Agency (released under the OGL on EA Geostore).

The Centre for Ecology & Hydrology (CEH) produce a dataset which provides an estimate of daily rainfall on a 1km grid for Great Britain and Northern Ireland from all the way back to 1890 onward to 2014. The data is hosted in NetCDF format hosted on an OPeNDAP server. At the time of writing only data to 2012 is available via OPenDAP. The script below demonstrates how to access this dataset from Python using the netCDF4 module.

At some point you’re going to care about how long it takes your geoprocessing algorithm to run. While the Fiona and Shapely libraries make efforts to be as efficient as possible, a lot will depend on the algorithms you use them in. I’ve written before about how spatial indexing can speed up intersection and nearest neighbour queries. At the end of the previous article I mentioned a trade-off between speed and memory usage when the same features are requested more than once.

Update: As of 2016-03-31 the Oxontime service has changed provider and the API is no longer functional. Presumably a new API will be made available at some point. This post remains for posterity. The Oxontime Real Time Information (RTI) system provides real time predictions for bus departures in Oxfordshire, which are displayed on screens at many bus stops. The data is also available via an API. I thought it would be fun to use an Adafruit 16x2 LCD display and a Raspberry Pi to make my own personal display (photo below).

This post shows how to find the version number of an ArcGIS map document (.mxd) programmatically using Python. It seems that arcpy does not provide a method to do this itself. ArcGIS map documents are actually Microsoft OLE2 files (also called Structured Storage, Compound File Binary Format or Compound Document File Format). There is a Python module called oletools which can read this format, which is available on PyPI. To install oletools using pip (from a command prompt):

This post shows how to extract data from a raster only where it intersects a vector feature using Rasterio. Fiona is used to read the vector data, and Shapely is used only as a convenience to calculate the feature’s bounding box. GDAL provides a function to “burn” vector shapes into rasters (i.e., to rasterize the geometries). This functionality is accessed from Rasterio using the rasterize function. This tool can be used to create a raster mask for another raster layer.

Embedding a map in a Qt application using Leaflet is surprisingly easy. The QtWebKit.QWebView object can be used to create a browser window within a Qt dialog. Within this browser window you can create an interactive map using Leaflet just as you would for any other browser. Communication between Qt and the Leaflet map is also possible, allowing you to control the map from the main application, or extract information from Leaflet.