Geocoding of Chicago crime data

Question

The City of Chicago publishes some rich crime data, available e.g. here:

https://catalog.data.gov/dataset/crimes-2001-to-present-398a4

The address of event crime is partly obscured, so as to read for example "010XX N CENTRAL PARK AVE". The data also provides coordinates and longitude / latitude data. These agree if you use the correct projection (in feet). I would have expected each coordinate to, for example, correspond to the middle of the block, but this is not so.

Screen shot from QGIS with OpenLayers as the basemap:

So what we see is clusters of events, corresponding to the correct block. What I find puzzling is that the mid-point of these clusters does not appear to be the mid-point the block. The pattern is repeated, seemingly, across the whole dataset (so in other parts of the city, with residential buildings on the avenues, you get north/south clusters, again, not centred). Could the centre of each cluster perhaps correspond to the centroid (projected onto the middle of the road) of all the buildings in the block?

From some messing about in Python, and simulation, the clusters look like they are normally distributed (not uniformly distributed).

Interestingly, I've now looked at the complete data set, from 2001 onwards, and the crimes reported in 2001 (only) seem to show a rather different pattern:

Frankly, this looks pretty realistic to me! Is it possible they only started obscuring the real coordinates after 2001?

Does anyone know the exact way the location coordinates are actually generated?

Answer 1

Since I asked this, I have made some progress.

If you ask around (or craft the appropriate web search terms) it is possible to find copies of the data which are old. In the past (I think, 2015 and before) the data was geocoded to building level! This obviously is rather at odds with reporting addresses like "010XX N CENTRAL PARK AVE" when it's possible to look at the coordinates and work out the exact address. I am guessing that the new, unusual, distribution of coordinates is to remove this breach of privacy.

Anyway, once you have an old copy of the data, you can join the datasets, and see how the coordinates have moved. I analysed this in an IPython notebook which you can see here: https://github.com/QuantCrimAtLeeds/PredictCode/blob/master/examples/Chicago/Old%20Chicago%20Data.ipynb The plots at the end might be interesting.

Answer 2

I worked extensively with this data when I worked at the Chicago Tribune, including building crime.chicagotribune.com

According to a conversation I had with a leader of Chicago's data publishing efforts, the geocodes for crime data are not intended to be precise, because that would be at odds with "redacting" the address down to the block level. My understanding is that "jitter" of up to 1/8 mile is applied to every point in the dataset. Since this was conversation some years ago, this should be treated as anecdotal unless directly verified with the Chicago Police Department, but it squares with my experience.

More importantly, remember that this data has been collected by humans and has a high risk of incidental errors. For one project, we compared geocodes of crimes marked at CTA train stations with the actual locations of train stations, and found many cases where the crime incident geocodes were much further than 0.125 mile from any train station. Those problems are more likely with the location type tagging than the geocodes, but still, proceed with caution.

Answer 3

Two things that may help us understand the question at hand better:

• What data source, script or tool are you using to geocode the crime reports by city block? The quality of geocoding can vary widely depending upon the method you use.
• It's not entirely clear if possibly the first three supplied digits at the start of the address aren't the cause of the discrepancy in locations. Do any of the incidents occurring on the same block have different preceding street numbers?

That might help us understand what's causing the non-uniform distribution.

Answer 4

And the coordinates are identical precisely down to the last decimal point for the incidents?

If so, while I'm not familiar with ArcGIS's plotting functionality (I run OSX), I would assume it's just ArcGIS' method of plotting multiple points. Something like MarkerClusterer would be necessary.