# Geocoding of Chicago crime data

The City of Chicago publishes some rich crime data, available e.g. here:

https://catalog.data.gov/dataset/crimes-2001-to-present-398a4

The address of event crime is partly obscured, so as to read for example "010XX N CENTRAL PARK AVE". The data also provides coordinates and longitude / latitude data. These agree if you use the correct projection (in feet). I would have expected each coordinate to, for example, correspond to the middle of the block, but this is not so.

Screen shot from QGIS with OpenLayers as the basemap:

So what we see is clusters of events, corresponding to the correct block. What I find puzzling is that the mid-point of these clusters does not appear to be the mid-point the block. The pattern is repeated, seemingly, across the whole dataset (so in other parts of the city, with residential buildings on the avenues, you get north/south clusters, again, not centred). Could the centre of each cluster perhaps correspond to the centroid (projected onto the middle of the road) of all the buildings in the block?

From some messing about in Python, and simulation, the clusters look like they are normally distributed (not uniformly distributed).

Interestingly, I've now looked at the complete data set, from 2001 onwards, and the crimes reported in 2001 (only) seem to show a rather different pattern:

Frankly, this looks pretty realistic to me! Is it possible they only started obscuring the real coordinates after 2001?

Does anyone know the exact way the location coordinates are actually generated?

## migrated from gis.stackexchange.comMay 27 '17 at 20:17

This question came from our site for cartographers, geographers and GIS professionals.

• For questions about what seems to be open data I recommend researching/asking at the Open Data Stack Exchange. – PolyGeo May 12 '17 at 10:54
• It probably isn't the best plan to start a question with an assumption about your audience, especially an unwarranted one. The first sentence of a question should be designed to lure in more readers, not drive them away. Instead, just state facts about your data and ask your question. – Vince May 12 '17 at 11:56
• Since this got migrated, it is now a duplicate of my question: opendata.stackexchange.com/questions/11190 Sorry... – Matthew Daws May 29 '17 at 10:39

Since I asked this, I have made some progress.

If you ask around (or craft the appropriate web search terms) it is possible to find copies of the data which are old. In the past (I think, 2015 and before) the data was geocoded to building level! This obviously is rather at odds with reporting addresses like "010XX N CENTRAL PARK AVE" when it's possible to look at the coordinates and work out the exact address. I am guessing that the new, unusual, distribution of coordinates is to remove this breach of privacy.

Anyway, once you have an old copy of the data, you can join the datasets, and see how the coordinates have moved. I analysed this in an IPython notebook which you can see here: https://github.com/QuantCrimAtLeeds/PredictCode/blob/master/examples/Chicago/Old%20Chicago%20Data.ipynb The plots at the end might be interesting.

• Something similar is how the UK Police API gives the locations of crimes: they assign the crime to an unspecified landmark or street. For example, there may be a murder on or near nightclub or on or near Smithfield Street. I don't know if that helps – Beta Decay May 29 '17 at 17:43

I worked extensively with this data when I worked at the Chicago Tribune, including building crime.chicagotribune.com

According to a conversation I had with a leader of Chicago's data publishing efforts, the geocodes for crime data are not intended to be precise, because that would be at odds with "redacting" the address down to the block level. My understanding is that "jitter" of up to 1/8 mile is applied to every point in the dataset. Since this was conversation some years ago, this should be treated as anecdotal unless directly verified with the Chicago Police Department, but it squares with my experience.

More importantly, remember that this data has been collected by humans and has a high risk of incidental errors. For one project, we compared geocodes of crimes marked at CTA train stations with the actual locations of train stations, and found many cases where the crime incident geocodes were much further than 0.125 mile from any train station. Those problems are more likely with the location type tagging than the geocodes, but still, proceed with caution.

Two things that may help us understand the question at hand better:

• What data source, script or tool are you using to geocode the crime reports by city block? The quality of geocoding can vary widely depending upon the method you use.
• It's not entirely clear if possibly the first three supplied digits at the start of the address aren't the cause of the discrepancy in locations. Do any of the incidents occurring on the same block have different preceding street numbers?

That might help us understand what's causing the non-uniform distribution.

• Sorry, I am somehow not making myself clear. I am not doing the geocoding: the data comes with lon/lat (and projected variants) coordinates. The "block" level of the address does seem to always correspond correctly to the coordinates. – Matthew Daws May 29 '17 at 10:38

And the coordinates are identical precisely down to the last decimal point for the incidents?

If so, while I'm not familiar with ArcGIS's plotting functionality (I run OSX), I would assume it's just ArcGIS' method of plotting multiple points. Something like MarkerClusterer would be necessary.

• Hi Carl. I'm not using ArcGIS. The plots are from QGIS (but look the same in some plots I made manually in python). The coordinates are not the same in the input file (so no postprocessing is done to make the plot). – Matthew Daws May 29 '17 at 11:31
• Please use comments for conversations and, unless there's a particular reason, consolidate your actual answers to a question to a single "answer". – Joe Germuska May 30 '17 at 20:59