Is there (or should there be) a standard way to categorize procurement data at the municipal level?

Question

One of the data sets that I see being released by cities is information about what services cities are procuring. For example, the City of Chicago lists their contract award data like this: https://data.cityofchicago.org/Administration-Finance/Contracts/rsxa-ify5

What if we had the same data schema in multiple cities so that we could start to compare procurement across different cities? Is NY getting a better deal on paper than Philadelphia? Are there contracts that stand out from other cities once you actually look at them side by side?

The first question is, has anyone else tried to standardize this type of data? If there's not a standard yet, would there be enough support in the opengovernment community to try and create one?

Answer 1

I think it would be foolish to try and replace the NAICS system. NAICS is the federal government's categorization system, and in my experience, it is also in use at the municipal level in the United States. Here's a longer description:

NAICS was developed under the auspices of the Office of Management and Budget (OMB), and adopted in 1997 to replace the Standard Industrial Classification (SIC) system. It was developed jointly by the U.S. Economic Classification Policy Committee (ECPC), Statistics Canada , and Mexico's Instituto Nacional de Estadistica y Geografia , to allow for a high level of comparability in business statistics among the North American countries.

It's easy to hate on a system that only gets as specific as "Custom computer programming services", but in reality, it was developed with a lot of effort by a bunch of smart folks.

In my opinion, there are two challenges when trying to compare prices across governments:

• NAICS is not always specific enough, especially when it comes to technology.
• Units are not standardized.

While I would love to see a data standard that fixes these issues, think it's inevitable that there is always going to be a large amount of discretion/manual comparison involved. Because of this, I'd be much more interested in a system that allows a user to view aggregated pricing data from multiple cities, filtered by query or NAICS code. It would pull from multiple sources like the City of Chicago link in the original post.

As far as NAICS goes -- maybe we could create additional subcategories for codes that are way too broad, such as "Custom computer programming services"? Or taking this idea further, would there be a way to create a superset of NAICS that extends it so it never goes out of date?

EDIT: There's a discussion going on about this at https://github.com/dobtco/NAICS/issues/1

Answer 2

There's plenty of classifications for public procurement used in different countries, Common Procurement Vocabulary (mentioned above) being the standard in EU member states. To give you some links:

• Jose María Alvarez Rodríguez (while at the WESO research group) worked on product scheme classifications used in public procurement. This diagram shows his coverage; including CPV, NAICS, NACE or UNSPSC. Many of these datasets are listed at the Data Hub, however, a lot of these contain dead links.
• Recently, I did some work with U.S. procurement data from FedBizOpps.gov and USASpending. These datasets use NAICS as the classification (available here in RDF) and Federal Acquisition Regulation code lists (available here in RDF).

Answer 3

I think NAICS is useful, but it depends on what you're trying to categorize. In practice, NAICS is a self-reported industrial classification of the contractor, and not necessarily a classification of what is being purchased. In fact it was developed to make comparisons across business statistics easier.

The federal contracting data has Product and Service codes in addition to NAICS that detail the category of the actual item purchased and were explicitly designed for this purpose. A big contractor like General Dynamics may self-report it's industrial classification as whatever is on it's tax form or D&B registration, but they could be selling the government something that is classified under a totally different industrial category.

I believe this is the most recent Product and Service Code manual. It gets revised every few years and they appear to keep a relatively detailed changelog.

Answer 4

While I don't know about the US, a fallback may be looking at the Common Procurement Vocabulary (CPV) used throughout the European Union. It's a bit weird but fairly detailed and since it's already in use in the EU member states and a few countries beyond that, it may be a nice thing to use further:

https://github.com/opented/opented/tree/master/cpvcodes

Answer 5

You hit on a particularly relevant pain-point when it comes to procurement data. The purchasing thresholds differ wildly across the map, and there isn't any universal standard for classification.

I maintain an open-source project at http://openprocure.us that, while far from being close to comprehensive, is an effort in the right direction.