15

You have been misled, CBSA files are in fact what you are looking for. Check the documentation detailing what each type of shapefile contains. Want more verification? Here is the description of the GEOID field for the CBSA shapefile: 2010 Census metropolitan statistical area/micropolitan statistical area code found in CBSAFP10. And here is the description ...


14

Some general advice: Invest in community managers and evangelists. The worst thing you can do is create a forum or invite feedback and then not have anyone with a mandate to respond to it in an official capacity; I'm seeing this happen right now with the newly launched project-open-data in which they've invited contributions and have no one to actually ...


13

Here is most comprehensive list of all .gov /Developer pages compiled by Gray Brooks at GSA http://gsa.github.io/slash-developer-pages/ You can suggest new additions if some info is missing. It is currently well maintained.


12

Resource.org has gathered reports dating back to 2002 and says that they process new data monthly. Bulk data can be pulled from here. For those who prefer a simple search, the Economic Research Institute (ERI) has a database of forms dating back to 2003. In at least one case, a journalist on the NICAR-L list reported back that there was a form in the ERI ...


12

Currently Data.gov is run on bunch of technologies. Specifically one of the primary data platforms is Socrata - explore.data.gov. There is API for it defined here http://dev.socrata.com/, but essentially you can call this endpoint and iterate through all pages of master list of datasets https://explore.data.gov/api/dcat.json?page=1 I have sample scrapers ...


12

Full disclosure: I am a GSA employee and the Tech Lead for FBOpen, a website and API for search and discovery of federal business opportunities. There is indeed both bulk data and an fbo.gov API available, although I can only offer experience with the former. There are two different versions of the bulk FTP downloads, weekly files and nightly files. These ...


10

On the Data.gov team, we'll be blogging and posting here as well about some of the upcoming changes. The intent is to have all of you give us feedback as we start to evolve ideas, rather than just seeing what happens at the end of the design process. My blog above shows the catalog upgrade using CKAN 2.0 (it's the demonstration site, so don't worry if you ...


9

The bifurcated view of open data on Data.gov will evaporate later this month and there will be an API to the entire catalog. You can also separate that catalog along geo and non-geo data if that's your preference. To get an early look at the direction we are heading, check out: http://geo.gov.ckan.org/dataset Raw data is being merged with geo-data as ...


9

There's also a good list just released from ProPublica starting for 2012 http://projects.propublica.org/nonprofits/ And you can always request a 990 directly from an entity -- it is required to give it to you. Also, GuideStar aggregates them and allows you to obtain one-at-a-time at no cost. Aggregate data will cost, however.


8

I think the question, as phrased, is impossible to answer well, but I will try. Q: "How does a consumer know they are getting good data?" A: Let me answer with more questions. How does a consumer know they are getting a good search result from Google? How do they know when the news is of high quality? It depends. As consumers get more interested and ...


8

You've asked three separate questions in one question, I don't think this fits into Stack Exchange's one question, one answer model, should have been three separate posts maybe. But anyway.. It appears that every API response includes a JSON piece with help information. can this be excluded? No :) Not as far as I know. Just ignore it. I agree it seems a ...


8

The native raw datapoints comprising some of the more popular tables related to your search are accessible through sites at both agencies. As examples: Summary reports on current employment statistics and detailed data behind the report from the Bureau of Labor Statistics Summary reports on firm size, employees, and payroll and detailed data behind the ...


8

My guess is that the census tiger products are going to have what you're looking for. http://www2.census.gov/geo/pdfs/maps-data/data/tiger/tgrshp2016/TGRSHP2016_TechDoc_Ch2.pdf "The legal entities included in these shapefiles are: American Indian off-reservation trust lands American Indian reservations (both federally and state-recognized) American Indian ...


7

On Data.gov we haven't looked at trying to aggregate a forward-looking schedule. In general we encourage agencies to release data as quickly as possible. Some data releases occur on a very regular basis, but there is often some fluctuation based on the internal approval processes. I expect as agencies respond to the actions to comply with the new Open Data ...


7

Block level data is only available from the Decennial Census, which means that you can only get fundamental demographic and housing data. This amounts to age, sex, race, household/family structure, home ownership and home vacancy rates. Some Decennial Census tables are only tabulated at the census tract level and higher, because of restrictions based on ...


7

There are several sources you could compile information from. City data portals that list gas station locations: New Orleans: https://data.nola.gov/Administrative-Data/NOLA-Gas-Stations-Map/ic3z-bztr Washington, DC: https://opendata.socrata.com/dataset/Gas-Stations-in-Washington-DC/tk2x-chx8 State data portals that list alternative fuel stations: ...


7

Each of the datasets on Data.gov describes the license used (see the upper left items on the dataset page). The intent for data provided by the U.S. Government (whether it is on Data.gov or not) is to have an open license, as defined by Project Open Data. The license field in the Data.gov metadata schema is defined as well. In most cases, the license is "...


7

For now the Data.gov CKAN API redirects package_list to package_search and for package_search the relevant Solr parameters to limit your query are rows and start. For example, http://catalog.data.gov/api/3/action/package_search?rows=1000&start=0 And then page through. If you or anyone has additional feedback on what Data.gov CKAN API documentation ...


7

There are quite a few cities that publish this data. A quick way to find it, is via Data.gov, which provides local government as well as federal government data. There are 53 city crime statistics datasets published, most of which include location data: All 53 city crime datasets, which are from 9 cities, but broken down by multiple factors for better ...


7

The Open Knowledge Foundation keeps a list of Cities that publish crime datasets. Most datasets have location data. In some cases the location is in State Plane coordinates instead of Lat/Lon. They list 51 cities. http://us-city.census.okfn.org/dataset/crime-stats Below is a blog posting I posted a year ago on methods for doing crime analysis: http://www....


7

I recommend Occupational Employment Statistics. Includes approximately 800 job categories/titles (Standard Occupational Classification). Reports estimated employment, average wages and 10th/25th/50th/75th/90th wage percentiles. The upside is that it's pretty easy to download or hit the API. The downside is that there are quite a few "holes" in the data, ...


6

This is a complex opportunity to which many of us in open data are trying to respond. (Disclaimer: I'm the Evangelist for Data.gov and this answer addresses examples from my work there and at NASA.) There are two broad kinds of communities to engage in around open data. One is the technical community, like many of you here, of users and consumers of the ...


6

The datasets that you might be looking for are all school districts and/or the School Universe Survey. To provide feedback on any of the datasets on Data.gov, you can contact the dataset owner listed on the page (contact email, which in this case is jane.clark@ed.gov) or contact Data.gov at the contact link at the bottom of every Data.gov page. The URL you ...


6

Data.gov is down because of the US government shutdown. You can see almost identcal language on the census.gov website, which explains that the shutdown is causing the site to go offline. Here is an article explaining more details about what is and is not available during the shutdown. And if you are interested in seeing what APIs are available for when ...


6

There's an Open Plant Hardiness Zones (OPHZ) project on Github where various people have reverse-engineered a pdf of the hardiness zones (pdf, really large) to produce a GIS file (SHP) of the zone boundaries. The ophz-c version is the latest. It's public domain. You could use GIS software or tools, and a list of the center point of each zip, to find the ...


6

Just released on Github!! Frostline A parser and resulting dataset for USDA plant hardiness zones. It uses the ZIP-querying API built into the U.S. Department of Agriculture’s Plant Hardiness Zone website (e.g., http://planthardiness.ars.usda.gov/PHZMWeb/ZipProxy.ashx?ZipCode=55555 Full data set: CSV (1 MB) (archive.org link) MySQL (1.8 MB) (archive....


6

I am not sure if this is something that is required to be reported. Data like does this does exist for campaign/PAC contributions and lobbying money spent by corporations. The Sunlight Foundation has a great site called InflueceExplorer.com (which has a web UI, bulk downloads, and API). For example, a search for Microsoft. I also heard from a contact well ...


6

Apparently, here: Source: http://www.usgs.gov/blogs/features/files/2015/01/free-and-open-data-policy-graph-no-footnote.jpg Here is the original post, noting cost savings from Landsat. http://landsat.gsfc.nasa.gov/?p=9654


5

See this blog post about their move to the CKAN platform: http://www.data.gov/blog/under-hood-open-data-engine


5

The list of all organizations that are contributing data (or metadata) to Data.gov can be found in two different places. One place is on the Federal Agency Participation link at the bottom of the page. Clicking on an agency or sub-agency name takes you to a page with all of the raw datasets and tools submitted by that agency. There is also a listing of ...


Only top voted, non community-wiki answers of a minimum length are eligible