Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
63

The Open Product Data project is a comprehensive source for open barcode data. As of May 2014, they have close to a million products in their database. The data is accessible online, through an Android app (source code), and available for download under the Open Data Commons Open Database License (ODbL).


38

The complete USDA National Nutrient Database for Standard Reference can be downloaded as ASCII text files from https://www.ars.usda.gov/Services/docs.htm?docid=8964 — no PDF scraping necessary :) Regarding product barcodes, have a look at Open Product Data, a new project by the Open Knowledge Foundation.


32

Conferences: TransparencyCamp, by the Sunlight Foundation, annually since 2009. Open Knowledge Conference (OKCon), annually since 2005. Open data has been central since its inception - in 2012 this expanded to be the Open Knowledge Festival (OKFestival). Open Government Data Camp, by the Open Knowledge Foundation, annually since 2010. From 2012 the camp has ...


30

I recently set out to do this. https://www.datakick.org/ The mission: Build a database of every product in the world that is free to everyone. It's licensed under CC0 (Public Domain Dedication) and all the information is available for download. There aren't many products yet, but it's a start for those who want to make this a reality.


24

Sci-Hub is a paywall-bypassing website that uses "shared" user credentials to provide PDF or HTML scientific papers. The website itself doesn't store any papers. (There are interesting comments on your same question on another site.) But LibGen is said to archive each PDF retrieved by Sci-Hub. http://gen.lib.rus.ec has a downloads page - (click the tab to ...


20

The European Centre for Medium-Range Weather Forecasts (ECWMF) has a rather impressive data collection available via batch scripts, downloadable files or even tailored formats. The first two are freely available (under specific conditions) for non-commercial research but registration is necessary. Just to give an example, the GRIB dataset lists 4115 ...


20

The Historical price data link for Bitcoins on CoinDesk.com lets you download a CSV starting 18 Jul 2010. I found this by simply Googling 'bitcoin history'


19

Using the Census Bureau's API, you can retrieve quite a bit of information about counties, but you need to learn where it's stored. You can request up to 50 different variables in a single call, and you can ask for all counties in the US with one call. To the census, "cities" are "places," and that includes things that you and I might think of as a city but ...


19

Check out: An official source for agency information: https://www.federalregister.gov/api/v1/agencies GSA API: http://www.usa.gov/About/developer-resources/federal-agency-directory/index.shtml Source: https://github.com/unitedstates/orgchart/issues/1


19

In short, no, there is no comprehensive open database of EAN 13 barcodes. I worked on a project that required UPC information and I found out that this data is not only hard to get, but what is there is often incomplete or poorly formed. I subscribed to SimpleUPC's API service to do some testing on matching products to parent companies. Manufacturer names ...


18

The IRS website has an Exempt Organizations Select Check that allows the user to search for organizations that: Are eligible to receive tax-deductible contributions Were automatically revoked Have filed Form 990-N (e-Postcard) After selecting one of these options, a link will appear to download the entire database of organizations. The databases (plain-...


18

ConceptNet is a semantic network containing lots of things computers should know about the world, especially when understanding text written by people. Trying to reproduce your relation-sequences yields: finger PartOf hand IsA body part, which looks surprisingly "dead end". chair IsA seat RelatedTo furniture MadeOf wood ...


18

California Polytechnic State University has a downloaded dataset on fatal accidents on the national highway system for the year 2007: https://wiki.csc.calpoly.edu/datasets/wiki/HighwayAccidents Data.Gov has a number of datasets on accident data by state: https://explore.data.gov/catalog/raw?tags=crash The NHTSA has summary statistics for 2012 on a state ...


17

As it turns out there are several resources one can use for all main cryptocurrencies so I will post here the most relevant and flexible I was able to gather. All Cryptocurrencies Coinmetrics https://coinmetrics.io/data-downloads/ This page has data for: Bitcoin, Litecoin, Ethereum, NEM, Decred, ZCash (transparent transactions only), Dash, Dogecoin, ...


16

For the United States, the Census Bureau has lists of surnames from 1990 and 2000 censuses here. The US Census list for 2010 was, for a time, available on census.socrata.com but that site is no longer running. (You may be able to find it with this Wayback Machine link.) The Social Security Administration provides downloadable lists of first names by gender, ...


16

There is a pretty massive list of given (first) names (~50,000), and it's carefully curated (not machine generated). More details are available on another answer: The best source of international human given (first) names comes from a German computer magazine. The text file has nearly 50k names that are classified by likely gender, and how popular in ...


16

Have a look at OpenFoodFacts, which is a "free, open and collaborative database of food products from the entire world." It contains almost 320,000 items from around the world and may be helpful in solving your issue.


16

The "Common Core of Data" (CCD) maintained by the National Center for Education Statistics (NCES), U.S. Department of Education (ED) is your starting point. For Public Schools (PK-12), see http://www.ed.gov/developers and http://www.data.gov/education including: Public elementary/secondary school listing School district / Local education agency (LEA) ...


16

This question was previously answered for non-profits, and I think the same answer works here at least partially. In addition, you can find all U.S. companies at the Securities and Exchange Commission's EDGAR site. Technical documentation is also available. Information about Canadian companies is accessible through their open data site, but seems to be ...


16

You might want to check out the HUD zip code-county crosswalk (screenshot below). My gut feeling is that this feature would be in the US census TIGER product line https://www.census.gov/geo/maps-data/data/tiger.html maybe here: https://www.census.gov/geo/maps-data/data/relationship.html Please let us know what you find most useful.


16

This list is (and may always be) incomplete. Please add to it when you can. NOAA's ISD data (goes back to ~1900 in some cases): https://www.ncdc.noaa.gov/isd (home page) ftp://ftp.ncdc.noaa.gov/pub/data/noaa/ (direct link to data, some stations include multiple report per hour in some cases) https://www1.ncdc.noaa.gov/pub/data/noaa/isd-lite/ (direct link ...


15

OpenStreetMap's database has the ATM tag. Through the Overpass API, you can quickly access the Points of Interest: Map of ATMs in Paris (Overpass API). Click on the points to see additional metadata (mainly the name of the operating bank). Bonus: there is a good thread on XAPI call for all ATMS in OpenStreetMap's own Q&A plattform.


15

hrecipe (and microformats in general) are the bees knees and lucky for you are widely employed across the web; here's a list of sites actively publishing hrecipes in the wild; you can scrape and parse as you please! http://www.eat-vegan.rocks/ http://funcook.com/ http://www.therecipedepository.com http://sabores.sapo.pt/ http://www.epicurious.com/ http://www....


14

The USDA now provides an Open API for the National Nutrient Database for Standard Reference using data.gov. You need a data.gov API key in order to access this, and requests are sent to api.data.gov using the REST protocol.


14

There are several studies on the economical impact of Open Data. The most recent I know of is a study done in 2011 on data held by all public bodies in the European Union called Review of Recent PSI Re-Use Studies Published [docx] (PSI stands for Public Sector Information), the study is also know as the Vickery study. One of its main findings is that the EU'...


14

Might want to double check the license, but the baseline standard is the CMU Pronunciation dictionary, which is freely downloadable and also ships with many NLP libraries, like NLTK (python). For out-of-vocabulary words, I've had great success with Sequitur G2P, which is both trainable and under the GPL. edit: note that CMUDict (and many other speech ...


14

There are several different potential sources of information. I don't think any are completely comprehensive and few would count as strict "open data": apart from Open Access titles, licensing is likely to vary between publishers. Having said that you could look at some of the following sources: Springer API Nature Linked Data Platform PubMed API PLOS API ...


14

While you will find some exceptions, sharing archived tweets is against Twitter Developer Policy. Take all reasonable efforts to do the following, provided that when requested by Twitter, you must promptly take such actions: Delete Content that Twitter reports as deleted or expired; etc If a user deletes their own content, the archive ...


14

There is a list of soccer datasets and APIs here from this related SO question. Hope that helps. EDIT: Quoted text from the article linked above, as requested: openfootball has started a free (open source) public domain football database. The data is historical data, meaning no lives scores but the data does include the schedule, teams and players ...


14

Have you seen WikiQuote by the Wikimedia Foundation? There is an API endpoint at http://en.wikiquote.org/w/api.php which uses the standard MediaWiki API for there are API clients in many different languages. EDIT: Two WikiQuote API-specific links: https://stackoverflow.com/questions/13762688/wiki-quotes-api and http://bwgz57.wordpress.com/2013/02/14/in-...


Only top voted, non community-wiki answers of a minimum length are eligible