98
The short answer
DataPortals.org (previously DataCatalogs.org) provides a comprehensive list of open data portals from around the world. Their (meta-)data is in the public domain and available for download as CSV and JSON.
The longer answer
Data that is somehow related is usually grouped in datasets or databases, contained in files (e.g. CSV or ...
36
There's also http://datacatalogs.org/ which maintains a list of open data portals.
You can get a full machine readable list (JSON) from there via the API: http://datacatalogs.org/api/search/dataset
That's only the first 10, to get 100 do: http://datacatalogs.org/api/search/dataset?limit=100
And with full info: http://datacatalogs.org/api/search/dataset?...
22
I'd suggest using JSON Table Schema: http://www.dataprotocols.org/en/latest/json-table-schema.html
It's:
JSON-based
Super simple
Extendable
Here's a rough outline:
# fields is an ordered list of field descriptors
# one for each field (column) in the data
"fields": [
# a field-descriptor
{
"id": "field unique name / id",
"label": ...
18
Quandl is an index of datasets. It includes open datasets avaliable for free as well as premium databases which are only available for a cost.
It not only pulls them into one place for easier access, but provides an API for each dataset and packages for them to be pulled into the analysis tool of choice.
15
As of February 2019, you'll find a list of 2600+ open data portals around the world on OpenDataSoft's website. It lists as of now 2600+ portals ranging from the US (obviously) to Afghanistan.
The story behind it:
Working for a SaaS company in need of loads of structured data, our team started to compile a list of all open data portals around the world as ...
15
There are several studies on the economical impact of Open Data. The most recent I know of is a study done in 2011 on data held by all public bodies in the European Union called Review of Recent PSI Re-Use Studies Published [docx] (PSI stands for Public Sector Information), the study is also know as the Vickery study. One of its main findings is that the EU'...
14
As far as I know, the most complete dataset index is IOGDS: International Open Government Dataset Search. It contains info about >1 million datasets.
Check it out at http://logd.tw.rpi.edu/node/9903
12
While definitely not yet an established practice, another proposal to consider might be Jeni Tennison's Linked CSV (see examples), which adds "prolog lines" to describe the schema used in CSV files.
11
Actually, with the data portals everyone is talking about, people forgot about Google. We Google for most things in life, why not data?
Does no-one remember the early days of the web, when we had portals that neatly catalogued web pages. It made sense to have people manually curating lists of pages, carefully categorizing them. Portals were cool. Much ...
9
Our open-date site has links to about 600 useful datasets from US and Canadian government and a handful from the UN. The US federal government datasets are sorted by department.
http://www.opengeocode.org/cude1.1/index.php
We [opengeocode.org] maintain a catalog of open data portals across the world. We've categorized them under the categories:
Data ...
9
One of the latest published reviews / reports (a few days ago) is the Shakespeare review which contains a chapter on evidence page 20pp:
"...This figure comprises direct economic benefits estimated at around £1.8bn, and a wider social value of PSI conservatively estimated in excess of £5bn..."
There are various comments on this report online from Guardian ...
8
There are some great resources here. I would like to add http://knoema.com to this list.
They are possibly the most comprehensive and constantly updating resource. All their data is free to use and links back to source. Plus the data exploration tools and search engine is pretty useful.
Their dataset explorer is here: http://knoema.com/data
I found that ...
8
The Guardian data website has a variety and insights in to UK, European and World data.
The site also includes data visualisationa and application you can use.
Very enjoyable.
http://www.guardian.co.uk/data
8
databib.org
re3data.org
recently merged: Registry of Research Data Repositories i.e. RE3
"The consolidated registry contains information for more than 1,130 data repositories that are accessed by over 5,000 unique visitors each month. On average, 10 new repositories are added every week."
"A new REST-API is currently being beta tested that provides the ...
8
Dataverse uses the following dataset versioning strategy:
7
While SKOS certainly might be the best way to represent this information it does require more effort than I believe most will be willing to provide. Couldn't we start with a simple, practical, form for providing a data dictionary with another CSV file with 4 mandatory columns. DatasetName, FieldName, FieldValue or Code, ValueDefinition and an optional ...
7
Another system of interest for your list would be the OKFN Global Open Data Index:
https://index.okfn.org/place/
It provides national and sub-national catalog listings and categorization for a variety of data types, and an assessment of how open each country is. Similar to DataPortals.org, and run by the same people.
7
First Caveat, I am a co-founder of opengeocode.org
We have a few resources that might help you:
Catalog of Open Data sites around the world
Specification for open data vocabulary
Detailed specification on representing open data
Papers related to Open Data
6
There are two additional options here.
If you know of an open data portal for a country or topic you are interested in, many allow you to subscribe to their data updates or releases.
If you do not know that a source exists and are looking for one, several aggregator sites are referenced in a recent answer to your related questions on international ...
6
This wiki page from linkedgov.org, The economic impact of open data is:
...collecting case studies and references to the economic impact of open data.
The World Bank Knowledge Repository also has a collection of economic impact studies in the "How would my country benefit from Open Data?" section.
6
The New General Service List (updated just a few weeks ago) is a list of "the most important words for second language learners of English".
It ranks the top three thousand headwords (plus more words within each headword, such as plurals and tenses).
So it won't cover all the words in your list, and it's really indicating "usefullness" rather than ...
5
Please refer the link https://www.udacity.com/wiki/ml/resources
In the above given link there is a link which points to many sites where you can pull the data from open database.
5
The UCI Machine Learning Repository provides databases mostly aimed at machine learning researchers.
Amazon Public Data Sets lists all public data sets that can be seamlessly integrated into AWS cloud-based application (currently 55 data sets are available).
French government data: http://www.data.gouv.fr/
Also, this Quora thread lists quite a few others.
5
I wouldn't dismiss XML so lightly. In the first place, given the natural vagaries of data transmission (especially when considered world-wide), information should be put in XML to simplify error-catching -- the start- and end-point of every datum is unambiguously identified. In the second place, if your XML is self-documenting, as the XML spec intends, it ...
5
The National Information Exchange Model (NIEM) is an XML-based system for defining "data in motion" (i.e., an on-the-wire format.) What distinguishes NIEM is 2 things: it is for standardizing the semantics of the exchange, not just the syntax; and it as much a process model as it is a technical model. That is, it (the NIEM organization) has developed a ...
5
Open Data Barometer recently published this:
http://www.opendatabarometer.org/report/analysis/impact.html
also, the open data working group under the Open Government Partnership will be looking at impact:
http://www.opengovpartnership.org/groups/opendata/
5
datahub.io looks like a good way to find sources for open data.
It is based on CKAN:
CKAN is a powerful data management system that makes data accessible –
by providing tools to streamline publishing, sharing, finding and
using data. CKAN is aimed at data publishers (national and regional
governments, companies and organizations) wanting to make ...
5
Our hope with the Frictionless Data project is to create a set of standards that apply equally to open and non-open datasets. These standards did, in fact, grow out of experience with distributing open data.
Check out the Data Package spec here: http://specs.frictionlessdata.io/data-packages/
4
I have recently started using the Linked CSV proposed standard for generating CSV files from plural data sources. Below is a vocabulary definition for the columns/data types I am using. Perhaps others will find this useful/interesting approach:
http://www.opengeocode.org/cude1.1/LinkedCSV-Vocab.php
Update: the above link throws a 404, however it is ...
4
Have you looked into Linked Open Data (http://vimeo.com/36752317)? http://lod-cloud.net/ has a interactive diagram of open data sets (as at September 2011).
Only top voted, non community-wiki answers of a minimum length are eligible
