The short answer
DataPortals.org (previously DataCatalogs.org) provides a comprehensive list of open data portals from around the world. Their (meta-)data is in the public domain and available for download as CSV and JSON.
The longer answer
Data that is somehow related is usually grouped in datasets or databases, contained in files (e.g. CSV or ...
There's also http://datacatalogs.org/ which maintains a list of open data portals.
You can get a full machine readable list (JSON) from there via the API: http://datacatalogs.org/api/search/dataset
That's only the first 10, to get 100 do: http://datacatalogs.org/api/search/dataset?limit=100
And with full info: http://datacatalogs.org/api/search/dataset?...
Quandl is an index of datasets. It includes open datasets avaliable for free as well as premium databases which are only available for a cost.
It not only pulls them into one place for easier access, but provides an API for each dataset and packages for them to be pulled into the analysis tool of choice.
As of February 2019, you'll find a list of 2600+ open data portals around the world on OpenDataSoft's website. It lists as of now 2600+ portals ranging from the US (obviously) to Afghanistan.
The story behind it:
Working for a SaaS company in need of loads of structured data, our team started to compile a list of all open data portals around the world as ...
Some general advice:
Invest in community managers and evangelists. The worst thing you can do is create a forum or invite feedback and then not have anyone with a mandate to respond to it in an official capacity; I'm seeing this happen right now with the newly launched project-open-data in which they've invited contributions and have no one to actually ...
Actually, with the data portals everyone is talking about, people forgot about Google. We Google for most things in life, why not data?
Does no-one remember the early days of the web, when we had portals that neatly catalogued web pages. It made sense to have people manually curating lists of pages, carefully categorizing them. Portals were cool. Much ...
You have a few options for real time (or "near real time", which is when you have a delay between the collection & time to serve it, or for those that sample at a lower cadence)
There are a lot of considerations when dealing with 'real time' data:
Who is the intended audience? (and do they already have standards for serving this type of data?)
Is the ...
Figuring out the cost for running CKAN boils down to two different categories: the cost of running the software itself, as well as the cost of building and maintaining your open data catalog going forward.
The cost of the software itself is simpler to answer: CKAN itself is free/open-source, and available as a hosted solution.
If you go the open-source ...
Our open-date site has links to about 600 useful datasets from US and Canadian government and a handful from the UN. The US federal government datasets are sorted by department.
We [opengeocode.org] maintain a catalog of open data portals across the world. We've categorized them under the categories:
As you say, http://datahub.io is a particular instance powered by CKAN, so you can't really compare both. In terms of what is particular about datahub.io, it is a community centred and powered site where everybody should be able to host their data, rather than the more default scenario of one or more organizations publishing data on the portal.
It is in the ...
The Guardian data website has a variety and insights in to UK, European and World data.
The site also includes data visualisationa and application you can use.
The Data Hub, powered by CKAN, currently lists more than 6000 datasets, though not all of them are Open Data. These datasets come from all kinds of sources, not just governments and statistics institutes.
There is also the Linked Data community which collects datasets in the Linking Open Data Cloud group on the Data Hub. Most of these datasets are related ...
There are some great resources here. I would like to add http://knoema.com to this list.
They are possibly the most comprehensive and constantly updating resource. All their data is free to use and links back to source. Plus the data exploration tools and search engine is pretty useful.
Their dataset explorer is here: http://knoema.com/data
I found that ...
recently merged: Registry of Research Data Repositories i.e. RE3
"The consolidated registry contains information for more than 1,130 data repositories that are accessed by over 5,000 unique visitors each month. On average, 10 new repositories are added every week."
"A new REST-API is currently being beta tested that provides the ...
Another system of interest for your list would be the OKFN Global Open Data Index:
It provides national and sub-national catalog listings and categorization for a variety of data types, and an assessment of how open each country is. Similar to DataPortals.org, and run by the same people.
I would also add that The World Bank has a very active and well run open data program: http://data.worldbank.org/ In addition, organizations such as the African Development Bank http://opendataforafrica.org/ provide regional data.
Sounds like you could be interested in the Sensor Observation Service (SOS) standard published by the OGC. It's an open standard which describes a service to publish sensor readings and meta data.
I haven't been using it for a while now but I had an SOS server from 52North running successfully for a while.
Try https://geovation.uk/data-sources/ for a pretty comprehensive list of links to sources with crime data, retail data, consumer data, transport data and government data.
You can also refer to this list to fill in the gaps on English Heritage data, environment data and Ordnance Survey data.
I hope this helps!
In Scotland specifically, I would suggest that you have a look at:
Statistics.gov.scot - a very decent repository of publicly available data in Scotland, a lot of spatially aggregated.
NOMIS - spatially aggregated labour market data
UK Data Service - vast catalogue of research / survey data
Neigbhourhood Statistics - ONS product with neighbourhood ...
This is a complex opportunity to which many of us in open data are trying to respond. (Disclaimer: I'm the Evangelist for Data.gov and this answer addresses examples from my work there and at NASA.)
There are two broad kinds of communities to engage in around open data. One is the technical community, like many of you here, of users and consumers of the ...
To add to amercader's answer, if you have some data to publish and are wondering whether to use datahub.io or your own CKAN instance, there are various considerations.
Using datahub.io is free and requires no setup, site hosting, etc.
With your own instance, you can have a data portal 'homed' at your own web address (e.g. data.myorganisation.org). You can ...
There's the IATI Registry which aggregates data related to Aid projects published as part of the IATI initiative. This is obviously rather narrow.
There's also http://africaopendata.org/ which aggregates data from government and elsewhere from across Africa. This hub is community run coordinated by the Open Institute in Kenya.
These might be of interest:
Also cited in other answers:
Africa Open Data
Open Data LatinoAmerica
And some traditional official providers:
International Labour Organization (labour force surveys)
International Monetary Fund (IMF)
World Health Organization
Here's the rest of my list with ...
The UCI Machine Learning Repository provides databases mostly aimed at machine learning researchers.
Amazon Public Data Sets lists all public data sets that can be seamlessly integrated into AWS cloud-based application (currently 55 data sets are available).
French government data: http://www.data.gouv.fr/
Also, this Quora thread lists quite a few others.
Open Data looks a bit like a jungle at first but you'll find tons of awesome data and tons of ideas to create stuff with it!
The first questions would be: are you searching some particuliar data or are you just curious? what do you want to do with those data?
Then I would say:
the first step is find some data near you. The OKFN runs a service called ...
I've compiled an index/catalog of government open data portals around the world. So far, there are over 700 sites in the catalog. I've setup the catalog for crowdsourcing, so feel free to send us your suggested portals to add to the catalog.
You can download the index as a CSV file as well:
The Registry of Research Data Repositories just issued a press release announcing their existance, but their scope isn't necessarily all data, only 'research data'.
I assume they also contain closed data, based on one of their metadata fields:
Type of access to data: open
Here is a list of US portals that is slightly more up to date than datacatalogs.org, if one is missing, submit a pull request (but also notify datacatalogs.org!) https://github.com/sunlightpolicy/opendata/blob/master/USlocalopendataportals.csv
A list of portals, by vendor: https://github.com/tlevine/openprism/blob/gh-pages/src/index.js
Chicago's 'Hidden Open ...