Suppose that I have some sort of specialized data, perhaps that I've collected myself or been a part of the collection. And suppose that nothing prevents me from handing this data out to people. In what method should I go about distributing/storing this data so that others will be able to find it and use it, whenever this time may be?
Targeting specialised ...
Quandl is an index of datasets. It includes open datasets avaliable for free as well as premium databases which are only available for a cost.
It not only pulls them into one place for easier access, but provides an API for each dataset and packages for them to be pulled into the analysis tool of choice.
Community wiki to collect data sets available to download/seed on bit torrent
DNS Census, the DNS registration dataset snapshot taken in 2013 (compressed ~15GB and uncompressed 157GB).
The DNS Census 2013 is an attempt to provide a public dataset of registered domains and DNS records. It was inspired by the Internet Census 2012 which showed that ...
As of February 2019, you'll find a list of 2600+ open data portals around the world on OpenDataSoft's website. It lists as of now 2600+ portals ranging from the US (obviously) to Afghanistan.
The story behind it:
Working for a SaaS company in need of loads of structured data, our team started to compile a list of all open data portals around the world as ...
I'd recommend looking to see how that specific scientific discipline handles their data.
Some common methods include:
Discipline Registry -- you continue hosting the data elsewhere, but notify the service of the availability of the data, and characteristics so that other people can tell if it's of value to them. (eg, Anthropological, Heliophysics)
Actually, with the data portals everyone is talking about, people forgot about Google. We Google for most things in life, why not data?
Does no-one remember the early days of the web, when we had portals that neatly catalogued web pages. It made sense to have people manually curating lists of pages, carefully categorizing them. Portals were cool. Much ...
You have a few options for real time (or "near real time", which is when you have a delay between the collection & time to serve it, or for those that sample at a lower cadence)
There are a lot of considerations when dealing with 'real time' data:
Who is the intended audience? (and do they already have standards for serving this type of data?)
Is the ...
Take a look at what OpenStreetMap does. There's a page describing the nature of tile server disk usage. If you go up to zoom level 18 worldwide, you're talking about 91,625,968,981 tiles, which would take around 54000GB of disk space, but would mostly never be viewed.
So I'm not sure if it would ever be a sensible approach, but having said that, I heard ...
The other answers so far are all terrific. I'll reiterate one point, and make a new one:
The openness of an API is always important, but when complete, quality bulk data is available some of these access issues become a lot more tolerable. An API is not a substitute for bulk data. The federal government has become very API focused, and many of them have ...
Example: I only know of the company Cloudmade because they provide (a now outdated, but fine at the time) download portal of ready-to-use shapefiles derived from OSM data. Though this service might have helped its competitors, they probably earned much more in terms of visibility, which might spawn e.g. development contracts for custom-made solutions.
Ideally, like @Andrew - OpenGeoCode mentions, you would release it in multiple formats.
I would really suggest you look into organizing it into a e-book written in Markdown and hosted on Github. There are several advantages to this such as a built-in change log, being able to let people to (publicly) fork your document(s) and share their changes with the ...
Our open-date site has links to about 600 useful datasets from US and Canadian government and a handful from the UN. The US federal government datasets are sorted by department.
We [opengeocode.org] maintain a catalog of open data portals across the world. We've categorized them under the categories:
GTFS is what everyone is using. Other vendors, and even open platforms that consume transit data, consume GTFS. Open Trip Planner is an example.
(As an aside: was there a particular need that had that you feel GTFS doesn't address? Perhaps we could give you a more focused answer if you clarified.)
I think it would be foolish to try and replace the NAICS system. NAICS is the federal government's categorization system, and in my experience, it is also in use at the municipal level in the United States. Here's a longer description:
NAICS was developed under the auspices of the Office of Management and Budget (OMB), and adopted in 1997 to replace the ...
The W3C offers a collection of tools that convert from CSV to RDF. However, there is no explicit mention of RDFa in any of these CSV converters. Personally, I'd give the RDF Refine plugin for OpenRefine a try.
Once you have your open data in an RDF format, you could use the RDF Translator to turn it into RDFa.
Well my comment received a number of up votes which I take as a signal of quality and I am posting the links here so they are more visible to future visitors:
opendata.socrata.com - you can upload a number of different file types here, create visualizations, link to them, and take advantage of a very mature set of APIs for data consumption and publishing
There are some great resources here. I would like to add http://knoema.com to this list.
They are possibly the most comprehensive and constantly updating resource. All their data is free to use and links back to source. Plus the data exploration tools and search engine is pretty useful.
Their dataset explorer is here: http://knoema.com/data
I found that ...
recently merged: Registry of Research Data Repositories i.e. RE3
"The consolidated registry contains information for more than 1,130 data repositories that are accessed by over 5,000 unique visitors each month. On average, 10 new repositories are added every week."
"A new REST-API is currently being beta tested that provides the ...
The Guardian data website has a variety and insights in to UK, European and World data.
The site also includes data visualisationa and application you can use.
Regarding this question, I think this post by Denny Vrandečić (the project leader of Wikimedia's Wikidata) is well worth reading: https://plus.google.com/104177144420404771615/posts/cvGay9eDSSK
Denny knows very well what he is talking about. I'll just quote the first sentences as a teaser:
tl;dr - If you publish data, attach the CC0 license to it, but ...
Here are the three options listed on opendatacommons.org, the licenses FAQ also has a bunch of really good information.
Public Domain Dedication and License (PDDL) — “The PDDL places the data(base) in the public domain (waiving all rights)”
Attribution License (ODC-By) — “Attribution for data/databases”
Open Database License (ODC-ODbL) — “Attribution Share-...
Transmodel is a not very widely used format for schedule data (alternative to GTFS).
For real time data (alternative to GTFS-realtime): SIRI is an XML protocol used most heavily in Europe.
You'll want to consider what formats developers are most aware of and any possible performance issues.
TRANSMODEL has been adopted as the European experimental ...
I think the question, as phrased, is impossible to answer well, but I will try.
Q: "How does a consumer know they are getting good data?"
A: Let me answer with more questions. How does a consumer know they are getting a good search result from Google? How do they know when the news is of high quality? It depends. As consumers get more interested and ...
The basics for FOIA are:
be very clear in your request
Remember that while there are government officials who are hostile to records requests, not all of them are. There are many people in government who want to do the right thing.
There are a few websites designed to help you file FOIA requests. In the US, Muck Rock has been ...
The main benefit is moving the attribution requirement from being a legal part of the license, i.e. something that you MUST do, to being a norm, i.e. should that you OUGHT to do (to be polite).
The reason why this is beneficial is because attribution can be difficult:
Data publishers don't always indicate how they wish to be attributed
Where they do, then ...
Another system of interest for your list would be the OKFN Global Open Data Index:
It provides national and sub-national catalog listings and categorization for a variety of data types, and an assessment of how open each country is. Similar to DataPortals.org, and run by the same people.
I don't think there is such a thing. Near as I can tell, before Google came along and prodded them, a machine-readable transit data format for transit agencies didn't exist.
There is a Transit Developers Google Group, you might want to check there as well.
Great question! Cities follow many different paths, but some best practices are starting to emerge. Two particularly helpful guides/roadmaps are published by:
Open Data Field Guide from Socrata
Open Data Handbook from the Open Knowledge Foundation (available in multiple languages)
Both provide a nice how-to guide and future plans for cities and localities ...