Google now offers Dataset Search (beta)

https://toolbox.google.com/datasetsearch

Dataset Search enables users to find datasets stored across the Web through a simple keyword search. The tool surfaces information about datasets hosted in thousands of repositories across the Web, making these datasets universally accessible and useful.

How can I get my dataset indexed there?

and specifically, if I don't want to host the data myself:

Are there any data platforms/portals that allow user uploads and will automatically index my dataset?


(related: top-voted question A database of open databases? and the Dataset Search answer)

  • We stored our gap-filled MODIS Land Surface Temperature data at Zenodo (zenodo.org/record/1135230) and it got picked up automatically. Maybe a place to host your data as well? – markusN Oct 25 at 10:10

You may want to stop and ask why you should want to be indexed there.

Dataset Search seems like a poorly planned effort to try and corner dataset search. Why the push isn't for open linked datasets/integration with search reveals many details. Performing a search doesn't even tell you how many results there are.

"making these datasets universally accessible and useful"

Try hitting the tab key on the dataset search; neither a11y or useful for keyboard users. Considering the results list navigation cannot even be bothered to use anchor elements for links, I'll go ahead and say the experience for screenreaders is less than ideal too.
Dataset Search results does know how to use anchor elements, but only when it helps them. Scroll to the end of the search results list and you should see:

"Not seeing a result you expected? Learn how you can add new datasets to our index."

Which links to Google Developers Search documentation about datasets; essentially explaining the standards you can utilize to publish your data for it to be properly indexed. Unsurprisingly, all of the example datasets on that page don't render to the page/look broken. You can read more here: Facilitating the discovery of public datasets

Search is wonky to say the least as well; searching for "Virginia" will not return results for the "Virginia Water Lead Levels" dataset that exists in the dataset search tool, but will return numerous datasets for "West Virginia". Numerous results return with incredibly detailed and informative titles, such as "Table 8"; that was sarcasm. Willing to bet Google's tool is working correctly here, these datasets are just poorly labelled. That said, automation is never going to be the end all, particularly in these cases where human inspection/intervention is necessary for curating the desired effects. Considering how much Google values automation is yet another reason why this product, is and will be, supremely inferior to what is available/possible. Poor metadata still works in search, as most of us are looking for vague/whatever sticks results. Dataset search is seeking refined, and rich sources of particular data, and even if the metadata is poor due to user error, poor results are not going to provide much, if any value in this context.

That said, I have a few datasets that have been integrated into Dataset Search without having to do anything on my end.

I uploaded this dataset, Racial Housing Patterns for Norfolk--Virginia Beach-- Newport News, VA--NC MSA in 2014 to datahub.io. I'm assuming datahub.io is using standards and schemas to publish their data that makes it very easy to consume. Also, really pathetic Google can't link to the dataset provider in the results.

Another dataset I have indexed, Virginia Water Lead Levels is coming from data.world. Seems like most of the datasets are being indexed/harvested from data portals, which I will again assume are publishing them with standards and schemas. I see Socrata and OpenDataSoft for numerous entries, so your best bet is to use a portal, preferably running CKAN.

Using a portal that is publishing datasets using standards and schemas properly is the answer you are seeking; Data.world and datahub.io are free to a certain limit, and OpenDataSoft maybe as well.
Publishing datasets using schema.org/dataset, DCAT, CSVW, in JSON-LD, CSV, HTML, etc., is ideal and most likely what the mentioned portals are using.

EDIT/UPDATE:
DCAT is what is powering CKAN datasets in Google Dataset Search, via this tweet which points to ckannext-dcat repository's README.md. The readme has pertinent information about this process.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.