Skip to main content
added 541 characters in body
Source Link
albert
  • 11.9k
  • 4
  • 31
  • 58

EDIT 2019-05-24:
Reddit AMA - Chris Gorgolewski from Google Dataset Search confirms much of this in my opinion, like not having dedicated devrel . Google Dataset Search Reddit AMA Quote Highlight - "We don't have a dedicated developer relationship person".

"making these datasets universally accessible and useful"

"making these datasets universally accessible and useful"

EDIT 2019-05-24:
Reddit AMA - Chris Gorgolewski from Google Dataset Search confirms much of this in my opinion, like not having dedicated devrel . Google Dataset Search Reddit AMA Quote Highlight - "We don't have a dedicated developer relationship person".

"making these datasets universally accessible and useful"

added 344 characters in body
Source Link
albert
  • 11.9k
  • 4
  • 31
  • 58

You may want to stop and ask why you should want to be indexed there.

Dataset Search seems like a poorly planned effort to try and corner dataset search. Why the push isn't for open linked datasets/integration with search reveals many details. Performing a search doesn't even tell you how many results there are.

"making these datasets universally accessible and useful"

Try hitting the tab key on the dataset search; neither a11y or useful for keyboard users. Considering the results list navigation cannot even be bothered to use anchor elements for links, I'll go ahead and say the experience for screenreaders is less than ideal too.
Dataset Search results does know how to use anchor elements, but only when it helps them. Scroll to the end of the search results list and you should see:

"Not seeing a result you expected? Learn how you can add new datasets to our index."

Which links to Google Developers Search documentation about datasets; essentially explaining the standards you can utilize to publish your data for it to be properly indexed. Unsurprisingly, all of the example datasets on that page don't render to the page/look broken. You can read more here: Facilitating the discovery of public datasets

Search is wonky to say the least as well; searching for "Virginia" will not return results for the "Virginia Water Lead Levels" dataset that exists in the dataset search tool, but will return numerous datasets for "West Virginia". Numerous results return with incredibly detailed and informative titles, such as "Table 8"; that was sarcasm. Willing to bet Google's tool is working correctly here, these datasets are just poorly labelled. That said, automation is never going to be the end all, particularly in these cases where human inspection/intervention is necessary for curating the desired effects. Considering how much Google values automation is yet another reason why this product, is and will be, supremely inferior to what is available/possible. Poor metadata still works in search, as most of us are looking for vague/whatever sticks results. Dataset search is seeking refined, and rich sources of particular data, and even if the metadata is poor due to user error, poor results are not going to provide much, if any value in this context.

That said, I have a few datasets that have been integrated into Dataset Search without having to do anything on my end.

I uploaded this dataset, Racial Housing Patterns for Norfolk--Virginia Beach-- Newport News, VA--NC MSA in 2014 to datahub.io. I'm assuming datahub.io is using standards and schemas to publish their data that makes it very easy to consume. Also, really pathetic Google can't link to the dataset provider in the results.

Another dataset I have indexed, Virginia Water Lead Levels is coming from data.world. Seems like most of the datasets are being indexed/harvested from data portals, which I will again assume are publishing them with standards and schemas. I see Socrata and OpenDataSoft for numerous entries, so your best bet is to use a portal, preferably running CKAN.

Using a portal that is publishing datasets using standards and schemas properly is the answer you are seeking; Data.world and datahub.io are free to a certain limit, and OpenDataSoft maybe as well.
Publishing datasets using schema.org/dataset, DCAT, CSVW, in JSON-LD, CSV, HTML, etc., is ideal and most likely what the mentioned portals are using.

EDIT/UPDATE:
DCAT is what is powering CKAN datasets in Google Dataset Search, via this tweet which points to ckannext-dcat repository's README.md. The readme has pertinent information about this process.

You may want to stop and ask why you should want to be indexed there.

Dataset Search seems like a poorly planned effort to try and corner dataset search. Why the push isn't for open linked datasets/integration with search reveals many details. Performing a search doesn't even tell you how many results there are.

"making these datasets universally accessible and useful"

Try hitting the tab key on the dataset search; neither a11y or useful for keyboard users. Considering the results list navigation cannot even be bothered to use anchor elements for links, I'll go ahead and say the experience for screenreaders is less than ideal too.
Dataset Search results does know how to use anchor elements, but only when it helps them. Scroll to the end of the search results list and you should see:

"Not seeing a result you expected? Learn how you can add new datasets to our index."

Which links to Google Developers Search documentation about datasets; essentially explaining the standards you can utilize to publish your data for it to be properly indexed. Unsurprisingly, all of the example datasets on that page don't render to the page/look broken. You can read more here: Facilitating the discovery of public datasets

Search is wonky to say the least as well; searching for "Virginia" will not return results for the "Virginia Water Lead Levels" dataset that exists in the dataset search tool, but will return numerous datasets for "West Virginia". Numerous results return with incredibly detailed and informative titles, such as "Table 8"; that was sarcasm. Willing to bet Google's tool is working correctly here, these datasets are just poorly labelled. That said, automation is never going to be the end all, particularly in these cases where human inspection/intervention is necessary for curating the desired effects. Considering how much Google values automation is yet another reason why this product, is and will be, supremely inferior to what is available/possible. Poor metadata still works in search, as most of us are looking for vague/whatever sticks results. Dataset search is seeking refined, and rich sources of particular data, and even if the metadata is poor due to user error, poor results are not going to provide much, if any value in this context.

That said, I have a few datasets that have been integrated into Dataset Search without having to do anything on my end.

I uploaded this dataset, Racial Housing Patterns for Norfolk--Virginia Beach-- Newport News, VA--NC MSA in 2014 to datahub.io. I'm assuming datahub.io is using standards and schemas to publish their data that makes it very easy to consume. Also, really pathetic Google can't link to the dataset provider in the results.

Another dataset I have indexed, Virginia Water Lead Levels is coming from data.world. Seems like most of the datasets are being indexed/harvested from data portals, which I will again assume are publishing them with standards and schemas. I see Socrata and OpenDataSoft for numerous entries, so your best bet is to use a portal, preferably running CKAN.

Using a portal that is publishing datasets using standards and schemas properly is the answer you are seeking; Data.world and datahub.io are free to a certain limit, and OpenDataSoft maybe as well.
Publishing datasets using schema.org/dataset, DCAT, CSVW, in JSON-LD, CSV, HTML, etc., is ideal and most likely what the mentioned portals are using.

You may want to stop and ask why you should want to be indexed there.

Dataset Search seems like a poorly planned effort to try and corner dataset search. Why the push isn't for open linked datasets/integration with search reveals many details. Performing a search doesn't even tell you how many results there are.

"making these datasets universally accessible and useful"

Try hitting the tab key on the dataset search; neither a11y or useful for keyboard users. Considering the results list navigation cannot even be bothered to use anchor elements for links, I'll go ahead and say the experience for screenreaders is less than ideal too.
Dataset Search results does know how to use anchor elements, but only when it helps them. Scroll to the end of the search results list and you should see:

"Not seeing a result you expected? Learn how you can add new datasets to our index."

Which links to Google Developers Search documentation about datasets; essentially explaining the standards you can utilize to publish your data for it to be properly indexed. Unsurprisingly, all of the example datasets on that page don't render to the page/look broken. You can read more here: Facilitating the discovery of public datasets

Search is wonky to say the least as well; searching for "Virginia" will not return results for the "Virginia Water Lead Levels" dataset that exists in the dataset search tool, but will return numerous datasets for "West Virginia". Numerous results return with incredibly detailed and informative titles, such as "Table 8"; that was sarcasm. Willing to bet Google's tool is working correctly here, these datasets are just poorly labelled. That said, automation is never going to be the end all, particularly in these cases where human inspection/intervention is necessary for curating the desired effects. Considering how much Google values automation is yet another reason why this product, is and will be, supremely inferior to what is available/possible. Poor metadata still works in search, as most of us are looking for vague/whatever sticks results. Dataset search is seeking refined, and rich sources of particular data, and even if the metadata is poor due to user error, poor results are not going to provide much, if any value in this context.

That said, I have a few datasets that have been integrated into Dataset Search without having to do anything on my end.

I uploaded this dataset, Racial Housing Patterns for Norfolk--Virginia Beach-- Newport News, VA--NC MSA in 2014 to datahub.io. I'm assuming datahub.io is using standards and schemas to publish their data that makes it very easy to consume. Also, really pathetic Google can't link to the dataset provider in the results.

Another dataset I have indexed, Virginia Water Lead Levels is coming from data.world. Seems like most of the datasets are being indexed/harvested from data portals, which I will again assume are publishing them with standards and schemas. I see Socrata and OpenDataSoft for numerous entries, so your best bet is to use a portal, preferably running CKAN.

Using a portal that is publishing datasets using standards and schemas properly is the answer you are seeking; Data.world and datahub.io are free to a certain limit, and OpenDataSoft maybe as well.
Publishing datasets using schema.org/dataset, DCAT, CSVW, in JSON-LD, CSV, HTML, etc., is ideal and most likely what the mentioned portals are using.

EDIT/UPDATE:
DCAT is what is powering CKAN datasets in Google Dataset Search, via this tweet which points to ckannext-dcat repository's README.md. The readme has pertinent information about this process.

Source Link
albert
  • 11.9k
  • 4
  • 31
  • 58

You may want to stop and ask why you should want to be indexed there.

Dataset Search seems like a poorly planned effort to try and corner dataset search. Why the push isn't for open linked datasets/integration with search reveals many details. Performing a search doesn't even tell you how many results there are.

"making these datasets universally accessible and useful"

Try hitting the tab key on the dataset search; neither a11y or useful for keyboard users. Considering the results list navigation cannot even be bothered to use anchor elements for links, I'll go ahead and say the experience for screenreaders is less than ideal too.
Dataset Search results does know how to use anchor elements, but only when it helps them. Scroll to the end of the search results list and you should see:

"Not seeing a result you expected? Learn how you can add new datasets to our index."

Which links to Google Developers Search documentation about datasets; essentially explaining the standards you can utilize to publish your data for it to be properly indexed. Unsurprisingly, all of the example datasets on that page don't render to the page/look broken. You can read more here: Facilitating the discovery of public datasets

Search is wonky to say the least as well; searching for "Virginia" will not return results for the "Virginia Water Lead Levels" dataset that exists in the dataset search tool, but will return numerous datasets for "West Virginia". Numerous results return with incredibly detailed and informative titles, such as "Table 8"; that was sarcasm. Willing to bet Google's tool is working correctly here, these datasets are just poorly labelled. That said, automation is never going to be the end all, particularly in these cases where human inspection/intervention is necessary for curating the desired effects. Considering how much Google values automation is yet another reason why this product, is and will be, supremely inferior to what is available/possible. Poor metadata still works in search, as most of us are looking for vague/whatever sticks results. Dataset search is seeking refined, and rich sources of particular data, and even if the metadata is poor due to user error, poor results are not going to provide much, if any value in this context.

That said, I have a few datasets that have been integrated into Dataset Search without having to do anything on my end.

I uploaded this dataset, Racial Housing Patterns for Norfolk--Virginia Beach-- Newport News, VA--NC MSA in 2014 to datahub.io. I'm assuming datahub.io is using standards and schemas to publish their data that makes it very easy to consume. Also, really pathetic Google can't link to the dataset provider in the results.

Another dataset I have indexed, Virginia Water Lead Levels is coming from data.world. Seems like most of the datasets are being indexed/harvested from data portals, which I will again assume are publishing them with standards and schemas. I see Socrata and OpenDataSoft for numerous entries, so your best bet is to use a portal, preferably running CKAN.

Using a portal that is publishing datasets using standards and schemas properly is the answer you are seeking; Data.world and datahub.io are free to a certain limit, and OpenDataSoft maybe as well.
Publishing datasets using schema.org/dataset, DCAT, CSVW, in JSON-LD, CSV, HTML, etc., is ideal and most likely what the mentioned portals are using.

Post Made Community Wiki by albert