3

If a government or a company publish their data as Open Data, you don't have a (theoretical) reason to not trust their data.

But what happens when the data is a result of crowdsourcing. Can we trust them with the same way we trust the data from official publishers? Is there any way to evaluate them? Can we use them on research or contests about Open Data?

A recent example: A local municipality has published a dataset of points of interest. Wikimapia has also a dataset for the same area but it is bigger and with more details than the official one. If I want to create an application, I can choose the dataset from the municipality and I will have an official "stamp" for the data and better "social acceptance", but my application will be misfunctioned. Or I can choose the "unofficial" dataset from crowdsourcing and create an application without the official "stamp".

Improve this question
3
  • I think your assumption that government data is more trustworthy than crowdsourced data is highly culturally-dependent. Not sure what your reference to "theoretical" is supposed to mean in that first sentence.
    – Megan Squire
    Commented Jan 23, 2014 at 14:13
  • If a gov publishes data about the coords of all the local park it will do it in base of official data. With crowdsource, I will have to find the signal with a GPS or Google maps and use this data. I use the word theoretical for the reason you said. Gov data isn't always trustworthy.
    – Tasos
    Commented Jan 23, 2014 at 16:03
  • Even if we work under the assumption that a government is publishing data to the best of its ability, it may lack the resources (monetary, human, or other) to provide "up to the moment" data, regardless of the official nature of the government. As with any data, it is just a snapshot in time. It is important to not only check the date the publishing, but also be aware of the frequency. Frequency of publication and available resources for collection is one area where crowdsourced data may result in higher quality data.
    – RyanKDalton
    Commented Jan 26, 2014 at 0:32

2 Answers 2

Reset to default
2

I aggregate datasets from different sources together frequently. I sometimes use a hybrid database/table representation, that follows some adaption of this basic approach:

  1. Split the fields into two table schemas. A. one for things that are invariants (e.g., place name, ISO codes, etc) B. one for things that are variants (e.g., population, coordinates, etc). C. the 2nd table schema uses a foreign key to tie its records to the first table schema.

  2. I then make an instance of the 2nd table for each dataset source.

  3. When querying, I join the first table with an instance of the second table depending on which data source I want to use.

I avoid blending data for the same field from different data sources if there is a potential to be different, and I hate the idea of adding N fields for the same value in one table to cover each dataset. This way, if I decide to add a new dataset source, I can simply create another instance of the table schema. Other benefits I have:

  1. Don't have to update existing table schemas.
  2. Don't have to re-optimize storage of in-production databases.
  3. Don't have to modify queries in my middle-ware.
Improve this answer
0

Why not have the possibility to have more than one stamp? A dataset can then have

  1. the 'official' stamp
  2. the 'social acceptance' stamp
  3. both 1. and 2.
Improve this answer
2
  • Sometimes it isn't feasible. What if the same data, has different values for each database? You cannot use both of them.
    – Tasos
    Commented Jan 23, 2014 at 13:04
  • If the same data has different values for each database, then it is not the same data.
    – philshem
    Commented Jan 23, 2014 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.