3

If a government or a company publish their data as Open Data, you don't have a (theoretical) reason to not trust their data.

But what happens when the data is a result of crowdsourcing. Can we trust them with the same way we trust the data from official publishers? Is there any way to evaluate them? Can we use them on research or contests about Open Data?

A recent example: A local municipality has published a dataset of points of interest. Wikimapia has also a dataset for the same area but it is bigger and with more details than the official one. If I want to create an application, I can choose the dataset from the municipality and I will have an official "stamp" for the data and better "social acceptance", but my application will be misfunctioned. Or I can choose the "unofficial" dataset from crowdsourcing and create an application without the official "stamp".

  • I think your assumption that government data is more trustworthy than crowdsourced data is highly culturally-dependent. Not sure what your reference to "theoretical" is supposed to mean in that first sentence. – Megan Squire Jan 23 '14 at 14:13
  • If a gov publishes data about the coords of all the local park it will do it in base of official data. With crowdsource, I will have to find the signal with a GPS or Google maps and use this data. I use the word theoretical for the reason you said. Gov data isn't always trustworthy. – Tasos Jan 23 '14 at 16:03
  • Even if we work under the assumption that a government is publishing data to the best of its ability, it may lack the resources (monetary, human, or other) to provide "up to the moment" data, regardless of the official nature of the government. As with any data, it is just a snapshot in time. It is important to not only check the date the publishing, but also be aware of the frequency. Frequency of publication and available resources for collection is one area where crowdsourced data may result in higher quality data. – RyanDalton Jan 26 '14 at 0:32
2

I aggregate datasets from different sources together frequently. I sometimes use a hybrid database/table representation, that follows some adaption of this basic approach:

  1. Split the fields into two table schemas. A. one for things that are invariants (e.g., place name, ISO codes, etc) B. one for things that are variants (e.g., population, coordinates, etc). C. the 2nd table schema uses a foreign key to tie its records to the first table schema.

  2. I then make an instance of the 2nd table for each dataset source.

  3. When querying, I join the first table with an instance of the second table depending on which data source I want to use.

I avoid blending data for the same field from different data sources if there is a potential to be different, and I hate the idea of adding N fields for the same value in one table to cover each dataset. This way, if I decide to add a new dataset source, I can simply create another instance of the table schema. Other benefits I have:

  1. Don't have to update existing table schemas.
  2. Don't have to re-optimize storage of in-production databases.
  3. Don't have to modify queries in my middle-ware.
0

Why not have the possibility to have more than one stamp? A dataset can then have

  1. the 'official' stamp
  2. the 'social acceptance' stamp
  3. both 1. and 2.
  • Sometimes it isn't feasible. What if the same data, has different values for each database? You cannot use both of them. – Tasos Jan 23 '14 at 13:04
  • If the same data has different values for each database, then it is not the same data. – philshem Jan 23 '14 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.