Crowdsourcing Data Submission

Question

I have been thinking about how it would be possible to crowdsource the collection of information. There are already many data sources, but often for the desired application the data is incomplete and practically impossible for a single person or a group to complete the missing informations. The good solution for this problem, in my opinion, is to unify the data source and enable end-users ("data consumers") to fill the data themselves.

In order to clarify my question I'll give two examples:

The first is the car driver identification from the car's licence plate. These data are already available for law enforcement agencies but let's say we want to establish in parallel the data source which would be filled in by end users. Let's say you know who is the driver of a car with certain licence plate and you also noticed that car driving carelessly close tho school area and you want to make this information public. Such an application might not be a good example from the standpoint of privacy, but the idea is to establish the data source filled in by other drivers.

The second example is the data source from which you could retrieve food's nutrition facts by its barcode. There already are some data sources but there will be always new products and the it is hard for a centralized system to catch up with new products in the market. So the idea here is to add information by the end user if it doesn't exist in the database.

So to summarize the idea in the form of questions:

• Is it practically possible to establish a unified dataset which could be filled in by end users (if yes, how)?
• Is there already any initiative dealing with this idea?
• Is there any algorithm (such as PageRank for web pages) to "rate" crowdsourced data by credibility and relevance?

Answer 1

Indeed, quite typically, for transport data, it can be very interesting to gather open data and users feedbacks (tweets, posts, ...) to improve quality and restore confidence towards public services.

There are some tools mixing Open Data and crowdsourcing: OpenStreetMap, OpenEcoMaps

At some midpoint between Open Data and crowdsourcing, you have FixMyStreet: The data released by the users can be used to hydrate future open data DB's.

Btw, see this blog post: http://blog.okfn.org/2011/05/23/can-crowdsourcing-improve-open-data/

Answer 2

There are several ideas that go together with crowdsourcing:

• Crowdsourcing exploits non-monetary motivation.

• To combat spam, and other kinds of opportunistic behavior, you need to have community moderation. Community moderation practically forces you to forego using any ulterior motives (such as spite in your careless driving example). What is left is:

  • vanity (community-assessed reputation in StackExchange),
  • influence (Wikipedia-style official positions with all the associated cabal stuff),
  • genuine desire to help the world,
  • hobby-level obsession with a particular subject (as Joe helpfully demonstrated, actresses in the case of IMDB).

The strength of the community and its motivation determine the viability of data crowdsourcing. If there are no people who are genuinely obsessed with accuracy of nutritional content information (your second example) you're truly hosed, since data crowdsourcing will be unviable.

Users should be able to get some value from the dataset right away. If you don't provide ability to enter arbitrary queries, they lose a bit of their motivation.

If you plan to monetize the dataset (I hope you don't), you'll be surprised to learn that the community has evaporated overnight and you'll be forced to hire people to do the stuff they used to do for free.

Now, to feasibility: acquiring the data and verifying them should be cheap in terms of time and money (preferably free as far as money is concerned). That is, users don't have to buy barcode readers or pay-per-view to fix some fact.

Last, be prepared to contend with lawsuits from all directions and excruciating pressure (FUD, lawsuits, sabotage) from commercial providers of competing datasets.

Answer 3

There is one source to quality that I don't believe has been mentioned -- duplication of effort.

As an example, the reCAPTCHA and Galaxy Zoo efforts can assign a level of confidence to their data, as it's been analyzed by more than one person. When there's agreement, they can mark it as valid.

Galaxy Zoo and the other Zooniverse projects track the people entering information, so that should there seem to be someone doing a poor job, they can easy flag all of their entries.

So, to add to Deer Hunter's list, although these would be more accurately described as analysis rather than collection:

• Gamification : making a game out of the effort (Zooniverse)

• Tolls : requiring an effort to gain access to something else desired (reCaptcha)

And in the case of reCaptcha, porn and warez have been involved, with groups using the promise of them to get people to OCR the text ... which was being used to protect other sites.

Answer 4

The answer to the second scenario is commercially available through a variety of platforms. See, e.g., http://platform.fatsecret.com/.