4

I have been thinking about how it would be possible to crowdsource the collection of information. There are already many data sources, but often for the desired application the data is incomplete and practically impossible for a single person or a group to complete the missing informations. The good solution for this problem, in my opinion, is to unify the data source and enable end-users ("data consumers") to fill the data themselves.

In order to clarify my question I'll give two examples:

The first is the car driver identification from the car's licence plate. These data are already available for law enforcement agencies but let's say we want to establish in parallel the data source which would be filled in by end users. Let's say you know who is the driver of a car with certain licence plate and you also noticed that car driving carelessly close tho school area and you want to make this information public. Such an application might not be a good example from the standpoint of privacy, but the idea is to establish the data source filled in by other drivers.

The second example is the data source from which you could retrieve food's nutrition facts by its barcode. There already are some data sources but there will be always new products and the it is hard for a centralized system to catch up with new products in the market. So the idea here is to add information by the end user if it doesn't exist in the database.

So to summarize the idea in the form of questions:

  • Is it practically possible to establish a unified dataset which could be filled in by end users (if yes, how)?
  • Is there already any initiative dealing with this idea?
  • Is there any algorithm (such as PageRank for web pages) to "rate" crowdsourced data by credibility and relevance?
  • Re: careless driving - You'll be sued for libel. Contact your lawyer for more information. Re: crowd sourcing in general - TANSTAAFL. You will get spammed into oblivion before anything useful emerges from the cr@p. Look at Wikipedia - it's crowdsourcing at its best, but it took a great deal of sweat from gullible hardworking editors to turn a would-be p0rn site into an encyclopaedia. – Deer Hunter Jun 18 '13 at 12:22
  • @DeerHunter : IMDB and CDDB were originally crowdsourced. CDDB was forked as OpenDB. (and not necessarily porn, but IMDB was originally the "Actress List", and then later came actors, directors, etc.) – Joe Jun 18 '13 at 13:01
2

Indeed, quite typically, for transport data, it can be very interesting to gather open data and users feedbacks (tweets, posts, ...) to improve quality and restore confidence towards public services.

There are some tools mixing Open Data and crowdsourcing: OpenStreetMap, OpenEcoMaps

At some midpoint between Open Data and crowdsourcing, you have FixMyStreet: The data released by the users can be used to hydrate future open data DB's.

Btw, see this blog post: http://blog.okfn.org/2011/05/23/can-crowdsourcing-improve-open-data/

2

There are several ideas that go together with crowdsourcing:

  • Crowdsourcing exploits non-monetary motivation.
  • To combat spam, and other kinds of opportunistic behavior, you need to have community moderation. Community moderation practically forces you to forego using any ulterior motives (such as spite in your careless driving example). What is left is:

    • vanity (community-assessed reputation in StackExchange),
    • influence (Wikipedia-style official positions with all the associated cabal stuff),
    • genuine desire to help the world,
    • hobby-level obsession with a particular subject (as Joe helpfully demonstrated, actresses in the case of IMDB).

The strength of the community and its motivation determine the viability of data crowdsourcing. If there are no people who are genuinely obsessed with accuracy of nutritional content information (your second example) you're truly hosed, since data crowdsourcing will be unviable.

Users should be able to get some value from the dataset right away. If you don't provide ability to enter arbitrary queries, they lose a bit of their motivation.

If you plan to monetize the dataset (I hope you don't), you'll be surprised to learn that the community has evaporated overnight and you'll be forced to hire people to do the stuff they used to do for free.

Now, to feasibility: acquiring the data and verifying them should be cheap in terms of time and money (preferably free as far as money is concerned). That is, users don't have to buy barcode readers or pay-per-view to fix some fact.

Last, be prepared to contend with lawsuits from all directions and excruciating pressure (FUD, lawsuits, sabotage) from commercial providers of competing datasets.

0

There is one source to quality that I don't believe has been mentioned -- duplication of effort.

As an example, the reCAPTCHA and Galaxy Zoo efforts can assign a level of confidence to their data, as it's been analyzed by more than one person. When there's agreement, they can mark it as valid.

Galaxy Zoo and the other Zooniverse projects track the people entering information, so that should there seem to be someone doing a poor job, they can easy flag all of their entries.

So, to add to Deer Hunter's list, although these would be more accurately described as analysis rather than collection:

  • Gamification : making a game out of the effort (Zooniverse)

  • Tolls : requiring an effort to gain access to something else desired (reCaptcha)

And in the case of reCaptcha, porn and warez have been involved, with groups using the promise of them to get people to OCR the text ... which was being used to protect other sites.

-1

The answer to the second scenario is commercially available through a variety of platforms. See, e.g., http://platform.fatsecret.com/.

  • Link-only answers to non-open data sources are not helpful here at Open Data SE. Please have a look at meta.opendata.stackexchange.com/q/32/70 – Patrick Hoefler Jun 24 '13 at 12:29
  • I'm not sure I understand how that dataset/API is not open. It's free and available for commercial uses and merely requires attribution. – dsmorgan77 Jun 24 '13 at 13:36
  • For example: "1.5 You may not continue to use and must immediately remove or replace (by re-requesting) any Content in your possession or under your control not explicitly identified as being storable indefinitely within 24 hours after the time at which you obtained the Content, or such other time as FatSecret may specify to you from time to time." (Source: FatSecret Platform API Terms of Use) Sorry, but by any definition I know, this is not Open Data. – Patrick Hoefler Jun 24 '13 at 13:45
  • Even worse: "1.2 Your Application must be generally accessible to users. You may require users to log in to your Application but you must not (unless you have entered into a separate agreement with FatSecret or obtained FatSecret's written permission) operate only behind a firewall or only on an internal network (except during the development and testing phase)." – Patrick Hoefler Jun 24 '13 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.