14

The Zooniverse team has had amazing success with the OldWeather project to crowdsource coding of old ships logs to extract climate data. And I believe Sunlight did some crowdsourcing (with Mechanical Turk?) to build the Political Party Time database. Are there other services, libraries, etc., that can be used to support crowdsourced volunteers in performing similar types of tasks in open data applications?

I'm imagining tasks like checking a certain percentage of scraped data against the original source, or doing data entry when some documents are in non-machine-readable PDFs. At a minimum I'm assuming a library or service would be able to deal with the user registration and management, setting up task overlap for accuracy checking and user scoring, etc.

4

PyBossa is an open platform for crowd-sourcing. You write a bit of HTML/JS that is the microtask. They have examples including PDF transcribing. Features include user registration, user credits, statistics.

http://crowdcrafting.org/about

1

If the data is in tables within pdfs, I've created a script in R for splitting the tables into cells and using OCR. You could use the cell images the script creates and use pybossa or mechanical turk to crowdsource each cell and merge them all back into a table.

https://github.com/hansthompson/pdfHarvester

1

Amazon Mechanical Turk is pretty well setup for these kinds of tasks. Of course, you have to pay workers to complete the tasks, but there are plenty of open-source clients to access the service. You might especially want to look at boto for Python and MTurkR for R (note: I am the developer of this package).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.