4

Recently I participated in a digital humanities hackathon where we created programs for automatic handling of texts. For example, one program involved automatic detection of Biblical phrases in modern prose and poetry. For this and similar projects, we needed quick access to text in a table format, such as, a table of all verses in the Bible. Although such table can be created automatically from Wikisource using their API, it is much more convenient to be able to download the table itself in a computer-readable format, such as comma-seperated-values.

The problem is, a CSV file of the Bible (or other large texts) would be too large to put in a Wikisource page. Also, the Media-Wiki software does not allow to upload CSV or any other database files.

So, my question is: do you know of a wiki-like website where uses can create and edit data tables in a format that is convenient both for human editors and for computers?

migrated from webapps.stackexchange.com Apr 27 '14 at 17:04

This question came from our site for power users of web applications.

  • Have you seen datatables.net ? – philshem Apr 29 '14 at 7:33
  • @philshem this looks like a plugin - not a Wikipedia-like website... – Erel Segal-Halevi Apr 29 '14 at 14:54
  • i would think the bible in csv would be a collection of csvs...granted you could make one, but to your point, its huge.... – albert Apr 29 '14 at 17:57
3

scraperwiki lets you do this, plus push to ckan databases. there are limitations for free accounts. it will break your huge file up into multiple csvs.
google drive also lets you do this, a little bit more limited, but you can still do what you want by creating a new doc (spreadsheet) and then importing your data.
tabula is another sweet tool, published by knight foundation, that you can use on your desktop...i've had minimal problems with file size, but for each one, if i simply wait it out, as in let the program consume my resources as much as it needs....it always works.

as for hosting: datahub.io, google drive, will host off the top of my head. so i guess on that note, amazon's free service and dropbox too

  • I uploaded my table to ScraperWiki, but didn't find a way to edit it collaboratively online (it is possible to code online, but not edit the raw text, nor share with other users). Tabula seems to be a program for extracting tables from PDF files - not for collaborative editing. Currently Google Drive looks like the best option. – Erel Segal-Halevi May 11 '14 at 11:21
  • you can always use scraperwiki to pull the data out, then save it to google drive. whatever works for you. – albert May 11 '14 at 17:51
4

A hopeful candidate is Dat, which aspires to become what Git (and GitHub) is for code. With this in mind, searching for "git for data" yields a lot of interesting results. In that context, the article We Don't Need a GitHub for Data is an good read which stresses the thought that it's (almost) never data that needs versioning, but data transformations - which usually are code...

Back to now: a proven way to share (non-big) data in a collaborative way right now is just to use GitHub. A good example is the Cooper-Hewitt's Collection Database, a collection of metadata on modern art exhibits. Practical note: try to use a least-surprise solution when creating the data structure. A good way might be to follow a published recommendation like the Tabular Data Package (previously called Simple Data Format) published by the Open Knoweldge Foundation.

And for the scientific data publisher: The Dataverse Network Project aims to become a repository of citable scientific data, analoguous to text publications that receive a unique identifier alreday.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.