24

Wikidata is a new Wikimedia project:

It centralizes access to and management of structured data, such as interwiki references and statistical information.

This data would be of enormous interest to the Open Data community. Does anyone know if it is possible to download a complete dump of the Wikidata database?

Update: Thanks to everyone who has pointed me to the Wikidata wiki dumps. However, I am interested in a more usable data format that doesn't require parsing to get to the actual data. An RDF dump, for example, would do nicely.

  • 2
    Not sure if you noticed that you can already download some data dumps: dumps.wikimedia.org/wikidatawiki Most of these files lack some decent description but as you can see, they're actively uploading new documents. – Robert Smith May 9 '13 at 4:25
  • @RobertSmith, thanks for the link! As far as I understand, these dumps contain the Wikidata wiki, but not the Wikidata data – I know, it sounds a little confusing ;) Am I wrong? If so, which one of the dumps is the one with the relevant data? – Patrick Hoefler May 9 '13 at 8:57
  • btw have you checked this answer? – RSFalcon7 May 9 '13 at 9:05
  • Thanks, @RSFalcon7! However, I'm interested in the data from the Wikidata project, not the database dumps from Wikipedia – though they can also be very helpful in certain cases. – Patrick Hoefler May 9 '13 at 9:39
  • @PatrickHoefler I think you're right but it seemed to me they are just uploading their own data first. – Robert Smith May 10 '13 at 15:10
12

Wikidata RDF dumps have been available since 3-Aug-2013. http://www.mail-archive.com/wikidata-l@lists.wikimedia.org/msg02553.html

If you're interested in topics related to Wikidata, I'd suggest subscribing to their mailing list.

Of course, both Freebase and DBpedia have had much larger dumps available for years.

11

The wikidata dump is already available. As of now, the last (mostly) complete dump is from 5 May 2013 and it includes the dump of pages in the important namespaces (pages-articles.xml).

  • Thanks for the links! However, I'm not interested in the dumps of the wiki pages, but in the dumps of the data collected by the Wikidata project. – Patrick Hoefler May 22 '13 at 17:09
  • 5
    @PatrickHoefler Well, that data is in that dump too. Wikidata stores its data as JSON page content. For example, if you look at the XML, the third <page> there is Q15 (Africa). – svick May 22 '13 at 17:15
6

The Wikidata database is currently offered for download in several different formats:

5

Depending on your use-case Wikidata toolkit is a Java tool that allows you to iterate over all the items in a dump. In fact it auto-downloads the latest dump and translates the entire model into Java objects. This means everything is abstracted and you just have to write the call-back to be applied to each item. I wrote a tutorial on how to use it in conjunction with Python pandas here.

2

I have no information about Wikidata, but those are two ways to download structured information from Wikipedia:

Images and uploaded files are stored elsewhere, also downloadable

Also answered here

  • Thanks for the links! However, I'm especially interested in the data from the Wikidata project, not the database dumps from Wikipedia – though they can also be very helpful in certain cases. – Patrick Hoefler May 9 '13 at 9:33
  • 1
    This is not structured information, it's semi-structured. It's mostly prose text which has no structure but is supplemented with things like infoboxes which are structured to varying degrees and always require clean up. – hippietrail Aug 10 '13 at 3:04
2

This guy describes three ways of getting the data of which two are already mentioned here

Not mentioned is the WDA tool.

2

Additionally to what is described in other answers, there is a Wikidata RDF dump which is used for the Wikidata query service. The format is described in RDF data format article.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.