3

I only want to use the link structure and articles’ content of Wikipedia. Therefore, I want to downlaod the dump that contains only the following details.

  • articles content
  • link structure (i.e. re-directed page, out-link categories, disambiguation pages etc.)

Is there any link where I can download them directly?

2 Answers 2

2

Data dumps are at https://dumps.wikimedia.org/enwiki/20181201/ (currently). Article content (source code) is in the pages-articles files (e.g. enwiki-20181201-pages-articles.xml.bz2 is all pages in a single file, enwiki-20181201-pages-articles1.xml-p10p30302.bz2 is page IDs 10 to 30302, enwiki-20181201-pages-articles-multistream.xml.bz2 is a single file with many separate bz2 streams). If you need all pages you are probably better off with the single-file plain bz2 format. Note the data might be in the TB range after uncompressing.

Link information is available as SQL dumps from the same page (towards the end of the list, files with templatelinks, imagelinks, redirects etc. in the name). These are 1:1 dumps of the corresponding MediaWiki tables so see DB schema information for details.

If you need the page content as HTML, your best bet is probably processing one of the Kiwix dumps (they are in ZIM) which contain a stripped down version of article HTML.

2

For the later (categories, pages,...) I recommend using the various dumps that are available from the Wikidata project. The dumps are available at the Wikimedia dumps source. The RDF format is described pretty well in the Wikidata RDF dump description.

Use the Wikidata Toolkit to process it and get the specific information you want. Or import it into a triple store like blazegraph.

For the Wikipedia article content, there are dumps provided by certain mirror servers in different formats and chunks. A promising example is this mirror. Others can be found at the Wikimedia XML dump project

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.