Mapping all Wikipedia URLs to Wikidata entity ids

Question

Is there a Wikidata dump that maps all Wikipedia URLs to their corresponding Wikidata entity ids, e.g. the Wikipedia URL https://en.wikipedia.org/wiki/Tom_Hanks to the Wikidata entity id Q2263?

Answer 1

If you want to do this from the dumps, you can use the wb_items_per_site dump (current version, 625 MB; most recent dumps)

For each Wikidata item, this contains rows with the corresponding page name on a given wiki project.

For individual articles, you can use &action=info on Wikipedia as Ainali suggests, but you have to use the /w/index.php?title=PAGENAME&action=info URL format, not /wiki/PAGENAME&action=info

With your example: https://en.wikipedia.org/w/index.php?title=Tom%20Hanks&action=info

Answer 2

You can also use the SPARQL service with query like:

prefix schema: <http://schema.org/>
SELECT * WHERE {
  <https://en.wikipedia.org/wiki/Tom_Hanks> schema:about ?item .
}

Answer 3

You could also use the API. The call https://en.wikipedia.org/w/api.php?action=query&prop=pageprops&format=json&titles=Tom%20Hanks will give you the id on Wikidata (as "wikibase_item").

Answer 4

You can also use the OpenRefine reconciliation API, which has an additional feature: it will resolve Wikipedia redirects for you.

For instance, say you want to retrieve the Qid corresponding to the page LNCS, which is a redirection to Lecture Notes in Computer Science. You can get it as follows:

https://tools.wmflabs.org/openrefine-wikidata/en/api?query=https://en.wikipedia.org/wiki/LNCS

Retrieving Qids of redirected pages will not work with the SPARQL and MediaWiki API.

Answer 5

First, download wikidata json dumps from https://www.wikidata.org/wiki/Wikidata:Database_download .

Then start reading the json file e.g. using Python's json library and for each entity, find entity['sitelinks']['enwiki']['value'] which gives you the page title.

Then replace space with _ and concatenate it to wikipedia.org/wiki/...

Now you have the complete URL.

Answer 6

I created a Python package and command line tool to deal with the issue called wikimapper. It can be installed via pip install wikimapper. It uses the Wikipedia SQL dumps to create an index that then can be used to map many times very fast (much faster than the Wikidata SPARQL endpoint). You could either use one of my precomputed indices and use this sqlite3 database or use the package to map Wikipedia page titles/Wikipedia URLs to Wikidata IDs and vice versa. This should be much faster than going to the network like other answers suggest and also works in 2019.