26

On 18. April 2016 the ticket "More efficient SPARQL queries for sitelinks" brought us a better way: schema:isPartOf. ?sitelink schema:about ?item . ?sitelink schema:inLanguage "en" . ?sitelink schema:isPartOf <https://en.wikipedia.org/> . Your full query now looks like this: PREFIX schema: <http://schema.org/> PREFIX wikibase: <http://...


17

Try this one: prefix schema: <http://schema.org/> PREFIX wikibase: <http://wikiba.se/ontology#> PREFIX wd: <http://www.wikidata.org/entity/> PREFIX wdt: <http://www.wikidata.org/prop/direct/> SELECT ?cid ?country ?article WHERE { ?cid wdt:P31 wd:Q3624078 . OPTIONAL { ?cid rdfs:label ?country filter (lang(?country) = "...


11

The dependency is supposed to be the opposite, i.e. that Wikipedia should rely on data in Wikidata. This is however not fully implemented yet. If you look at the infobox on this cheese in French, all data comes from Wikidata, but not in English (yet). We are far from full implementation, so expect to see any combination of people pulling data from Wikipedia ...


8

If you want to do this from the dumps, you can use the wb_items_per_site dump (current version, 625 MB; most recent dumps) For each Wikidata item, this contains rows with the corresponding page name on a given wiki project. For individual articles, you can use &action=info on Wikipedia as Ainali suggests, but you have to use the /w/index.php?title=...


7

The Wikipedia Special Export feature does exactly this. More details in this answer.


6

The wikitextparser Python module has good support for converting tables into structured formats. Example: import wikitextparser as wtp import requests import json r = requests.get('http://en.wikipedia.org/w/index.php?title=' + 'List_of_country_calling_codes&action=raw') wt = wtp.parse(r.text) print(json.dumps({'root': wt.tables[1].getdata()[0:3]}, ...


6

Right now, there is no good solution. The options I can think of are: Just download the normal dump whenever it becomes available. This means you won't get daily updates at all. For the English Wikipedia, a new dump is generated about once a month. Use the adds/changes dumps (already mentioned by ojdo). There are two problems with using this: It doesn't ...


6

First of all, if you want the page content, the best way to do this isn't by scraping, but by processing one of the database dumps - you can do this locally, and it'll allow you to get the raw page content rather than rendered HTML. To identify the pages, there are a number of possible options. First, categories - these are comparable to index pages, but ...


6

You could also use the API. The call https://en.wikipedia.org/w/api.php?action=query&prop=pageprops&format=json&titles=Tom%20Hanks will give you the id on Wikidata (as "wikibase_item").


5

You can also use the SPARQL service with query like: prefix schema: <http://schema.org/> SELECT * WHERE { <https://en.wikipedia.org/wiki/Tom_Hanks> schema:about ?item . }


5

As far as I can see, there is no link from the category Japanese people to the article John Andru (at least based on dump from 1 October 2013), so I'm not sure why did the tool tell you otherwise. But the category structure is pretty messed up. For example, John Andru is in the category 18th-century German writers, through chain of categories like this: ...


5

Using this SPARQL end-point: http://dbpedia.org/sparql you can enter this query: SELECT DISTINCT ?YA ?A ?YB ?B where { ?A dbo:influenced ?B. ?A a dbo:Philosopher. ?B a dbo:Philosopher. ?A dbo:birthYear ?YSA. ?B dbo:birthYear ?YSB. BIND( SUBSTR( xsd:string(?YSA), 1, 4) AS ?YA ) BIND( SUBSTR( xsd:string(?YSB), 1, 4) AS ?YB ) } LIMIT 10000 which ...


5

Wikidata doesn't automatically use data from Wikipedia. Data from Wikipedia is uploaded to Wikidata by hand or using some more or less automated tools to do batch uploads but ultimately under control of a user, as PetScan or Wikidata Game. The uploading work is in progress with thousands of articles in Wikipedias without an item in Wikidata and millions of ...


4

There exists an experimental Wikimedia adds/changes dump service. Another approach is generating the differential dump yourself using deltapack, discussed about in this thread in the Wikitech mailing list.


4

According to v0.4 of the Wikidata inclusion syntax, this feature has not been implemented yet. The progress of this feature is available in the Wikimedia bug tracking system.


4

Since the 2016 season is about to start I figure I'll update this to hopefully help some others that stumble upon this page. The best resource for getting free and comprehensive nfl statistics is the nfldb and nflgame python APIs. The developer pulls the information directly from json feeds used by NFL.com's gamecenter data, which is about as accurate and ...


4

By using WikidataQuery API you can get all items which have occupation (P106) soccer player (Q937857) and link to English Wikipedia: enwiki http://wdq.wmflabs.org/api?q=claim[106:937857] AND link[enwiki] Currently 134,037 soccer player Wikidata items are linked to enwiki. After that you can use MediaWiki API for Wikidata with wbgetentities to get the ...


4

The best is to host your mapping directly on Wikidata. The property is https://www.wikidata.org/wiki/Property:P1482 For instance, https://www.wikidata.org/wiki/Q859221 represents Java Swing, and has https://stackoverflow.com/tags/swing as a Stack Exchange tag property: [...] License: public domain


4

There is dbp:iso3166code, but it seems it's not set for a lot of countries.


4

SELECT ?itemLabel ?population ?date { ?item wdt:P31 wd:Q50337; p:P1082 [pq:P585 ?date; ps:P1082 ?population] FILTER NOT EXISTS {?item p:P1082 [pq:P585 ?date_] FILTER (?date_ > ?date)} SERVICE wikibase:label { bd:serviceParam wikibase:language "en". } } Try it If you really need GROUP BY: SELECT ?itemLabel ?date ?population { { SELECT ?item ...


3

You can also use the OpenRefine reconciliation API, which has an additional feature: it will resolve Wikipedia redirects for you. For instance, say you want to retrieve the Qid corresponding to the page LNCS, which is a redirection to Lecture Notes in Computer Science. You can get it as follows: https://tools.wmflabs.org/openrefine-wikidata/en/api?query=...


3

To get a long list of soccer players who where born TODAY, ordered by birthdate , youngest first, Go to http://dbpedia.org/sparql and enter this query into the big text box: PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX dbo: <http://dbpedia.org/ontology/> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> SELECT ...


3

According to DBpedia as Tables, DBpedia uses the following data to create the tables: Each instance is described by its URI, an English label and a short abstract, the mapping-based infobox data describing the instance (extracted from the English edition of Wikipedia), geo-coordinates, and external links. DBpedia basically provides two different types of ...


3

It wouldn't give you complete updates, but there is also dbpedia live. Dbpedia only covers the stuctured text boxes in the wikipedia entries. It is monitoring the changes on Wikipedia though.


3

And just to add to the others, now it is available on English Wikipedia, and you would use it like this: {{#property:P1096|from=Q313}}


3

Sometimes it pays off to look one step ahead, i.e. to check if the items and its properties represented by a Wikipedia table are also available in the Wikidata knowledge base. Depending on the data, instead of parsing the wiki text table, it might be easier to query Wikidata for that data. As an example, when trying to extract - say - country calling codes ...


3

DBpedia The answer provided by @svick will probably be most useful one in the long run, but for the time being dbp:iso3166code is unusable. However, I noticed that (apparently) all the country pages are the targets of redirects of this kind: dbr:ISO_3166-1:DO dbo:wikiPageRedirects dbr:Dominican_Republic Therefore, I can extract ISO 3166-1 code like this: ...


3

(When) will the following be possible: Search in their Phabricator. For a wikipedia page: get the list of all wikipedia links on that page with their respective wikidata IDs in a single query/API call. To some extent, it is possible right now. There are two ways. 1 PREFIX mw: <http://tools.wmflabs.org/mw2sparql/ontology#> SELECT ?link_to ?...


3

You can use the links API property to get the titles of all the links on the page: https://en.wikipedia.org/w/api.php?action=query&format=jsonfm&titles=List_of_American_scientists&prop=links&pllimit=500. Note that this gives only the first 500, so you will need to make multiple queries to get them all, using plcontinue.


Only top voted, non community-wiki answers of a minimum length are eligible