1

I am wondering if it is possible to view outlink categories of a given article in Wikipedia? For example, consider the Wikipedia article about 'Data Mining'. What are the outlink categories of "Data Mining" article and how to find them in the Wikipedia page?

According to the paper "Using Wikipedia knowledge to improve text classification "; the outlink categories and their count of "Data Mining", "Machine Learning" and "Computer Network" are as follows.

Outlink categories and their count

However, I still could not discover how they obtained these outlink categories and their counts. Please help me.

  • 1
    "The values correspond to the numbers of out-link articles which belong to the corresponding category" – Stanislav Kralin Dec 28 '18 at 17:48
  • 1
    Basically, like this – Stanislav Kralin Dec 28 '18 at 18:11
  • @StanislavKralin Thank you so much. This is exactly what I was looking for. However, I would like to do this offline. Do you know which data dumps of Wikipedia I should download to facilitate this task? – Emi Dec 28 '18 at 23:33
  • @StanislavKralin Please let me know if you know an answer for this stackoverflow.com/questions/55869194/… Looking forward to hearing from you. Thank you very much :) – Emi Apr 28 at 0:40
2

Just get the list of linked articles, get the categories for each and sum up. If you want to do it with the official API in a single request (well, single sequence of requests because the data will be too large for one response so you'll have to use query continuation, it would look like this:

https://en.wikipedia.org/w/api.php?action=query&format=json&prop=categories&titles=Machine%20learning%7CComputer%20network%7CData%20mining&generator=links&clshow=!hidden&cllimit=max&gpllimit=max

  • Thank you very much. However, I am not clear what that last url represent. Can you please tell me what that is? – Emi Dec 28 '18 at 23:33
  • 1
    It is a call the the Wikpedia API. Here's the sandbox version. It does all kinds of batching which makes it a bit complex. A less effective but simpler equivalent is for each article fetch all the links (example)... – Tgr Dec 29 '18 at 0:53
  • 1
    ...and then for each link fetch the non-hidden categories of the linked article (example). – Tgr Dec 29 '18 at 0:55
  • 1
    The minimum-effort version is downloading the SQL dumps for the pagelinks and categorylinks tables. – Tgr Dec 29 '18 at 22:03
  • 1
    The page_props dump can tell you which pages are disambiguation pages (you can probably use the categorylinks or templatelinks tables to the same effect as well). Beyond that, there isn't any easy way as far as I know - you'd have to do some lightweight parsing to locate all the bullet points on the page and then if they have multiple links hope that the first is the disambiguated page. – Tgr Dec 31 '18 at 5:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.