1

I am wondering if it is possible to view outlink categories of a given article in Wikipedia? For example, consider the Wikipedia article about 'Data Mining'. What are the outlink categories of "Data Mining" article and how to find them in the Wikipedia page?

According to the paper "Using Wikipedia knowledge to improve text classification "; the outlink categories and their count of "Data Mining", "Machine Learning" and "Computer Network" are as follows.

Outlink categories and their count

However, I still could not discover how they obtained these outlink categories and their counts. Please help me.

Improve this question
4
  • 1
    "The values correspond to the numbers of out-link articles which belong to the corresponding category"
    – Stanislav Kralin
    Commented Dec 28, 2018 at 17:48
  • 1
    Basically, like this
    – Stanislav Kralin
    Commented Dec 28, 2018 at 18:11
  • @StanislavKralin Thank you so much. This is exactly what I was looking for. However, I would like to do this offline. Do you know which data dumps of Wikipedia I should download to facilitate this task?
    – EmJ
    Commented Dec 28, 2018 at 23:33
  • @StanislavKralin Please let me know if you know an answer for this stackoverflow.com/questions/55869194/… Looking forward to hearing from you. Thank you very much :)
    – EmJ
    Commented Apr 28, 2019 at 0:40

1 Answer 1

Reset to default
2

Just get the list of linked articles, get the categories for each and sum up. If you want to do it with the official API in a single request (well, single sequence of requests because the data will be too large for one response so you'll have to use query continuation, it would look like this:

https://en.wikipedia.org/w/api.php?action=query&format=json&prop=categories&titles=Machine%20learning%7CComputer%20network%7CData%20mining&generator=links&clshow=!hidden&cllimit=max&gpllimit=max

Improve this answer
9
  • Thank you very much. However, I am not clear what that last url represent. Can you please tell me what that is?
    – EmJ
    Commented Dec 28, 2018 at 23:33
  • 1
    It is a call the the Wikpedia API. Here's the sandbox version. It does all kinds of batching which makes it a bit complex. A less effective but simpler equivalent is for each article fetch all the links (example)...
    – Tgr
    Commented Dec 29, 2018 at 0:53
  • 1
    ...and then for each link fetch the non-hidden categories of the linked article (example).
    – Tgr
    Commented Dec 29, 2018 at 0:55
  • 1
    The minimum-effort version is downloading the SQL dumps for the pagelinks and categorylinks tables.
    – Tgr
    Commented Dec 29, 2018 at 22:03
  • 1
    The page_props dump can tell you which pages are disambiguation pages (you can probably use the categorylinks or templatelinks tables to the same effect as well). Beyond that, there isn't any easy way as far as I know - you'd have to do some lightweight parsing to locate all the bullet points on the page and then if they have multiple links hope that the first is the disambiguated page.
    – Tgr
    Commented Dec 31, 2018 at 5:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.