Get section and page title from a list of articles in Wikipedia via API

Question

I want to get the different sections with their pages from the Wikipedia:Featured articles.

By following this answer, I know I can get page titles of links as follows:

https://en.wikipedia.org/w/api.php?action=query&format=json&titles=Wikipedia:Featured articles&prop=revisions&rvprop=content

The result looks something like this:

{
  "continue": {
    "plcontinue": "5921878|0|Barack_Obama",
    "continue": "||"
  },
  "query": {
    "pages": {
      "5921878": {
        "pageid": 5921878,
        "ns": 4,
        "title": "Wikipedia:Featured articles",
        "links": [
          {
            "ns": 0,
            "title": "...And Justice for All (album)"
          },
...

However, like that I don't get the section name under which each page/link is. For instance, "...And Justice for All (album)" lies under the Music section/header.

Is there a way to get i) the section each page belongs to or ii) the pages belonging to a section in the same or in a different query?

Answer 1

It turns out I can get the section and then the pages/links for that section, in two separate queries like so:

1) Get the sections

https://en.wikipedia.org/w/api.php?action=parse&format=json&page=Wikipedia:Featured articles&prop=sections

With answer:

{
  "parse": {
    "title": "Wikipedia:Featured articles",
    "pageid": 5921878,
    "sections": [
...
      {
        "toclevel": 1,
        "level": "2",
        "line": "Music",
        "number": "21",
        "index": "33",
        "fromtitle": "Wikipedia:Featured_articles",
        "byteoffset": 121596,
        "anchor": "Music"
      },
...
}

2) Get the pages/links for a given section

https://en.wikipedia.org/w/api.php?action=parse&format=json&page=Wikipedia:Featured articles&section=33&prop=links

To get:

{
  "parse": {
    "title": "Wikipedia:Featured articles",
    "pageid": 5921878,
    "links": [
      {
        "ns": 0,
        "exists": "",
        "*": "...And Justice for All (album)"
      },
...
}

I wish I could pipe this into a single query but I can't figure out how to use generators to do just that, maybe it's not supported.

In that case, parsing the wikitext/HTML text as suggested by svick may be the easiest way to go.

Answer 2

No, MediaWiki does not track where in the page does a link come from, so you can't get this information directly from the API.

What you would need to do is to retrieve the article text (either as wikitext or HTML), and parse that yourself.