Hot answers tagged

6

The wikitextparser Python module has good support for converting tables into structured formats. Example: import wikitextparser as wtp import requests import json r = requests.get('http://en.wikipedia.org/w/index.php?title=' + 'List_of_country_calling_codes&action=raw') wt = wtp.parse(r.text) print(json.dumps({'root': wt.tables[1].getdata()[0:3]}, ...


6

here's how i automated the process within R setwd( "C:/path/to/PDFs" ) year <- 2008 domain_name <- "anthem.com" archive_org_captures <- readLines( paste0( "http://web.archive.org/cdx/search/cdx?url=" , domain_name , "/about/&matchType=host&limit=10000000&from=" , year , ...


4

Here is one way, although the API probably has more efficient methods. You can append * to the end of a URL in the Wayback Machine and it will return all of the saved URLs for that domain. The link below does this, although you can't see the asterisk because markdown is dumb. This is actually a great case for why its dumb. After wasting a few minutes ...


3

Sometimes it pays off to look one step ahead, i.e. to check if the items and its properties represented by a Wikipedia table are also available in the Wikidata knowledge base. Depending on the data, instead of parsing the wiki text table, it might be easier to query Wikidata for that data. As an example, when trying to extract - say - country calling codes ...


2

I don't usually do things by manually, but if you only need to do it once, it's sometimes cost effective. In this case, you can go to the Edit page and copy the text. Since it's formatted as a table, you can use command line tools (or a text editor like Notepad++ to parse into a CSV). .... See also the [[List of television stations in the United States by ...


2

another option, and imo the easiest to implement, although that comes with a tradeoff in regards to owning your data outright. sign up for basic (free) scraperwiki.org account, and then log in. select create a new dataset, select extract data tables, then place the wikipedia url (any url) into the input form control and click extract tables. now your ...


2

Using the official wayback_machine_downloader tool: wayback_machine_downloader https://www.anthem.com --from 2005 --to 2008 --only "/\.(pdf)$/i"


1

HTML is machine-readable/can be parsed, so not sure what you mean by "unprocesable". The document containing the information you desire may as well be valid markup, but more importantly proves its parseable. Alternatively, if you view source, there is a script html-dfn.js, and if you follow that you can see that it calls xrefs.json, which I'm assuming ...


1

After long hours of searching, I could not find one. The short answer was to crawl many different websites, separate the data, and use multi-nomial bayes or a radial basis function depending on how awful the results were to generate my own corpus via cluster/grouping with the manually classified data forming a kernel.


1

TableTools2 https://addons.mozilla.org/en-US/firefox/addon/tabletools2/ It's a Firefox addon that enables right-click copying of the table like this: It works quite well and is very stable.


1

You can try with http://pasty.link/ that it has ad hoc function to export in csv and json file.


Only top voted, non community-wiki answers of a minimum length are eligible