Questions tagged [scraping]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
3 answers
80 views

Wikipedia referenced articles in computer readable format?

A huge number of news related wiki pages contain article references (for example this article on the 2017 Equifax data breach) and I would like to obtain the following data from the references section ...
TheMightyLlama's user avatar
1 vote
1 answer
569 views

How do I get historical data from coinmarketcap.com?

If you scroll down on this page: https://coinmarketcap.com/historical/20150503/ There is something that says: Total Market Cap: $3,863,780,096 Notice the date is given in the URL. How can I loop ...
user23089's user avatar
1 vote
1 answer
63 views

Programmatically retrieving VFR/IFR Charts from FAA

I am trying do the following: Subscribe and be alerted when a new version of the VFR and IFR charts are available from the following websites: https://www.faa.gov/air_traffic/flight_info/aeronav/...
gadgetgirl's user avatar
1 vote
0 answers
102 views

Shapefile for Petrol Fueling stations across Europe

I'm looking for petrol stations across Europe either in a shapefile format, coordinate data as .csv or addresses from a webpage which I can scrape. I have had a look and couldn't find anything that ...
joe_bill.dollar's user avatar
1 vote
0 answers
30 views

GIS Data on US College Dormitories

I am trying to find spatial data on all (or as many as possible) US college dormitories. Specifically, I want to find the location as a polygon or centroid. I have tried a few methods to acquire this ...
gwydion93's user avatar
  • 111
1 vote
0 answers
36 views

Seeking live bar/ restauarant UK data API for ML application

Looking for an API I can use to create a bar/restaurant recommendation app in the UK, with the possibility of commercial use. Will need to train a model on this data which may necessitate the "...
Ben Jones's user avatar
  • 111
1 vote
0 answers
16 views

Options for analyzing twitter hashtag use by various factors - date of account creation, location, username strings and even defined user lists

I am not sure if this question is better suited for data science but I'm mostly looking for an outline of what currently is or is not possible and a direction for where I should learn more. My first ...
MacKenzie's user avatar
0 votes
1 answer
34 views

Identifying parameters accepted by API request

I'm playing around with the data.gov API and was trying to reconstruct a API request from the websearch. ex: COVID-19 datasets filtered by Texas my url looks like https://catalog.data.gov/dataset?q=...
Warren Manuel's user avatar
0 votes
0 answers
15 views

API access to PDF documents comprising approval pages

I am looking to programmatically find, enumerate, and download the PDF files associated with a drug approval package. For example, the drug HADLIMA has several files in its approval package, which ...
Gregory Hassett's user avatar
0 votes
1 answer
29 views

Is there a way to retrieve navigation data from Google Maps e.g. to R?

How would an HTTP request or API request retrieve navigation data from Google Maps? Are there some simple lines of code for that? Let's say I want to give an input of origin and destination address, ...
Rudolf Nyitray's user avatar
0 votes
1 answer
578 views

Scraping sec filings

I'm trying to scrap all the historical filings of a particular Central Index Key (CIK) using python but I get http error codes. First, I got 403 error, I resolved by using requests session and timing ...
user1050619's user avatar