3

I am looking to obtain data on daily (or even weekly) airfares but am not sure if this type of data publicly exists or not. The bureau of transportation seems to have some of this type of data, but the data is all over the place and not in one readily available file.

Also, would the available data out there probably be only for major airports?

If this data isn't publicly available would the only feasible solution be to set-up a webscraping program to collect this type of data?

Ideally I would like something like this: rita.dot.gov/bts/airfares/compare/airports-metropolitan-area‌​s but instead of quarterly data I would prefer weekly data.

Improve this question

2 Answers 2

Reset to default
1

Daily airfares from where to where on which airline? Every flight everywhere or a specific flight on a specific airline...The data is public just go to a carriers site and do a query. If you a trying to build a travel site and need cost in real time, then you are going to have to build a large scrapper., that's a different question.

You could always use the googles QPX Express API. The is an SO thread with this question, you can use the google API for free up to a certain amount of queries.

Improve this answer
4
  • I am not so worried about getting data for today, but rather would like to see historical data. And I guess I would also like to see as many airports as possible, although if it is historical data, then I can't imagine there being data available for all airports (I hope I am wrong!)
    – RustyStatistician
    Commented Aug 17, 2016 at 15:06
  • How far back are you trying to go? You could build your own archive system specific tailored to collect the data you need in order to answer the question you are posing. Did you check and see if the google QPX Express API handles historical data? Its doing it to some level if it can calc delay percentages..which it does. looks like hotwire API has historical data as well programmableweb.com/news/… Ctrl+F historical
    – risail
    Commented Aug 17, 2016 at 16:06
  • No date in mind, just as far back as I can go, and for free :)
    – RustyStatistician
    Commented Aug 17, 2016 at 16:09
  • "I want all data for all time for free"...can't be answered sorry
    – risail
    Commented Aug 17, 2016 at 16:10
1

I don't know the scope of airports you are after (US only, EU or WW) but I can't think of any public sources.

I think that you will need to find a comparison site like kayak or expedia then use a combination of rselenium (to drive the website) and rvest (to scrape pages).

It depends on the use you are putting the data to, if it is going to be published / sold / concentrated, there will probably be some legal aspects you need to consider.

Improve this answer
3
  • Ideally I would like something like this: rita.dot.gov/bts/airfares/compare/airports-metropolitan-areas but instead of quarterly data I would prefer weekly data.
    – RustyStatistician
    Commented Aug 17, 2016 at 15:37
  • I'm into R, so I would write an R process to collect the above data automated to a week. Unfortunately with that method, you won't get historic data. I can't see a way to get this information, as the airlines keep it pretty under wraps. Also, the added complication is that the price you see depends on how many times you view the same thing on the same browser / ip address, etc.
    – Marcus D
    Commented Aug 17, 2016 at 15:46
  • 1
    Ok. I use R as well. I haven't ever done data scraping and so will have to look into how easy it is to implement.
    – RustyStatistician
    Commented Aug 17, 2016 at 15:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.