4

As part of my thesis on Visual Analytics I need to find a financial dataset to work on. The purpose of the thesis is to analyze a big-data-grade dataset and then visualize it in different ways. However so far I haven't a reliable source of financial big data. Where could one find such data?

migrated from datascience.stackexchange.com May 26 '15 at 6:35

This question came from our site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field.

  • I work in the field and would be happy to give you and answer, but your question is extremely broad. Could you please be more specific about what you are looking for. Examples; Stock price data, earnings data, financial metrics data, exchange level trading data, derivatives data... the possibilities go on and on. One strong suggestions I will make is that you be extremely careful chosing the data you use. The EDGAR XBRL data set from the SEC contains over 15GB of historic data filings, but it is an intensly messy data set. You will spend a HUGE amount of time cleaning it. – drobertson Apr 3 '16 at 0:45
6

I'm an accounting major and I know that for a number of my classes I've had to use various financial databases that have just tons of information available, but they're pay-services. However, I'd check out what your school has and see what can be downloaded.

All publicly traded companies also have to submit annual reports (10-K) and quarterly reports (10-Q) which you can find here: http://www.sec.gov/edgar.shtml

The Yahoo finance API for Python is also extremely convenient: https://pypi.python.org/pypi/yahoo-finance/1.2.1 It includes being able to get historical stock information with some very simple function calls.

Of course, there's no dearth of sources available since the government kind of makes a big deal about financial data having to be public. This was the number two result on my google search for "financial datasets": http://www.kdnuggets.com/datasets/index.html

  • How many records have each of these datasets? – Nicolas Raoul May 26 '15 at 7:59
4

Take a look at Yahoo! Finance for daily, weekly, monthly data on S&P500 and other tickers. However, if you want microaggregated data (e.g. seconds, minutes, hours ---> real big data) you have to pay (a lot!).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy