23
As it turns out there are several resources one can use for all main cryptocurrencies so I will post here the most relevant and flexible I was able to gather.
All Cryptocurrencies
Coinmetrics
https://coinmetrics.io/data-downloads/
This page has data for: Bitcoin, Litecoin, Ethereum, NEM, Decred, ZCash (transparent transactions only), Dash, Dogecoin, ...
13
No leak
The Panama Papers were not leaked at all. They were handed over to a German media company which probably paid a good price (translation/summary here) for that deal.
This means, the panama papers generate a lot of media attention, and probably already paid off for the Süddeutsche Zeitung but it wont have much impact as long as the full papers are ...
12
What you are looking for is the Issuer Identifier Number (IIN), which are the first 6 digits of the credit card. Like @pndfam05 mentioned, there are many commercial databases that charge for more than 10 lookups.
But, there is binlist.net that offers either
a data dump on their github repo (raw CSV link), (it seems it is a limited file)
or a freely ...
10
The Panama Papers database has just been published at this link
https://www.occrp.org/en/panamapapers/database.html
This ICIJ database contains information on almost 320,000 offshore entities that are part of the Panama Papers and the Offshore Leaks investigations. The data covers nearly 40 years up to the end of 2015 and links to people and companies in ...
9
I had a business need for this. A quick wikipedia read
https://en.wikipedia.org/wiki/ISO/IEC_7812#cite_note-ANSI-3
showed that the American Banker's Association was the ultimate holder of the "official" version of the dataset. (It only took 2 holes of data collection golf (phonecalls); a longer one to ANSI where I ended up getting the phone/email for the ...
9
There are many good resources described in the comments to your question (1 & 2). One that combines many different types of financial products and has some open access is Quandl.
Data browser tool
API documentation
Quandl provides a single easy-to-use API for stock prices and fundamentals. Coverage includes end-of-day prices, harmonized fundamentals, ...
9
While the data is only as good as the community contributes, OpenStreetMap (OSM) has some ATM data.
In OSM, points can be tagged with "amenity=atm". You can extract these nodes using these steps:
Download OSM data. This page has a "British isles" extract, among others. This page explains more generally how this kind of data is packaged.
Download and ...
9
I work at Quandl and stumbled upon this post. I'm happy to share an update: we now have two databases which contain OHLCV (open-high-low-close-volume) data for bitcoin and other cryptocurrencies:
1.Brave New Coin Digital Currency Indexed EOD: https://www.quandl.com/data/BNC2 Contains historical global price indexes for a number of cryptocurrencies
2.Brave ...
8
There is no comprehensive source of open data as you describe, but there are public and restricted-access open data sets that can illuminate certain aspects of private company finances. Privately-held companies often disclose information to the public in accordance with federal, state, and local laws. Some of this information directly involves company ...
8
You should take a look at Rank and Filed, a relatively new (Feb 2014) site about EDGAR filings, with lots of data export options.
7
I don't know if the SEC has datasets for what you are looking for, but here is a link to the SEC's open datasets home page:
https://www.sec.gov/open/datasets.shtml
Below is a link and contacts to a university research paper (2012) on insider trading. You may want to consider contacting the researchers on where they got their data and methodologies on using ...
7
Update: (October 2017) Kimono Labs API no longer available
Kimono Labs released a free open API for current and recent SEC filings with financial statements and valuation, profitability, leverage and liquidity ratios. You can find it here: http://kimonolabs.com/sec/explorer
7
Dev.Last10K.com has a RESTful API for SEC filing data.
Disclaimer: I work for Last10K.com
7
I'm an accounting major and I know that for a number of my classes I've had to use various financial databases that have just tons of information available, but they're pay-services. However, I'd check out what your school has and see what can be downloaded.
All publicly traded companies also have to submit annual reports (10-K) and quarterly reports (10-Q) ...
6
I've used the Yahoo Finance API in the past, but apparently it doesn't exist (although it works fine). There is no documentation.
For a site with documentation, I'd suggest Open Exchange Rates, which contains forex values that you are looking for, via an API.
There are many API methods and in particular for your case, you can request a time series.
From ...
6
In the U.S., bankruptcy data is found in several places. Depending on which parameters you decide are important for your tool and algorithm, some of the following datasets should be helpful to you:
Recent bankruptcies, findings, and maps from the U.S. Courts
Bankruptcy statistics from the U.S. Department of Justice
Public bankruptcy cases from the U.S. ...
6
After taking a look at the source code of the widget on a sample recipe page (selected at random by me), it appears that they are using http://corp.groceryserver.com/ as a data provider
For example, the recipe at http://allrecipes.com/Recipe/Sherry-Braised-Beef-Short-Ribs/Detail.aspx makes an AJAX request to http://allrecipes.groceryserver.com/groceryserver/...
6
As @philshem suggested I wrote some python to gather the data
import pandas as pd
import requests
import re
from bs4 import BeautifulSoup
# Create date range for historical snapshots
Date = pd.date_range(start='20130428', end='20171210', freq='7D').strftime('%Y%m%d')
# Retrieve market cap value in dollars
market_cap = []
for date in Date:
# Retrieve ...
5
The SEC publishes standardized machine readable data in XBRL format on public companies and other entities, such as mutual funds. Public company financial statements are among the data sets available. These data sets are available on the interactive data home page of the SEC, www.xbrl.sec.gov.
In addition, the SEC maintains a market structure and data ...
5
Yes, there is.
There are several ways to get this list:
You can, in Wikidata website, look at every entity that has a link to currency (Q8142): https://www.wikidata.org/wiki/Special:WhatLinksHere/Q8142. The problem of this is that you will have some entities that makes a link to currency other than instance of.
The best way is to use an external tool ...
5
You can find aggregated data on charitable donations from the IRS's Statistics of Income (SOI). Here are some links to look around:
http://www.irs.gov/uac/Tax-Stats-2
http://www.irs.gov/uac/SOI-Tax-Stats-Statistics-of-Income
In the UK, the National Council for Voluntary Organisations (NCVO) and the Charities Aid Foundation (CAF), has been conducting and ...
4
I think you are looking for the Crédit Agricole API, which has a page here. As far as I can tell, it's designed for app designers so each account would require individual authentication. Maybe
There are tons of news stories about it, check out examples here, here or here.
4
Have you seen this project opencorporates?
Probably you will find a lot of data there.
4
if you can use united states residents, you are looking for the survey of income and program participation. this microdata set contains monthly income about persons, families, and households. enjoy!
http://www.asdfree.com/search/label/survey%20of%20income%20and%20program%20participation%20%28sipp%29
4
if you need a free bin database, you can get it here:
https://getcreditcardonline.com/free-bin-database-download/
Data in CSV, JSON, SQL, TXT, Excel formats, etc.
4
The UN has several sources of datasets relating to government expenditures and trade that are free to use for commercial and non-commercial usage. Generally, the terms require attribution.
Note, not all UN datasets are free to use.
UN Data - data.un.org
Terms of Use - http://data.un.org/Host.aspx?Content=UNdataUse
All data and metadata provided on ...
4
great question.
The first thing to note is that the IEG database that you're referencing is only one of several that the World Bank provides. You might want to also check out the Bank's projects portal: http://www.worldbank.org/projects. In particular, check out specific country pages for a lot of good details, as well as many project documents (in pdf) ...
4
You may get better answers on CodeReview, but just glancing at your code I have one suggestion that is to split the codes into a scraping and parsing steps. This will help with reproducibility and also by decoupling the steps you can easier debug.
Scraping - the goal should be to save HTML files saved on the disk with names describing what they are and when ...
Only top voted, non community-wiki answers of a minimum length are eligible