As it turns out there are several resources one can use for all main cryptocurrencies so I will post here the most relevant and flexible I was able to gather.
This page has data for: Bitcoin, Litecoin, Ethereum, NEM, Decred, ZCash (transparent transactions only), Dash, Dogecoin, ...
The NOAA / National Weather Service has an archive from the Weather Prediction Center, but it only started archiving most products a few years ago:
Most of what's archived are weather maps (like what ...
forecast.io also provides historical forecasts through their API. You can call a past date and time and still get the daily, hourly, and minute forecasts (in addition to actual forecasts). That is, ask for the weather on April 16th and will include forecasts for April 20th in the daily summaries.
It's free for under 1,000 calls per day, and then $1 for 10,...
There are many good resources described in the comments to your question (1 & 2). One that combines many different types of financial products and has some open access is Quandl.
Data browser tool
Quandl provides a single easy-to-use API for stock prices and fundamentals. Coverage includes end-of-day prices, harmonized fundamentals, ...
I work at Quandl and stumbled upon this post. I'm happy to share an update: we now have two databases which contain OHLCV (open-high-low-close-volume) data for bitcoin and other cryptocurrencies:
1.Brave New Coin Digital Currency Indexed EOD: https://www.quandl.com/data/BNC2 Contains historical global price indexes for a number of cryptocurrencies
I found the spatial history project, which give month to month changes of the front lines. The article covers all interesting information and you can download the dataset (after the first figure) - viz., EuropeanBorders_WWII.zip (~260 MB).
For US data, the NHGIS project is by far the best source. From their home page:
The National Historical Geographic Information System (NHGIS)
provides, free of charge, aggregate census data and GIS-compatible
boundary files for the United States between 1790 and 2014.
I also found that the Association of American Geographers runs this Historical GIS ...
Great question. I agree that permanent archiving of blog posts and other digital content is an important challenge in open data. It might be helpful to break this down into parts:
Having a persistent address at which potential users/machines can reference your content is crucial to good archiving, and most of the issues you list ...
The data you want are in these files.
The primary source for this data was a US Census Bureau dataset of ~7500 incorporated cities whose populations surpassed 2500 people at some point in their existence. Additional cities were added from a variety of sources (...)
Publishing the Data
If you don't want to go down the full CKAN-style route a really simple option if you've got a bunch of CSV files is just to put them online and turn them into "simple data format" data packages by adding a tiny bit of metadata in the form of a datapackage.json. Building a catalog out of this is really easy (and can be done with just a ...
The Sunlight Foundation makes a lot of Congressional Record data available through its Capitol Words API:
It's focused on speeches, and doesn't bring much order to chaos beyond that.
But you're also looking for votes, and bills, and bill text? Rather than plumb the depths of the Congressional Record, you can get that from ...
www.keepa.com is a similar website to camelcamelcamel. I cannot find a way to download the data but maybe you will.
In addition, Terapeak provide a free package with limited api calls (500/months) where you can find historical prices.
The "FOOD AND AGRICULTURE ORGANIZATION OF THE UNITED NATIONS - Statistics Division" (link) provides historical data for commodity prices and agricultural production.
You can download for a single country or for all countries. Some data sets go back to at least the 1960s. You'll have to investigate which data sets may be best suited to ...
You can find datasets like this from Universities (particularly history depts.). Here's a few:
University of Pittsburg: http://www.dataverse.pitt.edu/external/datasets.php
University of North Texas: http://www.paulhensel.org/icowcol.html
University of Wisconsin-Madison: http://www.sage.wisc.edu/download/crop1700/hist_croplands.html
Uppsalla University: ...
As @philshem suggested I wrote some python to gather the data
import pandas as pd
from bs4 import BeautifulSoup
# Create date range for historical snapshots
Date = pd.date_range(start='20130428', end='20171210', freq='7D').strftime('%Y%m%d')
# Retrieve market cap value in dollars
market_cap = 
for date in Date:
# Retrieve ...
The Office for National Statistics website is going to be your friend here.
The current Population Estimates Dataset includes (under 'Mid-2016 detailed time series') national and regional population estimates going back to 1838.
For GDP there are several options, so it's probably best to look through the available data by searching the ONS website for '...
An application that uses a dataset like this one is Day Like Today from the OKFN Greek chapter. It makes use of DBpedia and retrieves information from Wikipedia infoboxes. For example dates of wars, deaths of popular persons etc.
Thus, I do not know a database that you can download and use, but you can use DBpedia to retrieve historical events from ...
I created a git repo with the data I have collected from the past few seasons for the NCAA mens basketball division I.
Check it out here:
I would checkout govtrack.us. They make their datasets available as 'public domain'.
Any data files we make available from the Source Data page for which we own the copyright we release into the public domain.
You can also scrape (or better yet contact and ask the folks at) https://thetracktor.com which seems to have some great historical data.
EDIT - Adding example:
https://thetracktor.com/ajax/prices/?id=881549&days=90 is the JSON response powering the graph at https://thetracktor.com/detail/B0054JJ0QW/, for example.
For German daily data, start here at the Deutschewettersdienst website.
Open the file KL_Tageswerte_Beschreibung_Stationen.txt to find a station with data for the location and time period that you want. For Bonn, it looks like station ID 599 "Bonn-Friesdorf" covers 1932 through ...
The historical forecast archive of the World Bank has the following note:
please also note that in the past, commodities forecasts have not appeared in stand-alone publications, but rather as chapters or annexes of other publications, namely Global Economic Prospects (in January of each year) and Global Development Finance (in June of each year)
Have you tried Google BigQuery?
GBQ has now loaded climate data worldwide from the National Oceanic and Atmospheric Association (NOAA) from over 9,000 weather stations. The data includes 2015 and 2016.
You can query the climate data set using SQL. This queries 14 days of max temperatures (tested just now):
SELECT year, mo, da, temp FROM [bigquery-public-...
I recommend trying rnoaa https://cran.rstudio.com/web/packages/rnoaa/ - It's an interface to many different data sources from NOAA.
df <- data.frame(
id = letters[1:6],
time = c("2016-06-01 10:35:26", "2016-06-01 10:35:57", "2016-06-01 10:37:30",
"2016-06-01 19:21:34", "2016-06-01 19:22:36", "2016-06-01 19:23:07"),
latitude = c(41....