30
votes
Accepted
Bulk download Sci-Hub papers
Sci-Hub is a paywall-bypassing website that uses "shared" user credentials to provide PDF or HTML scientific papers. The website itself doesn't store any papers. (There are interesting comments on ...
- 17.6k
23
votes
Accepted
Cryptocurrency historical prices
As it turns out there are several resources one can use for all main cryptocurrencies so I will post here the most relevant and flexible I was able to gather.
All Cryptocurrencies
Coinmetrics
https:...
- 1,022
23
votes
Accepted
Nicknames database
Nicknames and Diminutive Names Lookup (CSV)
CSV, Database of Common Nicknames and its GitHub Repository: Common Nickname CSV (Database)
Name to Nickname CSV and Nickname to Name CSV
- 11.9k
16
votes
Vehicles Data and Images API
http://vpic.nhtsa.dot.gov/api/
The National Highway Traffic Safety Administration has a free API that provides VIN decoding and lists of Makes/Models/Manufacturers/Parts/etc.
XML, JSON, CSV
Here ...
- 271
16
votes
COVID19 - Coronavirus case data (country summary data)
The JHU dashboard you linked has their data available as CSV on Github:
https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data
in particular these CSV files, which contain all ...
- 17.6k
15
votes
Open Source alternative to IMDB
Sadly, Freebase has been made "read only" March 31st 2015.
Alternatives are:
https://www.themoviedb.org/
Actively maintained by a large community and used by a broad range of apps under a ...
- 250
14
votes
Downloading the Panama Papers
No leak
The Panama Papers were not leaked at all. They were handed over to a German media company which probably paid a good price (translation/summary here) for that deal.
This means, the panama ...
- 290
12
votes
List of public holidays by countries?
You can use the Nager.Date project its supports over 100 countries (US, DE, FR, RU, UK, ...). The project is open source and available on GitHub.
Some data sources are available
WebApi (Json)
CSV ...
- 231
12
votes
Accepted
Age, Weight and Height dataset
Try the Demographic and Health Survey (DHS). They have what you are looking for, for a large set of countries. On top of that you will in part be able to identify family ties, and you can even ...
- 1,169
11
votes
Database world cities with their names in all languages of the world
Wikidata has a category called "city" which will give you a list of world cities
https://www.wikidata.org/wiki/Q515
You can use SPARQL to query Wikidata
https://www.mediawiki.org/wiki/...
- 17.6k
11
votes
Accepted
COVID19 - Coronavirus case data (country summary data)
Index of Countries (community wiki)
Global
Johns Hopkins University
WHO
ECDC
OurWorldInData
Worlddata.AI
Europe
Belgium
Germany and another Germany
Switzerland
Spain
Italy
United Kingdom
United ...
10
votes
Clickstream sample dataset
Dataset 1:
Wikipedia Releases Clickstream Data
Wikipedia has released a data set of clickstream data for January 2015. A clickstream is the path a user requests to get to a desired web page or ...
- 221
10
votes
Accepted
Downloading the Panama Papers
The Panama Papers database has just been published at this link
https://www.occrp.org/en/panamapapers/database.html
This ICIJ database contains information on almost 320,000 offshore entities that ...
- 332
10
votes
Accepted
Database of cities with coordinates and timezone
The GeoNames database gives you both the geo-coordinates and the timezone for each city with a population greater than 1k.
Here's a version of that DB with a direct access to an API and a map on ...
- 1,157
10
votes
Accepted
Customer Service (Call Center) Audio datasets
Audio datasets of call center recordings are hard to find as those are usually privately owned and subject to various privacy laws (which differ from one country to another). Unfortunately, it is ...
- 691
9
votes
Open Source alternative to IMDB
Wikidata has an increasingly complete coverage of many topics, including movies
Here's a sample query to get title.
Use the query builder (top right) to see what data is available.
(my source)
- 17.6k
9
votes
Cryptocurrency historical prices
I work at Quandl and stumbled upon this post. I'm happy to share an update: we now have two databases which contain OHLCV (open-high-low-close-volume) data for bitcoin and other cryptocurrencies:
1....
- 1,036
9
votes
Database world cities with their names in all languages of the world
I guess the Geonames project is the closest answer to your question. You can download the data directly on the website or you can also find it here and download it/use the api.
Cheers
- 1,157
9
votes
Age, Weight and Height dataset
Okay, this is a very specific dataset for the !Kung San people, but it has height, weight, sex, and age fields. Reference was found in McElreath:
"The data contained in data (Howell1) are partial ...
9
votes
Sources of weather data
http://www.ecmwf.int/en/research/climate-reanalysis/era-interim
A remodel of weather since 1979, based on many many measurements. The resulting dataset contains daily to hourly values around the globe ...
- 1,453
9
votes
Sources of weather data
Get the data from BigQuery:
https://cloud.google.com/bigquery/public-data/noaa-gsod
Sample query:
#standardSQL
SELECT name, stn, a.wban, TIMESTAMP(CONCAT(year, '-', mo, '-', da)) date, temp, prcp
...
- 211
9
votes
Accepted
Dataset of planets and moons in the solar system
Devstronomy project provides the datasets for planets (plus Pluto) and their satellites(moons) in accessible formats (CSV, JSON, SQL), available for download. The datasets originate from the NASA Jet ...
- 106
9
votes
Accepted
Where can I get shapefiles for changing German territory during the Second World War?
I found the spatial history project, which give month to month changes of the front lines. The article covers all interesting information and you can download the dataset (after the first figure) - ...
- 299
9
votes
COVID19 - Coronavirus case data (country summary data)
The Italian data from the Civil Protection Agency is updated daily at
https://github.com/pcm-dpc/COVID-19
There are a few data sets (CSV) in there. Aggregated data is published at http://...
- 211
9
votes
Dataset on coronavirus government countermeasures (per country/region/city?)
We started a worldwide crowdsourcing data collection on policy measures, including lockdowns at the admin1 level (region) and the country level. Currently we coded data for ~13 countries and are ...
- 1,169
8
votes
Where can I download those bible verses in JSON or XML or SQL formatted file?
Disclaimer: I am the author of below github projects.
Two very good (and complete) sources are:
https://github.com/scrollmapper/bible_databases :
This has many database formats including all ...
8
votes
Accepted
List of abbreviations and acronyms
I found a github repository that scrapes the wiki list of roughly 6000 acronyms
https://github.com/krishnakt031990/Crawl-Wiki-For-Acronyms
raw acronyms file
and then I wrote this python script to ...
- 17.6k
8
votes
Paraphrase data sets
You definitely have the big ones with MSRP and PPDB.
A few others, grouped by maintainers.
CLiC at the Universitat de Barcelona
Maintain several highly annotated corpora.
These are available from ...
Only top scored, non community-wiki answers of a minimum length are eligible