Hot answers tagged

8 votes

List of United States cities

The US Census Bureau also has a national gazeteer file with state and city info, including incorporation information. For metadata/excel file column please have a look at HERE.
user avatar
  • 201
6 votes
Accepted

List of United States cities

I have a CSV file you can download that lists all US cities by state. The data was compiled from the US Census 2013 Gazetteer. http://www.opengeocode.org/download.php#statecity In case you must ...
user avatar
5 votes
Accepted

What does “lower status” mean in “Boston house prices dataset”?

From dataset documentation in Table IV of Harrison & Rubin (1978), LSTAT is defined as: Proportion of population that is lower status = 1/2 (proportion of adults without, some high school ...
user avatar
  • 1,179
5 votes
Accepted

Average Monthly temperatures for cities around the world

If there is no well documented API for weatherbase.com, it might be interesting to scrape the information. Here is the list of cities the site contains with their URL. You can select some cities and ...
user avatar
5 votes

List of United States cities

You might try using data from the Geonames.org database. You can download gazateer data by country from this list. Another potential source is the the MaxMind GeoLite2 dataset.
user avatar
4 votes

Historical Twitter Data

Unfortunately, the search index for searching tweets only goes back 7 days. Link to the documentation here, it can be found under 'until'. You might be able to collect tweets from individual users' ...
user avatar
4 votes

Datasets based on casual conversations for chatterbots

tl;dr - find and use AIML files There is an open-source chat-box project called A.L.I.C.E. Link to Demo A.L.I.C.E. uses the AIML format AIML (Artificial Intelligence Markup Language) is an XML-...
user avatar
  • 17.1k
4 votes
Accepted

I get 'certificate verify failed' URLError when trying to download data from openFDA

I'm not sure what underlying HTTP packages pandas uses, but I found that the default settings of the requests package in python does not work with SSL and this site: import requests url = 'https://...
user avatar
  • 17.1k
3 votes
Accepted

to which extend we can use the data of healthsite.io - can we use maps etc. too?

Great question! The data license is the same as the ODbL of OpenStreetMap. ## ODC Open Database License (ODbL) Preamble The Open Database License (ODbL) is a license agreement intended to allow ...
user avatar
2 votes

Scraping vehicle reporting data from regulator's website

Selenium is a good option to interact with the site and eventually get to the page where you can download/scrape the data you need. I've used the python (https://selenium-python.readthedocs.org/) ...
user avatar
  • 171
2 votes

CKAN - ckanext-scheming - Add dataset button

You can override the template snippets/add_dataset.html if you just want to have multiple add buttons. Here is what you can do (tested on CKAN 2.8.3): {# Adds 'Add Dataset' button #} {% set ...
user avatar
2 votes

Python crashing when trying to read zipped json files

For those who might get to this question and look for an answer, here is the link to my other post with the answer. https://stackoverflow.com/questions/40824807/reading-zipped-json-files
user avatar
  • 41
2 votes
Accepted

Access ECMWF Data via python script

I know this topic is six months old, but for reference: the ECMWF API is now supported on Windows. Please see here for details The OP created a file '.ecmwfapirc.txt'. This is wrong, it has to be ...
user avatar
2 votes

Access ECMWF Data via python script

You can create a UNIX environment from Windows by several ways. Cygwin is a Unix-like environment and command-line interface for Microsoft Windows. Cygwin provides native integration of Windows-...
user avatar
  • 17.1k
2 votes

Load NSF Research Award Abstracts in Python or R

This is one way of working with the dataset in Python 3.5. The code and document are available here: https://github.com/HamidehIraj/ReadNSF Import Libraries import re import os Loading Data indir ...
user avatar
  • 261
2 votes
Accepted

Lobby meetings of European Commission - Integrity Watch

With a hint from a person from Transparency International and some further browsing through the source code this link eventually made it work: http://static.tttp.eu/doi/data/meeting.csv
user avatar
  • 153
2 votes

How do I get historical data from coinmarketcap.com?

Here's a quick python3 script to get the table from each html page as a local csv file. basically, you have to create a loop over all dates and then make a web requests. the python library pandas has ...
user avatar
  • 17.1k
1 vote

Looking for Databases with gender of names and Ethnicity information

I think you can start with general source of knowledge nowadays, wikipedia, namely the article on List of popular given names . This will give you enough information to extrapolate from country to ...
user avatar
1 vote

Is it possible to link drugs, with protein targets, using OpenFDA?

I'm Jack with the openFDA team. From my understanding, you have a list of target proteins and want to link it with drugs in one of the openFDA endpoints. As I see it, the problem you'll run into is ...
user avatar
1 vote

Not able to view private datasets in CKAN using Nginx

Well done on this issue / question. It's more succinct than before and focuses more closely on 1 problem. I'll focus on your 3 issues together as they seem to be interrelated. You're right, I think ...
user avatar
1 vote

Cannot get data from FPDS atom feed using pyfpds

I was actually using the wrong format for date. It should be something like this: records = c.get(agency_name=o['name'], last_modified_date="["+str(next_date)+","+str(next_date)+"]") or records = c....
user avatar
  • 111
1 vote

KEEPA API Experience

I’m assuming you’re looking for something to make this api work easier, rather than resorting to hand coding. I would suggest Talend’s Data Services Platform. I think Talend is the best tool around; ...
user avatar
1 vote

I get 'certificate verify failed' URLError when trying to download data from openFDA

I would try to do a request of the data first. import requests json_df = pd.read_json(requests.get(url).json())
user avatar
1 vote
Accepted

World Bank via Pandas - looking up country regions

You can pull the metadata directly from this URL (xml returned by default): http://api.worldbank.org/countries It looks like it has what you need. Regions/groups are treated like countries but you ...
user avatar
  • 431
1 vote

Looking for open dataset containing data for disease and symptoms

There is an international coding system that lists and codes an enormous range of diseases/symptoms called ICD10. This is how Wikipedia describes it: ICD-10 is the 10th revision of the ...
user avatar
1 vote

Looking for open dataset containing data for disease and symptoms

JATS articles are "open dataset" in that sense... The main source at PubMed Central is the bulk data at http://www.ncbi.nlm.nih.gov/pmc/tools/ftp/ But your approach must start by the "candidate ...
user avatar
1 vote

Looking for open dataset containing data for disease and symptoms

Maybe you can try extract it from PubMed. Or maybe you can use/refer to this paper by Barabasi. Got some data on the supplement section. http://www.nature.com/ncomms/2014/140626/ncomms5212/abs/...
user avatar
  • 191
1 vote

Open database for JEL code crawling (using article names)

Try Transpose.com. It uses a drag and drop platform with a bunch of different fields so you can create the exact database you need. It also uses search analytics so you find anything you've input by ...
user avatar
  • 11

Only top scored, non community-wiki answers of a minimum length are eligible