8
The US Census Bureau also has a national gazeteer file with state and city info, including incorporation information.
For metadata/excel file column please have a look at HERE.
6
I have a CSV file you can download that lists all US cities by state. The data was compiled from the US Census 2013 Gazetteer.
http://www.opengeocode.org/download.php#statecity
In case you must have the data from that domain, each state's information is listed on the states index pagef, and follows similar naming conventions. For example, here's Virginia'...
5
You might try using data from the Geonames.org database. You can download gazateer data by country from this list.
Another potential source is the the MaxMind GeoLite2 dataset.
4
I'm not sure what underlying HTTP packages pandas uses, but I found that the default settings of the requests package in python does not work with SSL and this site:
import requests
url = 'https://api.fda.gov/drug/event.json?'
print requests.get(url)
gives an error like:
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:...
4
Unfortunately, the search index for searching tweets only goes back 7 days. Link to the documentation here, it can be found under 'until'.
You might be able to collect tweets from individual users' timelines, if you have an idea about what users are using those terms in their tweets. You can then use the User Timeline part of the REST API to get up to 3200 ...
4
tl;dr - find and use AIML files
There is an open-source chat-box project called A.L.I.C.E.
Link to Demo
A.L.I.C.E. uses the AIML format
AIML (Artificial Intelligence Markup Language) is an XML-compliant language that's easy to learn, and makes it possible for you to begin customizing an Alicebot or creating one from scratch within minutes.
The ...
4
If there is no well documented API for weatherbase.com, it might be interesting to scrape the information. Here is the list of cities the site contains with their URL. You can select some cities and paste their rows into a csv file. This Python 2 script will read the .csv and extract the average temperature in February for each city. The code is probably ...
2
For those who might get to this question and look for an answer, here is the link to my other post with the answer.
https://stackoverflow.com/questions/40824807/reading-zipped-json-files
2
I know this topic is six months old, but for reference:
the ECMWF API is now supported on Windows. Please see here for details
The OP created a file '.ecmwfapirc.txt'. This is wrong, it has to be named '.ecmwfapirc' (without extension).
From what I can see your retrieval script is correct.
There is a web interface to help you create your retrieval script:...
2
You can create a UNIX environment from Windows by several ways.
Cygwin
is a Unix-like environment and command-line interface for Microsoft Windows. Cygwin provides native integration of Windows-based applications, data, and other system resources with applications, software tools, and data of the Unix-like environment. (source)
Run Linux in VirtualBox. ...
2
This is one way of working with the dataset in Python 3.5. The code and document are available here: https://github.com/HamidehIraj/ReadNSF
Import Libraries
import re
import os
Loading Data
indir = './Part1/'
# indir = './Part1-Complete/'
file_list=[]
year_list=[]
abstract_list=[]
for root, dirs, filenames in os.walk(indir):
for file in filenames:
...
2
You can override the template snippets/add_dataset.html if you just want to have multiple add buttons.
Here is what you can do (tested on CKAN 2.8.3):
{# Adds 'Add Dataset' button #}
{% set dataset_type = dataset_type if dataset_type else 'dataset' %}
{#
{% if group %}
{% link_for _('Add Dataset'), controller='package', action='new', group=group, ...
2
Selenium is a good option to interact with the site and eventually get to the page where you can download/scrape the data you need. I've used the python (https://selenium-python.readthedocs.org/) version with good success.
Start with:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support ...
2
SELECT ?nameLabel ?typeLabel ?langLabel WHERE {
?type wdt:P279* wd:Q202444 .
?name wdt:P31 ?type .
?name wdt:P407 ?lang
SERVICE wikibase:label{bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en".}
}
The wikidata query above (http://tinyurl.com/y9t44r59) yields 12841 results and may provide a helpful starting point.
2
From dataset documentation in Table IV of Harrison & Rubin (1978), LSTAT is defined as:
Proportion of population that is lower status = 1/2
(proportion of adults without, some high school education and proportion of male workers classified as
laborers). The logarithmic specification implies that
socioeconomic status distinctions mean more in the
...
2
With a hint from a person from Transparency International and some further browsing through the source code this link eventually made it work:
http://static.tttp.eu/doi/data/meeting.csv
2
Here's a quick python3 script to get the table from each html page as a local csv file.
basically, you have to create a loop over all dates and then make a web requests. the python library pandas has a nice function .read_html(), which in this case parses the table directly into a dataframe. .read_html() returns an array of dataframes, and since the code ...
1
This is not an answer.
If you visit https://sdmdataaccess.sc.egov.usda.gov/QueryHelp.aspx and download the "Table Column Descriptions Report (741K)" PDF, you will get a list of all tables (and column names) for your data.
You can then go to https://sdmdataaccess.sc.egov.usda.gov/Query.aspx and make queries like SELECT * FROM chaashto. You will have to ...
1
I'm Jack with the openFDA team. From my understanding, you have a list of target proteins and want to link it with drugs in one of the openFDA endpoints. As I see it, the problem you'll run into is that to my knowledge there's no field in openFDA that details target proteins. If you already know which drugs you're looking for and are searching for more ...
1
Well done on this issue / question. It's more succinct than before and focuses more closely on 1 problem. I'll focus on your 3 issues together as they seem to be interrelated.
You're right, I think you having to login in the preview section is totally incorrect and feels a bit odd :) and I can't seem to replicate this specific issue.
In your screenshot I ...
1
I was actually using the wrong format for date. It should be something like this:
records = c.get(agency_name=o['name'], last_modified_date="["+str(next_date)+","+str(next_date)+"]")
or
records = c.get(agency_name=o['name'], last_modified_date="[2018/12/17,2018/12/21]")
1
I’m assuming you’re looking for something to make this api work easier, rather than resorting to hand coding. I would suggest Talend’s Data Services Platform.
I think Talend is the best tool around; but of course that’s why I work for a Talend...
For an unbiased viewpoint, Forrester gives Talend the highest score among all data fabrics. =)
1
I think you can start with general source of knowledge nowadays, wikipedia, namely the article on List of popular given names . This will give you enough information to extrapolate from country to gender/ethnicity and if the classification is feasible.
After that, you can either dig deeper into country census data or use pseudo-labelling techniques to ...
1
I would try to do a request of the data first.
import requests
json_df = pd.read_json(requests.get(url).json())
1
There is an international coding system that lists and codes an enormous range of diseases/symptoms called ICD10. This is how Wikipedia describes it:
ICD-10 is the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD), a medical classification list by the World Health Organization (WHO). It contains ...
1
JATS articles are "open dataset" in that sense... The main source at PubMed Central is the bulk data at http://www.ncbi.nlm.nih.gov/pmc/tools/ftp/
But your approach must start by the "candidate article selection", because the universe is so big... "Containing data for disease and symptoms" is so big...
(aleatory) Examples of cientific paper with data ...
1
Maybe you can try extract it from PubMed.
Or maybe you can use/refer to this paper by Barabasi. Got some data on the supplement section. http://www.nature.com/ncomms/2014/140626/ncomms5212/abs/ncomms5212.html#figures
1
Try Transpose.com. It uses a drag and drop platform with a bunch of different fields so you can create the exact database you need. It also uses search analytics so you find anything you've input by just typing it in.
1
Your issue is on line 2 of the "This doesn't work" snippet (the error is including the /api/3). It should be, as follows, in full:
import ckanapi
ckan = ckanapi.RemoteCKAN('https://data.noaa.gov')
search_params = {
'q': 'tags:"sea_water_temperature" AND metadata_modified:[2012-06-01T00:00:00.000Z TO NOW]',
...
1
You can pull the metadata directly from this URL (xml returned by default):
http://api.worldbank.org/countries
It looks like it has what you need. Regions/groups are treated like countries but you can distinguish them based on empty or "NA" field values.
Only top voted, non community-wiki answers of a minimum length are eligible
