Questions tagged [python]
For all questions and requests concerning or related to Python, a highly readable and dynamic, general purpose programming language.
56
questions
0
votes
0
answers
13
views
data.dol.gov/get 'invalid API key' (python)
I keep getting an 'Invalid API key' response when trying to make a call to the data.dol.gov v2 API. Here is what my call looks like, please help!
base_endpoint = 'https://data.dol.gov/get/'
...
- 1
0
votes
0
answers
12
views
Extreme hyperparameter tuning for XGBoost
One important skill is to know how to build models that would produce good results, and not take a whole day to run at the same time. That said, I have time, and I want to try lots of hyperparameter ...
- 1
0
votes
1
answer
58
views
Wikidata dump in another language
I am aware that you can download a Wikidata dump file from here
But is it possible to only include results for a specific language?I have in the past worked with a json dump file and used python to ...
- 1
0
votes
0
answers
12
views
2020 FBI Hate Crime Tables?
I'm sure most of us are familiar with the data the FBI provides. A while back the FBI "released" the 2020 data, but it's missing from their website. If you go to the link
https://ucr.fbi.gov/...
- 113
0
votes
1
answer
24
views
augment/enrich latitude longitude data
I am looking for ways to augment/enrich latitude longitude data, to obtain for example information like this (for the EU/Europe and the UK):
rural/urban area
distance to lake/sea
distance to nearest ...
- 497
0
votes
1
answer
164
views
how I can read the content of the dataset in pkl file and return 67 sparse matrix of type '<class 'numpy.int64'>'in python?
I have a dataset in pkl file, and I write the following code to unpkl the file and read the data
import pickle
f=open("data_cdg.pkl","rb")
a = pickle.load(f)
f.close()
print(a)
...
1
vote
0
answers
21
views
GIS Data on US College Dormitories
I am trying to find spatial data on all (or as many as possible) US college dormitories. Specifically, I want to find the location as a polygon or centroid. I have tried a few methods to acquire this ...
- 111
0
votes
1
answer
499
views
Scraping sec filings
I'm trying to scrap all the historical filings of a particular Central Index Key (CIK) using python but I get http error codes.
First, I got 403 error, I resolved by using requests session and timing ...
- 101
0
votes
1
answer
219
views
Extracting data from OpenFDA using python to save it locally
I have been looking to extract all data from FDA so I can use it all.
I created this code below that works but seems it is too much data it fails. This way it just gives me 25000 records.
url = '...
1
vote
0
answers
33
views
Accessing DOL State Unemployment Insurance Weekly Claims Data
I would like to access weekly data for unemployment claims through the DOL API (Department of Labor), listed here: https://oui.doleta.gov/unemploy/claims.asp
I'm able to get national data by using ...
- 11
1
vote
0
answers
42
views
Getting partial data from department of labor API
In trying get data from this API (I have a token..):
https://devtools.dol.gov/APISampler/Home/Index1?datasetName=OUI%20Initial%20Claims
It seems that I'm not getting all the data, the latest year I ...
- 11
1
vote
1
answer
280
views
Wind forecast over land accessible through python API
I would like to automatically fetch wind speed forecast data over my town with a python script.
I am looking for an available dataset and API to do so.
It is for a small personal project, so it is ...
- 111
1
vote
1
answer
43
views
to which extend we can use the data of healthsite.io - can we use maps etc. too?
to which extend we can use the data of healthsite.io
update:: well i guess that it is evident:
since the data in Healthsites.io are indeed under the ODBL licence: https://github.com/healthsites/...
- 309
1
vote
1
answer
23
views
Missing ACS Variables
Looks like a range of ACS variables regarding housing are missing for the ACS:
B06013_001E
BO6013_002E
BO6013_003E
BO6013_004E
and others in this group as well.
What is one to do when they are ...
- 143
1
vote
2
answers
134
views
How to Format API Requests to Loop through All Tracts in a County
I wrote this code to get specific ACS data for a set of years for all the tracts in Wake County (id 37187) in the state of North Carolina. I've read the census example of how to format the code to get ...
- 143
1
vote
1
answer
89
views
State Area for 2010 Census AREALAND variable not found
I was following this tutorial for requesting from the 2010 Decennial Census. It specifies the following code for printing a list of state names ("NAME"), state areas ("AREALAND"), and state total ...
- 143
1
vote
1
answer
434
views
How do I get historical data from coinmarketcap.com?
If you scroll down on this page:
https://coinmarketcap.com/historical/20150503/
There is something that says: Total Market Cap: $3,863,780,096
Notice the date is given in the URL. How can I loop ...
- 11
0
votes
1
answer
37
views
Writing a query to obtain soil data from lat/lon in USA?
I would like to access soil data by lat/lon. It looks like this is the US soil web service here. However, I am not sure how to set up the query, the Help page should be helpful but is 100's of pages ...
- 101
3
votes
1
answer
5k
views
What does “lower status” mean in “Boston house prices dataset”?
scikit-learn comes with Boston house prices dataset. One of the features is LSTAT, which means "Percentage of lower status of the population".
What does "lower status" mean there?
Does it mean ...
- 211
1
vote
0
answers
22
views
data with video to text transcription
I search data and script (java, python) where from video, text is transcripted into documents.I.E recognize speech from video to text
i found this
https://github.com/Naki21/google-speech-to-text
but ...
- 31
1
vote
0
answers
11
views
Is there any conversational dataset for creating a chat bot with deep learning [duplicate]
I am creating a chatbot in Python-3 and I was looking for conversational data set for training the bot. I want the bot to effectively answer general questions and interact with the user with casual ...
- 11
1
vote
0
answers
32
views
Where can I find data to create small sample drug database?
For my diploma thesis I want to build an application to control medicine dosage and to do this I think about making sample database with like 1000 medicines with info most importantly about a dosage (...
- 11
1
vote
0
answers
21
views
Is there any API, tool, websites that i can find how many German people moved to United States in recent 10 years?
I need to do geographical population analysis on how many German people recently moved to US and started to living there, especially to the California. I know US government has open data policy but so ...
1
vote
0
answers
61
views
Open source weather database for predict crop disease?
I have started to work on forecasting models for precision agriculture. My goal is to train a model which uses temperature, relative humidity, and rainfall to predict when late blight (Phytophthora ...
- 111
1
vote
0
answers
124
views
average ingredients for each drug in each year for a specific manufacturer
I want to count the average number of ingredients in each drug produced by a specific manufacturer in each year.
The results I want should be like this:
year | drug_names | avg_number_of_ingredients
...
- 111
1
vote
0
answers
109
views
CKAN : ValueError: VDM only works with SQLAlchemy versions 0.4 through 0.9, not: 1.1.2
Do I need to upgrade the package vdm to make it compatible with sqlalchemy version greater then 0.9?
This is a CKAN question.
- 31
2
votes
1
answer
31
views
Which is the best way to work with NCBI data obtaining online partial information from the whole
I was trying to create a database from NCBI Nucleotide bank. I did a query which gave me 1124 results.
From each single result I was wanting to obtain only the items realted to Accesion Country and ...
Paulo Cecco
0
votes
2
answers
61
views
Is it possible to link drugs, with protein targets, using OpenFDA?
I have a set of protein targets. The absolute ideal return for these targets for me, would be a .tsv file that looks like this:
target_name fda_approved_drug
target1 drug1
target2 -
...
- 101
1
vote
2
answers
88
views
Dataset of lines at n miles from the coast
I'm using Folium to generate some interactive maps with ocean data. One of the restrictions I am facing is that the objects, I plot has to be at-least at 5, 6, .., n miles from the coast, hence I am ...
1
vote
1
answer
2k
views
How to check whether a word is abbreviated or not using Python? [closed]
How do I check whether a word is abbreviated or not in a dataframe column using Python? For instance, I need to detect the value "U.S.A." as an abbreviation.
Is there any dictionary present for this?
SUHASINI KL
3
votes
1
answer
306
views
Not able to view private datasets in CKAN using Nginx
I have installed CKAN v2.7.2(served under HTTPs using Apache web server),Nginx using docker. The task I am doing requires me to mount ckan on /ckan mount point, thus have added
ckan.root_path = /ckan/...
- 43
0
votes
1
answer
117
views
Cannot get data from FPDS atom feed using pyfpds
I have a script which uses the pyfpds wrapper (https://github.com/18F/pyfpds) to extract contract data from the FPDS Atom Feed (www.fpds.gov).
Following is the code I have and this used to work till ...
- 111
1
vote
0
answers
23
views
Are there any available lexical databases for English, containing predicate-argument (relation-role) information?
I am looking for a database (something like WordNet) containing predicate-arguments information (also known as relations-roles). I.e. what are the arguments of a given predicate?
For example,
loves(...
- 11
0
votes
1
answer
536
views
KEEPA API Experience
I'm trying to pull historical prices and BSR from about 3,000 ASINs on Amazon, but need something built to interface with Keepa's API. Does anyone have any experience with this? If so, is there a ...
1
vote
1
answer
226
views
Dict not able to return data from college scorecard API
So I've been working on a page which displays a school's name, url, city, state, zip, and student size via College Scorecard API but I'm getting tons of errors instead. However, the program is able to ...
- 21
0
votes
2
answers
577
views
Looking for Databases with gender of names and Ethnicity information
I want to build a classifier which can classify gender and ethnicity based on the names.
I am looking for datasets to download where I take samples for supervised learning.
If there are no open ...
- 167
4
votes
1
answer
2k
views
Average Monthly temperatures for cities around the world
I am looking to find the average historical temperature in april (not the average temperature in april of 1988, for example) in different cities (about 400 cities across the world) and store that data ...
- 88
1
vote
1
answer
1k
views
Python crashing when trying to read zipped json files [closed]
I downloaded the 2015 adverse drug events data from openfda and I want to run some analysis with python. I am a python newbie, but have become comfortable with the scikit-learn libraries, but I cannot ...
- 41
1
vote
0
answers
77
views
To fetch bigger dataset from twitter
I need twitter data for my project, but with streaming api I am able to get only 15 min window frame.Hence a very smaller sample space.
I have a written code in python as follows
import tweep
from ...
- 41
3
votes
2
answers
1k
views
Access ECMWF Data via python script
I want to access ECMWF data via a python script. I am not sure how to do that.
I followed the instruction given on the website:
Accessing ECMWF data server in batch
Unfortunately it says: "This ...
- 143
4
votes
1
answer
215
views
Load NSF Research Award Abstracts in Python or R
I want to work on NSF Research Award Abstracts 1990-2003 Data Set in R. Every record of the data is stored in a text file and text files are grouped in folders (See data files here)
Is there any easy ...
- 261
3
votes
0
answers
373
views
Scraping data from a PDF in HTML - table recognition
I have a simple looking table in a pdf file: one column containing company names. I don't know how many there are in each group but groups are separated by a bold line (the lower border of a table).
...
- 153
1
vote
0
answers
53
views
How can I deploy a sci-kit learn model when the data in use are .csv files?
I am dealing with accelerometer data (csv) as well as audio data stored in FTP server coming from an android app. Data is loaded from FTP in my notebook for exploratory analysis to extract features ...
- 111
5
votes
3
answers
5k
views
I get 'certificate verify failed' URLError when trying to download data from openFDA
I'm using Python and pandas to try to download the data in the FDA's 'Adverse drug event reports since 2004'.
Here is my code:
import pandas as pd
import json
from pandas.io.json import ...
- 153
1
vote
0
answers
632
views
Using Google's Custom Search api [closed]
I am currently working for a social media site. We would like to use Google Custom Search Engine API and derive analytical insights from the results. My choice of development platform is R or Python. ...
- 11
4
votes
1
answer
86
views
Lobby meetings of European Commission - Integrity Watch
The staff of the European Commission is obliged to keep a register of their meetings with lobbyists. The site Integrity Watch administered by Transparency International keeps track of these meetings ...
- 153
8
votes
1
answer
653
views
Historical Twitter Data
I'm trying to get historical twitter data starting from 2010 with specific keywords related to S&P500 companies. I will probably end up using their tickers as keywords once I finalize my list of ...
- 81
4
votes
1
answer
719
views
Datasets based on casual conversations for chatterbots
I am building a chatterbot that can answer questions related to tennis. I wish to know if there any any available datasets than can answer casual questions like "How are you", "How is the weather ...
- 41
6
votes
3
answers
15k
views
List of United States cities
I'm looking to get some data of this page
http://www.greatschools.org/california/san-francisco/schools/?gradeLevels=e&page=2
To do it I need to create the links in this format:
http://www....
2
votes
0
answers
63
views
data.gov CKAN API ignores row parameter, returns duplicate results for different search terms
Using the data.gov CKAN api, this search via the web GUI returns 566 results: children
However, post requests via the CKAN API returns 10 results despite the row parameter being set and, incidentally,...
- 121