Questions tagged [python]

For all questions and requests concerning or related to Python, a highly readable and dynamic, general purpose programming language.

Filter by
Sorted by
Tagged with
-3 votes
0 answers
23 views

What is Upgini? What are its uses? [closed]

Does anyone used upgini python library for open data search? What is your feedback about it?
user avatar
  • 3
0 votes
0 answers
12 views

2020 FBI Hate Crime Tables?

I'm sure most of us are familiar with the data the FBI provides. A while back the FBI "released" the 2020 data, but it's missing from their website. If you go to the link https://ucr.fbi.gov/...
user avatar
0 votes
1 answer
14 views

augment/enrich latitude longitude data

I am looking for ways to augment/enrich latitude longitude data, to obtain for example information like this (for the EU/Europe and the UK): rural/urban area distance to lake/sea distance to nearest ...
user avatar
  • 497
0 votes
1 answer
31 views

how I can read the content of the dataset in pkl file and return 67 sparse matrix of type '<class 'numpy.int64'>'in python?

I have a dataset in pkl file, and I write the following code to unpkl the file and read the data import pickle f=open("data_cdg.pkl","rb") a = pickle.load(f) f.close() print(a) ...
user avatar
0 votes
0 answers
10 views

How do I add a square root curve to my data into my regression model?

I want to add a square root term into my regression model, but I'm not exactly sure how to do it. from scipy.optimize import curve_fit import statsmodels.formula.api as smf def square_root(x,a,b,c): ...
user avatar
1 vote
0 answers
18 views

GIS Data on US College Dormitories

I am trying to find spatial data on all (or as many as possible) US college dormitories. Specifically, I want to find the location as a polygon or centroid. I have tried a few methods to acquire this ...
user avatar
  • 111
0 votes
1 answer
43 views

Where can I find real world salt and pepper noise image?

I am looking for real world salt and pepper or gaussian noise images. Where can I find them?
user avatar
  • 1
0 votes
1 answer
331 views

Scraping sec filings

I'm trying to scrap all the historical filings of a particular Central Index Key (CIK) using python but I get http error codes. First, I got 403 error, I resolved by using requests session and timing ...
user avatar
0 votes
1 answer
112 views

Extracting data from OpenFDA using python to save it locally

I have been looking to extract all data from FDA so I can use it all. I created this code below that works but seems it is too much data it fails. This way it just gives me 25000 records. url = '...
user avatar
1 vote
0 answers
27 views

Accessing DOL State Unemployment Insurance Weekly Claims Data

I would like to access weekly data for unemployment claims through the DOL API (Department of Labor), listed here: https://oui.doleta.gov/unemploy/claims.asp I'm able to get national data by using ...
user avatar
1 vote
0 answers
29 views

Getting partial data from department of labor API

In trying get data from this API (I have a token..): https://devtools.dol.gov/APISampler/Home/Index1?datasetName=OUI%20Initial%20Claims It seems that I'm not getting all the data, the latest year I ...
user avatar
1 vote
1 answer
202 views

Wind forecast over land accessible through python API

I would like to automatically fetch wind speed forecast data over my town with a python script. I am looking for an available dataset and API to do so. It is for a small personal project, so it is ...
user avatar
  • 111
1 vote
1 answer
39 views

to which extend we can use the data of healthsite.io - can we use maps etc. too?

to which extend we can use the data of healthsite.io update:: well i guess that it is evident: since the data in Healthsites.io are indeed under the ODBL licence: https://github.com/healthsites/...
user avatar
  • 309
1 vote
1 answer
22 views

Missing ACS Variables

Looks like a range of ACS variables regarding housing are missing for the ACS: B06013_001E BO6013_002E BO6013_003E BO6013_004E and others in this group as well. What is one to do when they are ...
user avatar
1 vote
2 answers
105 views

How to Format API Requests to Loop through All Tracts in a County

I wrote this code to get specific ACS data for a set of years for all the tracts in Wake County (id 37187) in the state of North Carolina. I've read the census example of how to format the code to get ...
user avatar
1 vote
0 answers
66 views

State Area for 2010 Census AREALAND variable not found

I was following this tutorial for requesting from the 2010 Decennial Census. It specifies the following code for printing a list of state names ("NAME"), state areas ("AREALAND"), and state total ...
user avatar
1 vote
1 answer
364 views

How do I get historical data from coinmarketcap.com?

If you scroll down on this page: https://coinmarketcap.com/historical/20150503/ There is something that says: Total Market Cap: $3,863,780,096 Notice the date is given in the URL. How can I loop ...
user avatar
0 votes
1 answer
30 views

Writing a query to obtain soil data from lat/lon in USA?

I would like to access soil data by lat/lon. It looks like this is the US soil web service here. However, I am not sure how to set up the query, the Help page should be helpful but is 100's of pages ...
user avatar
  • 101
3 votes
1 answer
3k views

What does “lower status” mean in “Boston house prices dataset”?

scikit-learn comes with Boston house prices dataset. One of the features is LSTAT, which means "Percentage of lower status of the population". What does "lower status" mean there? Does it mean ...
user avatar
  • 211
1 vote
0 answers
22 views

data with video to text transcription

I search data and script (java, python) where from video, text is transcripted into documents.I.E recognize speech from video to text i found this https://github.com/Naki21/google-speech-to-text but ...
user avatar
  • 21
1 vote
0 answers
11 views

Is there any conversational dataset for creating a chat bot with deep learning [duplicate]

I am creating a chatbot in Python-3 and I was looking for conversational data set for training the bot. I want the bot to effectively answer general questions and interact with the user with casual ...
user avatar
  • 11
1 vote
0 answers
31 views

Where can I find data to create small sample drug database?

For my diploma thesis I want to build an application to control medicine dosage and to do this I think about making sample database with like 1000 medicines with info most importantly about a dosage (...
user avatar
1 vote
0 answers
20 views

Is there any API, tool, websites that i can find how many German people moved to United States in recent 10 years?

I need to do geographical population analysis on how many German people recently moved to US and started to living there, especially to the California. I know US government has open data policy but so ...
user avatar
1 vote
0 answers
59 views

Open source weather database for predict crop disease?

I have started to work on forecasting models for precision agriculture. My goal is to train a model which uses temperature, relative humidity, and rainfall to predict when late blight (Phytophthora ...
user avatar
  • 111
1 vote
0 answers
121 views

average ingredients for each drug in each year for a specific manufacturer

I want to count the average number of ingredients in each drug produced by a specific manufacturer in each year. The results I want should be like this: year | drug_names | avg_number_of_ingredients ...
user avatar
  • 111
1 vote
0 answers
103 views

CKAN : ValueError: VDM only works with SQLAlchemy versions 0.4 through 0.9, not: 1.1.2

Do I need to upgrade the package vdm to make it compatible with sqlalchemy version greater then 0.9? This is a CKAN question.
user avatar
  • 31
2 votes
1 answer
24 views

Which is the best way to work with NCBI data obtaining online partial information from the whole

I was trying to create a database from NCBI Nucleotide bank. I did a query which gave me 1124 results. From each single result I was wanting to obtain only the items realted to Accesion Country and ...
user avatar
0 votes
2 answers
59 views

Is it possible to link drugs, with protein targets, using OpenFDA?

I have a set of protein targets. The absolute ideal return for these targets for me, would be a .tsv file that looks like this: target_name fda_approved_drug target1 drug1 target2 - ...
user avatar
  • 101
1 vote
2 answers
81 views

Dataset of lines at n miles from the coast

I'm using Folium to generate some interactive maps with ocean data. One of the restrictions I am facing is that the objects, I plot has to be at-least at 5, 6, .., n miles from the coast, hence I am ...
user avatar
1 vote
1 answer
1k views

How to check whether a word is abbreviated or not using Python? [closed]

How do I check whether a word is abbreviated or not in a dataframe column using Python? For instance, I need to detect the value "U.S.A." as an abbreviation. Is there any dictionary present for this?
user avatar
3 votes
1 answer
262 views

Not able to view private datasets in CKAN using Nginx

I have installed CKAN v2.7.2(served under HTTPs using Apache web server),Nginx using docker. The task I am doing requires me to mount ckan on /ckan mount point, thus have added ckan.root_path = /ckan/...
user avatar
  • 43
0 votes
1 answer
100 views

Cannot get data from FPDS atom feed using pyfpds

I have a script which uses the pyfpds wrapper (https://github.com/18F/pyfpds) to extract contract data from the FPDS Atom Feed (www.fpds.gov). Following is the code I have and this used to work till ...
user avatar
  • 111
1 vote
0 answers
23 views

Are there any available lexical databases for English, containing predicate-argument (relation-role) information?

I am looking for a database (something like WordNet) containing predicate-arguments information (also known as relations-roles). I.e. what are the arguments of a given predicate? For example, loves(...
user avatar
0 votes
1 answer
479 views

KEEPA API Experience

I'm trying to pull historical prices and BSR from about 3,000 ASINs on Amazon, but need something built to interface with Keepa's API. Does anyone have any experience with this? If so, is there a ...
user avatar
1 vote
1 answer
205 views

Dict not able to return data from college scorecard API

So I've been working on a page which displays a school's name, url, city, state, zip, and student size via College Scorecard API but I'm getting tons of errors instead. However, the program is able to ...
user avatar
  • 21
0 votes
2 answers
415 views

Looking for Databases with gender of names and Ethnicity information

I want to build a classifier which can classify gender and ethnicity based on the names. I am looking for datasets to download where I take samples for supervised learning. If there are no open ...
user avatar
  • 167
4 votes
1 answer
1k views

Average Monthly temperatures for cities around the world

I am looking to find the average historical temperature in april (not the average temperature in april of 1988, for example) in different cities (about 400 cities across the world) and store that data ...
user avatar
1 vote
1 answer
1k views

Python crashing when trying to read zipped json files [closed]

I downloaded the 2015 adverse drug events data from openfda and I want to run some analysis with python. I am a python newbie, but have become comfortable with the scikit-learn libraries, but I cannot ...
user avatar
  • 41
1 vote
0 answers
74 views

To fetch bigger dataset from twitter

I need twitter data for my project, but with streaming api I am able to get only 15 min window frame.Hence a very smaller sample space. I have a written code in python as follows import tweep from ...
user avatar
3 votes
2 answers
1k views

Access ECMWF Data via python script

I want to access ECMWF data via a python script. I am not sure how to do that. I followed the instruction given on the website: Accessing ECMWF data server in batch Unfortunately it says: "This ...
user avatar
  • 143
4 votes
1 answer
205 views

Load NSF Research Award Abstracts in Python or R

I want to work on NSF Research Award Abstracts 1990-2003 Data Set in R. Every record of the data is stored in a text file and text files are grouped in folders (See data files here) Is there any easy ...
user avatar
  • 261
3 votes
0 answers
365 views

Scraping data from a PDF in HTML - table recognition

I have a simple looking table in a pdf file: one column containing company names. I don't know how many there are in each group but groups are separated by a bold line (the lower border of a table). ...
user avatar
  • 153
1 vote
0 answers
52 views

How can I deploy a sci-kit learn model when the data in use are .csv files?

I am dealing with accelerometer data (csv) as well as audio data stored in FTP server coming from an android app. Data is loaded from FTP in my notebook for exploratory analysis to extract features ...
user avatar
5 votes
3 answers
5k views

I get 'certificate verify failed' URLError when trying to download data from openFDA

I'm using Python and pandas to try to download the data in the FDA's 'Adverse drug event reports since 2004'. Here is my code: import pandas as pd import json from pandas.io.json import ...
user avatar
  • 153
1 vote
0 answers
628 views

Using Google's Custom Search api [closed]

I am currently working for a social media site. We would like to use Google Custom Search Engine API and derive analytical insights from the results. My choice of development platform is R or Python. ...
user avatar
  • 11
4 votes
1 answer
84 views

Lobby meetings of European Commission - Integrity Watch

The staff of the European Commission is obliged to keep a register of their meetings with lobbyists. The site Integrity Watch administered by Transparency International keeps track of these meetings ...
user avatar
  • 153
8 votes
2 answers
637 views

Historical Twitter Data

I'm trying to get historical twitter data starting from 2010 with specific keywords related to S&P500 companies. I will probably end up using their tickers as keywords once I finalize my list of ...
user avatar
  • 81
4 votes
1 answer
698 views

Datasets based on casual conversations for chatterbots

I am building a chatterbot that can answer questions related to tennis. I wish to know if there any any available datasets than can answer casual questions like "How are you", "How is the weather ...
user avatar
6 votes
3 answers
14k views

List of United States cities

I'm looking to get some data of this page http://www.greatschools.org/california/san-francisco/schools/?gradeLevels=e&page=2 To do it I need to create the links in this format: http://www....
user avatar
2 votes
0 answers
61 views

data.gov CKAN API ignores row parameter, returns duplicate results for different search terms

Using the data.gov CKAN api, this search via the web GUI returns 566 results: children However, post requests via the CKAN API returns 10 results despite the row parameter being set and, incidentally,...
user avatar
  • 121