Questions tagged [python]

For all questions and requests concerning or related to Python, a highly readable and dynamic, general purpose programming language.

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

What is the fastest way (e.g. Python, Java, PDF Miner) to download several hundred .pdf files from a url, then convert them to HTML text?

Picture any standard .pdf Create a "robot" to download over 500 .pdf files and convert to a .zip. Unpack said .zip file, then convert specific portions (e.g. 100 words) of the .pdf to HTML ...
0
votes
0answers
10 views

healthsites, healthcare - ressources providing data for the world

healthsites, healthcare - ressources providing data for the world I've been working with healthcare in the past, but most of what I have seen is proprietary. I'm interested in reproducible research ...
0
votes
0answers
34 views

How can I export or query all hospitals from Open Street Map?

How can I export or query all hospitals from Open Street Map? 'd like to extract all the hospital locations in the from Open Street Map as a reference for my geocoding project. I know how to get all ...
0
votes
0answers
11 views

Extracting required fields from multiple text(pdf extracted)file?

What's the best practice of extracting information(fields) from a large number of text files, which may be formatted differently? For example, I have extracted text from multiple pdf files to multiple ...
1
vote
0answers
11 views

Accessing DOL State Unemployment Insurance Weekly Claims Data

I would like to access weekly data for unemployment claims through the DOL API (Department of Labor), listed here: https://oui.doleta.gov/unemploy/claims.asp I'm able to get national data by using ...
0
votes
0answers
14 views

Getting partial data from department of labor API

In trying get data from this API (I have a token..): https://devtools.dol.gov/APISampler/Home/Index1?datasetName=OUI%20Initial%20Claims It seems that I'm not getting all the data, the latest year I ...
1
vote
1answer
24 views

Wind forecast over land accessible through python API

I would like to automatically fetch wind speed forecast data over my town with a python script. I am looking for an available dataset and API to do so. It is for a small personal project, so it is ...
1
vote
1answer
21 views

to which extend we can use the data of healthsite.io - can we use maps etc. too?

to which extend we can use the data of healthsite.io update:: well i guess that it is evident: since the data in Healthsites.io are indeed under the ODBL licence: https://github.com/healthsites/...
1
vote
1answer
15 views

Missing ACS Variables

Looks like a range of ACS variables regarding housing are missing for the ACS: B06013_001E BO6013_002E BO6013_003E BO6013_004E and others in this group as well. What is one to do when they are ...
1
vote
2answers
41 views

How to Format API Requests to Loop through All Tracts in a County

I wrote this code to get specific ACS data for a set of years for all the tracts in Wake County (id 37187) in the state of North Carolina. I've read the census example of how to format the code to get ...
0
votes
0answers
10 views

State Area for 2010 Census AREALAND variable not found

I was following this tutorial for requesting from the 2010 Decennial Census. It specifies the following code for printing a list of state names ("NAME"), state areas ("AREALAND"), and state total ...
1
vote
1answer
47 views

How do I get historical data from coinmarketcap.com?

If you scroll down on this page: https://coinmarketcap.com/historical/20150503/ There is something that says: Total Market Cap: $3,863,780,096 Notice the date is given in the URL. How can I loop ...
0
votes
1answer
26 views

Writing a query to obtain soil data from lat/lon in USA?

I would like to access soil data by lat/lon. It looks like this is the US soil web service here. However, I am not sure how to set up the query, the Help page should be helpful but is 100's of pages ...
0
votes
1answer
619 views

What does “lower status” mean in “Boston house prices dataset”?

scikit-learn comes with Boston house prices dataset. One of the features is LSTAT, which means "Percentage of lower status of the population". What does "lower status" mean there? Does it mean ...
0
votes
0answers
15 views

data with video to text transcription

I search data and script (java, python) where from video, text is transcripted into documents.I.E recognize speech from video to text i found this https://github.com/Naki21/google-speech-to-text but ...
0
votes
0answers
9 views

Is there any conversational dataset for creating a chat bot with deep learning [duplicate]

I am creating a chatbot in Python-3 and I was looking for conversational data set for training the bot. I want the bot to effectively answer general questions and interact with the user with casual ...
1
vote
0answers
24 views

Where can I find data to create small sample drug database?

For my diploma thesis I want to build an application to control medicine dosage and to do this I think about making sample database with like 1000 medicines with info most importantly about a dosage (...
1
vote
0answers
13 views

Is there any API, tool, websites that i can find how many German people moved to United States in recent 10 years?

I need to do geographical population analysis on how many German people recently moved to US and started to living there, especially to the California. I know US government has open data policy but so ...
0
votes
0answers
38 views

Open source weather database for predict crop disease?

I have started to work on forecasting models for precision agriculture. My goal is to train a model which uses temperature, relative humidity, and rainfall to predict when late blight (Phytophthora ...
1
vote
0answers
103 views

average ingredients for each drug in each year for a specific manufacturer

I want to count the average number of ingredients in each drug produced by a specific manufacturer in each year. The results I want should be like this: year | drug_names | avg_number_of_ingredients ...
0
votes
0answers
76 views

CKAN : ValueError: VDM only works with SQLAlchemy versions 0.4 through 0.9, not: 1.1.2

Do I need to upgrade the package vdm to make it compatible with sqlalchemy version greater then 0.9? This is a CKAN question.
2
votes
1answer
18 views

Which is the best way to work with NCBI data obtaining online partial information from the whole

I was trying to create a database from NCBI Nucleotide bank. I did a query which gave me 1124 results. From each single result I was wanting to obtain only the items realted to Accesion Country and ...
0
votes
2answers
52 views

Is it possible to link drugs, with protein targets, using OpenFDA?

I have a set of protein targets. The absolute ideal return for these targets for me, would be a .tsv file that looks like this: target_name fda_approved_drug target1 drug1 target2 - ...
1
vote
2answers
69 views

Dataset of lines at n miles from the coast

I'm using Folium to generate some interactive maps with ocean data. One of the restrictions I am facing is that the objects, I plot has to be at-least at 5, 6, .., n miles from the coast, hence I am ...
0
votes
1answer
424 views

How to check whether a word is abbreviated or not using Python? [closed]

How do I check whether a word is abbreviated or not in a dataframe column using Python? For instance, I need to detect the value "U.S.A." as an abbreviation. Is there any dictionary present for this?
3
votes
1answer
143 views

Not able to view private datasets in CKAN using Nginx

I have installed CKAN v2.7.2(served under HTTPs using Apache web server),Nginx using docker. The task I am doing requires me to mount ckan on /ckan mount point, thus have added ckan.root_path = /ckan/...
0
votes
1answer
71 views

Cannot get data from FPDS atom feed using pyfpds

I have a script which uses the pyfpds wrapper (https://github.com/18F/pyfpds) to extract contract data from the FPDS Atom Feed (www.fpds.gov). Following is the code I have and this used to work till ...
1
vote
0answers
19 views

Are there any available lexical databases for English, containing predicate-argument (relation-role) information?

I am looking for a database (something like WordNet) containing predicate-arguments information (also known as relations-roles). I.e. what are the arguments of a given predicate? For example, loves(...
0
votes
1answer
397 views

KEEPA API Experience

I'm trying to pull historical prices and BSR from about 3,000 ASINs on Amazon, but need something built to interface with Keepa's API. Does anyone have any experience with this? If so, is there a ...
0
votes
1answer
155 views

Dict not able to return data from college scorecard API

So I've been working on a page which displays a school's name, url, city, state, zip, and student size via College Scorecard API but I'm getting tons of errors instead. However, the program is able to ...
0
votes
2answers
87 views

Looking for Databases with gender of names and Ethnicity information

I want to build a classifier which can classify gender and ethnicity based on the names. I am looking for datasets to download where I take samples for supervised learning. If there are no open ...
3
votes
1answer
1k views

Average Monthly temperatures for cities around the world

I am looking to find the average historical temperature in april (not the average temperature in april of 1988, for example) in different cities (about 400 cities across the world) and store that data ...
1
vote
1answer
539 views

Python crashing when trying to read zipped json files [closed]

I downloaded the 2015 adverse drug events data from openfda and I want to run some analysis with python. I am a python newbie, but have become comfortable with the scikit-learn libraries, but I cannot ...
1
vote
0answers
65 views

To fetch bigger dataset from twitter

I need twitter data for my project, but with streaming api I am able to get only 15 min window frame.Hence a very smaller sample space. I have a written code in python as follows import tweep from ...
3
votes
2answers
758 views

Access ECMWF Data via python script

I want to access ECMWF data via a python script. I am not sure how to do that. I followed the instruction given on the website: Accessing ECMWF data server in batch Unfortunately it says: "This ...
4
votes
1answer
172 views

Load NSF Research Award Abstracts in Python or R

I want to work on NSF Research Award Abstracts 1990-2003 Data Set in R. Every record of the data is stored in a text file and text files are grouped in folders (See data files here) Is there any easy ...
3
votes
0answers
354 views

Scraping data from a PDF in HTML - table recognition

I have a simple looking table in a pdf file: one column containing company names. I don't know how many there are in each group but groups are separated by a bold line (the lower border of a table). ...
1
vote
0answers
45 views

How can I deploy a sci-kit learn model when the data in use are .csv files?

I am dealing with accelerometer data (csv) as well as audio data stored in FTP server coming from an android app. Data is loaded from FTP in my notebook for exploratory analysis to extract features ...
5
votes
3answers
3k views

I get 'certificate verify failed' URLError when trying to download data from openFDA

I'm using Python and pandas to try to download the data in the FDA's 'Adverse drug event reports since 2004'. Here is my code: import pandas as pd import json from pandas.io.json import ...
1
vote
0answers
594 views

Using Google's Custom Search api [closed]

I am currently working for a social media site. We would like to use Google Custom Search Engine API and derive analytical insights from the results. My choice of development platform is R or Python. ...
4
votes
1answer
82 views

Lobby meetings of European Commission - Integrity Watch

The staff of the European Commission is obliged to keep a register of their meetings with lobbyists. The site Integrity Watch administered by Transparency International keeps track of these meetings ...
7
votes
2answers
491 views

Historical Twitter Data

I'm trying to get historical twitter data starting from 2010 with specific keywords related to S&P500 companies. I will probably end up using their tickers as keywords once I finalize my list of ...
4
votes
1answer
667 views

Datasets based on casual conversations for chatterbots

I am building a chatterbot that can answer questions related to tennis. I wish to know if there any any available datasets than can answer casual questions like "How are you", "How is the weather ...
6
votes
3answers
10k views

List of United States cities

I'm looking to get some data of this page http://www.greatschools.org/california/san-francisco/schools/?gradeLevels=e&page=2 To do it I need to create the links in this format: http://www....
2
votes
0answers
55 views

data.gov CKAN API ignores row parameter, returns duplicate results for different search terms

Using the data.gov CKAN api, this search via the web GUI returns 566 results: children However, post requests via the CKAN API returns 10 results despite the row parameter being set and, incidentally,...
5
votes
4answers
6k views

Looking for open dataset containing data for disease and symptoms

I am doing a data mining project on "health prediction system". So, Is there any open dataset containing data for disease and symptoms.
1
vote
1answer
227 views

Scraping vehicle reporting data from regulator's website

Website in question: http://www-odi.nhtsa.dot.gov/ewr/qb/index.cfm Automakers are required to report on a quarterly basis to NHTSA any claims brought against them that involve death, injury or ...
5
votes
1answer
70 views

Open database for JEL code crawling (using article names)

I'm currently working on a project to create a database (for analysis later on) of 100 journals, and all the articles for these journals between 1969 and 2014. This of course results in over half a ...
1
vote
2answers
383 views

Querying CKAN in Python using ckanapi

I used to query with ckanclient thusly: import ckanclient ckan = ckanclient.CkanClient('http://catalog.data.gov/api/3') search_params = { 'q': 'tags:"...
4
votes
1answer
267 views

CKAN - ckanext-scheming - Add dataset button

I've been following the instructions and the examples to configure ckanext-scheming extension (ckanext-scheming). I am able to access by URL to the different schemas (as the example shows) that I've ...