Questions tagged [python]

For all questions and requests concerning or related to Python, a highly readable and dynamic, general purpose programming language.

Filter by
Sorted by
Tagged with
7
votes
2answers
424 views

Historical Twitter Data

I'm trying to get historical twitter data starting from 2010 with specific keywords related to S&P500 companies. I will probably end up using their tickers as keywords once I finalize my list of ...
6
votes
3answers
8k views

List of United States cities

I'm looking to get some data of this page http://www.greatschools.org/california/san-francisco/schools/?gradeLevels=e&page=2 To do it I need to create the links in this format: http://www....
5
votes
2answers
2k views

I get 'certificate verify failed' URLError when trying to download data from openFDA

I'm using Python and pandas to try to download the data in the FDA's 'Adverse drug event reports since 2004'. Here is my code: import pandas as pd import json from pandas.io.json import ...
5
votes
4answers
5k views

Looking for open dataset containing data for disease and symptoms

I am doing a data mining project on "health prediction system". So, Is there any open dataset containing data for disease and symptoms.
4
votes
1answer
615 views

Datasets based on casual conversations for chatterbots

I am building a chatterbot that can answer questions related to tennis. I wish to know if there any any available datasets than can answer casual questions like "How are you", "How is the weather ...
4
votes
1answer
130 views

Load NSF Research Award Abstracts in Python or R

I want to work on NSF Research Award Abstracts 1990-2003 Data Set in R. Every record of the data is stored in a text file and text files are grouped in folders (See data files here) Is there any easy ...
4
votes
1answer
80 views

Lobby meetings of European Commission - Integrity Watch

The staff of the European Commission is obliged to keep a register of their meetings with lobbyists. The site Integrity Watch administered by Transparency International keeps track of these meetings ...
4
votes
1answer
68 views

Open database for JEL code crawling (using article names)

I'm currently working on a project to create a database (for analysis later on) of 100 journals, and all the articles for these journals between 1969 and 2014. This of course results in over half a ...
4
votes
0answers
226 views

CKAN - ckanext-scheming - Add dataset button

I've been following the instructions and the examples to configure ckanext-scheming extension (ckanext-scheming). I am able to access by URL to the different schemas (as the example shows) that I've ...
3
votes
1answer
801 views

Average Monthly temperatures for cities around the world

I am looking to find the average historical temperature in april (not the average temperature in april of 1988, for example) in different cities (about 400 cities across the world) and store that data ...
3
votes
2answers
611 views

Access ECMWF Data via python script

I want to access ECMWF data via a python script. I am not sure how to do that. I followed the instruction given on the website: Accessing ECMWF data server in batch Unfortunately it says: "This ...
3
votes
1answer
104 views

Not able to view private datasets in CKAN using Nginx

I have installed CKAN v2.7.2(served under HTTPs using Apache web server),Nginx using docker. The task I am doing requires me to mount ckan on /ckan mount point, thus have added ckan.root_path = /ckan/...
3
votes
1answer
231 views

World Bank via Pandas - looking up country regions

I am getting country data from World Bank via Pandas Remote Data API. Is there a way to get metadata for each country? In particular: Is it a country, or a region/group (e.g. Arab World or Upper ...
3
votes
0answers
329 views

Scraping data from a PDF in HTML - table recognition

I have a simple looking table in a pdf file: one column containing company names. I don't know how many there are in each group but groups are separated by a bold line (the lower border of a table). ...
2
votes
0answers
19 views

Are there any available lexical databases for English, containing predicate-argument (relation-role) information?

I am looking for a database (something like WordNet) containing predicate-arguments information (also known as relations-roles). I.e. what are the arguments of a given predicate? For example, loves(...
2
votes
0answers
53 views

data.gov CKAN API ignores row parameter, returns duplicate results for different search terms

Using the data.gov CKAN api, this search via the web GUI returns 566 results: children However, post requests via the CKAN API returns 10 results despite the row parameter being set and, incidentally,...
1
vote
1answer
314 views

Python crashing when trying to read zipped json files [closed]

I downloaded the 2015 adverse drug events data from openfda and I want to run some analysis with python. I am a python newbie, but have become comfortable with the scikit-learn libraries, but I cannot ...
1
vote
1answer
208 views

Scraping vehicle reporting data from regulator's website

Website in question: http://www-odi.nhtsa.dot.gov/ewr/qb/index.cfm Automakers are required to report on a quarterly basis to NHTSA any claims brought against them that involve death, injury or ...
1
vote
1answer
314 views

Querying CKAN in Python using ckanapi

I used to query with ckanclient thusly: import ckanclient ckan = ckanclient.CkanClient('http://catalog.data.gov/api/3') search_params = { 'q': 'tags:"...
1
vote
0answers
13 views

Where can I find data to create small sample drug database?

For my diploma thesis I want to build an application to control medicine dosage and to do this I think about making sample database with like 1000 medicines with info most importantly about a dosage (...
1
vote
0answers
35 views

average ingredients for each drug in each year for a specific manufacturer

I want to count the average number of ingredients in each drug produced by a specific manufacturer in each year. The results I want should be like this: year | drug_names | avg_number_of_ingredients ...
1
vote
1answer
13 views

Which is the best way to work with NCBI data obtaining online partial information from the whole

I was trying to create a database from NCBI Nucleotide bank. I did a query which gave me 1124 results. From each single result I was wanting to obtain only the items realted to Accesion Country and ...
1
vote
1answer
49 views

Dataset of lines at n miles from the coast

I'm using Folium to generate some interactive maps with ocean data. One of the restrictions I am facing is that the objects, I plot has to be at-least at 5, 6, .., n miles from the coast, hence I am ...
1
vote
0answers
61 views

To fetch bigger dataset from twitter

I need twitter data for my project, but with streaming api I am able to get only 15 min window frame.Hence a very smaller sample space. I have a written code in python as follows import tweep from ...
1
vote
0answers
44 views

How can I deploy a sci-kit learn model when the data in use are .csv files?

I am dealing with accelerometer data (csv) as well as audio data stored in FTP server coming from an android app. Data is loaded from FTP in my notebook for exploratory analysis to extract features ...
1
vote
0answers
533 views

Using Google's Custom Search api [closed]

I am currently working for a social media site. We would like to use Google Custom Search Engine API and derive analytical insights from the results. My choice of development platform is R or Python. ...
0
votes
2answers
60 views

Looking for Databases with gender of names and Ethnicity information

I want to build a classifier which can classify gender and ethnicity based on the names. I am looking for datasets to download where I take samples for supervised learning. If there are no open ...
0
votes
2answers
38 views

Is it possible to link drugs, with protein targets, using OpenFDA?

I have a set of protein targets. The absolute ideal return for these targets for me, would be a .tsv file that looks like this: target_name fda_approved_drug target1 drug1 target2 - ...
0
votes
1answer
39 views

Cannot get data from FPDS atom feed using pyfpds

I have a script which uses the pyfpds wrapper (https://github.com/18F/pyfpds) to extract contract data from the FPDS Atom Feed (www.fpds.gov). Following is the code I have and this used to work till ...
0
votes
1answer
231 views

KEEPA API Experience

I'm trying to pull historical prices and BSR from about 3,000 ASINs on Amazon, but need something built to interface with Keepa's API. Does anyone have any experience with this? If so, is there a ...
0
votes
0answers
6 views

How to install alpha_vantage for python 3 using Anaconda [migrated]

I cannot install alpha_vantage using conda install -c manaliss alpha_vantage because of the error message below. I have tried updating each of the libraries listed using 'conda update' and they all ...
0
votes
0answers
11 views

Is there any API, tool, websites that i can find how many German people moved to United States in recent 10 years?

I need to do geographical population analysis on how many German people recently moved to US and started to living there, especially to the California. I know US government has open data policy but so ...
0
votes
0answers
29 views

Open source weather database for predict crop disease?

I have started to work on forecasting models for precision agriculture. My goal is to train a model which uses temperature, relative humidity, and rainfall to predict when late blight (Phytophthora ...
0
votes
0answers
15 views

CKAN : ValueError: VDM only works with SQLAlchemy versions 0.4 through 0.9, not: 1.1.2

Do I need to upgrade the package vdm to make it compatible with sqlalchemy version greater then 0.9? This is a CKAN question.
0
votes
1answer
48 views

How to check whether a word is abbreviated or not using Python? [closed]

How do I check whether a word is abbreviated or not in a dataframe column using Python? For instance, I need to detect the value "U.S.A." as an abbreviation. Is there any dictionary present for this?
0
votes
1answer
90 views

Dict not able to return data from college scorecard API

So I've been working on a page which displays a school's name, url, city, state, zip, and student size via College Scorecard API but I'm getting tons of errors instead. However, the program is able to ...