0

I have been looking to extract all data from FDA so I can use it all.

  1. I created this code below that works but seems it is too much data it fails. This way it just gives me 25000 records.
url = 'https://api.fda.gov/drug/ndc.json?sort=dea_schedule:desc&limit={}&skip={}'

all_data_df = []

limit = 100
for skip in range(0, total, limit):
    query = url.format( limit, skip)
    print('query:', query)
    data = pd.read_json(query, orient='values', typ='series', convert_dates=False)
    data = data['results']
    all_data_df = pd.DataFrame(all_data_df.append(data))

I need all data. FDA site says to use paging here the link https://open.fda.gov/apis/paging/ This below will give me the next page that they mention but I am not sure how to do put it together. Any help is appreciated

re = requests.get('https://api.fda.gov/drug/ndc.json?sort=dea_schedule:desc&limit=100')
url =re.links['next']['url']
0

It appears your best option would be to use the downloadable JSON files: https://open.fda.gov/data/downloads/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.