I just downloaded the openFDA source code from the github, as well as all the event endpoint JSON files, and I wanted to know which directory to save the JSON endpoints. Some also seemed to have the same file name, e.g. when both 2004Q1 part1 and 2004Q2 part1 are extracted, they result in drug-event-0001-of-0002.json. How should I distinguish them within openFDA?
4
-
I am not sure I understand your question. What do you mean by "How should I distinguish them within openFDA?"– Hans NelsenJul 1, 2016 at 18:51
-
Sorry about that. I just read the README inside /api/faers in the source code and it says "Get an ES instance running locally with data in the FAERS mapping format." How do I do this after downloading the endpoints from openFDA?– kdavid2Jul 1, 2016 at 19:14
-
The downloads are meant to be used instead of the API. If you want to have the API up and running locally, you will need to run the pipelines locally, which will take a lot of resources. If you want to post a question "How do I run the openFDA API locally?" I can try to answer that.– Hans NelsenJul 1, 2016 at 19:22
-
Alright, just posted another question!– kdavid2Jul 1, 2016 at 19:31
Add a comment
|
1 Answer
Just to be clear, the downloads cannot be shoehorned into the API code. They are meant to be used in place of the API. From what I understand, they will be perfect for you. Once you have downloaded in this fashion, you can glob over all the zips and process each one for your pairs.
If I understand you question (that the file names are the same) then you will have to do something like the following:
import os
import requests
URL = 'https://api.fda.gov/download.json'
BASE_DIR = './downloads'
os.system('mkdir -p %s' % BASE_DIR)
downloads = requests.get(URL).json()
drug_events = downloads.get('results', {}).get('drug', {}).get('event', {}).get('partitions', [])
for event in drug_events:
zip_url = event.get('file', None)
if zip_url:
parsed_url = requests.utils.urlparse(zip_url)
target_file = os.path.basename(parsed_url.path)
target_base = parsed_url.path.replace(target_file, '')[1:]
target_dir = os.path.join(BASE_DIR, target_base)
os.system('mkdir -p %s' % target_dir)
os.system('curl --progress-bar %s > %s' % (zip_url, os.path.join(target_dir, target_file)))
