I am totally lost with the OpenFDA API. I've downloaded the API from Github, run the bootstrap.sh, run python setup.py install, and I have no clue what to do now.

My specific questions:

1. Downloading data

The API Reference page mentions a way to write code to download the data automatically, and then it gives me a nice bit of "download API query" code without every telling me how to use this code from my machine (this means: to download the dataset, not as a query to retrieve database information using search or connect. This was already covered in excellent detail and I understand it quite clearly).

2. Running the API Locally

When I navigate to api/faers/ and open the README.md file, the first instruction tells me to "Get an ES instance running locally with data in the FAERS mapping format."

Um, how do I do that?

If I proceed with the other steps, I get connection refused errors on step 4.

3. Querying from my machine

Similarly to question #1, I'm confused about how to query the database from my machine and especially how this relates to the API key. Can someone give me some example code or point to a tutorial on this---I can't seem to find any.

As a final note, I am not interested in any links to sites you have made that have already done this for me; I am only asking for help with setting it up for myself.

Thank you!

  • 1
    I can put together an answer for you. I just need to know a few things first. What OS are you running locally? What are you trying to accomplish by querying the API? Do you need all of the APIs (there are around 9) or just a certain one? Jun 20, 2016 at 17:06
  • Can you describe what it is that you're trying to do? In other words, what are you planning to build/make with openFDA? That will help narrow the answers to your questions. As it stands, #1 is a question about downloading the current data in JSON format from the existing API. #2 is about setting up your own custom COPY of the whole API server etc.
    – Adam
    Jun 20, 2016 at 18:09
  • add this as a comment. its not an answer
    – albert
    Jun 20, 2016 at 18:16
  • This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - From Review
    – philshem
    Jun 20, 2016 at 19:00

1 Answer 1

  1. Downloading data: there is an endpoint: https://api.fda.gov/download.json. The response is a list of all of the downloads. It is meant to be used in lieu of scraping the page for the links. Once you have the paths, you can use something like python requests library to grab the data.
  2. Running the API locally: you will need to have node installed, run npm install from the ./api/faers directory, have elasticsearch running locally (or port forward localhost:9200 to a running instance), finally, run node api.js from within the ./api/faers directory. This presumes you have run the pipeline and loaded an elasticsearch instance. This is why I asked to see what exactly you need, because /drug/event.json and /device/event.json take awhile to run. The rest run in short order, so running them on a laptop is not a big deal.
  3. Querying: The examples on the documentation site show you everything you need to know. https://open.fda.gov/drug/event/ has the example: https://api.fda.gov/drug/event.json?search=receivedate:[20040101+TO+20160601]&count=receivedate. To call this with an api key, just add that on as a parameter &api_key=xxxxxxxxxx. If you are running the api locally, just replace the https://api.fda.gov/ with http://localhost:8000/. Everything you need to know about querying the API is here: https://open.fda.gov/api/

A python example for calling the api:

import requests

URL = 'https://api.fda.gov/drug/event.json?search=receivedate:[20040101+TO+20160601]&count=receivedate'

data = requests.get(URL).json()

for result in data.get('results', []):
  print result

There are limits to what you can do with the API. I would suggest seeing if the one that is maintained by the FDA works for you, since it will save you a lot of time and money. If you need more than 5000 on a skip and limit loop, then you can either use the downloads to get all the API output as zips (question #1) or you can standup your own server and remove the limits (question #2).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.