I'm currently working on a project that would greatly be assisted by running the openFDA API locally. I tried to follow the READMEs but I can't get it to work.
-
Can you give me a little background on what this is for and what kind of machine you plan on running it on? Also, how comfortable are you with things like unix, docker, python and nodejs?– Hans NelsenCommented Jul 1, 2016 at 19:36
-
I'm working on a project that needs to count the number of reports per drug for a specific MedDRA adverse drug event. Currently the online API sets the limit to 1000 results, but when this occurs, the final drug count for an ADE would still be very large. In fact, I need to get all the ADE-drug report counts greater than 10, so I would be losing out on a lot of reports. This is important since I'm going to be calculating PRRs for the project. How strong of a machine do I need; we might have some open machines to run it on. I'm actually only comfortable with python and not much of the rest.– kdavid2Commented Jul 1, 2016 at 19:42
-
It seems like setting up the API for that would be overkill. The /drug/event.json pipeline can take over 24 hours to run. Then you would have to hack the limits that are baked into the API (nodejs code). It seems that the downloads are your best for that--it is pretty easy to loop over all the events in about an hour, especially if you have already downloaded everything. I posted a python script on one of your other questions that would be easy to alter to get what you need. Does that make sense?– Hans NelsenCommented Jul 1, 2016 at 21:06
-
Yes, alright I'll see how it does.– kdavid2Commented Jul 2, 2016 at 18:32
Add a comment
|
1 Answer
Given the API call limitations, and your need to run analytics on the entire dataset offline, it looks like you should opt to download the FAERS (adverse events database) instead of making repeated API calls. You can get it from here: http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm082193.htm
It will take you about a week of coding to clean, normalize and prepare the files for analysis in Python. The upside is that then you can do whatever analysis you need using pandas or any other Python library. You'll also need to take care of updating the data, if your project requires it. This is work that others have done already (you can get this data at http://www.johnsnowlabs.com).
-
Yes that is what I ended up doing; I just downloaded the JSON files used by openFDA and ran my python scripts there.– kdavid2Commented Jul 28, 2016 at 19:14