I have been working to build a download and unzipping "pipeline" going (as a noob) with OpenFDA data, and I saw with interest @h.das' closed post about zips - "Python crashing when trying to read zipped json files", but this issue to me seems more like an issue with the source file than my noob status (tho of course I'm out a limb there) or the lack of RAM (I threw a TON of RAM at this, btw).

Programmatically with python 2.7's zipfile library and on the plain ol' bash cli on a ubuntu machine as well as through chrome on a win10 laptop I get (just now 2017-03-17 9am ish) a failed file error of one kind or another with the following URL: https://download.open.fda.gov/drug/event/2011q4/drug-event-0006-of-0007.json.zip

It's easy enough to pass over with exception handling but I don't want to miss some 300+MB of data in the middle of what seems like exactly what I want to look at.

  • on os x/firefox, i can download it by itself. and open it up. its 1.5gb json file. could there be file size limitations/restrictions in what you are doing? – albert Mar 17 '17 at 13:33
  • 1
    a failed file error of one kind or another is not an error message. If you want help, you have to be specific. And the 1.503.379.837 byte JSON file in it unpacks just fine on Windows. – Jan Doggen Mar 17 '17 at 13:48
  • 1
    Thanks for the responses, and I'm happy to be as specific as I need to be. I will look into this being a network limitation or some setting somewhere -- if others are actually able to download and work with the json, then it seems unlikely it's a bad file -- so that helps...I think. As for being more specific on the error messages, I get a network error on chrome in win10 and a badzip file error type through the exception construct in the python script I wrote -- does not seem like a need here* to get into those specifics. I'll look for filesize "switches" configs from other sources. Thanks – ksf Mar 17 '17 at 13:59
  • Hi @ksf, this is Jack from the openFDA team. I'm not able to confirm any issue with the file and was able to extract it without issue. As others have mentioned, it is a large file, which may be causing some problems. Let me know if you need any more assistance dealing with the openFDA API and data. – Jack Finch Mar 18 '17 at 21:31
  • Thanks, @Jack Finch, really appreciate it--and the gang. I'm looking into the file size thing . It looks like the reported error messages (esp the python exception) were sending me in the wrong direction. I'll shout if I have something. Best ... – ksf Mar 21 '17 at 13:04

Hi this is Jack from the openFDA team. I'm not able to confirm any issue with the file and was able to extract it without issue. As others have mentioned, it is a large file, which may be causing some problems. Let me know if you need any more assistance dealing with the openFDA API and data.

--Reposted as an answer for posterity--

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.