I am trying to do some classification and clustering on the openFDA dataset. I started by downloading a year's worth of data and realized I run into system limits and would need to solve for that.
Then I decided to query for just one drug indication, the subset I am looking at has about 422k records. But I realized that I hit the skip limit and cannot get the data through the API.
I then decided to run the api locally so as to push through the limits, but I am unable to find clear instructions on how to do it. I have installed node, elasticsearch, and run bootstrap and npm install? I believe I need to run the pipeline, do I just run all-pipelines.sh?