I was working with the API and trying to paginate the results and I ran into an error which says 'Skip value must 5000 or less'. It would be nice if you guys could add this to your documentation because I had no idea that this limit was being imposed until I started to try and clean the data. Now I must go and download it all over again to make sure I've captured all the data.
-
this looks like a comment, not a question....albeit a good comment, i'm going to vote to close this if that is the case and you should post this on meta– albertJun 15, 2015 at 18:49
Add a comment
|
3 Answers
Thanks for your note. The skip limit was, until recently, unbounded. Unfortunately, we found that queries with large skip values were overloading the servers. To ensure the service remained available for everyone, we introduced the skip limits.
We are currently investigating if we can support large skip values without compromising the servers. We will update the documentation accordingly.
A workaround would be to modify the searching strategy. For example, breaking down the data you wanted in accordance with a date range.
Is this issue going to be resolved soon? Is there any workaround? I need to access data through multiple years which requires skip values over 5000, as the limit is only 100 for the API
