I was working with the API and trying to paginate the results and I ran into an error which says 'Skip value must 5000 or less'. It would be nice if you guys could add this to your documentation because I had no idea that this limit was being imposed until I started to try and clean the data. Now I must go and download it all over again to make sure I've captured all the data.
Thanks for your note. The skip limit was, until recently, unbounded. Unfortunately, we found that queries with large skip values were overloading the servers. To ensure the service remained available for everyone, we introduced the skip limits.
We are currently investigating if we can support large skip values without compromising the servers. We will update the documentation accordingly.