Lately I have been playing around with the Amazon Product Advertising API, and it made me very curious about how Amazon-price checking sites like Keepa and camelcamelcamel actually work.
For a small number of requests, the Amazon Product Advertising API is quite straightforward. However, even if you are pushing a lot of revenue to Amazon, I don't see how you could pull the amount of data that these sites have based on the published API limits. The relevant docs from amazon imply that the absolute maximum number of requests is about 900,000 (ten per second per day):
The tracking sites, on the other hand, claims that they are pulling daily prices for nearly seven million items, and that they are pulling prices much more frequently for the most popular items.
So how are they doing it? Do they have a sweetheart deal with Amazon, or are they screenscraping or something? Any ideas or insider info?