2

I am looking to retrieve every ASIN in Amazon's product database. In setting up a sample scrapy web scraping script - it became clear that crawling is not the efficient solution.

Looking at answers about the Product Advertising API I found a similar question: Amazon ASIN and Category

If the goal is to retrieve all of the physical products currently listed on Amazon is the following method the most efficient:

  • Get List of All Parent Categories
  • Find List of Sub Categories of the Parents (http://www.amazon.com/gp/site-directory/ref=nav_shopall_btn)
  • From that list of all sub categories that I am interested in use the Product Advertising API to find the "browsenodes" of each.
  • Use a recurring loop on each browse node until there are no more children nodes
  • Use that list of categories with api's ItemSearch to return all of the ASIN's in that node

This would be run as a python script - due to the immense number of categories/subcategories/sub-sub-categories and ultimately ASIN's I know this will be a huge collection of product ASINs.

Is there a resource that already has this information? Or is the usage of Amazon's API the most efficient method?

2

I found a large collection of product ASINs here
But it is pretty old. (belongs to 2015)

  • Amarnath thanks for the link - like you mentioned the list is quite old. We had to abandon the project because scraping really was the only method after the initial API request was made. The number of pages to scrape would take weeks to complete. – W00tW00t111 Jul 21 '16 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.