3

I am looking to retrieve every ASIN in Amazon's product database. In setting up a sample scrapy web scraping script - it became clear that crawling is not the efficient solution.

Looking at answers about the Product Advertising API I found a similar question: Amazon ASIN and Category

If the goal is to retrieve all of the physical products currently listed on Amazon is the following method the most efficient:

  • Get List of All Parent Categories
  • Find List of Sub Categories of the Parents (http://www.amazon.com/gp/site-directory/ref=nav_shopall_btn)
  • From that list of all sub categories that I am interested in use the Product Advertising API to find the "browsenodes" of each.
  • Use a recurring loop on each browse node until there are no more children nodes
  • Use that list of categories with api's ItemSearch to return all of the ASIN's in that node

This would be run as a python script - due to the immense number of categories/subcategories/sub-sub-categories and ultimately ASIN's I know this will be a huge collection of product ASINs.

Is there a resource that already has this information? Or is the usage of Amazon's API the most efficient method?

Improve this question

1 Answer 1

Reset to default
2

I found a large collection of product ASINs here
But it is pretty old. (belongs to 2015)

Improve this answer
1
  • Amarnath thanks for the link - like you mentioned the list is quite old. We had to abandon the project because scraping really was the only method after the initial API request was made. The number of pages to scrape would take weeks to complete.
    – W00tW00t111
    Commented Jul 21, 2016 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.