1

enter image description here

Can anyone give me pointers on how I can extract price and sales rank data from camelcamelcamel.com? They display only a chart showing the evolution of prices/sales rank over time, but I'm trying to extract the underlying data that generates this chart. I tried looking at the HTML, but can't find the relevant call based on which the chart is computed. Any help will be much appreciated!

(Disclaimer: This is related to a research project that I'm doing, and I don't seek to clone the website or to scrape it en masse in any way - I just need price data for a small set of products (1000 at max)).

Edit: Here's a screenshot of the networks tab of a sample page - I tried to see if any of these files potentially have the data, but can't find it at first glance, if anyone has any pointers that would be much appreciated!

Improve this question
3
  • this requires a log-in, which is the primary reason why this isn't open data, therefore i am flagging this for being closed for not following this site's rules around questions. i tried checking it out anyways to help, but the charts are behind a log-in and i'm not going to sign up. if you can provide a url/example, i'm happy to look into it for you. you can ping me at jalbertbowden - google's email service
    – albert
    Oct 25, 2017 at 16:47
  • 2
    I'm voting to close this question as off-topic because its not about open data.
    – albert
    Oct 25, 2017 at 16:47
  • Thanks @albert - the price data actually doesn't require a log in (although the sales rank data does - sorry for posting it here!). Here's a sample url that should display the price history chart without a log in: camelcamelcamel.com/product. If you have any pointers wrt extracting the underlying data, that would be awesome. Thanks and sorry again!
    – Shrabastee Banerjee
    Oct 26, 2017 at 16:54

1 Answer 1

Reset to default
2

As nearly as I can tell, the image is generated server-side (not HTML5 canvas or anything). A sample image:

https://charts.camelcamelcamel.com/us/B001444G5Q/amazon.png?force=1&zero=0&w=725&h=440&desired=false&legend=1&ilt=1&tp=all&fo=0&lang=en

In other words, the database and underlying data may not be available to end users at all.

You could try parsing the image itself, using a combination of OCR and looking for green (the decimal RGB values appear to be 99,168,164, identified by https://www.beautycolorcode.com/63a8a4 as "moderate cyan"), but that would be ugly.

You could also try contacting them directly.

Improve this answer
4
  • Thanks @Barry Carter, I guess getting in touch with them would be the easiest thing to do at this point!
    – Shrabastee Banerjee
    Oct 26, 2017 at 16:57
  • the data is definitely in the document...look @ the tables below the images. i just found this piece of the puzzle in network tab: us-ads.openx.net/w/1.0/…
    – albert
    Oct 26, 2017 at 18:42
  • hi albert, i had tried going through many entries in the network tab but couldn't find anything that potentially had the data - how do you suggest making use of the us-ads link you posted to try and get at the data?
    – Shrabastee Banerjee
    Oct 26, 2017 at 22:13
  • idk man. actually i don't think the us ads is part of it..sorry
    – albert
    Oct 26, 2017 at 22:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.