I have a discussion in a Google Group about the next thing:
One person wants to scrape multiple web sites to gather prices of different types of technology products (cell phones, TVs, computers, tablets, etc.) and then provide this collected information as Open Data.

I said that this is not open data, because the person does not own the source of the data. However, other people say that it is open data, because the prices are available to the public, and they give examples of multiple companies that do similar stuff of scraping multiple sites.

Can scraping specific data from multiple websites and providing this in a database be considered as open data? Where can I find information about this activity? I want to read articles where this action is accepted as Open Data OR specifying that this is not open data, but just an extraction and cannot be reused.

Many sites explain OpenData for governments, but very few talk about open data for the private sector.

(The Google group is the Spanish OpenDATA Productos de consumo)


There's a big difference between “available to the public” and “belonging to the public”. By accessing a privately-owned website, you are accepting its terms and conditions. These conditions typically preclude scraping and aggregation. Here's an example from Amazon's (Canada) Licence and Access terms:

Subject to your compliance with these Conditions
of Use and your payment of any applicable fees,
Amazon.ca or its content providers grant you a limited,
non-exclusive, non-transferable, non-sublicensable
licence to access and make personal and non-commercial
use of the Amazon.ca Services. This licence does
not include any resale or commercial use of any
Amazon.ca Service or its contents; any collection
and use of any product listings, descriptions, or
prices; any derivative use of any Amazon.ca Service
or its contents; any downloading or copying of account
information for the benefit of another merchant; or any
use of data mining, robots, or similar data-gathering
and extraction tools.

The last clause is pretty clear that you can't legally scrape from their website.

  • 1
    I concur with the response. US Copyright Laws extends to catalogs (collections) of information that is otherwise public. The originator owns the right to their catalog of information. – Andrew - OpenGeoCode Feb 8 '16 at 19:14

I think it depends of the nature of the site, not only the terms and conditions. In the case of privately owned health medicine information, it could be an exception because of the nature itself of information or in the case of government site with all rights reserved default license, it could be argued that there is enough legal ground to try particular data as public one, even if there is not a explicit permission. So I would recommend go on a case per case base seeing exceptions and limitations of the data according to the nature of the data itself and the site, instead of a single answer fits all, by some canonical definition (like Open Knowledge's or others).

  • 1
    This is a dangerous assumption to make. Consult a lawyer before attempting this. – scruss Feb 9 '16 at 1:36
  • Totally agree: Consult a lawyer. Here is a research on published data by government agencies, where I take part with the data visualization and the information was not scrapped, but because of being public and made by govs on meds, collected analyzed data (not original one) will be republished as open data/research. So see how much of this applies for case like yours. There is some consumer association or gov regulation caring about publishing this data in your country? – Offray Feb 9 '16 at 15:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.