There's a big difference between “available to the public” and “belonging to the public”. By accessing a privately-owned website, you are accepting its terms and conditions. These conditions typically preclude scraping and aggregation. Here's an example from Amazon's (Canada) Licence and Access terms:
Subject to your compliance with these Conditions
of Use ...
My best guess is World petrol prices on MyTravelCost.com. They give you a large, configurable bar chart of gasoline prices. However, their source description can be described vague at best:
The data are drawn from a variety of sources including official government materials, oil companies, online resources specializing in gas prices, and others. These ...
The "FOOD AND AGRICULTURE ORGANIZATION OF THE UNITED NATIONS - Statistics Division" (link) provides historical data for commodity prices and agricultural production.
You can download for a single country or for all countries. Some data sets go back to at least the 1960s. You'll have to investigate which data sets may be best suited to ...
www.keepa.com is a similar website to camelcamelcamel. I cannot find a way to download the data but maybe you will.
In addition, Terapeak provide a free package with limited api calls (500/months) where you can find historical prices.
The data you're looking for is available here -:
It has historical data from 1918, it is free and open for use.
P.S- I work for datazar, an open source data library, where this file is hosted.
You are looking for the variables in MEPS (the Medical Expenditure Panel Survey from AHRQ) with the text slf in them. There are both utilization-level files as well as a consolidated person-level file available every year.
Index Mundi has a lot of commodity data (present and historical) available to the public.
But I am not sure if you would find toilet paper on the list (lol) - it does not exactly fit the definition for a commodity:
"a raw material or primary agricultural product that can be bought and sold, such as copper or coffee."
You can get release dates of versions/iterations at http://en.wikipedia.org/wiki/Adobe_Creative_Suite but I think you'll need to use a combination of Google and The Wayback Machine by archive.org to find pricing.
Also see http://www.computerworld.com/article/2517120/enterprise-applications/adobe-creative-suite--the-history.html
You can also scrape (or better yet contact and ask the folks at) https://thetracktor.com which seems to have some great historical data.
EDIT - Adding example:
https://thetracktor.com/ajax/prices/?id=881549&days=90 is the JSON response powering the graph at https://thetracktor.com/detail/B0054JJ0QW/, for example.
Check out these databases on Quandl:
Australian Bureau of Statistics:
Organisation for Economic Co-Operation and Development with data on Australia: https://www.quandl.com/data/OECD?keyword=construction%20cost%20australia
National Institute of Statistics and Economic Studies with data on France:
This might be helpful for other people if not for OP.
The problem here is that US Census data is based on census tracts, not zip codes. So for the US here is HUD "crosswalk" data from www.Huduser.gov that converts zip codes to census tracts, and vice versa. This will get you part way there, and may be very useful for other people. It's also recent, as of ...
Living Wage Calculator by MIT provides cost of living data down to locality, which you could mesh into zip codes, but since zip codes are not unique to a locality, you'd have to give or take something. perhaps use census zcta's? I'm not 100% on that, that's just the first thing that comes to mind. I don't see any bulk download options, but they do use a ...
Try looking at Numbeo. It is a crowd source project on user contributed real-time data. The cost-of-living section has data on gasoline prices.
Creative Commons Attribution-Sharealike 3.0 Unported License (CC-BY-SA) and GNU Free Documentation License (GFDL)
Medicare has a public dataset that may provide the information you are looking for:
Medicare Provider Utilization and Payment Data: Physician and Other
As part of the Obama Administration’s efforts to make our healthcare
system more transparent, affordable, and accountable, the Centers for
Medicare & Medicaid Services (CMS) has ...
The World Bank did this kind of project in 2012-2013:
The objective of the pilot was to study the feasibility of
crowd-sourced price data collection. Non-professional price collectors
used personal computers and mobile phones for collecting data and
entering it in a multilingual ...
Zillow is an online real estate database company and offers research data which is aggregated, and you may find meets some of your needs.
I've found a couple of sources of retail prices in open access:
http://data.worldbank.org/data-catalog/crowd-sourced-price-collection (same here https://app.enigma.io/table/org.worldbank.crowd-sourced.price.collection.csv)
But the World Bank covers only 8 developing countries. And PriceStats shares only US prices with a 10-day ...
A source at the US federal government level is GSA Advantage! which allows schedule holders (government contractors) to list products and services that government agencies can purchase much like an online shopping cart.
This is by no means clean or normalized data and runs into the issues described in the comment thread above about discounts/sales.
The World Bank provides data, but it is aggregated on the yearly level. Their source is German Agency for International Cooperation (GIZ).
Pump price for gasoline (US$ per liter)
Fuel prices refer to the pump prices of the most widely sold grade of gasoline. Prices have been converted from the local currency to U.S. dollars.
There is a download button ...
Wikipedia has the information for all countries:
Creative Commons Attribution-ShareAlike License
There is also http://www.globalpetrolprices.com/gasoline_prices/ but it has a much more restrictive BY-NC-ND license.
The US Bureau of Transportation Statistics provides various stats and datasets for historical airline ticket prices.
General overview, datasets and statistics
From the Route Fares page:
If you go through the PDF report (2012 to Q1 2014), you can find the corresponding raw data (CSV/Excel) for each table.
How about a government jurisdiction's ledger? This would be more granular than for each contract and instead be for each check written.
State of Iowa: Ledger data at https://data.iowa.gov/d/9sag-ywd8; Exploratory UI at http://checkbook.iowa.gov/
City of Danvilla, VA: Ledger data at https://data.danvilleva.gov/d/2b9c-d6ip; Exploratory UI at ...
I think it depends of the nature of the site, not only the terms and conditions. In the case of privately owned health medicine information, it could be an exception because of the nature itself of information or in the case of government site with all rights reserved default license, it could be argued that there is enough legal ground to try particular ...
Some kind soul scraped the newspaper listings and put the results on github here.
However, several of the things you ask are much more involved, require analysis and subjective judgment to furninsh, and won't be found in standard real estate datasets. For example, Quality of neighborhood can mean any number of things, even for hedonic modeling purposes. ...
You can see what days the Wayback Machine has archived a given URL by changing the date portion of the URL to an asterisk:
The problem is that they've only ever crawled that specific URL once, and it returned a redirect (status 302).
It's possible that there's some other service ...