11
Not sure if this would classify as a comment or an answer, but it's useful information nonethelss:
So in reading this question I HAVE to point this out - ever heard of the paper?:
Arvind Narayanan and Vitaly Shmatikov. "Robust De-anonymization of Large Datasets (How to Break Anonymity of the Netflix Prize Dataset)".
The University of Texas at Austin ...
9
The databases at the International Labour Organization (specifically ILOSTAT and LABORSTA) are tantalisingly close to what you're after. For example, go to LABORSTA and select Employment, then select Employment for detailed occupational groups by sex (SEGREGAT). This allows you to select a country and view a breakdown of detailed occupational groups which ...
9
I found an API from Numbeo (documentation) that claims
1,485,832 prices in 4,786 cities entered by 177,368 users
(information updated 2014-11-25)
Here is their page related to the Cost of Living.
And an overview of the data for Zurich.
Their Terms of Service state that the license is CC BY-SA 3.0 and GNU Free Document License (GFDL).
What I can't ...
7
Quick answer with what comes to my mind. They might be several others.
Advantages
Since you lower the barrier to access the data (no more scraping), you will see more people using it and new usage of your data will appear
You will also receive feedback regarding your data quality (since there is more eyes looking at it under different angles)
This can be a ...
6
I'm not sure about specific examples in the USA, but the OpenStreetMap had some outstanding success in various countries. There are bound to be disaster stories in the US as well.
A famous example is the Haiti earthquake. It also has nice visuals.
Less known is perhaps the community mapping in Dar es Salaam, Tanzania:
http://www.opendta.org/Pages/...
6
I'm clear you're looking for open data and NOT vendors, but I've found some of these vendors do give attribution to their sources and thus aid your search from their marketing materials:
Panjiva
Import Genius
Datamyne
Goodwill
Fleetmon
Marine Traffic
Vessel Finder
Container Ship
Maritime Connector
Marine Vessel Traffic
Air Nav Systems
CMA
Marine Vessel ...
6
International organisations provide this kind of data. As you want to cover the whole World, have a look at organisations with the same geographical scope:
The World Bank has a database that covers a wide range of topics.
If you are more focussed on economic indicators, you can also have a look at the World Economic Outlook Database of the International ...
6
They answer (partially) in the faq: https://www.cia.gov/library/publications/resources/the-world-factbook/docs/faqs.html
Quoting them:
What is The World Factbook’s source for a specific subject field?
The Factbook staff uses many different sources to publish what we judge are the most reliable and consistent data for any particular category. Space ...
6
Lots of data from the US Federal "Office of Personnel Management" (OPM).
https://www.opm.gov/data/
OPM is the focal point for providing statistical information about the Federal civilian workforce. OPM's FedScope is an online tool which allows customers to access and analyze the most popular data elements from OPM's Enterprise Human Resources Integration ...
5
yes. the less barriers to accessibility, the easier it is to be consumed. the same principle applies to content.
open.all.the.things.
5
I don't know of any existing standards (de facto or not) for extending NAICS, but what I would do is start with the NAICS Index File, which ties over 19,000 industry names to the standard 1,000 or so NAICS codes. Take the list of index entries for each NAICS, give each one a 2-digit sequential ID, and tack this onto the standard NAICS to make it 8-digit (...
5
Open Data Barometer recently published this:
http://www.opendatabarometer.org/report/analysis/impact.html
also, the open data working group under the Open Government Partnership will be looking at impact:
http://www.opengovpartnership.org/groups/opendata/
5
The "FOOD AND AGRICULTURE ORGANIZATION OF THE UNITED NATIONS - Statistics Division" (link) provides historical data for commodity prices and agricultural production.
Production
Prices
You can download for a single country or for all countries. Some data sets go back to at least the 1960s. You'll have to investigate which data sets may be best suited to ...
5
First, note that ZIP codes are not geographies (see this other answer).
Also, as Kotebiya notes in a comment, there aren't pseudo-geographies for ZIP+4 and if there were, they would probably represent too small a group of people to allow data sharing without concern for individual privacy.
There may be commercial services that approximate this level of ...
5
This data is available from stats.oecd.org (for OECD countries plus Russia)
You seem to be able to get "mean disposable income" by the following age groups:
0 to 17
18 to 25
26 to 40
41 to 50
51 to 65
66 to 75
76+
Below is a screenshot that should help you get to the data.. unfortunately they do not have a way to save your query without logging in.
4
New research published at the weekend suggests that the UK currently has 270,000 digital companies (http://www.bbc.co.uk/news/business-23399134) out of about 2.1 million total businesses (http://www.ons.gov.uk/ons/rel/bus-register/uk-business/2011/sum-ukbusiness-2011.html).
This suggests that in the UK, at least, digital companies (most likely to emply ...
4
In Oakland we used a mix of open data and proprietary to plan out and implement a Community Land Trust- the data (foreclosures- public data sold by private firms, not open) helped to make the case for federal funding, and allowed us to target neighborhoods of high need (assessors data-open) as well as to purchase foreclosed homes while avoiding the most ...
4
Yes there is- I was part of a small review with Code for America folks of the McKinsey report looking at global value of open data- it's now published here: Open data: Unlocking innovation and performance with liquid information
4
Aswath Damodaran, Professor of Finance at the Stern School of Business at New York University, has been compiling corporate data on corporations worldwide into (FREE) datasets and providing them online since 1998.
You can find this information on Japanese firms (3258 companies), as well as other countries at this page:
http://pages.stern.nyu.edu/~adamodar/...
4
The UN has several sources of datasets relating to government expenditures and trade that are free to use for commercial and non-commercial usage. Generally, the terms require attribution.
Note, not all UN datasets are free to use.
UN Data - data.un.org
Terms of Use - http://data.un.org/Host.aspx?Content=UNdataUse
All data and metadata provided on ...
4
For the US, this information is generally obtained through the Bureau of Labor (BLS) jointly with the US Census. Data is collected on a regional, state and metropolitan statistical area (MSA). Here are some links:
BLS: http://www.bls.gov/data/#prices
BEA (Bureau of Economic Analysis): http://www.bea.gov/newsreleases/regional/rpp/2014/rpp0414.htm
4
Here's a link to the Consumer Price Index on Data.gov, which has Department of Labor (BLS) and other U.S. Federal agency data set up for download.
Cheers,
Martin
4
Data about imports and exports to the United States is available at several sources, but not necessarily at the shipping container level. You can see:
U.S. international trade in goods and services since 1960 from the U.S. Census (open data)
There are two sources referenced by @szxk earlier, which provide more detailed data, but are only free for the ...
4
Quandl has a dataset from the Bureau of Economic Analysis that might be helpful for you. It has many economic metrics at the state level.
Quandl is a great resource for datasets like this.
4
I can certainly help you with the income estimates. The Census Bureau publishes mean/average income at the ZCTA level. For cities and ZCTA's, the estimate come from a pooled 5-year sample that ranges from 2010-2014 back to 2005-2009.
You might only need the first table (S1901), but I figured posting the table giving information on the estimated aggregate sum ...
4
Wolfram Mathematica comes with curated data build in. For this answer I'll restrict the data collection to only a couple of counties but it is easily expanded to more.
We will use the AdministrativeDivisionData function for reference and the "PerCapitaIncome" property. Expand the Details section to see the list of data points available from this function. ...
3
This EU documentation shows various Value Added Tax rates for countries in Europe:
http://ec.europa.eu/taxation_customs/resources/documents/taxation/vat/how_vat_works/rates/vat_rates_en.pdf
3
I think the world is moving towards open data, the sooner you get on the wagon the better. If someone needs your data they're going to use 1001 trick to get it and if you want to stop them you'd probably end up in a legal battle where only the lawyers win!
3
The World Bank did this kind of project in 2012-2013:
http://data.worldbank.org/data-catalog/crowd-sourced-price-collection
The objective of the pilot was to study the feasibility of
crowd-sourced price data collection. Non-professional price collectors
used personal computers and mobile phones for collecting data and
entering it in a multilingual ...
3
Gas Buddy, the Price of Weed website (SE does not allow a link) and Craigslist all come to mind.
Only top voted, non community-wiki answers of a minimum length are eligible