I'm looking for a free/open source downloadable database/API of all the businesses in North America. The sort of data that I'm looking for includes: name, address, industry, email address, website, ...
The license should preferably be compatible with commercial use. Does anyone know of such a database?
This question was previously answered for non-profits, and I think the same answer works here at least partially.
In addition, you can find all U.S. companies at the Securities and Exchange Commission's EDGAR site. Technical documentation is also available.
Information about Canadian companies is accessible through their open data site, but seems to be segmented by industry.
Mexico does not seem to have corporate content available publicly, but an explanation of where they are is at the Open Data Index.
There's a nice overview on Programmable about ways to access this type of information.
NOTE I'm not a lawyer and this is only a suggestion, please do consult a professional.
I've been in your shoes before, If you're willing to compromise precision for the sake of completeness in terms of the number of businesses, I would suggest scraping yellow pages and/or sites like yelp. There has been a long debate over the legality of this type of data scraping, and it seems to be in a gray area more towards being legal. Then again you should definitely see a lawyer for more info on that.
In my opinion the best method would be to use many sources and try to consolidate them using internal and/or external standards. For example, you can get a very sound dataset from yellow pages focused on small businesses, and some data on public businesses from NASDAQ (as mentioned in one of the answers) and try to normalize them for your own use.
Data/information seems to be very contextual and you usually have to do some mass operation on your datasets, why not use different sources while you're at it?
NASDAQ provides a CSV dataset for all corporations traded on NASDAQ, AMEX and NYSE. You can get some basic information on these publicly traded companies from the datasets.
http://www.nasdaq.com/screening/company-list.aspx
Below is a link on our OpenGeoCode website where we've downloaded the datasets and converted them to our CUDE CSV format.
http://www.opengeocode.org/cude1.1/US%20Stock%20Exchange/Stocks/index.php
I doubt that this exists. Lexis-Nexus provides access to US Federal Tax ID info (FEIN), but I imagine there are license restrictions. Public Companies are required to file much of this info and it should be available via the SEC Edgar interface. GuideStar provides access to nonprofit 990 data (which would include much of what you are looking for EIN data but the bulk data services are fee-based. ESRI provides access to business data for both the US and Canada via their Business Analyst service, but again, these are fee-for-service products. Verisign provides access to domain name registries, but I'm not sure what the license agreements are and obviously not all businesses have registered domains.
Finally, the issue of defining industry is challenging. If you mean NAICS/SIC, it can be quite challenging to drill down to individual level data. Often the BLS/BEA redacts industry data, particularly in non-metro areas, because the small number of establishments results in a defacto release of individual establishment data.
Good luck.
One thing you can do is to download the open source datasets of every city (most large cities in north america offer such datasets on their websites), they usually have datasets of business licenses issued which has a lot of info associated with them. I hope this can be starting point.
Also, be sure to check free database records usually compiled by each Secretary of State in each of our 50 states. All businesses who incorporate in each state can be found there.
I often use Georgia's Secretary of State incorporated business database. You just have to get creative with the keywords you enter.
As an aside, all British Company data is available at Companies House via the web interface or via CSV files: http://download.companieshouse.gov.uk/en_output.html
There are many commercial data sources for company data. Data quality varies widely. INFOT (infotdata.com) sells reasonably good databases for most major countries, but it's Yellow Pages quality. Many businesses are missing, and there's a fair amount of bogus information. In particular, web site names for companies are often inaccurate. It's good enough for marketing purposes, but marginal for business validation.
SEC EDGAR is useful, but it only covers publicly held companies in the US. We use it, and update our own database from it daily.
For the UK, there's one central source - Companies House - and they offer a full UK company list for free download, updated monthly.
The gold standard is D&B Firmographics. That will cost you about $800K/year just for the US.
https://brex.io offers API access to query business data from the US States. It's not 100% complete but only a few states are missing.
