I think this is what you are looking for. It is a DNS registration dataset snapshot taken in 2013. Compressed - it is ~15GB and uncompressed 157GB.
They claim it contains: Dataset containing 2,676,380,336 DNS records and 106,928,034 domains
Wikipedia Releases Clickstream Data
Wikipedia has released a data set of clickstream data for January 2015. A clickstream is the path a user requests to get to a desired web page or article by using a referer—clicking on a link or performing a search. The dataset contains 22 million referer-article pairs from the English language, desktop ...
This isn't a complete dataset, but you will find all .gov.uk domain names with the following info:
Next Renewal Date
I think you're unlikely to find this as true "open data," since there are substantial privacy concerns to internet traffic monitoring, and the volume of data involved to do proper analysis is considerable.
That said, there is an organization, the Center for Applied Internet Data Analysis (CAIDA), which makes datasets available to researchers under oversight....
The DNS Census 2013 is an attempt to provide a public dataset of registered domains and DNS records.
The torrent is about 15 GB (uncompressed: 157 GB).
I am pretty sure it contains domain names and IPs, but I am not 100% sure as I have not downloaded it to check. Please let us know, thanks!
You can extract TLD from domain name.
This may be for only an old version (4.1), but...
ClickStream Example Database Wayback Archive
Example Database File Locations
The example databases are installed in:
Update: Download instructions (link denied due to registration)
Some projects such as the Open Data Monitor aim to answer this question. For convenience, I split the answer into four sections.
The amount of open data released by governments is reasonably documented. Perhaps not in the sense of the exact amount of Gb.
The Open Data Index by the Open Knowledge Foundation
The Open Data ...
The caniuse.com project includes compatabilities for many browsers versus CSS, HTML, JS API, SVG, and Other categories.
"Can I use" provides up-to-date browser support tables for support of front-end web technologies on desktop and mobile web browsers.
License is CC BY-NC 3.0.
Raw data is on GitHub.
I don't think the coverage is very complete, but Open Street Maps has a key called internet_access.
With this key you can use tags like wlan to find open WLAN.
To find free WLAN, add the internet_access:fee=no option
I just snagged it off the wayback machine here:
but more files can be found here by searching for '.tar.gz' or '.db.gz'
Download the data
Download links from ...
BroadbandMap.gov, by the FCC, has a raster map of fiber to the end user at http://www.broadbandmap.gov/technology/fiber-to-the-end-user
Unfortunately it does not seem to have the data available as anything other than a tile service.
Edit 1: You might consider requesting the data from a FCC Data Officer
Edit 2: Also, have you seen http://www....
I suggest you use data from Caida (Cooperative Association for Internet Data Analysis). In particular their Macroscopic Internet Topology Data Kit (ITDK) may suite your needs.
Quoting from the above page:
The ITDK contains data about connectivity and routing gathered from a large cross-section of the ...
This image is prominent on the Wikipedia article for Internet backbone.
Each line is drawn between two nodes, representing two IP addresses. This is a small look at the backbone of the Internet.
The source and raw data can be obtained from http://www.opte.org/maps/.
Unfortunately, the data is from 2005.
If you can tolerate the HTML, you can find long ...
I would suggest characterizing the availability not by the quantity of data but by the expansion of open data portals. Here are some suggestions:
Visit the various open data catalogs which list (government and non-government) open data portals.
A. Estimate the number of open data portals that are available.
B. Look at when they were launched (...
To expand on the answer of Anastasios Ventouris, I found an official list for US .gov domains.
Federal Executive Agency Internet Domains as of 03272014
There are also many filter, view, and export options (i.e. JSON). I don't know how the URL will be updated since it includes date information.
This does not resolve the conflict, but traces its origin.
One of the first links that appears from your Google query is for
Internet Live Stats that also presents the 84.2% number and what seems to be a justification by giving the total population and the number of internet users. That source cites "Elaboration of data by International Telecommunication ...
Yes, there are (U.S.) public agencies that are currently (2017-03-07) streaming real-time video data. I'm not aware of a list of them (would LOVE it, if one such exists), but here's one:
Live Video Stream - NOAA Ship Okeanos Explorer
I'm not familiar with DataTurbine, nor aware of similar tools, but that's not to say they don't exist.
Seattle has been ...
There are an estimated 1.6 billion public websites in the world, with 200 million being active. But no one really knows, because a website can be just an IP address with no domain, or an .onion link, or temporary or short-lived, or weather-dependent, ...
One way would be to use known DNS records (domain names) from the 2013 DNS Census.
It is a DNS ...
There is the good old Linux counter project1 which estimates around 69 million linux users. Now this is not exactly the same as number of online linux machines, but it should at least give some reference for what the number might be.
1 Actually this is a newer incarnation of the original counter.li.org.
Another source of domain names is https://www.spamhaus.org/ a non-profit organization that tracks spam related activities.
They have a Domain Block List (DBL) that includes spam domains:
I'm afraid this is generally the kind of data which is not centrally collected by a government agency, and not freely given by businesses that set up to collect it for their own use or for sale to third-parties.
That said, NASDAQ has a list of companies by industry that is 6500+ entries (not limited to the NASDAQ exchange). I'm not really sure how they ...
DUNS is the best business list that I can think of -- but I have no idea if it has industry info in it. Years ago, when you wanted an SSL certificate, registrars wanted your DUNS number to verify you weren't mascarading as another company. Unfortunately, it's not an open list.
For something that's more likely to be free, for the U.S., there's a the EIN, ...
I publish the Google Play Store statistics of one of my apps:
See for instance the installs per device type. It is fragmented by day, with metrics such as installs/uninstalls/upgrades.
It is less than 100K rows, though, as data is pre-aggregated.
I'm not sure if it fits your criteria, but consider the A Week in the Life of a Browser data set, which combines browsing history with survey data.
Test duration: 7 days
Test type: Global
Firefox versions covered: Fx 3.5 and Fx 3.6, Fx4 Beta
Data submission: 527,817 test sets submitted in November 2010.
Table Name: 'users'
Main table of ...
There is a good source of "unofficial" internet data from the 2012 Internet Census.
While playing around with the Nmap Scripting Engine (NSE) we discovered an amazing number of open embedded devices on the Internet. Many of them are based on Linux and allow login to standard BusyBox with empty or default credentials. We used these devices to build ...