Barry, could you abuse this site's "answer your own question" feature to create a community wiki answer for sources of weather data (both current and historical), since it gets asked so often?
This list is (and may always be) incomplete. Please add to it when you can.
Per minute data: ftp://ftp.ncei.noaa.gov/pub/data/asos-onemin
NOAA's ISD data (goes back to ~1900 in some cases):
https://www.ncdc.noaa.gov/isd (home page)
ftp://ftp.ncdc.noaa.gov/pub/data/noaa/ (direct link to data, some stations include multiple report per hour in some cases)
https://www1.ncdc.noaa.gov/pub/data/noaa/isd-lite/ (direct link to "lighter" version of data-- hourly observations only)
Global Historical Climatology Network (GHCN) observations, some dating back to the mid 1700s:
http://mesowest.utah.edu/ includes solar radiation and snowfall accumulation, which some other sites don't. Sample:
The "raw" data as it comes into NOAA from airports, surface weather stations, boats, and buoys:
You can read more about these "cycle" files at: http://www.nws.noaa.gov/tg/datahelp.php Note these files are updated every few minutes and have not been curated, so data can be inaccurate. Reports in these cycle files are sometimes updated (and thus invalidated) in later cycle files.
To translate station codes to physical locations:
- The file "isd-history.csv" in ftp://ftp.ncdc.noaa.gov/pub/data/noaa/
- Commercial source (but some portions are free): http://www.weathergraphics.com/identifiers/
Well known sites:
Get the data from BigQuery:
#standardSQL SELECT name, stn, a.wban, TIMESTAMP(CONCAT(year, '-', mo, '-', da)) date, temp, prcp FROM `bigquery-public-data.noaa_gsod.gsod2*` a JOIN ( SELECT * FROM `bigquery-public-data.noaa_gsod.stations` WHERE country='US' AND name LIKE '%HONOLULU%' AND state LIKE 'HI' LIMIT 1000 ) b ON a.stn=b.usaf AND a.wban=b.wban WHERE prcp!=99.99 ORDER BY name, date
Related: Finding the rainiest US cities
A remodel of weather since 1979, based on many many measurements. The resulting dataset contains daily to hourly values around the globe for many meteorological and climatological parameters such as temperatures, water content, wind, radiation and various fluxes.
The most up to date version of this dataset is ERA 5: https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5
A project similar to ERA 5.
http://www.esrl.noaa.gov/ The Earth System Research Laboratory, with data and information or aerosols, as well as other temperature, precipitation etc. Within this organisation:
(closed data) A commercial service that hosts a lot of high detail weather data, and allows a limited number (1000/day) of requests for free: https://darksky.net/. Darksky has been acquired by Apple and their API is no longer available.
- WorldClim: Climate data for past, present and predicted future conditions. Includes temperature (min, max, mean) and precipitation. 1km x 1km resolution.
- NCAR GIS Climate Change Scenarios: Lots of data from the National Centre for Atmospheric Research models, including data used by the IPCC in their reports. Registration required
- CRU Climate Datasets: Various datasets including temperature, precipitation, pressure, drought, paleoclimate and others, from the (infamous) Climate Research Unit at the University of East Anglia. High and low resolution datasets available.
- Downscaled GCM Portal: A wide range of downscaled (that is, higher resolution data) created from the outputs of a wide range of GCMs. Covers the majority of important climate variables.
- European Climate Assessment and Dataset: Gridded observation data across Europe, including cloudiness, temperature, precipitation, humidity, pressure, snow and sunshine.
- Global Potential Evapotranspiration and Aridity Index: Potential Evapotranspiration data and indices of aridity at 30 arc-second resolution in raster format.
- GlobAerosol: Aerosol data in raster format (provided as NetCDF files), including estimations of AOD at 550nm and the Angstrom coefficient.
- Global Aerosol Climatology Project: Monthly averages of Aerosol Optical Thickness and the Angstrom exponent from 1981 to 2006.
- Satellite Application Facility on Climate Monitoring: Provides near real-time and retroactively-generated datasets of cloud cover, type and temperature, surface radiation budget and temperatures, amongst others.
- Climate Analysis Indicators Tool: Carbon Dioxide emissions data by country, and by US state, including measures of adaption.
- International Satellite Cloud Climatology Project: Monthly averages of a number of cloud variables including cloud cover (and associated breakdown by frequency), cloud top temperature, cloud water path, cloud optical thickness etc.
- Climatological Database for the world's oceans (CLIWOC): Data compiled from ships logs compiled during voyages between 1750 and 1850. Includes various meteorological observations, broken down by date, ship and year.
- IRI/LDEO Climate Data Library: A wide range of over 300 datasets from various climate models and datasets.
- World Ozone and Ultraviolet Radiation Data Center: Extensive set of data collected from many stations across the world, recording ozone information (from ozonesonde ascents) and ground-level ultraviolet irradiance.
- Gridded climatic data for North America, South America and Europe: A huge range of climatic data at 1km and 4km resolution, derived from various models, including temperature, precipitation, snow and derived variables such as water deficit and continentality.
Also available through the Wayback Machine (added on April 5, 2019).
The resources below are not from Robin Wilson's site:
There are several modified versions of the Köppen-Geiger climate classification. One of the more popular modifications is the Trewartha climate classification
Holdridge Life Zones, an alternative climate classification based on precipitation, humidity, and potential evapotranspiration ratio.
I recently found this website for downloading weather data: https://rp5.ru/ which is the website of Reliable Prognosis, a company from St-Petersburg. The website is mainly dedicated to weather forecast, but you can find historical weather data (weather archive). There are interesting features for historical weather data:
- Available data from "8400 SYNOP weather stations and 5200 METAR weather stations" (according to https://rp5.ru/docs/about/en).
- Hourly, 3-hourly or daily data available (depending on the station).
- Many weather variables available : temperature, precipitation, pression, cloud cover, wind velocity & direction, ... (depending on the stations) + observations (textual information).
- A nice stuff - data is very frequently updated: data from METAR stations appears on the website 10 min. after recordings.
- Data licence? I did not find information about the data licence.
I've used data from this website for one station in France + had a quick look at other ones in Belgium. If someone else has more experience with this website, please comment!
I have recently worked on GWeather a command-line tool/library that helps users to work with climate data from all over the world. The main goal is to be able to work on a dataset where data from PSL will be presented in a friendly way combining location names from simplemaps.
The application supports the next features:
- Download climate datasets from PSL
- Download cities/countries dataset from simplemaps.com
- Merge/join above datasets by location's coordinates
- Export merged data in several formats
Comments remarks and pull requests are all welcome.