26

Barry, could you abuse this site's "answer your own question" feature to create a community wiki answer for sources of weather data (both current and historical), since it gets asked so often?

Improve this question
5
  • 4
    I see answers creeping in with data from models. Can we please distinguish real data and model data? (I suggest limiting this question to observational data only).
    – user4293
    Nov 28, 2016 at 10:04
  • 3
    We also need (with each link) good descriptions of the data in the set and its historic period. Can I suggest additions in the format [precipitation, wind speeds, temperatures][1960-current]? Maybe add time resolution as well?
    – user4293
    Nov 28, 2016 at 10:06
  • 2
    Note that there is significant overlap with questions on Earth Science. I have not investigated if they have a canonical/community question over there, it might be wise to team up. Start a proposal in meta on both sites?
    – user4293
    Nov 28, 2016 at 10:08
  • .. and geographical area (can no longer edit my second comment).
    – user4293
    Nov 28, 2016 at 10:12
  • @JanDoggen the distinction between 'real data' and 'model data' is not always clear (b/c, e.g. processing of radar requires models, as do historical reprocessing and reanalysis); 'station data' would be closest to 'real' although this is generally re-processed to remove artifacts and fill gaps ... using models) perhaps a better distinction would be 'best guess at historical reality' vs. model predictions (or historical vs future)?
    – David LeBauer
    Nov 28, 2017 at 17:17

6 Answers 6

Reset to default
20

This list is (and may always be) incomplete. Please add to it when you can.

enter image description here

You can read more about these "cycle" files at: http://www.nws.noaa.gov/tg/datahelp.php Note these files are updated every few minutes and have not been curated, so data can be inaccurate. Reports in these cycle files are sometimes updated (and thus invalidated) in later cycle files.

To translate station codes to physical locations:

Well known sites:

  • weather.gov
  • wunderground.com
  • weather.com
Improve this answer
6
  • Thank you for the immensely helpful link. I am planning on using this link for historical hourly temperatures: ftp.ncdc.noaa.gov/pub/data/noaa It contains yearly folders, e.g. 2017 within which each station has its own file, e.g. 010010-99999-2017 The only question I have: how to get the location of the station, 010010 in this case? Lat and long is fine, even zip code would be OK. I tried to find such a list but was not successful.
    – Kevin P.
    Dec 19, 2018 at 20:23
  • 1
    @KevinP. I've added a paragraph near the end re sources to identify station "metadata". For your use case, isd-history.txt should work
    – user3856
    Dec 19, 2018 at 23:03
  • Thank you again, Barry. I was able to easily use wget to get this path: ftp.ncdc.noaa.gov/pub/data/noaa/isd-history.csv
    – Kevin P.
    Dec 23, 2018 at 1:12
  • A further question: Does anyone know if the hour field is for the local time zone of the weather station (I would assume so) and if it has daylight savings already applied to it? I read through all the documentation but it doesn't address it one way or the other.
    – Kevin P.
    Dec 29, 2018 at 1:26
  • 1
    @KevinP.I'm pretty sure it 's all UTC. If you google isd "utc" site:.gov (as quoted) and then look at the cached version of the pages (the main ones are down due to US govt shutdown), you might find a canonical confirmation (I found a lot that suggests all times are UTC, but nothing canonical for isd-lite specifically)
    – user3856
    Dec 29, 2018 at 19:53
9

Get the data from BigQuery:

Sample query:

#standardSQL
SELECT name, stn, a.wban, TIMESTAMP(CONCAT(year, '-', mo, '-', da)) date, temp, prcp
FROM `bigquery-public-data.noaa_gsod.gsod2*` a
JOIN (
  SELECT *
  FROM `bigquery-public-data.noaa_gsod.stations` 
  WHERE country='US'
  AND name LIKE '%HONOLULU%'
  AND state LIKE 'HI'
  LIMIT 1000
) b 
ON a.stn=b.usaf AND a.wban=b.wban
WHERE prcp!=99.99
ORDER BY name, date

enter image description here

Related: Finding the rainiest US cities

Improve this answer
9
Improve this answer
5

There's a good list of weather data sources on this excellent website created by Robin Wilson, which I duplicate here in case that site ever goes away:

There are hundreds more links to geospatial data on other topics on the main site, as well as a link to donate and keep that site alive.

Also available through the Wayback Machine (added on April 5, 2019).

The resources below are not from Robin Wilson's site:

  • The Köppen-Geiger climate classification, which divides the Earth's land area into 30 climatological regions based on location, temperature, and precipitation. A high-resolution map is available here

  • There are several modified versions of the Köppen-Geiger climate classification. One of the more popular modifications is the Trewartha climate classification

  • Holdridge Life Zones, an alternative climate classification based on precipitation, humidity, and potential evapotranspiration ratio.

Improve this answer
3
  • Since this answer seems very climate-focused, could you add the Koeppen-Geiger classification scheme: koeppen-geiger.vu-wien.ac.at/present.htm
    – user3856
    Jul 11, 2019 at 17:25
  • @BarryCarter Thanks for the suggestion, I added it to the list. Can you add a description?
    – csk
    Jul 11, 2019 at 17:30
  • Done (+ added a couple more sources + made it clear these sources are not from Wilson's site). Edit will be invisible until peer-reviewed, though.
    – user3856
    Jul 12, 2019 at 4:18
4

I recently found this website for downloading weather data: https://rp5.ru/ which is the website of Reliable Prognosis, a company from St-Petersburg. The website is mainly dedicated to weather forecast, but you can find historical weather data (weather archive). There are interesting features for historical weather data:

  • Available data from "8400 SYNOP weather stations and 5200 METAR weather stations" (according to https://rp5.ru/docs/about/en).
  • Hourly, 3-hourly or daily data available (depending on the station).
  • Many weather variables available : temperature, precipitation, pression, cloud cover, wind velocity & direction, ... (depending on the stations) + observations (textual information).
  • A nice stuff - data is very frequently updated: data from METAR stations appears on the website 10 min. after recordings.
  • Data licence? I did not find information about the data licence.

I've used data from this website for one station in France + had a quick look at other ones in Belgium. If someone else has more experience with this website, please comment!

J.

Improve this answer
0

I have recently worked on GWeather a command-line tool/library that helps users to work with climate data from all over the world. The main goal is to be able to work on a dataset where data from PSL will be presented in a friendly way combining location names from simplemaps.

The application supports the next features:

  • Download climate datasets from PSL
  • Download cities/countries dataset from simplemaps.com
  • Merge/join above datasets by location's coordinates
  • Export merged data in several formats

Comments remarks and pull requests are all welcome.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.