19

Barry, could you abuse this site's "answer your own question" feature to create a community wiki answer for sources of weather data (both current and historical), since it gets asked so often?

  • 4
    I see answers creeping in with data from models. Can we please distinguish real data and model data? (I suggest limiting this question to observational data only). – Jan Doggen Nov 28 '16 at 10:04
  • 3
    We also need (with each link) good descriptions of the data in the set and its historic period. Can I suggest additions in the format [precipitation, wind speeds, temperatures][1960-current]? Maybe add time resolution as well? – Jan Doggen Nov 28 '16 at 10:06
  • 2
    Note that there is significant overlap with questions on Earth Science. I have not investigated if they have a canonical/community question over there, it might be wise to team up. Start a proposal in meta on both sites? – Jan Doggen Nov 28 '16 at 10:08
  • .. and geographical area (can no longer edit my second comment). – Jan Doggen Nov 28 '16 at 10:12
  • @JanDoggen the distinction between 'real data' and 'model data' is not always clear (b/c, e.g. processing of radar requires models, as do historical reprocessing and reanalysis); 'station data' would be closest to 'real' although this is generally re-processed to remove artifacts and fill gaps ... using models) perhaps a better distinction would be 'best guess at historical reality' vs. model predictions (or historical vs future)? – David LeBauer Nov 28 '17 at 17:17
16

This list is (and may always be) incomplete. Please add to it when you can.

enter image description here

You can read more about these "cycle" files at: http://www.nws.noaa.gov/tg/datahelp.php Note these files are updated every few minutes and have not been curated, so data can be inaccurate. Reports in these cycle files are sometimes updated (and thus invalidated) in later cycle files.

To translate station codes to physical locations:

Well known sites:

  • weather.gov
  • wunderground.com
  • weather.com
  • Thank you for the immensely helpful link. I am planning on using this link for historical hourly temperatures: ftp.ncdc.noaa.gov/pub/data/noaa It contains yearly folders, e.g. 2017 within which each station has its own file, e.g. 010010-99999-2017 The only question I have: how to get the location of the station, 010010 in this case? Lat and long is fine, even zip code would be OK. I tried to find such a list but was not successful. – Kevin P. Dec 19 '18 at 20:23
  • 1
    @KevinP. I've added a paragraph near the end re sources to identify station "metadata". For your use case, isd-history.txt should work – Barry Carter Dec 19 '18 at 23:03
  • Thank you again, Barry. I was able to easily use wget to get this path: ftp.ncdc.noaa.gov/pub/data/noaa/isd-history.csv – Kevin P. Dec 23 '18 at 1:12
  • A further question: Does anyone know if the hour field is for the local time zone of the weather station (I would assume so) and if it has daylight savings already applied to it? I read through all the documentation but it doesn't address it one way or the other. – Kevin P. Dec 29 '18 at 1:26
  • 1
    @KevinP.I'm pretty sure it 's all UTC. If you google isd "utc" site:.gov (as quoted) and then look at the cached version of the pages (the main ones are down due to US govt shutdown), you might find a canonical confirmation (I found a lot that suggests all times are UTC, but nothing canonical for isd-lite specifically) – Barry Carter Dec 29 '18 at 19:53
9

Get the data from BigQuery:

Sample query:

#standardSQL
SELECT name, stn, a.wban, TIMESTAMP(CONCAT(year, '-', mo, '-', da)) date, temp, prcp
FROM `bigquery-public-data.noaa_gsod.gsod2*` a
JOIN (
  SELECT *
  FROM `bigquery-public-data.noaa_gsod.stations` 
  WHERE country='US'
  AND name LIKE '%HONOLULU%'
  AND state LIKE 'HI'
  LIMIT 1000
) b 
ON a.stn=b.usaf AND a.wban=b.wban
WHERE prcp!=99.99
ORDER BY name, date

enter image description here

Related: Finding the rainiest US cities

8
5

There's a good list of weather data sources on this excellent website created by Robin Wilson, which I duplicate here in case that site ever goes away:

There are hundreds more links to geospatial data on other topics on the main site, as well as a link to donate and keep that site alive.

Also available through the Wayback Machine (added on April 5, 2019).

The resources below are not from Robin Wilson's site:

  • The Köppen-Geiger climate classification, which divides the Earth's land area into 30 climatological regions based on location, temperature, and precipitation. A high-resolution map is available here

  • There are several modified versions of the Köppen-Geiger climate classification. One of the more popular modifications is the Trewartha climate classification

  • Holdridge Life Zones, an alternative climate classification based on precipitation, humidity, and potential evapotranspiration ratio.

  • Since this answer seems very climate-focused, could you add the Koeppen-Geiger classification scheme: koeppen-geiger.vu-wien.ac.at/present.htm – Barry Carter Jul 11 at 17:25
  • @BarryCarter Thanks for the suggestion, I added it to the list. Can you add a description? – csk Jul 11 at 17:30
  • Done (+ added a couple more sources + made it clear these sources are not from Wilson's site). Edit will be invisible until peer-reviewed, though. – Barry Carter Jul 12 at 4:18
4

I recently found this website for downloading weather data: https://rp5.ru/ which is the website of Reliable Prognosis, a company from St-Petersburg. The website is mainly dedicated to weather forecast, but you can find historical weather data (weather archive). There are interesting features for historical weather data:

  • Available data from "8400 SYNOP weather stations and 5200 METAR weather stations" (according to https://rp5.ru/docs/about/en).
  • Hourly, 3-hourly or daily data available (depending on the station).
  • Many weather variables available : temperature, precipitation, pression, cloud cover, wind velocity & direction, ... (depending on the stations) + observations (textual information).
  • A nice stuff - data is very frequently updated: data from METAR stations appears on the website 10 min. after recordings.
  • Data licence? I did not find information about the data licence.

I've used data from this website for one station in France + had a quick look at other ones in Belgium. If someone else has more experience with this website, please comment!

J.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.