This is a list of weather and climate datasets preprocessed for AI research. This can include benchmarks, competitions or ML papers with published data. The list is in alphabetical order.
AI for Earth System Science Summer School Hackathon¶
Code and Data: https://github.com/NCAR/ai4ess-hackathon-2020
Source: NCAR, Lawrence Berkeley Lab, and NOAA
Description: 5 challenge problems related to prediction and emulation. GOES challenge problem focuses on predicting lightning from GOES-16 satellite imagery. GECKO-A challenge problem focuses on emulating the GECKO-A chemistry model from a large set of model time series. Microphysics challenge problem focuses on emulating the TAU bin microphysics scheme. HOLODEC challenge problem focuses on estimating rain drop distribution properties from synthetic holographic diffraction patterns. ENSO challenge problem focuses on predicting ENSO from gridded model output.
AMS Solar Energy Prediction Contest¶
Source data: GEFS forecasts and Mesonet solar observations
Description: Predict total daily solar irradiance from GEFS and Oklahoma Mesonet Data
AQ-Bench: A Benchmark Dataset for Machine Learning on Global Air Quality Metrics¶
Paper: Betancourt, C., Stomberg, T., Stadtler, S., Roscher, R., and Schultz, M. G.: AQ-Bench: A Benchmark Dataset for Machine Learning on Global Air Quality Metrics, Earth Syst. Sci. Data Discuss. (preprint), https://doi.org/10.5194/essd-2020-380, in review, 2021.
Source data: Database of the Tropospheric Ozone Assessment Report (TOAR)
Description: Aggregated air quality data from the years 2010–2014 and metadata at more than 5500 air quality monitoring stations all over the world. A well-defined task, a suitable evaluation metric and baseline scores are provided.
CAMELS: CATCHMENT ATTRIBUTES AND METEOROLOGY FOR LARGE-SAMPLE STUDIES¶
Source data: Weather models (Daymet, NLDAS, Maurer), streamflow observations (USGS), catchment attributes (USGS, MODIS, Daymet, STATSGO, Global Lithological Map (GLiM), GLobal Hydrogeology Maps (GLHYMPS))
Description: Weather drivers, streamflow observations, and catchment attributes for 671 catchments across the continental US.
ClimateNet: an expert-labelled open dataset and Deep Learning architecture for enabling high-precision analyses of extreme weather¶
Code and data: https://portal.nersc.gov/project/ClimateNet/
Source data: Climate Model simulations and expert labels
Description: Detect atmospheric rivers and tropical cyclones from climate model simulations. Tool for labeling along with dataset of expert labelled data.
CloudCast: A large-scale dataset and baseline for forecasting clouds¶
Code and data: https://vision.eng.au.dk/cloudcast-dataset/
Source data: Meteosat-11 with cloud types annotated on a pixel-level
Description: The CloudCast dataset contains 70080 images with 10 different cloud types for multiple layers of the atmosphere annotated on a pixel level. The dataset has a spatial size of 928 x 1530 pixels recorded with 15-min intervals for the period 2017-2018, with a 3.0 km resolution.
CUMULO: a benchmark dataset for training and evaluating global cloud classification models¶
Code and data: https://github.com/FrontierDevelopmentLab/CUMULO
Description: the dataset provides the global 1km-resolution imagery of the MODIS aligned with the accurately measured cloud properties of the CloudSat products. It contains three years of 1354 x 2030 pixel hyperspectral images combined with pixel-width ‘tracks’ of cloud labels, corresponding to the eight World Meteorological Organization genera.
Deepti: Deep-Learning-Based Tropical Cyclone Intensity Estimation System (+ Competition)¶
Code and data: http://registry.mlhub.earth/10.34911/rdnt.xs53up/
Source data: GOES
Description: A collection of tropical storms in the Atlantic and East Pacific Oceans from 2000 to 2019 with corresponding maximum sustained surface wind speed. This dataset is split into training and test categories for the purpose of a competition. The train set consists of 70,257 images and the test set consists of 44,377 image, each one being 366 x 366 pixels
EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts¶
Code and data: https://www.earthnet.tech/
Source data: Sentinel 2
Description: Curated dataset containing target spatio-temporal Sentinel 2 satellite imagery at 20 m resolution, matched with high-resolution topography and mesoscale (1.28 km) weather variables. With over 32000 samples it is suitable for training deep neural networks.
The ExtremeWeather Dataset¶
Source data: CAM5
Description: Consists of 768 × 1152 images of the global atmospheric state with a spatial resolution of 25 km and separated by 6 hour intervals from 1979 to 2005. There are 16 channels of images that correspond to different variables such as surface pressure, surface temperature and humidity of the reference altitude. In addition, there are boundary boxes and class labels for 4 types of extreme weather events: Tropical Depressions, Tropical Cyclones, Extratropical Cyclones and Atmospheric Rivers.
FlowDB: A new large scale river flow, flash flood, and precipitation dataset¶
Code and data: https://flow-forecast.atlassian.net/wiki/spaces/FF/pages/33456135/FlowDB+Dataset (Not public)
Source data: USGS, SNOTEL, NOAA, ASOS,EcoNet
Description: An hourly river flow and precipitation dataset and a second subset of flash flood events with damage estimates and injury counts. Created for general stream flow forecasting and flash flood damage estimation.
How Much Did It Rain I and II¶
MeteoNet, an open reference weather dataset by METEO FRANCE¶
Code and data: https://github.com/meteofrance/meteonet
Source data: AROME/ARPEGE forecasts, radar reflectivity and ground stations over France
Description: Multi source dataset of forecasts and observations over France spanning 3 years
Neural Networks for Postprocessing Ensemble Weather Forecasts¶
Paper: Rasp and Lerch 2018
Code and data: https://github.com/slerch/ppnn
Source data: TIGGE forecasts and station observations over Germany
Description: Ensemble temperature postprocessing of station observations over Germany. 9 years of data at 500 stations. Predictors include temperature as well as a range of other variables.
RainBench: Towards Global Precipitation Forecasting from Satellite Imagery¶
Code and data: https://github.com/frontierdevelopmentlab/pyrain
Source data: IMERG, ERA5 and SimSat
Description: Multi-modal benchmark dataset for data-driven precipitation forecasting at 3 different spatial resolutions: 0.1deg (IMERG and SimSat) and 0.5deg (ERA5). Presented along an efficient dataloading pipeline: Pyrain
SVRIMG - SeVere Reflectivity IMaGe Dataset¶
Code and data: https://svrimg.org/
Source data: GridRad (which in turn is sourced from NOAA NEXRAD Level II archives)
Description: over 500,000 data rich, geospatial, radar reflectivity images centered on high-impact weather events. These images have consistent dimensions and intensity values on a grid with relatively low spatial distortion over the Conterminous United States. Also includes crowd-sourced labeling.
TAASRAD19, a high-resolution weather radar reflectivity dataset for precipitation nowcasting¶
Code and data:https://github.com/MPBA/TAASRAD19
Source data: Official public meteorological agency of the civil protection department of the Autonomous Province of Trento (Italy)
Description: Benchmark dataset for radar nowcasting with deep learning. The dataset contains 1,732 radar sequences labeled with precipitation type spanning from 2010 to 2019, for a total of 362,233 radar images. Image size is 480 x 480 at 500m resolution (UTM grid) covering a complex orographic area in the Italian Alps.
Understanding Clouds from Satellite Images¶
Code and data: https://www.kaggle.com/c/understanding_cloud_organization
Source data: TERRA and AQUA MODIS visible images
Description: Cloud classification challenge of 4 human-designed shallow cloud patterns of organization: Sugar, Flower, Fish and Gravel with 30,000 human labels
VALUE: A framework to validate downscaling approaches for climate change studies¶
Code and data: http://www.value-cost.eu/
Source data: Station observations
Description: Framework for evaluating climate model downscaling methods. Validation observations are provided.
Papers using this dataset: Many
WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting¶
Code and data: https://github.com/pangeo-data/WeatherBench
Source data: ERA5 and TIGGE for baselines
Description: Benchmark dataset for medium-range (3 and 5 day) forecasting of global pressure, temperature and precipitation with preprocessed data (40 years), evaluation and baselines
Code and data:
Papers using this dataset: