DOE OSTI · 2555905
Navigating the Noise: Bringing Clarity to ML Parameterization Design With O $\boldsymbol{\mathcal{O}}$(100) Ensembles
Abstract
Abstract Machine‐learning (ML) parameterizations of subgrid processes (here of turbulence, convection, and radiation) may one day replace conventional parameterizations by emulating high‐resolution physics without the cost of explicit simulation. However, uncertainty about the relationship between offline and online performance (i.e., when integrated with a large‐scale general circulation model) hinders their development. Much of this uncertainty stems from limited sampling of the noisy, emergent effects of upstream ML design decisions on downstream online hybrid simulation. Our work rectifies the sampling issue via the construction of a semi‐automated, end‐to‐end pipeline for size ensembles of hybrid simulations, revealing important nuances in how systematic reductions in offline error manifest in changes to online error and online stability. For example, removing dropout and switching from a Mean Squared Error to a Mean Absolute Error loss both reduce offline error, but they have opposite effects on online error and online stability. Other design decisions, like incorporating memory, converting moisture input from specific humidity to relative humidity, using batch normalization, and training on multiple climates do not come with any such compromises. Finally, we show that ensemble sizes of may be necessary to reliably detect causally relevant differences online. By enabling rapid online experimentation at scale, we can empirically settle debates regarding subgrid ML parameterization design that would have otherwise remained unresolved in the noise.
Keep this discovery
Explore connections, maps & timelines
Lin, Jerry [Department of Earth System Sciences University of California at Irvine Irvine CA USA] (ORCID:0000000317789426), Yu, Sungduk [Multimodal Cognitive AI Intel Labs Santa Clara CA USA] (ORCID:0000000245063887), Peng, Liran [Department of Earth System Sciences University of California at Irvine Irvine CA USA] (ORCID:0000000329935208), Beucler, Tom [Faculty of Geosciences and Environment University of Lausanne Lausanne Switzerland, Expertise Center for Climate Extremes University of Lausanne Lausanne Switzerland] (ORCID:0000000257311040), Wong‐Toi, Eliot [Department of Statistics University of California at Irvine Irvine CA USA], Hu, Zeyuan [Department of Earth and Planetary Sciences Harvard University Cambridge MA USA, NVIDIA Research Santa Clara CA USA] (ORCID:000000032041879X), Gentine, Pierre [LEAP Science and Technology Center School of Engineering and Applied Sciences Climate School Columbia University New York NY USA] (ORCID:0000000208458345), Geleta, Margarita [Department of Electrical Engineering and Computer Science Berkeley AI Research (BAIR) University of California at Berkeley Berkeley CA USA, Department of Biomedical Data Science Stanford University School of Medicine Palo Alto CA USA], Pritchard, Mike [Department of Earth System Sciences University of California at Irvine Irvine CA USA, NVIDIA Research Santa Clara CA USA] (ORCID:0000000203406327). 2025-04-12. Navigating the Noise: Bringing Clarity to ML Parameterization Design With O $\boldsymbol{\mathcal{O}}$(100) Ensembles. https://doi.org/10.1029/2024ms004551
Cite the original work for its findings. Save a collection to share your selection of sources.