Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning and data science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

High-Resolution Model Intercomparison Project phase 2 (HighResMIP2) towards CMIP7

Abstract. Robust projections and predictions of climate variability and change, particularly at regional scales, rely on the driving processes being represented with fidelity in model simulations. Consequently, the role of enhanced horizontal resolution in improved process representation in all components of the climate system continues to be of great interest. Recent simulations suggest the possibility of significant changes in both large-scale aspects of the ocean and atmospheric circulations and in the regional responses to climate change, as well as improvements in representations of small-scale processes and extremes, when resolution is enhanced. The first phase of the High-Resolution Model Intercomparison Project (HighResMIP1) was successful at producing a baseline multi-model assessment of global simulations with model grid spacings of 25–50 km in the atmosphere and 10–25 km in the ocean, a significant increase when compared to models with standard resolutions on the order of 1° that are typically used as part of the Coupled Model Intercomparison Project (CMIP) experiments. In addition to over 250 peer-reviewed manuscripts using the published HighResMIP1 datasets, the results were widely cited in the Intergovernmental Panel on Climate Change report and were the basis of a variety of derived datasets, including tracked cyclones (both tropical and extratropical), river discharge, storm surge, and impact studies. There were also suggestions from the few ocean eddy-rich coupled simulations that aspects of climate variability and change might be significantly influenced by improved process representation in such models. The compromises that HighResMIP1 made should now be revisited, given the recent major advances in modelling and computing resources. Aspects that will be reconsidered include experimental design and simulation length, complexity, and resolution. In addition, larger ensemble sizes and a wider range of future scenarios would enhance the applicability of HighResMIP. Therefore, we propose the High-Resolution Model Intercomparison Project phase 2 (HighResMIP2) to improve and extend the previous work, to address new science questions, and to further advance our understanding of the role of horizontal resolution (and hence process representation) in state-of-the-art climate simulations. With further increases in high-performance computing resources and modelling advances, along with the ability to take full advantage of these computational resources, an enhanced investigation of the drivers and consequences of variability and change in both large- and synoptic-scale weather and climate is now possible. With the arrival of global cloud-resolving models (currently run for relatively short timescales), there is also an opportunity to improve links between such models and more traditional CMIP models, with HighResMIP providing a bridge to link understanding between these domains. HighResMIP also aims to link to other CMIP projects and international efforts such as the World Climate Research Program lighthouse activities and various digital twin initiatives. It also has the potential to be used as training and validation data for the fast-evolving machine learning climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Evaluation of UAV-derived multimodal remote sensing data for biomass prediction and drought tolerance assessment in bioenergy sorghum

Screening for drought tolerance is critical to ensure high biomass production of bioenergy sorghum in arid or semi-arid environments. The bottleneck in drought tolerance selection is the challenge of accurately predicting biomass for a large number of genotypes. Although biomass prediction by low-altitude remote sensing has been widely investigated on various crops, the performance of the predictions are not consistent, especially when applied in a breeding context with hundreds of genotypes. In some cases, biomass prediction of a large group of genotypes benefited from multimodal remote sensing data; while in other cases, the benefits were not obvious. In this study, we evaluated the performance of single and multimodal data (thermal, RGB, and multispectral) derived from an unmanned aerial vehicle (UAV) for biomass prediction for drought tolerance assessments within a context of bioenergy sorghum breeding. The biomass of 360 sorghum genotypes grown under well-watered and water-stressed regimes was predicted with a series of UAV-derived canopy features, including canopy structure, spectral reflectance, and thermal radiation features. Biomass predictions using canopy features derived from the multimodal data showed comparable performance with the best results obtained with the single modal data with coefficients of determination (R 2 ) ranging from 0.40 to 0.53 under water-stressed environment and 0.11 to 0.35 under well-watered environment. The significance in biomass prediction was highest with multispectral followed by RGB and lowest with the thermal sensor. Finally, two well-recognized yield-based drought tolerance indices were calculated from ground truth biomass data and UAV predicted biomass, respectively. Results showed that the geometric mean productivity index outperformed the yield stability index in terms of the potential for reliable predictions by the remotely sensed data. Collectively, this study demonstrated a promising strategy for the use of different UAV-based imaging sensors to quantify yield-based drought tolerance.

09 BIOMASS FUELS↗
DOE OSTI2024

Sub-millisecond keyhole pore detection in laser powder bed fusion using sound and light sensors and machine learning

Laser powder bed fusion is a mainstream additive manufacturing technology widely used to manufacture complex parts in prominent sectors, including aerospace, biomedical, and automotive industries. However, during the printing process, the presence of an unstable vapor depression can lead to a type of defect called keyhole porosity, which is detrimental to the part quality. In this study, we developed an effective approach to locally detect the generation of keyhole pores during the printing process by leveraging machine learning and a suite of optical and acoustic sensors. Simultaneous synchrotron x-ray imaging allows the direct visualization of pore generation events inside the sample, offering high-fidelity ground truth. A neural network model adopting SqueezeNet architecture using single-sensor data was developed to evaluate the fidelity of each sensor for capturing keyhole pore generation events. Our comparative study shows that the near infrared images gave the highest prediction accuracy, followed by 100 kHz and 20 kHz microphones, and the photodiode sensitive to processing laser wavelength had the lowest accuracy. Using a single sensor, over 90% prediction accuracy can be achieved with a temporal resolution as short as 0.1 ms. A data fusion scheme was also developed with features extracted using SqueezeNet neural network architecture and classification using different machine learning algorithms. Our work demonstrates the correlation between the characteristic optical and acoustic emissions and the keyhole oscillation behavior, and thereby provides strong physics support for the machine learning approach.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Automated annotation of scientific texts for ML-based keyphrase extraction and validation

Advanced omics technologies and facilities generate a wealth of valuable data daily; however, the data often lack the essential metadata required for researchers to find, curate, and search them effectively. The lack of metadata poses a significant challenge in the utilization of these data sets. Machine learning (ML)–based metadata extraction techniques have emerged as a potentially viable approach to automatically annotating scientific data sets with the metadata necessary for enabling effective search. Text labeling, usually performed manually, plays a crucial role in validating machine-extracted metadata. However, manual labeling is time-consuming and not always feasible; thus, there is a need to develop automated text labeling techniques in order to accelerate the process of scientific innovation. This need is particularly urgent in fields such as environmental genomics and microbiome science, which have historically received less attention in terms of metadata curation and creation of gold-standard text mining data sets. In this paper, we present two novel automated text labeling approaches for the validation of ML-generated metadata for unlabeled texts, with specific applications in environmental genomics. Our techniques show the potential of two new ways to leverage existing information that is only available for select documents within a corpus to validate ML models, which can then be used to describe the remaining documents in the corpus. The first technique exploits relationships between different types of data sources related to the same research study, such as publications and proposals. The second technique takes advantage of domain-specific controlled vocabularies or ontologies. In this paper, we detail applying these approaches in the context of environmental genomics research for ML-generated metadata validation. Our results show that the proposed label assignment approaches can generate both generic and highly specific text labels for the unlabeled texts, with up to 44% of the labels matching with those suggested by a ML keyword extraction algorithm.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Tournament-Based Pretraining to Accelerate Federated Learning

Advances in hardware, proliferation of compute at the edge, and data creation at unprecedented scales have made federated learning (FL) necessary for the next leap forward in pervasive machine learning. For privacy and network reasons, large volumes of data remain stranded on endpoints located in geographically austere (or at least austere network-wise) locations. However, challenges exist to the effective use of these data. To solve the system and functional level challenges, we present an three novel variants of a serverless federated learning framework. We also present tournament-based pretraining, which we demonstrate significantly improves model performance in some experiments. Overall, these extensions to FL and our novel training method enable greater focus on science rather than ML development.

Baughman, Matt↗
DOE OSTI2023

Measuring photometric redshifts for high-redshift radio source surveys

With the advent of deep, all-sky radio surveys, the need for ancillary data to make the most of the new, high-quality radio data from surveys like the Evolutionary Map of the Universe (EMU), GaLactic and Extragalactic All-sky Murchison Widefield Array survey eXtended, Very Large Array Sky Survey, and LOFAR Two-metre Sky Survey is growing rapidly. Radio surveys produce significant numbers of Active Galactic Nuclei (AGNs) and have a significantly higher average redshift when compared with optical and infrared all-sky surveys. Thus, traditional methods of estimating redshift are challenged, with spectroscopic surveys not reaching the redshift depth of radio surveys, and AGNs making it difficult for template fitting methods to accurately model the source. Machine Learning (ML) methods have been used, but efforts have typically been directed towards optically selected samples, or samples at significantly lower redshift than expected from upcoming radio surveys. This work compiles and homogenises a radio-selected dataset from both the northern hemisphere (making use of Sloan Digital Sky Survey optical photometry) and southern hemisphere (making use of Dark Energy Survey optical photometry). We then test commonly used ML algorithms such as k-Nearest Neighbours (kNN), Random Forest, ANNz, and GPz on this monolithic radio-selected sample. We show that kNN has the lowest percentage of catastrophic outliers, providing the best match for the majority of science cases in the EMU survey. We note that the wider redshift range of the combined dataset used allows for estimation of sources up to z = 3 before random scatter begins to dominate. When binning the data into redshift bins and treating the problem as a classification problem, we are able to correctly identify ≈ 76% of the highest redshift sources—sources at redshift z > 2.51 —as being in either the highest bin (z > 2.51) or second highest (z = 2.25).

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Self-Supervised Cloud Classification

Abstract Low-level marine clouds play a pivotal role in Earth’s weather and climate through their interactions with radiation, heat and moisture transport, and the hydrological cycle. These interactions depend on a range of dynamical and microphysical processes that result in a broad diversity of cloud types and spatial structures, and a comprehensive understanding of cloud morphology is critical for continued improvement of our atmospheric modeling and prediction capabilities moving forward. Deep learning has recently accelerated our ability to study clouds using satellite remote sensing, and machine learning classifiers have enabled detailed studies of cloud morphology. A major limitation of deep learning approaches to this problem, however, is the large number of hand-labeled samples that are required for training. This work applies a recently developed self-supervised learning scheme to train a deep convolutional neural network (CNN) to map marine cloud imagery to vector embeddings that capture information about mesoscale cloud morphology and can be used for satellite image classification. The model is evaluated against existing cloud classification datasets and several use cases are demonstrated, including training cloud classifiers with very few labeled samples, interrogation of the CNN’s learned internal feature representations, cross-instrument application, and resilience against sensor calibration drift and changing scene brightness. The self-supervised approach learns meaningful internal representations of cloud structures and achieves comparable classification accuracy to supervised deep learning methods without the expense of creating large hand-annotated training datasets. Significance Statement Marine clouds heavily influence Earth’s weather and climate, and improved understanding of marine clouds is required to improve our atmospheric modeling capabilities and physical understanding of the atmosphere. Recently, deep learning has emerged as a powerful research tool that can be used to identify and study specific marine cloud types in the vast number of images collected by Earth-observing satellites. While powerful, these approaches require hand-labeling of training data, which is prohibitively time intensive. This study evaluates a recently developed self-supervised deep learning method that does not require human-labeled training data for processing images of clouds. We show that the trained algorithm performs competitively with algorithms trained on hand-labeled data for image classification tasks. We also discuss potential downstream uses and demonstrate some exciting features of the approach including application to multiple satellite instruments, resilience against changing image brightness, and its learned internal representations of cloud types. The self-supervised technique removes one of the major hurdles for applying deep learning to very large atmospheric datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Perspectives on the integration between first-principles and data-driven modeling

Efficiently embedding and/or integrating mechanistic information with data-driven models is essential if it is desired to simultaneously take advantage of both engineering principles and data-science. Further the opportunity for hybridization occurs in many scenarios, such as the development of a faster model of an accurate high-fidelity computer model; the correction of a mechanistic model that does not fully-capture the physical phenomena of the system; or the integration of a data-driven component approximating an unknown correlation within a mechanistic model. At the same time, different techniques have been proposed and applied in different literatures to achieve this hybridization, such as hybrid modeling, physics-informed Machine Learning (ML) and model calibration. In this paper we review the methods, challenges, applications and algorithms of these three research areas and discuss them in the context of the different hybridization scenarios. Moreover, we provide a comprehensive comparison of the hybridization techniques with respect to their differences and similarities, as well as advantages and limitations and future perspectives. Finally, we apply and illustrate hybrid modeling, physics-informed ML and model calibration via a chemical reactor case study.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Automated Gold Nanorod Spectral Morphology Analysis Pipeline

The development of a colloidal synthesis procedure to produce nanomaterials with high shape and size purity is often a time-consuming, iterative process. This is often due to quantitative uncertainties in the required reaction conditions and the time, resources, and expertise intensive characterization methods required for quantitative determination of nanomaterial size and shape. Absorption spectroscopy is often the easiest method for colloidal nanomaterial characterization. However, due to the lack of a reliable method to extract nanoparticle shapes from absorption spectroscopy, it is generally treated as a more qualitative measure for metal nanoparticles. This work demonstrates a gold nanorod (AuNR) spectral morphology analysis tool, called AuNR-SMA, which is a fast and accurate method to extract quantitative structural information from colloidal AuNR absorption spectra. To demonstrate the practical utility of this model, we apply it to three distinct applications. First, we demonstrate this model's utility as an automated analysis tool in a high-throughput AuNR synthesis procedure by generating quantitative size information from optical spectra. Second, we use the predictions generated by this model to train a machine learning model to predict the resulting AuNR size distributions under specified reaction conditions. Third, we apply this model to spectra extracted from the literature where no size distributions are reported and impute unreported quantitative information on AuNR synthesis. This approach can potentially be extended to any other nanocrystal system where absorption spectra are size dependent, and accurate numerical simulation of absorption spectra is possible. In addition, this pipeline could be integrated into automated synthesis apparatuses to provide interpretable data from simple measurements, help explore the synthesis science of nanoparticles in a rational manner, or facilitate closed-loop workflows.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

Near real-time air quality forecasts using the NASA GEOS model

NASA's Global Modeling and Assimilation Office (GMAO) produces high-resolution global forecasts for weather, aerosols, and air quality. The NASA Global Earth Observing System (GEOS) model has been expanded to provide global near-real-time5-day forecasts of atmospheric composition at unprecedented horizontal resolution of 0.25 degrees (~25 km). This composition forecast system (GEOS-CF) combines the operational GEOS weather forecasting model with the state-of-the-science GEOS-Chem chemistry module (version 12) to provide detailed analysis of a wide range of air pollutants such as ozone, carbon monoxide, nitrogen oxides, and fine particulate matter (PM2.5). Satellite observations are assimilated into the system for improved representation of weather and smoke. The assimilation system is being expanded to include chemically reactive trace gases. We discuss current capabilities of the GEOS Constituent Data Assimilation System (CoDAS) to improve atmospheric composition modeling and possible future directions, notably incorporating new observations (TROPOMI, geostationary satellites) and machine learning techniques. We show how machine learning techniques can be used to correct for sub-grid-scale variability, which further improves model estimates at a given observation site.

Air Quality Forecast↗
DOE OSTI2023

Molecular Simulations of the Chain Length Dependent Adsorption of C7‐C14 n ‐Alkanes in ZIF‐8

Recent experiments show that the diffusivities of C8 and C10 n-alkanes in ZIF-8 are higher than those of C7 and C9, respectively. We investigated this unusual ‘odd-even' effect by simulating the adsorption of C7-C14 n-alkanes in ZIF-8 using hybrid Monte Carlo molecular simulations. The resultant adsorption isotherms, guest-host energies, isosteric heats, and chain length distributions are analyzed for trends among the n-alkanes. ZIF-8 cages filled with n-alkanes are characterized using a combination of image processing and data science to quantify the differences in packing that occur with chain length. In conclusion, results indicate that packing changes drastically from C7 to C8 and from C9 to C10, which is consistent with the diffusion trends previously reported in experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Integrating Reanalysis and Satellite Cloud Information to Estimate Surface Downward Long-Wave Radiation

The estimation of downward long-wave radiation (DLR) at the surface is very important for the understanding of the Earth’s radiative budget with implications in surface–atmosphere exchanges, climate variability, and global warming. Theoretical radiative transfer and observationally based studies identify the crucial role of clouds in modulating the temporal and spatial variability of DLR. In this study, a new machine learning algorithm that uses multivariate adaptive regression splines (MARS) and the combination of near-surface meteorological data with satellite cloud information is proposed. The new algorithm is compared with the current operational formulation used by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) Satellite Application Facility on Land Surface Analysis (LSA-SAF). Both algorithms use near-surface temperature and dewpoint temperature along with total column water vapor from the latest European Centre for Medium-range Weather Forecasts (ECMWF) reanalysis ERA5 and satellite cloud information from the Meteosat Second Generation. The algorithms are trained and validated using both ECMWF-ERA5 and DLR acquired from 23 ground stations as part of the Baseline Surface Radiation Network (BSRN) and the Atmospheric Radiation Measurement (ARM) user facility. Results show that the MARS algorithm generally improves DLR estimation in comparison with other model estimates, particularly when trained with observations. When considering all the validation data, root mean square errors (RMSEs) of 18.76, 23.55, and 22.08 W·m –2 are obtained for MARS, operational LSA-SAF, and ERA5, respectively. The added value of using the satellite cloud information is accessed by comparing with estimates driven by ERA5 total cloud cover, showing an increase of 17% of the RMSE. The consistency of MARS estimate is also tested against an independent dataset of 52 ground stations (from FLUXNET2015), further supporting the good performance of the proposed model.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Investigation of the Effect of Framework Flexibility on Adsorption in SIFSIX-3-Cu using a Machine-Learned Force Field

Metal-organic frameworks (MOFs) are a promising class of adsorbents. The performance of MOF sorbents relies on high selectivity and low regeneration energy. This work focuses on the use of machine learned force fields (MLFFs) to model adsorption in a flexible MOF, SIFSIX-3-Cu. A DeePMD-based MLFF was trained to reproduce DFT (PBE+D3) energies, forces, and stresses, using an iterative sampling scheme combining sampling based on molecular dynamics, Monte Carlo, and geometry optimization to capture both attractive and repulsive regions of the potential energy surface. Flexibility of the MOF was explicitly included in this model. Hybrid Monte Carlo/molecular dynamics (MC/MD) simulations using the MLFF predicted adsorption isotherms in good agreement with experimental data for a range of pressures (40 Pa – 104 Pa) in contrast to rigid models, which overpredict CO2 adsorption at low pressures. The improvement was the result of a description of the variability of fluorine-fluorine diagonal distances at adsorption sites. This detailed description of flexibility afforded by the MLFF resulted in more accurate predictions adsorption isotherms when compared to the experimentally measured values. These results underscore the importance of including framework flexibility when modeling adsorption phenomena in MOFs, particularly for low pressure applications and provide a robust procedure for training MLFF models for MOFs.

Atomistic Simulation↗
DOE OSTI2020

Predicting lake surface water phosphorus dynamics using process-guided machine learning

Phosphorus (P) loading to lakes is degrading the quality and usability of water globally. Accurate predictions of lake P dynamics are needed to understand whole-ecosystem P budgets, as well as the consequences of changing lake P concentrations for water quality. However, complex biophysical processes within lakes, along with limited observational data, challenge our capacity to reproduce short-term lake dynamics needed for water quality predictions, as well as long-term dynamics needed to understand broad scale controls over lake P. Here we use an emerging paradigm in modeling, process-guided machine learning (PGML), to produce a phosphorus budget for Lake Mendota (Wisconsin, USA) and to accurately predict epilimnetic phosphorus over a time range of days to decades. In our implementation of PGML, which we term a Process-Guided Recurrent Neural Network (PGRNN), we combine a process-based model for lake P with a recurrent neural network, and then constrain the predictions with ecological principles. We test independently the process-based model, the recurrent neural network, and the PGRNN to evaluate the overall approach. The process-based model accounted for most of the observed pattern in lake P; however it missed the long-term trend in lake P and had the worst performance in predicting winter and summer P in surface waters. The root mean square error (RMSE) for the process-based model, the recurrent neural network, and the PGRNN was 33.0 μg P L -1 , 22.7 μg P L -1 , and 20.7 μg P L -1 , respectively. All models performed better during summer, with RMSE values for the three models (same order) equal to 14.3 μg P L -1 , 10.9 μg P L -1 , and 10.7 μg P L -1 . Although the PGRNN had only marginally better RMSE during summer, it had lower bias and reproduced long-term decreases in lake P missed by the other two models. For all seasons and all years, the recurrent neural network had better predictions than process alone, with root mean square error (RMSE) of 23.8 μg P L -1 and 28.0 μg P L -1 , respectively. The output of PGRNN indicated that new processes related to water temperature, thermal stratification, and long term changes in external loads are needed to improve the process model. By using ecological knowledge, as well as the information content of complex data, PGML shows promise as a technique for accurate prediction in messy, real-world ecological dynamics, while providing valuable information that can improve our understanding of process.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Software-Defined Network for End-to-end Networked Science at the Exascale

Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The work we presen here is motivated by a vision for a new smart network and smart application ecosystem that will provide a more deterministic and interactive environment for domain science workflows. The Software-Defined Network for End-to-end Networked Science at Exascale (SENSE) system includes a model-based architecture, implementation, and deployment which enables automated end-to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements, an intelligent orchestrator and resource control systems allow for custom tailoring of scalability and real-time responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource as is the current practice for instruments, compute, and storage systems. Deployment and experiments on production networks and testbeds have validated SENSE functions and performance. Emulation based testing verified the scalability needed to support research and education infrastructures. Key contributions of this work include an architecture definition, reference implementation, and deployment. This provides the basis for further innovation of smart network services to accelerate scientific discovery in the era of big data, cloud computing, machine learning and artificial intelligence.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Prediction of the SYM-H Index Using a Bayesian Deep Learning Method With Uncertainty Quantification

We propose a novel deep learning framework, named SYMHnet, which employs a graph neural network and a bidirectional long short-term memory network to cooperatively learn patterns from solar wind and interplanetary magnetic field parameters for short-term forecasts of the SYM-H index based on 1- and 5-min resolution data. SYMHnet takes, as input, the time series of the parameters' values provided by NASA's Space Science Data Coordinated Archive and predicts, as output, the SYM-H index value at time point t + w hours for a given time point t where w is 1 or 2. By incorporating Bayesian inference into the learning framework, SYMHnet can quantify both aleatoric (data) uncertainty and epistemic (model) uncertainty when predicting future SYM-H indices. Experimental results show that SYMHnet works well at quiet time and storm time, for both 1- and 5-min resolution data. The results also show that SYMHnet generally performs better than related machine learning methods. For example, SYMHnet achieves a forecast skill score (FSS) of 0.343 compared to the FSS of 0.074 of a recent gradient boosting machine (GBM) method when predicting SYM-H indices (1 hr in advance) in a large storm (SYM-H = -393 nT) using 5-min resolution data. When predicting the SYM-H indices (2 hr in advance) in the large storm, SYMHnet achieves an FSS of 0.553 compared to the FSS of 0.087 of the GBM method. In addition, SYMHnet can provide results for both data and model uncertainty quantification, whereas the related methods cannot.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Crowdsourcing the Frontier: Advancing Hybrid Physics‐ML Climate Simulation via a $\$$50,000 Kaggle Competition

Subgrid machine-learning (machine learning [ML]) parameterizations have the potential to introduce a new generation of climate models that incorporate the effects of higher-resolution physics without incurring the prohibitive computational cost associated with more explicit physics-based simulations. However, important issues, ranging from online instability to inconsistent online performance, have limited their operational use for long-term climate projections. To more rapidly drive progress in solving these issues, domain scientists and ML researchers opened up the offline aspect of this problem to the broader ML and data science community with the release of ClimSim, a NeurIPS Data sets and Benchmarks publication, and an associated Kaggle competition. This paper reports on the downstream results of the Kaggle competition by coupling emulators inspired by the winning teams' architectures to an interactive climate model (including full cloud microphysics, a regime historically prone to online instability) and systematically evaluating their online performance. Our results demonstrate that online stability in the low-resolution real-geography setting is reproducible across multiple diverse architectures, which we consider a key milestone. All tested architectures exhibit strikingly similar offline and online biases, though their responses to architecture-agnostic design choices (e.g., expanding the list of input variables) can differ significantly. Multiple Kaggle-inspired architectures achieve state-of-the-art results on certain metrics such as zonal mean bias patterns and global Root Mean Squared Error, indicating that crowdsourcing the essence of the offline problem is one path to improving online performance in hybrid physics-AI climate simulation.

Environmental sciences↗
DOE OSTI2020

Software-defined network for end-to-end networked science at the exascale

Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The work presented here is motivated by a vision for a new smart network and smart application ecosystem that will provide a more deterministic and interactive environment for domain science workflows. The Software-Defined Network for End-to-end Networked Science at Exascale (SENSE) system includes a model-based architecture, implementation, and deployment which enables automated end- to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements, an intelligent orchestrator and resource control systems allow for custom tailoring of scalability and real-time responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource as is the current practice for instruments, compute, and storage systems. Deployment and experiments on production networks and testbeds have validated SENSE functions and performance. Emulation based testing verified the scalability needed to support research and education infrastructures. Key contributions of this work include an architecture definition, reference implementation, and deployment. This provides the basis for further innovation of smart network services to accelerate scientific discovery in the era of big data, cloud computing, machine learning and artificial intelligence.

47 OTHER INSTRUMENTATION↗