Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Traditional Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Stable Machine‐Learning Parameterization of Subgrid Processes in a Comprehensive Atmospheric Model Learned From Embedded Convection‐Permitting Simulations

Modern climate projections often suffer from inadequate spatial and temporal resolution due to computational limitations, resulting in inaccurate representations of sub-grid processes. A promising technique to address this is the multiscale modeling framework (MMF), which embeds a kilometer-resolution cloud-resolving model (CRM) within each atmospheric column of a host climate model to replace traditional convection and cloud parameterizations. Machine learning offers a unique opportunity to make MMF more accessible by emulating the embedded CRM and reducing its substantial computational cost. Although many studies have demonstrated proof-of-concept success of achieving stable hybrid simulations, it remains a challenge to achieve near operational-level success with real geography and comprehensive variable emulation that includes, for example, explicit cloud condensate coupling. In this study, we present a stable hybrid model capable of integrating for at least 5 years with near operational-level complexity, including coarse-grid geography, seasonality, explicit cloud condensate and wind predictions, and land coupling. Our model demonstrates skillful online performance, achieving a 5-year zonal mean tropospheric temperature bias within 2 K, water vapor bias within 1 g/kg, and a precipitation root mean square error of 0.96 mm/day. Key factors contributing to our online performance include an expressive U-Net architecture and physical thermodynamic constraints for microphysics. With microphysical constraints mitigating unrealistic cloud formation, our work is the first to demonstrate realistic multi-year cloud condensate climatology under the MMF framework. Despite these advances, online diagnostics reveal persistent biases in certain regions, highlighting the need for innovative strategies to further optimize online performance.

Hu, Zeyuan [NVIDIA Corporation, Santa Clara, CA (U↗

A systematic review of machine learning in groundwater monitoring

With increasing concerns about water scarcity, groundwater has become crucial since this resource provides most of the freshwater needs. However, various human and natural activities often contaminate the groundwater, making it unsuitable for use. Over the years, scientists and engineers have used many methods to predict and track groundwater contamination as part of environmental monitoring. Consequently, there is an urgent need for improved methods, particularly in the face of increasing contamination. Machine learning has sometimes been used to monitor groundwater, air quality, and climate. Traditional methods must be improved due to the complexity and large amount of environmental data. This includes using hybrid models that combine traditional and new techniques. Despite the use of machine learning in many scientific areas, there is a lack of comprehensive reviews focusing on its use in environmental monitoring, especially groundwater monitoring. We aim to fill this gap by exploring machine-learning applications in groundwater monitoring. We discuss relevant methods, their limitations, and future potential. We summarize research on automating data processing and model training using groundwater sensor data. Our research underscores the transformative potential of machine learning to revolutionize long-term groundwater monitoring and contamination detection, providing valuable insights for future research and practical applications.

AI/ML↗

Characterizing Machine Learning I/O Workloads on Leadership Scale HPC Systems

High performance computing (HPC) is no longer solely limited to traditional workloads such as simulation and modeling. With the increase in the popularity of machine learning (ML) and deep learning (DL) technologies, we are observing that an increasing number of HPC users are incorporating ML methods into their workflow and scientific discovery processes, across a wide spectrum of science domains such as biology, earth science, and physics. This gives rise to a diverse set of I/O patterns than the traditional checkpoint/restart-based HPC I/O behavior. The details of the I/O characteristics of such ML I/O workloads have not been studied extensively for large-scale leadership HPC systems. This paper aims to fill that gap by providing an in-depth analysis to gain an understanding of the I/O behavior of ML I/O workloads using darshan - an I/O characterization tool designed for lightweight tracing and profiling. We study the darshan logs of more than 23, 000 HPC ML I/O jobs over a time period of one year running on Summit - the second-fastest supercomputer in the world. This paper provides a systematic I/O characterization of ML I/O jobs running on a leadership scale supercomputer to understand how the I/O behavior differs across science domains and the scale of workloads, and analyze the usage of parallel file system and burst buffer by ML I/O workloads.

Paul, Arnab↗

Machine Learning-Enabled Image Classification for Automated Electron Microscopy

Abstract Traditionally, materials discovery has been driven more by evidence and intuition than by systematic design. However, the advent of “big data” and an exponential increase in computational power have reshaped the landscape. Today, we use simulations, artificial intelligence (AI), and machine learning (ML) to predict materials characteristics, which dramatically accelerates the discovery of novel materials. For instance, combinatorial megalibraries, where millions of distinct nanoparticles are created on a single chip, have spurred the need for automated characterization tools. This paper presents an ML model specifically developed to perform real-time binary classification of grayscale high-angle annular dark-field images of nanoparticles sourced from these megalibraries. Given the high costs associated with downstream processing errors, a primary requirement for our model was to minimize false positives while maintaining efficacy on unseen images. We elaborate on the computational challenges and our solutions, including managing memory constraints, optimizing training time, and utilizing Neural Architecture Search tools. The final model outperformed our expectations, achieving over 95% precision and a weighted F-score of more than 90% on our test data set. This paper discusses the development, challenges, and successful outcomes of this significant advancement in the application of AI and ML to materials discovery.

Materials Science↗

Computationally Accelerated Discovery and Experimental Demonstration of High-Performance Materials for Advanced Solar Thermochemical Hydrogen Production

This project achieved its overarching goal of accelerating the discovery and validation of solar thermochemical hydrogen (STCH) materials through a tightly integrated approach that combined high-throughput computational screening, advanced machine learning (ML), and experimental testing. Guided by the objectives outlined in the Statement of Project Objectives (SOPO), our work fulfilled all major milestones across four technical tasks and delivered scientific breakthroughs and practical tools that significantly exceeded the original scope of the project. We began by addressing the challenge of predicting material phase stability through machine learning. A novel Python module was developed to generate thousands of meaningful features from composition, structure, and electronic properties, enabling rapid and reproducible ML model development. Using these tools, we trained a model to predict temperature-dependent Gibbs energies (G(T)) for inorganic crystalline materials with near-chemical accuracy—roughly 40 meV/atom—marking the first such descriptor of its kind. We also introduced a new machine-learned tolerance factor, τ, that accurately predicted perovskite formability with over 90% success, outperforming traditional heuristic models, such as the Goldschmidt tolerance factor. These capabilities allowed for rapid and accurate predictions of phase stability across a vast oxide composition space, setting the stage for high-throughput thermodynamic screening. Building on this foundation, we conducted an extensive computational screening of candidate STCH oxide materials. Over 1.1 million perovskite compositions were evaluated using the τ descriptor, leading to the identification of more than 27,000 predicted stable structures. Using density functional theory (DFT), we refined over 68,000 multinary perovskite structures and computed oxygen vacancy formation energies for over 1,300 ternary and double perovskites. These calculations enabled us to isolate compounds with redox behavior consistent with STCH requirements and resulted in a public dataset now hosted on the Materials Project. Recognizing that thermodynamic screening alone is insufficient, we addressed kinetic limitations by developing a suite of tools to estimate transition state (TS) energies for key redox reactions. We implemented a novel bounding approach that provides lower and upper estimates of TS energies with dramatically reduced computational cost, requiring less than 10% of the CPU time of a full nudged elastic band (NEB) calculation while maintaining high accuracy. This enabled rapid evaluation of over 200 reaction pathways across 90 materials. To further accelerate screening, we developed a SISSO-based ML model to predict diffusion barriers with a 96.7% success rate in classifying fast vs. slow materials, supporting a robust, data-driven framework for assessing redox kinetics. Experimental validation was critical to confirming the predictive power of our models. We synthesized and tested a wide array of candidate materials, including Mn-doped hercynite and several Gd- and La-based perovskites. Notably, Sr 0.4 Gd 0.6 Mn 0.6 Al 0.4 O 3 (SGMA) and Gd 0.5 La 0.5 Co 0.5 Fe 0.5 O 3 (GLCF) emerged as leading STCH materials, exhibiting robust redox cycling and high hydrogen yields exceeding 150 µmol H 2 /g per cycle. These materials also retained over 50% of their hydrogen productivity under high-conversion conditions (H 2 O:H 2 = 1333:1), demonstrating strong thermodynamic favorability and promising performance under industrially relevant scenarios. Additional candidates, such as La 2 MnNiO 6 (L2MN), were found to produce even higher yields than ceria under standard STCH conditions. Our collaborators at Sandia National Laboratories confirmed these findings using high-temperature X-ray diffraction and thermogravimetric analysis, observing stable phase evolution and reversible redox activity. In several respects, the project went beyond the goals initially outlined in the SOPO. We published 17 peer-reviewed articles, including a large dataset of over 66,000 theoretical perovskites and a new structure prediction method (SPuDS-DFT) that accurately identifies ground-state structures at a fraction of the cost of traditional DFT. We demonstrated that our machine-learned G(T) model offers accuracy rivaling quasiharmonic calculations while being orders of magnitude faster. In partnership with the Materials Project, we made our datasets openly available, providing a powerful new resource for the broader materials science community. The combined computational and experimental advances of this project represent a significant advance in STCH materials discovery. By creating a robust, generalizable, and open workflow for thermodynamic and kinetic screening, and validating key findings through synthesis and reactor testing, we have provided a practical and scalable pathway for the rapid identification of new redox-active materials. The tools, data, and materials developed under this project are already supporting ongoing research and have laid the groundwork for the next generation of solar fuel technologies.

08 HYDROGEN↗

Scientific machine learning for closure models in multiscale problems: A review

Here, closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.

97 MATHEMATICS AND COMPUTING↗

The Application of Machine Learning Techniques to Meteorological Forecasting

Fog and inland-penetrating sea-breezes occur often at SRS and have a strong impact on site operations. Site personnel therefore require accurate forecasts of these events, but both are difficult to forecast using traditional techniques. Our goal is to apply machine learning (ML) techniques to the problem of forecasting fog and the sea breeze at the Savannah River Site. We apply several such techniques - decision trees, regression, and a series of classification/regression techniques – and train them using the large datasets collected by our group at SRS and from external organizations that maintain databases of regional meteorological variables.

54 ENVIRONMENTAL SCIENCES↗

Neural network accelerator for quantum control

Efficient quantum control is necessary for practical quantum computing implementations with current technologies. Conventional algorithms for determining optimal control parameters are computationally expensive, largely excluding them from use outside of the simulation. Existing hardware solutions structured as lookup tables are imprecise and costly. By designing a machine learning model to approximate the results of traditional tools, a more efficient method can be produced. Such a model can then be synthesized into a hardware accelerator for use in quantum systems. In this study, we demonstrate a machine learning algorithm for predicting optimal pulse parameters. This algorithm is lightweight enough to fit on a low-resource FPGA and perform inference with a latency of 175 ns and pipeline interval of 5 ns with > 0.99 gate fidelity. In the long term, such an accelerator could be used near quantum computing hardware where traditional computers cannot operate, enabling quantum control at a reasonable cost at low latencies without incurring large data bandwidths outside of the cryogenic environment.

43 PARTICLE ACCELERATORS↗

Machine-Learning Architecture for Ultrasonic Thermometry

Temperature distribution in solids can be inverted from the speed of sound (SOS) measurements, as has been shown feasible by timing the propagation of the excitation pulse and the train of echoes in ultrasonically segmented waveguides (WGs) and metal components. However, complicated geometries and closely-space echogenic features (EFs) create complex waveforms, from which the segmental time of flights (TOFs) are impossible to estimate using traditional methods. This work describes a machine learning architecture shown to extract temperature information from complex ultrasonic waveforms without explicit measurements of segmental TOFs. We accomplish this by using an autoencoder neural network (NN) to map ultrasonic waveforms into a low-dimensional latent space. A second NN then maps the latent space into unknown temperature distribution along the WG. The proposed architecture was tested in simulations and experimentally. The autoencoder accurately reconstructs the waveforms from their latent representation, several orders of magnitude lower in dimensionality. The obtained latent space was successfully mapped into the temperature of the propagation path.

John, Mason↗

Discovery of structure–property relations for molecules via hypothesis-driven active learning over the chemical space

The discovery of the molecular candidates for application in drug targets, biomolecular systems, catalysts, photovoltaics, organic electronics, and batteries necessitates the development of machine learning algorithms capable of rapid exploration of chemical spaces targeting the desired functionalities. Here, we introduce a novel approach for active learning over the chemical spaces based on hypothesis learning. We construct the hypotheses on the possible relationships between structures and functionalities of interest based on a small subset of data followed by introducing them as (probabilistic) mean functions for the Gaussian process. This approach combines the elements from the symbolic regression methods, such as SISSO and active learning, into a single framework. The primary focus of constructing this framework is to approximate physical laws in an active learning regime toward a more robust predictive performance, as traditional evaluation on hold-out sets in machine learning does not account for out-of-distribution effects which may lead to a complete failure on unseen chemical space. Here, we demonstrate it for the QM9 dataset, but it can be applied more broadly to datasets from both domains of molecular and solid-state materials sciences.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Machine Learning Approach to Quantitative Analysis of Enamel Microstructure from Scanning Electron Microscopy Images

Dental enamel, the outermost tissue of mammalian teeth, must withstand a lifetime of wear and cyclic contact. To meet this demand, enamel possesses a combination of high hardness and resistance to fracture, properties that are typically mutually exclusive. The impressive damage tolerance has been attributed largely to decussation of the enamel rods, the principal unit of its microstructure. As such, enamel is inspiring the design of next‐generation structural materials. However, quantitative descriptions of the decussated enamel rod microstructure remain limited due to challenges encountered in applying computed tomography and in acquiring quality images appropriate for traditional digital processing methods. Here, a machine learning segmentation method is applied to images of the enamel obtained using scanning electron microscopy to support quantitative analysis of the microstructure. A pretrained convolutional neural network is used to expand the input training image dataset to allow the training of a random forest classifier, which ultimately segments the image with a very small training set ( n = 3 images). A validation of this segmentation method is presented, in addition to its application to calculate relevant microstructural parameters for images of tooth enamel from selected mammalian species. The methodology applied here is equally applicable to other hard tissues.

36 MATERIALS SCIENCE↗

Uncovering obscured phonon dynamics from powder inelastic neutron scattering using machine learning

The study of phonon dynamics is pivotal for understanding material properties, yet it faces challenges due to the irreversible information loss inherent in powder inelastic neutron scattering spectra and the limitations of traditional analysis methods. In this study, we present a machine learning framework designed to reveal obscured phonon dynamics from powder spectra. Using a variational autoencoder, we obtain a disentangled latent representation of spectra and successfully extract force constants for reconstructing phonon dispersions. Notably, our model demonstrates effective applicability to experimental data even when trained exclusively on physics-based simulations. The fine-tuning with experimental spectra further mitigates issues arising from domain shift. Analysis of latent space underscores the model’s versatility and generalizability, affirming its suitability for complex system applications. Furthermore, our framework’s two-stage design is promising for developing a universal pre-trained feature extractor. This approach has the potential to revolutionize neutron measurements of phonon dynamics, offering researchers a potent tool to decipher intricate spectra and gain valuable insights into the intrinsic physics of materials.

domain adaptation↗

ClimGen: Learning the Forcing-Response Relationship in Climate System

Solar Radiation Management (SRM) is emerging as a potential geoengineering strategy to address the anthropogenic impact on climate, but its effective implementation requires an iterative and large ensemble of highly accurate and efficient climate projections. Traditional climate projections rely on executing computationally demanding and time-consuming numerical climate models. Recent advances in machine learning (ML) aim to enhance these approaches by emulating traditional methods. In this work, we propose a novel framework for directly learning the relationship between solar radiation flux at the top of the atmosphere and the corresponding surface temperature response. To evaluate the feasibility of this direct ML-based projection, we developed a dataset using an intermediate complexity model, incorporating a comprehensive suite of different forcing patterns and evaluation metrics to rigorously assess the ML model’s performance. We introduce a Conditional Denoising Diffusion Probabilistic Model (cDDPM) for this task, which demonstrates encouraging skill in representing climate statistics under previously unseen forcing patterns. This approach provides a promising pathway for direct climate projections by accurately learning the forcing-response relationship, with a wide range of applications in impact mitigation, emissions policy design, and SRM strategies.

Chen, Tse-Chun [BATTELLE (PACIFIC NW LAB)] (ORCID:↗

Fluorine spillover for ceria- vs silica-supported palladium nanoparticles: A MD study using machine learning potentials

Supported metallic nanoparticles play a central role in catalysis. However, predictive modeling is particularly challenging due to the structural and dynamic complexity of the nanoparticle and its interface with the support, given that the sizes of interest are often well beyond those accessible via traditional ab initio methods. With recent advances in machine learning, it is now feasible to perform MD simulations with potentials retaining near-density-functional theory (DFT) accuracy, which can elucidate the growth and relaxation of supported metal nanoparticles, as well as reactions on those catalysts, at temperatures and time scales approaching those relevant to experiments. Furthermore, the surfaces of the support materials can also be modeled realistically through simulated annealing to include effects such as defects and amorphous structures. We study the adsorption of fluorine atoms on ceria and silica supported palladium nanoparticles using machine learning potential trained by DFT data using the DeePMD framework. We show defects on ceria and Pd/ceria interfaces are crucial for the initial adsorption of fluorine, while the interplay between Pd and ceria and the reverse oxygen migration from ceria to Pd control spillover of fluorine from Pd to ceria at later stages. In contrast, silica supports do not induce fluorine spillover from Pd particles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Supplementary Data for "Evaluation of the Economic Implications of Varied Pressure Drawdown Strategies Generated Using a Real-time, Rapid Predictive, Multi-fidelity Model for Unconventional Oil and Gas Wells" by Bello, K., Vikara, D., Sheriff, A., Viswanathan, H., Carr, T., Sweeney, M., O'Malley, D., Marquis, M., Vactor, R.T., and Cunha, L.

The Bello et al. study evaluates the impact of contrasting pressure drawdown on gas productivity and the resulting economics of a well in the Marcellus Shale of the Appalachian Basin. This research applies a techno-economic analysis approach to help identify potential ways pressure management strategies can be used to improve cumulative recovery of hydraulically fractured horizontal wells while maintaining project profitability. Gas production forecast outlook scenarios of the Marcellus Shale Energy and Environment Laboratory Laboratory's MIP-3H well were generated under varying pressure drawdowns using two approaches: 1) a novel physics-informed machine learning (PIML) workflow and 2) via traditional reservoir simulation in Computer Modeling Group’s (CMG) GEM Compositional & Unconventional Simulator. Cash flow and other economic metrics of interest were compiled on the production outlook using the U.S. Department of Energy's (DOE) National Energy Technology Laboratory (NETL) Unconventional Shale Well Economic Model (UShWEM).The sheets within this Microsoft ExcelTM workbook provide the economic metric outputs for the baseline condition and the one-at-a-time (OAT) sensitivity analysis of UShWEM's input parameters for each of the production scenarios evaluated.

Fracture Network Model↗

Sensor Reduction for Diversion Detection in a Realistic Heat Pipe Microreactor Using Supervised Machine Learning

Microreactors are designed as a smaller, cheaper, and safer alternative to traditional nuclear power plants. Their non-traditional characteristics and prospect of mass production and deployment will likely require new approaches to nuclear safeguards. The primary proliferation concern with microreactors is the diversion of fuel material. Such diversion may produce measurable defects in key physical attributes like neutron flux, which may in turn be detectable using machine learning models. Preliminary work has demonstrated this ability for modeled nominal and diversion scenarios using large quantities of energy integrated neutron flux data. In practice, the number of available sensors for such measurements will be limited and energy integrated flux information will not be available. This work explores the ability of tree-based gradient boosted ensemble models to classify a given microreactor core is nominal or diversion, and determine the number of fuel pins diverted in the case of diversion with reduced numbers of sensors and more realistic detector responses. Classification accuracy of greater than 98% and regression errors as low as 5% of the total number of fuel pins were achieved with as few as 15 sensors, compared to 99% and 4.1% with a maximum of 240 sensors.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

A Graph Dynamical neural network approach for decoding dynamical states in ferroelectrics.

Ferroelectric materials such as BaTiO 3 show tremendous potential for emerging advances in memory devices, particular neuromorphic type devices. High density of memory can be obtained by stabilising polar domain walls at the nanoscale, regions of discontinuity between the well-defined polarization order parameter, but little is known about what controls their structure and dynamics in real nanoscale materials. Indeed, chiral polar domain walls have been observed in heterogeneous ferroelectrics, such as oxygen-deficient BaTiO 3 , but very little is known about how such polar-domains walls interact with defects. Indeed, a critical understanding of how dynamics of domain-walls depend on point-defects is crucial to create engineered ferroelectric memory devices. For this work, we perform large-scale simulations of nansocale domain-wall dynamics in pristine and defective BaTiO 3 using reactive force-field developed by us earlier (Phys. Chem. Chem. Phys., 2019, 21, 18240–18249), and capture their dynamical dependence on point defects using a graph dynamical neural-network approach, which we adapted to interrogate solids with well-defined order-parameters, and implemented using Pytorch based libraries. Our machine learning (ML) approach goes beyond the traditional post-processing methods to capture both spatial and temporal heterogeneities of large-scale molecular dynamics simulations of complex defective ferroelectric oxide materials. We crucially find that isolated oxygen vacancies introduce very localized spatial regions (~1–2 unit-cell in length) that show slow dipole relaxation due to formation of defect-dipoles, and that these defect-dipoles in turn slow the intrinsic dynamics of domain walls. Further, the roughness of domain walls, also influenced by vacancies, introduce dynamic heterogeneity along the domain-wall. As such we find a novel mechanism by which quenched disorder due to defects introduce dynamic heterogeneity thereby influencing response to external fields (particularly time varying fields) in a ferroelectric. Our study also emphasizes the need for creating digital twins of dynamical quantities to achieve autonomous in operando control of nanoscale switching.

42 ENGINEERING↗

Performance on HPC Platforms Is Possible Without C++

Computing at large scales has become extremely challenging due to increasing heterogeneity in both hardware and software. More and more scientific workflows must tackle a range of scales and use machine learning and AI intertwined with more traditional numerical modeling methods, placing more demands on computational platforms. These constraints indicate a need to fundamentally rethink the way computational science is done and the tools that are needed to enable these complex workflows. The current set of C++-based solutions may not suffice, and relying exclusively upon C++ may not be the best option, especially because several newer languages and boutique solutions offer more robust design features to tackle the challenges of heterogeneity. In June 2023, we held a mini symposium that explored the use of newer languages and heterogeneity solutions that are not tied to C++ and that offer options beyond template metaprogramming and Parallel. For for performance and portability. In conclusion, we describe some of the presentations and discussion from the mini symposium in this article.

97 MATHEMATICS AND COMPUTING↗