Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Learning with errors”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

DeepShake: Shaking Intensity Prediction Using Deep Spatiotemporal RNNs for Earthquake Early Warning

Abstract We propose a deep spatiotemporal recurrent neural network, DeepShake, to project future shaking intensity directly from current ground-motion observations. DeepShake is a network-based forecasting model, able to predict future shaking intensity at all stations within a network given previously measured ground shaking. The model is not given any a priori knowledge of station locations; instead, it learns wave propagation amplitudes and delays solely from training data. We developed DeepShake with the 35,679 earthquakes from the 2019 Ridgecrest sequence. Tasked with alerting for modified Mercalli intensity (MMI) IV+ shaking on 3568 validation earthquakes at least 5 s in advance, DeepShake achieves an equal error rate of 11.4%. For the Mw 7.1 earthquake that hit Ridgecrest on 5 July 2019, DeepShake was able to provide targeted alerts to all stations inside the network 5 s prior to the arrival of MMI IV+ waveforms. DeepShake demonstrates that deep spatiotemporal neural networks can effectively provide one-step earthquake early warning with reasonable accuracy and latency.

Geochemistry & Geophysics↗
DOE OSTI2022

Entangling Quantum Generative Adversarial Networks

Generative adversarial networks (GANs) are one of the most widely adopted machine learning methods for data generation. In this work, we propose a new type of architecture for quantum generative adversarial networks (an entangling quantum GAN, EQ-GAN) that overcomes limitations of previously proposed quantum GANs. Leveraging the entangling power of quantum circuits, the EQ-GAN converges to the Nash equilibrium by performing entangling operations between both the generator output and true quantum data. In the first multiqubit experimental demonstration of a fully quantum GAN with a provably optimal Nash equilibrium, we use the EQ-GAN on a Google Sycamore superconducting quantum processor to mitigate uncharacterized errors, and we numerically confirm successful error mitigation with simulations up to 18 qubits. Finally, we present an application of the EQ-GAN to prepare an approximate quantum random access memory and for the training of quantum neural networks via variational datasets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

A hybrid Penman-Monteith and machine learning model for simulating evapotranspiration and its components

Integrating physical processes with machine learning has advanced evapotranspiration (ET) simulation, yet most hybrid models fail to partition total ET into its components: soil evaporation (E) and vegetation transpiration (T). This study introduces Residual Neural Network–Penman–Monteith (RNN-PM), a novel hybrid dual-source ET model designed to overcome this limitation. The model synergizes the physically-based Penman–Monteith framework with three specialized residual neural networks trained to estimate key conductance parameters (canopy conductance, soil surface conductance, and aerodynamic conductance). Furthermore this explicit parameterization allows for the direct partitioning of total ET. Validation at National Ecological Observatory Network (NEON) flux sites using high-frequency partitioned E and T shows that RNN-PM reliably reproduces ET and the transpiration fraction (T/ET). For ET, the model achieves an average Kling–Gupta efficiency (KGE) of 0.89 and a root-mean-square error (RMSE) of 0.55 mm/day; for T/ET, the KGE is 0.87 with an RMSE of 0.06. Furthermore, RNN-PM demonstrates robust generalization, accurately simulating ET and its components well beyond the initial training dataset, even under extreme climatic conditions. This study extended the analysis by comparing the RNN-PM model with seven established dual-source ET models. The results indicate that RNN-PM outperforms both conventional machine learning models and purely physical process-based models in simulating ET components in most cases. Among the purely physical process-based dual-source models, those based on surface temperature decomposition showed improved performance as the leaf area index (LAI) decreased when evaluated against high-frequency ET component datasets. In contrast, the performance of conductance-based dual-source models declined with decreasing LAI. Although purely machine learning-based models can produce relatively accurate simulations of ET components, they often exhibit limited generalization capability, an issue that the RNN-PM model effectively overcomes. Ultimately, the RNN-PM model represents a significant advance in simulating ET components, offering a novel and scalable approach for improving the representation of land–atmosphere interactions in Earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction

Abstract The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified Swin Transformer V2 (SwinV2) on ERA5 data and find that it attains superior skill in terms of mean-square errors of deterministic forecasts when compared against the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS). Almost all DL–NWP systems share a core set of hyperparameters and design decisions. To aid and expedite future DL–NWP research, we present an in-depth, systematic exploration of different loss functions, model sizes and depths, patch sizes, and multistep training objectives. We also examine the model performance with metrics beyond the typical accuracy (ACC) and RMSE and investigate how the performance scales with model size. Through our open-source code, scoring pipelines, and models, we share our findings on key aspects of the training pipeline. These ablations reduce the necessity for expensive hyperparameter tuning and lower the barrier to entry for future DL–NWP research. Significance Statement This study investigates the potential of using large-scale transformer-based models for weather prediction, showing that it is possible to achieve high forecast accuracy with simpler, off-the-shelf architectures. By training a minimally modified SwinV2 transformer on ERA5 data, we show that the model achieves competitive forecast skill in terms of mean-square error for key variables, outperforming the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS) at all lead times. Our findings suggest that effective training strategies, such as multistep fine-tuning and channel-weighted losses, significantly enhance the model’s performance. However, we also highlight that these improvements come with trade-offs in other areas, such as ensemble spread and high-frequency spatial detail. This work highlights the promise of deep learning in improving weather forecasts, which could lead to better preparedness and response to weather events, ultimately benefiting society by providing more reliable weather predictions.

Willard, Jared D. [Lawrence Berkeley National Labo↗
DOE OSTI2024

Data-scarce surrogate modeling of shock-induced pore collapse process

Understanding the mechanisms of shock-induced pore collapse is of great interest in various disciplines in sciences and engineering, including materials science, biological sciences, and geophysics. However, numerical modeling of the complex pore collapse processes can be costly. To this end, a strong need exists to develop surrogate models for generating economic predictions of pore collapse processes. Here, in this work, we study the use of a data-driven reduced-order model, namely dynamic mode decomposition, and a deep generative model, namely conditional generative adversarial networks, to resemble the numerical simulations of the pore collapse process at representative training shock pressures. Since the simulations are expensive, the training data are scarce, which makes training an accurate surrogate model challenging. To overcome the difficulties posed by the complex physics phenomena, we make several crucial treatments to the plain original form of the methods to increase the capability of approximating and predicting the dynamics. In particular, physics information is used as indicators or conditional inputs to guide the prediction. In realizing these methods, the training of each dynamic mode composition model takes only around 30 s on CPU. In contrast, training a generative adversarial network model takes 8 h on GPU. Moreover, using dynamic mode decomposition, the final-time relative error is around 0.3% in the reproductive cases. We also demonstrate the predictive power of the methods at unseen testing shock pressures, where the error ranges from 1.3 to 5% in the interpolatory cases and 8 to 9% in extrapolatory cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Deep learning approaches for instantaneous laser absorptance prediction in additive manufacturing

Abstract The quantification of absorbed light is essential for understanding laser-material interactions and melt pool dynamics in order to minimize defects in additively manufactured metal components. The geometry of a vapor depression formed during laser melting is closely related to laser energy absorption. This relationship has been observed by the state-of-the-art in situ high-speed synchrotron X-ray visualization and integrating sphere radiometry. These two techniques create a temporally resolved dataset consisting of vapor depression images and corresponding laser absorptance. In this work, we propose two different approaches to predict instantaneous laser absorptance. The end-to-end approach uses deep convolutional neural networks to learn implicit features of X-ray images automatically and predict the laser energy absorptance. The two-stage approach uses a semantic segmentation model to engineer geometric features and predict absorptance using classical regression models. While having distinct advantages, both approaches achieved a consistently low mean absolute error of less than 3.3%.

Chemistry↗
DOE OSTI2026

EXERGETIC: De-Risking Next-Generation Resilient Geothermal Hybrids via At-Scale Evaluation Using Virtual Emulation Digital Twin Environment for Efficient Operation

The DOE-GTO-funded project, award number 5.1.2.12, entitled "EXERGETIC - De-risking Next Generation Resilient Geothermal Hybrids via at-Scale Evaluation Using a Virtual Emulation Digital Twin Environment for Efficient Operation," advances the solution to these challenges by developing and validating a geothermal co-emulation environment implemented at the National Laboratory of the Rockies (NLR)'s Advanced Research on Integrated Energy Systems (ARIES) platform. This framework enables the de-risking of next-generation geothermal and geothermal hybrid systems through high-fidelity modeling, real-time digital emulation, advanced control strategies, and techno-economic assessment. The project focused on geothermal hybrid configurations that integrate geothermal power plants with concentrated solar power and underground thermal energy storage, enabling enhanced efficiency, flexibility, and grid support capabilities. The main goal of this project was the development of a geothermal digital co-emulation environment to demonstrate the technical and economic value of geothermal hybrid systems and their contribution to grid stability and flexibility. The EXERGETIC framework combined physics-based models, controls, and real assets at ARIES, including digital real-time simulators (DRTS), a 20-MW-scale controllable grid interface (CGI), and a 2-MW conventional generator. Detailed transient models were developed for the key subsystems of a hybrid geothermal plant, including parabolic trough solar collectors, reservoir thermal energy storage (RTES), and a binary Organic Rankine Cycle (ORC) power plant. The ORC model explicitly captured thermal inertia and off-design operation and integrated control strategies to dynamically respond to electric load profiles. The models were validated against published experimental and numerical studies, demonstrating strong agreement and confirming the accuracy and robustness of the modeling approach. The resulting digital twin represents geothermal-solar-storage systems at multiple scales (1 MW to 100 MW) and enables realistic emulation of grid-connected operation. The control architecture allows the geothermal resource to provide stable baseload generation, while solar and stored thermal energy supply flexible, dispatchable support during periods of high demand or variable grid conditions. A key contribution of the EXERGETIC project is the demonstration that geothermal hybrid systems can be designed to be active grid assets rather than passive baseload generators. Using the ARIES platform, the digital twin was evaluated under multiple grid scenarios, including load following, voltage support at the distribution level, and frequency response at the transmission level. Results show that hybrid geothermal systems can respond effectively to dynamic grid conditions, providing inertia-like behavior, primary frequency support, and voltage regulation through coordinated control. In addition to the performance and grid services capability analysis of geothermal and hybrid geothermal systems, the EXERGETIC project also focused on scalability and techno-economic analysis of geothermal hybrid plants. In particular, for the scalability analysis, machine-learning (ML)-based surrogate models were trained using data generated from the geothermal digital twin under different grid-connected scenarios and plant capacities. These ML models demonstrated strong interpolation and extrapolation capabilities across plant sizes, accurately reproducing both steady-state and transient responses with very low errors. Regarding the techno-economic analysis, plant performance results were integrated with cost models for hybrid geothermal systems, and the levelized cost of electricity (LCOE) was used as the main economic metric to evaluate system performance across a range of system capacities, solar shares, solar multiples, and storage durations. Results indicate that economies of scale significantly reduce geothermal LCOE as plant capacity increases, with large-scale systems (25-100 MW) achieving substantially lower costs than small plants. Hybridization with solar thermal energy and storage further improves economic performance by increasing capacity utilization and enabling flexible dispatch. In addition, thermal storage plays a critical role in reducing LCOE by maximizing geothermal, solar, and stored energy resources. In summary, the results from this project demonstrate that geothermal hybrid systems represent a promising alternative for increasing the energy conversion efficiency of geothermal technologies, contributing to the preservation of geothermal resources, and supporting the transition of geothermal plants from traditional baseload resources into flexible, resilient, and cost-competitive energy conversion technologies.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Application of machine learning for optical emission spectroscopy data in NAGDIS-II

In this study, we applied machine learning to optical emission spectroscopy (OES) data and device parameters from the linear plasma device NAGDIS-II to explore the potential application of machine learning for predicting electron density, $n$ e , and temperature, $T$ e . The covered ranges of $n$ e and $T$ e , which were measured by an electrostatic probe, are 3.6 × 10 17 –2.4 × 10 19 m -3 and 0.3–7.1 eV, respectively. A three hidden layer neural network (NN) is introduced to model the relationship between $n$ e /$T$ e and the combination of line intensities, radial position, and device parameters. It is shown that the errors in $n$ e and $T$ e become 18.0 and 18.8%, respectively, which were almost the same level for the electrostatic probe, using all available data. Lasso regression and greedy algorithm are used to select the necessary line emissions. In conclusion, it is shown that four- or five-line intensities are sufficient to obtain almost the same quality as the one with all the other lines.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Running Ensemble Workflows at Extreme Scale: Lessons Learned and Path Forward

The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.

Mehta, Kshitij↗
DOE OSTI2021

Measuring the electron temperature and identifying plasma detachment using machine learning and spectroscopy

A machine learning approach has been implemented to measure the electron temperature directly from the emission spectra of a tokamak plasma. This approach utilized a neural network (NN) trained on a dataset of 1865 time slices from operation of the DIII-D tokamak using extreme ultraviolet/vacuum ultraviolet emission spectroscopy matched with high-accuracy divertor Thomson scattering measurements of the electron temperature, T e . This NN is shown to be particularly good at predicting T e at low temperatures (T e < 10 eV) where the NN demonstrated a mean average error of less than 1 eV. Trained to detect plasma detachment in the tokamak divertor, a NN classifier was able to correctly identify detached states (T e < 5 eV) with a 99% accuracy (an F 1 score of 0.96) at an acquisition rate 10× faster than the Thomson scattering measurement. The performance of the model is understood by examining a set of 4800 theoretical spectra generated using collisional radiative modeling that was also used to predict the performance of a low-cost spectrometer viewing nitrogen emission in the visible wavelengths. Furthermore, these results provide a proof-of-principle that low-cost spectrometers leveraged with machine learning can be used to boost the performance of more expensive diagnostics on fusion devices and be used independently as a fast and accurate T e measurement and detachment classifier.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Bayesian learning for rapid prediction of lithium-ion battery-cycling protocols

Advancing lithium-ion battery technology requires the optimization of cycling protocols. A new data-driven methodology is demonstrated for rapid, accurate prediction of the cycle life obtained by new cycling protocols using a single test lasting only 3 cycles, enabling rapid exploration of cycling protocol design spaces with orders of magnitude reduction in testing time. We achieve this by combining lifetime early prediction with a hierarchical Bayesian model (HBM) to rapidly predict performance distributions without the need for extensive repetitive testing. The methodology is applied to a comprehensive dataset of lithium-iron-phosphate/graphite comprising 29 different fast-charging protocols. HBM alone provides high protocol-lifetime prediction performance, with 6.5% of overall test average percent error, after cycling only one battery to failure. Here, by combining HBM with a battery lifetime prediction model, we achieve a test error of 8.8% using a single 3-cycle test. In addition, the generalizability of the HBM approach is demonstrated for lithium-manganese-cobalt-oxide/graphite cells.

25 ENERGY STORAGE↗
DOE OSTI2024

Haar-Like Wavelets on Hierarchical Trees

Here, discrete wavelet methods, originally formulated in the setting of regularly sampled signals, can be adapted to data defined on a point cloud if some multiresolution structure is imposed on the cloud. A wide variety of hierarchical clustering algorithms can be used for this purpose, and the multiresolution structure obtained can be encoded by a hierarchical tree of subsets of the cloud. Prior work introduced the use of Haar-like bases defined with respect to such trees for approximation and learning tasks on unstructured data. This paper builds on that work in two directions. First, we present an algorithm for constructing Haar-like bases on general discrete hierarchical trees. Second, with an eye towards data compression, we present thresholding techniques for data defined on a point cloud with error controlled in the $L$ $\infty$ norm and in a Hölder-type norm. In a concluding trio of numerical examples, we apply our methods to compress a point cloud dataset, study the tightness of the $L$ $\infty$ error bound, and use thresholding to identify MNIST classifiers with good generalizability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Neural Architecture and Feature Search for Predicting the Ridership of Public Transportation Routes

Accurately predicting the ridership of public-transit routes provides substantial benefits to both transit agencies, who can dispatch additional vehicles proactively before the vehicles that serve a route become crowded, and to passengers, who can avoid crowded vehicles based on publicly available predictions. The spread of the coronavirus disease has further elevated the importance of ridership prediction as crowded vehicles now present not only an inconvenience but also a public-health risk. At the same time, accurately predicting ridership has become more challenging due to evolving ridership patterns, which may make all data except for the most recent records stale. One promising approach for improving prediction accuracy is to fine-tune the hyper-parameters of machine-learning models for each transit route based on the characteristics of the particular route, such as the number of records. However, manually designing a machine-learning model for each route is a labor-intensive process, which may require experts to spend a significant amount of their valuable time. To help experts with designing machine-learning models, we propose a neural-architecture and feature search approach, which optimizes the architecture and features of a deep neural network for predicting the ridership of a public-transit route. Our approach is based on a randomized local hyper-parameter search, which minimizes both prediction error as well as the complexity of the model. We evaluate our approach on real-world ridership data provided by the public transit agency of Chattanooga, TN, and we demonstrate that training neural networks whose architectures and features are optimized for each route provides significantly better performance than training neural networks whose architectures and features are generic.

Ayman, Afiya↗
DOE OSTI2021

Machine Learning Tools Set for Natural Gas Fuel Cell System Design

This study is focusing on leveraging the system design tools set for the next-generation solid oxide fuel cell (SOFC) based natural gas fuel cell (NGFC) system. Conventionally, system design and optimization of NGFC systems rely heavily on traditional reduced order model (ROM) techniques and designers’ experience level. For overcoming the technical barriers of system design, multiple multi-physics models and machine learning (ML) tools have been utilized to automate the conceptual design process and enhance the reliability of solutions for the NGFC system. The proposed tools set includes a physics-informed ML tool for automated ROM construction that leverages advances in deep neural networks to significantly reduce ROM prediction error for the NGFC power island compared to traditional approaches. The constructed physics-informed ML ROM can be used in system design, and optimization tools set Institute for the Design of Advanced Energy Systems (IDAES) Process Systems Engineering (PSE) framework. The tools set also provides a user-friendly graphic user interface built within Jupyter Notebooks, and the complete tools set is open-source public available.

Wang, Dewei↗
DOE OSTI2022

Projecting the Thermal Response in a HTGR-Type System during Conduction Cooldown Using Graph-Laplacian Based Machine Learning

Accurate prediction of an off-normal event in a nuclear reactor is dependent upon the availability of sensory data, reactor core physical condition, and understanding of the underlying phenomenon. This work presents a method to project the data from some discrete sensory locations to the overall reactor domain during conduction cooldown scenarios similar to High Temperature Gas-cooled Reactors (HTGRs). The existing models for conductive cooldown in a heterogeneous multi-body system, such as an assembly of prismatic blocks or pebble beds relies on knowledge of the thermal contact conductance, requiring significant knowledge of local thermal contacts and heat transport possibilities across those contacts. With a priori knowledge of bulk geometry features and some discrete sensors, a machine learning approach was devised. The presented work uses an experimental facility to mimic conduction cooldown with an assembly of 68 cylindrical rods initially heated to 1200 K. High-fidelity temperature data were collected using an infrared (IR) camera to provide training data to the model and validate the predicted temperature data. The machine learning approach used here first converts the macroscopic bulk geometry information into Graph-Laplacian, and then uses the eigenvectors of the Graph-Laplacian to develop Kernel functions. Support vector regression (SVR) was implemented on the obtained Kernels and used to predict the thermal response in a packed rod assembly during a conduction cooldown experiment. The usage of SVR modeling differs from most models today because of its representation of thermal coupling between rods in the core. When trained with thermographic data, the average normalized error is less than 2% over 400 s, during which temperatures of the assembly have dropped by more than 500 K. The rod temperature prediction performance was significantly better for rods in the interior of the assembly compared to those near the exterior, likely due to the model simplification of the surroundings.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Operation-adversarial scenario generation

This paper proposes a modified conditional generative adversarial network (cGAN) model to generate net load scenarios for power systems that are statistically credible, conditioned by given labels (e.g., seasons), and, at the same time, “stressful” to the system operations and dispatch decisions. The measure of stress used in this paper is based on the operating cost increases due to net load changes. The proposed operation-adversarial cGAN (OA-cGAN) internalizes a DC optimal power flow model and seeks to maximize the operating cost and achieve a worst-case data generation. The training and testing stages employed in the proposed OA-cGAN use historical day-ahead net load forecast errors and has been implemented for the realistic NYISO 11-zone system. In conclusion, our numerical experiments demonstrate that the generated operation-adversarial forecast errors lead to more cost-effective and reliable dispatch decisions.

42 ENGINEERING↗
DOE OSTI2025

Prediction of hydration energies of adsorbates at Pt(111) and liquid water interfaces using machine learning

Aqueous phase heterogeneous catalysis is important to various industrial processes, including biomass conversion, Fischer–Tropsch synthesis, and electrocatalysis. Accurate calculation of solvation thermodynamic properties is essential for modeling the performance of catalysts for these processes. Explicit solvation methods employing multiscale modeling, e.g., involving density functional theory and molecular dynamics have emerged for this purpose. Although accurate, these methods are computationally intensive. This study introduces machine learning (ML) models to predict solvation thermodynamics for adsorbates on a Pt(111) surface, aiming to enhance computational efficiency without compromising accuracy. In particular, ML models are developed using a combination of molecular descriptors and fingerprints and trained on previously published water–adsorbate interaction energies, energies of solvation, and free energies of solvation of adsorbates bound to Pt(111). These models achieve root mean square error values of 0.09 eV for interaction energies, 0.04 eV for energies of solvation, and 0.06 eV for free energies of solvation, demonstrating accuracy within the standard error of multiscale modeling. Feature importance analysis reveals that hydrogen bonding, van der Waals interactions, and solvent density, together with the properties of the adsorbate, are critical factors influencing solvation thermodynamics. Furthermore, these findings suggest that ML models can provide rapid and reliable predictions of solvation properties. This approach not only reduces computational costs but also offers insights into the solvation characteristics of adsorbates at Pt(111)–water interfaces.

Adsorption↗