Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “MACHINE LEARNING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Gearbox bearing crack growth prognostics and uncertainty quantification with physics-informed machine learning

This paper introduces the extreme theory of functional connections (X-TFC), a physics-informed machine learning algorithm, and tailors it to estimate the remaining useful life (RUL) of wind turbine gearbox bearings experiencing fatigue crack growth. Unlike purely data-driven methods, X-TFC embeds a physics model, based on Head's theory in this work, into its training objective. The core of X-TFC is a random-projection single-layer neural network trained via an extreme learning machine, which requires only limited damage progression data and solves for output weights with a least-squares optimization algorithm. A composite loss function balances the network's fit to observed degradation data against the residuals of the governing crack growth differential equation, ensuring the learned damage trajectory remains physically plausible. When applied to a vibration-based health-index (HI) dataset measured during the growth of a crack on the inner ring of a high-speed bearing in a wind turbine gearbox (Bechhoefer and Dubé, 2020), X-TFC achieves near-zero prediction bias. Even when trained on only the first 10 %–20 % of the damage progression data, with sufficient physics weighting its predictions remain monotonic and smooth, delivering high prognosability and trendability. To quantify the epistemic uncertainty, we employ a Monte Carlo ensemble of independently initialized X-TFC models trained on noise-perturbed data, which yields confidence intervals around each RUL estimate and captures both model-parameter and epistemic uncertainty. In addition to a vibration-based HI, we demonstrate that the proposed framework can be directly applied to a supervisory control and data acquisition (SCADA) data-based HI (Eftekhari Milani et al., 2026) measured during similar wind turbine gearbox bearing crack faults, preserving its accuracy and interpretability. This extension shows the versatility of our approach, which is applicable to bearings of multiple gearbox manufacturers, models, and ratings using only SCADA data. By integrating domain knowledge with machine learning, X-TFC offers a rapid, reliable tool for crack prognostics. Its adaptability to other bearing failure modes, such as pitch bearing ring cracks, positions X-TFC as a powerful enabler of data-driven, physics-informed asset management in the wind energy sector and beyond.

17 WIND ENERGY↗
DOE OSTI2023

GOOML (Geothermal Operational Optimization with Machine Learning) [SWR-23-01]

The Geothermal Operational Optimization with Machine Learning (GOOML) is a partnership between NREL and Upflow, NZ, awarded in response to the U.S. Department of Energy's Geothermal Technologies Office's Funding Opportunity Announcement (FOA) to expand the role of advanced analytics and automation in geothermal operations through machine learning. Partnering with industry (Contact Energy Limited ("Contact"), Ngati Tuwharetoa Geothermal Assets Limited ("NTGA"), Ormat Technologies Inc. ("Ormat") and Flow State Solutions Limited ("FSS"), GOOML was created to improve the operational efficiency of geothermal power plant steam fields through the analysis of historical operational data and the application of custom machine learning algorithms. NREL's contributions include machine learning, coding, and data management expertise as well as access to high-performance compute solutions. GOOML can increase geothermal operational efficiency through development of a digital system twin that can be utilized to provide optimal geothermal operating conditions for real-world geothermal fields. GOOML allows users to analyze field production histories in detail, develop models, and train machine learning algorithms to identify opportunities for increased geothermal efficiency, detect potential trouble, and allow predictive scenario modeling. Preliminary experiments have demonstrated a potential to increase total generation by as much as 12% through ML optimization of the utilization of existing steam field resources.

Buster, Grant↗
DOE OSTI2021

Beyond the hubble sequence – exploring galaxy morphology with unsupervised machine learning

We explore unsupervised machine learning for galaxy morphology analyses using a combination of feature extraction with a vector-quantized variational autoencoder (VQ-VAE) and hierarchical clustering (HC). We propose a new methodology that includes: (1) consideration of the clustering performance simultaneously when learning features from images; (2) allowing for various distance thresholds within the HC algorithm; (3) using the galaxy orientation to determine the number of clusters. This set-up provides 27 clusters created with this unsupervised learning that we show are well separated based on galaxy shape and structure (e.g. Sérsic index, concentration, asymmetry, Gini coefficient). These resulting clusters also correlate well with physical properties such as the colour–magnitude diagram, and span the range of scaling relations such as mass versus size amongst the different machine-defined clusters. When we merge these multiple clusters into two large preliminary clusters to provide a binary classification, an accuracy of $\sim 87{{\ \rm per\ cent}}$ is reached using an imbalanced data set, matching real galaxy distributions, which includes 22.7 per cent early-type galaxies and 77.3 per cent late-type galaxies. Comparing the given clusters with classic Hubble types (ellipticals, lenticulars, early spirals, late spirals, and irregulars), we show that there is an intrinsic vagueness in visual classification systems, in particular galaxies with transitional features such as lenticulars and early spirals. Based on this, the main result in this work is not how well our unsupervised method matches visual classifications and physical properties, but that the method provides an independent classification that may be more physically meaningful than any visually based ones.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Exploiting Machine Learning in Multiscale Modelling of Materials

Recent developments in efficient machine learning algorithms have spurred significant interest in the materials community. The inherently complex and multiscale problems in Materials Science and Engineering pose a formidable challenge. The present scenario of machine learning research in Materials Science has a clear lacunae, where efficient algorithms are being developed as a separate endeavour, while such methods are being applied as ‘black-box’ models by others. The present article aims to discuss pertinent issues related to the development and application of machine learning algorithms for various aspects of multiscale materials modelling. The authors present an overview of machine learning of equivariant properties, machine learning-aided statistical mechanics, the incorporation of ab initio approaches in multiscale models of materials processing and application of machine learning in uncertainty quantification. In addition to the above, the applicability of Bayesian approach for multiscale modelling will be discussed. Critical issues related to the multiscale materials modelling are also discussed.

42 ENGINEERING↗
DOE OSTI2022

Special Issue: Geostatistics and Machine Learning

Abstract Recent years have seen a steady growth in the number of papers that apply machine learning methods to problems in the earth sciences. Although they have different origins, machine learning and geostatistics share concepts and methods. For example, the kriging formalism can be cast in the machine learning framework of Gaussian process regression. Machine learning, with its focus on algorithms and ability to seek, identify, and exploit hidden structures in big data sets, is providing new tools for exploration and prediction in the earth sciences. Geostatistics, on the other hand, offers interpretable models of spatial (and spatiotemporal) dependence. This special issue on Geostatistics and Machine Learning aims to investigate applications of machine learning methods as well as hybrid approaches combining machine learning and geostatistics which advance our understanding and predictive ability of spatial processes.

58 GEOSCIENCES↗
DOE OSTI2024

An Efficient Checkpointing System for Large Machine Learning Model Training

As machine learning models increase in size and complexity rapidly, the cost of checkpointing in ML training became a bottleneck in storage and performance (time). For example, the latest GPT-4 model has massive parameters at the scale of 1.76 trillion. It is highly time and storage consuming to frequently writes the model to checkpoints with more than 1 trillion floating point values to storage. This work aims to understand and attempt to mitigate this problem. First, we characterize the checkpointing interface in a collection of representative large machine learning/language models with respect to storage consumption and performance overhead. Second, we propose the two optimizations: i) A periodic cleaning strategy that periodically cleans up outdated checkpoints to reduce the storage burden; ii) A data staging optimization that coordinates checkpoints between local and shared file systems for performance improvement.

machine learning, artificial intelligence↗
DOE OSTI2025

Results and lessons learned from accelerating radio frequency modeling using machine learning [slides]

The “advanced tokamak” reactor concept is a leading candidate for a steady state fusion pilot plant. An advanced tokamak (AT) sustains a majority of the required plasma current with effects resulting from maintenance of the peaked pressure at the device center. This current is augmented by auxiliary current drive sources. These auxiliary actuators may consist of neutral particle beams and/or radio frequency (RF) systems such as lower hybrid current drive (LHCD) and high harmonic fast wave (HHFW) current drive using radio and microwaves from antennas. The primary focus of this work is to develop models of RF current profile control suitable for use in integrated modeling frameworks and for real-time control in experiments. Direct physics models of RF current drive can be computationally intensive. In order to achieve predictive times appropriate for the thousands of calls needed in real-time control of experiments and for use in integrated models, we will apply modern machine learning (ML) techniques to accelerate these models and interpolate their results. To generate the fast and accurate models for use in control level algorithms and integrated modeling we need to replace present models with high dimensional interpolation of their results. We will perform additional simulations across a broader parameter range for EAST and other tokamaks in different physics regimes (Alcator C-Mod, DIII-D, WEST, CFETR, ARC, ITER) and combine them into a larger database for training and testing of the ML models. Further testing of the control level models with experimental current profile data from EAST and C-Mod tokamaks will provide additional confirmation of the control level model before integration in a tokamak control system or integrated modeling suite. ML will be used to optimize the selection of training data consisting of RF current driven at different values of density profile, temperature profile, plasma current, and wavenumber. ML will also be used to facilitate classification of current drive from these input data. The output of this effort will be a validated classifier capable of determining the current drive profiles for HHFW CD and LHCD on a mille-second timescale. This will provide a breakthrough capability enabling real-time control of RF driven current profiles in experiments including ITER ICRF and use integrated modeling frameworks requiring thousands of current profile calculations in discharge simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

SMALE: Enhancing Scalability of Machine Learning Algorithms on Extreme-Scale Computing Platforms

Deployment and execution of machine learning tasks on extreme-scale computing platforms face several significant technical challenges: 1) High computing cost incurred by dense networks – The computing workload of deep networks with densely-connected topology increases rapidly with the network size, imposing a non-scalable computing model of extreme-scale computing platforms; 2) Non-optimized workload distribution – Many advanced deep learning algorithms, e.g., sparsification and irregular net-work topology, produce very unbalanced workload distribution on extreme-scale computing platforms. The computation efficiency is greatly hindered by the incurred data and computation redundancies as well as long tails of the node with extensive workload; 3) Constraints in data movement and I/O bottle-neck – Inter-node data movement in extreme-scale computing platforms are associated with high energy and latency costs, and subject to the constraints of I/O bandwidth; and 4) Generalization of algorithm realization and acceleration on computing platforms – The large varieties of machine learning algorithms and structures of extreme-scale computing platforms make the derivation of a generalized algorithm realization and acceleration method very challenging, which, however, is the requirement by domain scientists and interested users. We call the above challenges Smale’s Problems in Machine Learning and Understanding for High-Performance Computing Scientific Discovery. The objective of our three-year research project is to develop a holistic innovation set at structure, assembly, and acceleration layers of machine learning algorithms to address the above challenges in algorithm deployment and execution. Three tasks are particularly performed, including: At the algorithm structure level, we investigate the techniques that can structurally sparsify on the topology of deep networks for computing workload reduction. We also study clustering and pruning techniques that can optimize the workload distributions over the extreme-scale computing platforms; At the algorithm assembly level, we derive a unified learning framework for unsupervised transfer learning and dynamic growing capabilities. Novel training methods are also exploited to enhance the training efficiency of the proposed framework; At the algorithm acceleration level, we will develop a series of techniques that can accelerate the computation of sparse matrix operations, which are one of the core executions in deep learning and optimize memory access of the concerned platforms. Our proposed techniques attack the fundamental problems in machine learning algorithms running on extreme-scale computing platforms by vertically integrating the solutions at three closely entangled layers, paving the long-term scaling path of machine learning applications under DOE context. Three tasks corresponding to the above respective research orientations are performed during the three-year project period with our collaborators at ORNL. The outcome of the proposed project is anticipated to form a holistic solution set of novel algorithms and network topologies, efficient training techniques, and fast acceleration methods to promote the computing scalability of the machine learning applications of particular interest to DOE.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Small angle scattering of diblock copolymers profiled by machine learning

We outline a machine learning strategy for quantitively determining the conformation of AB-type diblock copolymers with excluded volume effects using small angle scattering. Complemented by computer simulations, a correlation matrix connecting conformations of different copolymers according to their scattering features is established on the mathematical framework of a Gaussian process, a multivariate extension of the familiar univariate Gaussian distribution. We show that the relevant conformational characteristics of copolymers can be probabilistically inferred from their coherent scattering cross sections without any restriction imposed by model assumptions. This work not only facilitates the quantitative structural analysis of copolymer solutions but also provides the reliable benchmarking for the related theoretical development of scattering functions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Full-stack Quantification of Variability in Predicting Ion Transport Properties using Machine-learned Interatomic Potentials

Machine-learned interatomic potentials (MLIPs) have become the state-of-the-art for performing accurate, scalable molecular dynamics (MD) simulations. It is therefore crucial to understand and quantify the reliability of MLIPs for downstream property predictions. Uncertainty in predicted properties can arise from limitations in first-principles training data, intrinsic MLIP model errors in representing the data, and the statistical noise introduced during subsequent MD simulations. Using ion transport in Li7P3S11 as a case study, we systematically assess the impact of training set size and selection, neural network stochasticity, and MD sampling statistics on predicted diffusivity and activation energy. We find that when using equivariant MLIP architectures with standard MD protocols, uncertainty arising from MD sampling dominates over model-induced errors. In contrast, MLIP errors relative to the underlying first-principles data are consistently minor. Given this, there are two main routes to improving the accuracy of predictions based on MLIP potentials: adopting higher accuracy reference data generation methods, and improving the MD sampling statistics.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Machine Learning for Sustainable Energy Systems

In recent years, machine learning has proven to be a powerful tool for deriving insights from data. In this review, we describe ways in which machine learning has been leveraged to facilitate the development and operation of sustainable energy systems. We first provide a taxonomy of machine learning paradigms and techniques, along with a discussion of their strengths and limitations. We then provide an overview of existing research using machine learning for sustainable energy production, delivery, and storage. Finally, we identify gaps in this literature, propose future research directions, and discuss important considerations for deployment.

Environmental Sciences & Ecology↗
DOE OSTI2025

Open-source generation of sigma profiles: impact of quantum chemistry and solvation treatment on machine learning performance

The combination of machine learning (ML) models with chemistry-related tasks requires the description of molecular structures in a machine-readable way. The nature of these so-called molecular descriptors has a direct and major impact on the performance of ML models and remains an open problem in the field. Structural descriptors like SMILES strings or molecular graphs lack size-independence and can be memory intensive. Machine-learned descriptors can be of low dimensionality and constant size but lack physical significance and human interpretability. Sigma profiles, which are unnormalized histograms of the surface charge distributions of solvated molecules, combine physical significance with low dimensionality and size-independence, making them a suitable candidate for a universal molecular descriptor. However, their widespread adoption in ML applications requires open access to sigma profile generation, which is currently not available. This work details the development of OpenSPGen – an open-source tool for generating sigma profiles. Also presented are studies on the effect of different settings on the efficacy of the generated sigma profiles at predicting thermophysical material properties when used as inputs to a Gaussian process as a simple surrogate ML model. We find that a higher level of theory does not translate to more accurate results. We also provide further recommendations for sigma profile calculation and use in ML models.

Salih, Fathya Y. M. [University of Notre Dame, IN ↗
DOE OSTI2026

Quantum-Inspired Bayesian Sampling for Uncertainty Quantification and Machine Learning (Final Technical Report)

With increasing simulation and measurement data, machine learning and artificial intelligence have been widely used in computational decision-making of complex engineering systems. The resulting tools, such as uncertainty quantification solvers, reinforcement learning, and physics-informed machine learning, have achieved great success in critical DOE tasks such as material discovery and design, energy system modeling and control, and numerical weather and climate prediction. A core topic in scientific machine learning and artificial intelligence is Bayesian inference: given an observed data set, people want to estimate the posterior distribution of a (possibly large) number of hidden parameters. Due to the flexibility and weak assumptions, Bayesian sampling has been the mainstream Bayesian inference solvers despite the rapid progress of approximate Bayesian inference. Classical Bayesian sampling methods such as Markov-chain Monte Carlo suffer from a low-acceptance rate due to the random walk nature, therefore state-of-the-art techniques use Hamiltonian Monte Carlo and its variants to efficiently draw posterior samples in a high dimension. The key idea of Hamiltonian Monte Carlo and its variants is to simulate the Hamiltonian dynamics of a classical particle with a fixed mass, and their performance significantly degrades when the posterior distribution is highly spiky or has multiple modes. Leveraging the idea of quantum physics, this project has investigated new theory, algorithms and applications of Bayesian inference (especially Bayesian sampling). The main results include: (1) novel quantum-inspired Bayesian sampling methods that can lead to better accuracy for challenging multi-modal or spiky distributions, (2) more scalable machine learning framework leveraging tensor-compressed Bayesian inference, and (3) Bayesian and sampling approaches for verifying the robustness of continuous and binary neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Physics guided machine learning using simplified theories

Recent applications of machine learning, in particular deep learning, motivate the need to address the generalizability of the statistical inference approaches in physical sciences. In this Letter, we introduce a modular physics guided machine learning framework to improve the accuracy of such data-driven predictive engines. The chief idea in our approach is to augment the knowledge of the simplified theories with the underlying learning process. To emphasize their physical importance, our architecture consists of adding certain features at intermediate layers rather than in the input layer. To demonstrate our approach, we select a canonical airfoil aerodynamic problem with the enhancement of the potential flow theory. We include the features obtained by a panel method that can be computed efficiently for an unseen configuration in our training procedure. By addressing the generalizability concerns, our results suggest that the proposed feature enhancement approach can be effectively used in many scientific machine learning applications, especially for the systems where we can use a theoretical, empirical, or simplified model to guide the learning module.

42 ENGINEERING↗
DOE OSTI2026

Hierarchical transfer learning: an agile and equitable strategy for machine-learning interatomic models

Machine-learned interatomic models are growing in popularity due to their ability to afford near quantum-accurate predictions for complex phenomena with orders-of-magnitude greater computational efficiency. However, these models struggle when applied to systems of many element types due to the approximately exponential increase in number of parameters that must be determined. To mitigate this challenge, we present a new hierarchical transfer learning approach that allows the fitting problem to be decomposed into smaller independent and reusable parameter blocks that enable development of explicitly chemically extensible ML-IAM. Application of this strategy is demonstrated for C and N mixtures under conditions ranging from nominally ambient to ~10,000 K and 200 GPa for compositions from 0 to 100% N. Ultimately, this strategy makes model generation for chemically complex systems more tractable and efficient, facilitates comprehensive model validation, and makes ML-IAM development for problems of this nature more accessible to users with limited access to extreme computing infrastructure.

Lindsey, Rebecca K. [Univ. of Michigan, Ann Arbor,↗
DOE OSTI2022

Quantum machine learning for chemistry and physics

Machine learning (ML) has emerged as a formidable force for identifying hidden but pertinent patterns within a given data set with the objective of subsequent generation of automated predictive behavior. In recent years, it is safe to conclude that ML and its close cousin, deep learning (DL), have ushered in unprecedented developments in all areas of physical sciences, especially chemistry. Not only classical variants of ML, even those trainable on near-term quantum hardwares have been developed with promising outcomes. Such algorithms have revolutionized materials design and performance of photovoltaics, electronic structure calculations of ground and excited states of correlated matter, computation of force-fields and potential energy surfaces informing chemical reaction dynamics, reactivity inspired rational strategies of drug designing and even classification of phases of matter with accurate identification of emergent criticality. In this review we shall explicate a subset of such topics and delineate the contributions made by both classical and quantum computing enhanced machine learning algorithms over the past few years. We shall not only present a brief overview of the well-known techniques but also highlight their learning strategies using statistical physical insight. The objective of the review is not only to foster exposition of the aforesaid techniques but also to empower and promote cross-pollination among future research in all areas of chemistry which can benefit from ML and in turn can potentially accelerate the growth of such algorithms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

A Data-Agnostic, Continuous Machine Learning Framework for Application in High Energy Physics and Beyond: Phase 1 Final Scientific/Technical Report

This Phase 1 effort has focused on the development of continual learning frameworks for use in machine learning, specifically in the applied context of High Energy Physics (HEP). Machine learning (ML) is a transformative technology by which computers, typically through the use of neural networks, are able to perform tasks with proficiency that rivals or surpasses that of human users. Model Degradation & Catastrophic Forgetting are two undesired phenomena which can occur in ML where the performance of a model degrades when either deployed on novel data streams, or trained on novel data which are sufficiently different than the data the models were initially trained on. A natural example where these sorts of effects can be observed is in the performance of detectors in harsh environments, where the detector signature may change over the lifetime of the detector as it ages and deteriorates — precisely what occurs in the experiments conducted in HEP. Real world HEP data is therefore an excellent test-ground and use-case for Continual Learning paradigms, which are techniques used in ML to counteract these problems. Ensemble learning is one such technique, where multiple smaller models are trained on subsets of the overall data and are ensembled together during inference. The intuition behind this technique is that, although there are shifts in the distributions which govern the incoming data streams, these shifts are not expected to be homogeneous or global. If a sufficient diversity in solutions within the various sub-models has been achieved, then at least one sub-model is expected to retain its performance within the overall ensemble. One further strength of this approach is that the architectures of the various models do not need to be identical, and in fact even different modalities of data can naturally be combined in this way. This work focused on applying ensemble learning techniques to derive results using two main datasets, anomaly detection in HEP data & time-series forecasting in semiconductor manufacturing data. Semiconductor manufacturing involves data with surprising similarity to that of HEP (e.g. wafer maps look very similar to digi-occupancy maps) and Cerium Lab’s prominence within the semiconductor industry makes semiconductor manufacturing a natural opportunity for commercialization of this work. Our efforts have led to two strong results. The first is that we evaluated the proposed ensembling techniques using previously proposed machine learning architectures for use in anomaly detection, namely AutoEncoder based models and their derivatives. We also developed new architectures which have not been evaluated in this context before. In fact, this work marks the first use of Vision Transformers for anomaly detection in HEP. Second, we demonstrated that ensemble learning significantly improves model performance in scenarios prone to degradation, validating its effectiveness across both HEP and semiconductor datasets. These results further support ensemble learning as a powerful strategy for mitigating catastrophic forgetting and maintaining robust performance in evolving data environments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗