Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learned”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,585 records · Page 88

Deep multiscale model learning

The objective of this article is to design novel multi-layer neural networks for multiscale simulations of flows taking into account the observed fine data and physical modeling concepts. Our approaches use deep learning techniques combined with local multiscale model reduction methodologies to predict flow dynamics. Using reduced-order model concepts is important for constructing robust deep learning architectures since the reduced-order models provide fewer degrees of freedom. We consider flow dynamics in porous media as multi-layer networks in this work. More precisely, the solution (e.g., pressures and saturation) at the time instant n+1 depends on the solution at the time instant n and input parameters, such as permeability fields, forcing terms, and initial conditions. One can regard the solution as a multi-layer network, where each layer, in general, is a nonlinear forward map and the number of layers relates to the internal time steps. We will rely on rigorous model reduction concepts to define unknowns and connections between layers. It is critical to use reduced-order models for this purpose, which will identify the regions of influence and the appropriate number of variables. Furthermore, due to the lack of available observed fine data, the reduced-order model can provide us sufficient inexpensive data as needed. The designed deep neural network will be trained using both coarse simulation data which is obtained from the reduced-order model and observed fine data. We will present the main ingredients of our approach and numerical examples. Numerical results show that using deep learning with data generated from multiscale models as well as available observed fine data, we can obtain an improved forward map which can better approximate the fine scale model.

97 MATHEMATICS AND COMPUTING↗

Mesoscale informed parameter estimation through machine learning: A case-study in fracture modeling

Scale bridging is a critical need in computational sciences, where the modeling community has developed accurate physics models from first principles, of processes at lower length and time scales that influence the behavior at the higher scales of interest. However, it is not computationally feasible to incorporate all of the lower length scale physics directly into upscaled models. This is an area where machine learning has shown promise in building emulators of the lower length scale models, which incur a mere fraction of the computational cost of the original higher fidelity models. We demonstrate the use of machine learning using an example in materials science estimating continuum scale parameters by emulating, with uncertainties, complicated mesoscale physics. Additionally, we describe a new framework to emulate the fine scale physics, especially in the presence of microstructures, using machine learning, and showcase its usefulness by providing an example from modeling fracture propagation. Our approach can be thought of as a data-driven dimension reduction technique that yields probabilistic emulators. Our results show well-calibrated predictions for the quantities of interests in a low-strain simulation of fracture propagation at the mesoscale level. Furthermore, on average, we achieve ~10% relative errors on time-varying quantities like total damage and maximum stresses. Successfully replicating mesoscale scale physics within the continuum models is a crucial step towards predictive capability in multi-scale problems.

36 MATERIALS SCIENCE↗

Leveraging interpolation models and error bounds for verifiable scientific machine learning

Effective verification and validation techniques for modern scientific machine learning workflows are challenging to devise. Statistical methods are abundant and easily deployed, but often rely on speculative assumptions about the data and methods involved. Error bounds for classical interpolation techniques can provide mathematically rigorous estimates of accuracy, but often are difficult or impractical to determine computationally. Here, in this work, we present a best-of-both-worlds approach to verifiable scientific machine learning by demonstrating that (1) multiple standard interpolation techniques have informative error bounds that can be computed or estimated efficiently; (2) comparative performance among distinct interpolants can aid in validation goals; (3) deploying interpolation methods on latent spaces generated by deep learning techniques enables some interpretability for black-box models. We present a detailed case study of our approach for predicting lift-drag ratios from airfoil images. Code developed for this work is available in a public Github repository.

97 MATHEMATICS AND COMPUTING↗

What do physics-informed DeepONets learn? Understanding and improving training for scientific computing applications

Physics-informed deep operator networks (DeepONets) have emerged as a promising approach toward numerically approximating the solution of partial differential equations (PDEs). In this work, we aim to develop further understanding of what is being learned by physics-informed DeepONets by assessing the universality of the extracted basis functions and demonstrating their potential toward model reduction with spectral methods. Results provide clarity about measuring the performance of a physics-informed DeepONet through the decays of singular values and expansion coefficients. In addition, we propose a transfer learning approach for improving training for physics-informed DeepONets between parameters of the same PDE as well as across different, but related, PDEs where these models struggle to train well. This approach results in significant error reduction and learned basis functions that are more effective in representing the solution of a PDE.

Deep operator networks↗

A hybrid Penman-Monteith and machine learning model for simulating evapotranspiration and its components

Integrating physical processes with machine learning has advanced evapotranspiration (ET) simulation, yet most hybrid models fail to partition total ET into its components: soil evaporation (E) and vegetation transpiration (T). This study introduces Residual Neural Network–Penman–Monteith (RNN-PM), a novel hybrid dual-source ET model designed to overcome this limitation. The model synergizes the physically-based Penman–Monteith framework with three specialized residual neural networks trained to estimate key conductance parameters (canopy conductance, soil surface conductance, and aerodynamic conductance). Furthermore this explicit parameterization allows for the direct partitioning of total ET. Validation at National Ecological Observatory Network (NEON) flux sites using high-frequency partitioned E and T shows that RNN-PM reliably reproduces ET and the transpiration fraction (T/ET). For ET, the model achieves an average Kling–Gupta efficiency (KGE) of 0.89 and a root-mean-square error (RMSE) of 0.55 mm/day; for T/ET, the KGE is 0.87 with an RMSE of 0.06. Furthermore, RNN-PM demonstrates robust generalization, accurately simulating ET and its components well beyond the initial training dataset, even under extreme climatic conditions. This study extended the analysis by comparing the RNN-PM model with seven established dual-source ET models. The results indicate that RNN-PM outperforms both conventional machine learning models and purely physical process-based models in simulating ET components in most cases. Among the purely physical process-based dual-source models, those based on surface temperature decomposition showed improved performance as the leaf area index (LAI) decreased when evaluated against high-frequency ET component datasets. In contrast, the performance of conductance-based dual-source models declined with decreasing LAI. Although purely machine learning-based models can produce relatively accurate simulations of ET components, they often exhibit limited generalization capability, an issue that the RNN-PM model effectively overcomes. Ultimately, the RNN-PM model represents a significant advance in simulating ET components, offering a novel and scalable approach for improving the representation of land–atmosphere interactions in Earth system models.

54 ENVIRONMENTAL SCIENCES↗

Stability boundary and optimal operating parameter identification in milling using Bayesian learning

Here, this paper describes a novel Bayesian learning approach for stability boundary and optimal parameter identification in milling without the knowledge of the underlying tool dynamics or material cutting force coefficients. The paper is divided into two parts. First, a Bayesian learning method for stability lobe identification using test results is described. Each axial depth and spindle speed combination is characterized by a probability of stability which is updated using Bayes’ rule when a test result (stable or unstable) is made available. A novel likelihood function is defined which incorporates knowledge of the stability behavior. Numerical results show convergence to the analytical stability lobe diagram. Second, an adaptive experimental strategy to identify optimal operating parameters that maximize material removal rate is described. Numerical evaluation shows convergence to the optimal operating point with error less than 15 % within ten tests on average. The approach is validated using experimental results. Results show that the proposed method is an efficient and robust learning method to identify the stability lobe diagram and optimal operating parameters with a limited number of tests/data points.

Bayesian machine learning↗

Generalizability analysis of tool condition monitoring ensemble machine learning models

Tool condition monitoring (TCM) is an essential research area for the optimization and automation of metal machining processes, and could help manufacturers reduce costs, production time, machine downtime, energy use, and part scrappage. However, TCM systems developed in prior studies have struggled to reach the high level of generalizability which is necessary for industrial applications. This study addresses TCM system generalizability to new machining conditions, how variations in machining and environmental conditions may be used to improve model generalizability, and ensemble machine learning techniques for TCM. Further, milling tool life experiments were conducted using various machining conditions, and the processes' sound, spindle power, and axial load signals were collected. Different machine learning models were evaluated for the prediction of tool wear levels, including four individual models and five ensemble models. Changes in cutting speed were found to display a large effect on model performance, while the chip load showed some effect, and the feed rate had little effect. A simulated noise data augmentation technique for model improvement is applied within TCM for the first time, and resulted in increased model generalizability and reduced overfitting. Across several performance metrics the extremely randomized trees ensemble machine learning model generally performed the best for this application, achieving a leave-one-group-out cross validation accuracy score of 92.4 %, a 10-fold cross validation score of 98.9 %, and an averaged accuracy across 11 generalizability tests of 87.3 %.

42 ENGINEERING↗

Machine learning-based real-time monitoring system for smart connected worker to improve energy efficiency

Recent advances in machine learning and computer vision brought to light technologies and algorithms that serve as new opportunities for creating intelligent and efficient manufacturing systems. In this study, the real-time monitoring system of manufacturing workflow for the Smart Connected Worker (SCW) is developed for the small and medium-sized manufacturers (SMMs), which integrates state-of-the-art machine learning techniques with the workplace scenarios of advanced manufacturing systems. Specifically, object detection and text recognition models are investigated and adopted to ameliorate the labor-intensive machine state monitoring process, while artificial neural networks are introduced to enable real-time energy disaggregation for further optimization. The developed system achieved efficient supervision and accurate information analysis in real-time for prolonged working conditions, which could effectively reduce the cost related to human labor, as well as provide an affordable solution for SMMs. The competent experiment results also demonstrated the feasibility and effectiveness of integrating machine learning technologies into the realm of advanced manufacturing systems.

42 ENGINEERING↗

Battery state-of-health diagnostics during fast cycling using physics-informed deep-learning

Rapid, in-situ Li-ion battery state-of-health (SOH) quantification is challenging. Li-ion battery aging can vary significantly with chemistry, operating conditions, cycling demands, electrode design, and operation history. As a cell ages, optimal and safe operating conditions need to be adapted to account for battery degradation by tracking critical aging modes such as loss-of-lithium-inventory (LLI), loss-of-active-material (LAM) in either electrode, and/or impedance rise. This manuscript describes a framework for identifying battery aging modes in-operando using fast-rate voltage charge/discharge responses. The framework uses a physically based Li-ion battery model to produce synthetic high-rate responses at aged states. The aging model is calibrated against experimental data from cells with different electrode loadings and cycled under a variety of fast-charging conditions (1 h, 15 min, 10 min, and 7 min charging). The synthetically generated high-rate responses at aged states are then used to train a deep-learning model to identify real cell state-of-health from fast charge/discharge battery voltage responses. The synthetically trained deep-learning model performance is validated by comparing to standard incremental capacity analysis and half-cell measurements. Finally, the framework demonstrates the benefits of using high-rate physics-based models to generate synthetic data for training deep-learning models.

25 ENERGY STORAGE↗

Dynamic machine learning-based optimization algorithm to improve boiler efficiency

With decreasing computational costs, improvement in algorithms, and the aggregation of large industrial and commercial datasets, machine learning is becoming a ubiquitous tool for process and business innovations. Machine learning is still lacking applications in the field of dynamic optimization for real-time control. This work presents a novel framework for performing constrained dynamic optimization using a recurrent neural network model combined with a metaheuristic optimizer. The framework is designed to augment an existing control system and is purely data-driven, like most industrial Model Predictive Control applications. Several recurrent neural network models are compared as well as several metaheuristic optimizers. Hyperparameters and optimizer parameters are tuned with parameter sweeps, and the resulting values are reported. Further, the best parameters for each optimizer and model combination are demonstrated in closed-loop control of a dynamic simulation, and several recommendations are made for generalizing this framework to other systems. Up to 0.953% improvement is realized over the non-optimized case for a simulated coal-fired boiler. While this is not a large improvement in percentage, the total economic impact is $991,000 per year, and this study builds a foundation for future machine learning with dynamic optimization.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Machine learning-enabled multiscale modeling of mechanical deformation of aluminum and Al-SiC nanocomposites

A machine learning-enabled multiscale framework is developed for modeling the mechanical response of both pure metal and nanoparticle-reinforced metal matrix nanocomposites (MMNCs). Using aluminum–silicon carbide (Al-SiC) as an example MMNC, atomistic simulations reveal three distinct deformation mechanisms (i.e., defect-free, dislocation-based, and interface separation) governed by the interfaces between the Al matrix and SiC nanoparticles. As compared with single crystal Al, the lattice undergoes a more abrupt failure once the dislocation network becomes extensive and void nucleation initiates, whereas in Al-SiC, nanoparticle interfaces enable a more gradual progression of damage. These mechanisms are captured through a combined classification-regression neural network surrogate model that bridges atomic-scale insights with continuum-scale finite element analysis. Machine learning-enabled multiscale modeling of pure Al accurately predicted strain localization and confirmed by in-situ scanning electron microscopic tensile testing on perforated Al specimens. This study underscores the promise of integrating physics-informed machine learning with hierarchical modeling to capture the interface dominated phenomena and guide the design of advanced MMNCs.

Al-SiC↗

Fast-RF-Shimming: Accelerate RF shimming in 7T MRI using deep learning

Ultrahigh field (UHF) Magnetic Resonance Imaging (MRI) offers an elevated signal-to-noise ratio (SNR), enabling exceptionally high spatial resolution that benefits both clinical diagnostics and advanced research. However, the jump to higher fields introduces complications, particularly transmit radiofrequency (RF) field ($B^{+}_{1}$) inhomogeneities, manifesting as uneven flip angles and image intensity irregularities. These artifacts can degrade image quality and impede broader clinical adoption. Traditional RF shimming methods, such as Magnitude Least Squares (MLS) optimization, effectively mitigate $B^{+}_{1}$ inhomogeneity, but remain time-consuming. Recent machine learning approaches, including RF Shim Prediction by Iteratively Projected Ridge Regression and other deep learning architectures, suggest alternative pathways. Although these approaches show promise, challenges such as extensive training periods, limited network complexity, and practical data requirements persist. In this paper, we introduce a holistic learning-based framework called Fast-RF-Shimming, which achieves a 5000 ​× ​speed-up compared to the traditional MLS method. In the initial phase, we employ random-initialized Adaptive Moment Estimation (Adam) to derive the desired reference shimming weights from multi-channel $B^{+}_{1}$ fields. Next, we train a Residual Network (ResNet) to map $B^{+}_{1}$ fields directly to the ultimate RF shimming outputs, incorporating the confidence parameter into its loss function. Finally, we design Non-uniformity Field Detector (NFD), an optional post-processing step, to ensure the extreme non-uniform outcomes are identified. Comparative evaluations with standard MLS optimization underscore notable gains in both processing speed and predictive accuracy, which indicates that our technique shows a promising solution for addressing persistent inhomogeneity challenges.

Deep learning↗

A comparison of machine learning surrogate models of street-scale flooding in Norfolk, Virginia

Low-lying coastal cities, exemplified by Norfolk, Virginia, face the challenge of street flooding caused by rainfall and tides, which strain transportation and sewer systems and can lead to personal and property damage. While high-fidelity, physics-based simulations provide accurate predictions of urban pluvial flooding, their computational complexity renders them unsuitable for real-time applications. Using data from Norfolk rainfall events between 2016 and 2018, this study compares the performance of a previous surrogate model based on a random forest algorithm with two deep learning models: Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The comparison of deep learning to the random forest algorithm is motivated by the desire to utilize a machine learning architecture that allows for the future inclusion of common uncertainty quantification techniques and the effective integration of relevant, multi-modal features.

54 ENVIRONMENTAL SCIENCES↗

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various applications. Such temporal graphs exhibit heterogeneous transient dynamics, varying time intervals, and highly evolving node features throughout their evolution. Hence, incorporating long-range dependencies from the historical graph context plays a crucial role in accurately learning their temporal dynamics. In this paper, we develop a graph embedding model with uncertainty quantification, TransformerG2G, by exploiting the advanced transformer encoder to first learn intermediate node representations from its current state (t) and previous context (over timestamps [t–1,t–l], l is the length of context). Moreover, we employ two projection layers to generate lower-dimensional multivariate Gaussian distributions as each node's latent embedding at timestamp t. We consider diverse benchmarks with varying levels of "novelty" as measured by the TEA (Temporal Edge Appearance) plots. Here, our experiments demonstrate that the proposed TransformerG2G model outperforms conventional multi-step methods and our prior work (DynG2G) in terms of both link prediction accuracy and computational efficiency, especially for high degree of novelty. Furthermore, the learned time-dependent attention weights across multiple graph snapshots reveal the development of an automatic adaptive time stepping enabled by the transformer. Importantly, by examining the attention weights, we can uncover temporal dependencies, identify influential elements, and gain insights into the complex interactions within the graph structure. For example, we identified a strong correlation between attention weights and node degree at the various stages of the graph topology evolution.

97 MATHEMATICS AND COMPUTING↗

A machine learning approach for particle accelerator errant beam prediction using spatial phase deviation

Particle accelerators are extremely complex systems that are expected to operate on high availability. Predicting impending failures only by utilizing data collected from diagnostic equipment already on board can help operators to avoid installing expensive sensors, unscheduled downtime and associated costs. For this purpose we explore the predictive power of Machine Learning algorithms to detect faulty beams prior to the failure. In this study, we propose a Machine Learning approach to model mapping from a pair of sensors located across the accelerator. While the model is trained to represent normal operation, we evaluate the predictive performance on known faulty beam pulses. We also investigate the model performance on unseen data through k-fold cross-validation. Then we recap the analysis with a neural architecture search and hyperparameter optimization study to fine tune our initial model. In conclusion, this paper will also introduce a sustainable framework that can standardize Machine Learning workflow applied to particle accelerators.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Expanded analysis of machine learning models for nuclear transient identification using TPOT

Industries around the world are becoming more and more data driven. The nuclear field is no exception with several different applications being proposed. One popular area of research is the use of machine learning in transient detection. This paper seeks to build upon a previous study which made use of the AutoML package TPOT to train traditional machine learning models to classify transient events occurring with a reactor. Synthetic data was once again collected using a GPWR reactor simulator. Data on 12 different events was collected using 15 different initial conditions. Here, a dataset consisting of over 100,000 data points was compiled and used to train 7 different machine learning models using a pre-defined TPOT dictionary with 12 different preprocessing techniques. Three of the trained models were able to produce validation results in the 90s with the expanded dataset. Once the models were trained, it was possible to look into where during the simulation, misclassifications occurred. Using these three models, analysis was done to determine if TPOT could be used to train models that were effective if important features were missing. The results from this were positive with the newly trained models scoring close to the original models. Finally, to conclude this study, the three high performing models were retrained using different random states to see if there was any major variation when different states were used.

42 ENGINEERING↗

Predicting the superconducting critical temperature in transition metal carbides and nitrides using machine learning

Transition metal carbides and nitrides have unique mechanical and chemical characteristics. At low temperatures many of them also exhibit superconductivity, which can be controlled by substitutions into both the transition metal and carbon/nitrogen sites. To investigate the factors governing the superconducting state, we apply machine learning methods. Here we collected a dataset containing 147 materials, which was used to create a pipeline for predicting their superconducting critical temperature. When this pipeline is applied to a randomly selected test set, it shows a good performance, with of 0.82 and RMSE of 1.9 K. To explore the limits of the machine learning approach, we also use it to predict entire substitution series within the dataset. This represents a realistic test for the predictive models, which can be extremely useful when applied to new substitutions in materials systems. The performance of the pipeline in this case is much more uneven, with good predictions for some series, while for others the model shows minimal predictive power. We discuss possible reasons for these results, as well as methods to estimate the performance of machine learning on new substitution series.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Time-series learning of latent-space dynamics for reduced-order model closure

In this work, we study the performance of long short-term memory networks (LSTMs) and neural ordinary differential equations (NODEs) in learning latent-space representations of dynamical equations for an advection-dominated problem given by the viscous Burgers equation. Our formulation is devised in a nonintrusive manner with an equation-free evolution of dynamics in a reduced space with the latter being obtained through a proper orthogonal decomposition. In addition, we leverage the sequential nature of learning for both LSTMs and NODEs to demonstrate their capability for closure in systems that are not completely resolved in the reduced space. We assess our hypothesis for two advection-dominated problems given by the viscous Burgers equation. We observe that both LSTMs and NODEs are able to reproduce the effects of the absent scales for our test cases more effectively than does intrusive dynamics evolution through a Galerkin projection. This result empirically suggests that time-series learning techniques implicitly leverage a memory kernel for coarse-grained system closure as is suggested through the Mori–Zwanzig formalism.

97 MATHEMATICS AND COMPUTING↗