Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Leveraging transfer learning and leaf spectroscopy for leaf trait prediction with broad spatial, species, and temporal applicability

Accurate and reliable prediction of leaf traits is crucial for understanding plant adaptations to environmental variation, monitoring terrestrial ecosystems, and enhancing comprehension of functional diversity and ecosystem functioning. Currently, various approaches (e.g., statistical, physical models) have been developed to estimate leaf traits through hyperspectral remote sensing and leaf spectroscopy. However, the absence of high-performing, transferable, and stable models across various domains of space, plant functional types (PFTs) and seasons hinder our ability to quantify and comprehend spatiotemporal variations in leaf traits. This study proposes robust and highly transferable models for better predicting leaf traits with hyperspectral reflectance. Initially, three datasets were assembled, pairing common leaf traits — chlorophyll (Chla+b), carotenoids (Ccar), leaf mass per area (LAM), equivalent water thickness (EWT) — with leaf spectra measurements collected across diverse geographic locations in the U.S. and Europe, PFTs, and seasons. Measurements were acquired using spectroradiometers (e.g., ASD FieldSpec 3/4/Pro and SVC HR-1024i) with integrating spheres, leaf clips, and contact probes. Here, we then developed transfer learning-based hybrid models that incorporated the domain knowledge of radiative transfer models (RTMs) through pretraining processes and were well-constrained by fine-tuning with field measurements. Through comparison with other state-of-the-art statistical models, including partial-least squares regression (PLSR) and Gaussian Process Regression (GPR), as well as pure physical models, we found that the proposed transfer learning models achieved better predictive performance and higher transferability. Specifically, compared to other statistical models and pure RTMs, the transfer learning model exhibited higher coefficient of determination (R 2 ) values with range of 0.01 to 0.79, lower normalized root mean square error (NRMSE) with range of 0.06 % to 33.25 % in model performance. Additionally, the models exhibited improved transferability, with higher R 2 values range from 0.04 to 0.32, lower NRMSE range from 0.08 % to 30.81 %. The findings underscore that transfer learning models through integrating domain knowledge from RTMs and limited observations, can harness the advantages of both RTMs and statistical models and serve as a promising approach for effectively predicting leaf traits.

59 BASIC BIOLOGICAL SCIENCES↗

Tracking atomic structure evolution during directed electron beam induced Si-atom motion in graphene via deep machine learning

Using electron beam manipulation, we enable deterministic motion of individual Si atoms in graphene along predefined trajectories. Structural evolution during the dopant motion was explored, providing information on changes of the Si atom neighborhood during atomic motion and providing statistical information of possible defect configurations. The combination of a Gaussian mixture model and principal component analysis applied to the deep learning-processed experimental data allowed disentangling of the atomic distortions for two different graphene sublattices. This approach demonstrates the potential of e-beam manipulation to create defect libraries of multiple realizations of the same defect and explore the potential of symmetry breaking physics. The rapid image analytics enabled via a deep learning network further empowers instrumentation for e-beam controlled atom-by-atom fabrication. Here, the analysis described in the paper can be reproduced via an interactive Jupyter notebook at https://git.io/JJ3Bx.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Domain Adaptive Graph Neural Networks for Constraining Cosmological Parameters Across Multiple Data Sets

State of the art astronomical simulations have provided datasets which enabled the training of novel deep learning techniques for constraining cosmological parameters. However, differences in subgrid physics implementation and numerical approximations among simulation suites lead to differences in simulated datasets, which pose a hard challenge when trying to generalize across diverse data domains and ultimately when applying models to observational data. Recent work reveals deep learning algorithms are able to extract more information from complex cosmological simulations than summary statistics like power spectra. We introduce Domain Adaptive Graph Neural Networks (DA-GNNs), trained on CAMELS data, inspired by CosmoGraphNet (Villanueva-Domingo et al 2023). By utilizing GNNs, we can capitalize on their capacity to capture both astrophysical and topological features of galaxy distributions. Mixing these capabilities with domain adaptation techniques such as Maximum Mean Discrepancy (MMD), which enable extraction of domain-invariant features, our framework demonstrates enhanced accuracy and robustness. We present experimental results, including the alignment of distributions across domains through data visualization. These findings suggest that DA-GNNs are an efficient way of extracting domain independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.

79 ASTRONOMY AND ASTROPHYSICS↗

U-net architected deep material network training with microstructure local field information

The Deep Material Network (DMN) has recently emerged as a powerful reduced-order modeling framework for simulating the mechanical response of heterogeneous materials such as composites. Unlike most data-driven approaches that directly learn a material’s response under prescribed loading, the DMN acts as a homogenization operator, learning the kinematic constraints and mechanical interactions of the underlying microstructure. However, traditional DMN training relies exclusively on homogenized effective properties derived from Direct Numerical Simulations (DNS), discarding the rich local field data that govern microstructural interactions. In this work, we extend the DMN framework to incorporate such local field information into the offline training process. Utilizing a U-Net architecture, we augment the DMN training objective to include the first and second statistical moments of the local stress fields obtained from linear DNS. This ensures that the learned network topology not only fits the effective stiffness but also accurately reflects the internal local stress and strain partitioning of the microstructure. The results confirm that supervising the localization process during training yields a superior surrogate model, reducing local prediction errors by an order of magnitude and significantly improving generalization to unseen nonlinear constitutive behaviors compared to traditional DMNs.

36 MATERIALS SCIENCE↗

Data-Driven Learning for the Mori--Zwanzig Formalism: A Generalization of the Koopman Learning Framework

A theoretical framework which unifies the conventional Mori--Zwanzig formalism and the approximate Koopman learning of deterministic dynamical systems from noiseless observation is presented. In this framework, the Mori--Zwanzig formalism, developed in statistical mechanics to tackle the hard problem of construction of reduced-order dynamics for high-dimensional dynamical systems, can be considered as a natural generalization of the Koopman description of the dynamical system. We next show that, similar to the approximate Koopman learning methods, data-driven methods can be developed for the Mori--Zwanzig formalism with Mori's linear projection operator. We have developed two algorithms to extract the key operators, the Markov and the memory kernel, using time series of a reduced set of observables in a dynamical system. We have adopted the Lorenz `96 system as a test problem and solved for the above operators. These operators exhibit complex behaviors, which are unlikely to be captured by traditional modeling approaches in Mori--Zwanzig analysis. The nontrivial generalized fluctuation-dissipation relationship, which relates the memory kernel with the two-time correlation statistics of the orthogonal dynamics, was numerically verified as a validation of the solved operators. Here we present numerical evidence that the generalized Langevin equation, a key construct in the Mori--Zwanzig formalism, is more advantageous in predicting the evolution of the reduced set of observables than the conventional approximate Koopman operators.

97 MATHEMATICS AND COMPUTING↗

Generalization error guaranteed auto-encoder-based nonlinear model reduction for operator learning

Many physical processes in science and engineering are naturally represented by operators between infinite-dimensional function spaces. The problem of operator learning, in this context, seeks to extract these physical processes from empirical data, which is challenging due to the infinite or high dimensionality of data. An integral component in addressing this challenge is model reduction, which reduces both the data dimensionality and problem size. In this paper, we utilize low-dimensional nonlinear structures in model reduction by investigating Auto-Encoder-based Neural Network (AENet). AENet first learns the latent variables of the input data and then learns the transformation from these latent variables to corresponding output data. Our numerical experiments validate the ability of AENet to accurately learn the solution operator of nonlinear partial differential equations. Furthermore, we establish a mathematical and statistical estimation theory that analyzes the generalization error of AENet. Finally, our theoretical framework shows that the sample complexity of training AENet is intricately tied to the intrinsic dimension of the modeled process, while also demonstrating the robustness of AENet to noise.

Auto-encoder↗

Integrated machine learning-molecular dynamics framework for electrolyte property prediction

Electrochemical stability windows determine the operating range of battery electrolytes, yet accurate prediction remains challenging because stability emerges from statistical ensembles of local solvation environments rather than single ground-state molecular structures. Traditional density functional theory calculations on energy-minimized clusters cannot capture the thermal variations in local coordination environments and geometries that govern decomposition, while SMILES-based machine learning methods lack explicit representation of three-dimensional solvation structure and ion pairing. Here, we introduce a structure-aware machine learning framework that predicts frontier orbital energies (HOMO and LUMO) directly from molecular dynamics-sampled solvation configurations, achieving sub-0.6 eV accuracy at computational costs 3–4 orders of magnitude lower than first-principles methods. Across twelve representative battery electrolytes, we demonstrate that solvent-separated and contact ion pairs exhibit strong size- and local chemistry dependent electronic stability, with variations in coordination shifts of HOMO or LUMO level by 2–3 eV, and that extended solvation structure and partially desolvated environment further modulate stability by up to 3 eV. By encoding the statistical nature of electrochemical failure through ensemble sampling of explicit solvation geometries, our approach enables high-throughput screening and rational design of next-generation battery electrolytes with mechanistic understanding of structure–property relationships.

Energy - Storage↗

Learning to Predict Crystal Plasticity at the Nanoscale: Deep Residual Networks and Size Effects in Uniaxial Compression Discrete Dislocation Simulations

The density and configurational changes of crystal dislocations during plastic deformation influence the mechanical properties of materials. These influences have become clearest in nanoscale experiments, in terms of strength, hardness and work hardening size effects in small volumes. The mechanical characterization of a model crystal may be cast as an inverse problem of deducing the defect population characteristics (density, correlations) in small volumes from the mechanical behavior. In this work, we demonstrate how a deep residual network can be used to deduce the dislocation characteristics of a sample of interest using only its surface strain profiles at small deformations, and then statistically predict the mechanical response of size-affected samples at larger deformations. As a testbed of our approach, we utilize high-throughput discrete dislocation simulations for systems of widths that range from nano- to micro- meters. We show that the proposed deep learning model significantly outperforms a traditional machine learning model, as well as accurately produces statistical predictions of the size effects in samples of various widths. By visualizing the filters in convolutional layers and saliency maps, we find that the proposed model is able to learn the significant features of sample strain profiles.

36 MATERIALS SCIENCE↗

Exploring Advanced Computational Tools and Techniques with Artificial Intelligence and Machine Learning in Operating Nuclear Plants

This report presents the project Idaho National Laboratory conducted for Nuclear Regulatory Commission to explore the advanced computational tools and techniques, such as artificial intelligence (AI) and machine learning (ML), for operating nuclear plants. The report reviews the nuclear data sources, with the focus on the operating experience data, that could be applied by advanced computational tools and techniques. Plant-specific and generic (national and international) data from different sources are described. The report describes the relationships between statistics and AI/ML and then introduces the most widely used AI/ML algorithms in both supervised and unsupervised learning. The report reviews the recent applications of advanced computational tools and techniques in various fields of nuclear industry, such as reactor system design and analysis, plant operation and maintenance, and nuclear safety and risk analysis. Finally, the report presents the insights from the project on the potential applicability of AI/ML techniques in improving advanced computational capabilities, how the advanced tools and techniques could contribute to the understanding of safety and risk, and what information would be needed to provide meaningful insights to decision makers. The report also documents an NRC survey on the current state of commercial nuclear power operations relative to the use of AI and ML tools as well as the role of AI/ML tools in nuclear power operations was published by the NRC as in FRN NRC-2021-0048 in April 2021. A summary of the survey including the survey questions, survey participants, survey responses, and the conclusions and insights derived from the survey is provided in the report. Finally, the report investigates potential applications of using AI/ML in operating NPPs and advanced reactors (both advanced LWRs and advanced NLWRs) to improve nuclear plant safety and efficiency. Three main application fields are defined and discussed: (1) plant safety and security assessments; (2) plant degradation modeling, fault and accident diagnosis and prognosis; and (3) plant operation and maintenance efficiency improvement.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Thermochemical Data Fusion Using Graph Representation Learning

Large databases are required for “Big Data” applications in catalysis and materials science. Thermochemical databases can be created by combining data from various sources and by correcting low-fidelity datasets to higher accuracy with minimal computation. To achieve this “data fusion”, thermochemical quantities of interest, calculated at various levels of density functional theory (DFT), need to be mapped to the same, high levels of theory. In this work, a graph theoretical, statistical framework is proposed for such tasks. Subgraph frequencies are shown to provide a natural representation for learning these fusion maps. The maps are linear and are learnt with automated descriptor selection. Using a dataset of as few as ~1% from the QM9 database of 133,885 molecules, these models can predict multiple thermochemical quantities at a higher level of theory with an accuracy of 1 kcal/mol. Here, the method is explainable, generalizable, and provides a diagnostic tool for outlier identification

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Bayesian Physics Informed Spatio-Temporal Network for Streamflow Data Imputation

Reliable reconstruction of incomplete streamflow records is critical for improving hydrological forecasting, flood preparedness, and water resource management. However, large observational gaps and uncertainties in governing physical parameters limit the accuracy of traditional statistical and machinelearning imputation frameworks. To address these challenges, we develop a Bayesian Physics-Informed Spatio-Temporal Network (BPI-STNet) that jointly captures spatial and temporal dependencies while enforcing hydrologic consistency through embedded physical constraints. The framework integrates a GraphSAGE-LSTM architecture to model spatial connectivity across gauges and temporal flow dynamics, coupled with a Bayesian update mechanism to estimate uncertain parameters in a simplified water-balance framework. Unlike conventional physics-informed networks that rely on sampling-based posterior estimation, BPI-STNet derives an analytic solution to the inverse problem, allowing closed-form Bayesian updates of uncertain parameters Λ={α,β,k} using Gaussian priors and likelihoods. Applied to daily observations from the Susquehanna River Basin (1980-2022), BPI-STNet achieves substantial improvements over a purely data-driven RGNN baseline, which reduced RMSE by 23 % and MAE by 9 %, and achieving an average NSE values up to 0.96. The results demonstrate that coupling Bayesian inference with physics-informed learning yields physically consistent, uncertainty-aware reconstructions that preserve the temporal persistence and statistical distribution of observed flows. The proposed framework establishes a generalizable paradigm for data-sparse hydrologic systems where both data fidelity and physical interpretability are essential.

Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗

Tractable Learning in Underexcited Power Grids

Estimating the structure of physical flow networks, such as power grids, is critical to secure delivery of energy. This article discusses statistical structure estimation in power grids in the “underexcited” regime, where a subset of internal nodes has zero injection fluctuations. Prior estimation algorithms based on nodal voltages fail for such grids as the voltage covariance matrix is not invertible. Here, we propose a novel topology learning algorithm for learning underexcited general networks. Our algorithm uses physics-informed conservation laws to first identify the zero-injection buses and their neighbors, and then estimates the remaining edges in the grid. We prove the asymptotic correctness of our algorithm for grids with nonadjacent internal zero-injection nodes. More important, we theoretically analyze our algorithm’s efficacy under noisy measurements, and determine bounds on maximum noise under which asymptotically correct recovery is guaranteed. Our approach is validated through simulations with voltage samples generated on test distribution grids with real injection data and nonlinear power flow models.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Process prediction and detection of faults using probabilistic bidirectional recurrent neural networks on real plant data

Attaining Industry 4.0 for manufacturing operations requires advanced monitoring systems and real-time data analytics of plant data, among other topics. We propose a Probabilistic Bidirectional Recurrent Network (PBRN) for industrial process monitoring for the early detection of faults. The model is based on a Gated Recurrent Unit (GRU) neural network that allows the model to retain long-term dependencies between sensor data along a time horizon, hence learning the dynamic behavior of the process. To reduce the false-positive detection rate of the model, we compel the model to learn from a highly noisy sensor reading while outputting noise-free sensor outputs. The performance of the proposed model is compared to other data-driven statistical process monitoring schemes using real plant data from an industrial Air Separations Unit (ASU) containing noisy sensor readings. We show that the model can learn from noisy data without reducing its performance. Using two different fault cases, we demonstrate the model’s ability to carry out early fault detection with average false-positive rates of 2.9% and 4.9% for both fault cases. The missed detection rates are 0.1% and 0.2%, respectively.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Uncertainty Quantification for Dissimilar Material Joints Under Corrosion Environment

Abstract Self-Piercing Riveting (SPR) is one of the most commonly used methods for joining dissimilar materials in the automotive industry. These joints are popular due to their adaptability, high performance and short cycle time. However, since these joints involve two dissimilar materials, they are susceptible to galvanic corrosion in the presence of an electrolyte which is driven by the difference in the equilibrium potential of the metals. This can affect the safety and resilience of these joints. In this paper, we focus on galvanic corrosion in Al-Fe SPR joints. A Machine learning (ML) based surrogate model, which is based off of FE simulations, for statistical corrosion analysis is developed. This model enables the resilience and reliability analysis of SPR joints under corrosion environment. In this study, first a physics-based finite element (FE) corrosion model has been developed to simulate the galvanic corrosion between a Fe cathode and an Al anode of a SPR joint. This model takes into account the effect of the crystal microstructure of the Al anode and the precipitation of the corrosion product. Several geometric and environmental factors including crevice gap, roughness of anode, conductivity, pH and the temperature of the electrolyte that effect corrosion are investigated. A thorough Uncertainty Quantification (UQ) analysis is conducted for the overall corrosion behavior of the Fe-Al SPR joints using a novelistic Probabilistic Confidence-Based Adaptive Sampling (PCAS) technique. PCAS is used to train the surrogate model by identifying the critical sampling points and thus reducing the overall computational costs. It is found that the electrolyte temperature has the largest effects on the material loss and needs to be managed closely for better corrosion control. By understanding the corrosion performance and resultant uncertainty impact on SPR joints, the reliability and resilience of these joints can be improved.

36 MATERIALS SCIENCE↗

Optimal dimensionality selection for independent component analysis of transcriptomic data

Independent component analysis is an unsupervised machine learning algorithm that separates a set of mixed signals into a set of statistically independent source signals. Applied to high-quality gene expression datasets, independent component analysis effectively reveals both the source signals of the transcriptome as co-regulated gene sets, and the activity levels of the underlying regulators across diverse experimental conditions. Two major variables that affect the final gene sets are the diversity of the expression profiles contained in the underlying data, and the user-defined number of independent components, or dimensionality, to compute. Availability of high-quality transcriptomic datasets has grown exponentially as high-throughput technologies have advanced; however, optimal dimensionality selection remains an open question. We computed independent components across a range of dimensionalities for four gene expression datasets with varying dimensions (both in terms of number of genes and number of samples). We computed the correlation between independent components across different dimensionalities to understand how the overall structure evolves as the number of user-defined components increases. We then measured how well the resulting gene clusters reflected known regulatory mechanisms, and developed a set of metrics to assess the accuracy of the decomposition at a given dimension. We found that over-decomposition results in many independent components dominated by a single gene, whereas under-decomposition results in independent components that poorly capture the known regulatory structure. From these results, we developed a new method, called OptICA, for finding the optimal dimensionality that controls for both over- and under-decomposition. Specifically, OptICA selects the highest dimension that produces a low number of components that are dominated by a single gene. We show that OptICA outperforms two previously proposed methods for selecting the number of independent components across four transcriptomic databases of varying sizes. OptICA avoids both over-decomposition and under-decomposition of transcriptomic datasets resulting in the best representation of the organism’s underlying transcriptional regulatory network.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Classification and Fusion of Two Disparate Data Streams and Nuclear Dissolutions Application

We consider two streams of data or measurements with disparate qualities and time resolutions that need to be classified. The first stream consists of higher quality data at a coarser time resolution, and the other consists of lower quality data at a finer time resolution. We present a fuser-switch method that fuses the set of classifiers of each stream separately and switches between them. We show that this method provides classification decisions at a finer time resolution with superior detection and false alarm probabilities compared to individual classifiers, under the statistical independence and time resolution ratio conditions. When classifiers are trained using machine learning methods, we show that this superior performance is guaranteed with a confidence probability specified by the classifiers' generalization equations. We use these results to provide analytical foundations for previous practical results that achieved significant performance improvements in classifying Pu/Np target dissolution events at a radiochemical processing facility.

Rao, Nageswara↗

Expansion of Machine-Learning Method for Classifying Neutron Resonances

The understanding of astrophysics processes and the performance of nuclear reactors and other nuclear systems depend on a precise description of the neutron interaction cross sections for materials and nuclei present in these environments. At low neutron energies, these cross sections exhibit resonance structure represented by sharp enhancements when the neutron energy is sufficiently close to excited levels in a compound nucleus. Such resonances can be characterized by their quantum numbers relative to angular momenta, which are often deduced in an ad hoc and irreproducible manner from the shape of the cross sections. The correct assignment of the quantum numbers of neutron resonances is therefore of paramount importance. To address this we have developed a machine-learning method to automate the identification and correction of these spin assignments. The algorithm is trained from simulated data, generated from statistical properties of resonance data for a given nucleus, to mimic the errors found in real data. In this project we describe five independent approaches to further develop and expand the applicability of the machine-learning spin classifier: i) Feature impact; ii) Integration with the Atlas; iii) Training optimization; iv) Spacings systematics; and v) Validation with polarized data. The premises, methods, results, and future perspectives are discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

FY21 Progress Report: SRNL Analysis of ICCWR LCM and WAMS data for Corrosion and Cracking

The development of algorithms for machine learning and data analysis for the 3013 Surveillance Program is a collaborative effort by the Savannah River National Laboratory (SRNL) and the University of South Carolina (USC). For corrosion detection, Laser Confocal Microscope (LCM) or Wide Area 3D Measurement System (WAMS) data is extracted from large binary files, with software written to convert the data to physical attributes (e.g., height, color and grayscale values; all as functions of a location in a plane projection). A user-friendly Matlab Graphical User Interface (GUI) that reads data from either LCM or WAMS files was developed to integrate input data with software developed for processing and evaluation. The GUI can selectively download binary data, interrogate data attributes, label data, flag significant features, execute Machine Learning (ML) algorithms, output parameters for trained ML algorithms, report ML model accuracy with respect to labeled data, and generate graphical representations for various analyses. Features can be called out by user-specified thresholds, manual labeling or machine learning algorithms when they have been completed. The ability to rapidly label data is important because of the volume of data required for training machine learning algorithms. The GUI has the flexibility to allow addition of improved ML algorithms, methods for data visualization, and statistical computations. Statistical analyses via the GUI include areas of pits within a defined range of pit depths, correlations between Red-Green-Blue (RGB) or grayscale intensity and relative surface height, covariances between values associated with features, and feature histograms. The development of supervised machine learning algorithms, however, has been hindered by a lack of training data. The machine learning algorithms for crack identification are being refined but require improvements to the true positive rate for crack detection. This shortcoming is an artifact of the limited training data currently available, perhaps more so than the structure of the neural networks. At present, the best results are had from a consensus over an ensemble of randomly generated Deep Neural Network (DNN) or Convolutional Neural Network (CNN) algorithms. Although the consensus accuracy method has yielded optimum true positive and true negative rates in excess of 80%, additional validation testing is necessary. In addition to the suite of LCM data that was initially used, and which represents the majority of the work presented in this report, WAMS image data was also reviewed at a preliminary level. The review included a comparison between image resolution and dynamic range for each method. WAMS (ZON file) image data was found to have a pixel pitch of 3.69μm compared to 1 μm for the LCM (vk4 file) data, which implies a lower resolution for the WAMS images. Conversely, the ratio of dynamic range of the WAMS data to the LCM data was approximately 41:20 for height data, suggesting that information from WAMS should more accurately determine the depth of pits. At present, the significance of the greater dynamic range of the WAMS data relative to the LCM data has not yet been evaluated.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗