Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning in Network Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Robust errant beam prognostics with conditional modeling for particle accelerators

Abstract Particle accelerators are complex and comprise thousands of components, with many pieces of equipment running at their peak power. Consequently, they can fault and abort operations for numerous reasons, lowering efficiency and science output. To avoid these faults, we apply anomaly detection techniques to predict unusual behavior and perform preemptive actions to improve the total availability. Supervised machine learning (ML) techniques such as siamese neural network models can outperform the often-used unsupervised or semi-supervised approaches for anomaly detection by leveraging the label information. One of the challenges specific to anomaly detection for particle accelerators is the data’s variability due to accelerator configuration changes within a production run of several months. ML models fail at providing accurate predictions when data changes due to changes in the configuration. To address this challenge, we include the configuration settings into our models and training to improve the results. Beam configurations are used as a conditional input for the model to learn any cross-correlation between the data from different conditions and retain its performance. We employ conditional siamese neural network (CSNN) models and conditional variational auto encoder (CVAE) models to predict errant beam pulses at the spallation neutron source under different system configurations and compare their performance. We demonstrate that CSNNs outperform CVAEs in our application.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Assessments of epistemic uncertainty using Gaussian stochastic weight averaging for fluid-flow regression

Here, we use Gaussian stochastic weight averaging (SWAG) to assess the epistemic uncertainty associated with neural-network-based function approximation relevant to fluid flows. SWAG approximates a posterior Gaussian distribution of each weight, given training data, and a constant learning rate. Having access to this distribution, it is able to create multiple models with various combinations of sampled weights, which can be used to obtain ensemble predictions. The average of such an ensemble can be regarded as the 'mean estimation', whereas its standard deviation can be used to construct 'confidence intervals', which enable us to perform uncertainty quantification (UQ) with regard to the training process of neural networks. We utilize representative neural-network-based function approximation tasks for the following cases: (i) a two-dimensional circular-cylinder wake; (ii) the DayMET dataset (maximum daily temperature in North America); (iii) a three-dimensional square-cylinder wake; and (iv) urban flow, to assess the generalizability of the present idea for a wide range of complex datasets. SWAG-based UQ can be applied regardless of the network architecture, and therefore, we demonstrate the applicability of the method for two types of neural networks: (i) global field reconstruction from sparse sensors by combining convolutional neural network (CNN) and multi-layer perceptron (MLP); and (ii) far-field state estimation from sectional data with two-dimensional CNN. We find that SWAG can obtain physically-interpretable confidence-interval estimates from the perspective of epistemic uncertainty. This capability supports its use for a wide range of problems in science and engineering.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Flexible machine-learning interatomic potential for simulating structural disordering behavior of Li 7 La 3 Zr 2 O 12 solid electrolytes

Batteries based on solid-state electrolytes, including Li 7 La 3 Zr 2 O 12 (LLZO), promise improved safety and increased energy density; however, atomic disorder at grain boundaries and phase boundaries can severely deteriorate their performance. Machine-learning (ML) interatomic potentials offer a uniquely compelling solution for simulating chemical processes, rare events, and phase transitions associated with these complex interfaces by mixing high scalability with quantum-level accuracy, provided that they can be trained to properly address atomic disorder. To this end, we report the construction and validation of an ML potential that is specifically designed to simulate crystalline, disordered, and amorphous LLZO systems across a wide range of conditions. The ML model is based on a neural network algorithm and is trained using ab initio data. Performance tests prove that the developed ML potential can predict accurate structural and vibrational characteristics, elastic properties, and Li diffusivity of LLZO comparable to ab initio simulations. As a demonstration of its applicability to larger systems, we show that the potential can correctly capture grain boundary effects on diffusivity, as well as the thermal transition behavior of LLZO. Here these examples show that the ML potential enables simulations of transitions between well-defined and disordered structures with quantum-level accuracy at speeds thousands of times faster than ab initio methods.

25 ENERGY STORAGE↗
DOE OSTI2020

Deeply uncertain: comparing methods of uncertainty quantification in deep learning algorithms

We present a comparison of methods for uncertainty quantification (UQ) in deep learning algorithms in the context of a simple physical system. Three of the most common uncertainty quantification methods - Bayesian Neural Networks (BNN), Concrete Dropout (CD), and Deep Ensembles (DE) - are compared to the standard analytic error propagation. We discuss this comparison in terms endemic to both machine learning ("epistemic" and "aleatoric") and the physical sciences ("statistical" and "systematic"). The comparisons are presented in terms of simulated experimental measurements of a single pendulum - a prototypical physical system for studying measurement and analysis techniques. Our results highlight some pitfalls that may occur when using these UQ methods. For example, when the variation of noise in the training set is small, all methods predicted the same relative uncertainty independently of the inputs. This issue is particularly hard to avoid in BNN. On the other hand, when the test set contains samples far from the training distribution, we found that no methods sufficiently increased the uncertainties associated to their predictions. This problem was particularly clear for CD. In light of these results, we make some recommendations for usage and interpretation of UQ methods.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

A Machine Learning Approach to Determine Surface Radiative Fluxes based on CERES Observations

The Clouds and Earth’s Radiant Energy System (CERES) projects provides satellite-based observations of the radiative fluxes and clouds systems. CERES climate quality data products typically take several months of calibration and validation before release to the public. An alternative data product, Fast Longwave and Shortwave radiative Flux (FLASHFlux), was created to provide data to the applied sciences and educational users. FLASHFlux provides Top-of-Atmosphere radiative fluxes, Clouds properties, and parameterized surface radiative fluxes within four days for footprint (Level 2) data. We investigate the use of Artificial Neural Network (ANN) using MODerate resolution Imaging Spectroradiometer (MODIS) derived clouds properties and meteorology from the Global Assimilation and Meteorology Office (GMAO) scaled to the CERES footprint from the CERES Clouds Radiative Swath (CRS) data product to compute surface radiative fluxes. We test ANN produce fluxes against surface fluxes produced from the Fu-Liou model used in CRS and the Langley Parameterized Shortwave Algorithm (LPSA) and Langley Parameterized Longwave Algorithm (LPLA) used in FLASHFlux. We also validated each model with ground-based observations. Furthermore, we investigate Leave-One-Feature-Out Importance (LOFO) to evaluate the significance of each feature in our training and provide insight for future models. Advances in machine learning, along with increases in computational capabilities and available data allow us to estimate effects of unresolved processes in our climate without direct modeling. This work evaluates the ability to create accurate data-driven models to supplement or replace current models that estimate surface radiative fluxes.

Climatology↗
DOE OSTI2026

A hybrid Penman-Monteith and machine learning model for simulating evapotranspiration and its components

Integrating physical processes with machine learning has advanced evapotranspiration (ET) simulation, yet most hybrid models fail to partition total ET into its components: soil evaporation (E) and vegetation transpiration (T). This study introduces Residual Neural Network–Penman–Monteith (RNN-PM), a novel hybrid dual-source ET model designed to overcome this limitation. The model synergizes the physically-based Penman–Monteith framework with three specialized residual neural networks trained to estimate key conductance parameters (canopy conductance, soil surface conductance, and aerodynamic conductance). Furthermore this explicit parameterization allows for the direct partitioning of total ET. Validation at National Ecological Observatory Network (NEON) flux sites using high-frequency partitioned E and T shows that RNN-PM reliably reproduces ET and the transpiration fraction (T/ET). For ET, the model achieves an average Kling–Gupta efficiency (KGE) of 0.89 and a root-mean-square error (RMSE) of 0.55 mm/day; for T/ET, the KGE is 0.87 with an RMSE of 0.06. Furthermore, RNN-PM demonstrates robust generalization, accurately simulating ET and its components well beyond the initial training dataset, even under extreme climatic conditions. This study extended the analysis by comparing the RNN-PM model with seven established dual-source ET models. The results indicate that RNN-PM outperforms both conventional machine learning models and purely physical process-based models in simulating ET components in most cases. Among the purely physical process-based dual-source models, those based on surface temperature decomposition showed improved performance as the leaf area index (LAI) decreased when evaluated against high-frequency ET component datasets. In contrast, the performance of conductance-based dual-source models declined with decreasing LAI. Although purely machine learning-based models can produce relatively accurate simulations of ET components, they often exhibit limited generalization capability, an issue that the RNN-PM model effectively overcomes. Ultimately, the RNN-PM model represents a significant advance in simulating ET components, offering a novel and scalable approach for improving the representation of land–atmosphere interactions in Earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Integrated analysis of X-ray diffraction patterns and pair distribution functions for machine-learned phase identification

Abstract To bolster the accuracy of existing methods for automated phase identification from X-ray diffraction (XRD) patterns, we introduce a machine learning approach that uses a dual representation whereby XRD patterns are augmented with simulated pair distribution functions (PDFs). A convolutional neural network is trained directly on XRD patterns calculated using physics-informed data augmentation, which accounts for experimental artifacts such as lattice strain and crystallographic texture. A second network is trained on PDFs generated via Fourier transform of the augmented XRD patterns. At inference, these networks classify unknown samples by aggregating their predictions in a confidence-weighted sum. We show that such an integrated approach to phase identification provides enhanced accuracy by leveraging the benefits of each model’s input representation. Whereas networks trained on XRD patterns provide a reciprocal space representation and can effectively distinguish large diffraction peaks in multi-phase samples, networks trained on PDFs provide a real space representation and perform better when peaks with low intensity become important. These findings underscore the importance of using diverse input representations for machine learning models in materials science and point to new avenues for automating multi-modal characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Evaluating the Trustworthiness of Explainable Artificial Intelligence (XAI) Methods Applied to Regression Predictions of Arctic Sea Ice Motion

Abstract Recent advances in explainable artificial intelligence (XAI) methods show promise for understanding predictions made by machine learning (ML) models. XAI explains how the input features are relevant or important for the model predictions. We train linear regression (LR) and convolutional neural network (CNN) models to make 1-day predictions of sea ice velocity in the Arctic from inputs of present-day wind velocity and previous-day ice velocity and concentration. We apply XAI methods to the CNN and compare explanations to variance explained by LR. We confirm the feasibility of using a novel XAI method [i.e., global layerwise relevance propagation (LRP)] to understand ML model predictions of sea ice motion by comparing it to established techniques. We investigate a suite of linear, perturbation-based, and propagation-based XAI methods in both local and global forms. Outputs from different explainability methods are generally consistent in showing that wind speed is the input feature with the highest contribution to ML predictions of ice motion, and we discuss inconsistencies in the spatial variability of the explanations. Additionally, we show that the CNN relies on both linear and nonlinear relationships between the inputs and uses nonlocal information to make predictions. LRP shows that wind speed over land is highly relevant for predicting ice motion offshore. This provides a framework to show how knowledge of environmental variables (i.e., wind) on land could be useful for predicting other properties (i.e., sea ice velocity) elsewhere. Significance Statement Explainable artificial intelligence (XAI) is useful for understanding predictions made by machine learning models. Our research establishes trustability in a novel implementation of an explainable AI method known as layerwise relevance propagation for Earth science applications. To do this, we provide a comparative evaluation of a suite of explainable AI methods applied to machine learning models that make 1-day predictions of Arctic sea ice velocity. We use explainable AI outputs to understand how the input features are used by the machine learning to predict ice motion. Additionally, we show that a convolutional neural network uses nonlinear and nonlocal information in making its predictions. We take advantage of the nonlocality to investigate the extent to which knowledge of wind on land is useful for predicting sea ice velocity elsewhere.

Hoffman, Lauren [Scripps Institution of Oceanograp↗
DOE OSTI2022

Exploration of Quantum Machine Learning and AI Accelerators for Fusion Science

The dawn of the noisy intermediate-scale quantum (NISQ) era sparked rapid development in variational quantum algorithms. These algorithms, utilizing parameterized quantum circuit optimized by classical computers with feedback, are practical under the constraints of the current hardware and can potentially show quantum advantage. In recent years, these variational circuit are applied to neural networks, hoping to boost the triumphant success of deep learning. We simulate various of quantum-classical hybrid neural networks applied to scientific tasks, and seek to understand their capabilities and limitations. Specifically, we devise quantum convolutional layers and apply them to deep neural networks used for predicting plasma disruption in fusion reactors.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Deep learning enhanced super-resolution x-ray fluorescence microscopy by a dual-branch network

X-ray fluorescence (XRF) microscopy is a powerful technique for quantifying the distribution of elements in complex materials, which makes it a crucial imaging technique across a wide range of disciplines in physical and biological sciences, including chemistry, materials science, microbiology, and geosciences. However, as a scanning microscopy technique, the spatial resolution of XRF imaging is inherently constrained by the x-ray probe profile and scanning step size. Here we propose a dual-branch machine learning (ML) model, which can extract scale-variant features and bypass abundant low-frequency information separately, to enhance the spatial resolution of the XRF images by mitigating the effects of blurring from the probe profile. The model is trained by simulated natural images, and a two-stage training strategy is used to overcome the domain gap between the natural images and experimental data. The tomography reconstruction from enhanced XRF projections shows an improvement in resolution by a scale factor of four and reveals distinct internal features invisible in low-resolution XRF within a battery sample. This study offers a promising approach for obtaining high-resolution XRF imaging from its low-resolution version, paving the way for future investigations in a broader range of disciplines and materials.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Machine learning to accelerate screening for Marcus reorganization energies

Understanding and predicting the charge transport properties of π-conjugated materials is an important challenge for designing new organic electronic devices, such as solar cells, plastic transistors, light-emitting devices, and chemical sensors. A key component of the hopping mechanism of charge transfer in these materials is the Marcus reorganization energy which serves as an activation barrier to hole or electron transfer. While modern density functional methods have proven to accurately predict trends in intramolecular reorganization energy, such calculations are computationally expensive. In this work, we outline active machine learning methods to predict computed intramolecular reorganization energies of a wide range of polythiophenes and their use toward screening new compounds with low internal reorganization energies. Our models have an overall root mean square error (RMSE) of ±0.113 eV, but a much smaller RMSE of only ±0.036 eV on the new screening set. Since the larger error derives from high-reorganization energy compounds, the new method is highly effective to screen for compounds with potentially efficient charge transport parameters.

14 SOLAR ENERGY↗
NASA NTRSDate not supplied

Biological Research and Space Health Enabled by Machine Learning to Support Deep Space Missions

A key science goal of the NASA “Moon to Mars” campaign is to understand how biology responds to the Lunar, Martian, and deep space environments in order to advance fundamental knowledge, reduce risk, and support safe, productive human space missions. Through the powerful emerging computer science approaches of artificial intelligence (AI) and machine learning (ML), a paradigm shift has begun in biomedical science and engineered astronaut health systems, to enable Earth-independence and autonomy of mission operations. We present a decadal view of AI/ML architecture to support deep space mission goals, developed in concert with leaders in the field. We describe current AI/ML methods to support 1) fundamental biology, 2) in situ analytics, 3) high performance computing hardware, 4) automated science, 5) self-driving labs, 6) remote data management, 7) integrated real-time mission biomonitoring, and 8) a Precision Space Health system. Cutting-edge AI/ML approaches that can be integrated to support these domains include active learning, explainable AI, adaptive learning, causal inference, knowledge graphs, federated learning, transfer learning, and large language models. Finally, we present results from several current ML projects that are underway in the field to address key challenges of small sample n, high feature count, heterogeneity, and sparse data. These include 1) connecting omics data to phenotypic data using an ensemble model to infer causality of spaceflight rodent liver health disruption, 2) usage of explainable ML to interrogate the muscular underpinnings of spaceflight muscle atrophy, 3) ML models analyzing and determining directed acyclic graphs of human space health risk leveraging rodent bone datasets, 4) usage of large pre-trained models connecting biomedical knowledgebases with small spaceflight datasets to understand gene-to-gene interaction networks, and 5) a suite of benchmarked open science datasets (spaceflight mouse liver; radiation DNA damage) enabling programmers to identify the best ML algorithms to answer space biological science questions.

space biology↗
DOE OSTI2020

Hierarchical, rotation‐equivariant neural networks to select structural models of protein complexes

Abstract Predicting the structure of multi‐protein complexes is a grand challenge in biochemistry, with major implications for basic science and drug discovery. Computational structure prediction methods generally leverage predefined structural features to distinguish accurate structural models from less accurate ones. This raises the question of whether it is possible to learn characteristics of accurate models directly from atomic coordinates of protein complexes, with no prior assumptions. Here we introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes, without using any precomputed physics‐inspired or statistical terms. Our neural network architecture combines multiple ingredients that together enable end‐to‐end learning from molecular structures containing tens of thousands of atoms: a point‐based representation of atoms, equivariance with respect to rotation and translation, local convolutions, and hierarchical subsampling operations. When used in combination with previously developed scoring functions, our network substantially improves the identification of accurate structural models among a large set of possible models. Our network can also be used to predict the accuracy of a given structural model in absolute terms. The architecture we present is readily applicable to other tasks involving learning on 3D structures of large atomic systems.

Eismann, Stephan↗
DOE OSTI2025

Multi-fidelity learning for interatomic potentials: low-level forces and high-level energies are all you need

The promise of machine learning interatomic potentials (MLIPs) has led to an abundance of public quantum mechanical (QM) training datasets. The quality of an MLIP is directly limited by the accuracy of the energies and atomic forces in the training dataset. Unfortunately, most of these datasets are computed with relatively low-accuracy QM methods, e.g. density functional theory with a moderate basis set. Due to the increased computational cost of more accurate QM methods, e.g. coupled-cluster theory with a complete basis set (CBS) extrapolation, most high-accuracy datasets are much smaller and often do not contain atomic forces. The lack of high-accuracy atomic forces is quite troubling, as training with force data greatly improves the stability and quality of the MLIP compared to training to energy alone. Because most datasets are computed with a unique level of theory, traditional single-fidelity (SF) learning is not capable of leveraging the vast amounts of published QM data. In this study, we apply multi-fidelity learning (MFL) to train an MLIP to multiple QM datasets of different levels of accuracy, i.e. levels of fidelity. Specifically, we perform three test cases to demonstrate that MFL with both low-level forces and high-level energies yields an extremely accurate MLIP—far more accurate than a SF MLIP trained solely to high-level energies and almost as accurate as a SF MLIP trained directly to high-level energies and forces. Therefore, MFL greatly alleviates the need for generating large and expensive datasets containing high-accuracy atomic forces and allows for more effective training to existing high-accuracy energy-only datasets. Indeed, low-accuracy atomic forces and high-accuracy energies are all that are needed to achieve a high-accuracy MLIP with MFL.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

Biological Research and Space Health Enabled by Machine Learning to Support Deep Space Missions

A key science goal of the NASA “Moon to Mars” campaign is to understand how biology responds to the Lunar, Martian, and deep space environments in order to advance fundamental knowledge, reduce risk, and support safe, productive human space missions. Through the powerful emerging computer science approaches of artificial intelligence (AI) and machine learning (ML), a paradigm shift has begun in biomedical science and engineered astronaut health systems, to enable Earth-independence and autonomy of mission operations. We present a decadal view of AI/ML architecture to support deep space mission goals, developed in concert with leaders in the field. We describe current AI/ML methods to support 1) fundamental biology, 2) in situ analytics, 3) high performance computing hardware, 4) automated science, 5) self-driving labs, 6) remote data management, 7) integrated real-time mission biomonitoring, and 8) a Precision Space Health system. Cutting-edge AI/ML approaches that can be integrated to support these domains include active learning, explainable AI, adaptive learning, causal inference, knowledge graphs, federated learning, transfer learning, and large language models. Finally, we present results from several current ML projects that are underway in the field to address key challenges of small sample n, high feature count, heterogeneity, and sparse data. These include 1) connecting omics data to phenotypic data using an ensemble model to infer causality of spaceflight rodent liver health disruption, 2) usage of explainable ML to interrogate the muscular underpinnings of spaceflight muscle atrophy, 3) ML models analyzing and determining directed acyclic graphs of human space health risk leveraging rodent bone datasets, 4) usage of large pre-trained models connecting biomedical knowledgebases with small spaceflight datasets to understand gene-to-gene interaction networks, and 5) a suite of benchmarked open science datasets (spaceflight mouse liver; radiation DNA damage) enabling programmers to identify the best ML algorithms to answer space biological science questions.

space biology↗
DOE OSTI2022

Multi-Kernel Adaptive Support Vector Machine for Scalable Predictive Maintenance

Application of data-driven solutions across an industry is challenging, since the data are often stored locally, and increasing privacy and security concerns restrict access to the data. In addition, it is highly unlikely that all potential data patterns are captured in a single data source. Because it is highly unlikely that all potential data patterns are captured in a single data source, machine learning (ML) models developed from a single source cannot be robust enough. An alternative is to train the ML model at each source and develop a distributed knowledge discovery and aggregation approach to build global knowledge. In this paper, we develop and demonstrate a distributed ML model, federated transfer learning (FTL), using a multi-kernel-based adaptive support vector machine (MK-A-SVM). For federated learning (FL), the multi-kernel (MK) approach enables feature-specific model aggregation under data heterogeneity; whereas for transfer learning (TL) the adaptive model enables utilization of an aggregated model from a different task. The proposed approach is validated using nuclear power plant (NPP) vertical motor-driven pump data to predict the health condition of vertical motor-driven pumps as an anomaly detection. The efficiency of the proposed approach is also quantified and compared with neural network.

42 ENGINEERING↗