Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning in Network Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Towards a more general understanding of the algorithmic utility of recurrent connections

Lateral and recurrent connections are ubiquitous in biological neural circuits. Yet while the strong computational abilities of feedforward networks have been extensively studied, our understanding of the role and advantages of recurrent computations that might explain their prevalence remains an important open challenge. Foundational studies by Minsky and Roelfsema argued that computations that require propagation of global information for local computation to take place would particularly benefit from the sequential, parallel nature of processing in recurrent networks. Such “tag propagation” algorithms perform repeated, local propagation of information and were originally introduced in the context of detecting connectedness, a task that is challenging for feedforward networks. Here, we advance the understanding of the utility of lateral and recurrent computation by first performing a large-scale empirical study of neural architectures for the computation of connectedness to explore feedforward solutions more fully and establish robustly the importance of recurrent architectures. In addition, we highlight a tradeoff between computation time and performance and construct hybrid feedforward/recurrent models that perform well even in the presence of varying computational time limitations. We then generalize tag propagation architectures to propagating multiple interacting tags and demonstrate that these are efficient computational substrates for more general computations of connectedness by introducing and solving an abstracted biologically inspired decision-making task. Our work thus clarifies and expands the set of computational tasks that can be solved efficiently by recurrent computation, yielding hypotheses for structure in population activity that may be present in such tasks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

GANpiler

Focal Area(s): Rather than augment or replace physical models with machine learning, we instead preserve the existing models and augment the underlying code with faster surrogate models created by generative adversarial networks (GAN). To leverage the performance optimization, we also propose a runtime system and user interfaces that allow prediction and tracking of accumulated error as well as dynamic, per-process decision making as to which model (if any) to use for each iteration. Science Challenge: This proposal sits at the nexus of two hard problems. First, climate models based on machine learning will be, by their nature, difficult to trust once their predictions begin diverging from the consensus. Second, compilers and hardware have been making only incremental performance gains for decades. GPGPUs have provided a welcome performance boost for codes that can take advantage of them, but there is no similar technology on the horizon to provide the next performance leap.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Design, Control and Application of Next Generation Qubits

Design, Control and Application of Next Generation Qubits Arun Bansil, Northeastern University (Principal Investigator) Claudio Chamon, Boston University (Co-Investigator) Adrian Feiguin, Northeastern University (Co-Investigator) Liang Fu, MIT (Co-Investigator) Eduardo Mucciolo, Univ. of Central Florida (Co-Investigator) Qimin Yan, Temple University (Co-Investigator) The quest for developing technologies for manipulating and storing information quantum mechanically is currently led by approaches that include Josephson-junctions, ion-traps, and qubits generated by defect spins in solids. Topological qubits, however, are inherently more robust to decoherence by environmental effects, and should be able to sprint ahead once practical barriers have been overcome. At the present stage of the development of the field, it is important to explore a variety of architectures and materials beyond the conventional paradigms in order to seed breakthroughs toward building a scalable quantum computer. Our comprehensive theoretical research program involved four interconnected thrusts as follows. • A materials discovery effort in two-dimensional compounds in search of materials to support Majorana zero modes and defect structures suitable as qubits. • Exploration of architectures for topological quantum computation by investigating both superconducting Majorana qubits, and robust platforms for braiding with new “meta-materials” built of arrays of Majorana qubits. • Investigation of properties of hybrid metal-organic qubits based on transition-metal centers in graphene, and molecular crystals of polyaromatic complexes with embedded transition-metal atoms. • Development of tensor-network and semiclassical approaches to study decoherence in the presence of random and dispersive spin baths, and NV centers in diamond. The full spectrum of theoretical and numerical approaches was used to address the goals of this project including first-principles, density-matrix-renormalization group, tensor networks, and data-driven high-throughput approaches using materials database and machine-learning.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Development of neural network force fields for corrosion studies

To fully understand the chemistry and physics of corrosion, novel methods of simulation must be developed. One approach is designing machine learning (ML) algorithms integrated with density functional theory to develop adaptive force fields to gain insight into corrosion behavior namely at the surface of metal oxides. Current methods of modeling corrosion are slow due to the computational cost of resolving both reaction mechanics and mass transport processes. Machine learning methods can be implemented to obtain structure-activity relationships at both the molecular and bulk scale while still retaining the accuracy of density functional theory (DFT) and significantly decreasing the time needed for simulations of complex chemical processes in the various environments of corrosion. Multiscale models are needed for corrosion studies to fully understand its processes not only at the atomic length scale (chemical bonding, energies, and forces), but also at the nano and meso length scales (solid-state physics and material science processes). Current methods of study include DFT, molecular dynamics, and Monte Carlo. The limitation of DFT is that only a small number of atoms or molecules can be simulated at that level of theory. Density functional theory is used to study the electronic structure of atoms and molecules, and calculate the force component of each atom. However, these calculations are limited to about 1000 atoms. Custom periodic boundary conditions (PBC) can be used to describe the various environments and defects that affect the atomic forces to produce a large data set from which a training set can be derived. Machine learning can be utilized to overcome the barrier of modeling macroscopic and multi-scale processes from ab initio calculations through the development of adaptive force fields. Local environments determine the atomic forces of a given system, therefore adaptive force fields must be created to produce reliable quantum mechanical calculations. This can be achieved by developing a learning algorithm that uses the mapped atomic forces or fingerprint as an input to produce energies and magnetic moments as output. A systematic approach was used to begin to build a data set in order to accurately describe the atomic forces in various environments. In Figure 4 below, a simple PBC cell of Fe{sub 2}O{sub 3} was first optimized. A surface optimization was performed next, followed by a hydroxylated surface optimization. Once this calculation has converged, the adsorption of halide species to the hydroxylated surface will be investigated. TensorFlow is an open source platform for machine learning developed by Google. Using a high level application program interface (API) such as Keras allows for building and training ML models easily in a number of different environments and languages. For this project, a neural network was developed within Anaconda in Python. Future Work: Further development of reference data set; Refining neural network and learning algorithm; Fingerprinting atomic environment to enable mapping of atomic force components; Choosing appropriate training set from reference data; Learning from training set and enabling non-linear mapping of training set fingerprints and the atomic forces; Estimation of uncertainty to identify ranges of outside applicability; Testing and analysis of molecular dynamic simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Semantic segmentation with a sparse convolutional neural network for event reconstruction in MicroBooNE

We present the performance of a semantic segmentation network, SparseSSNet, that provides pixel-level classification of MicroBooNE data. The MicroBooNE experiment employs a liquid argon time projection chamber for the study of neutrino properties and interactions. SparseSSNet is a submanifold sparse convolutional neural network, which provides the initial machine learning based algorithm utilized in one of MicroBooNE's ν e -appearance oscillation analyses. The network is trained to categorize pixels into five classes, which are re-classified into two classes more relevant to the current analysis. The output of SparseSSNet is a key input in further analysis steps. This technique, used for the first time in liquid argon time projection chambers data and is an improvement compared to a previously used convolutional neural network, both in accuracy and computing resource utilization. Here, the accuracy achieved on the test sample is ≥ 99%. For full neutrino interaction simulations, the time for processing one image is ≈ 0.5 sec, the memory usage is at 1 GB level, which allows utilization of most typical CPU worker machine.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Integrating Analytical Solutions and U-Net Model for Predicting Groundwater Contaminant Plumes in Pump-and-Treat Systems

Pump-and-treat (P&T) is a common technique for groundwater remediation involving the extraction and treatment of contaminated water above ground. Optimizing the design and operation of the P&T well network is essential for maximizing the system’s effectiveness and efficiency. However, this optimization often necessitates many model evaluations, leading to computationally demanding tasks. This study introduces a novel approach that integrates analytical solutions for groundwater dynamics with the U-Net (Ronneberger et al., 2015) deep learning framework to predict groundwater contaminant plume migration under dynamic pumping conditions. By incorporating the Thiem equation (Thiem, 1906) into the input preprocessing, the U-Net model transforms sparse well data into a continuous spatial field that captures the hydraulic impacts of pumping activities. This integration enables the model to leverage both deep learning capabilities and classical physics-based groundwater theories, enhancing prediction accuracy and computational efficiency. These advancements can facilitate rapid, large-scale evaluations of P&T optimization simulations, allowing for timely and effective decision-making in well placement and system management. We demonstrate the model's robust performance across both simplified transient 2D models and a more complex 3D heterogeneous site model at the 200 West P&T facility at the Hanford Site. The U-Net-based model offers substantial computational advantages, reducing simulation times significantly compared to full physics-based models and providing a powerful tool for rapid site evaluation and P&T system optimization, such as evaluating alternative P&T well network designs. Our findings highlight the potential of advanced machine learning models to significantly enhance the efficiency and sustainability of groundwater remediation efforts, offering a novel application of U-Net architecture in environmental science.

Pump-and-treat↗
DOE OSTI2024

SPIDARman: System-Level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors

In nuclear power plants (NPPs), anomalies arising from sensors or human errors (HEs) can undermine the performance and reliability of plant operations. Anomaly detection models can be employed to detect sensor errors and HEs. Additionally, physics-informed machine learning models can utilize the known physics of the system, as described by mathematical equations, to ensure that sensor values are consistent with physical laws. Hence, we propose SPIDARman: System-level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors, a holistic physics-informed anomaly detection approach based on generative adversarial networks (GANs) to detect anomalies in both automatically collected sensor data and manually collected surveillance data. Here we test our approach on data collected from a flow loop testbed, showcasing its potential to detect anomalies. Results demonstrate that the proposed model performs better than the baseline GAN-based models in detecting sensor and surveillance anomalies, suggesting the potential of physics-informed anomaly detection GAN models in NPPs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Advanced Data Science Model for Detecting Intelligent Malware

This study focused on developing a robust artificial intelligence (AI) model capable of detecting and characterizing advanced malware in Internet of Things (IoT) devices using network data. By analyzing network traffic with various machine learning (ML) models, our AI model can identify and characterize malicious activities to significantly improve malware detection accuracy and reliability as compared to traditional methods. The developed AI/ML model was trained using network data from IoT devices, leveraging classifiers such as Random Forest, Gradient Boosting, AdaBoost, and others to optimize detection performance. This project demonstrates a scalable framework for real-time malware detection and characterization in IoT networks, capable of identifying infected devices and facilitating the necessary steps to remove or isolate them, thereby preventing further infections. Although digital twin (DT) integration is not yet implemented in the current model, it represents a promising future enhancement. By creating a virtual replica of physical IoT devices, DT technology would allow for real-time monitoring and analysis without directly accessing operational technology, thus reducing the risk of compromising or reducing the performance of actual devices. This integration would further enhance the security of IoT ecosystems, combining AI technology to better flag and detect indications of malware-infected devices within a nuclear system environment.

42 ENGINEERING↗
DOE OSTI2026

Attention-based explainability for structure–property relationships

Machine learning methods are emerging as a universal paradigm for constructing correlative structure–property relationships in materials science based on multimodal characterization. However, this necessitates the development of methods for the physical interpretability of the resulting correlative models. Here, we demonstrate the potential of attention-based neural networks for revealing structure–property relationships and the underlying physical mechanisms, using the ferroelectric properties of PbTiO3 thin films as a case study. Through the analysis of attention scores, we disentangle the influence of distinct domain patterns on the polarization switching process. The attention-based Transformer model is explored both as a direct interpretability tool and as a surrogate for explaining representations learned via unsupervised machine learning, enabling the identification of physically grounded correlations. We compare attention-derived interpretability scores with classical SHapley Additive exPlanations analysis and show that, in contrast to applications in natural language processing, attention mechanisms in materials science exhibit high efficiency in highlighting meaningful structural features.

Slautin, Boris [Independent Researcher]↗
DOE OSTI2022

Data-centric machine learning in quantum information science

Abstract We propose a series of data-centric heuristics for improving the performance of machine learning systems when applied to problems in quantum information science. In particular, we consider how systematic engineering of training sets can significantly enhance the accuracy of pre-trained neural networks used for quantum state reconstruction without altering the underlying architecture. We find that it is not always optimal to engineer training sets to exactly match the expected distribution of a target scenario, and instead, performance can be further improved by biasing the training set to be slightly more mixed than the target. This is due to the heterogeneity in the number of free variables required to describe states of different purity, and as a result, overall accuracy of the network improves when training sets of a fixed size focus on states with the least constrained free variables. For further clarity, we also include a ‘toy model’ demonstration of how spurious correlations can inadvertently enter synthetic data sets used for training, how the performance of systems trained with these correlations can degrade dramatically, and how the inclusion of even relatively few counterexamples can effectively remedy such problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients

Mortality prediction for intensive care unit (ICU) patients is crucial for improving outcomes and efficient utilization of resources. Accessibility of electronic health records (EHR) has enabled data-driven predictive modeling using machine learning. However, very few studies rely solely on unstructured clinical notes from the EHR for mortality prediction. In this work, we propose a framework to predict short, mid, and long-term mortality in adult ICU patients using unstructured clinical notes from the MIMIC III database, natural language processing (NLP), and machine learning (ML) models. Depending on the statistical description of the patients’ length of stay, we define the short-term as 48-hour and 4-day period, the mid-term as 7-day and 10-day period, and the long-term as 15-day and 30-day period after admission. We found that by only using clinical notes within the 24 hours of admission, our framework can achieve a high area under the receiver operating characteristics (AU-ROC) score for short, mid and long-term mortality prediction tasks. The test AU-ROC scores are 0.87, 0.83, 0.83, 0.82, 0.82, and 0.82 for 48-hour, 4-day, 7-day, 10-day, 15-day, and 30-day period mortality prediction, respectively. We also provide a comparative study among three types of feature extraction techniques from NLP: frequency-based technique, fixed embedding-based technique, and dynamic embedding-based technique. Lastly, we provide an interpretation of the NLP-based predictive models using feature-importance scores.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

E-PINNs: Epistemic Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) have demonstrated promise as a framework for solving forward and inverse problems involving partial differential equations. Despite recent progress in the field, it remains challenging to quantify uncertainty in these networks. While techniques such as Bayesian PINNs (B-PINNs) provide a principled approach to capturing epistemic uncertainty through Bayesian inference, they can be computationally expensive for large-scale applications. In this work, we propose Epistemic Physics-Informed Neural Networks (E-PINNs), a framework that uses a small network, the epinet, to efficiently quantify epistemic uncertainty in PINNs. The proposed approach works as an add-on to existing, pre-trained PINNs with a small computational overhead. We demonstrate the applicability of the proposed framework in various test cases and compare the results with B-PINNs using Hamiltonian Monte Carlo (HMC) posterior estimation and dropout-equipped PINNs (Dropout-PINNs). In our experiments, E-PINNs achieve calibrated coverage with competitive sharpness at substantially lower cost. We demonstrate that when B-PINNs produce narrower bands, they under-cover in our tests. E-PINNs also show better calibration than Dropout-PINNs in these examples, indicating a favorable accuracy-efficiency trade-off.

AI for Science↗
DOE OSTI2021

Probing lattice defects in crystalline battery cathode using hard X-ray nanoprobe with data-driven modeling

Lattice defects, e.g., dislocations and grain boundaries, critically impact the properties of crystalline battery cathode materials. A longstanding challenge is to probe the meso–scale heterogeneity and evolution of lattice defects with sensitivity to atomic-scale details. Herein, we tackle this issue with a unique combination of X-ray nanoprobe diffractive imaging and advanced machine learning techniques. The domains with different lattice defect configuration within a single-crystalline LiCoO2 cathode particle are faithfully revealed using our approach. We further visualize the rearrangement of grain boundaries and local crystallinity upon mild thermal annealing. Furthermore, these results pave a direct way to the understanding of crystalline battery materials’ response under external stimuli with high fidelity, which provides valuable empirical guidance to defect-engineering strategies for improving the cathode materials against aggressive battery operation.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Validation and parameterization of a novel physics-constrained neural dynamics model applied to turbulent fluid flow

We report, in fluid physics, data-driven models to enhance or accelerate time to solution are becoming increasingly popular for many application domains, such as alternatives to turbulence closures, system surrogates, or for new physics discovery. In the context of reduced order models of high-dimensional time-dependent fluid systems, machine learning methods grant the benefit of automated learning from data, but the burden of a model lies on its reduced-order representation of both the fluid state and physical dynamics. In this work, we build a physics-constrained, data-driven reduced order model for Navier–Stokes equations to approximate spatiotemporal fluid dynamics in the canonical case of isotropic turbulence in a triply periodic box. The model design choices mimic numerical and physical constraints by, for example, implicitly enforcing the incompressibility constraint and utilizing continuous neural ordinary differential equations for tracking the evolution of the governing differential equation. We demonstrate this technique on a three-dimensional, moderate Reynolds number turbulent fluid flow. In assessing the statistical quality and characteristics of the machine-learned model through rigorous diagnostic tests, we find that our model is capable of reconstructing the dynamics of the flow over large integral timescales, favoring accuracy at the larger length scales. More significantly, comprehensive diagnostics suggest that physically interpretable model parameters, corresponding to the representations of the fluid state and dynamics, have attributable and quantifiable impact on the quality of the model predictions and computational complexity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Using artificial intelligence to detect human errors in nuclear power plants: A case in operation and maintenance

Human error (HE) is an important concern in safety-critical systems such as nuclear power plants (NPPs). HE has played a role in many accidents and outage incidents in NPPs. Despite the increased automation in NPPs, HE remains unavoidable. Hence, the need for HE detection is as important as HE prevention efforts. In NPPs, HE is rather rare. Hence, anomaly detection, a widely used machine learning technique for detecting rare anomalous instances, can be repurposed to detect potential HE. In this study, we develop an unsupervised anomaly detection technique based on generative adversarial networks (GANs) to detect anomalies in manually collected surveillance data in NPPs. More specifically, our GAN is trained to detect mismatches between automatically recorded sensor data and manually collected surveillance data, and hence, identify anomalous instances that can be attributed to HE. We test our GAN on both a real-world dataset and an external dataset obtained from a testbed, and we benchmark our results against state-of-the-art unsupervised anomaly detection algorithms, including one-class support vector machine and isolation forest. Our results show that the proposed GAN provides improved anomaly detection performance. Our study is promising for the future development of artificial intelligence based HE detection systems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Machine-learning based approach to examine ecological processes influencing the diversity of riverine dissolved organic matter composition

Dissolved organic matter (DOM) assemblages in freshwater rivers are formed from mixtures of simple to complex compounds that are highly variable across time and space. These mixtures largely form due to the environmental heterogeneity of river networks and the contribution of diverse allochthonous and autochthonous DOM sources. Most studies are, however, confined to local and regional scales, which precludes an understanding of how these mixtures arise at large, e.g., continental, spatial scales. The processes contributing to these mixtures are also difficult to study because of the complex interactions between various environmental factors and DOM. Here we propose the use of machine learning (ML) approaches to identify ecological processes contributing toward mixtures of DOM at a continental-scale. We related a dataset that characterized the molecular composition of DOM from river water and sediment with Fourier-transform ion cyclotron resonance mass spectrometry to explanatory physicochemical variables such as nutrient concentrations and stable water isotopes ( 2 H and 18 O). Using unsupervised ML, distinctive clusters for sediment and water samples were identified, with unique molecular compositions influenced by environmental factors like terrestrial input and microbial activity. Sediment clusters showed a higher proportion of protein-like and unclassified compounds than water clusters, while water clusters exhibited a more diversified chemical composition. We then applied a supervised ML approach, involving a two-stage use of SHapley Additive exPlanations (SHAP) values. In the first stage, SHAP values were obtained and used to identify key physicochemical variables. These parameters were employed to train models using both the default and subsequently tuned hyperparameters of the Histogram-based Gradient Boosting (HGB) algorithm. The supervised ML approach, using HGB and SHAP values, highlighted complex relationships between environmental factors and DOM diversity, in particular the existence of dams upstream, precipitation events, and other watershed characteristics were important in predicting higher chemical diversity in DOM. Our data-driven approach can now be used more generally to reveal the interplay between physical, chemical, and biological factors in determining the diversity of DOM in other ecosystems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Deep Multitask Learning Models for Radiation Estimation at High Energy Accelerator Facility

Controlling the dose of radiation exposure in potential radioactive facilities is critical for ensuring the safety of staff and the public. Here, in this paper, we developed machine learning models to estimate radiation exposure efficiently at the Thomas Jefferson National Accelerator Facility (JLab), aiming to enhance safety in both accelerator facilities and public areas. Multiple sensors were deployed around the three experimental halls at JLab. Data on single-beam currents, energy levels, and radiation values at the sensor locations were collected during accelerator operation. We proposed a multi-task learning model for radiation estimation, utilizing either one-dimensional convolutional neural networks (1-D CNNs) or long short-term memory networks (LSTMs) as the backbone. The proposed model was trained to simultaneously estimate radiation levels at the sensor locations. Experimental results demonstrated that the proposed model with LSTM backbone achieved the best estimation performance, with an average R 2 score of 0.7557 for estimation within the same year and 0.7157 for estimation across different years. These results significantly surpassed those of competing models.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗