Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning in Network Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Converting tabular data into images for deep learning with convolutional neural networks

Abstract Convolutional neural networks (CNNs) have been successfully used in many applications where important information about data is embedded in the order of features, such as speech and imaging. However, most tabular data do not assume a spatial relationship between features, and thus are unsuitable for modeling using CNNs. To meet this challenge, we develop a novel algorithm, image generator for tabular data (IGTD), to transform tabular data into images by assigning features to pixel positions so that similar features are close to each other in the image. The algorithm searches for an optimized assignment by minimizing the difference between the ranking of distances between features and the ranking of distances between their assigned pixels in the image. We apply IGTD to transform gene expression profiles of cancer cell lines (CCLs) and molecular descriptors of drugs into their respective image representations. Compared with existing transformation methods, IGTD generates compact image representations with better preservation of feature neighborhood structure. Evaluated on benchmark drug screening datasets, CNNs trained on IGTD image representations of CCLs and drugs exhibit a better performance of predicting anti-cancer drug response than both CNNs trained on alternative image representations and prediction models trained on the original tabular data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

BUTTER-E - Energy Consumption Data for the BUTTER Empirical Deep Learning Dataset

The BUTTER-E - Energy Consumption Data for the BUTTER Empirical Deep Learning Dataset adds node-level energy consumption data from watt-meters to the primary sweep of the BUTTER - Empirical Deep Learning Dataset. This dataset contains energy consumption and performance data from 63,527 individual experimental runs spanning 30,582 distinct configurations: 13 datasets, 20 sizes (number of trainable parameters), 8 network "shapes", and 14 depths on both CPU and GPU hardware collected using node-level watt-meters. This dataset reveals the complex relationship between dataset size, network structure, and energy use, and highlights the impact of cache effects. BUTTER-E is intended to be joined with the BUTTER dataset (see "BUTTER - Empirical Deep Learning Dataset on OEDI" resource below) which characterizes the performance of 483k distinct fully connected neural networks but does not include energy measurements.

Array↗
DOE OSTI2021

COVID-19 dynamics across the US: A deep learning study of human mobility and social behavior

This paper presents a deep learning framework for epidemiology system identification from noisy and sparse observations with quantified uncertainty. The proposed approach employs an ensemble of deep neural networks to infer the time-dependent reproduction number of an infectious disease by formulating a tensor-based multi-step loss function that allows us to efficiently calibrate the model on multiple observed trajectories. The method is applied to a mobility and social behavior-based SEIR model of COVID-19 spread. The model is trained on Google and Unacast mobility data spanning a period of 66 days, and is able to yield accurate future forecasts of COVID-19 spread in 203 US counties within a time-window of 15 days. Interestingly, a sensitivity analysis that assesses the importance of different mobility and social behavior parameters reveals that attendance of close places, including workplaces, residential, and retail and recreational locations, has the largest impact on the effective reproduction number. Furthermore, the model enables us to rapidly probe and quantify the effects of government interventions, such as lock-down and re-opening strategies. Taken together, the proposed framework provides a robust workflow for data-driven epidemiology model discovery under uncertainty and produces probabilistic forecasts for the evolution of a pandemic that can judiciously provide information for policy and decision making. All codes and data accompanying this manuscript are available at https://github.com/PredictiveIntelligenceLab/DeepCOVID19.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Missing Wedge Completion via Unsupervised Learning with Coordinate Networks

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input projections. This eliminates the need for pretraining, reducing reconstruction runtime by 3–20× compared to supervised methods. Our in silico results show improved shape completion and reduction of missing wedge artifacts, assessed through several voxel-based image quality metrics in real space and a novel directional Fourier Shell Correlation (FSC) metric. Our study illuminates benefits and considerations of both supervised and unsupervised approaches, guiding the development of improved reconstruction strategies.

42 ENGINEERING↗
DOE OSTI2024

Machine learning assisted search for Fe–Co–C ternary compounds with high magnetic anisotropy

We employ a machine learning (ML)-guided framework to explore rare earth free magnetic materials, specifically focusing on Fe–Co–C ternary compounds for potential use in permanent magnets. Utilizing a specifically trained crystal graph convolutional neural network model, we efficiently screen a vast space of nearly a million substitutional structures to select 620 promising structures for further investigation by first-principles calculation. We predict five low-energy metastable Fe–Co–C compounds with formation energy less than 150 meV/atom above the convex hull. These compounds exhibit high magnetization (Js > 1.0 T) and significant magnetic anisotropy (K1 > 1.0 MJ/m3), making them promising candidates for permanent magnet applications. The phonon calculations indicate these compounds are dynamically stable. Our ML-guided framework demonstrates the utility of rapidly identifying novel materials with tailored magnetic properties.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Hybrid interatomic potential for Sn

To design materials for extreme applications, it is important to understand and predict phase transitions and their influence on material properties under high pressures and temperatures. Atomistic modeling can be a useful tool to assess these behaviors. However, this can be difficult due to the lack of fidelity of the interatomic potentials in reproducing this high pressure and temperature extreme behavior. Here, in this work, a hybrid EAM-R—which is the combination of embedded atom method (EAM) and rapid artificial neural network potential—for Tin (Sn) is described which is capable of accurately modeling the complex sequence of phase transitions between different metallic polymorphs as a function of pressure. This hybrid approach ensures that a basic empirical potential like EAM is used as a lower energy bound. By using the final activation function, the neural network contribution to energy must be positive, assuring stability over the whole configuration space. This implementation has the capacity to reproduce density functional theory results at 6 orders of magnitude slower than a pair potential for molecular dynamics simulation, including elastic and plastic characteristics and relative energies of each phase. Using calculations of the Gibbs free energy, it is demonstrated that the potential precisely predicts the experimentally observed phase changes at temperatures and pressures across the whole phase diagram. At 10.2 GPa, the present potential predicts a first-order phase transition between body-centered tetragonal (BCT) β-Sn and another polymorph of BCT-Sn. This structure transforms into body-centered cubic near the experimentally reported value at 33 GPa. Thus, the Sn potential developed in this paper can be used to study complex deformation mechanisms under extreme conditions of high pressure and strain rates unlike existing potentials. Moreover, the framework developed in this paper can be extended for different material systems with complex phase diagrams.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Towards a more general understanding of the algorithmic utility of recurrent connections

Lateral and recurrent connections are ubiquitous in biological neural circuits. Yet while the strong computational abilities of feedforward networks have been extensively studied, our understanding of the role and advantages of recurrent computations that might explain their prevalence remains an important open challenge. Foundational studies by Minsky and Roelfsema argued that computations that require propagation of global information for local computation to take place would particularly benefit from the sequential, parallel nature of processing in recurrent networks. Such “tag propagation” algorithms perform repeated, local propagation of information and were originally introduced in the context of detecting connectedness, a task that is challenging for feedforward networks. Here, we advance the understanding of the utility of lateral and recurrent computation by first performing a large-scale empirical study of neural architectures for the computation of connectedness to explore feedforward solutions more fully and establish robustly the importance of recurrent architectures. In addition, we highlight a tradeoff between computation time and performance and construct hybrid feedforward/recurrent models that perform well even in the presence of varying computational time limitations. We then generalize tag propagation architectures to propagating multiple interacting tags and demonstrate that these are efficient computational substrates for more general computations of connectedness by introducing and solving an abstracted biologically inspired decision-making task. Our work thus clarifies and expands the set of computational tasks that can be solved efficiently by recurrent computation, yielding hypotheses for structure in population activity that may be present in such tasks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

GANpiler

Focal Area(s): Rather than augment or replace physical models with machine learning, we instead preserve the existing models and augment the underlying code with faster surrogate models created by generative adversarial networks (GAN). To leverage the performance optimization, we also propose a runtime system and user interfaces that allow prediction and tracking of accumulated error as well as dynamic, per-process decision making as to which model (if any) to use for each iteration. Science Challenge: This proposal sits at the nexus of two hard problems. First, climate models based on machine learning will be, by their nature, difficult to trust once their predictions begin diverging from the consensus. Second, compilers and hardware have been making only incremental performance gains for decades. GPGPUs have provided a welcome performance boost for codes that can take advantage of them, but there is no similar technology on the horizon to provide the next performance leap.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Design, Control and Application of Next Generation Qubits

Design, Control and Application of Next Generation Qubits Arun Bansil, Northeastern University (Principal Investigator) Claudio Chamon, Boston University (Co-Investigator) Adrian Feiguin, Northeastern University (Co-Investigator) Liang Fu, MIT (Co-Investigator) Eduardo Mucciolo, Univ. of Central Florida (Co-Investigator) Qimin Yan, Temple University (Co-Investigator) The quest for developing technologies for manipulating and storing information quantum mechanically is currently led by approaches that include Josephson-junctions, ion-traps, and qubits generated by defect spins in solids. Topological qubits, however, are inherently more robust to decoherence by environmental effects, and should be able to sprint ahead once practical barriers have been overcome. At the present stage of the development of the field, it is important to explore a variety of architectures and materials beyond the conventional paradigms in order to seed breakthroughs toward building a scalable quantum computer. Our comprehensive theoretical research program involved four interconnected thrusts as follows. • A materials discovery effort in two-dimensional compounds in search of materials to support Majorana zero modes and defect structures suitable as qubits. • Exploration of architectures for topological quantum computation by investigating both superconducting Majorana qubits, and robust platforms for braiding with new “meta-materials” built of arrays of Majorana qubits. • Investigation of properties of hybrid metal-organic qubits based on transition-metal centers in graphene, and molecular crystals of polyaromatic complexes with embedded transition-metal atoms. • Development of tensor-network and semiclassical approaches to study decoherence in the presence of random and dispersive spin baths, and NV centers in diamond. The full spectrum of theoretical and numerical approaches was used to address the goals of this project including first-principles, density-matrix-renormalization group, tensor networks, and data-driven high-throughput approaches using materials database and machine-learning.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Semantic segmentation with a sparse convolutional neural network for event reconstruction in MicroBooNE

We present the performance of a semantic segmentation network, SparseSSNet, that provides pixel-level classification of MicroBooNE data. The MicroBooNE experiment employs a liquid argon time projection chamber for the study of neutrino properties and interactions. SparseSSNet is a submanifold sparse convolutional neural network, which provides the initial machine learning based algorithm utilized in one of MicroBooNE's ν e -appearance oscillation analyses. The network is trained to categorize pixels into five classes, which are re-classified into two classes more relevant to the current analysis. The output of SparseSSNet is a key input in further analysis steps. This technique, used for the first time in liquid argon time projection chambers data and is an improvement compared to a previously used convolutional neural network, both in accuracy and computing resource utilization. Here, the accuracy achieved on the test sample is ≥ 99%. For full neutrino interaction simulations, the time for processing one image is ≈ 0.5 sec, the memory usage is at 1 GB level, which allows utilization of most typical CPU worker machine.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Integrating Analytical Solutions and U-Net Model for Predicting Groundwater Contaminant Plumes in Pump-and-Treat Systems

Pump-and-treat (P&T) is a common technique for groundwater remediation involving the extraction and treatment of contaminated water above ground. Optimizing the design and operation of the P&T well network is essential for maximizing the system’s effectiveness and efficiency. However, this optimization often necessitates many model evaluations, leading to computationally demanding tasks. This study introduces a novel approach that integrates analytical solutions for groundwater dynamics with the U-Net (Ronneberger et al., 2015) deep learning framework to predict groundwater contaminant plume migration under dynamic pumping conditions. By incorporating the Thiem equation (Thiem, 1906) into the input preprocessing, the U-Net model transforms sparse well data into a continuous spatial field that captures the hydraulic impacts of pumping activities. This integration enables the model to leverage both deep learning capabilities and classical physics-based groundwater theories, enhancing prediction accuracy and computational efficiency. These advancements can facilitate rapid, large-scale evaluations of P&T optimization simulations, allowing for timely and effective decision-making in well placement and system management. We demonstrate the model's robust performance across both simplified transient 2D models and a more complex 3D heterogeneous site model at the 200 West P&T facility at the Hanford Site. The U-Net-based model offers substantial computational advantages, reducing simulation times significantly compared to full physics-based models and providing a powerful tool for rapid site evaluation and P&T system optimization, such as evaluating alternative P&T well network designs. Our findings highlight the potential of advanced machine learning models to significantly enhance the efficiency and sustainability of groundwater remediation efforts, offering a novel application of U-Net architecture in environmental science.

Pump-and-treat↗
DOE OSTI2024

SPIDARman: System-Level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors

In nuclear power plants (NPPs), anomalies arising from sensors or human errors (HEs) can undermine the performance and reliability of plant operations. Anomaly detection models can be employed to detect sensor errors and HEs. Additionally, physics-informed machine learning models can utilize the known physics of the system, as described by mathematical equations, to ensure that sensor values are consistent with physical laws. Hence, we propose SPIDARman: System-level Physics-Informed Detection of Anomalies in Reactor Collected Data Considering Human Errors, a holistic physics-informed anomaly detection approach based on generative adversarial networks (GANs) to detect anomalies in both automatically collected sensor data and manually collected surveillance data. Here we test our approach on data collected from a flow loop testbed, showcasing its potential to detect anomalies. Results demonstrate that the proposed model performs better than the baseline GAN-based models in detecting sensor and surveillance anomalies, suggesting the potential of physics-informed anomaly detection GAN models in NPPs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Advanced Data Science Model for Detecting Intelligent Malware

This study focused on developing a robust artificial intelligence (AI) model capable of detecting and characterizing advanced malware in Internet of Things (IoT) devices using network data. By analyzing network traffic with various machine learning (ML) models, our AI model can identify and characterize malicious activities to significantly improve malware detection accuracy and reliability as compared to traditional methods. The developed AI/ML model was trained using network data from IoT devices, leveraging classifiers such as Random Forest, Gradient Boosting, AdaBoost, and others to optimize detection performance. This project demonstrates a scalable framework for real-time malware detection and characterization in IoT networks, capable of identifying infected devices and facilitating the necessary steps to remove or isolate them, thereby preventing further infections. Although digital twin (DT) integration is not yet implemented in the current model, it represents a promising future enhancement. By creating a virtual replica of physical IoT devices, DT technology would allow for real-time monitoring and analysis without directly accessing operational technology, thus reducing the risk of compromising or reducing the performance of actual devices. This integration would further enhance the security of IoT ecosystems, combining AI technology to better flag and detect indications of malware-infected devices within a nuclear system environment.

42 ENGINEERING↗
DOE OSTI2026

Attention-based explainability for structure–property relationships

Machine learning methods are emerging as a universal paradigm for constructing correlative structure–property relationships in materials science based on multimodal characterization. However, this necessitates the development of methods for the physical interpretability of the resulting correlative models. Here, we demonstrate the potential of attention-based neural networks for revealing structure–property relationships and the underlying physical mechanisms, using the ferroelectric properties of PbTiO3 thin films as a case study. Through the analysis of attention scores, we disentangle the influence of distinct domain patterns on the polarization switching process. The attention-based Transformer model is explored both as a direct interpretability tool and as a surrogate for explaining representations learned via unsupervised machine learning, enabling the identification of physically grounded correlations. We compare attention-derived interpretability scores with classical SHapley Additive exPlanations analysis and show that, in contrast to applications in natural language processing, attention mechanisms in materials science exhibit high efficiency in highlighting meaningful structural features.

Slautin, Boris [Independent Researcher]↗
DOE OSTI2022

Data-centric machine learning in quantum information science

Abstract We propose a series of data-centric heuristics for improving the performance of machine learning systems when applied to problems in quantum information science. In particular, we consider how systematic engineering of training sets can significantly enhance the accuracy of pre-trained neural networks used for quantum state reconstruction without altering the underlying architecture. We find that it is not always optimal to engineer training sets to exactly match the expected distribution of a target scenario, and instead, performance can be further improved by biasing the training set to be slightly more mixed than the target. This is due to the heterogeneity in the number of free variables required to describe states of different purity, and as a result, overall accuracy of the network improves when training sets of a fixed size focus on states with the least constrained free variables. For further clarity, we also include a ‘toy model’ demonstration of how spurious correlations can inadvertently enter synthetic data sets used for training, how the performance of systems trained with these correlations can degrade dramatically, and how the inclusion of even relatively few counterexamples can effectively remedy such problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients

Mortality prediction for intensive care unit (ICU) patients is crucial for improving outcomes and efficient utilization of resources. Accessibility of electronic health records (EHR) has enabled data-driven predictive modeling using machine learning. However, very few studies rely solely on unstructured clinical notes from the EHR for mortality prediction. In this work, we propose a framework to predict short, mid, and long-term mortality in adult ICU patients using unstructured clinical notes from the MIMIC III database, natural language processing (NLP), and machine learning (ML) models. Depending on the statistical description of the patients’ length of stay, we define the short-term as 48-hour and 4-day period, the mid-term as 7-day and 10-day period, and the long-term as 15-day and 30-day period after admission. We found that by only using clinical notes within the 24 hours of admission, our framework can achieve a high area under the receiver operating characteristics (AU-ROC) score for short, mid and long-term mortality prediction tasks. The test AU-ROC scores are 0.87, 0.83, 0.83, 0.82, 0.82, and 0.82 for 48-hour, 4-day, 7-day, 10-day, 15-day, and 30-day period mortality prediction, respectively. We also provide a comparative study among three types of feature extraction techniques from NLP: frequency-based technique, fixed embedding-based technique, and dynamic embedding-based technique. Lastly, we provide an interpretation of the NLP-based predictive models using feature-importance scores.

60 APPLIED LIFE SCIENCES↗