Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning in Network Science”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Developing a simulator-based satellite dataset for using machine learning techniques to derive aerosol-cloud-precipitation interactions in models and observations in a consistent framework

Aerosol-cloud-precipitation interactions (ACPI) remain a major uncertainty in understanding the Earth’s radiation budget and water cycle (including extremes). After decades of active research, various observationally based metrics have been developed to constrain ACPI in Earth System Models (ESMs), but direct comparison of model and data estimates can confound scientific understanding because limitations and uncertainties in sampling and retrieval procedures may combine with model deficiencies in process representations of ACPI to obstruct understanding. Furthermore, conventional ACPI metrics often vary from one regime to another, and the ACPI process representation in ESMs is also typically derived based on only a limited area/regime (even though the parameterization applies globally). To bridge the gap between models and data and to correctly describe ACPI in all regimes, we propose to construct a new CALIPSO-CloudSat merged dataset that is produced by the same algorithms used in satellite simulators in ESMs, and to use machine learning techniques to derive new ACPI metrics that can be accurately estimated by satellites and can provide meaningful constraints on cloud microphysical process representations in ESMs. The dataset will include measured and retrieved variables for aerosol, cloud, and precipitation from CALIPSO and CloudSat, and environmental variables from meteorological reanalysis. The data will be used to train a neural network to construct the ACPI metrics as a function of environmental conditions. The new ACPI formula will be used to constrain the ACPI in the Energy Exascale Earth System Model (E3SM), and to augment/reformulate the ACPI process representation in the E3SM to improve the simulation of the evolution of the atmosphere under different environmental conditions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Accurate Thermochemistry of Complex Lignin Structures via Density Functional Theory, Group Additivity, and Machine Learning

A molecular-level understanding of lignin structures and bond dissociation energies could facilitate depolymerization technologies. Still, this information is currently limited due to the lack of databases and the simplification of surrogate models. Here, substitution effects on seven common linkages in lignin polymers are systematically investigated. An automated reaction network generator is employed to create a database of structures. A new group additivity (GA) model based on principal component analysis (PCA) descriptors is introduced and trained on gas-phase density functional theory data of 4100 species at the M06-2X/6-311++G(d,p) level. Hydrogen bonds, local steric, and nonaromatic ring contributions are also incorporated. Lastly, we improve the accuracy of the group additivity model to reach the G4 theory by computing a data set of 770 species at this level and using a data fusion approach.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Deep learning for NLTE spectral opacities

Computer simulations of high energy density science experiments are computationally challenging, consisting of multiple physics calculations including radiation transport, hydrodynamics, atomic physics, nuclear reactions, laser–plasma interactions, and more. To simulate inertial confinement fusion (ICF) experiments at high fidelity, each of these physics calculations should be as detailed as possible. However, this quickly becomes too computationally expensive even for modern supercomputers, and thus many simplifying assumptions are made to reduce the required computational time. Much of the research has focused on acceleration techniques for the various packages in multiphysics codes. In this work, we explore a novel method for accelerating physics packages via machine learning. The non-local thermodynamic equilibrium (NLTE) package is one of the most expensive calculations in the simulations of indirect drive inertial confinement fusion, taking several tens of percent of the total wall clock time. We explore the use of machine learning to accelerate this package, by essentially replacing the physics calculation with a deep neural network that has been trained to emulate the physics code. Overall, we demonstrate the feasibility of this approach on a simple problem and perform a side-by-side comparison of the physics calculation and the neural network inline in an ICF Hohlraum simulation. We show that the neural network achieves a 10× speed up in NLTE computational time while achieving good agreement with the physics code for several quantities of interest.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Deep Learning for In-Situ Layer Quality Monitoring during Laser-Based Directed Energy Deposition (LB-DED) Additive Manufacturing Process

Defects are a leading issue for the rejection of parts manufactured through the Directed Energy Deposition (DED) Additive Manufacturing (AM) process. In an attempt to illuminate and advance in situ quality monitoring and control of workpieces, we present an innovative data-driven method that synchronously collects sensing data and AM process parameters with a low sampling rate during the DED process. The proposed data-driven technique determines the important influences that individual printing parameters and sensing features have on prediction at the inter-layer qualification to perform feature selection. Three Machine Learning (ML) algorithms including Random Forest (RF), Support Vector Machine (SVM), and Convolutional Neural Network (CNN) are used. During post-production, a threshold is applied to detect low-density occurrences such as porosity sizes and quantities from CT scans that render individual layers acceptable or unacceptable. This information is fed to the ML models for training. Training/testing are completed offline on samples deemed “high-quality” and “low-quality”, utilizing only features recorded from the build process. CNN results show that the classification of acceptable/unacceptable layers can reach between 90% accuracy while training/testing on a “high-quality” sample and dip to 65% accuracy when trained/tested on “low-quality”/“high-quality” (respectively), indicating over-fitting but showing CNN as a promising inter-layer classifier.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Protein sequence design with a learned potential

The task of protein sequence design is central to nearly all rational protein engineering problems, and enormous effort has gone into the development of energy functions to guide design. Here, we investigate the capability of a deep neural network model to automate design of sequences onto protein backbones, having learned directly from crystal structure data and without any human-specified priors. The model generalizes to native topologies not seen during training, producing experimentally stable designs. We evaluate the generalizability of our method to a de novo TIM-barrel scaffold. The model produces novel sequences, and high-resolution crystal structures of two designs show excellent agreement with in silico models. Our findings demonstrate the tractability of an entirely learned method for protein sequence design.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Graph Neural Networks for Particle Reconstruction in High Energy Physics detectors

Pattern recognition problems in high energy physics are notably different from traditional machine learning applications in computer vision. Reconstruction algorithms identify and measure the kinematic properties of particles produced in high energy collisions and recorded with complex detector systems. Two critical applications are the reconstruction of charged particle trajectories in tracking detectors and the reconstruction of particle showers in calorimeters. These two problems have unique challenges and characteristics, but both have high dimensionality, high degree of sparsity, and complex geometric layouts. Graph Neural Networks (GNNs) are a relatively new class of deep learning architectures which can deal with such data effectively, allowing scientists to incorporate domain knowledge in a graph structure and learn powerful representations leveraging that structure to identify patterns of interest. In this work we demonstrate the applicability of GNNs to these two diverse particle reconstruction problems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

A robust synthetic data generation framework for machine learning in high-resolution transmission electron microscopy (HRTEM)

Machine learning techniques are attractive options for developing highly-accurate analysis tools for nanomaterials characterization, including high-resolution transmission electron microscopy (HRTEM). However, successfully implementing such machine learning tools can be difficult due to the challenges in procuring sufficiently large, high-quality training datasets from experiments. In this work, we introduce Construction Zone, a Python package for rapid generation of complex nanoscale atomic structures which enables fast, systematic sampling of realistic nanomaterial structures and can be used as a random structure generator for large, diverse synthetic datasets. Using Construction Zone, we develop an end-to-end machine learning workflow for training neural network models to analyze experimental atomic resolution HRTEM images on the task of nanoparticle image segmentation purely with simulated databases. Further, we study the data curation process to understand how various aspects of the curated simulated data—including simulation fidelity, the distribution of atomic structures, and the distribution of imaging conditions—affect model performance across three benchmark experimental HRTEM image datasets. Using our workflow, we are able to achieve state-of-the-art segmentation performance on these experimental benchmarks and, further, we discuss robust strategies for consistently achieving high performance with machine learning in experimental settings using purely synthetic data. Construction Zone and its documentation are available at https://github.com/lerandc/construction_zone.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Single Cell RNA-Seq and Machine Learning Reveal Novel Subpopulations in Low-Grade Inflammatory Monocytes With Unique Regulatory Circuits

Subclinical doses of LPS (SD-LPS) are known to cause low-grade inflammatory activation of monocytes, which could lead to inflammatory diseases including atherosclerosis and metabolic syndrome. Sodium 4-phenylbutyrate is a potential therapeutic compound which can reduce the inflammation caused by SD-LPS. To understand the gene regulatory networks of these processes, we have generated scRNA-seq data from mouse monocytes treated with these compounds and identified 11 novel cell clusters. We have developed a machine learning method to integrate scRNA-seq, ATAC-seq, and binding motifs to characterize gene regulatory networks underlying these cell clusters. Using guided regularized random forest and feature selection, our method achieved high performance and outperformed a traditional enrichment-based method in selecting candidate regulatory genes. Our method is particularly efficient in selecting a few candidate genes to explain observed expression pattern. In particular, among 531 candidate TFs, our method achieves an auROC of 0.961 with only 10 motifs. Finally, we found two novel subpopulations of monocyte cells in response to SD-LPS and we confirmed our analysis using independent flow cytometry experiments. Our results suggest that our new machine learning method can select candidate regulatory genes as potential targets for developing new therapeutics against low grade inflammation.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Exploring far-from-equilibrium ultrafast polarization control in ferroelectric oxides with excited-state neural network quantum molecular dynamics

Ferroelectric materials exhibit a rich range of complex polar topologies, but their study under far-from-equilibrium optical excitation has been largely unexplored because of the difficulty in modeling the multiple spatiotemporal scales involved quantum-mechanically. To study optical excitation at spatiotemporal scales where these topologies emerge, we have performed multiscale excited-state neural network quantum molecular dynamics simulations that integrate quantum-mechanical description of electronic excitation and billion-atom machine learning molecular dynamics to describe ultrafast polarization control in an archetypal ferroelectric oxide, lead titanate. Far-from-equilibrium quantum simulations reveal a marked photo-induced change in the electronic energy landscape and resulting cross-over from ferroelectric to octahedral tilting topological dynamics within picoseconds. The coupling and frustration of these dynamics, in turn, create topological defects in the form of polar strings. The demonstrated nexus of multiscale quantum simulation and machine learning will boost not only the emerging field of ferroelectric topotronics but also broader optoelectronic applications.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Rapid Inference of Logic Gate Neural Networks for Anomaly Detection in High Energy Physics

The increasing data rates and complexity of detectors at the Large Hadron Collider (LHC) necessitate fast and efficient machine learning models, particularly for rapid selection of what data to store, known as triggering. Building on recent work in differentiable logic gates, we present a public implementation of a Convolutional Differentiable Logic Gate Neural Network (CLGN). We apply this to detecting anomalies at the Level-1 Trigger at CMS using public data from the CICADA project. We demonstrate that the CLGN achieves physics performance on par with or superior to conventional quantized neural networks. We also synthesize an LGN for a Field-Programmable Gate Array (FPGA) and show highly promising FPGA characteristics, notably zero Digital Signal Processor (DSP) resource usage. This work highlights the potential of logic gate networks for high-speed, on-detector inference in High Energy Physics and beyond.

FOS: Physical sciences↗
DOE OSTI2024

Deep-Learning Interatomic Potential Connects Molecular Structural Ordering to the Macroscale Properties of Polyacrylonitrile

Polyacrylonitrile (PAN) is an important commercial polymer, bearing atactic stereochemistry resulting from nonselective radical polymerization. As such, an accurate, fundamental understanding of governing interactions among PAN molecular units is indispensable for advancing the design principles of final products at reduced processability costs. While ab initio molecular dynamics (AIMD) simulations can provide the necessary accuracy for treating key interactions in polar polymers, such as dipole–dipole interactions and hydrogen bonding, and analyzing their influence on the molecular orientation, their implementation is limited to small molecules only. Herein, we show that the neural network interatomic potentials (NNIPs) that are trained on the small-scale AIMD data (acquired for oligomers) can be efficiently employed to examine the structures and properties at large scales (polymers). NNIP provides critical insight into intra- and interchain hydrogen-bonding and dipolar correlations and accurately predicts the amorphous bulk PAN structure validated by modeling the experimental X-ray structure factor. Furthermore, the NNIP-predicted PAN properties, such as density and elastic modulus, are in good agreement with their experimental values. Overall, the trend in the elastic modulus is found to correlate strongly with the PAN structural orientations encoded in the Hermans orientation factor. In conclusion, this study enables the ability to predict the structure–property relations for PAN and analogues with sustainable ab initio accuracy across scales.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Embedding hard physical constraints in neural network coarse-graining of three-dimensional turbulence

In recent years, deep learning approaches have shown much promise in modeling complex systems in the physical sciences. A major challenge in deep learning of partial differential equations is enforcing physical constraints and boundary conditions. In this work, we propose a general framework to directly embed the notion of an incompressible fluid into convolutional neural networks, and apply this to coarse-graining of turbulent flow. These physics-embedded neural networks leverage interpretable strategies from numerical methods and computational fluid dynamics to enforce physical laws and boundary conditions by taking advantage the mathematical properties of the underlying equations. Here, we demonstrate results on three-dimensional fully developed turbulence, showing that this technique drastically improves local conservation of mass, without sacrificing performance according to several other metrics characterizing the fluid flow.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Pulsed Thermal Tomography Nondestructive Examination of Additively Manufactured Reactor Materials and Components. Third Annual Progress Report

Additive manufacturing (AM) of high-strength corrosion resistance alloys for nuclear energy applications, such as stainless steel and Inconel, is currently based on laser powder bed fusion (LPBF) process. Some of the challenges with using LPBF method for nuclear manufacturing include the possibility of introducing pores into metallic structures. Probability of crack initiation at the pore depends on size, shape, and orientation of the defect. Pulsed Infrared Thermography Imaging (PIT) provides a capability for non-destructive evaluation (NDE) of sub-surface defects in arbitrary size structures. The PIT method is based on recording material surface temperature transients with infrared (IR) camera following thermal pulse delivered on material surface with flash light. The PIT method has advantages for NDE of actual AM structures because the method involves one-sided non-contact measurements and fast processing of large sample areas captured in one image. Following initial qualification of an AM component for deployment in a nuclear reactor, a PIT system can also be used for in-service nondestructive evaluation (NDE) applications. In this report, we describe recent progress in enhancing PIT capabilities in detecting microscopic subsurface defects in metals, and classifying shapes and orientation of pores in thermal images. For detection of microscopic defects in PIT imaging data, we have developed Spatial Temporal Denoised Thermal Source Separation (STDTSS) unsupervised machine learning (ML) image processing algorithm. We show that flat bottom hole (FBH) defects as small as 200µm in SS316 and IN718 specimens, can be detected with STDTSS algorithm. To the best of our knowledge, these are the smallest detected defects which are reported in literature. For classification of defects shapes, we have previously developed thermal tomography (TT) algorithm to obtain depth reconstructions of material defects from data cube of sequentially recorded surface temperatures. However, interpretation of TT images is non-trivial because of blurring with increasing depth. To address this challenge, we have developed a deep learning convolutional neural network (CNN) to classify size and orientation subsurface defects in simulated TT images.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Multi-Task Learning of Scanning Electron Microscopy and Synthetic Thermal Tomography Images for Detection of Defects in Additively Manufactured Metals

One of the key challenges in laser powder bed fusion (LPBF) additive manufacturing of metals is the appearance of microscopic pores in 3D-printed metallic structures. Quality control in LPBF can be accomplished with non-destructive imaging of the actual 3D-printed structures. Thermal tomography (TT) is a promising non-contact, non-destructive imaging method, which allows for the visualization of subsurface defects in arbitrary-sized metallic structures. However, because imaging is based on heat diffusion, TT images suffer from blurring, which increases with depth. We have been investigating the enhancement of TT imaging capability using machine learning. In this work, we introduce a novel multi-task learning (MTL) approach, which simultaneously performs the classification of synthetic TT images, and segmentation of experimental scanning electron microscopy (SEM) images. Synthetic TT images are obtained from computer simulations of metallic structures with subsurface elliptical-shaped defects, while experimental SEM images are obtained from imaging of LPBF-printed stainless-steel coupons. MTL network is implemented as a shared U-net encoder between the classification and the segmentation tasks. Results of this study show that the MTL network performs better in both the classification of synthetic TT images and the segmentation of SEM images tasks, as compared to the conventional approach when the individual tasks are performed independently of each other.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Deep learning approaches for neural decoding across architectures and recording modalities

Decoding behavior, perception or cognitive state directly from neural signals is critical for brain–computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. Here, we describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Journey over Destination: Dynamic Sensor Placement Enhances Generalization

Reconstructing complex, high-dimensional global fields from limited data points is a challenge across various scientific and industrial domains. This is particularly important for recovering spatio-temporal fields using sensor data from, for example, laboratory-based scientific experiments, weather forecasting, or drone surveys. Given the prohibitive costs of specialized sensors and the inaccessibility of
certain regions of the domain, achieving full field coverage is typically not feasible. Therefore, the development of machine learning algorithms trained to reconstruct fields given a limited dataset is of critical importance. In this study, we introduce a general
approach that employs moving sensors to enhance data exploitation during the training of an attention based neural network, thereby improving field reconstruction. The training of sensor locations is accomplished using an end-to-end workflow, ensuring
differentiability in the interpolation of field values associated to the sensors, and is simple to implement using differentiable programming. Additionally, we have incorporated a correction mechanism to prevent sensors from entering invalid regions within the domain. We evaluated our method using two distinct datasets; the results show that our approach enhances learning, as evidenced by improved test scores.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Automated Segmentation of Porous Thermal Spray Material CT Scans with Geometric Uncertainty Estimation

Thermal sprayed metal coatings are used in many industrial applications, and characterizing the structure and performance of these materials is vital to understanding their behavior in the field. X-ray Computed Tomography (CT) machines enable volumetric, nondestructive imaging of these materials, but precise segmentation of this grayscale image data into discrete material phases is necessary to calculate quantities of interest related to material structure. In this work, we present a methodology to automate the CT segmentation process as well as quantify uncertainty in segmentations via deep learning. Neural networks (NNs) are shown to accurately segment full resolution CT scans of thermal sprayed materials and provide maps of uncertainty that conservatively bound the predicted geometry. These bounds are propagated through calculations of material properties such as porosity that may provide an understanding of anticipated behavior in the field.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Review of internal cyber attacks in nuclear facilities and an artificial neural network model for implementing internal cyberforensics

Deployment of digital technologies within a modern shift in cyber defense systems is essential for protecting the energy production units. One of the important components of defense is cyberforensics: once an attack has been detected to locate its origin. In this paper, a review of well-known cyberattacks in nuclear facilities is provided, with the lessons learned leading to the development of a machine learning approach implementing identification of internal at- tacks in the facility's data networks. Our approach may be seen as one of the layers in a defense-in-depth strategy that identifies if the attack comes from inside, which may result in identifying faster the attacker's origin. The presented model exploits network packet examination to cast accurate predictions on detailing the origin of malicious network connections. The approach fuses multiple mathematical functions within an artificial neural network to provide a response in the form of 0/1, i. e., whether the attack is identified as internal or not. The utilization of a variety of test cases is developed to explore the relevance and validity of the predictive approach. The proposed implementation is examined with network data packet variance, and the results obtained exhibit a highly accurate detection rate.

Nuclear Science & Technology↗