Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Distributed fiber sensor and machine learning data analytics for pipeline protection against extrinsic intrusions and intrinsic corrosions

This paper presents an integrated technical framework to protect pipelines against both malicious intrusions and piping degradation using a distributed fiber sensing technology and artificial intelligence. A distributed acoustic sensing (DAS) system based on phase-sensitive optical time-domain reflectometry (φ-OTDR) was used to detect acoustic wave propagation and scattering along pipeline structures consisting of straight piping and sharp bend elbow. Signal to noise ratio of the DAS system was enhanced by femtosecond induced artificial Rayleigh scattering centers. Data harnessed by the DAS system were analyzed by neural network-based machine learning algorithms. The system identified with over 85% accuracy in various external impact events, and over 94% accuracy for defect identification through supervised learning and 71% accuracy through unsupervised learning.

Peng, Zhaoqiang↗
DOE OSTI2022

Python Codebase and Jupyter Notebooks - Applications of Machine Learning Techniques to Geothermal Play Fairway Analysis in the Great Basin Region, Nevada

Git archive containing Python modules and resources used to generate machine-learning models used in the "Applications of Machine Learning Techniques to Geothermal Play Fairway Analysis in the Great Basin Region, Nevada" project. This software is licensed as free to use, modify, and distribute with attribution. Full license details are included within the archive. See "documentation.zip" for setup instructions and file trees annotated with module descriptions.

Brown, Stephen↗
DOE OSTI2024

Real-Time Neuromorphic Processing of Spatiotemporal Data for Scientific Discovery

Spiking Neural Networks (SNNs) are brain-inspired computing models incorporating unique temporal dynamics and event-driven processing. Rich dynamics in both space and time offer great challenges and opportunities for efficient processing of sparse spatiotemporal data compared with conventional artificial neural networks (ANNs). Under this context, the goal of this project is to develop spiking neural network based neuromorphic computing to enable energy-efficient real-time learning and processing of spatiotemporal data. This report summarizes the key results on network architecture design, training methods, and SNN hardware acceleration achieved under this project, demonstrating the promise of spiking neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

cTULIP: application of a human-based RNA-seq primary tumor classification tool for cross-species primary tumor classification in canine

The domestic dog, Canis familiaris, is quickly gaining traction as an advantageous model for use in the study of cancer, one of the leading causes of death worldwide. Naturally occurring canine cancers share clinical, histological, and molecular characteristics with the corresponding human diseases. In this study, we take a deep-learning approach to test how similar the gene expression profile of canine glioma and bladder cancer (BLCA) tumors are to the corresponding human tumors. We likewise develop a tool for identifying misclassified or outlier samples in large canine oncological datasets, analogous to that which was developed for human datasets. We test a number of machine learning algorithms and found that a convolutional neural network outperformed logistic regression and random forest approaches. We use a recently developed RNA-seq-based convolutional neural network, TULIP, to test the robustness of a human-data-trained primary tumor classification tool on cross-species primary tumor prediction. Our study ultimately highlights the molecular similarities between canine and human BLCA and glioma tumors, showing that protein-coding one-to-one homologs shared between humans and canines, are sufficient to distinguish between BLCA and gliomas. The results of this study indicate that using protein-coding one-to-one homologs as the features in the input layer of TULIP performs good primary tumor prediction in both humans and canines. Furthermore, our analysis shows that our selected features also contain the majority of features with known clinical relevance in BLCA and gliomas. Our success in using a human-data-trained model for cross-species primary tumor prediction also sheds light on the conservation of oncological pathways in humans and canines, further underscoring the importance of the canine model system in the study of human disease.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Deep Learning for Land Cover Classification Using Only a Few Bands

There is an emerging interest in using hyperspectral data for land cover classification. The motivation behind using hyperspectral data is the notion that increasing the number of narrowband spectral channels would provide richer spectral information and thus help improve the land cover classification performance. Although hyperspectral data with hundreds of channels provide detailed spectral signatures, the curse of dimensionality might lead to degradation in the land cover classification performance. Moreover, in some practical applications, hyperspectral data may not be available due to cost, data storage, or bandwidth issues, and RGB and near infrared (NIR) could be the only image bands available for land cover classification. Light detection and ranging (LiDAR) data is another type of data to assist land cover classification especially if the land covers of interest have different heights. In this paper, we examined the performance of two Convolutional Neural Network (CNN)-based deep learning algorithms for land cover classification using only four bands (RGB+NIR) and five bands (RGB+NIR+LiDAR), where these limited number of image bands were augmented using Extended Multi-attribute Profiles (EMAP). The deep learning algorithms were applied to a well-known dataset used in the 2013 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion Contest. With EMAP augmentation, the two deep learning algorithms were observed to achieve better land cover classification performance using only four bands as compared to that using all 144 hyperspectral bands.

58 GEOSCIENCES↗
DOE OSTI2022

Device-Centric Ransomware Detection using Machine Learning-Based Memory Forensics for Smart Inverters

Ransomware attacks are the fastest-growing form of cyberattacks worldwide. Recently, ransomware attacks have targeted industrial control systems (ICSs), including power grids. Lessons learned from recent incidents in ICSs show that ransomware groups can deliver ransomware into not only the organization’s control servers, but also the operational technology (OT) devices such as smart inverters and smart grid devices. This paper proposes a machine learning (ML)- based memory forensics method enabling the detection of ransomware binaries stored in the memory of a commercial smart inverter. Device firmware binary files are extracted from a Serial Peripheral Interface (SPI) flash memory, and samples of both benign and ransomware binaries are generated by a binary manipulation method and a real-world ransomware encryption, separately. A deep transfer learning (DTL) method is used to retrain a convolutional neural network (CNN)-based ransomware detection algorithm using the generated samples. The experimental result validates that the proposed ML-based memory forensics method can accurately detect ransomware files.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A Deep Neural Network for Simultaneous Estimation of b Jet Energy and Resolution

We describe a method to obtain point and dispersion estimates for the energies of jets arising from b quarks produced in proton–proton collisions at an energy of $\sqrt{s}=13\,\text {TeV} $ at the CERN LHC. The algorithm is trained on a large sample of simulated b jets and validated on data recorded by the CMS detector in 2017 corresponding to an integrated luminosity of 41 $\,\text {fb}^{-1}$. A multivariate regression algorithm based on a deep feed-forward neural network employs jet composition and shape information, and the properties of reconstructed secondary vertices associated with the jet. The results of the algorithm are used to improve the sensitivity of analyses that make use of b jets in the final state, such as the observation of Higgs boson decay to $\hbox {b}\bar{\hbox {b}}$.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Wire-cell 3D pattern recognition techniques for neutrino event reconstruction in large LArTPCs: algorithm description and quantitative evaluation with MicroBooNE simulation

Wire-Cell is a 3D event reconstruction package for liquid argon time projection chambers. Through geometry, time, and drifted charge from multiple readout wire planes, 3D space points with associated charge are reconstructed prior to the pattern recognition stage. Pattern recognition techniques, including track trajectory and dQ/dx (ionization charge per unit length) fitting, 3D neutrino vertex fitting, track and shower separation, particle-level clustering, and particle identification are then applied on these 3D space points as well as the original 2D projection measurements. A deep neural network is developed to enhance the reconstruction of the neutrino interaction vertex. Compared to traditional algorithms, the deep neural network boosts the vertex efficiency by a relative 30% for charged-current ν e interactions. Therefore, this pattern recognition achieves 80–90% reconstruction efficiencies for primary leptons, after a 65.8% (72.9%) vertex efficiency for charged-current ν e (ν μ ) interactions. Based on the resulting reconstructed particles and their kinematics, we also achieve 15-20% energy reconstruction resolutions for charged-current neutrino interactions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Dynamic Model Agnostic Reliability Evaluation of Machine-Learning Models Integrated in Instrumentation & Control Systems

In recent years, the field of data-driven neural network-based machine learning (ML) algorithms has grown significantly and spurred research in its applicability to instrumentation and control systems. While they are promising in operational contexts, the trustworthiness of such algorithms is not adequately assessed. Failures of ML-integrated systems are poorly understood; the lack of comprehensive risk modeling can degrade the trustworthiness of these systems. In recent reports by the National Institute for Standards and Technology, trustworthiness in ML is a critical barrier to adoption and will play a vital role in intelligent systems' safe and accountable operation. Thus, in this work, we demonstrate a real-time model-agnostic method to evaluate the relative reliability of ML predictions by incorporating out-of-distribution detection on the training dataset. It is well documented that ML algorithms excel at interpolation (or near-interpolation) tasks but significantly degrade at extrapolation. This occurs when new samples are "far" from training samples. The method, referred to as the Laplacian distributed decay for reliability (LADDR), determines the difference between the operational and training datasets, which is used to calculate a prediction's relative reliability. LADDR is demonstrated on a feedforward neural network-based model used to predict safety significant factors during different loss-of-flow transients. LADDR is intended as a "data supervisor" and determines the appropriateness of well-trained ML models in the context of operational conditions. Ultimately, LADDR illustrates how training data can be used as evidence to support the trustworthiness of ML predictions when utilized for conventional interpolation tasks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Stanford Thermal Earth Model for the Conterminous United States

Provided here are various forms of the Stanford Thermal Earth Model, as well as the data and methods used for its creation. The predictions produced by this model were visualized in two-dimensional spatial maps across the modeled depths (0-7 km) for the conterminous United States. The thermal earth model is made available as an application programming interface (API) and as feature layers on ArcGIS, which are both provided via links below. A data-driven spatial interpolation algorithm based on physics-informed graph neural networks was used to develop these national temperature-at-depth maps. The model satisfied the three-dimensional heat conduction law by predicting subsurface temperature, surface heat flow, and rock thermal conductivity. Many physical quantities, including bottomhole temperature, depth, geographic coordinates, elevation, sediment thickness, magnetic anomaly, gravity anomaly, gamma-ray flux of radioactive elements, seismicity, and electric conductivity were used as model inputs. Surface heat flow, temperature, and thermal conductivity predictions were constructed for depths of 0-7 km at an interval of 1 km with spatial resolution of 18 km2 per grid cell. The model showed superior temperature, surface heat flow and thermal conductivity mean absolute errors of 4.8C, 8.1 mW/m2 and 0.07 W/(C-m), respectively..

15 GEOTHERMAL ENERGY↗
DOE OSTI2021

Convolutional Neural Networks for the CHIPS Neutrino Detector R&D Project.

The CHerenkov detectors In mine PitS (Chips) neutrino detector R&D project aims to develop novel strategies and technologies for very large yet ‘cheap as chips’ water Cherenkov neutrino detectors. Via deployment in a body of water, use of commercially available components, and instrumentation coverage optimisation for the study of exclusively accelerator beam neutrinos, Chips will enable megaton scale detectors to become a reality at the cost of $200k-$300k per kt of sensitive mass. During the summer of 2019 a prototype Chips detector, Chips-5, was deployed into the Wentworth 2W disused mine pit in northern Minnesota, 7 mrad off the NuMI beam axis. A novel data acquisition system was introduced using cheap single-board computers and open-source software. This work presents a novel approach to water Cherenkov neutrino detector event reconstruction and classification. Three forms of a Convolutional Neural Network, a type of deep learning algorithm, have been trained to reject cosmic muon events, classify beam events, and estimate neutrino energies, all using only the raw detector event as input. When evaluated on the expected distribution of Chips-5 events, this new approach is shown to be robust and explainable as well as providing a significant performance increase over the standard likelihood-based reconstruction and simple neural network classification. Promisingly, the performance presented here is comparable to the more complex (and expensive) neutrino oscillation experiments within the field.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Uncertainty quantification and propagation in lithium-ion battery electrodes using bayesian convolutional neural networks

The complex nature of manufacturing processes stipulates electrodes to possess high variability with increased heterogeneity during production. X-ray computed tomography imaging has proved to be critical in visualizing the complicated stochastic particle distribution of as-manufactured electrodes in lithium-ion batteries. However, accurate prediction of their electrochemical performance necessitates precise evaluation of kinetic and transport properties from real electrodes. Image segmentation that characterizes voxels to particle/pore phase is often meticulous and fraught with subjectivity owing to a myriad of unconstrained choices and filter algorithms. Here we utilize a Bayesian convolutional neural network to tackle segmentation subjectivity and quantify its pertinent uncertainties. Otsu inter-variance and Blind/Referenceless Imaging Spatial Quality Evaluator are used to assess the relative image quality of grayscale tomograms, thus evaluating the uncertainty in the derived microstructural attributes. We analyze how image uncertainty is correlated with the uncertainties and magnitude of kinetic and transport properties of an electrode, further identifying pathways of uncertainty propagation within microstructural attributes. The coupled effect of spatial heterogeneity and microstructural anisotropy on the uncertainty quantification of transport parameters is also understood. This work demonstrates a novel methodology to extract microstructural descriptors from real electrode images through quantification of associated uncertainties and discerning the relative strength of their propagation, thus facilitating feedback to manufacturing processes from accurate image based electrochemical simulations.

25 ENERGY STORAGE↗
DOE OSTI2022

Towards a more general understanding of the algorithmic utility of recurrent connections

Lateral and recurrent connections are ubiquitous in biological neural circuits. Yet while the strong computational abilities of feedforward networks have been extensively studied, our understanding of the role and advantages of recurrent computations that might explain their prevalence remains an important open challenge. Foundational studies by Minsky and Roelfsema argued that computations that require propagation of global information for local computation to take place would particularly benefit from the sequential, parallel nature of processing in recurrent networks. Such “tag propagation” algorithms perform repeated, local propagation of information and were originally introduced in the context of detecting connectedness, a task that is challenging for feedforward networks. Here, we advance the understanding of the utility of lateral and recurrent computation by first performing a large-scale empirical study of neural architectures for the computation of connectedness to explore feedforward solutions more fully and establish robustly the importance of recurrent architectures. In addition, we highlight a tradeoff between computation time and performance and construct hybrid feedforward/recurrent models that perform well even in the presence of varying computational time limitations. We then generalize tag propagation architectures to propagating multiple interacting tags and demonstrate that these are efficient computational substrates for more general computations of connectedness by introducing and solving an abstracted biologically inspired decision-making task. Our work thus clarifies and expands the set of computational tasks that can be solved efficiently by recurrent computation, yielding hypotheses for structure in population activity that may be present in such tasks.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Continual Learning for Pattern Recognizers using Neurogenesis Deep Learning

Deep neural networks have emerged as a leading set of algorithms to infer information from a variety of data sources such as images and time series data. In their most basic form, neural networks lack the ability to adapt to new classes of information. Continual learning is a field of study attempting to give previously trained deep learning models the ability to adapt to a changing environment. Previous work developed a CL method called Neurogenesis for Deep Learning (NDL). Here, we combine NDL with a specific neural network architecture (the Ladder Network) to produce a system capable of automatically adapting a classification neural network to new classes of data. The NDL Ladder Network was evaluated against other leading CL methods. While the NDL and Ladder Network system did not match the cutting edge performance achieved by other CL methods, in most cases it performed comparably and is the only system evaluated that can learn new classes of information with no human intervention.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Imaging of Calibrated Defects in Additively Manufactured Materials: Pulsed Thermal Tomography Nondestructive Examination of Additively Manufactured Reactor Materials and Components

Additive manufacturing (AM, or 3D printing) for commercial nuclear energy applications is an emerging method for cost-efficient manufacturing aimed at replacing aging nuclear reactor parts and reducing costs for new construction. Known material flaws in AM include low-density regions consisting of non-sintered powder, which have to be detected to ensure the safety of long-term performance nuclear reactor components. Currently, limited options are available for nondestructive evaluation (NDE), either during or post manufacturing. As a solution to NDE of AM, we are developing pulsed thermal imaging which is non-contact, one-sided, and scalable to arbitrary size and shape of the AM parts. Pulsed thermography (PT) system utilizes a high intensity flash lamp to rapidly heat surface of sample, and a high sensitivity fast-frame megapixel infrared (IR) camera to capture data of surface temperature variations. The acquired data cube consists of a stack of surface temperature images taken at different times. Information about material internal defects is extracted by analyzing the data cube. This report provides results of preliminary performance evaluation of pulsed thermal imaging capability in detection of imprinted flaws in AM metallic structures. The flaws were introduced into AM parts as imprinted hemispherical low density regions, consisting of trapped un-sintered metallic powder. Specimens for developed for this study consisted of AM stainless steel 316 and Inconel 718 plates. The diameters of imprinted defects varied from 1mm to 8mm, and their depths below the plate flat surface varied between 1mm and 6mm. Pulsed thermal tomography (PTT) processes the measured data cube to obtain 3D reconstructions of material effusivity using a unique inversion algorithm developed at Argonne. PTT has been previously used in imaging of similar size flat bottom hole (FBH) simulated defects in stainless steel 316 and Inconel 718 specimens. In the study involving AM specimens, PTT imaging results have shown that 1mm-diameter defects located 1mm and 2mm below the surface of specimens were detectable. Larger size defects were detectable at greater depth. We also explored an alternative approach to detection of material flaws in PT data cube, which is using neural learning-based approach to blind source separation. Detection of small material defects requires finding features in the data cube which have signal contrast levels approaching sensitivity limit of IR camera. In this study, an optimized Neural Learning based Blind Source Separation (NLBSS) algorithm, including Principal Component Analysis (PCA), and Independent Component Analysis (ICA) is demonstrated to automatically extract principal temporal and spatial features of thermography frames to enhance flaw detection. By using the NLBSS algorithm, material internal defects can be automatically detected. Furthermore, this processing approach compensates for experimental thermal imaging artifacts, such as noise and uneven heating. By merging artificial intelligence with phtotothermics, the NDE system detects internal calibrated defects of various sizes and depths in AM nuclear-grade metallic alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2022

A Decision-Making Machine Learning Approach in Hermite Spectral Approximations of Partial Differential Equations

The accuracy and effectiveness of Hermite spectral methods for the numerical discretization of partial differential equations on unbounded domains are strongly affected by the amplitude of the Gaussian weight function employed to describe the approximation space. This is particularly true if the problem is under-resolved, i.e., there are no enough degrees of freedom. The issue becomes even more crucial when the equation under study is time-dependent, forcing in this way the choice of Hermite functions where the corresponding weight depends on time. In order to adapt dynamically the approximation space, it is here proposed an automatic decision-making process that relies on machine learning techniques, such as deep neural networks and support vector machines. The algorithm is numerically tested with success on a simple 1D problem, but the main goal is its exportability in the context of more serious applications. Here we also show at the end an application in the framework of plasma physics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Optimizing temperature distributions for training neural quantum states using parallel tempering

Parametrized artificial neural networks (ANNs) can be very expressive ansatzes for variational algorithms, reaching state-of-the-art energies on many quantum many-body Hamiltonians. Nevertheless, the training of the ANN can be slow and stymied by the presence of local minima in the parameter landscape. One approach to mitigate this issue is to use parallel tempering methods, and in this work, we focus on the role played by the temperature distribution of the parallel tempering replicas. Using an adaptive method that adjusts the temperatures in order to equate the exchange probability between neighboring replicas, we show that this temperature optimization can significantly increase the success rate of the variational algorithm with negligible computational cost by eliminating bottlenecks in the replicas' random walk. Furthermore, we demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network, which we use to study a toy problem based on a permutation invariant Hamiltonian with a pernicious local minimum and the 𝐽 1 −𝐽 2 model on a rectangular lattice.

Neural network simulations↗
DOE OSTI2022

An algorithm for physics informed scan path optimization in additive manufacturing

Site specific microstructure control is a critical research area within the field of additive manufacturing due to its potential to revolutionize part performance. One way to achieve site specific microstructure control is through control of the solidification conditions via the construction of intricate scan paths; however, the search space for such a problem is large. Previous attempts only considered the solidification conditions at the top surface while also requiring either lots of manual-fine tuning or large amounts of computational resources. This paper introduces a general method for scan path optimization which considers the solidification conditions in the bulk of the material without an increase in computational expense. This method consists of three core components:1. A heat transfer model for simulating the temperature field at a given time.2. A surrogate model which takes scan pattern information and temperature data and predicts the solidification conditions of the bulk as well as the meltpool depths for a spot melt.3. A decision algorithm to decide which spot melt should be printed next based on the outputs of the surrogate model.Each of these components can be changed without changing the overall method. Within this work, this method is applied in the creation of an algorithm containing a semi-analytic heat transfer model to simulate the temperature field, a fully convolutional neural network (FCNN) as the surrogate model, and a greedy decision algorithm. The resulting algorithm produced complex scan patterns which gave strong results for simulated microstructure control.

36 MATERIALS SCIENCE↗