Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

DeepGhostBusters: Using Mask R-CNN to Detect and Mask Ghosting and Scattered-Light Artifacts from Optical Survey Images

Wide-field astronomical surveys are often affected by the presence of undesirable reflections (often known as "ghosting artifacts" or "ghosts") and scattered-light artifacts. The identification and mitigation of these artifacts is important for rigorous astronomical analyses of faint and low-surface-brightness systems. However, the identification of ghosts and scattered-light artifacts is challenging due to a) the complex morphology of these features and b) the large data volume of current and near-future surveys. In this work, we use images from the Dark Energy Survey (DES) to train, validate, and test a deep neural network (Mask R-CNN) to detect and localize ghosts and scattered-light artifacts. We find that the ability of the Mask R-CNN model to identify affected regions is superior to that of conventional algorithms and traditional convolutional neural networks methods. We propose that a multi-step pipeline combining Mask R-CNN segmentation with a classical CNN classifier provides a powerful technique for the automated detection of ghosting and scattered-light artifacts in current and near-future surveys.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Graph Neural Networks and Applied Linear Algebra v.1.0

SAND2024-01365O The Graph Neural Networks and Applied Linear Algebra is companion software for the educational article with the same title. The software provides illustrative examples of graph neural networks in Matlab and Python. These stand-alone algorithms are for educational purposes. The software also includes graph neural network-based algorithms for a trainable Jacobi iteration as well as diffusion coefficient estimation. The software provides human-interpretable implementations of Graph Neural Networks in Matlab and Python. These implementations are not optimized for performance and instead emphasize readability. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
DOE OSTI2021

Evolutionary NAS in Light of Model Stability for Accurate Continual Learning

Continual learning, the capability to learn new knowledge from streaming data without forgetting the previous knowledge, is a critical requirement for dynamic learning systems, especially for emerging edge devices such as self-driving cars and drones. However, continual learning is still facing the catastrophic forgetting problem. Previous work illustrate that model performance on continual learning is not only related to the learning algorithms but also strongly dependent on the inherited model, i.e., the model where continual learning starts. The better stability of the inherited model, the less catastrophic forgetting and thus, the inherited model should be elaborately selected. Inspired by this finding, we develop an evolutionary neural architecture search (ENAS) algorithm that emphasizes the Stability of the inherited model, namely ENAS-S. ENAS-S aims to find optimal architectures for accurate continual learning on edge devices. On CIFAR-10 and CIFAR-100, we present that ENAS-S achieves competitive architectures with lower catastrophic forgetting and smaller model size when learning from a data stream, as compared with handcrafted DNNs.

Du, Xiaocong↗
DOE OSTI2024

Angular Dependence of Drell-Yan in the SeaQuest Experiment using Deep Neural Network-Based Reconstruction

The SeaQuest and SpinQuest experiments at Fermilab were designed to probe the internal dynamics of protons and neutrons via the angular dependence of muons created by colliding 120 GeV protons into stationary targets. To do this, it is necessary to translate the detector data into coherent information about the particles detected in the experiment. This dissertation describes the development and implementation of QTracker, a neural network-based algorithm designed to reconstruct muons. The application of QTracker to experimental data yields improved reconstruction of muon tracks, allowing for a more precise investigation of the transverse momentum distributions and angular modulations in the Drell-Yan process. This work highlights the potential of neural network-based methods to advance particle tracking and enhance our understanding of nucleon structure.

Conover, Arthur↗
DOE OSTI2025

Enabling Real-Time Communication in Multi-Agent Systems: A Graph Neural Network Based Approach

Global connectivity enables effective coordination in Multi-Agent Systems (MAS). Solving these connection problems under hardware constraints is an NP-hard non-Euclidean Degree Constrained Minimum Spanning Tree (DCMST) problem. Prior MAS controllers coordinate team movement for task completion and collision avoidance; some considering Line-of-Sight (LOS) maintenance but prioritizing flexibility over guarantees. Evolutionary Algorithms (EA) have been shown to find good solutions for DCMST, but their performance degrades with larger populations required to support a large MAS. We present a method based on edge graph attention networks, trained offline to reduce online computation times. Empirical comparisons with greedy polynomial-time solvers and EA show that our method leverages latent graph information to consistently find constraint-satisfying solutions in less time.

connectivity maintenance↗
DOE OSTI2024

Neural architecture search via similarity adaptive guidance

Evolutionary neural network architecture search (ENAS) has attracted the attention of many experts due to its global optimization capabilities to automatically search for convolutional neural network architectures based on the target task. The current search space for ENAS is not to design a fully structured network, but to search for smaller cell architectures to reduce search costs. However, blind search strategies do not effectively utilize the potential experience of the population. In order to utilize the potential experience learned by the current population to guide the evolutionary search of the population, we propose a similarity guided neural network architecture search algorithm based on cell architecture, which utilizes the similarity between pairwise architectures in the population as empirical knowledge learned by the population. Our proposed algorithm provides a novel method for calculating architecture similarity, which calculates architecture similarity separately from the cell and macro-structure. Then we decouple the connections and operations in the cell and calculate connection and operation similarity separately. In addition, we propose adaptive similarity selection and binary tournament selection strategies to enhance the algorithm’s global and local search capabilities and effectively explore the search space. Finally, we design an improved single-point crossover operator to enhance the local search ability of the evolutionary operator. The experimental results show that SAGNAS is a competitive algorithm that achieves 97.44% and 81.60% in CIFAR10 and CIFAR100 with only 1.9 GPU-days spent.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Neuromorphic learning with Mott insulator NiO

Habituation and sensitization (nonassociative learning) are among the most fundamental forms of learning and memory behavior present in organisms that enable adaptation and learning in dynamic environments. Emulating such features of intelligence found in nature in the solid state can serve as inspiration for algorithmic simulations in artificial neural networks and potential use in neuromorphic computing. In this work, we demonstrate nonassociative learning with a prototypical Mott insulator, nickel oxide (NiO), under a variety of external stimuli at and above room temperature. Similar to biological species such as Aplysia, habituation and sensitization of NiO possess time-dependent plasticity relying on both strength and time interval between stimuli. A combination of experimental approaches and first-principles calculations reveals that such learning behavior of NiO results from dynamic modulation of its defect and electronic structure. An artificial neural network model inspired by such nonassociative learning is simulated to show advantages for an unsupervised clustering task in accuracy and reducing catastrophic interference, which could help mitigate the stability–plasticity dilemma. Mott insulators can therefore serve as building blocks to examine learning behavior noted in biology and inspire new learning algorithms for artificial intelligence.

42 ENGINEERING↗
DOE OSTI2023

Frost prediction using machine learning and deep neural network models

This study describes accurate, computationally efficient models that can be implemented for practical use in predicting frost events for point-scale agricultural applications. Frost damage in agriculture is a costly burden to farmers and global food security alike. Timely prediction of frost events is important to reduce the cost of agricultural frost damage and traditional numerical weather forecasts are often inaccurate at the field-scale in complex terrain. In this paper, we developed machine learning (ML) algorithms for the prediction of such frost events near Alcalde, NM at the point-scale. ML algorithms investigated include deep neural network, convolution neural networks, and random forest models at lead-times of 6–48 h. Our results show promising accuracy (6-h prediction RMSE = 1.53–1.72°C) for use in frost and minimum temperature prediction applications. Seasonal differences in model predictions resulted in a slight negative bias during Spring and Summer months and a positive bias in Fall and Winter months. Additionally, we tested the model transferability by continuing training and testing using data from sensors at a nearby farm. We calculated the feature importance of the random forest models and were able to determine which parameters provided the models with the most useful information for predictions. We determined that soil temperature is a key parameter in longer term predictions (>24 h), while other temperature related parameters provide the majority of information for shorter term predictions. The model error compared favorable to previous ML based frost studies and outperformed the physically based High Resolution Rapid Refresh forecasting system making our ML-models attractive for deployment toward real-time monitoring of frost events and damage at commercial farming operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

MHD mode tracking using high-speed cameras and deep learning

Abstract We present a new algorithm to track the amplitude and phase of rotating magnetohydrodynamic (MHD) modes in tokamak plasmas using high speed imaging cameras and deep learning. This algorithm uses a convolutional neural network (CNN) to predict the amplitudes of the n = 1 sine and cosine mode components using solely optical measurements from one or more cameras. The model was trained and tested on an experimental dataset consisting of camera frame images and magnetic-based mode measurements from the High Beta Tokamak - Extended Pulse (HBT-EP) device, and it outperformed other, more conventional, algorithms using identical image inputs. The effect of different input data streams on the accuracy of the model’s predictions is also explored, including using a temporal frame stack or images from two cameras viewing different toroidal regions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Retina-inspired narrowband perovskite sensor array for panchromatic imaging

The retina is the essential part of the human visual system that receives light, converts it to neural signal, and transmits to brain for visual recognition. The red, green, and blue (R/G/B) cone retina cells are natural narrowband photodetectors (PDs) sensitive to R/G/B lights. Connecting with these cone cells, a multilayer neuro-network in the retina provides neuromorphic preprocessing before transmitting to brain. Inspired by this sophistication, we develop the narrowband (NB) imaging sensor combining R/G/B perovskite NB sensor array (mimicking the R/G/B photoreceptors) with a neuromorphic algorithm (mimicking the intermediate neural network) for high-fidelity panchromatic imaging. Compared to commercial sensors, we use perovskite “intrinsic” NB PD to exempt the complex optical filter array. In addition, we use an asymmetric device configuration to collect photocurrent without external bias, enabling a power-free photodetection feature. These results display a promising design for efficient and intelligent panchromatic imaging.

42 ENGINEERING↗
DOE OSTI2023

Photon Reconstruction in the Belle II Calorimeter Using Graph Neural Networks

We present the study of a fuzzy clustering algorithm for the Belle II electromagnetic calorimeter using Graph Neural Networks. We use a realistic detector simulation including simulated beam backgrounds and focus on the reconstruction of both isolated and overlapping photons. We find significant improvements of the energy resolution compared to the currently used reconstruction algorithm for both isolated and overlapping photons of more than 30% for photons with energies E γ < 0.5 GeV and high levels of beam backgrounds. Overall, the GNN reconstruction improves the resolution and reduces the tails of the reconstructed energy distribution and therefore is a promising option for the upcoming high luminosity running of Belle II.

Calorimeter↗
DOE OSTI2020

Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs

We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, and tracking performance of our implementations based on a benchmark dataset. We find a considerable speedup over CPU-based execution is possible, potentially enabling such algorithms to be used effectively in future computing workflows and the FPGA-based Level-1 trigger at the CERN Large Hadron Collider.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Quantifying Atomically Dispersed Catalysts Using Deep Learning Assisted Microscopy

The catalytic performance of atomically dispersed catalysts (ADCs) is greatly influenced by their atomic configurations, such as atom–atom distances, clustering of atoms into dimers and trimers, and their distributions. Scanning transmission electron microscopy (STEM) is a powerful technique for imaging ADCs at the atomic scale; however, most STEM analyses of ADCs thus far have relied on human labeling, making it difficult to analyze large data sets. Here, we introduce a convolutional neural network (CNN)-based algorithm capable of quantifying the spatial arrangement of different adatom configurations. The algorithm was tested on different ADCs with varying support crystallinity and homogeneity. Results show that our algorithm can accurately identify atom positions and effectively analyze large data sets. Here, this work provides a robust method to overcome a major bottleneck in STEM analysis for ADC catalyst research. We highlight the potential of this method to serve as an on-the-fly analysis tool for catalysts in future in situ microscopy experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Training neural networks using Metropolis Monte Carlo and an adaptive variant

Abstract We examine the zero-temperature Metropolis Monte Carlo (MC) algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis MC can train a neural net with an accuracy comparable to that of gradient descent (GD), if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network’s structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm (aMC) to overcome these limitations. The intrinsic stochasticity and numerical stability of the MC method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by GD. MC methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A real-time energy and cost efficient vehicle route assignment neural recommender system

Here, this paper presents a neural network recommender system algorithm for assigning vehicles to routes based on energy and cost criteria. In this work, we applied this new approach to efficiently identify the most cost-effective medium and heavy duty truck (MDHDT) powertrain technology, from a total cost of ownership (TCO) perspective, for given trips. We employ a machine learning based approach to efficiently estimate the energy consumption of various candidate vehicles over given routes, defined as sequences of links (road segments), with little information known about internal dynamics, i.e. using high level macroscopic route information. A complete recommendation logic is then developed to allow for real-time optimum assignment for each route, subject to the operational constraints of the fleet. We show how this framework can be used to (1) efficiently provide a single trip recommendation with a top-k vehicles star ranking system, and (2) engage in more general assignment problems where n vehicles need to be deployed over m (m ≤ n) trips. This new assignment system has been deployed and integrated into the POLARIS. Transportation System Simulation Tool for use in research conducted by the Department of Energy's Systems and Modeling for Accelerated Research in Transportation (SMART) Mobility Consortium (SMART, 2024).

Energy consumption↗
DOE OSTI2021

Probabilistic Deep Learning Approach to Automate the Interpretation of Multi-phase Diffraction Spectra

Autonomous synthesis and characterization of inorganic materials requires the automatic and accurate analysis of X-ray diffraction spectra. For this task, we designed a probabilistic deep learning algorithm to identify complex multi-phase mixtures. At the core of this algorithm lies an ensemble convolutional neural network trained on simulated diffraction spectra, which are systematically augmented with physics-informed perturbations to account for artifacts that can arise during experimental sample preparation and synthesis. In this work, larger perturbations associated with off-stoichiometry are also captured by supplementing the training set with hypothetical solid solutions. Spectra containing mixtures of materials are analyzed with a newly developed branching algorithm that utilizes the probabilistic nature of the neural network to explore suspected mixtures and identify the set of phases that maximize confidence in the prediction. Our model is benchmarked on simulated and experimentally measured diffraction spectra, showing exceptional performance with accuracies exceeding those given by previously reported methods based on profile matching and deep learning. We envision that the algorithm presented here may be integrated in experimental workflows to facilitate the high-throughput and autonomous discovery of inorganic materials.

36 MATERIALS SCIENCE↗
DOE OSTI2021

ECNet is an evolutionary context-integrated deep learning framework for protein engineering

Abstract Machine learning has been increasingly used for protein engineering. However, because the general sequence contexts they capture are not specific to the protein being engineered, the accuracy of existing machine learning algorithms is rather limited. Here, we report ECNet (evolutionary context-integrated neural network), a deep-learning algorithm that exploits evolutionary contexts to predict functional fitness for protein engineering. This algorithm integrates local evolutionary context from homologous sequences that explicitly model residue-residue epistasis for the protein of interest with the global evolutionary context that encodes rich semantic and structural features from the enormous protein sequence universe. As such, it enables accurate mapping from sequence to function and provides generalization from low-order mutants to higher-order mutants. We show that ECNet predicts the sequence-function relationship more accurately as compared to existing machine learning algorithms by using ~50 deep mutational scanning and random mutagenesis datasets. Moreover, we used ECNet to guide the engineering of TEM-1 β-lactamase and identified variants with improved ampicillin resistance with high success rates.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Identification of high-dielectric constant compounds from statistical design

Abstract The discovery of high-dielectric materials is crucial to increasing the efficiency of electronic devices and batteries. Here, we report three previously unexplored materials with very high dielectric constants (69 < ϵ < 101) and large band gaps (2.9 < E g (eV) < 5.5) obtained by screening materials databases using statistical optimization algorithms aided by artificial neural networks (ANN). Two of these new dielectrics are mixed-anion compounds (Eu 5 SiCl 6 O 4 and HoClO) and are shown to be thermodynamically stable against common semiconductors via phase diagram analysis. We also uncovered four other materials with relatively large dielectric constants (20 < ϵ < 40) and band gaps (2.3 < E g (eV) < 2.7). While the ANN training-data are obtained from the Materials Project, the search-space consists of materials from the Open Quantum Materials Database (OQMD)—demonstrating a successful implementation of cross-database materials design. Overall, we report the dielectric properties of 17 materials calculated using ab initio calculations, that were selected in our design workflow. The dielectric materials with high-dielectric properties predicted in this work open up further experimental research opportunities.

36 MATERIALS SCIENCE↗