Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS2023

Neural-Based Compression Scheme for Solar Image Data

Studying the solar system and especially the Sun relies on the data gathered daily from space missions. These missions are data-intensive and compressing this data to make them efficiently transferable to the ground station is a twofold decision to make. Stronger compression methods, by distorting the data, can increase data throughput at the cost of accuracy which could affect scientific analysis of the data. On the other hand, preserving subtle details in the compressed data requires a high amount of data to be transferred, reducing the desired gains from compression. In this work, we propose a neural network-based lossy compression method to be used in NASA’s data-intensive imagery missions. We chose NASA’s Solar Dynamics Observatory (SDO) mission which transmits 1.4 terabytes of data each day as a proof of concept for the proposed algorithm. In this work, we propose an adversarially trained neural network, equipped with local and non-local attention modules to capture both the local and global structure of the image resulting in a better trade-off in rate-distortion (RD) compared to conventional hand-engineered codecs. The RD variational autoencoder used in this work is jointly trained with a channel-dependent entropy model as a shared prior between the analysis and synthesis transforms to make the entropy coding of the latent code more effective. We also studied how optimizing perceptual losses could help our neural compressor to preserve high-frequency details of the data in the reconstructed compressed image. Our neural image compression algorithm outperforms currently-in-use and state-of-the-art codecs such as JPEG and JPEG-2000 in terms of the RD performance when compressing extreme-ultraviolet (EUV) data. As a proof of concept for use of this algorithm in SDO data analysis, we have performed coronal hole (CH) detection using our compressed images, and generated consistent segmentations, even at a compression rate of ∼ 0.1 bits per pixel (compared to 8 bits per pixel on the original data) using EUV data from SDO.

Image coding↗
DOE OSTI2024

Super Resolving Unrolled Neural Networks for Remote Sensing

In remote sensing systems, the capabilities of the system are constrained by the complex interactions between size, weight, and power (SWAP) of potential designs. In electro-optical (EO) systems, examples of these critical parameters include the system’s sensitivity and resolution. Those parameters can be increased by ever larger optical apertures and focal planes but at the cost of more SWAP. Multi-image super resolution (MISR) techniques allow resolution to be enhanced via computation rather than more sophisticated optical hardware. These algorithms combine multiple images together into a single, higher resolution image, trading temporal resolution and computation for spatial resolution. Fielded MISR techniques, such as Drizzle, can require several hundred images to create a single super resolved image, implying reduced temporal resolution, increased data acquisition load, and limiting mission applications. Iterative techniques, such as model-based image reconstruction and compressive sensing, have been shown to create super resolved images using fewer images than Drizzle. They do this by posing an optimization problem that balances accuracy between a highly accurate physical model and an image model. In the case of super resolution, the physical model is defined by the relation between low resolution input images and the desired high resolution output image. The image model encodes some assumptions about the super resolved image. These assumptions are meant to suppress reconstruction artifacts that arise due to deterministic physical model error, stochastic measurement noise, and potential undersampling. In practice, the performance of iterative methods are limited by imaging models compatible with optimization. Deep learning-based methods can effectively learn image models of arbitrary complexity, but lack the theoretical explainability and robustness of iterative techniques. Consensus equilibrium (CE) generalizes the iterative techniques beyond optimization, enabling blackbox algorithms such as traditional and neural image denoisers to be used as the image model. CE-based approaches retain much of the explainability and robustness of iterative techniques while allowing the expressiveness of machine learning image models to be used. Additionally, by unrolling iterations of CE with an embedded image denoiser, the image denoiser can be further trained and specialized to the specific application with potentially higher quality reconstructions. Under this project, we demonstrated the feasibility of training an unrolled neural network based upon CE. While we didn’t train one, we showed that the CE process is differentiable and its gradient can be tractably computed. We also explored the usage of a variants of CE akin to generative neural works. Most importantly, we applied the CE framework to a number of problems including non-blind deconvolution, upsampling, single-image super resolution, MISR, event-based sensing, and saturated deconvolution. Our MISR prototype creates high quality reconstructions with an order of magnitude fewer images than previous approaches and, critically, produces these reconstructions fast enough for practical usage.

47 OTHER INSTRUMENTATION↗
NASA NTRS2004

Real-Time Adaptive Color Segmentation by Neural Networks

Artificial neural networks that would utilize the cascade error projection (CEP) algorithm have been proposed as means of autonomous, real-time, adaptive color segmentation of images that change with time. In the original intended application, such a neural network would be used to analyze digitized color video images of terrain on a remote planet as viewed from an uninhabited spacecraft approaching the planet. During descent toward the surface of the planet, information on the segmentation of the images into differently colored areas would be updated adaptively in real time to capture changes in contrast, brightness, and resolution, all in an effort to identify a safe and scientifically productive landing site and provide control feedback to steer the spacecraft toward that site. Potential terrestrial applications include monitoring images of crops to detect insect invasions and monitoring of buildings and other facilities to detect intruders. The CEP algorithm is reliable and is well suited to implementation in very-large-scale integrated (VLSI) circuitry. It was chosen over other neural-network learning algorithms because it is better suited to realtime learning: It provides a self-evolving neural-network structure, requires fewer iterations to converge and is more tolerant to low resolution (that is, fewer bits) in the quantization of neural-network synaptic weights. Consequently, a CEP neural network learns relatively quickly, and the circuitry needed to implement it is relatively simple. Like other neural networks, a CEP neural network includes an input layer, hidden units, and output units (see figure). As in other neural networks, a CEP network is presented with a succession of input training patterns, giving rise to a set of outputs that are compared with the desired outputs. Also as in other neural networks, the synaptic weights are updated iteratively in an effort to bring the outputs closer to target values. A distinctive feature of the CEP neural network and algorithm is that each update of synaptic weights takes place in conjunction with the addition of another hidden unit, which then remains in place as still other hidden units are added on subsequent iterations. For a given training pattern, the synaptic weight between (1) the inputs and the previously added hidden units and (2) the newly added hidden unit is updated by an amount proportional to the partial derivative of a quadratic error function with respect to the synaptic weight. The synaptic weight between the newly added hidden unit and each output unit is given by a more complex function that involves the errors between the outputs and their target values, the transfer functions (hyperbolic tangents) of the neural units, and the derivatives of the transfer functions.

Duong, Tuan A.↗
DOE OSTI2025

Digital twin framework for PIP-II linac: AI-driven multi-scale modeling from ion source to 800 MeV

The PIP-II superconducting linac at Fermilab is designed to deliver multi-megawatt proton beams for neutrino physics and other high-intensity applications. To expedite commissioning and enhance operational reliability, we have developed an EPICS-based data flow framework that seamlessly integrates digital twins (DT) with physical twins (PT). These digital twins comprise high-fidelity beam dynamics models or data-driven surrogate models connected to their physical counterparts through real-time diagnostics and advanced machine-learning algorithms.Central to this framework is Linac_Gen, an accelerated simulation tool that incorporates convolutional neural networks, random forests, and genetic algorithms to provide up to a tenfold speedup in optimizing the accelerator geometry model. An EPICS translator layer ensures interoperability by efficiently mapping lattice parameters across diverse simulation platforms.Our EPICS-based framework supports multiple operational modes—monitoring, passive learning, closed-loop control, and online learning—covering the entire machine lifecycle. By leveraging HPC resources and multi-objective optimization techniques, the digital twin enables adaptive trajectory correction, real-time fault detection, and predictive modeling of beam stability. This comprehensive approach paves the way for robust, high-intensity operation and data-driven accelerator R&D at Fermilab.

Pathak, Abhishek [Fermilab]↗
NASA NTRS1992

Comparison of crisp and fuzzy character networks in handwritten word recognition

Experiments involving handwritten word recognition on words taken from images of handwritten address blocks from the United States Postal Service mailstream are described. The word recognition algorithm relies on the use of neural networks at the character level. The neural networks are trained using crisp and fuzzy desired outputs. The fuzzy outputs were defined using a fuzzy k-nearest neighbor algorithm. The crisp networks slightly outperformed the fuzzy networks at the character level but the fuzzy networks outperformed the crisp networks at the word level.

Gader, Paul↗
DOE OSTI2021

Primordial non-Gaussianity from the completed SDSS-IV extended Baryon Oscillation Spectroscopic Survey – I: Catalogue preparation and systematic mitigation

ABSTRACT We investigate the large-scale clustering of the final spectroscopic sample of quasars from the recently completed extended Baryon Oscillation Spectroscopic Survey (eBOSS). The sample contains 343 708 objects in the redshift range 0.8 < z < 2.2 and 72 667 objects with redshifts 2.2 < z < 3.5, covering an effective area of $4699\, {\rm deg}^{2}$. We develop a neural network-based approach to mitigate spurious fluctuations in the density field caused by spatial variations in the quality of the imaging data used to select targets for follow-up spectroscopy. Simulations are used with the same angular and radial distributions as the real data to estimate covariance matrices, perform error analyses, and assess residual systematic uncertainties. We measure the mean density contrast and cross-correlations of the eBOSS quasars against maps of potential sources of imaging systematics to address algorithm effectiveness, finding that the neural network-based approach outperforms standard linear regression. Stellar density is one of the most important sources of spurious fluctuations, and a new template constructed using data from the Gaia spacecraft provides the best match to the observed quasar clustering. The end-product from this work is a new value-added quasar catalogue with the improved weights to correct for non-linear imaging systematic effects, which will be made public. Our quasar catalogue is used to measure the local-type primordial non-Gaussianity in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2022

Radiation Tolerance and Mitigation for Neuromorphic Processors

Neuromorphic processors are designed to execute Deep Neural Networks (DNNs) at very high speed using only a fraction of the electrical power needed to run a DNN on a traditional CPU or GPU. This unique capability makes Neuromorphic processors a prime candidate for space systems, where advanced computational tasks like image analysis, depth map reconstruction, or rover control need to be executed in a power-starved environment. In contrast to the growing number of applications of Neuromorphic processors in smart phones, the automotive and robotics domain, the space environment is unforgiving because of extreme temperatures and high levels of radiation. Any space system, operating beyond LEO requires computing hardware that is resilient against radiation effects. However, Neuromorphic processors have not yet been designed or tested for their radiation tolerance. In this report, we consider traditional methods of detection of radiation events and mitigation via redundancy and gauge their effectiveness on DNNs. In contrast to traditional flight software, however, neural networks represent a statistical algorithm, which might affect its resilience against radiation events. We will focus on the analysis of the tolerance of DNNs with respect to radiation events and discuss techniques to detect radiation hits using on-chip triple modular redundancy (TMR) on an Intel Loihi neuromorphic processor and to mitigate radiation damage. We describe an architecture for on-chip TMR for the Intel Loihi and present results of initial experiments.

Neural Networks↗
NASA NTRS2002

A Bayesian Approach to Estimating Coupling Between Neural Components: Evaluation of the Multiple Component, Event-Related Potential (mcERP) Algorithm

Accurate measurement of single-trial responses is key to a definitive use of complex electromagnetic and hemodynamic measurements in the investigation of brain dynamics. We developed the multiple component, Event-Related Potential (mcERP) approach to single-trial response estimation. To improve our resolution of dynamic interactions between neuronal ensembles located in different layers within a cortical region and/or in different cortical regions. The mcERP model assets that multiple components defined as stereotypic waveforms comprise the stimulus-evoked response and that these components may vary in amplitude and latency from trial to trial. Maximum a posteriori (MAP) solutions for the model are obtained by iterating a set of equations derived from the posterior probability. Our first goal was to use the ANTWERP algorithm to analyze interactions (specifically latency and amplitude correlation) between responses in different layers within a cortical region. Thus, we evaluated the model by applying the algorithm to synthetic data containing two correlated local components and one independent far-field component. Three cases were considered: the local components were correlated by an interaction in their single-trial amplitudes, by an interaction in their single-trial latencies, or by an interaction in both amplitude and latency. We then analyzed the accuracy with which the algorithm estimated the component waveshapes and the single-trial parameters as a function of the linearity of each of these relationships. Extensions of these analyses to real data are discussed as well as ongoing work to incorporate more detailed prior information.

Shah, Ankoor S.↗
DOE OSTI2024

Bridging Hydrological Ensemble Simulation and Learning Using Deep Neural Operators

Ensemble-based simulation and learning (ESnL) has long been used in hydrology for parameter inference, but computational demands of process-based ESnL can be quite high. To address this issue, we propose a deep neural operator learning approach. Neural operators are generic machine learning algorithms that can learn functional mappings between infinite-dimensional spaces, providing a highly flexible tool for scientific machine learning. Our approach is built upon DeepONet, a specific deep neural operator, and is designed to address several common problems in hydrology, namely, model parameter estimation, prediction at ungaged locations, and uncertainty quantification. Here we demonstrate the effectiveness of our DeepONet-based workflow using an existing large model ensemble created for an eastern U.S. watershed that is instrumented with 10 streamflow gages. Results suggest DeepONet achieves high efficiency in learning an ML surrogate model from the model ensemble, with the modified Kling-Gupta Efficiency exceeding 0.9 on holdout test sets. Parameter inference, carried out using the trained DeepONet surrogate model and genetic algorithm, also yields robust results. Additionally, we formulate and train a separate DeepONet model for physics-informed, seq-to-seq streamflow forecasting, which further reduces biases in the pre-trained DeepONet surrogate model. While this study focuses primarily on a single watershed, our approach is general and may be extended to enable learning from model ensembles across multiple basins or models. Thus, this research represents a significant contribution to the application of hybrid machine learning in hydrology.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1999

Systems Engineering Design Via Experimental Operation Research: Complex Organizational Metric for Programmatic Risk Environments (COMPRE)

Unique and innovative graph theory, neural network, organizational modeling, and genetic algorithms are applied to the design and evolution of programmatic and organizational architectures. Graph theory representations of programs and organizations increase modeling capabilities and flexibility, while illuminating preferable programmatic/organizational design features. Treating programs and organizations as neural networks results in better system synthesis, and more robust data modeling. Organizational modeling using covariance structures enhances the determination of organizational risk factors. Genetic algorithms improve programmatic evolution characteristics, while shedding light on rulebase requirements for achieving specified technological readiness levels, given budget and schedule resources. This program of research improves the robustness and verifiability of systems synthesis tools, including the Complex Organizational Metric for Programmatic Risk Environments (COMPRE).

Mog, Robert A.↗
DOE OSTI2021

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Here using a model heat engine, we show that neural-network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Development of Hopfield Artificial Neural Network for Anomaly Detection in Environmental Gamma Radiation Background: Consortium on Nuclear Security Technologies (CONNECT) (Q2 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore supervised machine learning (ML) algorithms for development of a Hopfield Neural Network (HNN) in conjunction with an image processing algorithm for detection and identification of weak nuisances and anomalies events in the presence of a highly fluctuating background.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Reducing Communication in Graph Neural Network Training

Graph Neural Networks (GNNs) are powerful and flexible neural networks that use the naturally sparse connectivity information of the data. GNNs represent this connectivity as sparse matrices, which have lower arithmetic intensity and thus higher communication costs compared to dense matrices, making GNNs harder to scale to high concurrencies than convolutional or fully-connected neural networks. Here, we introduce a family of parallel algorithms for training GNNs and show that they can asymptotically reduce communication compared to previous parallel GNN training methods. We implement these algorithms, which are based on 1D, 1. 5D, 2D, and 3D sparse-dense matrix multiplication, using torch.distributed on GPU-equipped clusters. Our algorithms optimize communication across the full GNN training pipeline. We train GNNs on over a hundred GPUs on multiple datasets, including a protein network with over a billion edges.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

Retrieval of ice thickness from polarimetric SAR data

We describe a potential procedure for retrieving ice thickness from multi-frequency polarimetric SAR data for thin ice. This procedure includes first masking out the thicker ice types with a simple classifier and then deriving the thickness of the remaining pixels using a model-inversion technique. The technique used to derive ice thickness from polarimetric observations is provided by a numerical estimator or neural network. A three-layer perceptron implemented with the backpropagation algorithm is used in this investigation with several improved aspects for a faster convergence rate and a better accuracy of the neural network. These improvements include weight initialization, normalization of the output range, the selection of offset constant, and a heuristic learning algorithm. The performance of the neural network is demonstrated by using training data generated by a theoretical scattering model for sea ice matched to the database of interest. The training data are comprised of the polarimetric backscattering coefficients of thin ice and the corresponding input ice parameters to the scattering model. The retrieved ice thickness from the theoretical backscattering coefficients is compare with the input ice thickness to the scattering model to illustrate the accuracy of the inversion method. Results indicate that the network convergence rate and accuracy are higher when multi-frequency training sets are presented. In addition, the dominant backscattering coefficients in retrieving ice thickness are found by comparing the behavior of the network trained backscattering data at various incidence angels. After the neural network is trained with the theoretical backscattering data at various incidence anges, the interconnection weights between nodes are saved and applied to the experimental data to be investigated. In this paper, we illustrate the effectiveness of this technique using polarimetric SAR data collected by the JPL DC-8 radar over a sea ice scene.

Kwok, R.↗
DOE OSTI2026

Search for the pair production of long-lived supersymmetric partners of the tau lepton in proton-proton collisions at $\sqrt{s}=13$ TeV

Gauge-mediated supersymmetry-breaking models provide a strong motivation to search for a supersymmetric partner of the tau lepton (stau) with a macroscopic lifetime. Long-lived stau decays produce tau leptons that are displaced from the primary proton-proton interaction vertex, leading to an unconventional signature. This paper presents a search for the direct production of long-lived staus decaying within the CMS tracker volume in proton-proton collisions at $\sqrt{s}=13$ TeV, performed for the first time with an identification algorithm based on a graph neural network dedicated to displaced tau leptons. The data sample, corresponding to an integrated luminosity of 138 fb −1 , was recorded with the CMS experiment at the CERN LHC between 2016 and 2018. This search excludes, at 95% confidence level, stau masses, m~ τ , in the 126–260 (90–425) GeV range for a proper decay length of 50 mm in the maximally mixed (mass-degenerate) scenario, while for m~ τ = 200 GeV, stau proper decay lengths are excluded in the range 21–94 (6–333) mm. These results improve the exclusion limits compared to previous searches, and extend the parameter space explored in the context of supersymmetry.

Hadron-Hadron Scattering↗
DOE OSTI2023

Improved calorimetric particle identification in NA62 using machine learning techniques

Measurement of the ultra-rare $K^+$ → $π$ +$ν$$\overline{ν}$ decay at the NA62 experiment at CERN requires high-performance particle identification to distinguish muons from pions. Calorimetric identification currently in use, based on a boosted decision tree algorithm, achieves a muon misidentification probability of 1.2×10 -5 for a pion identification efficiency of 75% in the momentum range of 15–40 GeV/c. In this work, calorimetric identification performance is improved by developing an algorithm based on a convolutional neural network classifier augmented by a filter. Muon misidentification probability is reduced by a factor of six with respect to the current value for a fixed pion-identification efficiency of 75%. Alternatively, pion identification efficiency is improved from 72% to 91% for a fixed muon misidentification probability of 10 -5 .

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗