Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing overhead where inputs need to be zero-padded to the maximum sentence length in the batch to accommodate the parallel computing platforms. This paper targets the field-programmable gate array (FPGA) and proposes a coherent sequence length adaptive algorithm–hardware co-design for Transformer acceleration. Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm. The proposed sparse attention operator brings the complexity of attention-based models down to linear complexity and alleviates the off-chip memory traffic. The proposed length-aware resource hardware scheduling algorithm dynamically allocates the hardware resources to fill up the pipeline slots and eliminates bubbles for NLP tasks. Experiments show that our design has very small accuracy loss and has 80.2 × and 2.6 × speedup compared to CPU and GPU implementation, and 4 × higher energy efficiency than state-of-the-art GPU accelerator optimized via CUBLAS GEMM.

Peng, Hongwu↗
NASA NTRS1994

VLSI neuroprocessors

Electronic and optoelectronic hardware implementations of highly parallel computing architectures address several ill-defined and/or computation-intensive problems not easily solved by conventional computing techniques. The concurrent processing architectures developed are derived from a variety of advanced computing paradigms including neural network models, fuzzy logic, and cellular automata. Hardware implementation technologies range from state-of-the-art digital/analog custom-VLSI to advanced optoelectronic devices such as computer-generated holograms and e-beam fabricated Dammann gratings. JPL's concurrent processing devices group has developed a broad technology base in hardware implementable parallel algorithms, low-power and high-speed VLSI designs and building block VLSI chips, leading to application-specific high-performance embeddable processors. Application areas include high throughput map-data classification using feedforward neural networks, terrain based tactical movement planner using cellular automata, resource optimization (weapon-target assignment) using a multidimensional feedback network with lateral inhibition, and classification of rocks using an inner-product scheme on thematic mapper data. In addition to addressing specific functional needs of DOD and NASA, the JPL-developed concurrent processing device technology is also being customized for a variety of commercial applications (in collaboration with industrial partners), and is being transferred to U.S. industries. This viewgraph p resentation focuses on two application-specific processors which solve the computation intensive tasks of resource allocation (weapon-target assignment) and terrain based tactical movement planning using two extremely different topologies. Resource allocation is implemented as an asynchronous analog competitive assignment architecture inspired by the Hopfield network. Hardware realization leads to a two to four order of magnitude speed-up over conventional techniques and enables multiple assignments, (many to many), not achievable with standard statistical approaches. Tactical movement planning (finding the best path from A to B) is accomplished with a digital two-dimensional concurrent processor array. By exploiting the natural parallel decomposition of the problem in silicon, a four order of magnitude speed-up over optimized software approaches has been demonstrated.

Kemeny, Sabrina E.↗
DOE OSTI2024

Improved Subseasonal Forecasting of Extreme Polar Vortices Using Machine Learning

Our research was focused on forecasting the position and shape of the winter stratospheric polar vortex at a subseasonal timescale of 15 days in advance. To achieve this, we employed both statistical and neural network machine learning techniques. The analysis was performed on 42 winter seasons of reanalysis data provided by NASA giving us a total of 6,342 days of data. The state of the polar vortex for determined by using geometric moments to calculate the centroid latitude and the aspect ratio of an ellipse fit onto the vortex. Timeseries for thirty additional precursors were calculated to help improve the predictive capabilities of the algorithm. Feature importance of these precursors was performed using random forest to measure the predictive importance and the ideal number of precursors. Then, using the precursors identified as important, various statistical methods were tested for predictive accuracy with random forest and nearest neighbor performing the best. An echo state network, a type of recurrent neural network that features sparsely connected hidden layer and a reduced number of trainable parameters that allows for rapid training and testing, was also implemented for the forecasting problem. Hyperparameter tuning was performed for each methods using a subset of the training data. The algorithms were trained and tuned on the first 41 years of data, then tested for accuracy on the final year. In general, the centroid latitude of the polar vortex proved easier to predict than the aspect ratio across all algorithms. Random forest outperformed other statistical forecasting algorithms overall but struggled to predict extreme values. Forecasting from echo state network suggested a strong predictive capability past 15 days, but further work is required to fully realize the potential of recurrent neural network approaches.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2017

H31G-1596: DeepSAT's CloudCNN: A Deep Neural Network for Rapid Cloud Detection from Geostationary Satellites

Cloud and cloud shadow detection has important applications in weather and climate studies. It is even more crucial when we introduce geostationary satellites into the field of terrestrial remote sensing. With the challenges associated with data acquired in very high frequency (10-15 mins per scan), the ability to derive an accurate cloud shadow mask from geostationary satellite data is critical. The key to the success for most of the existing algorithms depends on spatially and temporally varying thresholds,which better capture local atmospheric and surface effects.However, the selection of proper threshold is difficult and may lead to erroneous results. In this work, we propose a deep neural network based approach called CloudCNN to classify cloudshadow from Himawari-8 AHI and GOES-16 ABI multispectral data. DeepSAT's CloudCNN consists of an encoderdecoder based architecture for binary-class pixel wise segmentation. We train CloudCNN on multi-GPU Nvidia Devbox cluster, and deploy the prediction pipeline on NASA Earth Exchange (NEX) Pleiades supercomputer. We achieved an overall accuracy of 93.29% on test samples. Since, the predictions take only a few seconds to segment a full multispectral GOES-16 or Himawari-8 Full Disk image, the developed framework can be used for real-time cloud detection, cyclone detection, or extreme weather event predictions.

GOES-1↗
DOE OSTI2023

Convergence of Hyperbolic Neural Networks Under Riemannian Stochastic Gradient Descent

Abstract We prove, under mild conditions, the convergence of a Riemannian gradient descent method for a hyperbolic neural network regression model, both in batch gradient descent and stochastic gradient descent. We also discuss a Riemannian version of the Adam algorithm. We show numerical simulations of these algorithms on various benchmarks.

Whiting, Wes (ORCID:0000000247505060)↗
DOE OSTI2024

DASEventNet: AI‐Based Microseismic Detection on Distributed Acoustic Sensing Data From the Utah FORGE Well 16A (78)‐32 Hydraulic Stimulation

Abstract Distributed acoustic sensing (DAS) has emerged as a promising seismic technology for monitoring microearthquakes (MEQs) with high spatial resolution. Efficient algorithms are needed for processing large DAS data volumes. This study introduces a deep learning (DL) model based on a Residual Convolutional Neural Network (ResNet) for detecting MEQs using DAS data, named as DASEventNet. The test data were collected from the Utah FORGE 16A (78)‐32 hydraulic stimulation experiments conducted in April 2022. The DASEventNet model achieves a remarkable accuracy of 100% when discriminating MEQs from noise in the raw test set of 260 examples. Surprisingly, the model identified weak MEQ signatures that have been manually categorized as noise. The decision‐making process with the model is decoded by the classic activation map, which illuminates learning features of the DASEventNet model. These features provide clear illustrations of weak MEQs and varied noise types. Finally, we apply the trained model to the entire period (∼7 days) of continuous DAS recordings and find that it discovers >5,700 new MEQs, previously unregistered in the public Silixa DAS catalog. The DASEventNet model significantly outperforms the traditional seismic method Short‐Term Average/Long‐Term Average (STA/LTA), which detected only 1,307 MEQs. The DASEventNet detection threshold is M w −1.80 compared to the minimum magnitude of M w −1.14 detected by STA/LTA. The spatiotemporal distribution of the newly identified MEQs defines an extensive stimulation zone and more accurately characterizes fracture geometry. Our results highlight the potential of DL for long‐term, real‐time microseismic monitoring that can improve enhanced geothermal systems and other activities that include subsurface hydraulic fracturing.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Simultaneous control of the electron temperature and safety factor profiles in DIII-D using model-based optimal control techniques

Future tokamak power plants will likely operate using a single, well-defined plasma scenario, either in steady state or for very long pulse lengths. In order to enhance the robustness of the scenario, feedback controllers for a variety of plasma properties will be necessary to counteract any disturbances and ensure safe operation. However, only a limited set of actuators will be available to control many different quantities. Because of this, it is necessary to develop controllers that are able to regulate multiple plasma properties using a limited set of actuators. To this end, a controller has been developed for the simultaneous regulation of both the electron temperature and safety factor profiles in DIII-D. This algorithm uses a linear quadratic integral control synthesis approach based on a linearized model of the dynamics of the two profiles. Two neural network surrogate models, NubeamNet and MMMnet, are included to improve the fidelity of the model. Furthermore, the controller has been tested in simulation using COTSIM, and has demonstrated the ability to simultaneously track changes in both the electron temperature and safety factor targets, including changes in both the magnitude and the shape of the profiles.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Unsupervised quantum circuit learning in high energy physics

Unsupervised training of generative models is a machine learning task that has many applications in scientific computing. Here, in this work, we evaluate the efficacy of using quantum circuit-based generative models to generate synthetic data of high energy physics processes. We use nonadversarial, gradient-based training of quantum circuit Born machines to generate joint distributions over two and three variables.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2001

Do nonlinearities play a significant role in short term, beat-to-beat variability?

Numerous studies of short-term beat-to-beat variability in cardiovascular signals have not resolved the debate about the completeness of linear analysis techniques. This aim of this paper is to evaluate further the role of nonlinearities in short-term, beat-to-beat variability. We compared linear autoregressive moving average (ARMA) and nonlinear neural network (NN) models for predicting instantaneous heart rate (HR) and mean arterial blood pressure (BP) from past HR and BP. To evaluate these models, we used HR and BP time series from the MIMIC database. Experimental results indicate that NN-based nonlinearities do not play a significant role and suggest that ARMA linear analysis techniques provide adequate characterization of the system dynamics responsible for generating short-term, beat-to-beat variability.

NASA Discipline Cardiopulmonary↗
NASA NTRS1998

Design of Neural Networks for Fast Convergence and Accuracy

A novel procedure for the design and training of artificial neural networks, used for rapid and efficient controls and dynamics design and analysis for flexible space systems, has been developed. Artificial neural networks are employed to provide a means of evaluating the impact of design changes rapidly. Specifically, two-layer feedforward neural networks are designed to approximate the functional relationship between the component spacecraft design changes and measures of its performance. A training algorithm, based on statistical sampling theory, is presented, which guarantees that the trained networks provide a designer-specified degree of accuracy in mapping the functional relationship. Within each iteration of this statistical-based algorithm, a sequential design algorithm is used for the design and training of the feedforward network to provide rapid convergence to the network goals. Here, at each sequence a new network is trained to minimize the error of previous network. The design algorithm attempts to avoid the local minima phenomenon that hampers the traditional network training. A numerical example is performed on a spacecraft application in order to demonstrate the feasibility of the proposed approach.

Maghami, Peiman G.↗
DOE OSTI2024

Visibility-enhanced model-free deep reinforcement learning algorithm for voltage control in realistic distribution systems using smart inverters

Increasing integration of distributed solar photovoltaic (PV) into distribution networks could result in adverse effects on grid operation. Traditional model-based control algorithms require accurate model information that is difficult to acquire and thus are challenging to implement in practice. Here, this paper proposes a surrogate model-enabled grid visibility scheme to empower deep reinforcement learning (DRL) approach for distribution network voltage regulation using PV inverters with minimal system knowledge. In contrast to existing DRL methods, this paper presents and corroborates the adverse impact of missing load information on DRL performance and, based on this finding, proposes a surrogate model methodology to impute load information utilizing observable data. Additionally, a multi-fidelity neural network is utilized to construct the DRL training environment, chosen for its efficient data utilization and enhanced robustness to data uncertainty. The feasibility and effectiveness of the proposed algorithm are assessed by considering DRL testing across varying degrees of observable load information and diverse training environments on a realistic power system.

14 SOLAR ENERGY↗
DOE OSTI2026

Wilson loops with neural networks

Wilson loops are essential objects in QCD and have been pivotal in scale setting and demonstrating confinement. Various generalizations are crucial for computations needed in effective field theories. In lattice gauge theory, Wilson loop calculations face challenges, including excited-state contamination at short times and the signal-to-noise ratio issue at longer times. To address these problems, we develop a new method by using neural networks to parametrize interpolators for the static quark-antiquark pair. We construct gauge-equivariant layers for the network and train it to find the ground state of the system. The trained network itself is then treated as our new observable for the inference. Our results demonstrate a significant improvement in the signal compared to traditional Wilson loops, performing as well as Coulomb-gauge Wilson-line correlators while maintaining gauge invariance. Additionally, we present an example where the optimized ground state is used to measure the static force directly, as well as another example combining this method with the multilevel algorithm. Finally, we extend the formalism to find excited-state interpolators for static quark-antiquark systems. To our knowledge, this work is the first study of neural networks with a physically motivated loss function for Wilson loops.

Bellscheidt, Verena [Massachusetts Inst. of Techno↗
NASA NTRS1993

The probabilistic neural network architecture for high speed classification of remotely sensed imagery

In this paper we discuss a neural network architecture (the Probabilistic Neural Net or the PNN) that, to the best of our knowledge, has not previously been applied to remotely sensed data. The PNN is a supervised non-parametric classification algorithm as opposed to the Gaussian maximum likelihood classifier (GMLC). The PNN works by fitting a Gaussian kernel to each training point. The width of the Gaussian is controlled by a tuning parameter called the window width. If very small widths are used, the method is equivalent to the nearest neighbor method. For large windows, the PNN behaves like the GMLC. The basic implementation of the PNN requires no training time at all. In this respect it is far better than the commonly used backpropagation neural network which can be shown to take O(N6) time for training where N is the dimensionality of the input vector. In addition the PNN can be implemented in a feed forward mode in hardware. The disadvantage of the PNN is that it requires all the training data to be stored. Some solutions to this problem are discussed in the paper. Finally, we discuss the accuracy of the PNN with respect to the GMLC and the backpropagation neural network (BPNN). The PNN is shown to be better than GMLC and not as good as the BPNN with regards to classification accuracy.

Chettri, Samir R.↗
DOE OSTI2023

Invertible neural networks for real-time control of extrusion additive manufacturing

Material extrusion additive manufacturing (AM) has enabled an elegant fabrication pathway for a vast material library. Nonetheless, each material requires optimization of printing parameters generally determined through significant trial-and-error testing. To eliminate arduous, iteration-based optimization approaches, many researchers have used machine learning (ML) algorithms which provide opportunities for automated process optimization. Here, in this work, we demonstrate the use of an ML-driven approach for real-time material extrusion print-parameter optimization through in-situ monitoring of printed line geometry. To do this, we use deep invertible neural networks (INNs) which can solve both forward and inverse, or optimization, problems using a single network. By combining in-situ computer vision and deep INNs, the printing parameters can be autonomously optimized to print a target line width in 1.2 s. Furthermore, defects that occur during printing can be rapidly identified and corrected autonomously. The methods developed and presented in this work eliminate user-intensive, time-consuming, and iterative parameter discovery approaches that currently limit accelerated implementation of extrusion-based AM processes. Furthermore, the presented approach can be generalized to provide real-time monitoring and optimization pathways for increasingly complex AM environments.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Toward fusion plasma scenario planning for NSTX-U using machine-learning-accelerated models

One of the most promising devices for realizing power production through nuclear fusion is the tokamak. To maximize performance, it is preferable that tokamak reactors achieve advanced operating scenarios characterized by good plasma confinement, improved magnetohydrodynamic (MHD) stability, and a largely non-inductively driven plasma current. Such scenarios could enable steady-state reactor operation with high \emph{fusion gain} --- the ratio of produced fusion power to the external power provided through the plasma boundary. Precise and robust control of the evolution of the plasma boundary shape as well as the spatial distribution of the plasma current, density, temperature, and rotation will be essential to achieving and maintaining such scenarios. The complexity of the evolution of tokamak plasmas, arising due to nonlinearities and coupling between various parameters, motivates the use of model-based control algorithms that can account for the system dynamics. In this work, a learning-based accelerated model trained on data from the National Spherical Torus Experiment Upgrade (NSTX-U) is employed to develop planning and control strategies for regulating the density and temperature profile evolution around desired trajectories. The proposed model combines empirical scaling laws developed across multiple devices with neural networks trained on empirical data from NSTX-U and a database of first-principles-based computationally intensive simulations. The reduced execution time of the accelerated model will enable practical application of optimization algorithms and reinforcement learning approaches for scenario planning and control development. An initial demonstration of applying optimization approaches to the learning-based model is presented, including a strategy for mitigating the effect of leaving the finite validity range of the accelerated model. The approach shows promise for actuator planning between experiments and in real-time.

machine learning↗
NASA NTRS2000

Neural network recognition of chemical class information in mobility spectra obtained at high temperatures

A minimal neural network was applied to a large library of high-temperature mobility spectra drawn from 16 chemical classes including 154 substances with 2000 spectra at various concentrations. A genetic algorithm was used to create a representative subset of points from the mobility spectrum as input to a cascade-type back-propagation network. This network demonstrated that significant information specific to chemical class was located in the spectral region near the reactant ions. This network failed to generalize the solution to unfamiliar compounds necessitating the use of complete spectra in network processing. An extended back-propagation network classified unfamiliar chemicals by functional group with a mean for average values of 0.83 without sulfides and 0.79 with sulfides. Further experiments confirmed that chemical class information was resident in the spectral region near the reactant ions. Deconvolution of spectra demonstrated the presence of ions, merged with the reactant ion peaks that originated from introduced samples. The ability of the neural network to generalize the solution to unfamiliar compounds suggests that these ions are distinct and class specific.

NASA Discipline Environmental Health↗
NASA NTRS1989

Learning to read aloud: A neural network approach using sparse distributed memory

An attempt to solve a problem of text-to-phoneme mapping is described which does not appear amenable to solution by use of standard algorithmic procedures. Experiments based on a model of distributed processing are also described. This model (sparse distributed memory (SDM)) can be used in an iterative supervised learning mode to solve the problem. Additional improvements aimed at obtaining better performance are suggested.

Joglekar, Umesh Dwarkanath↗