SEARCH · Engineering Papers

Results for “Neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

All sources NASA NTRS DOE OSTI

DOE OSTI2018

Fast inference of deep neural networks in FPGAs for particle physics

Here, recent results at the Large Hadron Collider (LHC) have pointed to enhanced physics capabilities through the improvement of the real-time event processing techniques. Machine learning methods are ubiquitous and have proven to be very powerful in LHC physics, and particle physics as a whole. However, exploration of the use of such techniques in low-latency, low-power FPGA (Field Programmable Gate Array) hardware has only just begun. FPGA-based trigger and data acquisition systems have extremely low, sub-microsecond latency requirements that are unique to particle physics. We present a case study for neural network inference in FPGAs focusing on a classifier for jet substructure which would enable, among many other physics scenarios, searches for new dark sector particles and novel measurements of the Higgs boson. While we focus on a specific example, the lessons are far-reaching. A companion compiler package for this work is developed based on High-Level Synthesis (HLS) called hls4ml to build machine learning models in FPGAs. The use of HLS increases accessibility across a broad user community and allows for a drastic decrease in firmware development time. We map out FPGA resource usage and latency versus neural network hyperparameters to identify the problems in particle physics that would benefit from performing neural network inference with FPGAs. For our example jet substructure model, we fit well within the available resources of modern FPGAs with a latency on the scale of 100 ns.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Effective cosmic density field reconstruction with convolutional neural network

ABSTRACT We present a cosmic density field reconstruction method that augments the traditional reconstruction algorithms with a convolutional neural network (CNN). Following previous work, the key component of our method is to use the reconstructed density field as the input to the neural network. We extend this previous work by exploring how the performance of these reconstruction ideas depends on the input reconstruction algorithm, the reconstruction parameters, and the shot noise of the density field, as well as the robustness of the method. We build an eight-layer CNN and train the network with reconstructed density fields computed from the Quijote suite of simulations. The reconstructed density fields are generated by both the standard algorithm and a new iterative algorithm. In real space at z = 0, we find that the reconstructed field is 90 per cent correlated with the true initial density out to $k\sim 0.5 \, \mathrm{ h}\, \rm {Mpc}^{-1}$, a significant improvement over $k\sim 0.2 \, \mathrm{ h}\, \rm {Mpc}^{-1}$ achieved by the input reconstruction algorithms. We find similar improvements in redshift space, including an improved removal of redshift space distortions at small scales. We also find that the method is robust across changes in cosmology. Additionally, the CNN removes much of the variance from the choice of different reconstruction algorithms and reconstruction parameters. However, the effectiveness decreases with increasing shot noise, suggesting that such an approach is best suited to high density samples. This work highlights the additional information in the density field beyond linear scales as well as the power of complementing traditional analysis approaches with machine learning techniques.

Astronomy & Astrophysics↗

NASA NTRS1991

Electronic neural network for dynamic resource allocation

A VLSI implementable neural network architecture for dynamic assignment is presented. The resource allocation problems involve assigning members of one set (e.g. resources) to those of another (e.g. consumers) such that the global 'cost' of the associations is minimized. The network consists of a matrix of sigmoidal processing elements (neurons), where the rows of the matrix represent resources and columns represent consumers. Unlike previous neural implementations, however, association costs are applied directly to the neurons, reducing connectivity of the network to VLSI-compatible 0 (number of neurons). Each row (and column) has an additional neuron associated with it to independently oversee activations of all the neurons in each row (and each column), providing a programmable 'k-winner-take-all' function. This function simultaneously enforces blocking (excitatory/inhibitory) constraints during convergence to control the number of active elements in each row and column within desired boundary conditions. Simulations show that the network, when implemented in fully parallel VLSI hardware, offers optimal (or near-optimal) solutions within only a fraction of a millisecond, for problems up to 128 resources and 128 consumers, orders of magnitude faster than conventional computing or heuristic search methods.

Thakoor, A. P.↗

DOE OSTI2023

What’s the Difference? The Potential for Convolutional Neural Networks for Transient Detection without Template Subtraction

Abstract We present a study of the potential for convolutional neural networks (CNNs) to enable separation of astrophysical transients from image artifacts, a task known as “real–bogus” classification, without requiring a template-subtracted (or difference) image, which requires a computationally expensive process to generate, involving image matching on small spatial scales in large volumes of data. Using data from the Dark Energy Survey, we explore the use of CNNs to (1) automate the real–bogus classification and (2) reduce the computational costs of transient discovery. We compare the efficiency of two CNNs with similar architectures, one that uses “image triplets” (templates, search, and difference image) and one that takes as input the template and search only. We measure the decrease in efficiency associated with the loss of information in input, finding that the testing accuracy is reduced from ∼96% to ∼91.1%. We further investigate how the latter model learns the required information from the template and search by exploring the saliency maps. Our work (1) confirms that CNNs are excellent models for real–bogus classification that rely exclusively on the imaging data and require no feature engineering task and (2) demonstrates that high-accuracy (>90%) models can be built without the need to construct difference images, but some accuracy is lost. Because, once trained, neural networks can generate predictions at minimal computational costs, we argue that future implementations of this methodology could dramatically reduce the computational costs in the detection of transients in synoptic surveys like Rubin Observatory's Legacy Survey of Space and Time by bypassing the difference image analysis entirely.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Predicting Geologic Behavior in Carbon Storage Projects Using Graph Neural Network

This study was invited to presented at NVIDIA's GTC conference to highlight the potential of Graph Neural Network as a novel and promising methodology for predicting pressure and saturation evolution in carbon storage projects. Carbon capture and storage (CCS) technology plays a pivotal role in mitigating greenhouse gas emissions, facilitating the transition to a low-carbon future. Effective management of subsurface reservoirs is essential to ensure the safe and efficient storage of captured carbon dioxide (CO₂). Accurate predictions of pressure and saturation over time are critical for evaluating the long-term performance and integrity of CCS projects. In recent years, Graph Neural Network (GNN) has emerged as a powerful framework for analyzing complex data in graph-structured domains. This abstract explores the application of GNN to forecast pressure and saturation evolution in carbon storage projects. Traditional numerical simulations of subsurface reservoirs have proven successful in providing pressure and saturation forecasts. However, these simulations involve massive amounts of computational effort and require extensive domain expertise for proper model calibration and validation. Graph Neural Operator offers an alternative approach that harnesses the inherent graph structure of reservoirs, where nodes represent reservoir grid cells and edges represent the geological connectivity between them.

Shih, Chung Yan↗

DOE OSTI2023

Improved Tomographic Binning of 3 × 2 pt Lens Samples: Neural Network Classifiers and Optimal Bin Assignments

Abstract Large imaging surveys, such as the Legacy Survey of Space and Time, rely on photometric redshifts and tomographic binning for 3 × 2 pt analyses that combine galaxy clustering and weak lensing. In this paper, we propose a method for optimizing the tomographic binning choice for the lens sample of galaxies. We divide the CosmoDC2 and Buzzard simulated galaxy catalogs into a training set and an application set, where the training set is nonrepresentative in a realistic way, and then estimate photometric redshifts for the application sets. The galaxies are sorted into redshift bins covering equal intervals of redshift or comoving distance, or with an equal number of galaxies in each bin, and we consider a generalized extension of these approaches. We find that bins of equal comoving distance produce the highest dark energy figure of merit of the initial binning choices, but that the choice of bin edges can be further optimized. We then train a neural network classifier to identify galaxies that are either highly likely to have accurate photometric redshift estimates or highly likely to be sorted into the correct redshift bin. The neural network classifier is used to remove poor redshift estimates from the sample, and the results are compared to the case when none of the sample is removed. We find that the neural network classifiers are able to improve the figure of merit by ∼13% and are able to recover ∼25% of the loss in the figure of merit that occurs when a nonrepresentative training sample is used.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

Abstract Motivation Millions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. Results We developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementation The source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2016

Top-of-the-Atmosphere Shortwave Flux Estimation from Satellite Observations: An Empirical Neural Network Approach Applied with Data from the A-Train Constellation

Estimates of top-of-the-atmosphere (TOA) radiative flux are essential for the understanding of Earth's energy budget and climate system. Clouds, aerosols, water vapor, and ozone (O3) are among the most important atmospheric agents impacting the Earth's shortwave (SW) radiation budget. There are several sensors in orbit that provide independent information related to these parameters. Having coincident information from these sensors is important for understanding their potential contributions. The A-train constellation of satellites provides a unique opportunity to analyze data from several of these sensors. In this paper, retrievals of cloud/aerosol parameters and total column ozone (TCO) from the Aura Ozone Monitoring Instrument (OMI) have been collocated with the Aqua Clouds and Earth's Radiant Energy System (CERES) estimates of total reflected TOA outgoing SW flux (SWF). We use these data to develop a variety of neural networks that estimate TOA SWF globally over ocean and land using only OMI data and other ancillary information as inputs and CERES TOA SWF as the output for training purposes. OMI-estimated TOA SWF from the trained neural networks reproduces independent CERES data with high fidelity. The global mean daily TOA SWF calculated from OMI is consistently within 1% of CERES throughout the year 2007. Application of our neural network method to other sensors that provide similar retrieved parameters, both past and future, can produce similar estimates TOA SWF. For example, the well-calibrated Total Ozone Mapping Spectrometer (TOMS) series could provide estimates of TOA SWF dating back to late 1978.

CERES data with high fidelity↗

DOE OSTI2017

Testing autonomous cyber-physical systems using fuzzing features from convolutional neural networks: work-in-progress

Autonomous cyber-physical systems rely on modern machine learning methods such as deep neural networks to control their interactions with the physical world. Testing of such intelligent cyber-physical systems is a challenge due to the huge state space associated with high-resolution visual sensory inputs. We demonstrate how fuzzing the input using patterns obtained from the convolutional filters of an unrelated convolutional neural network can be used to test computer vision algorithms implemented in intelligent cyber-physical systems. Our method discovers interesting counterexamples to a pedestrian detection algorithm implemented in the popular OpenCV library. Our approach also unearths counterexamples to the correct behavior of an autonomous car similar to NVIDIA's end-to-end self-driving deep neural net running on the Udacity open-source simulator.

Raj, Sunny↗

DOE OSTI2023

Video frame prediction of microbial growth with a recurrent neural network

The recent explosion of interest and advances in machine learning technologies has opened the door to new analytical capabilities in microbiology. Using experimental data such as images or videos, machine learning, in particular deep learning with neural networks, can be harnessed to provide insights and predictions for microbial populations. This paper presents such an application in which a Recurrent Neural Network (RNN) was used to perform prediction of microbial growth for a population of two Pseudomonas aeruginosa mutants. The RNN was trained on videos that were acquired previously using fluorescence microscopy and microfluidics. Of the 20 frames that make up each video, 10 were used as inputs to the network which outputs a prediction for the next 10 frames of the video. The accuracy of the network was evaluated by comparing the predicted frames to the original frames, as well as population curves and the number and size of individual colonies extracted from these frames. Overall, the growth predictions are found to be accurate in metrics such as image comparison, colony size, and total population. Yet, limitations exist due to the scarcity of available and comparable data in the literature, indicating a need for more studies. Both the successes and challenges of our approach are discussed.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS1992

Optical computing and neural networks; Proceedings of the Meeting, National Chiao Tung Univ., Hsinchu, Taiwan, Dec. 16, 17, 1992

The present conference discusses optical neural networks, photorefractive nonlinear optics, optical pattern recognition, digital and analog processors, and holography and its applications. Attention is given to bifurcating optical information processing, neural structures in digital halftoning, an exemplar-based optical neural net classifier for color pattern recognition, volume storage in photorefractive disks, and microlaser-based compact optical neuroprocessors. Also treated are the optical implementation of a feature-enhanced optical interpattern-associative neural network model and its optical implementation, an optical pattern binary dual-rail logic gate module, a theoretical analysis for holographic associative memories, joint transform correlators, image addition and subtraction via the Talbot effect, and optical wavelet-matched filters. (No individual items are abstracted in this volume)

Hsu, Ken-Yuh↗

NASA NTRS1998

Predicting Ecologically Important Vegetation Variables from Remotely Sensed Optical/Radar Data Using Neural Networks

A number of satellite sensor systems will collect large data sets of the Earth's surface during NASA's Earth Observing System (EOS) era. Efforts are being made to develop efficient algorithms that can incorporate a wide variety of spectral data and ancillary data in order to extract vegetation variables required for global and regional studies of ecosystem processes, biosphere-atmosphere interactions, and carbon dynamics. These variables are, for the most part, continuous (e.g. biomass, leaf area index, fraction of vegetation cover, vegetation height, vegetation age, spectral albedo, absorbed photosynthetic active radiation, photosynthetic efficiency, etc.) and estimates may be made using remotely sensed data (e.g. nadir and directional optical wavelengths, multifrequency radar backscatter) and any other readily available ancillary data (e.g., topography, sun angle, ground data, etc.). Using these types of data, neural networks can: 1) provide accurate initial models for extracting vegetation variables when an adequate amount of data is available; 2) provide a performance standard for evaluating existing physically-based models; 3) invert multivariate, physically based models; 4) in a variable selection process, identify those independent variables which best infer the vegetation variable(s) of interest; and 5) incorporate new data sources that would be difficult or impossible to use with conventional techniques. In addition, neural networks employ a more powerful and adaptive nonlinear equation form as compared to traditional linear, index transformations, and simple nonlinear analyses. These neural networks attributes are discussed in the context of the authors' investigations of extracting vegetation variables of ecological interest.

Kimes, Daniel S.↗

NASA NTRS1993

Synthesis of Optimal Nonlinear Feedback Laws for Dynamic Systems Using Neural Networks

Open-loop solutions of dynamical optimization problems can be numerically computed usingexisting software packages. The computed time histories of the state and control variables, formultiple sets of end conditions can then be used to train a neural network to 'recognize' the optimal,nonlinear feedback relation between the states and controls of the system. The 'learned' network canthen be used to output an approximate optimal control given a full set (or a partial set) of measuredsystem states. With simple neural networks, we have successfully demonstrated the efficacy of theproposed approach using a minimum-time orbit injection problem. The usefulness and limitations ofthis novel approach on real-life optimal guidance and control problems, with many state and control variables as well as path inequality constraints, remain to be seen.

Orbit injection problem↗

DOE OSTI2021

Enhancement of Distribution System State Estimation Using Pruned Physics-Aware Neural Networks

Realizing complete observability in the three-phase distribution system remains a challenge that hinders the implementation of classic state estimation algorithms. In this paper, a new method, called the pruned physics-aware neural network (P2N2), is developed to improve the voltage estimation accuracy in the distribution system. The method relies on the physical grid topology, which is used to design the connections between different hidden layers of a neural network model. To verify the proposed method, a numerical simulation based on one- year smart meter data of load consumptions for three-phase power flow is developed to generate the measurement and voltage state data. The IEEE 123-node system is selected as the test network to benchmark the proposed algorithm against the classic weighted least squares (WLS). Numerical results show that P2N2 outperforms WLS in terms of data redundancy and estimation accuracy.

distribution system state estimation↗

NASA NTRS1995

Improved Adjoint-Operator Learning For A Neural Network

Improved method of adjoint-operator learning reduces amount of computation and associated computational memory needed to make electronic neural network learn temporally varying pattern (e.g., to recognize moving object in image) in real time. Method extension of method described in "Adjoint-Operator Learning for a Neural Network" (NPO-18352).

Toomarian, Nikzad↗

DOE OSTI2023

Neural network surrogate models for equations of state

Equation of state (EOS) data provide necessary information for accurate multiphysics modeling, which is necessary for fields such as inertial confinement fusion. Here, we suggest a neural network surrogate model of energy and entropy and use thermodynamic relationships to derive other necessary thermodynamic EOS quantities. We incorporate phase information into the model by training a phase classifier and using phase-specific regression models, which improves the modal prediction accuracy. Our model predicts energy values to 1% relative error and entropy to 3.5% relative error in a log-transformed space. Although sound speed predictions require further improvement, the derived pressure values are accurate within 10% relative error. Our results suggest that neural network models can effectively model EOS for inertial confinement fusion simulation applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Measuring thermal profiles in high explosives using neural networks

We present a new method for calculating the temperature profile of high explosive (HE) material using a Convolutional Neural Network (CNN). To train/test the CNN, we have developed a hybrid experiment/simulation method for collecting acoustic and temperature data. We experimentally heat cylindrical containers of HE material until detonation/deflagration, where we continuously measure the acoustic bursts through the HE using multiple acoustic transducers lined around the exterior container circumference. However, measuring the temperature profile in the HE in an experiment would require inserting a large number of thermal probes, which would disrupt the heating process. Thus, we use two thermal probes, one at the HE center and one at the wall. We then use numerical simulation of the heating process to calculate the temperature distribution and correct the simulated temperatures based on the experimental center and wall temperatures. We calculate temperature errors on the order of 15 °C, which is ∼12% of the range of temperatures in the experiment. We also investigate how the algorithm’s accuracy is affected by the number of acoustic receivers used to collect each measurement and the resolution of the temperature prediction. This work provides a means of assessing the safety status of HE material, which cannot be achieved using existing temperature measurement methods. In addition, it has implications for a range of other applications where internal temperature profile measurements would provide critical information. These applications include detecting chemical reactions, observing thermodynamic processes such as combustion, monitoring metal or plastic casting, determining the energy density in thermal storage capsules, and identifying abnormal battery operations.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1994

Application of neural networks to unsteady aerodynamic control

The problem under consideration in this viewgraph presentation is to understand, predict, and control the fluid mechanics of dynamic maneuvers, unsteady boundary layers, and vortex dominated flows. One solution is the application of neural networks demonstrating closed-loop control. Neural networks offer unique opportunities: simplify modeling of three dimensional, vortex dominated, unsteady separated flow fields; are effective means for controlling unsteady aerodynamics; and address integration of sensors, controllers, and time lags into adaptive control systems.

Faller, William E.↗