Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Harnessing Neural Networks for Elucidating X-ray Absorption Structure–Spectrum Relationships in Amorphous Carbon

Improved understanding of structural and chemical properties through local experimental probes, such as X-ray absorption near-edge structure (XANES) spectroscopy, is crucial for the understanding and design of functional materials. In recent years, significant advancements have been made in the development of data science approaches for the automated interpretation of XANES structure–spectrum relationships. However, existing studies have primarily focused on crystalline solids and small molecules, while fewer efforts have been devoted to disordered systems. Thus, in this work, we demonstrate the development of neural network models for predicting and interpreting XANES spectra of amorphous carbon (a-C) from local structural descriptors. Comparison between different structural descriptors expectedly shows that the inclusion of both bond length and bond angle information is necessary for an accurate prediction of the spectra. Among the descriptors considered in this work, we find that the local many-body tensor representation yields the highest accuracy and greatest interpretability so that it can be leveraged to understand the importance of structural motifs in determining XANES spectra. Furthermore, we also discuss performance of neural network models for predicting both local structure features, such as bond lengths and bond angles, and global chemical composition, such as the sp:sp 2 :sp 3 ratio.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Fast convolutional neural networks on FPGAs with hls4ml

We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on field-programmable gate arrays (FPGAs). By extending the hls4ml library, we demonstrate an inference latency of 5 µs using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Numbers Dataset, we demonstrate various methods for model compression in order to fit the computational constraints of a typical FPGA device used in trigger and data acquisition systems of particle detectors. In particular, we discuss pruning and quantization-aware training, and demonstrate how resource utilization can be significantly reduced with little to no loss in model accuracy. We show that the FPGA critical resource consumption can be reduced by 97% with zero loss in model accuracy, and by 99% when tolerating a 6% accuracy degradation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

3D-equivariant graph neural networks for protein model quality assessment

Quality assessment (QA) of predicted protein tertiary structure models plays an important role in ranking and using them. With the recent development of deep learning end-to-end protein structure prediction techniques for generating highly confident tertiary structures for most proteins, it is important to explore corresponding QA strategies to evaluate and select the structural models predicted by them since these models have better quality and different properties than the models predicted by traditional tertiary structure prediction methods. We develop EnQA, a novel graph-based 3D-equivariant neural network method that is equivariant to rotation and translation of 3D objects to estimate the accuracy of protein structural models by leveraging the structural features acquired from the state-of-the-art tertiary structure prediction method—AlphaFold2. We train and test the method on both traditional model datasets (e.g. the datasets of the Critical Assessment of Techniques for Protein Structure Prediction) and a new dataset of high-quality structural models predicted only by AlphaFold2 for the proteins whose experimental structures were released recently. Our approach achieves state-of-the-art performance on protein structural models predicted by both traditional protein structure prediction methods and the latest end-to-end deep learning method—AlphaFold2. It performs even better than the model QA scores provided by AlphaFold2 itself. The results illustrate that the 3D-equivariant graph neural network is a promising approach to the evaluation of protein structural models. Integrating AlphaFold2 features with other complementary sequence and structural features is important for improving protein model QA.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Detection of Isotopes in Urban Source Search Low-Count Gamma Spectra Using Hopfield Neural Networks

Source search campaigns involve measurements of background gamma-ray spectra with a mobile detector-spectrometer traveling along arbitrarily chosen trajectories over a wide screening area. Radiation counts are typically measured with a tellurium-doped sodium iodide [NaI(Tl)] scintillator detector-spectrometer in short acquisition intervals, usually 1 s. The objective is to detect orphan isotopes with half-lives shorter than those of the isotopes in the natural background. In principle, radioisotopes can be identified by their unique gamma emission spectrum. However, detecting orphan isotopes in search data is challenging because low counts measured in short acquisition intervals result in incomplete spectral lines. In this study, we investigate the performance of a Hopfield neural network (HNN) that implements an auto-associative memory for the detection of isotopes of interest in an urban search campaign. The HNN is trained on one example of gamma spectra with well-resolved spectral lines of each isotope of interest. During testing, the auto-associative memory implementation of the HNN processes low-count gamma spectra with partially complete isotopic lines by matching incoming measurements to the closest one of its memory-stored patterns. The testing database consisted of almost 10 000 1-s gamma spectra, including measurements of orphan isotopes 137 Cs, 241 Am, and 131 I, obtained during two urban search surveys with a NaI(Tl) detector. The performance of the HNN detection algorithm was evaluated using precision, recall, and F1 scores, and benchmarked with a multiple linear regression (MLR) identification algorithm. In conclusion, the test results demonstrate that HNN outperforms MLR in the detection of all the isotopes of interest.

Auto associative memory↗
DOE OSTI2024

Machine learning the electric field response of condensed phase systems using perturbed neural network potentials

Abstract The interaction of condensed phase systems with external electric fields is of major importance in a myriad of processes in nature and technology, ranging from the field-directed motion of cells (galvanotaxis), to geochemistry and the formation of ice phases on planets, to field-directed chemical catalysis and energy storage and conversion systems including supercapacitors, batteries and solar cells. Molecular simulation in the presence of electric fields would give important atomistic insight into these processes but applications of the most accurate methods such as ab-initio molecular dynamics (AIMD) are limited in scope by their computational expense. Here we introduce Perturbed Neural Network Potential Molecular Dynamics (PNNP MD) to push back the accessible time and length scales of such simulations. We demonstrate that important dielectric properties of liquid water including the field-induced relaxation dynamics, the dielectric constant and the field-dependent IR spectrum can be machine learned up to surprisingly high field strengths of about 0.2 V Å −1 without loss in accuracy when compared to ab-initio molecular dynamics. This is remarkable because, in contrast to most previous approaches, the two neural networks on which PNNP MD is based are exclusively trained on molecular configurations sampled from zero-field MD simulations, demonstrating that the networks not only interpolate but also reliably extrapolate the field response. PNNP MD is based on rigorous theory yet it is simple, general, modular, and systematically improvable allowing us to obtain atomistic insight into the interaction of a wide range of condensed phase systems with external electric fields.

Science & Technology - Other Topics↗
DOE OSTI2025

Improved heavy-ion PID using scintillation light detector with neural network analysis: a Monte Carlo simulation study

The photon collection efficiency of gaseous scintillator detectors varies according to the position of the impinging charged particles in the medium that generates scintillation light. Thus, when impinging particles are distributed over a large area, the intrinsic photon-number resolution of the system is affected by a large variation. This work presents and discusses a method for adjusting the total number of detected photons to account for variation in the photon collection efficiency as a function of the position of the light source within the scintillating medium. The method was developed and validated by processing data from systematic simulation studies based on GEANT4 that model the response of the Energy Loss Optical Scintillation System (ELOSS) detector. The position of the charged particle is calculated using a deep neural network algorithm. This is accomplished by analyzing the distribution of scintillation light recorded by the array of photosensors. The estimated particle position is then used to calculate the correction factor and adjust the amount of captured light to account for variations in the photon collection efficiency. The neural network algorithm provides excellent tracking capabilities, achieving sub-millimeter position resolution and an angular resolution of 12 mrad, approaching the performance of traditional tracking detectors (e.g., drift chambers). The present method can be generalized to any optical scintillation system where the photon collection efficiency depends on the position of the impinging particle.

Heavy-ion detectors↗
NASA NTRS2019

Carbon Nanotube Gas Sensor Using Neural Networks

The need to identify the presence and quantify the concentrations of gases and vapors is ubiquitous in NASA missions and societal applications. Sensors for air quality monitoring in crew cabins and ISS have been actively under development (Ref. 1). In particular, measuring the concentration of CO2 and NH3 is important because high concentrations of these gases pose a risk to ISS crew health. Detection of fuel and oxidant leaks in crew vehicles is critical for ensuring mission safety. Accurate gas and vapor concentrations can be measured, but this typically requires bulky and expensive instrumentation. Recently, inexpensive sensors with low power demands have been fabricated for use on the International Space Station (ISS). Carbon Nanotube (CNT) based chemical sensors are one type of these sensors. CNT sensors meet the requirements for low cost and ease of fabrication for deployment on the ISS. However, converting the measured signal from the sensors to human readable indicators of atmospheric air quality and safety is challenging. This is because it is difficult to develop an analytical model that maps the CNT sensor output signal to gas concentration. Training a neural network on CNT sensor data to predict gas concentration is more effective than developing an analytic approach to calculate the concentration from the same data set. With this in mind a neural network was created to tackle this challenge of converting the measured signal into CO2 and NH3 concentration values.

Norman, Thaddeus J.↗
DOE OSTI2022

Fast Gaussian Process Estimation for Large-Scale In Situ Inference using Convolutional Neural Networks

Exascale computing will bring with it significant I/O limitations. One foreseeable consequence of such restrictions is that the user can save only a small fraction of complex simulation data to disk for subsequent analysis. An alternative is to fit statistical models to data in situ, that is, inside the simulation as it runs. This option requires extremely fast statistical estimation to avoid slowing down the simulation. Gaussian processes (GPs) have state-of-the-art predictive performance for modeling spatial data. However, standard estimation methods for GPs scale quite poorly to large data sets as parameter estimation requires inverting a covariance matrix to the size of the data set. In the presented work, we use a convolutional neural network (CNN) to predict the GP parameters for a spatial data set, from a simulation or otherwise, rather than optimize the parameters directly. Here, our presented case study models spatial data from E3SM, the Department of Energy’s Exascale climate model. The CNN is trained on synthetic data simulated from GP models with known parameters and then applied to data from the climate simulation. In the presented examples, the neural network scheme produces parameter estimates that compare well with standard methods such as maximum likelihood estimation in predictive performance but is obtained four orders of magnitude faster.

big data↗
NASA NTRS1995

Lyapunov exponents from CHUA's circuit time series using artificial neural networks

In this paper we present the general problem of identifying if a nonlinear dynamic system has a chaotic behavior. If the answer is positive the system will be sensitive to small perturbations in the initial conditions which will imply that there is a chaotic attractor in its state space. A particular problem would be that of identifying a chaotic oscillator. We present an example of three well known different chaotic oscillators where we have knowledge of the equations that govern the dynamical systems and from there we can obtain the corresponding time series. In a similar example we assume that we only know the time series and, finally, in another example we have to take measurements in the Chua's circuit to obtain sample points of the time series. With the knowledge about the time series the phase plane portraits are plotted and from them, by visual inspection, it is concluded whether or not the system is chaotic. This method has the problem of uncertainty and subjectivity and for that reason a different approach is needed. A quantitative approach is the computation of the Lyapunov exponents. We describe several methods for obtaining them and apply a little known method of artificial neural networks to the different examples mentioned above. We end the paper discussing the importance of the Lyapunov exponents in the interpretation of the dynamic behavior of biological neurons and biological neural networks.

Gonzalez, J. Jesus↗
DOE OSTI2024

Neural Networks to Find the Optimal Forcing for Offsetting the Anthropogenic Climate Change Effects

Abstract Of great relevance to climate engineering is the systematic relationship between the radiative forcing to the climate system and the response of the system, a relationship often represented by the linear response function (LRF) of the system. However, estimating the LRF often becomes an ill-posed inverse problem due to high-dimensionality and nonunique relationships between the forcing and response. Recent advances in machine learning make it possible to address the ill-posed inverse problem through regularization and sparse system fitting. Here, we develop a convolutional neural network (CNN) for regularized inversion. The CNN is trained using the surface temperature responses from a set of Green’s function perturbation experiments as imagery input data together with data sample densification. The resulting CNN model can infer the forcing pattern responsible for the temperature response from out-of-sample forcing scenarios. This promising proof of concept suggests a possible strategy for estimating the optimal forcing to negate certain undesirable effects of climate change. The limited success of this effort underscores the challenges of solving an inverse problem for a climate system with inherent nonlinearity. Significance Statement Predicting the climate response for a given climate forcing is a direct problem, while inferring the forcing for a given desired climate response is often an inverse, ill-posed, problem, posing a new challenge to the climate community. This study makes the first attempt to infer the radiative forcing for a given target pattern of global surface temperature response using a deep learning approach. The resulting deeply trained convolutional neural network inversion model shows promise in capturing the forcing pattern corresponding to a given surface temperature response, with a significant implication on the design of an optimal solar radiation management strategy for curbing global warming. This study also highlights the technical challenges that future research should prioritize in seeking feasible solutions to the inverse climate problem.

Ren, Huiying↗
DOE OSTI2021

Identifying Genomic Islands with Deep Neural Networks

Background Horizontal gene transfer is the main source of adaptability for bacteria, through which genes are obtained from different sources including bacteria, archaea, viruses, and eukaryotes. This process promotes the rapid spread of genetic information across lineages, typically in the form of clusters of genes referred to as genomic islands (GIs). Different types of GIs exist, and are often classified by the content of their cargo genes or their means of integration and mobility. While various computational methods have been devised to detect different types of GIs, no single method is capable of detecting all types. Results We propose a method, which we call Shutter Island, that uses a deep learning model (Inception V3, widely used in computer vision) to detect genomic islands. The intrinsic value of deep learning methods lies in their ability to generalize. Via a technique called transfer learning, the model is pre-trained on a large generic dataset and then re-trained on images that we generate to represent genomic fragments. We demonstrate that this image-based approach generalizes better than the existing tools. Conclusions We used a deep neural network and an image-based approach to detect the most out of the correct GI predictions made by other tools, in addition to making novel GI predictions. The fact that the deep neural network was re-trained on only a limited number of GI datasets and then successfully generalized indicates that this approach could be applied to other problems in the field where data is still lacking or hard to curate.

Computer Vision↗
NASA NTRS1993

Application of a neural network as a potential aid in predicting NTF pump failure

The National Transonic Facility has three centrifugal multi-stage pumps to supply liquid nitrogen to the wind tunnel. Pump reliability is critical to facility operation and test capability. A highly desirable goal is to be able to detect a pump rotating component problem as early as possible during normal operation and avoid serious damage to other pump components. If a problem is detected before serious damage occurs, the repair cost and downtime could be reduced significantly. A neural network-based tool was developed for monitoring pump performance and aiding in predicting pump failure. Once trained, neural networks can rapidly process many combinations of input values other than those used for training to approximate previously unknown output values. This neural network was applied to establish relationships among the critical frequencies and aid in predicting failures. Training pairs were developed from frequency scans from typical tunnel operations. After training, various combinations of critical pump frequencies were propagated through the neural network. The approximated output was used to create a contour plot depicting the relationships of the input frequencies to the output pump frequency.

Rogers, James L.↗
DOE OSTI2022

Discovering Ca II absorption lines with a neural network

Quasar absorption line analysis is critical for studying gas and dust components and their physical and chemical properties as well as the evolution and formation of galaxies in the early universe. Calcium II (Ca II ) absorbers, which are one of the dustiest absorbers and are located at lower redshifts than most other absorbers, are especially valuable when studying physical processes and conditions in recent galaxies. However, the number of known quasar Ca II absorbers is relatively low due to the difficulty of detecting them with traditional methods. In this work, we developed an accurate and quick approach to search for Ca II absorption lines using deep learning. In our deep learning model, a convolutional neural network, tuned using simulated data, is used for the classification task. The simulated training data are generated by inserting artificial Ca II absorption lines into original quasar spectra from the Sloan Digital Sky Survey (SDSS), while an existing Ca II catalogue is adopted as the test set. The resulting model achieves an accuracy of 96 per cent on the real data in the test set. Our solution runs thousands of times faster than traditional methods, taking a fraction of a second to analyse thousands of quasars, while traditional methods may take days to weeks. The trained neural network is applied to quasar spectra from SDSS’s DR7 and DR12 and discovered 399 new quasar Ca II absorbers. In addition, we confirmed 409 known quasar Ca II absorbers identified previously by other research groups through traditional methods.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

EFIT-Prime: Probabilistic and physics-constrained reduced-order neural network model for equilibrium reconstruction in DIII-D

We introduce EFIT-Prime, a novel machine learning surrogate model for EFIT (Equilibrium FIT) that integrates probabilistic and physics-informed methodologies to overcome typical limitations associated with deterministic and ad hoc neural network architectures. EFIT-Prime utilizes a neural architecture search-based deep ensemble for robust uncertainty quantification, providing scalable and efficient neural architectures that comprehensively quantify both data and model uncertainties. Physically informed by the Grad–Shafranov equation, EFIT-Prime applies a constraint on the current density J tor and a smoothness constraint on the first derivative of the poloidal flux, ensuring physically plausible solutions. Furthermore, the spatial location of the diagnostics is explicitly incorporated in the inputs to account for their spatial correlation. Extensive evaluations demonstrate EFIT-Prime's accuracy and robustness across diverse scenarios, most notably showing good generalization on negative-triangularity discharges that were excluded from training. Timing studies indicate an ensemble inference time of 15 ms for predicting a new equilibrium, offering the possibility of plasma control in real-time, if the model is optimized for speed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS2006

Structural Health Monitoring and Impact Detection Using Neural Networks for Damage Characterization

Detection of damage due to foreign object impact is an important factor in the development of new aerospace vehicles. Acoustic waves generated on impact can be detected using a set of piezoelectric transducers, and the location of impact can be determined by triangulation based on the differences in the arrival time of the waves at each of the sensors. These sensors generate electrical signals in response to mechanical motion resulting from the impact as well as from natural vibrations. Due to electrical noise and mechanical vibration, accurately determining these time differentials can be challenging, and even small measurement inaccuracies can lead to significant errors in the computed damage location. Wavelet transforms are used to analyze the signals at multiple levels of detail, allowing the signals resulting from the impact to be isolated from ambient electromechanical noise. Data extracted from these transformed signals are input to an artificial neural network to aid in identifying the moment of impact from the transformed signals. By distinguishing which of the signal components are resultant from the impact and which are characteristic of noise and normal aerodynamic loads, the time differentials as well as the location of damage can be accurately assessed. The combination of wavelet transformations and neural network processing results in an efficient and accurate approach for passive in-flight detection of foreign object damage.

Ross, Richard W.↗
NASA NTRS2003

System Identification of X-33 Neural Network

Modern flight control research has improved spacecraft survivability as its goal. To this end we need to have a failure detection system on board. In case the spacecraft is performing imperfectly, reconfiguration of control is needed. For that purpose we need to have parameter identification of spacecraft dynamics. Parameter identification of a system is called system identification. We treat the system as a black box which receives some inputs that lead to some outputs. The question is: what kind of parameters for a particular black box can correlate the observed inputs and outputs? Can these parameters help us to predict the outputs for a new given set of inputs? This is the basic problem of system identification. The X33 was supposed to have the onboard capability of evaluating the current performance and if needed to take the corrective measures to adapt to desired performance. The X33 is comprised of both rocket and aircraft vehicle design characteristics and requires, in general, analytical methods for evaluating its flight performance. Its flight consists of four phases: ascent, transition, entry and TAEM (Terminal Area Energy Management). It spends about 200 seconds in ascent phase, reaching an altitude of about 180,000 feet and a speed of about 10 to 15 Mach. During the transition phase which lasts only about 30 seconds, its altitude may increase to about 190,000 feet but its speed is reduced to about 9 Mach. At the beginning of this phase, the Main Engine is Cut Off (MECO) and the control is reconfigured with the help of aerosurfaces (four elevons, two flaps and two rudders) and reaction control system (RCS). The entry phase brings down the altitude of X33 to about 90,000 feet and its speed to about Mach 3. It spends about 250 seconds in this phase. Main engine is still cut off and the vehicle is controlled by complex maneuvers of aerosurfaces. The last phase TAEM lasts for about 450 seconds and the altitude and speed, both are reduced to zero. The present attempt, as a start, focuses only on the entry phase. Since the main engine remains cut off in this phase, there is no thrust acting on the system. This considerably simplifies the equations of motion. We introduce another simplification by assuming the system to be linear after some non-linearities are removed analytically from our consideration. Under these assumptions, the problem could be solved by Classical Statistics by employing the least sum of squares approach. Instead we chose to use the Neural Network method. This method has many advantages. It is modern, more efficient, can be adapted to work even when the assumptions are diluted. In fact, Neural Networks try to model the human brain and are capable of pattern recognition.

Aggarwal, Shiv↗
DOE OSTI2020

Neural network based prediction of no-wall β N limits due to ideal external kink instabilities

Multi-layer perceptron based neural networks (NNs) are trained to predict the Troyon no-wall beta limits due to the onset of low-n (n = 1 – 3, n is the toroidal mode number) ideal external kink instabilities in tokamak plasmas. It is revealed that a well-trained NN can predict the n = 1 no-wall beta limit within 10% relative error. The NN performance is somewhat worse for the n = 2 and 3 no-wall beta limits, but still a relative error within 20% is achievable. The trained NNs well reproduce the known dependences of the no-wall beta limits on the plasma pressure (pressure peaking factor) and current (plasma internal inductance) profiles. Other scalings are further easily established with NNs, for parametric dependences such as on the aspect ratio, the elongation and triangularity of the plasma boundary shape. These NN-based Troyon beta limit predictors can be incorporated into integrated modeling platforms, or directly implemented as a real time stability estimator for the purpose of disruption avoidance or mitigation during experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Multi-task graph neural networks for simultaneous prediction of global and atomic properties in ferromagnetic systems *

Abstract We introduce a multi-tasking graph convolutional neural network, HydraGNN, to simultaneously predict both global and atomic physical properties and demonstrate with ferromagnetic materials. We train HydraGNN on an open-source ab initio density functional theory (DFT) dataset for iron-platinum with a fixed body centered tetragonal lattice structure and fixed volume to simultaneously predict the mixing enthalpy (a global feature of the system), the atomic charge transfer, and the atomic magnetic moment across configurations that span the entire compositional range. By taking advantage of underlying physical correlations between material properties, multi-task learning (MTL) with HydraGNN provides effective training even with modest amounts of data. Moreover, this is achieved with just one architecture instead of three, as required by single-task learning (STL). The first convolutional layers of the HydraGNN architecture are shared by all learning tasks and extract features common to all material properties. The following layers discriminate the features of the different properties, the results of which are fed to the separate heads of the final layer to produce predictions. Numerical results show that HydraGNN effectively captures the relation between the configurational entropy and the material properties over the entire compositional range. Overall, the accuracy of simultaneous MTL predictions is comparable to the accuracy of the STL predictions. In addition, the computational cost of training HydraGNN for MTL is much lower than the original DFT calculations and also lower than training separate STL models for each property.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗