Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural Networks (Computer)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65

Super resolution convolutional neural network for feature extraction in spectroscopic data

Two dimensional (2D) peak finding is a common practice in data analysis for physics experiments, which is typically achieved by computing the local derivatives. However, this method is inherently unstable when the local landscape is complicated or the signal-to-noise ratio of the data is low. In this work, we propose a new method in which the peak tracking task is formalized as an inverse problem, which thus can be solved with a convolutional neural network (CNN). In addition, we show that the underlying physics principle of the experiments can be used to generate the training data. By generalizing the trained neural network on real experimental data, we show that the CNN method can achieve comparable or better results than traditional derivative based methods. This approach can be further generalized in different physics experiments when the physical process is known.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection

Deep learning has recently demonstrated state-of-the art performance on key tasks related to the maintenance of computer systems, such as intrusion detection, denial of service attack detection, hardware and software system failures, and malware detection. In these contexts, model interpretability is vital for administrator and analyst to trust and act on the automated analysis of machine learning models. Deep learning methods have been criticized as black box oracles which allow limited insight into decision factors. In this work we seek to "bridge the gap'' between the impressive performance of deep learning models and the need for interpretable model introspection. To this end we present recurrent neural network (RNN) language models augmented with attention for anomaly detection in system logs. Our methods are generally applicable to any computer system and logging source. By incorporating attention variants into our RNN language models we create opportunities for model introspection and analysis without sacrificing state-of-the art performance. We demonstrate model performance and illustrate model interpretability on an intrusion detection task using the Los Alamos National Laboratory (LANL) cyber security dataset, reporting upward of 0.99 area under the receiver operator characteristic curve despite being trained only on a single day's worth of data.

Anomaly detection, Attention, Recurrent Neural Net↗

Moiré Patterns in Pt Overlayers on Gold: A Graph Neural Network Interatomic Potential Study

Overlayer structures in bimetallic catalysts are relevant to a variety of catalytic reactions, particularly in electrocatalysis for fuel cell applications. Previous computational studies largely consider these overlayer structures to be those of a pseudomorphic overlayer, where there is a 1:1 atomic ratio between the overlayer and the support metal. Our previous work based on density functional theory (DFT) has shown that there exist nonstoichiometric overlayer structures that are more stable than the stoichiometric ones. Here, in this work, we developed a graph neural network interatomic potential (GNN-IP) to analyze structures formed in Pt overlayers on Au(111). The GNN-IP was used to explore properties of nonstoichiometric overlayers at length scales that are prohibitively expensive to pursue with planewave DFT calculations. In particular, we examined large Pt islands on top of a 48 × 48 Au(111) unit cell to explore the influence of the rotational angle (α) between the Pt overlayer and support Au(111) on both the stability of and the preferred atomic density in the overlayer. Island structures with smaller rotational angles between the Pt overlayer and Au(111) tend to be more stable. Further, smaller rotational angles tend to result in lower atomic density in the Pt overlayer.

chemical structure↗

Augmented cognition tool for rapid military decision making.

This report describes the laboratory directed research and development work to model relevant areas of the brain that associate multi-modal information for long-term storage for the purpose of creating a more effective, and more automated, association mechanism to support rapid decision making. Using the biology and functionality of the hippocampus as an analogy or inspiration, we have developed an artificial neural network architecture to associate k-tuples (paired associates) of multimodal input records. The architecture is composed of coupled unimodal self-organizing neural modules that learn generalizations of unimodal components of the input record. Cross modal associations, stored as a higher-order tensor, are learned incrementally as these generalizations form. Graph algorithms are then applied to the tensor to extract multi-modal association networks formed during learning. Doing so yields a novel approach to data mining for knowledge discovery. This report describes the neurobiological inspiration, architecture, and operational characteristics of our model, and also provides a real world terrorist network example to illustrate the model's functionality.

99 GENERAL AND MISCELLANEOUS↗

Boosting the Performance of Plug-and-Play Priors via Denoiser Scaling

Plug-and-play priors (PnP) is an image reconstruction framework that utilizes an image denoiser as an imaging prior. Unlike traditional regularized inversion, PnP does not require the prior to be expressible in the form of a regularization function. This flexibility enables PnP algorithms to exploit the most effective image denoisers, leading to their state-of-the-art performance in various imaging tasks. However, many powerful denoisers, such as the ones based on convolutional neural networks (CNNs), do not have tunable parameters that would allow controlling their influence within PnP. To address this issue, in this paper, we introduce a scaling parameter that adjusts the magnitude of the denoiser input and output. We theoretical justify the denoiser scaling from the perspectives of proximal optimization, statistical estimation, and consensus equilibrium. Finally, we provide numerical experiments demonstrating the ability of denoiser scaling to systematically improve the performance of PnP for denoising CNN priors that do not have explicitly tunable parameters.

Boosting↗

Stimulus Sensitivity of a Spiking Neural Network Model

Some recent papers relate the criticality of complex systems to their maximal capacity of information processing. In the present paper, we consider high dimensional point processes, known as age-dependent Hawkes processes, which have been used to model spiking neural networks. Using mean-field approximation, the response of the network to a stimulus is computed and we provide a notion of stimulus sensitivity. It appears that the maximal sensitivity is achieved in the sub-critical regime, yet almost critical for a range of biologically relevant parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Particle hit clustering and identification using point set transformers in liquid argon time projection chambers

Liquid argon time projection chambers are often used in neutrino physics and dark-matter searches because of their high spatial resolution. The images generated by these detectors are extremely sparse, as the energy values detected by most of the detector are equal to 0, meaning that despite their high resolution, most of the detector is unused in a particular interaction. Instead of representing all of the empty detections, the interaction is usually stored as a sparse matrix, a list of detection locations paired with their energy values. Traditional machine learning methods that have been applied to particle reconstruction such as convolutional neural networks (CNNs), however, cannot operate over data stored in this way and therefore must have the matrix fully instantiated as a dense matrix. Operating on dense matrices requires a lot of memory and computation time, in contrast to directly operating on the sparse matrix. We propose a machine learning model using a point set neural network that operates over a sparse matrix, greatly improving both processing speed and accuracy over methods that instantiate the dense matrix, as well as over other methods that operate over sparse matrices. Compared to competing state-of-the-art methods, our method improves classification performance by 14%, segmentation performance by more than 22%, while taking 80% less time and using 66% less memory. Compared to state-of-the-art CNN methods, our method improves classification performance by more than 86%, segmentation performance by more than 71%, while reducing runtime by 91% and reducing memory usage by 61%.

calibration and fitting methods↗

A gimbal platform stabilization for topographic applications

The aim of this work is the stabilization of a Gimbal platform for optical sensors acquisitions in topographic applications using mobile vehicles. The stabilization of the line of sight (LOS) consists in tracking the command velocity in presence of nonlinear noise due to the external environment. The hardware architecture is characterized by an Ardupilot platform that allows the control of both the mobile device and the Gimbal. Here we developed a new approach to stabilize the Gimbal platform, which is based on neural network. For the control system, we considered a plant that represents the transfer function of the servo system control model for an inertial stabilized Gimbal platform. The transductor used in the feed-back line control is characterized by the Rate Gyro transfer function installed onboard of Ardupilot. For the simulation and investigation of the system performance, we used the Simulink tool of Matlab. Results show that the hardware/software approach is efficient, reliable and cheap for direct photogrammetry, as well as for general purpose applications using mobile vehicles.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Physics informed machine learning for chemistry tabulation

Modeling of turbulent combustion system requires modeling the underlying chemistry and the turbulent transport. Solving both systems simultaneously is computationally prohibitive. Instead, given the difference in scales at which the two sub-systems evolve, the two sub-systems are typically (re)solved separately. Popular approaches such as the Flamelet Generated Manifolds (FGM) use a two-step strategy where the governing reaction kinetics are pre-computed and mapped to a low-dimensional manifold, characterized by a few reaction progress variables (model reduction) and the manifold is then “looked-up” during the run-time to estimate the high-dimensional system state by the turbulent transport system. While existing works have focused on these two steps independently, in this work we show that joint learning of the progress variables and the look-up model, can yield more accurate results. Here, we build on the base formulation and implementation to include the dynamically generated Thermochemical State Variables (Lower Dimensional Dynamic Source Terms). We discuss the challenges in the implementation of this deep neural network architecture and experimentally demonstrate its superior performance.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Identifying Planetary Transit Candidates in TESS Full-frame Image Light Curves via Convolutional Neural Networks

The Transiting Exoplanet Survey Satellite(TESS)mission measured light from stars in∼75% of the sky throughout its 2 yr primary mission, resulting in millions of TESS 30-minute-cadence light curves to analyze in the search for transiting exoplanets. To search this vast data trove for transit signals, we aim to provide an approach that both is computationally efficient and produces highly performant predictions. This approach minimizes the required human search effort. We present a convolutional neural network, which we train to identify planetary transit signals and dismiss false positives. To make a prediction for a given light curve, our network requires no prior transit parameters identified using other methods. Our network performs inference on a TESS 30-minute-cadence light curve in∼5 ms on a single GPU, enabling large-scale archival searches. We present 181 new planet candidates identified by our network, which pass subsequent human vetting designed to rule out false positives.Our neural network model is additionally provided as open-source code for public use and extension

Gregory Olmschenk↗

Efficient data acquisition and training of collisional-radiative model artificial neural network surrogates through adaptive parameter space sampling

Abstract Effective plasma transport modeling of magnetically confined fusion devices relies on having an accurate understanding of the ion composition and radiative power losses of the plasma. Generally, these quantities can be obtained from solutions of a collisional-radiative (CR) model at each time step within a plasma transport simulation. However, even compact, approximate CR models can be computationally onerous to evaluate, and in-situ evaluation of these models within a larger plasma transport code can lead to a rigid bottleneck. As a way to bypass this bottleneck, we propose deploying artificial neural network (ANN) surrogates to allow rapid evaluation of the necessary plasma quantities. However, one issue with training an accurate ANN surrogate is the reliance on a sufficiently large and representative training and validation data set, which can be time-consuming to generate. In this work we explore a data-driven active learning and training routine to allow autonomous adaptive sampling of the problem parameter space to ensure a sufficiently large and meaningful set of training data is assembled for the network training. As a result, we can demonstrate approximately order-of-magnitude savings in required training data samples to produce an accurate surrogate.

97 MATHEMATICS AND COMPUTING↗

Modeling design and control problems involving neural network surrogates

Here, we consider nonlinear optimization problems that involve surrogate models represented by neural networks. We demonstrate first how to directly embed neural network evaluation into optimization models, highlight a difficulty with this approach that can prevent convergence, and then characterize stationarity of such models. We then present two alternative formulations of these problems in the specific case of feedforward neural networks with ReLU activation: as a mixed-integer optimization problem and as a mathematical program with complementarity constraints. For the latter formulation we prove that stationarity at a point for this problem corresponds to stationarity of the embedded formulation. Each of these formulations may be solved with state-of-the-art optimization methods, and we show how to obtain good initial feasible solutions for these methods. We compare our formulations on three practical applications arising in the design and control of combustion engines, in the generation of adversarial attacks on classifier networks, and in the determination of optimal flows in an oil well network.

97 MATHEMATICS AND COMPUTING↗

Acquisition and Representation of Spatio-Temporal Signals in Polychronizing Spiking Neural Networks

The ability of an intelligent agent to process complex signals such as those found in audio or video depends heavily on the nature of the internal representation of the relevant information. Our report explores the mechanisms underlying this process by investigating theories inspired by the function of the neocortex. Specifically, we focus on the phenomenon of polychronization, which describes the self-organization in a spiking neural network resulting from the interplay between network structure, driven spiking activity, and synaptic plasticity. What emerges are groups of neurons that exhibit reproducible, time-locked patterns of spiking activity. We propose that this representation is well suited to spatio-temporal signal processing, as it naturally resembles patterns found in real-world signals. We explore the computational properties of this method and demonstrate the ability of a simple polychronizing network to learn different spatio-temporal signals.

97 MATHEMATICS AND COMPUTING↗

Neuromorphic computing for temporal scientific data classification

In this work, we apply a spiking neural network model and an associated memristive neuromorphic implementation to an application in classifying temporal scientific data. We demonstrate that the spiking neural network model achieves comparable results to a previously reported convolutional neural network model, with significantly fewer neurons and synapses required.

Schuman, Catherine↗

Seismic Waveform Inversion Capability on Resource-Constrained Edge Devices

Seismic full wave inversion (FWI) is a widely used non-linear seismic imaging method used to reconstruct subsurface velocity images, however it is time consuming, has high computational cost and depend heavily on human interaction. Recently, deep learning has accelerated it’s use in several data-driven techniques, however most deep learning techniques suffer from overfitting and stability issues. In this work, we propose an edge computing-based data-driven inversion technique based on supervised deep convolutional neural network to accurately reconstruct the subsurface velocities. Deep learning based data-driven technique depends mostly on bulk data training. In this work, we train our deep convolutional neural network (DCN) (UNet and InversionNet) on the raw seismic data and their corresponding velocity models during the training phase to learn the non-linear mapping between the seismic data and velocity models. The trained network is then used to estimate the velocity models from new input seismic data during the prediction phase. The prediction phase is performed on a resource-constrained edge device such as Raspberry Pi. Raspberry Pi provides real-time and on-device computational power to execute the inference process. In addition, we demonstrate robustness of our models to perform inversion in the presence on noise by performing both noise-aware and no-noise training and feeding the resulting trained models with noise at different signal-to-noise (SNR) ratio values. We make great efforts to achieve very feasible inference times on the Raspberry Pi for both models. Specifically, the inference times per prediction for UNet and InversionNet models on Raspberry Pi were 22 and 4 s respectively whilst inference times for both models on the GPU were 2 and 18 s which are very comparable. Finally, we have designed a user-friendly interactive graphical user interface (GUI) to automate the model execution and inversion process on the Raspberry Pi.

Manu, Daniel (ORCID:0000000154982677)↗