Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data Using Unstructured Spatial Discretizations

In this work, we propose a nonlinear manifold learning technique based on deep convolutional autoencoders that is appropriate for model order reduction of physical systems in complex geometries. Convolutional neural networks have proven to be highly advantageous for compressing data arising from systems demonstrating a slow-decaying Kolmogorov n-width. However, these networks are restricted to data on structured meshes. Unstructured meshes are often required for performing analyses of real systems with complex geometry. Our custom graph convolution operators based on the available differential operators for a given spatial discretization effectively extend the application space of deep convolutional autoencoders to systems with arbitrarily complex geometry that are typically discretized using unstructured meshes. We propose sets of convolution operators based on the spatial derivative operators for the underlying spatial discretization, making the method particularly well suited to data arising from the solution of partial differential equations. We demonstrate the method using examples from heat transfer and fluid mechanics and show better than an order of magnitude improvement in accuracy over linear methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimizing High-Throughput Inference on Graph Neural Networks at Shared Computing Facilities with the NVIDIA Triton Inference Server

Abstract With machine learning applications now spanning a variety of computational tasks, multi-user shared computing facilities are devoting a rapidly increasing proportion of their resources to such algorithms. Graph neural networks (GNNs), for example, have provided astounding improvements in extracting complex signatures from data and are now widely used in a variety of applications, such as particle jet classification in high energy physics (HEP). However, GNNs also come with an enormous computational penalty that requires the use of GPUs to maintain reasonable throughput. At shared computing facilities, such as those used by physicists at Fermi National Accelerator Laboratory (Fermilab), methodical resource allocation and high throughput at the many-user scale are key to ensuring that resources are being used as efficiently as possible. These facilities, however, primarily provide CPU-only nodes, which proves detrimental to time-to-insight and computational throughput for workflows that include machine learning inference. In this work, we describe how a shared computing facility can use the NVIDIA Triton Inference Server to optimize its resource allocation and computing structure, recovering high throughput while scaling out to multiple users by massively parallelizing their machine learning inference. To demonstrate the effectiveness of this system in a realistic multi-user environment, we use the Fermilab Elastic Analysis Facility augmented with the Triton Inference Server to provide scalable and high-throughput access to a HEP-specific GNN and report on the outcome.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Flare Statistics for Young Stars from a Convolutional Neural Network Analysis of TESS Data

All-sky photometric time-series missions have allowed for the monitoring of thousands of young (t age < 800 Myr) stars in order to understand the evolution of stellar activity. In this work, we developed a convolutional neural network (CNN), stella, specifically trained to find flares in Transiting Exoplanet Survey Satellite (TESS) short-cadence data. We applied the network to 3200 young stars in order to evaluate flare rates as a function of age and spectral type. The CNN takes a few seconds to identify flares on a single light curve. We also measured rotation periods for 1500 of our targets and find that flares of all amplitudes are present across all spot phases, suggesting high spot coverage across the entire surface. Additionally, flare rates and amplitudes decrease for stars t age > 50 Myr across all temperatures T eff ≥ 4000 K, while stars from 2300 ≤ T eff < 4000 K show no evolution across 800 Myr. Stars of T eff ≤ 4000 K also show higher flare rates and amplitudes across all ages. We investigate the effects of high flare rates on photoevaporative atmospheric mass loss for young planets. In the presence of flares, planets lose 4%–7% more atmosphere over the first 1 Gyr. stella is an open-source Python toolkit hosted on GitHub and PyPI.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Reconstructing High Resolution ESM Data Through a Novel Fast Super Resolution Convolutional Neural Network (FSRCNN)

In this work, we present the first application of a fast super resolution convolutional neural network (FSRCNN) based approach for downscaling earth system model (ESM) simulations. Unlike other SR approaches, FSRCNN uses the same input feature dimensions as the low resolution input. This allows it to have smaller convolution layers, avoiding over-smoothing, and reducing computational costs. We adapt the FSRCNN to improve reconstruction on ESM data, we term the FSRCNN-ESM. We use high-resolution (~0.25°) monthly averaged model output of five surface variables over North America from the US Department of Energy's Energy Exascale Earth System Model's control simulation. These high-resolution and corresponding coarsened low-resolution (~1°) pairs of images are used to train the FSRCNN-ESM and evaluate its use as a downscaling approach. We find that FSRCNN-ESM outperforms FSRCNN and other super-resolution methods in reconstructing high resolution images producing finer spatial scale features with better accuracy for surface temperature, surface radiative fluxes, and precipitation.

58 GEOSCIENCES↗
DOE OSTI2023

Graph Neural Network for Object Reconstruction in Liquid Argon Time Projection Chambers

The Exa.TrkX project presents a graph neural network (GNN) technique for low-level reconstruction of neutrino interactions in a Liquid Argon Time Projection Chamber (LArTPC). GNNs are still a relatively novel technique, and have shown great promise for similar reconstruction tasks in the Large Hadron Collider (LHC). Graphs describing particle interactions are formed by treating each detector hit as a node, with edges describing the relationships between hits. We utilise a multi-head attention message passing network which performs graph convolutions in order to label each node with a particle type.We present an updated variant of our GNN architecture, with several improvements. After testing the model on more realistic simulation with regions of unresponsive wires, the target was modified from edge classification to node classification in order to increase robustness. Removing edges as a classification target opens up a broader possibility space for edge-forming techniques; we explore the model’s performance across a variety of approaches, such as Delaunay triangulation, kNN, and radius-based methods. We also extend this model to the 3D context, sharing information between detector views. By using reconstructed 3D spacepoints to map detector hits from each wire plane, the model naively constructs 2D representations that are independent yet fully consistent.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Structures of neural network effective theories

We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations. Published by the American Physical Society 2024

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Neural network based emulation of galaxy power spectrum covariances: A reanalysis of BOSS DR12 data

We train neural networks to quickly generate redshift-space galaxy power spectrum covariances from a given parameter set (cosmology and galaxy bias). This covariance emulator utilizes a combination of traditional fully connected network layers and transformer architecture to accurately predict covariance matrices for the high redshift, north galactic cap sample of the BOSS DR12 galaxy catalog. We run simulated likelihood analyses with emulated and brute-force computed covariances, and we quantify the network’s performance via two different metrics: (1) difference in Χ 2 and (2) likelihood contours for simulated BOSS DR 12 analyses. We find that the emulator returns excellent results over a large parameter range. We then use our emulator to perform a reanalysis of the BOSS HighZ NGC galaxy power spectrum, and find that varying covariance with cosmology along with the model vector produces Ω m = $0.27⁢6$$^{+0.013}_{–0.015}$, H 0 = 70.2 ± 1.9 km/s/Mpc, and σ 8 = $0.67⁢4$$^{+0.058}_{–0.077}$. These constraints represent an average 0.46⁢σ shift in best-fit values and a 5% increase in constraining power compared to fixing the covariance matrix (Ω m = 0.293 ± 0.017, H 0 = 70.3 ± 2.0 km/s/Mpc, σ 8 = $0.70⁢2$$^{+0.063}_{–0.075}$). As a result, this work demonstrates that emulators for more complex cosmological quantities than second-order statistics can be trained over a wide parameter range at sufficiently high accuracy to be implemented in realistic likelihood analyses.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Efficient I/O for Neural Network Training with Compressed Data

FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and propose mechanisms for selecting compressors for both approaches. Using FanStore, the same storage hardware can host 2-13x more data for example applications without significant runtime overhead. Empirically, our experiments show that FanStore scales to 512 compute nodes with near linear performance scalability.

Zhang, Zhao↗
DOE OSTI2022

Physics Informed Neural Networks as Computational Physics Emulators

This report is a brief overview and evaluation of Physics Informed Neural Networks (PINNs). Karniadakis and co-workers, e.g., Karniadakis et al. (2021) assert that the PINNs approach integrates seamlessly both data and mathematical physics models, even in partially understood, uncertain and high-dimensional contexts. They further claim that PINNs are effective and efficient for ill-posed and inverse problems, and when combined with domain decomposition, are scalable to large problems and a tool to discover hidden physics. While they demonstrate the capabilities in specific academic instances, their overarching claims about PINNs seem to be an overstatement, at least at the current time. We have briefly considered a few of the limitations of PINNs in this investigation. It is not clear to us if the PINNs approach can ever be competitive with approaches that use specialized algorithms to achieve high-accuracy solutions of governing equations and other techniques that can combine observational data with such solutions. As an example of the latter, consistent with the principles of Bayesian inference, data assimilation, or more generally data-model fusion, is a process that fuses observational data typically with a computational model that respects certain constraints such as conservation laws. For example, improvements in observational network combined with data assimilation have been key in improving weather predictions over the past four decades Kalnay (2003).

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1987

Neural network based feed-forward high density associative memory

A novel thin film approach to neural-network-based high-density associative memory is described. The information is stored locally in a memory matrix of passive, nonvolatile, binary connection elements with a potential to achieve a storage density of 10 to the 9th bits/sq cm. Microswitches based on memory switching in thin film hydrogenated amorphous silicon, and alternatively in manganese oxide, have been used as programmable read-only memory elements. Low-energy switching has been ascertained in both these materials. Fabrication and testing of memory matrix is described. High-speed associative recall approaching 10 to the 7th bits/sec and high storage capacity in such a connection matrix memory system is also described.

Daud, T.↗
NASA NTRS1990

Connectivity strategies for higher-order neural networks applied to pattern recognition

Different strategies for non-fully connected HONNs (higher-order neural networks) are discussed, showing that by using such strategies an input field of 128 x 128 pixels can be attained while still achieving in-plane rotation and translation-invariant recognition. These techniques allow HONNs to be used with the larger input scenes required for practical pattern-recognition applications. The number of interconnections that must be stored has been reduced by a factor of approximately 200,000 in a T/C case and about 2000 in a Space Shuttle/F-18 case by using regional connectivity. Third-order networks have been simulated using several connection strategies. The method found to work best is regional connectivity. The main advantages of this strategy are the following: (1) it considers features of various scales within the image and thus gets a better sample of what the image looks like; (2) it is invariant to shape-preserving geometric transformations, such as translation and rotation; (3) the connections are predetermined so that no extra computations are necessary during run time; and (4) it does not require any extra storage for recording which connections were formed.

Spirkovska, Lilly↗
NASA NTRS1992

Learning and optimization with cascaded VLSI neural network building-block chips

To demonstrate the versatility of the building-block approach, two neural network applications were implemented on cascaded analog VLSI chips. Weights were implemented using 7-b multiplying digital-to-analog converter (MDAC) synapse circuits, with 31 x 32 and 32 x 32 synapses per chip. A novel learning algorithm compatible with analog VLSI was applied to the two-input parity problem. The algorithm combines dynamically evolving architecture with limited gradient-descent backpropagation for efficient and versatile supervised learning. To implement the learning algorithm in hardware, synapse circuits were paralleled for additional quantization levels. The hardware-in-the-loop learning system allocated 2-5 hidden neurons for parity problems. Also, a 7 x 7 assignment problem was mapped onto a cascaded 64-neuron fully connected feedback network. In 100 randomly selected problems, the network found optimal or good solutions in most cases, with settling times in the range of 7-100 microseconds.

Duong, T.↗
NASA NTRSDate not supplied

Artificial Neural Networks and AI in high Assurance Applications: Gaps and Techniques

In recent years, capabilties of Deep Neural Networks (DNN) and Artificial Intelligence (AI) systems have grown tremendously. They are now applied in many areas ranging from game playing, social media, science, to robotics, automotive, and aerospace applications.Based upon requirements for safety of DNN and AI in high assurance automotive and aerospace applications, I will discuss the necessity to ensure that AI technqiues for the analysis of Earth observation data and reasoning are working correctly and reliably.In this talk I will present modern techniques for the verification and validation (V&V) of DNN and other AI components as well as approaches for interpretable AI. I will discuss how these techniques can help to ensure quality of the AI results, improve confidence in their application, and facilitate human-AI interaction and collaboration.

Johann Schumann↗
DOE OSTI2022

Multimodel ensemble predictions of precipitation using bayesian neural networks

Multimodel ensembling improves predictions and considers model uncertainties. In this study, we present a Bayesian Neural Network (BNN) ensemble approach for large-scale precipitation predictions based on a set of climate models. BNN infers spatiotemporally varying model weights and biases through the calibration against observations. This ensemble scheme of BNN sufficiently leverages individual model skill for accurate predictions as well as provides interpretability about which models contribute more to the ensemble prediction at which locations and times to inform model development. Additionally, BNN accurately quantifies epistemic uncertainty to avoid overconfident projections. We demonstrate BNN’s superior prediction performance to three state-of-the-art ensemble approaches and discuss its interpretability and uncertainty quantification.

Fan, Patrick↗
DOE OSTI2022

Multimodel Ensemble Predictions of Precipitation using Bayesian Neural Networks

Multimodel ensembling improves predictions and considers model uncertainties. In this study, we present a Bayesian Neural Network (BNN) ensemble approach for large-scale precipitation predictions based on a set of climate models. BNN infers spatiotemporally varying model weights and biases through the calibration against observations. This ensemble scheme of BNN sufficiently leverages individual model skill for accurate predictions as well as provides interpretability about which models contribute more to the ensemble prediction at which locations and times to inform model development. Additionally, BNN accurately quantifies epistemic uncertainty to avoid overconfident projections. We demonstrate BNN’s superior prediction performance to three state-of-the-art ensemble approaches and discuss its interpretability and uncertainty quantification.

Fan, Patrick↗
NASA NTRS2022

Instantaneous Photosynthetically Available Radiation (IPAR) prediction models based on Neural Network for ocean waters.

Instantaneous photosynthetically available radiation (IPAR) at the ocean surface and its vertical profile below the surface play a critical role in models to calculate net primary productivity of marine phytoplankton. In this work, we report two IPAR prediction models based on neural network (NN) approach, one for open ocean and the other for coastal waters. These models are trained, validated, and tested using a large volume of synthetic datasets for open ocean and coastal waters simulated by a radiative transfer model. Our NN models are designed to predict IPAR under a large range of atmospheric and oceanic conditions. The NN models can compute subsurface IPAR profile very accurately up to euphotic zone depth. The root mean square errors associated with the diffuse attenuation coefficient of IPAR are less than 0.011 𝑚−1 and 0.036 𝑚−1 for open ocean and coastal waters respectively. The performance of the NN models is better than presently available semi analytical models, with significant superiority in coastal waters.

PACE↗
DOE OSTI2025

Physics-Informed Neural Networks for PDE-Constrained Optimization and Control

The goal of optimal control is to determine a sequence of inputs for maximizing or minimizing a given performance criterion subject to the dynamics and constraints of the system under observation. This work introduces Control Physics-Informed Neural Networks (PINNs), which simultaneously learn both the system states and the optimal control signal in a single-stage framework that leverages the system’s underlying physical laws. While prior approaches often follow a two-stage process-modeling, the system first and then devising its control—the presented novel framework embeds the necessary optimality conditions directly into the network architecture and loss function. We demonstrate the effectiveness of the novel methodology by solving various open-loop optimal control problems governed by analytical, one-dimensional, and two-dimensional partial differential equations (PDEs).

97 MATHEMATICS AND COMPUTING↗