Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,225 records · Page 68
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Tackling the curse of dimensionality in fractional and tempered fractional PDEs with physics-informed neural networks

Fractional and tempered fractional partial differential equations (PDEs) are effective models of long-range interactions, anomalous diffusion, and non-local effects. Traditional numerical methods for these problems are mesh-based, thus struggling with the curse of dimensionality (CoD). Physics-informed neural networks (PINNs) offer a promising solution due to their universal approximation, generalization ability, and mesh-free training. In principle, Monte Carlo fractional PINN (MC-fPINN) estimates fractional derivatives using Monte Carlo methods and thus could lift CoD. However, this may cause significant variance and errors, hence affecting convergence; in addition, MC-fPINN is sensitive to hyperparameters. In general, numerical methods and specifically PINNs for tempered fractional PDEs are under-developed. Herein, we extend MC-fPINN to tempered fractional PDEs to address these issues, resulting in the Monte Carlo tempered fractional PINN (MC-tfPINN). To reduce possible high variance and errors from Monte Carlo sampling, we replace the one-dimensional (1D) Monte Carlo with 1D Gaussian quadrature, applicable to both MC-fPINN and MC-tfPINN. We validate our methods on various forward and inverse problems of fractional and tempered fractional PDEs, scaling up to 100,000 dimensions. Our improved MC-fPINN/MC-tfPINN using quadrature consistently outperforms the original versions in accuracy and convergence speed in very high dimensions.

42 ENGINEERING↗
DOE OSTI2023

Using limited neural networks to assess relative mechanistic influence on shock heating in granular solids

The rapid compaction of granular media results in localized heating that can induce chemical reactions, phase transformations, and melting. However, there are numerous mechanisms in play that can be dependent on a variety of microstructural features. Machine learning techniques such as neural networks offer a ubiquitous method to develop models for physical processes. Limiting what kind of microstructural information is used as an input and assessing normalized changes in network error, the relative importance of different mechanisms can be inferred. Here we utilize binned, initial density information as network inputs to predict local shock heating in a granular high explosive trained from large-scale molecular dynamics simulations. In this study, the spatial extent of the density field used in the network is altered to assess the importance and relevant length scales of the physical mechanisms in play, where different microstructural features result in different predictive capabilities.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Neural-network model for force prediction in multi-principal-element alloys

Atomistic simulations can provide useful insights into the physical properties of multi-principal-element alloys. However, classical potentials mostly fail to capture key quantum (electronic-structure) effects. Here we present a deep 3D convolutional neural network (3D CNN) based framework combined with a voxelization technique to design interatomic potentials for chemically complex alloys. We highlight the performance of the 3D CNN model and its efficacy in computing potentials using the medium-entropy alloy TaNbMo. In order to provide insights into the effect of voxel resolution, we implemented two approaches based on the inner and outer bounding boxes. An efficient 3D CNN model, which is as accurate as the density-functional theory (DFT) approach, for calculating potentials will provide a promising schema for accurate atomistic simulations of structure and dynamics of general multi-principle element alloys.

36 MATERIALS SCIENCE↗
NASA NTRSDate not supplied

SPIKE-Dx : A Low-Power High-Throughput Fault Diagnostics Tool using Spiking Neural Networks for Constrained Systems

Diagnostic systems are important for many aerospace systems, which are severely limited in available power, like cubesats or UAVs. Therefore, traditional diagnostics systems cannot be used due to their substantial footprint and constraints. In this paper, we present our very low power diagnostic tool SPIKE-DX to monitor critical systems with constrained computational and energy resources. This is made possible through spiking neural networks (SNNs), which are executable within optimized simulation environments and further implemented on on cutting-edge neuromorphic hardware. Based upon FMEA (Failure Mode and Effect Analysis) framework, Diagnostic Bayesian Networks (DBNs) can be constructed that provide powerful means for diagnostic reasoning. In this paper, we describe such DBNs and a method to automatically translate the DBN into highly structured networks of spiking neurons for execution in SPIKE-DX.

Spiking Neural Networks↗
DOE OSTI2024

On the Generalizability of Time-of-Flight Convolutional Neural Networks for Noninvasive Acoustic Measurements

Bulk wave acoustic time-of-flight (ToF) measurements in pipes and closed containers can be hindered by guided waves with similar arrival times propagating in the container wall, especially when a low excitation frequency is used to mitigate sound attenuation from the material. Convolutional neural networks (CNNs) have emerged as a new paradigm for obtaining accurate ToF in non-destructive evaluation (NDE) and have been demonstrated for such complicated conditions. However, the generalizability of ToF-CNNs has not been investigated. In this work, we analyze the generalizability of the ToF-CNN for broader applications, given limited training data. We first investigate the CNN performance with respect to training dataset size and different training data and test data parameters (container dimensions and material properties). Furthermore, we perform a series of tests to understand the distribution of data parameters that need to be incorporated in training for enhanced model generalizability. This is investigated by training the model on a set of small- and large-container datasets regardless of the test data. We observe that the quantity of data partitioned for training must be of a good representation of the entire sets and sufficient to span through the input space. The result of the network also shows that the learning model with the training data on small containers delivers a sufficiently stable result on different feature interactions compared to the learning model with the training data on large containers. To check the robustness of the model, we tested the trained model to predict the ToF of different sound speed mediums, which shows excellent accuracy. Furthermore, to mimic real experimental scenarios, data are augmented by adding noise. We envision that the proposed approach will extend the applications of CNNs for ToF prediction in a broader range.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Bayesian Inference with Latent Hamiltonian Neural Networks (L-HNNs)

When sampling for Bayesian inference, one popular approach is to use Hamiltonian Monte Carlo (HMC) and the No-U-Turn Sampler (NUTS). However, HMC and NUTS can require numerous numerical gradients of the target density and can prove slow in practice. We propose Hamiltonian neural networks (HNNs) with HMC and NUTS for solving Bayesian inference problems [1, 2]. Once trained, HNNs do not require gradients of the target density while sampling. Moreover, they satisfy important properties such as perfect time reversibility and Hamiltonian conservation, making them well suited for use within HMC and NUTS because stationarity can be shown. We also propose an HNN extension called latent HNNs (L-HNNs), which predict latent variable outputs. Compared to HNNs, L-HNNs offer improved expressivity and a reduction in integration errors. Finally, we propose employing L-HNNs in NUTS with an online error monitoring scheme to prevent degeneracy of the sampling in regions of low probability density. We demonstrate L-HNNs in NUTS with online error monitoring by using several example cases involving complex, heavy-tailed, and high local curvature probability densities. Overall, L-HNNs in NUTS with online error monitoring satisfactorily inferred these probability densities. Compared to traditional NUTS, L-HNNs in NUTS with online error monitoring improved the effective sample size (ESS) per gradient by an order of magnitude.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Explainable Bayesian Neural Network for Probabilistic Transient Stability Analysis Considering Wind Energy

While several data-driven models have been developed for transient stability assessment, how to consider the uncertainties from load and renewable generations and provide interpretation of data-driven assessment results are still open. This paper proposes an explainable Bayesian Neural Network (BNN) for probabilistic transient stability assessment (TSA). By extracting the uncertainties from loads and wind farms, the BNN model can make a reliable prediction and quantify the prediction uncertainties. We also develop the Gradient Shap algorithm to make the global and local explanations for the probabilistic TSA model, a significant advantage over existing black-box data-driven methods. Numerical results on the modified IEEE 39-bus system show that the proposed method outperforms the existing methods in terms of prediction accuracy and uncertainty quantification capabilities. The explainability of the proposed method allows system operators to design preventive controls for enhancing system stability.

Bayesian Neural Network↗
DOE OSTI2023

Graph neural networks for detecting anomalies in scientific workflows

Identifying and addressing anomalies in complex, distributed systems can be challenging for reliable execution of scientific workflows. We model these workflows as directed acyclic graphs (DAGs), where the nodes and edges of the DAGs represent jobs and their dependencies, respectively. We develop graph neural networks (GNNs) to learn patterns in the DAGs and to detect anomalies at the node (job) and graph (workflow) levels. We investigate workflow-specific GNN models that are trained on a particular workflow and workflow-agnostic GNN models that are trained across the workflows. Our GNN models, which incorporate both individual job features and topological information from the workflow, show improved accuracy and efficiency compared to conventional learning methods for detecting anomalies. While joint trained with multiple scientific workflows, our GNN models reached an accuracy more than 80% for workflow level and 75% for job level anomalies. In addition, we illustrate the importance of hyperparameter tuning method in our study that can significantly improve the metric(s) measure of evaluating the GNN models. Finally, we integrate explainable GNN methods to provide insights on job features in the workflow that cause an anomaly.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

SAFE-OCC: A novelty detection framework for Convolutional Neural Network sensors and its application in process control

Herein we present a novelty detection framework for Convolutional Neural Network (CNN) sensors that we call Sensor-Activated Feature Extraction One-Class Classification (SAFE-OCC). We show that this framework enables the safe use of computer vision sensors in process control architectures. Emergent control applications use CNN models to map visual data to a state signal that can be interpreted by the controller. Incorporating such sensors introduces a significant system operation vulnerability because CNN sensors can exhibit high prediction errors when exposed to novel (abnormal) visual data. Unfortunately, identifying such novelties in real-time is nontrivial. To address this issue, the SAFE-OCC framework leverages the convolutional blocks of the CNN to create an effective feature space to conduct novelty detection using a desired one-class classification technique. This approach engenders a feature space that directly corresponds to that used by the CNN sensor and avoids the need to derive an independent latent space. We demonstrate the effectiveness of SAFE-OCC via simulated control environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Sivers extraction with Neural Network

Pseudo-data with simulated experimental errors can be generated to train an ensemble of Artificial Neural Networks (ANN) implemented on a regression to extract Transverse Momentum-dependent Distributions (TMDs). A preliminary analysis is presented on the reliability in extraction of the Sivers function imposed in the pseudo-data given the bounds on the experimental errors, data sparsity, and complexity of phase-space.

Fernando, Ishara↗
DOE OSTI2021

Large-scale Gravitational Lens Modeling with Bayesian Neural Networks for Accurate and Precise Inference of the Hubble Constant

We investigate the use of approximate Bayesian neural networks (BNNs) in modeling hundreds of time delay gravitational lenses for Hubble constant (H 0 ) determination. Here, our BNN was trained on synthetic Hubble Space Telescope quality images of strongly lensed active galactic nuclei with lens galaxy light included. The BNN can accurately characterize the posterior probability density functions (PDFs) of model parameters governing the elliptical power-law mass profile in an external shear field. We then propagate the BNN-inferred posterior PDFs into an ensemble H 0 inference, using simulated time delay measurements from a plausible dedicated monitoring campaign. Assuming well-measured time delays and a reasonable set of priors on the environment of the lens, we achieve a median precision of 9.3% per lens in the inferred H 0 . A simple combination of a set of 200 test lenses results in a precision of 0.5 km s –1 Mpc –1 (0.7%), with no detectable bias in this H 0 recovery test. The computation time for the entire pipeline—including the generation of the training set, BNN training and H0 inference—translates to 9 minutes per lens on average for 200 lenses and converges to 6 minutes per lens as the sample size is increased. Being fully automated and efficient, our pipeline is a promising tool for exploring ensemble-level systematics in lens modeling for H 0 inference.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1986

Comparison between sparsely distributed memory and Hopfield-type neural network models

The Sparsely Distributed Memory (SDM) model (Kanerva, 1984) is compared to Hopfield-type neural-network models. A mathematical framework for comparing the two is developed, and the capacity of each model is investigated. The capacity of the SDM can be increased independently of the dimension of the stored vectors, whereas the Hopfield capacity is limited to a fraction of this dimension. However, the total number of stored bits per matrix element is the same in the two models, as well as for extended models with higher order interactions. The models are also compared in their ability to store sequences of patterns. The SDM is extended to include time delays so that contextual information can be used to cover sequences. Finally, it is shown how a generalization of the SDM allows storage of correlated input pattern vectors.

Keeler, James D.↗
NASA NTRS1991

Neural network representation and learning of mappings and their derivatives

Discussed here are recent theorems proving that artificial neural networks are capable of approximating an arbitrary mapping and its derivatives as accurately as desired. This fact forms the basis for further results establishing the learnability of the desired approximations, using results from non-parametric statistics. These results have potential applications in robotics, chaotic dynamics, control, and sensitivity analysis. An example involving learning the transfer function and its derivatives for a chaotic map is discussed.

White, Halbert↗
NASA NTRS1991

Multi-channel holographic birfurcative neural network system for real-time adaptive EOS data analysis

Viewgraphs on multi-channel holographic bifurcative neural network system for real-time adaptive Earth Observing System (EOS) data analysis are presented. The objective is to research and develop an optical bifurcating neuromorphic pattern recognition system for making optical data array comparisons and to evaluate the use of the system for EOS data classification, reduction, analysis, and other applications.

Liu, Hua-Kuang↗
NASA NTRS1996

VLSI Neural Networks Help To Compress Video Signals

Advanced analog/digital electronic system for compression of video signals incorporates artificial neural networks. Performs motion-estimation and image-data-compression processing. Effectively eliminates temporal and spatial redundancies of sequences of video images; processes video image data, retaining only nonredundant parts to be transmitted, then transmits resulting data stream in form of efficient code. Reduces bandwidth and storage requirements for transmission and recording of video signal.

Fang, Wai-Chi↗
DOE OSTI2020

Partitioning net carbon dioxide fluxes into photosynthesis and respiration using neural networks

The eddy covariance (EC) technique is used to measure the net ecosystem exchange (NEE) of CO 2 between ecosystems and the atmosphere, offering a unique opportunity to study ecosystem responses to climate change. NEE is the difference between the total CO 2 release due to all respiration processes (RECO), and the gross carbon uptake by photosynthesis (GPP). These two gross CO 2 fluxes are derived from EC measurements by applying partitioning methods that rely on physiologically based functional relationships with a limited number of environmental drivers. However, the partitioning methods applied in the global FLUXNET network of EC observations do not account for the multiple co-acting factors that modulate GPP and RECO flux dynamics. To overcome this limitation, we developed a hybrid data-driven approach based on combined neural networks (NN C-part ). NN C-part incorporates process knowledge by introducing a photosynthetic response based on the light-use efficiency (LUE) concept, and uses a comprehensive dataset of soil and micrometeorological variables as fluxes drivers. We applied the method to 36 sites from the FLUXNET2015 dataset and found a high consistency in the results with those derived from other standard partitioning methods for both GPP (R 2 > .94) and RECO (R 2 > .8). High consistency was also found for (a) the diurnal and seasonal patterns of fluxes and (b) the ecosystem functional responses. NNC-part performed more realistic than the traditional methods for predicting additional patterns of gross CO 2 fluxes, such as: (a) the GPP response to VPD, (b) direct effects of air temperature on GPP dynamics, (c) hysteresis in the diel cycle of gross CO 2 fluxes, (d) the sensitivity of LUE to the diffuse to direct radiation ratio, and (e) the post rain respiration pulse after a long dry period. In conclusion, NN C-part is a valid data-driven approach to provide GPP and RECO estimates and complementary to the existing partitioning methods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Power System Event Classification and Localization Using a Convolutional Neural Network

Detection and timely identification of power system disturbances are essential for situation awareness and reliable electricity grid operation. Because records of actual events in the system are limited, ensemble simulation-based events are needed to provide adequate data for building event-detection models through deep learning; e.g., a convolutional neural network (CNN). An ensemble numerical simulation-based training data set have been generated through dynamic simulations performed on the Polish system with various types of faults in different locations. Such data augmentation is proven to be able to provide adequate data for deep learning. The synchronous generators’ frequency signals are used and encoded into images for developing and evaluating CNN models for classification of fault types and locations. With a time-domain stacked image set as the benchmark, two different time-series encoding approaches, i.e., wavelet decomposition-based frequency-domain stacking and polar coordinate system-based Gramian Angular Field (GAF) stacking, are also adopted to evaluate and compare the CNN model performance and applicability. The various encoding approaches are suitable for different fault types and spatial zonation. With optimized settings of the developed CNN models, the classification and localization accuracies can go beyond 84 and 91%, respectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Predicting critical heat flux with uncertainty quantification and domain generalization using conditional variational autoencoders and deep neural networks

Deep generative models (DGMs) can generate synthetic data samples that closely resemble the original dataset, addressing data scarcity. In this work, we developed a conditional variational autoencoder (CVAE) to augment critical heat flux (CHF) data used for the 2006 Groeneveld lookup table. To compare with traditional methods, a fine-tuned deep neural network (DNN) regression model was evaluated on the same dataset. Both models achieved small mean absolute relative errors, with the CVAE showing more favorable results. Uncertainty quantification (UQ) was performed using repeated CVAE sampling and DNN ensembling. The DNN ensemble improved performance over the baseline, while the CVAE maintained consistent results with less variability and higher confidence. Both models achieved small errors inside and outside the training domain, with slightly larger errors outside. Altogether, the CVAE performed better than the DNN in predicting CHF and exhibited better uncertainty behavior.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗