Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORK”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

Neural network approximation of Bayesian models for the inference of ion and electron temperature profiles at W7-X

Here, we describe a method for training a neural network (NN) to approximate the full model Bayesian inference of plasma profiles from x-ray imaging diagnostic measurements. The modeling is carried out within the Minerva Bayesian modeling framework where models are defined as a set of assumptions, prior beliefs on parameter values and physics knowledge. The goal is to use NNs for fast ion and electron temperature profile inversion from measured image data. The NN is trained solely on artificial data generated by sampling from the joint distribution of the free parameters and model predictions. The training is carried out in such a way that the mapping learned by the network constitutes an approximation of the full model Bayesian inference. The analysis is carried out on images constituted of 20 × 195 pixels corresponding to binned lines of sight and spectral channels, respectively. Through the full model inference, it is possible to infer electron and ion temperature profiles as well as impurity density profiles. When the network is utilized for the inference of the temperature profiles, the analysis time can be reduced down to a few tens of microseconds for a single time point, which is a drastic improvement if compared to the ≈4 h long Bayesian inference. The procedure developed for the generation of the training set does not rely on diagnostic-specific features, and therefore it is in principle applicable to any other model developed within the Minerva framework. The trained NN has been tested on data collected during the first operational campaign at W7-X, and compared to the full model Bayesian inference results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Improving ideal MHD equilibrium accuracy with physics-informed neural networks

We present a novel approach to compute three-dimensional magnetohydrodynamic equilibria with isotropic pressure profiles and nested surfaces by parametrizing Fourier modes with artificial neural networks (NNs). The full nonlinear global force residual of single equilibria across the volume in real space is then minimized with first order optimizers and compared to equilibria computed by conventional solvers. Already, we observe competitive computational cost to arrive at the same minimum residuals computable with existing codes. With increased computational cost, lower minima of the residual are computable with the NNs than with any other tested solver, establishing a new lower bound for the force residual. We use minimally complex NNs, and we expect significant improvements for solving not only single equilibria with NNs, but also for creating NN models valid over continuous distributions of equilibria.

ideal magnetohydrodynamics↗

Neural Network Identification For a C5 Parallel Robot

This paper presents the design and analysis of a neural network-based identification of the inverse dynamic model of a C5 parallel robot. The identification structure is designed using the black box form (the dynamic model is completely unknown). This identification uses real data acquired on the C5 parallel robot by applying a nominal control scheme (PD). The desired trajectories of this scheme are based on Fourier series and the coefficients are chosen in a heuristic way. We have used this type of desired trajectories to obtain exciting trajectories for identification procedure. Three identification schemes are tested and compared. The comparison is performed based on the number of parameters used in each architecture and the quality of the generalization error. The used neural network is of MLP type and composed of one hidden layer.

42 ENGINEERING↗

Exploring the Value of Nodes with Multicommunity Membership for Classification with Graph Convolutional Neural Networks

Sampling is an important step in the machine learning process because it prioritizes samples that help the model best summarize the important concepts required for the task at hand. The process of determining the best sampling method has been rarely studied in the context of graph neural networks. In this paper, we evaluate multiple sampling methods (i.e., ascending and descending) that sample based off different definitions of centrality (i.e., Voterank, Pagerank, degree) to observe its relation with network topology. We find that no sampling method is superior across all network topologies. Additionally, we find situations where ascending sampling provides better classification scores, showing the strength of weak ties. Two strategies are then created to predict the best sampling method, one that observes the homogeneous connectivity of the nodes, and one that observes the network topology. In both methods, we are able to evaluate the best sampling direction consistently.

97 MATHEMATICS AND COMPUTING↗

Modern deep neural networks for Direct Normal Irradiance forecasting: A classification approach

The escalating energy demand and the adverse environmental impacts of fossil-fuel use necessitate a shift towards cleaner and renewable alternatives. Concentrated Solar Power (CSP) technology emerges as a promising solution, offering a carbon-free alternative for power generation. The efficiency and profitability of CSP depend on the Direct Normal Irradiance (DNI) component of solar radiation; hence, accurate DNI forecasting can help optimize CSP plants’ operations and performance. The unpredictable nature of weather phenomena, particularly cloud cover, introduces uncertainty into DNI projections. Existing DNI forecasting models use meteorological factors, which are both challenging to estimate numerically over short prediction windows and expensive to model through data at a sufficiently high spatial and temporal resolution. This research addresses the challenge by presenting a novel approach that formulates DNI prediction as a multi-class classification problem, departing from conventional regression-based methods. The primary objective of this classification framework is to identify optimal periods aligning with specific operational thresholds for CSP plants, contributing to enhanced dispatch optimization strategies. We model the DNI classification problem using four advanced deep neural networks – rectified linear unit (ReLU) networks, 1D residual networks (ResNets), bidirectional long short-term memory (BiLSTM) networks, and transformers – achieving accuracies up to 93.5% without requiring meteorological parameters.

14 SOLAR ENERGY↗

Deep neural networks for energy and position reconstruction in EXO-200

Here, we apply deep neural networks (DNN) to data from the EXO-200 experiment. In the studied cases, the DNN is able to reconstruct the relevant parameters—total energy and position—directly from raw digitized waveforms, with minimal exceptions. For the first time, the developed algorithms are evaluated on real detector calibration data. The accuracy of reconstruction either reaches or exceeds what was achieved by the conventional approaches developed by EXO-200 over the course of the experiment. Most existing DNN approaches to event reconstruction and classification in particle physics are trained on Monte Carlo simulated events. Such algorithms are inherently limited by the accuracy of the simulation. We describe a unique approach that, in an experiment such as EXO-200, allows to successfully perform certain reconstruction and analysis tasks by training the network on waveforms from experimental data, either reducing or eliminating the reliance on the Monte Carlo.

97 MATHEMATICS AND COMPUTING↗

Identification of tau leptons using a convolutional neural network with domain adaptation

A tau lepton identification algorithm,DeepTau, based on convolutional neural network techniques, has been developed in the CMS experiment to discriminate reconstructed hadronic decays of tau leptons (τ h ) from quark or gluon jets and electrons and muons that are misreconstructed as τ h candidates. The latest version of this algorithm, v2.5, includes domain adaptation by backpropagation, a technique that reduces discrepancies between collision data and simulation in the region with the highest purity of genuine τh candidates. Additionally, a refined training workflow improves classification performance with respect to the previous version of the algorithm, with a reduction of 30–50% in the probability for quark and gluon jets to be misidentified as τ h candidates for given reconstruction and identification efficiencies. This paper presents the novel improvements introduced in theDeepTau algorithm and evaluates its performance in LHC proton-proton collision data at √(s) = 13 and 13.6 TeV collected in 2018 and 2022 with integrated luminosities of 60 and 35 fb -1 , respectively. Techniques to calibrate the performance of the τ h identification algorithm in simulation with respect to its measured performance in real data are presented, together with a subset of results among those measured for use in CMS physics analyses.

Large detector-systems performance↗

Nuclear Responses with Neural-Network Quantum States

We introduce a variational Monte Carlo framework that combines neural-network quantum states with the Lorentz integral transform technique to compute the dynamical properties of self-bound quantum many-body systems in continuous Hilbert spaces. While broadly applicable to various quantum systems, including atoms and molecules, in this initial application we focus on the photoabsorption cross section of light nuclei, where benchmarks against numerically exact techniques are available. Our accurate theoretical predictions are complemented by robust uncertainty quantification, enabling meaningful comparisons with experiments. Here, we demonstrate that a relatively simple nuclear Hamiltonian—based on a leading-order pionless EFT expansion and known to accurately reproduce ground-state energies of nuclei with 𝐴 ≤ 40—also provides a reliable description of the photoabsorption cross section.

Ab initio calculations↗

Filter pruning of Convolutional Neural Networks for text classification: A case study of cancer pathology report comprehension

Convolutional Neural Networks (CNN) have recently demonstrated effective performance in many Natural Language Processing tasks. In this study, we explore a novel approach for pruning a CNN's convolution filters using our new data-driven utility score. We have applied this technique to an information extraction task of classifying a dataset of cancer pathology reports by cancer type, a highly imbalanced dataset. Compared to standard CNN training, our new algorithm resulted in a nearly .07 increase in the micro-averaged F1-score and a strong .22 increase in the macro-averaged F1-score using a model with nearly a third fewer network weights. We show how directly utilizing a network's interpretation of data can result in strong performance gains, particularly with severely imbalanced datasets.

Yoon, Hong-Jun↗

Inferring Convolutional Neural Networks' Accuracies from Their Architectural Characterizations

The challenge of choosing an appropriate convolutional neural network (CNN) architecture for specific applications and different data sets is still poorly understood in the literature. This is problematic, since CNNs have shown strong promise for analyzing scientific data from many domains including particle imaging detectors. In this paper, we proposed a systematic language that is useful for comparison between different CNN's architectures before training time. This helps us predict whether a network can perform better than a certain threshold accuracy before training up to 70% accuracy using simple machine learning models. Additionally, we found a coefficient of determination of 0.966 for an Ordinary Least Squares model in a regression task to predict accuracy of a large population of networks.

Hoang, Duc↗

DeepDataProfiler: A Platform and Methodology for the Analysis and Interpretation of Neural Networks

The DeepDataProfiler is a methodology and framework for providing interpretability to trained neural networks. Its approach is to decompose a network into a weighted graph of neurons and synapses and link the components of the graph to human identifiable concepts. By identifying concepts important to the network and tracking the decision process employed by the network, the network becomes more transparent and less like a black box. Spurious decisions and poor generalization strategies can be identified and a measure of trustworthiness can be established.

97 MATHEMATICS AND COMPUTING↗

Nuclear responses with neural-network quantum states

We introduce a variational Monte Carlo framework that combines neural-network quantum states with the Lorentz integral transform technique to compute the dynamical properties of self-bound quantum many-body systems in continuous Hilbert spaces. While broadly applicable to various quantum systems, including atoms and molecules, in this initial application we focus on the photoabsorption cross section of light nuclei, where benchmarks against numerically exact techniques are available. Our accurate theoretical predictions are complemented by robust uncertainty quantification, enabling meaningful comparisons with experiments. We demonstrate that a simple nuclear Hamiltonian, based on a leading-order pionless effective field theory expansion and known to accurately reproduce the ground-state energies of nuclei with $A\leq 20$ nucleons also provides a reliable description of the photoabsorption cross section.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Ground-state coding in partially connected neural networks

Patterns over (-1,0,1) define, by their outer products, partially connected neural networks, consisting of internally strongly connected, externally weakly connected subnetworks. The connectivity patterns may have highly organized structures, such as lattices and fractal trees or nests. Subpatterns over (-1,1) define the subcodes stored in the subnetwork, that agree in their common bits. It is first shown that the code words are locally stable stares of the network, provided that each of the subcodes consists of mutually orthogonal words or of, at most, two words. Then it is shown that if each of the subcodes consists of two orthogonal words, the code words are the unique ground states (absolute minima) of the Hamiltonian associated with the network. The regions of attraction associated with the code words are shown to grow with the number of subnetworks sharing each of the neurons. Depending on the particular network architecture, the code sizes of partially connected networks can be vastly greater than those of fully connected ones and their error correction capabilities can be significantly greater than those of the disconnected subnetworks. The codes associated with lattice-structured and hierarchical networks are discussed in some detail.

Baram, Yoram↗

A preliminary attempt to use neural networks for turbulent eddy classification

This note describes an attempt to use standard neural network tools to fashion a means of detecting eddy patterns in the wall region of a turbulent flow. The research was motivated by the desire to formulate a means to use only flow parameters that can be sensed on the wall to describe the passing eddy structure. If a simple formulation can be obtained, it could conceivably be utilized to control actuators embedded in the wall. Such actuators have been developed by Jacobson and Reynolds (1993a), Blackwelder and Liu (1994), Tung et al. (1995), and others. These actuators have the common characteristics that they are small and are typically flush with the wall when not deployed. When they are activated, it is assumed that they will be able to interact constructively with the turbulent eddies near their location to either decrease the wall shear stress, enhance or reduce the mixing, etc. At present, there is only a nascent understanding of the interaction dynamics between the actuators and the eddies in the flow. Nevertheless, for such interaction to succeed, methods to couple the actuators to the oncoming flow must be obtained. General methods must be found that will detect the space and temporal location of the desired structure. In particular, it will be necessary to know when the eddies will arrive at the location of the actuator. This research attempted to use the shear stress measurements on the wall in the vicinity of an actuator location to predict when a particular eddy pattern would arrive and/or occur at the designated location. In this work the eddy pattern to be detected was identified by its velocity signature only.

Blackwelder, Ron F.↗

Concept-based Analysis of Neural Networks via Vision-Language Models

The analysis of vision-based deep neural networks (DNNs) is highly desirable but it is very challenging due to the difficulty of expressing formal specifications for vision tasks and the lack of efficient verification procedures. In this paper, we propose to leverage emerging multimodal, vision-language, foundation models (VLMs) as a lens through which we can reason about vision models. VLMs have been trained on a large body of images accompanied by their textual description, and are thus implicitly aware of high-level, human-understandable concepts describing the images. We describe a logical specification language Con spec designed to facilitate writing specifications in terms of these concepts. To define and formally check Con spec specifications, we build a map between the internal representations of a given vision model and a VLM, leading to an efficient verification procedure of natural-language properties for vision models. We demonstrate our techniques on a ResNet-based classifier trained on the RIVAL-10 dataset using CLIP as the multimodal model.

Large Vision Language Models↗