Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Haze Mitigation in High-Resolution Satellite Imagery using Enhanced Style-Transfer Neural Network and Normalization Across Multiple GPUs

Despite recent advances in deep learning approaches, haze mitigation in large satellite images is still a challenging problem. Due to amorphous nature of haze, object detection or image segmentation approaches are not applicable. Also it is practically infeasible to obtain ground truths for training. Bounded memory capacity of GPUs is another constraint that limits the size of image to be processed. In this paper, we propose a style transfer based neural network approach to mitigate haze in a large overhead imagery. The network is trained without paired ground truths; further, perception loss is added to restore vivid colors, enhance contrast and minimize artifacts. The paper also illustrates our use of multiple GPUs in a collective way to produce a single coherent clear image where each GPU dehazes different portions of a large hazy image.

Park, Byung↗
DOE OSTI2024

Using graph neural networks to reconstruct charged pion showers in the CMS High Granularity Calorimeter

A novel method to reconstruct the energy of hadronic showersin the CMS High Granularity Calorimeter (HGCAL) is presented. TheHGCAL is a sampling calorimeter with very fine transverse andlongitudinal granularity. The active media are silicon sensors andscintillator tiles readout by SiPMs and the absorbers are acombination of lead and Cu/CuW in the electromagnetic section, andsteel in the hadronic section. The shower reconstruction method isbased on graph neural networks and it makes use of a dynamicreduction network architecture. It is shown that the algorithm isable to capture and mitigate the main effects that normally hinderthe reconstruction of hadronic showers using classicalreconstruction methods, by compensating for fluctuations in themultiplicity, energy, and spatial distributions of the shower'sconstituents. The performance of the algorithm is evaluated usingtest beam data collected in 2018 prototype of the CMS HGCALaccompanied by a section of the CALICE AHCAL prototype. Thecapability of the method to mitigate the impact of energy leakagefrom the calorimeter is also demonstrated.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Using a Neural Network Classifier to Select Galaxies with the Most Accurate Photometric Redshifts

The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) will produce several billion photometric redshifts (photo-z's), enabling cosmological analyses to select a subset of galaxies with the most accurate photo-z. For this work, we perform initial redshift fits on Subaru Strategic Program galaxies with deep grizy photometry using Trees for Photo-Z (TPZ) before applying a custom neural network classifier (NNC) tuned to select galaxies with (z phot – z spec )/(1 + z spec ) < 0.10. We consider four cases of training and test sets ranging from an idealized case to using data augmentation to increase the representation of dim galaxies in the training set. Selections made using the NNC yield significant further improvements in outlier fraction and photo-z scatter (σ z ) over those made with typical photo-z uncertainties. As an example, when selecting the best third of the galaxy sample, the NNC achieves a 35% improvement in outlier rate and a 23% improvement in σ z compared to using uncertainties from TPZ. For cosmology and galaxy evolution studies, this method can be tuned to retain a particular sample size or to achieve a desired photo-z accuracy; our results show that it is possible to retain more than a third of an LSST-like galaxy sample while reducing σ z by a factor of 2 compared to the full sample, with one-fifth as many photo-z outliers. For surveys like LSST that are not limited by shot noise, this method enables a larger number of tomographic redshift bins and hence a significant increase in the total signal to noise of galaxy angular power spectra.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Neural Networks for Prediction of Complex Chemistry in Water Treatment Process Optimization

Water chemistry plays a critical role in the design and operation of water treatment processes. Detailed chemistry modeling tools use a combination of advanced thermodynamic models and extensive databases to predict phase equilibria and reaction phenomena. The complexity and formulation of these models preclude their direct integration in equation-oriented modeling platforms, making it difficult to use their capabilities for rigorous water treatment process optimization. Neural networks (NN) can provide a pathway for integrating the predictive capability of chemistry software into equation-oriented models and enable optimization of complex water treatment processes across a broad range of conditions and process designs. Herein, we assess how NN architecture and training data impact their accuracy and use in equation-oriented water treatment models. We generate training data using PhreeqC software and determine how data generation and sample size impact the accuracy of trained NNs. The effect of NN architecture on optimization is evaluated by optimizing hypothetical black-box desalination processes using a range of feed compositions from USGS brackish water data set, tracking the number of successful optimizations, and testing the impact of initial guess on the final solution. Our results clearly demonstrate that data generation and architecture impact NN accuracy and viability for use in equation-oriented optimization problems.

Dudchenko, Alexander V↗
DOE OSTI2024

Physics-informed graphical neural network for power system state estimation

State estimation is highly critical for accurately observing the dynamic behavior of the power grids and minimizing risks from cyber threats. However, existing state estimation methods encounter challenges in accurately capturing power system dynamics, primarily because of limitations in encoding the grid topology and sparse measurements. Here, this paper proposes a physics-informed graphical learning state estimation method to address these limitations by leveraging both domain physical knowledge and a graph neural network (GNN). We employ a GNN architecture that can handle the graph-structured data of power systems more effectively than traditional data-driven methods. The physics-based knowledge is constructed from the branch current formulation, making the approach adaptable to both transmission and distribution systems. The validation results of three IEEE test systems show that the proposed method can achieve lower mean square error more than 20% than the conventional methods.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Neural-network based collision operators for the Boltzmann equation

Kinetic gas dynamics in rarefied and moderate-density regimes have complex behavior associated with collisional processes. These processes are generally defined by convolution integrals over a high-dimensional space (as in the Boltzmann operator), or require evaluating complex auxiliary variables (as in Rosenbluth potentials in Fokker-Planck operators) that are challenging to implement and computationally expensive to evaluate. In this work, we develop a data-driven neural network model that augments a simple and inexpensive BGK collision operator with a machine-learned correction term, which improves the fidelity of the simple operator with a small overhead to overall runtime. The composite collision operator has a tunable fidelity and, in this work, is trained using and tested against a direct-simulation Monte-Carlo (DSMC) collision operator.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Decoding the protein–ligand interactions using parallel graph neural networks

Abstract Protein–ligand interactions (PLIs) are essential for biochemical functionality and their identification is crucial for estimating biophysical properties for rational therapeutic design. Currently, experimental characterization of these properties is the most accurate method, however, this is very time-consuming and labor-intensive. A number of computational methods have been developed in this context but most of the existing PLI prediction heavily depends on 2D protein sequence data. Here, we present a novel parallel graph neural network (GNN) to integrate knowledge representation and reasoning for PLI prediction to perform deep learning guided by expert knowledge and informed by 3D structural data. We develop two distinct GNN architectures: $$\hbox {GNN}_{\mathrm{F}}$$ GNN F is the base implementation that employs distinct featurization to enhance domain-awareness, while $$\hbox {GNN}_{\mathrm{P}}$$ GNN P is a novel implementation that can predict with no prior knowledge of the intermolecular interactions. The comprehensive evaluation demonstrated that GNN can successfully capture the binary interactions between ligand and protein’s 3D structure with 0.979 test accuracy for $$\hbox {GNN}_{\mathrm{F}}$$ GNN F and 0.958 for $$\hbox {GNN}_{\mathrm{P}}$$ GNN P for predicting activity of a protein–ligand complex. These models are further adapted for regression tasks to predict experimental binding affinities and $$\hbox {pIC}_{\mathrm{50}}$$ pIC 50 crucial for compound’s potency and efficacy. We achieve a Pearson correlation coefficient of 0.66 and 0.65 on experimental affinity and 0.50 and 0.51 on $$\hbox {pIC}_{\mathrm{50}}$$ pIC 50 with $$\hbox {GNN}_{\mathrm{F}}$$ GNN F and $$\hbox {GNN}_{\mathrm{P}}$$ GNN P , respectively, outperforming similar 2D sequence based models. Our method can serve as an interpretable and explainable artificial intelligence (AI) tool for predicted activity, potency, and biophysical properties of lead candidates. To this end, we show the utility of $$\hbox {GNN}_{\mathrm{P}}$$ GNN P on SARS-Cov-2 protein targets by screening a large compound library and comparing the prediction with the experimentally measured data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Coupling a recurrent neural network to SPAD TCSPC systems for real-time fluorescence lifetime imaging

Fluorescence lifetime imaging (FLI) has been receiving increased attention in recent years as a powerful diagnostic technique in biological and medical research. However, existing FLI systems often suffer from a tradeoff between processing speed, accuracy, and robustness. Inspired by the concept of Edge Artificial Intelligence (Edge AI), we propose a robust approach that enables fast FLI with no degradation of accuracy. This approach couples a recurrent neural network (RNN), which is trained to estimate the fluorescence lifetime directly from raw timestamps without building histograms, to SPAD TCSPC systems, thereby drastically reducing transfer data volumes and hardware resource utilization, and enabling real-time FLI acquisition. We train two variants of the RNN on a synthetic dataset and compare the results to those obtained using center-of-mass method (CMM) and least squares fitting (LS fitting). Results demonstrate that two RNN variants, gated recurrent unit (GRU) and long short-term memory (LSTM), are comparable to CMM and LS fitting in terms of accuracy, while outperforming them in the presence of background noise by a large margin. To explore the ultimate limits of the approach, we derive the Cramer-Rao lower bound of the measurement, showing that RNN yields lifetime estimations with near-optimal precision. To demonstrate real-time operation, we build a FLI microscope based on an existing SPAD TCSPC system comprising a 32 x 32 SPAD sensor named Piccolo. Four quantized GRU cores, capable of processing up to 4 million photons per second, are deployed on the Xilinx Kintex-7 FPGA that controls the Piccolo. Powered by the GRU, the FLI setup can retrieve real-time fluorescence lifetime images at up to 10 frames per second. The proposed FLI system is promising and ideally suited for biomedical applications, including biological imaging, biomedical diagnostics, and fluorescence-assisted surgery, etc.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

The expressivity of classical and quantum neural networks on entanglement entropy

Abstract Analytically continuing the von Neumann entropy from Rényi entropies is a challenging task in quantum field theory. While then-th Rényi entropy can be computed using the replica method in the path integral representation of quantum field theory, the analytic continuation can only be achieved for some simple systems on a case-by-case basis. In this work, we propose a general framework to tackle this problem using classical and quantum neural networks with supervised learning. We begin by studying several examples with known von Neumann entropy, where the input data is generated by representing$${\text {Tr}}\rho _A^n$$ Tr ρ A n with a generating function. We adopt KerasTuner to determine the optimal network architecture and hyperparameters with limited data. In addition, we frame a similar problem in terms of quantum machine learning models, where the expressivity of the quantum models for the entanglement entropy as a partial Fourier series is established. Our proposed methods can accurately predict the von Neumann and Rényi entropies numerically, highlighting the potential of deep learning techniques for solving problems in quantum information theory.

Physics↗
DOE OSTI2020

Short-term load demand forecasting through rich features based on recurrent neural networks

With the emerging penetration of renewables and dynamic loads, the understanding of grid edge loading conditions becomes increasingly substantial. Load modelling researches commonly consist of explicitly expressed load models and non-explicitly expressed techniques, of which artificial intelligence approaches turn out to be the major path. This paper reveals the artificial intelligence-based load modelling technique to enhance the knowledge of current and future load information considering geographical and weather dependencies. This paper presents a recurrent neural network based sequence to sequence (Seq2Seq) model to forecast the short-term power loads. Also, a feature attention mechanism, which is along channel and time directions, is developed to improve the efficiency of feature learning. The experiments over three publicly available datasets demonstrate the accuracy and effectiveness of the proposed model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Photometric redshift estimation with convolutional neural networks and galaxy images: Case study of resolving biases in data-driven methods

Deep-learning models have been increasingly exploited in astrophysical studies, but these data-driven algorithms are prone to producing biased outputs that are detrimental for subsequent analyses. In this work, we investigate two main forms of biases: class-dependent residuals, and mode collapse. We do this in a case study, in which we estimate photometric redshift as a classification problem using convolutional neural networks (CNNs) trained with galaxy images and associated spectroscopic redshifts. We focus on point estimates and propose a set of consecutive steps for resolving the two biases based on CNN models, involving representation learning with multichannel outputs, balancing the training data, and leveraging soft labels. The residuals can be viewed as a function of spectroscopic redshift or photometric redshift, and the biases with respect to these two definitions are incompatible and should be treated individually. We suggest that a prerequisite for resolving biases in photometric space is resolving biases in spectroscopic space. Experiments show that our methods can better control biases than benchmark methods, and they are robust in various implementing and training conditions with high-quality data. Our methods hold promises for future cosmological surveys that require a good constraint of biases, and they may be applied to regression problems and other studies that make use of data-driven models. Nonetheless, the bias-variance tradeoff and the requirement of sufficient statistics suggest that we need better methods and optimized data usage strategies.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Kulkarni, Shruti↗
DOE OSTI2021

Short Term Plasticity for Artificial Neural Networks

Achieving efficient learning for AI systems was identified as a major challenge in the DOE's recently released, AI for Science , report. The human brain is capable of efficient and low-powered learning. It is likely that implementing brain-like principles will lead to more efficient AI systems. In this LDRD, I aim to contribute to this goal by creating a foundation for implementing and studying a brain phenomenon termed short term plasticity (STP) in spiking artificial neural networks within Sandia. First, data collected by the Allen Institute for Brain Science (AIBS) was analyzed to see if STP could be classified into types using the data collected. Although the data was inadequate at the time, AIBS has updated their database and created models that could be utilized in the future. Second, I began creating a software package to assess the ability of a Boltzmann machine utilizing STP to sample from national security data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

The Feasibility of Incorporating a 3D Velocity Model Into Earthquake Location Around Salt Lake City, UT Using a Physics Informed Neural Network

Earthquake location algorithms typically require travel time calculation. Doing this calculation in 3D, despite advances in algorithm efficiency and computational power, can still be prohibitively expensive in terms of resources and storage. Implementation of high-resolution 3D models in routine earthquake location would be a significant step forward in most of the world. Machine learning algorithms have potential to act as substitutes for travel time calculation algorithms or stored travel time tables. We investigate EikoNet - a physics informed neural network machine learning model that estimates travel times very quickly and comes with negligible memory-overhead. Specifically, we apply EikoNet to the Wasatch Fault Community Velocity Model (WFCVM), a highly detailed and complex 3D velocity model of the Salt Lake City, UT region. While routine locations in the area and studies of the 2020 Magna, UT earthquake sequence used a 1D velocity model, a 3D model may help better our understanding the structure of the major fault in the region. Our primary goal was to test the speed, memory requirements, and accuracy of EikoNet compared to a reference eikonal solver. We find that while the EikoNet is exceedingly fast and requires little memory overhead, achieving acceptable accuracy in estimated travel times is difficult and requires extensive computational resources.

58 GEOSCIENCES↗
DOE OSTI2020

Direct detection of dark matter substructure in strong lens images with convolutional neural networks

Strong gravitational lensing is a potential way of uncovering the nature of dark matter, by finding perturbations to images that cannot be accounted for well by modeling the lens galaxy without additional structure, be it subhalos (smaller halos within the smooth lens) or line-of-sight (LOS) halos. Here, we introduce results attempting to infer the presence of substructure from images without requiring an intermediate step in which a smooth model has to be subtracted, using a simple convolutional neural network (CNN). We find that the network is only able to infer the presence of subhalos with greater than 75% accuracy when they have masses of greater than or equal to 5 × 10 9 M ⊙ if they lie within the main lens galaxy. Since less massive foreground LOS halos can have the same effect as higher-mass subhalos, the CNN can probe lower masses in the halo mass function. The accuracy does not improve significantly if we add a population of less massive subhalos. With the expectation of experiments such as Hubble Space Telescope and Euclid yielding thousands of high-quality strong lensing images in the next years, having a way of analyzing images quickly to identify candidates that merit further analysis to determine individual subhalo properties while preventing extensive resources being used for images that would yield null detections could be very useful. By understanding the sensitivity as a function of substructure mass, nondetections could be combined with the information from images with substructure to constrain the cold dark matter scenario, in particular if the sensitivity can be pushed to lower masses.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Accelerating iterative ptychography with an integrated neural network

Electron ptychography is a powerful and versatile tool for high-resolution and dose-efficient imaging. Iterative reconstruction algorithms are powerful but also computationally expensive due to their relative complexity and the many hyperparameters that must be optimised. Gradient descent-based iterative ptychography is a popular method, but it may converge slowly when reconstructing low spatial frequencies. Here, in this work, we present a method for accelerating a gradient descent-based iterative reconstruction algorithm by training a neural network (NN) that is applied in the reconstruction loop. The NN works in Fourier space and selectively boosts low spatial frequencies, thus enabling faster convergence in a manner similar to accelerated gradient descent algorithms. We discuss the difficulties that arise when incorporating a NN into an iterative reconstruction algorithm and show how they can be overcome with iterative training. We apply our method to simulated and experimental data of gold nanoparticles on amorphous carbon and show that we can significantly speed up ptychographic reconstruction of the nanoparticles.

4DSTEM↗
DOE OSTI2024

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Invertible Neural Networks for Airfoil Design

We report the airfoil design problem, in which an engineer seeks a shape with desired performance characteristics, is fundamental to aerodynamics. Design workflows traditionally rely on iterative optimization methods using low-fidelity integral boundary-layer methods as higher-fidelity adjoint-based computational fluid dynamics methods are computationally expensive. Surrogate-based approaches can accelerate the design process but still rely on some iterative inverse design procedure. In this work, we leverage emerging invertible neural network (INN) tools to enable the rapid inverse design of airfoil shapes for wind turbines. INNs are specialized deep-learning models with well-defined inverse mappings. When trained appropriately, INN surrogate models are capable of forward prediction of aerodynamic and structural quantities for a given airfoil shape as well as inverse recovery of airfoil shapes with specified aerodynamic and structural characteristics. The INN approach offers a roughly 100 times speed-up compared to adjoint-based methods for inverse design. We demonstrate the INN tool for inverse design on three test cases of 100 airfoils each that satisfy the performance characteristics close to those of airfoils used in wind-turbine blades. All generated shapes satisfy the desired aerodynamic characteristics, demonstrating the success of the INN approach for inverse design of airfoils.

17 WIND ENERGY↗