SEARCH · Engineering Papers

Results for “Generative Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Deblending galaxies with variational autoencoders: A joint multiband, multi-instrument approach

ABSTRACT Blending of galaxies has a major contribution in the systematic error budget of weak-lensing studies, affecting photometric and shape measurements, particularly for ground-based, deep, photometric galaxy surveys, such as the Rubin Observatory Legacy Survey of Space and Time (LSST). Existing deblenders mostly rely on analytic modelling of galaxy profiles and suffer from the lack of flexible yet accurate models. We propose to use generative models based on deep neural networks, namely variational autoencoders (VAE), to learn probabilistic models directly from data. We train a VAE on images of centred, isolated galaxies, which we reuse, as a prior, in a second VAE-like neural network in charge of deblending galaxies. We train our networks on simulated images including six LSST bandpass filters and the visible and near-infrared bands of the Euclid satellite, as our method naturally generalizes to multiple bands and can incorporate data from multiple instruments. We obtain median reconstruction errors on ellipticities and r-band magnitude between ±0.01 and ±0.05, respectively, in most cases, and ellipticity multiplicative bias of 1.6 per cent for blended objects in the optimal configuration. We also study the impact of decentring and prove the method to be robust. This method only requires the approximate centre of each target galaxy, but no assumptions about the number of surrounding objects, pointing to an iterative detection/deblending procedure we leave for future work. Finally, we discuss future challenges about training on real data and obtain encouraging results when applying transfer learning.

Arcelin, Bastien↗

DOE OSTI2024

A Suppression-based STDP Rule Resilient to Jitter Noise in Spike Patterns for Neuromorphic Computing

Multi-spike models of synaptic plasticity, such as the triplet and suppression spike-timing-dependent plasticity (STDP) rules, exhibit better alignment with neurophysiological data in the brain compared to the pair-based STDP rule. Previous studies have empirically shown that the pair-based STDP rule can detect spatiotemporal spike patterns hidden in equally dense distractor spike trains in an unsupervised manner. However, it fails to detect spike patterns influenced by jitter noise. Given that spiking neural networks (SNNs) exhibit variability in generated spike trains in response to the same inputs, it becomes imperative to have learning rules capable of detecting spike patterns even in the presence of jitter noise. In this study, we introduce a simplified suppression-based STDP rule that demonstrates significantly enhanced tolerance to jitter in spike patterns compared to the pair-based STDP rule. Unlike the ideal suppression STDP rule, characterized by an exponential learning window and requiring high-resolution synapses, the simplified rule limits the synaptic efficacy update to a single bit at any given instant. Moreover, it employs 4-bit fixed-point synapses, facilitating straightforward implementation in neuromorphic hardware.

Gautam, Ashish [ORNL]↗

DOE OSTI2022

An MLIR-based Compiler Flow for System-Level Design and Hardware Acceleration

The generation of custom hardware accelerators for applications implemented within high-level productive programming frameworks requires considerable manual effort. To automate this process, we introduce \sodaopt, a compiler tool that extends the MLIR infrastructure. \sodaopt automatically searches, outlines, tiles, and pre-optimizes relevant code regions to generate high-quality accelerators through high-level synthesis. \sodaopt can support any high-level programming framework and domain-specific language that interface with the MLIR infrastructure. By leveraging MLIR, \sodaopt solves compiler optimization problems with specialized abstractions. Backend synthesis tools connect to \sodaopt through progressive intermediate representation lowerings. \sodaopt interfaces to a design space exploration engine to identify the combination of compiler optimization passes and options that provides high-performance generated designs for different backends and targets. We demonstrate the practical applicability of the compilation flow by exploring the automatic generation of accelerators for deep neural networks operators outlined at arbitrary granularity and by combining outlining with tiling on large convolution layers. Experimental results with kernels from the PolyBench benchmark show that \sodaopt high-level optimizations improve execution delays of synthesized accelerators up to 60x. We also show that for the selected kernels, our solution outperforms the current of state-of-the art in more than 70% of the benchmarks and provides better average speedup in 55% of them.

Bohm Agostini, Nicolas↗

DOE OSTI2022

pnnl/Active-Sampling-for-Atomistic-Potentials

This software contains a routine for active sampling of a DFT-generated dataset for training a neural network potential (SchNet). Major updates to the base SchNet code are also included to accommodate large systems (100s of atoms). We also provide code to apply the neural network potential in dynamic shear simulations.

Sprueill, Henry↗

DOE OSTI2025

Structure–Property Linkage in Alloys Using Graph Neural Network and Explainable Artificial Intelligence

Deep learning tools have recently shown significant potential for accelerating the prediction of microstructure–property linkage in materials. While deep neural networks like convolution neural networks (CNNs) can extract physics information from 3D microstructure images, they often require a large network architecture and substantial training time. In this research, we trained a graph neural network (GNN) using phase field generated microstructures of Ni-Al alloys to predict the evolution of mechanical properties. We found that a single GNN is capable of accurately predicting the strengthening of Ni-Al alloys with microstructures of varying sizes and dimensions, which cannot otherwise be done with a CNN. Additionally, GNN requires significantly less GPU utilization than CNN and offers more interpretable explanation of predictions using saliency analysis as features are manually defined in the graph. We also utilize explainable artificial intelligence tool Bayesian Inference to determine the coefficients in the power law equation that governs coarsening of precipitates. Overall, our work demonstrates the ability of the GNN to accurately and efficiently extract relevant information from material microstructures without having restrictions on microstructure size or dimension and offers an interpretable explanation.

Chemistry↗

NASA NTRS1992

Design and evaluation of a robust dynamic neurocontroller for a multivariable aircraft control problem

The design of a dynamic neurocontroller with good robustness properties is presented for a multivariable aircraft control problem. The internal dynamics of the neurocontroller are synthesized by a state estimator feedback loop. The neurocontrol is generated by a multilayer feedforward neural network which is trained through backpropagation to minimize an objective function that is a weighted sum of tracking errors, and control input commands and rates. The neurocontroller exhibits good robustness through stability margins in phase and vehicle output gains. By maintaining performance and stability in the presence of sensor failures in the error loops, the structure of the neurocontroller is also consistent with the classical approach of flight control design.

Troudet, T.↗

NASA NTRS1992

Design and evaluation of a robust dynamic neurocontroller for a multivariable aircraft control problem

Troudet, T.↗

NASA NTRS2017

Very High Resolution Tree Cover Mapping for Continental United States using Deep Convolutional Neural Networks

Uncertainties in input land cover estimates contribute to a significant bias in modeled above ground biomass (AGB) and carbon estimates from satellite-derived data. The resolution of most currently used passive remote sensing products is not sufficient to capture tree canopy cover of less than ca. 10-20 percent, limiting their utility to estimate canopy cover and AGB for trees outside of forest land. In our study, we created a first of its kind Continental United States (CONUS) tree cover map at a spatial resolution of 1-m for the 2010-2012 epoch using the USDA NAIP imagery to address the present uncertainties in AGB estimates. The process involves different tasks including data acquisition ingestion to pre-processing and running a state-of-art encoder-decoder based deep convolutional neural network (CNN) algorithm for automatically generating a tree non-tree map for almost a quarter million scenes. The entire processing chain including generation of the largest open source existing aerial satellite image training database was performed at the NEX supercomputing and storage facility. We believe the resulting forest cover product will substantially contribute to filling the gaps in ongoing carbon and ecological monitoring research and help quantifying the errors and uncertainties in derived products.

High Resolution↗

DOE OSTI2022

A machine learning pipeline for membrane segmentation of cryo-electron tomograms

We describe how to use several machine learning techniques organized in a learning pipeline to segment and identify cell membrane structures from cryo electron tomograms. These tomograms are difficult to analyze with traditional segmentation tools. The learning pipeline in our approach starts from supervised learning via a special convolutional neural network trained with simulated data. It continues with semi-supervised reinforcement learning and/or a region merging technique that tries to piece together disconnected components belonging to the same membrane structure. A parametric or non-parametric fitting procedure is then used to enhance the segmentation results and quantify uncertainties in the fitting. Domain knowledge is used in generating the training data for the neural network and in guiding the fitting procedure through the use of appropriately chosen priors and constraints. We demonstrate that the approach proposed here works well for extracting membrane surfaces in two real tomogram datasets.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1990

Satellite image analysis using neural networks

The tremendous backlog of unanalyzed satellite data necessitates the development of improved methods for data cataloging and analysis. Ford Aerospace has developed an image analysis system, SIANN (Satellite Image Analysis using Neural Networks) that integrates the technologies necessary to satisfy NASA's science data analysis requirements for the next generation of satellites. SIANN will enable scientists to train a neural network to recognize image data containing scenes of interest and then rapidly search data archives for all such images. The approach combines conventional image processing technology with recent advances in neural networks to provide improved classification capabilities. SIANN allows users to proceed through a four step process of image classification: filtering and enhancement, creation of neural network training data via application of feature extraction algorithms, configuring and training a neural network model, and classification of images by application of the trained neural network. A prototype experimentation testbed was completed and applied to climatological data.

Sheldon, Roger A.↗

DOE OSTI2022

Physics Informed Reinforcement Learning for Power Grid Control using Augmented Random Search

Wide adoption of deep reinforcement learning need to overcome several challenges in energy system domain, including scalability, learning from limited samples, and high-dimensional continuous state and action spaces. In this paper, we integrated physics-based information from the normal generator operation state formula in the reinforcement learning agent's neural network loss function, and applied an augmented random search agent to optimize the generator control under dynamic contingency. Simulation results demonstrated the reliability performance improvements in training speed, reward convergence, sampling efficiency, scalability, and transferability.

physics informed ML, Physics Informed Neural Netwo↗

DOE OSTI2024

Geometry-complete diffusion for 3D molecule generation and optimization

Abstract Generative deep learning methods have recently been proposed for generating 3D molecules using equivariant graph neural networks (GNNs) within a denoising diffusion framework. However, such methods are unable to learn important geometric properties of 3D molecules, as they adopt molecule-agnostic and non-geometric GNNs as their 3D graph denoising networks, which notably hinders their ability to generate valid large 3D molecules. In this work, we address these gaps by introducing the Geometry-Complete Diffusion Model (GCDM) for 3D molecule generation, which outperforms existing 3D molecular diffusion models by significant margins across conditional and unconditional settings for the QM9 dataset and the larger GEOM-Drugs dataset, respectively. Importantly, we demonstrate that GCDM’s generative denoising process enables the model to generate a significant proportion of valid and energetically-stable large molecules at the scale of GEOM-Drugs, whereas previous methods fail to do so with the features they learn. Additionally, we show that extensions of GCDM can not only effectively design 3D molecules for specific protein pockets but can be repurposed to consistently optimize the geometry and chemical composition of existing 3D molecules for molecular stability and property specificity, demonstrating new versatility of molecular diffusion models. Code and data are freely available on GitHub .

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Hybrid Cyber-attack Detection in Photovoltaic Farms

Here, to address the cyber-physical security in PV farms, a hybrid cyber-attack detection is proposed in this manuscript. To secure PV farms, the proposed method integrates model-based and data-driven methods by fusing the detection score at the device and system levels. First, a model-based cyber-attack detection method is developed for each PV inverter. A residual between the estimation of the Kalman filter and measurement is calculated. By leveraging the calculated residual from all inverters, a squared Mahalanobis distance is developed for device detection score generation. At the system level, a convolutional neural network (CNN) is proposed to detect cyber-attack using the waveform data at the point of common coupling (PCC) in PV farms. To improve the CNN detection accuracy, a set of well-designed features are extracted from the raw waveform data. Finally, a weighted detection score fusion method is proposed to combine device and system detection scores by using their complementary strength. The feasibility and robustness of the proposed method are validated by testing cases and a comparative experiment.

14 SOLAR ENERGY↗

NASA NTRS1992

Fault diagnosis for the Space Shuttle main engine

A conceptual design of a model-based fault detection and diagnosis system is developed for the Space Shuttle main engine. The design approach consists of process modeling, residual generation, and fault detection and diagnosis. The engine is modeled using a discrete time, quasilinear state-space representation. Model parameters are determined by identification. Residuals generated from the model are used by a neural network to detect and diagnose engine component faults. Fault diagnosis is accomplished by training the neural network to recognize the pattern of the respective fault signatures. Preliminary results for a failed valve, generated using a full, nonlinear simulation of the engine, are presented. These results indicate that the developed approach can be used for fault detection and diagnosis. The results also show that the developed model is an accurate and reliable predictor of the highly nonlinear and very complex engine.

Duyar, Ahmet↗

DOE OSTI2022

Distilling physical origins of hardness in multi-principal element alloys directly from ensemble neural network models

Abstract Despite a plethora of data being generated on the mechanical behavior of multi-principal element alloys, a systematic assessment remains inaccessible via Edisonian approaches. We approach this challenge by considering the specific case of alloy hardness, and present a machine-learning framework that captures the essential physical features contributing to hardness and allows high-throughput exploration of multi-dimensional compositional space. The model, tested on diverse datasets, was used to explore and successfully predict hardness in Al x Ti y (CrFeNi) 1- x - y , Hf x Co y (CrFeNi) 1- x - y and Al x (TiZrHf) 1- x systems supported by data from density-functional theory predicted phase stability and ordering behavior. The experimental validation of hardness was done on TiZrHfAl x . The selected systems pose diverse challenges due to the presence of ordering and clustering pairs, as well as vacancy-stabilized novel structures. We also present a detailed model analysis that integrates local partial-dependencies with a compositional-stimulus and model-response study to derive material-specific insights from the decision-making process.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2020

Adaptive 3D convolutional neural network-based reconstruction method for 3D coherent diffraction imaging

We present a novel adaptive machine-learning based approach for reconstructing three-dimensional (3D) crystals from coherent diffraction imaging. We represent the crystals using spherical harmonics (SH) and generate the corresponding synthetic diffraction patterns. We utilize 3D convolutional neural networks (CNNs) to learn a mapping between 3D diffraction volumes and the SH, which describe the boundary of the physical volumes from which they were generated. We use the 3D CNN-predicted SH coefficients as the initial guesses, which are then fine-tuned using adaptive model-independent feedback for improved accuracy. We also adaptively tune the locations, intensities, and decay rates of collections of radial basis functions in order to reproduce the non-uniform internal structure of 3D objects and demonstrate the method for a synthetic volume that has an internal void and a density ramp.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R 2 -score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(10 4 ) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(10 5 ) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(10 5 ) – O(10 7 )) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

58 GEOSCIENCES↗

DOE OSTI2022

Revenue prediction for integrated renewable energy and energy storage system using machine learning techniques

Revenue estimation for integrated renewable energy and energy storage systems is important to support plant owners or operators’ decisions in battery sizing selection that leads to maximized financial performances. A common approach to optimizing revenues of a hybrid hydro and energy storage system is using mixed-integer linear programming (MILP). Although MILP models can provide accurate production cost estimations, they are typically very computationally expensive. To provide a fast yet accurate first-step information to hydropower plant owners or operators who consider integrating energy storage systems, we propose an innovative approach to predicting optimal revenues of an integrated energy generation and storage system. In this study, we examined the performance of two prediction techniques: Generalized Additive Models (GAMs) and machine learning (ML) models developed based on artificial neural networks (ANN). Predictive equations and models are generated based on optimized solutions from a market participation optimization model, the Conventional Hydropower Energy and Environmental Resource System (CHEERS) model. The two predicting techniques reduce the computational time to evaluate annual revenue for one set of battery configurations from 3 h to 1 to 4 min per run while also being implementable with significantly less data. The model validation prediction errors of developed GAMs and ML models are generally below 5%; for model testing predictions, the ML models consistently outperform the regression equations in terms of root mean square errors. This new approach allows plant owners, operators, or potential investors to quickly access multiple battery configurations under different energy generation and market scenarios. This new revenue prediction method will therefore help reduce the barriers, and thereby promoting the deployment of battery hybridization with existing renewable energy sources.

13 HYDRO ENERGY↗