SEARCH · Engineering Papers

Results for “Generative Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Autoencoders for semivisible jet detection

The production of dark matter particles from confining dark sectors may lead to many novel experimental signatures. Depending on the details of the theory, dark quark production in proton-proton collisions could result in semivisible jets of particles: collimated sprays of dark hadrons of which only some are detectable by particle collider experiments. The experimental signature is characterised by the presence of reconstructed missing momentum collinear with the visible components of the jets. This complex topology is sensitive to detector inefficiencies and mis-reconstruction that generate artificial missing momentum. With this work, we propose a signal-agnostic strategy to reject ordinary jets and identify semivisible jets via anomaly detection techniques. A deep neural autoencoder network with jet substructure variables as input proves highly useful for analyzing anomalous jets. The study focuses on the semivisible jet signature; however, the technique can apply to any new physics model that predicts signatures with anomalous jets from non-SM particles.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2026

Integrating very-high-resolution imagery, Sentinel-2 time-series data, and machine learning to map shrub fractional abundance across arid and semi-arid ecosystems in China

Shrub fractional abundance (SFA), the proportion of shrub cover per unit area, serves as a critical indicator of environmental aridity and ecosystem health in arid and semi-arid regions, particularly across the Mongolian steppe. However, large-scale SFA mapping in Mongolian steppe ecosystems remains challenging due to the small crown size of shrubs, their sparse distribution, and spectral overlap with coexisting low vegetation (e.g., grasses and herbs), which hinders accurate detection using coarser-resolution satellite data or traditional field surveys. To address these challenges, we developed a two-step approach that integrates very-high-resolution (VHR) imagery, time-series Sentinel-2 data, and deep learning techniques. First, we generated high-accuracy benchmark maps of individual shrub crowns from 0.5 m VHR imagery by combining manual segmentation with a hybrid deep learning framework (Dino V2 and convolutional neural networks). Second, we used these shrub crown maps as training data to build an XGBoost model for predicting SFA from 20 m Sentinel-2 time-series data, leveraging phenological information to improve estimation. We validated our approach across 70 sites (1km 2 each) in the Inner Mongolia Autonomous Region, which is representative of Mongolian steppe ecosystems. From VHR imagery, we mapped 1.31 million shrub crowns with an accuracy of R 2 = 0.92. Scaling up with Sentinel-2 data yielded regional SFA maps with an R 2 = 0.60. Further SHAP (SHapley Additive exPlanations) analysis on the developed XGBoost model revealed that phenological metrics (particularly observations in early-May, mid-July, and late-September), which distinguish shrub phenology from that of other land cover types (e.g., grasses and bare soil), were the most influential predictors of SFA. Finally, our regional SFA maps uncovered unimodal relationships between shrub distribution and climate variables, peaking at mean annual minimum temperatures near 0 °C and annual precipitation around 200 mm. Collectively, these findings demonstrate how the integration of multi-source remote sensing and machine learning can overcome historical limitations in SFA mapping, enabling accurate, spatially continuous assessments across vast Inner-Mongolian steppe ecosystems. Our framework has the potential to be applied to other steppe ecosystems and dryland ecosystems across the Mongolian steppe and beyond, offering a foundation for improved monitoring and ecological impact assessments in the face of global climate changes.

Arid and semi-arid landscapes↗

DOE OSTI2021

DeepSZ: identification of Sunyaev–Zel’dovich galaxy clusters using deep learning

Galaxy clusters identified via the Sunyaev–Zel’dovich (SZ) effect are a key ingredient in multiwavelength cluster cosmology. In this paper, we present and compare three methods of cluster identification: the standard matched filter (MF) method in SZ cluster finding, a convolutional neural networks (CNN), and a ‘combined’ identifier. We apply the methods to simulated millimeter maps for several observing frequencies for a survey similar to SPT-3G, the third-generation camera for the South Pole Telescope. The MF requires image pre-processing to remove point sources and a model for the noise, while the CNN requires very little pre-processing of images. Additionally, the CNN requires tuning of hyperparameters in the model and takes cut-out images of the sky as input, identifying the cut-out as cluster-containing or not. We compare differences in purity and completeness. The MF signal-to-noise ratio depends on both mass and redshift. Our CNN, trained for a given mass threshold, captures a different set of clusters than the MF, some with signal-to-noise-ratio below the MF detection threshold. However, the CNN tends to mis-classify cut-out whose clusters are located near the edge of the cut-out, which can be mitigated with staggered cut-out. We leverage the complementarity of the two methods, combining the scores from each method for identification. The purity and completeness are both 0.61 for MF, and 0.59 and 0.61 for CNN. The combined method yields 0.60 and 0.77, a significant increase for completeness with a modest decrease in purity. We advocate for combined methods that increase the confidence of many low signal-to-noise clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

A Weakly-Supervised, Multitask Deep Learning Framework for Shadow Mitigation in Remote Sensing Imagery

We propose a weakly-supervised, multitask framework for training a convolutional neural network to solve the problem of cloud shadow mitigation given only cloud and shadow masks as labels. The network minimizes the Wasserstein distance between shadows and their proximal sunlit neighborhoods, generating a supervisory signal directly from within the input image. We extract further utility from the shadow mask through multitask learning by introducing an auxiliary task of shadow segmentation. Our approach is advantageous since it performs mitigation in an end-to-end framework which requires only a shadowed image for inference. We apply this process to the Landsat 8 OLI SPARCS validation data set and demonstrate plausible results.

Couwenhoven, Scott↗

DOE OSTI2020

Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks

One of the open problems in scientific computing is the long-time integration of nonlinear stochastic partial differential equations (SPDEs), especially with arbitrary initial data. We address this problem by taking advantage of recent advances in scientific machine learning and the spectral dynamically orthogonal (DO) and borthogonal (BO) methods for representing stochastic processes. The recently introduced DO/BO methods reduce the SPDE to solving a system of deterministic PDEs and a system of stochastic ordinary differential equations. Specifically, we propose two new physics-informed neural networks (PINNs) for solving time-dependent SPDEs, namely the neural network (NN)-DO/BO methods. The proposed methods incorporate the DO/BO constraints into the loss function (along with the modal decomposition of the SPDE) with an implicit form instead of generating explicit expressions for the temporal derivatives of the DO/BO modes. Hence, the NN-DO/BO methods can overcome some of the drawbacks of the original DO/BO methods. For example, we do not need the assumption that the covariance matrix of the random coefficients is invertible as in the original DO method, and we can remove the assumption of no eigenvalue crossing as in the original BO method. Moreover, the NN-DO/BO methods can be used to solve time-dependent stochastic inverse problems with the same formulation and same computational complexity as for forward problems. Furthermore, we demonstrate the capability of the proposed methods via several numerical examples, namely: (1) A linear stochastic advection equation with deterministic initial condition: we obtain good results with the proposed methods, while the original DO/BO methods cannot be applied directly in this case. (2) Long-time integration of the stochastic Burgers' equation: we show the good performance of NN-DO/BO methods, especially the effectiveness of the NN-BO approach for such problems with many eigenvalue crossings during the whole time evolution, while the original BO method fails. (3) Nonlinear reaction diffusion equation: we consider both the forward problem and the inverse problems, including very noisy initial point values, to investigate the flexibility of the NN-DO/BO methods in handling inverse and mixed type problems. Taken together, these simulation results demonstrate that the NN-DO/BO methods can be employed to effectively quantify uncertainty propagation in a wide range of physical problems, but future work should address the efficiency issue of PINNs for forward problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Evaluation of the PhaseNet Model Applied to the IMS Seismic Network

Producing a complete and accurate set of signal detections is essential for automatically building and characterizing seismic events of interest for nuclear explosion monitoring. Signal detection algorithms have been an area of research for decades, but still produce large quantities of false detections and misidentify real signals that must be detected to produce a complete global catalog of events of interest. Deep learning methods have shown promising capabilities in effectively characterizing seismic signals for complex tasks such as identifying phase arrival times. We use the PhaseNet model, a UNet-based Neural Network, trained on local distance data from northern California to predict seismic arrivals on data from the International Monitoring System (IMS) global network. We use an analyst-curated bulletin generated from this data set to compare the performance of PhaseNet to that of the Short-Term Average/Long-Term Average (STA/LTA) algorithm. We find that PhaseNet has the potential of outperforming traditional processing methods and recommend the training of a new model with the IMS data to achieve optimal performance.

58 GEOSCIENCES↗

DOE OSTI2023

From Images to Dark Matter: End-to-end Inference of Substructure from Hundreds of Strong Gravitational Lenses

Abstract Constraining the distribution of small-scale structure in our universe allows us to probe alternatives to the cold dark matter paradigm. Strong gravitational lensing offers a unique window into small dark matter halos (<10 10 M ⊙ ) because these halos impart a gravitational lensing signal even if they do not host luminous galaxies. We create large data sets of strong lensing images with realistic low-mass halos, Hubble Space Telescope (HST) observational effects, and galaxy light from HST’s COSMOS field. Using a simulation-based inference pipeline, we train a neural posterior estimator of the subhalo mass function (SHMF) and place constraints on populations of lenses generated using a separate set of galaxy sources. We find that by combining our network with a hierarchical inference framework, we can both reliably infer the SHMF across a variety of configurations and scale efficiently to populations with hundreds of lenses. By conducting precise inference on large and complex simulated data sets, our method lays a foundation for extracting dark matter constraints from the next generation of wide-field optical imaging surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Advanced control techniques for modern inertia based inverters

In this research three artificial intelligent (AI)-based techniques are proposed to regulate the voltage and frequency of a grid-connected inverter. The increase in the penetration of renewable energy sources (RESs) into the power grid has led to the increase in the penetration of fast-responding inertia-less power converters. The increase in the penetration of these power electronics converters changes the nature of the conventional grid, in which the existing kinetic inertia in the rotating parts of the enormous generators plays a vital role. The concept of virtual inertia control scheme is proposed to make the behavior of grid connected inverters more similar to the synchronous generators, by mimicking the mechanical behavior of a synchronous generator. Conventional control techniques lack to perform optimally in nonlinear, uncertain, inaccurate power grids. Besides, the decoupled control assumption in conventional VSGs makes them nonoptimal in resistive grids. The neural network predictive controller, the heuristic dynamic programming, and the dual heuristic dynamic programming techniques are presented in this research to overcome the draw backs of conventional VSGs. The nonlinear characteristics of neural networks, and the online training enable the proposed methods to perform as robust and optimal controllers. The simulation and the experimental laboratory prototype results are provided to demonstrate the effectiveness of the proposed techniques.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

Modeling design and control problems involving neural network surrogates

Here, we consider nonlinear optimization problems that involve surrogate models represented by neural networks. We demonstrate first how to directly embed neural network evaluation into optimization models, highlight a difficulty with this approach that can prevent convergence, and then characterize stationarity of such models. We then present two alternative formulations of these problems in the specific case of feedforward neural networks with ReLU activation: as a mixed-integer optimization problem and as a mathematical program with complementarity constraints. For the latter formulation we prove that stationarity at a point for this problem corresponds to stationarity of the embedded formulation. Each of these formulations may be solved with state-of-the-art optimization methods, and we show how to obtain good initial feasible solutions for these methods. We compare our formulations on three practical applications arising in the design and control of combustion engines, in the generation of adversarial attacks on classifier networks, and in the determination of optimal flows in an oil well network.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

DownScaleBench for developing and applying a deep learning based urban climate downscaling- first results for high-resolution urban precipitation climatology over Austin, Texas

Abstract Cities need climate information to develop resilient infrastructure and for adaptation decisions. The information desired is at the order of magnitudes finer scales relative to what is typically available from climate analysis and future projections. Urban downscaling refers to developing such climate information at the city (order of 1 – 10 km) and neighborhood (order of 0.1 – 1 km) resolutions from coarser climate products. Developing these higher resolution (finer grid spacing) data needed for assessments typically covering multiyear climatology of past data and future projections is complex and computationally expensive for traditional physics-based dynamical models. In this study, we develop and adopt a novel approach for urban downscaling by generating a general-purpose operator using deep learning. This ‘DownScaleBench’ tool can aid the process of downscaling to any location. The DownScaleBench has been generalized for both in situ (ground- based) and satellite or reanalysis gridded data. The algorithm employs an iterative super-resolution convolutional neural network (Iterative SRCNN) over the city. We apply this for the development of a high-resolution gridded precipitation product (300 m) from a relatively coarse (10 km) satellite-based product (JAXA GsMAP). The high-resolution gridded precipitation datasets is compared against insitu observations for past heavy rain events over Austin, Texas, and shows marked improvement relative to the coarser datasets relative to cubic interpolation as a baseline. The creation of this Downscaling Bench has implications for generating high-resolution gridded urban meteorological datasets and aiding the planning process for climate-ready cities.

Singh, Manmeet (ORCID:0000000233747149)↗

DOE OSTI2023

Peri-Net-Pro: the neural processes with quantified uncertainty for crack patterns

Abstract This paper develops a deep learning tool based on neural processes (NPs) called the Peri-Net-Pro, to predict the crack patterns in a moving disk and classifies them according to the classification modes with quantified uncertainties. In particular, image classification and regression studies are conducted by means of convolutional neural networks (CNNs) and NPs. First, the amount and quality of the data are enhanced by using peridynamics to theoretically compensate for the problems of the finite element method (FEM) in generating crack pattern images. Second, case studies are conducted with the prototype microelastic brittle (PMB), linear peridynamic solid (LPS), and viscoelastic solid (VES) models obtained by using the peridynamic theory. The case studies are performed to classify the images by using CNNs and determine the suitability of the PMB, LBS, and VES models. Finally, a regression analysis is performed on the crack pattern images with NPs to predict the crack patterns. The regression analysis results confirm that the variance decreases when the number of epochs increases by using the NPs. The training results gradually improve, and the variance ranges decrease to less than 0.035. The main finding of this study is that the NPs enable accurate predictions, even with missing or insufficient training data. The results demonstrate that if the context points are set to the 10th, 100th, 300th, and 784th, the training information is deliberately omitted for the context points of the 10th, 100th, and 300th, and the predictions are different when the context points are significantly lower. However, the comparison of the results of the 100th and 784th context points shows that the predicted results are similar because of the Gaussian processes in the NPs. Therefore, if the NPs are employed for training, the missing information of the training data can be supplemented to predict the results.

Mathematics↗

DOE OSTI2025

Prediction of electric and magnetic fields from spectral data using machine learning algorithms for Doppler-free saturation spectroscopy diagnostics

The prediction of electric and magnetic field amplitudes from atomic spectral data is critical for plasma control in fusion devices such as tokamaks. Conventional approaches that rely on physics-based models are computationally expensive and unsuitable for real-time applications. In this work, we develop and benchmark three machine learning algorithms—simulation-based inference (SBI), fully connected neural networks (FCNN), and histogram-based gradient boosting regression (GBR-Hist)—to infer field intensities directly from Doppler-free saturation spectroscopy (DFSS) spectra. Synthetic datasets of spectra were generated using the EZSSS code and evaluated both with and without added Poisson noise to mimic experimental conditions. We find that SBI achieves the highest accuracy and robustness, FCNN provides a strong balance of accuracy and computational efficiency for real-time applications, and GBR-Hist offers the fastest inference but is more sensitive to noise. Furthermore, these results demonstrate the potential of machine learning to accelerate DFSS analysis and enhance its utility for plasma diagnostics and control.

Doppler-free saturation spectroscopy↗

DOE OSTI2023

Enhancing Nanoparticle Detection in Interferometric Scattering (iSCAT) Microscopy Using a Mask R-CNN

Interferometric scattering microscopy (iSCAT) is a label-free optical microscopy technique that enables imaging of individual nano-objects such as nanoparticles, viruses, and proteins. Essential to this technique is the suppression of background scattering and identification of signals from nano-objects. In the presence of substrates with high roughness, scattering heterogeneities in the background, when coupled with tiny stage movements, cause features in the background to be manifested in background-suppressed iSCAT images. Traditional computer vision algorithms detect these background features as particles, limiting the accuracy of object detection in iSCAT experiments. Here, in this paper, we present a pathway to improve particle detection in such situations using supervised machine learning via a mask region-based convolutional neural network (mask R-CNN). Using a model iSCAT experiment of 19.2 nm gold nanoparticles adsorbing to a rough layer-by-layer polyelectrolyte film, we develop a method to generate labeled datasets using experimental background images and simulated particle signals and train the mask R-CNN using limited computational resources via transfer learning. We then compare the performance of the mask R-CNN trained with and without inclusion of experimental backgrounds in the dataset against that of a traditional computer vision object detection algorithm, Haar-like feature detection, by analyzing data from the model experiment. Results demonstrate that including representative backgrounds in training datasets improved the mask R-CNN in differentiating between background and particle signals and elevated performance by markedly reducing false positives. The methodology for creating a labeled dataset with representative experimental backgrounds and simulated signals facilitates the application of machine learning in iSCAT experiments with strong background scattering and thus provides a useful workflow for future researchers to improve their image processing capabilities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Training data selection for accuracy and transferability of interatomic potentials

Abstract Advances in machine learning (ML) have enabled the development of interatomic potentials that promise the accuracy of first principles methods and the low-cost, parallel efficiency of empirical potentials. However, ML-based potentials struggle to achieve transferability, i.e., provide consistent accuracy across configurations that differ from those used during training. In order to realize the promise of ML-based potentials, systematic and scalable approaches to generate diverse training sets need to be developed. This work creates a diverse training set for tungsten in an automated manner using an entropy optimization approach. Subsequently, multiple polynomial and neural network potentials are trained on the entropy-optimized dataset. A corresponding set of potentials are trained on an expert-curated dataset for tungsten for comparison. The models trained to the entropy-optimized data exhibited superior transferability compared to the expert-curated models. Furthermore, the models trained to the expert-curated set exhibited a significant decrease in performance when evaluated on out-of-sample configurations.

36 MATERIALS SCIENCE↗

DOE OSTI2022

PolyODENet: Deriving mass-action rate equations from incomplete transient kinetics data

Kinetics of a reaction network that follows mass-action rate laws can be described with a system of ordinary differential equations (ODEs) with polynomial right-hand side. However, it is challenging to derive such kinetic differential equations from transient kinetic data without knowing the reaction network, especially when the data are incomplete due to experimental limitations. We introduce a program, PolyODENet, toward this goal. Based on the machine-learning method Neural ODE, PolyODENet defines a generative model and predicts concentrations at arbitrary time. As such, it is possible to include unmeasurable intermediate species in the kinetic equations. Importantly, we have implemented various measures to apply physical constraints and chemical knowledge in the training to regularize the solution space. Here, using simple catalytic reaction models, we demonstrate that PolyODENet can predict reaction profiles of unknown species and doing so even reveal hidden parts of reaction mechanisms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

PolyODENet

Wu, Qin [Brookhaven National Lab. (BNL), Upton, NY↗

DOE OSTI2023

Explaining machine-learning models for gamma-ray detection and identification

As more complex predictive models are used for gamma-ray spectral analysis, methods are needed to probe and understand their predictions and behavior. Recent work has begun to bring the latest techniques from the field of Explainable Artificial Intelligence (XAI) into the applications of gamma-ray spectroscopy, including the introduction of gradient-based methods like saliency mapping and Gradient-weighted Class Activation Mapping (Grad-CAM), and black box methods like Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). In addition, new sources of synthetic radiological data are becoming available, and these new data sets present opportunities to train models using more data than ever before. In this work, we use a neural network model trained on synthetic NaI(Tl) urban search data to compare some of these explanation methods and identify modifications that need to be applied to adapt the methods to gamma-ray spectral data. We find that the black box methods LIME and SHAP are especially accurate in their results, and recommend SHAP since it requires little hyperparameter tuning. We also propose and demonstrate a technique for generating counterfactual explanations using orthogonal projections of LIME and SHAP explanations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2021

Seismic savanna: machine learning for classifying wildlife and behaviours using ground‐based vibration field recordings

Abstract We develop a machine learning approach to detect and discriminate elephants from other species, and to recognise important behaviours such as running and rumbling, based only on seismic data generated by the animals. We demonstrate our approach using data acquired in the Kenyan savanna, consisting of 8000 h seismic recordings and 250 k camera trap pictures. Our classifiers, different convolutional neural networks trained on seismograms and spectrograms, achieved 80%–90% balanced accuracy in detecting elephants up to 100 m away, and over 90% balanced accuracy in recognising running and rumbling behaviours from the seismic data. We release the dataset used in this study: SeisSavanna represents a unique collection of seismic signals with the associated wildlife species and behaviour. Our results suggest that seismic data offer substantial benefits for monitoring wildlife, and we propose to further develop our methods using dense arrays that could result in a seismic shift for wildlife monitoring.

Szenicer, Alexandre↗