Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “convolutional neural network model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Temporal Interpolation of Geostationary Satellite Imagery with Task Specific Optical Flow

Applications of satellite data in areas such as weather tracking and modeling, ecosystem monitoring, wildfire detection, and land-cover change are heavily dependent on the trade-offs to spatial, spectral and temporal resolutions of observations. In weather tracking, high-frequency temporal observations are critical and used to improve forecasts, study severe events, and extract atmospheric motion, among others. However, while the current generation of geostationary satellites have hemispheric coverage at 10-15 minute intervals, higher temporal frequency observations are ideal for studying mesoscale severe weather events. In this work, we apply a task specific optical flow approach to temporal up-sampling using deep convolutional neural networks. We apply this technique to 16-bands of GOES-R/Advanced Baseline Imager mesoscale dataset to temporally enhance full disk hemispheric snapshots of different spatial resolutions from 15 minutes to 1 minute. Experiments show the effectiveness of task specific optical flow and multi-scale blocks for interpolating high-frequency severe weather events relative to bilinear and global optical flow baselines. Lastly, we demonstrate strong performance in capturing variability during convective precipitation events.

Optical flow, temporal interpolation, geostationar↗
DOE OSTI2024

Uncertainty-aware Continuous Implicit Neural Representations for Remote Sensing Object Counting

Many existing object counting methods rely on density map estimation (DME) of the discrete grid representation by decoding extracted image semantic features from designed convolutional neural networks (CNNs). Relying on discrete density maps not only leads to information loss dependent on the original image resolution, but also has a scalability issue when analyzing high-resolution images with cubically increasing memory complexity. Furthermore, none of the existing methods can offer reliable uncertainty quantification (UQ) for the derived count estimates. To overcome these limitations, we design UNcertainty-aware, hypernetwork-based Implicit neural representations for Counting (UNIC) to assign probabilities and the corresponding counting confidence over continuous spatial coordinates. We derive a sampling-based Bayesian counting loss function and develop the corresponding model training algorithm. UNIC outperforms existing methods on the Remote Sensing Object Counting (RSOC) dataset with reliable UQ and improved interpretability of the derived count estimates. Our code is available at https://github.com/SiyuanXu-tamu/UNIC.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Out-of-distribution detection with non-parametric density estimation for models predicting processing history of uranium ore concentrates

The rapid advancement in machine learning (ML) and computer vision (CV) coincides with the growth of interest in deploying these ML/CV models in numerous fields from medicine to social science. Similar to those areas, we have witnessed a great number of works in materials science employing ML/CV models – neural networks in particular – in their studies in recent years. These models have proven to obtain accurate performance in various tasks. However, these models struggle to attain a similar performance when encountering test samples coming from a distribution that is different from the training set. More importantly, they fail without providing any warning to the users. Therefore, we propose a framework for detecting out-of-distribution (OOD) samples to alert users when a human intervention might be necessary in this work. Specifically, we explore the use of a non-parametric density estimation method to detect OOD samples. Here, we assess OOD detection capability of the proposed framework on ML models developed for categorizing precipitation routes of U 3 O 8 when encountering OOD datasets that contain samples (1) undergone different imaging acquisition process, (2) undergone different material synthesis process, and (3) different materials than ID set. Through those experiments, we achieve an average area under the receiver operating characteristic (AUROC) of at least 91% on average in detecting OOD samples. With minimal overhead cost and superior performance, the proposed framework enables a reliable and safe system when deploying in real-world scenarios.

Convolutional neural networks↗
DOE OSTI2023

Deep Learning Approach for High-accuracy Electron Counting of Monolithic Active Pixel Sensor-type Direct Electron Detectors at Increased Electron Dose

Abstract Electron counting can be performed algorithmically for monolithic active pixel sensor direct electron detectors to eliminate readout noise and Landau noise arising from the variability in the amount of deposited energy for each electron. Errors in existing counting algorithms include mistakenly counting a multielectron strike as a single electron event, and inaccurately locating the incident position of the electron due to lateral spread of deposited energy and dark noise. Here, we report a supervised deep learning (DL) approach based on Faster region-based convolutional neural network (R-CNN) to recognize single electron events at varying electron doses and voltages. The DL approach shows high accuracy according to the near-ideal modulation transfer function (MTF) and detector quantum efficiency for sparse images. It predicts, on average, 0.47 pixel deviation from the incident positions for 200 kV electrons versus 0.59 pixel using the conventional counting method. The DL approach also shows better robustness against coincidence loss as the electron dose increases, maintaining the MTF at half Nyquist frequency above 0.83 as the electron density increases to 0.06 e−/pixel. Thus, the DL model extends the advantages of counting analysis to higher dose rates than conventional methods.

Materials Science↗
DOE OSTI2023

Spatio-Temporal Anomaly Detection with Graph Networks for Data Quality Monitoring of the Hadron Calorimeter

The Compact Muon Solenoid (CMS) experiment is a general-purpose detector for high-energy collision at the Large Hadron Collider (LHC) at CERN. It employs an online data quality monitoring (DQM) system to promptly spot and diagnose particle data acquisition problems to avoid data quality loss. In this study, we present a semi-supervised spatio-temporal anomaly detection (AD) monitoring system for the physics particle reading channels of the Hadron Calorimeter (HCAL) of the CMS using three-dimensional digi-occupancy map data of the DQM. We propose the GraphSTAD system, which employs convolutional and graph neural networks to learn local spatial characteristics induced by particles traversing the detector and the global behavior owing to shared backend circuit connections and housing boxes of the channels, respectively. Recurrent neural networks capture the temporal evolution of the extracted spatial features. We validate the accuracy of the proposed AD system in capturing diverse channel fault types using the LHC collision data sets. The GraphSTAD system achieves production-level accuracy and is being integrated into the CMS core production system for real-time monitoring of the HCAL. We provide a quantitative performance comparison with alternative benchmark models to demonstrate the promising leverage of the presented system.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Toward direct spatial and intensity characterization of ultra-high-intensity laser pulses using ponderomotive scattering of free electrons

Spatial distributions of electrons ionized and scattered from ultra-low-pressure gases are proposed and experimentally demonstrated as a method to directly measure the intensity of an ultra-high-intensity laser pulse. Analytic models relating the peak scattered electron energy to the peak laser intensity are derived and compared to paraxial Runge–Kutta simulations highlighting two models suitable for describing electrons scattered from weakly paraxial beams (f#>5) for intensities in the range of 1018−1021 W cm−2. Scattering energies are shown to be dependent on gas species, emphasizing the need for specific gases for given intensity ranges. Direct measurements of the laser intensity at full power of two laser systems are demonstrated, both showing a good agreement between indirect methods of intensity measurement and the proposed method. One experiment exhibited the role of spatial aberrations in the scattered electron distribution, motivating a qualitative study on the effect. We propose the use of convolutional neural networks as a method for extracting quantitative information on the spatial structure of the laser at full power. We believe the presented technique to be a powerful tool that can be immediately implemented in many high-power laser facilities worldwide.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Graph Neural Network for Object Reconstruction in Liquid Argon Time Projection Chambers

The Exa.TrkX project presents a graph neural network (GNN) technique for low-level reconstruction of neutrino interactions in a Liquid Argon Time Projection Chamber (LArTPC). GNNs are still a relatively novel technique, and have shown great promise for similar reconstruction tasks in the Large Hadron Collider (LHC). Graphs describing particle interactions are formed by treating each detector hit as a node, with edges describing the relationships between hits. We utilise a multi-head attention message passing network which performs graph convolutions in order to label each node with a particle type.We present an updated variant of our GNN architecture, with several improvements. After testing the model on more realistic simulation with regions of unresponsive wires, the target was modified from edge classification to node classification in order to increase robustness. Removing edges as a classification target opens up a broader possibility space for edge-forming techniques; we explore the model’s performance across a variety of approaches, such as Delaunay triangulation, kNN, and radius-based methods. We also extend this model to the 3D context, sharing information between detector views. By using reconstructed 3D spacepoints to map detector hits from each wire plane, the model naively constructs 2D representations that are independent yet fully consistent.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Defect graph neural networks for materials discovery in high-temperature clean-energy applications

We present a graph neural network approach that fully automates the prediction of defect formation enthalpies for any crystallographic site from the ideal crystal structure, without the need to create defected atomic structure models as input. Here we used density functional theory reference data for vacancy defects in oxides, to train a defect graph neural network (dGNN) model that replaces the density functional theory supercell relaxations otherwise required for each symmetrically unique crystal site. Interfaced with thermodynamic calculations of reduction entropies and associated free energies, the dGNN model is applied to the screening of oxides in the Materials Project database, connecting the zero-kelvin defect enthalpies to high-temperature process conditions relevant for solar thermochemical hydrogen production and other energy applications. The dGNN approach is applicable to arbitrary structures with an accuracy limited principally by the amount and diversity of the training data, and it is generalizable to other defect types and advanced graph convolution architectures. In conclusion, it will help to tackle future materials discovery problems in clean energy and beyond.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

High–throughput measurement of plant fitness traits with an object detection method using Faster R–CNN

Revealing the contributions of genes to plant phenotype is frequently challenging because loss-of-function effects may be subtle or masked by varying degrees of genetic redundancy. Such effects can potentially be detected by measuring plant fitness, which reflects the cumulative effects of genetic changes over the lifetime of a plant. However, fitness is challenging to measure accurately, particularly in species with high fecundity and relatively small propagule sizes such as Arabidopsis thaliana. An image segmentation-based method using the software ImageJ and an object detection-based method using the Faster Region-based Convolutional Neural Network (R-CNN) algorithm were used for measuring two Arabidopsis fitness traits: seed and fruit counts. The segmentation-based method was error-prone (correlation between true and predicted seed counts, r 2 = 0.849) because seeds touching each other were undercounted. By contrast, the object detection-based algorithm yielded near perfect seed counts (r 2 = 0.9996) and highly accurate fruit counts (r 2 = 0.980). Comparing seed counts for wild-type and 12 mutant lines revealed fitness effects for three genes; fruit counts revealed the same effects for two genes. Our study provides analysis pipelines and models to facilitate the investigation of Arabidopsis fitness traits and demonstrates the importance of examining fitness traits when studying gene functions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Completion design improvement using a deep convolutional network

Maximizing stimulated natural and hydraulic fracture network is one of the primary hydraulic fracturing concerns for economic production from a horizontal shale gas well. Geomechanical facies and preexisting fractures in each stage are identified based on similarities in formation characteristics to optimize the locations of perforation clusters. This often requires analyzing large volumes of drilling, Logging While Drilling (LWD) and Measurement While Drilling (MWD) data. In this paper, we develop a methodology that calculates the mechanical specific energy (MSE) using real-time drill string acceleration signals directly from its definition. High resolution vibration signals have been collected using a tri-axial accerlometer, which was an auxiliary tool included in acoustic borehole imager. This technique provides a cost-efficient solution for engineered completion design. Furthermore, we adopt deep Convolutional Neural Network (CNN) with signal processing to build a data pipeline that effectively extracts patterns from dynamic acceleration signals for rock lateral MSE classification. First, we apply discrete wavelet transform and Short-Time Fourier Transform (STFT) for signal denoising and pattern recognition. Then we construct an image dataset using multi-scale image fusion at pixel level from 3 sensor channels, including axial, lateral acceleration spectrograms and zero-padded revolutions per minute (RPM). The resulted RGB image dataset includes 4,000 images of 5 MSE ranges with various rock strength conditions. Our results demonstrate that the proposed deep learning model can achieve more than 90% classification accuracy. The deep learning results, as a reference source, were applied in selected Marcellus Shale Energy and Environmental Lab (MSEEL) wells engineered completion located in the Marcellus shale gas site.

03 NATURAL GAS↗
DOE OSTI2023

ZeoNet: 3D convolutional neural networks for predicting adsorption in nanoporous zeolites

Zeolites are one of the most widely used materials in the chemical industry due to their nanometer-sized pores that can adsorb and react upon molecules selectively. With hundreds of known framework topologies and hundreds of thousands of computationally predicted structures, the ability to rapidly predict zeolite performance allows researchers to prioritize their efforts on the most promising structures for a given application. Although the accuracy of forcefield-based atomistic simulations has advanced significantly in the past two decades, these simulations can be computationally expensive, especially for long-chain, complex molecules. Here, we present ZeoNet, a representation learning framework using convolutional neural networks (ConvNets) and 3D volumetric representations for predicting adsorption in zeolites. ZeoNet was trained on the task of predicting Henry's constants for adsorption, k H , of n-octadecane in more than 330 000 known and predicted zeolite materials. Employing a 3D grid based on the distances to solvent-accessible surfaces, a volumetric representation that can be generated efficiently, the best-performing ZeoNet achieved a correlation coefficient r 2 = 0.977 and a mean-squared error MSE = 3.8 in ln k H , which corresponds to an error of 9.3 kJ mol -1 in adsorption free energy. In comparison, a model based on hand-designed geometric features has values of r 2 = 0.783 and MSE = 35.7. ZeoNet is also relatively efficient and can process ≈8 structures per second on an Nvidia RTX 2080TI GPU, orders of magnitude faster than forcefield-based simulations. A systematic analysis was conducted to investigate how the choice of ConvNet architectures, the linear dimension (L) and spatial resolution (Δd) of the distance grids, batch size, optimizer, and learning rate impact the model performance. We found that ConvNets based on the ResNet architecture offer the best tradeoff between expressiveness and efficiency. The performance for all models reaches a plateau at L = 30–45 Å and depends less sensitively on grid resolution, with a small benefit around Δd = 0.30–0.45 Å. Finally, saliency maps were visualized to identify which regions of the materials contributed the most to model predictions. It was found, interestingly, that the predictions are driven primarily by the accessible pore volume rather than the region occupied by the framework atoms.

36 MATERIALS SCIENCE↗
DOE OSTI2023

AutoSourceID-Classifier: Star-galaxy classification using a convolutional neural network with spatial information

Aims.Traditional star-galaxy classification techniques often rely on feature estimation from catalogs, a process susceptible to introducing inaccuracies, thereby potentially jeopardizing the classification’s reliability. Certain galaxies, especially those not manifesting as extended sources, can be misclassified when their shape parameters and flux solely drive the inference. We aim to create a robust and accurate classification network for identifying stars and galaxies directly from astronomical images. Methods.The AutoSourceID-Classifier (ASID-C) algorithm developed for this work uses 32x32 pixel single filter band source cutouts generated by the previously developed AutoSourceID-Light (ASID-L) code. By leveraging convolutional neural networks (CNN) and additional information about the source position within the full-field image, ASID-C aims to accurately classify all stars and galaxies within a survey. Subsequently, we employed a modified Platt scaling calibration for the output of the CNN, ensuring that the derived probabilities were effectively calibrated, delivering precise and reliable results. Results.We show that ASID-C, trained on MeerLICHT telescope images and using the Dark Energy Camera Legacy Survey (DECaLS) morphological classification, is a robust classifier and outperforms similar codes such as SourceExtractor. To facilitate a rigorous comparison, we also trained an eXtreme Gradient Boosting (XGBoost) model on tabular features extracted by SourceExtractor. While this XGBoost model approaches ASID-C in performance metrics, it does not offer the computational efficiency and reduced error propagation inherent in ASID-C’s direct image-based classification approach. ASID-C excels in low signal-to-noise ratio and crowded scenarios, potentially aiding in transient host identification and advancing deep-sky astronomy.

Astronomy & Astrophysics↗
DOE OSTI2021

Quantitative imaging and automated fuel pin identification for passive gamma emission tomography

Compliance of member States to the Treaty on the Non-Proliferation of Nuclear Weapons is monitored through nuclear safeguards. The Passive Gamma Emission Tomography (PGET) system is a novel instrument developed within the framework of the International Atomic Energy Agency (IAEA) project JNT 1510, which included the European Commission, Finland, Hungary and Sweden. The PGET is used for the verification of spent nuclear fuel stored in water pools. Advanced image reconstruction techniques are crucial for obtaining high-quality cross-sectional images of the spent-fuel bundle to allow inspectors of the IAEA to monitor nuclear material and promptly identify its diversion. In this work, we have developed a software suite to accurately reconstruct the spent-fuel cross sectional image, automatically identify present fuel rods, and estimate their activity. Unique image reconstruction challenges are posed by the measurement of spent fuel, due to its high activity and the self-attenuation. While the former is mitigated by detector physical collimation, we implemented a linear forward model to model the detector responses to the fuel rods inside the PGET, to account for the latter. The image reconstruction is performed by solving a regularized linear inverse problem using the fast-iterative shrinkage-thresholding algorithm. We have also implemented the traditional filtered back projection (FBP) method based on the inverse Radon transform for comparison and applied both methods to reconstruct images of simulated mockup fuel assemblies. Higher image resolution and fewer reconstruction artifacts were obtained with the inverse-problem approach, with the mean-square-error reduced by 50%, and the structural-similarity improved by 200%. We then used a convolutional neural network (CNN) to automatically identify the bundle type and extract the pin locations from the images; the estimated activity levels finally being compared with the ground truth. The proposed computational methods accurately estimated the activity levels of the present pins, with an associated uncertainty of approximately 5%.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Predicting beam transmission using 2-dimensional phase space projections of hadron accelerators

We present a method to compress the 2D transverse phase space projections from a hadron accelerator and use that information to predict the beam transmission. This method assumes that obtaining at least three projections of the 4D transverse phase space is possible and that an accurate simulation model is available for the beamline. Using a simulated model, we show that—a computer can train a convolutional autoencoder to reduce phase-space information which can later be used to predict the beam transmission. Finally, we argue that although using projections from a realistic nonlinear distribution produces less accurate results, the method still generalizes well.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Application of a Physics-Informed Convolutional Neural Network for Monitoring the Temperature Fields in High-Temperature Gas Reactors

Here, this work presents current advances in applying a physics-informed convolutional neural network (CNN) to evaluate temperature distributions in advanced reactors. Our goal is to demonstrate that the CNN can reconstruct temperature fields within the solid region of a prismatic fuel assembly in a high-temperature gas reactor (HTGR) with sensor data available in only a few cooling channels. Before that, we showcase the superior performance of the physics-informed CNN in comparison to a purely data-driven multilayer perceptron (MLP), considering a canonical heated channel setup. This analysis shows the advantages of our approach and justifies its choice. The datasets employed here are obtained upon numerical simulations performed with codes under the Nuclear Energy Advanced Modeling and Simulation program. This work is important, as industry experience indicates that the assembly material in HTGR concepts is prone to large thermal-mechanical loads nearing operational limits. This makes it crucial to characterize peak temperatures and their distributions near hot spots. Modern thermocouples are unreliable in these types of harsh environments because of the high neutron fluxes and elevated temperatures involved. The CNN-based field reconstruction represents an attractive solution, enabling sensor arrays in less aggressive locations and augmenting indirect predictions for less accessible regions. The results show that the CNN reduces prediction errors by orders of magnitude in comparison to the MLP, considering the simple yet well-representative heated channel case. In the case of the HTGR fuel assembly, the CNN can successfully reconstruct temperature fields over various cooling regimes. Furthermore, we also explore the algorithm’s ability to detect abnormalities. Interestingly, the CNN proves it has the capacity to detect blockage in one of the noninstrumented cooling channels.

Machine learning↗
NASA NTRS2024

Sensor Fault Detection in Smart Extraterrestrial Habitats Using Unsupervised Learning

Various types of sensors are needed to monitor the health state of smart deep-space habitats. However, measured data can be affected by sensor faults, which influence the health management system and consequently the decision-making. In this paper, an unsupervised learning approach based on convolutional autoencoders (CAEs) is developed to detect anomalies in temperature and pressure sensors. The proposed method is systematically investigated using a habitat simulator (HabSim). Several illustrative examples are demonstrated in the nominal and hazardous states of the habitat, including micrometeorite impact and fire scenarios. The performance of the proposed method using CAEs is compared with that of existing methods using auto-associative neural networks (AANNs) and variational autoencoders. This comparison is based on typical evaluation metrics, including precision, recall, F1 score, training time, and testing time. The effect of temperature–pressure coupling on the detection performance of CAEs and AANNs is explored by training different data-driven models, including one with temperature sensors, one with pressure sensors, and one with both temperature and pressure sensors. The effect of the number of faulty sensors on the performance of CAEs is studied, as with an increase in the number of faulty sensors, redundant information among the sensors is reduced. The capability of CAEs to change the number of sensors without redesigning the network architecture and retraining the neural network is investigated and demonstrated. The capabilities and limitations of the proposed solution are discussed.

Zixin Wang↗
DOE OSTI2023

Galaxy Spin Classification. I. Z-wise versus S-wise Spirals with the Chirality Equivariant Residual Network

The angular momentum of galaxies (galaxy spin) contains rich information about the initial condition of the universe, yet it is challenging to efficiently measure the spin direction for the tremendous amount of galaxies that are being mapped by ongoing and forthcoming cosmological surveys. We present a machine-learning-based classifier for the Z-wise versus S-wise spirals, which can help to break the degeneracy in the galaxy spin direction measurement. The proposed chirality equivariant residual network (CE-ResNet) is manifestly equivariant under a reflection of the input image, which guarantees that there is no inherent asymmetry between the Z-wise and S-wise probability estimators. We train the model with Sloan Digital Sky Survey images, with the training labels given by the Galaxy Zoo 1 project. A combination of data augmentation techniques is used during the training, making the model more robust to be applied to other surveys. We find an ~30% increase in both types of spirals when Dark Energy Spectroscopic Instrument (DESI) images are used for classification, due to the better imaging quality of DESI. We verify that the ~7σ difference between the numbers of Z-wise and S-wise spirals is due to human bias, since the discrepancy drops to <1.8σ with our CE-ResNet classification results. We discuss the potential systematics relevant to future cosmological applications.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Time-lapse seismic inversion for CO 2 saturation with SeisCO2Net: An application to Frio-II site

Seismic monitoring of geological CO 2 storage (GCS) involves highly nonlinear seismic inversion and petrophysical inversion, making it challenging to estimate CO 2 volume efficiently and detect possible early CO 2 leakages. Deep learning (DL) using convolutional neural networks (CNNs) has shown promise in solving highly nonlinear seismic inversion problems. However, direct estimation of CO 2 plume extent/saturation from time-lapse seismic gathers using DL is still underexplored, with no reported field applications to date. The investigation of field data is primarily hindered by scarcity of field data for neural network training. Other obstacles include highly nonlinear seismic-petrophysics inverse relationship, and presence of noise in field seismic data. We introduce SeisCO2Net, a deep CNN that predicts CO 2 saturation maps directly from time-lapse full waveform shot gathers. For training, we use site-specific geological information, fluid flow physics, rock physics, and seismic modeling to generate synthetic datasets that closely resemble the CO 2 storage site. Synthetic tests show promising results, inspiring us to apply SeisCO2Net's trained weights on field data collected at Frio-II GCS site by leveraging transfer learning principles. As reference, we compare SeisCO2Net's predicted CO 2 saturation maps with results obtained from physics-based inversion. Our analyses show both methods display similar CO 2 plume shapes, reasonable CO 2 plume characteristics, and comparable saturation values. Our results suggest pre-training CNNs on physics-informed synthetic datasets and then applying the learned weights to field data is a viable approach to estimating field CO 2 saturation. This method effectively addresses the scarcity of field training data, thus encouraging the feasibility of long-term GCS monitoring.

58 GEOSCIENCES↗