Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “convolutional neural network model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

Journey Over Destination: Differentiable Sensor Placement Enhances Generalization [Poster]

The challenge of reconstructing spatial fields that change over time from limited sensor data has been a focal point for many research studies. Various machine learning methods have been used in attempts to address this complex issue, including convolutional neural networks. All of the proposed methods share a common requirement that the user needs to manually determine the sensor positions. This requirement remains a limiting factor in the ongoing quest for efficient learning and accurate field reconstruction. This study aims to present a method that enables a model to optimize sensor positions via backpropagation, thereby facilitating the model’s exploration of the spatial domain and enhancing sensor positioning effectively. Indexing naturally incorporates discrete decisions. This operation is nondifferentiable which is a requirement for the application of gradient-based optimization methods. We showcased its effectiveness by training an attention-based neural network, which achieved top-tier performance on two separate datasets. To our knowledge, this represents the first fully end-to-end differentiable workflow for enhancing sensor placement within a neural network model.

58 GEOSCIENCES↗

Adaptive 3D convolutional neural network-based reconstruction method for 3D coherent diffraction imaging

We present a novel adaptive machine-learning based approach for reconstructing three-dimensional (3D) crystals from coherent diffraction imaging. We represent the crystals using spherical harmonics (SH) and generate the corresponding synthetic diffraction patterns. We utilize 3D convolutional neural networks (CNNs) to learn a mapping between 3D diffraction volumes and the SH, which describe the boundary of the physical volumes from which they were generated. We use the 3D CNN-predicted SH coefficients as the initial guesses, which are then fine-tuned using adaptive model-independent feedback for improved accuracy. We also adaptively tune the locations, intensities, and decay rates of collections of radial basis functions in order to reproduce the non-uniform internal structure of 3D objects and demonstrate the method for a synthetic volume that has an internal void and a density ramp.

36 MATERIALS SCIENCE↗

Inverse Analysis with Variational Autoencoders: A Comparison of Shallow and Deep Networks

Inverse problems are applied to determine unknown properties by matching observational data with a physical model that often takes many parameters as input. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented involving regularization with a technique known as a variational autoencoder (VAE), which is trained to map a high-dimensional parameter space with a complex structure to a low-dimensional latent space with a simple structure. We apply this approach to unconditioned realizations of the parameters (heterogeneous hydraulic fields) for a hydrogeological inverse problem. Two types of hydraulic conductivity fields are used to evaluate the characterization for the different levels of heterogeneity complexity of the physical inputs. This approach keeps the computational cost of generating the training data low. The reason is unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any groundwater flow model forward runs. In addition, this approach applies regularization on a low-dimensional latent space from the VAE and increases optimization efficiency through automatic differentiation. Furthermore, two different neural network (NN) structures are tested for their utility in using the VAE for inverse analysis. The performance of a deep, convolutional neural network strongly depends on the dimensionality of the latent space, which requires tuning. In contrast, a shallow, dense neural network provides consistently accurate characterization without tuning. Furthermore, our approach evaluates the advantages of the shallow, dense neural network over the deep, convolutional one and enables future application to a wide range of inverse problems.

97 MATHEMATICS AND COMPUTING↗

Web-Based Tools for Data-Informed Remedy Optimization: Software Theory and User Guide

This report documents the development and application of two web-based decision-support tools for pump-and-treat (P&T) groundwater remediation systems: PTOLEMY (Pump-and-Treat Optimized Location Evaluation to Maximize Yields) and OPTIMA (Optimization for Pump-and-Treat Implementation, Management, & Assessment). These tools enhance remedy design and management by leveraging advanced computational methods – specifically deep learning and multi-objective optimization – within a user-friendly platform. By integrating data-driven models with established hydrogeological knowledge, PTOLEMY and OPTIMA enable more efficient evaluation of well placement and operational strategies, helping site managers balance multiple remediation objectives under complex conditions. Both tools are implemented as modules within the SOCRATES (Suite Of Comprehensive Rapid Analysis Tools for Environmental Sites) web platform, which provides data access, visualization, and analytics to support remedy optimization across sites in the U.S. Department of Energy Office of Environmental Management complex. PTOLEMY is a rapid screening module designed to identify promising locations for new extraction wells. It employs a multi-channel three-dimensional convolutional neural network (MC3D-CNN) trained on high-fidelity simulation data to predict the relative performance (in terms of contaminant mass recovery) of potential well sites. Through an interactive web interface, PTOLEMY visualizes the probability of high performance across a site, highlighting areas where an extraction well is likely to yield above-threshold contaminant removal over a multi-year period. PTOLEMY’s map-based displays and exportable results support transparent communication of screening analyses. By focusing attention on the most favorable candidate locations, the tool augments traditional engineering judgment and physics-based modeling, providing a data informed basis for subsequent detailed evaluations. OPTIMA is a multi objective optimization module designed to find wellfield layouts and operating schedules that meet various cleanup goals. It quickly evaluates thousands of candidate setups – combinations of well locations, timing, and rates – and returns a small set of best trade-off options for comparison. At its core, OPTIMA uses a U-Net-based surrogate model – a deep-learning emulator of a groundwater flow and transport simulator – to dramatically accelerate scenario evaluations. Coupling this fast surrogate with the NSGA-II (Non-dominated Sorting Genetic Algorithm II) evolutionary algorithm, OPTIMA explores a wide decision space of well locations and schedules to identify Pareto-optimal solutions that trade off key objectives (e.g., minimizing cleanup time, maximizing contaminant mass removal, and minimizing plume extent). The tool outputs a family of optimal configurations and visualizes their trade-offs (Pareto frontiers of cleanup metrics and maps of optimized well placements). Site managers can use these results to understand the range of viable strategies and to select candidate designs for more detailed verification. OPTIMA is currently under active development and not yet fully released; this guide provides early documentation to support planning and gather user feedback.

54 ENVIRONMENTAL SCIENCES↗

Ensemble models for circuit topology estimation, fault detection and classification in distribution systems

This paper presents a methodology for simultaneous fault detection, classification, and topology estimation for adaptive protection of distribution systems. The methodology estimates the probability of the occurrence of each one of these events by using a hybrid structure that combines three sub-systems, a convolutional neural network for topology estimation, a fault detection based on predictive residual analysis, and a standard support vector machine with probabilistic output for fault classification. The input to all these sub-systems is the local voltage and current measurements. A convolutional neural network uses these local measurements in the form of sequential data to extract features and estimate the topology conditions. The fault detector is constructed with a Bayesian stage (a multitask Gaussian process) that computes a predictive distribution (assumed to be Gaussian) of the residuals using the input. Since the distribution is known, these residuals can be transformed into a Standard distribution, whose values are then introduced into a one-class support vector machine. The structure allows using a one-class support vector machine without parameter cross-validation, so the fault detector is fully unsupervised. Finally, a support vector machine uses the input to perform the classification of the fault types. All three sub-systems can work in a parallel setup for both performance and computation efficiency. In conclusion, we test all three sub-systems included in the structure on a modified IEEE123 bus system, and we compare and evaluate the results with standard approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Probing Slow Earthquakes With Deep Learning

Abstract Slow earthquakes may trigger failure on neighboring locked faults that are stressed sufficiently to break, and slow slip patterns may evolve before a nearby great earthquake. However, even in the clearest cases such as Cascadia, slow earthquakes and associated tremor have only been observed in intermittent and discrete bursts. By training a convolutional neural network to detect known tremor on a single seismic station in Cascadia, we isolate and identify tremor and slip preceding and following known larger slow events. The deep neural network can be used for the detection of quasi‐continuous tremor, providing a proxy that quantifies the slow slip rate. Furthermore, the model trained in Cascadia recognizes tremor in other subduction zones and also along the San Andreas Fault at Parkfield, suggesting a universality of waveform characteristics and source processes, as posited from experiments and theory.

58 GEOSCIENCES↗

Particle hit clustering and identification using point set transformers in liquid argon time projection chambers

Liquid argon time projection chambers are often used in neutrino physics and dark-matter searches because of their high spatial resolution. The images generated by these detectors are extremely sparse, as the energy values detected by most of the detector are equal to 0, meaning that despite their high resolution, most of the detector is unused in a particular interaction. Instead of representing all of the empty detections, the interaction is usually stored as a sparse matrix, a list of detection locations paired with their energy values. Traditional machine learning methods that have been applied to particle reconstruction such as convolutional neural networks (CNNs), however, cannot operate over data stored in this way and therefore must have the matrix fully instantiated as a dense matrix. Operating on dense matrices requires a lot of memory and computation time, in contrast to directly operating on the sparse matrix. We propose a machine learning model using a point set neural network that operates over a sparse matrix, greatly improving both processing speed and accuracy over methods that instantiate the dense matrix, as well as over other methods that operate over sparse matrices. Compared to competing state-of-the-art methods, our method improves classification performance by 14%, segmentation performance by more than 22%, while taking 80% less time and using 66% less memory. Compared to state-of-the-art CNN methods, our method improves classification performance by more than 86%, segmentation performance by more than 71%, while reducing runtime by 91% and reducing memory usage by 61%.

calibration and fitting methods↗

DeepSZ: identification of Sunyaev–Zel’dovich galaxy clusters using deep learning

Galaxy clusters identified via the Sunyaev–Zel’dovich (SZ) effect are a key ingredient in multiwavelength cluster cosmology. In this paper, we present and compare three methods of cluster identification: the standard matched filter (MF) method in SZ cluster finding, a convolutional neural networks (CNN), and a ‘combined’ identifier. We apply the methods to simulated millimeter maps for several observing frequencies for a survey similar to SPT-3G, the third-generation camera for the South Pole Telescope. The MF requires image pre-processing to remove point sources and a model for the noise, while the CNN requires very little pre-processing of images. Additionally, the CNN requires tuning of hyperparameters in the model and takes cut-out images of the sky as input, identifying the cut-out as cluster-containing or not. We compare differences in purity and completeness. The MF signal-to-noise ratio depends on both mass and redshift. Our CNN, trained for a given mass threshold, captures a different set of clusters than the MF, some with signal-to-noise-ratio below the MF detection threshold. However, the CNN tends to mis-classify cut-out whose clusters are located near the edge of the cut-out, which can be mitigated with staggered cut-out. We leverage the complementarity of the two methods, combining the scores from each method for identification. The purity and completeness are both 0.61 for MF, and 0.59 and 0.61 for CNN. The combined method yields 0.60 and 0.77, a significant increase for completeness with a modest decrease in purity. We advocate for combined methods that increase the confidence of many low signal-to-noise clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

Deep-learning-aided forward optical coherence tomography endoscope for percutaneous nephrostomy guidance

Percutaneous renal access is the critical initial step in many medical settings. In order to obtain the best surgical outcome with minimum patient morbidity, an improved method for access to the renal calyx is needed. In our study, we built a forward-view optical coherence tomography (OCT) endoscopic system for percutaneous nephrostomy (PCN) guidance. Porcine kidneys were imaged in our experiment to demonstrate the feasibility of the imaging system. Three tissue types of porcine kidneys (renal cortex, medulla, and calyx) can be clearly distinguished due to the morphological and tissue differences from the OCT endoscopic images. To further improve the guidance efficacy and reduce the learning burden of the clinical doctors, a deep-learning-based computer aided diagnosis platform was developed to automatically classify the OCT images by the renal tissue types. Convolutional neural networks (CNN) were developed with labeled OCT images based on the ResNet34, MobileNetv2 and ResNet50 architectures. Nested cross-validation and testing was used to benchmark the classification performance with uncertainty quantification over 10 kidneys, which demonstrated robust performance over substantial biological variability among kidneys. ResNet50-based CNN models achieved an average classification accuracy of 82.6%±3.0%. The classification precisions were 79%±4% for cortex, 85%±6% for medulla, and 91%±5% for calyx and the classification recalls were 68%±11% for cortex, 91%±4% for medulla, and 89%±3% for calyx. Interpretation of the CNN predictions showed the discriminative characteristics in the OCT images of the three renal tissue types. The results validated the technical feasibility of using this novel imaging platform to automatically recognize the images of renal tissue structures ahead of the PCN needle in PCN surgery.

Wang, Chen↗

RU-net for automatic characterization of TRISO fuel cross sections

During irradiation, phenomena such as kernel swelling and buffer densification may impact the performance of tristructural isotropic (TRISO) particle fuel. Post-irradiation microscopy is often used to identify these irradiation-induced morphologic changes. However, each fuel compact generally contains thousands of TRISO particles. Manually performing the work to get statistical information on these phenomena is cumbersome and subjective. Here, to reduce the subjectivity inherent in that process and to accelerate data analysis, we used convolutional neural networks (CNNs) to automatically segment cross-sectional images of microscopic TRISO layers. CNNs are a class of machine-learning algorithms specifically designed for processing structured grid data. They have gained popularity in recent years due to their remarkable performance in various computer vision tasks, including image classification, object detection, and image segmentation. In this research, we generated a large irradiated TRISO layer dataset with more than 2,000 microscopic images of cross-sectional TRISO particles and the corresponding annotated images. Based on these annotated images, we used different CNNs to automatically segment different TRISO layers. These CNNs include RU-Net (developed in this study), as well as three existing architectures: U-Net, Residual Network (ResNet), and Attention U-Net. The preliminary results show that the model based on RU-Net performs best in terms of Intersection over Union (IoU). Using CNN models, we can expedite the analysis of TRISO particle cross sections, significantly reducing the manual labor involved and improving the objectivity of the segmentation results.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Global Nuclear Explosion Discrimination Using a Convolutional Neural Network

Using P-wave seismograms, we trained a seismic source classifier using a Convolutional Neural Network. We trained for three classes: earthquake P-wave, underground nuclear explosion (UNE) P-wave, and noise. With the current absence of nuclear testing by countries that have signed the Comprehensive Test Ban Treaty, high quality seismic data from UNEs is limited. Even with limited training data, our model can accurately characterize most events recorded at regional and teleseismic distances, finding over 95% signals in the validation set. We applied the model on holdout datasets of the North Korean test explosions to evaluate the performance on unique region and station-source pairs, with promising results. Additionally, we tested on the Source Physics Experiment events to investigate the potential for chemical explosions to act as a surrogate for nuclear explosions. We anticipate that machine-learning models like our classifier system can have broad application for other seismic signals including volcanic and non-volcanic tremor, anomalous earthquakes, ice-quakes or landslide-quakes.

58 GEOSCIENCES↗

Artificial Intelligence and Computer Vision for Cost-Effective Benthic Habitat Characterizations

Machine learning applied to computer vision and pattern recognition is a type of artificial intelligence that has advanced rapidly in the last 10 to 15 years, spurred forward by breakthroughs in deep convolutional neural networks. These state-of-the-art methods are poised to become widely used in environmental monitoring applications as a result of the increasing abundance of data available from different imaging platforms (e.g., fixed-point cameras, drone surveys, high-resolution satellite data) that can be analyzed to observe, model, and understand environmental conditions. Computer vision and pattern recognition (CVPR) tools advance our ability to use imagery and camera-based tools in cost-effective ways for environmental monitoring. Although these techniques offer great potential, some challenges remain, such as the need for large sets of labeled images for model training and validation and optimized hardware and software to ensure that the models can be trained effectively and in a reasonable amount of time. Here, we have overcome many of these rate-limiting challenges by using a diverse image library built across multiple projects coupled with staff expertise and onsite computing resources. We are working toward a fully automated SPI processing system and also are moving to develop CVPR analytical tools for other imaging platforms and data sets.

02 PETROLEUM↗

Deep Learning Systems for Increased Safeguards Surveillance Review Productivity

Nuclear safeguards inspectors expend significant time and maintain intense focus in reviewing video surveillance for safeguards relevant events. To increase efficiency and reduce the time burden of safeguards inspectors performing surveillance data review, this paper presents a novel deep learning (DL) systems concept to integrate generalized DL models into the safeguards surveillance review workflow. The Agency is investigating several DL algorithms for object recognition, localization, tracking, and flagging relevant activities. The project team is working closely with nuclear safeguards inspectors to identify review use cases (based on specific safeguards objectives) and collect their associated requirements. We focused on CANDU and LWR Nuclear Power Plants (NPPs) and their associated dry storage areas as these present a particularly heavy burden on the inspector surveillance review process due to the number of these facilities under safeguards worldwide. Initial DL algorithm results on safeguards data are promising. Using a convolutional neural network, the team attained a mean average precision (mAP) of 92.9% identifying and localizing spent fuel (SF) casks from a 475 surveillance image dataset. Further, the team had initial success in training a recurrent neural network to identify reactor area activities in video clips, successfully indicating when SF casks enter or exit a pool. We discuss how such DL algorithms would be integrated into the Next Generation Surveillance Review (NGSR) software application. Another issue impacting review productivity is the long time inspectors may have to wait when running these algorithms in NGSR. We present a concept to pre-process remotely collected surveillance data with DL models as the data arrives to IAEA headquarters so that results are already available when starting a new review in NGSR. The proposed DL system concept shows a pathway and workflow for increasing an inspector’s surveillance review productivity by quickly and accurately identifying declared and undeclared safeguards relevant objects and activities in large quantities of surveillance imagery data.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Predicting Elastic Properties of Materials from Electronic Charge Density Using 3D Deep Convolutional Neural Networks

Materials representation plays a key role in machine learning-based prediction of materials properties and new materials discovery. Currently both graph and three-dimensional (3D) voxel representation methods are based on the heterogeneous elements of the crystal structures. Here, we propose to use electronic charge density (ECD) as a generic unified 3D descriptor for materials property prediction with the advantage of possessing close relation with the physical and chemical properties of materials. We developed an ECD-based 3D convolutional neural networks (CNNs) for predicting the elastic properties of materials, in which CNNs can learn effective hierarchical features with multiple convolving and pooling operations. Extensive benchmark experiments over 2170 $Fm\bar3m$ face-centered-cubic materials show that our ECD-based CNNs can achieve good performance for elasticity prediction. Especially, our CNN models based on the fusion of elemental Materials-Agnostic Platform for Informatics and Exploration features and ECD descriptors achieved the best fivefold cross-validation performance. More importantly, we showed that our ECD-based CNN models can achieve significantly better extrapolation performance when evaluated over nonredundant data sets, where there are few neighbor-training samples around test samples. As an additional validation, we evaluated the predictive performance of our models on 329 materials of space group $Fm\bar3m$ by comparing to density functional theory calculated values, which shows a better prediction power of our model for bulk modulus than shear modulus. Because of the unified representation power of ECD, it is expected that our ECD-based CNN approach can also be applied to predict other physical and chemical properties of crystalline materials.

36 MATERIALS SCIENCE↗

Explore Spatio‐Temporal Learning of Large Sample Hydrology Using Graph Neural Networks

Abstract Streamflow forecasting over gauged and ungauged basins play a vital role in water resources planning, especially under the changing climate. Increased availability of large sample hydrology data sets, together with recent advances in deep learning techniques, has presented new opportunities to explore temporal and spatial patterns in hydrological signatures for improving streamflow forecasting. The purpose of this study is to adapt and benchmark several state‐of‐the‐art graph neural network (GNN) architectures, including ChebNet, Graph Convolutional Network (GCN), and GraphWaveNet, for end‐to‐end graph learning. We explicitly represent river basins as nodes in a graph, learn the spatiotemporal nodal dependencies, and then use the learned relations to predict streamflow simultaneously across all nodes in the graph. The efficacy of the developed GNN models is investigated using the Catchment Attributes and MEteorology for Large‐sample Studies (CAMELS) data set under two settings, fixed graph topology (transductive learning), and variable graph topology (inductive learning), with the latter applicable to prediction in ungauged basins (PUB). Results indicate that GNNs are generally robust and computationally efficient, achieving similar or better performance than a baseline model trained using the long short‐term memory (LSTM) network. Further analyses are conducted to interpret the graph learning process at the edge and node levels and to investigate the effect of different model configurations. We conclude that graph learning constitutes a viable machine learning‐based method for aggregating spatiotemporal information from a multitude of sources for streamflow forecasting

Sun, Alexander Y.↗

A Weakly Supervised Machine Learning Procedure for Magnet Quench Diagnostics

Voltage taps remain the standard and reliable diagnostic tool for detecting quenches in superconducting magnets. However, they identify a quench only at the time of voltage rise and do not provide information on earlier physical precursors. In this work, we investigate whether acoustic emission data can reveal precursor activity that occurs before conventional voltage detection using machine learning techniques. We introduce an event selection method and a weakly supervised machine learning procedure to learn data-driven criteria for identifying potential acoustic precursors to quenches. Two Convolutional Neural Network (CNN) architectures are trained: one on acoustic sensor events from our selection procedure and one on the Fast Fourier Transforms (FFTs) of these events. Both networks are trained iteratively using confidence-weighted loss functions to associate certain subsets of training data with a precursor label. We evaluate the performance of these models by examining the time distribution of events classified as potential precursors relative to the quench onset. Results indicate that the proposed approach can possibly distinguish acoustic emission events occurring closer to the quench from earlier acoustic activity during ramping, suggesting the potential for flagging quench precursors in acoustic data.

Khan, Maira [Fermilab] (ORCID:0009000891602387)↗

Survey of Image Denoising Methods for Medical Image Classification

Medical imaging devices, such as X-ray machines, inherently produce images that suffer from visual noise. Our objectives were to (i.) determine the effect of image denoising on a medical image classification task, and (ii.) determine if there exists a correlation between image denoising performance and medical image classification performance. We performed the medical image classification task on chest X-rays using the DenseNet-121 convolutional neural network (CNN) and used the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics as the image denoising performance measures. We first found that different denoising methods can make a statistically significant difference in classification performance for select labels. We also found that denoising methods affect fine-tuned models more than randomly-initialized models and that fine-tuned models have significantly higher and more uniform performance than randomly-initialized models. Lastly, we found that there is no significant correlation between PSNR and SSIM values and classification performance for our task.

Michael, Peter↗

FY24 Progress Report: SRNL Analysis of ICCWR LCM and WAMS data for Corrosion and Cracking

Algorithms for Machine Learning (ML) and data analysis for the 3013 Surveillance Program have been developed in an ongoing collaborative effort by the Savannah River National Laboratory (SRNL) and the University of South Carolina (USC). The objective of the algorithms is to automate the identification of corrosion and crack formation in the Inner Container Closure Weld Region (ICCWR) of the canister system used to store Pu-bearing material. Data for corrosion and cracking is collected from large binary files generated by a Laser Confocal Microscope (LCM), the Wide Area 3D Measurement System (WAMS), or,in a recent proposal, by a Scanning Electron Microscope (SEM). The ML software uses the physical attributes in the data files (e.g., one or more of: height, color, and 16-bit grayscale values as functions of position in a plane projection) to detect signs of surface corrosion and cracking after being trained on similar data, with the features to be detected. Although the initial scope included screening for broader indicators of corrosion, e.g., pitting, identification of potential cracks was prioritized for the past several years at the request of program leadership. Labeled training data is essential to developing the ML algorithm, and enhancements to data labeling capability have been developed to address this essential precursor to application of ML routines. Efficient labeling is particularly important in view of the large volume of data required to train ML algorithms and the relative rarity of cracks in the ICCWR data set. The updated program will read binary data from either LCM, WAMS or SEM files, interrogate data attributes, facilitate user labeling of data for training ML algorithms, execute ML algorithms, output parameters from trained ML algorithms, report ML model accuracy with respect to labeled data, and generate graphical representations for various analyses. In FY24, hourglass neural networks (HNNs) that were initiated in FY22 were further developed and tested using available LCM data, and their performance was tested against that of the alternative U-Net Neural Network algorithm structure. HNNs along with previously developed Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) comprise a suite of ML tools for identification of cracks in the ICCWR

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗