SEARCH · Engineering Papers

Results for “convolutional neural network model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Attention-based convolutional capsules for evapotranspiration estimation at scale

Evapotranspiration (ET) measures the amount of water lost from the Earth's surface to the atmosphere and is an integral metric for both agricultural and environmental sciences. Understanding and quantifying ET is critical for achieving effective management of freshwater and irrigation systems. However, current ET estimation models suffer from a trade-off between accuracy and spatial coverage. In this study, we introduce our model Quench, a neural network architecture that achieves highly-accurate ET estimates over large continuous spatial extents. Quench uses our novel Attention-Based Convolutional Capsule for its neural network layers to identify areas of focus and efficiently extract ET information from satellite imagery. Benchmarks that profile our model's performance show substantive improvements in accuracy, with up to 128% increase in accuracy compared to traditional convolutional-based and process-based models. Finally, Quench also demonstrates consistent model performance over high geospatial variability and a diverse array of regions, seasons, climates, and vegetations.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Enhancing Nanoparticle Detection in Interferometric Scattering (iSCAT) Microscopy Using a Mask R-CNN

Interferometric scattering microscopy (iSCAT) is a label-free optical microscopy technique that enables imaging of individual nano-objects such as nanoparticles, viruses, and proteins. Essential to this technique is the suppression of background scattering and identification of signals from nano-objects. In the presence of substrates with high roughness, scattering heterogeneities in the background, when coupled with tiny stage movements, cause features in the background to be manifested in background-suppressed iSCAT images. Traditional computer vision algorithms detect these background features as particles, limiting the accuracy of object detection in iSCAT experiments. Here, in this paper, we present a pathway to improve particle detection in such situations using supervised machine learning via a mask region-based convolutional neural network (mask R-CNN). Using a model iSCAT experiment of 19.2 nm gold nanoparticles adsorbing to a rough layer-by-layer polyelectrolyte film, we develop a method to generate labeled datasets using experimental background images and simulated particle signals and train the mask R-CNN using limited computational resources via transfer learning. We then compare the performance of the mask R-CNN trained with and without inclusion of experimental backgrounds in the dataset against that of a traditional computer vision object detection algorithm, Haar-like feature detection, by analyzing data from the model experiment. Results demonstrate that including representative backgrounds in training datasets improved the mask R-CNN in differentiating between background and particle signals and elevated performance by markedly reducing false positives. The methodology for creating a labeled dataset with representative experimental backgrounds and simulated signals facilitates the application of machine learning in iSCAT experiments with strong background scattering and thus provides a useful workflow for future researchers to improve their image processing capabilities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

High-resolution mapping of spatial heterogeneity in ice wedge polygon geomorphology near Prudhoe Bay, Alaska

It is well known that microtopography associated with ice wedge polygons drives pronounced, meter-scale spatial gradients in hydrologic and ecological processes on the tundra. However, highresolution maps of polygonal geomorphology are rarely available, due to the complexity and subtlety of ice wedge polygon relief at landscape scales. Here we present a sub-meter resolution map of >106 discrete ice wedge polygons across a ~1200 km2 landscape, delineated within a lidar-derived digital elevation model. The delineation procedure relies on a convolutional neural network paired with a set of common image processing operations and permits explicit measurement of relative elevation at the center of each ice wedge polygon. The resulting map visualizes meter- to kilometer-scale spatial gradients in polygonal geomorphology across an extensive landscape with unprecedented detail. This high-resolution inventory of polygonal geomorphology provides rich spatial context for extrapolating observations of environmental processes across the landscape. The map also represents an extensive baseline dataset for quantifying contemporary land surface deformation (i.e., thermokarst) at the survey area, through future topographic surveys.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Identifying precursors of daily to seasonal hydrological extremes over the USA using deep learning techniques and climate model ensembles

Focal Area(s): We focus on two areas of crosscutting interest for DOE: 1) predictability of extreme precipitation and drought in the USA and 2) the integration of climate models with new AI tools, such as convolutional neural networks (CNN) and methods to understand their output (e.g. layer-wise relevance propagation; LRP). This project fits into focus area 3 of this call for white paper using AI to gain insight from complex data, including explainable AI tools. Science Challenge: Predicting hydrological extremes is important due to their impacts on people, agriculture and infrastructure. This prediction is difficult due to the infrequent occurrence of extremes and their complexity. However, extreme events can be related to more predictable conditions in the ocean, such as El Nino, long-term soil moisture or large scale modes of climate variability, such as the North Atlantic Oscillation (NAO).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Machine Learning for DUNE Supernova Trigger

One of the major scientific goals of the Deep Underground Neutrino Experiment (DUNE) is to detect and measure the neutrino flux originating from galactic core-collapse supernovae. These neutrinos provide an opportunity to study the end of life evolution of massive stars, and reveal information about the structure of core-collapse that is not visible in observations of the electromagnetic spectrum. Because of the rarity of these events, it is crucial that DUNE is able to detect supernova neutrino interactions when they occur. However, this will require sifting through a large quantity of data, motivating the development of a trigger algorithm to identify significant events and discard irrelevant data. Machine learning provides a potential approach to building this trigger. This project generates ADC and ground truth images of simulated neutrino interactions in a LArTPC detector to be used for machine learning, and uses them to train a sparse Convolutional Neural Network (C NN). The performance of this model when applied to the task of pixel classification based on interaction type is examined. This project found that the sparse CNN approach has the potential to have high accuracy in pixel classification, meaning it may be highly relevant to the development of a supernova neutrino trigger for the DUNE far detector.

Damish, S.↗

DOE OSTI2025

Introducing the DREAMS Project: DaRk mattEr and Astrophysics with Machine Learning and Simulations

We introduce the DaRk mattEr and Astrophysics with Machine learning and Simulations (DREAMS) project, an innovative approach to understanding the astrophysical implications of alternative dark matter (DM) models and their effects on galaxy formation and evolution. The DREAMS project will ultimately comprise thousands of cosmological hydrodynamic simulations that simultaneously vary over DM physics, astrophysics, and cosmology in modeling a range of systems—from galaxy clusters to ultra-faint satellites. Such extensive simulation suites can provide adequate training sets for machine-learning-based analyses. This paper introduces two new cosmological hydrodynamical suites of warm dark matter (WDM), each comprising 1024 simulations generated using the arepo code. One suite consists of uniform-box simulations covering a ${(25\,{h}^{-1}\,{\rm{Mpc}})}^{3}$ volume, while the other consists of Milky Way zoom-ins with sufficient resolution to capture the properties of classical satellites. For each simulation, the WDM particle mass is varied along with the initial density field and several parameters controlling the strength of baryonic feedback within the IllustrisTNG model. We provide two examples, separately utilizing emulators and convolutional neural networks, to demonstrate how such simulation suites can be used to disentangle the effects of DM and baryonic physics on galactic properties. The DREAMS project can be extended further to include different DM models, galaxy formation physics, and astrophysical targets. In this way, it will provide an unparalleled opportunity to characterize uncertainties on predictions for small-scale observables, leading to robust predictions for testing the particle physics nature of DM on these scales.

cosmological parameters↗

DOE OSTI2022

A comparison of neural network architectures for data-driven reduced-order modeling

The popularity of deep convolutional autoencoders (CAEs) has engendered new and effective reduced-order models (ROMs) for the simulation of large-scale dynamical systems. Despite this, it is still unknown whether deep CAEs provide superior performance over established linear techniques or other network-based methods in all modeling scenarios. To elucidate this, the effect of autoencoder architecture on its associated ROM is studied through the comparison of deep CAEs against two alternatives: a simple fully connected autoencoder, and a novel graph convolutional autoencoder. Through benchmark experiments, it is shown that the superior autoencoder architecture for a given ROM application is highly dependent on the size of the latent space and the structure of the snapshot data, with the proposed architecture demonstrating benefits on data with irregular connectivity when the latent space is sufficiently large.

42 ENGINEERING↗

DOE OSTI2020

An image-driven machine learning approach to kinetic modeling of a discontinuous precipitation reaction

Micrograph quantification is an essential component of several materials science studies. Machine learning methods, in particular convolutional neural networks, have previously demonstrated performance in image recognition tasks across several disciplines (e.g. materials science, medical imaging, facial recognition). Here, we apply these well-established methods to develop an approach to microstructure quantification for kinetic modeling of a discontinuous precipitation reaction in a case study on the uranium-molybdenum system. Prediction of material processing history based on image data (classification), calculation of area fraction of phases present in the micrographs (segmentation), and kinetic modeling from segmentation results were performed. Results indicate that convolutional neural networks represent microstructure image data well, and segmentation using the k-means clustering algorithm yields results that agree well with manually annotated images. Classification accuracies of original and segmented images are both 94% for a 5-class classification problem. Kinetic modeling results agree well with previously reported data using manual thresholding. The image quantification and kinetic modeling approach developed and presented here aims to reduce researcher bias introduced into the characterization process, and allows for leveraging information in limited image data sets.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Investigation of 3D printed lightweight hybrid composites via theoretical modeling and machine learning

Hybrid composites combine two or more different fillers to achieve multifunctional or advanced material properties, such as lightweight and enhanced mechanical properties. The properties of the composites significantly depend on their microstructures, which can be tailored via advanced 3D printing processes. Understanding the process-structure-property relationships is critical to enable the design and engineering of novel hybrid composites for applications in aerospace, automotive, and protective coatings. Here, for this work, we develop 3D printable and lightweight hybrid composites and leverage the conventional design of experiments, a theoretical hybrid model, and an image-driven machine learning (ML) method to investigate their mechanical behaviors. The hybrid composites are formulated with elastomer matrix, microfillers, and thin-shell particles, enabling a significant degree of design freedom of microstructures with densities and mechanical properties varying up to 70% and 91%, respectively. Our statistical analysis indicates that the 3D printing path direction and the microfibers fraction are dominating process parameters with contribution percentages of 45.3% and 57.7% on the specific stiffness and strength, respectively. A hybrid mechanics model is developed based on a simple Weibull distribution function and classical single-filler models to effectively capture the variations in mechanical properties, however, it overestimates the values due to its statistical constraints and idealization of experimental uncertainty. The image-driven ML model leverages the microscale images directly without losing the structural details, shows more accurate predictions with experimental data, and has 48.6% lower root mean square error than the theoretical model.

3D printing↗

DOE OSTI2023

The LSST AGN Data Challenge: Selection Methods

Abstract Development of the Rubin Observatory Legacy Survey of Space and Time (LSST) includes a series of Data Challenges (DCs) arranged by various LSST Scientific Collaborations that are taking place during the project's preoperational phase. The AGN Science Collaboration Data Challenge (AGNSC-DC) is a partial prototype of the expected LSST data on active galactic nuclei (AGNs), aimed at validating machine learning approaches for AGN selection and characterization in large surveys like LSST. The AGNSC-DC took place in 2021, focusing on accuracy, robustness, and scalability. The training and the blinded data sets were constructed to mimic the future LSST release catalogs using the data from the Sloan Digital Sky Survey Stripe 82 region and the XMM-Newton Large Scale Structure Survey region. Data features were divided into astrometry, photometry, color, morphology, redshift, and class label with the addition of variability features and images. We present the results of four submitted solutions to DCs using both classical and machine learning methods. We systematically test the performance of supervised models (support vector machine, random forest, extreme gradient boosting, artificial neural network, convolutional neural network) and unsupervised ones (deep embedding clustering) when applied to the problem of classifying/clustering sources as stars, galaxies, or AGNs. We obtained classification accuracy of 97.5% for supervised models and clustering accuracy of 96.0% for unsupervised ones and 95.0% with a classic approach for a blinded data set. We find that variability features significantly improve the accuracy of the trained models, and correlation analysis among different bands enables a fast and inexpensive first-order selection of quasar candidates.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2021

A Physics-Constrained Deep Learning Model for Simulating Multiphase Flow in 3D Heterogeneous Porous Media

Physics-based simulators for multiphase flow in porous media emulate nonlinear processes with coupled physics, and usually require extensive computational resources for software development, maintenance and simulation execution. As a result, a huge demand exists for fast modeling of coupled processes in a wide range of subsurface applications including geological sequestration, hydrocarbon recovery and geothermal energy extraction. In this work, an efficient physics-constrained deep learning model is developed for solving multiphase flow in 3-Dimensional (3D) heterogeneous porous media. The model fully leverages the spatial topology predictive capability of convolutional neural networks, specifically U-Net with successive contracting and expansive steps, and is coupled with an efficient continuity-based smoother to predict flow responses that need spatial continuity. Furthermore, the transient regions are penalized to steer the training process such that the model can accurately capture flow in these regions. The model takes inputs including properties of porous media, fluid properties and well controls, and predicts the temporal-spatial evolution of the state variables (pressure and saturation). While maintaining the continuity of fluid flow, the 3D spatial domain is decomposed into 2D images for reducing training cost, and the decomposition results in an increased number of training data samples and better training efficiency. Additionally, a surrogate model is separately constructed as a postprocessor to calculate well flow rate based on the predictions of state variables from the deep learning model. We use the example of CO 2 injection into saline aquifers, and apply the physics-constrained deep learning model that is trained from physics-based simulation data and emulates the physics process. The model performs prediction with a speedup of ~ 1400 times compared to physics-based simulations, and the average temporal errors of predicted pressure and saturation plumes are 0.27% and 0.099% respectively. Furthermore, water production rate is efficiently predicted by a surrogate model for well flow rate, with a mean error less than 5%. Therefore, with its unique scheme to cope with the fidelity in fluid flow in porous media, the physics-constrained deep learning model can become an efficient predictive model for computationally demanding inverse problems or other coupled processes.

58 GEOSCIENCES↗

DOE OSTI2025

Morphotype-resolved characterization of microalgal communities in a nutrient recovery process with ARTiMiS flow imaging microscopy

Microalgae-driven nutrient recovery represents a promising technology for phosphorus removal from wastewater while simultaneously generating biomass that can be valorized to offset treatment costs. As full-scale processes come online, system parameters including biomass composition must be carefully monitored to optimize performance and prevent culture crashes. In this study, flow imaging microscopy (FIM) was leveraged to characterize microalgal community composition in near real-time at a full-scale municipal wastewater treatment plant (WWTP) in Wisconsin, USA, and population and morphotype dynamics were examined to identify relationships between water chemistry, biomass composition, and system performance. Two FIM technologies, FlowCam and ARTiMiS, were evaluated as monitoring tools. ARTiMiS provided a more accurate estimate of total system biomass, and estimates derived from particle area as a proxy for biovolume yielded better approximations than particle counts. Deep learning classification models trained on annotated image libraries demonstrated equivalent performance between FlowCam and ARTiMiS, and convolutional neural network (CNN) classifiers proved significantly more accurate when compared to feature table-based dense neural network (DNN) models. Across a two-year study period, Scenedesmus spp. appeared most important for phosphorus removal, and were negatively impacted by elevated temperatures and increase in nitrite/nitrate concentrations. Chlorella and Monoraphidium also played an important role in phosphorus removal. For both Scenedesmus and Chlorella, smaller morphological types were more often associated with better system performance, whereas larger morphotypes likely associated with stress response(s) correlated with poor phosphorus recovery rates. Furthermore, these results demonstrate the potential of FIM as a critical technology for high-resolution characterization of industrial microalgal processes.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2022

Region-based convolutional neural network for wind turbine wake characterization from scanning lidars

A convolutional neural network is applied to lidar scan images from three experimental campaigns to identify and characterize wind turbine wakes. Initially developed as a proof-of-concept model and applied to a single data set in complex terrain, the model is now improved and generalized and applied to two other unique lidar data sets, one located near an escarpment and one located offshore. The model, initially developed using lidar scans collected in predominantly westerly flow, exhibits sensitivity to wind flow direction. The model is thus successfully generalized through implementing a standard rotation process to scan images before input into the convolutional neural network to ensure the flow is westerly. The sample size of lidar scans used to train the model is increased, and along with the generalization process, these changes to the model are shown to enhance accuracy and robustness when characterizing dissipating and asymmetric wakes. Applied to the offshore data set in which nearly 20 wind turbine wakes are included per scan, the improved model exhibits a 95% success rate in characterizing wakes and a 74% success rate in characterizing dissipating wake fragments. The improved model is shown to generalize well to the two new data sets, although an increase in wake characterization accuracy is offset by an increase in model sensitivity and false positive wake identifications.

17 WIND ENERGY↗

DOE OSTI2025

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗

DOE OSTI2021

Predictive Data-driven Platform for Subsurface Energy Production

Subsurface energy activities such as unconventional resource recovery, enhanced geothermal energy systems, and geologic carbon storage require fast and reliable methods to account for complex, multiphysical processes in heterogeneous fractured and porous media. Although reservoir simulation is considered the industry standard for simulating these subsurface systems with injection and/or extraction operations, reservoir simulation requires spatio-temporal “Big Data” into the simulation model, which is typically a major challenge during model development and computational phase. In this work, we developed and applied various deep neural network-based approaches to (1) process multiscale image segmentation, (2) generate ensemble members of drainage networks, flow channels, and porous media using deep convolutional generative adversarial network, (3) construct multiple hybrid neural networks such as convolutional LSTM and convolutional neural network-LSTM to develop fast and accurate reduced order models for shale gas extraction, and (4) physics-informed neural network and deep Q-learning for flow and energy production. We hypothesized that physicsbased machine learning/deep learning can overcome the shortcomings of traditional machine learning methods where data-driven models have faltered beyond the data and physical conditions used for training and validation. We improved and developed novel approaches to demonstrate that physics-based ML can allow us to incorporate physical constraints (e.g., scientific domain knowledge) into ML framework. Outcomes of this project will be readily applicable for many energy and national security problems that are particularly defined by multiscale features and network systems.

58 GEOSCIENCES↗

DOE OSTI2025

Dynamical Sketching for Enhanced Communication Efficiency in Federated Learning

Federated learning (FL) has revolutionized distributed machine learning by enabling collaborative model training without sharing local data. However, communication efficiency and privacy guarantees remain significant challenges. This paper introduces a dynamic sketching mechanism in FL, optimizing the trade-off between communication efficiency and model accuracy. By dynamically selecting the sketch matrix size, our approach adapts to the evolving characteristics of the data and the model, ensuring optimal performance across diverse scenarios. We leverage Bayesian optimization to systematically tune the sketch parameters, achieving an effective balance between resource efficiency and model performance. Experimental results on the MNIST dataset using a convolutional neural network (CNN) architecture validate the proposed method's efficiency and scalability. Our dynamic sketching approach significantly outperforms fixed-size sketching techniques, achieving higher compression ratios (up to 62x) and providing better privacy guarantees while maintaining high model accuracy. These findings highlight the robustness and versatility of our approach and make it a valuable solution for privacy-preserving, communication-efficient federated learning.

Afrose, Sharmin [ORNL]↗

DOE OSTI2022

Deep Learning Based Superconducting Radio-Frequency Cavity Fault Classification at Jefferson Laboratory

This work investigates the efficacy of deep learning (DL) for classifying C100 superconducting radio-frequency (SRF) cavity faults in the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. CEBAF is a large, high-power continuous wave recirculating linac that utilizes 418 SRF cavities to accelerate electrons up to 12 GeV. Recent upgrades to CEBAF include installation of 11 new cryomodules (88 cavities) equipped with a low-level RF system that records RF time-series data from each cavity at the onset of an RF failure. Typically, subject matter experts (SME) analyze this data to determine the fault type and identify the cavity of origin. This information is subsequently utilized to identify failure trends and to implement corrective measures on the offending cavity. Manual inspection of large-scale, time-series data, generated by frequent system failures is tedious and time consuming, and thereby motivates the use of machine learning (ML) to automate the task. This study extends work on a previously developed system based on traditional ML methods (Tennant and Carpenter and Powers and Shabalina Solopova and Vidyaratne and Iftekharuddin, Phys. Rev. Accel. Beams, 2020, 23, 114601), and investigates the effectiveness of deep learning approaches. The transition to a DL model is driven by the goal of developing a system with sufficiently fast inference that it could be used to predict a fault event and take actionable information before the onset (on the order of a few hundred milliseconds). Because features are learned, rather than explicitly computed, DL offers a potential advantage over traditional ML. Specifically, two seminal DL architecture types are explored: deep recurrent neural networks (RNN) and deep convolutional neural networks (CNN). We provide a detailed analysis on the performance of individual models using an RF waveform dataset built from past operational runs of CEBAF. In particular, the performance of RNN models incorporating long short-term memory (LSTM) are analyzed along with the CNN performance. Furthermore, comparing these DL models with a state-of-the-art fault ML model shows that DL architectures obtain similar performance for cavity identification, do not perform quite as well for fault classification, but provide an advantage in inference speed.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Informative and non-informative decomposition of turbulent flow fields

Not all the information in a turbulent field is relevant for understanding particular regions or variables in the flow. Here, we present a method for decomposing a source field into its informative Φ I (x, t) and residual Φ R (x, t) components relative to another target field. The method is referred to as informative and non-informative decomposition (IND). All the necessary information for physical understanding, reduced-order modelling and control of the target variable is contained in Φ I (x, t), whereas Φ R (x, t) offers no substantial utility in these contexts. The decomposition is formulated as an optimisation problem that seeks to maximise the time-lagged mutual information of the informative component with the target variable while minimising the mutual information with the residual component. The method is applied to extract the informative and residual components of the velocity field in a turbulent channel flow, using the wall shear stress as the target variable. We demonstrate the utility of IND in three scenarios: (i) physical insight into the effect of the velocity fluctuations on the wall shear stress; (ii) prediction of the wall shear stress using velocities far from the wall; and (iii) development of control strategies for drag reduction in a turbulent channel flow using opposition control. In case (i), IND reveals that the informative velocity related to wall shear stress consists of wall-attached high- and low-velocity streaks, collocated with regions of vertical motions and weak spanwise velocity. This informative structure is embedded within a larger-scale streak–roll structure of residual velocity, which bears no information about the wall shear stress. In case (ii), the best-performing model for predicting wall shear stress is a convolutional neural network that uses the informative component of the velocity as input, while the residual velocity component provides no predictive capabilities. Finally, in case (iii), we demonstrate that the informative component of the wall-normal velocity is closely linked to the observability of the target variable and holds the essential information needed to develop successful control strategies.

97 MATHEMATICS AND COMPUTING↗