Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Convolutional neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Constraining the Multiscale Structure of Geophysical Fields in Machine Learning: The Case of Precipitation

The use of deep-learning algorithms for estimating the value of geophysical variables from remotely sensed information is rapidly expanding. The typical objective function minimized in such algorithms is the mean square error (MSE), which is known to lead to smooth estimates with compressed dynamical range as compared to the true distribution of the variable of interest. Here, we introduce and evaluate alternative objective functions, focusing on the retrieval of precipitation rates from satellite passive microwave radiometric measurements using a deep convolutional neural network. For this testbed application, the results show that explicitly imposing the preservation of the statistical distribution and spatial wavelet power spectrum of the target variable allows to accurately reproduce extreme values and sharp gradients across multiple scales in this study.

58 GEOSCIENCES↗
DOE OSTI2022

Bridging Python to Silicon: The SODA Toolchain

Systems performing scientific computing, data analysis, and machine learning tasks have a growing demand for application-specific accelerators that can provide high computational performance while meeting strict size and power requirements. However, the algorithms and applications that need to be accelerated are evolving at a rate that is incompatible with manual design processes based on hardware description languages. Agile hardware design tools based on compiler techniques can help by quickly producing an application-specific integrated circuit (ASIC) accelerator starting from a high-level algorithmic description. Here, we present the software-defined accelerator (SODA) synthesizer, a modular and open-source hardware compiler that provides automated end-to-end synthesis from high-level software frameworks to ASIC implementation, relying on multilevel representations to progressively lower and optimize the input code. Our approach does not require the application developer to write any register-transfer level code, and it is able to reach up to 364 giga floating point operations per second (GFLOPS)/W efficiency (32-bit precision) on typical convolutional neural network operators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Dynamic Validation of CNN-Based Surrogate Models for Inverter-Based Resources in Open-Source Solvers

Traditionally, distribution system planning has focused on steady-state analyses, with limited consideration of dynamic behavior. However, as large or medium-scale inverter-based resources (IBRs), particularly grid-following (GFL) inverters in commercial or industry buildings, become more prevalent, understanding their dynamic impact is essential for grid planning and operation. This article presents an innovative deep-learning (DL)-approach using convolutional neural networks technique to model the GFL inverters. Developed from real grid-tied commercial IBR transient data, these dynamic DL models overcome proprietary constraints by requiring minimal knowledge of internal converter physics while maintaining high accuracy and flexibility. To demonstrate their applicability, the models were incorporated into GridLAB-D, an open-source, three-phase distribution analysis tool. This integration enables dynamic simulations of large-scale distribution networks with high IBR penetration stability analysis. Rigorous testing and validation, aligned with industry standards, confirmed the reliability and efficiency of this approach, paving the way for enhanced planning and operational assessments of modern power systems.

Deep-learning↗
DOE OSTI2022

Occlusion-Perturbed Deep Learning for Probabilistic Solar Forecasting via Sky Images

Solar forecasting is shifting to the probabilistic paradigm due to the inherent uncertainty within the solar resource. Input uncertainty quantification is one of the widely used and best-performing ways to model solar uncertainty. However, compared to other sources of inputs, such as numerical weather prediction models, pure sky image-based probabilistic solar forecasting lags behind. In this research, an occlusion-perturbed convolutional neural network, named the PSolarNet, is developed. The PSolarNet provides very short-term deterministic forecasts, forecast scenarios, and probabilistic forecasts of the global horizontal irradiance from sky image sequences. Case studies based on 6 years of open-source data show that the developed PSolarNet is able to generate accurate 10-minute ahead deterministic forecasts with a 5.62% normalized root mean square error, realistic and diverse forecast scenarios with a 0.966 average correlation with the actual time series, and reliable and sharp probabilistic forecasts with a 2.77% normalized continuous ranked probability score.

Bayesian model averaging↗
DOE OSTI2024

CNN-Based Phase Fault Classification in Real and Simulated Power Systems Data

This study proposes a convolutional neural network (CNN)–based two-step phase fault detection and identification method to classify anomalies in the power grid signal. Specifically, the first step checks the fault’s existence and determines the need for the second step. Subsequently, in the case of anomalies in the power grid signal, the second step identifies the type of fault, including line-to-line, single-line-to-ground, double-line-to-ground, and triple-line. Accordingly, the CNN architecture is both designed for the classification layers and trained with simulated data. To provide maximum prediction accuracy with minimum processing time, this study investigates the combinations of various feature extraction (FE) techniques, such as fast Fourier transform (FFT), amplitude and phase (AP), auto-correlation function, power spectral density, and wavelet transform (WT). Consequently, simulated and real-world results demonstrate that the proposed two-step method outperforms conventional one-step techniques, with the best performance obtained by using the combination of AP-AP, AP-WT, FFT-AP, and FFT-WT–based FE methods.

Alaca, Ozgur↗
DOE OSTI2025

A Weakly Supervised Machine Learning Procedure for Magnet Quench Diagnostics

Voltage taps remain the standard and reliable diagnostic tool for detecting quenches in superconducting magnets. However, they identify a quench only at the time of voltage rise and do not provide information on earlier physical precursors. In this work, we investigate whether acoustic emission data can reveal precursor activity that occurs before conventional voltage detection using machine learning techniques. We introduce an event selection method and a weakly supervised machine learning procedure to learn data-driven criteria for identifying potential acoustic precursors to quenches. Two Convolutional Neural Network (CNN) architectures are trained: one on acoustic sensor events from our selection procedure and one on the Fast Fourier Transforms (FFTs) of these events. Both networks are trained iteratively using confidence-weighted loss functions to associate certain subsets of training data with a precursor label. We evaluate the performance of these models by examining the time distribution of events classified as potential precursors relative to the quench onset. Results indicate that the proposed approach can possibly distinguish acoustic emission events occurring closer to the quench from earlier acoustic activity during ramping, suggesting the potential for flagging quench precursors in acoustic data.

Khan, Maira [Fermilab] (ORCID:0009000891602387)↗
DOE OSTI2022

Making Invisible Visible: Data-Driven Seismic Inversion With Spatio-Temporally Constrained Data Augmentation

Deep learning and data-driven approaches have shown great potential in scientific domains. The promise of data-driven techniques relies on the availability of a large volume of high-quality training datasets. Due to the high cost of obtaining data through expensive physical experiments, instruments, and simulations, data augmentation techniques for scientific applications have emerged as a new direction for obtaining scientific data recently. However, existing data augmentation techniques originating from computer vision yield physically unacceptable data samples that are not helpful for the domain problems that we are interested in. In this article, we develop new data augmentation techniques based on convolutional neural networks. Specifically, our generative models leverage different physics knowledge (such as governing equations, observable perception, and physics phenomena) to improve the quality of the synthetic data. To validate the effectiveness of our data augmentation techniques, we apply them to solve a subsurface seismic full-waveform inversion using simulated CO 2 leakage data. Our interest is to invert for subsurface velocity models associated with very small CO 2 leakage. We validate the performance of our methods using comprehensive numerical tests. Here via comparison and analysis, we show that data-driven seismic imaging can be significantly enhanced by using our data augmentation techniques. Particularly, the imaging quality has been improved by 15% in test scenarios of general-sized leakage and 17% in small-sized leakage when using an augmented training set obtained with our techniques.

58 GEOSCIENCES↗
DOE OSTI2021

Real-Time and Adaptive Reservoir Computing With Application to Profile Prediction in Fusion Plasma

Nuclear fusion is a promising alternative to address the problem of sustainable energy production. The tokamak is an approach to fusion based on magnetic plasma confinement, constituting a complex physical system with many control challenges. Here we study the characteristics and optimization of reservoir computing (RC) for real-time and adaptive prediction of plasma profiles in the DIII-D tokamak. Our experiments demonstrate that RC achieves comparable results to state-of-the-art (deep) convolutional neural networks (CNNs) and long short-term memory (LSTM) models, with a significantly easier and faster training procedure. This efficient approach allows for fast and frequent adaptation of the model to new situations, such as changing plasma conditions or different fusion devices.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Learning-Based Real-Time Event Identification Using Rich Real PMU Data

A large-scale deployment of phasor measurement units (PMUs) that reveal the inherent physical laws of power systems from a data perspective enables an enhanced awareness of power system operation. However, the high-granularity and non-stationary nature of PMU data and imperfect data quality could bring great technical challenges for real-time system event identification. To address these challenges, this paper proposes a two-stage learning-based framework. In the first stage, a Markov transition field (MTF) algorithm is exploited to extract the latent data features by encoding temporal dependency and transition statistics of PMU data in graphs. Then, a spatial pyramid pooling (SPP)-aided convolutional neural network (CNN) is established to efficiently and accurately identify power events. The proposed method fully builds on and is also tested on a large real-world dataset from several tens of PMU sources (and the corresponding event logs), located across the U.S., with a time span of two consecutive years. We report the numerical results validate that our method has high identification accuracy while showing good robustness against poor data quality.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Uncertainty Quantification of Capacitor Switching Transient Location using Machine Learning

Identification of capacitor switching transient location provides valuable insight into the state of the associated equipment. Machine learning (ML) models, and convolutional neural networks (CNNs) in particular, have demonstrated remarkable performance in signal location. However, ML models are data driven whose predictions are affected by noise in data and may also suffer from large extrapolation errors when applied to new conditions. Uncertainty quantification (UQ) is necessary to ensure model trustworthiness and avoid overconfident predictions in extrapolation. Here, in this work, we propose a novel UQ method, called PI3NN, to quantify prediction uncertainty of ML models and integrate the method with CNNs for transient source location. PI3NN calculates Prediction Intervals by training 3 Neural Networks and uses root-finding methods to determine the interval precisely. Additionally, PI3NN can identify out-of-distribution (OOD) data in a nonstationary condition to avoid overconfident prediction. Results indicate that with PI3NN, transient signals are not only correctly identified, but when said signals are subject to corruptions characteristic of an actual power monitoring system (e.g. non-ideal sensors), the model recognizes when it is uncertain about its predictions, effectively letting the user know when to accept or discard the results.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Deep Learning-Based Failure Prognostic Model for PV Inverter Using Field Measurements

Here, this study presents a novel approach for the precise monitoring and prognosis of photovoltaic (PV) inverter status, which is crucial for the proactive maintenance of PV systems. It addresses the gaps in traditional model-based methods, which tend to neglect the overall reliability of inverters, and the limitations of data-driven approaches that largely depend on simulated data. This research presents a robust solution applicable to real-world scenarios. The proposed data-driven model for PV inverter failure prognosis employs actual inverter measurements, integrating various operational and weather-related factors based on domain knowledge. This approach effectively represents inverter stressors and operational status. Utilizing an Enhanced Siamese Convolutional Neural Network (ESCNN), the model merges operational data with domain knowledge features, redefining the prognosis challenge as a classification task. Furthermore, the paper discusses an ESCNN-based real-time inverter failure monitoring method developed on the well-trained model. The proposed models are rigorously trained and tested with real inverter data and a novel filtering method is included to address accidental failures in practical scenarios. The results validate the model's efficacy, and the directions for future research are also outlined.

42 ENGINEERING↗
DOE OSTI2021

A Visual Designer of Layer‐wise Relevance Propagation Models

Abstract Layer‐wise Relevance Propagation (LRP) is an emerging and widely‐used method for interpreting the prediction results of convolutional neural networks (CNN). LRP developers often select and employ different relevance backpropagation rules and parameters, to compute relevance scores on input images. However, there exists no obvious solution to define a “best” LRP model. A satisfied model is highly reliant on pertinent images and designers' goals. We develop a visual model designer, named as VisLRPDesigner, to overcome the challenges in the design and use of LRP models. Various LRP rules are unified into an integrated framework with an intuitive workflow of parameter setup. VisLRPDesigner thus allows users to interactively configure and compare LRP models. It also facilitates relevance‐based visual analysis with two important functions: relevance‐based pixel flipping and neuron ablation. Several use cases illustrate the benefits of VisLRPDesigner. The usability and limitation of the visual designer is evaluated by LRP users.

Huang, Xinyi↗
DOE OSTI2022

ASDFL: An adaptive super‐pixel discriminative feature‐selective learning for vehicle matching

Abstract There are a large number of cameras in modern transportation system that capture numerous vehicle images continuously. Therefore, automatic analysis of these vehicle images is helpful for traffic flow management, criminal investigations and vehicle inspections. Vehicle matching, which aims to determine whether two input images depict an identical vehicle, is one of the core tasks in vehicle analysis. Recent relevant studies have focused on local feature extraction instead of global extraction, since local details can provide crucial cues to distinguish between cars. However, these methods do not select local features; that is, they do not assign weights to local features. Therefore, in this research, we systematically study the vehicle matching task, and present a novel annotation‐free local‐based deep learning method called Adaptive super‐pixel discriminative feature‐selective learning (ASDFL) to address this issue. In ASDFL, vehicle images are segmented into clusters of super‐pixels of similar size by considering the location and colour similarities of pixels without using any component‐level annotation. These super‐pixels are deemed to be the virtual components of vehicles. Moreover, a convolutional neural network is used to extract the deep features of these virtual components. Thereafter, an instance‐specific mask generation module driven by the extracted global features is enhanced to produce a mask to select the most distinctive virtual components of each vehicle image pair in the feature space. Finally, the vehicle matching task is accomplished by classifying the selected virtual component features of each imaged vehicle pair. Extensive experiments on two popular vehicle identification benchmarks demonstrate that our method is 1.57% and 0.8% more accurate than the previous baselines in a vehicle matching task on the VeRi and VehicleID datasets, respectively, which demonstrates the effectiveness of our method.

Qin, Rong↗
DOE OSTI2022

High–throughput measurement of plant fitness traits with an object detection method using Faster R–CNN

Revealing the contributions of genes to plant phenotype is frequently challenging because loss-of-function effects may be subtle or masked by varying degrees of genetic redundancy. Such effects can potentially be detected by measuring plant fitness, which reflects the cumulative effects of genetic changes over the lifetime of a plant. However, fitness is challenging to measure accurately, particularly in species with high fecundity and relatively small propagule sizes such as Arabidopsis thaliana. An image segmentation-based method using the software ImageJ and an object detection-based method using the Faster Region-based Convolutional Neural Network (R-CNN) algorithm were used for measuring two Arabidopsis fitness traits: seed and fruit counts. The segmentation-based method was error-prone (correlation between true and predicted seed counts, r 2 = 0.849) because seeds touching each other were undercounted. By contrast, the object detection-based algorithm yielded near perfect seed counts (r 2 = 0.9996) and highly accurate fruit counts (r 2 = 0.980). Comparing seed counts for wild-type and 12 mutant lines revealed fitness effects for three genes; fruit counts revealed the same effects for two genes. Our study provides analysis pipelines and models to facilitate the investigation of Arabidopsis fitness traits and demonstrates the importance of examining fitness traits when studying gene functions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

The Value of Information From Horizontal Distributed Acoustic Sensing Compared to Multicomponent Geophones Via Machine Learning

Abstract Faults play an important role in recharging many geothermal reservoirs, and seismic information can image the locations of these faults. The value of information (VOI) metric is used to objectively quantify and compare the value of two types of seismic receiver data via a machine learning approach. The demonstrated VOI methodology is novel by including spatial models from seismic data and obtaining the information statistics from machine learning. Our two-dimensional numerical experiments compare images created from sparsely spaced (80 m), two-component geophone sampling to high spatial resolution (1 m), single-component DAS. We used a three-fold cross validation of a U-Net convolutional neural networks to achieve average classification statistics. The results suggest that when horizontal sources are utilized, geophones and DAS identify reflectors and non-reflectors at roughly the same rate. The average F1 score for horizontal DAS is 0.939 and 0.931 for geophones. For images created from a vertical source, DAS performed marginally better (F1 = 0.919) than geophones (F1 = 0.877). Our transferrable methodology can provide guidance on which acquisition scenarios can improve images of important structures in the subsurface and present an efficient method for obtaining reliability statistics from high-dimensional, spatial data.

Energy & Fuels↗
DOE OSTI2023

Data-Driven Approaches for Bead Geometry Prediction Via Melt Pool Monitoring

In the realm of additive manufacturing, the selection of process parameters to avoid over and under deposition entails a time-consuming and resource-intensive trial-and-error approach. Given the distinct characteristics of each part geometry, there is a pressing need for advancing real-time process monitoring and control to ensure consistent and reliable part dimensional accuracy. Here, this research shows that support vector regression (SVR) and convolutional neural network (CNN) models offer a promising solution for real-time process control due to the models’ abilities to recognize complex, non-linear patterns with high accuracy. A novel experiment was designed to compare the performance of SVR and CNN models to indirectly detect bead height from a coaxial image of a melt pool from a single-layer, single bead build. The study showed that both SVR and CNN models trained on melt pool data collected from a coaxial optical camera can accurately predict the bead height with a mean absolute percentage error of 3.67% and 3.68%, respectively.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Learning the Temporal Effect in Infrared Thermal Videos With Long Short-Term Memory for Quality Prediction in Resistance Spot Welding

With the advances of sensing technology, in-situ infrared thermal videos can be collected from Resistance Spot Welding (RSW) processes. Each video records the formulation process of a weld nugget. The nugget evolution creates a “temporal effect” across the frames, which can be leveraged for real-time, nondestructive evaluation (NDE) of the weld quality. Currently, quality prediction with imaging data mainly focuses on optical feature extraction with Convolutional Neural Network (CNN) but does not make the most of such temporal effect. In this study, pixels corresponding to critical locations on the weld nugget surface are extracted from a video to form multivariate time series (MTS). Multivariate Adaptive Regression Splines (MARS) is used in MTS processing to remove noisy signals related to uninformative frames. A Stacked Long Short-Term Memory (LSTM) model is developed to learn from the processed MTS and then predicts weld nugget size and thickness in real-time NDE. Results from a case study on RSW of Boron steel demonstrates the improvement in prediction accuracy and computational time with the proposed method, as compared to CNN-based weld quality prediction.

Guo, Shenghan↗
DOE OSTI2025

Machine-learning-enabled on-the-fly analysis of RHEED patterns during thin film deposition by molecular beam epitaxy

Thin film deposition is a fundamental technology for the discovery, optimization, and manufacturing of functional materials. Deposition by molecular beam epitaxy (MBE) typically employs reflection high-energy electron diffraction (RHEED) as a real-time in situ probe of the growing film. However, the state-of-the-art for RHEED analysis during deposition requires human observation. Here, we present an approach using machine learning (ML) methods to monitor, analyze, and interpret RHEED images on-the-fly during thin film deposition. In the analysis workflow, RHEED pattern images are collected at one frame per second and featurized using a pretrained deep convolutional neural network. The feature vectors are then statistically analyzed to identify changepoints; these changepoints can be related to changes in the deposition mode from initial film nucleation to a transition regime, smooth film deposition, and in some cases, an additional transition to a rough, islanded deposition regime. The feature vectors are additionally analyzed via graph analysis and community classification. The graph is quantified as a stabilization plot, and we show that inflection points in the stabilization plot correspond to changes in the growth regime. The full RHEED analysis workflow is termed RHAAPsody and includes data transfer and output to a visual dashboard. We demonstrate the functionality of RHAAPsody by analyzing the precaptured RHEED images from epitaxial depositions of anatase TiO2 on SrTiO3(001) and show that the analysis workflow can be executed in less than 1 s. Our approach shows promise as one component of ML-enabled real-time feedback control of the MBE deposition process.

36 MATERIALS SCIENCE↗