Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “PCA based deep neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

12 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Efficient Signal Processing in BOTDA: Utilizing PCA and PCA-Based Neural Networks for Temperature Monitoring

This work presents a comparative analysis of the various signal processing techniques used in the Brillouin gain spectrum (BGS) peak estimation. Traditional fitting methods such as Lorentzian curve fitting (LCF) are slow and less effective in noisy data. PCA-based methods were tested on the experimental data: A Euclidian distance-based approach, and a probabilistic deep neural network (PDNN) based approach, both using 5 principal components to represent a single BGS. Both methods significantly reduce computational time with respect to LCF, whereas PDNN offers uncertainty insights along with the parameter value. Measuring a range of temperatures, analyzing accuracy, and speed, it can be concluded that PCA trained PDNN outperforms other methods, and appears to be helpful in scenario where large datasets are generated.

Brillouin optical time domain analysis↗
DOE OSTI2024

Uncertainty Quantification and Sensitivity Analysis of Low-Dimensional Manifold via Co-Kurtosis PCA in Combustion Modeling

For multi-scale multi-physics applications e.g., the turbulent combustion code Pele, robust and accurate dimensionality reduction is crucial to solving problems at exascale and beyond. A recently developed technique, Co-Kurtosis based Principal Component Analysis (CoK-PCA) which leverages principal vectors of co-kurtosis, is a promising alternative to traditional PCA for complex chemical systems. To improve the effectiveness of this approach, we employ Artificial Neural Networks for reconstructing thermo-chemical scalars, species production rates, and overall heat release rates corresponding to the full state space. Our focus is on bolstering confidence in this deep learning based non-linear reconstruction through Uncertainty Quantification (UQ) and Sensitivity Analysis (SA). UQ involves quantifying uncertainties in inputs and outputs, while SA identifies influential inputs. One of the noteworthy challenges is the computational expense inherent in both endeavors. To address this, we employ the Monte Carlo methods to effectively quantify and propagate uncertainties in our reduced spaces while managing computational demands. Our research carries profound implications not only for the realm of combustion modeling but also for a broader audience in UQ. By showcasing the reliability and robustness of CoK-PCA in dimensionality reduction and deep learning predictions, we empower researchers and decision-makers to navigate complex combustion systems with greater confidence.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R 2 -score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(10 4 ) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(10 5 ) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(10 5 ) – O(10 7 )) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

58 GEOSCIENCES↗
DOE OSTI2025

Modeling MTS pyrolysis and SiC deposition kinetics using principal component analysis and neural networks

Accurate chemical kinetics modeling is crucial for improving the efficiency of chemical processing and synthesis of ceramic matrix composites. Detailed kinetic models are computationally expensive due to the large number of transported chemical species, while the simplified physics-based models, such as single-step global mechanisms, are efficient but often overlook key chemical intermediates and pathways. Recent deep learning approaches promise accurate and cost-effective models. Yet, they require additional closures for the transported nonlinear latent variables, complicating integration with existing solvers. In this work, we develop a hybrid linear—nonlinear reduced model for silicon carbide deposition from methyltrichlorosilane precursor by combining principal component analysis (PCA) and autoencoder (AE) neural network (NN) approaches. PCA is used to identify a smaller set of linear transport variables, enabling direct reuse of conventional transport solvers. NNs then reconstruct the full chemical state from these reduced variables. We demonstrate the method on a chemical vapor deposition reactor—comprising a gas-phase pyrolysis plug flow reactor and a heterogeneous surface reactor—over a wide range of temperatures, pressures, and residence times. Our PCA–AE model achieves high accuracy with only five transported scalars, achieving an eightfold cost reduction compared to detailed mechanisms, in both a priori (using data from the test set only) and a posteriori (coupled with a differential equation solver). In conclusion, notable errors arise primarily near training domain boundaries and for long residence times, indicating the need for domain shift indicators and better long-horizon predictions in future reduced chemistry model development.

autoencoder neural networks↗
DOE OSTI2023

A co-kurtosis PCA based dimensionality reduction with nonlinear reconstruction using neural networks

For turbulent reacting flow systems, identification of low-dimensional representations of the thermo-chemical state space is vitally important, primarily to significantly reduce the computational cost of device-scale simulations. Principal component analysis (PCA), and its variants, are a widely employed class of methods. Recently, an alternative technique that focuses on higher-order statistical interactions, co-kurtosis PCA (CoK-PCA), has been shown to effectively provide a low-dimensional representation by capturing the stiff chemical dynamics associated with spatiotemporally localized reaction zones. While its effectiveness has only been demonstrated based on a priori analyses with linear reconstruction, in this work, we employ nonlinear techniques to reconstruct the full thermo-chemical state and evaluate the efficacy of CoK-PCA compared to PCA. Specifically, we combine a CoK-PCA-/PCA-based dimensionality reduction (encoding) with an artificial neural network (ANN) based reconstruction (decoding) and examine, a priori, the reconstruction errors of the thermo-chemical state. In addition, we evaluate the errors in species production rates and heat release rates, which are nonlinear functions of the reconstructed state, as a measure of the overall accuracy of the dimensionality reduction technique. We employ four datasets to assess CoK-PCA/PCA coupled with ANN-based reconstruction: zero-dimensional (homogeneous) reactor for autoignition of an ethylene/air mixture that has conventional single-stage ignition kinetics, a dimethyl ether (DME)/air mixture which has two-stage (low and high temperature) ignition kinetics, a one-dimensional freely propagating premixed ethylene/air laminar flame, and a two-dimensional dataset representing turbulent autoignition of ethanol in a homogeneous charge compression ignition (HCCI) engine. Finally, results from the analyses demonstrate the robustness of the CoK-PCA based low-dimensional manifold with ANN reconstruction in accurately capturing the data, specifically from the reaction zones.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Variational encoder geostatistical analysis (VEGAS) with an application to large scale riverine bathymetry

Estimation of riverbed profiles, also known as bathymetry, plays a vital role in many applications, such as safe and efficient inland navigation, prediction of bank erosion, land subsidence, and flood risk management. The high cost and complex logistics of direct bathymetry surveys, i.e, depth imaging, have encouraged the use of indirect measurements such as surface flow velocities. However, estimating high-resolution bathymetry from indirect measurements is an inverse problem that can be computationally challenging. Here, we propose a reduced-order model (ROM) based approach that utilizes a variational autoencoder (VAE), a type of deep neural network with a narrow layer in the middle, to compress bathymetry and flow velocity information and accelerate bathymetry inverse problems from flow velocity measurements. In our application, the shallow-water equations (SWE) with appropriate boundary conditions (BCs), e.g., the discharge and/or the free surface elevation, constitute the forward problem, to predict flow velocity. Then, ROMs of the SWEs are constructed on a nonlinear manifold of low dimensionality through a variational encoder and the bathymetry inversion problem is derived on the low-dimensional latent space in a Hierarchical Bayesian setting. Further, the reformulation allows variational inference with a small number (e.g., $\mathscr{O}$ (100) of ROM runs and efficient uncertainty quantification. We have tested our inversion approach on a one-mile reach of the Savannah River, GA, USA. Once the neural network is trained (offline stage), the proposed technique can perform the inversion operation orders of magnitude faster than traditional inversion methods that are commonly based on linear projections, such as principal component analysis (PCA), or the principal component geostatistical approach (PCGA). Furthermore, tests show that the algorithm can estimate the bathymetry with good accuracy even with sparse flow velocity measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Deep learning for exploring ultra-thin ferroelectrics with highly improved sensitivity of piezoresponse force microscopy

Hafnium oxide-based ferroelectrics have been extensively studied because of their existing ferroelectricity, even in ultra-thin film form. However, studying the weak response from ultra-thin film requires improved measurement sensitivity. In general, resonance-enhanced piezoresponse force microscopy (PFM) has been used to characterize ferroelectricity by fitting a simple harmonic oscillation model with the resonance spectrum. However, an iterative approach, such as traditional least squares (LS) fitting, is sensitive to noise and can result in the misunderstanding of weak responses. In this study, we developed the deep neural network (DNN) hybrid with deep denoising autoencoder (DDA) and principal component analysis (PCA) to extract resonance information. The DDA/PCA-DNN improves the PFM sensitivity down to 0.3 pm, allowing measurement of weak piezoresponse with low excitation voltage in 10-nm-thick Hf 0.5 Zr 0.5 O 2 thin films. Our hybrid approaches could provide more chances to explore the low piezoresponse of the ultra-thin ferroelectrics and could be applied to other microscopic techniques.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Cy-Phy ADS: Cyber Physical Anomaly Detection Framework for EV Charging Systems

Today’s large-scale Electric Vehicle (EV) infrastructures are heavily dependent on information communication technologies to maintain their operation and to support communication within sub-system components as well as the outside world. These technologies are vulnerable to various cyber and physical threats. Timely identification and mitigation of these threats are critical for improving human safety, avoiding economic losses, and preventing catastrophic system failures. By addressing this, our work presents a ResNet Autoencoder (AE) based Cyber-Physical Anomaly Detection System (Cy-Phy ADS) for detecting anomalies in EV Controller Area Network (CAN) protocol communication. It consists of four main components: Cyber-Physical Feature Extractor, ResNet AE-based Anomaly Detection Framework, Cyber-Physical Health Metric (CPHM), and Visualization Dashboard. The presented framework was trained and tested using CAN data collected from the EV charging system testbed at the Idaho National Laboratory. The presented Cy-Phy ADS compared against six widely used unsupervised anomaly detection algorithms: One Class Support Vector Machine (OCSVM), Variational Autoencoder (VAE), LSTM Autoencoder (LSTM AE), Isolation Forest (IForest), Principle Component Analysis (PCA) and Local Outlier Factor (LOF). Here the presented approach showed the highest accuracy among the compared methods. Further, the proposed approach showed comparable performance in terms of precision, F1, and False positive rate. It also showed the lowest training and inference time compared to the neural network-based baseline algorithms compared against with. Additionally, the Cy-Phy ADS has advantages such as unsupervised training, the ability to provide a holistic metric for system health characterization, and non-linear feature extraction.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Cross-Feature Transfer Learning for Efficient Tensor Program Generation

Tuning tensor program generation involves navigating a vast search space to find optimal program transformations and measurements for a program on the target hardware. The complexity of this process is further amplified by the exponential combinations of transformations, especially in heterogeneous environments. This research addresses these challenges by introducing a novel approach that learns the joint neural network and hardware features space, facilitating knowledge transfer to new, unseen target hardware. A comprehensive analysis is conducted on the existing state-of-the-art dataset, TenSet, including a thorough examination of test split strategies and the proposal of methodologies for dataset pruning. Leveraging an attention-inspired technique, we tailor the tuning of tensor programs to embed both neural network and hardware-specific features. Notably, our approach substantially reduces the dataset size by up to 53% compared to the baseline without compromising Pairwise Comparison Accuracy (PCA). Furthermore, our proposed methodology demonstrates competitive or improved mean inference times with only 25–40% of the baseline tuning time across various networks and target hardware. The attention-based tuner can effectively utilize schedules learned from previous hardware program measurements to optimize tensor program tuning on previously unseen hardware, achieving a top-5 accuracy exceeding 90%. This research introduces a significant advancement in autotuning tensor program generation, addressing the complexities associated with heterogeneous environments and showcasing promising results regarding efficiency and accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Pulsed Thermal Tomography Nondestructive Examination of Additively Manufactured Reactor Materials and Components (Final Technical Report)

Metal Additive Manufacturing (AM) is a promising method for cost-efficient fabrication of complex shape structures for applications in harsh environment, such as in a nuclear reactor. However, internal defects (pores) occur in high-strength AM alloys, which are manufactured with Laser Powder Bed Fusion (LPBF) AM method. Pulsed Infrared Thermography (PIT) is an efficient nondestructive evaluation (NDE) method to examine actual structures, because this method offers one-sided non-contact measurements, and fast processing of large sample areas. However, imaging of material defects, particularly defects with sizes at microscopic level, is challenging. In this report, we benchmark the performance of several Unsupervised Learning (UL) algorithms designed to enhance imaging of microscopic defects in metals with PIT. UL aims to learn the latent principal patterns (dictionaries) in PIT data to detect defects with minimal human supervision. Performance of Independent Component Analysis (ICA), Sparse Coding (SC), Principal Component Analysis (PCA) and Exploratory Factor Analysis (EFA) was compared using F-score, UL model training time and defects reconstruction time. We obtained the average F-score of 0.75, and a highest F-score of 0.89 for the EFA algorithm. Overall, EFA outperforms other UL algorithms considered in this study. In another approach, we investigate Thermal Tomography (TT), which is a computational method for reconstruction of depth profile of internal material defects from PIT nondestructive evaluation (NDE). TT algorithm obtains depth reconstructions of thermal effusivity, which has been shown to provide visualization of subsurface internals defects in metals. In many applications, one needs to determine the defect shape and orientation from reconstructed effusivity images. Interpretation of TT images is non-trivial because of blurring, which increases with depth due to heat diffusion-based nature of image formation. We have developed a deep learning convolutional neural network (CNN) to classify size and orientation of subsurface material defects in TT images. CNN was trained with TT images produced with computer simulations of 2D metallic structures (thin plates) containing elliptical subsurface voids. Performance of CNN was investigated using test TT images developed with computer simulations of plates containing elliptical defects, and defects with shape imported from scanning electron microscopy (SEM) images. CNN demonstrated the ability to classify radii and angular orientation of elliptical defects in previously unseen test TT images. We have also demonstrated that CNN trained on TT images of elliptical defects is capable of classifying shape and orientation of irregular defects. Training the CNN on irregular defect shapes instead of on elliptical shapes would make the resulting classifications more descriptive of actual defect shapes. However, this requires a much higher volume of SEM images of material defects, which are difficult to obtain because of random occurrence of defects in LPBF. To address this challenge, we developed a generative adversarial network (GAN) to augment the existing dataset of SEM defect images. The GAN model is demonstrated to create novel yet realistic defect shapes that can be used as input for simulated PTT images to train CNN. We also investigate several approaches based on Gaussian Random Circle and Bezier Curves for constructing parametric models of irregular-shape defects.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Mallat Scattering Transformation based surrogate for Magnetohydrodynamics

Abstract A Machine and Deep Learning (MLDL) methodology is developed and applied to give a high fidelity, fast surrogate for 2D resistive MagnetoHydroDynamic (MHD) simulations of Magnetic Liner Inertial Fusion (MagLIF) implosions. The resistive MHD code is used to generate an ensemble of implosions with different liner aspect ratios, initial gas preheat temperatures (that is, different adiabats), and different liner perturbations. The liner density and magnetic field as functions of x , y , and z were generated. The Mallat Scattering Transformation (MST) is taken of the logarithm of both fields and a Principal Components Analysis (PCA) is done on the logarithm of the MST of both fields. The fields are projected onto the PCA vectors and a small number of these PCA vector components are kept. Singular Value Decompositions of the cross correlation of the input parameters to the output logarithm of the MST of the fields, and of the cross correlation of the SVD vector components to the PCA vector components are done. This allows the identification of the PCA vectors vis-a-vis the input parameters. Finally, a Multi Layer Perceptron (MLP) neural network with ReLU activation and a simple three layer encoder/decoder architecture is trained on this dataset to predict the PCA vector components of the fields as a function of time. Details of the implosion, stagnation, and the disassembly are well captured. Examination of the PCA vectors and a permutation importance analysis of the MLP show definitive evidence of an inverse turbulent cascade into a dipole emergent behavior. The orientation of the dipole is set by the initial liner perturbation. The analysis is repeated with a version of the MST which includes phase, called Wavelet Phase Harmonics (WPH). While WPH do not give the physical insight of the MST, they can and are inverted to give field configurations as a function of time, including field-to-field correlations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗