SEARCH · Engineering Papers
Results for “Variational autoencoder”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Application of the variational autoencoder to detect the critical points of the anisotropic Ising model
We generalize the previous study on the application of variational autoencoders to the two-dimensional Ising model to a system with anisotropy. Due to the self-duality property of the system, the critical points can be located exactly for the entire range of anisotropic coupling. This presents an excellent test bed for the validity of using a variational autoencoder to characterize an anisotropic classical model. Furthermore, we reproduce the phase diagram for a wide range of anisotropic couplings and temperatures via a variational autoencoder without the explicit construction of an order parameter. Considering that the partition function of ($d$ + 1)-dimensional anisotropic models can be mapped to that of the $d$-dimensional quantum spin models, the present study provides numerical evidence that a variational autoencoder can be applied to analyze quantum systems via the quantum Monte Carlo method.
A dimensionality reduction algorithm for mapping tokamak operational regimes using a variational autoencoder (VAE) neural network
Abstract A variational autoencoder (VAE) is a type of unsupervised neural network which is able to learn meaningful data representations in a reduced dimensional space. We present an application of VAE in identifying the operational stability boundary of tokamak plasma discharges. This model was implemented using a dataset of over 3000 discharges from the high beta tokamak-extended pulse (HBT-EP) device. We found the VAE model to be capable of forming a continuous low-dimensional operational space map and identifying the operational boundaries using a specified warning time window. By projecting the operational parameters onto the same reduced space, this provides an intuitive way for the machine operator or an automated control system to perform disruption avoidance using a relevant control actuator as a discharge approaches a boundary. Pre-programmed GPU control experiments were conducted to demonstrate this control technique using HBT-EP’s saddle control coils as a horizontal position actuator, showing the ability to avoid the oncoming disruptive event and extend the duration of the discharge.
Robust Spectral Anomaly Detection in EELS Spectral Images via 3D Convolutional Variational Autoencoders
Abstract A 3D Convolutional Variational Autoencoder (3D‐CVAE) is introduced for automated anomaly detection in electron energy‐loss spectroscopy spectrum imaging (EELS‐SI) data. This approach leverages the full 3D structure of EELS‐SI data to detect subtle spectral anomalies while preserving both spatial and spectral correlations across the datacube. By employing cross‐entropy loss and training on bulk spectra, the model learns to reconstruct bulk features characteristic of the defect‐free material. In exploring methods for anomaly detection, both the 3D‐CVAE approach and principal component analysis (PCA) are evaluated, testing their performance using FeL‐edge ΔEpeak shifts designed to simulate material defects. These results show that 3D‐CVAE achieves superior anomaly detection and maintains consistent performance across various shift magnitudes. The method demonstrates clear bimodal separation between bulk and anomalous spectra, enabling reliable classification. Further analysis verifies that lower‐dimensional representations are robust to anomalies in the data. While performance advantages over PCA diminish with decreasing anomaly concentration, our method maintains high reconstruction quality even in challenging, noise‐dominated spectral regions. This approach provides a robust framework for unsupervised automated detection of spectral anomalies in EELS‐SI data, particularly valuable for analyzing complex material systems.
Machine learning of Kondo physics using variational autoencoders and symbolic regression
In this work, we employ variational autoencoders to extract physical insight from a dataset of one-particle Anderson impurity model spectral functions. Autoencoders are trained to find a low-dimensional, latent space representation that faithfully characterizes each element of the training set, as measured by a reconstruction error. Variational autoencoders, a probabilistic generalization of standard autoencoders, further condition the learned latent space to promote highly interpretable features. In our study, we find that the learned latent variables strongly correlate with well known, but nontrivial, parameters that characterize emergent behaviors in the Anderson impurity model. In particular, one latent variable correlates with particle-hole asymmetry, while another is in near one-to-one correspondence with the Kondo temperature, a dynamically generated low-energy scale in the impurity model. Using symbolic regression, we model this variable as a function of the known bare physical input parameters and "rediscover" the nonperturbative formula for the Kondo temperature. The machine learning pipeline we develop suggests a general purpose approach, which opens opportunities to discover new domain knowledge in other physical systems.
An explainable variational autoencoder model for three-dimensional acoustic emission source localization in hollow cylindrical structures
We introduce an explainable variational autoencoder for three-dimensional (3D) localization of acoustic emission sources in hollow cylindrical structures, with an unsupervised approach. This research capitalizes on multi-arrival waveforms generated by helical path propagation in cylindrical geometries to enable efficient two-receiver localization. By integrating the modal characteristics of Lamb modes under multi-path conditions, we demonstrate that two sets of time-of-arrival differences and peak amplitudes extracted from one receiver can serve as effective localization features. This initial approach identifies four potential source locations, highlighting the feasibility of two-receiver source localization using traditional feature extraction methods. However, direct extraction can be challenging when mode overlaps occur, complicating the localization process. To address this, our work proposes a novel waveform-based method. This method leverages the consistent dispersion characteristics within isotropic materials, where each unique combination of mode arrival times and peak amplitudes constructs a distinct waveform. This distinctiveness overcomes the ambiguities associated with mode overlaps, significantly enhancing the method’s precision and robustness. Our approach adopts a data-driven strategy for waveform-based localization using variational autoencoder (VAE). VAE discerns waveform patterns for localization, while also addressing data uncertainties. The VAE’s encoder and decoder networks capture the localization process and the source’s influence on waveform generation, respectively, guiding latent variables to segregate waveforms by source in the latent space. The design of the learning process focuses on specific localization characteristics to enhance result explainability. Localization predictions are generated by projecting test waveforms, not included in the training set, onto a trained latent space. The prediction is determined using a nearest-neighbor approach based on the closest latent representation of a source. Validation with pencil-lead-break tests on a metallic pipe confirmed our method’s effectiveness, achieving an averaged 3D localization accuracy of 0.84.
Improving Variational Autoencoders for New Physics Detection at the LHC With Normalizing Flows
We investigate how to improve new physics detection strategies exploiting variational autoencoders and normalizing flows for anomaly detection at the Large Hadron Collider. As a working example, we consider the DarkMachines challenge dataset. We show how different design choices (e.g., event representations, anomaly score definitions, network architectures) affect the result on specific benchmark new physics models. Once a baseline is established, we discuss how to improve the anomaly detection accuracy by exploiting normalizing flow layers in the latent space of the variational autoencoder.
Particle-based fast jet simulation at the LHC with variational autoencoders
Abstract We study how to use deep variational autoencoders (VAEs) for a fast simulation of jets of particles at the Large Hadron Collider. We represent jets as a list of constituents, characterized by their momenta. Starting from a simulation of the jet before detector effects, we train a deep VAE to return the corresponding list of constituents after detection. Doing so, we bypass both the time-consuming detector simulation and the collision reconstruction steps of a traditional processing chain, speeding up significantly the events generation workflow. Through model optimization and hyperparameter tuning, we achieve state-of-the-art precision on the jet four-momentum, while providing an accurate description of the constituents momenta, and an inference time comparable to that of a rule-based fast simulation.
A Variational Autoencoder Model Toward Molecular Structure Representation Learning of Fuels
Here, in this work, a Variational Autoencoder (VAE)-based data-driven modeling framework is developed with the overarching goal of enabling fuel design. The VAE model is trained on a large dataset with several chemical species to learn a compressed latent space molecular representation. Chemical structure in the form of Simplified Molecular Input Line Entry System (SMILES) string is fed as input, encoded into the VAE latent space, and decoded back to the SMILES string using Long Short-Term Memory (LSTM) networks. Complexities of the VAE training loss function are thoroughly examined by varying the weightage (beta (𝜷) parameter) of the latent space regularization term, thereby assessing the balance between reconstruction accuracy and validity, and focusing on both accurate molecular structure reconstruction and latent space consistency. Two different strategies for 𝜷 variation are evaluated: linear annealing and cyclic annealing. In addition, the impact of total correlation adjustment and hierarchical priors is also studied with regard to the balance between reconstruction fidelity and latent space regularization, and potential issues such as posterior collapse, over-regularization, and poor disentanglement of latent variables. Overall, the best performance of the model is achieved with hierarchical priors and incrementally increasing 𝜷 from 0 to a threshold value of 0.25 over 75 epochs. The generative VAE model can be readily coupled with Quantitative Structure–Property Relationship (QSPR) analysis to develop an integrated end-to-end framework for fuel-property prediction and molecular design of novel promising fuels.
Learning and Predicting Photonic Responses of Plasmonic Nanoparticle Assemblies via Dual Variational Autoencoders
In this work, the application of machine learning is demonstrated for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. It is shown that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that hyperspectral darkfield microscopy can be used to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.
Evaluating the Efficacy of Conditional Variational Autoencoders in Generating Synthetic Single Nuclei RNA-Seq Data for Space Biology Research
Astronauts are subject to unique stressors during spaceflight, leading to changes in their cellular function. However, neither astronauts nor model organisms respond the same to spaceflight, and research implicates a contribution of omics components in differential responses. Understanding how gene expression affects astronaut health is critical for the success of long-term space missions, prompting interest in developing personalized predictive models leveraging artificial intelligence (AI) and machine learning (ML) techniques. Developing such models requires extensive data, which is challenging to obtain and share. This study explores the use of conditional variational autoencoders (CVAEs) to synthetically generate single-nuclei RNA-seq (snRNA-seq) data. CVAEs build on standard variational autoencoders (VAEs) by conditioning data generation on covariates like sample identity and mission parameters, enhancing the relevance of generated data for specific contexts. For our work, we built two CVAEs with varying degrees of sparsity to optimize both interpretability and generative power. We train and validate models on existing snRNA-seq data collected from the brain tissue of mice subjected to spaceflight conditions and their ground control counterparts. We evaluate model performance using statistical tests and visualizations to compare synthetic data to real data. We aim to demonstrate that these prototype CVAE architectures could be used in future space biology work and that this is a method worth further exploring.
Predicting critical heat flux with uncertainty quantification and domain generalization using conditional variational autoencoders and deep neural networks
Deep generative models (DGMs) can generate synthetic data samples that closely resemble the original dataset, addressing data scarcity. In this work, we developed a conditional variational autoencoder (CVAE) to augment critical heat flux (CHF) data used for the 2006 Groeneveld lookup table. To compare with traditional methods, a fine-tuned deep neural network (DNN) regression model was evaluated on the same dataset. Both models achieved small mean absolute relative errors, with the CVAE showing more favorable results. Uncertainty quantification (UQ) was performed using repeated CVAE sampling and DNN ensembling. The DNN ensemble improved performance over the baseline, while the CVAE maintained consistent results with less variability and higher confidence. Both models achieved small errors inside and outside the training domain, with slightly larger errors outside. Altogether, the CVAE performed better than the DNN in predicting CHF and exhibited better uncertainty behavior.
Deep learning on the 2-dimensional Ising model to extract the crossover region with a variational autoencoder
The 2-dimensional Ising model on a square lattice is investigated with a variational autoencoder in the non-vanishing field case for the purpose of extracting the crossover region between the ferromagnetic and paramagnetic phases. The encoded latent variable space is found to provide suitable metrics for tracking the order and disorder in the Ising configurations that extends to the extraction of a crossover region in a way that is consistent with expectations. The extracted results achieve an exceptional prediction for the critical point as well as agreement with previously published results on the configurational magnetizations of the model. The performance of this method provides encouragement for the use of machine learning to extract meaningful structural information from complex physical systems where little a priori data is available.
Loss of Control Detection for Commercial Transport Aircraft Using Conditional Variational Autoencoders
This work describes a detector for the loss of control condition of a commercial transport in flight. The detector has a belief state defined by the latent variable stochastic modeling of a conditional variational autoencoder (CVAE) constructed with bidirectional recurrent layers. In 2000, the Boeing Company and the NASA Langley Research Center jointly developed a quantitative set of metrics for defining loss-of-control (LOC) for a commercial transport. We use the thresholds for these quantitative metrics to define a condition vector for training the CVAE. First, we demonstrate through experimentation that reconstruction probability is an accurate indicator that the vehicle has shifted to an LOC state. Second, we introduce a technique for inferring that the vehicle is approaching a flight state change by measuring a shift in the sampling distributions of the CVAE latent space. The sampling distributions for flight observations that are approaching envelope limits are localized to external areas of the latent space. We provide an analysis of its applicability to flight data from NASA’s dynamically-scaled generic transport model (GTM) aircraft.
Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systems
We develop data-driven methods for incorporating physical information for priors to learn parsimonious representations of nonlinear systems arising from parameterized PDEs and mechanics. Our approach is based on Variational Autoencoders (VAEs) for learning nonlinear state space models from observations. We develop ways to incorporate geometric and topological priors through general manifold latent space representations. We investigate the performance of our methods for learning low dimensional representations for the nonlinear Burgers equation and constrained mechanical systems.
Optimizing training trajectories in variational autoencoders via latent Bayesian optimization approach *
Unsupervised and semi-supervised ML methods such as variational autoencoders (VAE) have become widely adopted across multiple areas of physics, chemistry, and materials sciences due to their capability in disentangling representations and ability to find latent manifolds for classification and/or regression of complex experimental data. Like other ML problems, VAEs require hyperparameter tuning, e.g. balancing the Kullback–Leibler and reconstruction terms. However, the training process and resulting manifold topology and connectivity depend not only on hyperparameters, but also their evolution during training. Because of the inefficiency of exhaustive search in a high-dimensional hyperparameter space for the expensive-to-train models, here we have explored a latent Bayesian optimization (zBO) approach for the hyperparameter trajectory optimization for the unsupervised and semi-supervised ML and demonstrated for joint-VAE with rotational invariances. We have demonstrated an application of this method for finding joint discrete and continuous rotationally invariant representations for modified national institute of standards and technology database (MNIST) and experimental data of a plasmonic nanoparticles material system. The performance of the proposed approach has been discussed extensively, where it allows for any high dimensional hyperparameter trajectory optimization of other ML models.
Exploring order parameters and dynamic processes in disordered systems via variational autoencoders
The local order parameter in a disordered system is explored with convolutional neural networks and variational autoencoder.
Inverse Analysis with Variational Autoencoders: A Comparison of Shallow and Deep Networks
Inverse problems are applied to determine unknown properties by matching observational data with a physical model that often takes many parameters as input. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented involving regularization with a technique known as a variational autoencoder (VAE), which is trained to map a high-dimensional parameter space with a complex structure to a low-dimensional latent space with a simple structure. We apply this approach to unconditioned realizations of the parameters (heterogeneous hydraulic fields) for a hydrogeological inverse problem. Two types of hydraulic conductivity fields are used to evaluate the characterization for the different levels of heterogeneity complexity of the physical inputs. This approach keeps the computational cost of generating the training data low. The reason is unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any groundwater flow model forward runs. In addition, this approach applies regularization on a low-dimensional latent space from the VAE and increases optimization efficiency through automatic differentiation. Furthermore, two different neural network (NN) structures are tested for their utility in using the VAE for inverse analysis. The performance of a deep, convolutional neural network strongly depends on the dimensionality of the latent space, which requires tuning. In contrast, a shallow, dense neural network provides consistently accurate characterization without tuning. Furthermore, our approach evaluates the advantages of the shallow, dense neural network over the deep, convolutional one and enables future application to a wide range of inverse problems.