Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Latent space”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A deep learning-based direct forecasting of CO 2 plume migration

Accurate and timely forecasts of CO 2 plume evolution in geological reservoirs are crucial for CO 2 migration detection, leakage risk assessment, and operation decision support. Conventional forecasting usually adopts a two-step strategy, first calibrating reservoir model parameters against observations using iterative inverse modeling (or history matching) and then applying the calibrated model for predictions. This method impedes real-time forecasts due to the heavy computational demand in inverse modeling and may suffer from poor prediction accuracy because of the limited observation data. In this work, we propose a deep learning-based latent space mapping framework to forecast CO 2 plume migration directly by avoiding the inverse modeling. We first use the convolutional autoencoder to map the high-dimensional complex plume extents onto low-dimensional latent space. Next, we use neural networks to learn the relationship between the observation variables and the prediction latent variables. And then for given observation data, we infer the prediction values directly. This one-step direct forecasting is computationally efficient which requires a few number of parallelizable reservoir simulations and it can provide accurate predictions with limited observations by learning the observation-prediction relationship in the reduced dimension. Therefore, our proposed method enables an in-time forecast of dynamic CO 2 plume distributions. In this work, we demonstrate the effectiveness and accuracy of our method in predicting the CO 2 plume migration using four metrics such as plume area, centroid movement distance, and plume spreading in the primary and secondary directions. And the spatio-temporal evolution patterns of plume migration under diverse geological complexities are also accurately quantified.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Denoising diffusion algorithm for inverse design of microstructures with fine-tuned nonlinear material properties

Here we introduce a denoising diffusion algorithm to discover microstructures with nonlinear fine-tuned properties. Denoising diffusion probabilistic models are generative models that use diffusion-based dynamics to gradually denoise images and generate realistic synthetic samples. By learning the reverse of a Markov diffusion process, we design an artificial intelligence to efficiently manipulate the topology of microstructures to generate a massive number of prototypes that exhibit constitutive responses sufficiently close to designated nonlinear constitutive behaviors. To identify the subset of microcstructures with sufficiently precise fine-tuned properties, a convolutional neural network surrogate is trained to replace high-fidelity finite element simulations to filter out prototypes outside the admissible range. Results of this study indicate that the denoising diffusion process is capable of creating microstructures of fine-tuned nonlinear material properties within the latent space of the training data. More importantly, this denoising diffusion algorithm can be easily extended to incorporate additional topological and geometric modifications by introducing high-dimensional structures embedded in the latent space. Numerical experiments are conducted on the open-source mechanical MNIST data set (Lejeune, 2020). Consequently, this algorithm is not only capable of performing inverse design of nonlinear effective media, but also learns the nonlinear structure–property map to quantitatively understand the multiscale interplay among the geometry, topology, and their effective macroscopic properties.

42 ENGINEERING↗
DOE OSTI2024

Deep Koopman Neural Network for Analyzing High-Energy-Density Simulations of Electrical Wire Explosions

Megaampere-scale electrical wire experiments (EWEs) provide a platform for studying magnetohydrodynamic (MHD) instability growth in magneto-inertial fusion (MIF) devices. Even when nonlinear simulations of these experiments can digitally reproduce much of the experimentally observed instability growth, interpreting the results and understanding mode growth and evolution can be non-trivial. As a first step toward providing better interpretation of these simulation features, this work investigates the use of a deep neural network that uses Koopman operator theory to analyze the dynamics of pulsed-power-driven explosions of EWEs. This deep neural network is trained on 1-D resistive MHD simulations of EWEs. This neural network learns to transform the nonlinear data into a lower-dimensional representation where the time dynamics are linear. Layers of this neural network are shown to learn features of the simulations, including the locations of shock waves and different physical regimes of the simulation. Using the learned features, the network can compress a time state of the simulation consisting of 5120 data point into a 36-parameter lower-dimensional latent space embedding. Furthermore, these embeddings are shown to be clustered in the latent space by initial radius and time state.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Forward sensitivity analysis and mode dependent control for closure modeling of Galerkin systems

Model reduction by projection-based approaches is often associated with losing some of the important features that contribute towards the dynamics of the retained scales. As a result, a mismatch occurs between the predicted trajectories of the original system and the truncated one. We put forth a framework to apply a continuous time control signal in the latent space of the reduced order model (ROM) to account for the effect of truncation. We set the control input using parameterized models by following energy transfer principles. Our methodology relies on observing the system behavior in the physical space and using the projection operator to restrict the feedback signal into the latent space. Then, we leverage the forward sensitivity method (FSM) to derive relationships between the feedback and the desired mode-dependent control. We test the performance of the proposed approach using two test cases, corresponding to viscous Burgers and vortex merger problems at high Reynolds number. Results show that the ROM trajectory with the applied FSM control closely matches its target values in both the data-dense and data-sparse regimes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5

Abstract. There is growing interest in data-driven weather prediction (DDWP), e.g., using convolutional neural networks such as U-NET that are trained on data from models or reanalysis. Here, we propose three components, inspired by physics, to integrate with commonly used DDWP models in order to improve their forecast accuracy. These components are (1) a deep spatial transformer added to the latent space of U-NET to capture rotation and scaling transformation in the latent space for spatiotemporal data, (2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and (3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit and feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the spatial-transformer-based U-NET (U-STN) clearly outperforms the U-NET, e.g., improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. This DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promise; for example, it reduces the average error by factors of 2–3. These results show the benefits and feasibility of these three components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the three components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast and assimilation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Latent Twins

Over the past decade, scientific machine learning has transformed the development of mathematical and computational frameworks for analyzing, modeling, and predicting complex systems. From inverse problems to numerical partial differential equations (PDEs), dynamical systems, and model reduction, these advances have pushed the boundaries of what can be simulated. Yet they have often progressed in parallel, with representation learning and algorithmic solution methods evolving largely as separate pipelines. With Latent Twins, we propose a unifying mathematical framework that creates a hidden surrogate in latent space for the underlying equations. Whereas digital twins mirror physical systems in the digital world, Latent Twins mirror mathematical systems in a learned latent space governed by operators. Through this lens, classical modeling, inversion, model reduction, and operator approximation all emerge as special cases of a single principle. We establish the fundamental approximation properties of Latent Twins for both ordinary differential equations (ODEs) and PDEs and demonstrate the framework across three representative settings: (i) canonical ODEs, capturing diverse dynamical regimes; (ii) a PDE benchmark using the shallow-water equations, contrasting Latent Twin simulations with deep operator network and forecasts with a four-dimensional variational method baseline; and (iii) a challenging real-data geopotential reanalysis dataset, reconstructing and forecasting from sparse, noisy observations. Latent Twins provide a compact, interpretable surrogate for solution operators that evaluate across arbitrary time gaps in a single-shot, while remaining compatible with scientific pipelines such as assimilation, control, and uncertainty quantification. Looking forward, this framework offers scalable, theory-grounded surrogates that bridge data-driven representation learning and classical scientific modeling across disciplines.

Latent Twins↗
DOE OSTI2026

Identifying Anomalous DESI Galaxy Spectra with a Variational Autoencoder

The tens of millions of spectra being captured by the Dark Energy Spectroscopic Instrument (DESI) provide tremendous discovery potential. In this work we show how Machine Learning, in particular Variational Autoencoders (VAE), can detect anomalies in a sample of approximately 200,000 DESI spectra comprising galaxies, quasars and stars. We demonstrate that the VAE can compress the dimensionality of a spectrum by a factor of 100, while still retaining enough information to accurately reconstruct spectral features. We then detect anomalous spectra as those with high reconstruction error and those which are isolated in the VAE latent representation. The anomalies identified fall into two categories: spectra with artefacts and spectra with unique physical features. Awareness of the former can help to improve the DESI spectroscopic pipeline; whilst the latter can lead to the identification of new and unusual objects. To further curate the list of outliers, we use the Astronomaly package which employs Active Learning to provide personalised outlier recommendations for visual inspection. In this work we also explore the VAE latent space, finding that different object classes and subclasses are separated despite being unlabelled. We demonstrate the interpretability of this latent space by identifying tracks within it that correspond to various spectral characteristics. For example, we find tracks that correspond to increasing star formation and increase in broad emission lines along the Balmer series. In upcoming work we hope to apply the methods presented here to search for both systematics and astrophysically interesting objects in much larger datasets of DESI spectra.

Nicolaou, C. [University Coll. London] (ORCID:0000↗
NASA NTRSDate not supplied

Loss of Control Detection for Commercial Transport Aircraft Using Conditional Variational Autoencoders

This work describes a detector for the loss of control condition of a commercial transport in flight. The detector has a belief state defined by the latent variable stochastic modeling of a conditional variational autoencoder (CVAE) constructed with bidirectional recurrent layers. In 2000, the Boeing Company and the NASA Langley Research Center jointly developed a quantitative set of metrics for defining loss-of-control (LOC) for a commercial transport. We use the thresholds for these quantitative metrics to define a condition vector for training the CVAE. First, we demonstrate through experimentation that reconstruction probability is an accurate indicator that the vehicle has shifted to an LOC state. Second, we introduce a technique for inferring that the vehicle is approaching a flight state change by measuring a shift in the sampling distributions of the CVAE latent space. The sampling distributions for flight observations that are approaching envelope limits are localized to external areas of the latent space. We provide an analysis of its applicability to flight data from NASA’s dynamically-scaled generic transport model (GTM) aircraft.

Loss of control↗
DOE OSTI2021

Machine learning of Kondo physics using variational autoencoders and symbolic regression

In this work, we employ variational autoencoders to extract physical insight from a dataset of one-particle Anderson impurity model spectral functions. Autoencoders are trained to find a low-dimensional, latent space representation that faithfully characterizes each element of the training set, as measured by a reconstruction error. Variational autoencoders, a probabilistic generalization of standard autoencoders, further condition the learned latent space to promote highly interpretable features. In our study, we find that the learned latent variables strongly correlate with well known, but nontrivial, parameters that characterize emergent behaviors in the Anderson impurity model. In particular, one latent variable correlates with particle-hole asymmetry, while another is in near one-to-one correspondence with the Kondo temperature, a dynamically generated low-energy scale in the impurity model. Using symbolic regression, we model this variable as a function of the known bare physical input parameters and "rediscover" the nonperturbative formula for the Kondo temperature. The machine learning pipeline we develop suggests a general purpose approach, which opens opportunities to discover new domain knowledge in other physical systems.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

Physics-constrained superresolution diffusion for six-dimensional phase space diagnostics

Adaptive physics-constrained superresolution diffusion is developed for noninvasive virtual diagnostics of the six-dimensional (6D) phase space density of charged particle beams. An adaptive variational autoencoder embeds initial beam condition images and scalar measurements to a low-dimensional latent space from which a 32 6 pixel 6D tensor representation of the beam's 6D phase space density is generated. Projecting from a 6D tensor generates physically consistent two-dimensional projections. Physics-guided superresolution diffusion transforms low-resolution images of the 6D density to high resolution 256 × 256 pixel images. Unsupervised adaptive latent space tuning enables tracking of time-varying beams without knowledge of time-varying initial conditions. The method is demonstrated with experimental data and multiparticle simulations at the HiRES UED. The general approach is applicable to a wide range of complex dynamic systems evolving in high-dimensional phase space. The method is shown to be robust to distribution shift without retraining. Published by the American Physical Society 2025

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

A comparison of neural network architectures for data-driven reduced-order modeling

The popularity of deep convolutional autoencoders (CAEs) has engendered new and effective reduced-order models (ROMs) for the simulation of large-scale dynamical systems. Despite this, it is still unknown whether deep CAEs provide superior performance over established linear techniques or other network-based methods in all modeling scenarios. To elucidate this, the effect of autoencoder architecture on its associated ROM is studied through the comparison of deep CAEs against two alternatives: a simple fully connected autoencoder, and a novel graph convolutional autoencoder. Through benchmark experiments, it is shown that the superior autoencoder architecture for a given ROM application is highly dependent on the size of the latent space and the structure of the snapshot data, with the proposed architecture demonstrating benefits on data with irregular connectivity when the latent space is sufficiently large.

42 ENGINEERING↗
DOE OSTI2024

An explainable variational autoencoder model for three-dimensional acoustic emission source localization in hollow cylindrical structures

We introduce an explainable variational autoencoder for three-dimensional (3D) localization of acoustic emission sources in hollow cylindrical structures, with an unsupervised approach. This research capitalizes on multi-arrival waveforms generated by helical path propagation in cylindrical geometries to enable efficient two-receiver localization. By integrating the modal characteristics of Lamb modes under multi-path conditions, we demonstrate that two sets of time-of-arrival differences and peak amplitudes extracted from one receiver can serve as effective localization features. This initial approach identifies four potential source locations, highlighting the feasibility of two-receiver source localization using traditional feature extraction methods. However, direct extraction can be challenging when mode overlaps occur, complicating the localization process. To address this, our work proposes a novel waveform-based method. This method leverages the consistent dispersion characteristics within isotropic materials, where each unique combination of mode arrival times and peak amplitudes constructs a distinct waveform. This distinctiveness overcomes the ambiguities associated with mode overlaps, significantly enhancing the method’s precision and robustness. Our approach adopts a data-driven strategy for waveform-based localization using variational autoencoder (VAE). VAE discerns waveform patterns for localization, while also addressing data uncertainties. The VAE’s encoder and decoder networks capture the localization process and the source’s influence on waveform generation, respectively, guiding latent variables to segregate waveforms by source in the latent space. The design of the learning process focuses on specific localization characteristics to enhance result explainability. Localization predictions are generated by projecting test waveforms, not included in the training set, onto a trained latent space. The prediction is determined using a nearest-neighbor approach based on the closest latent representation of a source. Validation with pencil-lead-break tests on a metallic pipe confirmed our method’s effectiveness, achieving an averaged 3D localization accuracy of 0.84.

Lee, Guan-Wei↗
DOE OSTI2024

Selecting Appropriate Model Complexity: An Example of Tracer Inversion for Thermal Prediction in Enhanced Geothermal Systems

Abstract A major challenge in the inversion of subsurface parameters is the ill‐posedness issue caused by the inherent subsurface complexities and the generally spatially sparse data. Appropriate simplifications of inversion models are thus necessary to make the inversion process tractable and meanwhile preserve the predictive ability of the inversion results. In this study, we investigate the effect of model complexity on fracture aperture inversion and thermal performance prediction in a field‐scale EGS model. Principal component analysis was used to map the aperture field to a low‐dimensional latent space. The complexity of the inversion model was quantitatively represented by the percentage of total variance in the original aperture fields preserved by the latent space. Tracer, pressure and flow rate data were used to invert for fracture aperture through an ensemble‐based inversion method, and the inferred aperture field was used to predict thermal performance. With an over‐simplified aperture model, ensemble collapse occurred. The inverted aperture models failed to resolve necessary flow and transport features, leading to a biased thermal performance prediction. A complex aperture model involved excessive features and was prone to overinterpreting the inversion data. Both the tracer/pressure/flow rate data reproduction and thermal prediction showed significant uncertainties, making it difficult to properly estimate long‐term thermal performance. Fortunately, our results indicate that there exists an appropriate model complexity which can simultaneously match inversion data and predict thermal performance with an acceptable uncertainty. The quality of the fit of tracer data appears to be a useful indicator of such an appropriate model complexity.

15 GEOTHERMAL ENERGY↗
DOE OSTI2025

Understanding latent timescales in neural ordinary differential equation models of advection-dominated dynamical systems

The neural ordinary differential equation (ODE) framework has shown considerable promise in recent years in developing highly accelerated surrogate models for complex physical systems characterized by partial differential equations (PDEs). For PDE-based systems, state-of-the-art neural ODE strategies leverage a two-step procedure to achieve this acceleration: a nonlinear dimensionality reduction step provided by an autoencoder, and a time integration step provided by a neural-network based model for the resultant latent space dynamics (the neural ODE). This work explores the applicability of such autoencoder-based neural ODE strategies for PDEs in which advection terms play a critical role. More specifically, alongside predictive demonstrations, physical insight into the sources of model acceleration (i.e., how the neural ODE achieves its acceleration) is the scope of the current study. Such investigations are performed by quantifying the effects of both autoencoder and neural ODE components on latent system time-scales using eigenvalue analysis of dynamical system Jacobians. To this end, the sensitivity of various critical training parameters – de-coupled versus end-to-end training, latent space dimensionality, and the role of training trajectory length, for example – to both model accuracy and the discovered latent system timescales is quantified. Furthermore, this work specifically uncovers the key role played by the training trajectory length (the number of rollout steps in the loss function during training) on the latent system timescales: larger trajectory lengths correlate with an increase in limiting neural ODE time-scales, and optimal neural ODEs are found to recover the largest time-scales of the full-order (ground-truth) system. Demonstrations are performed across fundamentally different unsteady fluid dynamics configurations influenced by advection: (1) the Kuramoto–Sivashinsky equations (2) Hydrogen-Air channel detonations (the compressible reacting Navier–Stokes equations with detailed chemistry), and (3) 2D Atmospheric flow.

Advection-dominated dynamical systems↗
DOE OSTI2022

Ig-VAE: Generative modeling of protein structure by direct 3D coordinate generation

While deep learning models have seen increasing applications in protein science, few have been implemented for protein backbone generation—an important task in structure-based problems such as active site and interface design. We present a new approach to building class-specific backbones, using a variational auto-encoder to directly generate the 3D coordinates of immunoglobulins. Our model is torsion- and distance-aware, learns a high-resolution embedding of the dataset, and generates novel, high-quality structures compatible with existing design tools. We show that the Ig-VAE can be used with Rosetta to create a computational model of a SARS-CoV2-RBD binder via latent space sampling. We further demonstrate that the model’s generative prior is a powerful tool for guiding computational protein design, motivating a new paradigm under which backbone design is solved as constrained optimization problem in the latent space of a generative model.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Variational autoencoders for at-source data reduction and anomaly detection in high energy particle detectors

Detectors in next-generation high-energy physics experiments face several daunting requirements, such as high data rates, damaging radiation exposure, and stringent constraints on power, space, and latency. To address these challenges, machine learning in readout electronics can be leveraged for smart detector designs, enabling intelligent inference and data reduction at-source. Variational autoencoders (VAEs) offer a variety of benefits for front-end readout; an on-sensor encoder can perform efficient lossy data compression while simultaneously providing a latent space representation that can be used for anomaly detection. Results are presented from low-latency and resource-efficient VAEs for front-end data processing in a futuristic silicon pixel detector. Encoder-based data compression is found to preserve good performance of off-detector analysis while significantly reducing the off-detector data rate as compared to a similarly sized data filtering approach. Furthermore, the latent space information is found to be a useful discriminator in the context of real-time sensor defect monitoring. Together, these results highlight the multifaceted utility of autoencoder-based front-end readout schemes and motivate their consideration in future detector designs.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Unsupervised physics-informed disentanglement of multimodal data

Here, we introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal datasets. Individual modalities are embedded into a shared latent space and fused through a product-of-experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture-of-experts decoder that imposes inductive biases from prior scientific knowledge and thereby imparts structured disentanglement of the latent space. This approach enables cross-modal inference and the discovery of features in high-dimensional heterogeneous datasets. Consequently, this approach provides a means to discover fingerprints in multimodal scientific datasets and to avoid traditional bottlenecks related to high-fidelity measurement and characterization of scientific datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Lorentz group equivariant autoencoders

Abstract There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $$\textrm{SO}^+(3,1)$$ SO + ( 3 , 1 ) , with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗