Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “autoencoders”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Elucidating proximity magnetism through polarized neutron reflectometry and machine learning

Polarized neutron reflectometry is a powerful technique to interrogate the structures of multilayered magnetic materials with depth sensitivity and nanometer resolution. However, reflectometry profiles often inhabit a complicated objective function landscape using traditional fitting methods, posing a significant challenge for parameter retrieval. In this work, we develop a data-driven framework to recover the sample parameters from polarized neutron reflectometry data with minimal user intervention. We train a variational autoencoder to map reflectometry profiles with moderate experimental noise to an interpretable, low-dimensional space from which sample parameters can be extracted with high resolution. We apply our method to recover the scattering length density profiles of the topological insulator–ferromagnetic insulator heterostructure Bi2Se3/EuS exhibiting proximity magnetism in good agreement with the results of conventional fitting. We further analyze a more challenging reflectometry profile of the topological insulator–antiferromagnet heterostructure (Bi,Sb)2Te3/Cr2O3 and identify possible interfacial proximity magnetism in this material. We anticipate that the framework developed here can be applied to resolve hidden interfacial phenomena in a broad range of layered systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Unsupervised learning of ferroic variants from atomically resolved STEM images

An approach for the analysis of atomically resolved scanning transmission electron microscopy data with multiple ferroic variants in the presence of imaging non-idealities and chemical variabilities based on a rotationally invariant variational autoencoder (rVAE) is presented. We show that an optimal local descriptor for the analysis is a sub-image centered at specific atomic units, since materials and microscope distortions preclude the use of an ideal lattice as a reference point. The applicability of unsupervised clustering and dimensionality reduction methods is explored and is shown to produce clusters dominated by chemical and microscope effects, with a large number of classes required to establish the presence of rotational variants. Comparatively, the rVAE allows extraction of the angle corresponding to the orientation of ferroic variants explicitly, enabling straightforward identification of the ferroic variants as regions with constant or smoothly changing latent variables and sharp orientational changes. This approach allows further exploration of the chemical variability by separating the rotational degrees of freedom via rVAE and searching for remaining variability in the system. The code used in this article is available at https://github.com/saimani5/ferroelectric_domains_rVAE.

30 DIRECT ENERGY CONVERSION↗
DOE OSTI2022

Physics-constrained deep learning of nonlinear normal modes of spatiotemporal fluid flow dynamics

In this study, we present a physics-constrained deep learning method to discover and visualize from data the invariant nonlinear normal modes (NNMs) which contain the spatiotemporal dynamics of the fluid flow potentially containing strong nonlinearity. Specifically, we develop a NNM-physics-constrained convolutional autoencoder (NNM-CNN-AE) integrated with a multi-temporal-step dynamics prediction block to learn the nonlinear modal transformation, the NNMs containing the spatiotemporal dynamics of the flow, and reduced-order reconstruction and long-time future-state prediction of the flow fields, simultaneously. In test cases, we apply the developed method to analyze different flow regimes past a cylinder, including laminar flows with low Reynolds number in transient and steady states (RD = 100) and high Reynolds number flow (RD = 1000), respectively. The results indicate that the identified NNMs are able to reveal the nonlinear spatiotemporal dynamics of these flows, and the NNMs-based reduced-order modeling consistently achieves better accuracy with orders of magnitudes smaller errors in construction and prediction of the nonlinear velocity and vorticity fields, compared to the linear proper orthogonal decomposition (POD) method and the Koopman-constrained-CNN-AE using the same number or dimension of modes. We perform an analysis of the modal energy distribution of NNMs and find that compared to POD modes, the few fundamental NNMs capture a very high level of total energy of the flow, which is advantageous for reduced-order modeling and representation of the complex flows. Finally, we discuss the potentials and limitations of the presented method.

Mechanics↗
DOE OSTI2023

Unsupervised machine learning discovery of structural units and transformation pathways from imaging data

We show that unsupervised machine learning can be used to learn chemical transformation pathways from observational Scanning Transmission Electron Microscopy (STEM) data. To enable this analysis, we assumed the existence of atoms, a discreteness of atomic classes, and the presence of an explicit relationship between the observed STEM contrast and the presence of atomic units. With only these postulates, we developed a machine learning method leveraging a rotationally invariant variational autoencoder (VAE) that can identify the existing molecular fragments observed within a material. The approach encodes the information contained in STEM image sequences using a small number of latent variables, allowing the exploration of chemical transformation pathways by tracing the evolution of atoms in the latent space of the system. The results suggest that atomically resolved STEM data can be used to derive fundamental physical and chemical mechanisms involved, by providing encodings of the observed structures that act as bottom-up equivalents of structural order parameters. The approach also demonstrates the potential of variational (i.e., Bayesian) methods in the physical sciences and will stimulate the development of more sophisticated ways to encode physical constraints in the encoder–decoder architectures and generative physical laws and causal relationships in the latent space of VAEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Discovering invariant spatial features in electron energy loss spectroscopy images on the mesoscopic and atomic levels

Over the last two decades, Electron Energy Loss Spectroscopy (EELS) imaging with a scanning transmission electron microscope has emerged as a technique of choice for visualizing complex chemical, electronic, plasmonic, and phononic phenomena in complex materials and structures. The availability of the EELS data necessitates the development of methods to analyze multidimensional data sets with complex spatial and energy structures. Traditionally, the analysis of these data sets has been based on analysis of individual spectra, one at a time, whereas the spatial structure and correlations between individual spatial pixels containing the relevant information of the physics of underpinning processes have generally been ignored and analyzed only via the visualization as 2D maps. Here, we develop a machine learning-based approach and workflows for the analysis of spatial structures in 3D EELS data sets using a combination of dimensionality reduction and multichannel rotationally invariant variational autoencoders. This approach is illustrated for the analysis of both the plasmonic phenomena in a system of nanowires and in the core excitations in functional oxides using low loss and core-loss EELS, respectively. The code developed in this manuscript is open sourced and freely available and provided as a Jupyter notebook for the interested reader.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Deep learning-based predictive models for laser direct drive at the Omega Laser Facility

The rich and complex physics of inertial confinement fusion provides a unique and challenging space for high-fidelity first-principles modeling. Consequently, simulation codes that are used to design experiments are computationally expensive and lack the predictive capability required for extensive parameter exploration in search of a high-performing design for laser direct drive. In this article, we present two deep-learning-based predictive models intended to address these difficulties. The first model (TL DNN) acts as a fast emulator of simulations as well as experiments at the Omega Laser Facility. This model is trained on a simulation database and subsequently calibrated on experimental data using transfer learning. To facilitate the development of this model, an autoencoder is developed to reduce the dimensionality of the input space by compressing the laser pulse input. The model predicts key experimental scalar observables of Omega experiments with high accuracy and minimal computational cost. This deep neural net enables rapid exploration of a high-dimensional input parameter space for an optimal implosion design. The second model (DNN SM+) aims to extend the statistical modeling work of Lees et al. [Phys. Rev. Lett. 127, 105001 (2021)], by increasing the complexity of the model space and allowing for coupling between degradation terms. Since the model capacity of DNN SM+ is higher than the model of Lees et al., DNN SM+ can potentially provide an improvement in predictive capability, and we use this model to provide insight into complicated degradation dependencies.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Toward ultra-efficient high-fidelity predictions of wind turbine wakes: Augmenting the accuracy of engineering models with machine learning

This study proposes a novel machine learning (ML) methodology for the efficient and cost-effective prediction of high-fidelity three-dimensional velocity fields in the wake of utility-scale turbines. The model consists of an autoencoder convolutional neural network with U-Net skipped connections, fine-tuned using high-fidelity data from large-eddy simulations (LES). The trained model takes the low-fidelity velocity field cost-effectively generated from the analytical engineering wake model as input and produces the high-fidelity velocity fields. The accuracy of the proposed ML model is demonstrated in a utility-scale wind farm for which datasets of wake flow fields were previously generated using LES under various wind speeds, wind directions, and yaw angles. Comparing the ML model results with those of LES, the ML model was shown to reduce the error in the prediction from 20% obtained from the Gauss Curl hybrid (GCH) model to less than 5%. In addition, the ML model captured the non-symmetric wake deflection observed for opposing yaw angles for wake steering cases, demonstrating a greater accuracy than the GCH model. The computational cost of the ML model is on par with that of the analytical wake model while generating numerical outcomes nearly as accurate as those of the high-fidelity LES.

Mechanics↗
DOE OSTI2025

Invariant discovery of features across multiple length scales: Applications in microscopy and autonomous materials characterization

Physical imaging is a foundational characterization method in areas from condensed matter physics and chemistry to astronomy and spans length scales from atomic to universe. Images encapsulate crucial data regarding atomic bonding, materials microstructures, and dynamic phenomena such as microstructural evolution and turbulence, among other phenomena. The challenge lies in effectively extracting and interpreting this information. Variational Autoencoders (VAEs) have emerged as powerful tools for identifying the underlying factors of variation in image data, providing a systematic approach to distilling meaningful patterns from complex data sets. However, a significant hurdle in their application is the definition and selection of appropriate descriptors reflecting local structures. Here, we introduce the scale-invariant VAE approach (SI-VAE) based on the progressive training of the VAE with the descriptors sampled at different length scales. The SI-VAE allows the discovery of the length scale-dependent factors of variation in the system. Here, we illustrate this approach using the ferroelectric domain images and generalize it to the movies of the electron-beam induced phenomena in graphene and topography evolution across combinatorial libraries. This approach can further be used to initialize the decision making in automated experiments including structure–property discovery and can be applied across a broad range of imaging methods. This approach is universal and can be applied to any spatially resolved data including both experimental imaging studies and simulations, and can be particularly useful for exploration of phenomena such as turbulence and scale-invariant transformation fronts.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Deciphering the small-angle scattering of polydisperse hard spheres using deep learning

We introduce a deep learning approach for analyzing the scattering function of the polydisperse hard sphere system. We use a variational autoencoder-based neural network to learn the bidirectional mapping between the scattering function and the system parameters, including the volume fraction and polydispersity. Such that the trained model serves both as a generator that produces a scattering function from the system parameters and an inferrer that extracts system parameters from the scattering function. We first generate a scattering dataset by carrying out molecular dynamics simulations of the polydisperse hard spheres modeled by the truncated-shifted Lennard-Jones model, then analyze the scattering function dataset using singular value decomposition to confirm the feasibility of dimensional compression. Then, we split the dataset into training and testing sets and train our neural network on the training set only. Our generator model produces a scattering function with significantly higher accuracy compared to the traditional Percus–Yevick approximation and β correction, and the inferrer model can extract the volume fraction and polydispersity with much higher accuracy than traditional model functions.

Ding, Lijie [ORNL] (ORCID:0000000227454606)↗
DOE OSTI2026

An interactive machine learning platform for analyzing multi-particle coincidence data from cold target recoil ion momentum spectroscopy

We present SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training), a comprehensive software platform for analyzing tabulated high-dimensional multi-particle coincidence data from Cold Target Recoil Ion Momentum Spectroscopy (COLTRIMS) experiments. The software addresses critical challenges in modern momentum spectroscopy by integrating advanced machine learning techniques with physics-informed analysis in an interactive web-based environment. SCULPT implements uniform manifold approximation and projection for non-linear dimensionality reduction to reveal correlations in high-dimensional data. We also discuss potential extensions to deep autoencoders for feature learning and genetic programming for automated discovery of physically meaningful observables. A novel adaptive confidence scoring system provides quantitative reliability assessments by evaluating user-selected clustering quality metrics with predefined weights that reflect each metric’s robustness. The platform features configurable molecular profiles for different experimental systems, interactive visualization with selection tools, and comprehensive data filtering capabilities. Utilizing a subset of SCULPT’s capabilities, we analyze photo-double-ionization data measured using the COLTRIMS method for three-body dissociation of the D 2 O molecule, revealing distinct fragmentation channels and their correlations with physics parameters. The software’s modular architecture and web-based implementation make it accessible to the broader atomic and molecular physics community, significantly reducing the time required for complex multi-dimensional analyses. This opens the door to finding and isolating rare events exhibiting non-linear correlations on the fly during experimental measurements, which can help steer exploration and improve the efficiency of experiments.

Artificial neural networks↗
DOE OSTI2023

Driving and characterizing nucleation of urea and glycine polymorphs in water

Crystal nucleation is relevant across the domains of fundamental and applied sciences. However, in many cases, its mechanism remains unclear due to a lack of temporal or spatial resolution. To gain insights into the molecular details of nucleation, some form of molecular dynamics simulations is typically performed; these simulations, in turn, are limited by their ability to run long enough to sample the nucleation event thoroughly. To overcome the timescale limits in typical molecular dynamics simulations in a manner free of prior human bias, here, we employ the machine learning-augmented molecular dynamics framework “reweighted autoencoded variational Bayes for enhanced sampling (RAVE).” We study two molecular systems—urea and glycine—in explicit all-atom water, due to their enrichment in polymorphic structures and common utility in commercial applications. From our simulations, we observe multiple back-and-forth nucleation events of different polymorphs from homogeneous solution; from these trajectories, we calculate the relative ranking of finite-sized polymorph crystals embedded in solution, in terms of the free-energy difference between the finite-sized crystal polymorph and the original solution state. We further observe that the obtained reaction coordinates and transitions are highly nonclassical.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics

A new paradigm for data-driven, model-agnostic new physics searches at colliders is emerging, and aims to leverage recent breakthroughs in anomaly detection and machine learning. In order to develop and benchmark new anomaly detection methods within this framework, it is essential to have standard datasets. To this end, we have created the LHC Olympics 2020, a community challenge accompanied by a set of simulated collider events. Participants in these Olympics have developed their methods using an R&D dataset and then tested them on black boxes: datasets with an unknown anomaly (or not). Furthermore, methods made use of modern machine learning tools and were based on unsupervised learning (autoencoders, generative adversarial networks, normalizing flows), weakly supervised learning, and semi-supervised learning. This paper will review the LHC Olympics 2020 challenge, including an overview of the competition, a description of methods deployed in the competition, lessons learned from the experience, and implications for data analyses with future datasets as well as future colliders.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

CaloChallenge 2022: a community challenge for fast calorimeter simulation

Here, we present the results of the ‘Fast Calorimeter Simulation Challenge 2022’—the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of increasing dimensionality, ranging from a few hundred voxels to a few tens of thousand voxels. The 31 individual submissions span a wide range of current popular generative architectures, including variational autoencoders (VAEs), generative adversarial networks (GANs), normalizing flows, diffusion models, and models based on conditional flow matching. We compare all submissions in terms of quality of generated calorimeter showers, as well as shower generation time and model size. To assess the quality we use a broad range of different metrics including differences in one-dimensional histograms of observables, KPD/FPD scores, AUCs of binary classifiers, and the log-posterior of a multiclass classifier. The results of the CaloChallenge provide the most complete and comprehensive survey of cutting-edge approaches to calorimeter fast simulation to date. In addition, our work provides a uniquely detailed perspective on the important problem of how to evaluate generative models. As such, the results presented here should be applicable for other domains that use generative AI and require fast and faithful generation of samples in a large phase space.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Reconstructing Quasar Spectra and Measuring the Lyα Forest with SpenderQ

Quasar spectra carry the imprint of foreground intergalactic medium (IGM) through absorption features. In particular, absorption caused by neutral hydrogen gas, the "Lyα forest," is a key spectroscopic tracer for cosmological analyses used to measure cosmic expansion and test physics beyond the standard model. Despite their importance, current methods for measuring Lyα absorption cannot directly derive the intrinsic quasar continuum and make strong assumptions on its shape, thus distorting the measured Lyα clustering. We present SpenderQ , a ML-based approach for directly reconstructing the intrinsic quasar spectra and measuring the Lyα forest from observations. SpenderQ uses the Spender spectrum autoencoder to learn a compact and redshift-invariant latent encoding of quasar spectra, combined with an iterative procedure to identify and mask absorption regions. To demonstrate its performance, we apply SpenderQ to 400,000 synthetic quasar spectra created to validate the Dark Energy Spectroscopic Instrument Year 1 Lyα cosmological analyses. SpenderQ accurately reconstructs the true intrinsic quasar spectra, including the broad Lyβ, Lyα, SiIV, CIV, and CIII emission lines. Redward of Lyα, SpenderQ provides percent-level reconstructions of the true quasar spectra. Blueward of Lyα, SpenderQ reconstructs the true spectra to < 5%. SpenderQ reproduces the shapes of individual quasar spectra more robustly than the current state-of-the-art. We, thus, expect it will significantly reduce biases in Lyα clustering measurements and enable studies of quasars and their physical properties. SpenderQ also provides informative latent variable encodings that can be used to, e.g., classify quasars with Broad Absorption Lines. Overall, SpenderQ provides a new data-driven approach for unbiased Lyα forest measurements in cosmological, quasar, and IGM studies.

Hahn, ChangHoon [Arizona U., Astron. Dept. - Stewa↗
DOE OSTI2024

Training quantum neural networks using the quantum information bottleneck method

Abstract We provide in this paper a concrete method for training a quantum neural network to maximize the relevant information about a property that is transmitted through the network. This is significant because it gives an operationally well founded quantity to optimize when training autoencoders for problems where the inputs and outputs are fully quantum. We provide a rigorous algorithm for computing the value of the quantum information bottleneck quantity within error ε that requires O ( log 2 ⁡ ( 1 / ϵ ) + 1 / δ 2 ) queries to a purification of the input density operator if its spectrum is supported on { 0 } ⋃ [ δ , 1 − δ ] for δ > 0 and the kernels of the relevant density matrices are disjoint. We further provide algorithms for estimating the derivatives of the QIB function, showing that quantum neural networks can be trained efficiently using the QIB quantity given that the number of gradient steps required is polynomial.

Çatlı, Ahmet Burak (ORCID:0000000152294141)↗
DOE OSTI2022

Towards automating structural discovery in scanning transmission electron microscopy *

Abstract Scanning transmission electron microscopy is now the primary tool for exploring functional materials on the atomic level. Often, features of interest are highly localized in specific regions in the material, such as ferroelectric domain walls, extended defects, or second phase inclusions. Selecting regions to image for structural and chemical discovery via atomically resolved imaging has traditionally proceeded via human operators making semi-informed judgements on sampling locations and parameters. Recent efforts at automation for structural and physical discovery have pointed towards the use of ‘active learning’ methods that utilize Bayesian optimization with surrogate models to quickly find relevant regions of interest. Yet despite the potential importance of this direction, there is a general lack of certainty in selecting relevant control algorithms and how to balance a priori knowledge of the material system with knowledge derived during experimentation. Here we address this gap by developing the automated experiment workflows with several combinations to both illustrate the effects of these choices and demonstrate the tradeoffs associated with each in terms of accuracy, robustness, and susceptibility to hyperparameters for structural discovery. We discuss possible methods to build descriptors using the raw image data and deep learning based semantic segmentation, as well as the implementation of variational autoencoder based representation. Furthermore, each workflow is applied to a range of feature sizes including NiO pillars within a La:SrMnO 3 matrix, ferroelectric domains in BiFeO 3 , and topological defects in graphene. The code developed in this manuscript is open sourced and will be released at github.com/nccreang/AE_Workflows .

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Differentiable Earth mover’s distance for data compression at the high-luminosity LHC

Abstract The Earth mover’s distance (EMD) is a useful metric for image recognition and classification, but its usual implementations are not differentiable or too slow to be used as a loss function for training other algorithms via gradient descent. In this paper, we train a convolutional neural network (CNN) to learn a differentiable, fast approximation of the EMD and demonstrate that it can be used as a substitute for computing-intensive EMD implementations. We apply this differentiable approximation in the training of an autoencoder-inspired neural network (encoder NN) for data compression at the high-luminosity LHC at CERN The goal of this encoder NN is to compress the data while preserving the information related to the distribution of energy deposits in particle detectors. We demonstrate that the performance of our encoder NN trained using the differentiable EMD CNN surpasses that of training with loss functions based on mean squared error.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Deep kernel methods learn better: from cards to process optimization

Abstract The ability of deep learning methods to perform classification and regression tasks relies heavily on their capacity to uncover manifolds in high-dimensional data spaces and project them into low-dimensional representation spaces. In this study, we investigate the structure and character of the manifolds generated by classical variational autoencoder (VAE) approaches and deep kernel learning (DKL). In the former case, the structure of the latent space is determined by the properties of the input data alone, while in the latter, the latent manifold forms as a result of an active learning process that balances the data distribution and target functionalities. We show that DKL with active learning can produce a more compact and smooth latent space which is more conducive to optimization compared to previously reported methods, such as the VAE. We demonstrate this behavior using a simple cards dataset and extend it to the optimization of domain-generated trajectories in physical systems. Our findings suggest that latent manifolds constructed through active learning have a more beneficial structure for optimization problems, especially in feature-rich target-poor scenarios that are common in domain sciences, such as materials synthesis, energy storage, and molecular discovery. The Jupyter Notebooks that encapsulate the complete analysis accompany the article.

97 MATHEMATICS AND COMPUTING↗