Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Autoencoder”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

In-Field Whole-Plant Maize Architecture Characterized by Subcanopy Rovers and Latent Space Phenotyping

Core Ideas * Subcanopy rovers enabled 3D characterization of thousands of hybrid maize plots. * Machine learning produces heritable latent traits that describe plant architecture. * Rover-based phenotyping is far more efficient than manual phenotyping. * Latent phenotypes from rovers are ready for application to plant biology and breeding. Collecting useful, interpretable, and biologically relevant phenotypes in a resource-efficient manner is a bottleneck to plant breeding, genetic mapping, and genomic prediction. Autonomous and affordable subcanopy rovers are an efficient and scalable way to generate sensor-based datasets of in-field crop plants. Rovers equipped with lidar can produce three-dimensional reconstructions of entire hybrid maize (Zea mays L.) fields. In this study, we collected 2103 lidar scans of hybrid maize field plots and extracted phenotypic data from them by latent space phenotyping. We performed latent space phenotyping by two methods, principal component analysis and a convolutional autoencoder, to extract meaningful, quantitative latent space phenotypes (LSPs) describing whole-plant architecture and biomass distribution. The LSPs had heritabilities of up to 0.44, similar to some manually measured traits, indicating that they can be selected on or genetically mapped. Manually measured traits can be successfully predicted by using LSPs as explanatory variables in partial least squares regression, indicating that the LSPs contain biologically relevant information about plant architecture. These techniques can be used to assess crop architecture at a reduced cost and in an automated fashion for breeding, research, or extension purposes, as well as to create or inform crop growth models.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Deep Set Auto Encoders for Anomaly Detection in Particle Physics

There is an increased interest in model agnostic search strategies for physics beyond the standard model at the Large Hadron Collider. We introduce a Deep Set Variational Autoencoder and present results on the Dark Machines Anomaly Score Challenge. We find that the method attains the best anomaly detection ability when there is no decoding step for the network, and the anomaly score is based solely on the representation within the encoded latent space. This method was one of the top-performing models in the Dark Machines Challenge, both for the open data sets as well as the blinded data sets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Modeling hadronization using machine learning

We present the first steps in the development of a new class of hadronization models utilizing machine learning techniques. We successfully implement, validate, and train a conditional sliced-Wasserstein autoencoder to replicate the Pythia generated kinematic distributions of first-hadron emissions, when the Lund string model of hadronization implemented in Pythia is restricted to the emissions of pions only. The trained models are then used to generate the full hadronization chains, with an IR cutoff energy imposed externally. The hadron multiplicities and cumulative kinematic distributions are shown to match the Pythia generated ones. We also discuss possible future generalizations of our results.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Deep Conservation: A latent dynamics model for exact satisfaction of physical conservation laws [Report]

This work proposes an approach for latent dynamics learning that exactly enforces physical conservation laws. The method comprises two steps. First, we compute a low-dimensional embedding of the high-dimensional dynamical-system state using deep convolutional autoencoders. This defines a low-dimensional nonlinear manifold on which the state is subsequently enforced to evolve. Second, we define a latent dynamics model that associates with a constrained optimization problem. Specifically, the objective function is defined as the sum of squares of conservation-law violations over control volumes in a finite-volume discretization of the problem; nonlinear equality constraints explicitly enforce conservation over prescribed subdomains of the problem. The resulting dynamics model—which can be considered as a projection-based reduced-order model—ensures that the time-evolution of the latent state exactly satisfies conservation laws over the prescribed subdomains. In contrast to existing methods for latent dynamics learning, this is the only method that both employs a nonlinear embedding and computes dynamics for the latent state that guarantee the satisfaction of prescribed physical properties. Numerical experiments on a benchmark advection problem illustrate the method's ability to significantly reduce the dimensionality while enforcing physical conservation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

AI Denoising to Accelerate Detector Simulation

Detector simulation is critical to experimental HEP; however this simulation (commonly done through toolkits such as Geant4) is computationally intensive. Performance can be improved somewhat through technical optimization, but more is needed. Using machine learning (ML) to accelerate simulation is a promising field, however efforts to use generative adversarial networks (GANs) or optimized autoencoders have faced issues. Using convolutional neural networks (CNNs) for denoising has been successful in non-HEP applications such as image processing. This poster investigates the efficacy of using CNNs to denoise Geant4 simulations. This could increase the accuracy of simulations performed under settings designed to increase computational efficiency.

Franklin, Lena↗
DOE OSTI2020

Reification of latent microstructures: On supervised unsupervised and semi-supervised deep learning applications for microstructures in materials informatics

Machine learning (ML), including deep learning (DL), has become increasingly popular in the last few years due to its continually outstanding performance. In this context, we apply machine learning techniques to "learn" the microstructure using both supervised and unsupervised DL techniques. In particular, we focus (1) on the localization problem bridging (micro)structure (localized) property using supervised DL and (2) on the microstructure reconstruction problem in latent space using unsupervised DL. The goal of supervised and semi-supervised DL is to replace crystal plasticity finite element model (CPFEM) that maps from (micro)structure (localized) property, and implicitly the (micro)structure (homogenized) property relationships, while the goal of unsupervised DL is (1) to represent high-dimensional microstructure images in a non-linear low-dimensional manifold, and (2) to discover a way to interpolate microstructures via latent space associating with latent microstructure variables. At the heart of this report is the applications of several common DL architectures, including convolutional neural networks (CNN), autoencoder (AE), and generative adversarial network (GAN), to multiple microstructure datasets, and the quest of neural architecture search for optimal DL architectures.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Tardigrade (NA-22 Quarterly Report)

Tardigrade is an effort to create tunable thermal expansion materials for use in a ruggedized lens housing. This requires new computational algorithms to model and predict thermal performance of meta-materials composed of multiple structured base materials and void-space, along with matching and developing 3D print technology to demonstrate the housing. The project is 24 months into the 36-month plan. We are continuing effort on 3 main technical thrusts: material design algorithm development (transitioning into full 3D design), multi-material additive manufacturing process exploration, and sensor/optics design and engineering. In the last quarter, we have focused on developing the code for 3D Topology Optimization, building data sets for 2D autoencoder training, and finishing lens housing engineering for the microbolometer based test sensor. We are exploring 2 potential multi-material AM technologies, a polymer-based technique in development at LLNL and a metal SLS system produced by Aconity, called the AeroSint deposition head.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Data-Driven Specification of Earth's Magnetic Field

As our society is increasingly dependent on technologies in space, predicting space weather becomes ever more important. We have come to rely on technologies such as cellphones, GPS, the Internet, and other commercial and military assets. In order to protect these systems a high priority is to develop better understanding of the harmful conditions of space weather encountered in the near-Earth radiation environment which are mostly dictated by the dynamics of energetic charged particles. The main objective of our proposal was to improve the specification and forecast of the particle distribution in the Earth’s inner magnetosphere using data assimilation together with machine learning (ML) algorithms in our ring current-atmosphere interactions model with self-consistent magnetic and electric fields (RAM-SCBE). Specifically, we explored the application of machine learning algorithms to emulate and improve, through infusion of observational data, the magnetic field (SCBE) of our ring current model. Our work advanced towards the characterization of the RAM-SCBE particle flux field and its potentially different simulation scenarios. We explored how the model behaves in a number of simulation scenarios in order to better understand how the model reacts to perturbations in the solar wind. We were able to capture this behavior through the simulation of an ensemble of RAM-SCB model solutions. The ensemble showed the important correlations between the particle fluxes and the magnetic field. We were able to start the training of a neural network (Autoencoder) to identify the dominant features of the magnetic field, and the main correlations (linear and non-linear) between the particle fluxes and the magnetic field. Our next steps consists of training a neural network to capture the time evolution of the dominant features and obtain a consistent surrogate model for the RAM-SCB.

42 ENGINEERING↗
DOE OSTI2022

Tardigrade (Quarterly Report Q1 FY2022)

Tardigrade is an effort to create tunable thermal expansion materials for use in a ruggedized lens housing. This requires new computational algorithms to model and predict thermal performance of meta-materials composed of multiple structured base materials and void-space, along with matching and developing 3D print technology to demonstrate the housing. The project is 27 months into the 36-month plan. We are continuing effort on 3 main technical thrusts: material design algorithm development (transitioning into full 3D design), multi-material additive manufacturing process exploration, and sensor/optics design and engineering. In the last quarter, we have focused on developing the code for 3D Topology Optimization, building data sets for 2D autoencoder training, and finishing lens housing engineering for the microbolometer based test sensor. We are exploring 2 potential multi-material AM technologies, a polymer-based technique in development at LLNL and a metal SLS system produced by Aconity, called the AeroSint deposition head.

36 MATERIALS SCIENCE↗
DOE OSTI2022

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The ongoing growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Through access to a large number of time series data sets from different power plants, we have found common issues that impede the modeling process. Namely, the time series data are hard to transfer between groups due to differences in variable nomenclature, and the quality of the data sets can vary. We address the issue of variable nomenclature by FAIRifying spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. We have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. With access to large time series data sets across many power plants, we can utilize spatiotemporal coherence between the sites in order to improve the quality of our data. To solve the issue of data missingness, we propose the use of Spatiotemporal-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗
DOE OSTI2023

Digital Signal Processing Using Deep Neural Networks

Currently there is great interest in the utility of deep neural networks (DNNs) for the physical layer of radio frequency (RF) communications. In this manuscript we describe a custom DNN specially designed to solve problems in the RF domain. Our model leverages the mechanisms of feature extraction and attention through the combination of an autoencoder convolutional network with a transformer network, to accomplish several important communications network and digital signals processing (DSP) tasks. We also present a new open dataset and physical data augmentation model that enables training of DNNs that can perform automatic modulation classification, infer, and correct transmission channel effects, and directly demodulate baseband RF signals.

42 ENGINEERING↗
DOE OSTI2022

Computational Imaging for Intelligence in Highly Scattering Aerosols (Final Report)

Natural and man-made degraded visual environments pose major threats to national security. The random scattering and absorption of light by tiny particles suspended in the air reduces situational awareness and causes unacceptable down-time for critical systems and operations. To improve the situation, we have developed several approaches to interpret the information contained within scattered light to enhance sensing and imaging in scattering media. These approaches were tested at the Sandia National Laboratory Fog Chamber facility and with tabletop fog chambers. Computationally efficient light transport models were developed and leveraged for computational sensing. The models are based on a weak angular dependence approximation to the Boltzmann or radiative transfer equation that appears to be applicable in both the moderate and highly scattering regimes. After the new model was experimentally validated, statistical approaches for detection, localization, and imaging of objects hidden in fog were developed and demonstrated. A binary hypothesis test and the Neyman-Pearson lemma provided the highest theoretically possible probability of detection for a specified false alarm rate and signal-to-noise ratio. Maximum likelihood estimation allowed estimation of the fog optical properties as well as the position, size, and reflection coefficient of an object in fog. A computational dehazing approach was implemented to reduce the effects of scatter on images, making object features more readily discernible. We have developed, characterized, and deployed a new Tabletop Fog Chamber capable of repeatably generating multiple unique fog-analogues for optical testing in degraded visual environments. We characterized this chamber using both optical and microphysical techniques. In doing so we have explored the ability of droplet nucleation theory to describe the aerosols generated within the chamber, as well as Mie scattering theory to describe the attenuation of light by said aerosols, and correlated the aerosol microphysics to optical properties such as transmission and meteorological optical range (MOR). This chamber has proved highly valuable and has supported multiple efforts inclusive to and exclusive of this LDRD project to test optics in degraded visual environments. Circularly polarized light has been found to maintain its polarization state better than linearly polarized light when propagating through fog. This was demonstrated experimentally in both the visible and short-wave infrared (SWIR) by imaging targets made of different commercially available retroreflective films. It was found that active circularly polarized imaging can increase contrast and range compared to linearly polarized imaging. We have completed an initial investigation of the capability for machine learning methods to reduce the effects of light scattering when imaging through fog. Previously acquired experimental long-wave images were used to train an autoencoder denoising architecture. Overfitting was found to be a problem because of lack of variability in the object type in this data set. The lessons learned were used to collect a well labeled dataset with much more variability using the Tabletop Fog Chamber that will be available for future studies. We have developed several new sensing methods using speckle intensity correlations. First, the ability to image moving objects in fog was shown, establishing that our unique speckle imaging method can be implemented in dynamic scattering media. Second, the speckle decorrelation over time was found to be sensitive to fog composition, implying extensions to fog characterization. Third, the ability to distinguish macroscopically identical objects on a far-subwavelength scale was demonstrated, suggesting numerous applications ranging from nanoscale defect detection to security. Fourth, we have shown the capability to simultaneously image and localize hidden objects, allowing the speckle imaging method to be effective without prior object positional information. Finally, an interferometric effect was presented that illustrates a new approach for analyzing speckle intensity correlations that may lead to more effective ways to localize and image moving objects. All of these results represent significant developments that challenge the limits of the application of speckle imaging and open important application spaces. A theory was developed and simulations were performed to assess the potential transverse resolution benefit of relative motion in structured illumination for radar systems. Results for a simplified radar system model indicate that significant resolution benefits are possible using data from scanning a structured beam over the target, with the use of appropriate signal processing.

58 GEOSCIENCES↗
DOE OSTI2024

Bridging the gap between experiments and simulations using machine learning

The physics of inertial confinement fusion is rich and complex. Simulation codes that are used to design experiments are computationally expensive and lack the predictive capability required for extensive parameter exploration in search of a high-performing design for laser direct drive. In this work we use deep learning to build a fast emulator of experiments. To facilitate the development of the deep-learning model, an autoencoder is used to reduce the dimensionality of the input space. Two deep learning models are developed. One model is trained on a vast array of simulation data and is subsequently calibrated to expensive and limited experimental data using a technique known as “transfer learning.” The other model is trained on a statistical model and is subsequently calibrated using experimental data. A comparative study of the two predictive models is carried out. The models potentially reproduce key experimental observables with high accuracy and unprecedented inference times relative to those achieved with simulation codes. These models facilitate rapid exploration of a high dimensional input parameter space.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

ML-Based Rock Properties and Seismic Volume Enhancement

This project aims to improve field-scale Carbon Capture and Storage (CCS) assessments by enhancing petrophysical and geophysical log predictions through machine learning and neural networks. In our work during EY23, we applied Conditional Variational Autoencoders (CVAEs) to predict compressional velocity (Vp) and assess CO2 saturation levels in geological formations at the Illinois Basin Decatur Project (IBDP). In another task, we improved full-waveform inversion (FWI) methods with machine-learning approaches using lithological constraints. Full-waveform inversion (FWI) of seismic data estimates the elastic properties of subsurface rocks with high spatial resolution.

Nathanail, Athanasios↗
DOE OSTI2024

Reduced Order Model for Guided Wave Propagation on Gas Pipelines to Enable Real-Time Simulation

Reduced order model for simulation of Guided wave propagation is presented here. The utilization of reduced order models ensures efficient data generation for a variety of parameters where it takes huge computational effort to simulate, crucial for timely monitoring and decision-making. Autoencoder based reduced order models are proposed here, which are trained on simulated data from open-source finite element framework, Firedrake.

Bukka, Sandeep Reddy↗
DOE OSTI2024

Decode the Workload: Training Deep Learning Models for Efficient Compute Cluster Representation

Monitoring the status of a high throughput computing cluster running computationally intensive production jobs is a crucial yet challenging system administration task due to the complexity of such systems. To this end, we train autoencoders using the Linux kernel CPU metrics of the cluster. Additionally, we explore assisting these models with graph neural networks to share information across threads within a compute node. The models are compared in terms of their ability to: 1) Produce a compressed latent representation that captures the salient features of the input, 2) Detect anomalous activity, and 3) Make distinction between different kinds of jobs run at Jefferson Lab. The goal is to have a robust encoder whose compressed embeddings are used for several downstream tasks. We extend this study further by deploying these models in a human-in-the-loop production-based setting for the anomaly detection task and discuss the associated implementation aspects such as continual learning and the criterion to generate alarms. This study represents a first step in the endeavor towards building self-supervised large-scale foundation models for computing centers.

Mohammed, Ahmed↗
DOE OSTI2023

Adaptive Discovery and Mixed-Variable Optimization of Next Generation Synthesizable Microelectronic Materials

Design of new microelectronic materials is characterized by several challenges such as high-dimensionality of the atomic structure-composition variable space, formidable cost of directly using high-fidelity simulations for design optimization, dispersity in literature-reported similar materials and synthesis methods, complex physical mechanisms, and mixed qualitative and quantitative design variables that lead to a disjointed design space. Even though machine learning (ML) techniques have been employed to expedite materials innovation, existing methods treat ML and design optimization as two separate processes, failing to resolve the fundamental challenges associated with high dimensionality and mixed-variable complexity. We have developed a ML enhanced mixed-variable material design optimization framework to efficiently extract useful information from existing data in literature and physics-based simulations to guide the autonomous search for optimal materials. Our proposed framework is composed of four computational modules: (1) a natural language processing (NLP) based virtual screening module, (2) classification based concept exploration module, (3) a density functional theory (DFT)-based high-fidelity evaluation model, and (4) a novel latent-variable Gaussian process (LVGP) ML model for mixed-variable problems with uncertainty quantification, which seamlessly integrates with Bayesian Optimization (BO) and achieves superb efficiency through embedded physics-based dimension reduction. Our approach is demonstrated and validated using the testbed of functional materials exhibiting metal-insulation transitions (MITs), with the targeted reversible resistivity changes (∼10^5) near room temperature. At the end of the 30-month project, we have developed a series of new ML techniques using NLP, conditional variational autoencoders, active learning, latent-variable Gaussian processes, integrated with Bayesian optimization. Our project has resulted in new predicted MITs compounds and improved understanding of MITs microscopic mechanisms, which in turn will revolutionize microelectronics science to provide energy-saving solutions. Our research has improved both creativity and efficiency in transforming rare-event discoveries of new functional materials to persistent innovations. In addition to open-sourcing the online MIT database and the classification model, the LVGP open source code has been downloaded more than 15,000 times within two years. More than 40 MIT compounds have been identified and many have been pursued experimentally via collaborators. The research results are published in close to 20 collaborative papers in high-impact journals, such as Chem. Mater., Appl. Phys. Rev., Sci. Rep., among others of design space.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Applications of Nickelate perovskites for neuromorphic computing from electronic structure and Machine Learning

While the limit of Moore's law is presently being reached with current microelectronic technologies, we need to develop new paradigms that overcome this limitation. In that respect, neuromorphic computing is a concept that emulates the neural behavior and response of the human brain, and it has been recognized as a promising alternative approach. In this research project, we will perform multi-fidelity scale bridging to explore the potential use of materials with metal to insulator transition for neuromorphic applications. In particular, rare earth nickelates are promising for such purposes, as the transition in these materials is quite sensitive to a broad set of different external stimuli. Our multi-fidelity approach will bridge the high-fidelity electronic structure calculations with classical potentials. We will bridge dynamical mean field theory with a classical atomistic representation via a deep learning force field. The neural network is trained with energies, charges, and forces obtained by accurate electronic structure theories based on Dynamical Mean Field Theory. The configurational space is generated from known crystal phases, ab initio molecular dynamics with exchange-correlation functionals corrected with the Hubbard model, disordered phases with different concentrations of oxygen vacancies, and nonsymmetrical positions and induced strain by grain interfaces or contact with a substrate. Strategies to train the model with a reduced number of training examples are obtained from active learning methods, and new structures for improving the learning process are generated by using machine learning autoencoders. This classical potential will be validated through a diversity of electronic structure methods and represents an important step to combine the flexibility and accuracy of first-principles with the speed of classical potentials. The generated multi-fidelity surrogate model will be used to understand the role of strain, oxygen vacancies, proton doping, the variation of the crystal phase, substrate effects, vibrational effects as the octahedral rotation, grain boundaries and defect effects on the response of a Metal to Insulator Transition (MIT) in correlated materials. Long time and large-scale simulations will help understand the role of different stimuli to control the hysteresis of the MIT, as it has been experimentally suggested. Selected configurations will be analyzed with higher-level theories to provide an accurate electronic description and to study how the orbitals and charges are rearranged under different conditions.

36 MATERIALS SCIENCE↗