Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural network embeddings”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Gray-Box Modeling for Distribution Systems With Inverter-Based Resources: Integrating Physics-Based and Data-Driven Approaches

Here, in this paper, we develop a novel gray-box modeling approach for distribution systems with inverter-based resources (IBRs). The proposed gray-box modeling method aims to improve estimation accuracy by taking advantages of both physics-based (white-box) and data-driven (black-box) modeling approaches. To this end, we utilize partial physical knowledge of the system, including the inverters’ structures and control diagrams, as well as the equivalent network model simplified through Kron reduction. The white-box model containing unknown parameters is then constructed with mathematical equations and an optimization-based method is subsequently employed to identify these unknown parameters within the white-box model. Next, the graybox modeling framework is then constructed by embedding the output variables of the white-box model into the input vector of a black-box model (represented using a neural network). Finally, the black-box section is trained using the collected input-output datasets and the gray-box model is then obtained. Furthermore, case studies demonstrate that our gray-box modeling approach effectively improves estimation accuracy compared to purely physics-based or data-driven methods.

42 ENGINEERING↗
DOE OSTI2024

Deep Koopman Neural Network for Analyzing High-Energy-Density Simulations of Electrical Wire Explosions

Megaampere-scale electrical wire experiments (EWEs) provide a platform for studying magnetohydrodynamic (MHD) instability growth in magneto-inertial fusion (MIF) devices. Even when nonlinear simulations of these experiments can digitally reproduce much of the experimentally observed instability growth, interpreting the results and understanding mode growth and evolution can be non-trivial. As a first step toward providing better interpretation of these simulation features, this work investigates the use of a deep neural network that uses Koopman operator theory to analyze the dynamics of pulsed-power-driven explosions of EWEs. This deep neural network is trained on 1-D resistive MHD simulations of EWEs. This neural network learns to transform the nonlinear data into a lower-dimensional representation where the time dynamics are linear. Layers of this neural network are shown to learn features of the simulations, including the locations of shock waves and different physical regimes of the simulation. Using the learned features, the network can compress a time state of the simulation consisting of 5120 data point into a 36-parameter lower-dimensional latent space embedding. Furthermore, these embeddings are shown to be clustered in the latent space by initial radius and time state.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗
DOE OSTI2023

A deep learning upscaling framework: Reactive transport and mineral precipitation in fracture-matrix systems

Pore-scale modeling has limited applicability at large scales due to its high computational cost. One common approach to upscale pore-scale models is the use of effective medium theories, which homogenize small-scale features in a porous structure and characterize the medium by macroscale properties (e.g., permeability) and equations (e.g., Darcy’s law). However, there are classes of physical processes for which effective medium approximations may become inaccurate, e.g., mineral precipitation and clogging during reactive transport. We have developed a deep learning upscaling framework, in which pore-scale modeling is directly employed in macroscale systems, without relying on effective medium approximations. The upscaling framework is first developed for general multiscale systems and then applied to modeling reactive transport with mineral precipitation in the altered layer in fracture-matrix structures. Solute transport from the fractures to the matrix is modeled as a wall boundary condition for the fractures, which, in turn, is predicted by recurrent neural networks using the concentration histories at the fracture-matrix boundary. Specifically, we consider a meter-scale fracture network embedded in sandstones, where the smallest feature is at the micron scale. Here the proposed framework allows us to span five orders of magnitude in length scales by capturing mineral precipitation in the altered layer of the rock matrix at the pore scale across the entire meter-scale fracture network.

42 ENGINEERING↗
DOE OSTI2025

Simulating Atmospheric Processes in Earth System Models and Quantifying Uncertainties With Deep Learning Multi‐Member and Stochastic Parameterizations

Abstract Deep learning is a powerful tool to represent subgrid processes in climate models, but many application cases have so far used idealized settings and deterministic approaches. Here, we develop stochastic parameterizations with calibrated uncertainty quantification to learn subgrid convective and turbulent processes and surface radiative fluxes of a superparameterization embedded in an Earth System Model (ESM). We explore three methods to construct stochastic parameterizations: (a) a single Deep Neural Network (DNN) with Monte Carlo Dropout; (b) a multi‐member parameterization; and (c) a Variational Encoder Decoder with latent space perturbation. We show that the multi‐member parameterization improves the representation of convective processes, especially in the planetary boundary layer, compared to individual DNNs. The respective uncertainty quantification illustrates that methods (b) and (c) are advantageous compared to a dropout‐based DNN parameterization regarding the spread of convective processes. Hybrid simulations with our best‐performing multi‐member parameterizations remained challenging and crash within the first days. Therefore, we develop a pragmatic partial coupling strategy relying on the superparameterization for condensate emulation. Partial coupling reduces the computational efficiency of hybrid Earth‐like simulations but enables model stability over 5 months with our multi‐member parameterizations. However, our hybrid simulations exhibit biases in thermodynamic fields and differences in precipitation patterns. Despite this, the multi‐member parameterizations enable improvements in reproducing tropical extreme precipitation compared to a traditional convection parameterization. Despite these challenges, our results indicate the potential of a new generation of multi‐member machine learning parameterizations leveraging uncertainty quantification to improve the representation of stochasticity of subgrid effects.

Behrens, Gunnar [Deutsches Zentrum für Luft‐ und R↗
DOE OSTI2026

SBND Shower Reconstruction with SPINE

The Short-Baseline Near Detector (SBND) is a liquid argon time projection chamber (LArTPC) neutrino detector in the Short-Baseline Neutrino (SBN) program at Fermilab. SBND is designed to investigate the Low-Energy Excess (LEE), an unexplained excess of electron-like events observed by previous short-baseline neutrino experiments that may point to physics beyond the Standard Model. In LArTPC detectors, precise shower reconstruction is essential for distinguishing electrons from photons, a key requirement for testing possible explanations of the LEE and improving $\nu_e$ event selection. In this poster, the reconstruction studies using the Scalable Particle Imaging with Neural Embeddings (SPINE), a machine learning based reconstruction framework for particle imaging detectors will be presented. SPINE combines sparse convolutional neural networks (CNN) and graph neural networks (GNN) to enable detailed reconstruction and characterization of neutrino interactions in LArTPC detectors. Shower calorimetry and kinematic reconstruction are performed in dedicated post-processing stages. Strong agreement between data and Monte Carlo simulation will be demonstrated, indicating high-precision detector calibration and reconstruction performance. The agreement between reconstructed and true electron shower energy will also be discussed, emphasizing the robustness of the shower reconstruction performance. These results demonstrate the unprecedented precision achievable with SPINE in SBND, highlighting their potential for future high-resolution neutrino measurements.

Fan, Castaly [Florida U.; Fermilab] (ORCID:0000000↗
DOE OSTI2022

Graph Neural Networks for Charged Particle Tracking on FPGAs

The determination of charged particle trajectories in collisions at the CERN Large Hadron Collider (LHC) is an important but challenging problem, especially in the high interaction density conditions expected during the future high-luminosity phase of the LHC (HL-LHC). Graph neural networks (GNNs) are a type of geometric deep learning algorithm that has successfully been applied to this task by embedding tracker data as a graph—nodes represent hits, while edges represent possible track segments—and classifying the edges as true or fake track segments. However, their study in hardware- or software-based trigger applications has been limited due to their large computational cost. In this paper, we introduce an automated translation workflow, integrated into a broader tool called hls4ml , for converting GNNs into firmware for field-programmable gate arrays (FPGAs). We use this translation tool to implement GNNs for charged particle tracking, trained using the TrackML challenge dataset, on FPGAs with designs targeting different graph sizes, task complexites, and latency/throughput requirements. This work could enable the inclusion of charged particle tracking GNNs at the trigger level for HL-LHC experiments.

Elabd, Abdelrahman↗
DOE OSTI2023

Systems and methods for customizing kernel machines with deep neural networks

A method including receiving an input data set. The input data set can include one of a feature domain set or a kernel matrix. The method also can include constructing dense embeddings using: (i) Nyström approximations on the input data set when the input data set comprises the kernel matrix, and (ii) clustered Nyström approximations on the input data set when the input data set comprises the feature domain set. The method additionally can include performing representation learning on each of the dense embeddings using a multi-layer fully-connected network for each of the dense embeddings to generate latent representations corresponding to each of the dense embeddings. The method further can include applying a fusion layer to the latent representations corresponding to the dense embeddings to generate a combined representation. The method additionally can include performing classification on the combined representation. Other embodiments of related systems and methods are also disclosed.

Song, Huan↗
DOE OSTI2021

Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders

A common strategy for the dimensionality reduction of nonlinear partial differential equations (PDEs) relies on the use of the proper orthogonal decomposition (POD) to identify a reduced subspace and the Galerkin projection for evolving dynamics in this reduced space. However, advection-dominated PDEs are represented poorly by this methodology since the process of truncation discards important interactions between higher-order modes during time evolution. In this study, we demonstrate that encoding using convolutional autoencoders (CAEs) followed by a reduced-space time evolution by recurrent neural networks overcomes this limitation effectively. We demonstrate that a truncated system of only two latent space dimensions can reproduce a sharp advecting shock profile for the viscous Burgers equation with very low viscosities, and a six-dimensional latent space can recreate the evolution of the inviscid shallow water equations. Additionally, the proposed framework is extended to a parametric reduced-order model by directly embedding parametric information into the latent space to detect trends in system evolution. Furthermore, our results show that these advection-dominated systems are more amenable to low-dimensional encoding and time evolution by a CAE and recurrent neural network combination than the POD-Galerkin technique.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1995

Optical fiber sensors for damage analysis in aerospace materials

Under this grant, fiber optic sensors were investigated for use in the nondestructive evaluation of aging aircraft. Specifically, optical fiber sensors for detection and location of impacts on a surface, and for detection of corrosion in metals were developed. The use of neural networks was investigated for determining impact location by processing the output of a network of fiberoptic strain sensors distributed on a surface. This approach employs triangulation to determine location by comparing the arrival times at several sensors, of the acoustic signal generated by the impact. For this study, a neural network simulator running on a personal computer was used to train a network using a back-propagation algorithm. Fiber optic extrinsic Fabry-Perot interferometer (EFPI) strain sensors are attached to or embedded in the surface, so that stress waves emanating from an impact can be detected. The ability of the network to determine impact location by time-or-arrival of acoustic signals was assessed by comparing network outputs with actual experimental results using impacts on a panel instrumented with optical fiber sensors. Using the neural network to process the sensor outputs, the impact location can be inferred to centimeter range accuracy directly from the arrival time data. In addition, the network can be trained to determine impact location, regardless of material anisotropy. Results demonstrate that a back-propagation network identifies impact location for an anisotropic graphite/bismaleimide plate with the same accuracy as that for an isotropic aluminum plate. Two different approaches were investigated for the development of fiber optic sensors for corrosion detection in metals, both utilizing optical fiber sensors with metal coatings. In the first approach, an extrinsic Fabry-Perot interferometric fiber optic strain sensor was placed under tensile stress, and while in the resulting strained position, a thick coating of metal was applied. Due to an increase in the quantity of material, the sensor does not return to its original position upon removal of the applied stress, and some residual strain is maintained within the sensor element. As the metal thickness decreases due to corrosion, this strain is released, providing the sensing mechanism for corrosion detection. In the second approach, photosensitive optical fibers with long period Bragg gratings in the core were coated with metal. The Bragg gratings serve to couple core modes at discrete wavelengths to cladding modes. Since cladding modes interact with the metal coating surrounding the fiber cladding, the specific wavelengths coupled from core to cladding depend on the refractive index of the metal coating. Therefore, as the metal corrodes, the resulting change in index of the coating may be measured by measuring the change in wavelength of the coupled mode. Results demonstrate that both approaches can be successfully used to track the loss in metal coating on the optical fiber sensors due to corrosion.

Schindler, Paul↗
DOE OSTI2021

Deep learning of parameterized equations with applications to uncertainty quantification

We propose a learning algorithm for discovering unknown parameterized dynamical systems by using observational data of the state variables. Our method is built upon and extends the recent work of discovering unknown dynamical systems, in particular those using deep neural network (DNN). We propose a DNN structure, largely based upon the residual network (ResNet), to not only learn the unknown form of the governing equation but also take into account the random effect embedded in the system, which is generated by the random parameters. Once the DNN model is successfully constructed, it is able to produce system prediction over longer term and for arbitrary parameter values. For uncertainty quantification, it allows us to conduct uncertainty analysis by evaluating solution statistics over the parameter space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Graph Convolutional Network-Based Topology Embedded Deep Reinforcement Learning for Voltage Stability Control

Topological variations in power system is a common phenomenon and can impose significant challenges to traditional controllers of power system. Recent study revealed the strength of deep reinforcement learning (DRL) based approaches in power system preventive and corrective control. But topological variations are difficult to capture using classical fully connected neural network (FCN) model and has not been explicitly modeled in previous work. Hence, we develop a Graph Convolutional Network (GCN) based DRL framework to tackle topology changes in control design of power system. The GCN model exploits the graph structure of the power network and helps the DRL agent to embed the topology information during learning process. Our GCN based approach is evaluated using the IEEE-39 bus system and it outperforms the FCN-based DRL scheme in terms of training convergence and control performance considering grid topology changes.

Hossain, Ramij Raja↗
NASA NTRS1990

Fiber optic smart structures and skins II; Proceedings of the Meeting, Boston, MA, Sept. 5-8, 1989

The present conference on embedded fiber-optics incorporating 'smart' structural systems and structural surfaces discusses topics in the nature and current status of university- and government-sponsored smart-structure development programs, manufacturing and cure-monitoring for composite smart structures, smart-structure damage assessment, smart-structure actuators, and smart-structure sensors and components. Attention is given to fiber-optic sensor selection, the optical properties of curing epoxies, the automated production of smart structures, damage-detection in composites with embedded fiber-optic interferometers, fiber-optic strain and impact sensors, dynamically-tunable smart composites, smart structures incorporating artificial neural networks, active structural acoustic control with smart structures, and fiber-optic shape sensing for flexible structures.

Udd, Eric↗
NASA NTRS1993

Design of fuzzy system by NNs and realization of adaptability

The issue of designing and tuning fuzzy membership functions by neural networks (NN's) was started by NN-driven Fuzzy Reasoning in 1988. NN-driven fuzzy reasoning involves a NN embedded in the fuzzy system which generates membership values. In conventional fuzzy system design, the membership functions are hand-crafted by trial and error for each input variable. In contrast, NN-driven fuzzy reasoning considers several variables simultaneously and can design a multidimensional, nonlinear membership function for the entire subspace.

Takagi, Hideyuki↗
NASA NTRS2022

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
NASA NTRSDate not supplied

Objectively Identifying Transverse Cirrus Bands in Tropical Cyclones using a Convolutional Neural Network

Transverse cirrus bands (TCBs) are a phenomenon that occurs in many mesoscale and synoptic weather systems, including tropical cyclones. TCBs are defined as "Bands of clouds oriented perpendicular to the flow in which they are embedded" (NOAA) and can be associated with low static stability and high environmental vertical wind shear in the upper levels (Kawashima 2021). TCBs in tropical cyclones are of particular interest due to their apparent occurrence and potential linkage to the tropical cyclone diurnal cycle, stronger tropical cyclones, and rapid intensification, where rapid intensification is an intensification rate of 30 knots or more in 24 hours (NHC).

John Mark Mayhall↗
DOE OSTI2021

Physics-informed machine learning of the Lagrangian dynamics of velocity gradient tensor

Reduced models describing the Lagrangian dynamics of the velocity gradient tensor (VGT) in homogeneous isotropic turbulence (HIT) are developed under the physics-informed machine learning (PIML) framework. We consider the VGT at both Kolmogorov scale and coarse-grained scale within the inertial range of HIT. Building reduced models requires resolving the pressure Hessian and subfilter contributions, which is accomplished by constructing them using the integrity bases and invariants of the VGT. The developed models can be expressed using the extended tensor basis neural network (TBNN) introduced by Ling et al. [J. Fluid Mech. 807, 155 (2016)]. Physical constraints, such as Galilean invariance, rotational invariance, and incompressibility condition, are thus embedded in the models explicitly. Our PIML models are trained on the Lagrangian data from a high-Reynolds number direct numerical simulation (DNS). To validate the results, we perform a comprehensive out-of-sample test. We observe that the PIML model provides an improved representation for the magnitude and orientation of the small-scale pressure Hessian contributions. Statistics of the flow, as indicated by the joint PDF of second and third invariants of the VGT, show good agreement with the “ground-truth” DNS data. A number of other important features describing the structure of HIT are reproduced by the model successfully. We have also identified challenges in modeling inertial range dynamics, which indicates that a richer modeling strategy is required. This helps us identify important directions for future research, in particular towards including inertial range geometry into the TBNN.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗