Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Neural state space models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Machine learning for deep elastic strain engineering of semiconductor electronic band structure and effective mass

Abstract The controlled introduction of elastic strains is an appealing strategy for modulating the physical properties of semiconductor materials. With the recent discovery of large elastic deformation in nanoscale specimens as diverse as silicon and diamond, employing this strategy to improve device performance necessitates first-principles computations of the fundamental electronic band structure and target figures-of-merit, through the design of an optimal straining pathway. Such simulations, however, call for approaches that combine deep learning algorithms and physics of deformation with band structure calculations to custom-design electronic and optical properties. Motivated by this challenge, we present here details of a machine learning framework involving convolutional neural networks to represent the topology and curvature of band structures in k -space. These calculations enable us to identify ways in which the physical properties can be altered through “deep” elastic strain engineering up to a large fraction of the ideal strain. Algorithms capable of active learning and informed by the underlying physics were presented here for predicting the bandgap and the band structure. By training a surrogate model with ab initio computational data, our method can identify the most efficient strain energy pathway to realize physical property changes. The power of this method is further demonstrated with results from the prediction of strain states that influence the effective electron mass. We illustrate the applications of the method with specific results for diamonds, although the general deep learning technique presented here is potentially useful for optimizing the physical properties of a wide variety of semiconductor materials.

Tsymbalov, Evgenii↗
DOE OSTI2021

Parallel physics-informed neural networks via domain decomposition

Here we develop a distributed framework for the physics-informed neural networks (PINNs) based on two recent extensions, namely conservative PINNs (cPINNs) and extended PINNs (XPINNs), which employ domain decomposition in space and in time-space, respectively. This domain decomposition endows cPINNs and XPINNs with several advantages over the vanilla PINNs, such as parallelization capacity, large representation capacity, efficient hyperparameter tuning, and is particularly effective for multi-scale and multi-physics problems. Here, we present a parallel algorithm for cPINNs and XPINNs constructed with a hybrid programming model described by MPI + X, where X ∈ {CPUs, GPUs}. The main advantage of cPINN and XPINN over the more classical data and model parallel approaches is the flexibility of optimizing all hyperparameters of each neural network separately in each subdomain. We compare the performance of distributed cPINNs and XPINNs for various forward problems, using both weak and strong scalings. Our results indicate that for space domain decomposition, cPINNs are more efficient in terms of communication cost but XPINNs provide greater flexibility as they can also handle time-domain decomposition for any differential equations, and can deal with any arbitrarily shaped complex subdomains. To this end, we also present an application of the parallel XPINN method for solving an inverse diffusion problem with variable conductivity on the United States map, using ten regions as subdomains.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Deep learning of parameterized equations with applications to uncertainty quantification

We propose a learning algorithm for discovering unknown parameterized dynamical systems by using observational data of the state variables. Our method is built upon and extends the recent work of discovering unknown dynamical systems, in particular those using deep neural network (DNN). We propose a DNN structure, largely based upon the residual network (ResNet), to not only learn the unknown form of the governing equation but also take into account the random effect embedded in the system, which is generated by the random parameters. Once the DNN model is successfully constructed, it is able to produce system prediction over longer term and for arbitrary parameter values. For uncertainty quantification, it allows us to conduct uncertainty analysis by evaluating solution statistics over the parameter space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

From disorganized data to emergent dynamic models: Questionnaires to partial differential equations

Starting with sets of disorganized observations of spatially varying and temporally evolving systems, obtained at different (also disorganized) sets of parameters, we demonstrate the data-driven derivation of parameter dependent, evolutionary partial differential equation (PDE) models capable of generating the data. This tensor type of data is reminiscent of shuffled (multidimensional) puzzle tiles. The independent variables for the evolution equations (their “space” and “time”) as well as their effective parameters are all emergent , i.e. determined in a data-driven way from our disorganized observations of behavior in them. We use a diffusion map based questionnaire approach to build a smooth parametrization of our emergent space/time/parameter space for the data. This approach iteratively processes the data by successively observing them on the “space,” the “time” and the “parameter” axes of a tensor. Once the data become organized, we use machine learning (here, neural networks) to approximate the operators governing the evolution equations in this emergent space. Our illustrative examples are based (i) on a simple advection–diffusion model; (ii) on a previously developed vertex-plus-signaling model of Drosophila embryonic development; and (iii) on two complex dynamic network models (one neuronal and one coupled oscillator model) for which no obvious smooth embedding geometry is known a priori. This allows us to discuss features of the process like symmetry breaking, translational invariance, and autonomousness of the emergent PDE model, as well as its interpretability.

generative models↗
NASA NTRSDate not supplied

Statistical learning framework for safety and failure analysis of a DNN-based autonomous aircraft system

Deep Neural Networks (DNNs) and Machine Learning technology is increasingly used for safety-critical applications in the Aerospace domain. To ensure safe operations, the DNN and the system must undergo rigorous verification and validation, including advanced statistical analyses. Performance and safety of the DNN and system behavior must not only be analyzed for the nominal case, but under numerous off-nominal and failure cases. In this paper we will describe how our statistical learning framework SYSAI can efficiently perform such analyses using the tool’s unique combination of advanced learning modeling and statistical analysis techniques. SYSAI can effectively explore the high-dimensional state and failure space of the system under test; geometrical shape detection of safety regions and boundaries support explainability of the results to the designer. In this paper, we report experiments and results obtained with a vision-based DNN control system (ACT) that is capable of autonomously steering an aircraft down a runway.

Yuning He↗
DOE OSTI2023

Lagrangian large eddy simulations via physics-informed machine learning

High-Reynolds number homogeneous isotropic turbulence (HIT) is fully described within the Navier–Stokes (NS) equations, which are notoriously difficult to solve numerically. Engineers, interested primarily in describing turbulence at a reduced range of resolved scales, have designed heuristics, known as large eddy simulation (LES). LES is described in terms of the temporally evolving Eulerian velocity field defined over a spatial grid with the mean-spacing correspondent to the resolved scale. This classic Eulerian LES depends on assumptions about effects of subgrid scales on the resolved scales. Here, we take an alternative approach and design LES heuristics stated in terms of Lagrangian particles moving with the flow. Our Lagrangian LES, thus L-LES, is described by equations generalizing the weakly compressible smoothed particle hydrodynamics formulation with extended parametric and functional freedom, which is then resolved via Machine Learning training on Lagrangian data from direct numerical simulations of the NS equations. The L-LES model includes physics-informed parameterization and functional form, by combining physics-based parameters and physics-inspired Neural Networks to describe the evolution of turbulence within the resolved range of scales. The subgrid-scale contributions are modeled separately with physical constraints to account for the effects from unresolved scales. We build the resulting model under the differentiable programming framework to facilitate efficient training. We experiment with loss functions of different types, including physics-informed ones accounting for statistics of Lagrangian particles. We show that our L-LES model is capable of reproducing Eulerian and unique Lagrangian turbulence structures and statistics over a range of turbulent Mach numbers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2003

Vector-Ordering Filter Procedure for Data Reduction

The vector-ordering filter (VOF) technique involves a procedure for sampling a large population of data vectors to select a subset of data vectors that fully characterize the state space of the large population. The VOF technique enables a large reduction of the volume of data that must be handled in the automated monitoring system and method discussed in the two immediately preceding articles. In so doing, the VOF technique enables the development of data-driven mathematical models of a monitored asset from sets of data that would otherwise exceed the memory capacities of conventional engineering computers. Data-driven mathematical models have been shown to offer high fidelity for purposes of control and monitoring of assets. In practice, a collection of asset-operating observations is acquired with the intention that the collection contain observations characteristic of the full dynamic range of operation of the asset. Often, such a collection contains an extremely large number of observations, many of which are redundant. The VOF technique fills the need for a means to extract, from the original collection of observational data, a reduced data matrix that excludes redundant data while maintaining the full statistical character and dynamic range of the original data. The reduced data matrix can then be used as the input data for development of a mathematical model of the monitored asset, or as training data for a neural-network substitute for an explicit mathematical model of the asset. Alternatively, the reduced data matrix can, itself, be used directly as a mathematical model of the monitored asset, as is commonly done in multivariate state-estimation techniques. The original data are collected from the asset over a range of operating states and are put in matrix form. Each column vector in the original data matrix represents the signal values acquired at a particular operational state of the asset. Thus, the number of columns of the original data matrix equals the number of observed states and the number of rows in this matrix equals the number of signals acquired at each observation. In the VOF technique, one extracts the reduced data matrix from the original data matrix through the selection of a representative subset of the column (state) vectors.

Bickford, Randall L.↗
NASA NTRS2008

A Globally Optimal Particle Tracking Technique for Stereo Imaging Velocimetry Experiments

An important phase of any Stereo Imaging Velocimetry experiment is particle tracking. Particle tracking seeks to identify and characterize the motion of individual particles entrained in a fluid or air experiment. We analyze a cylindrical chamber filled with water and seeded with density-matched particles. In every four-frame sequence, we identify a particle track by assigning a unique track label for each camera image. The conventional approach to particle tracking is to use an exhaustive tree-search method utilizing greedy algorithms to reduce search times. However, these types of algorithms are not optimal due to a cascade effect of incorrect decisions upon adjacent tracks. We examine the use of a guided evolutionary neural net with simulated annealing to arrive at a globally optimal assignment of tracks. The net is guided both by the minimization of the search space through the use of prior limiting assumptions about valid tracks and by a strategy which seeks to avoid high-energy intermediate states which can trap the net in a local minimum. A stochastic search algorithm is used in place of back-propagation of error to further reduce the chance of being trapped in an energy well. Global optimization is achieved by minimizing an objective function, which includes both track smoothness and particle-image utilization parameters. In this paper we describe our model and present our experimental results. We compare our results with a nonoptimizing, predictive tracker and obtain an average increase in valid track yield of 27 percent

McDowell, Mark↗
DOE OSTI2026

Scaling kinetic Monte-Carlo simulations of grain growth with combined convolutional and graph neural networks

Graph neural networks (GNN) have emerged as a promising machine learning method for microstructure simulations such as grain growth. However, accurate modeling of realistic grain boundary networks requires large simulation cells, which GNN has difficulty scaling up to. To alleviate the computational costs and memory footprint of GNN, we suggest a hybrid architecture combining a convolutional neural network (CNN) based bijective autoencoder to compress the spatial dimensions, and a GNN that evolves the microstructure in the latent space of reduced spatial sizes. Our results demonstrate that the new design significantly reduces computational costs with using fewer message passing layer (from 12 down to 3) compared with GNN alone. The reduction in computational cost becomes more pronounced as the spatial size increases, indicating strong computational scalability. For the largest mesh evaluated (160 3 ), our method reduces memory usage and runtime in inference by 117× and 115×, respectively, compared with GNN-only baseline. More importantly, it shows higher accuracy and stronger spatiotemporal capability than the GNN-only baseline, especially in long-term testing. Such combination of scalability and accuracy is essential for simulating realistic material microstructures over extended time scales. The improvements can be attributed to the bijective autoencoder’s ability to compress information losslessly from spatial domain into a high dimensional feature space, thereby producing more expressive latent features for the GNN to learn from, while also contributing its own spatiotemporal modeling capability. Training data are generated from stochastic grain growth simulations, providing realistic variability for learning robust microstructure evolution. Comprehensive system validation confirms that the model is accurate, robust, and scalable.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Computing Nonlinear Power Spectra Across Dynamical Dark Energy Model Space with Neural ODEs

I show how to compute the nonlinear power spectrum across the entire $w(z)$ dynamical dark energy model space. Using synthetic ΛCDM data, I train a neural ordinary differential equation (ODE) to infer the evolution of the nonlinear matter power spectrum as a function of the background expansion and mean matter density across ∼9 Gyr of cosmic evolution. After training, the model generalises to any dynamical dark energy model parameterised by $w(z)$. With little optimisation, the neural ODE is accurate to within 4% up to $k = 5\, h\, {\mathrm Mpc}^{−1}$. Unlike simulation rescaling methods, neural ODEs naturally extend to summary statistics beyond the power spectrum that are sensitive to the growth history.

cosmology↗
DOE OSTI2022

SAFE-OCC: A novelty detection framework for Convolutional Neural Network sensors and its application in process control

Herein we present a novelty detection framework for Convolutional Neural Network (CNN) sensors that we call Sensor-Activated Feature Extraction One-Class Classification (SAFE-OCC). We show that this framework enables the safe use of computer vision sensors in process control architectures. Emergent control applications use CNN models to map visual data to a state signal that can be interpreted by the controller. Incorporating such sensors introduces a significant system operation vulnerability because CNN sensors can exhibit high prediction errors when exposed to novel (abnormal) visual data. Unfortunately, identifying such novelties in real-time is nontrivial. To address this issue, the SAFE-OCC framework leverages the convolutional blocks of the CNN to create an effective feature space to conduct novelty detection using a desired one-class classification technique. This approach engenders a feature space that directly corresponds to that used by the CNN sensor and avoids the need to derive an independent latent space. We demonstrate the effectiveness of SAFE-OCC via simulated control environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Accelerating defect predictions in semiconductors using graph neural networks

First-principles computations reliably predict the energetics of point defects in semiconductors but are constrained by the expense of using large supercells and advanced levels of theory. Machine learning models trained on computational data, especially ones that sufficiently encode defect coordination environments, can be used to accelerate defect predictions. Here, we develop a framework for the prediction and screening of native defects and functional impurities in a chemical space of group IV, III–V, and II–VI zinc blende semiconductors, powered by crystal Graph-based Neural Networks (GNNs) trained on high-throughput density functional theory (DFT) data. Using an innovative approach of sampling partially optimized defect configurations from DFT calculations, we generate one of the largest computational defect datasets to date, containing many types of vacancies, self-interstitials, anti-site substitutions, impurity interstitials and substitutions, as well as some defect complexes. We applied three types of established GNN techniques, namely crystal graph convolutional neural network, materials graph network, and Atomistic Line Graph Neural Network (ALIGNN), to rigorously train models for predicting defect formation energy (DFE) in multiple charge states and chemical potential conditions. We find that ALIGNN yields the best DFE predictions with root mean square errors around 0.3 eV, which represents a prediction accuracy of 98% given the range of values within the dataset, improving significantly on the state-of-the-art. We further show that GNN-based defective structure optimization can take us close to DFT-optimized geometries at a fraction of the cost of full DFT. The current models are based on the semi-local generalized gradient approximation-Perdew–Burke–Ernzerhof (PBE) functional but are highly promising because of the correlation of computed energetics and defect levels with higher levels of theory and experimental data, the accuracy and necessity of discovering novel metastable and low energy defect structures at the PBE level of theory before advanced methods could be applied, and the ability to train multi-fidelity models in the future with new data from non-local functionals. The DFT-GNN models enable prediction and screening across thousands of hypothetical defects based on both unoptimized and partially optimized defective structures, helping identify electronically active defects in technologically important semiconductors.

Rahman, Md Habibur (ORCID:000000027705984X)↗
DOE OSTI2021

The deep-DRT: A deep neural network approach to deconvolve the distribution of relaxation times from multidimensional electrochemical impedance spectroscopy data

Electrochemical impedance spectroscopy (EIS) is an experimental technique ubiquitously used to study electrochemical systems. However, conventional EIS data interpretation through physical and equivalent circuit models is challenging because physical models are problem-specific, and equivalent circuits are often just lumped-element analogs lacking physical meaning. The distribution of relaxation times (DRT) has emerged as a complementary approach to resolve these issues. One drawback of conventional DRT deconvolution is that the EIS data is understood to be (only) a function of frequency (i.e. 1D data) and deconvolved accordingly. This work proposes a novel deconvolution method based on deep neural networks (DNNs), allowing the analysis of multidimensional EIS spectra to bridge data dependency on both frequencies and experimental conditions. Two particularly appealing traits of the deep-DRT method developed in this article are that neither regularization nor specific spacing on the state variables defining the experiment are required. Finally, leveraging DNN to examine complex EIS spectra and their dependence on experimental conditions, this work opens a new research direction in the area of EIS analysis and DRT deconvolution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Towards On-Chip Learning for Low Latency Reasoning with End-to-End Synthesis

The Software Defined Architectures (SODA) Synthesizer is an open-source compiler-based tool able to automatically generate domain-specialized systems targeting Application-Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs) starting from high-level programming. SODA is composed of a frontend, SODA-OPT, which leverages the multilevel intermediate representation (MLIR) framework to interface with productive programming tools (e.g., machine learning frame-works), identify kernels suitable for acceleration, and perform high-level optimizations, and of a state-of-the-art high-level synthesis backend, Bambu from the PandA framework, to generate custom accelerators. One specific application of the SODA Synthesizer is the generation of accelerators to enable ultra-low latency inference and control on autonomous systems for scientific discovery (e.g., electron microscopes, sensors in particle accelerators, etc.). This paper provides an overview of the flow in the context of the generation of accelerators for edge processing to be integrated in transmission electron microscopy (TEM) devices, focusing on use cases from precision material synthesis. We show the tool in action with an example of design space exploration for inference on reconfigurable devices with a conventional deep neural network model (LeNet). Finally, we discuss the research directions and opportunities enabled by SODA in the area of autonomous control for scientific experimental workflows.

Castellana, Vito G.↗
DOE OSTI2021

Entanglement transitions from restricted Boltzmann machines

The search for novel entangled phases of matter has lead to the recent discovery of a new class of “en- tanglement transitions,” exemplified by random tensor networks and monitored quantum circuits. Most known examples can be understood as some classical ordering transitions in an underlying statistical mechanics model, where entanglement maps onto the free-energy cost of inserting a domain wall. In this paper we study the possibility of entanglement transitions driven by physics beyond such statistical mechanics mappings. Motivated by recent applications of neural-network-inspired variational Ansätze, we investigate under what conditions on the variational parameters these Ansätze can capture an entanglement transition. We study the entanglement scaling of short-range restricted Boltzmann machine (RBM) quantum states with random phases. For uncorrelated random phases, we analytically demonstrate the absence of an entanglement transition and reveal subtle finite-size effects in finite-size numerical simulations. Introducing phases with correlations decaying as 1/r α in real space, we observe three regions with a different scaling of entanglement entropy depending on the exponent α. We study the nature of the transition between these regions, finding numerical evidence for critical behavior. Furthermore, our work establishes the presence of long-range correlated phases in RBM-based wave functions as a required ingredient for entanglement transitions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Code for the manuscript "Lagrangian Large Eddy Simulations vis Physics-informed Machine Learning"

We would like to disclose a code based on Python/Pytorch, in which we implement the data-driven physics-informed machine learning of Lagrangian large eddy simulation that is described in the manuscript "Lagrangian Large Eddy Simulations via Physics-Informed Machine Learning ". High-Reynolds number Homogeneous Isotropic Turbulence is fully described within the Navier-Stokes (NS) equations, which are notoriously difficult to solve numerically. Engineers, interested primarily in describing turbulence at a reduced range of resolved scales, have designed heuristics, known as Large Eddy Simulation (LES). LES is described in terms of the temporally evolving Eulerian velocity field defined over a spatial grid with the mean-spacing correspondent to the resolved scale. This classic Eulerian LES depends on assumptions about the effects of sub-grid scales on the resolved scales. Here, we take an alternative approach and design novel LES heuristics stated in terms of Lagrangian particles moving with the flow. Our \emph{Lagrangian LES}, thus L-LES, is described by equations generalizing the weakly compressible Smoothed Particle Hydrodynamics formulation with extended parametric and functional freedom, which is then resolved via Machine Learning training on Lagrangian data from Direct Numerical Simulations of the NS equations. The L-LES model includes physics-informed parameterization and functional form, by combining physics-based parameters and physics-inspired Neural Networks to describe the evolution of turbulence within the resolved range of scales. The sub-grid scale contributions are modeled separately with physical constraints to account for the effects from un-resolved scales. We build the resulting model under the Differentiable Programming framework to facilitate efficient training. We experiment with loss functions of different types, including physics-informed ones accounting for statistics of Lagrangian particles. We show that our Lagrangian LES model is capable of reproducing Eulerian and unique Lagrangian turbulence structures and statistics over a range of turbulent Mach numbers.

Tian, Yifeng↗
DOE OSTI2022

Influence of weather on gobbling activity of male wild turkeys

Gobbling activity of Eastern wild turkeys (Meleagris gallopavo silvestris; hereafter, turkeys) has been widely studied, focusing on drivers of daily variation. Weather variables are widely believed to influence gobbling activity, but results across studies are contradictory and often equivocal, leading to uncertainty in the relative contribution of weather variables to daily fluctuations in gobbling activity. Previous works relied on road-based auditory surveys to collect gobbling data, which limits data consistency, duration, and quantity due to logistical difficulties associated with human observers and restricted sampling frames. Development of new methods using autonomous recording units (ARUs) allows researchers to collect continuous data in more locations for longer periods of time, providing the opportunity to delve into factors influencing daily gobbling activity. We used ARUs from 1 March to 31 May to detail gobbling activity across multiple study sites in the southeastern United States during 2014–2018. We used state-space modeling to investigate the effects of weather variables on daily gobbling activity. Our findings suggest rainfall, greater wind speeds, and greater temperatures negatively affected gobbling activity, whereas increasing barometric pressure positively affected gobbling activity. Therefore, when using daily gobbling activity to make inferences relative to gobbling chronology, reproductive phenology, and hunting season frameworks, stakeholders should recognize and consider the potential influences of extended periods of inclement weather.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Automated Waterbox Inspection for Nuclear Power Plants Using Computer Vision - Based Change Detection

Nuclear power plant waterboxes require regular inspection for leaks, missing components, and structural damage during maintenance outages. Traditional manual inspection is time-consuming and poses safety risks from confined space entry. We developed an automated computer vision system for drone-based waterbox inspection in partnership with Florida Light and Power. Our approach uses feature detection and matching to identify critical changes between baseline and current inspection images, automatically flagging additions (leaks/debris), removals (missing plugs), and translations (displaced components) while compensating for drone movement and environmental variations. We systematically evaluated six feature matching methods, from classical approaches (SIFT+BF) to state-of-the-art neural networks (SuperPoint+SuperGlue), using both standard benchmarks (HPatches) and waterbox-specific validation with real-world augmentations. SuperPoint+SuperGlue achieved superior performance with 7.82 pixels RMSE and 100% success rate—2.8x better accuracy than our baseline. While the pre-trained model has commercial licensing restrictions for nuclear deployment, our findings validate this architecture for custom training. We implemented a real-time GUI demonstrating the SIFT+BF approach for immediate deployment, processing drone feeds at 30 FPS with color-coded change visualization. Future work includes training a custom SuperPoint+SuperGlue model on waterbox data and integrating Vision-Language Models for automated reporting and maintenance guidance.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗