Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “tensor networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Symbolic diagnostics to interpret and analyze neural network models

Embedded machine-learned models (EMLMs) have the promise to improve the predictive accuracy of engineering simulators in environments of national interest. EMLMs often comprise complex input-output maps (e.g., neural networks), which make them unamenable to rigorous analysis and generally difficult to interpret. In the face of decades of theory, this lack of interpretability is a significant barrier to building confidence in these models. This work outlines an approach to interpret EMLMs using sparse polynomial regression for comparison with theoretical understanding. To do so, we build on the concept of Locally Interpretable Model-agnostic Explanations (LIME) using physics-informed clustering, prototype selection, and library construction. While general, we demonstrate our method on tensor-basis neural networks used in Reynolds-Averaged Navier-Stokes simulations of hypersonic fluid flows. Results are presented for a simulated toy model and for direct numerical simulations (DNS) of turbulent flows over a flat plate.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Atypical phase-change alloy Ga 2 Te 3 : atomic structure, incipient nanotectonic nuclei, and multilevel writing

Emerging brain-inspired computing, including artificial optical synapses, photonic tensor cores, neuromorphic networks, etc., needs phase-change materials (PCMs) of the next generation with lower energy consumption and a wider temperature range for reliable long-term operation. Gallium tellurides with higher melting and crystallization temperatures appear to be promising candidates and enable achieving the necessary requirements. Here, using high energy X-ray diffraction and Raman spectroscopy supported by first-principles simulations, we show that vitreous g-Ga 2 Te 3 films essentially have a tetrahedral local structure and sp 3 hybridization, similar to those in the stable fcc Ga 2 Te 3 polymorph and in contrast to a vast majority of typical PCMs. Nevertheless, optical pump–probe laser experiments revealed high-contrast, fast and reversible multilevel SET-RESET transitions raising a question related to the phase change mechanism. A recently observed nanotectonic compression in bulk glassy Ga–Te alloys seems to be responsible for the PCM performance. Incipient nanotectonic nuclei, reminiscent of monoclinic high-pressure HP-Te II and rhombohedral HP-Ga 2 Te 3 , are present as minorities (2–4%) in g-Ga 2 Te 3 but are suggested to grow dramatically with increasing temperature while interacting with appropriate laser pulses. This leads to co-crystallization of HP-polymorphs amplified by a high internal local pressure reaching 4–8 GPa. The metallic HP-forms provide an increasing optical and electrical contrast, favorable for reliable PCM operations, and higher energy efficiency.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Physics-informed machine learning of the Lagrangian dynamics of velocity gradient tensor

Reduced models describing the Lagrangian dynamics of the velocity gradient tensor (VGT) in homogeneous isotropic turbulence (HIT) are developed under the physics-informed machine learning (PIML) framework. We consider the VGT at both Kolmogorov scale and coarse-grained scale within the inertial range of HIT. Building reduced models requires resolving the pressure Hessian and subfilter contributions, which is accomplished by constructing them using the integrity bases and invariants of the VGT. The developed models can be expressed using the extended tensor basis neural network (TBNN) introduced by Ling et al. [J. Fluid Mech. 807, 155 (2016)]. Physical constraints, such as Galilean invariance, rotational invariance, and incompressibility condition, are thus embedded in the models explicitly. Our PIML models are trained on the Lagrangian data from a high-Reynolds number direct numerical simulation (DNS). To validate the results, we perform a comprehensive out-of-sample test. We observe that the PIML model provides an improved representation for the magnitude and orientation of the small-scale pressure Hessian contributions. Statistics of the flow, as indicated by the joint PDF of second and third invariants of the VGT, show good agreement with the “ground-truth” DNS data. A number of other important features describing the structure of HIT are reproduced by the model successfully. We have also identified challenges in modeling inertial range dynamics, which indicates that a richer modeling strategy is required. This helps us identify important directions for future research, in particular towards including inertial range geometry into the TBNN.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

A General Spatiotemporal Imputation Framework for Missing Sensor Data

Many applications from precision agriculture, environmental monitoring and transportation networks rely on data collected across space and time over a large geographic area. Missing data poses a significant challenge for any data-driven inference and control tasks. Data imputation or the estimation of missing data can help fill these gaps by utilizing inherent spatial relationships and temporal patterns. A variety of spatiotemporal imputation models have been developed to address missing data in spatiotemporal datasets. However, these classical methods rely on the assumption that the underlying data follows a smooth trend and fail to provide accurate estimates when there is a large number of missing points in the data. Even though there are machine learning driven tensor completion approaches such as convolutional neural network based tensor completion (CoSTCo) that capture the non-linear relationships in the dataset, the transductive nature makes the algorithm less scalable. Thus, existing approaches for estimating the missing information do not effectively capture all dimensions of the spatiotemporal data structure, resulting in erroneous predictions and poor performance. The main contributions of this paper are: (1) We propose a novel inductive framework (G-LSTM) for missing data imputation that integrates a graph neural network with LSTMs to effectively capture both spatial and temporal dependencies. (2) Experimental results on a traffic dataset demonstrate that the proposed GNN integrated with an LSTM framework achieves improved imputation and maintains steady performance even when there are extreme missing conditions in comparison with the state-of-the-art imputation framework (i.e, CoSTCo). (3) The simulation results on a traffic network show up to 69% reduction in mean absolute error and 61% reduction in root mean square error when compared to CoSTCo.

Tharzeen, Aabila↗
DOE OSTI2020

Report for Department of State (DoS) V-Fund Project "Tying Moment Tensor Solutions to Explosive Yield"

The goal of this study was to use U.S. nuclear explosions with known source parameters (yield, depth, shot point material and/or parameters) to determine moment-derived yield estimates. This work has been accomplished by performing full moment tensor solutions using regional network data from the LLNL network, and other regional broadband stations. As part of this study, we have calculated solutions for 130 U.S. nuclear explosions and 12 additional chemical explosions. We then take several approaches to using moment tensors to estimate yield, considering both the full and isotropic moment tensors, doing straight regression analysis on the whole dataset, then successively refining the calibration with additional information about material and overburden. We have also tried a completely new approach of using the seismic moment to help estimate the radiated seismic energy and tying this to yield through a seismic efficiency. Results appear to be promising, but more work might be required to make this more useful in an operational sense.

58 GEOSCIENCES↗
DOE OSTI2024

Chapter 4: Physically informed deep learning networks for simulating microstructure evolution of 3D polycrystals

As discussed in the previous chapter, high energy diffraction microscopy (HEDM) is used to study the micromechanical evolution of a material during in situ loading. HEDM experiments have been used to verify crystal plasticity (CP) simulations [119, 91, 90, 120], for experimental planning, material design, and to further analyze experimental results. However, Fast Fourier transform-based CP (CP-FFT) or finite element-based CP (CP-FE) methods are often too slow to be used in real-time during an experiment. CP-FFT is faster than CP-FE simulations due to the absence of meshing, but can still take hours to simulate the response of a single volume depending on the size and number of strain steps [127]. Reducing computation time would create a larger exploration space in planning and design, and enable faster analysis of experimental results and real-time feedback during an experiment. This research expands upon previous works to develop a workflow for predicting the full-field evolution of a 3D polycrystal. The workflow is simplified from previous works to predict only orientation and elastic strain tensors (from which stress tensors are calculated). The network is physically informed through loss functions and network architecture for a more robust model. The orientation predictions are informed about the cubic crystal symmetry of the material by incorporating disorientation and misorientation information into the network architecture and loss. The Von Mises stress is used to enforce the correct stress-strain trends in the strain tensor predictions. Additional total strain steps from the elastic and elastoplastic region are included to better capture the stress-strain evolution at smaller total strain steps. Material and hardening parameters are additional inputs into the networks to further inform the network and to study the network’s ability to predict different materials other than those used for training.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Multiarea Distribution System State Estimation via Distributed Tensor Completion

Here, this paper proposes a model-free distribution system state estimation method based on tensor completion using canonical polyadic decomposition. In particular, we consider a setting where the network is divided into multiple areas. The measured physical quantities at buses located in the same area are processed by an area controller. A three-way tensor is constructed to collect these measured quantities. The measurements are analyzed locally to recover the full state information of the network. A distributed closed-form iterative algorithm based on the alternating direction method of multipliers is developed to obtain the low-rank factors of the whole network state tensor where information exchange happens only between neighboring areas. The convergence properties of the distributed algorithm and the sufficient conditions on the number of samples for each smaller network that guarantee the identifiability of the factors of the state tensor are presented. To demonstrate the efficacy of the proposed algorithm and to check the identifiability conditions, numerical simulations are carried out using the IEEE 123-bus system and a large-scale real utility feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Evaluating Physics-Informed Neural Network Performance for Seismic Discrimination between Earthquakes and Explosions

In this article, we evaluate adding a weak physics constraint, that is, a physics‐based empirical relationship, to the loss function with a physics‐informed manner in local distance explosion discrimination in the hope of improving the generalization capability of the machine learning (ML) model. We compare the proposed model with the two‐branch model we previously developed, as well as with a pure data‐driven model. Unexpectedly, the proposed model did not consistently outperform the pure data‐driven model. By varying the level of inconsistency in the training data, we find this approach is modulated by the strength of the physics relationship. In conclusion, this result has important implications for how to best incorporate physical constraints in ML models.

58 GEOSCIENCES↗
DOE OSTI2021

Quantum-Based Molecular Dynamics Simulations Using Tensor Cores

Tensor cores, along with tensor processing units, represent a new form of hardware acceleration specifically designed for deep neural network calculations in artificial intelligence applications. Tensor cores provide extraordinary computational speed and energy efficiency but with the caveat that they were designed for tensor contractions (matrix–matrix multiplications) using only low-precision floating-point operations. Despite this perceived limitation, we demonstrate how tensor cores can be applied with high efficiency to the challenging and numerically sensitive problem of quantum-based Born–Oppenheimer molecular dynamics, which requires highly accurate electronic structure optimizations and conservative force evaluations. The interatomic forces are calculated on-the-fly from an electronic structure that is obtained from a generalized deep neural network, where the computational structure naturally takes advantage of the exceptional processing power of the tensor cores and allows for high performance in excess of 100 Tflops on a single Nvidia A100 GPU. Stable molecular dynamics trajectories are generated using the framework of extended Lagrangian Born–Oppenheimer molecular dynamics, which combines computational efficiency with long-term stability, even when using approximate charge relaxations and force evaluations that are limited in accuracy by the numerically noisy conditions caused by the low-precision tensor core floating-point operations. A canonical ensemble simulation scheme is also presented, where the additional numerical noise in the calculated forces is absorbed into a Langevin-like dynamics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Efficient Mixed-Precision Matrix Factorization of the Inverse Overlap Matrix in Electronic Structure Calculations with AI-Hardware and GPUs

In recent years, a new kind of accelerated hardware has gained popularity in the artificial intelligence (AI) community which enables extremely high-performance tensor contractions in reduced precision for deep neural network calculations. In this article, we exploit Nvidia Tensor cores, a prototypical example of such AI-hardware, to develop a mixed precision approach for computing a dense matrix factorization of the inverse overlap matrix in electronic structure theory, S –1 . This factorization of S –1 , written as ZZT = S –1 , is used to transform the general matrix eigenvalue problem into a standard matrix eigenvalue problem. Here we present a mixed precision iterative refinement algorithm where Z is given recursively using matrix–matrix multiplications and can be computed with high performance on Tensor cores. To understand the performance and accuracy of Tensor cores, comparisons are made to GPU-only implementations in single and double precision. Additionally, we propose a nonparametric stopping criteria which is robust in the face of lower precision floating point operations. The algorithm is particularly useful when we have a good initial guess to Z, for example, from previous time steps in quantum-mechanical molecular dynamics simulations or from a previous iteration in a geometry optimization.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Challenge Problem 1: Preliminary Model Development and Assessment of Flexible Heat Transfer Modeling Approaches

This report presents the modeling progress of a university consortium studying flexible modeling for turbulent heat transfer. In the context of advanced nuclear reactor design, where challenges include non-unity Prandtl fluids, free and mixed convection, and transitional flow, computational fluid dynamics methods are underutilized due to low confidence in modeling approaches and high computational cost. This report evaluates the performance of engineering CFD models in diabatic, buoyant turbulent flow. It finds that all turbulence models including several variants of the k-ε and k-ω models struggle to predict accurate turbulent momentum and heat transfer in such flows. The Nusselt numbers have been compared between the models and the DNS data, where calculations have been performed for each case and trends have shown a good agreement between DNS estimated Nusselt numbers and available correlations as well as experimental data. A novel DNS correlation for high Pr cases as they are transitioning from mixed convection to natural convection has been proposed. While Nusselt number errors relative to DNS range from 20% to 50%, the models capture similar trends to DNS with respect to Nusselt suppression and amplification under varying levels of buoyancy effect. This report also highlights model form error as a significant contributor to CFD predictions and proposes a framework for quantifying model error and improving confidence in CFD calculations. To improve the predictive capability of engineering CFD models, data-driven approaches for turbulence models are investigated. Theoretical frameworks based on the invariant tensor / vector basis neural networks for prediction of Reynolds stresses and turbulent heat fluxes are employed. The models are developed using direct numerical simulations data for forced convection flows of different fluids in vertical planar channel domain. The framework is implemented in spectral element solvers Nek5000 / nekRS and has shown a potential for future development and consideration of mixed convection flows.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Multi-Area Model-Free State Estimation via Distributed Tensor Decomposition

This paper proposes a model-free method for distribution system state estimation based on tensor completion using canonical polyadic decomposition. In particular, we consider a setting where the network is divided into multiple areas. The measured physical quantities at buses located in the same area are processed by an area controller. A third-order tensor is constructed to collect these measured quantities. The measurements are analyzed locally to recover the full state information of the network. A closed-form iterative algorithm based on the alternating direction method of multipliers is developed to obtain the low-rank factors of the whole network state tensor where information exchange happens only between neighboring areas. To demonstrate the efficacy of the developed algorithm, numerical simulations are carried out using an IEEE test system.

alternating direction method of multipliers↗
DOE OSTI2024

Geometry-aware training of factorized layers in tensor Tucker format

Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during train and inference of large networks. Given its easy implementation and flexibility, one promising approach is layer factorization, which reshapes weight tensors into a matrix format and parameterizes it as the product of two rank-r matrices. However, this family of approaches often requires an initial full-model warm-up phase, prior knowledge of a feasible rank, and it is sensitive to parameter initialization.In this work, we introduce a novel approach to train the factors of a Tucker decomposition of the weight tensors. Our training proposal proves to be optimal in locally approximating the original unfactorized dynamics and stable for the initialization. Furthermore, the rank of each mode is dynamically updated during training.We provide a theoretical analysis of the algorithm, showing convergence, approximation and local descent guarantees. The method's performance is further illustrated through a variety of experiments, showing remarkable training compression rates and comparable or even better performance than the full baseline and alternative layer factorization strategies.

Zangrando, Emanuele [Gran Sasso Science Institute ↗
DOE OSTI2025

Atomic Structure, Dynamics, Changes in Chemical Bonding and Semiconductor-Metal Transition in Sb 2 Se 3 : A Remarkable Material for Quantum Networks and Energy Applications

Antimony sesquiselenide has become an outstanding functional material for photovoltaics, energy storage and transformation, memory and photonic applications. Sb 2 Se 3 is one of the most successful emerging solar light absorbers and has also been identified as a highly promising ultralow-loss phase-change material (PCM) for next-generation coherent nanophotonic processors, photonic tensor cores, quantum and neuromorphic networks. Unlike benchmark telluride PCMs, Sb 2 Se 3 features a quasi-one-dimensional (1D) crystalline structure consisting of (Sb 4 Se 6 ) ∞ ribbons, lacks the typical PCM chemical bonding, and undergoes an extended semiconductor-metal transition above the melting point. Consequently, the origin of high optical contrast between crystalline (SET) and amorphous (RESET) logic states remains elusive and presents a significant challenge. Using high-energy X-ray diffraction and Raman spectroscopy over a wide temperature range, supported by first-principles simulations and complemented by thermal, optical and electrical measurements, as well as by 121 Sb-Mossbauer spectroscopy, the quasi-1D network of orthorhombic antimony sesquiselenide was found to undergo significant evolution in amorphous and supercooled Sb 2 Se 3 , leading to lower coordination, shorter interatomic distances and a higher p-electron density on antimony, indicating changes in chemical bonding. The observed novel Sb 2 Se 3 nanocrystalline polymorph, characterized by trigonal antimony coordination and more isolated Sb-Se ribbons, could help reduce multiple trapping defect states in the bandgap, which are typical of orthorhombic Sb 2 Se 3 , thereby enhancing the power-conversion efficiency of photovoltaic devices. Semimetallic and metallic liquid Sb 2 Se 3 exhibit a gradual transformation into a denser 2D and/or 3D network with higher antimony coordination. Localized electron states in the pseudogap are becoming extended, leading to an increase in electronic conductivity σ following the relationship σ ∝ N(E F ) 2 . Liquid Sb 2 Se 3 also appears to be strongly fragile, with a nonmonotonic change in viscosity and higher atomic mobility in the metallic liquid. Furthermore, these results explain extraordinary functionalities of Sb 2 Se 3 for photonic and energy applications.

antimony↗
DOE OSTI2020

Model-Free State Estimation Using Low-Rank Canonical Polyadic Decomposition

As electric grids experience high penetration levels of renewable generation, fundamental changes are required to address real-time situational awareness. Here, we utilize unique traits of tensors to devise a model-free situational awareness and energy forecasting framework for distribution networks. This work formulates the state of the network at multiple time instants as a three-way tensor; hence, recovering full state information of the network is tantamount to estimating all the values of the tensor. Given measurements received from µphasor measurement units and/or smart meters, the recovery of unobserved quantities is carried out using the low-rank canonical polyadic decomposition of the state tensor—that is, the state estimation task is posed as a tensor imputation problem utilizing observed patterns in measured (sampled) quantities. Two structured sampling schemes are considered, namely, asynchronous slab and fiber sampling. For both schemes, we present sufficient conditions on the number of sampled slabs and fibers that guarantee identifiability of the factors of the state tensor. Numerical results demonstrate the ability of the proposed framework to achieve high estimation accuracy in multiple sampling scenarios.

42 ENGINEERING↗
NASA NTRS1991

Optical Computation Of Matrices From Vectors

Proposed optical apparatus generates rectangular pattern of light and dark areas, brightnesses of which represent elements of matrix product of two vectors. Photorefractive effect gives rise to four-wave mixing, which generates output beam modulated spatially by matrix product xy(SupT). Optical multipliers used as real-time analog tensor generators, components of neural networks, and generators of patterns to steer electromagnetic beams by diffraction or to effect temporary interconnections in very-large-scale integrated circuits.

Liu, Hua-Kuang↗