Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Tensor network methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Hutchinson Trace Estimation for high-dimensional and high-order Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have proven effective in solving partial differential equations (PDEs), especially when some data are available by seamlessly blending data and physics. However, extending PINNs to high-dimensional and even high-order PDEs encounters significant challenges due to the computational cost associated with automatic differentiation in the residual loss function calculation. Herein, we address the limitations of PINNs in handling high-dimensional and high-order PDEs by introducing the Hutchinson Trace Estimation (HTE) method. Starting with the second-order high-dimensional PDEs, which are ubiquitous in scientific computing, HTE is applied to transform the calculation of the entire Hessian matrix into a Hessian vector product (HVP). This approach not only alleviates the computational bottleneck via Taylor-mode automatic differentiation but also significantly reduces memory consumption from the Hessian matrix to an HVP’s scalar output. We further showcase HTE’s convergence to the original PINN loss and its unbiased behavior under specific conditions. Comparisons with the Stochastic Dimension Gradient Descent (SDGD) highlight the distinct advantages of HTE, particularly in scenarios with significant variability and variance among dimensions. We further extend the application of HTE to higher-order and higher-dimensional PDEs, specifically addressing the biharmonic equation. By employing tensor-vector products (TVP), HTE efficiently computes the colossal tensor associated with the fourth-order high-dimensional biharmonic equation, saving memory and enabling rapid computation. The effectiveness of HTE is illustrated through experimental setups, demonstrating comparable convergence rates with SDGD under memory and speed constraints. Additionally, HTE proves valuable in accelerating the Gradient-Enhanced PINN (gPINN) version as well as the Biharmonic equation. Overall, HTE opens up a new capability in scientific machine learning for tackling high-order and high-dimensional PDEs.

Curse of dimensionality↗

Geometry-aware training of factorized layers in tensor Tucker format

Reducing parameter redundancies in neural network architectures is crucial for achieving feasible computational and memory requirements during train and inference of large networks. Given its easy implementation and flexibility, one promising approach is layer factorization, which reshapes weight tensors into a matrix format and parameterizes it as the product of two rank-r matrices. However, this family of approaches often requires an initial full-model warm-up phase, prior knowledge of a feasible rank, and it is sensitive to parameter initialization.In this work, we introduce a novel approach to train the factors of a Tucker decomposition of the weight tensors. Our training proposal proves to be optimal in locally approximating the original unfactorized dynamics and stable for the initialization. Furthermore, the rank of each mode is dynamically updated during training.We provide a theoretical analysis of the algorithm, showing convergence, approximation and local descent guarantees. The method's performance is further illustrated through a variety of experiments, showing remarkable training compression rates and comparable or even better performance than the full baseline and alternative layer factorization strategies.

Zangrando, Emanuele [Gran Sasso Science Institute ↗

Automated Generation of Integrated Digital and Spiking Neuromorphic Machine Learning Accelerators

The growing numbers of application areas for artificial intelligence (AI) methods have led to an explosion of domain-specific accelerators that could support every new machine learning (ML) algorithm advancement, clearly highlighting the need for a capability to quickly and automatically transition from algorithm definition to hardware implementation and explore design space along a variety of SWaP (size, weight and Power). The software defined architectures (SODA) synthesizer implements a compiler-based modular infrastructure for the end-to-end generation of machine learning accelerators from high-level frameworks to hardware description language. At the same time, neuromorphic computing, by mimicking how the brain operates, promises to perform artificial intelligence tasks at efficiencies orders of magnitude higher than the current conventional tensor-processing based accelerators, as demonstrated by a variety of specialized designs leveraging Spiking Neural Networks (SNNs). Nevertheless, the mapping of an artificial neural network (ANN) to solutions supporting SNNs is still a non-trivial and very device-specific task, and completely lack the possibility to design hybrid systems that integrate conventional and spiking neural models. In this paper we discuss the support for such an integrated generation leveraging the SODA Synthesizer framework and its modular structure. In particular, we present a new MLIR dialect (part of the SODA frontend) that allows expressing spiking neural network features (e.g., available resources, spiking sequences, analog signal reading, etc.) and illustrate how it enables mapping to Spiking Neurons and deployment to the related specialized hardware (which, in the digital domain, could be generated through the other existing layers of the SODA Synthesizer). We then discuss the opportunities for even deeper integration afforded by the hardware compilation infrastructure, providing a path towards the generation of complex heterogeneous artificial intelligence systems.

Curzel, Serena↗

Quantum real-time evolution using tensor renormalization group methods

We introduce an approach for approximate real-time evolution of quantum systems using tensor renormalization group (TRG) methods originally developed for imaginary time. We use higher-order TRG to generate a coarse-grained time evolution operator for a 1+1⁢D transverse Ising model with a longitudinal field. We show that the standard tensor norm used for the singular value decomposition-based truncation is degenerate and propose an alternate method to discriminate. We show that it is effective and efficient in evolving Gaussian wave packets for one and two particles in the disordered phase, while ordered phase behavior is more challenging to capture. We compare our algorithm with local simulators for universal quantum computers and discuss possible benchmarking in the near future.

lattice gauge theory↗

E(n)-Equivariant cartesian tensor message passing interatomic potential

Machine learning potential (MLP) has been a popular topic in recent years for its capability to replace expensive first-principles calculations in some large systems. Meanwhile, message passing networks have gained significant attention due to their remarkable accuracy, and a wave of message passing networks based on Cartesian coordinates has emerged. However, the information of the node in these models is usually limited to scalars, and vectors. In this work, we propose High-order Tensor message Passing interatomic Potential (HotPP), an E(n) equivariant message passing neural network that extends the node embedding and message to an arbitrary order tensor. By performing some basic equivariant operations, high order tensors can be coupled very simply and thus the model can make direct predictions of high-order tensors such as dipole moments and polarizabilities without any modifications. The tests in several datasets show that HotPP not only achieves high accuracy in predicting target properties, but also successfully performs tasks such as calculating phonon spectra, infrared spectra, and Raman spectra, demonstrating its potential as a tool for future research.

97 MATHEMATICS AND COMPUTING↗

Robust training of machine learning interatomic potentials with dimensionality reduction and stratified sampling

Abstract Machine learning interatomic potentials (MLIPs) enable accurate simulations of materials at scales beyond that accessible by ab initio methods and play an increasingly important role in the study and design of materials. However, MLIPs are only as accurate and robust as the data on which they are trained. Here, we present DImensionality-Reduced Encoded Clusters with sTratified (DIRECT) sampling as an approach to select a robust training set of structures from a large and complex configuration space. By applying DIRECT sampling on the Materials Project relaxation trajectories dataset with over one million structures and 89 elements, we develop an improved materials 3-body graph network (M3GNet) universal potential that extrapolates more reliably to unseen structures. We further show that molecular dynamics (MD) simulations with the M3GNet universal potential can be used instead of expensive ab initio MD to rapidly create a large configuration space for target systems. We combined this scheme with DIRECT sampling to develop a reliable moment tensor potential for titanium hydrides without the need for iterative augmentation of training structures. This work paves the way for robust high-throughput development of MLIPs across any compositional complexity.

Qi, Ji (ORCID:0000000158089931)↗

Network sensitivity solutions for regional moment-tensor inversions

Well-resolved moment-tensor solutions reveal information about the sources of seismic waves. In this paper,we introduce a newly of assessing confidence in the regional full moment-tensor inversion via the introduction of the network sensitivity solution (NSS). The NSS takes into account the unique station distribution, frequency band, and signal-to-noise ratio of a given event scenario. The NSS compares both a hypothetical pure source (for example, an explosion or an earthquake) and the actual data with several thousand sets of synthetic data from a uniform distribution of all possible sources. The comparison with a hypothetical pure source provides the theoretically best-constrained source-type distribution for a given set of stations; and with it, one can determine whether further analysis with the data is warranted. The NSS that employs the actual data gives a direct comparison of all other source types with the best fit source. In this way, one can choose a threshold level of fit in which the solution is comfortably constrained. The method is tested for the well-recorded nuclear test, JUNCTION, at the Nevada Test Site. Sources that fit comparably well to a hypothetical pure explosion recorded with no noise at the JUNCTION data stations have a large volumetric component and are not described well by a double-couple (DC) source. The NSS using the real data from JUNCTION is even more tightly constrained to an explosion because the data contain some energy that precludes fitting with any type of deviator source. We also calculate the NSS for the October 2006 North Korea test and a nearby earthquake, where the station coverage is poor and the event magnitude is small. As a result, the earthquake solution is very well fit by a DC source, and the best-fit solution to the nuclear test (M w 4.1) is dominantly explosion.

58 GEOSCIENCES↗

Effective many-body interactions in reduced-dimensionality spaces through neural network models

Accurately describing properties of challenging problems in physical sciences often requires complex mathematical models that are unmanageable to tackle head on. Therefore, developing reduced-dimensionality representations that encapsulate complex correlation effects in many-body systems is crucial to advance the understanding of these complicated problems. However, a numerical evaluation of these predictive models can still be associated with a significant computational overhead. To address this challenge, in this paper we discuss a combined framework that integrates recent advances in the development of active-space representations of coupled cluster (CC) downfolded Hamiltonians with neural network approaches. The primary objective of this effort is to train neural networks to eliminate the computationally expensive steps required for evaluating hundreds or thousands of Hugenholtz diagrams, which correspond to multidimensional tensor contractions necessary for evaluating a many-body form of downfolded effective Hamiltonians. Using small molecular systems (the H 2 O and HF molecules) as examples, we demonstrate that training neural networks employing effective Hamiltonians for a few nuclear geometries of molecules can accurately interpolate or extrapolate their forms to other geometrical configurations characterized by different intensities of correlation effects. We also discuss differences between effective interactions that define CC downfolded Hamiltonians with those of bare Hamiltonians defined by Coulomb interactions in the active spaces. Published by the American Physical Society 2024

97 MATHEMATICS AND COMPUTING↗

Computational Power of Random Quantum Circuits in Arbitrary Geometries

Empirical evidence for a gap between the computational powers of classical and quantum computers has been provided by experiments that sample the output distributions of two-dimensional quantum circuits. Many attempts to close this gap have utilized classical simulations based on tensor network techniques, and their limitations shed light on the improvements to quantum hardware required to frustrate classical simulability. In particular, quantum computers having in excess of approximately 50 qubits are primarily vulnerable to classical simulation due to restrictions on their gate fidelity and their connectivity, the latter determining how many gates are required (and, therefore, how much infidelity is suffered) in generating highly entangled states. Here, we describe recent hardware upgrades to Quantinuum’s H2 quantum computer, enabling it to operate on up to 56 qubits with arbitrary connectivity and 99.843(5)% two-qubit gate fidelity. We define a class of circuits with random geometries that become hard to classically simulate in very low depth and implement them utilizing the flexible connectivity of H2. A careful analysis demonstrating the fast saturation of classical simulation complexity with depth indicates that H2 can yield data well beyond the reach of state-of-the art classical simulation methods at unprecedented fidelities. We find that the considerable difficulty of classically simulating H2 is likely limited only by qubit number, demonstrating the promise and scalability of the quantum charge-coupled device architecture as continued progress is made toward building larger machines. Published by the American Physical Society 2025

DeCross, M.↗

H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture

Recently Graph Neural Networks (GNNs) have drawn tremendous attentions due to their unique capability to extend the Machine Learning (ML) approaches to broadly defined applications with unstructured data, especially graphs. Comparing with other ML modalities, the acceleration of GNNs is as critical but even more challenging due to the irregularity and heterogeneity from graph typologies that together limit the performance. Existing efforts mainly focus on handling graphs’ irregularity, however, have not studied the heterogeneity. To this end, in this work, we propose H-GCN, a PL-AIE-based hybrid accelerator that leverages the emerging heterogeneity of Xilinx Versal ACAPs to achieve high-performance GNN inference. In particular, H-GCN partitions each graph into three subgraphs based on its inherent heterogeneity and processes them using PL and the newly emerged AIE respectively. To further improve the performance, we explore the sparsity support of AIE and develop an efficient density-aware method to map tiles of SpMM onto the systolic tensor array automatically. Compared with the current state-of-the-art GCN accelerator, HGCN achieves on average 1.5× speedups.

Zhang, Chengming↗

A sequential DNN based Baseline Energy Prediction Framework with Long term Error Mitigation

In this paper,we aim to present a novel sequential framework containing two deep network architectures for baseline energy prediction, applied to a simulated commercial building dataset. We have observed that this sequential framework helps to improve long term prediction accuracy, there by mitigating prediction error accumulation over time. The proposed framework utilizes convolution layers to extract features from the input data space without changing spatial relations between variables. These features are memorized by tensor train based gated recurrent units for an accurate long-term prediction. Although, architecture comprising the amalgamation of convolution layers and memory cell have shown promising results in domains such as social media analysis, language modeling, video frame prediction and image recognition, this paper extends its scope to the context of energy applications. Furthermore, the addition of tensor train based gated recurrent units is motivated by the necessity of computational time reduction during the training process, there by making this framework more suitable for future in-field deployment. Lastly, results show the current framework outperforms other existing machine learning based methods, in both short-term and long-term prediction categories.

Chakraborty, Indrasis↗

A new generic method for the semi-automatic extraction of river and road networks in low and mid-resolution satellite images

This paper addresses the problem of semi-automatic extraction of road or hydrographic networks in satellite images. For that purpose, we propose an approach combining concepts arising from mathematical morphology and hydrology. The method exploits both geometrical and topological characteristics of rivers/roads and their tributaries in order to reconstruct the complete networks. It assumes that the images satisfy the following two general assumptions, which are the minimum conditions for a road/river network to be identifiable and are usually verified in low- to mid-resolution satellite images: (i) visual constraint: most pixels composing the network have similar spectral signature that is distinguishable from most of the surrounding areas; (ii) geometric constraint: a line is a region that is relatively long and narrow, compared with other objects in the image. While this approach fully exploits local (roads/rivers are modeled as elongated regions with a smooth spectral signature in the image and a maximum width) and global (they are structured like a tree) characteristics of the networks, further directional information about the image structures is incorporated. Namely, an appropriate anisotropic metric is designed by using both the characteristic features of the target network and the eigen-decomposition of the gradient structure tensor of the image. Following, the geodesic propagation from a given network seed with this metric is combined with hydrological operators for overland flow simulation to extract the paths which contain most line evidence and identify them with the target network.

47 OTHER INSTRUMENTATION↗

Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques

Large volumes of data generated by scientific simulations, genome sequencing, and other applications need to be moved among clusters for data collection/analysis. Data compression techniques have effectively reduced data storage and transfer costs. However, users' requirements on interactively controlling both data quality and compression ratios are non-trivial to fulfill. Here, we propose a novel Compression-as-a-Service (CaaS) platform called Ocelot with four important contributions: (1) It offers real-time visualization, interactive compression, and transfer of scientific datasets. (2) It incorporates new strategies for compressing diverse types of datasets more effectively than traditional methods. (3) It provides an effective method for estimating the compression ratio and execution time of compression tasks. (4) Experiments on multiple real-world datasets on geographically distributed computers show that Ocelot can significantly improve data transfer efficiency with a performance gain of more than 10x in computing clusters with relatively slow networks.

compression as a service (CaaS)↗

Three-Receiver Quantum Broadcast Channels: Classical Communication with Quantum Non-unique Decoding

In network communication, it is common in broadcasting scenarios for there to exist a hierarchy among receivers based on information they decode due, for example, to different physical conditions or premium subscriptions. This hierarchy may result in varied information quality, such as higher-quality video for certain receivers. This is modeled mathematically as a degraded message set, indicating a hierarchy between messages to be decoded by different receivers, where the default quality corresponds to a common message intended for all receivers, a higher quality is represented by a message for a smaller subset of receivers, and so forth. We extend these considerations to quantum communication, exploring three-receiver quantum broadcast channels with two- and three-degraded message sets. Our technical tool involves employing quantum non-unique decoding, a technique we develop by utilizing the simultaneous pinching method. Here, we construct one-shot codes for various scenarios and find achievable rate regions relying on various quantum Rényi mutual information error exponents. Our investigation includes a comprehensive study of pinching across tensor product spaces, presenting our findings as the asymptotic counterpart to our one-shot codes. By employing the non-unique decoding, we also establish a simpler proof to Marton’s inner bound for two-receiver quantum broadcast channels without the need for more involved techniques. Additionally, we derive no-go results and demonstrate their tightness in special cases.

Salek, Farzin [Technical University of Munich (Ger↗

Source Characterization of Underground Explosions from Combined Regional Moment Tensor and First-Motion Analysis

Here in this study, we investigate the 14 September 1988 U.S.–Soviet Joint Verification Experiment nuclear test at the Semipalatinsk test site in eastern Kazakhstan and two nuclear explosions conducted less than 10 years later at the Chinese Lop Nor test site. These events were very sparsely recorded by stations located within 1600 km, and in each case only three or four stations were available in the regional distance range. We have utilized a regional distance seismic waveform method fitting long-period, complete, three-component waveforms jointly with first-motion observations from regional stations and teleseismic arrays. The combination of long-period waveforms and first-motion observations provides a unique discrimination of these sparsely recorded events in the context of the Hudson et al. (1989) source-type diagram. We demonstrate through a series of jackknife tests and sensitivity analyses that the source type of the explosions is well constrained. One event, a 1996 Lop Nor shaft explosion, displays large Love waves and possibly reversed Rayleigh waves at one station, indicative of a large F-factor. We show the combination of long-period waveforms and P-wave first motions are able to discriminate this event as explosion-like and distinct from earthquakes and collapses. We further demonstrate the behavior of network sensitivity solutions for models of tectonic release and spall-based tensile damage over a range of F-factors and K-factors.

58 GEOSCIENCES↗

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗

Algebraic Bethe Circuits

The Algebraic Bethe Ansatz (ABA) is a highly successful analytical method used to exactly solve several physical models in both statistical mechanics and condensed-matter physics. Here we bring the ABA into unitary form, for its direct implementation on a quantum computer. This is achieved by distilling the non-unitary R matrices that make up the ABA into unitaries using the QR decomposition. Our algorithm is deterministic and works for both real and complex roots of the Bethe equations. We illustrate our method on the spin-$\frac{1}{2}$ XX and XXZ models. We show that using this approach one can efficiently prepare eigenstates of the XX model on a quantum computer with quantum resources that match previous state-of-the-art approaches. We run small-scale error mitigated implementations on the IBM quantum computers, including the preparation of the ground state for the XX and XXZ models on 4 sites. Finally, we derive a new form of the Yang-Baxter equation using unitary matrices, and also verify it on a quantum computer.

97 MATHEMATICS AND COMPUTING↗

Identifying Isotropic Events Using a Regional Moment Tensor Inversion

In our previous work the deviatoric and isotropic source components for 17 explosions at the Nevada Test Site, as well as 12 earthquakes and 4 collapses in the surrounding region of the western US, were calculated using a regional time-domain full waveform inversion for the complete moment tensor (Dreger et al., 2008; Ford et al., 2008; Ford et al., 2009a). The events separate into specific populations according to their deviation from a pure double-couple and ratio of isotropic to deviatoric energy. The separation allows for anomalous event identification and discrimination between explosions, earthquakes, and collapses. Confidence regions of the model parameters are estimated from the data misfit by assuming normally distributed parameter values. We developed a new Network Sensitivity Solution (NSS) in which the fit of sources distributed over a source-type plot (Hudson et al., 1989) show the resolution of the source parameters. The NSS takes into account the unique station distribution, frequency band, and signal-to-noise ratio of a given event scenario. The NSS compares both a hypothetical pure source (for example an explosion or an earthquake) and the actual data with several thousand sets of synthetic data from a uniform distribution of all possible sources. The comparison with a hypothetical pure source provides the theoretically best-constrained source-type region for a given set of stations, and with it one can determine whether further analysis with the data is warranted. We apply the NSS to a NTS nuclear explosion, and earthquake, as well as the 2006 North Korean explosion, and a nearby earthquake. The results show that explosions and earthquakes are distinguishable, however the solution space depends strongly on the station coverage. Finally, on May 25, 2009 a second North Korean test took place. Our preliminary results show that the explosive nature of the event may be determined using the regional distance moment tensor method. Results indicate that the 2009 event is approximately 5-6 times larger than the earlier test, with an isotropic moment of about 1.8e+22 dyne cm. We perform a series of inversions for pure double-couple, pure explosion, combined double-couple and explosion, full moment tensor, and damped moment tensor inversions to assess the resolution of the isotropic moment of the event.

58 GEOSCIENCES↗