SEARCH · Engineering Papers

Results for “tensor program generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Cross-Feature Transfer Learning for Efficient Tensor Program Generation

Tuning tensor program generation involves navigating a vast search space to find optimal program transformations and measurements for a program on the target hardware. The complexity of this process is further amplified by the exponential combinations of transformations, especially in heterogeneous environments. This research addresses these challenges by introducing a novel approach that learns the joint neural network and hardware features space, facilitating knowledge transfer to new, unseen target hardware. A comprehensive analysis is conducted on the existing state-of-the-art dataset, TenSet, including a thorough examination of test split strategies and the proposal of methodologies for dataset pruning. Leveraging an attention-inspired technique, we tailor the tuning of tensor programs to embed both neural network and hardware-specific features. Notably, our approach substantially reduces the dataset size by up to 53% compared to the baseline without compromising Pairwise Comparison Accuracy (PCA). Furthermore, our proposed methodology demonstrates competitive or improved mean inference times with only 25–40% of the baseline tuning time across various networks and target hardware. The attention-based tuner can effectively utilize schedules learned from previous hardware program measurements to optimize tensor program tuning on previously unseen hardware, achieving a top-5 accuracy exceeding 90%. This research introduces a significant advancement in autotuning tensor program generation, addressing the complexities associated with heterogeneous environments and showcasing promising results regarding efficiency and accuracy.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

A High Performance Sparse Tensor Algebra Compiler in MLIR

Sparse tensor algebra is widely used in many applications, including scientific computing, machine learning, and data analytics. The performance of sparse tensor algebra kernels strongly depends on the intrinsic characteristics of the input tensors, hence many storage formats are designed for tensors to achieve optimal performance for particular applications/architectures, which makes it challenging to implement and optimize every tensor operation of interest on a given architecture. We propose a tensor algebra domain-specific language (DSL) and compiler framework to automatically generate kernels for mixed sparse-dense tensor algebra operations. The proposed DSL provides high-level programming abstractions that resemble the familiar Einstein notation to represent tensor algebra operations. The compiler introduces a new Sparse Tensor Algebra dialect built on top of LLVM's extensible MLIR compiler infrastructure for efficient code generation while covering a wide range of tensor storage formats. Our compiler also leverages input-dependent code optimization to enhance data locality for better performance. Our results show that the performance of automatically generated kernels outperforms the state-of-the-art sparse tensor algebra compiler, with up to 20.92x, 6.39x, and 13.9x performance improvement over state-of-the-art tensor algebra compilers, for parallel SpMV, SpMM, and TTM, respectively.

Tian, Ruiqin↗

NASA NTRS1974

Development of solution techniques for nonlinear structural analysis

Nonlinear structural solution methods in the current research literature are classified according to order of the solution scheme, and it is shown that the analytical tools for these methods are uniformly derivable by perturbation techniques. A new perturbation formulation is developed for treating an arbitrary nonlinear material, in terms of a finite-difference generated stress-strain expansion. Nonlinear geometric effects are included in an explicit manner by appropriate definition of an applicable strain tensor. A new finite-element pilot computer program PANES (Program for Analysis of Nonlinear Equilibrium and Stability) is presented for treatment of problems involving material and geometric nonlinearities, as well as certain forms on nonconservative loading.

Vos, R. G.↗

DOE OSTI2016

Simit: A Language for Physical Simulation

With existing programming tools, writing high-performance simulation code is labor intensive and requires sacrificing readability and portability. Here, the alternative is to prototype simulations in a high-level language like Matlab, thereby sacrificing performance. The Matlab programming model naturally describes the behavior of an entire physical system using the language of linear algebra. However, simulations also manipulate individual geometric elements, which are best represented using linked data structures like meshes. Translating between the linked data structures and linear algebra comes at significant cost, both to the programmer and to the machine. High-performance implementations avoid the cost by rephrasing the computation in terms of linked or index data structures, leaving the code complicated and monolithic, often increasing its size by an order of magnitude. In this article, we present Simit, a new language for physical simulations that lets the programmer view the system both as a linked data structure in the form of a hypergraph and as a set of global vectors, matrices, and tensors depending on what is convenient at any given time. Simit provides a novel assembly construct that makes it conceptually easy and computationally efficient to move between the two abstractions. Using the information provided by the assembly construct, the compiler generates efficient in-place computation on the graph. We demonstrate that Simit is easy to use: a Simit program is typically shorter than a Matlab program; that it is high performance: a Simit program running sequentially on a CPU performs comparably to hand-optimized simulations; and that it is portable: Simit programs can be compiled for GPUs with no change to the program, delivering 4 to 20× speedups over our optimized CPU code.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2012

Empirical Performance Model-Driven Data Layout Optimization and Library Call Selection for Tensor Contraction Expressions

Empirical optimizers like ATLAS have been very effective in optimizing computational kernels in libraries. The best choice of parameters such as tile size and degree of loop unrolling is determined by executing different versions of the computation. In contrast, optimizing compilers use a model-driven approach to program transformation. While the model-driven approach of optimizing compilers is generally orders of magnitude faster than ATLAS-like library generators, its effectiveness can be limited by the accuracy of the performance models used. In this paper, we describe an approach where a class of computations is modeled in terms of constituent operations that are empirically measured, thereby allowing modeling of the overall execution time. The performance model with empirically determined cost components is used to perform data layout optimization together with the selection of library calls and layout transformations in the context of the Tensor Contraction Engine, a compiler for a high-level domain-specific language for expressing computational models in quantum chemistry. The effectiveness of the approach is demonstrated through experimental measurements on representative computations from quantum chemistry.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2019

Recursion relation for general 3d blocks

We derive closed-form expressions for all ingredients of the Zamolodchikov-like recursion relation for general spinning conformal blocks in 3-dimensional conformal field theory. This result opens a path to efficient automatic generation of conformal block tables, which has immediate applications in numerical conformal bootstrap program. Our derivation is based on an understanding of null states and conformally-invariant differential operators in momentum space, combined with a careful choice of the relevant tensor structures bases. This derivation generalizes straightforwardly to higher spacetime dimensions d, provided the relevant Clebsch-Gordan coefficients of Spin (d) are known.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Gleipnir: toward practical error analysis for Quantum programs

Practical error analysis is essential for the design, optimization, and evaluation of Noisy Intermediate-Scale Quantum(NISQ) computing. However, bounding errors in quantum programs is a grand challenge, because the effects of quantum errors depend on exponentially large quantum states. In this work, we present Gleipnir, a novel methodology toward practically computing verified error bounds in quantum programs. Gleipnir introduces the (ρ,δ)-diamond norm, an error metric constrained by a quantum predicate consisting of the approximate state ρ and its distance δ to the ideal state ρ. This predicate (ρ,δ) can be computed adaptively using tensor networks based on the Matrix Product States. Gleipnir features a lightweight logic for reasoning about error bounds in noisy quantum programs, based on the (ρ,δ)-diamond norm metric. Furthermore, our experimental results show that Gleipnir is able to efficiently generate tight error bounds for real-world quantum programs with 10 to 100 qubits, and can be used to evaluate the error mitigation performance of quantum compiler transformations.

Tao, Runzhou↗

NASA NTRS1988

Symbolic generation of constitutive equations

The use of the symbolic manipulation program MACSYMA for the automatic generation of constitutive equations describing the thermomechanical behavior of complex materials such as metal-matrix composites is demonstrated. Problem-specific algorithms are developed; e.g., for partial differentiation by the chain rule, automatic generation of tree structures, special tensor analysis, and the simplification of expressions. The resulting subprogram package, SDICE, is shown to be fully applicable to both isotropic and anisotropic materials; its potential usefulness as a basis for FEM analyses is indicated.

Tan, H. Q.↗

DOE OSTI2023

Interfacing electron and neutrino quasielastic scattering cross sections with the spectral function in GENIE

Progress in neutrino-nucleus cross section models is being driven by the need for highly accurate predictions for the neutrino oscillation community. These sophisticated models are being developed within a microscopic description of the nucleus with the goal of encompassing all reaction modes relevant for the accelerator neutrino program. The disconnect between these microscopic models and the event generators that will be used in the next generation of experiments represents a critical obstacle that must be overcome in order to precisely measure the neutrino oscillation parameters. To this end we have developed a hadron tensor interface for lepton-nucleus quasielastic (QE) scattering within the GENIE event generator as a proof of principle, with the broader goal of creating an efficient pipeline for incorporating advanced theoretical models in event generators. As a demonstration of this interface we have implemented the spectral function model into GENIE by connecting theorist provided fortran code through the hadron tensor interface. The spectral function model offers a more complete description of the nuclear ground state, as well as the ability to provide quantifiable theoretical uncertainties. Finally, we validate this implementation and compare its predictions against data and against QE models already available in GENIE.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS1991

Enhancement of surface definition and gridding in the EAGLE code

Algorithms for smoothing of curves and surfaces for the EAGLE grid generation program are presented. The method uses an existing automated technique which detects undesirable geometric characteristics by using a local fairness criterion. The geometry entity is then smoothed by repeated removal and insertion of spline knots in the vicinity of the geometric irregularity. The smoothing algorithm is formulated for use with curves in Beta spline form and tensor product B-spline surfaces.

Thompson, Joe F.↗

NASA NTRS1978

Numerical solution of the Navier-Stokes equations for super-sonic flows with strong shocks

The numerical solution of the full Navier-Stokes Equations for viscous flows with high Mach numbers and a strong detached bow shock was obtained. Two dimensional flows around a circular cylinder, and a circular cylinder with an aft-body in the form of a fairing, were considered. The solution of the compressible N.S. equations was accomplished by the method of finite differences. An implicit scheme of solution, the S.O.R., was used with the optimum acceleration parameters determined by trial and error. The tensor notation was used in writing the N-S Equations transformed into general curvilinear coordinates. The equations for the generation of the coordinate system were solved, followed by the solution of the N.S. equations, at the end of a set of given number of time steps. "Wiggles", constituted the one major problem that needed to be overcome. These oscillations give rise to quantities such as negative temperatures, which ultimately caused the computational program to break down. Certain dissipative finite-difference schemes damped these oscillations.

Devarayalu, K.↗

DOE OSTI2024

FTTN: Feature-Targeted Testing for Numerical Properties of NVIDIA & AMD Matrix Accelerators

While NVIDIA has been the dominant provider of GPUs for HPC and ML, now AMD has several offerings of GPUs. This encourages programmers to try out AMD GPUs for new codes and also port existing codes over. Unfortunately, without understanding the floating-point differences between these GPU types, software development or porting can introduce bugs—and currently such an understanding is lacking. The magnitude of this open question becomes clear if one imagines the the number of floating-point precision choices (FP16, FP32, etc.), floating-point formats (standard floats, brain-float, etc.), and execution units available (elementary units, matrix/tensor cores, etc.) Questions such as rounding modes and subnormal support are also important. Most of these answers are unknown today or are hard to access. We provide the first testing-guided approach that answers a significant number of these questions. We also devise tests to reveal internal information (e.g., extra bits kept) to make sure that our findings are reliable. Many of our tests employ systematically generated random-programs, others apply fast-math flags and some involve fused multiplyadd. Especially for tensor/matrix cores, the tests have nontrivial logic that we present Our testing approach is reusable for the plethora of GPUs yet to be introduced. Our findings include up to 7 ulps of difference between NVIDIA and AMD for sin and cos at FP32 precision and 3 ulp at FP64. In our study of matrix cores (NVIDIA) and tensor cores (AMD), we have extensively characterized rounding modes (truncation versus round-to-nearest), the number of extra internal bits kept (whether 3 bits are kept or not), subnormal support for inputs and outputs across four different floating-point formats and across NVIDIA A100 and AMD MI250X GPUs. We believe that this wealth of data becoming available for the first time may help avoid significant porting bugs when migrating code across these platforms.

Li, Xinyi↗

DOE OSTI2019

Mesoscale Design of Magnetoelectric Heterostructures and Nanocomposites

Multiferroic materials have a coexistence of at least two ferroic orders. The coupling interaction between the different order parameters can produce new phenomena, such as the magnetoelectric (ME) effect. The ME response is the appearance of an electric polarization P upon applying a magnetic field H, and/or vice versa. Magnetoelectricity has been observed as an intrinsic effect in some natural materials at low temperature; however, such single phase materials suffer from extremely weak ME exchange. Better alternatives are two-phase composites consisting of magnetostrictive and piezoelectric phases, which have been shown to have large ME coefficients. Said composites exploit a unique product ME tensor property that depends upon the individual strictions of the piezoelectric and magnetostrictive phases, the elastic stiffness and integrity of the interphase interfaces, the phase distribution and the dimensionality of its connectivity, and any elastic constraint; amongst other things. This research program between D. Viehland PI and S.Priya at Virginia Tech investigated the mesoscale physics and chemistry of such two-phase ME heterostructures. The results of the proposed research are relevant to new generations of multifunctional materials involving heterogeneous components. How the product ME property be optimized? The answers lies with the interphase-interfaces, where the atomic layers coordinate stress transfer from one phase to another. It was the primary goal of this research program to discover such fundamental relationship for two phase ME heterostructures and nanocomposites. This research program integrated all the components required to fully investigate the ME phenomenon: starting from synthesis of nanomaterials with diversified interfacial characteristics; to full electrical, magnetic, and structural characterizations.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Applying Quantum Computing to Simulate Power System Dynamics

Power system dynamics are generally modeled by high dimensional nonlinear differential-algebraic equations due to a large number of generators, loads, and transmission lines. Thus, its computational complexity grows exponentially with the system size. This paper demonstrates the potential use of quantum computing algorithms to model the power system dynamics. Leveraging a symbolic programming framework, we equivalently convert the power system dynamics’ differential algebraic equations (DAEs) into ordinary differential equations (ODEs), where the data of the state vector can be encoded into quantum computers via amplitude encoding. The system's nonlinearity is captured by Taylor polynomial expansion, the quantum state tensor, and Hamiltonian simulation, whereas state variables can be updated by a quantum linear equation solver. Our results show that quantum computing can simulate the dynamics of the power system with high accuracy, whereas its complexity is polynomial in the logarithm of the system dimension. Our work also illustrates the use of scientific machine learning tools for implementing scientific computing concepts, e.g., Taylor expansion, DAEs/ODEs transform, and quantum computing solver, in the field of power engineering.

Tran, Huynh↗

DOE OSTI2021

xSDK: Building an ecosystem of highly efficient math libraries for exascale

Current efforts to build increasingly powerful computer architectures are opening up new avenues for more complex and higher fidelity simulations coupled with data analytics and learning, leading to new scientific insights and deeper understanding. At one extreme, exascale computers will be much faster than previous computer generations (performing 10 18 operations per second—that is, 1,000 times faster than petascale). To achieve these performance improvements, computer architectures are becoming increasingly complex, with deep memory hierarchies, very high node and core counts, and heterogeneous features such as graphics processing units (GPUs). Such architectural changes impact the full breadth of computing scales, as heterogeneity pervades even current-generation laptops, workstations, and moderate-sized clusters. While emerging advanced architectures provide unprecedented opportunities, they also present significant challenges for developers of scientific applications, such as multiphysics and multiscale codes, who must adapt their software to handle disruptive changes in architectures and new programming models that have not yet stabilized. Developers must consider increasing concurrency while reducing communication and synchronization, and other complexities such as the potential for using mixed precision to leverage the compute power available in low-precision tensor cores. On one hand, developers must implement new scientific capabilities, which in turn increase code complexity. On the other hand, the codes must be ported to new architectures, requiring the inclusion of new programming models and the restructuring of code to achieve good performance. Addressing these issues is beyond the capability of any single person or team—leading to the need for collaboration among many teams, who encapsulate their expertise in reusable software and work together to create sustainable software ecosystems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2012

Wavelet-Smoothed Interpolation of Masked Scientific Data for JPEG 2000 Compression [Slides]

How should we manage scientific data with 'holes'? Some applications, like JPEG 2000, expect logically rectangular data, but some sources, like the Parallel Ocean Program (POP), generate data that isn't defined on certain subsets. We refer to grid points that lack well-defined, scientifically meaningful sample values as 'masked' samples. Wavelet-smoothing is a highly scalable interpolation scheme for regions with complex boundaries on logically rectangular grids. Computation is based on forward/inverse discrete wavelet transforms, so runtime complexity and memory scale linearly with respect to sample count. Efficient state-of-the-art minimal realizations yield small constants (O(10)) for arithmetic complexity scaling, and in-situ implementation techniques make optimal use of memory. Implementation in two dimensions using tensor product filter banks is straighsorward and should generalize routinely to higher dimensions. No hand-tuning required when the interpolation mask changes, making the method aeractive for problems with time-varying masks. Well-suited for interpolating undefined samples prior to JPEG 2000 encoding. The method outperforms global mean interpolation, as judged by both SNR rate-distortion performance and low-rate artifact mitigation, for data distributions whose histograms do not take the form of sharply peaked, symmetric, unimodal probability density functions. These performance advantages can hold even for data whose distribution differs only moderately from the peaked unimodal case, as demonstrated by POP salinity data. The interpolation method is very general and is not tied to any particular class of applications, could be used for more generic smooth interpolation.

42 ENGINEERING↗

DOE OSTI2014

Polarized deuterium physics with EIC

We summarize the prospects for measurements of deep-inelastic scattering and related processes on polarized deuterium with a future Electron-Ion Collider (EIC). Such measurements are needed to map the quark spin structure of the neutron and quantify the effect of nuclear binding on the nucleon's quark and gluon distributions. They can also reveal coherent two-nucleon effects through tensor polarization observables. Of particular interest are deep-inelastic measurements in which the spectator nucleon (proton or neutron) is detected in the forward direction. They allow one to identify the active nucleon and control its quantum state, offering unprecedented control and precision for theoretical analysis. The JLab MEIC detector and interaction region design provides excellent capabilities for "spectator tagging" and would enable a rich physics program with polarized deuterium beams. We report about first results of a JLab 2014 LDRD project aimed at developing simulation tools for tagged measurements on polarized deuterium with EIC (cross section models, Monte-Carlo generators) and demonstrating their feasibility.

Weiss, Christian↗

NASA NTRS1980

Alternatives for jet engine control

Alternatives to linear quadratic regulator theory in the linear case are examined along with nonlinear modelling and optimization approaches for global control. Context for the studies has been set by the DYNGEN digital simulator and by models generated for various phases of the F100 Multivariable Control Synthesis Program. With respect to the linear alternatives, the multivariable frequency domain is stressed. Progress is reported in both the direct algebraic approach to exact model matching, by means of stimulating work on the basic computational issues, and in the indirect generalized Nyquist approach. With respect to nonlinear modelling and optimization, the emphasis is twofold: the development of analytical nonlinear models of the jet engine and the use of these models in conjunction with techniques of mathematical programming in order to study global control over nonincremental portions of the flight envelope. The possibility of using tensor methods is explored.

Sain, M. K.↗