Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “limited memory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Autonomous nondestructive evaluation of resistance spot welded joints

The application of non-destructive evaluation approaches has attracted strong interests in modern automotive industries. Here, we present an autonomous deep-computing framework to analyze raw videos from infrared systems and to predict weld nugget shape and size with unprecedented accuracy and speed. In a comprehensive training and testing experiment with 90 videos (seven sets of welding material stack-ups), a new method was developed to assemble sufficient datasets for neural network training. Our framework successfully predicts all the nugget shapes with F1 scores that range from 0.84 to 0.92. The total training time on Nvidia DGX station takes less than 10 min for each set of welding material stack-up. The real inference time of an individual dataset (with 30 video frames) takes about 0.005 s. The procedure and methods developed in the study can be applied to other image-based weld property prediction, as well as other manufacturing processes. Furthermore, our well-trained neural networks take limited memory resources (2.3 MB) and are suitable for embedded microprocessors for in-situ welding quality control as edge computing within an intelligent welding framework.

42 ENGINEERING↗
DOE OSTI2022

WUS256: An Adjoint Waveform Tomography Model of the Crust and Upper Mantle of the Western United States for Improved Waveform Simulations

Abstract We report a new model (WUS256) of radially anisotropic seismic wavespeeds of the crust and upper mantle of the western United States (WUS) obtained from adjoint waveform tomography for the purpose of improving synthetic waveform fits to observed data. WUS256 is based on inversion of over 94,000 waveforms from 72 earthquakes recorded by nearly 3,400 stations. We started with the SPiRaL global model (Simmons et al., 2021, https://doi.org/10.1093/gji/ggab277 ) and waveforms in the period band of 50–120 s. We followed a conservative multiscale inversion approach with eight stages and 256 total inversion iterations which enabled monotonic misfit reduction to 20‐s minimum‐period waves. WUS256 relied on time‐frequency (TF) phase misfits and a trust region limited memory Broyden–Fletcher–Goldfarb–Shanno (L‐BFGS) optimization. Hessian‐vector products were used to qualitatively assess model resolution. Results indicate that WUS256 has good coverage of the continental regions to depths of about 150 km and is able to resolve features on lateral scales of about 200 km. We quantify waveform fits by the reduction in TF and normalized amplitude difference misfits between WUS256 and the SPiRaL starting model. WUS256 significantly improves waveform fits with misfit reduction 64% for both inversion and validation data sets compared to the SPiRaL starting model and shows even better fits compared to other models. Waveform fits illustrate that WUS256 reproduces body‐waves, fundamental mode surface waves as well as late arriving dispersed and/or scattered short period surface waves. The improvement in waveform fit indicates that WUS256 can be used to reproduce path effects on regional complete waveforms and moment tensor inversions.

58 GEOSCIENCES↗
DOE OSTI2026

Ground and excited state gradients with end-to-end differentiable semiempirical quantum chemistry

Accurate and efficient gradients of molecular energy with respect to nuclear degrees of freedom are essential for geometry optimization and molecular dynamics, including simulations that go beyond the Born–Oppenheimer regime. A common approach involves deriving analytical formulas for new electronic structure methods, which is often conceptually difficult and requires tedious coding. Here, we implement analytical, semi-numerical, and automatic differentiation (AD)-based gradient pathways for semiempirical Hamiltonian models in the PYSEQM software package, leveraging both graphics processing unit (GPU) and central processing unit (CPU) architectures. We further extend these capabilities to excited states calculated using the configuration interaction singles and time-dependent Hartree–Fock ansätze. We benchmark wall time, peak memory usage, and accuracy across three molecular families of varying chemical complexity, including systems of up to a thousand atoms. For ground-state simulations, analytical and AD gradients achieve near-identical GPU runtimes, while semi-numerical gradients are slower on GPU but remain competitive on CPU. For excited states, both analytical and custom AD approaches using implicit differentiation show similar performance and low memory requirements, whereas gradients with full AD are memory-limited. AD gradients match analytical ones in accuracy across all tested systems, aided by a quaternion-based diatomic frame rotation for two-center quantities that ensures smooth energy surfaces. Overall, automatic differentiation emerges as a practical alternative to analytical gradients in semiempirical quantum chemistry, offering high accuracy while allowing seamless integration in AI-driven workflows and popular packages, such as PyTorch and JAX. Our results provide actionable guidance for selecting optimal gradient strategies in large-scale ground- and excited-state molecular dynamics simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Pre-conditioned BFGS-based uncertainty quantification in elastic full-waveform inversion

SUMMARY Full-waveform inversion has become an essential technique for mapping geophysical subsurface structures. However, proper uncertainty quantification is often lacking in current applications. In theory, uncertainty quantification is related to the inverse Hessian (or the posterior covariance matrix). Even for common geophysical inverse problems its calculation is beyond the computational and storage capacities of the largest high-performance computing systems. In this study, we amend the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm to perform uncertainty quantification for large-scale applications. For seismic inverse problems, the limited-memory BFGS (L-BFGS) method prevails as the most efficient quasi-Newton method. We aim to augment it further to obtain an approximate inverse Hessian for uncertainty quantification in FWI. To facilitate retrieval of the inverse Hessian, we combine BFGS (essentially a full-history L-BFGS) with randomized singular value decomposition to determine a low-rank approximation of the inverse Hessian. Setting the rank number equal to the number of iterations makes this solution efficient and memory-affordable even for large-scale problems. Furthermore, based on the Gauss–Newton method, we formulate different initial, diagonal Hessian matrices as pre-conditioners for the inverse scheme and compare their performances in elastic FWI applications. We highlight our approach with the elastic Marmousi benchmark model, demonstrating the applicability of pre-conditioned BFGS for large-scale FWI and uncertainty quantification.

58 GEOSCIENCES↗
DOE OSTI2026

Tula: Optimizing Time, Cost, and Generalization in Distributed Large-Batch Training

Distributed training increases the number of batches processed per iteration either by scaling-out (adding more nodes) or scaling-up (increasing the batch-size). However, the largest configuration does not necessarily yield the best performance. Horizontal scaling introduces additional communication overhead, while vertical scaling is constrained by computation cost and device memory limits. Thus, simply increasing the batch-size leads to diminishing returns: training time and cost decrease initially but eventually plateaus, creating a knee-point in the time/cost vs. batch-size pareto curve. The optimal batch-size therefore depends on the underlying model, data and available compute resources. Large batches also suffer from worse model quality due to the well-known “generalization gap”. In this paper, we present Tula, an online service that automatically optimizes time, cost, and convergence quality for large-batch training of convolutional models. It combines parallel-systems modeling with statistical performance prediction to identify the optimal batchsize. Tula predicts training time and cost within 7.5−14% error across multiple models, and achieves up to 20× overall speedup and improves test accuracy by ≈9% on average over standard large-batch training on various vision tasks, thus successfully mitigating the generalization gap and accelerating training at the same time.

Tyagi, Sahil [ORNL] (ORCID:0009000783144745)↗
DOE OSTI2024

LATTE: Los Alamos TravelTime package based on Eikonal equation

This Fortran code focuses on traveltime computation and tomography based on eikonal equation. Specifically, the package provides three major functionalities: (1) forward modeling of traveltime from single-point or ensemble source based on factorized eikonal equation, (2) adjoint-state first-arrival traveltime tomography based on picked first arrival traveltime using steepest descent, conjugate gradient, or limited-memory BFGS inversion scheme, and (3) adjoint-state joint transmission-reflection tomography based on picked first-arrival and reflection traveltimes. The package applies to forward modeling and tomography based on traveltime in 2D and 3D isotropic regular-grid models. We name this package LATTE – Los Alamos TravelTime package based on Eikonal equation. * The code is for accompanying a journal paper under preparation. The paper will be submitted via LA-UR separately later.

Gao, Kai↗
DOE OSTI2025

Sequence length scaling in vision transformers for scientific images on frontier

Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence lengths. While transformers for text have inspired scaling sequence lengths in ViTs, adapting these for ViTs introduces unique challenges. We develop distributed sequence parallelism for ViTs, enabling them to handle up to 1M tokens. Our approach, leveraging DeepSpeed-Ulysses and Long-Sequence-Segmentation with model sharding, is the first to apply sequence parallelism in ViT training, achieving a 94% batch scaling efficiency on 2,048 AMD-MI250X GPUs. Evaluating sequence parallelism in ViTs, particularly in models up to 10B parameters, highlighted substantial bottlenecks. We countered these with hybrid sequence, pipeline, and flash attention strategies, to scale beyond single GPU memory limits. Our method significantly enhances climate modeling accuracy by 20% in temperature predictions, marking the first training of a vision transformer model to convergence with a sequence length of 188K tokens, using full self-attention.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
DOE OSTI2024

Advancing attenuation estimation through integration of the Hessian in multiparameter viscoacoustic full-waveform inversion

Accurate seismic attenuation models of subsurface structures not only enhance subsequent migration processes by improving fidelity, resolution, and facilitating amplitude-compliant angle gather generation but also provide valuable constraints on subsurface physical properties. Leveraging full-wavefield information, multiparameter viscoacoustic full-waveform inversion ( Q-FWI) simultaneously estimates seismic velocity and attenuation ( Q) models. However, a major challenge in Q-FWI is the contamination of crosstalk artifacts, where inaccuracies in the velocity model are mistakenly mapped to the inverted attenuation model. While incorporating the Hessian is expected to mitigate these artifacts, the explicit implementation is prohibitively expensive due to its formidable computational cost. In this study, we formulate and develop a Q-FWI algorithm via the Newton-conjugate gradient (CG) framework, where the search direction at each iteration is determined through an internal CG loop. In particular, the Hessian is integrated into each CG step in a matrix-free fashion using the second-order adjoint-state method. We find through synthetic experiments that our Newton-CG Q-FWI significantly mitigates crosstalk artifacts compared with the limited-memory Broyden-Fletcher-Goldfarb-Shanno method and the CG method, albeit with a notable computational cost. In the discussion of several key implementation details, we also determine the significance of the approximate Gauss-Newton Hessian, the second-order adjoint-state method, and the two-stage inversion strategy.

Geochemistry & Geophysics↗
DOE OSTI2023

Multiphysics Simulations of MSRE with NEAMS Thermal Hydraulics Tools

This report documents the benchmarks being developed and simulations performed using tools and codes developed under the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, utilizing MSRE experimental data. In FY23, three main work scopes were investigated under the NEAMS MSR work package at ANL. The first scope investigated the Griffin-SAM coupling model for simulating the pump startup transient experiment of MSRE. The analyses start with a simple model (single-channel, single-lattice), gradually adding more details (multi-channel, full-core) into the model. The results show that the reactivity loss curve is very sensitive to the axial boundary conditions and the radial core discretization. The simple model can predict a similar reactivity trend as that of the more sophisticated model, which is likely due to error cancellation. Accurately modeling the axial boundary condition may further improve the reactivity trend but would require significant efforts to generate the mesh of the MSRE inlet and upper plenum. The core channel radial discretization for the Griffin-SAM coupled model also depends on the flow distribution. Given the complex geometry in the inlet plenum, the flow distribution needed to be calculated from CFD analysis, which was performed using the NekRS code. This analysis employed a MSRE CAD model developed by Copenhagen Atomics. The CAD model was disassembled to keep the inlet plenum region only, which was subsequently cleaned and modified so that the mesh generated is under the memory limit. The results are merged to a few radial regions to show that the flow rate is highest in the central region. This would be useful for future improvement of the Griffin-SAM coupling model of the MSRE core. The last task investigated is tritium transport modeling using the standalone SAM code. This task aimed to initiate the effort to demonstrate and validate the tritium transport model implemented in SAM. The preliminary investigation employed an MSRE model consisting of the primary loop. Three tritium transport pathways were examined including the retention in the graphite, the permeation through the HX tube wall, and the removal from the off-gas system. The results compare well with the MSRE data, but improvements are still needed on the initial conditions (i.e., the present state may not have reached equilibrium), the boundary conditions, the off-gas system modeling, and a better numerical strategy to reach the equilibrium state.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Towards reverse mode automatic differentiation of Kokkos-based codes

Derivative computation is a key component of optimization, sensitivity analysis, uncertainty quantification, and the solving of nonlinear problems. Automatic differentiation (AD) is a powerful technique for evaluating such derivatives, and in recent years, has been integrated into programming environments such as Jax, PyTorch, and TensorFlow to support derivative computations needed for training of machine learning models, facilitating wide-spread use of these technologies. The C++ language has become the de facto standard for scientific computing due to numerous factors, yet language complexity has made the wide-spread adoption of AD technologies for C++ difficult, hampering the incorporation of powerful differentiable programming approaches into C++ scientific simulations. This is exacerbated by the increasing emergence of architectures, such as GPUs, with limited memory capabilities and requiring massive thread-level concurrency. C++ AD tools must effectively use these environments to bring novel scientific simulations to next-generation DOE experimental and observational facilities. In this project, we investigated source transformation-based automatic differentiation using LLVM compiler infrastructure to automatically generate portable and efficient gradient computations of Kokkos-based code. We have demonstrated that our proposed strategy is feasible by investigating the usage of a prototype LLVM-based source transformation tool to generate gradients of simple functions made of sequences of simple Kokkos parallel regions. Speedups of up to 500x compared to Sacado were observed on NVIDIA V100 GPU.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Phase‐Change‐Memory Process at the Limit: A Proposal for Utilizing Monolayer Sb 2 Te 3

Abstract One central task of developing nonvolatile phase change memory (PCM) is to improve its scalability for high‐density data integration. In this work, by first‐principles molecular dynamics, to date the thinnest PCM material possible (0.8 nm), namely, a monolayer Sb 2 Te 3 , is proposed. Importantly, its SET (crystallization) process is a fast one‐step transition from amorphous to hexagonal phase without the usual intermediate cubic phase. An increased spatial localization of electrons due to geometrical confinement is found to be beneficial for keeping the data nonvolatile in the amorphous phase at the 2D limit. The substrate and superstrate can be utilized to control the phase change behavior: e.g., with passivated SiO 2 (001) surfaces or hexagonal Boron Nitride, the monolayer Sb 2 Te 3 can reach SET recrystallization in 0.54 ns or even as fast as 0.12 ns, but with unpassivated SiO 2 (001), this would not be possible. Besides, working with small volume PCM materials is also a natural way to lower power consumption. Therefore, the proposed PCM working process at the 2D limit will be an important potential strategy of scaling the current PCM materials for ultrahigh‐density data storage.

2D limit↗
DOE OSTI2022

MemHC: An Optimized GPU Memory Management Framework for Accelerating Many-body Correlation

The many-body correlation function is a fundamental computation kernel in modern physics computing applications, e.g., Hadron Contractions in Lattice quantum chromodynamics (QCD). This kernel is both computation and memory intensive, involving a series of tensor contractions, and thus usually runs on accelerators like GPUs. Existing optimizations on many-body correlation mainly focus on individual tensor contractions (e.g., cuBLAS libraries and others). In contrast, this work discovers a new optimization dimension for many-body correlation by exploring the optimization opportunities among tensor contractions. More specifically, it targets general GPU architectures (both NVIDIA and AMD) and optimizes many-body correlation’s memory management by exploiting a set of memory allocation and communication redundancy elimination opportunities: first, GPU memory allocation redundancy: the intermediate output frequently occurs as input in the subsequent calculations; second, CPU-GPU communication redundancy: although all tensors are allocated on both CPU and GPU, many of them are used (and reused) on the GPU side only, and thus, many CPU/GPU communications (like that in existing Unified Memory designs) are unnecessary; third, GPU oversubscription: limited GPU memory size causes oversubscription issues, and existing memory management usually results in near-reuse data eviction, thus incurring extra CPU/GPU memory communications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

COLLABORATIVE DEVELOPMENT PROJECTS - PHOTONIC MEMORY CONTROLLER MODULE (P-MCM)

As computational density for high-performance computing and big-data services continues to scale, performance scalability of next generation computing systems is becoming increasingly constrained by limitations in memory access, power dissipation and chip packaging. The processor-memory communication bottleneck, a major challenge in current multicore processors due to limited pin-out and power budget, presents a detrimental scaling barrier to data-intensive computing. A consortium team of small businesses and leading researchers that includes experts from photonics processor-memory architecture, III/V photonic laser design/fabrication, silicon photonics design/fabrication, photonics packaging and assembly, and FPGA-based high-performance memory controller IP development – to collaboratively develop a commercialization path for a Photonic Memory Controller Module (P-MCM).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

R-Adaptivity to Enable Compression of Elementary Computations in Extreme-Scale Finite Element Simulators

Modern computing systems are capable of exascale calculations, which are revolutionizing the development and application of high-fidelity numerical models in computational science and engineering. While these systems continue to grow in processing power, the available system memory has not increased commensurately, and electrical power consumption continues to grow. A predominant approach to limit the memory usage in large-scale applications is to exploit the abundant processing power and continually recompute many low-level simulation quantities, rather than storing them. However, this approach can adversely impact the throughput of the simulation and diminish the benefits of modern computing architectures. We present three novel contributions to reduce the memory burden while maintaining, and sometimes improving, performance in simulations based on finite element discretizations. The first contribution develops dictionary-based data compression schemes that detect and exploit the structure of the discretization, due to redundancies across the finite element mesh. While these schemes are shown to reduce memory requirements by more than 99% on meshes with large numbers of identical mesh cells, there are applications where this structure does not exist. The second contribution leverages a recently developed augmented Lagrangian optimization algorithm to enable r-adaptivity for meshes with the goal of enhancing the redundancies in the mesh. The third contribution extends these methods to patch-based linear solvers and preconditioners by compressing local matrices. Numerical results demonstrate the effectiveness of the proposed methods to detect, enhance and exploit mesh structure on a suite of examples inspired by large-scale applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Shape memory embolectomy devices and systems

An embolectomy device comprised of an expansion unit and a support unit is disclosed. The expansion unit can be actuated in response to one or more external stimuli, and the support unit, located proximately to the expansion unit, provides a force to hold the expansion unit in place and to further induce the expansion unit's radial expansion. The radial expansion of the expansion unit causes the expansion unit to physically contact a blood clot, enabling the blood clot to be removed. In some embodiments, the expansion unit can be fabricated from a shape memory polymer foam. In some embodiments the support unit can be fabricated from any elastic material including, without limitation, shape memory alloys.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Analysis of Vector Particle-In-Cell (VPIC) memory usage optimizations on cutting-edge computer architectures

Vector Particle-In-Cell (VPIC) is one of the fastest plasma simulation codes in the world, with particle numbers ranging from one trillion on the first petascale system, Roadrunner, to ten trillion particles on the more recent Blue Waters supercomputer. As supercomputers continue to grow rapidly in size, so too does the gap between computing capability and memory capability. Current memory systems limit VPIC simulations greatly as the maximum number of particles that can be simulated directly depends on the available memory. In this study, we present a suite of VPIC memory optimizations (i.e., particle weight, half-precision, and fixed-point optimizations) that enable a significant increase in the number of particles in VPIC simulations. Here, we assess the optimizations’ impact on memory and runtime performance for a suite of cutting-edge computer architectures such has the NVIDIA V100 GPU, the IBM Power9, and the Fujitsu A64FX architectures. Our optimizations enable a 31.25% reduction in memory usage and up to 40% increase in the number of particles. This paper extends our work on developing particle storage format optimizations Tan et al.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

UltraLiM: In-Memory Boolean Logic Architecture Using UltraRAM

Conventional computing architectures encounter ‘von Neumann’ and ‘memory wall’ bottlenecks which arise due to the back-and-forth data movement between the physically separate memory and processing units and the speed mismatch between them, respectively. These bottlenecks hurt both energy efficiency and the throughput of computing systems. To address these challenges, in-memory computing architectures have emerged as a promising alternative. They reduce the need for frequent data movement by executing different computing tasks inside the memory system. Here, we present UltraLiM, a logic-in-memory architecture using the UltraRAM-based memory system. UltraRAM holds the promise of developing a ‘universal memory’, overcoming the limitations of charge-based memories thanks to their non-volatile behavior with lower operating voltage. This work presents an in-memory computing architecture that integrates an UltraRAM-based memory array with a custom-designed peripheral circuitry. With this architecture, we can perform various in-memory Boolean logic operations (such as NOT, NAND, NOR, and XOR) in a single cycle. Leveraging the separate read-write paths in the UltraRAM-based memory array, we optimize read operations without encountering design conflicts. This optimization enhances the sense margin, enabling the use of simpler peripheral circuitry for in-memory logic operations.

Alam, Shamiul [University of Tennessee, Knoxville ↗
DOE OSTI2026

Linear complexity

We present factorization and solution phases for a new linear complexity direct solver designed for concurrent batch operations on fine-grained parallel architectures, for matrices amenable to hierarchical representation. We focus on the strong-admissibility-based $\mathscr{H}^{2}$ format, where strong recursive skeletonization factorization compresses remote interactions. We build upon previous implementations of $\mathscr{H}^{2}$ matrix construction for efficient factorization and solution algorithm design, which are illustrated graphically in stepwise detail. The algorithms are ‘blackbox’ in the sense that the only inputs are the matrix and right-hand side, without analytical or geometrical information about the origin of the system. We demonstrate linear complexity scaling in both time and memory on four representative families of dense matrices up to one million in size. Parallel scaling up to 16 threads is enabled by a multi-level matrix graph coloring and avoidance of dynamic memory allocations thanks to prefix-sum memory management. An experimental backward error analysis is included. We break down the timings of different phases, identify phases that are memory-bandwidth limited, and discuss alternatives for phases that may be sensitive to the trend to employ lower precisions for performance.

Boukaram, Wajih↗