Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “limited memory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

LATTE: Los Alamos TravelTime package based on Eikonal equation

This Fortran code focuses on traveltime computation and tomography based on eikonal equation. Specifically, the package provides three major functionalities: (1) forward modeling of traveltime from single-point or ensemble source based on factorized eikonal equation, (2) adjoint-state first-arrival traveltime tomography based on picked first arrival traveltime using steepest descent, conjugate gradient, or limited-memory BFGS inversion scheme, and (3) adjoint-state joint transmission-reflection tomography based on picked first-arrival and reflection traveltimes. The package applies to forward modeling and tomography based on traveltime in 2D and 3D isotropic regular-grid models. We name this package LATTE – Los Alamos TravelTime package based on Eikonal equation. * The code is for accompanying a journal paper under preparation. The paper will be submitted via LA-UR separately later.

Gao, Kai↗

Sequence length scaling in vision transformers for scientific images on frontier

Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence lengths. While transformers for text have inspired scaling sequence lengths in ViTs, adapting these for ViTs introduces unique challenges. We develop distributed sequence parallelism for ViTs, enabling them to handle up to 1M tokens. Our approach, leveraging DeepSpeed-Ulysses and Long-Sequence-Segmentation with model sharding, is the first to apply sequence parallelism in ViT training, achieving a 94% batch scaling efficiency on 2,048 AMD-MI250X GPUs. Evaluating sequence parallelism in ViTs, particularly in models up to 10B parameters, highlighted substantial bottlenecks. We countered these with hybrid sequence, pipeline, and flash attention strategies, to scale beyond single GPU memory limits. Our method significantly enhances climate modeling accuracy by 20% in temperature predictions, marking the first training of a vision transformer model to convergence with a sequence length of 188K tokens, using full self-attention.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗

Advancing attenuation estimation through integration of the Hessian in multiparameter viscoacoustic full-waveform inversion

Accurate seismic attenuation models of subsurface structures not only enhance subsequent migration processes by improving fidelity, resolution, and facilitating amplitude-compliant angle gather generation but also provide valuable constraints on subsurface physical properties. Leveraging full-wavefield information, multiparameter viscoacoustic full-waveform inversion ( Q-FWI) simultaneously estimates seismic velocity and attenuation ( Q) models. However, a major challenge in Q-FWI is the contamination of crosstalk artifacts, where inaccuracies in the velocity model are mistakenly mapped to the inverted attenuation model. While incorporating the Hessian is expected to mitigate these artifacts, the explicit implementation is prohibitively expensive due to its formidable computational cost. In this study, we formulate and develop a Q-FWI algorithm via the Newton-conjugate gradient (CG) framework, where the search direction at each iteration is determined through an internal CG loop. In particular, the Hessian is integrated into each CG step in a matrix-free fashion using the second-order adjoint-state method. We find through synthetic experiments that our Newton-CG Q-FWI significantly mitigates crosstalk artifacts compared with the limited-memory Broyden-Fletcher-Goldfarb-Shanno method and the CG method, albeit with a notable computational cost. In the discussion of several key implementation details, we also determine the significance of the approximate Gauss-Newton Hessian, the second-order adjoint-state method, and the two-stage inversion strategy.

Geochemistry & Geophysics↗

Stan : A Probabilistic Programming Language

Stan is a probabilistic programming language for specifying statistical models. A Stan program imperatively defines a log probability function over parameters conditioned on specified data and constants. As of version 2.14.0, Stan provides full Bayesian inference for continuous-variable models through Markov chain Monte Carlo methods such as the No-U-Turn sampler, an adaptive form of Hamiltonian Monte Carlo sampling. Penalized maximum likelihood estimates are calculated using optimization methods such as the limited memory Broyden-Fletcher-Goldfarb-Shanno algorithm. Stan is also a platform for computing log densities and their gradients and Hessians, which can be used in alternative algorithms such as variational Bayes, expectation propagation, and marginal inference using approximate integration. To this end, Stan is set up so that the densities, gradients, and Hessians, along with intermediate quantities of the algorithm such as acceptance probabilities, are easily accessible. Stan can also be called from the command line using the cmdstan package, through R using the rstan package, and through Python using the pystan package. All three interfaces support sampling and optimization-based inference with diagnostics and posterior analysis. rstan and pystan also provide access to log probabilities, gradients, Hessians, parameter transforms, and specialized plotting.

97 MATHEMATICS AND COMPUTING↗

High-Order Preconditioners for Modern Computing Architectures: Line-Based Discontinuous Galerkin Method on GPUs

The goals of our project are to be able to overcome issues that plague computing in modern architecture such as limited memory bandwidth, badly conditioned Jacobians, and restrictive time steps. We begin by describing the line-based discontinuous Galerkin (line DG) method for a scalar two-dimensional problem. Line DG has been shown to provide a significant increase in sparsity for Jacobian matrix, especially in the diagonal block elements. In addition to this, the implicit time integration scheme helps alleviate the time step restriction but requires solving a (potentially badly conditioned) linear system. We can precondition the linear system to facilitate the solution. Since we want to solve the system on the GPU, the increase in sparsity from the line DG method is helpful to alleviate the memory issue. We successfully implemented a computation of the Jacobian on the CPU and are currently working on the GPU implementation. Finally, I discuss how this opportunity to intern at Lawrence Livermore National Laboratory through the National Science Foundation Mathematical Sciences Graduate Internship has affected my personal and professional growth.

97 MATHEMATICS AND COMPUTING↗

Multiphysics Simulations of MSRE with NEAMS Thermal Hydraulics Tools

This report documents the benchmarks being developed and simulations performed using tools and codes developed under the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, utilizing MSRE experimental data. In FY23, three main work scopes were investigated under the NEAMS MSR work package at ANL. The first scope investigated the Griffin-SAM coupling model for simulating the pump startup transient experiment of MSRE. The analyses start with a simple model (single-channel, single-lattice), gradually adding more details (multi-channel, full-core) into the model. The results show that the reactivity loss curve is very sensitive to the axial boundary conditions and the radial core discretization. The simple model can predict a similar reactivity trend as that of the more sophisticated model, which is likely due to error cancellation. Accurately modeling the axial boundary condition may further improve the reactivity trend but would require significant efforts to generate the mesh of the MSRE inlet and upper plenum. The core channel radial discretization for the Griffin-SAM coupled model also depends on the flow distribution. Given the complex geometry in the inlet plenum, the flow distribution needed to be calculated from CFD analysis, which was performed using the NekRS code. This analysis employed a MSRE CAD model developed by Copenhagen Atomics. The CAD model was disassembled to keep the inlet plenum region only, which was subsequently cleaned and modified so that the mesh generated is under the memory limit. The results are merged to a few radial regions to show that the flow rate is highest in the central region. This would be useful for future improvement of the Griffin-SAM coupling model of the MSRE core. The last task investigated is tritium transport modeling using the standalone SAM code. This task aimed to initiate the effort to demonstrate and validate the tritium transport model implemented in SAM. The preliminary investigation employed an MSRE model consisting of the primary loop. Three tritium transport pathways were examined including the retention in the graphite, the permeation through the HX tube wall, and the removal from the off-gas system. The results compare well with the MSRE data, but improvements are still needed on the initial conditions (i.e., the present state may not have reached equilibrium), the boundary conditions, the off-gas system modeling, and a better numerical strategy to reach the equilibrium state.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Towards reverse mode automatic differentiation of Kokkos-based codes

Derivative computation is a key component of optimization, sensitivity analysis, uncertainty quantification, and the solving of nonlinear problems. Automatic differentiation (AD) is a powerful technique for evaluating such derivatives, and in recent years, has been integrated into programming environments such as Jax, PyTorch, and TensorFlow to support derivative computations needed for training of machine learning models, facilitating wide-spread use of these technologies. The C++ language has become the de facto standard for scientific computing due to numerous factors, yet language complexity has made the wide-spread adoption of AD technologies for C++ difficult, hampering the incorporation of powerful differentiable programming approaches into C++ scientific simulations. This is exacerbated by the increasing emergence of architectures, such as GPUs, with limited memory capabilities and requiring massive thread-level concurrency. C++ AD tools must effectively use these environments to bring novel scientific simulations to next-generation DOE experimental and observational facilities. In this project, we investigated source transformation-based automatic differentiation using LLVM compiler infrastructure to automatically generate portable and efficient gradient computations of Kokkos-based code. We have demonstrated that our proposed strategy is feasible by investigating the usage of a prototype LLVM-based source transformation tool to generate gradients of simple functions made of sequences of simple Kokkos parallel regions. Speedups of up to 500x compared to Sacado were observed on NVIDIA V100 GPU.

97 MATHEMATICS AND COMPUTING↗

CRIC and Rucio integration for CMS Data Management

The Large Hadron Collider (LHC) at CERN smashes protons together at close to the speed of light with seven times the energy of the most powerful accelerators built up to now. Some of the collision energy is turned into mass, creating new particles which are observed in the Compact Muon Solenoid (CMS) particle detector. CMS data is analyzed by scientists around the world to build up a picture of what happened at the heart of the collision. This will help us answer questions such as: ”what is the Universe really made of and what forces act within it?” and ”what gives everything substance?”. All the data produced by the experiment are collected and stored in a distributed system, since it must provide an high throughput and performing access to these data to scientists and physicists spread all over the world. In order to answer the previous questions and many others, CMS users must first obtain a copy of a portion of these data, and then to analyze it, therefore the first challenge is to decide in which storage site to save this duplicate so they can access it as quickly as possible. The second issue is each storage site shares a limited memory space with many CMS users, and each one needs to be assigned to a well defined personal amount of storage to pin data he needs. The main goal of my Summer Internship is to make scalable and automatic this user-site mapping which is based on a preexisting policy, proving a structured and organized algorithm to accomplish this task. Hence we are going to define, and solve, the main challenges encountered, and the technical solution used to generalize the algorithm to handle different policies for different countries.

43 PARTICLE ACCELERATORS↗

Poblano v1.0 : a Matlab toolbox for gradient-based optimization.

We present Poblano v1.0, a Matlab toolbox for solving gradient-based unconstrained optimization problems. Poblano implements three optimization methods (nonlinear conjugate gradients, limited-memory BFGS, and truncated Newton) that require only first order derivative information. In this paper, we describe the Poblano methods, provide numerous examples on how to use Poblano, and present results of Poblano used in solving problems from a standard test collection of unconstrained optimization problems.

99 GENERAL AND MISCELLANEOUS↗

Land surface model parameter optimisation using in situ flux data: comparison of gradient-based versus random search algorithms (a case study using ORCHIDEE v1.9.5.2)

Land surface models (LSMs), which form the land component of earth system models, rely on numerous processes for describing carbon, water and energy budgets, often associated with highly uncertain parameters. Data assimilation (DA) is a useful approach for optimising the most critical parameters in order to improve model accuracy and refine future climate predictions. In this study, we compare two different DA methods for optimising the parameters of seven plant functional types (PFTs) of the ORCHIDEE LSM using daily averaged eddy-covariance observations of net ecosystem exchange and latent heat flux at 78 sites across the globe. We perform a technical investigation of two classes of minimisation methods – local gradient-based (the L-BFGS-B algorithm, limited memory Broyden–Fletcher–Goldfarb–Shanno algorithm with bound constraints) and global random search (the genetic algorithm) – by evaluating their relative performance in terms of the model–data fit and the difference in retrieved parameter values. We examine the performance of each method for two cases: when optimising parameters at each site independently (“single-site” approach) and when simultaneously optimising the model at all sites for a given PFT using a common set of parameters (“multi-site” approach). We find that for the single site case the random search algorithm results in lower values of the cost function (i.e. lower model–data root mean square differences) than the gradient-based method; the difference between the two methods is smaller for the multi-site optimisation due to a smoothing of the cost function shape with a greater number of observations. The spread of the cost function, when performing the same tests with 16 random first-guess parameters, is much larger with the gradient-based method, due to the higher likelihood of being trapped in local minima. When using pseudo-observation tests, the genetic algorithm results in a closer approximation of the true posterior parameter value in the L-BFGS-B algorithm. We demonstrate the advantages and challenges of different DA techniques and provide some advice on using it for the LSM parameter optimisation.

58 GEOSCIENCES↗

An optimized treatment for algorithmic differentiation of an important glaciological fixed-point problem

We apply an optimized method to the adjoint generation of a time-evolving land ice model through algorithmic differentiation (AD). The optimization involves a special treatment of the fixed-point iteration required to solve the nonlinear stress balance, which differs from a straightforward application of AD software, and leads to smaller memory requirements and in some cases shorter computation times of the adjoint. The optimization is done via implementation of the algorithm of Christianson (1994) for reverse accumulation of fixed-point problems, with the AD tool OpenAD. For test problems, the optimized adjoint is shown to have far lower memory requirements, potentially enabling larger problem sizes on memory-limited machines. In the case of the land ice model, implementation of the algorithm allows further optimization by having the adjoint model solve a sequence of linear systems with identical (as opposed to varying) matrices, greatly improving performance. Finally, the methods introduced here will be of value to other efforts applying AD tools to ice models, particularly ones which solve a hybrid shallow ice/shallow shelf approximation to the Stokes equations.

58 GEOSCIENCES↗

A technique for interpretation of multispectral remote sensor data

The author has identified the following significant results. The U.S. Army Engineer Waterways Experiment Station is engaged in a study to detect from ERTS-1 satellite data alterations to the absorption and scattering properties caused by movement of suspended particles and solutes in selected areas of the Chesapeake Bay and to correlate the data to determine the feasibility of delineating flow patterns, flushing action of the estuary, and sediment and pollutant dispersion. As a part of this study, ADP techniques have been developed that permit automatic interpretation of data from any multispectral remote sensor with computer systems which have limited memory capacity and computing speed. The multispectral remote sensor is considered as a reflectance spectrophotometer. The data which define the spectral reflectance characteristics of a scene are scanned pixel by pixel. Each pixel whose spectral reflectance matches a reference spectrum is identified, and the results are shown in a map that identifies the locations where spectrum matches were detected and spectrum that was matched. The interpretation technique is described and an example of interpreted data from ERTS-1 is presented.

Williamson, A. N.↗

A model of the human in a cognitive prediction task.

The human decision maker's behavior when predicting future states of discrete linear dynamic systems driven by zero-mean Gaussian processes is modeled. The task is on a slow enough time scale that physiological constraints are insignificant compared with cognitive limitations. The model is basically a linear regression system identifier with a limited memory and noisy observations. Experimental data are presented and compared to the model.

Rouse, W. B.↗

Models of man as a suboptimal predictor

Models of man making predictions of future states of discrete linear dynamic systems are considered. The task is forced-pace, but the pace is slow enough to eliminate the effects of reaction time and neuromuscular lag. The best of the several models considered includes the constraints of limited memory and observation noise.

Rouse, W. B.↗

Accuracy of estimating the masses of Phobos and Deimos from multiple Viking orbiter encounters

The problem was investigated of estimating the masses of Phobos and Deimos from Doppler and onboard optical measurements during the Viking extended mission. A Kalman filter was used to analyze the effects of gravitational uncertainties and nongravitational accelerations. These accelerations destroy the dynamical integrity of the orbit, and multibatch or limited memory filtering is preferred to single batch processing. Optical tracking is essential to improve the relative orbit geometry. The masses can be determined to about 10% and 25% respectively for Phobos and Deimos, assuming satellite densities of about 3 gr/cu cm.

Tolson, R. H.↗

Accuracy of estimating the masses of Phobos and Deimos from multiple Viking orbiter encounters

This paper addresses the problem of estimating the masses of Phobos and Deimos from Doppler and onboard optical measurements during the Viking extended mission. A Kalman filter is used to analyze the effects of gravitational uncertainties and nongravitational accelerations. These accelerations destroy the dynamical integrity of the orbit and multi-batch or limited memory filtering is preferred to single batch processing. Optical tracking is essential to improve the relative orbit geometry. The masses can be determined to about 10% and 25% respectively for Phobos and Deimos, assuming satellite densities of about 3 g per cu cm.

Tolson, R. H.↗

Conservative zonal schemes for patched grids in 2 and 3 dimensions

The computation of flow over complex geometries, such as realistic aircraft configurations, poses difficult grid generation problems for computational aerodynamicists. The creation of a traditional, single-module grid of acceptable quality about an entire configuration may be impossible even with the most sophisticated of grid generation techniques. A zonal approach, wherein the flow field is partitioned into several regions within which grids are independently generated, is a practical alternative for treating complicated geometries. This technique not only alleviates the problems of discretizing a complex region, but also facilitates a block processing approach to computation thereby circumventing computer memory limitations. The use of such a zonal scheme, however, requires the development of an interfacing procedure that ensures a stable, accurate, and conservative calculation for the transfer of information across the zonal borders.

Hessenius, Kristin A.↗

System identification in the microcomputer environment

An implementation of an algorithm for system identification, the eigenvalue realization algorithm (ERA), on a microcomputer is reported. Test results are presented to show that accurate structural identification can be performed in a microcomputer environment. It is also shown that the memory limits (640 K) of small machines are generally adequate for solving large scale problems (e.g., Hankel matrices with 15,000 elements), with run times being on the order of several minutes.

Garcia, Ephrahim↗