Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “grid computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

WLCG Transition from X.509 to Tokens. Status, Plans, and Timeline

Since 2017, the Worldwide LHC Computing Grid (WLCG) has been working towards enabling token-based authentication and authorization throughout its entire middleware stack. Following the initial publication of the WLCG Token Schema v1.0 in 2019, OAuth2.0 token workflows have been integrated across grid middleware. There are many complex challenges to be addressed before the WLCG can be end-to-end token-based, including not just technical hurdles but also interoperability with the wider authentication and authorization landscape. This paper presents the status of the WLCG coordination and deployment work, and how it relates to software providers and partner communities. The authors also detail how the WLCG token transition timeline has progressed, and how it has changed since its publication.

97 MATHEMATICS AND COMPUTING↗

System-Level Virtualization Research at Oak Ridge National Laboratory

System-level virtualization is today enjoying a rebirth as a technique to effectively share what were then considered large computing resources to subsequently fade from the spotlight as individual workstations gained in popularity with a one machine - one user approach. One reason for this resurgence is that the simple workstation has grown in capability to rival that of anything available in the past. Thus, computing centers are again looking at the price/performance benefit of sharing that single computing box via server consolidation. However, industry is only concentrating on the benefits of using virtualization for server consolidation (enterprise computing) whereas our interest is in leveraging virtualization to advance high-performance computing (HPC). While these two interests may appear to be orthogonal, one consolidating multiple applications and users on a single machine while the other requires all the power from many machines to be dedicated solely to its purpose, we propose that virtualization does provide attractive capabilities that may be exploited to the benefit of HPC interests. This does raise the two fundamental questions of: is the concept of virtualization (a machine sharing technology) really suitable for HPC and if so, how does one go about leveraging these virtualization capabilities for the benefit of HPC. To address these questions, this document presents ongoing studies on the usage of system-level virtualization in a HPC context. These studies include an analysis of the benefits of system-level virtualization for HPC, a presentation of research efforts based on virtualization for system availability, and a presentation of research efforts for the management of virtual systems. The basis for this document was material presented by Stephen L. Scott at the Collaborative and Grid Computing Technologies meeting held in Cancun, Mexico on April 12-14, 2007.

99 GENERAL AND MISCELLANEOUS↗

FTS3: Data Movement Service in containers deployed in OKD [Slides]

The File Transfer Service (FTS3) is a data movement service developed at CERN which is used to distribute the majority of the Large Hadron Collider's data across the Worldwide LHC Computing Grid (WLCG) infrastructure. At Fermilab, we have deployed FTS3 instances for Intensity Frontier experiments (e.g. DUNE) to transfer data in America and Europe, using a container-based strategy. In this article we summarize our experience building docker images based on work from the SLATE project (slateci.io) and deployed in OKD, the community distribution of Red Hat OpenShift. Additionally, we discuss our method of certificate management and maintenance utilizing Kubernetes CronJobs. Finally, we also report on the two different configurations currently running at Fermilab, comparing and contrasting a Docker-based OKD deployment against a traditional RPM-based deployment.

97 MATHEMATICS AND COMPUTING↗

Supporting multiple hardware architectures at CMS: the integration and validation of POWER9

Computing resources in the Worldwide LHC Computing Grid (WLCG) have been based entirely on the x86 architecture for more than two decades. In the near future, however, heterogeneous non-x86 resources, such as ARM, POWER and Risc-V, will become a substantial fraction of the resources that will be provided to the LHC experiments, due to their presence in existing and planned world-class HPC installations. The CMS experiment, one of the four large detectors at the LHC, has started to prepare for this situation, with the CMS software stack (CMSSW) already compiled for multiple architectures. In order to allow for a production use, the tools for workload management and job distribution need to be extended to be able to exploit heterogeneous architectures. Profiting from the opportunity to exploit the first sizable IBM Power9 allocation available on Marconi100 HPC system at CINECA, CMS developed all the needed modifications to the CMS workload management system. After a successful proof of concept, a full physics validation has been performed in order to bring the system in production. The experiences are of very high value, when it comes to commissioning of the similar (even larger) Summit HPC system at Oak Ridge, where CMS is also expecting a resource allocation. Moreover the compute power of those systems is being provided also via GPUs and this represents an extremely valuable opportunity to exploit the offloading capability already implemented in CMSSW. The status of the current integration including the exploitation of the GPUs, the results of the validation as well as the future plans will be shown and discussed.

Boccali, Tommaso [INFN, Pisa]↗

A fully-integrated lattice Boltzmann method for fluid–structure interaction

Here we present a fully-integrated lattice Boltzmann (LB) method for fluid–structure interaction (FSI) simulations that efficiently models deformable solids in complex suspensions and active systems. Our Eulerian method (LBRMT) couples finite-strain solids to the LB fluid on the same fixed computational grid with the reference map technique (RMT). An integral part of the LBRMT is a new LB boundary condition for moving deformable interfaces across different densities. With this fully Eulerian solid–fluid coupling, the LBRMT is well-suited for parallelization and simulating multi-body contact without remeshing or extra meshes. We validate its accuracy via a benchmark of a deformable solid in a lid-driven cavity, then showcase its versatility through examples of soft solids rotating and settling. The LBRMT achieves a spatial convergence rate between first-order and second-order for FSI simulations and is designed for low to intermediate Reynolds number flows with finite inertia at small Mach numbers. With simulations of complex suspensions mixing, we highlight the potential of the LBRMT for studying collective behavior in soft matter and biofluid dynamics.

97 MATHEMATICS AND COMPUTING↗

A Fast Butterfly-Compressed Hadamard–Babich Integrator for High-Frequency Helmholtz Equations in Inhomogeneous Media with Arbitrary Sources

Here we present a butterfly-compressed representation of the Hadamard-Babich (HB) ansatz for the Green's function of the high-frequency Helmholtz equation in smooth inhomogeneous media. For a computational domain discretized with Nv discretization cells, the proposed algorithm first solves and tabulates the phase and HB coefficients via eikonal and transport equations with observation points and point sources located at the Chebyshev nodes using a set of much coarser computation grids, and then butterfly compresses the resulting HB interactions from all Nv cell centers to each other. The overall CPU time and memory requirement scale as O(Nv log2 Nv) for any bounded two-dimensional (2D) domains with arbitrary excitation sources. A direct extension of this scheme to bounded 3D domains yields an O(Nv4/3) CPU complexity, which can be further reduced to quasi-linear complexities with proposed remedies. The scheme can also efficiently handle scattering problems involving inclusions in inhomogeneous media. Although the current construction of our HB integrator does not accommodate caustics, the resulting HB integrator itself can be applied to certain sources, such as concave-shaped sources, to produce caustic effects. Compared to finite-difference frequency domain methods, the proposed HB integrator is free of numerical dispersion and requires fewer discretization points per wavelength. As a result, it can solve wave propagation problems well beyond the capability of existing solvers. Remarkably, the proposed scheme can accurately model wave propagation in 2D domains with 640 wavelengths per direction and in 3D domains with 54 wavelengths per direction on a state-of-the-art supercomputer at Lawrence Berkeley National Laboratory.

Hadamard--Babich ansatz↗

The ATLAS Simulation Infrastructure

The simulation software for the ATLAS Experiment at the Large Hadron Collider is being used for large-scale production of events on the LHC Computing Grid. This simulation requires many components, from the generators that simulate particle collisions, through packages simulating the response of the various detectors and triggers. All of these components come together under the ATLAS simulation infrastructure. In this paper, that infrastructure is discussed, including that supporting the detector description, interfacing the event generation, and combining the GEANT4 simulation of the response of the individual detectors. Also described are the tools allowing the software validation, performance testing, and the validation of the simulated output against known physics processes.

43 PARTICLE ACCELERATORS↗

Discretization limits of lattice‐Boltzmann methods for studying immiscible two‐phase flow in porous media

Summary Digital images of porous media often include features approaching the image resolution length scale. The behavior of numerical methods at low resolution is therefore important even for well‐resolved systems. We study the behavior of the Shan‐Chen (SC) and Rothman‐Keller (RK) multicomponent lattice‐Boltzmann models in situations where the fluid‐fluid interfacial radius of curvature and/or the feature size of the medium approaches the discrete unit size of the computational grid. Various simple, small‐scale test geometries are considered, and a drainage test is also performed in a Bentheimer sandstone sample. We find that both RK and SC models show very high ultimate limits: in ideal conditions the models can simulate static fluid configuration with acceptable accuracy in tubes as small as three lattice units across for RK model (six lattice units for SC model) and with an interfacial radius of curvature of two lattice units for RK and SC models. However, the stability of the models is affected when operating in these extreme discrete limits: in certain circumstances the models exhibit behaviors ranging from loss of accuracy to numerical instability. We discuss the circumstances where these behaviors occur and the ramifications for larger‐scale fluid displacement simulations in porous media, along with strategies to mitigate the most severe effects. Overall we find that the RK model, with modern enhancements, exhibits fewer instabilities and is more suitable for systems of low fluid‐fluid miscibility. The shortcomings of the SC model seem to arise predominantly from the high, strongly pressure‐dependent miscibility of the two fluid components.

Li, Zhe↗

Interface and embedded joint methods for modeling dynamic fracture opening by explosive products [Two methods for modeling dynamic fracture opening by explosive products]

Two computational approaches are proposed in the paper to model dynamic fracture opening by explosive products. The first method assumes that the fractures may be modeled using flow elements embedded along the mesh lines. This method models crack opening in a straightforward way by splitting the nodes of the computational grid. It can account for crack branching; however, the crack directions are constrained by existing mesh faces, which may lead to mesh dependence. Also, the stress in flow elements is calculated explicitly separate from the surrounding solid elements that can impose additional limits on the time step stability condition for explicit integration. The second approach uses embedded flow elements to model the cracks. Typical thickness of the cracks is much smaller than the element size. Therefore, gas pressure in the cracks is assumed to be in stress equilibrium with the element stress. To achieve this, the crack thickness and the state of the gas is updated simultaneously with the state of the solid element which contains the crack. Therefore, the time step is controlled by the explicit solver applied for the solid and does not depend on the thickness of the crack. In conclusion, the main disadvantage of the second approach is due to the complexity of modeling multiple intersecting cracks, which go through the same element. We discuss the areas of possible applications of these 2 methods and the ways to improve and enhance them for future practical applications.

58 GEOSCIENCES↗

Database for Validation of Design Load Extrapolation Techniques

Extrapolation techniques used for predicting long-term wind turbine loads have produced highly variable loading estimates dependent on the individual designer implementation. To reduce such variability, more precise definition and validation of these techniques are necessary. As part of a wider effort to test loads extrapolation techniques used in wind turbine design, two data sets were created. The first data set was created as an example typical of what designers use to extrapolate loads according to wind turbine design standards. The second data set was a series of year-long simulations that could be used to quantify the accuracy of extrapolation methods. Due to the large number of simulations, care was taken not to reproduce random samplings in individual time series. Additionally, a grid computing architecture was used to run the simulations in a reasonable time frame. The wind speeds where loads were highest were identified, which varied with loading type. The identification of dominant wind speeds is important to ensure an adequate number of simulations at such speeds, which often influence extrapolated loads. In-plane loads and deflections tended to be dominated by high wind speeds near cut-out wind speed. Out-of-plane loads and deflections were most influenced by winds that were near the rated wind speed. Some loads were influenced by a range of wind speeds. The loads dominated by high wind speeds had greater variability in the extreme values, which could be a reflection of the greatly varying wind and also greater sensitivity to higher energy content of the wind at such speeds.

17 WIND ENERGY↗

Dynamo: Handling Scientific Data Across Sites and Storage Media

Abstract Dynamo is a full-stack software solution for scientific data management. Dynamo’s architecture is modular, extensible, and customizable, making the software suitable for managing data in a wide range of installation scales, from a few terabytes stored at a single location to hundreds of petabytes distributed across a worldwide computing grid. This article documents the core system design of Dynamo and describes the applications that implement various data management tasks. A brief report is also given on the operational experiences of the system at the CMS experiment at the CERN Large Hadron Collider and at a small-scale analysis facility.

Iiyama, Yutaro (ORCID:0000000282975930)↗

A particle resolved simulation approach for studying shock interactions with moving, colliding solid particles

This work applies a new combination of techniques for the fully resolved simulation of compressible, gas–particle multiphase flows. Here, the adaptive wavelet collocation method is used to dynamically, and efficiently, adapt the computational grid to localized flow features and the particles. A characteristic-based volume penalization method that imposes arbitrary Dirichlet, Neumann, or Robin-type immersed boundary conditions, is used to enforce the no-slip condition at particle surfaces. A hard-sphere collision model is applied to capture the particle–particle collisions. Proof of concept test cases are presented, showcasing the dynamic grid adaptation and fully resolved two-way coupling between the phases that is possible with this approach. Results for a shock-driven single cylinder under viscous and inviscid conditions are presented along with a demonstration of a shock interacting with a cloud of randomly distributed cylinders and spheres.

42 ENGINEERING↗

Fully Coupled Well Models for Fluid Injection and Production

Wells are the primary engineered component of geologic sequestration systems with deep subsurface reservoirs. Wells provide a conduit for injecting greenhouse gases and producing reservoirs fluids, such as brines, natural gas, and crude oil, depending on the target reservoir. Well trajectories, well pressures, and fluid flow rates are parameters over which well engineers and operators have control during the geologic sequestration process. Current drilling practices provided well engineers flexibility in designing well trajectories and controlling screened intervals. Injection pressures and fluids can be used to purposely fracture the reservoir formation or to purposely prevent fracturing. Numerical simulation of geologic sequestration processes involves the solution of multifluid transport equations within heterogeneous geologic media. These equations that mathematically describe the flow of fluid through the reservoir formation are nonlinear in form, requiring linearization techniques to resolve. In actual geologic settings fluid exchange between a well and reservoir is a function of local pressure gradients, fluid saturations, and formation characteristics. In numerical simulators fluid exchange between a well and reservoir can be specified using a spectrum of approaches that vary from totally ignoring the reservoir conditions to fully considering reservoir conditions and well processes. Well models are a numerical simulation approach that account for local conditions and gradients in the exchange of fluids between the well and reservoir. As with the mathematical equations that describe fluid flow in the reservoir, variation in fluid properties with temperature and pressure yield nonlinearities in the mathematical equations that describe fluid flow within the well. To numerically simulate the fluid exchange between a well and reservoir the two systems of nonlinear multifluid flow equations must be resolved. The spectrum of numerical approaches for resolving these equations varies from zero coupling to full coupling. In this paper we describe a fully coupled solution approach for well model that allows for a flexible well trajectory and screened interval within a structured hexahedral computational grid. In this scheme the nonlinear well equations have been fully integrated into the Jacobian matrix for the reservoir conservation equations, minimizing the matrix bandwidth.

coupled well model↗

Parameterizing hot electron energy distributions for tabular emissivities and opacities

Cold Kα and Kβ emission spectroscopy is a powerful diagnostic for plasma conditions and distributions of energetic electrons produced in short pulse laser experiments. SPECT3D post-processes hydrodynamics code output and simulates high-resolution spectra and images for plasmas containing arbitrary distributions of hot electrons, however these calculations can become prohibitively expensive for plasmas requiring complex atomic data and detailed computational grids. To significantly increase calculation speed, it is desirable to use pre-configured opacity tables that include information about hot electrons, rather than calculating emissivity and opacity for every grid point by solving collisional-radiative atomic kinetic rate equations. However, to tabulate such opacities, a general parameter-based description of hot electron energy distributions must be found. To be practical, hot electrons should be represented with an analytic function of just a few parameters which, in addition to temperature and density, can be used to define parameter space for opacity tables. In this paper, we present a method that allows for describing a set of arbitrary binned hot electron distributions with a simple analytic function, with only a modest impact on the accuracy of the simulations. Increased calculation speeds of 50–100x are demonstrated. In conclusion, we also discuss details of opacity calculations and present synthetic spectroscopy data based on post-processing of detailed particle-in-cell simulations relevant to the ongoing experimental campaign at the Omega EP laser facility.

74 ATOMIC AND MOLECULAR PHYSICS↗

Feasibility of a Multigroup Deterministic Solution Method for Three-Dimensional Radiotherapy Dose Calculations

Purpose: To investigate the potential of a novel deterministic solver, Attila, for external photon beam radiotherapy dose calculations. Methods and Materials: Two hypothetical cases for prostate and head-and-neck cancer photon beam treatment plans were calculated using Attila and EGSnrc Monte Carlo simulations. Open beams were modeled as isotropic photon point sources collimated to specified field sizes. The sources had a realistic energy spectrum calculated by Monte Carlo for a Varian Clinac 2100 operated in a 6-MV photon mode. The Attila computational grids consisted of 106,000 elements, or 424,000 spatial degrees of freedom, for the prostate case, and 123,000 tetrahedral elements, or 492,000 spatial degrees of freedom, for the head-and-neck cases. Results: For both cases, results demonstrate excellent agreement between Attila and EGSnrc in all areas, including the build-up regions, near heterogeneities, and at the beam penumbra. Dose agreement for 99% of the voxels was within the 3% (relative point-wise difference) or 3-mm distance-to-agreement criterion. Localized differences between the Attila and EGSnrc results were observed at bone and soft-tissue interfaces and are attributable to the effect of voxel material homogenization in calculating dose-to-medium in EGSnrc. For both cases, Attila calculation times were <20 central processing unit minutes on a single 2.2-GHz AMD Opteron processor. Conclusions: The methods in Attila have the potential to be the basis for an efficient dose engine for patient-specific treatment planning, providing accuracy similar to that obtained by Monte Carlo.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Wave energy resources along the Hawaiian Island chain

Hawaii is subject to direct approach of swells from distant storms as well as seas generated by trade winds all year round. The island chain creates a localized weather system that modifies the wave energy resources from the far field. Here, we implement a nested computational grid across the major Hawaiian Islands at 3-min resolution in the global WaveWatch3 (WW3) model and utilize the Weather Research Forecast (WRF) model to provide high-resolution mesoscale wind forcing over the Hawaii region. Two hindcast studies representative of the year-round conditions provide a quantitative assessment of the regional wind and wave patterns as well as the wave energy resources along the Hawaiian Island chain. These events of approximately two weeks each have a range of wind speeds, ground swells, and wind waves for validation of the modeling system with satellite and buoy measurements prior to its general application.

13 HYDRO ENERGY↗

Development of a Filtered CFD-DEM Drag Model with Multiscale Markers Using an Artificial Neural Network and Nonlinear Regression

Here, the accuracy of coarse-grained Euler-Lagrangian simulations of fluidized beds heavily depends on the mesoscale drag models to account for the influences of the unresolved sub-grid structures. Traditional filtered drag models are regressed with mesoscale markers such as voidage and slip velocities. In this research, a filtered drag was regressed with both mesoscale and macro-scale markers using fine grid Computational Fluid Dynamics - Discrete Element Method (CFD-DEM) simulations. The traditional non-linear regression method was compared with machine learning regression using an Artificial Neural Network (ANN) implemented in PyTorch and coupled with MFiX. The new drag showed higher accuracy than the Wen-Yu drag and another filtered drag derived from the two-fluid model. The nonlinear regression shows slightly better results than ANN regression in cases with similar R 2 values. The utilization of the gas inlet velocity as an additional macro-scale marker reduced the errors by up to 55.3% in the tested cases.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Microstructure, Transport, and Mechanics of Compacted Clay Simulated at the 0.1 μm Scale (1400 Smectite Clay Particles) Using a Coarse-Grained Model with Explicit Counterions

Clay-rich geomaterials play a critical role in many subsurface systems. The macroscale properties of these materials (low permeability, high ionic conductivity, high swelling pressure, etc.) are sensitive to molecular-level adsorption and hydration interactions at clay−water interfaces. Efforts to develop multiscale simulation approaches to predict these properties reveal a scale gap between atomistic simulations (typically limited to systems smaller than 10 nm) and continuum-scale models (which use computational grid elements with dimensions ≳ 10 μm). In this study, we present a coarse-grained (CG) framework that partly bridges this gap by simulating compacted smectite clay assemblages with dimensions of 0.1 μm containing 1,400 clay particles across a range of dry densities (1,050 to 1,850 kg·m −3 ) and Na/Ca counterion compositions (Na fraction ranging from 0.2 to 1). The simulated systems, along with their reconstructed binary three-dimensional pore networks, are used to evaluate the microstructure, pore size distribution, tortuosity, ion diffusivity, and swelling pressure of compacted smectite clay. Results show that our approach captures important features of the mesoscale heterogeneity of compacted clays, including tactoid formation, hierarchical porosity, and anisotropic pore networks. Results also reveal how compaction and counterion composition govern emergent behaviors, including dominant pore sizes, directional transport, and electrochemical response. This work highlights the potential of CG simulations to bridge molecular and continuum scales and to advance geotechnical and environmental applications involving clay-rich materials as well as related nanoporous media such as geopolymers and calcium-silicate-hydrate. However, the results also suggest that accurate prediction of certain microstructural and mechanical properties (e.g., swelling pressure) may require even larger-scale systems on the order of 1 μm.

02 PETROLEUM↗