Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “grid computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Networking: the view from HEP

Networks have played a critical role in high-energy physics (HEP), enabling us to access and effectively utilize globally distributed resources to meet the needs of our physicists. National and global-scale collaborations that characterize HEP would not be feasible without ubiquitous capable networks. Because of their importance in enabling our grid computing infrastructure many physicists have taken leading roles in research and education (R&E) networking, participating in, and even convening, network related meetings and research programs with the broader networking community worldwide. This has led to HEP benefiting from excellent global networking capabilities for little to no direct cost. However, as other science domains ramp-up their need for similar networking it becomes less clear that this situation will continue unchanged. This paper will briefly discuss the history of networking in HEP, the current activities and challenges we are facing, and try to provide some understanding of where networking may be going in the next 5 to 10 years.

97 MATHEMATICS AND COMPUTING↗

Networks in ATLAS

Networks have played a critical role in high-energy physics (HEP), enabling us to access and effectively utilize globally distributed resources to meet the needs of our physicists. Because of their importance in enabling our grid computing infrastructure many physicists have taken leading roles in research and education (R&E) networking, participating in, and even convening, network related meetings and research programs with the broader networking community worldwide. This has led to HEP benefiting from excellent global networking capabilities for little to no direct cost. However, as other science domains ramp-up their need for similar networking it becomes less clear that this situation will continue unchanged. What this means for ATLAS in particular needs to be understood. ATLAS has evolved its computing model since the LHC started based upon its experience with using globally distributed resources. The most significant theme of those changes has been increased reliance upon, and use of, its networks. We will report on a number of networking initiatives in ATLAS including participation in the global perfSONAR network monitoring and measuring efforts of WLCG and OSG, the collaboration with the LHCOPN/LHCONE effort, the integration of network awareness into PanDA, the use of the evolving ATLAS analytics framework to better understand our networks and the changes in our DDM system to allow remote access to data.

97 MATHEMATICS AND COMPUTING↗

The machine/job features mechanism

Within the HEPiX virtualization group and the Worldwide LHC Computing Grid’s Machine/Job Features Task Force, a mechanism has been developed which provides access to detailed information about the current host and the current job to the job itself. This allows user payloads to access meta information, independent of the current batch system or virtual machine model. The information can be accessed either locally via the filesystem on a worker node, or remotely via HTTP(S) from a webserver. This paper describes the final version of the specification from 2016 which was published as an HEP Software Foundation technical note, and the design of the implementations of this version for batch and virtual machine platforms. We discuss early experiences with these implementations and how they can be exploited by experiment frameworks.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

General relativistic simulations of collapsing binary neutron star mergers with Monte Carlo neutrino transport

Recent gravitational wave observations of neutron-star-neutron-star and neutron-star-black-hole binaries appear to indicate that massive neutron stars may not be too uncommon in merging systems. These discoveries have led to an increased interest in the simulation of merging compact binaries involving massive stars. In this paper, we present a first set of evolution of massive neutron star binaries using Monte Carlo radiation transport for the evolution of neutrinos. Here, we study a range of systems, from nearly symmetric binaries that collapse to a black hole before forming a disk or ejecting material, to more asymmetric binaries in which tidal disruption of the lower mass star leads to the production of more interesting postmerger remnants. For the latter type of systems, we additionally study the impact of viscosity on the properties of the outflows, and compare our results to two recent simulations of identical binaries performed with the whiskyTHC code. We find agreement on the black hole properties, disk mass, and mass and velocity of the outflows within expected numerical uncertainties, and some minor but noticeable differences in the evolution of the electron fraction when using a subgrid viscosity model, with viscosity playing a more minor role in our simulations. The method used to account for r-process heating in the determination of the outflow properties appears to have a larger impact on our result than those differences between numerical codes. We also use the simulation with the most ejected material to verify that our newly implemented Lagrangian tracers provide a reasonable sampling of the matter outflows as they leave the computational grid. We note that, given the lack of production of hot outflows in these mergers, the main role of neutrinos in these systems is to set the composition of the postmerger remnant. One of the main potential uses of our simulations is, thus, as improved initial conditions for longer evolutions of such remnants.

79 ASTRONOMY AND ASTROPHYSICS↗

Multipatch methods in general relativistic astrophysics: Hydrodynamical flows on fixed backgrounds

Many systems of interest in general relativistic astrophysics, including neutron stars, accreting compact objects in x-ray binaries and active galactic nuclei, core collapse, and collapsars, are assumed to be approximately spherically symmetric or axisymmetric. In Newtonian or fixed-background relativistic approximations it is common practice to use spherical polar coordinates for computational grids; however, these coordinates have singularities and are difficult to use in fully relativistic models. We present, in this series of papers, a numerical technique which is able to use effectively spherical grids by employing multiple patches. We provide detailed instructions on how to implement such a scheme, and present a number of code tests for the fixed-background case, including an accretion torus around a black hole.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

An Accurate Fast Fluid Dynamics Model for Data Center Applications

Traditional CFD is broadly useful in the design and operation of reliable and efficient data centers. It is nevertheless computationally expensive, particularly when employed for design optimization, which usually requires multiple simulations. To speed-up calculations while retaining essential airflow physics, several researchers have turned to an alternative CFD methodology, namely, fast fluid dynamics (FFD). FFD has been reported to be much faster than traditional CFD, but at an assumed (acceptable) trade-off of reduced accuracy. However, a recent comparison of FFD and traditional CFD for data center plenum applications produced nearly indistinguishable results and suggested that previously-reported FFD/traditional-CFD differences were due primarily to inconsistent: 1) advection schemes, 2) computational grids, and 3) turbulence models. The present paper extends this FFD/traditional-CFD comparison to the data center whitespace and confirms the finding that a true like-for-like comparison produces nearly identical predictions. Our FFD implementation utilizes a first-order upwind finite-volume scheme for advection like traditional CFD, so alternative advection schemes (e.g., semi-Lagrangian) are not considered further here. Likewise, the effect of grid choice on traditional-CFD predictions is well known. However, the effect of turbulence model for data center applications has not been reported extensively so we do consider this topic further here. We compare the standard k-e and a simpler algebraic model to benchmark experimental data from a real data center. We find that the algebraic turbulence model predicts rack-inlet temperatures at a similar level-of-accuracy as the k-e model for our fairly-simple-airflow reference data center.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Quantifying the effects of varietal types × management on the spatial variability of sorghum biomass across US environments

Abstract Regional‐scale estimations of sorghum biomass production allow identification of optimum genotype×environment×management (G×E×M) combinations for bioenergy generation. The objective of this study was to determine the degree of contributions of G, E, and M toward variability in sorghum biomass in the United States. Using the Agricultural Production Systems sIMulator in a grid computing platform, biomass was simulated for irrigated and rainfed conditions for 30 years across the United States for four sorghum varietal types (grain—GS, sudangrass—SS, photosensitive—PS, and photo‐insensitive—PI). Simulated biomass was assessed by environments clustered using the sum of intercepted solar radiation ( ir ), mean of temperature stress factor ( tp ) and water stress factor ( sw ). Simulated biomass ranged from 5.8 t ha −1 (GS‐rainfed) to 27.5 t ha −1 (PI‐irrigated). Under high‐temperature environments (mean annual temperature = 25°C), rainfed biomass between 40 and 80 days after planting (DAP) was strongly correlated with sw ( r = 0.64–0.86) and irrigated biomass with ir ( r = 0.68–0.81). Under low‐temperature environments (mean annual temperature = 18°C) after 40 DAP, tp and ir had greater effects than sw ( r = 0.55–0.82). Biomass variance was mainly explained by varietal type (50%–76%) in all environments×irrigation combinations, except in the high‐ and mid‐temperature environments under rainfed conditions where rainfall had the major effect (25%–45%). However, when mean temperature during the growing season decreased from 25°C (high environments) to 18°C (low environments), the contribution of mean temperature to biomass variance increased from 7% to 34% (rainfed) and from 4% to 36% (irrigated). Varietal type had the larger interactions with other factors independently of the environment and irrigation. We demonstrated a need to quantify (i) the main G×E×M drivers of biomass variability based on environmental stress factors and (ii) the variance contribution of these drivers on sorghum biomass. Our regional‐scale estimations are key inputs for future robust biomass projections of energy sorghum genotypes integrating G×E×M under climate change scenarios.

Ojeda, Jonathan J.↗

Calibrating constitutive models with full‐field data via physics informed neural networks

Abstract The calibration of solid constitutive models with full‐field experimental data is a long‐standing challenge, especially in materials that undergo large deformations. In this paper, we propose a physics‐informed deep‐learning framework for the discovery of hyperelastic constitutive model parameterizations given full‐field surface displacement data and global force‐displacement data. Contrary to the majority of recent literature in this field, we work with the weak form of the governing equations rather than the strong form to impose physical constraints upon the neural network predictions. The approach presented in this paper is computationally efficient, suitable for irregular geometric domains, and readily ingests displacement data without the need for interpolation onto a computational grid. A selection of canonical hyperelastic material models suitable for different material classes is considered including the Neo–Hookean, Gent, and Blatz–Ko constitutive models as exemplars for general non‐linear elastic behaviour, elastomer behaviour with finite strain lock‐up, and compressible foam behaviour, respectively. We demonstrate that physics informed machine learning is an enabling technology and may shift the paradigm of how full‐field experimental data are utilized to calibrate constitutive models under finite deformations.

Hamel, Craig M.↗

Transparent Asynchronous Zero-copy Remote I/O (TAZeR)

Transparent Asynchronous Zero-copy Remote I/O (TAZeR) is a framework built to facilitate grid computing by easing the memory movement and management across geographically separated computing resources. TAZeR employs a multiclient/ multi-server paradigm where data is stored at on a remote server. Users preload an interposing library which upon application start-up, stands up an ephemeral lightweight client. The client captures POSIX I/O calls to TAZeR meta-files which contains the IP address/port of the remote server, the location of the file on remote server, and additional meta-data. The client connects to the server using this information and begins downloading the file in parts optionally compressing the data.

Friese, Ryan↗

Generic Discretization Library

The GenDiL library is a collection of C++ software abstractions designed to discretize and solve partial differential equations (PDEs) for high-performance computing (HPC) applications. Its primary focus is on modern C++ generic programming, which helps ensure portability across various hardware architectures. The central idea behind the library is to provide building blocks for numerical algorithms-such as discretization methods and iteration patterns-so that domain experts can focus on the math, rather than the low-level details of hardware or implementation. By defining abstractions for data types, iteration over computational grids, and scheduling of operations, the library isolates the high-level PDE algorithms from the platform-specific optimizations needed to achieve efficient performance.

Dudouit, Yohann [Lawrence Livermore National Labor↗

caGrid 1.0 : an enterprise Grid infrastructure for biomedical research.

To develop software infrastructure that will provide support for discovery, characterization, integrated access, and management of diverse and disparate collections of information sources, analysis methods, and applications in biomedical research. Design: An enterprise Grid software infrastructure, called caGrid version 1.0 (caGrid 1.0), has been developed as the core Grid architecture of the NCI-sponsored cancer Biomedical Informatics Grid (caBIG{trademark}) program. It is designed to support a wide range of use cases in basic, translational, and clinical research, including (1) discovery, (2) integrated and large-scale data analysis, and (3) coordinated study. Measurements: The caGrid is built as a Grid software infrastructure and leverages Grid computing technologies and the Web Services Resource Framework standards. It provides a set of core services, toolkits for the development and deployment of new community provided services, and application programming interfaces for building client applications. Results: The caGrid 1.0 was released to the caBIG community in December 2006. It is built on open source components and caGrid source code is publicly and freely available under a liberal open source license. The core software, associated tools, and documentation can be downloaded from the following URL: <https://cabig.nci.nih.gov/workspaces/Architecture/caGrid>.

59 BASIC BIOLOGICAL SCIENCES↗

Wavelet-Smoothed Interpolation of Masked Scientific Data for JPEG 2000 Compression [Slides]

How should we manage scientific data with 'holes'? Some applications, like JPEG 2000, expect logically rectangular data, but some sources, like the Parallel Ocean Program (POP), generate data that isn't defined on certain subsets. We refer to grid points that lack well-defined, scientifically meaningful sample values as 'masked' samples. Wavelet-smoothing is a highly scalable interpolation scheme for regions with complex boundaries on logically rectangular grids. Computation is based on forward/inverse discrete wavelet transforms, so runtime complexity and memory scale linearly with respect to sample count. Efficient state-of-the-art minimal realizations yield small constants (O(10)) for arithmetic complexity scaling, and in-situ implementation techniques make optimal use of memory. Implementation in two dimensions using tensor product filter banks is straighsorward and should generalize routinely to higher dimensions. No hand-tuning required when the interpolation mask changes, making the method aeractive for problems with time-varying masks. Well-suited for interpolating undefined samples prior to JPEG 2000 encoding. The method outperforms global mean interpolation, as judged by both SNR rate-distortion performance and low-rate artifact mitigation, for data distributions whose histograms do not take the form of sharply peaked, symmetric, unimodal probability density functions. These performance advantages can hold even for data whose distribution differs only moderately from the peaked unimodal case, as demonstrated by POP salinity data. The interpolation method is very general and is not tied to any particular class of applications, could be used for more generic smooth interpolation.

42 ENGINEERING↗

Final Report. Evaluating the Climate Sensitivity of Dissipative Subgrid-Scale Mixing Processes and Variable Resolution in NCAR's Community Earth System Model

The goals of this project were to (1) assess and quantify the sensitivity and scale-dependency of unresolved subgrid-scale mixing processes in NCAR’s Community Earth System Model (CESM), and (2) to improve the accuracy and skill of forthcoming CESM configurations on modern cubed-sphere and variable-resolution computational grids. The research thereby contributed to the description and quantification of uncertainties in CESM’s dynamical cores and their physics-dynamics interactions.

58 GEOSCIENCES↗

Toward a Unified Representation of Atmospheric Convection in Variable-Resolution Climate Models

The purpose of this project was to improve the representation of convection in atmospheric weather and climate models that employ computational grids with spatially-variable resolution. Specifically, our work targeted models whose grids are fine enough over selected regions that convection is resolved explicitly, while over other regions the grid is coarser and convection is represented as a subgrid-scale process. The working criterion for a successful scheme for representing convection over this range of grid resolution was that identical convective environments must produce very similar convective responses (i.e., the same precipitation amount, rate, and timing, and the same modification of the atmospheric profile) regardless of grid scale. The need for such a convective scheme has increased in recent years as more global weather and climate models have adopted variable resolution meshes that are often extended into the range of resolving convection in selected locations.

54 ENVIRONMENTAL SCIENCES↗

Final Report: High Energy Physics at the Energy Frontier at Louisiana Tech

The Louisiana Tech University High Energy Physics group has developed a research program aimed at experimentally testing the Standard Model of particle physics and searching for new phenomena through a focused set of analyses in collaboration with the ATLAS experiment at the Large Hadron Collider (LHC) at the CERN laboratory in Geneva. This research program includes involvement in the current operation and maintenance of the ATLAS experiment and full involvement in Phase 1 and Phase 2 upgrades in preparation for future high luminosity (HL-LHC) operation of the LHC. Our focus is solely on the ATLAS experiment at the LHC, with some related detector development and software efforts. We have established important service roles on ATLAS in five major areas: Triggers, especially jet triggers; Data Quality monitoring; grid computing; GPU applications for upgrades; and radiation testing for upgrades. Our physics research is focused on multijet measurements and top quark physics in final states containing tau leptons, which we propose to extend into related searches for new phenomena. Focusing on closely related topics in the jet and top analyses and coordinating these analyses in our group has led to high efficiency and increased visibility inside the ATLAS collaboration and beyond. Based on our work in the DØ experiment in Run II of the Fermilab Tevatron Collider, Louisiana Tech has developed a reputation as one of the leading institutions pursuing jet physics studies. Currently we are applying this expertise to the ATLAS experiment, with several multijet analyses in progress.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Evaluating Large-Eddy Simulations of Boundary-Layer Cloud Response to Climate Change using MAGIC Observations

Over the cool waters off California, a persistent and extensive deck of low-lying clouds forms atop a shallow, turbulent layer of moist air extending 500-2000 meters above the ocean, above which the air is much warmer and drier. Its west edge varies with weather and season and is typically half way to Hawaii. This cloud deck and similar cloud decks elsewhere reflect sunlight back to space, cooling the underlying ocean and indeed the entire planet. They must be accurately represented by a climate model to correctly simulate our current climate or future global warming. Most climate models suggest these cloud decks will thin and break up in a warmer CO 2 -enriched climate, so extra sunlight would reach the ocean surface and amplify warming worldwide. But these models have difficulty accurately simulating the cloud thickness and extent because the turbulent cloud-forming air currents are much too small to be simulated by their coarse computational grid. In 2012-2013, the DOE Atmospheric Radiation Measurement MAGIC program instrumented a container ship transiting between California and Hawaii to observe clouds and rain, small particles (aerosols), and weather conditions. Data were obtained from 20 4-6 day cruises in each direction. This study tested whether a sophisticated ‘large-eddy simulation’ (LES) of the turbulent air motions that create the clouds in this region accurately predicts the variations in cloud properties seen across the different cruises. If so, that model is more believable for predicting cloud changes in this region in a future warmer climate. Our LES used a numerical mesh over a 6.4x6.4 km region with 50 m horizontal and 5 m vertical grid spacing to simulate the cloud-forming eddies and the sharp overlying transition between moist and dry air. To better compare with MAGIC observations, we configured our model to simulate a region that moved along with the ship. We used a weather forecast model to specify effects of the large-scale air flow. We compared our results with balloon ascents, a radar and other instruments pointed upward at the clouds, measurements of downward sunlight and infrared radiation, aerosols, and weather variables for all suitable cruise periods. Our LES was remarkably skillful in predicting the cloudiness variations measured by the ship. MAGIC also sampled mesoscale cells, regions of cloud thickening and thinning over distances of 10-50 km. An LES run over a larger region 100 km on a side shows these cells form spontaneously, enhanced by strong radiative heat loss at the cloud tops.

54 ENVIRONMENTAL SCIENCES↗

ASCR@40

One page flyers on: A Revolution in Modeling and Simulation; Applying Equations to Complex Problems; Uncertainty Quantification; Moving Big Data; Grid Computing for High-Speed Collaboration; Making Sense of Big Data; Overcoming Scaling Challenges; Building Better Computers; World-Leading Computing Facilities; Supporting Science Through Open-Source Software; and Building the Computational Workforce. Content provided by Department of Energy multiprogram laboratory researchers. Prepared by the Krell Institute for the ASCAC Subcommittee on the 40-year History of ASCR.

97 MATHEMATICS AND COMPUTING↗

Operational Evolution of FTS3: A DevOps Driven Approach to Elastic Operations

The File Transfer Service (FTS3) is a distributed data movement service developed at CERN and widely used to transfer data across the Worldwide LHC Computing Grid (WLCG). At Fermilab, FTS3 supports data transfers for multiple experiments, including Intensity Frontier experiments such as DUNE, enabling reliable data movement between WebDAV endpoints in Europe and the Americas.​ At CHEP 2021, we reported on the initial containerized deployment of FTS3 on OKD, the community Kubernetes distribution of Red Hat OpenShift. In this work, we present the subsequent evolution of this deployment, focusing on new operational capabilities introduced to improve scalability, robustness, and long-term maintainability.​ We describe the adoption of more secure and reproducible container build workflows, the integration of DevOps-driven operational practices, and enhancements in monitoring and automation. A key new result is the introduction of horizontal scaling and elastic resource management, allowing FTS3 components to dynamically adapt to workload variations while maintaining service reliability. We also discuss improvements in fault tolerance and operational procedures derived from production experience.​ Finally, we summarize lessons learned from operating FTS3 as a Kubernetes-native service and outline how these developments have improved the resilience and efficiency of data movement operations at Fermilab.

Munoz Flores, Victor Leopoldo [Fermilab]↗