Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Grid computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Communication Lower Bounds and Optimal Algorithms for Multiple Tensor-Times-Matrix Computation

Multiple tensor-times-matrix (Multi-TTM) is a key computation in algorithms for computing and operating with the Tucker tensor decomposition, which is frequently used in multidimensional data analysis. Here, we establish communication lower bounds that determine how much data movement is required (under mild conditions) to perform the Multi-TTM computation in parallel. The crux of the proof relies on analytically solving a constrained, nonlinear optimization problem. We also present a parallel algorithm to perform this computation that organizes the processors into a logical grid with twice as many modes as the input tensor. We show that, with correct choices of grid dimensions, the communication cost of the algorithm attains the lower bounds and is therefore communication optimal. Finally, we show that our algorithm can significantly reduce communication compared to the straightforward approach of expressing the computation as a sequence of tensor-times-matrix operations when the input and output tensors vary greatly in size.

HBL-inequalities↗
DOE OSTI2019

Enabling Technologies for Multinational Interconnected Smart Grid

The multinational interconnected smart grid is a crucial part for the construction of global energy internet, which requires various technologies to support. Based on the key technologies of multinational grid, this paper first states the importance of emerging information communication technologies, such as distributed cloud computing and blockchain, to the safe and stable operation of the grid. Then this paper discusses smart-grid control technologies for power dispatch and grid security. Finally, this paper talks about enabling power electronic technology and analyzes the relationship between power electronics, flexible AC transmission and HVDC transmission technology. To conclude, all these technologies play key roles in enabling multinational interconnected power systems.

Yang, Jianlin↗
DOE OSTI2019

Random Force Perturbations: A New Extension of the Cell Perturbation Method for Turbulence Generation in Multiscale Atmospheric Boundary Layer Simulations

Coupling between mesoscale and large eddy simulation (LES) is critically important for many atmospheric model applications, from predictions of wind energy to fire propagation. The grid-nesting technique enables bridging between vastly different scales without incurring prohibitive computational costs. Yet, the transition from coarser to finer resolutions often requires a large number of grid points from inflow boundaries for the development of fine-scale turbulence features in the LES domain. Recently, the cell perturbation method (CPM) was developed to reduce the turbulence development region with high computational efficiency. Herein, we explore a new method based on the CPM that uses force perturbations in both the horizontal and vertical directions (Force Cell Perturbation Method) instead of the potential temperature perturbations in the original CPM, as an attempt to further explore the performance of the random perturbation techniques. This method is tested for a neutral and a convective atmospheric boundary layer under idealized conditions. Overall, similar performance is found between the optimal configurations of the CPM and the Force Cell Perturbation Method pointing to the robustness of this family of perturbation methods in accelerating turbulence generation in nested domains. Vertical force perturbations performed better than horizontal force perturbations for both atmospheric stability conditions. The CPM performed best under convective stability conditions. The combination of the force and potential temperature perturbations is found to provide no additional performance improvement over the stand-alone application of the individual methods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Scalable implementation of polynomial filtering for density functional theory calculation in PARSEC

In this work, we present an efficient implementation of polynomial filtering methods in PARSEC, a real-space pseudopotential based Kohn–Sham density functional theory solver. The implementation described here improves upon a Chebyshev-filtered subspace iteration algorithm used in the previous version of PARSEC. We present a hybrid polynomial filtering scheme that combines Chebyshev-filtered subspace iteration and a spectrum slicing method that partitions the spectrum into several spectral slices and uses bandpass-filtered subspace iteration to compute approximate eigenpairs within each interior slice simultaneously. We describe a procedure to partition a spectrum and construct polynomial filters. We also discuss a number of practical issues such as the use of appropriate data layouts for carrying out the computation on a two-dimensional process grid and how to achieve good load balance by allocating an appropriate number of process groups to each spectral slice. Numerical examples are presented to demonstrate the effectiveness of the hybrid polynomial filtering method as well as the superior parallel scalability of spectrum slicing in comparison to that of Chebyshev-filtered subspace iteration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

An Integrated Platform for Wind Power Plant Operations: From Atmosphere to Electrons to the Grid (A2e2g)

The research objective for the Atmosphere to Electrons to the Grid (A2e2g) project was to design a platform that merges forecasting tools with aerodynamic and economic models. The value proposition is that expanding wind power plant operation to include grid services allows those plants to operate in markets for grid services as well as energy markets, increasing revenue streams for wind plant operators while contributing to reliable grid operation.

17 WIND ENERGY↗
DOE OSTI2026

Integrating AI Data Centers with the Power Grid

The rapid expansion of artificial intelligence (AI) has triggered an unprecedented surge in electricity demand, with US data center energy use projected to double or triple 2023 levels by 2028. This exponential growth places strain on grid infrastructure, which can hinder timely construction of desired computing capacity. To bridge this supply-demand gap, utilities and AI developers are increasingly turning to demand flexibility, a strategy that incentivizes shifting or reducing power use during peak periods of grid stress. Data centers are uniquely equipped for flexible operations due to their digital workloads, built-in redundancy, and onsite energy assets. This article outlines four primary mechanisms to enable data center flexibility: computational load flexibility (shifting tasks temporally or geographically), flexible use of core facility infrastructure adjustments, energy storage utilization, and onsite electricity generation. To encourage adoption, utilities are deploying new tariff designs, including voluntary interruptible service riders, mandated flexibility requirements, and streamlined interconnection processes for flexible loads. For the highly capitalized and rapidly growing AI industry, the primary motivators for embracing these strategies are expediting facility interconnection, satisfying emerging regulatory mandates, and mitigating community resistance. While demand flexibility cannot substitute the long-term need for new bulk power generation, it serves as an essential, immediate solution for enabling near-term deployment. By transforming data centers from grid stressors into stabilizing assets, flexible operations can ensure reliable grid integration, ease market pressures, and support a resilient power system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Axisymmetric hydrodynamics in numerical relativity using a multipatch method

In this paper, we describe a method of implementing the axisymmetric evolution of general-relativistic hydrodynamics and magnetohydrodynamics through modification of a multipatch grid scheme. In order to ease the computational requirements required to evolve the post-merger phase of systems involving binary compact massive objects in numerical relativity, it is often beneficial to take advantage of these system's tendency to rapidly settle into states that are nearly axisymmetric, allowing for 2D evolution of secular timescales. We implement this scheme in the spectral Einstein code and show the results of application of this method to four test systems including viscosity, magnetic fields, and neutrino radiation transport. Our results show that this method can be used to quickly allow already existing 3D infrastructure that makes use of local coordinate system transformations to be made to run in axisymmetric 2D with the flexible grid creation capabilities of multipatch methods. Our code tests include a simple model of a binary neutron star postmerger remnant, for which we confirm the formation of a massive torus which is a promising source of post-merger ejecta.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Improving the Capabilities and Computational Efficiency of the RTE+RRTMGP Radiation Code (Final Report)

This report details progress on the RTE+RRTMGP radiation codes made during the period of performance. RTE+RRTMGP is a set of codes for computing radiative fluxes in planetary atmospheres. RRTMGP uses a k-distribution to provide an optical description (absorption and possibly Rayleigh optical depth) of the gaseous atmosphere, along with the relevant source functions, on a pre-determined spectral grid given temperatures, pressures, and gas concentration. RTE computes fluxes given spectrally-resolved optical descriptions and source functions. Spectrally-resolved fluxes are summarized (“reduced”) via a user extensible class. The initial release of the code and the design choices are described in Pincus et al. 2019; the codes are available on Github. Although RRTMGP was based on current (at the time) empirical spectroscopic data, RTE and RRTMGP were developed in large part to modernize software practices. The design focused on flexibility broadly interpreted: by separating code from data and allowing data to drive computation; in coupling to the host model (e.g. the coupling of clouds to radiative fluxes is user-controlled); with respect to programming languages (computational tasks are accessed via widely-compatible C interfaces); and with respect to hardware (the codes run on a range of CPU and GPU architectures). The code also puts an emphasis on modularity and clarity. RTE+RRTMGP v1.0 was released in September 20219. This award supported the evolution of the RTE+RRTMGP code base to support greater flexibility, accuracy, and efficiency.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Privacy-preserving Information Security for the Energy Grid of Things

Smart grid infrastructure relies on information exchange between multiple actors in order to ensure system reliability. These actors include but are not limited to smart loads, grid control, and energy management technologies. Further, as information exchange between these actors is susceptible to cyber-attacks, security and privacy issues are indispensable to ensure a reliable and stable grid. This position paper proposes a privacy-preserving, trust-augmented secure scheme for a smart grid implementation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

An embedded 3D fracture modeling approach for simulating fracture-dominated fluid flow and heat transfer in geothermal reservoirs

In this paper, we describe an efficient modeling approach, named embedded discrete fracture method (EDFM), for incorporating arbitrary 3D, discrete fractures, such as hydraulic fractures or faults, into modeling fracture-dominated fluid flow and heat transfer in fractured geothermal reservoirs. This technique allows 3D discrete fractures to be discretized independently from surrounding rock volume and inserted explicitly into a primary fracture/matrix grid, generated without including 3D discrete fractures in prior. An effective computational algorithm is developed to discretize these 3D discrete fractures and construct local connections between 3D fractures and fracture/matrix grid blocks representing the surrounding rock volume. The constructed gridding information on 3D fractures is then added to the primary grid. This embedded fracture modeling approach can be directly implemented into a developed geothermal reservoir simulator via the integral finite difference (IFD) method or with TOUGH2 technology. This embedded fracture modeling approach is very promising and computationally efficient to handle realistic 3D discrete fractures with complicated geometries, connections, and spatial distributions. Compared with other fracture modeling approaches, it avoids cumbersome 3D unstructured, local refining procedures, and increases computational efficiency by simplifying Jacobian matrix size and sparsity, while maintaining enough accuracy. Several numeral simulations are presented to demonstrate the utility and robustness of the proposed technique. Our numerical experiments show that this approach captures all the key patterns about fluid flow and heat transfer dominated by fractures in these cases. Thus, this approach is readily available to the simulation of fractured geothermal reservoirs with both artificial and natural fractures.

15 GEOTHERMAL ENERGY↗
DOE OSTI2018

An Embedded 3D Fracture Modeling Approach for Simulating Fracture-Dominated Fluid Flow and Heat Transfer in Geothermal Reservoirs

An efficient modeling approach is described for incorporating arbitrary 3D, discrete fractures, such as hydraulic fractures or faults, into modeling fracture-dominated fluid flow and heat transfer in fractured geothermal reservoirs. This technique allows 3D discrete fractures to be discretized independently from surrounding rock volume and inserted explicitly into a primary fracture/matrix grid, generated without including 3D discrete fractures in prior. An effective computational algorithm is developed to discretize these 3D discrete fractures and construct local connections between 3D fractures and fracture/matrix grid blocks of representing the surrounding rock volume. The constructed gridding information on 3D fractures is then added to the primary grid. This embedded fracture modeling approach can be directly implemented into a developed geothermal reservoir simulator via the integral finite difference (IFD) method or with TOUGH2 technology This embedded fracture modeling approach is very promising and computationally efficient to handle realistic 3D discrete fractures with complicated geometries, connections, and spatial distributions. Compared with other fracture modeling approaches, it avoids cumbersome 3D unstructured, local refining procedures, and increases computational efficiency by simplifying Jacobian matrix size and sparsity, while keeps sufficient accuracy. Several numeral simulations are present to demonstrate the utility and robustness of the proposed technique. Our numerical experiments show that this approach captures all the key patterns about fluid flow and heat transfer dominated by fractures in these cases. Thus, this approach is readily available to simulation of fractured geothermal reservoirs with both artificial and natural fractures.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Parallax-corrected VISST-derived pixel-level products from satellite GOES-16

The NASA Langley group led by William Smith produced GOES-16 satellite cloud retrievals over an approximate 10 by 10 degree region over the CACTI field campaign location. These retrievals are described here: https://www.arm.gov/capabilities/vaps/visst and are available for download here . They use algorithms historically called VISST that are now referred to as SatCORPS. More information can be found in Trepte et al. (2019), Minnis et al. (2021), and Yost et al. (2021). If using this dataset, please cite these references, the CACTI VISST dataset DOI found at the download link above, and this dataset’s DOI. The CACTI VISST pixel-level retrievals are on a 2 km spatial grid and available every 15 minutes (every 10 minutes late in the campaign), producing 21,765 files for the entire field campaign between October 2018 and April 2019. They are not corrected for parallax error, which is an offset in the actual geographical location of a cloud above the surface due to the satellite viewing the cloud partly from the side off nadir. This dataset applies a correction for parallax using the location relative to the satellite and the retrieved cloud top height above the surface, which allows the dataset to be geo-located with surface-based observations. The parallax correction for each location depends on the longitude, latitude and cloud top height above ground level (AGL) for that longitude and latitude in the original VISST files. The cloud top height AGL requires first computing the surface elevation at each VISST grid point. Data from the Advanced Spaceborne Thermal Emission and Reflection (ASTER) Global Digital Elevation Map Version 3 at 30-m resolution is projected onto the VISST grid using conservative coarsening (conserving surface elevation) in the xESMF Python package. The surface elevation is then subtracted from the VISST-retrieved cloud top height above mean sea level. These cloud top heights AGL are then combined with longitude and latitude to estimate the latitude and longitude corrections. Due to variability in cloud top height, the parallax shifts produce an irregular grid of values since higher cloud tops are shifted further than lower cloud tops. A ball tree-based neighbor search with Haversine distance is performed using the Python-based scikit-learn library to find the nearest VISST grid point to each parallax correction-shifted point. The data value of the shifted point is then assigned to that VISST grid point. In this manner, the irregular geographical shifts to correct for parallax are projected back to the rectilinear VISST grid. Because relatively higher clouds should obscure lower clouds, the variable values for the highest cloud top are preferentially chosen if two or more values are assigned to a grid point. The parallax correction should be viewed as an improved but still imperfect estimation of the cloud top locations, largely because the cloud top height is an imperfect retrieval. Please see the attached README document for further information. Users are encouraged to contact the authors with any additional questions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

SLoG: Large-Scale Logging Middleware for HPC and Big Data Convergence

Cloud developers traditionally rely on purpose-specific services to provide the storage model they need for an application. In contrast, HPC developers have a much more limited choice, typically restricted to a centralized parallel file system for persistent storage. Unfortunately, these systems often offer low performance when subject to highly concurrent, conflicting I/O patterns. This makes difficult the implementation of inherently concurrent data structures such as distributed shared logs. Yet, this data structure is key to applications such as computational steering, data collection from physical sensor grids, or discrete event generators. In this paper we tackle this issue. We present SLoG, shared log middleware providing a shared log abstraction over a parallel file system, designed to circumvent the aforementioned limitations. We evaluate SLoG’s design on up to 100,000 cores of the Theta supercomputer: the results show high append velocity at scale while also providing substantial benefits for other persistent backend storage systems.

big data↗
DOE OSTI2021

Using Pilot Jobs and CernVM File System for Simplified Use of Containers and Software Distribution

High Energy Physics (HEP) experiments entail an abundance of computing resources, i.e. sites, to run simulations and analyses by processing data. This requirement is fulfilled by local batch farms, grid sites, private/commercial clouds, and supercomputing centers via High Throughput Computing (HTC). The growing needs of such experiments and resources being prone to trends of heterogeneity make it difficult for physicists to handle these resources directly. Additionally, HEP collaborations heavily rely on data and software releases, typically in the order of tens of gigabytes, while conducting simulations and analyses. Hence, aspects of scalability, reliability, and maintenance become crucial with regards to the distribution of the necessary data and software stack. The GlideinWMS [4] framework helps with the resource management problem by using pilot jobs, aka Glideins, to provision reliable elastic virtual clusters. Glideins are submitted to unreliable heterogeneous resources which are validated and customized by the Glideins to make the worker nodes available for end-user job execution. On the other hand, the CernVM File System (CernVM-FS or CVMFS) [1] helps with data distribution. It is a write-once, read-everywhere filesystem used to deploy scientific software to thousands of nodes on a worldwide distributed computing infrastructure. CVMFS is based on the Hyper Text Transfer Protocol and has been widely used within the particle physics community for (1) distributing experiment software and data such as calibrations, and (2) facilitating containerization by efficiently hosting container images along with providing containerization software, especially Singularity [3] GlideinWMS relies on CVMFS installed locally on the computing resources to satisfy the experiments' software needs. This requires system administrators' effort to install and maintain CVMFS at the sites and limits the use of sites, especially HPC resources, that do not have CVMFS installed. This poster presents a solution, taking advantage of Glideins to provide CVMFS at most sites without the need for a local installation. Doing so expands the pool of resources available for HEP experiments and reduces the effort of system administrators for current resources. Additionally, the proposed solution allows GlideinWMS to also start Singularity [3], a containerization software that can run unprivileged, on sites where neither CVMFS nor Singularity are available, including HPC sites. The benefits provided by this solution are: (1) lower overhead for site administrators in that they have less software to install, (2) an expanded pool of resources that run user jobs with easy access to software and data provided by CVMFS, thus making life easier for the scientists, and (3) improved flexibility to use HPC resources by enabling GlideinWMS pilot jobs to support HPC sites.

Urs, Namratha↗
DOE OSTI2023

Optimizing Communication in 2D Grid-Based MPI Applications at Exascale

The new reality of exascale computing faces many challenges in achieving optimal performance on large numbers of nodes. A key challenge is the efficient utilization of the message-passing interface (MPI), a critical component for process communication. This paper explores communication optimization strategies to harness the GPU-accelerated architectures of these supercomputers. We focus on MPI applications where processors form a two-dimensional process grid, a common arrangement in applications involving dense matrix operations. This configuration offers a unique opportunity to implement innovative strategies to improve performance and maintain effective load distribution. We study two applications— Dist-FW (Apsp:all-pair-shortest-path) and HPL-MxP (LU factorization with Mixed precision)—on two accelerated systems: Summit (IBM Power and NVIDIA V100) and Frontier (AMD EPYC and MI250X). These supercomputers are operated by the Oak Ridge Leadership Computing Facility (OLCF) and are currently ranked #1 and #5 on the Top500 list. We show how to scale up both applications to exascale levels and tackle the MPI challenges related to implementation, synchronization, and performance. We also compare the performance of several communication strategies at an unprecedented scale. Accurately predicting application performance becomes crucial for cost reduction as the computational scale grows. To address this, we suggest a hyperbolic model as a better alternative to the traditional one-sided asymptotic model for predicting future application performance at such large scales.

Lu, Hao↗
DOE OSTI2020

Grid-based minimization at scale: Feldman-Cousins corrections for SBN

We present a computational model for the construction of Feldman-Cousins (FC) corrections frequently used in High Energy Physics (HEP) analysis. The program contains a grid-based minimization and is written in C++. Our algorithms exploit vectorization through Eigen3, yielding a single-core speed-up of 350 compared to the original implementation, and achieve MPI data parallelism by using DIY. We demonstrate the application to scale very well at High Performance Computing (HPC) sites. We use HDF5 in conjunction with HighFive to write results of the calculation to file.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2014

Efficient algorithm for locating and sizing series compensation devices in large power transmission grids: II. Solutions and applications

In a companion manuscript, we developed a novel optimization method for placement, sizing, and operation of Flexible Alternating Current Transmission System (FACTS) devices to relieve transmission network congestion. Specifically, we addressed FACTS that provide Series Compensation (SC) via modification of line inductance. In this manuscript, this heuristic algorithm and its solutions are explored on a number of test cases: a 30-bus test network and a realistically-sized model of the Polish grid (~ 2700 nodes and ~ 3300 lines). The results on the 30-bus network are used to study the general properties of the solutions including non-locality and sparsity. The Polish grid is used as a demonstration of the computational efficiency of the heuristics that leverages sequential linearization of power flow constraints and cutting plane methods that take advantage of the sparse nature of the SC placement solutions. Using these approaches, the algorithm is able to solve an instance of Polish grid in tens of seconds. We explore the utility of the algorithm by analyzing transmission networks congested by (a) uniform load growth, (b) multiple overloaded configurations, and (c) sequential generator retirements.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Powered by dsgrid [Slides]

NREL's demand-side grid (dsgrid) toolkit harnesses decades of sector-specific energy modeling expertise to understand current and future U.S. electricity load for power systems analyses. The primary purpose of dsgrid is to create comprehensive electricity load data sets at high temporal, geographic, sectoral, and end-use resolution. These data sets enable detailed analyses of current patterns and future projections of end-use loads. This presentation will include NREL power grid researcher Elaine Hale.

24 POWER TRANSMISSION AND DISTRIBUTION↗