SBIR Phase II Final Technical Report - Precise and Scalable Race Detection for Extreme-Scale Computing
Explore the source record for details and available documents.
97 MATHEMATICS AND COMPUTING↗
SEARCH · Engineering Papers
Results for “Extreme scale”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Real-time Bayesian inference at extreme scale: Tsunami early warning applied to the Cascadia subduction zone
Explore the source record for details and available documents.
Mathematics and Computing↗
Field tests of a highly flexible downwind ultralight rotor to mimic a 13-MW turbine rotor
Offshore extreme-scale turbines of 20–25 MW in size may offer reduced energy costs. The technical barriers at these extreme scales include escalating blade masses with increased flexibility as well as high gravity loads and tower-strike issues. These barriers may be addressed with a load-aligning downwind turbine. To investigate this type of design, a field test campaign was conducted with an aeroelastically scaled rotor, termed the Segmented Ultralight Morphing Rotor Demonstrator (SUMR-D). The tests were conducted on the Controls Advanced Research Turbine at the National Renewable Energy Laboratory. The paper gives an overview of the experimental diagnostics, blade design, and results of the field campaign, as well as makes conclusions and recommendations regarding extreme-scale highly flexible downwind rotors.
17 WIND ENERGY↗
Advancing Scientific Productivity through Better Scientific Software: Developer Productivity and Software Sustainability Report
The Exascale Computing Project (ECP) provides a unique opportunity to advance computational science and engineering (CSE) through an accelerated growth phase in extreme-scale computing. Central to the project is the development of next-generation applications and software technologies that can exploit emerging architectures for optimal performance and provide high-fidelity, multiphysics, multiscale capabilities. However, disruptive changes in computer architectures and the complexities of tackling new frontiers in extreme-scale modeling, simulation, and analysis present daunting challenges to the productivity of software developers and the sustainability of software artifacts. Members of the CSE community - especially at extreme scales but more broadly at all scales of computing - face an urgent need to improve developer productivity, positively impacting product quality, development time, and staffing resources, and software sustainability, reducing the cost of maintaining, sustaining, and evolving software capabilities.
97 MATHEMATICS AND COMPUTING↗
Data Transfer Tool (HPC Annual Report V.3.1)
Sandia has been developing and supporting data transfer tools for over 20 years and has the expertise to take DOE into the Extreme Scale era. In looking at Exascale and beyond (Extreme Scale Computing), data sets can be thousands of 500TBs in size, a single file can be in the 100TB range, and billions of files are expected. Huge bursts of data need to be transferred, even today. While data archiving is often not thought about, it is an integral part of the full data management path when data is generated on HPC systems. In order to move generated data to its final resting place (data archive) or to transfer between file systems, a capable data transfer tool is required.
97 MATHEMATICS AND COMPUTING↗
Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective
Emerging workloads such as artificial intelligence, big data analytics and complex multi-step workflows alongside future exascale applications are anticipated future HPC workloads, which will result in a more diverse I/O system workload and even less predictable I/O behavior and access patterns. Along with the ever increasing gap between the compute and storage performance capabilities, the in-depth understanding of extreme-scale I/O behavior and the I/O performance modeling and prediction are essential tools of the large-scale I/O evaluation process for addressing the needs of extreme-scale hybrid workloads. In this survey article, we focus on the state-of-the-art of the I/O behavior and performance analysis process for HPC systems in a 5-year time window and identify future research challenges.
Neuwirth, Sarah↗
Holistic Measurement Driven Resilience: Combining Operational Fault and Failure Measurements and Fault Injection for Quantifying Fault Detection, Propagation and Impact. Final report
For HPC systems to date, application resilience to faults and failures has been accomplished by the brute- force method of checkpoint/restart, which allows an application to make forward progress in the face of system and application faults, errors, and failures independent of root cause or end result. It has remained the primary resilience mechanism because we lack a way to identify faults and anticipate consequences early enough to take meaningful mitigating action. However, checkpoint/restart implementations put a tremendous burden on system resources and on the applications themselves and is becoming less feasible at scale. Because we have not yet operated at scales at which checkpoint/restart fails to provide forward progress, despite increasing costs, vendors have had little motivation to provide the instrumentation necessary for early identification of faults and failures. However, as we move from petascale to exascale, component mean time to failure (MTTF) will render the existing techniques ineffectual and/or too expensive. Furthermore, fault recovery mechanisms such as failover and/or error correction introduce performance inconsistency. Instrumentation allowing early indication of problems and tools to enable use of such information by systems, operating systems, and applications offer an alternative, more scalable and less costly solution. In the HMDR project, we built on our experience and expertise developed and accumulated over years of research on design, monitoring, measurement, and assessment of resilient computing systems. Analysis of field data on the current and past generations of extreme-scale systems revealed several challenges that, if not addressed in increasingly larger and more complex systems, may hinder the effectiveness of future exascale computing systems. Specifically, i) file systems and interconnects in current-generation large-scale systems already operate at the margins of resiliency, including consistent performance, and may not scale to larger deployments; ii) automated, software-based failover mechanisms are frequently inadequate and can introduce wider failures, such that failures during recovery may lead to system/application failures, including system-wide outages; and iii) silent data corruption represents a critical fault mode and will require efficient detection mechanisms if next-generation applications are to take full advantage of exascale hardware. To address the above challenges, we assembled a team of world-renowned experts in resilient extreme- scale computing from the University of Illinois (Electrical and Computer Engineering, Computer Science, and NCSA), SNL, LANL, NERSC, and Cray. Our team includes representatives from centers that house many of the largest HPC resources in the world, both today and over the coming years. The team has a unique track record of research in i) system and application failure characterization based on the analysis of field data, ii) data-driven design of fault/error detection mechanisms, and iii) experimental characterization of system/application resiliency. The team includes system owners/operators who provide continuous data collection and access and ensure installation of appropriate analysis tools.
97 MATHEMATICS AND COMPUTING↗
Optimizing Data Movement for GPU-Based In-Situ Workflow Using GPUDirect RDMA
The extreme-scale computing landscape is increasingly dominated by GPU-accelerated systems. At the same time, in-situ workflows that employ memory-to-memory inter-application data exchanges have emerged as an effective approach for leveraging these extreme-scale systems. In the case of GPUs, GPUDirect RDMA enables third-party devices, such as network interface cards, to access GPU memory directly and has been adopted for intra-application communications across GPUs. In this paper, we present an interoperable framework for GPU-based in-situ workflows that optimizes data movement using GPUDirect RDMA. Specifically, we analyze the characteristics of the possible data movement pathways between GPUs from an in-situ workflow perspective, and design a strategy that maximizes throughput. Furthermore, we implement this approach as an extension of the DataSpaces data staging service, and experimentally evaluate its performance and scalability on a current leadership GPU cluster. The performance results show that the proposed design reduces data-movement time by up to 53% and 40% for the sender and receiver, respectively, and maintains excellent scalability for up to 256 GPUs.
Zhang, Bo↗
Message-Based SPDK Transport Utility (TED Report) [Slides]
The driving goal is to investigate the feasibility of a message-based data transport for SPDK. All of SPDK’s existing data transport protocols are “connection-based”, requiring a target to hold state for each connected client / initiator. At extreme scale, juggling these connections from a large number of initiators creates a performance bottleneck. Under a message-based paradigm, we can bypass this behavior, as all initiators share a global communication channel, held open for the entire run. In theory, this approach should minimize connection management overhead at extreme scale.
97 MATHEMATICS AND COMPUTING↗
Two new SciDAC institutes promote mathematical tools and software technology for high-performance computing
Bigger is often said to be better, and the newest extreme-scale computers certainly are bigger, with millions of processing units. Moreover, the breadth of science performed on the U.S. Department of Energy (DOE) computing facilities is expanding, with new technology such as artificial intelligence emerging. These advances are exciting, creating new opportunities for scientific discovery; however, they also raise new questions for scientists who want to exploit these advances for tackling more complex problems. Will my simulation code be able to utilize the accelerators in extreme-scale computing systems? Can I take advantage of the deepening memory hierarchy in heterogeneous processors? Is there a way around bottlenecks caused by the widening ratio of peak floating-point operations per second to I/0 bandwidth? How can I manage my huge amounts of data effectively? Can I analyze data in situ, or must I transfer it to offline storage for later analysis? To address such questions, DOE announced that it is providing $57.5 million over the next five years for two multidisciplinary teams — FASTMath and RAPIDS2 — to develop new tools and techniques to harness supercomputers for scientific discovery. The teams, called SciDAC Institutes, are part of the Scientific Discovery through Advanced Computing program.
97 MATHEMATICS AND COMPUTING↗
cuZ-Checker: A GPU-Based Ultra-Fast Assessment System for Lossy Compressions
Lossy compression is becoming an indispensable technique for the success of today's extreme-scale high-performance computing projects that produce vast volumes of data during scientific simulations or instrument data acquisitions. Comprehensively understanding the compression quality and performance of different lossy compressors is critical to selecting the best-fit compressors and using them properly and efficiently in practice. A few lossy compression assessment tools (e.g., Z-checker) have been developed, but none of them support the execution in a GPU environment. This is a significant gap because many recent extreme-scale applications and lossy compressors (e.g., cuSZ) can run entirely within GPUs. In this work, we develop an efficient lossy compression measuring system (called cuZ-Checker) on the GPU platform, which aims to perform the lossy compression quality and performance assessment completely within the GPU environment. Our contribution is threefold. (1) We develop a novel GPU-based lossy compression measuring framework using a computation pattern-based design approach. This approach classifies the computing-intensive metrics into three categories based on their patterns which creates large opportunities for kernel fusion and data reuse. (2) For each pattern in cuZ-Checker, we develop a CUDA kernel and provide fine-grained optimizations to boost its performance. (3) We thoroughly evaluate our cuZ-checker on a V100 GPU using four real-world scientific application datasets. Experiments show that cuZ-Checker can significantly accelerate the overall lossy compression assessment performance by 23X similar to 31X compared with the OpenMP-based multithreading CPU performance. To the best of our knowledge, this is the first lossy compression measuring system designed for GPU devices.
GPU↗
ECP Software Technology Capability Assessment Report V3.0
The Exascale Computing Project (ECP) Software Technology (ST) focus area is responsible for (1) developing critical software capabilities that will enable the successful execution of ECP applications and (2) providing key components of a productive and sustainable exascale computing ecosystem that will position the US Department of Energy (DOE) and the broader high-performance computing (HPC) community with a firm foundation for future extreme-scale computing capabilities. This ECP ST Capability Assessment Report (CAR) provides an overview and assessment of current ECP ST capabilities and activities, giving stakeholders and the broader HPC community information that can be used to assess ECP ST progress and plan their own efforts accordingly. ECP ST leaders commit to updating this document on regular basis (every 6–12 months). Highlights from this version of the report are presented here. This version of the CAR contains the following updates relative to the previous revision: (1) This report highlights the progress with the Extreme-scale Scientific Software Stack (E4S) efforts. In particular, this report discusses how E4S continues to gain traction as a first-class entity in the HPC ecosystem, enabling new conversations with users, facilities, vendors, other US agencies, and international partners. (2) The several-page summaries of each ECP Level 4 project were updated to reflect recent progress and next steps (Section 4). Of particular note are the experiences of our teams on early-access systems for Frontier. (3) The E4S is described further. E4S is now updated via quarterly releases. E4S is the primary integration and delivery vehicle for ECP ST capabilities (Section 2.1.1). (4) The ECP ST software development kit (SDK) effort further refined its groupings (Section 2.1.2). The ECP ST focus area represents the key bridge between exascale systems and the scientists developing applications that will run on those platforms. ECP ST efforts contribute to approximately 70 software products (Section 2.1.3) in six technical areas (Table 1). Since publishing the previous revision of the CAR, the team has continued to evolve the product dictionary of official product names, which enables more rigorous mapping of ECP ST deliverables to stakeholders (Section 2.1.4).
97 MATHEMATICS AND COMPUTING↗
Design and analysis of a segmented blade for a 50 MW wind turbine rotor
Extreme-size wind turbines face logistical challenges due to their sheer size. A solution, segmentation, is examined for an extreme-scale 50 MW wind turbine with 250 m blades using a systematic approach. Segmentation poses challenges regarding minimizing joint mass, transferring loads between segments and logistics. We investigate the feasibility of segmenting a 250 m blade by developing design methods and analyzing the impact of segmentation on the blade mass and blade frequencies. This investigation considers various variables such as joint types (bolted and bonded), adhesive materials, joint locations, number of joints and taper ratios (ply dropping). Segmentation increases blade mass by 4.1%–62% with bolted joints and by 0.4%–3.6% with bonded joints for taper ratios up to 1:10. Cases with large mass growth significantly reduce blade frequencies potentially challenging the control design. We show that segmentation of an extreme-scale blade is possible but mass reduction is necessary to improve its feasibility.
Escalera Mendoza, Alejandra S.↗
Ultra-Broadband Interleaver for Extreme Wavelength Scaling in Silicon Photonic Links
Not Available
Rizzo, Anthony (ORCID:000000034752797X)↗
Observational and numerical studies of extreme frontal scale contraction
The general objective of this effort is to increase understanding of how frontal scale contraction processes may create and sustain intense mesoscale precipitation along intensifying cold fronts. The five-part project (an expansion of the originally proposed two-part project) employed conventional meteorological data, special mesoscale data, remote sensing measurements, and various numerical models. First an idealized hydrostatic modeling study of the scale contraction effects of differential cloud cover on low-level frontal structure and dynamics was completed and published in a peer-reviewed journal. The second objective was to complete and publish the results from a three dimensional numerical model simulation of a cold front in which differential sensible heating related to cloud coverage patterns was apparently crucial in the formation of a severe frontal squall line. The third objective was to use a nonhydrostatic model to examine the nonlinear interactions between the transverse circulation arising from inhomogeneous cloud cover, the adiabatic frontal circulation related to semi-geostrophic forcing, and diabatic effects related to precipitation processes, in the development of a density current-like microstructure at the leading edge of cold fronts. Although the development of a frontal model that could be used to initialize such a primitive equation model was begun, we decided to focus our efforts instead on a project that could be successfully completed in this short time, due to the lack of prospects for continued NASA funding beyond this first year (our proposal was not accepted for future funding). Thus, a fourth task was added, which was to use the nonhydrostatic model to test tentative hypotheses developed from the most detailed observations ever obtained on a density current (primarily sodar and wind profiler data). These simulations were successfully completed, the findings were reported at a scientific conference, and the results have recently been submitted to a peer-reviewed journal. The fifth objective was to complete the analysis of data collected during the Cooperative Oklahoma Profiler Studies (COPS-91) field project, which was supported by NASA. The analysis of the mesoscale surface and sounding data, Doppler radar imagery, and other remote sensing data from multi frequency wind profiler, microwave radiometer, and the Radio Acoustic Sounding System has been completed. This study is a unique investigation of processes that caused the contraction of a cold front to a microscale zone exhibiting an undular bore-like structure. Results were reported at a scientific conference and are being prepared for publication. In summary, considerable progress has been achieved under NASA funding in furthering our understanding of frontal scale contraction and density current - gravity wave interaction processes, and in utilizing models and remotely sensed data in such studies.
Koch, Steven E.↗
Parked aeroelastic field rotor response for a 20% scaled demonstrator of a 13‐MW downwind turbine
Abstract Aeroelastic parked testing of a unique downwind two‐bladed subscale rotor was completed to characterize the response of an extreme‐scale 13‐MW turbine in high‐wind parked conditions. A 20% geometric scaling was used resulting in scaled 20‐m‐long blades, whose structural and stiffness properties were designed using aeroelastic scaling to replicate the nondimensional structural aeroelastic deflections and dynamics that would occur for a lightweight, downwind 13‐MW rotor. The subscale rotor was mounted and field tested on the two‐bladed Controls Advanced Research Turbine (CART2) at the National Renewable Energy Laboratory's Flatiron Campus (NREL FC). The parked testing of these highly flexible blades included both pitch‐to‐run and pitch‐to‐feather configurations with the blades in the horizontal braked orientation. The collected experimental data includes the unsteady flapwise root bending moments and tip deflections as a function of inflow wind conditions. The bending moments are based on strain gauges located in the root section, whereas the tip deflections are captured by a video camera on the hub of the turbine pointed toward the tip of the blade. The experimental results are compared against computational predictions generated by FAST, a wind turbine simulation software, for the subscale and full‐scale models with consistent unsteady wind fields. FAST reasonably predicted the bending moments and deflections of the experimental data in terms of both the mean and standard deviations. These results demonstrate the efficacy of the first such aeroelastically scaled turbine test and demonstrate that a highly flexible lightweight downwind coned rotor can be designed to withstand extreme loads in parked conditions.
17 WIND ENERGY↗
How Frequent Will the Rarest Daily Rainfall Records of Hurricane Ida’s Remnants Be in the Future?
Abstract Gaining continued insights into the impact of global warming on the occurrence of hurricane-associated intense record downpours is essential for building climate resilient communities. This study investigates projected future changes in extreme rainfall over the Northeast United States, as represented by extreme daily amounts during Hurricane Ida in 2021. We used historical control simulations of Weather Research and Forecasting (WRF) Model generated from 40 years of weather events (1980–2014, 12 km) forced by the fifth generation European Centre for Medium-Range Weather Forecasts atmospheric reanalysis. These simulations are thermodynamically modified (2060–2100) via an imposed warming for the high-emission scenario of shared socioeconomic pathway (SSP585) from a range of general circulation models. Ground observations from the Global Historical Climatology Network (1950–2014) and WRF simulations (historical, 1980–2014, and future, 2060–2100) are integrated into a nonstationary generalized extreme value (GEV) framework to assess the frequency of Ida’s heaviest daily rain rates under the SSP585 scenario. Results show that Ida’s daily maximum rainfall recorded at different observation locations was higher than the single highest September daily maximum observed (1950–2014) for 5 out of 17 stations (∼30% of the stations). Ida-like extreme daily rain rates are projected to be, on average, more than 2 times more likely to occur at the end of the century in the simulations (with some regions as high as 5 times). This work demonstrates that integrating a high-resolution atmospheric model’s present-day and thermodynamically modified future simulations along with ground observations, within a nonstationary statistical framework, is crucial for understanding changing characteristics of extreme weather events. Significance Statement Daily scale extreme precipitation is expected to become more frequent and severe, as evidenced by observations and model simulations. While it is important to investigate how these intensifying heavy rainfall events affect current engineering standards, fewer studies have contextualized how warming impacts the most extreme rainfall from a single storm event relative to historical heavy downpours. In this study, we focused on the daily extreme rainfall associated with the extratropical transition of Hurricane Ida (2021), particularly over the northeastern United States—some of which exceeded the commonly used hydrologic design criteria for a 100-yr storm. Using a high-resolution atmospheric model simulation, we investigated how continued warming may influence the frequency of such daily rain rates. Under a high-emission scenario, these events are projected to become up to 5 times more likely at the end of the twenty-first century.
Dollan, Ishrat J↗
PETSc DMNetwork: A Library for Scalable Network PDE-Based Multiphysics Simulations
We present DMNetwork, a high-level package included in the PETSc library for the simulation of multiphysics phenomena over large-scale networked systems. The library aims at applications that have networked structures such as those in electrical, gas, and water distribution systems. DMNetwork provides data and topology management, parallelization for multiphysics systems over a network, and hierarchical and composable solvers to exploit the problem structure. DMNetwork eases the simulation development cycle by providing the necessary infrastructure through simple abstractions to define and query the network components. This article presents the design of DMNetwork, illustrates its user interface, and demonstrates its ability to solve multiphysics systems, such as an electric circuit, a network of power grid and water subnetworks, and transient hydraulic systems over large networks with more than 2 billion variables on extreme-scale computers using up to 30,000 processors.
extreme-scale computing↗