Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Staging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Understanding the Impact of Data Staging for Coupled Scientific Workflows

We report the rate of data generated by cutting-edge experimental science facilities and large-scale simulations enabled by current high-performance computing (HPC) systems has continued to grow at a far greater pace than the development of the network and storage capabilities on which these systems rely. To cope with this challenge, scientist are moving toward the creation of autonomous experiments and HPC simulations using machine learning. However, efficiently moving, storing, and processing large amounts of data away from the point of origin presents an incredible challenge. In-memory computing, in situ analysis, data staging, and data streaming are recognized viable alternatives to traditional file-based methods for transferring data between coupled workflows. However, the performance trade-offs and limitations for these methods are not fully understood when used in HPC applications. This article presents a comprehensive performance assessment of the current solutions for data staging when applied to applications that are not necessary I/O intensive which makes them not ideal candidates for these methods. Our study is based on experiments running at scale on Oak Ridge National Laboratory's Summit supercomputer using applications and simulations that cover typical computational motifs and patterns. We investigated the usability and cost/benefit trade-offs of staging algorithms for HPC applications under different scenarios and highlight opportunities for optimizing the dataflow between coupled simulation workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

CoREC: Scalable and Resilient In-memory Data Staging for In-situ Workflows

The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. Here, the experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1956

Experimental Investigation of a Five-stage Axial-flow Research Compressor with Transonic Rotors in All Stages III : Interstage Data and Individual Stage Performance Characteristics

For use in computing the detailed performance parameters of individual blade rows of a multistage compressor, radial distributions of total pressure, total temperature, static pressure and air-flow angle are tabulated. The data cover a range of air flow from choke to the approximate surge limit for equivalent rotor speeds from 70 to 100 percent of design. Equivalent stage performance curves determined from the radial survey measurements indicate both the individual performance and the matching of the stages at the speeds investigated.

Sandercock, Donald M↗
NASA NTRS1972

Single stage evaluation of highly loaded high Mach number compressor stages. 6: Data and performance of cantilevered stator

A compressor stage with a rotor tip speed of 1600 ft/sec was tested to evaluate its performance with a cantilevered stator and a rotating inner shroud beneath the stator. Both the rotor blades and the stator vanes were composed of multiple circular arc airfoil sections. Comparison of data taken during this test of the cantilevered stator and previous tests with the same compressor and airfoil geometry in a shroud stator configuration showed only slight differences in stage performance with no significant effect on overall efficiency. However, the severity of the stator wake near the rotating hub was decreased at all flows including the near surge condition. Stall and wise open discharge corrected weight flows were the same as for the shrouded stator configuration.

Merrow, A. S.↗
NASA NTRS1991

Analyzing the I/O behavior of supercomputer applications

Supercomputer applications of I/Os are classified here into three categories - required, checkpoint, and data staging - and it is shown how memory size and CPU speed are likely to affect each category. A data analysis shows that data staging I/O dominates when it is present. If an application requires data staging I/O, its read/write ratio is greater than one; otherwise, its read/write ratio is less than one. This information can be used to anticipate a program's I/O demands and thus reduce the load on the I/O system.

Miller, Ethan L.↗
NASA NTRS1972

Upper stage technology evaluation studies

Studies to evaluate advanced technology relative to chemical upper stages and orbit-to-orbit stages are reported. The work described includes: development of LH2/LOX stage data, development of data to indicate stage sensitivity to engine tolerance, modified thermal routines to accommodate storable propellants, added stage geometries to computer program for monopropellant configurations, determination of the relative gain obtainable through improvement of stage mass fraction, future propulsion concepts, effect of ultrahigh chamber-pressure increases, and relative gains obtainable through improved mass fraction.

Source record↗
NASA NTRS1972

Single-stage evaluation of highly-loaded high-Mach-number compressor stages 5. Data and performance of baseline, corner-blow wall suction and combined corner blow wall suction stator

A single-stage compressor with a rotor tip speed of 1600 ft/sec and a 0.5 hub tip ratio was used to investigate the effects of several stator endwall treatment methods on stage range and performance. These endwall treatment methods consisted of stator corner-blow, annular wall suction upstream of stator leading edge, and combined corner-blow and annular wall suction. The overall stage performance with corner blow was essentially the same as the baseline performance. The performance for the annular wall suction and the combined corner-blow and wall suction showed a reduction in peak efficiency of 2.5 percentage points compared to the baseline data.

Nikkanen, J. P.↗