Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “computer system performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Institute for Sustained Performance, Energy, and Resilience (SUPER): University of Utah Final Report

Over the course of this project, computational scientists working on behalf of the Department of Energy's Office of Science (DOE SC) were exploiting a new generation of petascale computing resources to make previously inaccessible discoveries in a broad range of disciplines including physics, chemistry and material science. The computational systems underpinning this work increased in performance potential from tens to hundreds of Pflop/s, and evolved significantly from those in use at the beginning of the project in terms of available concurrency; wider use of accelerators such as graphics processing units(GPUs); and changes to the memory hierarchy including the incorporation of a new generation of persistent devices (e.g., phase change memory). To ensure that DOE's computational scientists can successfully exploit this emerging generation of high performance computing (HPC) systems, the University of Southern California (USC) assembled a broad team of computer scientists with the expertise to address the most pressing challenges of DOE's computing systems: (a) end-to-end performance optimization, including single-node performance, interprocessor communication, load balancing and I/O; (b) performance portability for heterogeneous systems, including accelerators; (c) minimizing energy consumption; (d) resilient computation; and, (e) exploiting new memory technology.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Design and implementation of I/O performance prediction scheme on HPC systems through large-scale log analysis

Abstract Large-scale high performance computing (HPC) systems typically consist of many thousands of CPUs and storage units used by hundreds to thousands of users simultaneously. Applications from large numbers of users have diverse characteristics, such as varying computation, communication, memory, and I/O intensity. A good understanding of the performance characteristics of each user application is important for job scheduling and resource provisioning. Among these performance characteristics, I/O performance is becoming increasingly important as data sizes rapidly increase and large-scale applications, such as simulation and model training, are widely adopted. However, predicting I/O performance is difficult because I/O systems are shared among all users and involve many layers of software and hardware stack, including the application, network interconnect, operating system, file system, and storage devices. Furthermore, updates to these layers and changes in system management policy can significantly alter the I/O behavior of applications and the entire system. To improve the prediction of the I/O performance on HPC systems, we propose integrating information from several different system logs and developing a regression-based approach to predict the I/O performance. Our proposed scheme can dynamically select the most relevant features from the log entries using various feature selection algorithms and scoring functions, and can automatically select the regression algorithm with the best accuracy for the prediction task. The evaluation results show that our proposed scheme can predict the write performance with up to 90% prediction accuracy and the read performance with up to 99% prediction accuracy using the real logs from the Cori supercomputer system at NERSC.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1966

Computer program determines performance efficiency of remote measuring systems

Computer programs control and evaluate instrumentation system performance for numerous rocket engine test facilities and prescribe calibration and maintenance techniques to maintain the systems within process specifications. Similar programs can be written for other test equipment in an industry such as the petrochemical industry.

Merewether, E. K.↗
DOE OSTI2025

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗
DOE OSTI2014

Collective input/output under memory constraints

Compared with current high-performance computing (HPC) systems, exascale systems are expected to have much less memory per node, which can significantly reduce necessary collective input/output (I/O) performance. In this study, we introduce a memory-conscious collective I/O strategy that takes into account memory capacity and bandwidth constraints. The new strategy restricts aggregation data traffic within disjointed subgroups, coordinates I/O accesses in intranode and internode layers, and determines I/O aggregators at run time considering memory consumption among processes. We have prototyped the design and evaluated it with commonly used benchmarks to verify its potential. The evaluation results demonstrate that this strategy holds promise in mitigating the memory pressure, alleviating the contention for memory bandwidth, and improving the I/O performance for projected extreme-scale systems. Given the importance of supporting increasingly data-intensive workloads and projected memory constraints on increasingly larger scale HPC systems, this new memory-conscious collective I/O can have a significant positive impact on scientific discovery productivity.

collective input/output↗
DOE OSTI2019

Can I/O Variability Be Reduced on QoS-Less HPC Storage Systems?

For a production high-performance computing (HPC) system, where storage devices are shared between multiple applications and managed in a best effort manner, I/O contention is often a major problem. In this paper, we propose a balanced messaging-based re-routing in conjunction with throttling at the middleware level. This work tackles two key challenges that have not been fully resolved in the past: whether I/O variability can be reduced on a QoS-less HPC storage system, and how to design a runtime scheduling system that can scale up to a large amount of cores. The proposed scheme uses a two-level messaging system to re-route I/O requests to a less congested storage location so that write performance is improved, while limiting the impact on read by throttling re-routing. An analytical model is derived to guide the setup of optimal throttling factor. We thoroughly analyze the virtual messaging layer overhead and explore whether the in-transit buffering is effective in managing I/O variability. Contrary to the intuition, in-transit buffer cannot completely solve the problem. It can reduce the absolute variability but not the relative variability. The proposed scheme is verified against a synthetic benchmark as well as being used by production applications.

Huang, Dan↗
DOE OSTI2018

Toward Transparent Data Management in Multi-layer Storage Hierarchy for HPC Systems

Upcoming exascale high performance computing (HPC) systems are expected to comprise multi-tier storage hierarchy, and thus will necessitate innovative storage and I/O mechanisms. Traditional disk and block-based interfaces and file systems face severe challenges in utilizing capabilities of storage hierarchies due to the lack of hierarchy support and semantic interfaces. Object-based and semantically-rich data abstractions for scientific data management on large scale systems offer a sustainable solution to these challenges. Such data abstractions can also simplify users involvement in data movement. Here, we take the first steps of realizing such an object abstraction and explore storage mechanisms for these objects to enhance I/O performance, especially for scientific applications. We explore how an object-based interface can facilitate next generation scalable computing systems by presenting the mapping of data I/O from two real world HPC scientific use cases: a plasma physics simulation code (VPIC) and a cosmology simulation code (HACC). Our storage model stores data objects in different physical organizations to support data movement across layers of memory/storage hierarchy. Our implementation sclaes well to 16K parallel processes, and compared to the state of the art, such as MPI-IO and HDF5, our object-based data abstractions and data placement strategy in multi-level storage hierarchy achieves up to 7 X I/O performance improvement for scientific data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Geomancy: Automated Performance Enhancement through Data Layout Optimization

Large distributed storage systems such as high- performance computing (HPC) systems used by national or international laboratories require sufficient performance and scale for demanding scientific workloads and must handle shifting workloads with ease. Ideally, data is placed in locations to optimize performance, but the size and complexity of large storage systems inhibit rapid effective restructuring of data layouts to maintain performance as workloads shift. To address these issues, we have developed Geomancy, a tool that models the placement of data within a distributed storage system and reacts to drops in performance. Using a combination of machine learning techniques suitable for temporal modeling, Geomancy determines when and where a bottleneck may happen due to changing workloads and suggests changes in the layout that mitigate or prevent them. Our approach to optimizing throughput offers benefits for storage systems such as avoiding potential bottlenecks and increasing overall I/O throughput from 11% to 30%.

Bel, Oceane M.↗
DOE OSTI2014

Non-volatile memory for checkpoint storage

A system, method and computer program product for supporting system initiated checkpoints in high performance parallel computing systems and storing of checkpoint data to a non-volatile memory storage device. The system and method generates selective control signals to perform checkpointing of system related data in presence of messaging activity associated with a user application running at the node. The checkpointing is initiated by the system such that checkpoint data of a plurality of network nodes may be obtained even in the presence of user applications running on highly parallel computers that include ongoing user messaging activity. In one embodiment, the non-volatile memory is a pluggable flash memory card.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

FY 18 Status Report on the Computing Systems for the Yucca Mountain Project TSPA-LA Models and Inventory of Software used for Process Models

Sandia National Laboratories (SNL) continued evaluation of total system performance assessment (TSPA) computing systems for the previously considered Yucca Mountain Project (YMP). This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment (TSPA) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. This work is a continuation of the ongoing readiness evaluation reported in Lee and Hadgu (2014), Hadgu et al. (2015) and Hadgu and Appel (2016), and Hadgu et al. (2017). The TSPA computing hardware (2014 server cluster -CL2014) and storage system described in Hadgu et al. (2015) were used for the current analysis. One floating license of Gold Sim with Versions 9.60.300, 10.5, 11.1 and 12.0 was installed on the cluster head node, and its distributed processing capability was mapped on the cluster processors. Other supporting software were tested and installed to support the TSPA-type analysis on the server cluster. The FY18 task included developing an inventory of software used for the Yucca Mountain Project process models and preliminary assessment of status of the software; enhancing security of the cluster and setting a backup system. The 2014 server cluster and supporting software systems are fully operational to support TSPA-LA type analysis.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

FY19 Status Report on the Computing Systems for the Yucca Mountain Project TSPA-LA Models and Preliminary Testing of a Selected Process Model

Sandia National Laboratories continued evaluation of total system performance assessment (TSPA) computing systems for the previously considered Yucca Mountain Project. This was done to maintain the operational readiness of the computing infrastructure (computer hardware and software) and knowledge capability for total system performance assessment) type analysis, as directed by the National Nuclear Security Administration (NNSA), DOE 2010. The FY19 task included continued operation of the cluster; maintenance of the TSPA-LA models (with Gold Sim 9.60.300); preliminary assessment of the status of the Infiltration Model (a process model that feeds the TSPA-LA). In addition, precautionary actions were needed to extend the life of the cluster hardware. To do that, three new nodes were added to the cluster. In the event any of the original nodes fail they will be replaced with the new nodes, thereby maintaining the core capability. The 2014 cluster and supporting software systems are currently fully operational to support TSPA-LA type analysis.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2002

Monitoring the Microgravity Environment Quality On-board the International Space Station Using Soft Computing Techniques: Preliminary System Performance Results - Part 2

This paper presents the preliminary performance results of the artificial intelligence monitoring system in full operational mode using near real time acceleration data downlinked from the International Space Station. Preliminary microgravity environment characterization analysis result for the International Space Station (Increment-2), using the monitoring system is presented. Also, comparison between the system predicted performance based on ground test data for the US laboratory "Destiny" module and actual on-orbit performance, using measured acceleration data from the U.S. laboratory module of the International Space Station is presented. Finally, preliminary on-orbit disturbance magnitude levels are presented for the Experiment of Physics of Colloids in Space, which are compared with on ground test data. The ground test data for the Experiment of Physics of Colloids in Space were acquired from the Microgravity Emission Laboratory, located at the NASA Glenn Research Center, Cleveland, Ohio. The artificial intelligence was developed by the NASA Glenn Principal Investigator Microgravity Services Project to help the principal investigator teams identify the primary vibratory disturbance sources that are active, at any moment of time, on-board the International Space Station, which might impact the microgravity environment their experiments are exposed to. From the Principal Investigator Microgravity Services' web site, the principal investigator teams can monitor via a dynamic graphical display, implemented in Java, in near real time, which event(s) is/are on, such as crew activities, pumps, fans, centrifuges, compressor, crew exercise, structural modes, etc., and decide whether or not to run their experiments, whenever that is an option, based on the acceleration magnitude and frequency sensitivity associated with that experiment. This monitoring system detects primarily the vibratory disturbance sources. The system has built-in capability to detect both known and unknown vibratory disturbance sources. Several soft computing techniques such as Kohonen's Self-Organizing Feature Map, Learning Vector Quantization, Back-Propagation Neural Networks, and Fuzzy Logic were used to design the system.

Jules, Kenol↗
NASA NTRS1985

Research in the design of high-performance reconfigurable systems

Computer aided design and computer aided manufacturing have the potential for greatly reducing the cost and lead time in the development of VLSI components. This potential paves the way for the design and fabrication of a wide variety of economically feasible high level functional units. It was observed that current computer systems have only a limited capacity to absorb new VLSI component types other than memory, microprocessors, and a relatively small number of other parts. The first purpose is to explore a system design which is capable of effectively incorporating a considerable number of VLSI part types and will both increase the speed of computation and reduce the attendant programming effort. A second purpose is to explore design techniques for VLSI parts which when incorporated by such a system will result in speeds and costs which are optimal. The proposed work may lay the groundwork for future efforts in the extensive simulation and measurements of the system's cost effectiveness and lead to prototype development.

Mcewan, S. D.↗
NASA NTRS1993

Measurements over distributed high performance computing and storage systems

A strawman proposal is given for a framework for presenting a common set of metrics for supercomputers, workstations, file servers, mass storage systems, and the networks that interconnect them. Production control and database systems are also included. Though other applications and third part software systems are not addressed, it is important to measure them as well.

Williams, Elizabeth↗
NASA NTRS1993

New computing systems, future computing environment, and their implications on structural analysis and design

Recent advances in computer technology that are likely to impact structural analysis and design of flight vehicles are reviewed. A brief summary is given of the advances in microelectronics, networking technologies, and in the user-interface hardware and software. The major features of new and projected computing systems, including high performance computers, parallel processing machines, and small systems, are described. Advances in programming environments, numerical algorithms, and computational strategies for new computing systems are reviewed. The impact of the advances in computer technology on structural analysis and the design of flight vehicles is described. A scenario for future computing paradigms is presented, and the near-term needs in the computational structures area are outlined.

Noor, Ahmed K.↗
DOE OSTI2017

Preemptive resource management for dynamically arriving tasks in an oversubscribed heterogeneous computing system

We design resource management heuristics that assign serial tasks to the nodes of a heterogeneous high performance computing (HPC) system. The value of completing these tasks is modeled using monotonically decreasing utility functions that represent the time-varying importance of the task. The value of completing a task is equal to its utility function at the time of its completion. The overall performance of this system is measured using the total utility earned by all tasks during some interval of time. To maximize the performance of such a system where the preemption of tasks is possible, we have designed, analyzed, and compared a set of resource allocation heuristic techniques. We combine two utility-aware heuristics with three different preemption techniques to create six preemption-capable heuristics. We also consider the two utility-aware heuristics without preemption. We use simulation studies to evaluate this set of eight heuristics and compare them with an FCFS heuristic, which is often used in real systems, and random assignments. In general, our set of eight heuristics is able to significantly outperform the comparison heuristics, and the preemption-capable heuristics are able to significantly increase the utility earned compared to the heuristics that do not use preemption. We analyze the performance tradeoffs among the different preemption-capable heuristics under a variety of oversubscribed workload environments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

The Institute for Sustained Performance, Energy, and Resilience Final Project Report

Over the course of the five years of the Scientific Discovery through Advanced Computation (SciDAC-3) program, computational scientists working on behalf of the Department of Energy's Office of Science (DOE SC) exploited a new generation of petascale computing resources to make previously inaccessible discoveries in a broad range of disciplines including physics, chemistry, and materials science. The computing systems underpinning this work increased in performance potential from tens to hundreds of PFlop/s, but in the process evolving significantly from architectures in 2011, when SciDAC-3 began. Although Moore's law continues unabated, the end of Dennard scaling has necessitated a fundamental shift in computer architecture focused on power efficiency. To that end, processors are increasingly varied as they strive to satisfy performance, productivity, reliability, and energy efficiency in the face of divergent computational requirements. The diversity among these machines presents a number of challenges to merely porting today's scientific applications, much less achieving good performance. Therefore, at the beginning of SciDAC-3, USC and its collaborators anticipated that both vastly increased scale (e.g., more chips, more cores per chip, wider SIMD) and heterogeneity would exacerbate performance optimization challenges while simultaneously promoting the issues of energy and heterogeneity will exacerbate performance optimization challenges while simultaneously promoting the issues of energy consumption and resilience to the forefront. To ensure that DOE's computational scientists could successfully exploit emerging high performance computing (HPC) systems, DOE SC’s Advanced Scientific Computing Research Program (ASCR) funded the SciDAC-3 Institute for Sustained Performance, Energy, and Resilience (SUPER). Led by USC, this was a broadly based project with expertise in compilers and system tools, performance engineering, energy management, and resilience. SUPER followed the successful model developed in the previous SciDAC-2 Performance Engineering Research Institute (PERI) of leveraging the research investments DOE and others have made and integrating the results to create new capabilities beyond the reach of any one group. To accomplish this, SUPER was organized into multi-institution teams conducting the research activities in performance portability, energy efficiency, exascale resilience, and overall optimization. SUPER researchers also collaborated with other SciDAC-3 institutes, 13 of the 19 Scientific Computation Application Partnerships (SAP), as well as the broader DOE computational sciences community. These collaborations both focused our research on the real challenges facing petascale scientific computing as well as ensured the broad and immediate impact of the results of our research.

97 MATHEMATICS AND COMPUTING↗