Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “I/O performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2013

A Lightweight I/O Scheme to Facilitate Spatial and Temporal Queries of Scientific Data Analytics

In the era of petascale computing, more scientific applications are being deployed on leadership scale computing platforms to enhance the scientific productivity. Many I/O techniques have been designed to address the growing I/O bottleneck on large-scale systems by handling massive scientific data in a holistic manner. While such techniques have been leveraged in a wide range of applications, they have not been shown as adequate for many mission critical applications, particularly in data post-processing stage. One of the examples is that some scientific applications generate datasets composed of a vast amount of small data elements that are organized along many spatial and temporal dimensions but require sophisticated data analytics on one or more dimensions. Including such dimensional knowledge into data organization can be beneficial to the efficiency of data post-processing, which is often missing from exiting I/O techniques. In this study, we propose a novel I/O scheme named STAR (Spatial and Temporal AggRegation) to enable high performance data queries for scientific analytics. STAR is able to dive into the massive data, identify the spatial and temporal relationships among data variables, and accordingly organize them into an optimized multi-dimensional data structure before storing to the storage. This technique not only facilitates the common access patterns of data analytics, but also further reduces the application turnaround time. In particular, STAR is able to enable efficient data queries along the time dimension, a practice common in scientific analytics but not yet supported by existing I/O techniques. In our case study with a critical climate modeling application GEOS-5, the experimental results on Jaguar supercomputer demonstrate an improvement up to 73 times for the read performance compared to the original I/O method.

Temporal Queries↗
NASA NTRS1996

Cooperative high-performance storage in the accelerated strategic computing initiative

The use and acceptance of new high-performance, parallel computing platforms will be impeded by the absence of an infrastructure capable of supporting orders-of-magnitude improvement in hierarchical storage and high-speed I/O (Input/Output). The distribution of these high-performance platforms and supporting infrastructures across a wide-area network further compounds this problem. We describe an architectural design and phased implementation plan for a distributed, Cooperative Storage Environment (CSE) to achieve the necessary performance, user transparency, site autonomy, communication, and security features needed to support the Accelerated Strategic Computing Initiative (ASCI). ASCI is a Department of Energy (DOE) program attempting to apply terascale platforms and Problem-Solving Environments (PSEs) toward real-world computational modeling and simulation problems. The ASCI mission must be carried out through a unified, multilaboratory effort, and will require highly secure, efficient access to vast amounts of data. The CSE provides a logically simple, geographically distributed, storage infrastructure of semi-autonomous cooperating sites to meet the strategic ASCI PSE goal of highperformance data storage and access at the user desktop.

Gary, Mark↗
NASA NTRS1996

Performance of the Galley Parallel File System

As the input/output (I/O) needs of parallel scientific applications increase, file systems for multiprocessors are being designed to provide applications with parallel access to multiple disks. Many parallel file systems present applications with a conventional Unix-like interface that allows the application to access multiple disks transparently. This interface conceals the parallism within the file system, which increases the ease of programmability, but makes it difficult or impossible for sophisticated programmers and libraries to use knowledge about their I/O needs to exploit that parallelism. Furthermore, most current parallel file systems are optimized for a different workload than they are being asked to support. We introduce Galley, a new parallel file system that is intended to efficiently support realistic parallel workloads. Initial experiments, reported in this paper, indicate that Galley is capable of providing high-performance 1/O to applications the applications that rely on them. In Section 3 we describe that access data in patterns that have been observed to be common.

Nieuwejaar, Nils↗
NASA NTRS2019

Leveraging STARE for Co-aligned Data Locality with netCDF and Python MPI

We have leveraged STARE indexing to package partitioned data chunks from diverse datasets into netCDF files, distributed them on a cluster of 16 lightweight nodes with their placements spatiotemporally co-aligned, and demonstrated a few integrative analyses using netCDF parallel I/O and Python MPI, with single-user performance and scalability comparable to, or even better than, that of a parallel array database management system (ADBMS) such as SciDB. However, records of the node location and STARE index ranges for each data chunk, similar to the chunk maps of SciDB, must be maintained and consulted by the I/O and analysis code for coordinating the analytic operations in parallel, in order to achieve the good performance and scalability.

Kwo-Sen Kuo↗
NASA NTRS1992

A real-time diagnostic and performance monitor for UNIX

There are now over one million UNIX sites and the pace at which new installations are added is steadily increasing. Along with this increase, comes a need to develop simple efficient, effective and adaptable ways of simultaneously collecting real-time diagnostic and performance data. This need exists because distributed systems can give rise to complex failure situations that are often un-identifiable with single-machine diagnostic software. The simultaneous collection of error and performance data is also important for research in failure prediction and error/performance studies. This paper introduces a portable method to concurrently collect real-time diagnostic and performance data on a distributed UNIX system. The combined diagnostic/performance data collection is implemented on a distributed multi-computer system using SUN4's as servers. The approach uses existing UNIX system facilities to gather system dependability information such as error and crash reports. In addition, performance data such as CPU utilization, disk usage, I/O transfer rate and network contention is also collected. In the future, the collected data will be used to identify dependability bottlenecks and to analyze the impact of failures on system performance.

Dong, Hongchao↗
NASA NTRS1975

System applications of the fault tolerant memory

Conventional memory technologies currently employed in aerospace applications contribute at least fifty percent to system unreliability (where the system includes CPU, I/O and memory). A fault tolerant memory performs both error correction and memory replacement at the bit plane level. To determine the effects of system design of using a fault tolerant memory in space applications, analysis was performed to determine tradeable hardware configurations that meet the reliability goals of each program. The candidate configurations, which included redundant elements of the computer system with both conventional and fault tolerant memories, were then traded in terms of selection criteria of cost, weight, volume, and power. These trade studies demonstrated that a fault tolerant memory provided significant advantages in terms of cost, weight, and volume. The memory selected for this analysis was a recently developed five fault tolerant memory.

Murphy, L. J.↗
NASA NTRS1992

Data storage and retrieval system abstract

The STX mass storage system design is intended for environments requiring high speed access to large volumes of data (terabyte and greater). Prior to commitment to a product design plan, STX conducted an exhaustive study of the commercially available off-the-shelf hardware and software. STX also conducted research into the area of emerging technologies in networks and storage media so that the design could easily accommodate new interfaces and peripherals as they came on the market. All the selected system elements were brought together in a demo suite sponsored jointly by STX and ALLIANT where the system elements were evaluated based on actual operation using a client-server mirror image configuration. Testing was conducted to assess the various component overheads and results were compared against vendor data claims. The resultant system, while adequate to meet our capacity requirements, fell short of transfer speed expectations. A product team lead by STX was assembled and chartered with solving the bottleneck issues. Optimization efforts yielded a 60 percent improvement in throughput performance. The ALLIANT computer platform provided the I/O flexibility needed to accommodate a multitude of peripheral interfaces including the following: up to twelve 25MB/s VME I/O channels; up to five HiPPI I/O full duplex channels; IPI-s, SCSI, SMD, and RAID disk array support; standard networking software support for TCP/IP, NFS, and FTP; open architecture based on standard RISC processors; and V.4/POSIX-based operating system (Concentrix). All components including the software are modular in design and can be reconfigured as needs and system uses change. Users can begin with a small system and add modules as needed in the field. Most add-ons can be accomplished seamlessly without revision, recompilation or re-linking of software.

Matheson, Barbara↗
NASA NTRS1994

Dynamic file-access characteristics of a production parallel scientific workload

Multiprocessors have permitted astounding increases in computational performance, but many cannot meet the intense I/O requirements of some scientific applications. An important component of any solution to this I/O bottleneck is a parallel file system that can provide high-bandwidth access to tremendous amounts of data in parallel to hundreds or thousands of processors. Most successful systems are based on a solid understanding of the expected workload, but thus far there have been no comprehensive workload characterizations of multiprocessor file systems. This paper presents the results of a three week tracing study in which all file-related activity on a massively parallel computer was recorded. Our instrumentation differs from previous efforts in that it collects information about every I/O request and about the mix of jobs running in a production environment. We also present the results of a trace-driven caching simulation and recommendations for designers of multiprocessor file systems.

Kotz, David↗
NASA NTRS1979

Discharge characteristics of 300 ampere-hour Ni-Zn traction cells

Preliminary tests were performed on 300 amphere-hour nickel-zinc cells containing the Lewis improved inorganic-organic (I/O) separator. These cells also have other design features included to optimize performance and cycle life. The tests carried out were formation tests and characteristic discharge tests. Information obtained include case temperature and maximum power delivered.

Ewashinka, J. G.↗
NASA NTRS2012

High-Performance, Multi-Node File Copies and Checksums for Clustered File Systems

Modern parallel file systems achieve high performance using a variety of techniques, such as striping files across multiple disks to increase aggregate I/O bandwidth and spreading disks across multiple servers to increase aggregate interconnect bandwidth. To achieve peak performance from such systems, it is typically necessary to utilize multiple concurrent readers/writers from multiple systems to overcome various singlesystem limitations, such as number of processors and network bandwidth. The standard cp and md5sum tools of GNU coreutils found on every modern Unix/Linux system, however, utilize a single execution thread on a single CPU core of a single system, and hence cannot take full advantage of the increased performance of clustered file systems. Mcp and msum are drop-in replacements for the standard cp and md5sum programs that utilize multiple types of parallelism and other optimizations to achieve maximum copy and checksum performance on clustered file systems. Multi-threading is used to ensure that nodes are kept as busy as possible. Read/write parallelism allows individual operations of a single copy to be overlapped using asynchronous I/O. Multinode cooperation allows different nodes to take part in the same copy/checksum. Split-file processing allows multiple threads to operate concurrently on the same file. Finally, hash trees allow inherently serial checksums to be performed in parallel. Mcp and msum provide significant performance improvements over standard cp and md5sum using multiple types of parallelism and other optimizations. The total speed-ups from all improvements are significant. Mcp improves cp performance over 27x, msum improves md5sum performance almost 19x, and the combination of mcp and msum improves verified copies via cp and md5sum by almost 22x. These improvements come in the form of drop-in replacements for cp and md5sum, so are easily used and are available for download as open source software at http://mutil.sourceforge.net.

Kolano, Paul Z.↗
NASA NTRSDate not supplied

NASA and Blue Origin Collaborative Assessment of Precision Landing Algorithms and Computing

NASA’s Safe and Precise Landing Integrated Capabilities Evolution (SPLICE) project is developing sensor, algorithm, and compute technologies for precision landing and hazard avoidance. These technologies are being tested as an integrated Precision Landing and Hazard Avoidance (PL&HA) system on Blue Origin’s New Shephard suborbital vehicle. A key goal for the computing element of this technology development is to characterize the performance of the SPLICE software workloads on the project’s Descent and Landing Computer (DLC). The DLC is a multi-core processor designed as a surrogate for NASA’s High-Performance Space Computer (HPSC). Measurements of the SPLICE workload performance on the DLC provides NASA insight on how PL&HA capabilities will perform on the HPSC, and guidance on how the SPLICE algorithms can be implemented to best utilize the DLC platform. This insight can also be used to derive requirements to guide trade studies on candidate computing architectures, for use on platforms like Blue Moon. NASA and Blue Origin are collaborating under an agreement to pursue this mutual benefit. Performance metrics collected are based on measurement of common compute resources such as percentage used of memory bandwidth, I/O utilization, interrupt latency, and kernel vs. user space code residency. Where possible existing performance counters and metrics that are part of the operating system kernel are used. As the design has a significant FPGA component, performance counters are identified and instantiated in the fabric to measure DMA performance and interface metrics. Collection of metrics is performed on the DLC with a representative workload that simulates a full landing cycle of the Blue Origin New Shepard vehicle. Consideration is given to the other compute implementations and whether they can run SPLICE algorithms at the same rate and with the same latency as the DLC. One option being considered is the use of a RISC-V soft core instantiated in a radiation resilient FPGA fabric such as the Xilinx KU60. Select algorithms from the SPLICE code will be run for comparison with the DLC. This paper describes how the DLC is instrumented to collect performance measurements of the SPLICE workloads, preliminary results from these measurements, and their implications on SPLICE algorithm implementation. The results of experimentation to derive candidate requirements for architecture trades on a PL&HA computing system are also presented.

computer performance↗
NASA NTRS1988

The Transportable Applications Environment - An interactive design-to-production development system

An account is given of the design philosophy and architecture of the Transportable Applications Environment (TAE), an executive program binding a system of applications programs into a single, easily operable whole. TAE simplifies the job of a system developer by furnishing a stable framework for system-building; it also integrates system activities, and cooperates with the host operating system in order to perform such functions as task-scheduling and I/O. The initial TAE human/computer interface supported command and menu interfaces, data displays, parameter-prompting, error-reporting, and online help. Recent extensions support graphics workstations with a window-based, modeless user interface.

Perkins, Dorothy C.↗
NASA NTRS1991

Performance of a disk array prototype

A prototype disk array's component-performance limits are examined using SCSI bus traces, Sprite operating system traces, and user programs. The array performs successfully for a workload of small, random I/O operations, achieving 275 I/Os per second on 14 disks before the Sun4/280 host becomes CPU-limited. The prototype is less successful in delivering high throughput for large, sequential operations. Memory system contention on the Sun4/280 host limits throughput to 2.3 MBytes/sec under the Sprite Operating System. Throughput is also limited by the bandwidth supported by the VME backplane, disk controller and disks, and overheads associated with the SCSI protocol. We conclude that merely using a powerful host CPU and many disks will not provide the full bandwidth possible from disk arrays. Host memory bandwidth and throughput of disk controllers are equally important. In addition, operating systems should avoid unnecessary copy and cache flush operations that can saturate the host memory system.

Chervenak, Ann L.↗
NASA NTRS2015

Identifying the "Right Stuff": An Exploration-Focused Astronaut Job Analysis

Industrial and organizational (I/O) psychologists play a key role in NASA astronaut candidate selection through the identification of the competencies necessary to successfully engage in the astronaut job. A set of psychosocial competencies, developed by I/O psychologists during a prior job analysis conducted in 1996 and updated in 2003, were identified as necessary for individuals working and living in the space shuttle and on the International Space Station (ISS). This set of competencies applied to the space shuttle and applies to current ISS missions, but may not apply to longer-duration or long-distance exploration missions. With the 2015 launch of the first 12- month ISS mission and the shift in the 2020s to missions beyond low earth orbit, the type of missions that astronauts will conduct and the environment in which they do their work will change dramatically, leading to new challenges for these crews. To support future astronaut selection, training, and research, I/O psychologists in NASA's Behavioral Health and Performance (BHP) Operations and Research groups engaged in a joint effort to conduct an updated analysis of the astronaut job for current and future operations. This project will result in the identification of behavioral competencies critical to performing the astronaut job, along with relative weights for each of the identified competencies, through the application of job analysis techniques. While this job analysis is being conducted according to job analysis best practices, the project poses a number of novel challenges. These challenges include the need to identify competencies for multiple mission types simultaneously, to evaluate jobs that have no incumbents as they have never before been conducted, and working with a very limited population of subject matter experts. Given these challenges, under the guidance of job analysis experts, we used the following methods to conduct the job analysis and identify the key competencies for current and potential future missions.

Barrett, J. D.↗
NASA NTRS2019

Using Big Data Technologies with Earth Science Data in HDF5: HDF5 Scalable Solutions

HDF5 (Hierarchical Data Format 5) is open-source, high-performance software that consists of an abstract data model, library, and fileformat used for storing and managing extremely large and/or complex data collections. NASA Earth Observing System (EOS) Data and Information Systems use HDF5 as an archival format to store remote sensing data from EOS satellites. HDF5 is also used to store other types of Geoscience and Strophysical data, e.g., seismic data and data from Low-Frequency Array (LOFAR) radio telescopes. Data stored in HDF5 has reached tens of petabytes and is growing at an accelerated rate.With the growing amout of HDF5 Earth Science data to analyze and process, scientists need to adopt big data technologies including new storage paradigms such as cloud and object storage. To run models and perform data analysis they also need to utilizied efficient and diverse ways to access data, from high-performance computing's (HPC) Message Passing Interface (MPI) I/O and deep memory hierarchies (DMH) to non-HPC frameworks such as Apache Hadoop, Spark, and Drill. The HDF Group continually works to enable usage of big data technologies in HDF software.

Knox, Larry↗
NASA NTRS1992

Efficient computation of inviscid flow fields around complex configurations using a multiblock multigrid method

The applicability of a multigrid technique to block-structured, body-fitted meshes is examined focusing on three different strategies. In the first strategy data are exchanged between blocks in each stage of a five-stage Runge-Kutta time-stepping scheme which keeps a possible time lag between blocks to a minimum, but requires a large amount of I/O operations and storage. The second strategy is based on performing a complete Runge-Kutta cycle within a block before switching to the next. In the third strategy both a complete Runge-Kutta cycle and the residual evaluation for the restriction operator are done within a block, allowing a minimum of I/O and storage. The inviscid flow around a wing-body/engine-pylon configuration was computed on a mesh consisting of 11 computational blocks. It was found that both the first and the second strategies delivered converged results, but the third failed due to larger time lag between blocks.

Rossow, C.-C.↗
NASA NTRS1983

Signal processor architecture for backscatter radars

Real time signal processing for backscatter radars which requires computational throughput and I/O rates is discussed. The operations that are usually performed in real time are highly repetitive simple accumulations of samples or of products of samples. The control logic does not depend on the values of the data and general purpose computers are not required for the initial high speed processing. The implications of these facts on the architectures of preprocessors for backscatter radars are explored and applied to the design of the Radar Signal Compender.

Swartz, W. E.↗
NASA NTRS1993

Storage media pipelining: Making good use of fine-grained media

This paper proposes a new high-performance paradigm for accessing removable media such as tapes and especially magneto-optical disks. In high-performance computing the striping of data across multiple devices is a common means of improving data transfer rates. Striping has been used very successfully for fixed magnetic disks improving overall system reliability as well as throughput. It has also been proposed as a solution for providing improved bandwidth for tape and magneto-optical subsystems. However, striping of removable media has shortcomings, particularly in the areas of latency to data and restricted system configurations, and is suitable primarily for very large I/Os. We propose that for fine-grained media, an alternative access method, media pipelining, may be used to provide high bandwidth for large requests while retaining the flexibility to support concurrent small requests and different system configurations. Its principal drawback is high buffering requirements in the host computer or file server. This paper discusses the possible organization of such a system including the hardware conditions under which it may be effective, and the flexibility of configuration. Its expected performance is discussed under varying workloads including large single I/O's and numerous smaller ones. Finally, a specific system incorporating a high-transfer-rate magneto-optical disk drive and autochanger is discussed.

Vanmeter, Rodney↗