SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
POSIX is an evolving set of operating system interface standards, whose parts are in varying stages of production in a number of standards working groups. This presentation separates the real-time POSIX standards work in process from the rest and provides an overview of the purpose, status, dependencies, content, and project schedule of each.
This is a set of POSIX device driver level abstractions in the RTEMS RTOS (Real-Time Executive for Multiprocessor Systems real-time operating system) to SBand radio hardware devices that have been instantiated in an FPGA (field-programmable gate array). These include A/D (analog-to-digital) sample capture, D/A (digital-to-analog) sample playback, PLL (phase-locked-loop) tuning, and PWM (pulse-width-modulation)-controlled gain. This software interfaces to Sband radio hardware in an attached Xilinx Virtex-2 FPGA. It uses plug-and-play device discovery to map memory to device IDs. Instead of interacting with hardware devices directly, using direct-memory mapped access at the application level, this driver provides an application programming interface (API) offering that easily uses standard POSIX function calls. This simplifies application programming, enables portability, and offers an additional level of protection to the hardware. There are three separate device drivers included in this package: sband_device (ADC capture and DAC playback), pll_device (RF front end PLL tuning), and pwm_device (RF front end AGC control).
Explore the source record for details and available documents.
EScp is a high speed transfer tool with a similar command line syntax to scp. Unlike SCP it is designed to transfer files at high speed, thus far we have been able to show 100gbit/s transfers, although I expect that the throughput should scale in proportion to the network interface, i.e. I expect 400gbit/s performance on our 400gbit/s test bed. EScp achieves good performance through an innovative design (multithreaded, zero copy transfers), along with pluggable filters and I/O engines. As an example, you can switch from POSIX i/O to UIO by checking a different engine. It also natively supports encryption, and cheksums for file verification and transport security. AAA is through standard SSH (same as SCP). By taking advantage of filters, EScp supports transferring unstructured data and/or I/O to non-posix data sources. Examples include streaming data (i.e. from equipment), transferring data to the cloud, and/or supporting non-posix file systems (like HPSS).
We benchmark the DAOS filesystem on Argonne's Aurora supercomputer (127 nodes, 4,064 targets) using fio, IOR, mdtest, and IO500 to characterize I/O and metadata performance across the DFS API and DFuse+POSIX. Single-client fio shows POSIX bandwidth saturating at 1–2 MiB I/O sizes, with write-heavy workloads outperforming reads. Multi-node IOR shows DFS bandwidth scaling well up to ~32 tasks/node, with write latency growing faster than read latency. An 8-node IO500 evaluation shows DFS achieving ~5x higher bandwidth and ~190x higher IOPS than POSIX. Results indicate DAOS is well-suited to read-heavy workloads like AI training data loading, given appropriately sized transfers and concurrency.
NASA is seeking an operating system interface definition (OSID) for the Space Station Program (SSP) in order to take advantage of the commercial off-the-shelf (COTS) products available today and the many that are expected in the future. NASA would also like to avoid the reliance on any one source for operating systems, information system, communication system, or instruction set architecture. The use of the Portable Operating System Interface for Computer Environments (POSIX) is examined as a possible solution to this problem. Since Ada is already the language of choice for SSP, the question of an Ada/POSIX binding is addressed. The intent of the binding is to provide access to the POSIX standard operation system (OS) interface and environment, by which application portability of Ada applications will be supported at the source code level. A guiding principle of Ada/POSIX binding development is a clear conformance of the Ada interface with the functional definition of POSIX. The interface is intended to be used by both application developers and system implementors. The objective is to provide a standard that allows a strictly conforming application source program that can be compiled to execute on any conforming implementation. Special emphasis is placed on first providing those functions and facilities that are needed in a wide variety of commercial applications
The Multithreaded Microbenchmarks for Processor-In-Memory (PIM) Compilers, Simulators, and Hardware are computer programs arranged in a series for use in testing the performances of PIM computing systems, including compilers, simulators, and hardware. The programs at the beginning of the series test basic functionality; the programs at subsequent positions in the series test increasingly complex functionality. The programs are intended to be used while designing a PIM system, and can be used to verify that compilers, simulators, and hardware work correctly. The programs can also be used to enable designers of these system components to examine tradeoffs in implementation. Finally, these programs can be run on non-PIM hardware (either single-threaded or multithreaded) using the POSIX pthreads standard to verify that the benchmarks themselves operate correctly. [POSIX (Portable Operating System Interface for UNIX) is a set of standards that define how programs and operating systems interact with each other. pthreads is a library of pre-emptive thread routines that comply with one of the POSIX standards.]
Abstract—High-performance computing (HPC) applications generate and consume substantial amounts of data, typically managed by parallel file systems. These applications access file systems either through the POSIX interface or by using highlevel I/O libraries. While the POSIX consistency model remains dominant in HPC, emerging file systems and popular I/O libraries increasingly adopt alternative consistency models that relax semantics in various ways, creating significant challenges for correctness and portability. This paper addresses these challenges by proposing a trace-driven I/O consistency verification workflow, implemented in our open-source tool, VerifyIO, which collects execution traces, detects data conflicts, and verifies proper synchronization against specified consistency models. Our extensive evaluation of 91 test case executions across three widely used I/O libraries with four I/O consistency models reveals critical consistency issues at both application and implementation levels.
Using parallel file systems efficiently is a tricky problem due to inter-dependencies among multiple layers of I/O software, including high-level I/O libraries (HDF5, netCDF, etc.), MPI-IO, POSIX, and file systems (GPFS, Lustre, etc.). Profiling tools such as Darshan collect traces to help understand the I/O performance behavior. However, there are significant gaps in analyzing the collected traces and then applying tuning options offered by various layers of I/O software. Seeking to connect the dots between I/O bottleneck detection and tuning, we propose DXT Explorer, an interactive log analysis tool. In this paper, we present a case study using our interactive log analysis tool to identify and apply various I/O optimizations. We report an evaluation of performance improvement achieved for four I/O kernels extracted from science applications.
The semantics of HPC storage systems are defined by the consistency models to which they abide. Storage consistency models have been less studied than their counterparts in memory systems, with the exception of the POSIX standard and its strict consistency model. The use of POSIX consistency imposes a performance penalty that becomes more significant as the scale of parallel file systems increases and the access time to storage devices, such as node-local solid storage devices, decreases. While some efforts have been made to adopt relaxed storage consistency models, these models are often defined informally and ambiguously as by-products of a particular implementation. Here in this work, we establish a connection between memory consistency models and storage consistency models and revisit the key design choices of storage consistency models from a high-level perspective. Further, we propose a formal and unified framework for defining storage consistency models and a layered implementation that can be used to easily evaluate their relative performance for different I/O workloads. Finally, we conduct a comprehensive performance comparison of two relaxed consistency models on a range of commonly seen parallel I/O workloads, such as checkpoint/restart of scientific applications and random reads of deep learning applications. We demonstrate that for certain I/O scenarios, a weaker consistency model can significantly improve the I/O performance. For instance, in small random reads that are typically found in deep learning applications, session consistency achieved a 5x improvement in I/O bandwidth compared to commit consistency, even at small scales.
SAND2024-00944O Qthread Support for MPICH is software that provides additions needed to enable the use of the Qthreads library. The high-performance message passing interface (MPICH) is an open-source implementation of MPI mainly developed and distributed by Argonne National Laboratory. Qthreads is a lightweight, user-level threading library developed and distributed by Sandia National Laboratories. MPICH currently supports Posix threads, Windows threads, and Argobots. This software enables parallel programs built with the MPICH implementation of MPI to use Qthreads user-level threads rather than Posix system-level threads. The software uses existing infrastructure in MPICH to interface to the Qthreads library. The existing interfaces in MPICH allow for creating, destroying, and managing the execution of multiple threads within a process and this software translates these calls to the equivalent Qthreads library functions. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
We benchmark DAOS on a 127-node, 4,064-target pool using fio, IOR, IO500, and mdtest, comparing the DFS API against DFuse+POSIX. Single-client fio sweeps show larger block sizes and writes yield higher bandwidth, with similar random/sequential performance. Multi-node IOR (1–32 nodes, 6–48 tasks/node) shows bandwidth saturating around 32 tasks/node, with large transfer sizes hurting read throughput at high concurrency and write latency growing faster than read latency. An 8-node IO500 run shows DFS achieving ~5x higher bandwidth and ~190x higher IOPS than POSIX. We also detail operational issues: SCM target-filling under mdtest, intermittent NA_HOSTUNREACH errors, fio CPU-affinity failures, and a pool error storm resolved by recreation.
Software technologies relevant to workstation executives are discussed. Evaluations of problems, potential or otherwise, seen with IBM's Workstation Executive (WEX) 2.5 preliminary design and applicable portions of the 2.5 critical design are presented. Diverse graphics requirements of the Johnson Space Center's Mission Control Center Upgrade (MCCU) are also discussed. The key is to use tools that are portable, compatible with the X window system, and best suited to the requirements of the associated application. This will include a User Interface Language (UIL), an interactive display builder, and a graphic plotting/modeling system. Work sheets are provided for POSIX 1003.4 real-time extensions and the requirements for the Center's automated information systems security plan, referred to as POSIX 1003.6, are discussed.
NASA Goddard Space Flight Center has built an open architecture, 24 channel spaceflight Global Positioning System (GPS) receiver. The compact PCI PiVoT GPS receiver card is based on the Mitel/GEC Plessey Builder 2 board. PiVoT uses two Plessey 2021 correlators to allow tracking of up to 24 separate GPS SV's on unique channels. Its four front ends can support four independent antennas, making it a useful card for hosting GPS attitude determination algorithms. It has been built using space quality, radiation tolerant parts. The PiVoT card works at a lower signal to noise ratio than the original Builder 2 board. It also hosts an improved clock oscillator. The PiVoT software is based on the original Piessey Builder 2 software ported to the Linux operating system. The software is posix compliant and can be easily converted to other posix operating systems. The software is open source to anyone with a licensing agreement with Plessey. Additional tasks can be added to the software to support GPS science experiments or attitude determination algorithms. The next generation PiVoT receiver will be a single radiation hardened compact PCI card containing the microprocessor and the GPS receiver optimized for use above the GPS constellation.
NASA Goddard Space Flight Center has built an open architecture, 24 channel space flight GPS receiver. The CompactPCI PiVoT GPS receiver card is based on the Mitel/GEC Plessey Builder-2 board. PiVoT uses two Plessey 2021 correlators to allow tracking of up to 24 separate GPS SV's on unique channels. Its four front ends can support four independent antennas, making it a useful card for hosting GPS attitude determination algorithms. It has been built using space quality, radiation tolerant parts. The PiVoT card will track a weaker signal than the original Builder 2 board. It also hosts an improved clock oscillator. The PiVoT software is based on the original Plessey Builder 2 software ported to the Linux operating system. The software is POSIX complaint and can easily be converted to other POSIX operating systems. The software is open source to anyone with a licensing agreement with Plessey. Additional tasks can be added to the software to support GPS science experiments or attitude determination algorithms. The next generation PiVoT receiver will be a single radiation hardened CompactPCI card containing the microprocessor and the GPS receiver optimized for use above the GPS constellation. PiVoT was flown successfully on a balloon in July, 2001, for its first non-simulated flight.
A powerful high-performance computer program for simulating and analyzing adaptive and controlled optical systems has been developed by modifying the serial version of the Modeling and Analysis for Controlled Optical Systems (MACOS) program to impart capabilities for multithreaded parallel processing on computing systems ranging from supercomputers down to Symmetric Multiprocessing (SMP) personal computers. The modifications included the incorporation of OpenMP, a portable and widely supported application interface software, that can be used to explicitly add multithreaded parallelism to an application program under a shared-memory programming model. OpenMP was applied to parallelize ray-tracing calculations, one of the major computing components in MACOS. Multithreading is also used in the diffraction propagation of light in MACOS based on pthreads [POSIX Thread, (where "POSIX" signifies a portable operating system for UNIX)]. In tests of the parallelized version of MACOS, the speedup in ray-tracing calculations was found to be linear, or proportional to the number of processors, while the speedup in diffraction calculations ranged from 50 to 60 percent, depending on the type and number of processors. The parallelized version of MACOS is portable, and, to the user, its interface is basically the same as that of the original serial version of MACOS.
With the appearance of multi-/many core machines, applications and runtime systems have evolved in order to exploit the new on-node concurrency brought by new software paradigms. POSIX threads (Pthreads) was widely-adopted for that purpose and it remains as the most used threading solution in current hardware. Lightweight thread (LWT) libraries emerged as an alternative offering lighter mechanisms to tackle the massive concurrency of current hardware. In this article, we analyze in detail the most representative threading libraries including Pthread- and LWT-based solutions. In addition, to examine the suitability of LWTs for different use cases, we develop a set of microbenchmarks consisting of OpenMP patterns commonly found in current parallel codes, and we compare the results using threading libraries and OpenMP implementations. Moreover, we study the semantics offered by threading libraries in order to expose the similarities among different LWT application programming interfaces and their advantages over Pthreads. This article exposes that LWT libraries outperform solutions based on operating system threads when tasks and nested parallelism are required.
Popular parallel I/O libraries, such as HDF5, provide tuning parameters to obtain superior performance. However, the selection of effective parameters on production systems is complex due to the interdependence of I/O software and file system layers. Hence, application developers typically use the default parameters and often experience poor I/O performance. This work conducts a benchmarking-based analysis on the HDF5 behaviors with a wide variety of I/O patterns to extract performance characteristics under the production workload. To make the analysis well controlled, we exercise I/O benchmarks on POSIX-IO, MPI-IO, and HDF5 using the same I/O patterns and in the same jobs. To address high performance variability in production environments, we repeat the benchmarks across I/O patterns, storage devices, and time intervals. Based on the results, we identified consistent HDF5 behaviors that appropriate configurations and operations on dataset layout and file-metadata placement can improve performance significantly. We apply our findings and evaluate the tuned I/O library on two supercomputers: Summit and Cori. The results show that our tuned parameters can achieve more than 10× I/O performance speedup than that with default parameters on both systems, suggesting the effectiveness, stability, and generality of our solution.