Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Asynchronous data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Real Time Phasor Analytics (RTPA) and RTPA-SCR System Strength Online Tool

This presentation showcases the Real-Time Phasor Analytics (RTPA) framework for monitoring inertia and assessing system strength in power grids. RTPA is an open-source tool designed to standardize access to data from Power Management Units (PMUs) and Phasor Data Concentrators (PDCs). It facilitates real-time connectivity to multiple PDCs in accordance with the IEEE C37.118-2 standard and supports asynchronous data stream integration. Additionally, RTPA can simulate a PDC server streaming C37.118-2 data and provides Python bindings for seamless interaction with the framework, eliminating the need for direct Rust programming.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Simplified Parallel Domain Traversal

Many data-intensive scientific analysis techniques require global domain traversal, which over the years has been a bottleneck for efficient parallelization across distributed-memory architectures. Inspired by MapReduce and other simplified parallel programming approaches, we have designed DStep, a flexible system that greatly simplifies efficient parallelization of domain traversal techniques at scale. In order to deliver both simplicity to users as well as scalability on HPC platforms, we introduce a novel two-tiered communication architecture for managing and exploiting asynchronous communication loads. We also integrate our design with advanced parallel I/O techniques that operate directly on native simulation output. We demonstrate DStep by performing teleconnection analysis across ensemble runs of terascale atmospheric CO{sub 2} and climate data, and we show scalability results on up to 65,536 IBM BlueGene/P cores.

99 GENERAL AND MISCELLANEOUS↗

A multiprocessing architecture for real-time monitoring

A multitasking architecture for performing real-time monitoring and analysis using knowledge-based problem solving techniques is described. To handle asynchronous inputs and perform in real time, the system consists of three or more distributed processes which run concurrently and communicate via a message passing scheme. The Data Management Process acquires, compresses, and routes the incoming sensor data to other processes. The Inference Process consists of a high performance inference engine that performs a real-time analysis on the state and health of the physical system. The I/O Process receives sensor data from the Data Management Process and status messages and recommendations from the Inference Process, updates its graphical displays in real time, and acts as the interface to the console operator. The distributed architecture has been interfaced to an actual spacecraft (NASA's Hubble Space Telescope) and is able to process the incoming telemetry in real-time (i.e., several hundred data changes per second). The system is being used in two locations for different purposes: (1) in Sunnyville, California at the Space Telescope Test Control Center it is used in the preflight testing of the vehicle; and (2) in Greenbelt, Maryland at NASA/Goddard it is being used on an experimental basis in flight operations for health and safety monitoring.

Schmidt, James L.↗

Real-time analysis of telemetry data

This paper descibes a knowledge-based system for performing real-time monitoring and analysis of telemetry data from the NASA Hubble Space Telescope (HST). In order to handle asynchronous inputs and perform in real time the system consists of three or more separate processes, which run concurrently and communicate via a message passing scheme. The data management process gathers, compresses, and scales the incoming telemetry data befoe sending it to the other tasks. The inferencing process uses the incoming data to perform a real-time analysis of the state and health of the Space Telescope. The I/O process receives telemetry monitors from the data management process, updates its graphical displays in real time, and acts as the interface to the console operator. The three processes may run on the same or different computers. This system is currently under development and is being used to monitor testcases produced by the Bass Telemetry System in the Hardware/Software Integration Facility at Lockheed Missile and Space Co. in Sunnyvale, California.

Kao, Simon A.↗

Lunar Reconnaissance Orbiter (LRO) Command and Data Handling Flight Electronics Subsystem

A document describes a high-performance, modular, and state-of-the-art Command and Data Handling (C&DH) system developed for use on the Lunar Reconnaissance Orbiter (LRO) mission. This system implements a complete hardware C&DH subsystem in a single chassis enclosure that includes a processor card, 48 Gbytes of solid-state recorder memory, data buses including MIL-STD-1553B, custom RS-422, SpaceWire, analog collection, switched power services, and interfaces to the Ka-Band and S-Band RF communications systems. The C&DH team capitalized on extensive experience with hardware and software with PCI bus design, SpaceWire networking, Actel FPGA design, digital flight design techniques, and the use of VxWorks for the real-time operating system. The resulting hardware architecture was implemented to meet the LRO mission requirements. The C&DH comprises an enclosure, a backplane, a low-voltage power converter, a single-board computer, a communications interface board, four data storage boards, a housekeeping and digital input/output board, and an analog data acquisition board. The interfaces between the C&DH and the instruments and avionics are connected through a SpaceWire network, a MIL-STD-1553 bus, and a combination of synchronous and asynchronous serial data transfers over RS-422 and LVDS (low-voltage differential-signaling) electrical interfaces. The C&DH acts as the spacecraft data system with an instrument data manager providing all software and internal bus scheduling, ingestion of science data, distribution of commands, and performing science operations in real time.

Nguyen, Quang↗

HiCOO: Hierarchical cooperation for scalable communication in Global Address Space programming models on Cray XT systems

Global Address Space (GAS) programming models enable a convenient, shared-memory style addressing model. Typically this is realized through one-sided operations that can enable asynchronous communication and data movement. With the size of petascale systems reaching 10,000s of nodes and 100,000s of cores, the underlying runtime systems face critical challenges in (1) scalably managing resources (such as memory for communication buffers), and (2) gracefully handling unpredictable communication patterns and any associated contention. For any solution that addresses these resource scalability challenges, equally important is the need to maintain the performance of GAS programming models. In this paper, we describe a Hierarchical COOperation (HiCOO) architecture for scalable communication in GAS programming models. HiCOO formulates a cooperative communication architecture: with inter-node cooperation amongst multiple nodes (a.k.a multinode) and hierarchical cooperation among multinodes that are arranged in various virtual topologies. We have implemented HiCOO for a popular GAS runtime library, Aggregate Remote Memory Copy Interface (ARMCI). By extensively evaluating different virtual topologies in HiCOO in terms of their impact to memory scalability, network contention, and application performance, we identify MFCG as the most suitable virtual topology. The resulting HiCOO architecture is able to realize scalable resource management and achieve resilience to network contention, while at the same time maintaining or enhancing the performance of scientific applications. In one case, it reduces the total execution time of an NWChem application by 52%.

Computer Science↗

UPC++ v1.0 Programmer’s Guide (Rev. 2023.9.0)

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

UPC++ v1.0 Programmer’s Guide (Revision 2022.3.0)

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

UPC++ v1.0 Programmer’s Guide, Revision 2023.3.0

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

UPC++ v1.0 Programmer’s Guide, Revision 2022.9.0

UPC++ is a C++ library that supports Partitioned Global Address Space (PGAS) programming. It is designed for writing efficient, scalable parallel programs on distributed-memory parallel computers. The key communication facilities in UPC++ are one-sided Remote Memory Access (RMA) and Remote Procedure Call (RPC). The UPC++ control model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. The PGAS memory model additionally provides one-sided RMA communication to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ also features Remote Procedure Call (RPC) communication, making it easy to move computation to operate on data that resides on remote processes. UPC++ was designed to support exascale high-performance computing, and the library interfaces and implementation are focused on maximizing scalability. In UPC++, all communication operations are syntactically explicit, which encourages programmers to consider the costs associated with communication and data movement. Moreover, all communication operations are asynchronous by default, encouraging programmers to seek opportunities for overlapping communication latencies with other useful work. UPC++ provides expressive and composable abstractions designed for efficiently managing aggressive use of asynchrony in programs. Together, these design principles are intended to enable programmers to write applications using UPC++ that perform well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗

Efficient Full-Field Vibration Measurements and Operational Modal Analysis Using Neuromorphic Event-Based Imaging

Traditional vibration measurement typically requires physically attached sensors, such as accelerometers and strain gauges. However, these discrete-point sensors provide only low spatial resolution vibration measurements, potentially foregoing valuable structural information such as localized damage. Noncontact optical measurement methods such as laser vibrometers can achieve high spatial resolution vibration measurements but only through time-consuming sequential measurements and are not cost-effective. As an alternative to traditional vibration measurement methods, digital video cameras are relatively low-cost, agile, and offer noncontact, simultaneous high spatial resolution measurements where every pixel on the structure becomes a measurement point. However, regular digital video cameras are frame-based where each pixel simultaneously performs temporally uniform (synchronous) measurements containing large amounts of redundant (background) data, which consumes considerable resources for video data measurement, management, and processing. To alleviate such a challenge, this work explores the use of event-based neuromorphic imagers, specifically silicon retinas, an efficient alternative to traditional frame-based video cameras, to perform full-field vibration measurements and operational modal analysis. By imitating biological vision, each silicon retina pixel independently and asynchronously records only intensity change events that contain structural motion information while excluding redundant (background) information. Such an asynchronous event-based data measurement mechanism allows for structural motion to be captured on the microsecond scale in an extremely data-efficient manner, which could benefit real-time vibration measurement and control applications. This study takes the first step toward these applications by formulating an existing video frame-based full-field operational modal analysis technique in the event-based, asynchronous silicon retina measurement framework. Specifically, local phase-based motion extraction and blind source separation are used to automatically and efficiently extract full-field vibration and dynamics parameters from silicon retina measurements. The developed method is validated by laboratory experiments on a bench-scale cantilever beam.

42 ENGINEERING↗

A multiprocessing architecture for real-time monitoring

A multiprocessing architecture for performing real time monitoring and analysis using knowledge-based problem solving techniques is discussed. To handle asynchronous inputs and perform in real time, the system consists of three or more separate processes which run concurrently on one or more processors and communicate via a message passing scheme. The Data Management Process gathers, compresses, scales and sends the incoming telemetry data to other tasks. The Inference Process consists of a proprietary high performance inference engine that runs at 1000 rules per second using telemetry data to perform real time analysis on the state and health of the Space Telescope. The multiprocessing architecture has been interfaced to a simulator and is able to process the incoming telemetry in real time.

Laffey, Thomas J.↗

Towards a distributed information architecture for avionics data

Avionics data at the National Aeronautics and Space Administration's (NASA) Jet Propulsion Laboratory (JPL consists of distributed, unmanaged, and heterogeneous information that is hard for flight system design engineers to find and use on new NASA/JPL missions. The development of a systematic approach for capturing, accessing and sharing avionics data critical to the support of NASA/JPL missions and projects is required. We propose a general information architecture for managing the existing distributed avionics data sources and a method for querying and retrieving avionics data using the Object Oriented Data Technology (OODT) framework. OODT uses XML messaging infrastructure that profiles data products and their locations using the ISO-11179 data model for describing data products. Queries against a common data dictionary (which implements the ISO model) are translated to domain dependent source data models, and distributed data products are returned asynchronously through the OODT middleware. Further work will include the ability to 'plug and play' new manufacturer data sources, which are distributed at avionics component manufacturer locations throughout the United States.

Information architecture↗

Transparent Asynchronous Zero-copy Remote I/O (TAZeR)

Transparent Asynchronous Zero-copy Remote I/O (TAZeR) is a framework built to facilitate grid computing by easing the memory movement and management across geographically separated computing resources. TAZeR employs a multiclient/ multi-server paradigm where data is stored at on a remote server. Users preload an interposing library which upon application start-up, stands up an ephemeral lightweight client. The client captures POSIX I/O calls to TAZeR meta-files which contains the IP address/port of the remote server, the location of the file on remote server, and additional meta-data. The client connects to the server using this information and begins downloading the file in parts optionally compressing the data.

Friese, Ryan↗

Extending I/O through High Performance Data Services

The complexity of HPC systems has increased the burden on the developer as applications scale to hundreds of thousands of processing cores. Moreover, additional efforts are required to achieve acceptable I/O performance, where it is important how I/O is performed, which resources are used, and where I/O functionality is deployed. Specifically, by scheduling I/O data movement and by effectively placing operators affecting data volumes or information about the data, tremendous gains can be achieved both in the performance of simulation output and in the usability of output data. Previous studies have shown the value of using asynchronous I/O, of employing a staging area, and of performing select operations on data before it is written to disk. Leveraging such insights, this paper develops and experiments with higher level I/O abstractions, termed 'data services', that manage output data from 'source to sink': where/when it is captured, transported towards storage, and filtered or manipulated by service functions to improve its information content. Useful services include data reduction, data indexing, and those that manage how I/O is performed, i.e., the control aspects of data movement. Our data services implementation distinguishes control aspects - the control plane - from data movement - the data plane, so that both may be changed separably. This results in runtime flexibility not only in which services to employ, but also in where to deploy them and how they use I/O resources. The outcome is consistently high levels of I/O performance at large scale, without requiring application change.

97 MATHEMATICS AND COMPUTING↗

Project Integration Architecture: Distributed Lock Management, Deadlock Detection, and Set Iteration

The migration of the Project Integration Architecture (PIA) to the distributed object environment of the Common Object Request Broker Architecture (CORBA) brings with it the nearly unavoidable requirements of multiaccessor, asynchronous operations. In order to maintain the integrity of data structures in such an environment, it is necessary to provide a locking mechanism capable of protecting the complex operations typical of the PIA architecture. This paper reports on the implementation of a locking mechanism to treat that need. Additionally, the ancillary features necessary to make the distributed lock mechanism work are discussed.

Jones, William Henry↗

LC-MEMENTO: A Memory Model for Accelerated Architectures

With the advent of heterogeneous architectures, in particular, with the ubiquity of multi-GPU systems, it is becoming increasingly important to manage device memory efficiently in order to reap the benefits of the additional core count. To date, such responsibility mainly falls on the programmer where device-to-host data communication (and vice versa), if not done properly, may incur costly memory transfer operations and synchronization. The problem may be compounded by additional requirement to maintain system-wide memory consistency that may involve expensive synchronization overhead. In this paper, we present Location Consistency Memory Model for Enhanced Transfer Operations (LC-MEMENTO). This framework considers incorporating runtime techniques for multi-GPU memory management to support relaxed synchronization semantics and memory transfer operations automatically. Specifically, we implement a relaxed form of a memory consistency model based on the Location Consistency (LC) in an Asynchronous Many-Task Runtime (ARTS) and demonstrate that, this memory model enables additional optimization opportunities for the three representative applications encompassing different computational patterns (scientific computation, graphs, data streaming, etc.).

Memory Models, Accelerators, Adaptive Optimization↗

Real-Time Rocket/Vehicle System Integrated Health Management Laboratory For Development and Testing of Health Monitoring/Management Systems

Pratt & Whitney Rocketdyne has developed a real-time engine/vehicle system integrated health management laboratory, or testbed, for developing and testing health management system concepts. This laboratory simulates components of an integrated system such as the rocket engine, rocket engine controller, vehicle or test controller, as well as a health management computer on separate general purpose computers. These general purpose computers can be replaced with more realistic components such as actual electronic controllers and valve actuators for hardware-in-the-loop simulation. Various engine configurations and propellant combinations are available. Fault or failure insertion capability on-the-fly using direct memory insertion from a user console is used to test system detection and response. The laboratory is currently capable of simulating the flow-path of a single rocket engine but work is underway to include structural and multiengine simulation capability as well as a dedicated data acquisition system. The ultimate goal is to simulate as accurately and realistically as possible the environment in which the health management system will operate including noise, dynamic response of the engine/engine controller, sensor time delays, and asynchronous operation of the various components. The rationale for the laboratory is also discussed including limited alternatives for demonstrating the effectiveness and safety of a flight system.

Aguilar, R.↗