Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data transport”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Common Column Identification for Table Similarity Detection in Electrified Transportation Data Lakes

Electrified transportation often requires researchers and operators to interact with datasets from a wide range of sources and disciplines, such as transportation, power systems, public health, policies, and regulations. These datasets vary in quality and format, making it difficult to understand, preprocess, and identify key columns representing real-world entities or values for indexing and joining, which can negatively impact downstream analysis and operation. Existing solutions are limited, requiring extensive manual customization or data expertise to utilize. In this article, we propose a multi-layered approach to automatically identify key columns to expedite preprocessing and aid in analysis of electrified transportation data. Our method leverages a dynamic ontology to identify common fields and an information theory-based strategy for edge cases that are difficult to generalize. Evaluations on a number of datasets from data.gov and kaggle.com show improved performance of our methods over several baseline techniques, and our ablation analyses illustrate the efficacy of individual components of our method. Our case studies also demonstrate that our methods have the potential to improve analysis of electrified transportation data and aid in automatic integration of such datasets.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

A Versatile Simulated Data Transport Layer for in Situ Workflows Performance Evaluation

In situ processing does not only allow scientific applications to face the explosion in data volume and velocity but also to address the time constraints of many simulation-analysis workflows by providing scientists with early insights about their applications at runtime. Multiple frameworks implement the concept of a data transport layer (DTL) to enable such in situ workflows. These tools are very versatile, directly or indirectly access the data generated on the same node, another node of the same compute cluster, or a completely distinct node, and allow data publishers and subscribers to run on the same computing resources or not. This versatility puts on researchers the onus of taking key decisions related to resource allocation and how to transport data to ensure the most efficient execution of their in situ workflows. However, domain scientists and workflow practitioners lack the appropriate tools to assess the respective performance of particular design and deployment options. In this paper we introduce a versatile simulated DTL designed to provide researchers with insights on the respective performance of different execution scenarios of in situ workflows. This open-source, standalone library builds on the SimGrid toolkit and can be linked to any SimGrid-based simulator. It facilitates the evaluation of the performance behavior, at scale, of different data transport configurations and the study of the effects of resource allocation strategies. We demonstrate the scalability, versatility, and accuracy of this simulated DTL by reproducing the execution of two synthetic benchmarks and of a real-world in situ workflow composed of an MPI application and a parallel data analysis. Results of simulations run on a single core show that the proposed library can simulate the interactions of tens of thousands of simulated processes deployed on two interconnected commodity clusters in a few seconds, and the execution by a thousand simulated processes of an in situ workflow in less than three minutes.

Suter, Fred [ORNL] (ORCID:0000000319021955)↗
NASA NTRS1987

Mass storage systems for data transport in the early space station era 1992-1998

NASA's Space Station Program will provide a vehicle to deploy an unprecedented number of data producing experiments and operational devices. Peak down link data rates are expected to be in the 500 megabit per second range and the daily data volume could reach 2.4 terabytes. Such startling requirements inspired an internal NASA study to determine if economically viable data storage solutions are likely to be available to support the Ground Data Transport segment of the NASA data system. To derive the requirements for data storage subsystems, several alternative data transport architectures were identified with different degrees of decentralization. Data storage operations at each subsystem were categorized based on access time and retrieval functions, and reduced to the following types of subsystems: First in First out (FIFO) storage, fast random access storage, and slow access with staging. The study showed that industry funded magnetic and optical storage technology has a reasonable probability of meeting these requirements. There are, however, system level issues that need to be addressed in the near term.

Carper, Richard↗
DOE OSTI2023

Methods and systems for evaluating data transportability in distribution grids

This document describes systems and techniques for evaluating and improving distribution-grid data transportability. These systems and techniques allow engineers to quantify the data transportability of a communication system within or connected to a distribution grid, which represents an ability to transport in real-time telemetry from source locations (e.g., sensors in the distribution grid) to control mechanisms. Distribution engineers can use the sensor readings to perform grid analytics, control operating parameters, and operate protection systems. Distribution engineers can also use the transportability of the communication system to evaluate the observability of the distribution grid, which represents an ability to combine actual measurements and various types of computations (e.g., analytics, estimators, forecasters) from a system model. Distribution engineers can then generate a sensor allocation plan that indicates the number and location of sensors to maximize observability for a fixed sensor cost and/or minimize sensor cost for predetermined observability.

Taft, Jeffrey D.↗
NASA NTRS1980

Data transport concepts for future NASA spacecraft

The anticipated increase in data quantity from NASA spacecraft over the next decade is expected to far exceed the capability of the present NASA data/information system to effectively manage and process that data. Conceptual designs aimed at providing more responsive and cost-effective methods for transporting and interpreting space-generated data are presented. Details of the Modular Data Transport System (MDTS) and Ancillary Data and Support Computing (ADSC) efforts done under the NASA End-to-End Data System (NEEDS) program are discussed. The common 'thread' throughout the data transport process involves the use of packet telemetry. Characteristics and benefits of packet telemetry are presented along with a discussion of techniques for generating and advantages of including ancillary data in each packet.

Lord, D. D.↗
NASA NTRS1988

Data Transport Subsystem - The SFOC glue

The design and operation of the Data Transport Subsystem (DTS) for the JPL Space Flight Operation Center (SFOC) are described. The SFOC is the ground data system under development to serve interplanetary space probes; in addition to the DTS, it comprises a ground interface facility, a telemetry-input subsystem, data monitor and display facilities, and a digital TV system. DTS links the other subsystems via an ISO OSI presentation layer and an LAN. Here, particular attention is given to the DTS services and service modes (virtual circuit, datagram, and broadcast), the DTS software architecture, the logical-name server, the role of the integrated AI library, and SFOC as a distributed system.

Parr, Stephen J.↗
DOE OSTI2025

In-Transit Data Transport Strategies for Coupled AI-Simulation Workflow Patterns

Coupled AI-Simulation workflows are becoming the major workloads for HPC facilities, and their increasing complexity necessitates new tools for performance analysis and prototyping of new in-situ workflows. We present SimAI-Bench, a tool designed to both prototype and evaluate these coupled workflows. In this paper, we use SimAI-Bench to benchmark the data transport performance of two common patterns on the Aurora supercomputer: a one-to-one workflow with co-located simulation and AI training instances, and a many-to-one workflow where a single AI model is trained from an ensemble of simulations. For the one-to-one pattern, our analysis shows that node-local and DragonHPC data staging strategies provide excellent performance compared Redis and Lustre file system. For the many-to-one pattern, we find that data transport becomes a dominant bottleneck as the ensemble size grows. Our evaluation reveals that file system is the optimal solution among the tested strategies for the many-to-one pattern.

Tummalapalli, Harikrishna [Argonne National Labora↗
DOE OSTI2025

ESnet-JLab FPGA Accelerated Transport (data plane) [EJFAT (udplb)] v1.0

The ESnet-JLab FPGA Accelerated Transport system is a solution for streaming high-speed scientific measurement data from Data Acquisition Systems (DAQs) to high-performance computing facilties. It is generally compatible with many science workflows, and makes no assumptions about the specifics of any particular experiment. This program (udplb) implements the data plane portion of the EJFAT system. It is an FPGA design that rewrites and forwards data packets from a UDP-based scientific workflow to high-performance compute nodes. It depends on another program (udplbd, disclosed separately) to implement the control system.

Bengough, Peter [Malleable Networks, Inc.]↗
DOE OSTI2026

Throughput Estimation of Data Transport Networks From Digital Twin Measurements

Digital twins of networked infrastructures, known as Virtual Infrastructure Twins (VITs), are increasingly used for software development, pre-deployment testing, and design space exploration. While VITs avoid the costs and potential disruptions associated with experiments on operational networks, their throughput measurements are typically not sufficiently accurate for performance profiling of wide-area networks that they emulate. Here, machine learning (ML) methods are developed to transform these inaccurate VIT network throughput measurements to closely match in peak and overall profile of those from a physical testbed or production network. First, a micro kernel network reflecting a physical network is utilized to collect one-time measurements on a host to support this ML transformation. Then, a generic multi-modal ML method is developed to learn a map that transforms measurements from subsequent VITs on the same host to match past, current and follow-on testbed and cloud networks. ML generalization equations are derived to establish its correctness and probabilistically guarantee its generalization accuracy. Experimental results are presented for a variety of VIT hosts with target testbed and cloud networks; they include a case study of a four-site science ecosystem wherein inaccurate convex VIT measurement profiles are transformed into accurate concave profiles of target networks.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1986

Mature data transport and command management services for the Space Station

The duplex space/ground/space data services for the Space Station are described. The need to separate the uplink data service functions from the command functions is discussed. Command management is a process shared by an operation control center and a command management system and consists of four functions: (1) uplink data communications, (2) management of the on-board computer, (3) flight resource allocation and management, and (4) real command management. The new data service capabilities provided by microprocessors, ground and flight nodes, and closed loop and open loop capabilities are studied. The need for and functions of a flight resource allocation management service are examined. The system is designed so only users can access the system; the problems encountered with open loop uplink access are analyzed. The procedures for delivery of operational, verification, computer, and surveillance and monitoring data directly to users are reviewed.

Carper, R. D.↗
NASA NTRS2010

Development of an e-VLBI Data Transport Software Suite with VDIF

We have developed a software library (KVTP-lib) for VLBI data transmission over the network with the VDIF (VLBI Data Interchange Format), which is the newly proposed standard VLBI data format designed for electronic data transfer over the network. The software package keeps the application layer (VDIF frame) and the transmission layer separate, so that each layer can be developed efficiently. The real-time VLBI data transmission tool sudp-send is an application tool based on the KVTP-lib library. sudp-send captures the VLBI data stream from the VSI-H interface with the K5/VSI PC-board and writes the data to file in standard Linux file format or transmits it to the network using the simple- UDP (SUDP) protocol. Another tool, sudp-recv , receives the data stream from the network and writes the data to file in a specific VLBI format (K5/VSSP, VDIF, or Mark 5B). This software system has been implemented on the Wettzell Tsukuba baseline; evaluation before operational employment is under way.

Sekido, Mamoru↗
DOE OSTI2022

Technical Characterization and Benefit Evaluation of 5G-Enabled Grid Data Transport and Applications

This report summarizes the Year 1 work of Pacific Northwest National Laboratory’s (PNNL’s) 5G Fabricated Resource and Asset Management Encompassment for energy infrastructure (Energy FRAME) project funded by the Department of Energy Office of Science’s Advanced Scientific Computing Research Program. 5G is a breakthrough technology that enables a fully mobile and connected society, and a 5G-enabled digital continuum will be one of the critical foundations for a clean energy economy and grid modernization. In collaboration with PNNL’s Advanced Wireless Communication team and Center for Advanced Technology Evaluation team, the project team has been evaluating the system performance of 5G testbeds in the PNNL 5G Innovation Studio, and has formulated a co-simulation test case of power system transmission, distribution, and communication (T&D&C) networks considering 5G technology and high penetration of distributed energy resources. The methodology developed in the 5G Energy FRAME project can be customized to fit different future grid scenarios to evaluate multiple (dynamic) configurations (computing, sensing, communication, environment) for different stakeholders. In summary, our main technical highlights in project Year 1 are as follows: 1) Technical characterization of 5G standalone architectures, 2) Formulation of co-simulation test case of T&D&C networks embedded with 5G, 3) Initial benefit evaluation of 5G communication platform for grid use cases, and 4) Additional extended discussions on edge computing, artificial intelligence and machine learning, and high-performance computing and cloud computing adoptions. In addition, a collection of system performance data is shared through the publicly available weblink, https://www.pnnl.gov/projects/5g-energy-frame/publications

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRSDate not supplied

Obtaining cryogenic molecular transport data for analysis

The James Webb Space Telescope (JWST) has a primary mirror, made of 18 segments, and a secondary mirror (SM) that are used to direct the light of desired targets. While on orbit, the secondary mirror assembly (SMA) was stowed for approximately 10 days and was subject to molecular contamination outgassing from the cavity of the secondary mirror support structure (SMSS) in-board hinge (IBH) which contains cables, motors, resolvers, and coatings. The main concern during this period before SMA deployment is the accumulation of ice due to the lack of a heater on the SMA. The temperature differentials between the IBH surfaces and SMA could cause redistribution of contaminants. To address this concern, single layer insulation (SLI) was used to create adequate venting and path redirection of outgassing from the IBH. Two separate thermal vacuum tests were performed to quantify this contamination: one Z307 ASTM E 1559 materials test of the radiator paint used on the motor of the IBH and a separate test on a primary mirror backplane assembly (PMBA) qualification engineering test unit (ETU) of the IBH. These tests approximately followed the predicted SMA pre-deployment thermal environment. To quantify source rates in case of a leak, the motor and resolver sides were separated and quartz crystal microbalances (QCM) were used to measure the deposition of water. The SLI redesign and implementation and outgassing measurements to understand leak effects from the IBH were essential to mitigate the deposition of contamination on the SMA.

James Webb Space Telescope, contamination, cleanro↗
NASA NTRSDate not supplied

Obtaining Cryogenic Molecular Transport Data for Analysis

The James Webb Space Telescope (JWST) has a primary mirror, made of 18 segments, and a secondary mirror (SM) that are used to direct the light of desired targets. While on orbit, the secondary mirror assembly (SMA) was stowed for approximately 10 days and was subject to molecular contamination outgassing from the cavity of the secondary mirror support structure (SMSS) in-board hinge (IBH) which contains cables, motors, resolvers, and coatings. The main concern during this period before SMA deployment is the accumulation of ice due to the lack of a heater on the SMA. The temperature differentials between the IBH surfaces and SMA could cause redistribution of contaminants. To address this concern, single layer insulation (SLI) was used to create adequate venting and path redirection of outgassing from the IBH. Two separate thermal vacuum tests were performed to quantify this contamination: one Z307 ASTM E 1559 materials test of the radiator paint used on the motor of the IBH and a separate test on a primary mirror backplane assembly (PMBA) qualification engineering test unit (ETU) of the IBH. These tests approximately followed the predicted SMA pre-deployment thermal environment. To quantify source rates in case of a leak, the motor and resolver sides were separated and quartz crystal microbalances (QCM) were used to measure the deposition of water. The SLI redesign and implementation and outgassing measurements to understand leak effects from the IBH were essential to mitigate the deposition of contamination on the SMA. "

James Webb Space Telescope↗