Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “File transfer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS2024

Smallsat 2024 - Starling Cubesat Swarm Technology Demonstration Flight Results

The Starling swarm of four 6U CubeSats launched in July 2023 to test four key technologies to enable future swarm missions: 1) Mobile Ad-Hoc Networking (MANET) over a crosslink radio network 2) Autonomous onboard decision-making for operations 3) Optical-based absolute and relative navigation 4) Autonomous maneuver planning and execution The Starling team implemented the Better Approach to Mobile Ad-hoc Networking (B.A.T.M.A.N.) protocol to automatically manage the crosslink network of four satellites. The B.A.T.M.A.N. protocol uses a decentralized approach to managing a multi-hop mesh network of devices, in this case, a satellite swarm. The four satellites were able to successfully establish a network at multiple data rates and demonstrate file transfer and command issuance between spacecraft over the network. Starling incorporated Distributed Spacecraft Autonomy's (DSA) software to demonstrate onboard decision-making. The DSA software takes L1/L2 band GPS measurements and uses them to estimate the relative Total Electron Count (TEC) in the ionosphere. The onboard software then determines if there are any features of interest and provides that information to the other satellites over the crosslink network. The swarm of satellites then reaches a consensus on the optimal TEC observation strategy and adjusts its measurement collection tactics autonomously. The Starling Formation-Flying Optical Experiment (StarFOX), produced by Stanford's Space Rendezvous Laboratory, uses the onboard star trackers to collect images of the other swarm spacecraft and produce angles-only navigation estimates. This system is envisioned to be valuable in applications in which Global Navigation Satellite Systems (GNSS) are not available, such as in cis-lunar or deep space. StarFOX successfully applied its algorithms to multiple simultaneous spacecraft targets using the star tracker imagery. Finally, Starling used Emergent Space's Cluster Flight Application (CFA) software suite for the Reconfiguration and Orbit Maintenance Experiments Onboard (ROMEO) demonstration of autonomously planning and executing propulsive maneuvers. Large swarms will need to be able to maintain formation requirements with minimal operator involvement, especially as the size of the swarm scales up. Results from the ROMEO experiment are presented. Starling is funded by the Small Spacecraft Technology (SST) program out of NASA's Space Technology Mission Directorate (STMD).

distributed systems↗
NASA NTRSDate not supplied

Communication Delays in Cislunar Space: A Lab Study Examining Human System Integration Architecture (HSIA) and Team Risk Concerns

BACKGROUND: Communication delays are an inherent challenge of space missions to the Moon and beyond. Past research studies showed that 50+ second delays adversely affected individual well-being, team cohesion, and overall task performance (Kintz et al., 2016; Larson et al., 2019). However, there is a dearth of research on the effects of shorter delays (on the order of 4-12 seconds one-way) that may present a more immediate challenge during the upcoming Artemis missions. Even these shorter delays could make it difficult or infeasible for ground control to provide real-time support to Artemis astronauts, especially in complex and time-critical tasks such as extra-vehicular activities (EVAs). Results from studies on longer or Mars-like delays cannot be directly applied to Artemis-like delays, due to the large differences in delay magnitude and task types between these mission categories. For example, real-time oversight and guidance from ground control is impossible under Mars-like delays but may be performed under Artemis-like delays, albeit with potentially high workload and communication difficulty. Thus, a better understanding of the effects of Artemis-like communication delays on collaborative task performance is needed. Additionally, there is a need to develop reliable task paradigms that can be used in future studies on communication delays. METHODS: This project will study the effects of Artemis-like communication delays on collaborative task performance in a simulated space-to-ground team task via a lunar Gateway-like interface prototype. Teams of two astronaut-like participants - one serving as “crewmember” and one as “flight controller” - will perform a spaceflight-relevant task under six delay conditions (i.e., 0, 4, 6, 8, 10, and 12 seconds). Audio, video, texting, and file transfer will be delayed between the participants to mimic lunar-like communication delays. Measures related to workload, situation awareness, system usability, task performance, team cohesion, well-being, and communication strategies will be collected. These measures will be compared across delay levels, participant roles, and off-nominal and nominal tasks. Participants will be given the choice to use any combination of video and text communication in order to study preferences in interaction modality. RESEARCH AIMS: One aim is to identify a delay level or range of levels at which there may be significant decrements in individual and team-based measures. Identifying such ranges may help design novel workload management or communication countermeasures for future space missions. Also, the spaceflight-relevant research tasks and corresponding communication delay technology developed as part of this effort may be used for future studies on communication delays. Results from this work are also expected to inform study scenarios in the Human Exploration Research Analog (HERA) Campaign.

S Upasani↗
DOE OSTI2021

FTS3: Data Movement Service in containers deployed in OKD

The File Transfer Service (FTS3) is a data movement service developed at CERN which is used to distribute the majority of the Large Hadron Collider's data across the Worldwide LHC Computing Grid (WLCG) infrastructure. At Fermilab, we have deployed FTS3 instances for Intensity Frontier experiments (e.g. DUNE) to transfer data in America and Europe, using a container-based strategy. In this article we summarize our experience building docker images based on work from the SLATE project (slateci.io) and deployed in OKD, the community distribution of Red Hat OpenShift. Additionally, we discuss our method of certificate management and maintenance utilizing Kubernetes CronJobs. Finally, we also report on the two different configurations currently running at Fermilab, comparing and contrasting a Docker-based OKD deployment against a traditional RPM-based deployment.

Lobato Pardavila, Lorena↗
DOE OSTI2024

Verifying Cyber Implementation Best Practices With Malcolm

Network traffic analysis can reveal a lot about what's right or wrong with a network's cybersecurity footing. Using Malcolm, a powerful open-source network traffic analysis tool suite for network security monitoring, cyber analysts and asset owners can validate cybersecurity best practices and uncover red flags in network configuration, including: proper network segmentation east-west (cross-segment) and north-south traffic unsecure or outdated network protocols authentication using clear text credentials rogue devices and services unexpected protocols (e.g., IPv6, DNS, DHCP, update checks, etc.) suspicious file transfers

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

CMS Token Transition

Within the LHC community, a momentous transition has been occurring in authorization. For nearly 20 years, services within the Worldwide LHC Computing Grid (WLCG) have authorized based on mapping an identity, derived from an X.509 credential, or a group/role, derived from a VOMS extension issued by the experiment. A fundamental shift is occurring to capabilities: the credential, a bearer token, asserts the authorizations of the bearer, not the identity. By the HL-LHC era, the CMS experiment plans for the transition to tokens, based on the WLCG Common JSON Web Token profile, to be complete. Services in the technology architecture include the INDIGO Identity and Access Management server to issue tokens; a HashiCorp Vault server to store and refresh access tokens for users and jobs; a managed token bastion server to push credentials to the HTCondor CredMon service; and HTCondor to maintain valid tokens in long-running batch jobs. We will describe the transition plans of the experiment, current status, configuration of the central authorization server, lessons learned in commissioning token-based access with sites, and operational experience using tokens for both job submissions and file transfers.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Fermilab's Transition to Token Authentication

Fermilab is the first High Energy Physics institution to transition from X.509 user certificates to authentication tokens in production systems. All the experiments that Fermilab hosts are now using JSON Web Token (JWT) access tokens in their grid jobs. Many software components have been either updated or created for this transition, and most of the software is available to others as open source. The tokens are defined using the WLCG Common JWT Profile. Token attributes for all the tokens are stored in the Fermilab FERRY system which generates the configuration for the CILogon token issuer. High security-value refresh tokens are stored in Hashicorp Vault configured by htvault-config, and JWT access tokens are requested by the htgettoken client through its integration with HTCondor. The Fermilab job submission system jobsub was redesigned to be a lightweight wrapper around HTCondor. The grid workload management system GlideinWMS which is also based on HTCondor was updated to use tokens for pilot job submission. For automated job submissions a managed tokens service was created to reduce duplication of effort and knowledge of how to securely keep tokens active. The existing Fermilab file transfer tool ifdh was updated to work seamlessly with tokens, as well as the Fermilab POMS (Production Operations Management System) which is used to manage automatic job submission and the RCDS (Rapid Code Distribution System) which is used to distribute analysis code via the CernVM FileSystem. The dCache storage system was reconfigured to accept tokens for authentication in place of X.509 proxy certificates. As some services and sites have not yet implemented token support, proxy certificates are still sent with jobs for backwards compatibility, but some experiments are beginning to transition to stop using them.

Dykstra, Dave [Fermilab] (ORCID:0000000326539015)↗
NASA NTRS1988

File-Format Program For Transferable Output ASCII Data

TOAD utilities machine-independent and require minimal central memory. Transferable Output ASCII Data (TOAD) file-format computer program facilitates transfer of data files from one computer installation to another. TOAD files preferred type and record length, easy to edit, read, and write on magnetic tape or transfer across communications networks. Applications programs write TOAD files directly and conform to all ANSI FORTRAN 77 standards.

Bingle, Bradford↗
DOE OSTI2018

Toward a smart data transfer node

Scientific computing systems are becoming significantly more complex, with distributed teams and complex workflows spanning resources from telescopes and light sources to fast networks and Internet of Things sensor systems. In such settings, no single, centralized administrative team and software stack can coordinate and manage all resources used by a single application. Indeed, we have reached a critical limit in manageability using current human-in-the-loop techniques. We therefore argue that resources must begin to respond automatically, adapting and tuning their behavior in response to observed properties of scientific workflows. Over time, machine learning methods can be used to identify effective strategies for autonomic, goal-driven management behaviors that can be applied end-to-end across the scientific computing landscape. Using the data transfer nodes that are widely deployed in modern research networks as an example, we explore the architecture, methods, and algorithms needed for a smart data transfer node to support future scientific computing systems that self-tune and self-manage.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Throughput Analytics of Data Transfer Infrastructures

To support increasingly distributed scientific and big-data applications, powerful data transfer infrastructures are being built with dedicated networks and software frameworks customized to distributed file systems and data transfer nodes. The data transfer performance of such infrastructures critically depends on the combined choices of file, disk, and host systems as well as network protocols and file transfer software, all of which may vary across sites. The randomness of throughput measurements makes it challenging to assess the impact of these choices on the performance of infrastructure or its parts. We propose regression-based throughput profiles by aggregating measurements from sites of the infrastructure, with RTT as the independent variable. The peak values and convex-concave shape of a profile together determine the overall throughput performance of memory and file transfers, and its variations show the performance differences among the sites. We then present projection and difference operators, and coefficients of throughput profiles to characterize the performance of infrastructure and its parts, including sites and file transfer tools. In particular, the utilization-concavity coefficient provides a value in the range [0, 1] that reflects overall transfer effectiveness. We present results of measurements collected using (i) testbed experiments over dedicated 0–366 ms 10 Gbps connections with combinations of TCP versions, file systems, host systems and transfer tools, and (ii) Globus GridFTP transfers over production infrastructure with varying site configurations.

Rao, Nageswara S.↗
NASA NTRS1987

Transferable Output ASCII Data (TOAD) file format description

Described is a format for writing ASCII data on a file to facilitate its transfer from one computer system to another. The TOAD format conforms to all ANSI FORTRAN 77 standards. There are two advantages in using the TOAD format. First, TOAD files are of the preferred type and record length to make them easy to edit, read from and write on magnetic tape, or transfer across communications networks. Secondly, application programs, using the TOAD format to write computational results, are more portable and the answer files easier to postprocess. TOAD utility software is listed in an appendix.

Bingel, Bradford↗
NASA NTRS1973

Program for transfer research and impact studies

Research activities conducted under the Program for Transfer Research and Impact Studies (TRIS) during 1972 included: (1) preparation of 10,196 TSP requests for TRIS application analysis; (2) interviews with over 500 individuals concerning the technical, economic, and social impacts of NASA-generated technology; (3) preparation of 38 new technology transfer example files and 101 new transfer cases; and (4) maintenance of a technology transfer library containing more than 2,900 titles. Six different modes of technology utilization are used to illustrate the pervasiveness of the transfer and diffusion of aerospace innovations. These modes also provide a basis for distinguishing the unique characteristics of the NASA Technology Utilization Program. An examination is reported of the ways in which NASA-generated technology is contributing to beneficial social change in five major areas of human concern: health, environment, safety, transportation, and communication.

Rusnak, J. J.↗
NASA NTRS1996

File Exchange Interface

None given. Paper is a pictorial representation of a proposed file exchange interface model.

file transfer distributed information information ↗
DOE OSTI2019

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗
DOE OSTI2020

USACE FUSRAP Records Management Challenges in the Digital Age - 20242

In late 2016, the U.S. Army Corps of Engineers (USACE) implemented a Headquarters-supported nationwide records management initiative managed by the USACE-New England District to address the digitization and organization of Formerly Utilized Sites Remedial Action Program (FUSRAP) project files. This initiative will prepare USACE FUSRAP for moving toward a fully electronic system of records management. Additionally, the records management initiative streamlines the process and prepares the Administrative Record (AR) and Permanent Record (PR) file for transfer from USACE to the U.S. Department of Energy (DOE)-Office of Legacy Management (LM) following project closeout. Over the past couple of years, USACE and DOE-LM have been working to finalize a joint guidance document to assist in the FUSRAP records transition process. However, there remain key records management topics that require further discussion and development. Specifically, these topics include the implementation, process, and procedure for the capture of social media records and compliance with Section 508 of the Rehabilitation Act of 1973 (29 USC 794d), as amended. Due to advances in technology and the ease of communication it provides, USACE is using social media more often to interface with the public than 5, 10 or 15 years ago. The identification and capture of these records are challenging because it requires each USACE district or producer of a record to track what has been released through social media so that it can be captured as part of the AR or PR. USACE is currently evaluating processes and procedures for social media records capture. This paper explores how social media is being used by USACE across FUSRAP projects and best practices for the capture of social media records. Public accessibility to the project AR is required under the Comprehensive Environmental Response, Compensation, and Liability Act, which USACE FUSRAP must follow. As USACE FUSRAP ARs are converted to a digital format, greater accessibility to the files can be provided through web sites or other electronic media. However, Section 508 of the Rehabilitation Act must be considered when releasing electronic files to the public. Under Section 508, government agencies must ensure that information is made available to the public in a format that provides equal access to persons with disabilities. This paper discusses the electronic records requirements under Section 508 and addresses broad implementation of compliance across large digital records sets. Records management challenges are presented in the context of the FUSRAP records management program; however, these records management considerations and best practices are applicable across all Federal agency records management programs. Examples presented will further aid in understanding electronic records requirements that may be included on new USACE FUSRAP contracts as well as upcoming National Archives Records Administration requirements that will move us toward an electronic government. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

WLCG transition from X.509 to Tokens: Progress and Outlook

Since 2017, the Worldwide LHC Computing Grid (WLCG) has been working towards enabling token-based authentication and authorization throughout its entire middleware stack.Taking guidance from the WLCG Token Transition Timeline, published in 2022, substantial progress has been achieved not only in making middleware compatible with the use of tokens, but also in understanding the limitations of the WLCG Common JWT Profiles, first published in 2019. Significant scalability experience has been gained from Data Challenge 2024, during which millions of files were transferred with tokens used as credentials - a significant percentage of the total transfers completed.Besides describing the state of affairs in the transition to tokens, revisions to the WLCG token profile, and the evolving road maps, this contribution also covers the corresponding transition from VOMS-Admin to INDIGO-IAM services, with continuing improvements in terms of functionality as well as deployment.

Dack, Thomas [Rutherford Appleton Laboratory]↗
NASA NTRS1976

Space benefits: The secondary application of aerospace technology in other sectors of the economy

A benefits briefing notebook is presented for the NASA Technology Utilization Office in which 515 applications of NASA aerospace technology to other sections of the economy are described. An overview of technology transfer is given. Benefit cases are cited in 19 categories along with pertinent information, such as communication link, DRI transfer example file, and individual case number. General, organization, geographic, and field center indexes are provided.

Source record↗
NASA NTRS1980

Space benefits: The secondary application of aerospace technology in other sectors of the economy

Over 580 examples of the beneficial use of NASA aerospace technology by public and private organizations are described to demonstrate the effects of mission-oriented programs on technological progress in the United States. General observations regarding technology transfer activity are presented. Benefit cases are listed in 20 categories along with pertinent information such as communication link with NASA; the DRI transfer example file number and individual case numbers associated with the technology and examples used; and the date of the latest contract with user organizations. Subject, organization, geographic, and field center indexes are included.

Source record↗
NASA NTRS1981

Space Benefits: The secondary application of aerospace technology in other sectors of the economy

Some 585 examples of the beneficial use of NASA aerospace technology by public and private organizations are described to demonstrate the effects of mission-oriented programs on technological progress in the United States. General observations regarding technology transfer activity are presented. Benefit cases are listed in 20 categories along with pertinent information such as communication link with NASA; the DRI transfer example file number; and individual case numbers associated with the technology and examples used; and the date of the latest contract with user organizations. Subject, organization, geographic, and field center indexes are included.

Source record↗