File Exchange Interface
None given. Paper is a pictorial representation of a proposed file exchange interface model.
file transfer distributed information information ↗
SEARCH · Engineering Papers
Results for “file transfer”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
USACE FUSRAP Records Management Challenges in the Digital Age - 20242
In late 2016, the U.S. Army Corps of Engineers (USACE) implemented a Headquarters-supported nationwide records management initiative managed by the USACE-New England District to address the digitization and organization of Formerly Utilized Sites Remedial Action Program (FUSRAP) project files. This initiative will prepare USACE FUSRAP for moving toward a fully electronic system of records management. Additionally, the records management initiative streamlines the process and prepares the Administrative Record (AR) and Permanent Record (PR) file for transfer from USACE to the U.S. Department of Energy (DOE)-Office of Legacy Management (LM) following project closeout. Over the past couple of years, USACE and DOE-LM have been working to finalize a joint guidance document to assist in the FUSRAP records transition process. However, there remain key records management topics that require further discussion and development. Specifically, these topics include the implementation, process, and procedure for the capture of social media records and compliance with Section 508 of the Rehabilitation Act of 1973 (29 USC 794d), as amended. Due to advances in technology and the ease of communication it provides, USACE is using social media more often to interface with the public than 5, 10 or 15 years ago. The identification and capture of these records are challenging because it requires each USACE district or producer of a record to track what has been released through social media so that it can be captured as part of the AR or PR. USACE is currently evaluating processes and procedures for social media records capture. This paper explores how social media is being used by USACE across FUSRAP projects and best practices for the capture of social media records. Public accessibility to the project AR is required under the Comprehensive Environmental Response, Compensation, and Liability Act, which USACE FUSRAP must follow. As USACE FUSRAP ARs are converted to a digital format, greater accessibility to the files can be provided through web sites or other electronic media. However, Section 508 of the Rehabilitation Act must be considered when releasing electronic files to the public. Under Section 508, government agencies must ensure that information is made available to the public in a format that provides equal access to persons with disabilities. This paper discusses the electronic records requirements under Section 508 and addresses broad implementation of compliance across large digital records sets. Records management challenges are presented in the context of the FUSRAP records management program; however, these records management considerations and best practices are applicable across all Federal agency records management programs. Examples presented will further aid in understanding electronic records requirements that may be included on new USACE FUSRAP contracts as well as upcoming National Archives Records Administration requirements that will move us toward an electronic government. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
WLCG transition from X.509 to Tokens: Progress and Outlook
Since 2017, the Worldwide LHC Computing Grid (WLCG) has been working towards enabling token-based authentication and authorization throughout its entire middleware stack.Taking guidance from the WLCG Token Transition Timeline, published in 2022, substantial progress has been achieved not only in making middleware compatible with the use of tokens, but also in understanding the limitations of the WLCG Common JWT Profiles, first published in 2019. Significant scalability experience has been gained from Data Challenge 2024, during which millions of files were transferred with tokens used as credentials - a significant percentage of the total transfers completed.Besides describing the state of affairs in the transition to tokens, revisions to the WLCG token profile, and the evolving road maps, this contribution also covers the corresponding transition from VOMS-Admin to INDIGO-IAM services, with continuing improvements in terms of functionality as well as deployment.
Dack, Thomas [Rutherford Appleton Laboratory]↗
Space benefits: The secondary application of aerospace technology in other sectors of the economy
A benefits briefing notebook is presented for the NASA Technology Utilization Office in which 515 applications of NASA aerospace technology to other sections of the economy are described. An overview of technology transfer is given. Benefit cases are cited in 19 categories along with pertinent information, such as communication link, DRI transfer example file, and individual case number. General, organization, geographic, and field center indexes are provided.
Source record↗
Space benefits: The secondary application of aerospace technology in other sectors of the economy
Over 580 examples of the beneficial use of NASA aerospace technology by public and private organizations are described to demonstrate the effects of mission-oriented programs on technological progress in the United States. General observations regarding technology transfer activity are presented. Benefit cases are listed in 20 categories along with pertinent information such as communication link with NASA; the DRI transfer example file number and individual case numbers associated with the technology and examples used; and the date of the latest contract with user organizations. Subject, organization, geographic, and field center indexes are included.
Source record↗
Space Benefits: The secondary application of aerospace technology in other sectors of the economy
Some 585 examples of the beneficial use of NASA aerospace technology by public and private organizations are described to demonstrate the effects of mission-oriented programs on technological progress in the United States. General observations regarding technology transfer activity are presented. Benefit cases are listed in 20 categories along with pertinent information such as communication link with NASA; the DRI transfer example file number; and individual case numbers associated with the technology and examples used; and the date of the latest contract with user organizations. Subject, organization, geographic, and field center indexes are included.
Source record↗
Geological terrain models
The initial phase of a program to determine the best interpretation strategy and sensor configuration for a radar remote sensing system for geologic applications is discussed. In this phase, terrain modeling and radar image simulation were used to perform parametric sensitivity studies. A relatively simple computer-generated terrain model is presented, and the data base, backscatter file, and transfer function for digital image simulation are described. Sets of images are presented that simulate the results obtained with an X-band radar from an altitude of 800 km and at three different terrain-illumination angles. The simulations include power maps, slant-range images, ground-range images, and ground-range images with statistical noise incorporated. It is concluded that digital image simulation and computer modeling provide cost-effective methods for evaluating terrain variations and sensor parameter changes, for predicting results, and for defining optimum sensor parameters.
Kaupp, V. H.↗
LaRC local area networks to support distributed computing
The Langley Research Center's (LaRC) Local Area Network (LAN) effort is discussed. LaRC initiated the development of a LAN to support a growing distributed computing environment at the Center. The purpose of the network is to provide an improved capability (over inteactive and RJE terminal access) for sharing multivendor computer resources. Specifically, the network will provide a data highway for the transfer of files between mainframe computers, minicomputers, work stations, and personal computers. An important influence on the overall network design was the vital need of LaRC researchers to efficiently utilize the large CDC mainframe computers in the central scientific computing facility. Although there was a steady migration from a centralized to a distributed computing environment at LaRC in recent years, the work load on the central resources increased. Major emphasis in the network design was on communication with the central resources within the distributed environment. The network to be implemented will allow researchers to utilize the central resources, distributed minicomputers, work stations, and personal computers to obtain the proper level of computing power to efficiently perform their jobs.
Riddle, E. P.↗
Mars relay operations: the technical challenges of operating a network at Mars
Explore the source record for details and available documents.
deep space communications proximity link file tran↗
Enhancements to the SSME transfer function modeling code
This report details the results of a one year effort by Ohio University to apply the transfer function modeling and analysis tools developed under NASA Grant NAG8-167 (Irwin, 1992), (Bartholomew, 1992) to attempt the generation of Space Shuttle Main Engine High Pressure Turbopump transfer functions from time domain data. In addition, new enhancements to the transfer function modeling codes which enhance the code functionality are presented, along with some ideas for improved modeling methods and future work. Section 2 contains a review of the analytical background used to generate transfer functions with the SSME transfer function modeling software. Section 2.1 presents the 'ratio method' developed for obtaining models of systems that are subject to single unmeasured excitation sources and have two or more measured output signals. Since most of the models developed during the investigation use the Eigensystem Realization Algorithm (ERA) for model generation, Section 2.2 presents an introduction of ERA, and Section 2.3 describes how it can be used to model spectral quantities. Section 2.4 details the Residue Identification Algorithm (RID) including the use of Constrained Least Squares (CLS) and Total Least Squares (TLS). Most of this information can be found in the report (and is repeated for convenience). Section 3 chronicles the effort of applying the SSME transfer function modeling codes to the a51p394.dat and a51p1294.dat time data files to generate transfer functions from the unmeasured input to the 129.4 degree sensor output. Included are transfer function modeling attempts using five methods. The first method is a direct application of the SSME codes to the data files and the second method uses the underlying trends in the spectral density estimates to form transfer function models with less clustering of poles and zeros than the models obtained by the direct method. In the third approach, the time data is low pass filtered prior to the modeling process in an effort to filter out high frequency characteristics. The fourth method removes the presumed system excitation and its harmonics in order to investigate the effects of the excitation on the modeling process. The fifth method is an attempt to apply constrained RID to obtain better transfer functions through more accurate modeling over certain frequency ranges. Section 4 presents some new C main files which were created to round out the functionality of the existing SSME transfer function modeling code. It is now possible to go from time data to transfer function models using only the C codes; it is not necessary to rely on external software. The new C main files and instructions for their use are included. Section 5 presents current and future enhancements to the XPLOT graphics program which was delivered with the initial software. Several new features which have been added to the program are detailed in the first part of this section. The remainder of Section 5 then lists some possible features which may be added in the future. Section 6 contains the conclusion section of this report. Section 6.1 is an overview of the work including a summary and observations relating to finding transfer functions with the SSME code. Section 6.2 contains information relating to future work on the project.
Irwin, R. Dennis↗
Accelerating shared file checkpoint with local burst buffers
A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.
Gooding, Thomas↗
Social Networking Adapted for Distributed Scientific Collaboration
Share is a social networking site with novel, specially designed feature sets to enable simultaneous remote collaboration and sharing of large data sets among scientists. The site will include not only the standard features found on popular consumer-oriented social networking sites such as Facebook and Myspace, but also a number of powerful tools to extend its functionality to a science collaboration site. A Virtual Observatory is a promising technology for making data accessible from various missions and instruments through a Web browser. Sci-Share augments services provided by Virtual Observatories by enabling distributed collaboration and sharing of downloaded and/or processed data among scientists. This will, in turn, increase science returns from NASA missions. Sci-Share also enables better utilization of NASA s high-performance computing resources by providing an easy and central mechanism to access and share large files on users space or those saved on mass storage. The most common means of remote scientific collaboration today remains the trio of e-mail for electronic communication, FTP for file sharing, and personalized Web sites for dissemination of papers and research results. Each of these tools has well-known limitations. Sci-Share transforms the social networking paradigm into a scientific collaboration environment by offering powerful tools for cooperative discourse and digital content sharing. Sci-Share differentiates itself by serving as an online repository for users digital content with the following unique features: a) Sharing of any file type, any size, from anywhere; b) Creation of projects and groups for controlled sharing; c) Module for sharing files on HPC (High Performance Computing) sites; d) Universal accessibility of staged files as embedded links on other sites (e.g. Facebook) and tools (e.g. e-mail); e) Drag-and-drop transfer of large files, replacing awkward e-mail attachments (and file size limitations); f) Enterprise-level data and messaging encryption; and g) Easy-to-use intuitive workflow.
Karimabadi, Homa↗
Automation of the CFD Process on Distributed Computing Systems
A script system was developed to automate and streamline portions of the CFD process. The system was designed to facilitate the use of CFD flow solvers on supercomputer and workstation platforms within a parametric design event. Integrating solver pre- and postprocessing phases, the fully automated ADTT script system marshalled the required input data, submitted the jobs to available computational resources, and processed the resulting output data. A number of codes were incorporated into the script system, which itself was part of a larger integrated design environment software package. The IDE and scripts were used in a design event involving a wind tunnel test. This experience highlighted the need for efficient data and resource management in all parts of the CFD process. To facilitate the use of CFD methods to perform parametric design studies, the script system was developed using UNIX shell and Perl languages. The goal of the work was to minimize the user interaction required to generate the data necessary to fill a parametric design space. The scripts wrote out the required input files for the user-specified flow solver, transferred all necessary input files to the computational resource, submitted and tracked the jobs using the resource queuing structure, and retrieved and post-processed the resulting dataset. For computational resources that did not run queueing software, the script system established its own simple first-in-first-out queueing structure to manage the workload. A variety of flow solvers were incorporated in the script system, including INS2D, PMARC, TIGER and GASP. Adapting the script system to a new flow solver was made easier through the use of object-oriented programming methods. The script system was incorporated into an ADTT integrated design environment and evaluated as part of a wind tunnel experiment. The system successfully generated the data required to fill the desired parametric design space. This stressed the computational resources required to compute and store the information. The scripts were continually modified to improve the utilization of the computational resources and reduce the likelihood of data loss due to failures. An ad-hoc file server was created to manage the large amount of data being generated as part of the design event. Files were stored and retrieved as needed to create new jobs and analyze the results. Additional information is contained in the original.
Tejnil, Ed↗
Globus service enhancements for exascale applications and facilities
Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications, particularly when leadership computing facilities are involved, can touch upon edge cases (e.g., terabyte files) that had not been a focus of previous Globus optimization work, which had emphasized rather the movement of many smaller (megabyte to gigabyte) files. We report here on how automated client-driven chunking can be used to accelerate both the movement of large files and the integrity checking operations that have proven to be essential for large data transfers. In conclusion, we present detailed performance studies that provide insights into the benefits of these modifications in a range of file transfer scenarios.
97 MATHEMATICS AND COMPUTING↗
Automated, reliable, and efficient continental-scale replication of 7.3 petabytes of computational simulation data: A case study
We report on our experiences replicating 7.3 petabytes (PB) of Earth System Grid Federation (ESGF) computational simulation data from Lawrence Livermore National Laboratory (LLNL) in California to Argonne National Laboratory (ANL) in Illinois and Oak Ridge National Laboratory (ORNL) in Tennessee—a task motivated by a need for increased reliability, capacity, and performance. This task presented significant challenges: the need to move 29 million files twice under time pressure from aging storage hardware; a source file system bottleneck limiting throughput to 1.5 GB/s; frequent site maintenance windows; and the need for complete reliability at scale. We addressed these challenges using a simple replication tool that invoked Globus to transfer large bundles of files while tracking progress in a database, dynamically rerouting transfers to work around maintenance periods and file system limitations. Under the covers, Globus organized transfers to make efficient use of the high-speed Energy Sciences network (ESnet) and the data transfer nodes deployed at participating sites, and also addressed security, integrity checking, and recovery from a variety of transient failures. This success demonstrates the considerable benefits that can accrue from the adoption of performant data replication infrastructure. The replication tool is available at https://github.com/esgf2-us/data-replication-tools.
Globus↗
Infinite possibilities: Computational structures technology
Computational Fluid Dynamics (or CFD) methods are very familiar to the research community. Even the general public has had some exposure to CFD images, primarily through the news media. However, very little attention has been paid to CST--Computational Structures Technology. Yet, no important design can be completed without it. During the first half of this century, researchers only dreamed of designing and building structures on a computer. Today their dreams have become practical realities as computational methods are used in all phases of design, fabrication and testing of engineering systems. Increasingly complex structures can now be built in even shorter periods of time. Over the past four decades, computer technology has been developing, and early finite element methods have grown from small in-house programs to numerous commercial software programs. When coupled with advanced computing systems, they help engineers make dramatic leaps in designing and testing concepts. The goals of CST include: predicting how a structure will behave under actual operating conditions; designing and complementing other experiments conducted on a structure; investigating microstructural damage or chaotic, unpredictable behavior; helping material developers in improving material systems; and being a useful tool in design systems optimization and sensitivity techniques. Applying CST to a structure problem requires five steps: (1) observe the specific problem; (2) develop a computational model for numerical simulation; (3) develop and assemble software and hardware for running the codes; (4) post-process and interpret the results; and (5) use the model to analyze and design the actual structure. Researchers in both industry and academia continue to make significant contributions to advance this technology with improvements in software, collaborative computing environments and supercomputing systems. As these environments and systems evolve, computational structures technology will evolve. By using CST in the design and operation of future structures systems, engineers will have a better understanding of how a system responds and lasts, more cost-effective methods of designing and testing models, and improved productivity. For informational and educational purposes, a videotape is being produced using both static and dynamic images from research institutions, software and hardware companies, private individuals, and historical photographs and drawings. The extensive number of CST resources indicates its widespread use. Applications run the gamut from simpler university-simulated problems to those requiring solutions on supercomputers. In some cases, an image or an animation will be mapped onto the actual structure to show the relevance of the computer model to the structure. Transferring the digital files to videotape presents a number of problems related to maintaining the quality of the original image, while still producing a broadcast quality videotape. Since researchers normally do not create a computer image using traditional composition theories or video production requirements, often the image loses some of its original digital quality and impact when transferred to videotape. Although many CST images are currently available, those that are edited into the final project must meet two important criteria: they must complement the narration, and they must be broadcast quality when recorded on videotape.
Beam, Sherilee F.↗
Medical Data Architecture (MDA) Project Status
The Medical Data Architecture (MDA) project supports the Exploration Medical Capability (ExMC) risk to minimize or reduce the risk of adverse health outcomes and decrements in performance due to in-flight medical capabilities on human exploration missions. To mitigate this risk, the ExMC MDA project addresses the technical limitations identified in ExMC Gap Med 07: We do not have the capability to comprehensively process medically-relevant information to support medical operations during exploration missions. This gap identifies that the current in-flight medical data management includes a combination of data collection and distribution methods that are minimally integrated with on-board medical devices and systems. Furthermore, there are a variety of data sources and methods of data collection. For an exploration mission, the seamless management of such data will enable a more medically autonomous crew than the current paradigm. The medical system requirements are being developed in parallel with the exploration mission architecture and vehicle design. ExMC has recognized that in order to make informed decisions about a medical data architecture framework, current methods for medical data management must not only be understood, but an architecture must also be identified that provides the crew with actionable insight to medical conditions. This medical data architecture will provide the necessary functionality to address the challenges of executing a self-contained medical system that approaches crew health care delivery without assistance from ground support. Hence, the products supported by current prototype development will directly inform exploration medical system requirements.In fiscal year 2018, the MDA project developed Test Bed 2, the second iteration in a series of prototypes with functionality focused on data security through role-based access control and encryption, integration with One Portal exercise software and ingestion of an ultrasound Digital Imaging and Communications in Medicine (DICOM) file and image display. Test Bed 2 advances the medical data system architecture framework by providing these functionalities in a scalable system that maintained a layered, modular design. The architecture framework uses a data services approach with role-based access to data in a customized medical record system suitable for space exploration. These functionalities were demonstrated as part of the Next Space Technologies for Exploration Partnerships (NextSTEP) ground test demonstrated at the NASA Johnson Space Center Integrated Power, Avionics and Software (iPAS) facility. Interfacing to a Core Flight Software (CFS) system, the MDA system, using Consultative Committee for Space Data Systems (CCSDS) protocol, transferred an exercise file from the simulated flight MDA system to a mirrored MDA system on the ground through the CFS system. The selection of data sources and demonstrations enabled the team to address stakeholder concerns throughout the development process. In the next iteration, the MDA team will work with stakeholders to identify additional relevant functionalities to further advance system data models, standards and principles that will inform the medical system requirements development.
medical data architecture↗
Streaming Data from Experimental Facilities to Supercomputers for Real-Time Data Processing
In this paper we demonstrate direct data streaming from instruments and detectors at a large-scale experimental facility to a supercomputer for real-time data processing and feedback. Streaming data to supercomputers introduces the potential for novel scientific applications and workflow models, including the ability to provide real-time feedback from very large datasets during an experiment and the integration of real-time ML training and inference at scale. We discuss a successful demonstration for real-time processing of data from the Advanced Photon Source (APS) on the Polaris supercomputer using an EPICS-based streaming framework. We describe the capabilities of the streaming framework itself, and outline the architecture that allows us to process experimentally derived data on a supercomputer without file-based data transfers. We present throughput measurements that are indicative of system performance capable of sustaining the expected data production rates of the facility, as well as discuss some outstanding challenges and our future directions.
real-time processing↗