SEARCH · Engineering Papers

Results for “FILES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2017

On Analytics of File Transfer Rates over Dedicated Wide-Area Connections

File transfers between the decentralized storage sites over dedicated wide-area connections are becoming increasingly important in high-performance computing and big data scenarios. Designing such scientific workflows for large file transfers is extremely challenging as they depend on the file, I/O, host, and local- and wide-area network subsystems, and their interactions. To gain insights into file-transfer rate profiles, we develop polynomial, bagging, and boosting regression models for Lustre and XFS file transfer measurements, which are collected using XDD over a suite of 10 Gbps connections with 0-366 ms round trip times (RTTs). In addition to overall trends and analytics, these regressions also provide file-transfer rate estimates for RTTs and number of parallel flows at which measurements might not have been collected. They show that bagging and boosting techniques provide closer data fits than the polynomial regression. We develop probabilistic bounds on the generalization error of these methods, which combined with the cross-validation error establish that former two are more accurate estimators than the polynomial regression. In addition, we present a method to efficiently determine the number of parallel flows to achieve a peak file-transfer rate using fewer than full sweep measurements; in our measurements, the peak is achieved in 96% of cases with 15-25% of measurements of a full sweep.

Sen, Satyabrata↗

DOE OSTI2018

Wide-area lustre file system using LNet routers

Scientific and big data computations are increasingly being distributed across wide-area networks, and they often require access to remote files. The file systems that are directly mounted over wide-area networks transparently support such computations, and also obviate the need for special purpose file transfer tools. In typical distributed file systems, the access is limited to local sites, and in particular, the reach of Lustre file system implemented over InfiniBand (IB) is limited to at most tens of miles due to 2.5ms latency bound. We describe LNet router methods that connect IB Lustre file system to remote Ethernet clients over wide-area networks. We collect extensive Lustre throughput measurements over 10Gbps connections with 0-366ms round-trip times. They demonstrate that Gbps throughput can be sustained over connections spanning the globe. We present Lustre throughput profiles over local and wide-area connections, which show the effects of various buffers and credits; in particular, they highlight the throughput limits for large transfers over wide-area connections. Furthermore, the measurements show the positive effects of pipelining in achieving higher throughput for successively file transfers compared to rates indicated by IOzone benchmark rates.

Rao, Nageswara↗

DOE OSTI2016

SU-F-T-172: A Method for Log File QA On An IBA Proteus System for Patient Specific Spot Scanning Quality Assurance

Purpose: The use of log files to perform patient specific quality assurance for both protons and IMRT has been established. Here, we extend that approach to a proprietary log file format and compare our results to measurements in phantom. Our goal was to generate a system that would permit gross errors to be found within 3 fractions until direct measurements. This approach could eventually replace direct measurements. Methods: Spot scanning protons pass through multi-wire ionization chambers which provide information about the charge, location, and size of each delivered spot. We have generated a program that calculates the dose in phantom from these log files and compares the measurements with the plan. The program has 3 different spot shape models: single Gaussian, double Gaussian and the ASTROID model. The program was benchmarked across different treatment sites for 23 patients and 74 fields. Results: The dose calculated from the log files were compared to those generate by the treatment planning system (Raystation). While the dual Gaussian model often gave better agreement, overall, the ASTROID model gave the most consistent results. Using a 5%–3 mm gamma with a 90% passing criteria and excluding doses below 20% of prescription all patient samples passed. However, the degree of agreement of the log file approach was slightly worse than that of the chamber array measurement approach. Operationally, this implies that if the beam passes the log file model, it should pass direct measurement. Conclusion: We have established and benchmarked a model for log file QA in an IBA proteus plus system. The choice of optimal spot model for a given class of patients may be affected by factors such as site, field size, and range shifter and will be investigated further.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

SU-F-T-288: Impact of Trajectory Log Files for Clarkson-Based Independent Dose Verification of IMRT and VMAT

Purpose: To investigate the effect of the trajectory files from linear accelerator for Clarkson-based independent dose verification in IMRT and VMAT plans. Methods: A CT-based independent dose verification software (Simple MU Analysis: SMU, Triangle Products, Japan) with a Clarksonbased algorithm was modified to calculate dose using the trajectory log files. Eclipse with the three techniques of step and shoot (SS), sliding window (SW) and Rapid Arc (RA) was used as treatment planning system (TPS). In this study, clinically approved IMRT and VMAT plans for prostate and head and neck (HN) at two institutions were retrospectively analyzed to assess the dose deviation between DICOM-RT plan (PL) and trajectory log file (TJ). An additional analysis was performed to evaluate MLC error detection capability of SMU when the trajectory log files was modified by adding systematic errors (0.2, 0.5, 1.0 mm) and random errors (5, 10, 30 mm) to actual MLC position. Results: The dose deviations for prostate and HN in the two sites were 0.0% and 0.0% in SS, 0.1±0.0%, 0.1±0.1% in SW and 0.6±0.5%, 0.7±0.9% in RA, respectively. The MLC error detection capability shows the plans for HN IMRT were the most sensitive and 0.2 mm of systematic error affected 0.7% dose deviation on average. Effect of the MLC random error did not affect dose error. Conclusion: The use of trajectory log files including actual information of MLC location, gantry angle, etc should be more effective for an independent verification. The tolerance level for the secondary check using the trajectory file may be similar to that of the verification using DICOM-RT plan file. From the view of the resolution of MLC positional error detection, the secondary check could detect the MLC position error corresponding to the treatment sites and techniques. This research is partially supported by Japan Agency for Medical Research and Development (AMED)

60 APPLIED LIFE SCIENCES↗

DOE OSTI2013

A model for optimizing file access patterns using spatio-temporal parallelism. In: UltraVis '13 Proceedings of the 8th International Workshop on Ultrascale Visualization, Article No. 4

For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. In this paper, we introduce a model that can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. Read times ultimately depend on how the file is stored and the access pattern used to read the file. The file access pattern will be dictated by the type of parallel decomposition used. We employ spatio-temporal parallelism, which combines both spatial and temporal parallelism, to provide greater flexibility to possible file access patterns. Using our model, we were able to configure the spatio-temporal parallelism to design optimized read access patterns that resulted in a speedup factor of approximately 400 over traditional file access patterns.

Nouanesengsy, Boonthanome↗

DOE OSTI2017

Scientific User Behavior and Data-Sharing Trends in a Petascale File System

The Oak Ridge Leadership Computing Facility (OLCF) runs the No. 4 supercomputer in the world, supported by a petascale file system, to facilitate scientific discovery. In this paper, using the daily file system metadata snapshots collected over 500 days, we have studied the behavioral trends of 1, 362 active users and 380 projects across 35 science domains. In particular, we have analyzed both individual and collective behavior of users and projects, highlighting needs from individual communities and the overall requirements to operate the file system. We have analyzed the metadata across three dimensions, namely (i) the projects' file generation and usage trends, using quantitative file system-centric metrics, (ii) scientific user behavior on the file system, and (iii) the data sharing trends of users and projects. To the best of our knowledge, our work is the first of its kind to provide comprehensive insights on user behavior from multiple science domains through metadata analysis of a large-scale shared file system. We envision that this OLCF case study will provide valuable insights for the design, operation, and management of storage systems at scale, and also encourage other HPC centers to undertake similar such efforts.

Lim, Seung-Hwan↗

DOE OSTI2024

Groundwater and Surface Water Flow (GSFLOW) model files to explore bedrock circulation depth and porosity in Copper Creek, Colorado

This data package contains integrated hydrological model input and output files for Copper Creek, Colorado (24 km2), a tributary of the East River located in the headwaters of the Upper Colorado River Basin. The model code is the U.S. Geological Survey (USGS) Groundwater and Surface Water Flow (GSFLOW) model. The model contains a 100-m grid resolution and a daily timestep. The land surface model is dynamically linked to a three-dimensional groundwater flow model that allows for streamflow gaining and losing conditions. The groundwater model contains 12 model layers and extends 400 m below land surface. The original Copper Creek model was modified to contain geologic layers representing saprolite, shallow bedrock, and deep bedrock. Endmember depth versus hydraulic conductivity relationships and porosity values for fractured crystalline rock are simulated. For the shallow case, median flow depths occur in the shallow saprolite at depths <8 m, while the deep case promotes a median groundwater flow depth of 100 m. With this modeling framework we compare streamflow response to a plausible worst-case drought lasting up to five years. Streamflow metrics of analysis include average streamflow, fraction of stream network that is dry, no-flow duration, average groundwater flow to streams and time to recovery following the drought. Results and implications are presented in a paper submitted to Geophysical Research Letters titled, "The role of bedrock circulation depth and porosity in mountain streamflow response to prolonged drought" by Rosemary WH. Carroll, Andrew H. Manning and Kenneth H Williams. A Readme.txt file provides instructions on how to download all model files and execute each model scenario. In addition to the GSFLOW output/prms/copper_drought.csv file containing daily basin water stores and fluxes (refer to GSFLOW manual) and the output/prms/copper_drought_statvar.dat file with output defined in the gsflow3.control file (refer to GSFLOW Manual), output files also include spatially distributed daily values of total evapotranspiration, canopy evaporation, precipitation, snowfall, infiltration, snow water equivalent, potential evapotranspiration, recharge, sublimation, soil moisture, contributing interflow, water table elevations, changes in groundwater storage, groundwater evapotranspiration, interbasin groundwater flow (limited to the alluvium below the stream outlet), and surface-groundwater exchanges within the river system.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Files and scripts to support manuscript Needham et al. Canopy Gradients of Respiration

This dataset includes the parameter files, relevant output files, and scripts to perform analysis with Jupyter notebooks that support the manuscript Needham et al 2025 “Canopy Gradients of Respiration Drive Plant Carbon Budgets and Leaf Area Index.” We add functionality to the Functionally Assembled Terrestrial Ecosystem Simulator (FATES) to allow flexible vertical gradients of leaf maintenance respiration (Rdark) and maximum carboxylation rate (Vcmax) through the canopy. We test the sensitivity of FATES to canopy gradients in Rdark, both in global simulations to assess broad scale impacts on leaf area index (LAI) and vegetation carbon, and in single site simulations where we assess impacts on plant functional type (PFT) competitive dynamics. Parameter files are netcdf files that can be converted to human readable .cdl files using NCO tools. Analysis scripts are Jupyter notebook files. These can be opened and run using the open source Jupyter notebook software. Model outputs are netcdf files.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2017

File format for normalizing radiological concentration exposure rate and dose rate data for the effects of radioactive decay and weathering processes

This report specifies the electronic file format that was agreed upon to be used as the file format for normalized radiological data produced by the software tool developed under this TI project. The NA-84 Technology Integration (TI) Program project (SNL17-CM-635, Normalizing Radiological Data for Analysis and Integration into Models) investigators held a teleconference on December 7, 2017 to discuss the tasks to be completed under the TI program project. During this teleconference, the TI project investigators determined that the comma-separated values (CSV) file format is the most suitable file format for the normalized radiological data that will be outputted from the normalizing tool developed under this TI project. The CSV file format was selected because it provides the requisite flexibility to manage different types of radiological data (i.e., activity concentration, exposure rate, dose rate) from other sources [e.g., Radiological Assessment and Monitoring System (RAMS), Aerial Measuring System (AMS), Monitoring and Sampling). The CSV file format also is suitable for the file format of the normalized radiological data because this normalized data can then be ingested by other software [e.g., RAMS, Visual Sampling Plan (VSP)] used by the NA-84’s Consequence Management Program.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Format reference for the Atlas of Neutron Resonances electronic files

The Atlas of Neutron Resonances is the direct successor to the vaunted BNL-325 series of references. The current Atlas is the culmination of the lifelong work of Said Mughabghab and within it Dr. Mughabghab strove to compile all known neutron resonances and related data. The current version of the Atlas is distributed in two volumes, covering isotopes with Z=1-60 in volume I and Z=61-104 in volume II. The Atlas is organized into two parts, a general reference for all things related to neutron resonance physics and listings of parameters for each element and/or isotope. The general reference is repeated in each volume. The individual elements and/or isotope listings contain a list of thermal cross section data, resonance properties, lists of resonance parameters for each isotope (and occasionally for elements in the event that resonances could not be attributed to individual isotopes) and an extensive bibliography. The entries of the Atlas are derived from two sets of files: txsrp files: these files contain listings of thermal properties, resonance properties, bibliographic references and coding that defines the columns in the corresponding cftma file. cfmta files: these files contain a list of compiled resonance parameters. There is one txsrp and one cfmta file for each element.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

Release of ENDF/B-VIII.1β1-Based ACE Data Files

On March 1 st , 2023, the National Nuclear Data Center (NNDC) released the ENDF/B-VIII.1β1 nuclear data library, which contained neutron, spontaneous and neutron-induced fission yield, and alpha sublibraries. The library was released in the standard Evaluated Nuclear Data File (ENDF) format. The files can be accessed on Brookhaven National Laboratory’s GitLab. The thermal scattering law (TSL) sublibrary will be released at a later date. The files provided in the neutron sublibrary were processed into A Compact ENDF (ACE)-formatted files, verified, and validated by the XCP-5 Nuclear Data Team. The resulting collection of ACE files will be referred to as the “e81b1” library. This report details the processing of these files and the quality assurance approach taken. This is not intended to be a full validation effort; rather, this library is intended to simply reproduce the released files for further validation testing by the community. The e81b1 ACE library will be released to the NNDC for access through GitLab and hosted locally on LANL computing resources. This library is only intended for analysis to support the future ENDF/B-VIII.1 release and should not be used for production results.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS1989

Deceit: A flexible distributed file system

Deceit, a distributed file system (DFS) being developed at Cornell, focuses on flexible file semantics in relation to efficiency, scalability, and reliability. Deceit servers are interchangeable and collectively provide the illusion of a single, large server machine to any clients of the Deceit service. Non-volatile replicas of each file are stored on a subset of the file servers. The user is able to set parameters on a file to achieve different levels of availability, performance, and one-copy serializability. Deceit also supports a file version control mechanism. In contrast with many recent DFS efforts, Deceit can behave like a plain Sun Network File System (NFS) server and can be used by any NFS client without modifying any client software. The current Deceit prototype uses the ISIS Distributed Programming Environment for all communication and process group management, an approach that reduces system complexity and increases system robustness.

Siegel, Alex↗

NASA NTRS1993

Using Transparent Informed Prefetching (TIP) to reduce file read latency

As processor performance gains continue to outstrip Input/Output gains, I/O performance is becoming critical to overall system performance. File read latency is the most significant bottleneck for high performance I/O. Other aspects of I/O performance benefit from recent advances in disk bandwidth and throughput resulting from disk arrays, and in write performance derived from buffered write behind and the Log-structured File System. The access gap problem limiting improvements in read latency is exacerbated by distributed file systems operating over networks with diverse bandwidth. Focus is on extending the power of caching and prefetching to reduce file read latencies by exploiting hints from high-levels of a system. Such Transparent Informed Prefetching, TIP, and its benefits are described. It is argued that hints that disclose high level knowledge are a means for transferring optimization information across, without violating, module boundaries. How TIP can be used to convert the high throughput of new technologies such as disk arrays and log-structured file systems into low latency for applications is discussed. Our preliminary experiments show reductions in wall - clock execution time of 13 percent and 20 percent for a multiple module compilation tool (make) accessing data on a local disk and remote Coda file server, respectively, and a reduction of 30 percent for a text search (grep) remotely accessing many small files.

Patterson, R. H.↗

NASA NTRS1993

Virtual file system for PSDS

This is a case study. It deals with the use of a 'virtual file system' (VFS) for Boeing's UNIX-based Product Standards Data System (PSDS). One of the objectives of PSDS is to store digital standards documents. The file-storage requirements are that the files must be rapidly accessible, stored for long periods of time - as though they were paper, protected from disaster, and accumulative to about 80 billion characters (80 gigabytes). This volume of data will be approached in the first two years of the project's operation. The approach chosen is to install a hierarchical file migration system using optical disk cartridges. Files are migrated from high-performance media to lower performance optical media based on a least-frequency-used algorithm. The optical media are less expensive per character stored and are removable. Vital statistics about the removable optical disk cartridges are maintained in a database. The assembly of hardware and software acts as a single virtual file system transparent to the PSDS user. The files are copied to 'backup-and-recover' media whose vital statistics are also stored in the database. Seventeen months into operation, PSDS is storing 49 gigabytes. A number of operational and performance problems were overcome. Costs are under control. New and/or alternative uses for the VFS are being considered.

Runnels, Tyson D.↗

NASA NTRS2010

Reporting Differences Between Spacecraft Sequence Files

A suite of computer programs, called seq diff suite, reports differences between the products of other computer programs involved in the generation of sequences of commands for spacecraft. These products consist of files of several types: replacement sequence of events (RSOE), DSN keyword file [DKF (wherein DSN signifies Deep Space Network)], spacecraft activities sequence file (SASF), spacecraft sequence file (SSF), and station allocation file (SAF). These products can include line numbers, request identifications, and other pieces of information that are not relevant when generating command sequence products, though these fields can result in the appearance of many changes to the files, particularly when using the UNIX diff command to inspect file differences. The outputs of prior software tools for reporting differences between such products include differences in these non-relevant pieces of information. In contrast, seq diff suite removes the fields containing the irrelevant pieces of information before processing to extract differences, so that only relevant differences are reported. Thus, seq diff suite is especially useful for reporting changes between successive versions of the various products and in particular flagging difference in fields relevant to the sequence command generation and review process.

Khanampompan, Teerapat↗

DOE OSTI2012

File I/O for MPI Applications in Redundant Execution Scenarios

As multi-petascale and exa-scale high-performance computing (HPC) systems inevitably have to deal with a number of resilience challenges, such as a significant growth in component count and smaller circuit sizes with lower circuit voltages, redundancy may offer an acceptable level of resilience that traditional fault tolerance techniques, such as checkpoint/restart, do not. Although redundancy in HPC is quite controversial due to the associated cost for redundant components, the constantly increasing number of cores-per-processor is tilting this cost calculation toward a system design where computation, such as for redundancy, is much cheaper and communication, needed for checkpoint/restart, is much more expensive. Recent research and development activities in redundancy for Message Passing Interface (MPI) applications focused on availability/reliability models and replication algorithms. This paper takes a first step toward solving an open research problem associated with running a parallel application redundantly, which is file I/O under redundancy. The approach intercepts file I/O calls made by a redundant application to employ coordination protocols that execute file I/O operations in a redundancy-oblivious fashion when accessing a node-local file system, or in a redundancy-aware fashion when accessing a shared networked file system. A proof-of concept prototype is presented and a number of coordination protocols are described and evaluated. The results show the performance impact for redundantly accessing a shared networked file system, but also demonstrate the capability to regain performance by utilizing MPI communication between replicas and parallel file I/O.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2013

A model for optimizing file access patterns using spatio-temporal parallelism

Boonthanome, Nouanesengsy↗

DOE OSTI2022

Scalable access to shared files in a distributed system

Logic may pre-populate cache of compute nodes allocated to a user's session. Logic may access an image file of a global filesystem, the image file to comprise a first set of one or more files associated with a user session and a second set of one or more files upon which execution of the first set will depend. Logic may store the image file in local data storage of each compute node of a spanning tree of compute nodes allocated for the user session. Logic may unpack the image file based on the data. Logic may configure the local environment of each compute node in the spanning tree to identify a location in local memory of each of the files. And logic may initiate execution of a local filesystem service in each of the compute nodes of the spanning tree.

Oganezov, Alexander↗