SEARCH · Engineering Papers

Results for “File Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

All sources NASA NTRS DOE OSTI

DOE OSTI2024

State Requirements for Electric Distribution System Planning

Utilities have conducted distribution planning since they first began building and operating electricity systems. But filing these plans for regulatory and stakeholder review is a relatively recent phenomenon. This report summarizes legislative and regulatory requirements for regulated electric utilities to file some type of distribution system plan in 20 U.S. jurisdictions. Some plans focus on expedited cost recovery for certain types of distribution system improvements; other plans focus on investments for grid modernization or distributed energy resources. Increasingly, states are adopting requirements for Integrated Distribution Plans. Such plans provide holistic grid investment strategies that address state and local policies and increasing complexity at the grid edge. The report covers the following topics for distribution system plans, highlighting advanced practices: -State goals and objectives -Procedural requirements -Forecasting loads and distributed energy resources -Hosting capacity analysis -Baseline information requirements -Grid modernization strategy -Grid needs assessment -Non-wires solutions -Reliability and resilience analyses -Stakeholder engagement -Equity -Pilots -Coordination with other planning processes The report includes links to legislation; regulatory requirements, proceedings, and orders; and filed utility plans. The U.S. Department of Energy’s Office of Electricity provided funding support.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1996

The medium is NOT the message or Indefinitely long-term file storage at Leeds University

Approximately 3 years ago we implemented an archive file storage system which embodies experiences gained over more than 25 years of using and writing file storage systems. It is the third in-house system that we have written, and all three systems have been adopted by other institutions. This paper discusses the requirements for long-term data storage in a university environment, and describes how our present system is designed to meet these requirements indefinitely. Particular emphasis is laid on experiences from past systems, and their influence on current system design. We also look at the influence of the IEEE-MSS standard. We currently have the system operating in five UK universities. The system operates in a multi-server environment, and is currently operational with UNIX (SunOS4, Solaris2, SGI-IRIX, HP-UX), NetWare3 and NetWare4. PCs logged on to NetWare can also archive and recover files that live on their hard disks.

Holdsworth, David↗

DOE OSTI2023

"PoliMOR: A Policy Engine \"Made-to-Order\" for Automated and Scalable Data Management in Lustre"

Modern supercomputing systems are increasingly reliant on hierarchical, multi-tiered file and storage system architectures due to cost-performance-capacity trade-offs. Within such multi-tiered systems, data management services are required to maintain healthy utilization, performance, and capacity levels. We present PoliMOR, a pragmatic and reliable policy-driven data management framework. PoliMOR is composed of modular, single-purpose agents that gather file system metadata and enforce policies on storage systems. PoliMOR facilitates automated and scalable data management with customizable agents tailored to HPC facility-specific storage systems and policies. Our evaluations demonstrate the scalability and performance of PoliMOR both by its individual agents and as a collective entity. We believe PoliMOR is widely applicable across HPC facilities with large-scale data management challenges and will garner interest from the HPC community, given its flexible and open-source nature.

George, Anjus↗

DOE OSTI2022

Scalable access to shared files in a distributed system

Logic may pre-populate cache of compute nodes allocated to a user's session. Logic may access an image file of a global filesystem, the image file to comprise a first set of one or more files associated with a user session and a second set of one or more files upon which execution of the first set will depend. Logic may store the image file in local data storage of each compute node of a spanning tree of compute nodes allocated for the user session. Logic may unpack the image file based on the data. Logic may configure the local environment of each compute node in the spanning tree to identify a location in local memory of each of the files. And logic may initiate execution of a local filesystem service in each of the compute nodes of the spanning tree.

Oganezov, Alexander↗

DOE OSTI2019

Systems and methods for generating slice files from native CAD geometries

A system and method for improving the production of objects with fully defined CAD models by generating CAD slice files from the native CAD geometries for use by AM machines. An electronic processing element receives the CAD model having a native and/or neutral format with metadata, determines a desired resolution, slices the CAD model to create the slice files having a slice format (e.g., point, edge, surface, volume) and retaining the metadata, and evaluates the slice files to determine whether the resolution has been achieved. If so, the slice files are sent to the AM machine. If not, a new desired resolution is determined and the CAD model is re-sliced, which may include combining the slice files to regenerate the CAD model. A support structure for the object may be analyzed, and if distortion is predicted, a modified CAD model may be created and sliced to create modified slice files.

Vernon, Gregory J.↗

NASA NTRSDate not supplied

Coupled Reactor and Engine Nuclear Thermal Propulsion Modeling Methodology

The design and development process of a Nuclear Thermal Propulsion (NTP) system requires extensive multiphysics modeling to couple the neutron physics and thermal feedback effects to determine the reactor’s power shape. Propulsion system performance codes utilize this power shape to determine NTP key performance parameters. While the power shape is heavily dependent on the temperature profile and geometry of the reactor, many analyses either assume a constant power shape, or use neutronics analysis to determine a power shape for a specific reactor configuration. The development of a coupling interface for a propulsion system performance code and a Monte Carlo neutron transport code (OpenMC) allows for the reactor power shape to be calculated in an iteration loop. The interface utilizes a file share system to transfer geometry dimensions, temperatures, and material identifiers to OpenMC, which is used to perform a neutron transport simulation of a design like the government Testing Reference Design reactor. The interface is then able to post-process the results from OpenMC and use the same file share system to share a power shape and other important neutron transport parameters to the system performance code. Initial results show that neglecting the changes to power shape when comparing reactor configurations can yield inaccurate results. Furthermore, utilizing propellants other than hydrogen gas can cause significant changes to the power shape, and thus, the thermal performance of a specific reactor design. This methodology is being expanded to allow for multiple families of NTP reactors to be analyzed, including block moderator, particle bed, and NERVA-derived reactors.

multiphysics coupling↗

NASA NTRSDate not supplied

Coupled Reactor and Engine Nuclear Thermal Propulsion Modeling Methodology

multiphysics coupling↗

DOE OSTI2024

A Framework for Addressing Hydropower Modeling Gaps in Electric Grid Planning and Operational Studies [Slides]

Realistic representation of hydropower in power system planning and operation studies is extremely important as it helps in avoiding under/over-estimation of the services the hydro power units can provide. A framework and tools have been developed to account for hydrological conditions and modify power system models accordingly. For example, HyDat – data and AI driven platform, that provides that data required for editing the power system model files Collaborative effort with V&R Energy. Also, development of POM based tools have been performed, a) Tool1: editing .raw files with realistic current and maximum hydrogeneration and redispatch of other units, and b) Tool2: Editing .dyr files to represent water head, current and maximum generation. Finally the framework, HASP framework combines HyDat and V&R energy tools to produce modified power system model files. Key Take-away points from contingency analysis studies include a) the number of critical contingencies increases as the water levels are reduced (70%>75%>80%), and b) both the extent and the quality of dynamic response of the Hydropower unit is different with varying water head levels.

13 HYDRO ENERGY↗

NASA NTRS2020

What Can We Learn from One Billion Ground System Log Messages?

Shortage of log-based data in a ground system they have traditionally been the under achievers in a satellite ground system. This is due to several factors: Once log messages scroll out of view on the TTC event console window they are soon forgotten. Application and system log files are scattered across directories within a system, across a multitude of servers, and across one or more databases making access cumbersome. Typical tools to perform log file content searching are generally crude and typically only employed as part of trouble-shooting exercises.As we move towards satellite constellations and fleets and add even more status information, the number of messages keeps growing. One mission now estimates that they could generate 3,000,000 messages per day 1 billion per year - for the life of their mission. What to do with those 1 billion messages? That is the challenge. With the recent technological advances in the management of large data sets, text-based processing, and data analytics, there are now capabilities that we can provide to the ground system engineers and satellite operators to address what we postulate are missed opportunities. Advanced real-time log analysis can allow us to be less reactionary in favor of being more proactive. Analytics goals include the ability to: Identify root cause of unexpected events, failures or error conditions enabled by correlating disparate data. Detect security breaches attempts before they are successful. Help admins ensure IT resources continue running optimally. Identify trends and patterns that may indicate impending failures or error conditions for valuable assets before they happen. Compare satellites in a fleet or constellation in terms of number of alarms, number of command sent to them, etc.. Answer questions like "Are the operations support needs increasing over the past year?" or "Have we seen this combination of alarm conditions before?" But really, once the tools are readily available the users will start realizing what can be done with their new powers. In this presentation we will show the results of analyzing millions of actual mission operations log messages, how the results can be displayed to the user, and how new products now available as open source can be applied to the challenges of large scale time-tagged text-based mission operations messages. Flight operations team members believe that this is a powerful new option for how they assess overall system and space asset health. Technical descriptions of the design, tools, and storage will be provided. One billion messages? Bring'em on!

Orsborne, Sharon↗

NASA NTRS1985

Planetary image conversion task

The Planetary Image Conversion Task group processed 12,500 magnetic tapes containing raw imaging data from JPL planetary missions and produced an image data base in consistent format on 1200 fully packed 6250-bpi tapes. The output tapes will remain at JPL. A copy of the entire tape set was delivered to US Geological Survey, Flagstaff, Ariz. A secondary task converted computer datalogs, which had been stored in project specific MARK IV File Management System data types and structures, to flat-file, text format that is processable on any modern computer system. The conversion processing took place at JPL's Image Processing Laboratory on an IBM 370-158 with existing software modified slightly to meet the needs of the conversion task. More than 99% of the original digital image data was successfully recovered by the conversion task. However, processing data tapes recorded before 1975 was destructive. This discovery is of critical importance to facilities responsible for maintaining digital archives since normal periodic random sampling techniques would be unlikely to detect this phenomenon, and entire data sets could be wiped out in the act of generating seemingly positive sampling results. Reccomended follow-on activities are also included.

Martin, M. D.↗

DOE OSTI2020

Methods and apparatus to modify a binary file for scalable dependency loading on distributed computing systems

Methods and apparatus to modify a binary file for scalable loading on distributed computing systems are disclosed. An example method to modify a binary file for scalable loading on a distributed computing system includes identifying, by executing an instruction with a processor, in the binary file a file name for a shared file, determining, by executing an instruction with the processor, whether the shared file can be loaded based on a path name specified in the binary file, and, responsive to a determination that the shared file can be loaded, generating, by executing an instruction with the processor, an identifier that can be used to access a local file scalably copied from a global file system, and modifying, by executing an instruction with the processor, the binary file by replacing, in the binary file, the file name with the generated identifier.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1984

Asynchronous file transfer to IBM PC's

The Asynchronous File Transfer System is used for interactively selecting and tramsmitting text files from the Langley Research Center's Business Data Systems Division (BDSD) host processor to an IBM Personal Computer. The IBM asynchronous communications support package is used for handling the communications on the personal computer. An application program (NATURAL, COBOL, etc.) is used for selecting and formatting the records to be transmitted, and three subroutine modules residing on the BDSD host processor are used for interfacing the application program and the communications software. Each record transmitted to the Personal Computer must be in the standard ASCII format. This record format is directly accessible by BASIC and many of the personal computer software packages provide utility programs for converting them to the format required for the particular package in question.

Hoerger, J.↗

DOE OSTI2022

Accelerating shared file checkpoint with local burst buffers

A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.

Gooding, Thomas↗

NASA NTRS1988

Archival-System Computer Program

Files stored with various degrees of relative performance. ARCHIVE system provides permanent storage area for files to which infrequent access is required. Routines designed to provide simple mechanism by which users store and retrieve files. User treats ARCHIVE as interface to "black box" where files are stored. There are five ARCHIVE user commands, though ARCHIVE employs standard VMS directives and VAX BACKUP utility program. Special care taken to provide security needed to insure integrity of files over period of years. ARCHIVE written in DEC VAX DCL.

Scott, Peter↗

DOE OSTI2025

Leveraging Pre-Built Catalogs and Object-Level Scheduling to Eliminate I/O Bottlenecks in HPC Environments

Modern High-Performance Computing (HPC) environments face mounting challenges due to the shift from large to small file datasets, along with an increasing number of users and parallelized applications. As HPC systems rely on Parallel File Systems (PFS), such as Lustre for data processing, performance bottlenecks stemming from Object Storage Target (OST) contention have become a significant concern. Existing solutions, such as LADS with its object-level scheduling approach, fall short in large-scale HPC environments due to their inability to effectively address metadata I/O bottlenecks and the growing number of I/O processes. This study highlights the pressing need for a comprehensive solution that tackles both OST contention and metadata I/O challenges in diverse HPC workloads. To address these challenges, we propose SwiftLoad, an object-level I/O scheduling framework that leverages a metadata catalog to enhance the performance and efficiency of parallel HPC utilities. The adoption of the metadata catalog mitigates the metadata I/O bottlenecks that commonly occur in HPC utilities, a challenge that is particularly pronounced in object-level I/O scheduling. SwiftLoad addresses OST contention and the uneven distribution of I/O processes across different OSTs through mathematical modeling and incorporates a Loader Configuration Module to regulate the number of I/O processes. Evaluated with two representative utilities—data deduplication profiling and data augmentation—SwiftLoad achieved performance improvements of up to 5.63x and 11.0x, respectively, on a production supercomputer.

HPC↗

DOE OSTI2017

Dynamic File Striping and Data Layout Transformation on Parallel System with Fluctuating I/O Workload

As the number of compute cores on modern parallel machines increases to more than hundreds of thousands, scalable and consistent I/O performance is becoming hard to obtain due to fluctuating file system performance. This fluctuation is often caused by rebuilding RAID disk from hardware failures or concurrent jobs competing for I/O. We present a mechanism that stripes across a dynamically-selected subset of I/O servers with the lightest workload to achieve the best I/O bandwidth available from the system. We implement this mechanism into an I/O software layer that enables memory-to-file data layout transformation and allows transparent file partitioning. File partitioning is a technique that divides data among a set of files and manages file access, making data appear as a single file to users. Experimental results on NERSC’s Hopper indicate that our approach effectively isolates I/O variation on shared systems and improves overall I/O performance significantly.

Son, Seung Woo↗

DOE OSTI2015

SU-E-T-363: Error Detection Comparison of EPID and MLC Log File Based IMRT QA Systems

Purpose: In this study we will compare the ability of three QA methods (Delta4, MU-EPID, Dynalog QA) to detect specific errors. Methods: A Varian Novalis Tx with a HD120 MLC and aS1000 Electronic Portal Imaging Device (EPID) was used in our study. Multi-leaf collimator (MLC) errors, gantry angle and dose errors were introduced into 5 volumetric arc therapy (VMAT) plans. 3D dose distributions calculated with data from the EPID and Dynalog QA methods were compared with the planned dose distribution. The gamma passing percentages as well as percentage error of planning target volume (PTV) dose were used for passing determination. Baselines for gamma passing percentages and PTV dose were established by measuring the original plan 5 times consecutively. Standard passing thresholds as well as thresholds derived from receiver operator characteristic (ROC) analysis and 2 standard deviation (SD) criteria were used. Results: When applying the standard 95% pass rate at 3%/3mm gamma analysis 14, 21 and 8 of 30 errors were detected by the Delta4, MU-EPID and Dynalog QA methods respectively. Thresholds set at 2 SD from our base line measurements resulted in the detection of 18, 9 and 14 of 30 errors for the Delta4, MU-EPID and Dynalog QA methods respectively. When using D2 of the PTV as a metric the Dynalog QA detected 20 of 30 errors while the EPID method detected 14 of 30 errors. Using D98 of the PTV, Dynalog QA detected 13 of 30 while the EPID detected 3 of 30 errors. Conclusion: Although MU-EPID detected the most errors at the standard 95% cutoff it also produced the most false detections in the baseline data. The Dynalog QA was the most effective when the ROC adjusted passing threshold was used. D2 was more effective as a metric for detecting errors than D98.

60 APPLIED LIFE SCIENCES↗

NASA NTRS1983

User-programmer dialogue: Guidelines for designing menus and help files for interactive computer systems

This document is a set of guidelines to aid a programmer in making the various decisions necessary for a clear user-programmer dialogue. Its goal is to promote an effective and efficient transfer of information between programmer and user. These guidelines are divided into four sections: (1) Format, (2) Sequence, (3) Audience, and (4) Aim. Format, in terms of this study, means the spatial and structural presentation of information. Sequence deals with the procedural aspects of multiple panel displays. This section looks at the issues of timelines of presentation, modularization of information, and patterns of user behavior. Audience looks at the relationship among programmer, user, and message. It covers the issues of analyzing the audience's knowledge, attitudes, and needs, anticipating the audience's inferences, and identifying textual ambiguities. The programmer's aim or intention shows up in everything from tone to format. Aim considers the programmer's purpose.

Carlson, P. A.↗