SEARCH · Engineering Papers

Results for “FILES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

DOE OSTI2022

DENOVA: Deduplication Extended NOVA File System

This paper shows mathematically and experimentally that inline deduplication is not suitable for file systems on ultra-low latency Intel Optane DC PM devices in terms of performance, and proposes DeNova, an offline deduplication specially designed for log-structured NVM file systems such as NOVA. DeNova offers high-performance and low-latency I/O processing and executes deduplication in the background without interfering with foreground I/Os. DeNova employs DRAM-free persistent deduplication metadata, favoring CPU cache line, and ensures failure consistency on any system failure. We implement DeNova in the NOVA file system. Evaluation with DeNova confirms a negligible performance drop of baseline NOVA of less than 1%, while gaining high storage space savings. Extensive experiments show DeNova is failure consistent in all failure scenario cases.

Khan, Awais↗

DOE OSTI2021

Reactive Transport Simulations of High-Tempertature Geologic Thermal Energy Storage (GeoTES) in Deep Saline Formations - I/O Files

Simulation input and output files, post-processed figures and excel tables, and tecplot layout files for generating figures. These simulations were run with TOUGHREACT V4.12 by Lawrence Berkeley National Laboratory in 2021. This work was completed as part of the geologic thermal energy storage (GeoTES) research project reported in the final report for Phase I of this work, which is linked below.

15 GEOTHERMAL ENERGY↗

DOE OSTI2021

Historical NTS digitizer file format

The historical NTS *.dig data formal is not widely known. I document the proper *.dig file format. I include the MATLAB code used to read *.dig files with the historical data analysis software, "eXreme Las Vegas".

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Architecture and performance of Perlmutter's 35 PB ClusterStor E1000 all-flash file system

NERSC's newest system, Perlmutter, features a 35 PB all-flash Lustre file system built on HPE Cray ClusterStor E1000. Here, we present its architecture, early performance figures, and performance considerations unique to this architecture. We demonstrate the performance of E1000 OSSes through low-level Lustre tests that achieve over 90% of the theoretical bandwidth of the SSDs at the OST and LNet levels. We also show end-to-end performance for both traditional dimensions of I/O performance (peak bulk-synchronous bandwidth) and nonoptimal workloads endemic to production computing (small, incoherent I/Os at random offsets) and compare them to NERSC's previous system, Cori, to illustrate that Perlmutter achieves the performance of a burst buffer and the resilience of a scratch file system. Finally, we discuss performance considerations unique to all-flash Lustre and present ways in which users and HPC facilities can adjust their I/O patterns and operations to make optimal use of such architectures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Design and implementation of dynamic I/O control scheme for large scale distributed file systems

In this paper, we have analyzed the input/output (I/O) activities of Cori, which is a high-performance computing system at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory. Our analysis results indicate that most users do not adjust storage configurations but rather use the default settings. In addition, owing to the interference from many applications running simultaneously, the performance varies based on the system status. To configure file systems autonomously in complex environments, we developed DCA-IO, a dynamic distributed file system configuration adjustment algorithm that utilizes the system log information to adjust storage configurations automatically. Our scheme aims to improve the application performance and avoid interference from other applications without user intervention. Moreover, DCA-IO uses the existing system logs and does not require code modifications, an additional library, or user intervention. To demonstrate the effectiveness of DCA-IO, we performed experiments using I/O kernels of real applications in both an isolated small-sized Lustre environment and Cori. Our experimental results shows that our scheme can improve the performance of HPC applications by up to 263% with the default Lustre configuration.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Radiation portal monitor data file format for comprehensive background radiation monitoring

Radiation portal monitors (RPMs) are widely used at border security checkpoints to detect the presence of radioactive materials in people, vehicles, and cargo. Typically, RPM detection systems consist of two pillars equipped with gamma and neutron detectors. To improve detection efficiency, RPMs employ techniques such as a limited energy window, dynamic alarm thresholds, and lead shielding. However, without continuous monitoring of background radiation, signal interpretation can be compromised, because environmental factors and mechanical failures can cause fluctuations. Here, we introduce a daily file format that logs gamma background and neutron background radiation levels continuously over a 24 h period; this format is different from traditional formats that record data only when the RPM is active or occupied. The approach enables RPM operators and analysts to (1) identify and diagnose malfunctioning components, (2) adjust system settings to account for dynamic environmental factors, and (3) use the recorded data to characterize outer space phenomena. Continuous background reporting is essential for identifying issues such as faulty connections, voltage divider failures, and errors in background updates. Continuous background reporting also enables the detection of external influences, including nearby X-ray scanners, temperature fluctuations, rainfall, cosmic radiation, and lunar phase changes. These data files are designed to be easily evaluated and parsed using common tools, and a quick review by an expert is often sufficient for problem diagnosis. We anticipate that continuous background radiation monitoring and these new strategies will significantly improve the accuracy and reliability of RPM systems, reducing the rate of false alarms and enhancing overall system performance.

Background radiation monitoring↗

DOE OSTI2018

Advanced TidGen Power System - OpenFOAM Version 5 CFD Case Files

The TidGen Power System generates emission-free electricity from tidal currents and connects directly into existing grids using smart grid technology. The power system consists of three major subsystems: shore-side power electronics, mooring system, and turbine generator unit (TGU) device. This submission contains supporting CFD files, case files and geometry for the Advanced TidGen. TSR = Tip speed ratio Cp = Power coefficient Cl = Lift coefficient Cd = Drag coefficient

16 TIDAL AND WAVE POWER↗

DOE OSTI2011

The Future of the Andrew File System

The talk will discuss the ten operational capabilities that have made AFS unique in the distributed file system space and how these capabilities are being expanded upon to meet the needs of the 21st century. Derrick Brashear and Jeffrey Altman will present a technical road map of new features and technical innovations that are under development by the OpenAFS community and Your File System, Inc. funded by a U.S. Department of Energy Small Business Innovative Research grant. The talk will end with a comparison of AFS to its modern days competitors.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2011

A History of the Andrew File System

Derrick Brashear and Jeffrey Altman will present a technical history of the evolution of Andrew File System starting with the early days of the Andrew Project at Carnegie Mellon through the commercialization by Transarc Corporation and IBM and a decade of OpenAFS. The talk will be technical with a focus on the various decisions and implementation trade-offs that were made over the course of AFS versions 1 through 4, the development of the Distributed Computing Environment Distributed File System (DCE DFS), and the course of the OpenAFS development community. The speakers will also discuss the various AFS branches developed at the University of Michigan, Massachusetts Institute of Technology and Carnegie Mellon University.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

pyEGAF: An open-source Python library for the Evaluated Gamma-ray Activation File

The Evaluated Gamma-ray Activation File (EGAF) is one of the most comprehensive resources for thermal neutron-capture data. This database contains data from prompt gamma activation analysis measurements carried out in a consistent manner using the same experimental configuration at the Budapest Research Reactor for 245 isotopes. Although these valuable datasets have been freely available for many years, one of the drawbacks is the outdated and cryptic Evaluated Nuclear Structure Data File (ENSDF) format that is currently adopted for dissemination, making it difficult for users unfamiliar with the format to access and utilize the data contained therein. Furthermore, the ENSDF format does not readily lend itself to modern computational technologies and a parser is required to interpret the complicated mixed-record format. To help overcome these challenges, we have developed a translator to convert the ENSDF-formatted datasets into an open standard JavaScript Object Notation (JSON) format enabling accessibility to applications using different programming languages running in different environments. To compliment this effort, we have also developed an open-source software package implemented in Python, pyEGAF, that is designed to interact with the JSON data structures for general purpose access, manipulation, and analysis of the neutron-capture $\gamma$-ray data in EGAF. The new format, together with the pyEGAF library, greatly enhances access to the wider applications community where EGAF data may be useful or is required.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2012

Automatic generation of IGUN{sup (C)} input files for enhanced mesh resolution at the plasma meniscus

The plasma-beam interface (meniscus) is highly nonlinear and its correct simulation needs a mesh resolution of the order of the Debye length. In high intensity ion sources, the plasma density is usually too high and the Debye length is too small for a sufficient mesh resolution. A well established method to overcome this dilemma is the use of a field line and an equipotential line to be created in a first run, in order to dissect the simulation problem into a plasma part with much higher mesh resolution and a transport part with usual resolution. In the past many users of IGUN have found it difficult to perform this dissection. Therefore, a new feature has been added to IGUN to automatically write new input files for the dissected areas. For this a field line starting point needs to be defined as well as the potential of a pseudo electrode. The field line then is used for the plasma part as a slanted and curved Neumann boundary, while the pseudo electrode will act as the extraction electrode. The trajectory end data then are used in the automatically generated concatenating run as ion starting input without any need for the user to adjust for positions or different mesh resolutions. Here we show as an example the simulation of the well-known CHORDIS ion source, the calculated field line, the pseudo equipotential line, and the resulting simulations for the automatically generated input files for the plasma and the transport parts.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2015

File-Based Data Flow in the CMS Filter Farm

During the LHC Long Shutdown 1, the CMS Data Acquisition system underwent a partial redesign to replace obsolete network equipment, use more homogeneous switching technologies, and prepare the ground for future upgrades of the detector front-ends. The software and hardware infrastructure to provide input, execute the High Level Trigger (HLT) algorithms and deal with output data transport and storage has also been redesigned to be completely file- based. This approach provides additional decoupling between the HLT algorithms and the input and output data flow. All the metadata needed for bookkeeping of the data flow and the HLT process lifetimes are also generated in the form of small “documents” using the JSON encoding, by either services in the flow of the HLT execution (for rates etc.) or watchdog processes. These “files” can remain memory-resident or be written to disk if they are to be used in another part of the system (e.g. for aggregation of output data). We discuss how this redesign improves the robustness and flexibility of the CMS DAQ and the performance of the system currently being commissioned for the LHC Run 2.

Andre, J. M.↗

DOE OSTI2017

Quasiphase Transition in a Single File of Water Molecules Encapsulated in (6,5) Carbon Nanotubes Observed by Temperature-Dependent Photoluminescence Spectroscopy

Molecules confined inside single-walled carbon nanotubes (SWCNTs) behave quite differently from their bulk analogues. In this work we present temperature-dependent (4.2 K up to room temperature) photoluminescence (PL) spectra of water-filled and empty single-chirality (6,5) SWCNTs. Superimposed on a linear temperature-dependent PL spectral shift of the empty SWCNTs, an additional stepwise PL spectral shift of the water-filled SWCNTs is observed at ~ 150 K . With the empty SWCNTs serving as an ideal reference system, we assign this shift to temperature-induced changes occurring in the single-file chain of water molecules encapsulated in the tubes. Our molecular dynamics simulations further support the occurrence of a quasiphase transition of the orientational order of the water dipoles in the single-file chain.

1-dimensional systems↗

DOE OSTI2023

UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage

We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.

Brim, Michael↗

DOE OSTI2013

Fast Global File Status

Fast Global File Status (FGFS) is a system software package that implimints a scalable mechanism to retrieve file information, such as its degree of distribution or replication and consistency.

Ahn, D.H.↗

DOE OSTI2018

Fasta File Splitter

The FASTA File Splitter program can be used to split apart a protein FASTA file into a number of sections. Although the splitting is random, each section will have a nearly identical number of residues.

Monroe, Matthew↗

DOE OSTI2018

MS File Info Scanner

The MS File Info Scanner can be used to scan a series of MS data files (or data folders) and extract the acquisition start and end times, number of spectra, and the total size of the data. In addition, it can create 2D-plot graphics of the m/z values vs. time for QA purposes.

Wilkins, Christopher↗

DOE OSTI2013

Lightweight File System (LWFS) v. 1.0

The Lightweight File System (LWFS) is a storage system that provides a minimal set of I/O-system functionality required by file system and/or I/O library implementations for massively parallel machines. In particular, the LWFS-core consists of a scalable security model, an efficient data-movement protocol, and a direct interface to object-based storage devices. Higher-level services such as namespace management, consistency semantics, reliability, and so forth are layered on top of the core services to provide application-specific functionality as needed. The LWFS code contains implementations of the core services and reference implementations of a number of supplemental services for namespace management and transaction support.

Oldfield, Ron↗