Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “FILES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Distributed metadata servers for cluster file systems using shared low latency persistent key-value metadata store

A cluster file system is provided having a plurality of distributed metadata servers with shared access to one or more shared low latency persistent key-value metadata stores. A metadata server comprises an abstract storage interface comprising a software interface module that communicates with at least one shared persistent key-value metadata store providing a key-value interface for persistent storage of key-value metadata. The software interface module provides the key-value metadata to the at least one shared persistent key-value metadata store in a key-value format. The shared persistent key-value metadata store is accessed by a plurality of metadata servers. A metadata request can be processed by a given metadata server independently of other metadata servers in the cluster file system. A distributed metadata storage environment is also disclosed that comprises a plurality of metadata servers having an abstract storage interface to at least one shared persistent key-value metadata store.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Architecture and Performance of Perlmutter’s 35 PB ClusterStor E1000 All-Flash File System

NERSC's newest system, Perlmutter, features a 35 PB all-flash Lustre file system built on HPE Cray ClusterStor E1000. We present its architecture, early performance figures, and performance considerations unique to this architecture. We demonstrate the performance of E1000 OSSes through low-level Lustre tests that achieve over 90% of the theoretical bandwidth of the SSDs at the OST and LNet levels. We also show end-to-end performance for both traditional dimensions of I/O performance (peak bulk-synchronous bandwidth) and non-optimal workloads endemic to production computing (small, incoherent I/Os at random offsets) and compare them to NERSC's previous system, Cori, to illustrate that Perlmutter achieves the performance of a burst buffer and the resilience of a scratch file system. Finally, we discuss performance considerations unique to all-flash Lustre and present ways in which users and HPC facilities can adjust their I/O patterns and operations to make optimal use of such architectures.

Lockwood, Glenn K↗
DOE OSTI2016

ARM Data File Standards Version 1.2

The U.S. Department of Energy (DOE)’s Atmospheric Radiation Measurement (ARM) Climate Research Facility performs routine in situ and remote-sensing observations to provide a detailed and accurate description of the Earth atmosphere in diverse climate regimes. The result is a huge archive of diverse data sets containing observational and derived data, currently accumulating at a rate of 30 terabytes (TB) of data and 150,000 different files per month (http://www.archive.arm.gov/stats/). Continuing the current processing while scaling this to even larger sizes is extremely important to the ARM Facility and requires consistent metadata and data standards. The standards described in this document will enable development of automated analysis and discovery tools for the ever growing data volumes. It will enable consistent analysis of the multiyear data, allow for development of automated monitoring and data health status tools, and allow future capabilities of delivering data on demand that can be tailored explicitly for the user needs. This analysis ability will only be possible if the data follows a minimum set of standards. This document proposes a hierarchy of required and recommended standards.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1996

File Exchange Interface

None given. Paper is a pictorial representation of a proposed file exchange interface model.

file transfer distributed information information ↗
DOE OSTI2023

Utilization of ACE nuclear data file toolkit ACEtk to calculate relative sensitivity coefficients of point-kinetics parameters

Sensitivity and uncertainty methods are quintessential for nuclear criticality safety and experiment design. This type of analysis relies on calculations of sensitivity coefficients; sensitivity coefficients of the effective neutron multiplication factor with respect to some nuclear data are predominantly calculated and used. As a part of the Laboratory Directed Research & Development project EUCLID (Experiments Underpinned by Computational Learning for Improvements in nuclear Data) at Los Alamos National Laboratory, sensitivity coefficients of many radiation detector measurement responses with respect to nuclear data were investigated. Specifically, this paper outlines a method to calculate point-kinetics parameters relative sensitivity coefficients with respect to nuclear data. Point-kinetics parameters such as the prompt neutron decay constant, effective delayed neutron fraction, and neutron generation time are especially important to experimenters and reactor operators designing systems with dynamic neutron populations. This method couples capabilities of the ACE (A Compact ENDF) nuclear data file toolkit, ACEtk, with the ability to load cross sections into the radiation transport code Monte Carlo N-Particle (MCNP). In conclusion, key aspects of optimizing this method for a particular application and sensitivity profiles of the Jezebel criticality experiment are examined and discussed.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2016

A survey of techniques for architecting and managing GPU register file

To support their massively-multithreaded architecture, GPUs use very large register file (RF) which has a capacity higher than even L1 and L2 caches. In total contrast, traditional CPUs use tiny RF and much larger caches to optimize latency. Due to these differences, along with the crucial impact of RF in determining GPU performance, novel and intelligent techniques are required for managing GPU RF. In this paper, we survey the techniques for designing and managing GPU RF. We discuss techniques related to performance, energy and reliability aspects of RF. To emphasize the similarities and differences between the techniques, we classify them along several parameters. Lastly, the aim of this paper is to synthesize the state-of-art developments in RF management and also stimulate further research in this area.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Formal Definitions and Performance Comparison of Consistency Models for Parallel File Systems

The semantics of HPC storage systems are defined by the consistency models to which they abide. Storage consistency models have been less studied than their counterparts in memory systems, with the exception of the POSIX standard and its strict consistency model. The use of POSIX consistency imposes a performance penalty that becomes more significant as the scale of parallel file systems increases and the access time to storage devices, such as node-local solid storage devices, decreases. While some efforts have been made to adopt relaxed storage consistency models, these models are often defined informally and ambiguously as by-products of a particular implementation. Here in this work, we establish a connection between memory consistency models and storage consistency models and revisit the key design choices of storage consistency models from a high-level perspective. Further, we propose a formal and unified framework for defining storage consistency models and a layered implementation that can be used to easily evaluate their relative performance for different I/O workloads. Finally, we conduct a comprehensive performance comparison of two relaxed consistency models on a range of commonly seen parallel I/O workloads, such as checkpoint/restart of scientific applications and random reads of deep learning applications. We demonstrate that for certain I/O scenarios, a weaker consistency model can significantly improve the I/O performance. For instance, in small random reads that are typically found in deep learning applications, session consistency achieved a 5x improvement in I/O bandwidth compared to commit consistency, even at small scales.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Utah FORGE Phase Native State FALCON Model Files

The submission includes FALCON input file and mesh for the an initial pressure-temperature simulation, and a second set for pressure-temperature-displacement simulation. All simulations are steady state. Data and input for the FORGE Phase 2 native state model were compiled from historic and FORGE specific characterization efforts. The simulations look to find the in-situ pressure, temperature, and stress distributions in the vicinity of the Utah FORGE site. The purpose of the native state model was to establish a baseline for all other modeling analyses—so that the whole modeling team can start from a consistent set of conditions which makes for comparison of results more straightforward and meaningful. Requires the FALCON simulator which can be found in the "FALCON Simulator" link in the submission. Instructions for compiling and running FALCON can be found on the FORGE branch of the FALCON git repository in the README.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Transitioning from File-Based HPC Workflows to Streaming Data Pipelines with openPMD and ADIOS2

This paper aims to create a transition path from file-based IO to streaming-based workflows for scientific applications in an HPC environment. By using the openPMP-api, traditional workflows limited by filesystem bottlenecks can be overcome and flexibly extended for in situ analysis. The openPMD-api is a library for the description of scientific data according to the Open Standard for Particle-Mesh Data (openPMD). Its approach towards recent challenges posed by hardware heterogeneity lies in the decoupling of data description in domain sciences, such as plasma physics simulations, from concrete implementations in hardware and IO. The streaming backend is provided by the ADIOS2 framework, developed at Oak Ridge National Laboratory. This paper surveys two openPMD-based loosely-coupled setups to demonstrate flexible applicability and to evaluate performance. In loose coupling, as opposed to tight coupling, two (or more) applications are executed separately, e.g. in individual MPI contexts, yet cooperate by exchanging data. This way, a streaming-based workflow allows for standalone codes instead of tightly-coupled plugins, using a unified streaming-aware API and leveraging high-speed communication infrastructure available in modern compute clusters for massive data exchange. We determine new challenges in resource allocation and in the need of strategies for a flexible data distribution, demonstrating their influence on efficiency and scaling on the Summit compute system. The presented setups show the potential for a more flexible use of compute resources brought by streaming IO as well as the ability to increase throughput by avoiding filesystem bottlenecks.

Poeschel, Franz↗
DOE OSTI2024

Rapid particle generation from an STL file and related issues in the application of material point methods to complex objects

Abstract In this paper, we focus on three issues related to applications of material point methods (MPMs) to objects with complex geometries. They are material point generation, compatibility of material points with a mesh, and sensitivity to mesh orientation. An efficient method of generating material points from a stereolithography (STL) file is introduced. This material point generation method is independent of the mesh used in MPM calculations. The compatibility between the material points and the mesh is then studied. We also show that the original MPM and the dual domain material point (DDMP) method are sensitive to mesh orientation. These issues are related to the calculation of the internal force and are concerns of the MPMs. They become more prominent when MPMs are applied to complex geometries. Our numerical results show that the recently developed local stress difference (LSD) algorithm (Perez et al. in J Comp Phys 498:112681, 2024) can be used to effectively address them.

36 MATERIALS SCIENCE↗
DOE OSTI2012

Comp Plan: A computer program to generate dose and radiobiological metrics from dose-volume histogram files

Treatment planning studies often require the calculation of a large number of dose and radiobiological metrics. To streamline these calculations, a computer program called Comp Plan was developed using MATLAB. Comp Plan calculates common metrics, including equivalent uniform dose, tumor control probability, and normal tissue complication probability from dose-volume histogram data. The dose and radiobiological metrics can be calculated for the original data or for an adjusted fraction size using the linear quadratic model. A homogeneous boost dose can be added to a given structure if desired. The final output is written to an Excel file in a format convenient for further statistical analysis. Comp Plan was verified by independent calculations. A lung treatment planning study comparing 45 plans for 7 structures using up to 6 metrics for each structure was successfully analyzed within approximately 5 minutes with Comp Plan. The code is freely available from the authors on request.

61 RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2014

Security in the CernVM File System and the Frontier Distributed Database Caching System

Both the CernVM File System (CVMFS) and the Frontier Distributed Database Caching System (Frontier) distribute centrally updated data worldwide for LHC experiments using http proxy caches. Neither system provides privacy or access control on reading the data, but both control access to updates of the data and can guarantee the authenticity and integrity of the data transferred to clients over the internet. CVMFS has since its early days required digital signatures and secure hashes on all distributed data, and recently Frontier has added X.509-based authenticity and integrity checking. In this paper we detail and compare the security models of CVMFS and Frontier.

Dykstra, D.↗
DOE OSTI2020

Generalized hydrodynamic analysis of transport through a finite open nanopore for two-component single-file systems

Single-file diffusion (SFD) in finite open nanopores is characterized by nonzero spatially varying tracer diffusion coefficients within a generalized hydrodynamic description. This contrasts with infinite SFD systems where tracer diffusivity vanishes. In standard tracer counterpermeation (TCP) analysis, two reservoirs, each containing a different species, are connected to opposite ends of a finite pore. Here, we implement an extended TCP analysis to allow the two reservoirs to contain slightly different mixtures of the two species. Then, determination of diffusion fluxes through the pore allows extraction of diffusion coefficients for near-constant partial concentrations of the two species. This analysis is applied for a lattice-gas model describing two-component SFD through a finite linear pore represented by a one-dimensional array of cells. Two types of particles, A and B, can hop only to adjacent empty cells with generally different rates, $h_A$ and $h_B$. Particles are noninteracting other than exclusion of multiple cell occupancy. Results reveal generalized hydrodynamic tracer diffusion coefficients which adopt small values inversely proportional to pore length in the pore center, but which are strongly enhanced near pore openings.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

Tracer counterpermeation analysis of diffusivity in finite-length nanopores with and without single-file dynamics

Here, we perform a tracer counterpermeation (TCP) analysis for a stochastic model of diffusive transport through a narrow linear pore where passing of species within the pore is inhibited or even excluded (single-file diffusion). TCP involves differently labeled but otherwise identical particles from two decoupled infinite reservoirs adsorbing into opposite ends of the pore, and desorbing from either end. In addition to transient behavior, we assess steady-state concentration profiles, spatial correlations, particle number fluctuations, and diffusion fluxes through the pore. From the profiles and fluxes, we determine a generalized tracer diffusion coefficient D tr (x), at various positions x within the pore. D tr (x) has a plateau value in the pore center scaling inversely with the pore length, but it is enhanced near the pore openings. The latter feature reflects the effect of fluctuations in adsorption and desorption, and it is also associated with a nontrivial scaling of the concentration profiles near the pore openings.

36 MATERIALS SCIENCE↗
DOE OSTI2019

A year in the life of a parallel file system

I/O performance is a critical aspect of data-intensive scientific computing. We seek to advance the state of the practice in understanding and diagnosing I/O performance issues through investigation of a comprehensive I/O performance data set that captures a full year of production storage activity at two leadership-scale computing facilities. We demonstrate techniques to identify regions of interest, perform focused investigations of both long-term trends and transient anomalies, and uncover the contributing factors that lead to performance fluctuation. We find that a year in the life of a parallel file system is comprised of distinct regions of long-term performance variation in addition to short-term performance transients. We demonstrate how systematic identification of these performance regions, combined with comprehensive analysis, allows us to isolate the factors contributing to different performance maladies at different time scales. From this, we present specific lessons learned and important considerations for HPC storage practitioners.

Lockwood, GK↗
DOE OSTI2013

Use of treatment log files in spot scanning proton therapy as part of patient-specific quality assurance

Purpose: The purpose of this work was to assess the monitor unit (MU) values and position accuracy of spot scanning proton beams as recorded by the daily treatment logs of the treatment control system, and furthermore establish the feasibility of using the delivered spot positions and MU values to calculate and evaluate delivered doses to patients. Methods: To validate the accuracy of the recorded spot positions, the authors generated and executed a test treatment plan containing nine spot positions, to which the authors delivered ten MU each. The spot positions were measured with radiographic films and Matrixx 2D ion-chambers array placed at the isocenter plane and compared for displacements from the planned and recorded positions. Treatment logs for 14 patients were then used to determine the spot MU values and position accuracy of the scanning proton beam delivery system. Univariate analysis was used to detect any systematic error or large variation between patients, treatment dates, proton energies, gantry angles, and planned spot positions. The recorded patient spot positions and MU values were then used to replace the spot positions and MU values in the plan, and the treatment planning system was used to calculate the delivered doses to patients. The results were compared with the treatment plan. Results: Within a treatment session, spot positions were reproducible within {+-}0.2 mm. The spot positions measured by film agreed with the planned positions within {+-}1 mm and with the recorded positions within {+-}0.5 mm. The maximum day-to-day variation for any given spot position was within {+-}1 mm. For all 14 patients, with {approx}1 500 000 spots recorded, the total MU accuracy was within 0.1% of the planned MU values, the mean (x, y) spot displacement from the planned value was (-0.03 mm, -0.01 mm), the maximum (x, y) displacement was (1.68 mm, 2.27 mm), and the (x, y) standard deviation was (0.26 mm, 0.42 mm). The maximum dose difference between calculated dose to the patient based on the plan and recorded data was within 2%. Conclusions: The authors have shown that the treatment log file in a spot scanning proton beam delivery system is precise enough to serve as a quality assurance tool to monitor variation in spot position and MU value, as well as the delivered dose uncertainty from the treatment delivery system. The analysis tool developed here could be useful for assessing spot position uncertainty and thus dose uncertainty for any patient receiving spot scanning proton beam therapy.

60 APPLIED LIFE SCIENCES↗