Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “FILES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

ARM Data File Standards Version: 1.0

The Atmospheric Radiation Measurement (ARM) Climate Research Facility performs routine in situ and remote-sensing observations to provide a detailed and accurate description of the Earth atmosphere in diverse climate regimes. The result is a diverse data sets containing observational and derived data, currently accumulating at a rate of 30 TB of data and 150,000 different files per month (http://www.archive.arm.gov/stats/storage2.html). Continuing the current processing while scaling this to even larger sizes is extremely important to the ARM Facility and requires consistent metadata and data standards. The standards described in this document will enable development of automated analysis and discovery tools for the ever-growing volumes of data. It also will enable consistent analysis of the multiyear data, allow for development of automated monitoring and data health status tools, and facilitate development of future capabilities for delivering data on demand that can be tailored explicitly to user needs. This analysis ability will only be possible if the data follows a minimum set of standards. This document proposes a hierarchy that includes required and recommended standards.

Kehoe, Kenneth↗

Request queues for interactive clients in a shared file system of a parallel computing system

Interactive requests are processed from users of log-in nodes. A metadata server node is provided for use in a file system shared by one or more interactive nodes and one or more batch nodes. The interactive nodes comprise interactive clients to execute interactive tasks and the batch nodes execute batch jobs for one or more batch clients. The metadata server node comprises a virtual machine monitor; an interactive client proxy to store metadata requests from the interactive clients in an interactive client queue; a batch client proxy to store metadata requests from the batch clients in a batch client queue; and a metadata server to store the metadata requests from the interactive client queue and the batch client queue in a metadata queue based on an allocation of resources by the virtual machine monitor. The metadata requests can be prioritized, for example, based on one or more of a predefined policy and predefined rules.

97 MATHEMATICS AND COMPUTING↗

OpenPET Hardware, Firmware, Software, and Board Design Files

OpenPET is an open source, flexible, high-performance, and modular data acquisition system for a variety of applications. The OpenPET electronics are capable of reading analog voltage or current signals from a wide variety of sensors. The electronics boards make extensive use of field programmable gate arrays (FPGAs) to provide flexibility and scalability. Firmware and software for the FPGAs and computer are used to control and acquire data from the system. The command and control flow is similar to the data flow, however, the commands are initiated from the computer similar to a tree topology (i.e., from top-to-bottom). Each node in the tree discovers its parent and children, and all addresses are configured accordingly. A user (or a script) initiates a command from the computer. This command will be translated and encoded to the corresponding child (e.g., SB, MB, DB, etc.). Consecutively, each node will pass the command to its corresponding child(ren) by looking at the destination address. Finally, once the command reaches its desired destination(s) the corresponding node(s) execute(s) the command and send(s) a reply, if required. All the firmware, software, and the electronics board design files are distributed through the OpenPET website (http://openpet.lbl.gov).

Abu-Nimeh, Faisal↗

Parallel compression of data chunks of a shared data object using a log-structured file system

Techniques are provided for parallel compression of data chunks being written to a shared object. A client executing on a compute node or a burst buffer node in a parallel computing system stores a data chunk generated by the parallel computing system to a shared data object on a storage node by compressing the data chunk; and providing the data compressed data chunk to the storage node that stores the shared object. The client and storage node may employ Log-Structured File techniques. The compressed data chunk can be de-compressed by the client when the data chunk is read. A storage node stores a data chunk as part of a shared object by receiving a compressed version of the data chunk from a compute node; and storing the compressed version of the data chunk to the shared data object on the storage node.

97 MATHEMATICS AND COMPUTING↗

Multiple Independent File Parallel I/O with HDF5

The HDF5 library has supported the I/O requirements of HPC codes at Lawrence Livermore National Labs (LLNL) since the late 90’s. In particular, HDF5 used in the Multiple Independent File (MIF) parallel I/O paradigm has supported LLNL code’s scalable I/O requirements and has recently been gainfully used at scales as large as O(10 6 ) parallel tasks.

97 MATHEMATICS AND COMPUTING↗

UNITY: Unified Memory and File Space

This paper describes the vision for UNITY, a new high-performance computing focused data storage abstraction that places the entire memory hierarchy, including both traditionally separated memoryand file-based data storage, into one storage continuum. Through the use of a novel API and a set of services centered around a smart runtime system, UNITY is able to provide a number of valuable and interesting benefits. The unified storage space provides a scalable and resilient data environment that dynamically manages the mapping of data onto available resources based on multiple factors, including desired persistence and energy budget considerations. By eliminating the need for high-performance computing domain scientists to develop architecture-dependent optimizations for rapidly evolving data storage technologies, UNITY addresses both ease-of-use and performance.

Otstott, Douglas↗

System and method of managing large data files

Disclosed are systems and software that provide a high-performance, extensible file format and web API for remote data access and a visual interface for data viewing, query, and analysis. The described system can support storage of raw spectroscopic data such as neural recording data, MSI data, metadata, and derived analyses in a single, self-describing format that may be compatible by a large range of analysis software.

Bowen, Benjamin P.↗

The Rucio File Catalog in DIRAC implemented for Belle II

DIRAC and Rucio are two standard pieces of software widely used in the HEP domain. DIRAC provides Workload and Data Management function- alities, among other things, while Rucio is a dedicated, advanced Distributed Data Management system. Many communities that already use DIRAC have expressed their interest in using DIRAC for Workload Management in combi- nation with Rucio for Data Management. In this paper, we describe the integra- tion of the Rucio File Catalog into DIRAC that was initially developed for the Belle II collaboration.

97 MATHEMATICS AND COMPUTING↗

Quota management in dCache or making a perfectly normal file system normal

dCache (https://dcache.org) is a highly scalable storage system providing location-independent access to data. The data are stored across multiple data servers as complete files presented to the end-user via a single-rooted namespace. From its inception, dCache has been designed as a caching disk buffer to a tertiary tape storage system with the assumption that the latter has virtually unlimited capacity. dCache can also be configured as a disk-only storage system with no tape backend. Owing to the idea that a tape resource is infinite, or purely physically limited by budget considerations, the system has never provided for any restrictions on how much data can be stored on tape. Likewise, in the disk-only configuration, the capacity of the system is only limited by the aggregate disk capacity of the data servers. In a multi-user environment, however, this has become problematic. This presentation will describe the design and implementation of a user- and group-based quota system, that allows to manage tape and disk space allocations, as part of dCache namespace.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Key Information File for Radioactive Waste Repositories - Preliminary Tests - 16169

The Key Information File (KIF) is a document designed to provide a summary of the existence, location and content of an engineered facility for the permanent disposal of radioactive wastes. It should be recognized as part of an internationally integrated system of records and memory. Its primary function would be to maintain awareness of the site, and long-term confidence in the effectiveness of the disposal system, so that the likelihood of unnecessary human disturbance is minimized. In order to ensure this document is accessible to a large audience, it would need to be limited in size. A draft format for the KIF has been proposed, and testing is being conducted on two cases, that of the 'Centre de stockage de la Manche', the closed French Repository (surface) and that of the planned Swedish Spent Fuel Repository (geological). Preliminary results show the potential for such a document. Comments and questions raised by the 'Records, Knowledge and Memory preservation' (RK and M) initiative, coordinated by the Nuclear Energy Agency (NEA) of OECD, are presented. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

New approaches to provide feedback from nuclear and covariance data adjustment for effective improvement of evaluated nuclear data files

A critical examination of the role of uncertainty assessment, target accuracies, role of integral experiment for validation and, consequently, of data adjustments methods is underway since several years at OECD-NEA, the objective being to provide criteria and practical approaches to use effectively the results of sensitivity analyses and cross section adjustments for feedback to evaluators and experimentalists in order to improve without ambiguities the knowledge of neutron cross sections, uncertainties, and correlations to be used in a wide range of applications and to meet new requirements and constraints for innovative reactor and fuel cycle system design. An approach will be described that expands as much as possible the use in the adjustment procedure of selected integral experiments that provide information on “elementary” phenomena, on separated individual physics effects related to specific isotopes or on specific energy ranges. An application to a large experimental data base has been performed and the results are discussed in the perspective of new evaluation projects like the CIELO initiative.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Collection of Disk Failure Events from Alpine, the Parallel File System for Summit Supercomputer

This dataset contains disk (HDD) failure events collected from the Alpine storage system of the Summit supercomputer, hosted at OLCF, spanning from January 4, 2019, to December 21, 2023 (a total of 4 years, 11 months, and 18 days), covering 89% of its operational lifetime. It includes 3,766 disk failure events, each recorded with its detection timestamp (in ISO 8601 format) and detailed by its location within the storage system - rack, enclosure, and drive slot number.

97 MATHEMATICS AND COMPUTING↗

SESAME: ASCII2 File Format

A new ASCII format for SESAME data is explicitly defined and dubbed ASCII2. This format fixes some of the onerous limitations of the legacy ASCII-styled format in addition to relaxing the FORTRAN style fixed-format layout of data tables. With the exception of the multi-tiered indexes of the legacy binary format, the new ASCII2 is more compatible with the binary in that it does not restrict the extent of various integer data (e.g., SESAME material and table numbers) to six digits nor does it restrict the extent or precision of the various floating point data. There will be very limited support of the legacy ASCII-styled formats in the future.

36 MATERIALS SCIENCE↗