Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “File Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

TaZr_BCC_SolidSolution_128atoms_VASP6

We performed density functional theory (DFT) calculations for body-centered-cubic (BCC) structures with 128 lattices sites of solid solution binary alloys tantalum-zirconium (Ta-Zr). The electronic structures of alloys have been calculated using Vienna Ab initio Simulation Package (VASP). Within this package the DFT approach is used to reduce many-body Schrodinger equation to set of single particle Kohn-Sham (KS) equations. The generalized electronic exchange-correlation functional is described by generalized gradient approximation with the Perdew-Burke-Ernzerhof parametrization. The electron-ion interactions is described by pseudopotentials developed within the plane-wave basis projector augmented-wave (PAW) approach \cite{PAW}. These pseudopotentials are available at the VASP portal (http://cms.mpi.univie.ac.at/vasp/). Our calculations have been run with the pseudopotentials treating s and p semi-core states as valence in case for the elements Ta and Zr. The electronic densities and potentials are expanded over plane-waves with energy cutoff of 350 eV. 2x2x2 k-mesh and normal precision were used. The alloys were modeled by supercell containing 128 randomly distributed atoms. At initial step the atoms occupy perfect bcc lattice cites. This initial structure was optimized until energy changes less than 1e-6 eV, while forces acting on atoms don't exceed 1e-2 eV/angstrom. The electron-ion interaction is described by PAW pseudopotentials. The calculations have been collected by sampling chemical compositions across the entire compositional range. The chemical compositions have been sampled by progressively changing the number of atoms per constituent by 4. For each chemical composition of binaries and ternaries, the first-principle calculations have been run for 100 randomized arrangements of the constituents on the BCC lattice sites. We collected data for a total of 3,100 randomized atomic structures over 31 chemical compositions. The calculations have been collected on NERSC-Perlmutter and OLCF-Summit using the VASP 6.3.2. The VASP calculations for every atomic structure have been performed in 2 main steps: 1. Starting from an ideal body-centered-cubic (BCC) structure, geometry optimization with low precision has been executed to perform a preliminary optimization of the atomic structure. The output for this calculations is available in the files 0.CONTCAR, 0.OUTCAR, rlx1.out. 2. Using the atomic structure resulting from the preliminary geometry optimization, a second geometry optimization has been performed using normal precision. The output for this calculations is available in the files CONTCAR, OUTCAR, rlx2.out, vaspout.h5, and vasprun.xml. Cases 1-10 have been run without generating the file 'vaspout.h5'. Every chemical composition sampled across the composition range in the dataset has its own directory. The convention used to name the directories for binary alloys is AXBY, where A and B refer to the constituents, whereas X and Y are positive integers that represent the number of atoms for each constituent and their values still sum up to 128. Each atomic structure associated with a specific chemical composition has its own sub-directory within the directory of the corresponding chemical composition. The sub-directories for each atomic structure for each chemical composition are named 'case-*', where * is a positive integer that spans all the values from 1 through 100, extremes included. The files contained in each sub-directory 'case-*' for each atomic structure are as follows: FILES contained in each subdirectory with name "case-N" where N ranges between 11 and 80, extremes included: 1. INCAR: input file that contains various parameters and settings for controlling the behavior of the electronic structure calculations 2. KPOINTS: input file that specifies the Bloch vectors (k points) used to sample the Brillouin zone 3. 0.POSCAR: input file that defines the atomic structure of a system 4. 0.CONTCAR: output file that provides the atomic positions and cell parameters after the first geometry optimization has been run with the precision variable set to PREC=Low in the INCAR file 5. 0.OUTCAR: output file that contains detailed information about the progress of a calculation after the first geometry optimization has been run with the precision variable set to PREC=Low in the INCAR file 6. rlx1.out: file with diagnostic information about the execution of the first geometry optimization with precision variable set to PREC=Low in the INCAR file 7. POSCAR: input file that defines the atomic structure of a system after the first geometry optimization has been run at low precision. This represents the input for the second geometry optimization run with the precision variable set to PREC=Normal in the INCAR file 8. CONTCAR: output file that provides the atomic positions and cell parameters after the second geometry optimization has been run with the precision variable set to PREC=Normal in the INCAR file 9. OUTCAR: output file that contains detailed information about the progress of a calculation after the second geometry optimization has been run with the precision variable set to PREC=Normal in the INCAR file 10. rlx2.out: file with diagnostic information about the execution of the second geometry optimization with precision variable set to PREC=Normal in the INCAR file 11. vaspout.h5: hierarchical HDF5 file containing the inputs and outputs of a VASP calculation. To analyze the data in this file we recommend using py4vasp. This file is only produced if the VASP version used is compiled with HDF5 support 12. vasprun.xml: contains similar information to OUTCAR, but in an xml format. 13. CHGCAR: contains the charge density data of a system. This data is crucial for analyzing electronic structures, calculating electrostatic potential, and studying the distribution of charge in a crystal or molecular system FILES contained in each subdirectory with name "case-N" where N ranges between 1 and 10 and between 81 and 100, extremes included: 1. INCAR: input file that contains various parameters and settings for controlling the behavior of the electronic structure calculations 2. KPOINTS: input file that specifies the Bloch vectors (k points) used to sample the Brillouin zone 3. {ID}.POSCAR: input file that defines the atomic structure of a system at the beginning of ID execution of geometry optimization with PREC=LOW 4. {ID}.CONTCAR: output file that provides the atomic positions and cell parameters at the end of ID execution of geometry optimization with PREC=LOW in the INCAR file 5. {ID}.OUTCAR: output file that contains detailed information about the progress of a calculation after the ID execution of geometry optimization that has been run with the precision variable set to PREC=Low in the INCAR file 6. rlx1.{ID}.out: file with diagnostic information about the execution of the ID execution of the geometry optimization with precision variable set to PREC=Low in the INCAR file 7. N{ID}.POSCAR: input file that defines the atomic structure of a system after the geometry optimization run at low precision. This represents the input for the ID execution of the second geometry optimization run with the precision variable set to PREC=Normal in the INCAR file 8. N{ID}.CONTCAR: output file that provides the atomic positions and cell parameters after the ID execution of the geometry optimization run with the precision variable set to PREC=Normal in the INCAR file 9. N{ID}.OUTCAR: output file that contains detailed information about the progress of a calculation after the ID execution of the geometry optimization run with the precision variable set to PREC=Normal in the INCAR file 10. rlx2.{ID}.out: file with diagnostic information about the ID execution of geometry optimization with precision variable set to PREC=Normal in the INCAR file 11. vaspout.h5: hierarchical HDF5 file containing the inputs and outputs of a VASP calculation. To analyze the data in this file we recommend using py4vasp. This file is only produced if the VASP version used is compiled with HDF5 support 12. vasprun.xml: contains similar information to OUTCAR, but in an xml format. 13. CHGCAR: contains the charge density data of a system. This data is crucial for analyzing electronic structures, calculating electrostatic potential, and studying the distribution of charge in a crystal or molecular system This research is sponsored by the Artificial Intelligence Initiative as part of the Laboratory Directed Research and Development (LDRD) Program of Oak Ridge National Laboratory, managed by UT-Battelle, LLC, for the US Department of Energy under contract DE-AC05-00OR22725. This work used resources of the Oak Ridge Leadership Computing Facility, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725, under Directorate Discretionary awards MAT025 (Materials Science) and LRN026 (Machine Learning), and INCITE award MAT201. This work also used resources of the National Energy Research Scientific Computing Center, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231, under award ERCAP0025216. REFERENCES (1) Kresse, G. & Hafner, J. Ab initio molecular dynamics for liquid metals. Phys. review B 47, 558 (1993). (2) Kresse, G. & Hafner, J. Ab initio molecular-dynamics simulation of the liquid-metal–amorphous-semiconductor transition in germanium. Phys. Rev. B 49, 14251 (1994) (3) Kresse, G. & Furthmüller, J. Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set. Comput. materials science 6, 15–50 (1996) (4) Kresse, G. & Furthmüller, J. Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set. Phys. review B 54, 11169 (1996) (5) Kresse, G. & Joubert, D. From ultrasoft pseudopotentials to the projector augmented-wave method. Phys. review b 59, 1758 (1999)

36 MATERIALS SCIENCE↗
DOE OSTI2021

Malware analysis and recovery

A system and method detects malware by processing notifications from an intrusion detection system and baseline snapshots from an image capture utility. The image capture utility constructs an image of the suspected malware intrusion and links the suspected malware intrusion to the baseline snapshots. The system and method propagates the image of the suspected malware intrusion across multiple networks before it distinguishes malicious code, device state, and files from benign code, device state, and files. Some systems and methods include a malware recovery system that executes machine learning instructions and heuristics to revert a client and/or a remote server to one or more baseline snapshots.

Smith, Jared M.↗
DOE OSTI2023

Isolated data processing modules

This disclosure relates to computer systems that isolate data processing modules and methods of operating the same. In one embodiment of a method, a computer system receives a first data processing request for a first data file. The computer system may determine that a first data type of the first data file is supported by an encapsulated data processing module and determine a first communication protocol from a set of communication protocols that can be used to receive the first data file. The computer system can then receive the first data file from the first storage location in accordance with the first communication protocol and the encapsulated data processing module may be executed to convert the first data file in the first data format into second data in a second data format.

Penton, Joshua Michael↗
DOE OSTI2015

Cloud object store for archive storage of high performance computing data using decoupling middleware

Cloud object storage is enabled for archived data, such as checkpoints and results, of high performance computing applications using a middleware process. A plurality of archived files, such as checkpoint files and results, generated by a plurality of processes in a parallel computing system are stored by obtaining the plurality of archived files from the parallel computing system; converting the plurality of archived files to objects using a log structured file system middleware process; and providing the objects for storage in a cloud object storage system. The plurality of processes may run, for example, on a plurality of compute nodes. The log structured file system middleware process may be embodied, for example, as a Parallel Log-Structured File System (PLFS). The log structured file system middleware process optionally executes on a burst buffer node.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Regen: An object layout regenerator on large-scale production HPC systems

This article proposes an object layout regenerator called Regen which regenerates and removes the object layout dynamically to improve the read performance of applications. Regen first detects frequent access patterns from the I/O requests of the applications. Second, Regen reorganizes the objects and regenerates or preallocates new object layouts according to the identified access patterns. Finally, Regen removes or reuses the obsolete or regenerated object layouts as necessary. As a result, Regen accelerates access to objects by providing a flexible object layout. We implement Regen as a framework on top of Proactive Data Container (PDC) and evaluate it on Cori supercomputer, a production-scale HPC system, by using realistic HPC I/O benchmarks. The experimental results show that Regen improves the I/O performance by up to 16.92 × compared with an existing system.

Distributed file system↗
DOE OSTI2014

Performance model-directed data sieving for high-performance I/O

Many scientific computing applications and engineering simulations exhibit noncontiguous I/O access patterns. Data sieving is an important technique to improve the performance of noncontiguous I/O accesses by combining small and noncontiguous requests into a large and contiguous request. It has been proven effective even though more data are potentially accessed than demanded. In this study, we propose a new data sieving approach namely performance model-directed data sieving, or PMD data sieving in short. It improves the existing data sieving approach from two aspects: (1) dynamically determines when it is beneficial to perform data sieving; and (2) dynamically determines how to perform data sieving if beneficial. It improves the performance of the existing data sieving approach considerably and reduces the memory consumption as verified by both theoretical analysis and experimental results. Given the importance of supporting noncontiguous accesses effectively and reducing the memory pressure in a large-scale system, the proposed PMD data sieving approach in this research holds a great promise and will have an impact on high-performance I/O systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

I/O load balancing for big data HPC applications

High Performance Computing (HPC) big data problems require efficient distributed storage systems. However, at scale, such storage systems often experience load imbalance and resource contention due to two factors: the bursty nature of scientific application I/O; and the complex I/O path that is without centralized arbitration and control. For example, the extant Lustre parallel file system-that supports many HPC centers-comprises numerous components connected via custom network topologies, and serves varying demands of a large number of users and applications. Consequently, some storage servers can be more loaded than others, which creates bottlenecks and reduces overall application I/O performance. Existing solutions typically focus on per application load balancing, and thus are not as effective given their lack of a global view of the system. In this paper, we propose a data-driven approach to load balance the I/O servers at scale, targeted at Lustre deployments. To this end, we design a global mapper on Lustre Metadata Server, which gathers runtime statistics from key storage components on the I/O path, and applies Markov chain modeling and a minimum-cost maximum-flow algorithm to decide where data should be placed. Evaluation using a realistic system simulator and a real setup shows that our approach yields better load balancing, which in turn can improve end-to-end performance.

Paul, Arnab K.↗
DOE OSTI2020

ESGF Future Architecture Report (V.1.1)

This document reports the findings of the Earth System Grid Federation Future Architecture Face-to-Face Meeting held in the UK 5-7 November 2019, attended by technical representatives from the ESGF partner organisations. ESGF is an International federated archive that provides access to Earth science data. It is fundamentally a distributed file management system with its main function being data dissemination through the provision of services for efficient replication and search. It has been in operation for over ten years and has undergone continuous improvements and enhancements over this time span. The Future Architecture is a major review of the software system including a fundamental reassessment of its requirements with a view to develop a new overall system architecture. This has been initiated in the light of challenges around the maintenance of the existing software legacy and a changing technology landscape and practices in the communities it serves. The work is being carried out under the oversight of the ESGF Executive Committee1,2 and is coordinated activity between ESGF and the European IS-ENES3 project.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Generalizable Web User Interface for Scalable and Streamlined Deployment of Building Energy Management Systems in Small and Medium-Sized Commercial Buildings

Small and medium-sized commercial buildings (SMCBs) comprise 94% of US commercial buildings yet face significant barriers to implementing building energy management systems despite advances in smart device technology. Existing solutions present critical limitations: cloud-based API solutions simplify deployment but create vendor lock-in constraints; commercial integrated software solutions ensure compatibility via standardized protocols but require substantial cost and technical expertise; open-source IoT platforms offer cost-effective vendor independence but provide insufficient standardized protocol support for commercial building automation. This research presents a generalizable web user interface framework that bridges the gap between evolving smart device capabilities and lagging software infrastructure for SMCBs. The proposed system integrates VOLTTRON open-source middleware with an automated configuration converter that transforms unified specifications written in YAML, a human-readable data-serialization format, into system-specific files, streamlining manual setup processes. The vendor-agnostic architecture supports industry-standard protocols (BACnet and Modbus) and semantic building models while providing adaptive web interfaces that dynamically adjust to various building configurations. Demonstrations through simulation-based testing and a field deployment show automatic interface adaptation across heterogeneous HVAC systems and multizone monitoring. The automated configuration converter also substantially reduces labor-intensive setup.

Chung, Jihoon [ORNL] (ORCID:0000000184880815)↗
DOE OSTI2025

DEDUPKV: A Space-Efficient and High-Performance Key-Value Store via Fine-Grained Deduplication

Log-Structured Merge Tree (LSM-tree) based key-value stores excel in write-intensive environments but suffer from data duplication, consuming up to 49% of storage space in LSM-tree-based key-value store deployments. Traditional solutions like compression and coarse-grained file system-level deduplication introduce overhead or have limited effectiveness. In this study, we propose DedupKV, a fine-grained deduplication framework tailored for LSM-tree, maximizing data reduction efficiency while minimizing write stalls and read overheads. DedupKV features three key innovations: (1) FLUSH-integrated inline deduplication, which removes duplicates during memory-to-storage writes; (2) WAL file-based offline deduplication, repurposing write-ahead logs to avoid double writes; and (3) elastic execution, dynamically balancing inline and offline deduplication based on memory pressure and workload intensity. Additionally, dynamic granularity management reduces deduplication metadata overhead. We implemented these four ideas in RocksDB for the first time and conducted experiments in a Linux environment. Our evaluation shows that WAL file-based offline deduplication and DedupKV outperform BlobDB by 33% and 23%, respectively, in write-heavy workloads, while reducing write amplification by 1.2 ×, 2 ×, and 1.6 × for real KV datasets.

Jamil, Safdar [Sogang University]↗
DOE OSTI2015

LADS: Optimizing Data Transfers using Layout-Aware Data Scheduling

While future terabit networks hold the promise of signifi- cantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today s 100 gigabit networks to real- ize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink. Data stor- age infrastructure at both the source and sink and its in- terplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this paper, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network en- vironment, and we present a new bulk data movement framework called LADS for terabit networks. LADS ex- ploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to use zero-copy, OS-bypass hardware when available. It can further im- prove data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared stor- age resource, improving I/O bandwidth, and data transfer rates across the high speed networks.

Data Management↗
DOE OSTI2016

Data Movement in Data-Intensive High Performance Computing.

The cost of executing a floating point operation has been decreasing for decades at a much higher rate than that of moving data. Bandwidth and latency, two key metrics that determine the cost of moving data, have degraded significantly relative to processor cycle time and execution rate. Despite the limitation of sub-micron processor technology and the end of Dennard scaling, this trend will continue in the short-term making data movement a performance-limiting factor and an energy/power efficiency concern. Even more so in the context of large-scale and data-intensive systems and workloads. This chapter gives an overview of the aspects of moving data across a system, from the storage system to the computing system down to the node and processor level, with case study and contributions from researchers at the San Diego Supercomputer Center, the Oak Ridge National Laboratory, the Pacific Northwest National Laboratory, and the University of Delaware.

Data Movement↗
DOE OSTI2023

Uncovering I/O demands on HPC platforms: Peeking under the hood of Santos Dumont

High-Performance Computing (HPC) platforms are required to solve the most diverse large-scale scientific problems in various research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific softwares, which have different requirements. These include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle mixed workload when storing data from the applications. Understanding the set of applications and their performance running in a supercomputer is paramount to understanding the storage system's usage, pinpointing possible bottlenecks, and guiding optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, identifying inefficient usage, problematic performance factors, and providing guidelines on how to tackle those issues.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Disk Failure Dataset from the Campaign Storage System

This dataset consists of 1,389 disk (HDD) failure events collected from the Campaign storage system at LANL. The Campaign system supported various compute platforms throughout its lifespan, including Cielo, Fire, Ice, and notably, the Trinity supercomputer. Each recorded event includes its detection timestamp (in ISO 8601 format) and details such as its location within the storage system—rack, enclosure, and drive slot number. The data, spanning from May 4, 2021, to July 25, 2023 (2 years, 2 months, and 22 days), represents failure events from the terminal years of Campaign's operational period, accounting for 26% of its total operational time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data for Bistline, et al. (2025) "Policy Implications of Net-Zero Emissions: A Multi-Model Analysis of United States Emissions and Energy System Impacts"

These files contain input assumptions, results, and figures associated with the Bistline, et al. (2025) article "Policy Implications of Net-Zero Emissions: A Multi-Model Analysis of United States Emissions and Energy System Impacts" in Energy and Climate Change as part of the Energy Modeling Forum 37 study. Please refer to the original paper for details.

climate policy↗