SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The Python code takes EnergyPlus input files and creates input for the View3D program, executes the View3D program, then inserts the respective view factors into the EnergyPlus input file, thus streamlining the process of view factor calculation and modification of the EnergyPlus input file.
PDQ is a tool for the management of the input and execution of batch jobs for simulation codes that use a text based input system. It accomplishes this goal by operating at two levels. First, it takes input file templates (commonly known at LANL as input deck templates) and creates multiple instantiations by performing substitutions of data from table files into symbols (variables) found in the template. Second, it provides commands to submit the created files to the SLURM batch system for execution. These two activities taken together produce a whole that is greater than the sum of its parts and provides an elegant way of executing studies across multiple similar simulations while minimizing the risk of typographical errors in the input files.
PDQ is a tool for the management of the input and execution of batch jobs for simulation codes that use a text based input system. It accomplishes this goal by operating at two levels. First, it takes input file templates (commonly known at LANL as input deck templates) and creates multiple instantiations by performing substitutions of data from table files into symbols (variables) found in the template. Second, it provides commands to submit the created files to the SLURM batch system for execution. These two activities taken together produce a whole that is greater than the sum of its parts and provides an elegant way of executing studies across multiple similar simulations while minimizing the risk of typographical errors in the input files. PDQ was originally developed as a job management system called XVS by Jeff McAninch while he was at LANL. Besides the capabilities described here, XVS had many other features specific for interactions with particular simulation codes. After Jeff’s departure, maintenance of XVS was taken over by Rendell Carver; he added some new features as well as kept it functioning as the batch system at LANL was changed from LSF to MOAB to SLURM. In 2017, Rob Pelak decided to develop a different version that removed the additional features (many of which were rendered obsolete with the retirement of the simulation code or batch system that they supported) and produced a cleaner “bare bones” version of XVS. A few other behaviors of XVS that Rob found irksome were altered. Rob gave the resulting code a new name: PDQ. In 2022 Danielle McDermott developed a version that runs under Python 3.X. As suggested by Rob, she used the python2to3 utility to identify most changes. Given that PDQ continues to operate with Python version 2.7 we have advanced the version number to 1.20.
This data package is associated with the publication “Allometric scaling of hyporheic respiration across basins in the Pacific Northwest USA” submitted to JGR-Biogeosciences (Regier et al. 2025).This study used reach-scale modeled estimates of hyporheic aerobic respiration made by the River Corridor Model (Fang et al. 2020) and watershed characteristics across the Willamette and Yakima River basins to explore potential allometric scaling (i.e., power-law relationships between size and function) of cumulative hyporheic respiration across catchment-to-basin scales. Scaling was explored quantitatively via the R2, slope, and y-intercept of relationships between cumulative hyporheic respiration and watershed area, divided into hyporheic exchange flux (HEF) quantiles. We also explored relationships between allometric scaling and other watershed characteristics through linear regression, spatial patterns, and mutual information analyses. Our results also suggest variability of hyporheic respiration allometry for middle exchange flux quantiles, and in relation to land-cover. Our findings provide initial evidence that allometric scaling may be useful for predicting hyporheic biogeochemical dynamics across watersheds from reach to basin scales. This data package is associated with the GitHub repository found at https://github.com/peterregier/rc_wrb_yrb_scaling. The data package is organized into several key directories. The “data” folder contains multiple CSV files, including landscape heterogeneity, scaling analysis, and watershed boundary data. The “figures” folder has all figure files in both PDF and PNG formats. Core analysis scripts and figure generation scripts are in the “scripts” directory, systematically numbered for sequential execution. The root directory includes essential project files; please see the file ending in “flmd.csv” for a list and description of all files contained in this data package and the file ending in “dd.csv” for data dictionaries used to describe tabular column headers.
Scientific workflows drive most modern large-scale science breakthroughs by allowing scientists to define their computations as a set of jobs executed in a given order based on their data dependencies. Workflow management systems (WMSs) have become key to automating scientific workflows-executing computational jobs and orchestrating data transfers between those jobs running on complex high-performance computing (HPC) platforms. Traditionally, WMSs use files to communicate between jobs: a job writes out files that are read by other jobs. However, HPC machines face a growing gap between their storage and compute capabilities. To address that concern, the scientific community has adopted a new approach called in situ, which bypasses costly parallel filesystem I/O operations with faster in-memory or in-network communications. When using in situ approaches, communication and computations can be interleaved. In this work, we leverage the Decaf in situ dataflow framework to accelerate task-based scientific workflows managed by the Pegasus WMS, by replacing file communications with faster MPI messaging. We propose a new execution engine that uses Decaf to manage communications within a sub-workflow (i.e., set of jobs) to optimize inter-job communications. We consider two workflows in this study: (i) a synthetic workflow that benchmarks and compares file- and MPI-based communication; and (ii) a realistic bioinformatics workflow that computes mu-tational overlaps in the human genome. Experiments show that in situ communication can improve the bioinformatics workflow execution time by 22% to 30% compared with file communication. Our results motivate further opportunities and challenges for bridging traditional WMSs with in situ frameworks.
High Energy Physics (HEP) experiments entail an abundance of computing resources, i.e. sites, to run simulations and analyses by processing data. This requirement is fulfilled by local batch farms, grid sites, private/commercial clouds, and supercomputing centers via High Throughput Computing (HTC). The growing needs of such experiments and resources being prone to trends of heterogeneity make it difficult for physicists to handle these resources directly. Additionally, HEP collaborations heavily rely on data and software releases, typically in the order of tens of gigabytes, while conducting simulations and analyses. Hence, aspects of scalability, reliability, and maintenance become crucial with regards to the distribution of the necessary data and software stack. The GlideinWMS [4] framework helps with the resource management problem by using pilot jobs, aka Glideins, to provision reliable elastic virtual clusters. Glideins are submitted to unreliable heterogeneous resources which are validated and customized by the Glideins to make the worker nodes available for end-user job execution. On the other hand, the CernVM File System (CernVM-FS or CVMFS) [1] helps with data distribution. It is a write-once, read-everywhere filesystem used to deploy scientific software to thousands of nodes on a worldwide distributed computing infrastructure. CVMFS is based on the Hyper Text Transfer Protocol and has been widely used within the particle physics community for (1) distributing experiment software and data such as calibrations, and (2) facilitating containerization by efficiently hosting container images along with providing containerization software, especially Singularity [3] GlideinWMS relies on CVMFS installed locally on the computing resources to satisfy the experiments' software needs. This requires system administrators' effort to install and maintain CVMFS at the sites and limits the use of sites, especially HPC resources, that do not have CVMFS installed. This poster presents a solution, taking advantage of Glideins to provide CVMFS at most sites without the need for a local installation. Doing so expands the pool of resources available for HEP experiments and reduces the effort of system administrators for current resources. Additionally, the proposed solution allows GlideinWMS to also start Singularity [3], a containerization software that can run unprivileged, on sites where neither CVMFS nor Singularity are available, including HPC sites. The benefits provided by this solution are: (1) lower overhead for site administrators in that they have less software to install, (2) an expanded pool of resources that run user jobs with easy access to software and data provided by CVMFS, thus making life easier for the scientists, and (3) improved flexibility to use HPC resources by enabling GlideinWMS pilot jobs to support HPC sites.
This dataset contains files storing results from classically-simulated quantum subroutines within a dynamical mean-field theory workflow, and jupyter notebooks processing the data in these files to generate plots. The files store: (1) Results from variational quantum eigensolver (VQE) simulations searching for optimal parameters allowing parametrized quantum circuits to prepare approximations to ground states of different Anderson impurity models (AIMs) (2) Results from simulations of a quantum Lanczos algorithm (QLA) estimating the Lanczos coefficients defining the continued-fraction representation of an (AIM) Green’s function Description: Any file named vqe_gs_results* stores approximations to the ground state and energy of a given AIM estimated using three different methods: (1) Numerical diagonalization (2) Ideal VQE simulation (3) VQE simulation with sampling noise For each VQE simulations metadata about the optimization (optimization results plus number of quantum circuits that would have been executed on real hardware) is also stored. Any file named qla_dos_results* estimations for the Lanczos coefficients defining the Green’s function of an AIM. The stored estimations are achieved using different methods: (1) Numerical Lanczos algorithm from initial states obtained from numerical diagonalization (2) Simulated quantum Lanczos algorithm from initial states prepared from parametrized quantum circuits yielded by corresponding ideal and noisy VQE subroutines. The dataset is used and described in M. Karabin et al., "Quantum solver for single-impurity Anderson models with particle-hole symmetry", Phys. Rev. Research 8, 033066 (2026). DOI: https://doi.org/10.1103/7ys3-tl4l
PoliMOR is a scalable, automated, and customizable policy engine framework for multi-tiered parallel file systems. It is composed of single-purpose agents that handle tasks such as gathering file metadata, making policy decisions, and then executing actions based on those policies. These agents are designed to communicate using distributed message queues, allowing the number of individual agents to be scaled up as needed. PoliMOR automates the data management tasks by precluding the need for admin intervention. The agents in PoliMOR can be customized to integrate any utilities/tools that perform tasks like metadata scanning and data placement management.
The proposed code, named FROG (Fusion neutron Generator) is built upon the open-source particle transport Monte Carlo toolkit Geant4. Geant4 provides C++ classes that can be leveraged to build application-specific codes dealing with the transport of particles through matter. Geant4-based codes are applied in high-energy particle physics experiments, medical applications, shielding, and space applications for example. The FROG code allows the user to define the geometry of a neutron converter device shaped as a hollow cylinder, where a neutron breeding material such as lithium deuteride (LiD) is cladded by two concentric cylinders. Such neutron converter is then placed inside a regular nuclear fission reactor, where thermal neutrons will react with the neutron breeder material (typically, Lithium 6), and through a series of reactions, will generate high-energy neutrons – neutrons whose kinetic energy are around 14 MeV. The hollowed central portion can hold a specimen that will be bombarded by high-energy neutrons created inside the neutron breeding material. Figuratively speaking, this type of device transforms neutrons from thermal (~0.625 eV) to fusion (~14 MeV) energies and is sometimes termed “fusion-to-thermal neutron converters” in the literature. The code consists of C++ source file compiled and linked to generate an executable. The user can select the dimensions of the converter (radius, length, and thickness of the breeder material), the breeder material type, the cladding material, and the specimen material that will be activated or irradiated. As input, the neutron flux for a specific location inside a reactor, for instance, positions in ATR, is required. As output, the code predicts the number of high-energy neutrons produced, the total neutron flux and fluence as well as its detailed spectrum. The physics involved in such device is very complex, as it requires modeling neutron transport, light-ion (tritons) transport, as well as fusion reactions. The Geant4 toolkit provides the required physical models.
The objective is to evaluate the capabilities of multiple CSDs (provided by NDG Systems) using Hadoop Filesystem and Apache Spark. The independent variables are: number of CSDs, 0, 1, 2, 4, or 6; size of dataset, 1 GB, 5 GB, 10 GB; type of dataset, one large file with all of the data, 10 files, 100 files. The dependent variables are: job time; execution time. the constants are operations on the dataset.
The DIF3D-VARIANT solver is part of the DIF3D code which was developed at ANL for engineering level predictive analysis of nuclear reactor cores. Over the course of the last 20 years, the DIF3D-VARIANT option of DIF3D has become increasingly more used because of improvements in computer memory capabilities and performance. This is beneficial to the users of DIF3D as DIF3D-VARIANT is able to apply both p- and h- spatial mesh refinement strategies in diffusion and transport theory. Given its greater use, utility programs were created to assist in post-processing the DIF3D output. This document discusses the basic execution process of DIF3D and the output files it creates but is primarily focused on being the manual for the utility programs EvaluateFlux, VARPOW, VARPEAK, and DIF3DtoVTK.
The DIF3D-VARIANT solver is part of the DIF3D code which was developed at ANL for engineering level predictive analysis of nuclear reactor cores. Over the course of the last 20 years, the DIF3D-VARIANT option of DIF3D has become increasingly more used because of improvements in computer memory capabilities and performance. This is beneficial to the users of DIF3D as DIF3D-VARIANT is able to apply both p- and h-spatial mesh refinement strategies in diffusion and transport theory. Given its greater use, utility programs were created to assist in post-processing the DIF3D output. This document discusses the basic execution process of DIF3D and the output files it creates but is primarily focused on being the manual for the utility programs EvaluateFlux, VARPOW, VARPEAK, and DIF3DtoVTK.
Abstract We present a database resulting from high throughput experimentation, primarily on metal oxide solid state materials. The central relational database, the Materials Provenance Store (MPS), manages the metadata and experimental provenance from acquisition of raw materials, through synthesis, to a broad range of materials characterization techniques. Given the primary research goal of materials discovery of solar fuels materials, many of the characterization experiments involve electrochemistry, along with optical, structural, and compositional characterizations. The MPS is populated with all information required for executing common data queries, which typically do not involve direct query of raw data. The result is a database file that can be distributed to users so that they can independently execute queries and subsequently download the data of interest. We propose this strategy as an approach to manage the highly heterogeneous and distributed data that arises from materials science experiments, as demonstrated by the management of over 30 million experiments run on over 12 million samples in the present MPS release.
This submission includes an input file, plot file, Fortran conversion file, and 3D data file from the simulation of the temperature profile within the Test Bed #1 of the EGS Collab project. The simulation was executed with PNNL's STOMP-GT simulator, which reads the input file, and produces the STOMP-GT Simulated 2D Temperature Distribution Data (plot.001302 file). The Fortran conversion file, converts the STOMP-GT Simulated 2D Temperature Distribution Data (plot.001302) 2D results in local coordinates to 3D results in Homestake coordinates.
Scientific applications frequently spend a large fraction of their execution time in reading and writing data on parallel file systems. Identifying these I/O performance bottlenecks and attributing root causes are critical steps toward devising optimization strategies. Several existing studies analyze I/O logs of a set of benchmarks or applications that were run with controlled behaviors. However, there is still a lack of general approach that systematically identifies I/O performance bottlenecks for applications running "in the wild" on production systems. In this study, we have developed an analysis approach of "zooming in" from platform-wide to application-wide to job-level I/O logs for identifying I/O bottlenecks in arbitrary scientific applications. We analyze the logs collected on a Cray XC40 system in production over a two-month period. This study results in several insights for application developers to use in optimizing I/O behavior.
This data package is associated with the publication “Modeling Microbial Regulatory Feedback in Organic Matter Decomposition Identifies Copiotrophic Traits as Key Drivers of Positive Priming” published as a preprint on BioRXiv by Ahamed et al. (2026); https://doi.org/10.1101/2024.08.11.607483. The package contains MATLAB scripts and saved simulation outputs used to implement a cybernetic model of microbial regulation during complex organic matter (OM) decomposition governing priming effects. It includes models of (i) single microbial functional groups (copiotrophic or oligotrophic degraders) and (ii) binary consortia composed of degraders and non-degraders with contrasting or common growth traits. Simulation results were generated using Monte Carlo analyses, with randomized key model parameters across a range of environmental mixing fractions of complex and labile OM. The dataset was created to provide a transparent and reusable computational framework for systematically exploring how microbial growth traits, metabolic regulation, and community composition influence OM decomposition dynamics and priming effects. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes the variable definitions. This package includes: (1) annotated MATLAB code implementing the system of ordinary differential equations and cybernetic control laws; (2) saved output files containing data (e.g., biomass, substrates, enzyme levels, priming metrics); and (3) scripts for processing saved outputs and regenerating figures. Specifically, the data package contains three main MATLAB scripts: runPrimingModel.m, runPlotData.m, and runPlotSuppFigS1.m, along with this readme and supporting documentation. Users should begin with runPrimingModel.m, which contains the annotated code implementing the system of ordinary differential equations and cybernetic control laws. This script runs the Monte Carlo simulations of microbial OM decomposition and allows users to modify microbial trait definitions, adjust parameter distributions, or define new community configurations. Simulation outputs are automatically saved as .mat files in the folder named SavedData, which stores all pre-generated results included in this package. The second script, runPlotData.m, reads files from the SavedData folder and processes them to regenerate the figures presented in the manuscript. The third script, runPlotSuppFigS1.m, specifically generates Figure S1 in the Supplementary Material of the manuscript. The package also includes the aforementioned files in non-proprietary .txt format. If users intend to use them, they should first save the files in their respective .m or .mat formats prior to execution in MATLAB.
A computer-executable method, system, and computer program product for managing a Parallel-Logged File System (PLFS) on a data storage system, wherein the data storage system is in communication with an application that updates the data storage system with checkpoint data, the computer-executable method, system, and computer program product comprising receiving a first checkpoint including data from the application; analyzing the data in the first checkpoint to determine whether a pattern exists in the checkpoint data; and upon a positive determination, adding the pattern into PLFS.
The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.