Persistent Data Staging Services for Data Intensive In-situ Scientific Workflows
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
EpetraExt is a set of extensions to Epetra. To allow Epetra to remain focused on its primary functionality as a Linear Algebra object support, EpetraExt was created to maintain additional support for such capabilities as transformations (permutations, subblock views, etc.), coloring support, partitioning (Zoltan), and I/O.
LDMS is a framework for collection and transport of system monitoring data from computational clusters. It utilizes existing data sources on the cluster (e.g., temperature sensors, network traffic counters). The collection and transport mechanisms of LDMS are lightweight and do not significantly adversely impact the performance of concurrently running applications.
The ProvEn Software is designed to analyze and monitor distributed scientific workflow application behavior in the context of its environment.
SSG is a group membership microservice based on the Mercury RPC system. It provides mechanisms for bootstrapping sets of Mercury processes into logical groups and for managing the membership of these process groups over time. At a high-level, each group collectively maintains a group view, which is just a mapping from group member identifiers to Mercury address information. The initial group membership view is specified completely when the group is bootstrapped (created).
POESIE is a Mochi microservice designed to run interpreters of various scripting languages (currently Lua and Python) and make them accessible remotely.
METS-R ADDSEVS simulator is a high fidelity, parallel, agent-based evacuation simulator for multi-modal energy-optimal trip scheduling in real-time (METS-R) at transportation hubs. It consists of two modules. The first one is the traffic simulator module; the second one is the high-performance computing (HPC) module. More details can be found at https://umnilab.github.io/METS-R_doc/.
As demand flexibility becomes more common, the mix and frequency of a range of demand flexibility strategies will introduce potential biases into baselines, i.e., the time period immediately prior to a LS event may include other LS or DF events. Baseline degradation assessment provides an approach to quantify how these biases begin to compromise predictive accuracy. Heatmaps, based on each month of the year provides a granular view of a given model's predictive capability. However, for comparing multiple algorithms, separately for weekdays and weekends, aggregating results to seasons provides a simpler means of comparison while still allowing for review of variations by time of day and time of year
The code is designed to run tests to generate and collect data from a Wi-Fi network using OPENWRT or a simulated a 5G network using Open5gs and UERANSIM. The tests simulate the network performing downloads or uploads of various files with a varying number of concurrent users. The tests use tcpdump to collect the network traffic but only stores the summarized data. The summarized datasets will be included.
Miscellany This LDMS Loftsman/Helm Chart horizontally scales LDMS daemons in order to achieve a 1Hz sample rate from over 5,000 nodes, collecting 38k metrics per minute on Perlmutter. This LMDS Configuration relies on already running `ldmsd` producers running on nodes, which produce metrics via sampler plugins. The Helm chart distributes the collection of metrics from producer acrross many aggregator and storage `ldmsd` daemons, ensuring no damon is overloaded and data loss is avoided.
AQDrop is a job management system designed to streamline access to the Advanced Quantum Testbed (AQT) at NERSC (National Energy Research Scientific Computing Center). It serves as a centralized middleware layer between researchers and quantum processing hardware. Key Features: AQDrop provides a FastAPI-based server backed by PostgreSQL for job submission, queue management, and role-based access control (members, operators, and administrators). Users submit Qiskit circuits via JSON payloads, which are queued, dispatched to the QPU through the Qubic API, and returned as measurement counts. A Python client library and web dashboard round out the interface options. Primary Use: Researchers submit quantum circuit jobs from a laptop or login node; an operator client executes those jobs on the AQT's physical QPU and returns results — all coordinated through the central API. Advantages: Compared to ad-hoc or direct hardware access, AQDrop adds structured queue management, auditable job-status tracking and OAuth2 authentication — reducing scheduling conflicts and unauthorized access. Its containerized deployment also improves reproducibility and scalability. Overall, AQDrop functions as a purpose-built quantum job broker tailored to NERSC's specific hardware and institutional access requirements.
Many extreme-scale applications require the movement of large quantities of data to, from, and among leadership computing facilities, as well as other scientific facilities and the home institutions of facility users. These applications, particularly when leadership computing facilities are involved, can touch upon edge cases (e.g., terabyte files) that had not been a focus of previous Globus optimization work, which had emphasized rather the movement of many smaller (megabyte to gigabyte) files. We report here on how automated client-driven chunking can be used to accelerate both the movement of large files and the integrity checking operations that have proven to be essential for large data transfers. In conclusion, we present detailed performance studies that provide insights into the benefits of these modifications in a range of file transfer scenarios.
Recent advances in high-volume sequencing technology and mining of genomes from metagenomic samples call for rapid and reliable genome quality evaluation. The current release of the PATRIC database contains over 220,000 genomes, and current metagenomic technology supports assemblies of many draft-quality genomes from a single sample, most of which will be novel. We have added two quality assessment tools to the PATRIC annotation pipeline. EvalCon uses supervised machine learning to calculate an annotation consistency score. EvalG implements a variant of the CheckM algorithm to estimate contamination and completeness of an annotated genome.We report on the performance of these tools and the potential utility of the consistency score. Additionally, we provide contamination, completeness, and consistency measures for all genomes in PATRIC and in a recent set of metagenomic assemblies. EvalG and EvalCon facilitate the rapid quality control and exploration of PATRIC-annotated draft genomes.
The concept, framework, process methodology and applications of parallel battery were proposed from both virtual and real aspects.The parallel battery was an application of ACP-based parallel intelligence in battery and related energy system areas.The real battery system was running with its equivalent, and the artificial battery system was in a virtual space, in a parallel and interactive manner.The artificial battery system contained the descriptive, predictive, and prescriptive functions on the real battery and related energy systems.There was a closed-loop workflow between the real battery system and the artificial battery system, which iteratively optimizes the battery and related energy systems, leading to a new paradigm of intelligent and ecological parallel battery system management.
Amidst the COVID-19 pandemic, restaurants become more reliant on no-contact pick-up or delivery ways for serving customers. As a result, they need to make tactical planning decisions such as whether to partner with online platforms, to form their own delivery team, or both. In this paper, we develop an integrated prediction-decision model to analyze the profit of combining the two approaches and to decide the needed number of drivers under stochastic demand. We first use the susceptible-infected-recovered (SIR) model to forecast future infected cases in a given region and then construct an autoregressive-moving-average (ARMA) regression model to predict food-ordering demand. Using predicted demand samples, we formulate a stochastic integer program to optimize food delivery plans. We conduct numerical studies using COVID-19 data and food-ordering demand data collected from local restaurants in Nuevo Leon, Mexico, from April to October 2020, to show results for helping restaurants build contingency plans under rapid market changes. Our method can be used under unexpected demand surges, various infection/vaccination status, and demand patterns. Here, our results show that a restaurant can benefit from partnering with third-party delivery platforms when (i) the subscription fee is low, (ii) customers can flexibly decide whether to order from platforms or from restaurants directly, (iii) customers require more efficient delivery, (iv) average delivery distance is long, or (v) demand variance is high.
Turbomolecular pumps (TMP) will be used with large amounts of tritium in future fusion machines like ITER, DEMO and in the KATRIN Experiment. In this work, a stress test of a large, magnetically levitated TMP (Leybold MAG W2800) with a tritium throughput of 1.1 kg over 384 days of operation was performed at TLK. After this, the pump was dismantled and the tritium uptake in several parts was deter-mined. Especially the non-metallic parts of the pump have absorbed large amounts of tritium and are most likely responsible for the observed pollution of the process gas. The total tritium uptake of the TMP was estimated with 0.1-1.1 TBq. No radiation-induced damages were found on the inner parts of the pump. The TMP showed no signs of functional limitations during the 384 days of operation. (authors)
We implement a traceable protocol for device-independent random number generation. Our demonstration involves transparently integrating a device-independent RNG into a public randomness beacon, and feeding 7434 random number pulses into the beacon over 39 days.