SEARCH · Engineering Papers

Results for “logs”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Pulse-shape discrimination against low-energy Ar-39 beta decays in liquid argon with 4.5 tonne-years of DEAP-3600 data

The DEAP-3600 detector searches for the scintillation signal from dark matter particles scattering on a 3.3 tonne liquid argon target. The largest background comes from 39 Ar beta decays and is suppressed using pulse-shape discrimination (PSD). We use two types of PSD estimator: the prompt-fraction, which considers the fraction of the scintillation signal in a narrow and a wide time window around the event peak, and the log-likelihood-ratio, which compares the observed photon arrival times to a signal and a background model. We furthermore use two algorithms to determine the number of photons detected at a given time: (1) simply dividing the charge of each PMT pulse by the mean single-photoelectron charge, and (2) a likelihood analysis that considers the probability to detect a certain number of photons at a given time, based on a model for the scintillation pulse shape and for afterpulsing in the light detectors. The prompt-fraction performs approximately as well as the log-likelihood-ratio PSD algorithm if the photon detection times are not biased by detector effects. We explain this result using a model for the information carried by scintillation photons as a function of the time when they are detected.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Access Patterns and Performance Behaviors of Multi-layer Supercomputer I/O Subsystems under Production Load

Scientific computing workloads at HPC facilities have been shifting from traditional numerical simulations to AI/ML applications for training and inference while processing and producing ever-increasing amounts of scientific data. To address the growing need for increased storage capacity, lower access latency, and higher bandwidth, emerging technologies such as non-volatile memory are integrated into supercomputer I/O subsystems. With these emerging trends, we need a better understanding of the multilayer supercomputer I/O systems and ways to use these subsystems efficiently. In this work, we study the I/O access patterns and performance characteristics of two representative supercomputer I/O subsystems. Through an extensive analysis of year-long I/O logs on each system, we report new observations in I/O reads and writes, unbalanced use of storage system layers, and new trends in user behaviors at the HPC I/O middleware stack.

Bez, JL↗

DOE OSTI2025

DEDUPKV: A Space-Efficient and High-Performance Key-Value Store via Fine-Grained Deduplication

Log-Structured Merge Tree (LSM-tree) based key-value stores excel in write-intensive environments but suffer from data duplication, consuming up to 49% of storage space in LSM-tree-based key-value store deployments. Traditional solutions like compression and coarse-grained file system-level deduplication introduce overhead or have limited effectiveness. In this study, we propose DedupKV, a fine-grained deduplication framework tailored for LSM-tree, maximizing data reduction efficiency while minimizing write stalls and read overheads. DedupKV features three key innovations: (1) FLUSH-integrated inline deduplication, which removes duplicates during memory-to-storage writes; (2) WAL file-based offline deduplication, repurposing write-ahead logs to avoid double writes; and (3) elastic execution, dynamically balancing inline and offline deduplication based on memory pressure and workload intensity. Additionally, dynamic granularity management reduces deduplication metadata overhead. We implemented these four ideas in RocksDB for the first time and conducted experiments in a Linux environment. Our evaluation shows that WAL file-based offline deduplication and DedupKV outperform BlobDB by 33% and 23%, respectively, in write-heavy workloads, while reducing write amplification by 1.2 ×, 2 ×, and 1.6 × for real KV datasets.

Jamil, Safdar [Sogang University]↗

DOE OSTI2020

Analysis of Electric Vehicle Charging Behavior Patterns with Function Principal Component Analysis Approach

This manuscript focused on analyzing electric vehicles’ (EV) charging behavior patterns with a functional data analysis (FDA) approach, with the goal of providing theoretical support to the EV infrastructure planning and regulation, as well as the power grid load management. 5-year real-world charging log data from a total of 455 charging stations in Kansas City, Missouri, was used. The focuses were placed on analyzing the daily usage occupancy variability, daily energy consumption variability, and station-level usage variability. Compared with the traditional discrete-based analysis models, the proposed FDA modeling approach had unique advantages in preserving the smooth function behavior of the data, bringing more flexibility in the modeling process with little required assumptions or background knowledge on independent variables, as well as the capability of handling time series data with different lengths or sizes. In addition to the patterns revealed in the EV charging station’s occupancy and energy consumption, the differences between EV driver’s charging time and parking time were analyzed and called for the needs for parking regulation and enforcement. The different usage patterns observed at charging stations located on different land-use types were also analyzed.

Engineering↗

DOE OSTI2022

Assessment of Spoilage Microbiota of Rainbow Trout (Oncorhynchus mykiss) during Storage by 16S rDNA Sequencing

Due to the high contents of protein and fat in rainbow trout, it is highly susceptible to spoilage, which limits the storage and transportation processes. Exploring the spoilage microbial community during rainbow trout storage is essential to develop an effective preservation method. Here, the changes in the total bacterial colony and total volatile base nitrogen (TVB-N) during the storage of rainbow trout were investigated. Storage at 0 °C can effectively slow down the spoilage process with bacterial counts and TVB-N contents decreased from 8.7 log CFU/g and 18.7 mg/100 g obtained at 4 °C to 5.6 log CFU/g and 14.5 mg/100 g, respectively. 16S rDNA high-throughput sequencing results showed that the diversity of microbial genera decreased during storage. Acinetobacter, Pseudomonas, and Shewanells gradually became the dominant spoilage genera with contents of 59.9%, 18.6%, and 1.7%, respectively, in the late stage of storage. The spoilage abilities of bacteria belonging to the Pseudomonas and Shewanells genera were analyzed. Shewanella sp. S5-52 showed the highest level of TVB-N content (100.6 mg/100 g) in sterile fish juice, indicating that it had a strong spoilage ability. This study confirmed the dominant spoilage bacterial genera and evaluated the spoilage abilities of isolated strains during the storage of rainbow trout, which laid the foundation for further investigation of the spoilage mechanism of rainbow trout and other aquatic products.

Du, Guangqing↗

DOE OSTI2020

Logic in Memory Emulator

Logic in Memory Emulator (LiME) is a hardware/software tool specially designed for memory system evaluation and experiment. Emerging memories display a wide range of bandwidths, latencies, and capacities, making it challenging for the computer architect to navigate the design space of potential memory configurations, and for the application developer to assess performance implications of using such memories. With the LiME framework, architectural ideas can be prototyped in great detail yet with sufficient performance to support realistic evaluation on long running applications. LiME consists of two fundamental components: 1) the hardware and OS infrastructure for the emulator, and 2) a suite of benchmark applications to assist in characterizing the performance of current and future computer architectures. Some of the applications have been collected from other open source projects. Uses: Logging, replay and analysis of an application's memory behavior Evaluate impact of emerging memory technology on application performance. Emulate complex memory interactions in whole applications orders of magnitude faster than software simulation. Emulate acceleration hardware co-located with the memory subsystem. Features: Capture and log external memory accesses to a separate off-chip memory device without affecting application execution. Memory traces include the address, length, timestamp, and optionally the data for each transaction. Captured trace data can be saved to an SD card for off-line analysis. Configure a wide range of memory latencies in sub-nanosecond increments that encompass highbandwidth and storage class memories. Specify regions of interest (ROI) in applications to reduce the amount of trace data captured for analysis. Currently supports execution on Xilinx Zynq SoC which integrates an ARM processor with FPGA logic on a single device. Applications can be run under Linux or in bare metal mode on the ARM cores.

Jain, AbhishekK↗

DOE OSTI2020

PAM module for OAuth 2.0 Device flow

This is a PAM module that lets you log in via SSH to servers using OpenID Connect credentials, instead of SSH Keys or a username and password combination. It uses the OAuth2 Device Flow, which means that during the login process, you will click a link and log in to your OpenID Connect Provider, which will then authenticate you for the SSH session. This module will then check if you're in the right group(s) or have a specified username, and allow or deny access.

Surkont, Jarosław↗

DOE OSTI2021

yuwangcn/C4_dynamic_model

A dynamic systems model of C4 photosynthesis was developed based on the previous NADP-ME metabolic model for maize (Wang et al., 2014 ab). The NADP-ME metabolic model is an ordinary differential equation model including all individual steps in C4 photosynthetic carbon metabolism. Here, the model is extended to include posttranslational regulation and temperature response of enzyme activities, dynamic stomata conductance, and leaf energy balance. This model is written in Matlab (R2019a) Steady-state and dynamic gas exchange data for maize (B73), sugarcane (CP88-1762) and sorghum (Tx430) were measured in a greenhouse in Urbana, IL from July 25, 2019 through August 8, 2019. Data include: CO2 response curves Light response curves Photosynthetic induction curves measured in the transition from darkness to high light (1800 μmol m-2 s-1), data logged every 1 min. Photosynthetic induction in the transition from darkness to high light (1800 μmol m-2 s-1) to determine the kinetics of rubisco activation in these C4 crops (τ_Rubisco), data logged every 10 s. Gas exchange under fluctuating light. After dark adaptation, the leaves undergo three light change steps, light intensity was set as 1800 µmol m-2 s-1, 200 µmol m-2 s-1 and 1800 µmol m-2 s-1 for each 1800 s step.

Wang, Yu↗

DOE OSTI2025

OLCF Test Harness

Acceptance and regression testing of a High Performance Computing (HPC) system requires an automated and reproducible framework and tool for running and logging results. Manually running tests across a system is labor intensive and prone to reproducibility errors. The OLCF Test Harness (OTH) provides a framework in which to document required tests for a HPC system. The OTH then provides tools to execute and log results of these tests in an automated fashion.

Dietz, Dan [Oak Ridge National Laboratory (ORNL), ↗

DOE OSTI2026

HPC ODA Commons [SWR-26-003]

HPC ODA Commons is a community-driven platform for standardizing HPC operational data analytics. HPC sites generate enormous volumes of operational data - scheduler logs, accounting records, monitoring streams - but turning that data into actionable insight is needlessly hard. Each site builds bespoke parsers, schemas, and evaluation pipelines. Results can't be compared across institutions. Promising analytics ideas stay siloed because there's no shared language for describing the data, the experiments, or the outcomes. HPC ODA Commons fixes this by establishing community-governed contracts - versioned schemas, canonical artifacts, and benchmark recipes - that make ODA workflows discoverable, reproducible, and comparable. It pairs these standards with a practical, CLI-first toolkit that lets operators and researchers go from raw logs to standardized results without sending data off-cluster.

Menear, Kevin [National Laboratory of the Rockies ↗

DOE OSTI2020

Formulation of autoconversion and drop spectra shape in shallow cumulus clouds

Two-moment autoconversion parameterizations as compared to accretion parameterizations exhibit significant errors suggesting that additional moments are needed to increase their accuracy. We develop a three-moment autoconversion parameterization using output from an LES model with size-resolved microphysics. Adding the third moment decreases the errors of parameterization and improves precipitation prediction. However, the errors are still significantly larger than errors of accretion rate. An analysis of the cloud drop size distributions (DSDs) in the simulated tropical convective cloud system reveals that most of DSDs have a significant fraction of cloud liquid water content (qc) in the mid-size droplet range (radii from 20 to 40 microns). Our data indicates that more than 30% of DSDs have over half of qc contained in the mid-size range and about 60% of spectra have, at least, one third of qc in this range. Even when the rain/drizzle mode is small (radar reflectivity Z < -10 dBZ), there is a significant number of spectra in which fraction of qc in the mid-size range is as large as 60%. These DSDs are more complex than the frequently used single-mode Gamma or Log-normal distributions, which usually do not have a considerable tail extending to the mid-size range and can be defined by three microphysical moments. The need to define DSDs by more than three moments explains the large errors in the three-moment autoconversion parameterization. The limitation of three-parameter Gamma or Log-normal distributions should be kept in mind when applying them in precipitating shallow Cu clouds.

Kogan, Yefim↗

DOE OSTI2020

On the Impact of Income, Age, and Travel Distance on the Value of Time

The value of time (VOT) is a fundamental component used in transportation modeling, policy analysis, and economic appraisal. Decades of research and practice have empirically estimated the VOT across many factors (e.g., mode, purpose, time, comfort, etc.), yet little is known about its underlying form. Although it is well established that VOT can vary, it is still unclear whether patterns exist in this variation. The objective of this paper is not to merely estimate the VOT, but to model the VOT across multiple continuous and interacting variables. The purpose is to reveal its functional form with respect to mode, age, gender, purpose, income, and time of day to provide a generalizable understanding for future research and practice. Such an understanding can help develop simpler models and reduce the need for bespoke estimations for every conceivable variable perturbation. This research utilized a household travel survey containing 14,159 reported trips with imputed travel time and costs for the alternative mode choices. The average overall estimated VOT is 40.32 $/h, with results showing VOT varying log-linearly with income and trip distance, but following a Gaussian function (normal curve) with age. Overall, the results show that travel distance dominates VOT variation, which increases exponentially at a rate that is 3.61 times higher per mile of distance than per $10,000 of income, and that VOT by age peaks at age 54. This basic understanding of how the VOT varies sets the foundation for answering the subsequent question for why it might vary.

Engineering↗

DOE OSTI2023

Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets

Abstract With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman’s rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. ( Sensors 2022 , 22 , 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman’s rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman’s rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).

Rasmussen, Maria H.↗

DOE OSTI2020

Vitality models found useful in modeling tag-failure times in acoustic-tag survival studies

Abstract Acoustic telemetry studies often rely on the assumption that premature tag failure does not affect the validity of inferences. However, in some cases this assumption is possibly or likely invalid and it is necessary to apply a correction to estimation procedures. The question of which approaches and specific models are best suited to modeling acoustic tag failures has received little research attention. In this short communication, we present a meta-analysis of 42 acoustic tag-life studies, originally used to correct survival studies involving outmigrating juvenile salmonids in the Columbia/Snake river basin. We compare the performance of nine alternative parametric models including common failure–time/survival models and the vitality models of Li and Anderson Theor Popul Biol 76:118–131, (2009) and Demogr Res 28:341–372, (2013). The tag-life studies used acoustic tags from three different tag manufacturers, had expected lifetimes between 12 and 61 days, and had dry weights ranging from 0.22 to 1.65 g. In 57% of the cases, the vitality models of Li and Anderson Theor Popul Biol 76:118–131, (2009) and Demogr Res 28:341–372, (2013) fit the tag-failure times best. The vitality models were also the second-best choices in 17% of the cases. Together, the vitality models, log-logistic, (19%), and gamma models (14%) accounted for 90% of the models selected. Unlike more traditional failure–time models (e.g., Weibull, Gompertz, gamma, and log-logistic), the vitality models are capable of characterizing both the early onset of tag failure due to manufacturing errors and the anticipated battery life. We provide further guidance on appropriate sample sizes (50–100 tags) and procedures to be considered when applying precise tag-life corrections in release–recapture survival studies.

Skalski, John R. (ORCID:0000000270702505)↗

DOE OSTI2022

CSEM Fluid Monitoring Methodology Using Real Data Examples

Conference presentation at International Meeting for Applied Geoscience & Energy (IMAGE), Houston, Texas, August 28 – September 1, 2023. Using field data from hydrocarbon and CO 2 applications, we illustrate the importance of a workflow and adaption to the target on hand. Verifying the geophysical acquisition and processing steps with 3D modeling and checking them against a 3D anisotropic log-derived model maintains confidence in the workflow and minimizes the influence on the data. This allows us to predict data validity and to certify the data with respect to the borehole logs.

20 FOSSIL-FUELED POWER PLANTS↗

DOE OSTI2021

Pore characterization of the Marcellus Shale by nitrogen adsorption and prediction of its gas storage capacity

In a shale gas reservoir, pore characterization is an important factor used to determine gas storage capacity. However, the nanometer (nm)-scale pore system in shale is difficult to explore by traditional optical, scanning electron microscopy, or even nuclear magnetic resonance well logging. We have investigated the pore structure and storage capacity of the Marcellus Shale through integration of petrophysical analysis from laboratory and well-logging data and nitrogen adsorption. The isotherm of Marcellus Shale is a composite isotherm, which has features of type I, type II, and type IV isotherms with type H4 of the hysteresis loop, suggesting slit-like pores developed in the Marcellus Shale. Quantitative analysis of pore volumes from the nitrogen adsorption indicates that density porosity may be more properly used to approximate the shale porosity and estimate the shale gas volume. In addition, the specific surface area, micropore, and mesopore volumes have a positive relationship with the kerogen volume and total organic content (TOC). By using the Langmuir and Brunauer-Emmet-Teller models, the simulated result indicates that the higher adsorbed quantity of the Marcellus Shale could be the result of the increase of micropore volume contributed, by the increase of kerogen or TOC content. The proposed equations rapidly compute TOC, a key parameter to predict gas storage capacity in overmature shale such as the Marcellus Shale.

Geochemistry & Geophysics↗

DOE OSTI2023

OLCF Summit Supercomputer GPU Snapshots During Double-Bit Errors and Normal Operations

As we move into the exascale era, the power and energy footprints of high-performance computing (HPC) systems have grown significantly larger. Due to the harsh power and thermal conditions the system, components are exposed to extreme operating conditions. Operation of such modern HPC systems requires deep insights into long term system behavior to maintain its efficiency as well as its longevity. To help the HPC community to gain such insights, we provide double-bit errors using system telemetry data and logs collected from the Summit supercomputer, equipped with 27,648 Tesla V100 GPUs with 2nd-generation high-bandwidth memory (HBM2). The dataset relies on Nvidia XID records internally collected by GPU firmware at the time of failure occurrence, on the reboot-time logs of each Summit node, on node-level job scheduler records collected after each job termination, and on a 1Hz data rate from the baseboard management controllers (BMCs) of each Summit compute node using the OpenBMC event subscription protocol. Technical details can be found in the paper Oles et. al “Understanding GPU Memory Corruption at Extreme Scale: The Summit Case Study” ICS’24 (https://doi.org/10.1145/3650200.3656615).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Summit Darshan Archival Dataset

Summit Darshan Archival Dataset contains 2021 Summit Darshan log data for 25 applications and is grouped into science domains. The dataset is processed, and all the propriety fields are anonymized. The resultant data is converted into a tabular structure and saved in parquet file format. In this notebook, we demonstrate how to access the data. Data Organization: The data is organized into two directories: Darshan total (`darshan_total`): List all the high levels generated by the `darshan-parser --total` command on `.darshan` files. There is one parquet file for each application. Note: `uid` and `exe` field are masked Darshan detail (`darshan_detail`): This data contains detailed job level log information extracted by command `darshan-parser` on the raw `.darshan` files. The data is sorted by directory hierarchy in the order of `year/month/day (2021/12/07)`. For instance, to get the data for a `job_id` 3819766 of application `App11`, which was executed on `2021-12-07`can be accessed as follows. Note:`uid` and `filename` fields are masked

97 MATHEMATICS AND COMPUTING↗