Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Logs”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

The SprayList: a scalable relaxed priority queue

High-performance concurrent priority queues are essential for applications such as task scheduling and discrete event simulation. Unfortunately, even the best performing implementations do not scale past a number of threads in the single digits. This is because of the sequential bottleneck in accessing the elements at the head of the queue in order to perform a DeleteMin operation. In this paper, we present the SprayList, a scalable priority queue with relaxed ordering semantics. Starting from a non-blocking SkipList, the main innovation behind our design is that the DeleteMin operations avoid a sequential bottleneck by ``spraying'' themselves onto the head of the SkipList list in a coordinated fashion. The spraying is implemented using a carefully designed random walk, so that DeleteMin returns an element among the first O(p log^3 p) in the list, with high probability, where p is the number of threads. We prove that the running time of a DeleteMin operation is O(log^3 p), with high probability, independent of the size of the list. Our experiments show that the relaxed semantics allow the data structure to scale for high thread counts, comparable to a classic unordered SkipList. As a result, we observe that, for reasonably parallel workloads, the scalability benefits of relaxation considerably outweigh the additional work due to out-of-order execution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Access Patterns and Performance Behaviors of Multi-layer Supercomputer I/O Subsystems under Production Load

Scientific computing workloads at HPC facilities have been shifting from traditional numerical simulations to AI/ML applications for training and inference while processing and producing ever-increasing amounts of scientific data. To address the growing need for increased storage capacity, lower access latency, and higher bandwidth, emerging technologies such as non-volatile memory are integrated into supercomputer I/O subsystems. With these emerging trends, we need a better understanding of the multilayer supercomputer I/O systems and ways to use these subsystems efficiently. In this work, we study the I/O access patterns and performance characteristics of two representative supercomputer I/O subsystems. Through an extensive analysis of year-long I/O logs on each system, we report new observations in I/O reads and writes, unbalanced use of storage system layers, and new trends in user behaviors at the HPC I/O middleware stack.

Bez, JL↗
DOE OSTI2025

DEDUPKV: A Space-Efficient and High-Performance Key-Value Store via Fine-Grained Deduplication

Log-Structured Merge Tree (LSM-tree) based key-value stores excel in write-intensive environments but suffer from data duplication, consuming up to 49% of storage space in LSM-tree-based key-value store deployments. Traditional solutions like compression and coarse-grained file system-level deduplication introduce overhead or have limited effectiveness. In this study, we propose DedupKV, a fine-grained deduplication framework tailored for LSM-tree, maximizing data reduction efficiency while minimizing write stalls and read overheads. DedupKV features three key innovations: (1) FLUSH-integrated inline deduplication, which removes duplicates during memory-to-storage writes; (2) WAL file-based offline deduplication, repurposing write-ahead logs to avoid double writes; and (3) elastic execution, dynamically balancing inline and offline deduplication based on memory pressure and workload intensity. Additionally, dynamic granularity management reduces deduplication metadata overhead. We implemented these four ideas in RocksDB for the first time and conducted experiments in a Linux environment. Our evaluation shows that WAL file-based offline deduplication and DedupKV outperform BlobDB by 33% and 23%, respectively, in write-heavy workloads, while reducing write amplification by 1.2 ×, 2 ×, and 1.6 × for real KV datasets.

Jamil, Safdar [Sogang University]↗
DOE OSTI2020

Analysis of Electric Vehicle Charging Behavior Patterns with Function Principal Component Analysis Approach

This manuscript focused on analyzing electric vehicles’ (EV) charging behavior patterns with a functional data analysis (FDA) approach, with the goal of providing theoretical support to the EV infrastructure planning and regulation, as well as the power grid load management. 5-year real-world charging log data from a total of 455 charging stations in Kansas City, Missouri, was used. The focuses were placed on analyzing the daily usage occupancy variability, daily energy consumption variability, and station-level usage variability. Compared with the traditional discrete-based analysis models, the proposed FDA modeling approach had unique advantages in preserving the smooth function behavior of the data, bringing more flexibility in the modeling process with little required assumptions or background knowledge on independent variables, as well as the capability of handling time series data with different lengths or sizes. In addition to the patterns revealed in the EV charging station’s occupancy and energy consumption, the differences between EV driver’s charging time and parking time were analyzed and called for the needs for parking regulation and enforcement. The different usage patterns observed at charging stations located on different land-use types were also analyzed.

Engineering↗
DOE OSTI2022

Assessment of Spoilage Microbiota of Rainbow Trout (Oncorhynchus mykiss) during Storage by 16S rDNA Sequencing

Due to the high contents of protein and fat in rainbow trout, it is highly susceptible to spoilage, which limits the storage and transportation processes. Exploring the spoilage microbial community during rainbow trout storage is essential to develop an effective preservation method. Here, the changes in the total bacterial colony and total volatile base nitrogen (TVB-N) during the storage of rainbow trout were investigated. Storage at 0 °C can effectively slow down the spoilage process with bacterial counts and TVB-N contents decreased from 8.7 log CFU/g and 18.7 mg/100 g obtained at 4 °C to 5.6 log CFU/g and 14.5 mg/100 g, respectively. 16S rDNA high-throughput sequencing results showed that the diversity of microbial genera decreased during storage. Acinetobacter, Pseudomonas, and Shewanells gradually became the dominant spoilage genera with contents of 59.9%, 18.6%, and 1.7%, respectively, in the late stage of storage. The spoilage abilities of bacteria belonging to the Pseudomonas and Shewanells genera were analyzed. Shewanella sp. S5-52 showed the highest level of TVB-N content (100.6 mg/100 g) in sterile fish juice, indicating that it had a strong spoilage ability. This study confirmed the dominant spoilage bacterial genera and evaluated the spoilage abilities of isolated strains during the storage of rainbow trout, which laid the foundation for further investigation of the spoilage mechanism of rainbow trout and other aquatic products.

Du, Guangqing↗
DOE OSTI2014

Sight Version 0.1

Enables applications to emit log information into an output file and produced a structured visual summary of the log data, as well as various statistical analyses of it. This makes it easier for developers to understand the behavior of their applications.

Bronevetsky, G.↗
DOE OSTI2018

Anno v1

The Anno log annotation system provides tools for creating and querying annotations to supercomputing log data.

Greiner, AnnetteM↗
DOE OSTI2020

Logic in Memory Emulator

Logic in Memory Emulator (LiME) is a hardware/software tool specially designed for memory system evaluation and experiment. Emerging memories display a wide range of bandwidths, latencies, and capacities, making it challenging for the computer architect to navigate the design space of potential memory configurations, and for the application developer to assess performance implications of using such memories. With the LiME framework, architectural ideas can be prototyped in great detail yet with sufficient performance to support realistic evaluation on long running applications. LiME consists of two fundamental components: 1) the hardware and OS infrastructure for the emulator, and 2) a suite of benchmark applications to assist in characterizing the performance of current and future computer architectures. Some of the applications have been collected from other open source projects. Uses: Logging, replay and analysis of an application's memory behavior Evaluate impact of emerging memory technology on application performance. Emulate complex memory interactions in whole applications orders of magnitude faster than software simulation. Emulate acceleration hardware co-located with the memory subsystem. Features: Capture and log external memory accesses to a separate off-chip memory device without affecting application execution. Memory traces include the address, length, timestamp, and optionally the data for each transaction. Captured trace data can be saved to an SD card for off-line analysis. Configure a wide range of memory latencies in sub-nanosecond increments that encompass highbandwidth and storage class memories. Specify regions of interest (ROI) in applications to reduce the amount of trace data captured for analysis. Currently supports execution on Xilinx Zynq SoC which integrates an ARM processor with FPGA logic on a single device. Applications can be run under Linux or in bare metal mode on the ARM cores.

Jain, AbhishekK↗
DOE OSTI2015

Phisherman v 1.0

Phisherman is an online software tool that was created to help experimenters study phishing. It can potentially be re-purposed to run other human studies. Phisherman enables studies to be run online, so that users can participate from their own computers. This means that experimenters can get data from subjects in their natural settings. Alternatively, an experimenter can also run the app online in a lab-based setting, if that is desired. The software enables the online deployment of a study that is comprised of three main parts: (1) a consent page, (2) a survey, and (3) an identification task, with instruction/transition screens between each part, allowing the experimenter to provide the user with instructions and messages. Upon logging in, the subject is taken to the permission page, where they agree to or do not agree to take part in the study. If the subject agrees to participate, then the software randomly chooses between doing the survey first (and identification task second) or the identification task first (and survey second). This is to balance possible order effects in the data. Procedurally, in the identification task, the software shows the stimuli to the subject, and asks if she thinks it is a phish (yes/no) and how confident she is about her answer. The subject is given 5 levels of certainty to select from, labeled "low" (1), to "medium" (3), to "high" (5), with the option of picking a level between low and medium (2), and between medium and high (4). After selecting his/her confidence level, then the "Next" button activates, allowing a user to move to the next email. The software saves a given subject's progress in the identification task, so that she may log in and out of the site. The consent page is a space for the experimenter to provide the subject with human studies board /internal review board information, and to formally consent to participate in the study. The survey is a space for the experimenter to provide questions and spaces for the users to input answers (allowing both multiple-choice and free-answer options). Phisherman includes administrative pages for managing the stimuli and users. This includes a tool for the experimenter to create, preview, edit, delete (if desired), and manage stimuli (emails). The stimuli may include pictures (uploaded to an appropriate folder) and links, for realism. The software includes a safety feature that prevents the user from going to any link location or opening a file/image. Instead of re-directing the subject's browser, the software provides a pop-up box with the URL location of where the user would have gone. Another administrative page may be used to create fake subject accounts for testing the software prior to deployment, as well as to delete subject accounts when necessary. Data from the experiment can be downloaded from another administrative page.

Kimura, Margot↗
DOE OSTI2020

PAM module for OAuth 2.0 Device flow

This is a PAM module that lets you log in via SSH to servers using OpenID Connect credentials, instead of SSH Keys or a username and password combination. It uses the OAuth2 Device Flow, which means that during the login process, you will click a link and log in to your OpenID Connect Provider, which will then authenticate you for the SSH session. This module will then check if you're in the right group(s) or have a specified username, and allow or deny access.

Surkont, Jarosław↗
DOE OSTI2021

yuwangcn/C4_dynamic_model

A dynamic systems model of C4 photosynthesis was developed based on the previous NADP-ME metabolic model for maize (Wang et al., 2014 ab). The NADP-ME metabolic model is an ordinary differential equation model including all individual steps in C4 photosynthetic carbon metabolism. Here, the model is extended to include posttranslational regulation and temperature response of enzyme activities, dynamic stomata conductance, and leaf energy balance. This model is written in Matlab (R2019a) Steady-state and dynamic gas exchange data for maize (B73), sugarcane (CP88-1762) and sorghum (Tx430) were measured in a greenhouse in Urbana, IL from July 25, 2019 through August 8, 2019. Data include: CO2 response curves Light response curves Photosynthetic induction curves measured in the transition from darkness to high light (1800 μmol m-2 s-1), data logged every 1 min. Photosynthetic induction in the transition from darkness to high light (1800 μmol m-2 s-1) to determine the kinetics of rubisco activation in these C4 crops (τ_Rubisco), data logged every 10 s. Gas exchange under fluctuating light. After dark adaptation, the leaves undergo three light change steps, light intensity was set as 1800 µmol m-2 s-1, 200 µmol m-2 s-1 and 1800 µmol m-2 s-1 for each 1800 s step.

Wang, Yu↗
DOE OSTI2025

OLCF Test Harness

Acceptance and regression testing of a High Performance Computing (HPC) system requires an automated and reproducible framework and tool for running and logging results. Manually running tests across a system is labor intensive and prone to reproducibility errors. The OLCF Test Harness (OTH) provides a framework in which to document required tests for a HPC system. The OTH then provides tools to execute and log results of these tests in an automated fashion.

Dietz, Dan [Oak Ridge National Laboratory (ORNL), ↗
DOE OSTI2026

HPC ODA Commons [SWR-26-003]

HPC ODA Commons is a community-driven platform for standardizing HPC operational data analytics. HPC sites generate enormous volumes of operational data - scheduler logs, accounting records, monitoring streams - but turning that data into actionable insight is needlessly hard. Each site builds bespoke parsers, schemas, and evaluation pipelines. Results can't be compared across institutions. Promising analytics ideas stay siloed because there's no shared language for describing the data, the experiments, or the outcomes. HPC ODA Commons fixes this by establishing community-governed contracts - versioned schemas, canonical artifacts, and benchmark recipes - that make ODA workflows discoverable, reproducible, and comparable. It pairs these standards with a practical, CLI-first toolkit that lets operators and researchers go from raw logs to standardized results without sending data off-cluster.

Menear, Kevin [National Laboratory of the Rockies ↗
DOE OSTI2020

Formulation of autoconversion and drop spectra shape in shallow cumulus clouds

Two-moment autoconversion parameterizations as compared to accretion parameterizations exhibit significant errors suggesting that additional moments are needed to increase their accuracy. We develop a three-moment autoconversion parameterization using output from an LES model with size-resolved microphysics. Adding the third moment decreases the errors of parameterization and improves precipitation prediction. However, the errors are still significantly larger than errors of accretion rate. An analysis of the cloud drop size distributions (DSDs) in the simulated tropical convective cloud system reveals that most of DSDs have a significant fraction of cloud liquid water content (qc) in the mid-size droplet range (radii from 20 to 40 microns). Our data indicates that more than 30% of DSDs have over half of qc contained in the mid-size range and about 60% of spectra have, at least, one third of qc in this range. Even when the rain/drizzle mode is small (radar reflectivity Z < -10 dBZ), there is a significant number of spectra in which fraction of qc in the mid-size range is as large as 60%. These DSDs are more complex than the frequently used single-mode Gamma or Log-normal distributions, which usually do not have a considerable tail extending to the mid-size range and can be defined by three microphysical moments. The need to define DSDs by more than three moments explains the large errors in the three-moment autoconversion parameterization. The limitation of three-parameter Gamma or Log-normal distributions should be kept in mind when applying them in precipitating shallow Cu clouds.

Kogan, Yefim↗
DOE OSTI2019

Using a Canopy Model Framework to Improve Large-Eddy Simulations of the Neutral Atmospheric Boundary Layer in the Weather Research and Forecasting Model

A canopy model framework is implemented in the Weather Research and Forecasting Model to improve the accuracy of large-eddy simulations (LES) of the atmospheric boundary layer (ABL). The model includes two options that depend on the scale of surface roughness elements. A resolved canopy model, typically used to model flow through vegetation canopies, is employed when roughness elements are resolved by the vertical LES grid. In the case of unresolved roughness, a modified “pseudocanopy model” is developed to distribute drag over a shallow layer above the surface. Both canopy model options are validated against idealized test cases in neutral stability conditions and are shown to improve surface layer velocity profiles relative to simulations employing Monin–Obukhov similarity theory (MOST), which is commonly used as a surface boundary condition in ABL models. Use of the canopy model framework also leads to increased levels of resolved turbulence kinetic energy and turbulent stresses. Because LES of the ABL has a well-known difficulty recovering the expected logarithmic velocity profile (log law) in the surface layer, particular focus is placed on using the pseudocanopy model to alleviate this issue over a range of model configurations. Tests with varying surface roughness values, LES closures, and grid aspect ratios confirm that the pseudocanopy model generally improves log-law agreement relative to simulations that employ a standard MOST boundary condition. The canopy model framework thus represents a low-cost, easy-to-implement method for improving LES of the ABL.

17 WIND ENERGY↗
DOE OSTI2020

On the Impact of Income, Age, and Travel Distance on the Value of Time

The value of time (VOT) is a fundamental component used in transportation modeling, policy analysis, and economic appraisal. Decades of research and practice have empirically estimated the VOT across many factors (e.g., mode, purpose, time, comfort, etc.), yet little is known about its underlying form. Although it is well established that VOT can vary, it is still unclear whether patterns exist in this variation. The objective of this paper is not to merely estimate the VOT, but to model the VOT across multiple continuous and interacting variables. The purpose is to reveal its functional form with respect to mode, age, gender, purpose, income, and time of day to provide a generalizable understanding for future research and practice. Such an understanding can help develop simpler models and reduce the need for bespoke estimations for every conceivable variable perturbation. This research utilized a household travel survey containing 14,159 reported trips with imputed travel time and costs for the alternative mode choices. The average overall estimated VOT is 40.32 $/h, with results showing VOT varying log-linearly with income and trip distance, but following a Gaussian function (normal curve) with age. Overall, the results show that travel distance dominates VOT variation, which increases exponentially at a rate that is 3.61 times higher per mile of distance than per $10,000 of income, and that VOT by age peaks at age 54. This basic understanding of how the VOT varies sets the foundation for answering the subsequent question for why it might vary.

Engineering↗
DOE OSTI2023

Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets

Abstract With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman’s rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. ( Sensors 2022 , 22 , 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman’s rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman’s rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).

Rasmussen, Maria H.↗
DOE OSTI2020

Vitality models found useful in modeling tag-failure times in acoustic-tag survival studies

Abstract Acoustic telemetry studies often rely on the assumption that premature tag failure does not affect the validity of inferences. However, in some cases this assumption is possibly or likely invalid and it is necessary to apply a correction to estimation procedures. The question of which approaches and specific models are best suited to modeling acoustic tag failures has received little research attention. In this short communication, we present a meta-analysis of 42 acoustic tag-life studies, originally used to correct survival studies involving outmigrating juvenile salmonids in the Columbia/Snake river basin. We compare the performance of nine alternative parametric models including common failure–time/survival models and the vitality models of Li and Anderson Theor Popul Biol 76:118–131, (2009) and Demogr Res 28:341–372, (2013). The tag-life studies used acoustic tags from three different tag manufacturers, had expected lifetimes between 12 and 61 days, and had dry weights ranging from 0.22 to 1.65 g. In 57% of the cases, the vitality models of Li and Anderson Theor Popul Biol 76:118–131, (2009) and Demogr Res 28:341–372, (2013) fit the tag-failure times best. The vitality models were also the second-best choices in 17% of the cases. Together, the vitality models, log-logistic, (19%), and gamma models (14%) accounted for 90% of the models selected. Unlike more traditional failure–time models (e.g., Weibull, Gompertz, gamma, and log-logistic), the vitality models are capable of characterizing both the early onset of tag failure due to manufacturing errors and the anticipated battery life. We provide further guidance on appropriate sample sizes (50–100 tags) and procedures to be considered when applying precise tag-life corrections in release–recapture survival studies.

Skalski, John R. (ORCID:0000000270702505)↗