Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Distributed File Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Leveraging Pre-Built Catalogs and Object-Level Scheduling to Eliminate I/O Bottlenecks in HPC Environments

Modern High-Performance Computing (HPC) environments face mounting challenges due to the shift from large to small file datasets, along with an increasing number of users and parallelized applications. As HPC systems rely on Parallel File Systems (PFS), such as Lustre for data processing, performance bottlenecks stemming from Object Storage Target (OST) contention have become a significant concern. Existing solutions, such as LADS with its object-level scheduling approach, fall short in large-scale HPC environments due to their inability to effectively address metadata I/O bottlenecks and the growing number of I/O processes. This study highlights the pressing need for a comprehensive solution that tackles both OST contention and metadata I/O challenges in diverse HPC workloads. To address these challenges, we propose SwiftLoad, an object-level I/O scheduling framework that leverages a metadata catalog to enhance the performance and efficiency of parallel HPC utilities. The adoption of the metadata catalog mitigates the metadata I/O bottlenecks that commonly occur in HPC utilities, a challenge that is particularly pronounced in object-level I/O scheduling. SwiftLoad addresses OST contention and the uneven distribution of I/O processes across different OSTs through mathematical modeling and incorporates a Loader Configuration Module to regulate the number of I/O processes. Evaluated with two representative utilities—data deduplication profiling and data augmentation—SwiftLoad achieved performance improvements of up to 5.63x and 11.0x, respectively, on a production supercomputer.

HPC↗
DOE OSTI2025

How initial conditions-, structural-, and parameter-based model uncertainty interact and influence predictions in permafrost ecosystems: Modeling Archive

This dataset contains model output and input data, as well as source code examples for the Terrestrial Ecosystem Model with the Dynamic Vegetation Model and Dynamic Organic Soil (DVM-DOS-TEM) for the field sites Imnavait creek and the Bonanza creek Long Term Ecological Research Network (LTER). The data covers simulations from the last glacial maximum (LGM) until 2100 for a selection of paleo scenarios, setting the mean temperature of the LGM up to 10°C lower than pre-industrial conditions. The model structure was modulated to represent various model versions, and this dataset contains the relevant changes in the source code. The raw output data, the processed statistical data, the setup and processing scripts as well as parameter value distribution files from a parameter sensitivity analysis are included as well. Model outputs include active layer depth, organic soil carbon, soil layer depths, gross primary productivity (GPP) with and without nitrogen limitation, net primary productivity (NPP), soil liquid water content, heterotrophic, maintenance, and growth respiration, soil temperature, and vegetation carbon (*.nc files). The Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) project is a research effort to reduce uncertainty in the Department of Energy’s Energy Exascale Earth System Model (E3SM) by developing a predictive understanding of Arctic tundra ecosystems underlain by permafrost and to quantify feedbacks from the Arctic tundra to the Earth system. NGEE Arctic is supported by the Department of Energy's Office of Biological and Environmental Research.Over Phases 1–3, observations made by the NGEE Arctic team across a gradient of permafrost landscapes in Arctic Alaska improved the representation of tundra processes in the land surface component of E3SM (the E3SM Land Model, ELM). Model improvements emphasized unique aspects of permafrost environments and explored reductions in model complexity while retaining predictive power. The Arctic-informed ELM developed by NGEE Arctic has been used to make novel predictions on processes ranging from permafrost thaw to soil biogeochemical cycling to Earth system feedbacks associated with the unique characteristics of tundra plants. In Phase 4, the NGEE Arctic team is evaluating our new predictive understanding under novel conditions across the Arctic domain. In collaboration with partners at long-term pan-Arctic research sites we are examining whether an Arctic-informed ELM can faithfully simulate interactions among surface and subsurface processes at site, regional, and pan-Arctic scales. In turn, we are using variety of tools to dynamically extend and evaluate ELM inference, with an emphasis on data synthesis and pan-Arctic model evaluation, reintegration of code with an evolving E3SM, scaling across heterogeneous Arctic landscapes, and the appropriate representation of the impacts of increasingly frequent Arctic disturbances.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Automatic log analysis with NLP for the CMS workflow handling

The central Monte-Carlo production of the CMS experiment utilizes the WLCG infrastructure and manages daily thousands of tasks, each up to thousands of jobs. The distributed computing system is bound to sustain a certain rate of failures of various types, which are currently handled by computing operators a posteriori. Within the context of computing operations, and operation intelligence, we propose a Machine Learning technique to learn from the operators with a view to reduce the operational workload and delays. This work is in continuation of CMS work on operation intelligence to try and reach accurate predictions with Machine Learning. We present an approach to consider the log files of the workflows as regular text to leverage modern techniques from Natural Language Processing (NLP). In general, log files contain a substantial amount of text that is not human language. Therefore, different log parsing approaches are studied in order to map the log files’ words to high dimensional vectors. These vectors are then exploited as feature space to train a model that predicts the action that the operator has to take. This approach has the advantage that the information of the log files is extracted automatically and the format of the logs can be arbitrary. In this work the performance of the log file analysis with NLP is presented and compared to previous approaches.

Layer, Lukas↗
DOE OSTI2024

Characterization of Soil Thermal and Electrical Properties along Multiple Hillslope Transects at Teller Road Site, Seward Peninsula, Alaska, 2017

This dataset has been acquired along five-119 m long transects located on the bottom part of the watershed hillslope at the NGEE Arctic Teller Road site at mile marker 27 (TL_MM27) on the Seward Peninsula, Alaska in July and September 2017. The Distributed Temperature Profiling (DTP) system dataset consist in vertically-resolved profile of soil temperature covering the top 0.8 m of soil with 8 cm interval. In addition to DPT data, electrical resistivity tomography (ERT) data, soil moisture, depth to rock or thaw layer thickness (no differentiation) and ground elevations data have been acquired along each of the transects. A UAV-based geotiff mosaic of the investigated site is also provided. The four data types provided with this dataset of 37 files (*.csv, *.tif, *.DATA): (1) soil temperature profiles, (2) ERT data, (3) the physical measurements of the thaw layer, and (4) an orthomosaic GeoTIFF of the transect study area.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

FERC Order No. 2222 and Considerations for Distributed Wind

The Federal Energy Regulatory Commission (FERC) issued Order No. 2222 in October 2020. The rule directs Regional Transmission Organizations and Independent System Operators (ISOs) to amend their tariffs and participation models to accommodate heterogeneous distributed energy resource (DER) aggregations in the wholesale energy markets that they operate, including capacity, energy, and ancillary service markets. The Commission issued the rule to better capture the benefits provided by DERs deployed in the United States, whose use has been expanding rapidly. The Commission defines DERs as “any resource located on the distribution system, any subsystem thereof or behind a customer meter,” including but not limited to “electric storage resources, distributed generation, demand response, energy efficiency, thermal storage, and electric vehicles and their supply equipment.” The rule aims to increase DER participation in wholesale markets by allowing the creation of DER aggregations, in which multiple DERs that are too small to meet minimum capacity requirements for wholesale markets individually would be able to participate in markets as a single unit. As of June 20, 2023, all ISOs have filed initial compliance plans and a number have begun implementation. Compliance dates range from 2024 to 2029, with Midcontinent ISO having the latest date of compliance proposed for 2029. Southwest Power Pool still has an outstanding date, having no final order yet from FERC, but a target date of the third quarter of calendar year 2025. The rule, which is technology agnostic and requires ISOs to create participation plans that accommodate different DERs, provides an opportunity for distributed wind market expansion. In addition, distributed wind can bring benefits to heterogenous DER aggregations. These benefits include resource diversity (i.e., a complementary generation profile to other types of distributed generation), its small footprint and ability to be co-located with load, and its potential to provide frequency response, voltage support, and black start services, among other ancillary services. This report provides a status update on FERC Order No. 2222, the current state of ISO compliance, and information relevant to the distributed wind industry as DER aggregators and other stakeholders expand their participation to wholesale energy markets.

17 WIND ENERGY↗
DOE OSTI2024

A Framework for Addressing Hydropower Modeling Gaps in Electric Grid Planning and Operational Studies [Slides]

Realistic representation of hydropower in power system planning and operation studies is extremely important as it helps in avoiding under/over-estimation of the services the hydro power units can provide. A framework and tools have been developed to account for hydrological conditions and modify power system models accordingly. For example, HyDat – data and AI driven platform, that provides that data required for editing the power system model files Collaborative effort with V&R Energy. Also, development of POM based tools have been performed, a) Tool1: editing .raw files with realistic current and maximum hydrogeneration and redispatch of other units, and b) Tool2: Editing .dyr files to represent water head, current and maximum generation. Finally the framework, HASP framework combines HyDat and V&R energy tools to produce modified power system model files. Key Take-away points from contingency analysis studies include a) the number of critical contingencies increases as the water levels are reduced (70%>75%>80%), and b) both the extent and the quality of dynamic response of the Hydropower unit is different with varying water head levels.

13 HYDRO ENERGY↗
DOE OSTI2020

Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications

User-space HPC data services are emerging as an appealing alternative to traditional parallel file systems, because of their ability to be tailored to application needs while eliminating unnecessary overheads incurred by POSIX compliance. The High Energy Physics (HEP) community is progressively turning towards such services to enable high-throughput accesses under heavy concurrency to billions of event data produced by instruments and consumed by subsequent analysis workflows. Such services would benefit from the possibility to be rescaled up and down to adapt to changing workloads, as experimental campaigns progress, in order to optimize resource usage. This paper formalizes rescaling a distributed storage system as a multi objective optimization problem considering three criteria: load balance, data balance, and duration of the rescaling operation. We propose a heuristic for rapidly finding a good approximate solution, while allowing users to weight the criteria as needed. The heuristic is evaluated with Pufferscale, a new rescaling manager for microservice-based distributed storage systems. To validate our approach in a real-world ecosystem, we showcase the use of Pufferscale as a means to enable storage malleability in the HEPnOS storage system for HEP applications.

Distributed Storage System↗
DOE OSTI2022

Outdoor Test Bed Performance of a Power Line Sensor Using a Real-Time Event Simulator

This report summarizes the design and application of an outdoor power line sensor testbed (OPLST) with a real-time simulator and power meter to compare a potential transformer (PT) and current transformer (CT) versus an advanced line-post sensor. The OPLST was created to validate advanced medium-voltage (20/34.5 kV) outdoor power line sensors used in electrical distribution systems. Electrical utilities have installed metering and relay-protection transformers like PTs and CTs for several decades. The PTs/CTs are iron core measurement transformers based on the electromagnetic induction principle and provide reliable data in normal grid operation. However, new outdoor power line sensors (OPLS) using other technologies like voltage divider, Rogowski coil and optical principles have become available and may have favorable performance and costs compared to PTs/CTs. Therefore, the importance of testing these technologies with the PT/CT, to compare the measured phase voltage/current at different power grid scenarios is crucial to understand the performance of these new OPLS. For this study a G&W Electric Model CVS-36-O power line sensor was chosen as the OPLS to test with voltage/current signals. An OPAL-RT Technologies Model OP4510 real-time simulator and Schweitzer Engineering Laboratories Model SEL-735 power meter were installed with the 20/34.5 kV OPLST to compare the measured transient events collected from an advanced OPLS and the PT/CT. This system is installed at the Distributed Energy Communications and Control (DECC) lab, Oak Ridge National Laboratory (ORNL). The simulator generated different power grid scenarios (electrical faults, capacitor bank operation, service restoration, etc.), and its analog-output signals were connected to the voltage/current amplifiers that feed the 20/34.5 kV aerial cable loop through the PT/CT devices. Additional PT/CT devices were also wired with the medium voltage aerial cable loop to measure the phase current/voltage signals and server as references. After each test, common format for transient data exchange (COMTRADE) files were collected from the SEL-735 power meter and used to compare the performance of the OPLS with the PT/CT. The behavior of analog signals, harmonic components, total harmonic distortion and crest factors were assessed and found favorable for the G&W sensor as compared to the reference PT/CT.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

WHOLESCALE: Coordinates of wells at San Emidio, Nevada

This dataset includes position coordinates and elevation information for wells at the WHOLESCALE San Emidio project location. Well positions in the attached file are characterized by UTM coordinates (Easting, Northing) in meters, and WHOLESCALE coordinates (Easting, Northing) relative in meters from a chosen reference point. The elevation of the top and bottom of open intervals within each well (representing locations of perforated or open-hole sections) are measured in meters (positive) above WGS84 geoid (mean sea level). The WHOLESCALE acronym stands for Water & Hole Observations Leverage Effective Stress Calculations and Lessen Expenses. The goal of the WHOLESCALE project is to simulate the spatial distribution and temporal evolution of stress in the geothermal system at San Emidio in Nevada, United States.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Non-growing Season Plant Nutrient Uptake Controls Arctic Tundra Vegetation Composition: Modeling Archive

This Modeling Archive is in support of a NGEE-Arctic publication: Riley et al. (2021) “Non-growing season plant nutrient uptake controls Arctic tundra vegetation composition under future climate” (Riley et al., 2021). The dataset contains “ecosys” model outputs reported in Riley et al. (2021) that examines the role of non-growing season (NGS) nutrient dynamics on 21st century vegetation composition. The study highlighted the importance of these nutrient dynamics on the emergent plant functional type distributions, focusing on their role in shrub expansion. The study showed that ignoring NGS nutrient dynamics led to a dramatic under-prediction of shrub expansion by year 2100, motivating the need to include these processes in Earth System Models. Included are modeled net primary production (NPP) and nitrogen (N) uptake for two scenarios: (1) baseline and (2) no NGS N uptake. The output is provided (1) at 25 km resolution across the North American tundra; (2) for years 2012 through 2100; and (3) PFT-specific. Dataset contains 20 NetCDF files in a zip file and one pdf.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

End-of-Winter Snow Depth, Temperature, Density, and SWE Measurements at Kougarok Road Site, Seward Peninsula, Alaska, 2022

Measurements of end-of-winter snow properties were collected at the NGEE Arctic Kougarok Road Site at mile marker 64 (KG_MM64) from April 2 to April 5, 2022. This dataset contains three *.csv data files of spatially distributed values of snow depth, snow water equivalent (SWE), and snow temperature at the snow surface and at the bottom of the snowpack. Data were collected toward the end of the winter season during early April, when the snowpack would be near its maximum. A Snow-Hydro MagnaProbe (http://www.snowhydro.com/products/column2.html) was used to improve collection efficiency and enhance spatial coverage. A user guide is included as a *.pdf. This dataset is a continuation of the previous end-of-winter snow surveys conducted at the Kougarok Road Site in 2018 (Wilson et al., 2021; https://doi.org/10.5440/1593874).The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

The generation of gridded emissions data for CMIP6

Abstract. Spatially distributed anthropogenic and open burning emissions are fundamental data needed by Earth system models. We describe the methods used for generating gridded datasets produced for use by the modeling community, particularly for the Coupled Model Intercomparison Project Phase 6. The development of three sets of gridded data for historical open burning, historical anthropogenic, and future scenarios was coordinated to produce consistent data over 1750–2100. Historical data up to 2014 were provided with annual resolution and future scenario data in 10-year intervals. Emissions are provided on a sectoral basis, along with additional files for speciated non-methane volatile organic compounds (NMVOCs). An automated framework was developed to produce these datasets to ensure that they are reproducible and facilitate future improvements. We discuss the methodologies used to produce these data along with limitations and potential for future work.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Wide Bandgap Generation (WBGen): Developing the Future Wide Bandgap Power Electronics Engineering Workforce

This Final Technical Report (FTR) summarizes the work conducted at the Center for Power Electronics Systems (CPES) under the Wide-Bandgap Generation (WBGen) fellowship and traineeship program established by the Advanced Manufacturing and Technologies Office (AMMTO) of the U.S. Department of Energy Office of Energy Efficiency and Renewable Energy (EERE) at Virginia Tech, which had as main objective to train the next generation of U.S. citizen power engineers with wide-bandgap (WBG) power semiconductor expertise, with the intent to aid in fulfilling the future workforce needs in this field. The latter was deemed of strategic importance given the fast-paced growth observed—and predicted—in the demand of this technical expertise, whose practitioners have become the enablers and executioners of the electrification transformation process that not just the U.S., but the whole world, is currently undergoing as it seeks for more effective and efficient ways to use energy. As such, the WBGen program set forth to achieve its educational goals, which in addition sought to broaden the range of WBG-based power electronics by conducting research and development on high-efficiency grid apparatus and high-efficiency electrical power systems, and to also enhance the power engineering curriculum by formalizing WBG-oriented design procedures replacing existent yet now obsolete design procedures developed for Silicon (Si) based power electronics. This effort led CPES to spearhead the development of a new major within The Bradley Electrical and Computer Engineering (ECE) Department at Virginia Tech, namely Electronic Power and Energy Systems (EPES), which coalesced power electronics and power systems courses to provide undergraduate students with a strong formation in the power engineering field, while creating a pipeline of graduate students that could pursue the WBG-based curriculum and conduct research at CPES. In all, in what is considered a true success, eight of the twenty WBGen fellows that graduated program were recruited from the ECE department undergraduate cohort. The traineeship emphasized as well, from its beginning, the partnership with industry and national laboratories, which took advantage of the successful industry consortium at CPES that has historically been formed by 80–90 power and energy companies working in close collaboration with the center. This gave WBGen fellows the accessibility and possibility to conduct internships at partner facilities during the summer months, focused solely on the evaluation, testing, and adoption of WBG devices, which were many times tightly related to their respective research work and plans. In addition, WBGen fellows conducted their main research work within the confines of research programs at CPES conducted with these industry partners, providing them with a unique opportunity to develop not just their technical expertise—while advancing their knowledge, but to also learn and practice a slew of skills needed for their professional growth. As such, the fellows tackled a variety of WBG-related research topics, from device reliability and capability aspects as well as packaging and integration, encompassing the use of advanced materials and new structures, current sharing challenges, and insulation systems, to advanced gate-drivers with integrated sensors and protection mechanisms and active current- and voltage-based control, to optimized layouts seeking to maximize the switching and power processing performance of these devices, to power processing solutions adopting Gallium-Nitride (GaN) and Silicon-Carbide (SiC) power semiconductors for a variety of applications; including radiation-hardened converters for space dc distribution systems, direct three-phase ac-to-ac power converters for aerospace systems, dc-ac inverters for automotive traction drives, high-frequency isolated dc-dc battery chargers also for heavy transportation systems, and medium-voltage dc-dc and dc-ac converters for distribution systems and future power grids. The WBGen program ultimately graduated a total of 20 power engineers, all experts on WBG-based power electronics, awarding 18 M.S. and 2 PhD degrees in the process. These fellows, all U.S. citizens, allowed CPES to increase the number of citizens students to 33 % at the peak of the program, as the traineeship made possible the recruitment of talent with more attractive graduate research assistantship (GRA) contracts. Unfortunately, the present U.S. citizen enrollment at CPES has declined back to historic levels—approximately 10 %, as the regular GRA rates are not competitive enough when compared with entry-level industry jobs. In all, WBGen fellows published a total of 7 peer-reviewed journal articles, 44 papers at international technical conferences, made 42 presentations at international technical conferences, and filed 4 invention disclosures and patent applications, which have since then been granted by the U.S. Patent and Trademark Office (USPTO). Their contribution to CPES, Virginia Tech, the United States, and the world, has been significant, and continues to yield results thanks to the exemplary career that the fellows have initiated at many of the partners of the program, which include Wolfspeed, Raytheon Technologies, Infineon, Lockheed Martin, Dominion Energy, Northrop Grumman, Rivian, Aerospace Corporation, Sandia National Laboratory, National Renewable Energy Laboratory, John Hopkins University, and Virginia Tech.

14 SOLAR ENERGY↗
DOE OSTI2023

End-of-Winter Snow Depth, Temperature, Density, and SWE Measurements at Teller Road Site, Seward Peninsula, Alaska, 2022

Measurements of end-of-winter snow properties were collected at the NGEE Arctic Teller Road Site at mile marker 27 (TL_MM27) from March 28 to April 1, 2022. This dataset contains three *.csv data files of spatially distributed values of snow depth, snow water equivalent (SWE), and temperature at the snow surface and at the bottom of the snowpack. Data were collected toward the end of the winter season from late March to early April, when the snowpack would be near its maximum. A Snow-Hydro MagnaProbe (http://www.snowhydro.com/products/column2.html) was used to improve collection efficiency and enhance spatial coverage. A user guide included as a *.pdf. This dataset is a continuation of the previous end-of-winter snow surveys conducted at the Teller Road Site in 2016-2018 (Wilson et al., 2020; https://doi.org/10.5440/1592103) and 2019 (Bennett et al., 2021; https://doi.org/10.5440/1798170). The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Using Parameter Sweep in WaterTAP to Analyze New Water Treatment Technologies

We describe a powerful and generalized parameter sweep tool in this report that was originally developed to analyze the performance of existing and novel water treatment models being developed in WaterTAP. Since WaterTAP is built upon IDAES and Pyomo, the parameter sweep tool can be used to systematically explore and debug the behavior of most Pyomo and IDAES numerical models. In order to enable meaningful analyses, the parameter sweep tool has been designed with the following features: 1) Model flexibility: The parameter sweep tool does not enforce any restrictions on the types of models that can be used with it. As long as a Pyomo model can be solved and the parameter is active and mutable, the tool only needs functions that describe how to run the model, the sweep parameters, and the output quantities of interest. 2) Flexible sampling: The parameter sweep tool has inbuilt functions to generate samples from a random distribution or a multidimensional Euclidean space. Furthermore, the users have to ability to supply samples generated from a tool of their choice. 3) Multiple sweep types: A user can choose from one of 3 types of parameter sweeps depending on their needs. 4) Detailed outputs: Outputs generated by the parameter sweep tool can be stored in detailed H5 file or user-friendly CSV files for post processing. 5) Parallel computing: The parameter sweep supports shared and distributed memory parallel computing to enable the use of high performance computers (HPC) for large-scale analyses. 6) Modular: The parameter sweep tool is self-contained and can easily be integrated within an outer-loop analysis or as desired by the user. 7) Ease of use: The tool is well documented and a simple sweep can be easily executed by following the online documentation in a few lines of code. We demonstrate the use of the parameter sweep tool on a simple water treatment system from the WaterTAP repository and show its parallel scaling performance on an Apple laptop and NREL's Eagle HPC. The parameter sweep tool is actively being used with models currently being developed within WaterTAP and we expect its use to grow beyond it to other IDAES and Pyomo models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

OCTOKV: An Agile Network-Based Key-Value Storage System with Robust Load Orchestration

In this paper, we propose OctoKV, an innovative network-based key-value storage system. OctoKV addresses the repetitive address translation overhead associated with traditional key-value stores running on file systems on the client side. To mitigate this overhead, we implemented the key-value store on the server side using NVMe-oF and a user-level NVMe driver. In particular, we employed fine-grained resource monitoring and load balancing based on heuristics to optimize I/O performance. OctoKV is deployed on a Linux cluster with Intel SPDK. The extensive evaluation shows that OctoKV achieves lower I/O response times in comparison to traditional approaches where key-value stores run on the client side. Also, the proposed load balancing strategies efficiently enhance I/O response times by equally distributing the workload from overloaded cores to other cores.

Khan, Awais↗
DOE OSTI2026

Tussock tundra surface temperatures, ambient air and incoming photosynthetically active radiation measured at the NGEE Arctic Council site, 2021 - 2023

This dataset contains temperature measurements carried out along two fiber optics cables/lines (150 m each) laid out along the ground at the Next Generation Ecosystem Experiment (NGEE) Arctic site near Council, Alaska. The lines traverse an heterogeneous part of the tussock tundra site including thermokarst features and lichen dominated sections of the tundra. Measurements were done using a Sensornet Oryx DTS, a Distributed Temperature sensor that was installed in September 2021 and taken down in August 2023. The sensor was powered by solar power with data being collected every 30 minutes at 1 m resolution. In addition to these measurements air temperature and incoming photosynthetically active radiation (PAR) are provided. These measurements are co-located with the NGEE Arctic Council eddy flux and meteorological station (AmeriFlux ID US-NGC). Included are six *.csv files (four data files and two reporting format files) and two *.kml files. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

Air temperature↗
DOE OSTI2026

Detector Interface for Streaming, Control, and Open-source integration (DISCO) v1.0.0

This suite consists of a multi-package ecosystem featuring detector emulators, EPICS areaDetector drivers, and remote server frameworks designed for the Advanced Light Source (ALS). Engineered for high-bandwidth devices—including VFCCD, Timepix3, Timepix4, and related pixel detectors—the software simulates hardware, wraps vendor SDKs into remote-callable servers, and integrates with open-source control systems. Key Capabilities: Distributed SDK Architecture: Server packages wrap hardware-specific SDKs, allowing areaDetector drivers to execute remote framework calls. This isolates proprietary libraries from the EPICS IOC, enhancing stability and enabling distributed computing across beamline networks. Device Support: Custom drivers for VFCCD, the Timepix family, and similar sensors optimize the data path from hardware control to high-speed transport. Full-Stack Emulation: Sophisticated emulator packages allow end-to-end pipeline testing and software development without requiring physical hardware or beam time. Integrated Workflows: Supports high-bandwidth streaming for real-time analysis and robust, metadata-rich file-based workflows (e.g., HDF5/NeXus). By standardizing interfaces across heterogeneous hardware, this suite reduces technical debt. It provides the ALS with a scalable, open-source solution to manage massive data rates within a unified control environment.

Mahl, Johannes [Lawrence Berkeley National Laborat↗