Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Effectiveness evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48

Epigenetic Footprints of CRISPR/Cas9-Mediated Genome Editing in Plants

CRISPR/Cas9 has been widely applied to various plant species accelerating the pace of plant genome editing and precision breeding in crops. Unintended effects beyond off-target nucleotide mutations are still somewhat unexplored. We investigated the degree and patterns of epigenetic changes after gene editing. We examined changes in DNA methylation in genome-edited promoters of naturally hypermethylated genes (AT1G72350 and AT1G09970) and hypomethylated genes (AT3G17320 and AT5G28770) from Arabidopsis. Transgenic plants were developed via Agrobacterium-mediated floral dip transformation. Homozygous edited lines were selected from segregated T2 plants by an in vitro digestion assay using ribonucleoprotein complex. Bisulfite sequencing comparisons were made between paired groups of edited and non-edited plants to identify changes in DNA methylation of the targeted loci. We found that directed mutagenesis via CRISPR/Cas9 resulted in no unintended morphological or epigenetic alterations. Phenotypes of wild-type, transgenic empty vector, and transgenic edited plants were similar. Epigenetic profiles revealed that methylation patterns of promoter regions flanking target sequences were identical among wild-type, transgenic empty vector, and transgenic edited plants. There was no effect of mutation type on epigenetic status. We also evaluated off-target mutagenesis effects in the edited plants. Potential off-target sites containing up to 4-bp mismatch of each target were sequenced. No off-target mutations were detected in candidate sites. Our results showed that CRISPR/Cas9 did not leave an epigenetic footprint on either the immediate gene-edited DNA and flanking DNA or introduce off-target mutations.

59 BASIC BIOLOGICAL SCIENCES↗

Safety in Artificial Intelligence: Challenges and Opportunities for the U.S. National Labs and Beyond

This report discusses the importance of the critical and underexplored topic of artificial intelligence (AI) safety, as highlighted during the “Strategy Alignment on AI Safety” workshop convened at Lawrence Livermore National Laboratory (LLNL) in April 2024. Through a summary of keynote talks, panel discussions, and breakout sessions, world-leading AI safety experts from academic, industry, national labs, and government agencies clearly agree on the need for and importance of large-scale investments for research and capabilities in AI safety. With the field innovating at unprecedented rates, there is increasing urgency to develop novel evaluation methodologies that allow full considerations of risks/threats of AI technologies in different domains. Quantitative metrics and effective methodologies that can evaluate and audit the “safeness” of how a given AI technology is trained, deployed, or regulated are, at best, nascent for certain scenarios or, more commonly, nonexistent. This maturation gap presents the possibility of serious threats to national security, and further inaction may have serious consequences. Additionally, the gap between the public’s and research community’s perceptions of AI risks/rewards is significant. While numerous voices from the AI community have expressed concern that the risks could be so high that future AI systems could inflict extinction-level damage to humanity if deployed incorrectly, the public largely is aware only of risk in low-impact scenarios. This discrepancy highlights the crucial need for researchers to articulate to governmental bodies what, why, and when various AI risks matter as part of motivating funding requests. Thus, the call to action for this community is to pursue AI safety as a “Big Science” project on a scale comparable to the Manhattan Project. High risks and high payoffs are on the table, but safe AI is a fast-moving target, and large-scale investments are needed to guide development of this technology in a responsible way. We highlight the need for a multilayered solution combining the development of new methods and algorithmic approaches to mitigate threats with an active participation of the government(s) in setting high industry standards and regulations based on state-of-the-art technology. The U.S. Department of Energy (DOE) national laboratories have served as leading institutions for scientific innovation in the U.S. for more than 70 years. Drawing on their expertise in the AI community and their history of safeguarding critical and sensitive information, and as we look to the future, national labs are the best choice for evaluating and safeguarding AI technologies.

97 MATHEMATICS AND COMPUTING↗

A Novel Spatial-Temporal Variational Quantum Circuit to Enable Deep Learning on NISQ Devices

Quantum computing presents a promising approach for machine learning with its capability for extremely parallel computation in high-dimension through superposition and entanglement. Despite its potential, existing quantum learning algorithms, such as Variational Quantum Circuits (VQCs), face challenges in handling more complex datasets, particularly those that are not linearly separable. What’s more, it encounters the deployability issue, making the learning models suffer a drastic accuracy drop after deploying them to the actual quantum devices. To overcome these limitations, this paper proposes a novel spatial-temporal design, namely “ST-VQC”, to integrate nonlinearity in quantum learning and improve the robustness of the learning model to noise. Specifically, ST-VQC can extract spatial features via a novel block-based encoding quantum sub-circuit coupled with a layer-wise computation quantum sub-circuit to enable temporal-wise deep learning. Additionally, a SWAP-Free physical circuit design is devised to improve robustness. These designs bring a number of hyperparameters. After a systematic analysis of the design space for each design component, an automated optimization framework is proposed to generate the ST-VQC quantum circuit. The proposed ST-VQC has been evaluated on two IBM quantum processors, ibm-cairo with 27 qubits and ibmq-lima with 7 qubits to assess its effectiveness. The results of the evaluation on the standard dataset for binary classification show that ST-VQC can achieve over 30% accuracy improvement compared with existing VQCs on actual quantum computers. Moreover, on a non-linear synthetic dataset, the STVQC outperforms a linear classifier by 27.9%, while the linear classifier using classical computing outperforms the existing VQC by 15.58%.

Li, Jinyang↗

Development of Stable Solid Oxide Electrolysis Cells for Low-Cost Hydrogen Production

The project objective was to demonstrate a solid oxide cell-based steam electrolysis stack that exhibits robustness, reliability, endurance, hydrogen purity, and produces hydrogen at elevated pressure of 2 to 3 bar. Innovative materials and processing methods were evaluated to improve degradation characteristics. Performance improvement focused on nearly all layers involved in the cell and stack assembly. Primary attention was paid to zirconia-ceria interface resistance control via sintering optimization and decrease in degradation from the oxygen electrode by evaluating low strontium (Sr) or Sr-free composition for both the oxygen electrode and current collection layer. Stack robustness was addressed by validating redox tolerance of fuel electrode, confirming capability of cells to survive repeated thermal cycles, studying the effect of pressure on performance and degradation, evaluating the effect of contamination on fuel and oxygen electrode performance and degradation, and identifying mitigation strategies to improve performance. The characterization included evaluation of electrochemical performance and stability followed by microstructural analysis. At the cell level, performance and stability improvements were achieved by incorporating a Sr-free oxygen electrode and a denser oxygen electrode barrier layer. At the stack level, pressurized operation reduces demand on first stage compression, the redox tolerant fuel electrode mitigates risk from service interruptions, and improvements to interconnect coating alleviate chromium (Cr) contamination effects. The denser barrier layer was achieved by adding a sintering aid to the samaria-doped ceria (SDC) composition that reduced sintering temperature by 150 °C. The resulting density was on par with the baseline SDC barrier layer density and the lower sintering temperature resulted in less resistive phase formation during sintering. Button cell tests did not demonstrate a change in performance when exposed to silicon (Si) or manganese (Mn) impurities to the fuel electrode and Cr impurity to the oxygen electrode. More detailed study however is warranted. The project addressed SOEC performance and stability at the cell and stack levels through a systematic approach to known sources of degradation that were combined and tested in three stack tests using an electrolyte supported cell design to allow for evaluation of a variety of fuel and oxygen electrode compositions. STK-82 and STK-83 had identical compositions. STK-100 incorporated the best materials and processing variables developed under this and concurrent projects, and was tested at elevated pressure in steam electrolysis. • STK-82 recovered performance after redox and thermal cycling, demonstrating the robustness of the stack and seals. It exhibited stable performance in testing for 500 hours in SOEC mode, followed by 300 hours of cycling between SOEC and SOFC tests. Degradation during SOEC operation was 1.8 %/ 1,000 hours. • STK-83 generated hydrogen at >80% steam conversion, and oxygen above 98.5 % purity during pressurized operation. Both hydrogen and oxygen were generated at 3 barg pressure without the use of a pressure vessel. In addition to balanced pressure, electrolysis operation at 1 bar differential pressure across anode and cathode was also demonstrated to substantial the robustness of the cell and seal. • STK-100 measured at initial ambient pressure conditions showed an area specific resistance of 1.1 ohm-cm 2 , and STK-83 had 1.3 ohm-cm 2 .

08 HYDROGEN↗

Analyzing the Impact of Future Weather Data on Energy Consumption in Weatherization Assistant

This study supports the mission of the U.S. Department of Energy’s Weatherization Assistance Program (WAP), which aims to increase the energy efficiency of dwellings and reduce their total residential expenditures. Specifically, we examine how projected future climate conditions may affect residential building energy performance by integrating future weather data into the National Energy Audit Tool (NEAT). Since WAP evaluates the cost-effectiveness of retrofit measures over lifespans of up to 30 years, accounting for evolving climate conditions is increasingly important. To reflect future household energy demands, this study replaces historically based Typical Meteorological Year (TMY3) weather inputs with Future Typical Meteorological Year (fTMY) datasets derived from global climate model (GCM) projections. A simulation-based framework was established to enable NEAT analysis under future weather conditions. This workflow involves converting EPW-format weather files into JSON inputs compatible with NEAT and generating degree-hour metrics needed for load calculations. The fTMY dataset used in this study was developed by Oak Ridge National Laboratory through downscaling of six GCMs under different emission scenarios and covers the period from 2020 to 2100. In contrast, the TMY3 dataset is based on historical weather data from 1961 to 1990. Simulations were conducted for benchmark single-family prototype buildings across ASHRAE climate zones 1–7, which cover all regions of the U.S. except the subarctic Zone 8 in northern Alaska, evaluating both heating and cooling loads under TMY3 and fTMY conditions. Four foundation types were tested, while heating systems were standardized, as NEAT does not differentiate thermal energy load by HVAC system type in its load calculations. Results show that fTMY weather input consistently yield lower heating loads and higher cooling loads across most locations, aligning with expected climate warming trends. Notably, colder regions such as zones 6A, 6B, and 7 experience marked reductions in heating load, while warmer and transitional zones, such as 2A (Lufkin, TX) and 3C (San Francisco, CA), have substantial increases in cooling loads. Although this study does not directly assess the performance of retrofit measures under future climate conditions, it provides a critical foundation for doing so. By quantifying shifts in baseline (i.e., pre-retrofit case) energy loads between historical and future weather files, the study highlights the importance of integrating climate-responsive data into audit tools. These findings will inform future efforts to evaluate the long-term effectiveness and cost-effectiveness of weatherization measures under changing climate conditions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

On the Uranium and Plutonium Nuclear Data Evaluations [Slides]

This presentation provides overviews of uraniums, 233 U and 235 U, plutonium research including motivations, current status in ENDF/B-VIII.0, the inclusion of sub-thermal data, and the inclusion of LANL ratio capture-to-fission data. Additionally, the preliminary fit of Mosby’s data as reported, fluctuating neutron multiplicities, uncertainty in evaluated libraries, and experimental effects are also presented. The presentation concludes by providing a summary of plans for the U and Pu evaluation work.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Toward practical issues: Identification and mitigation of the impurity effect in glyme solvents on the reversibility of Mg plating/stripping in Mg batteries

Reversible electrochemical magnesium plating/stripping processes are important for the development of high-energy-density Mg batteries based on Mg anodes. Ether glyme solutions such as monoglyme (G1), diglyme (G2), and triglyme (G3) with the MgTFSI 2 salt are one of the conventional and commonly used electrolytes that can obtain the reversible behavior of Mg electrodes. However, the electrolyte cathodic efficiency is argued to be limited due to the enormous parasitic reductive decomposition and passivation, which is governed by impurities. In this work, a systematic identification of the impurities in these systems and their effect on the Mg deposition–dissolution processes is reported. The mitigation methods generally used for eliminating impurities are evaluated, and their beneficial effects on the improved reactivity are also discussed. By comparing the performances, we proposed a necessary conditioning protocol that can be easy to handle and much safer toward the practical application of MgTFSI 2 /glyme electrolytes containing impurities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Experimental and Analytical Verification of ASME Section IiII Division 5 Creep-Fatigue Design Rules

The continuous advancement of structural materials and the growing demands for more reliable and economical structural components in high-temperature reactor applications have necessitated the development of comprehensive design methodologies and design rules. Mechanical degradation of structural components at elevated temperatures subjected to cyclic deformation is controlled by the creep-fatigue damage. Over the past few decades, diligent research efforts have been dedicated to refining the development of elevated temperature design rules in the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Code (BPVC), Section III, Division 5 and to develop conservative design rules that can effectively guard against the risk of creep-fatigue failure. In ASME Section III, Division 5, for a design to pass the creep-fatigue acceptance criteria, creep damage and fatigue damage are evaluated separately, and these damages must not violate the bi-linear creep-fatigue interaction diagram, i.e., the so-called D-diagram. The creep-fatigue damage evaluation procedure assumes that the effects of the actual cyclic loading sequence can be bounded by assuming that the individual loading cycles are uniformly distributed throughout the component design life. In this study, creep-fatigue experiments with variable amplitudes and loading sequencies were designed and performed on Alloy 617 at high temperatures. The results were analyzed to evaluate the loading history effect on creep-fatigue damage accumulation and to verify the assumptions for the creep-fatigue evaluation design rules.

36 - MATERIALS SCIENCE↗

Data Quality Analysis and Solutions for Distribution-Level PMUs

Synchrophasors provide promising enhancement to the distribution system monitoring. However, different from transmission systems, distribution systems are usually with higher noise, phase unbalance, and distributed energy resources. All of these could impair the data quality of distribution-level phasor measurement units (D-PMUs) and thus deteriorate the performance of D-PMU based applications. Therefore, the data quality of D-PMU should be evaluated to ensure the effectiveness of D-PMU based applications. In this paper, factors in distribution systems that could jeopardize D-PMU data quality are evaluated. The impact of data quality on D-PMU applications is analyzed. Finally, solutions are proposed to improve D-PMU data quality to satisfy these requirements. This work provides effective methods to improve the performance of distribution-level grid monitoring using synchrophasor measurements.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Radiative Corrections to Superallowed β Decays in Effective Field Theory

The accuracy of V u d determinations from superallowed β decays critically hinges on control over radiative corrections. Recently, substantial progress has been made on the single-nucleon, universal corrections, while nucleus-dependent effects, typically parametrized by a quantity δ NS , are much less well constrained. Here, we lay out a program to evaluate this correction from effective field theory (EFT), highlighting the dominant terms as predicted by the EFT power counting. Moreover, we compare the results to a dispersive representation of δ NS and show that the expected momentum scaling applies even in the case of low-lying intermediate states. Our EFT framework paves the way toward calculations of δ NS and thereby addresses the dominant uncertainty in V u d . Published by the American Physical Society 2024

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Combustion characteristics of diisopropoxymethane, a low-reactivity oxymethylene ether

Oxymethylene ethers (OMEs) have been studied for use as low-sooting diesel fuel additives or substitutes; very little literature discusses OMEs as spark-ignition (SI) fuels due to their typically high cetane numbers. In this work, a lower-reactivity, branched OME, diisopropoxymethane (DIPM), is evaluated to determine its effectiveness as a spark-ignition fuel, as it is the lowest-reactivity OME (as determined by Indicated Cetane Number) thus far evaluated in the literature. DIPM is synthesized in-house via acetalization from isopropanol (iPrOH) and trioxane using standard OME production practices. DIPM was then tested in a rapid compression machine (RCM) for autoignition and spark ignition characteristics, and in a modified CFR engine to determine effective octane numbers. In the RCM, an autoignition temperature sweep was performed at stoichiometric conditions from 1000/T = 1.7 - 1.0, at 5:1 inert ratio (comparable to approximately 25% EGR), where it was found that DIPM has ignition delay times 5–10x faster than isooctane and displays NTC ignition behavior. Blends with iPrOH indicate that reactivity can be matched with isooctane with low blend ratios of iPrOH in DIPM. Flame speeds were tested with a laser spark for ignition in the RCM, where the flame speed of DIPM and isooctane is determined to be comparable at engine relevant conditions. In the CFR engine, effective RON and MON based on pressure trace frequency domain measurements were determined for DIPM and a 15 vol% iPrOH in DIPM blend. Neat DIPM has (R+M)/2 = 59 and a negative sensitivity of S = -18, consistent with its NTC behavior and higher reactivity. Furthermore the DIPM/iPrOH blend has positive sensitivity and a pump-gasoline range (R+M)/2 = 89.3. DIPM on its own is unlikely to be an effective SI fuel, however, when blended with iPrOH as an ON booster, it may be a promising SI candidate fuel.

09 BIOMASS FUELS↗

A theoretical calculation of stacking fault energy of Ni alloys: The effects of temperature and composition

Combining cluster expansion (CE) method with one dimensional axial Ising model, this work investigated the effects of alloying elements and configurational variations due to temperature on the stacking fault energy (SFE) of FCC Ni binary alloys. Ensembles of large numbers of atomistic structures, each with more than ~400 atoms, were generated to consider sufficient long-range chemical disorder and temperature effects due to configurational entropy. Here, a Monte Carlo Metropolis algorithm was used to generate these structures, whose energies were then evaluated based on the effective cluster interactions obtained from CE. As a baseline, this work had shown the SFE of pure Ni and Al to be 127 mJ/m 2 and 137 mJ/m 2 , respectively, which agreed with the experimental values of 125 mJ/m 2 and 150 mJ/m 2 reported in the literature. Additions of Al, Ti, Cr and Co to pure Ni were found to decrease the SFE to different extents. Although temperature does not strongly influence the SFE of the FCC Ni-Al and Ni-Cr binary alloys, it can lead to significant changes to the SFE of the FCC Ni-Ti and Ni-Co alloys. While effects of temperature and composition on SFE observed in this work were calculated from binary Ni alloys, the general trends are nonetheless expected to be valid in the γ phase of multicomponent Ni superalloys.

36 MATERIALS SCIENCE↗

Report on FY 2020 Testing in Support of the Development and Extension of ASME Code Rules for Grade 91

This report summarizes the FY 2020 research and activities at Oak Ridge National Laboratory in support of the ASME code rule development and code extension for Grade 91 (Gr. 91) steel. Incorporation of the Gr. 91 temperature-dependent fatigue curves in Section III Division 5 was one major task, and the associated major aspects brought up during the course of the ASME balloting process are documented in this report. Experiments and analysis continued on the development of the alternative creep-fatigue evaluation based on elastic–perfectly plastic (EPP) plus simplified model test (SMT) methodology. The focus of the study in FY 2020 was the evaluation of the hold-time effect at high cycle and low strain range region. Baseline testing in support of the evaluation of hold time effects was started and will continue in FY 2021. Major on-going experiments and activities in support of the ASME code development and extension of Gr. 91 will be completed in FY 2021.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

The Early Career Research Program Annual Progress Report: Using ARM Data and Multiscale Models to Advance the Understanding of Liquid-Phase Cloud Response to Aerosol Perturbation over Ocean and Land

This project aims to study the response of liquid-phase clouds to aerosol perturbation and reduce the related uncertainty in the DOE Earth system model. Specific objectives include: Objective 1: Quantify the aerosol indirect effect on liquid-phase clouds using ARM observations and satellite retrievals while constraining the confounding large-scale conditions; Objective 2: Harness LES case studies, untangle the confounding meteorological impacts on the observed aerosol indirect effect, and identify key cloud processes controlling the net radiative forcing of aerosol-cloud interactions at different spatiotemporal scales; Objective 3: Evaluate the aerosol indirect effect on liquid-phase clouds with process-level understanding in E3SM multiscale model configurations.

54 ENVIRONMENTAL SCIENCES↗

Pavement condition and climatic data in southeast Texas: A dataset for evaluating flood impacts on pavement performance

Effective pavement maintenance is essential for economic stability, optimal network performance, and roadway safety. Achieving this requires thorough evaluation of pavement conditions, including structural integrity, surface roughness, and distress characteristics. Pavement performance indicators play a critical role in influencing vehicle safety and ride quality. Recent advances have emphasized the use of data-driven modeling to anticipate pavement behavior, with the goal of optimizing resource allocation and refining Maintenance and Rehabilitation (M&R) strategies through accurate condition assessment. A foundational requirement for these modeling efforts is the availability of standardized, high-quality datasets that can support robust and reproducible infrastructure analysis. This data article presents a comprehensive dataset assembled to facilitate pavement performance prediction, with a geographic focus on Southeast Texas, particularly the flood-vulnerable area of Beaumont. The dataset encompasses pavement and traffic attributes, meteorological records, flood simulation outputs, ground deformation measurements, and topographic indices, enabling detailed examination of both load-associated and non-load-associated degradation mechanisms. Data preprocessing was performed using ArcGIS Pro, Microsoft Excel, and Python to ensure consistency and usability in data-driven modeling applications, including machine learning workflows. Key contributions of this dataset include its utility in analyzing the climatic and environmental factors affecting pavement conditions, identifying critical predictive features, and enabling in-depth correlation analysis across diverse variables. By filling existing gaps in input variable selection resources, this dataset supports the development of predictive tools for estimating future maintenance demand and enhancing the resilience of pavement networks in flood-impacted areas. The resource highlights the importance of standardized datasets for advancing pavement management practices and provides a robust foundation for ongoing infrastructure performance modeling.

42 ENGINEERING↗

Applying cumulative effects to strategically advance large-scale ecosystem restoration

Programs restoring large terrestrial-aquatic ecosystems are expected to increase in area and number over the next decade. Factors ensuring long-term effectiveness at the program scale differ from those at the individual restoration project scale, necessitating a new synthesis. Here we contextualize hydrological, geomorphological, and biological effects of large-scale ecosystem restoration activities within a cumulative-effects paradigm. Evidence of eight modes of cumulative effects benefiting species and ecosystems was elucidated by reviewing restoration efforts in the greater Florida Everglades, Gulf Coast, lower Columbia River and estuary, Puget Sound estuary, Missouri River, northeastern coastal states, San Francisco Bay and Sacramento Delta. Additive and nonlinear cumulative effects of interacting restoration projects propagated across landscape and regional geographic scales, and throughout trophic levels, affecting population structures. Applying a cumulative-effects paradigm to planning, implementing, and evaluating large-scale restoration will avoid countervailing effects and capitalize on positive feedback, advancing evidence-based programs to recover priority species and ecosystems.

54 ENVIRONMENTAL SCIENCES↗

Integrating Experiments and Well Logs to Predict Caney Shale Static Mechanical Properties During Production with Supervised Machine Learning

Caney shale is one of the emerging oil reservoirs in Oklahoma. Understanding the impact of effective stress on its mechanical properties is critical for predicting hydraulic fracture geometry and overall hydrocarbon production. The objective of our study is to evaluate the impact of effective stress on the dynamic Young’s modulus using ultrasonic velocity measurements for Caney shale samples. A triaxial cell was utilized to measure ultrasonic (P-wave and S-wave) velocities for ten downhole Caney shale samples under various effective stresses to indirectly assess the impact of pore pressure change. The dynamic Young’s moduli estimated from these measurements were integrated with available conventional well logs (excluding sonic logs) and triaxial test results from Benge et al. (2021) to predict the static Young’s modulus using Random Forest (RF) and Extreme Gradient Boosting (XGBoost) models. The results showed that the estimated dynamic Young’s moduli from ultrasonic measurements were higher than the corresponding static Young’s modulus of cores from the same vertical well at similar depths. With increasing effective stress, the dynamic Young’s modulus increased for all samples. The estimated dynamic-to-static correction factor tended to be higher in zones of high neutron porosity (PHIN) and low density compared to other zones. Finally, SHapley Additive exPlanations (SHAP) for RF and XGBoost models identified depth, gamma ray (GR), and PHIN as key features for predicting the static Young’s modulus. This study enhances our understanding of the dynamic and static Young’s moduli for the Caney shale interval, as a function of effective stress and conventional well logs. The findings from this study can improve predictions of production throughout the well's lifespan by offering insights into the mechanical property degradation resulting from pore pressure depletion.

Kholy, Sherif M.↗

A globally sampled high-resolution hand-labeled validation dataset for evaluating surface water extent maps

Effective monitoring of global water resources is increasingly critical due to climate change and population growth. Advancements in remote sensing technology, specifically in spatial, spectral, and temporal resolutions, are revolutionizing water resource monitoring, leading to more frequent and high-quality surface water extent maps using various techniques such as traditional image processing and machine learning algorithms. However, satellite imagery datasets contain trade-offs that result in inconsistencies in performance, such as disparities in measurement principles between optical (e.g., Sentinel-2) and radar (e.g., Sentinel-1) sensors and differences in spatial and spectral resolutions among optical sensors. Therefore, developing accurate and robust surface water mapping solutions requires independent validations from multiple datasets to identify potential biases within the imagery and algorithms. However, high-quality validation datasets are expensive to build, and few contain information on water resources. For this purpose, we introduce a globally sampled, high-spatial-resolution dataset labeled using 3 m PlanetScope imagery. Our surface water extent dataset comprises 100 images, each with a size of 1024×1024 pixels, which were sampled using a stratified random sampling strategy covering all 14 biomes. We highlighted urban and rural regions, lakes, and rivers, including braided rivers and coastal regions. We evaluated two surface water extent mapping methods using our dataset – Dynamic World, based on Sentinel-2, and the NASA IMPACT model, based on Sentinel-1. Dynamic World achieved a mean intersection over union (IoU) of 72.16 % and F1 score of 79.70 %, while the NASA IMPACT model had a mean IoU of 57.61 % and F1 score of 65.79 %. Performance varied substantially across biomes, highlighting the importance of evaluating models on diverse landscapes to assess their generalizability and robustness. Our dataset can be used to analyze satellite products and methods, providing insights into their advantages and drawbacks. Our dataset offers a unique tool for analyzing satellite products, aiding the development of more accurate and robust surface water monitoring solutions. The dataset can be accessed via https://doi.org/10.25739/03nt-4f29.

54 ENVIRONMENTAL SCIENCES↗