Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “impact monitoring”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Evaluation of a coastal acoustic buoy for cetacean detections, bearing accuracy and exclusion zone monitoring

Abstract There is strong socio‐political support for offshore wind development in US territorial waters and construction is planned off several east coast states. Some of the planned development sites coincide with important habitat for critically endangered North Atlantic right whales. Both exclusion zones and passive acoustic monitoring are important tools for managing interactions between marine mammals and human activities. Understanding where animals are with respect to exclusion zones is important to avoid costly construction delays while minimizing the potential for negative impacts. Impact piling from construction of hundreds of offshore wind turbines likely require exclusion zones as large as 10 km. We have developed a three‐hydrophone passive acoustic monitoring system that provides bearing information along with marine mammal detections to allow for informed management decisions in real‐time. Multiple units form a monitoring system designed to determine whether marine mammal calls originate from inside or outside of an exclusion zone. In October 2021, we undertook a full system validation, with a focus on evaluating the detection range and bearing accuracy of the system with respect to right whale upcalls. Five units were deployed in Mid‐Atlantic waters and we played more than 3500 simulated right whale upcalls at known locations to characterize the detection function and bearing accuracy of each unit. The modelled results of the detection function error were then used to compare the effectiveness of a bearing‐based system to a single sensor that can only detect a signal but not ascertain directivity. Field trials indicated maximum detection ranges from 4–7.3 km depending on source and ambient noise levels. Simulations showed that incorporating bearing detections provide a substantial improvement in false alarm rates (6 to 12 times depending on number of units, placement and signal to noise conditions) for a small increase in the risk of missed detections inside of an exclusion zone (1%–3%). We show that the system can be used for monitoring exclusion zones and clearly highlight the value of including bearing estimation into exclusion zone monitoring plans while noting that placement and configuration of units should reflect anticipated ambient noise conditions.

17 WIND ENERGY↗
DOE OSTI2022

Characterizing Plug Load Energy Use and Savings Potential in Army Buildings

The Assistant Secretary of the Army (Installations, Energy and Environment) tasked the Pacific Northwest National Laboratory to examine plug loads in typical Army buildings. Plug loads (also known as miscellaneous electric loads (MELs)) represent the electricity used by appliances and devices that are plugged in or hardwired and serve functions outside of a building’s core end uses. Common plug loads include computers, printers, copiers, networking devices, refrigerators, and vending machines. They also include personal electronic devices such as televisions, smart phones, tablets, and gaming systems. Examples of hardwired MELs include elevators, air compressors, and fire and security systems. The findings from this study confirm that significant energy is consumed within Army buildings by plug load devices and hardwired MEL equipment. A number of opportunities are identified for reducing unnecessary energy use that could save the Army over $5 million per year when broadly applied. Army regulations clearly spell out expectations for the purchase and operation of information technology equipment (computers, laptops, monitors, printers, and multi-function devices). However, the policies regarding the shutdown or activation of sleep and other lower power modes after 30 minutes of inactivity (15 minutes for monitors) do not appear to be consistently followed. There are many effective approaches and pathways for impacting change as it relates to improving awareness, implementing measures, and adjusting behaviors to identify and reduce plug load energy use. The Army should prioritize and consider deploying all of these to better understand and manage plug load equipment to save energy and enhance resilience across their facilities. Engaging the building occupants who use these devices daily via outreach and education should be a strong component of the strategy. The focus should be on reducing waste without sacrificing productivity or the benefits that many of these devices provide. Continued evaluation of plug loads beyond that performed here is important to gather lessons from additional building and equipment types, and to stay aware of evolving device technology and management options. This will highlight additional needs for policies, best practices, control technologies, and education of personnel to achieve real reductions in energy waste from plug load equipment. It is recommended that this study may serve as the foundation for a broader and sustained focus on plug loads and MELs, towards simultaneously enhancing the productivity, readiness, and resilience of the Army while reducing energy use and demand, and freeing up resources to better support the mission.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Data from: "Reply to ‘The challenge of defining effectively-no-snow’"

This repository contains the data and code associated with the paper titled "Reply to ‘The challenge of defining effectively-no-snow’" published in Nature Reviews Earth and Environment, 2026. In this reply, we argue that the 10th percentile of peak SWE (Snow Water Equivalent), which we propose in the original article, can be used as intended given it's a standardized, impact-based benchmark for comparing snow conditions across regions, not as a literal measure of snow absence. We present new evidence with SNOwpack TELemetry (SNOTEL) data showing that years meeting the threshold are overwhelmingly associated with subsequent drought (given United States Drought Monitor conditions), supporting its hydrologic and societal relevance. We conclude that while the distinction between "effectively no snow" and "zero snow" should be clearly communicated, the original definition remains appropriate for assessing impacts on snow-dependent water systems. The file code_nree_ML_reply_2026.Rmd contains the main processing scripts which analyze the SNOTEL data. Data from the US Drought Monitor was downloaded at: https://usdmdataservices using the Get Drought Severity Statistics By Area Percent' option, saved to the *_HUC4_delineated.csv files (Hydrologic Unit Code), which are labeled accordingly. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

EARTH SCIENCE > TERRESTRIAL HYDROSPHERE > SNOW/ICE↗
DOE OSTI2024

Further adoption of conservation tillage can increase maize yields in the western US Corn Belt

Conservation tillage can reduce soil erosion, increase soil health, and decrease labor and fuel input costs. Despite these benefits, potential yield impacts remain an important concern for farmers considering adoption. Previous research suggests that conservation tillage is likely to have the largest yield benefits in more arid conditions, but a lack of field-level analyses across climatic, management and soil conditions limits confidence in such predictions. Satellite imagery provides the opportunity to monitor agricultural lands at sub-field resolution across large spatial scales and wide environmental gradients. Here we investigate the maize yield impacts of conservation tillage in the semi-arid western US Corn Belt, using sub-field resolution datasets on tillage practices and crop yields derived from satellite data spanning four states (Nebraska, Kansas, South Dakota, and North Dakota) between 2008 and 2020. On these datasets, we estimate heterogenous yield outcomes for several thousand maize fields across gradients in climate, soil quality and irrigation status by using a causal forests analysis, an adaptation of the random forests machine-learning algorithm for causal inference on observational data. We find that long-term adoption of conservation tillage increased rainfed maize yields by an average of 9.9% in the region. Impacts on irrigated yields were small and not statistically significant. These results, along with an analysis of variables related to greater than average yield benefits, indicate that improved water infiltration and retention are the primary reasons for conservation tillage benefits. Despite yield benefits, many fields estimated to see increased yields under long term low till have not adopted the practice. Therefore, we identify specific counties likely to benefit most from increased levels of adoption. Our results strengthen the understanding of the impacts of conservation agriculture on crop yields and help define environments and counties most likely to benefit from conservation tillage.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Process Anomaly Detection for Sparsely Labeled Events in Nuclear Power Plants

An essential aspect of online monitoring, subtle anomaly detection increases the detection lead time for equipment failure and enables a nuclear power plant (NPP) to mitigate unexpected partial or full outages, resulting in significant cost saving to the plant. Once an anomaly is detected by plant staff, its cause and severity are investigated. Because the vast majority of anomalies require some level of investigation, including some that require time-consuming examination, before they are passed over to the engineering organization for further analysis, plants are often equipped with tools to assist the staff in performing anomaly detection. Those tools operate as a black box and are often based on statistical methods that establish sensor correlations using preconfigured mathematical models and flag correlation deviations as anomalies. Due to the number of anomalies detected at a given NPP on a daily basis, a significant number of flagged anomalies usually await examination for days or weeks. A primary cause of this backlog is that the methods used by the tools generate many false positives. Though this is usually attributed to oversensitive model settings due to very narrow normal operation bands, it can also be associated with the model development being inadequate for the process being monitored, or with missing model inputs that could have explained misclassified positives. The performance of anomaly detection tools impacts their plant acceptance and utilization, especially when the effort to address false positives generated by the tool depletes the value or cost saved by using that tool. Thus, means to advance anomaly detection performance have been investigated by the Department of Energy’s Light Water Reactor Sustainability program. Previous and ongoing efforts have targeted unsupervised machine-learning (ML) methods, which do not require the labeling of any data fed into the ML model. By contrast, in supervised anomaly detection methods, every data point is labeled as either a normal or abnormal process condition, and the model is trained to replicate the classification process. Supervised methods usually outperform unsupervised methods, due to the added value in differentiating normal from anomalous states of the monitored process. An NPP’s corrective action program requires it to track and document, via a dedicated report, the resolution of any issues that occur within the plant. Once created, each report is reviewed by a plant screening committee, and several classifications and decisions are made. Recently, a collaborating NPP developed an artificial intelligence and ML-based classifier to categorize a condition report (CR) into classes that can serve to label the data as normal or anomalous. Applying CRs as labels represents a semi-supervised use case. Semi-supervised ML assumes that labels exist for some data points (i.e., labeled anomalies, in this case) but not for the rest. In this effort, semi-supervised ML methods were used to fuse data from CRs with anomaly detection methods in order to test the hypothesis that partially labeled anomalies would improve the accuracy of the anomaly detection methods. Specifically, two methods were used. The first is the deep Semi-supervised Anomaly Detection (deep SAD) method, which can handle labels ranging from fully unsupervised to fully supervised cases. The second is a newly designed ML method developed specifically for this effort and referred to as the high-order feature (HOF)-based method. To evaluate these two methods in controlled environments, synthetic data generators were developed and used. The first datasets used a spring-mass-damper (SMD) system simulator commonly found in mechanical engineering references. This was used to create two use cases: a one- and a three-mass system. Anomalies were introduced by changing the spring and damper coefficients while the system was actuated by random forces. The second datasets used the commercial Dymola-Modelica software to build a simplified nuclear reactor model. Anomalies were added in the form of corrupted sensor readings and/or control commands. The deep SAD method was tested using the SMD system, while the HOF method was tested using both datasets. Application of the deep SAD semi-supervised ML method demonstrated that labels can generate increased confidence in detecting true anomalies. This helped increase the number of true positives and decrease the number of false negatives—something that would aid in addressing the backlog of possible anomalies. Application of the HOF method demonstrated that labels can aid in down selecting from a candidate set of features to a more optimal subset in order to better differentiate between normal and anomalous conditions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

SPARTAN and IMPROVE Comparison Experiment (SPICE) Interim Campaign Report

SPICE (the SPARTAN and IMPROVE Comparison Experiment) aims to obtain and quantify comparisons between aerosol PM 2.5 mass concentration measurements from the University of Oklahoma (OU) Surface Particulate Matter Network (SPARTAN) station and the U.S. Environmental Protection Agency (EPA) Interagency Monitoring of Protected Visual Environments (IMPROVE) station hosted by the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) User Facility at ARM’s Southern Great Plains (SGP) observatory in Oklahoma. Aerosols—both natural and anthropogenic—affect human populations in multiple ways. Much of ARM’s research focuses on how aerosols influence weather and climate through their optical and radiative effects, as well as their impacts on clouds and precipitation. However, aerosols also pose direct risks to humans and other organisms through inhalation, with the severity of health impacts depending on particle size, chemical composition, and duration of exposure. The IMPROVE network was established by the U.S. EPA to monitor air quality, including visible clarity as well as total and chemically speciated aerosol mass concentrations. The ARM SGP site hosts the IMPROVE SOGP station. Separately, the SPARTAN network operates a globally distributed set of stations similar to IMPROVE but with an emphasis on remote deployment and semi-autonomous operation for use beyond the borders of the United States (IMPROVE only operates within the U.S.). The University of Oklahoma operates a SPARTAN station. To establish confidence in the OU SPARTAN instrumentation and measurement protocol relative to the EPA-certified IMPROVE station, the OU SPARTAN station is currently deployed at SGP in close proximity to the IMPROVE SOGP station. The SPICE campaign was envisioned as a contiguous calendar-year effort for 2025 to capture seasonal variation in mass loading as well as composition. However, independent of the SPICE campaign, the SPARTAN network adopted a new filter construction part-way through the year, interrupting our contiguous data set. Thus, to obtain a contiguous data set with a uniform consistent configuration, SPICE desires an extension through 2026.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Code Description for "Brief Communication: Monitoring snow depth using small, cheap, and easy-to-deploy ground surface temperature sensors"

Temporally continuous snow depth estimates are vital for understanding changing snow patterns and impacts on permafrost in the Arctic. We train a random forest machine learning model to predict snow depth from variability in ground surface temperature. To our knowledge, this is the first time that small ground surface temperature sensors have been used to estimate snow depth. The model performs well at sites where the model was trained and at pan-arctic evaluation sites (RMSE <= 0.15 m). Small temperature sensors are cheap and easy-to-deploy, so this technique enables spatially distributed and temporally continuous snowpack monitoring to an extent previously infeasible. The model is flexible and can be applied to datasets retroactively to retrieve snow depth estimates at additional sites. This code package includes a *.joblib file of the trained random forest model and a *.ipynb file showing how to clean input data, train the random forest model, and apply the model.

Bachand, Claire↗
DOE OSTI2021

The Impact of Behind-the-Meter Heterogeneous Distributed Energy Resources on Distribution Grids

The increasing integration of distributed energy resources (DERs) on the electric grid brings new challenges and opportunities for utility grid operations. With the rapid deployment of DERs, there is emerging interest in integrating these controllable devices with utility operations at all levels for monitoring and management. To understand the challenges with increasing behind-the-meter (BTM) DERs and to identify the needs in deploying advanced controls, a comprehensive grid impact study is indispensable. This paper presents the analysis which help visualize the DER impact on the grid, identify the challenges and provides an insight into the new distribution management and control needs to enable reliable and resilient grid operations.

27 ARPA - Advanced Research Projects Agency-Energy↗
DOE OSTI2022

Hydrologic investigations of radar-rainfall error propagation to rainfall-runoff model hydrographs

Rainfall is arguably the most important yet most variable input for rainfall-runoff hydrologic models. In this study, the authors search for the characteristics of radar-rainfall estimates that are most important for skillful streamflow predictions. They perform comprehensive hydrologic investigations of radar-rainfall characteristics, including spatiotemporal resolution, radar range visibility, statistical characterization of rainfall variability, all vis-a-vis basin characteristics such as size and river network topology. Since the true rainfall fields are unknown, the authors exploit a paradigm of using two independently constructed radar-rainfall products i.e., Multi-Radar Multi-Sensor and IFC-ZR used operationally by the Iowa Flood Center (IFC). Using the distributed hydrologic model called the Hillslope-Link Model for the domain of the state of Iowa, they evaluate streamflow prediction at 140 USGS gauge stations that monitor rivers in Iowa. Through spatial and temporal rainfall aggregation experiments, the authors show that the impact of spatial and temporal resolution of rainfall is significant typically for smaller basins while starts reducing significantly for basins larger than 1,000 km 2 . Other rainfall characteristics they explored do not reveal a strong signature in the relationship of rainfall differences between the two products and hydrograph errors. However, exploring the product similarities rather than differences reveals that the basin-wide rainfall volume has the most significant effect on streamflow prediction. The results from this study are generalizable for all rainfall observing systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Mobilization of trace metals from caprock and formation rocks at the Illinois Basin – Decatur Project demonstration site under geological carbon dioxide sequestration conditions

One concern for geologic CO 2 sequestration is the potential leakage of CO 2 or CO 2 -saturated brine containing trace metals into overlying aquifers, which poses the risk of adversely affecting underground sources of drinking water. In this work, rock and brine samples collected in the Illinois Basin – Decatur Project (IBDP), a large-scale geological CO 2 sequestration demonstration project in Decatur, Illinois, USA, as well as synthetic brine samples were used to understand the mobilization of trace metals as a result of CO 2 –rock–brine interactions under IBDP-specific conditions (50 °C and 20.6 MPa). Rock sample characterization indicated that, at the IBDP site, trace metal concentrations were greatest in cap rock samples from the Eau Claire Formation, compared with those from formations above and below this Formation. The natural brine samples collected from the Mt. Simon Sandstone reservoir were highly reducing and saline, with trace metal concentrations up to 680 times greater than the U.S. Environmental Protection Agency-prescribed drinking water standards. Batch leaching experiments indicated that both trace metal mobilization from rock and immobilization from the brine occurred when high-pressure CO 2 was introduced into the rock-brine system. The amounts of metals mobilized from the rock generally accounted for <5% of the total metals in the rock, but for some metals, including Ni, Pb, and Tl, up to 63% of the metals in the rock were mobilized to the brine. Leaching of trace metals into synthetic brines was different from that into the natural brine. The results of this study provided current information on the trace element sources and will help in risk simulations and experimental design to further evaluate potential impact of leakage on groundwater quality, which is important for future GCS projects to consider for monitoring and containment assurance purposes.

58 GEOSCIENCES↗
DOE OSTI2022

Multifunctional fiber-reinforced composites for passive sensing and energy harvesting with enhanced mechanical performance

Fiber-reinforced polymer composites (FRPCs) are valid alternatives to metals, especially in the aerospace industry, for their superior strength-to-weight ratio. FRPCs are generally manufactured by stacking layers of fibers and infusing them with epoxy-based adhesives to result in laminates. Such layer-by-layer structure often ensues a poor interlaminar property of the composites. Additionally, FRPCs can sustain complex damage modes because of their inherent anisotropic characteristics. For example, delamination can occur due to low-velocity impact at a subsurface level, which can result in premature catastrophic failures if undetected. Thus, structural monitoring is crucial to identify such damage. Point-based sensors (e.g., strain gauges, accelerometers, among many) can be embedded in the FRPCs for structural monitoring. However, their electrical power demand often embroils their usage. Therefore, the main objective of this research is to design and implement multifunctional composites that can simultaneously perform passive sensing and energy harvesting while exhibiting better mechanical performance. Previous research efforts have demonstrated that a continuous feed-through deposition of functional materials on fiber surfaces simultaneously enhanced the mechanical and sensing properties of the FRPCs. This study explores a similar approach to encode passive sensing and energy harvesting properties in the FRPCs by integrating ferroelectric microparticles on the fiber surfaces. Upon ensuring their superior interlaminar shear strength, sensing and energy harvesting properties were characterized through experimental studies. The outcome is a multifunctional composite fabricated by coating the fibers with functional microparticles through a high throughput, scalable, and low-cost approach that enables passive sensing, energy harvesting, and improved mechanical performance.

Gupta, Sumit↗
DOE OSTI2021

A Heterogeneous System for Eagle Detection, Deterrent, and Wildlife Collision Detection for Wind Turbines (Final Technical Report)

This report summarizes the design, implementation, and test of an integrated system for automated detection and deterrence of eagles, with included wind turbine blade strike detection and imaging functionality. A machine learning approach was used in conjunction with a 360° camera system for automated detection and classification of golden eagles. This was developed using footage obtained from trained golden eagles and other raptors, in collaboration with wildlife biologists and professional bird handlers. Oregon State University developed a visual deterrent system, which uses inflatable anthropomorphic sculptures with random, kinetic motion to deter eagles, and conducted limited field testing on live eagles; the deterrent can be triggered by the visual detection of eagles using the vision system. Finally, a multi-sensor module was developed that is mounted at the turbine blade root. This module measures vibration and other motions to detect blade strikes, and an integrated on-blade camera captures an image of any impacting objects. Long-term, this blade strike detection system is intended to support an automatic monitoring and certification system for the eagle detection and deterent system. Independent field testing of each system component is described. Testing of the integrated system on an operational wind turbine was conducted across three separate field tests. This includes multi-day fields tests on a General Electric 1.5MW wind turbine at the National Renewable Energy Laboratory (NREL) National Wind Technology Center (NWTC) in Boulder, CO in October 2018 and July 2019; installation procedures, test procedures, and a summary of collected data are presented. A third multi-day on-turbine field test is also presented, which was performed using a General Electric 1.5MW wind turbine at the North American Wind Research and Training Center (NAWRTC) at Mesalands Community College, Tucumcari, NM in April 2019. Across these field tests, the vision system was demonstrated using unmanned aerial vehicles (UAV), and the eagle classification algorithm was not tested; the visual deterrent system was demonstrated, including automatic, remote deployment following surrogate visual detections; and, multi-sensor on-blade data was recorded across multiple wind turbine operational conditions and through more than 100 surrogate blade strikes using soft projectiles, including the successful demonstration of automatic image capture of striking objects. This data set was also used for offline development and validation of enhanced collision detection algorithms. As summarized in this report, the development and field validation of an integrated detection, deterrent, and blade collision detection system represents a critical proof of concept for future technology development of related detection and deterrent technologies, where both deterrent as well as collision detection recording devices are needed for future siting, monitoring, and operation of wind turbine installations, both onshore and offshore.

17 WIND ENERGY↗
DOE OSTI2021

Laboratory Investigation of Gas Transport Through Variably Saturated Rock [Slides]

Project Overview: Noble gas fission products, such as xenon (Xe), are monitored to detect nuclear tests. For well-contained subsurface tests, transport to the surface will impact detected isotopic ratios, complicating event identification. Role of water saturation in subsurface transport is not well understood. Historic tests have taken place near or below the water table. To accurately predict breakthrough times and fractionation, gas transport must be characterized through intact rock with variable saturation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Parallel Simulation of Beam Dynamics in Particle Accelerators [Slides]

Particle accelerators are among the most versatile and important tools of scientific discovery. The Nation's accelerators are responsible for a wealth of advances in materials science, chemistry, the biosciences, particle physics, and nuclear physics. They also have important applications to national security, the environment, energy, medicine, and on the quality of people's lives. LANL has a long history of making pioneering contributions to Accelerator Science including key contributions to the field of Computational Accelerator Physics. These include the development of early beam dynamics codes with space charge (such as PARMILA and PARMELA), the development of rf cavity codes and magnet codes (including Poisson and Superfish), and the development and distribution of codes to the accelerator community through the Los Alamos Accelerator Code Group. LANL researchers also helped pioneer the development of massively parallel space-charge codes. In project t22_accelsim we have moved beyond electrostatic models of collective effects (i.e., solving the Poisson equation in the bunch frame) to fully electromagnetic models based on the Lienard-Wiechert formalism. This approach enables the large-scale simulation of radiation production and collective effects in high brightness electron beams. This is highly relevant to LANL given its future goal of developing an X-ray Free Electron Laser (XFEL). It also directly impacts a LANL LDRD project to develop an undulator-based non-invasive beam profile monitor for beams created in laser-plasma accelerator systems.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Effect of Humidity on Charge Decay in Varying Atmospheric Gases

Characterization of surface charge decay for varying materials – PTFE, Acrylic, and metals – of 100 mm diameter, radially symmetric bodies was performed. In a gaseous environment, charged materials will lose their charge depending on gas type, humidity, and other environmental factors. Eventually, the charge will be (1) neutralized by ions or electrons in the surrounding gas (2) distributed by conduction on the material surface or in its volume (3) or neutralized by ions or electrons in the material surface and volume. Metal and polymer samples are charged to a known potential in the tens of kilovolts range, and the decaying voltage is monitored with a non-contact probe until the potential reaches near zero. Humidity has an impact on both groups of materials. As a baseline, in air at 42% relative humidity in an enclosed chamber, a positive charge equivalent to 20 kV potential on a metal sphere of 50 mm radius decays to near zero within 25 minutes. In contrast, the same sphere negatively charged decays slightly faster within 20 minutes. Overall, the humidity has a significant effect on the charge decay time, such that an order of magnitude longer times (hours) are observed at low humidity. Effects of chamber size were also studied, drawing a connection to charge carriers in the air available for decay, with chambers of volume(s): 3.3 105 cm3 and 1.4 106 cm3, respectively.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

58 GEOSCIENCES↗
DOE OSTI2024

Rapid measurement of soluble xylo-oligomers using near-infrared spectroscopy (NIRS) and multivariate statistics: calibration model development and practical approaches to model optimization

Rapid monitoring of biomass conversion processes using techniques such as near-infrared (NIR) spectroscopy can be substantially quicker and less labor-, resource-, and energy-intensive than conventional measurement techniques such as gas or liquid chromatography (GC or LC) due to the lack of solvents and preparation methods, as well as removing the need to transfer samples to an external lab for analytical evaluation. The purpose of this study was to determine the feasibility of rapid monitoring of a biomass conversion process using NIR spectroscopy combined with multivariate statistical modeling, and to examine the impact of (1) subsetting the samples in the original dataset by process location and (2) reducing the spectral range used in the calibration model on model performance. We develop multivariate calibration models for the concentrations of soluble xylo-oligosaccharides (XOS), monomeric xylose, and total solids at multiple points in a biomass conversion process which produces and then purifies XOS compounds from sugar cane bagasse. A single model using samples from multiple locations in the process stream showed acceptable performance as measured by standard statistical measures. However, compared to the single model, we show that separate models built by segregating the calibration samples according to process location show improved performance. We also show that combining an understanding of the sample spectra with simple multivariate analysis tools can result in a calibration model with a substantially smaller spectral range that provides essentially equal performance to the full-range model. We demonstrate that real-time monitoring of soluble xylo-oligosaccharides (XOS), monomeric xylose, and total solids concentration at multiple points in a process stream using NIR spectroscopy coupled with multivariate statistics is feasible. Segregation of sample populations by process location improves model performance. Models using a reduced spectral range containing the most relevant spectral signatures show very similar performance to the full-range model, reinforcing the importance of performing robust exploratory data analysis before beginning multivariate modeling.

09 BIOMASS FUELS↗