Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “high harmonic generation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

The Future of X-ray Irradiation: Addressing Supply Chain Risks and Opportunities (UUR Edition)

This study supports the Office of Radiological Security’s (ORS) mission of eliminating cesium irradiators by analyzing the supply chain for self-shielded X-ray irradiators (SSXIs), identifying potential risks, and proposing mitigation measures. The research focuses on the primary components of SSXIs, including X-ray tubes, controllers, generators, and coolers or chillers, and evaluates their vulnerabilities using a comprehensive risk matrix framework. The methodology includes subject matter expert (SME) interviews with relevant manufacturers and major stakeholders, a deep literature review, and a meta-analysis of maintenance reports provided by SSXI end users. Results show that while the SSXI market is small, it’s growing, and the highly global nature of the supply chain may create vulnerabilities for critical SSXI components (X-ray tubes are the most vulnerable, followed by generators and controllers). This research communicates necessary information to address concerns of current and future end users, especially those interested in transitioning away from radioactive sources, and informs future policy aimed at supporting the irradiation industry.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2025

Ultimate brightness of a medium-energy synchrotron light source at operational beam intensity

Synchrotron light sources are key instruments of modern science, providing unique opportunities for groundbreaking studies in diverse scientific disciplines and driving innovation in numerous scientific and technological fields. Fourth-generation light sources provide unprecedented capabilities in imaging, spectroscopy and diffraction techniques. Ultimate brightness is the key to advancing to a smaller scale, faster response, and higher data measurement and processing rate. The brightness is primarily determined by the electron beam emittance and energy spread at operational intensity. A common feature of fourth-generation synchrotrons is the short length of the electron bunches combined with a very small transverse beam size. Consequently, the high particle density leads to strong collective effects that significantly increase the emittance and limit the achievable brightness at operational beam intensity. In this article, we summarize our studies of the emittance and brightness scaled with the beam energy and intensity, taking into account the effects of intrabeam scattering, beam-impedance interaction and bunch lengthening provided by higher-harmonic RF systems to identify optimal combinations of machine and beam parameters.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Surface structure of water from soft X-ray second harmonic generation

The microscopic structure of water’s surface is crucial to many natural and industrial processes, but studying its hydrogen bond (H-bond) network directly remains challenging due to the required interfacial sensitivity of experimental techniques. By leveraging advances in flat liquid sheet microjets and terawatt-scale attosecond soft X-ray pulses from the LCLS X-ray free electron laser, we employed soft X-ray second harmonic generation (SXSHG) spectroscopy to examine the liquid water/vapor interface. SXSHG combines the elemental selectivity of X-ray spectroscopies with the surface selectivity of SHG and gives access to the electronic structure of interfacial species. Here, we show the SXSHG spectrum differs from bulk water’s X-ray absorption, with its peak shifted several eV, indicating a vastly different electronic environment at the interface as compared to the bulk. First-principles electronic structure calculations show the signal is highly sensitive to H-bond interactions, such as water molecules accepting a single H-bond, which are surface abundant.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Validation of finite element analysis strategy to investigate acoustic levitation in a two-axis acoustic levitator

A two-axis acoustic levitator can be used to generate a standing pressure wave capable of levitating solid and liquid particles at appropriate input conditions. This work proposes a simulation framework to investigate the two-axis levitation particle stability using a commercial, computational fluid dynamics software based on the harmonic solution to the acoustic wave equation. The simulation produced predictions of the standing wave that include a strong "+" shaped pattern of nodes and anti-nodes that are aligned with the levitator axes. To verify the simulation, a levitator was built and used to generate the standing wave. The field was probed with a microphone and a motorized-scanning system. After scaling the simulated pressure to the measured pressure, the magnitudes of the sound pressure level at corresponding high-pressure locations were different by no more than 5%. This is the first time a measurement of a two-axis levitator standing pressure wave has been presented and shown to verify simulations. Furthermore, as an additional verification, the authors consulted high speed camera measurements of a reference-levitator transducer, which was found to have a maximum peak-to-peak displacement of 50 +/- 5 mu m. The reference-levitator is known to levitate water at 160 dB. The system for this work was simulated to match the operation of the reference-levitator so that it produced sound pressure levels of 160 dB. This pressure was achieved when the transducer maximum peak-to-peak displacement was 50.8 mu m. The agreement between the two levitators' displacements provides good justification that the modeling approach presented here produces reliable results.

2 axis levitator↗
DOE OSTI2025

Identification of Important Phenomena for Light Water Reactors During Heat Transport System Failure Events in Integrated Energy Systems

This work adapts historical literature and existing phenomena identification and ranking tables (PIRT) to be applicable to a novel nuclear power plant (NPP) and chemical or thermal process integrated energy system (IES), particularly focusing on the process heat and heat transport system failure events that are not a concern during normal NPP operation but become vital when an IES is considered. Nuclear energy has been suggested to go beyond base-load applications and be used for hydrogen co-generation systems, amongst other IESs. Prior to the implementation of nuclear IESs, sufficient analysis must be performed on accident events to ensure public safety. The events considered were deemed important because of their potential to damage systems, structures, and components (SSCs). Process thermal events of concern include loss of heat load and temperature transient events. Loss of heat load events were characterized as having high importance and being well understood. Temperature transient events may be further categorized by the cyclic loading and harmonics phenomena. Cyclic loading issues were classified as medium to high importance with knowledge gaps existing regarding fatigue and low power operation, while harmonics phenomena were classified as low importance and are well understood. Heat transport system failure events of concern include intermediate and process heat exchanger failures, mass addition to reactor coolant, ingress of material from thermal manifold/energy storage, and loss of intermediate fluid. Furthermore, these events tended to be of high or medium importance, with some knowledge gaps needing to be filled for individual reactor systems due to unique designs.

Integrated Energy System (IES)↗
DOE OSTI2024

Systems and methods that use harmonic drives for converting reciprocating axial motion to continuous rotary motion, helical drives for converting reciprocating rotary motion to reciprocating axial motion and combinations thereof for converting reciprocating rotary motion to continuous rotary motion

A harmonic drive assembly and fluid-powered linear motors with both axial pistons are rotary piston arrangements incorporating the harmonic drive assembly are disclosed. The motors may be used in downhole drilling applications, but the drive assembly and/or motors may be used in other applications. The assembly, motors and methods use advanced harmonic drives, advanced helical drives, and combinations thereof with 1) motors with axial pistons and reciprocating linear rings to convert reciprocative axial motion to continuous rotary motion, and 2) motors with rotary pistons and reciprocating linear rings to rectify reciprocative rotary motion to continual rotary motion to improve performance over prior configurations. Axial pistons provide a robust simple solution for generating rotation; Rotational pistons provide increased torque generation as the torque generated is proportional to motor length. Since downhole drills are long, a high-torque motor can be produced using this method.

Raymond, David W.↗
DOE OSTI2022

Comparison of Power System Current Sensors via Playback of Electrical Disturbances

The need to accurately measure high-frequency content in power system voltage and current phenomena is increasingly becoming more of a priority. As the amount of distributed energy resources (DER) and nonlinear loads penetrating the grid increases, so do challenges associated with traditional measurement and metering applications. In this paper, three commercially-available medium-voltage-level current sensors are characterized in terms of their harmonic amplitude and phase performance against reference signals that are “played back” through the sensors through the use of an arbitrary waveform generator. It is shown that none of the three sensors studied are able to faithfully replicate all of the input signals completely, though there are advantages and disadvantages to each in terms of noise, resonance, and induced phase drift. Additionally, the Goodness-of-Fit metric, typically used for PMU model validation, is used to generate side-by-side comparisons of sensor accuracy over a small window around the events under study.

Wilson, Aaron↗
DOE OSTI2026

Higher Efficiency, Demand Flexible Refrigerator with On-Demand Micro-Vibrational De-icing Technology

Refrigerator technology has advanced significantly over the last couple of decades. Today’s refrigerators use only about 25% of the energy that was required to power models built in 1975. Even as they continually improve efficiency to meet standards, refrigerators have increased in size by almost 20%, added energy-consuming features such as through-the-door ice, and provide more benefits than ever before. However, a few challenges and technology gaps are preventing further improvement of the demand responsiveness and efficiency of the refrigerators. One of the major technology gaps in existing refrigerators is their outdated de-icing process. When the evaporator generates frost, an old-fashioned resistive heating element melts the ice. Most refrigerators have a timed defrost cycle, rather than an active system that could monitor the state of the frost. In these systems, not only is the precious electricity used at its least efficient form of conversion (direct conversion of electricity to heat), but also all the latent heat associated with the ice is wasted during the melting process. On top of that, the refrigerator needs to work harder to pull the temperature down after defrosting, and, last but not least, the food quality is severely impacted by the temperature swings during the defrost cycle. According to a study, the EU alone wastes 89 million tons of food in the supply chain every year. Any temperature swing during defrosting (about 6F according to Emerson for low-temperature cases) can negatively impact the shelf life of meat and other products for multiple days. All these issues can happen during the peak demand time of the electric grid. Unlike the conventional systems, the proposed novel advanced micro-vibrational deicing process uses no heat for defrosting. Instead, it uses the micro vibrations generated by a piezoelectric or vibration-generating module to mechanically break ice from the heat exchanger almost instantaneously. The project titled “Higher Efficiency, Demand Flexible Refrigerator with On-Demand Micro-Vibrational De-icing Technology, performed by Ultrasonic Technology Solutions, LLC (UTS) of Knoxville, TN, in collaboration with Emerson (now Copeland), represents the final phase of a multi-year effort funded under the U.S. Department of Energy’s Building Technologies Office (BTO) BENEFIT FOA 2020. Initiated on October 1, 2021, and completed after a nine-month no-cost extension ending September 30, 2025, this project aimed to develop and validate a novel micro-vibrational mechanical defrosting system, achieving more than 25% improvement in defrosting energy efficiency over conventional baseline defrosting technologies. Over sixteen quarters, the project advanced from fundamental ice-mechanical characterization and prototype development to full-scale system integration and validation. Initial efforts established project management infrastructure and characterized ice adhesion properties, followed by the design and fabrication of early aluminum-based prototypes for resonance frequency testing. Subsequent quarters saw rapid technical progression, including the identification of optimal piezoelectric and motor-based vibration mechanisms, the demonstration of effective de-icing over 6x6-inch aluminum surfaces. The team achieved its Go/No-Go milestone by exceeding the 25% energy-efficiency improvement target—reaching up to 3,340% under optimized conditions—and later confirmed that motor-driven systems offered superior performance and energy efficiency compared to piezoelectric alternatives. Continued refinement led to the development of amplifier systems on printed circuit boards, improved control and instrumentation hardware, and integration into full-scale heat exchanger (HX) prototypes at both UTS and Copeland facilities. Multiple vibration-mounting studies and frost-growth experiments guided mechanical optimization and noise-mitigation strategies, achieving a 17.5 dB reduction in sound pressure level and verifying robust mechanical performance. Advanced analyses, including modal and harmonic simulations, established a quantitative understanding of vibrational behavior and de-icing efficiency across >1000 cm² systems. The final project phase successfully demonstrated scalable integration within reach-in and chest freezer prototypes, confirmed >25% efficiency improvements in large-area systems, and completed a comprehensive business model and scale-up strategy identifying electric defrost systems as the primary beachhead market. The culmination of this DOE-supported effort establishes micro-vibrational defrosting as a viable, high-efficiency, low-noise, and demand-flexible de-icing technology, paving the way for commercial deployment and broader application in next-generation refrigeration systems.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Quantum kinetic modeling of KEEN waves in a warm-dense regime

We report the first fully kinetic, quantum study of kinetic electrostatic electron nonlinear (KEEN) waves, showing that quantum diffraction systematically erodes the classical trapping mechanism, narrows harmonic locking to the fundamental, and hastens post-drive decay. Electrons are evolved with a second-order Strang-split 1D1V Wigner–Poisson solver that couples conservative semi-Lagrangian WENO advection to an analytic Fourier space update for the non-local Wigner term, while ions remain classical. We focus on collisionless dynamics in a weakly coupled regime, providing a controlled baseline before collisional extensions. Short, frequency-tuned ponderomotive pulses drive KEEN formation in a uniform Maxwellian plasma; as the dimensionless quantum parameter H rises from the classical limit to values relevant to warm-dense matter, doped semiconductors, and 2D electron systems, the drive threshold increases, higher harmonics are damped, trapped electron vortices diffuse, and the subplasma electrostatic energy relaxes to a lower stationary level, as confirmed by continuous wavelet analysis. These microscopic changes carry macroscopic weight. Ignition-scale capsules now compress matter to regimes where the electron de Broglie wavelength rivals the Debye length, making classical kinetic descriptions insufficient. By extending KEEN physics into this quantum domain, our results offer a potential diagnostic of non-equilibrium electron dynamics for next-generation inertial-confinement designs and high-energy-density platforms, indicating that predictive fusion modeling may benefit from the integration of kinetic fidelity with quantum effects.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Voltage waveform tailoring for high aspect ratio plasma etching of SiO 2 using Ar/CF 4 /O 2 mixtures: Consequences of ion and electron distributions on etch profiles

The quality of high aspect ratio (HAR) features etched into dielectrics for microelectronics fabrication using halogen containing low temperature plasmas strongly depends on the energy and angular distribution of the incident ions (IEAD) onto the wafer, as well as potentially that of the electrons (EEAD). Positive ions, accelerated to high energies by the sheath electric field, have narrow angular spreads and can penetrate deeply into HAR features. Electrons typically arrive at the wafer with nearly thermal energy and isotropic angular distributions and so do not directly penetrate deeply into features. These differences can lead to positive charging of the insides of the features that can slow etching rates and produce geometric defects such as twisting. In this work, we computationally investigated the plasma etching of HAR features into SiO 2 using tailored voltage waveforms in a geometrically asymmetric capacitively coupled plasma sustained in an Ar/CF 4 /O 2 mixture at 40 mTorr. The tailored waveform consisted of a sinusoidal wave and its higher harmonics with a fundamental frequency of 1 MHz. We found that some degree of control of the IEADs and EEADs is possible by adjusting the phase of higher harmonics φ through the resulting generation of electrical asymmetry and electric field reversal. However, the IEADs and EEADs cannot easily be separately controlled. The control of IEADs and EEADs is inherently linked. The highest quality feature was obtained with a phase angle φ = 0° as this value generated the largest (most negative) DC self-bias and largest electric field reversal for accelerating electrons into the feature. That said, the consequences of voltage waveform tailoring (VWT) on etched features are dominated by the change in the IEADs. Although VWT does produce EEADs with higher energy and narrower angular spread, the effect of these electrons on the feature compared to thermal electrons is not large. Finally, this smaller impact of VWT produced EEADs is attributed to thermal electrons being accelerated into the feature by electric fields produced by the positive in-feature charging.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Vacuum deposition of χ(2) nonlinear organic single crystal films on silicon

Integrating second order nonlinear (χ(2)) optical materials on chip is an ongoing challenge for Si photonics. Noncentrosymmetric molecular crystals have the potential to deliver high χ(2) nonlinearity with good thermal stability, but so far have been limited to growth from solution or the melt, which are both difficult to control and scale up in manufacturing. Here, we show that large (>100 μm) single crystal domains of the nonlinear molecule 2-[3-(4-hydroxystyryl)-5,5-dimethylcyclohex-2-enylidene] malononitrile (OH1) can be grown monolithically on either glass or Si via vacuum evaporation, followed by a short thermal annealing step. The crystallites are tens of nanometer thick and exhibit strong second harmonic generation with their primary χ(2) tensor component lying predominantly in plane. Remarkably, we find that a single domain can grow uninterrupted through nearby channels etched on a Si wafer, which may provide a path to integrate OH1 on Si or Si3N4 waveguides for a broad range of χ(2)-based photonic integrated circuit functionality.

Physics↗
DOE OSTI2022

Cross-scale sensing of field-level crop residue cover: Integrating field photos, airborne hyperspectral imaging, and satellite data

Conservation tillage practices can bring benefits to agricultural sustainability. Accurate spatial and temporal resolved information of field-scale crop residue cover, which reflects tillage intensity, is highly valuable for evaluating the outcomes of government conservation programs and voluntary ecosystem service markets, as well as facilitating agroecosystem modeling to quantify cropland biogeochemical processes. Remote sensing has the potential to cost-effectively detect crop residue cover, however, existing regional-scale studies were limited by insufficient ground truth data, scale mismatch between coarse satellite pixels and ground data, and the lack of key spectral data for detecting crop residues. Therefore, this study developed an innovative cross-sensing framework to integrate proximal sensing, airborne hyperspectral imaging, and satellite Earth Observation through deep learning to quantify field-level crop residue cover fractions at the regional scale. Specifically, we have collected intensive ground orthographic photos and conducted airborne hyperspectral surveys at corn and soybean fields of Champaign and nearby counties in Illinois, the heartland of the U.S. Corn Belt. Through semi-automatic labeling aided by ResNet-50 and superpixel image segmentation, we obtained 6719 records of ground residue fractions. With these ground data, we developed the 1-dimensional convolution neural network (CNN) model using airborne hyperspectral reflectance, which has 0.5m spatial resolution and 3–5 nm spectral resolution from 400 to 2400 nm, to predict residue fractions. By applying the CNN model to airborne pixels, we augmented “ground truth” data of crop residues and further combined them with Harmonized Landsat and Sentinel-2 (HLS) satellite data to quantify regional residue fractions at 30 m resolution. Results show that airborne hyperspectral imagery with CNN can accurately detect residue fractions (R 2 = 0.82, relative RMSE = 11.73%) to effectively generate quasi “ground truth” data to support satellite upscaling to all fields. With independent ground data for testing, we found that the ground-airborne-satellite integrative framework achieved better predictions in estimating crop residue cover (R 2 = 0.67, relative RMSE = 17.53%) than the conventional ground-satellite upscaling (R 2 = 0.22, relative RMSE = 32.09%). Here we also found that the shortwave infrared wavelengths, particularly 2100–2300 nm, are vital for predicting crop residue cover. Sentinel-2 and Landsat-8 data have a comparable capability to track residue fractions due to similar shortwave infrared wavelengths. This study highlights the high accuracy of hyperspectral imaging to detect agroecosystem tillage management practices and the advantages of cross-scale sensing to cost-effectively integrate multi-source data to quantify field-level agroecosystem variables across scales.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Multiresolution Quantum Chemistry: Nonlinear Response Properties at the Basis Set Limit

We benchmark the accuracy of Dunning correlation-consistent Gaussian basis sets for computing frequencydependent second-order hyperpolarizabilities relevant to second-harmonic generation (SHG), using multiresolution analysis (MRA) as a reference. Basis set errors are analyzed using a unit-sphere representation of the effective hyperpolarizability vector, enabling direct assessment of directional error structure. We introduce a relative RMS total error metric that integrates directional deviations over the unit sphere and complement it with signed projection errors that distinguish over- and underestimation. Unsupervised clustering based on these signed directional metrics reveals four distinct convergence behaviors across a set of 68 molecules. Unitsphere visualizations of representative systems show that basis set errors are often highly anisotropic and localized along specific bond directions, even when global error measures appear small. Doubly augmented basis sets consistently outperform singly augmented ones, and core-polarization functions are required for uniform convergence in second-row systems. Overall, this work demonstrates that directional analysis combined with clustering provides a robust framework for understanding basis set convergence in nonlinear optical response properties.

Basis sets↗
DOE OSTI2025

Azimuthal correlation anisotropies in p + p collisions simulated using Pythia

Stimulated by a keen interest in possible collective behavior in high-energy proton-proton and proton-nucleus collisions, we study two-particle angular correlations in pseudorapidity and azimuthal differences in simulated p + p interactions using the Pythia 8 event generator. Multi-parton interactions and color connection are included in these simulations, which have been perceived to produce collectivity in final-state particles. Meanwhile, contributions from genuine few-body nonflow correlations, not of collective flow behavior, are known to be severe in these small-system collisions. We present our Pythia correlation studies pedagogically and report azimuthal harmonic anisotropies analyzed using several methods. We observe anisotropies in these Pythia simulated events qualitatively and semi-quantitatively, similar to experimental data. Furthermore, our findings highlight the delicate nature of azimuthal anisotropies in small-system collisions and provide a benchmark that can aid in improving data analysis and interpreting experimental measurements in small-system collisions.

Pythia↗
DOE OSTI2020

Frequency-domain hot-wire sensor and 3D model for thermal conductivity measurements of reactive and corrosive materials at high temperatures

High temperature solids and liquids are becoming increasingly important in next-generation energy and manufacturing systems that seek higher efficiencies and lower emissions. Accurate measurements of thermal conductivity at high temperatures are required for the modeling and design of these systems, but commonly employed time-domain measurements can have errors from convection, corrosion, and ambient temperature fluctuations. Here, we describe the development of a frequency-domain hot-wire technique capable of accurately measuring the thermal conductivity of solid and molten compounds from room temperature up to 800 °C. Therefore, by operating in the frequency-domain, we can lock into the harmonic thermal response of the material and reject the influence of ambient temperature fluctuations, and we can keep the probed volume below 1 µl to minimize convection. The design of the microfabricated hot-wire sensor, electrical systems, and insulating wire coating to protect against corrosion is covered in detail. Furthermore, we discuss the development of a full three-dimensional multilayer thermal model that accounts for both radial conduction into the sample and axial conduction along the wire and the effect of wire coatings. The 3D, multilayer model facilitates the measurement of small sample volumes important for material development. A sensitivity analysis and an error propagation calculation of the frequency-domain thermal model are performed to demonstrate what factors are most important for thermal conductivity measurements. Finally, we show thermal conductivity measurements including model data fitting on gas (argon), solid (sulfur), and molten substances over a range of temperatures.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Passive Islanding Detection of Inverter-Based Resources in a Noisy Environment

Islanding occurs when a load is energized solely by local generators and can result in frequency and voltage instability, changes in current, and poor power quality. Poor power quality can interrupt industrial operations, damage sensitive electrical equipment, and induce outages upon the resynchronization of the island with the grid. This study proposes an islanding detection method employing a Duffing oscillator to analyze voltage fluctuations at the point of common coupling (PCC) under a high-noise environment. Unlike existing methods, which overlook the noise effect, this paper mitigates noise impact on islanding detection. Power system noise in PCC measurements arises from switching transients, harmonics, grounding issues, voltage sags and swells, electromagnetic interference, and power quality issues that affect islanding detection. Transient events like lightning-induced traveling waves to the PCC can also introduce noise levels exceeding the voltage amplitude by more than seven times, thus disturbing conventional detection techniques. The noise interferes with measurements and increases the nondetection zone (NDZ), causing failed or delayed islanding detection. The Duffing oscillator nonlinear dynamics enable detection capabilities at a high noise level. The proposed method is designed to detect the PCC voltage fluctuations based on the IEEE standard 1547 through the Duffing oscillator. For the voltages beyond the threshold, the Duffing oscillator phase trajectory changes from periodic to chaotic mode and sends an islanded operation command to the inverter. The proposed islanding detection method distinguishes switching transients and faults from an islanded operation. Experimental validation of the method is conducted using a 3.6 kW PV setup.

Energy & Fuels↗
DOE OSTI2020

Dispersion of the Nonlinear Susceptibility of MoS2 and WS2 from Second-Harmonic Scattering Spectroscopy

Dispersion of the absolute second-order susceptibility of both MoS2 and WS2 is assessed on a wide spectral excitation range (710–1300 nm) by using second-harmonic scattering spectroscopy (SHS). SHS is an accurate ensemble measurement here applied on well-dispersed suspensions of monodisperse liquid-exfoliated nanosheets showing a high monolayer content. The as-derived, high susceptibility values shed light on the discrepancies between available literature values while evidencing resonances associated with the main excitonic transitions.

CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND SU↗
DOE OSTI2020

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

58 GEOSCIENCES↗