SEARCH · Engineering Papers

Results for “randomized algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73

All sources NASA NTRS DOE OSTI

DOE OSTI2020

The importance of round-robin validation when assessing machine-learning-based vertical extrapolation of wind speeds

The extrapolation of wind speeds measured at a meteorological mast to wind turbine rotor heights is a key component in a bankable wind farm energy assessment and a significant source of uncertainty. Industry-standard methods for extrapolation include the power-law and logarithmic profiles. The emergence of machine-learning applications in wind energy has led to several studies demonstrating substantial improvements in vertical extrapolation accuracy in machine-learning methods over these conventional power-law and logarithmic profile methods. In all cases, these studies assess relative model performance at a measurement site where, critically, the machine-learning algorithm requires knowledge of the rotor-height wind speeds in order to train the model. This prior knowledge provides fundamental advantages to the site-specific machine-learning model over the power-law and log profiles, which, by contrast, are not highly tuned to rotor-height measurements but rather can generalize to any site. Furthermore, there is no practical benefit in applying a machine-learning model at a site where winds at the heights relevant for wind energy production are known; rather, its performance at nearby locations (i.e., across a wind farm site) without rotor-height measurements is of most practical interest. To more fairly and practically compare machine-learning-based extrapolation to standard approaches, we implemented a round-robin extrapolation model comparison, in which a random-forest machine-learning model is trained and evaluated at different sites and then compared against the power-law and logarithmic profiles. We consider 20 months of lidar and sonic anemometer data collected at four sites between 50 and 100 km apart in the central United States. We find that the random forest outperforms the standard extrapolation approaches, especially when incorporating surface measurements as inputs to include the influence of atmospheric stability. When compared at a single site (the traditional comparison approach), the machine-learning improvement in mean absolute error was 28 % and 23 % over the power-law and logarithmic profiles, respectively. Using the round-robin approach proposed here, this improvement drops to 20 % and 14 %, respectively. These latter values better represent practical model performance, and we conclude that round-robin validation should be the standard for machine-learning-based wind speed extrapolation methods.

17 WIND ENERGY↗

NASA NTRS2015

First SNPP Cal/Val Campaign: Satellite and Aircraft Sounding Retrieval Intercomparison

Satellite ultraspectral infrared sensors provide key data records essential for weather forecasting and climate change science. The Suomi National Polar-orbiting Partnership (SNPP) satellite Environmental Data Record (EDR) is retrieved from calibrated ultraspectral radiance so called Sensor Data Record (SDR). It is critical to understand the accuracy of retrieved EDRs, which mainly depends on SDR accuracy (e.g., instrument random noise and absolute accuracy), an ill-posed retrieval system, and radiative transfer model errors. There are few approaches to validate EDR products, e.g., some common methods are to rely on radiosonde measurements, ground-based measurements, and dedicated aircraft campaign providing in-situ measurements of atmosphere and/or employing similar ultraspectral interferometer sounders. Ultraspectral interferometer sounder aboard aircraft measures SDR to retrieve EDR, which is often used to validate satellite measurements of SDR and EDR. The SNPP Calibration/Validation Campaign was conducted during May 2013. The NASA high-altitude aircraft ER-2 that carried ultraspectral interferometer sounders such as the NASA Atmospheric Sounder Testbed-Interferometer (NAST-I) flew under the SNPP satellite that carries the Cross-track Infrared Sounder (CrIS). Here we inter-compare the EDRs produced with different retrieval algorithms from SDRs measured by the sensors from satellite and aircraft. The available dropsonde and radiosonde measurements together with the European Centre for Medium-Range Weather Forecasts (ECMWF) analysis were also used to draw the conclusion from this experiment.

Zhou, Daniel K.↗

NASA NTRS2005

An information maximization model of eye movements

We propose a sequential information maximization model as a general strategy for programming eye movements. The model reconstructs high-resolution visual information from a sequence of fixations, taking into account the fall-off in resolution from the fovea to the periphery. From this framework we get a simple rule for predicting fixation sequences: after each fixation, fixate next at the location that minimizes uncertainty (maximizes information) about the stimulus. By comparing our model performance to human eye movement data and to predictions from a saliency and random model, we demonstrate that our model is best at predicting fixation locations. Modeling additional biological constraints will improve the prediction of fixation sequences. Our results suggest that information maximization is a useful principle for programming eye movements.

NASA Discipline Neuroscience↗

DOE OSTI2025

Convergence and Quantum Advantage of Trotterized MERA for Strongly-Correlated Systems

Strongly-correlated quantum many-body systems are difficult to study and simulate classically. We recently proposed a variational quantum eigensolver (VQE) based on the multiscale entanglement renormalization ansatz (MERA) with tensors constrained to certain Trotter circuits. Here, we determine the scaling of computation costs for various critical spin chains which substantiates a polynomial quantum advantage in comparison to classical MERA simulations based on exact energy gradients or variational Monte Carlo. Algorithmic phase diagrams suggest an even greater separation for higher-dimensional systems. Hence, the Trotterized MERA VQE is a promising route for the efficient investigation of strongly-correlated quantum many-body systems on quantum computers. Furthermore, we show how the convergence can be substantially improved by building up the MERA layer by layer in the initialization stage and by scanning through the phase diagram during optimization. For the Trotter circuits being composed of single-qubit and two-qubit rotations, it is experimentally advantageous to have small rotation angles. We find that the average angle amplitude can be reduced considerably with negligible effect on the energy accuracy. Benchmark simulations suggest that the structure of the Trotter circuits for the TMERA tensors is not decisive; in particular, brick-wall circuits and parallel random-pair circuits yield very similar energy accuracies.

Miao, Qiang [Duke Quantum Center, Duke University,↗

DOE OSTI2021

Machine learning based algorithms for uncertainty quantification in numerical weather prediction models

Complex numerical weather prediction models incorporate a variety of physical processes, each described by multiple alternative physical schemes with specific parameters. The selection of the physical schemes and the choice of the corresponding physical parameters during model configuration can significantly impact the accuracy of model forecasts. There is no combination of physical schemes that works best for all times, at all locations, and under all conditions. It is therefore of considerable interest to understand the interplay between the choice of physics and the accuracy of the resulting forecasts under different conditions. This paper demonstrates the use of machine learning techniques to study the uncertainty in numerical weather prediction models due to the interaction of multiple physical processes. The first problem addressed herein is the estimation of systematic model errors in output quantities of interest at future times, and the use of this information to improve the model forecasts. The second problem considered is the identification of those specific physical processes that contribute most to the forecast uncertainty in the quantity of interest under specified meteorological conditions. In order to address these questions we employ two machine learning approaches, random forests and artificial neural networks. The discrepancies between model results and observations at past times are used to learn the relationships between the choice of physical processes and the resulting forecast errors. Numerical experiments are carried out with the Weather Research and Forecasting (WRF) model. The output quantity of interest is the model precipitation, a variable that is both extremely important and very challenging to forecast. The physical processes under consideration include various micro-physics schemes, cumulus parameterizations, short wave, and long wave radiation schemes. The experiments demonstrate the strong potential of machine learning approaches to aid the study of model errors.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1997

Particle Scattering in the Resonance Regime: Full-Wave Solution for Axisymmetric Particles with Large Aspect Ratios

Reliable descriptions of the optical properties of clouds and aerosols are essential for studies of radiative transfer in planetary atmospheres. The scattering algorithms provide accurate estimates of these properties for spherical particles with a wide range of sizes and refractive indices, but these methods are not valid for non-spherical particles (e.g., ice crystals, mineral dust, and smoke). Even though a host of methods exist for deriving the optical properties of nonspherical particles that are very small or very large compared with the wavelength, only a few methods are valid in the resonance regime, where the particle dimensions are comparable with the wavelength. Most such methods are not ideal for particles with sharp edges or large axial ratios. We explore the utility of an integral equation approach for deriving the single-scattering optical properties of axisymmetric particles with large axial ratios. The accuracy of this technique is shown for spheres of increasing size parameters and an ensemble of randomly oriented prolate spheroids of size parameter equal to 10.079368. In this last case our results are compared with published results obtained with the T-matrix approach. Next we derive cross sections, single-scattering albedos, and phase functions for cylinders, disks, and spheroids of ice with dimensions extending from the Rayleigh to the geometric optics regime. Compared with those for a standard surface integral equation method, the storage requirement and the computer time needed by this method are reduced, thus making it attractive for generating databases to be used in multiple-scattering calculations. Our results show that water ice disks and cylinders are more strongly absorbing than equivalent volume spheres at most infrared wavelengths. The geometry of these particles also affects the angular dependence of the scattering. Disks and columns with maximum linear dimensions larger than the wavelength scatter much more radiation in the forward and backward directions and much less radiation at intermediate phase angles than equivalent volume spheres.

Zuffada, Cinzia↗

NASA NTRS2005

Probabilisitc Geobiological Classification Using Elemental Abundance Distributions and Lossless Image Compression in Recent and Modern Organisms

Last year we presented techniques for the detection of fossils during robotic missions to Mars using both structural and chemical signatures[Storrie-Lombardi and Hoover, 2004]. Analyses included lossless compression of photographic images to estimate the relative complexity of a putative fossil compared to the rock matrix [Corsetti and Storrie-Lombardi, 2003] and elemental abundance distributions to provide mineralogical classification of the rock matrix [Storrie-Lombardi and Fisk, 2004]. We presented a classification strategy employing two exploratory classification algorithms (Principal Component Analysis and Hierarchical Cluster Analysis) and non-linear stochastic neural network to produce a Bayesian estimate of classification accuracy. We now present an extension of our previous experiments exploring putative fossil forms morphologically resembling cyanobacteria discovered in the Orgueil meteorite. Elemental abundances (C6, N7, O8, Na11, Mg12, Ai13, Si14, P15, S16, Cl17, K19, Ca20, Fe26) obtained for both extant cyanobacteria and fossil trilobites produce signatures readily distinguishing them from meteorite targets. When compared to elemental abundance signatures for extant cyanobacteria Orgueil structures exhibit decreased abundances for C6, N7, Na11, All3, P15, Cl17, K19, Ca20 and increases in Mg12, S16, Fe26. Diatoms and silicified portions of cyanobacterial sheaths exhibiting high levels of silicon and correspondingly low levels of carbon cluster more closely with terrestrial fossils than with extant cyanobacteria. Compression indices verify that variations in random and redundant textural patterns between perceived forms and the background matrix contribute significantly to morphological visual identification. The results provide a quantitative probabilistic methodology for discriminating putatitive fossils from the surrounding rock matrix and &om extant organisms using both structural and chemical information. The techniques described appear applicable to the geobiological analysis of meteoritic samples or in situ exploration of the Mars regolith. Keywords: cyanobacteria, microfossils, Mars, elemental abundances, complexity analysis, multifactor analysis, principal component analysis, hierarchical cluster analysis, artificial neural networks, paleo-biosignatures

Storrie-Lombardi, Michael C.↗

NASA NTRS1993

A petabyte size electronic library using the N-Gram memory engine

A model library containing petabytes of data is proposed by Triada, Ltd., Ann Arbor, Michigan. The library uses the newly patented N-Gram Memory Engine (Neurex), for storage, compression, and retrieval. Neurex splits data into two parts: a hierarchical network of associative memories that store 'information' from data and a permutation operator that preserves sequence. Neurex is expected to offer four advantages in mass storage systems. Neurex representations are dense, fully reversible, hence less expensive to store. Neurex becomes exponentially more stable with increasing data flow; thus its contents and the inverting algorithm may be mass produced for low cost distribution. Only a small permutation operator would be recalled from the library to recover data. Neurex may be enhanced to recall patterns using a partial pattern. Neurex nodes are measures of their pattern. Researchers might use nodes in statistical models to avoid costly sorting and counting procedures. Neurex subsumes a theory of learning and memory that the author believes extends information theory. Its first axiom is a symmetry principle: learning creates memory and memory evidences learning. The theory treats an information store that evolves from a null state to stationarity. A Neurex extracts information data without a priori knowledge; i.e., unlike neural networks, neither feedback nor training is required. The model consists of an energetically conservative field of uniformly distributed events with variable spatial and temporal scale, and an observer walking randomly through this field. A bank of band limited transducers (an 'eye'), each transducer in a bank being tuned to a sub-band, outputs signals upon registering events. Output signals are 'observed' by another transducer bank (a mid-brain), except the band limit of the second bank is narrower than the band limit of the first bank. The banks are arrayed as n 'levels' or 'time domains, td.' The banks are the hierarchical network (a cortex) and transducers are (associative) memories. A model Neurex was built and studied. Data were 50 MB to 10 GB samples of text, data base, and images: black/white, grey scale, and high resolution in several spectral bands. Memories at td, S(m(sub td)), were plotted against outputs of memories at td-1. S(m(sub td)) was Boltzman distributed, and memory frequencies exhibited self-organized criticality (SOC); i.e., 'l/f(sup beta)' after long exposures to data. Whereas output signals from level n may be encoded with B(sub output) = O(-log(2)f(sup beta)) bits, and input data encoded with B(sub input) = O((S(td)/S(td-1))(sup n)), B(sup output)/B(sub input) is much less than 1 always, the Neurex determines a canonical code for data and it is a lossless data compressor. Further tests are underway to confirm these results with more data types and larger samples.

Bugajski, Joseph M.↗

DOE OSTI2022

Kinetic theory of particle-in-cell simulation plasma and the ensemble averaging technique

Abstract We derive the kinetic theory of fluctuations in physically and numerically stable particle-in-cell (PIC) simulations of electrostatic plasmas. The starting point is the single-time correlation at the start of the simulation between the statistical fluctuations of the weighted densities of macroparticle centers in the plasma particle phase-space. The fluctuations are associated with different initial conditions, typically due to the random initial conditions (in velocity space) of the macroparticles/simulation plasma, assigned according to their initial distribution of probability. The single-time correlations at all time steps and in each spatial grid cell are then determined from the Laplace–Fourier transforms of the discretized Klimontovich-like equation for the macroparticles and Maxwell’s equations for the fields, as computed by modern PIC codes. We recover the expressions for the electrostatic field and the plasma particle density fluctuation autocorrelation spectra as well as the kinetic equations describing the average evolution of PIC-simulated plasma particles, first derived by Langdon (1970b Proc. 4th Conf. Numerical Simulation of Plasmas ) using a test macroparticle approach perturbing a discretized Vlasovian plasma and then averaging the obtained physical quantity over the initial macroparticle velocity distribution. We generalize and extend these results to the modern algorithms in PIC codes using arbitrary macroparticle weights. Analytical estimates of statistical fluctuation amplitudes are derived as a function of the plasma simulation parameters, using the central limit theorem in the limit of a large number of macroparticles per cell. The theory is then used to analyze the ensemble averaging technique of PIC simulations where statistical averages are performed over ensembles of PIC simulations, modeling the same plasma physics problem but using different statistical realizations of the initial distribution functions of the macroparticles. This method is illustrated by linear Landau damping uncovering (from noise, which is usually considered numerical) the physical fluctuations driven by a single small amplitude electrostatic wave perturbing a PIC simulation plasma in equilibrium.

fluctuations correlations↗

NASA NTRS1995

A new approach to the linear theory of single-species tearing in two-dimensional quasi-neutral sheets

We have developed the linear theory of collisionless ion tearing in a two-dimensional magnetotail equilibrium for a single resonant species. We have solved the normal mode problem for tearing instability by an algorithm that employs particle-in-cell simulation to calculate the orbit integrals in the Maxwell-Vlasov eigenmode equation. The results of our single-species tearing analysis can be applied to ion tearing where electron effects are not included. We have calculated the tearing growth rate as a function of the magnetic field component B(sub n) normal to the current sheet for thick and thin current sheets, and we show that marginal stability occurs when the normal gyrofrequency Omega(sub n) is comparable to the Harris neutral sheet growth rate. A cross-tail B(sub y) component has little effect on the growth rate for B(sub y) approximately = B(sub n). Even in the limit B(sub y) much greater than B(sub n), the mode is strongly stabilized by B(sub n). We report than random pitch angle scattering can overcome the stabilizing effect of B(sub n) and drive the growth rate up toward the Harris neutral sheet (B(sub n) = 0) value when the pitch angle diffusion rate is comparable to Omega(sub n).

Brittnacher, M.↗

NASA NTRS2011

Freeboard, Snow Depth and Sea-Ice Roughness in East Antarctica from In Situ and Multiple Satellite Data

In October 2003 a campaign on board the Australian icebreaker Aurora Australis had the objective to validate standard Aqua Advanced Microwave Scanning Radiometer (AMSR-E) sea-ice products. Additionally, the satellite laser altimeter on the Ice, Cloud and land Elevation Satellite (ICESat) was in operation. To capture the large-scale information on the sea-ice conditions necessary for satellite validation, the measurement strategy was to obtain large-scale sea-ice statistics using extensive sea-ice measurements in a Lagrangian approach. A drifting buoy array, spanning initially 50 km 100 km, was surveyed during the campaign. In situ measurements consisted of 12 transects, 50 500 m, with detailed snow and ice measurements as well as random snow depth sampling of floes within the buoy array using helicopters. In order to increase the amount of coincident in situ and satellite data an approach has been developed to extrapolate measurements in time and in space. Assuming no change in snow depth and freeboard occurred during the period of the campaign on the floes surveyed, we use buoy ice-drift information as well as daily estimates of thin-ice fraction and rough-ice vs smooth-ice fractions from AMSR-E and QuikSCAT, respectively, to estimate kilometer-scale snow depth and freeboard for other days. The results show that ICESat freeboard estimates have a mean difference of 1.8 cm when compared with the in situ data and a correlation coefficient of 0.6. Furthermore, incorporating ICESat roughness information into the AMSR-E snow depth algorithm significantly improves snow depth retrievals. Snow depth retrievals using a combination of AMSR-E and ICESat data agree with in situ data with a mean difference of 2.3 cm and a correlation coefficient of 0.84 with a negligible bias.

Markus, Thorsten↗

NASA NTRSDate not supplied

Parallelized Convolutional Interleaver Implementation for Efficient DDR Memory Access

Convolutional interleavers are used in many different communications systems to correct for burst errors due to atmospheric fades and scintillation. The interleaver size is related to the channel coherence time and the data rate. Small convolutional interleavers can be implemented in a field programmable gate array (FPGA) block random access memory (BRAM). However, large interleavers exceeding the size of the BRAM on the FPGA are necessary for channels with longer fades and higher data rates. Therefore, an implementation utilizing double data rate (DDR) memory external to the FPGA is necessary. Wide DDR memory data buses can make the use of DDR memory for convolutional interleavers inefficient when individual symbols are written to and read from the memory. DDR memory operational speeds can also limit the data rate of the interleaver. The Consultative Committee for Space Data Systems (CCSDS) Optical Communications High Photon Efficiency (HPE) standard utilizes a convolutional channel symbol interleaver. A previous implementation of the HPE standard utilized BRAM for the convolutional interleaver, but mission requirements for the upcoming Optical Artemis-2 Orion (O2O) communications demonstration dictate the use of an interleaver exceeding the size of the BRAM. An algorithm and method for implementing the convolutional interleaver in the FPGA with DDR memory is described in this paper.

optical communications↗

NASA NTRSDate not supplied

Parallelized convolutional interleaver implementation for efficient DDR memory access

Optical communications↗

DOE OSTI2020

Efficient numerical methods to solve sparse linear equations with application to PageRank

Over the last two decades, the PageRank problem has received increased interest from the academic community as an efficient tool to estimate web-page importance in information retrieval. Despite numerous developments, the design of efficient optimization algorithms for the PageRank problem is still a challenge. Here, we propose three new algorithms with a linear time complexity for solving the problem over a bounded-degree graph. The idea behind them is to set up the PageRank as a convex minimization problem over a unit simplex, and then solve it using iterative methods with small iteration complexity. Our theoretical results are supported by an extensive empirical justification using real-world and simulated data.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2013

Wind Retrieval Algorithms for the IWRAP and HIWRAP Airborne Doppler Radars with Applications to Hurricanes

Algorithms for the retrieval of atmospheric winds in precipitating systems from downward-pointing, conically-scanning airborne Doppler radars are presented. The focus in the paper is on two radars: the Imaging Wind and Rain Airborne Profiler(IWRAP) and the High-altitude IWRAP (HIWRAP). The IWRAP is a dual-frequency (Cand Ku band), multi-beam (incidence angles of 30 50) system that flies on the NOAAWP-3D aircraft at altitudes of 2-4 km. The HIWRAP is a dual-frequency (Ku and Kaband), dual-beam (incidence angles of 30 and 40) system that flies on the NASA Global Hawk aircraft at altitudes of 18-20 km. Retrievals of the three Cartesian wind components over the entire radar sampling volume are described, which can be determined using either a traditional least squares or variational solution procedure. The random errors in the retrievals are evaluated using both an error propagation analysis and a numerical simulation of a hurricane. These analyses show that the vertical and along-track wind errors have strong across-track dependence with values of 0.25 m s-1 at nadir to 2.0 m s-1 and 1.0 m s-1 at the swath edges, respectively. The across-track wind errors also have across-track structure and are on average, 3.0 3.5 m s-1 or 10 of the hurricane wind speed. For typical rotated figure four flight patterns through hurricanes, the zonal and meridional wind speed errors are 2 3 m s-1.Examples of measured data retrievals from IWRAP during an eyewall replacement cycle in Hurricane Isabel (2003) and from HIWRAP during the development of Tropical Storm Matthew (2010) are shown.

atmospheric winds↗

DOE OSTI2021

Application of Markov Chain Monte Carlo Methods for Uncertainty Quantification in Inverse Transport Problems

Determination of the components of a radioactive source/shield system using the system’s radiation signature is of great importance in homeland security, material safeguards, and waste management. Although significant progress has been made toward solving this inverse transport problem in recent years, work remains to be done to quantify the uncertainty in reconstructed results. In this article we apply two Markov chain Monte Carlo (MCMC) approaches, the delayed rejection adaptive metropolis (DRAM) and differential evolution adaptive metropolis (DREAM) methods, to solve inverse problems and quantify uncertainty. The DRAM method uses delayed rejection combined with global adaptation of the proposal covariance matrix. Furthermore, the DREAM method hybridizes MCMC sampling with the differential evolution (DE) algorithm. In numerical test cases, the DRAM and DREAM methods are shown to be superior to a first-order inverse Hessian approach for problems with noisy data and multiple unknown quantities, with DREAM converging to the posterior distribution more quickly than DRAM. The DREAM and DRAM results indicate that a full posterior distribution is required to quantify uncertainty in many inverse transport problems.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2020

Neural MUSE Analysis

Researchers at Oak Ridge National Laboratory (ORNL) created data as part of the MUSE (Multi-Agency Urban Search Experiment Detector and Algorithm Test Bed) project simulating illicit nuclear materials located in various buildings along a road. In the simulation, a truck containing a radiation detector drives down the road gathering listmode data (counting the and energy of incident gamma radiation). Building materials, source shielding, driving speed, truck direction, truck location on the road, source type, and source placement are all varied between runs of the data set. This data was created using deterministic neutron transport and Monte Carlo methods through a combination of SCALE, MAVRIC, MCNP, and GADRAS. As part of a follow-on NA-22 project, two Kaggle competitions were created to determine the best algorithms for finding and identifying gamma sources in this simulated urban environment. The winning algorithm was neural network-based and had a test accuracy of 76.4% accuracy for source identification. This work seeks to build upon this work and improve the results through the application of novel machine learning techniques. As a first step, the data was classified by a simple Convolutional Neural Network (CNN) To accomplish this, the data was first preprocessed into “waterfall plots.” These plots are composed of energy vs count plots that are stacked vertically to show progression in time. The horizontal axis indicating the particle energy incorporated user defined bin spacing with options for in linear-, logarithmic-, square root-, and user-spaced bins. The z or color dimension showed the number of counts corresponding the energy-time combination. This data was then used to generate more data, by generating a local estimate of the mean of the distribution for a bin and then randomly re-sampling that bin from a Poisson distribution. Once all of this data was generated, it was fed into a well-known CNN architecture, ResNet50. The output layer of this model was removed and replaced with layers corresponding to the shape desired isotope outputs. The provided training data was used to train the classifier and the remaining testing data was used to evaluate the model. Results are soon to be forthcoming.

61 RADIATION PROTECTION AND DOSIMETRY↗

NASA NTRS2024

Noise Reduction for Solar-Induced Fluorescence Retrievals Using Machine Learning and Principal Component Analysis: Simulations and Applications to GOME-2 Satellite Retrievals

We use a spectral-based approach that employs principal component analysis along with a relatively shallow artificial neural network (NN) to substantially reduce noise and other artifacts in terrestrial chlorophyll solar-induced fluorescence (SIF) retrievals. SIF is a very small emission at red and far-red wavelengths that is difficult to measure and is highly sensitive to random errors and systematic artifacts. Our approach relies upon an assumption that a trained NN can effectively reconstruct the total SIF signal from a relatively small number of leading principal components of the satellite-observed far-red radiance spectra without using information from the trailing modes that contain most of the random errors. We test the approach with simulated reflectance spectra produced with a full atmospheric and surface radiative transfer model using different observing and geophysical parameters and various noise levels. Resulting noisy and noise-reduced retrieved SIF values are compared with true values to assess performance. We then apply our noise reduction approach to SIF derived from two different satellite spectrometers. For evaluation, since the truth in this case is unknown, we compare SIF retrievals from two independent sensors with each other. We also compare the noise-reduced SIF temporal variations with those from an independent gross primary product (GPP) product that should display similar variations. Results show that our noise reduction approach improves capture of SIF seasonal and inter-annual variability. Our approach should be applicable to many noisy data products derived from spectral measurements. Our methodology does not replace the original retrieval algorithms; rather the original noisy retrievals are needed as the target for the NN training process.

noise reduction, solar-induced fluorescence, SIF, ↗