Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Retrieval Algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Evaluation of Physical Microphysical Property Retrieval Algorithms During the 2020 IMPACTS Field Campaign

The NASA Investigation of Microphysics and Precipitation for Atlantic Coast Threatening Snowstorms (IMPACTS) field campaign provides high-quality, high-altitude aircraft lidar (532 nm), radar (W-band) and in-cloud microphysical aircraft data taken during wintertime storm events impacting the United States. This study evaluates two mass-dimensional relationships (Brown and Francis (1995, BF95); Heymsfield (2014, H14) and two lidar-radar microphysical retrieval algorithms (Cloudsat and CALIPSO Ice Cloud Property Product (2C-ICE); VarPy (a variational method derived from the satellite lidar-radar data community)) to estimate aircraft-retrieved volume extinction coefficient (σ), ice water content (IWC), and effective radius (r e ) during the 2020 IMPACTS deployment. BF95 and H14 have a close 1:1 correlation (R 2 = 0.98) with in-situ observations of σ. However, only BF95 displays a linear, consistent, and almost temperature-independent low bias for IWC and r e , which likely arises from the environmental conditions used to determine each. Unlike the field-campaign-derived BF95 and H14 relationships, VarPy and 2C-ICE directly ingest the aircraft-based lidar and radar data to simulate σ, IWC, and r e . For all three microphysical parameters, VarPy and 2C-ICE retrieval errors became notably more pronounced around the dendritic growth zone (-15°C to -10°C) and near freezing (≥-5°C), which suggests that both algorithms experience difficulty addressing riming and aggregation processes and with larger particles (dendrites and plates) due in part to their simplified ice particle assumptions. However, the mean-melt diameter ice-particle assumption did yield more accurate IWC estimates, which led to slightly better overall results for VarPy.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Genetic-algorithm retrieval of the molecular alignment distribution with high-order harmonics generated from transiently aligned CO 2 molecules

Using genetic algorithm, we propose a method to retrieve the alignment distribution of transiently aligned CO 2 molecules from the high-order harmonic generation (HHG) spectra. The retrieval method is based on the quantitative rescattering (QRS) model where averaged photorecombination transition dipole can be factored out from the parallel (or perpendicular) harmonic spectra after the propagation of the harmonic fields in the gas medium. We examine how the retrieved alignment distributions are affected by uncertainty in alignment dependent ionization probability and on multiple orbital contribution to the HHG. We further confirm that alignment distribution is more accurately retrieved by using the minima in the HHG spectra driven by a long-wavelength laser. In addition, we show that earlier experimental data on the ratios between the perpendicular and the parallel HHG components of aligned CO 2 molecules are in better agreement with the QRS model if the macroscopic propagation and multiple orbital interference are included in the theoretical calculation.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2021

Version 4 CALIPSO Imaging Infrared Radiometer ice and liquid water cloud microphysical properties – Part I: The retrieval algorithms

Following the release of the version 4 Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) data products from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) mission, a new version (version 4; V4) of the CALIPSO Imaging Infrared Radiometer (IIR) Level 2 data products has been developed. The IIR Level 2 data products include cloud effective emissivities and cloud microphysical properties such as effective diameter and ice or liquid water path estimates. Dedicated retrievals for water clouds were added in V4, taking advantage of the high sensitivity of the IIR retrieval technique to small particle sizes. This paper (Part I) describes the improvements in the V4 algorithms compared to those used in the version 3 (V3) release, while results will be presented in a companion (Part II) paper. The IIR Level 2 algorithm has been modified in the V4 data release to improve the accuracy of the retrievals in clouds of very small (close to 0) and very large (close to 1) effective emissivities. To reduce biases at very small emissivities that were made evident in V3, the radiative transfer model used to compute clear-sky brightness temperatures over oceans has been updated and tuned for the simulations using Modern-Era Retrospective analysis for Research and Applications version 2 (MERRA-2) data to match IIR observations in clear-sky conditions. Furthermore, the clear-sky mask has been refined compared to V3 by taking advantage of additional information now available in the V4 CALIOP 5 km layer products used as an input to the IIR algorithm. After sea surface emissivity adjustments, observed and computed brightness temperatures differ by less than ±0.2 K at night for the three IIR channels centered at 08.65, 10.6, and 12.05 µm, and inter-channel biases are reduced from several tens of Kelvin in V3 to less than 0.1 K in V4. We have also improved retrievals in ice clouds having large emissivity by refining the determination of the radiative temperature needed for emissivity computation. The initial V3 estimate, namely the cloud centroid temperature derived from CALIOP, is corrected using a parameterized function of temperature difference between cloud base and top altitudes, cloud absorption optical depth, and CALIOP multiple scattering correction factor. As shown in Part II, this improvement reduces the low biases at large optical depths that were seen in V3 and increases the number of retrievals. As in V3, the IIR microphysical retrievals use the concept of microphysical indices applied to the pairs of IIR channels at 12.05 and 10.6 µm and at 12.05 and 08.65 µm. The V4 algorithm uses ice look-up tables (LUTs) built using two ice habit models from the recent “TAMUice2016” database, namely the single-hexagonal-column model and the eight-element column aggregate model, from which bulk properties are synthesized using a gamma size distribution. Four sets of effective diameters derived from a second approach are also reported in V4. Here, the LUTs are analytical functions relating microphysical index applied to IIR channels 12.05 and 10.6 µm and effective diameter as derived from in situ measurements at tropical and midlatitudes during the Tropical Composition, Cloud, and Climate Coupling (TC4) and Small Particles in Cirrus Science and Operations Plan (SPARTICUS) field experiments.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Optimal Estimation Retrievals and Their Uncertainties: What Every Atmospheric Scientist Should Know

Remote sensing instruments are heavily used to provide observations for both the operational and research communities. These sensors do not provide direct observations of the desired atmospheric variables, but instead, retrieval algorithms are necessary to convert the indirect observations into the variable of interest. It is critical to be aware of the underlying assumptions made by many retrieval algorithms, including that the retrieval problem is often ill posed and that there are various sources of uncertainty that need to be treated properly. In short, the retrieval challenge is to invert a set of noisy observations to obtain estimates of atmospheric quantities. The problem is often complicated by imperfect forward models, by imperfect prior knowledge, and by the existence of nonunique solutions. Optimal estimation (OE) is a widely used physical retrieval method that combines measurements, prior information, and the corresponding uncertainties based on Bayes’s theorem to find an optimal solution for the atmospheric state. Furthermore, OE also allows the relative contributions of the different sources of error to the uncertainty in the final retrieved atmospheric state to be understood. Here, we provide a novel Python library to illustrate the use of OE for inverse problems in the atmospheric sciences. We introduce two example problems: how to retrieve drop size distribution parameters from radar observations and how to retrieve the temperature profile from ground-based microwave sensors. Using these examples, we discuss common pitfalls, how the various error sources impact the retrieval, and how the quality of the retrieval results can be quantified.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Improving near real-time precipitation estimation using a U-Net convolutional neural network and geographical information

Reliable near real-time precipitation estimates are essential for monitoring and managing of natural disasters such as floods. Quality of inputs and capability of the retrieval algorithm are two important aspects for developing satellite-based precipitation datasets. Most retrieval algorithms utilize infrared (IR) information as their input due to its fine spatiotemporal resolution and near-instantaneous availability. However, their sole reliance on IR information limits their capability to learn different mechanisms of precipitation during training, resulting in less accurate estimates. Moreover, recent advances in the field of machine learning offer attractive opportunities to improve the precipitation retrieval algorithms. This work investigates the effectiveness of adding geographical information (i.e. latitude and longitude) to IR information and the application of a U-Net-based convolutional neural network for improving the accuracy of retrieval algorithms. This research suggests that applying an appropriate CNN architecture on geographical and IR information provides an opportunity to improve the satellite-based precipitation products.

42 ENGINEERING↗
DOE OSTI2020

Encoding the complete electric field of an ultraviolet ultrashort laser pulse in a near-infrared nonlinear-optical signal

We introduce a variation on the cross-correlation frequency-resolved optical gating (XFROG) technique that uses a near-infrared (NIR) nonlinear-optical signal to characterize pulses in the ultraviolet (UV). Using a transient-grating XFROG beam geometry, we create a grating using two copies of the unknown UV pulse and diffract a NIR reference pulse from it. We show that, by varying the delay between the UV pulses creating the grating, the UV pulse intensity-and-phase information can be encoded into a NIR signal. We also implemented a modified generalized-projections phase-retrieval algorithm for retrieving the UV pulses from these spectrograms. We performed proof-of-principle measurements of chirped pulses and double pulses, all at 400 nm. This approach should be extendable deeper into the UV and potentially even into the extreme UV or x-ray range.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

PyPhase – a Python package for X-ray phase imaging

X-ray propagation-based imaging techniques are well established at synchrotron radiation and laboratory sources. However, most reconstruction algorithms for such image modalities, also known as phase-retrieval algorithms, have been developed specifically for one instrument by and for experts, making the development and diffusion of such techniques difficult. Here, PyPhase, a free and open-source package for propagation-based near-field phase reconstructions, which is distributed under the CeCILL license, is presented. PyPhase implements some of the most popular phase-retrieval algorithms in a highly modular framework supporting its deployment on large-scale computing facilities. This makes the integration, the development of new phase-retrieval algorithms, and the deployment on different computing infrastructures straightforward. Its capabilities and simplicity are presented by application to data acquired at the synchrotron source MAX IV (Lund, Sweden).

36 MATERIALS SCIENCE↗
DOE OSTI2023

Retrieval of temperature and humidity profiles from ground-based high-resolution infrared observations using an adaptive fast iterative algorithm

Various retrieval algorithms have been developed for retrieving temperature and water vapor profiles from Atmospheric Emitted Radiance Interferometer (AERI) observations. The physical retrieval algorithm, named AERI Optimal Estimation (AERIoe), outperforms other retrieval algorithms in many aspects except the retrieval time, which is significantly increased due to the complex radiative transfer process. The calculation of the Jacobian matrix is the most computationally intensive step of the physical retrieval algorithm. Interestingly, an analysis of the change in AERI observations' information content with respect to Jacobians revealed that the AERIoe algorithm's performance presents negligible dependence on these metrics. Thus, the Jacobian matrix could remain unchanged when the variation in the atmospheric state is small in the retrieval process to reduce the most time-consuming computation. On the basis of the above findings, a fast physical–iterative retrieval algorithm was proposed by adaptively recalculating Jacobians in keeping with the changes in the atmospheric state. Experiments with synthetic observations demonstrate that the proposed method experiences an average reduction in retrieval time by an impressive 59 % compared to the original AERIoe algorithm while achieving maximum root-mean-square errors of less than 0.95 K and 0.22 log(ppmv) for heights below 3 km for the temperature and water vapor profile, respectively. Further analyses revealed that the fast-retrieval algorithm reached an acceptable convergence rate of 98.7 %, marginally lower than AERIoe's 99.9 % convergence rate for the 826 cases used in this study.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

An Algorithm to Retrieve Total Precipitable Water Vapor in the Atmosphere from FengYun 3D Medium Resolution Spectral Imager 2 (FY-3D MERSI-2) Data

The atmosphere has substantial effects on optical remote sensing imagery of the Earth’s surface from space. These effects come through the functioning of atmospheric particles on the radiometric transfer from the Earth’s surface through the atmosphere to the sensor in space. Precipitable water vapor (PWV), CO 2 , ozone, and aerosol in the atmosphere are very important among the particles through their functioning. This study presented an algorithm to retrieve total PWV from the Chinese second-generation polar-orbiting meteorological satellite FengYun 3D Medium Resolution Spectral Imager 2 (FY-3D MERSI-2) data, which have three near-infrared (NIR) water vapor absorbing channels, i.e., channel 16, 17, and 18. The algorithm was improved from the radiance ratio technique initially developed for Moderate-Resolution Imaging Spectroradiometer (MODIS) data. MODTRAN 5 was used to simulate the process of radiant transfer from the ground surfaces to the sensor at various atmospheric conditions for estimation of the coefficients of ratio technique, which was achieved through statistical regression analysis between the simulated radiance and transmittance values for FY-3D MERSI-2 NIR channels. The algorithm was then constructed as a linear combination of the three-water vapor absorbing channels of FY-3D MERSI-2. Measurements from two ground-based reference datasets were used to validate the algorithm: the sun photometer measurements of Aerosol Robotic Network (AERONET) and the microwave radiometer measurements of Energy’s Atmospheric Radiation Measurement Program (ARMP). The validation results showed that the algorithm performs very well when compared with the ground-based reference datasets. The estimated PWV values come with root mean square error (RMSE) of 0.28 g/cm 2 for the ARMP and 0.26 g/cm 2 for the AERONET datasets, with bias of 0.072 g/cm 2 and 0.096 g/cm 2 for the two reference datasets, respectively. The accuracy of the proposed algorithm revealed a better consistency with ground-based reference datasets. Thus, the proposed algorithm could be used as an alternative to retrieve PWV from FY-3D MERSI-2 data for various remote sensing applications such as agricultural monitoring, climate change, hydrologic cycle, and so on at various regional and global scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Characterization of single-shot attosecond pulses with angular streaking photoelectron spectra

Most of the traditional attosecond pulse retrieval algorithms are based on a so-called attosecond streak camera technique, in which the momentum of the electron is shifted by an amount depending on the relative time delay between the attosecond pulse and the streaking infrared pulse. Thus, temporal information of the attosecond pulse is encoded in the amount of momentum shift in the streaked photoelectron momentum spectrogram S(p,τ), where p is the momentum of the electron along the polarization direction and τ is the time delay. An iterative algorithm is then employed to reconstruct the attosecond pulse from the streaking spectrogram. This method, however, cannot be applied to attosecond pulses generated from free-electron x-ray lasers where each single shot is different and stochastic in time. However, using a circularly polarized infrared laser as the streaking field, a two (or three)-dimensional angular streaking electron spectrum can be used to retrieve attosecond pulses for each shot, as well as the time delay with respect to the circularly polarized IR field. Here we show that a retrieval algorithm previously developed for the traditional streaking spectrogram can be modified to efficiently characterize single-shot attosecond pulses. The methods have been applied to retrieve 188 single shots from recent experiments. We analyze the statistical behavior of these 188 pulses in terms of pulse duration, bandwidth, pulse peak energy, and time delay with respect to the IR field. Furthermore, the retrieval algorithm is efficient and can be easily used to characterize a large number of shots in future experiments for attosecond pulses at free-electron x-ray laser facilities.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2023

Bayesian Cloud Property Retrievals from ARM Active and Passive Measurements

The optimum use of the continuous measurements of thermodynamics, radiation, aerosols, clouds and precipitation from the DOE Atmospheric Radiation Measurement (ARM) program is key to achieve the DOE Atmospheric System Research (ASR)’s objectives. One of the key mission requirements is to retrieve cloud and precipitation properties, as well as vertical motion parameters, along the vertical cross- section defined by the profiling active sensors. Such retrievals are challenging to perform continuously in the entire spectrum of cloud and precipitation conditions due to the large natural microphysical and dynamical variability, the often-limited information content in the measurements, and the lack of proper characterization of measurement quality and uncertainty. Today, the acquisition of new remote and in-situ sensors by the ARM program creates opportunities to address the microphysical retrieval problem by exploiting new, more robust retrieval techniques and integrating various scattered advancements in both sensor techniques and retrieval algorithms. During this project, we constructed a robust Bayesian Markov chain Monte Carlo (MCMC) cloud property retrieval algorithm that includes a state of the art radar forward model. Our MCMC-based retrieval produces both the best estimate of height-resolved cloud and precipitation properties in the radar profile, as well as an estimate of the in-cloud vertical motion and turbulence. In addition, the MCMC algorithm automatically produces robust and flexible estimates of retrieval uncertainty. We tested the algorithm on several synthetic cloud profiles obtained from large eddy simulation (LES) models with bin-resolved microphysics.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Evaluation of four ground-based retrievals of cloud droplet number concentration in marine stratocumulus with aircraft in situ measurements

Abstract. Cloud droplet number concentration (Nd) is crucial for understanding aerosol–cloud interactions (ACI) and associated radiative effects. We present evaluations of four ground-based Nd retrievals based on comprehensive datasets from the Atmospheric Radiation Measurement (ARM) Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. The Nd retrieval methods use ARM ENA observatory ground-based remote sensing observations from a micropulse lidar, Raman lidar, cloud radar, and the ARM NDROP (Droplet Number Concentration) value-added product (VAP), all of which also retrieve cloud effective radius (re). The retrievals are compared against aircraft measurements from the fast cloud droplet probe (FCDP) and the cloud and aerosol spectrometer (CAS) obtained from low-level marine boundary layer clouds on 12 flight days during summer and winter seasons. Additionally, the in situ measurements are used to validate the assumptions and characterizations used in the retrieval algorithms. Statistical comparisons of the probability distribution function (PDF) of the Nd and cloud re retrievals with aircraft measurements demonstrate that these retrievals align well with in situ measurements for overcast clouds, but they may substantially differ for broken clouds or clouds with low liquid water path (LWP). The retrievals are applied to 4 years of ground-based remote sensing measurements of overcast marine boundary layer clouds at the ARM ENA observatory to find that Nd (re) values exhibit seasonal variations, with higher (lower) values during the summer season and lower (higher) values during the winter season. The ensemble of various retrievals using different measurements and retrieval algorithms such as those in this paper can help to quantify Nd retrieval uncertainties and identify reliable Nd retrieval scenarios. Of the retrieval methods, we recommend using the micropulse lidar-based method. This method has good agreement with in situ measurements, less sensitivity to issues arising from precipitation and low cloud LWP and/or optical depth, and broad applicability by functioning for both daytime and nighttime conditions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Low-pass spectral analysis of time-resolved serial femtosecond crystallography data

Low-pass spectral analysis (LPSA) is a recently developed dynamics retrieval algorithm showing excellent retrieval properties when applied to model data affected by extreme incompleteness and stochastic weighting. In this work, we apply LPSA to an experimental time-resolved serial femtosecond crystallography (TR-SFX) dataset from the membrane protein bacteriorhodopsin (bR) and analyze its parametric sensitivity. While most dynamical modes are contaminated by nonphysical high-frequency features, we identify two dominant modes, which are little affected by spurious frequencies. The dynamics retrieved using these modes shows an isomerization signal compatible with previous findings. We employ synthetic data with increasing timing uncertainty, increasing incompleteness level, pixel-dependent incompleteness, and photon counting errors to investigate the root cause of the high-frequency contamination of our TR-SFX modes. By testing a range of methods, we show that timing errors comparable to the dynamical periods to be retrieved produce a smearing of dynamical features, hampering dynamics retrieval, but with no introduction of spurious components in the solution, when convergence criteria are met. Using model data, we are able to attribute the high-frequency contamination of low-order dynamical modes to the high levels of noise present in the data. Finally, we propose a method to handle missing observations that produces a substantial dynamics retrieval improvement from synthetic data with a significant static component. Reprocessing of the bR TR-SFX data using the improved method yields dynamical movies with strong isomerization signals compatible with previous findings.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Cloud optical depth and liquid effective radius retrievals from SPHOT zenith radiances

A new three-channel (440, 870, and 1640 nm) retrieval algorithm for cloud optical depth has recently been developed by Christine Chiu and her co-workers [Chiu et al., 2012]. This VAP is to implement the three-channel cloud optical depth retrieval algorithm as an ARM operational VAP. First, simultaneously retrieved cloud optical depth and effective radius datasets will be generated by using the pre-calibrated zenith radiance measurements taken from the ARM Sun-Photometer (SPHOT) and the NASA AERONET and Satellite-based surface albedo derived from MODSI. Finally, the algorithm produces uncertainties along with the retrieval products. This VAP outputs a daily NetCDF file.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Evaluation of Four Ground-based Retrievals of Cloud Droplet Number Concentration in Marine Stratocumulus with Aircraft In Situ Measurements

Cloud droplet number concentration (N d ) is crucial for understanding aerosol-cloud interactions (ACI) and associated radiative effects. We present evaluations of four ground-based N d retrievals based on comprehensive datasets from the Atmospheric Radiation Measurements (ARM) Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. The N d retrieval methods use ARM ENA observatory ground-based remote sensing observations from a Micropulse lidar, Raman lidar, cloud radar, and the ARM NDROP Value-added Product (VAP), all of which also retrieve cloud effective radius (r e ). The retrievals are compared against aircraft measurements from the Fast-Cloud Droplet Probe (FCDP) and the Cloud and Aerosol Spectrometer (CAS) obtained from low-level marine boundary layer clouds on 12 flight days during summer and winter seasons. Additionally, the in situ measurements are used to validate the assumptions and characterizations used in the retrieval algorithms. Statistical comparisons of the probability distribution function (PDF) of the N d and cloud r e retrievals with aircraft measurements demonstrate that these retrievals align well with in situ measurements for overcast clouds, but they may substantially differ for broken clouds or clouds with low liquid water path (LWP). The retrievals are applied to four years of ground-based remote sensing measurements of overcast marine boundary layer clouds at the ARM ENA observatory to find that N d (r e ) values exhibit seasonal variations, with higher (lower) values during the summer season and lower (higher) values during the winter season. The ensemble of various retrievals using different measurements and retrieval algorithms such as those in this paper can help to quantify N d retrieval uncertainties and identify reliable N d retrieval scenarios. Of the retrieval methods, we recommend using the using the Micropulse lidar-based method given its good agreement with in situ measurements, it has less sensitivity to issues arising from precipitation and low cloud LWP/optical depth, and it has broad applicability by functioning for both day and nighttime conditions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Exploring Mission Design for Imaging Spectroscopy Retrievals for Land and Aquatic Ecosystems

The retrieval algorithms used for optical remote sensing satellite data to estimate Earth's geophysical properties have specific requirements for spatial resolution, temporal revisit, spectral range and resolution, and instrument signal-to-noise ratio (SNR) performance to meet biogeoscience objectives. Studies to estimate surface properties from hyperspectral data use a range of algorithms sensitive to various sources of spectroscopic uncertainty, which are in turn influenced by mission architecture choices. Retrieval algorithms vary across scientific fields and may be more or less sensitive to mission architecture choices that affect spectral, spatial, or temporal resolutions and spectrometer SNR. We used representative remote sensing algorithms across terrestrial and aquatic study domains to inform aspects of mission design that are most important for impacting accuracy in each scientific area. We simulated the propagation of uncertainties in the retrieval process including the effects of different instrument configuration choices. We found that retrieval accuracy and information content degrade consistently at >10 nm spectral resolution, >30 m spatial resolution, and >8-day revisit. In these studies, the noise reduction associated with lower spatial resolution improved accuracy vis à vis high spatial resolution measurements. The interplay between spatial resolution, temporal revisit, and SNR can be quantitatively assessed for imaging spectroscopy missions and used to identify key components of algorithm performance and mission observing criteria.

54 ENVIRONMENTAL SCIENCES↗