SEARCH · Engineering Papers

Results for “Retrieval Algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Phase retrieval for refraction-enhanced x-ray radiography using a deep neural network

X-ray refraction-enhanced radiography (RER) or phase contrast imaging is widely used to study internal discontinuities within materials. The resulting radiograph captures both the decrease in intensity caused by material absorption along the x-ray path, as well as the phase shift, which is highly sensitive to gradients in density. A significant challenge lies in effectively analyzing the radiographs to decouple the intensity and phase information and accurately ascertain the density profile. Conventional algorithms often yield ambiguous and unrealistic results due to difficulties in including physical constraints and other relevant information. We have developed an algorithm that uses a deep neural network to address these issues and applied it to extract the detailed density profile from an experimental RER. To generalize the applicability of our algorithm, we have developed a technique that quantitatively evaluates the complexity of the phase retrieval process based on the characteristics of the sample and the configuration of the experiment. Accordingly, this evaluation aids in the selection of the neural network architecture for each specific case. Beyond RER, the model has potential applications for other diagnostics where phase retrieval analysis is required.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2023

TRACER CLAMPS2 MWR TROPoe Retrievals

These files contain 24 hour periods of retrieved thermodynamic profiles derived from observations collected by the CLAMPS2 Atmospheric Emitted Radiance Interferometer (AERI) and microwave radiometer (MWR). The TROPoe algorithm is a python equivalent to the AERIoe algorithm (see Turner and Loehnert, 2014; Turner and Blumberg 2019). These data were collected during the TRACER-CUBIC project. MWR only and combined AERI and MWR retrievals are available. During this deployment the AERI had high noise so AERI only retrievals are unavailable and the combined AERI and MWR retrievals should be used with extreme caution.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

TRACER CLAMPS2 AERI_MWR TROPoe Retrievals

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Propagating information content: an example with advection

The mathematical algorithm to derive geophysical information from remote sensing observations is called a retrieval. The mathematics of many retrieval problems are ill-posed, and thus a priori information is used to help constrain the derived geophysical variable to realistic values. One quantity of interest, therefore, is the information content of the observation. Perfect information content in the observation would be achieved if the retrieval were able to capture any perturbation in the desired geophysical variable with the proper magnitude. Many new data products can be derived by combining geophysical variables retrieved from multiple different remote sensors. This paper explores, for the first time, how to derive the information content of these derived products. The approach uses traditional error propagation techniques to derive the uncertainty of the derived field twice, both when the observations are used in the retrieval and also when only the a priori information from each remote sensor is propagated. These two uncertainties are then used to provide an estimate of the information content of the derived geophysical variable. This study demonstrates how to propagate the uncertainties from six different instruments to provide the information content for water vapor and temperature advection. A multi-month analysis demonstrates that, in a mean sense, the information content for temperature advection is nearly unity for all heights below 700 m while, the information content for water vapor advection is somewhat more variable but still larger than 0.6 in the convective boundary layer.

Turner, David D. [National Oceanic and Atmospheric↗

DOE OSTI2024

Parallax-corrected VISST-derived pixel-level products from satellite GOES-16

The NASA Langley group led by William Smith produced GOES-16 satellite cloud retrievals over an approximate 10 by 10 degree region over the CACTI field campaign location. These retrievals are described here: https://www.arm.gov/capabilities/vaps/visst and are available for download here . They use algorithms historically called VISST that are now referred to as SatCORPS. More information can be found in Trepte et al. (2019), Minnis et al. (2021), and Yost et al. (2021). If using this dataset, please cite these references, the CACTI VISST dataset DOI found at the download link above, and this dataset’s DOI. The CACTI VISST pixel-level retrievals are on a 2 km spatial grid and available every 15 minutes (every 10 minutes late in the campaign), producing 21,765 files for the entire field campaign between October 2018 and April 2019. They are not corrected for parallax error, which is an offset in the actual geographical location of a cloud above the surface due to the satellite viewing the cloud partly from the side off nadir. This dataset applies a correction for parallax using the location relative to the satellite and the retrieved cloud top height above the surface, which allows the dataset to be geo-located with surface-based observations. The parallax correction for each location depends on the longitude, latitude and cloud top height above ground level (AGL) for that longitude and latitude in the original VISST files. The cloud top height AGL requires first computing the surface elevation at each VISST grid point. Data from the Advanced Spaceborne Thermal Emission and Reflection (ASTER) Global Digital Elevation Map Version 3 at 30-m resolution is projected onto the VISST grid using conservative coarsening (conserving surface elevation) in the xESMF Python package. The surface elevation is then subtracted from the VISST-retrieved cloud top height above mean sea level. These cloud top heights AGL are then combined with longitude and latitude to estimate the latitude and longitude corrections. Due to variability in cloud top height, the parallax shifts produce an irregular grid of values since higher cloud tops are shifted further than lower cloud tops. A ball tree-based neighbor search with Haversine distance is performed using the Python-based scikit-learn library to find the nearest VISST grid point to each parallax correction-shifted point. The data value of the shifted point is then assigned to that VISST grid point. In this manner, the irregular geographical shifts to correct for parallax are projected back to the rectilinear VISST grid. Because relatively higher clouds should obscure lower clouds, the variable values for the highest cloud top are preferentially chosen if two or more values are assigned to a grid point. The parallax correction should be viewed as an improved but still imperfect estimation of the cloud top locations, largely because the cloud top height is an imperfect retrieval. Please see the attached README document for further information. Users are encouraged to contact the authors with any additional questions.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Integration of Total-Sky Imager Data with a Physics-Based Smart Persistence Model for Intra-Hour Forecasting of Solar Radiation

Short-term solar forecasting models based solely on global horizontal irradiance (GHI) measurements are often unable to discriminate the forecasting of the factors affecting GHI from those that can be precisely computed by atmospheric models. Our previous study introduced a Physics-based Smart Persistence model for Intra-hour forecasting of solar radiation (PSPI) that decomposed the forecasting of GHI into the computation of extraterrestrial solar radiation and solar zenith angle and the forecasting of cloud albedo and cloud fraction. The extraterrestrial solar radiation and solar zenith angle were accurately computed by the Solar Position Algorithm (SPA) developed at the National Renewable Energy Laboratory (NREL). A cloud retrieval technique was used to estimate cloud albedo and cloud fraction from surface-based observations of GHI. With the assumption of persistent cloud structures, the cloud albedo and cloud fraction were predicted for future time steps using a two-stream approximation and a 5-minute exponential weighted moving average, respectively. The model evaluation indicated the estimation and forecast of cloud fraction mostly contributed to the uncertainty of the PSPI though it overcame the persistence and smart persistence models in all forecast time horizons between 5 and 60 minutes. This study aims to enhance the PSPI by ingesting surface-based observations of cloud fraction from a total sky imager (TSI). The estimation and forecast of cloud albedo is correspondingly improved by utilizing the cloud fraction observations and thus leads to more accurate GHI forecast. Various time-series analysis methods are also investigated on the forecasting of cloud fraction and cloud albedo for further improving the GHI forecast. These improvements are valuable for many applications, such as forecasting energy use for buildings, grid operations, and ultimately bringing down the cost of solar energy.

14 SOLAR ENERGY↗

DOE OSTI2021

Molecular structure retrieval directly from laboratory-frame photoelectron spectra in laser-induced electron diffraction

Ubiquitous to most molecular scattering methods is the challenge to retrieve bond distance and angle from the scattering signals since this requires convergence of pattern matching algorithms or fitting methods. This problem is typically exacerbated when imaging larger molecules or for dynamic systems with little a priori knowledge. Here, we employ laser-induced electron diffraction (LIED) which is a powerful means to determine the precise atomic configuration of an isolated gas-phase molecule with picometre spatial and attosecond temporal precision. We introduce a simple molecular retrieval method, which is based only on the identification of critical points in the oscillating molecular interference scattering signal that is extracted directly from the laboratory-frame photoelectron spectrum. The method is compared with a Fourier-based retrieval method, and we show that both methods correctly retrieve the asymmetrically stretched and bent field-dressed configuration of the asymmetric top molecule carbonyl sulfide (OCS), which is confirmed by our quantum-classical calculations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2020

One-dimensional phase retrieval: Regularization, box relaxation and uniqueness

Recovering a signal from its Fourier magnitude is referred to as phase retrieval, which occurs in different fields of engineering and applied physics. This paper gives a new characterization of the phase retrieval problem. Particularly useful is the analysis revealing that the common gradient-based regularization does not restrict the set of solutions to a smaller set. Specifically focusing on binary signals, we show that a box relaxation is equivalent to the binary constraint for Fourier-types of phase retrieval. Here, we further prove that binary signals can be recovered uniquely up to trivial ambiguities under certain conditions. Finally, we use the characterization theorem to develop an efficient denoising algorithm.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

NREL ASSIST Rhode Island / Thermodynamic retrievals TROPoe

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 11) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height from a collocated NREL Upgraded Galion lidar #1212-60 and temperature, RH, and pressure from collocated PNNL met station. The lidar beta is used to estimate the cloud base height through a modified algorithm by Newsom et al., 2019 used at ARM SGP. The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) which provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see e.g. Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at Upton, NY.

17 WIND ENERGY↗

DOE OSTI2020

Matching Crystal Structures Atom-to-Atom

Finding an optimal match between two different crystal structures underpins many important materials science problems, including describing solid-solid phase transitions and developing models for interface and grain boundary structures. Here, we formulate the matching of crystals as an optimization problem where the goal is to find the alignment and the atom-to-atom map that minimize a given cost function such as the Euclidean distance between the atoms. We construct an algorithm that directly solves this problem for large finite portions of the crystals and retrieves the periodicity of the match subsequently. We demonstrate its capacity to describe transformation pathways between known polymorphs and to reproduce experimentally realized structures of semi-coherent interfaces. Additionally, from our findings, we define a rigorous metric for measuring distances between crystal structures that can be used to properly quantify their geometric (Euclidean) closeness.

36 MATERIALS SCIENCE↗

DOE OSTI2026

ARM Radiosondes for SNPP/JPSS Validation Field Campaign Report

This field campaign extension has been a coordinated effort involving the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) User Facility, the University of Wisconsin at Madison, and the Joint Polar Satellite System (JPSS) project to validate NOAA Unique Combined Atmospheric Processing System (NUCAPS) temperature and moisture sounding products from the Cross-track Infrared Sounder (CrIS) and the Advanced Technology Microwave Sounder (ATMS) instruments on board the NOAA-20 and NOAA-21 satellite platforms. In this arrangement, funding for radiosondes and balloons was provided by the JPSS project to ARM. These radiosondes were launched coincident with NOAA-20 and NOAA-21 satellite overpasses at the ARM field sites at Eastern North Atlantic (ENA), North Slope Alaska (NSA), and Southern Great Plains (SGP). Combined with other ARM data, an assessment of the radiosonde data quality was performed and post-processing corrections applied. The dedicated radiosondes were integrated into the NOAA Products Validation System (NPROVS+), which collocated the radiosondes with satellite products (NOAA, National Aeronautics and Space Administration [NASA], European Organisation for the Exploitation of Meteorological Satellites [EUMETSAT], Geostationary Operational Environmental Satellite [GOES], Constellation Observing System for Meteorology, Ionosphere, and Climate [COSMIC]) and numerical weather prediction (NWP) forecasts for use in product assessment and algorithm development. This work is a part of the NOAA-20 and NOAA-21 satellite retrieval validation efforts and provides critical accuracy assessments of the temperature and water vapor soundings.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

AWAKEN 2023 CLAMPS2 MWR TROPoe Retrievals

These files contain 24 hour periods of retrieved thermodynamic profiles derived from observations collected by the CLAMPS2 Microwave Radiometer (MWR). The TROPoe algorithm is a python equivalent to the AERIoe algorithm (see Turner and Loehnert, 2014; Turner and Blumberg 2019). These data were collected during the AWAKEN project. MWR only data is available.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Quantum-parallel vectorized data encodings and computations on trapped-ion and transmon QPUs

Compact data representations in quantum systems are crucial for the development of quantum algorithms for data analysis. In this study, we present two innovative data encoding techniques, known as QCrank and QBArt, which exhibit significant quantum parallelism via uniformly controlled rotation gates. The QCrank method encodes a series of real-valued data as rotations on data qubits, resulting in increased storage capacity. On the other hand, QBArt directly incorporates a binary representation of the data within the computational basis, requiring fewer quantum measurements and enabling well-established arithmetic operations on binary data. We showcase various applications of the proposed encoding methods for various data types. Notably, we demonstrate quantum algorithms for tasks such as DNA pattern matching, Hamming weight computation, complex value conjugation, and the retrieval of a binary image with 384 pixels, all executed on the Quantinuum trapped-ion QPU. Furthermore, we employ several cloud-accessible QPUs, including those from IBMQ and IonQ, to conduct supplementary benchmarking experiments.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Performance of Cloud 3D Solvers in Ice Cloud Shortwave Radiation Closure Over the Equatorial Western Pacific Ocean

Abstract For retrieving cloud optical properties from satellite images or computing these properties from climate model output, computationally efficient treatments of cloud horizontal inhomogeneity include the Monte Carlo Independent Column Approximation (McICA) and the Tripleclouds method. Computationally efficient treatment of cloud horizontal radiation exchanges includes the SPeedy Algorithm for Radiative TrAnsfer through CloUd Sides (SPARTACUS). As a test to derive properties from satellite images, we collocate Moderate Resolution Imaging Spectroradiometer (MODIS) cloud retrievals with near‐nadir Cloud and the Earth's Radiant Energy System (CERES) footprints in July 2008 over an equatorial western Pacific Ocean region to compare the performance of the McICA, Tripleclouds, and SPARTACUS solvers to the conventional plane‐parallel homogeneous (PPH) treatment. PPH overestimates cloud albedo, and the three solvers effectively reduce overestimation with root mean square error of shortwave upwelling irradiance decreasing between 15.72 and 18.53 W m −2 , or about 22%–25%. Although cloud top variability does not get fed into the simulations, all three solvers also reduce the effect of cloud top variability on cloud albedo. Entrapment (energy reflected downward from clouds) and horizontal radiation transfer have opposite effects on the SPARTACUS cloud albedo simulation. The net effect depends on the cloud vertical extent, the unawareness of which limits the performance of the SPARTACUS solver.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

PhaseT3M: 3D imaging at 1.6 Å resolution via electron cryo-tomography with nonlinear phase retrieval

Electron cryo-tomography (cryo-ET) enables 3D imaging of complex, radiation-sensitive structures with molecular detail. However, image contrast from the interference of scattered electrons is nonlinear with atomic density and multiple scattering further complicates interpretation. These effects degrade resolution, particularly in conventional reconstruction algorithms, which assume linearity. Particle averaging can reduce such issues but is unsuitable for heterogeneous or dynamic samples ubiquitous in biology, chemistry, and materials sciences. Here, we develop a phase retrieval-based cryo-ET method, PhaseT3M. We experimentally demonstrate its application to an approximately 7 nm Co3O4 nanoparticle on an approximately 30 nm carbon substrate, achieving a maximum resolution of 1.6 Å, surpassing conventional limits using standard cryo-TEM equipment. PhaseT3M uses a multislice model for multiple scattering and Bayesian optimization for alignment and computational aberration correction, with a positivity constraint to recover ‘missing wedge’ information. Applied directly to biological particles, it enhances reconstruction quality and reduces artifacts, establishing a standard for routine 3D imaging with phase contrast.

Biophysics↗

DOE OSTI2020

Iterative X-ray spectroscopic ptychography

Spectroscopic ptychography is a powerful technique to determine the chemical composition of a sample with high spatial resolution. In spectro-ptychography, a sample is rastered through a focused X-ray beam with varying photon energy so that a series of phaseless diffraction data are recorded. Each chemical component in the material under investigation has a characteristic absorption and phase contrast as a function of photon energy. Using a dictionary formed by the set of contrast functions of each energy for each chemical component, it is possible to obtain the chemical composition of the material from high-resolution multi-spectral images. This paper presents SPA (spectroscopic ptychography with alternating direction method of multipliers), a novel algorithm to iteratively solve the spectroscopic blind ptychography problem. First, a nonlinear spectro-ptychography model based on Poisson maximum likelihood is designed, and then the proposed method is constructed on the basis of fast iterative splitting operators. SPA can be used to retrieve spectral contrast when considering either a known or an incomplete (partially known) dictionary of reference spectra. By coupling the redundancy across different spectral measurements, the proposed algorithm can achieve higher reconstruction quality when compared with standard state-of-the-art two-step methods. It is demonstrated how SPA can recover accurate chemical maps from Poisson-noised measurements, and its enhanced robustness when reconstructing reduced-redundancy ptychography data using large scanning step sizes is shown.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Lossy compression of statistical data using quantum annealer

Abstract We present a new lossy compression algorithm for statistical floating-point data through a representation learning with binary variables. The algorithm finds a set of basis vectors and their binary coefficients that precisely reconstruct the original data. The optimization for the basis vectors is performed classically, while binary coefficients are retrieved through both simulated and quantum annealing for comparison. A bias correction procedure is also presented to estimate and eliminate the error and bias introduced from the inexact reconstruction of the lossy compression for statistical data analyses. The compression algorithm is demonstrated on two different datasets of lattice quantum chromodynamics simulations. The results obtained using simulated annealing show 3–3.5 times better compression performance than the algorithm based on neural-network autoencoder. Calculations using quantum annealing also show promising results, but performance is limited by the integrated control error of the quantum processing unit, which yields large uncertainties in the biases and coupling parameters. Hardware comparison is further studied between the previous generation D-Wave 2000Q and the current D-Wave Advantage system. Our study shows that the Advantage system is more likely to obtain low-energy solutions for the problems than the 2000Q.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Holotomography Phase-Retrieval

Holotomography is a non-destructive x-ray imaging technique which allows to visualize the interior morphology of object without damaging them. It is similar to x-ray computed tomography (CT), but unlike conventional CT, which is based on the attenuation of x-rays passing through the object, holotomography exploits the phase shift caused by the sample. This phase-contrast imaging allows to increase image contrast by recovering refractive index of materials and is proven to be more sensitive to density variations than conventional absorption-based X-ray imaging when applied to low-Z materials. In holotomography, the phase is retrieved by measuring the variations in intensity of the object at different sample to detector distances. Several algorithms exist to perform this task such as, contrast transfer function (CTF), transport of intensity equation (TIE), and a mixed approach which combine both TIE and CTF.

Forien, Jean-Baptiste↗