Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Retrieval Algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Root Cause Correlation Analysis of Software Failures via Orthogonal Defect Classification and Natural Language Processing

Systems theoretic process analysis (STPA) is becoming an increasingly popular technique to assess how complex digital software systems can fail. Rather than defining failures by their observable failure events, which may be sparse especially for safety rated nuclear digital instrumentation and control systems (DI&C), failures are defined as postulated unsafe actions under specific contextual conditions. This permits a top-down analysis of system hazards and identifies whether imposed constraints and requirements can sufficiently address undesirable hazards. However, STPA is a qualitative approach at identifying inadequacies in the development process and cannot currently be used to quantify unsafe action likelihoods for probabilistic risk assessment. Therefore, in this work, we examine the root causes of software failure and explore whether a consistent correlation can be linked to specific unsafe action classes. We implement Lbl2Vec, an unsupervised document classification and retrieval algorithm, on a database of 4,096 software defect reports acquired from various open-source software systems. By analyzing sentence structure, embedded labels, and word vectors, we show that certain defect types positively correlate to specific unsafe action classes over others. The correlations developed can be used to estimate the failure probability of safety intended DI&C systems which provides a licensing basis for nuclear plant modernization efforts.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Root Cause Correlation Analysis of Software Failures via Orthogonal Defect Classification and Natural Language Processing

Systems theoretic process analysis (STPA) is becoming an increasingly popular technique to assess how complex digital software systems can fail. Rather than defining failures by their observable failure events, which may be sparse especially for safety rated nuclear digital instrumentation and control systems (DI&C), failures are defined as postulated unsafe actions under specific contextual conditions. This permits a top-down analysis of system hazards and identifies whether imposed constraints and requirements can sufficiently address undesirable hazards. However, STPA is a qualitative approach at identifying inadequacies in the development process and cannot currently be used to quantify unsafe action likelihoods for probabilistic risk assessment. Therefore, in this work, we examine the root causes of software failure and explore whether a consistent correlation can be linked to specific unsafe action classes. We implement Lbl2Vec, an unsupervised document classification and retrieval algorithm, on a database of 4,096 software defect reports acquired from various open-source software systems. By analyzing sentence structure, embedded labels, and word vectors, we show that certain defect types positively correlate to specific unsafe action classes over others. The correlations developed can be used to estimate the failure probability of safety intended DI&C systems which provides a licensing basis for nuclear plant modernization efforts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

CERES MODIS Cloud Product Retrievals for Edition 4--Part I: Algorithm Changes

The Edition 2 (Ed2) cloud property retrieval algorithm system was upgraded and applied to the MODerate-resolution Imaging Spectroradiometer (MODIS) data for the Clouds and the Earth's Radiant Energy System (CERES) Edition 4 (Ed4) products. New calibrations for solar channels and the use of the 1.24-μm channel for cloud optical depth (COD) over snow improve the daytime consistency between Terra and Aqua MODIS retrievals. Use of additional spectral channels and revised logic enhanced the cloud-top phase retrieval accuracy. A new ice crystal reflectance model and a CO₂-channel algorithm retrieved higher ice clouds, while a new regional lapse rate technique produced more accurate water cloud heights than in Ed2. Ice cloud base heights are more accurate due to a new cloud thickness parameterization. Overall, CODs increased, especially over the polar (PO) regions. The mean particle sizes increased slightly for water clouds, but more so for ice clouds in the PO areas. New experimental parameters introduced in Ed4 are limited in utility, but will be revised for the next CERES edition. As part of the Ed4 retrieval evaluation, the average properties are compared with those from other algorithms and the differences between individual reference data and matched Ed4 retrievals are explored. Part II of this article provides a comprehensive, objective evaluation of selected parameters. More accurate interpretation of the CERES radiation measurements has resulted from the use of the Ed4 cloud properties.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Efficient ptychographic phase retrieval via a matrix-free Levenberg-Marquardt algorithm

The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second order derivatives is even more laborious. Additionally, second order techniques often require the storage and inversion of large matrices of partial derivatives, with memory requirements that can be prohibitive for data-rich imaging modalities. We use a reverse-mode automatic differentiation (AD) framework to implement an efficient matrix-free version of the Levenberg-Marquardt (LM) algorithm, a longstanding method that finds popular use in nonlinear least-square minimization problems but which has seen little use in phase retrieval. Furthermore, we extend the basic LM algorithm so that it can be applied for more general constrained optimization problems (including phase retrieval problems) beyond just the least-square applications. Since we use AD, we only need to specify the physics-based forward model for a specific imaging application; the first and second-order derivative terms are calculated automatically through matrix-vector products, without explicitly forming the large Jacobian or Gauss-Newton matrices typically required for the LM method. We demonstrate that this algorithm can be used to solve both the unconstrained ptychographic object retrieval problem and the constrained “blind” ptychographic object and probe retrieval problems, under the popular Gaussian noise model as well as the Poisson noise model. We compare this algorithm to state-of-the-art first order ptychographic reconstruction methods to demonstrate empirically that this method outperforms best-in-class first-order methods: it provides excellent convergence guarantees with (in many cases) a superlinear rate of convergence, all with a computational cost comparable to, or lower than, the tested first-order algorithms.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Evaluation of cloud height, optical thickness, and phase retrievals from the CHROMA algorithm applied to Sentinel-3 OLCI data

We previously developed the Cloud Height Retrieval from O 2 Molecular Absorption (CHROMA) algorithm for the Ocean Color Instrument (OCI) on the new NASA Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission. Here, we apply CHROMA to observations from the Ocean Land Colour Instrument (OLCI) to guide expectations for PACE, as it will take some time to obtain large-scale validation data for OCI. We use cloud top height (CTH), phase, and (for liquid clouds) cloud optical thickness (COT) data from the ground-based Atmospheric Radiation Measurement (ARM) network to evaluate the OLCI retrievals. We found that OLCI and Moderate Resolution Imaging Spectroradiometer (MODIS) CTH compare similarly well to the ARM reference. OLCI has a tendency to underestimate CTH as CTH increases, and algorithm assumptions about cloud geometric thickness may contribute to this. ARM COT from multifilter shadowband radiometers (MFRSR) and Sun photometers are well-correlated with one another, albeit with a roughly 30 % offset on average; OLCI and MODIS COT agree more closely with the MFRSR data. OLCI retrieval uncertainty estimates show skill at telling low-uncertainty cases from high-uncertainty ones, although CTH uncertainties are underestimated. Additionally, we compare the OLCI data to satellite retrievals based on thermal infrared measurements from MODIS and Sea and Land Surface Temperature Radiometer (SLSTR) data. Differences are broadly consistent with physical expectations based on the A-band vs. thermal techniques, although one key challenge in such aggregated comparisons is different cloud masking sensitivities and algorithm failure rates meaning additional sampling differences are introduced. We conclude by discussing the transition to and possible enhancements for PACE OCI.

Sayer, Andrew M. [Univ. of Maryland Baltimore Coun↗
DOE OSTI2023

Three-dimensional imaging using coherent x rays at grazing incidence geometry

We have developed a three-dimensional coherent diffraction imaging algorithm to retrieve phases of diffraction patterns of samples in grazing incidence small angle x-ray scattering experiments. The algorithm interprets the diffraction patterns using the distorted-wave Born approximation instead of the Born approximation, as in this case, the existence of a reflected beam from the substrate causes the diffraction pattern to deviate significantly from the simple Fourier transform of the object. Detailed computer simulations show that the algorithm works. Verification with real experiments is planned.

Yang, Yi↗
DOE OSTI2023

Planetary Boundary Layer Height (PBLH) over SGP from 1998 to 2023

PBLH is a critical parameter influencing weather phenomena, air quality, and various meteorological processes. However, accurately retrieving PBLH has been a challenging task due to limitations such as coarse temporal resolution and measurement drift in traditional radiosonde observations. To address these limitations, we have devised a lidar-based methodology that capitalizes on a newly developed algorithm for PBLH retrieval. This algorithm demonstrates enhanced capabilities in capturing diurnal fluctuations in PBLH compared to existing lidar-based methods (Su et al. 2020). In addition, we have refined this algorithm specifically for PBLH retrieval under cloudy conditions through a novel scheme (Su et al. 2022). To ensure data reliability, a quality-control process has been implemented to filter out questionable data points. Accompanying the data set is a quality-control flag for ease of reference. It should be noted that we have assimilated all available radiosonde observations to provide a more robust estimate of PBLH, making the data set valuable for a variety of related studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

RHOD Site - NOAA PSL Wind Retrievals WINDoe / Derived Data

This dataset contains daily NetCDF files with horizontal wind profiles retrieved with the WINDoe retrieval (Gebauer and Bell 2024) at Rhode Island (RHOD). WINDoe retrievals datasets are also available at Nantucket Island (NANT, nant.windoe.z01.c1) and Block Island (BLOC, bloc.windoe.z01.c1). WINDoe is an optimal estimation algorithm to retrieve wind profiles combining multiple instruments. The code is available in this github repository (https://github.com/OAR-atmospheric-observations/WINDoe/tree/main) and the retrieval is described by Gebauer and Bell (2024). WINDoe allows combining the individual datasets and outputs into one profile taking into account the information and uncertainties of each dataset. The use of WINDoe minimizes data gaps and maximizes data availability, compared to using wind profiles from only one of the instruments. The regular height grid eases comparisons to numerical weather prediction models. Code modifications have been made that include reading in WFIP3 specific instruments, averaging Doppler lidar radial velocities at various azimuth angles to avoid overfitting, and allowing the user to define a height grid by the user in the vipfile. The instruments used as input to the retrieval are a radar wind profiler (low- and high resolution mode) providing data in and above the boundary layer, a scanning Doppler lidar usually providing data throughout the boundary layer, a profiling lidar providing data from 50 to 200 m at BLOC and NANT, and from 10 to 280 m at Rhode Island, and a surface tower (4 m at NANT and RHOD and 10 m at BLOC). From the scanning lidars, we used radial velocity measurements at 60 deg elevation angle at six different azimuth angles with a resolution of approximately 30 m along the line of sight and the lowest range gate at approximately 70 m. The wind profiles are retrieved with WINDoe up to 3.74 km with 10 m vertical resolution. The profiles are retrieved every 15 min at BLOC and NANT and every 60 min at RHOD.

17 WIND ENERGY↗
DOE OSTI2026

BLOC Site - NOAA PSL Wind Retrievals WINDoe / Derived Data

This dataset contains daily netcdf files with horizontal wind profiles retrieved with the WINDoe retrieval (Gebauer and Bell 2024) at Block Island (BLOC). WINDoe retrievals datasets are also available at Nantucket Island (NANT, nant.windoe.z01.c1) and Rhode Island (RHOD, rhod.windoe.z01.c1). WINDoe is an optimal estimation algorithm to retrieve wind profiles combining multiple instruments. The code is available in this github repository (https://github.com/OAR-atmospheric-observations/WINDoe/tree/main), and the retrieval is described by Gebauer and Bell (2024). WINDoe allows combining the individual datasets and outputs into one profile taking into account the information and uncertainties of each dataset. The use of WINDoe minimizes data gaps and maximizes data availability, compared to using wind profiles from only one of the instruments. The regular height grid eases comparisons to numerical weather prediction models. Code modifications have been made that include reading in WFIP3 specific instruments, averaging Doppler lidar radial velocities at various azimuth angles to avoid overfitting, and allowing the user to define a height grid by the user in the vipfile. The instruments used as input to the retrieval are a radar wind profiler (low- and high resolution mode) providing data in and above the boundary layer, a scanning Doppler lidar usually providing data throughout the boundary layer, a profiling lidar providing data from 50 to 200 m at BLOC and NANT, and from 10 to 280 m at Rhode Island, and a surface tower (4 m at NANT and RHOD and 10 m at BLOC). From the scanning lidars, we used radial velocity measurements at 60 deg elevation angle at six different azimuth angles with a resolution of approximately 30 m along the line of sight and the lowest range gate at approximately 70 m. The wind profiles are retrieved with WINDoe up to 3.74 km with 10 m vertical resolution. The profiles are retrieved every 15 min at BLOC and NANT and every 60 min at RHOD.

17 WIND ENERGY↗
DOE OSTI2026

NANT Site - NOAA PSL Wind Retrievals WINDoe / Derived Data

This dataset contains daily NetCDF files with horizontal wind profiles retrieved with the WINDoe retrieval (Gebauer and Bell 2024) at Nantucket Island (NANT). WINDoe retrievals datasets are also available at Block Island (BLOC, bloc.windoe.z01.c1) and Rhode Island (RHOD, rhod.windoe.z01.c1). WINDoe is an optimal estimation algorithm to retrieve wind profiles combining multiple instruments. The code is available in this github repository (https://github.com/OAR-atmospheric-observations/WINDoe/tree/main), and the retrieval is described by Gebauer and Bell (2024). WINDoe allows combining the individual datasets and outputs into one profile taking into account the information and uncertainties of each dataset. The use of WINDoe minimizes data gaps and maximizes data availability, compared to using wind profiles from only one of the instruments. The regular height grid eases comparisons to numerical weather prediction models. Code modifications have been made that include reading in WFIP3 specific instruments, averaging Doppler lidar radial velocities at various azimuth angles to avoid overfitting, and allowing the user to define a height grid by the user in the vipfile. The instruments used as input to the retrieval are a radar wind profiler (low- and high resolution mode) providing data in and above the boundary layer, a scanning Doppler lidar usually providing data throughout the boundary layer, a profiling lidar providing data from 50 to 200 m at BLOC and NANT, and from 10 to 280 m at Rhode Island, and a surface tower (4 m at NANT and RHOD and 10 m at BLOC). From the scanning lidars, we used radial velocity measurements at 60 deg elevation angle at six different azimuth angles with a resolution of approximately 30 m along the line of sight and the lowest range gate at approximately 70 m. The wind profiles are retrieved with WINDoe up to 3.74 km with 10 m vertical resolution. The profiles are retrieved every 15 min at BLOC and NANT and every 60 min at RHOD.

17 WIND ENERGY↗
DOE OSTI2022

Remote Sensing of Tundra Ecosystems Using High Spectral Resolution Reflectance: Opportunities and Challenges

Abstract Observing the environment in the vast regions of Earth through remote sensing platforms provides the tools to measure ecological dynamics. The Arctic tundra biome, one of the largest inaccessible terrestrial biomes on Earth, requires remote sensing across multiple spatial and temporal scales, from towers to satellites, particularly those equipped for imaging spectroscopy (IS). We describe a rationale for using IS derived from advances in our understanding of Arctic tundra vegetation communities and their interaction with the environment. To best leverage ongoing and forthcoming IS resources, including National Aeronautics and Space Administration’s Surface Biology and Geology mission, we identify a series of opportunities and challenges based on intrinsic spectral dimensionality analysis and a review of current data and literature that illustrates the unique attributes of the Arctic tundra biome. These opportunities and challenges include thematic vegetation mapping, complicated by low‐stature plants and very fine‐scale surface composition heterogeneity; development of scalable algorithms for retrieval of canopy and leaf traits; nuanced variation in vegetation growth and composition that complicates detection of long‐term trends; and rapid phenological changes across brief growing seasons that may go undetected due to low revisit frequency or be obscured by snow cover and clouds. We recommend improvements to future field campaigns and satellite missions, advocating for research that combines multi‐scale spectroscopy, from lab studies to satellites that enable frequent and continuous long‐term monitoring, to inform statistical and biophysical approaches to model vegetation dynamics.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Recovered supernova Ia rate from simulated LSST images

Aims.TheVera C. RubinObservatory’s Legacy Survey of Space and Time (LSST) will revolutionize time-domain astronomy by detecting millions of different transients. In particular, it is expected to increase the number of known type Ia supernovae (SN Ia) by a factor of 100 compared to existing samples up to redshift ∼1.2. Such a high number of events will dramatically reduce statistical uncertainties in the analysis of the properties and rates of these objects. However, the impact of all other sources of uncertainty on the measurement of the SN Ia rate must still be evaluated. The comprehension and reduction of such uncertainties will be fundamental both for cosmology and stellar evolution studies, as measuring the SN Ia rate can put constraints on the evolutionary scenarios of different SN Ia progenitors. Methods.We used simulated data from the Dark Energy Science Collaboration (DESC) Data Challenge 2 (DC2) and LSST Data Preview 0 to measure the SN Ia rate on a 15 deg 2 region of the “wide-fast-deep” area. We selected a sample of SN candidates detected in difference images, associated them to the host galaxy with a specially developed algorithm, and retrieved their photometric redshifts. We then tested different light-curve classification methods, with and without redshift priors (albeit ignoring contamination from other transients, as DC2 contains only SN Ia). We discuss how the distribution in redshift measured for the SN candidates changes according to the selected host galaxy and redshift estimate. Results.We measured the SN Ia rate, analyzing the impact of uncertainties due to photometric redshift, host-galaxy association and classification on the distribution in redshift of the starting sample. We find that we are missing 17% of the SN Ia, on average, with respect to the simulated sample. As 10% of the mismatch is due to the uncertainty on the photometric redshift alone (which also affects classification when used as a prior), we conclude that this parameter is the major source of uncertainty. We discuss possible reduction of the errors in the measurement of the SN Ia rate, including synergies with other surveys, which may help us to use the rate to discriminate different progenitor models.

Astronomy & Astrophysics↗
DOE OSTI2021

The Patterson function as auto-hologram and graph enables the direct solution to the phase problem for coherently illuminated atomistic structures

Abstract The coherent Patterson function, derived from the coherent scattering intensities from N atoms, can be described as an auto-holographic image, i.e., the superposition of N -holographic images in which each atom serves as the source of a different reference wave, and as a mathematical graph of the unknown structure. These insights indicate that the unknown structure is significantly over-determined by the inherent information content in a Patterson function by a factor of ( N − 1)/2 (for non-degenerate structures), independent of dimensionality. However, we also show that the ability to resolve the distinct features in the Patterson function depends strongly on details of the experimental design (data range, sampling frequency, and dimensionality). This re-interpretation suggests that the coherent Patterson function provides a natural context for describing the information content in coherent scattering, reveals that there is no inherent phase problem for coherently illuminated pointwise structures, and enables the design of an algorithm which retrieves the structure directly from the Patterson function, without using error metric minimization.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

TRACER CLAMPS1 Doppler Lidar VAD Data

These files contain 24 hour periods of data collected from the CLAMPS1 Halo Streamline XR Doppler lidar. The Doppler lidar conducts regular conical scans at a set elevation angle. These data are then passed through a typical VAD algorithm to retrieve horizontal wind speed and direction profiles. These data were collected during the TRACER project. For the TRACER campaign, the CLAMPS1 Doppler lidar collected PPI scans at 70 deg elevation every 20 minutes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

TRACER CLAMPS2 Doppler Lidar VAD Data

These files contain 24 hour periods of data collected from the CLAMPS2 Halo Streamline XR+ Doppler lidar. The Doppler lidar conducts regular conical scans at a set elevation angle. These data are then passed through a typical VAD algorithm to retrieve horizontal wind speed and direction profiles. These data were collected during the TRACER project.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

TRACER SPARC Doppler Lidar VAD Data

These files contain 24 hour periods of data collected from the SPARC Halo Streamline XR Doppler lidar. The Doppler lidar conducts regular conical scans at a set elevation angle. These data are then passed through a typical VAD algorithm to retrieve horizontal wind speed and direction profiles. These data were collected during the TRACER project.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

AWAKEN CLAMPS1 Doppler Lidar VAD Data

These files contain 24 hour periods of data collected from the CLAMPS1 Halo Streamline XR+ Doppler lidar. The Doppler lidar conducts regular conical scans at a set elevation angle. These data are then passed through a typical VAD algorithm to retrieve horizontal wind speed and direction profiles. These data were collected during the AWAKEN project.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

AWAKEN CLAMPS2 Doppler Lidar VAD Data

These files contain 24 hour periods of data collected from the CLAMPS2 Halo Streamline XR+ Doppler lidar. The Doppler lidar conducts regular conical scans at a set elevation angle. These data are then passed through a typical VAD algorithm to retrieve horizontal wind speed and direction profiles. These data were collected during the AWAKEN project.

54 ENVIRONMENTAL SCIENCES↗