Surface Landmark Matching and Selection in Natural Terrain
In this paper we present an algorithm for robust absolute position estimation in natural terrain based on landmarks extracted from dense 3-D surfaces.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
In this paper we present an algorithm for robust absolute position estimation in natural terrain based on landmarks extracted from dense 3-D surfaces.
Abstract not provided
When characterizing materials using transmission electron microscopy (TEM) images, detecting and quantifying small features in microstructures, such as cavities, pose significant challenges. Off-the-shelf object detection models, including YOLOv8, show considerable performance degradation, particularly when images vary in resolution and the objects of interest possess a low percentage of the total image region of interest. In this study, we introduce a novel detection pipeline that incorporates an implicit neural representation (INR)-based detection method, INR-TEM, and two-modality imaging (e.g., under-focused and over-focused images typically acquired during materials characterization) to improve object detection performance. The INR-TEM method incorporates a pixel-wise prediction principle inspired by pixel-wise centerness weighting. INR-TEM demonstrates superior robustness to resolution variability, maintaining high detection accuracy even at low image resolutions compared to YOLOv8. To leverage INR-TEM effectively in real-world two-modality characterization applications, we further integrate a two-stage motion correction pipeline designed explicitly for aligning multifocus TEM images. The alignment process, comprising keypoint (based on scale-invariant feature transform, SIFT) and intensity matching, significantly mitigates the adverse effects of perceived motion-induced image degradation during through-focal TEM imaging, directly enhancing INR-TEM’s detection capability over conventional single-focus images. Our integrated INR-TEM cavity detection framework notably improves performance across various cavity sizes, outperforming off-the-shelf YOLOv8 detections that rely on a single image modality.
A key component of Mars exploration is the operation of robotic instruments on the surface, such as those on board the Mars Exploration Rovers, the Mars Science Laboratory (MSL), and the planned Mars 2020 Rover. As the instruments carried by these rovers have become more advanced, the area targeted by some instruments becomes smaller, revealing more fine-grained details about the geology and chemistry of rocks on the surface. However, thermal fluctuations, rover settling or slipping, and inherent inaccuracies in pointing mechanisms all lead to pointing error that is on the order of the target size (several millimeters) or larger. We show that given a target located on a previously acquired image, the rover can align this with a new image to visually locate the target and refine the current pointing. Due to round-trip communication constraints, this visual targeting must be done efficiently on board the rover using relatively limited computing hardware. We employ existing ORB features for landmark-based image registration, describe and theoretically justify a novel approach to filtering false landmark matches, and employ a random forest classifier to automatically reject failed alignments. We demonstrate the efficacy of our approach using over 3,800 images acquired by Remote Micro-Imager on board the “Curiosity” rover.
The problem of image registration, or the alignment of two or more images representing the same scene or object, has to be addressed in various disciplines that employ digital imaging. In the area of remote sensing, just like in medical imaging or computer vision, it is necessary to design robust, fast, and widely applicable algorithms that would allow automatic registration of images generated by various imaging platforms at the same or different times and that would provide subpixel accuracy. One of the main issues that needs to be addressed when developing a registration algorithm is what type of information should be extracted from the images being registered, to be used in the search for the geometric transformation that best aligns them. The main objective of this paper is to evaluate several wavelet pyramids that may be used both for invariant feature extraction and for representing images at multiple spatial resolutions to accelerate registration. We find that the bandpass wavelets obtained from the steerable pyramid due to Simoncelli performs best in terms of accuracy and consistency, while the low-pass wavelets obtained from the same pyramid give the best results in terms of the radius of convergence. Based on these findings, we propose a modification of a gradient-based registration algorithm that has recently been developed for medical data. We test the modified algorithm on several sets of real and synthetic satellite imagery.
Determining the presence of widely separated substellar-mass companion is crucial to understand the dynamics of inner planets in extrasolar planetary systems (e.g., to explain their high mean eccentricity as inner planets are perturbed by the Kozai mechanism). We report the results of our Spitzer/Infrared Array Camera (IRAC) imaging search for widely separated (10''-25'') substellar-mass companions for 14 planet-host stars within 15 pc of the Sun. Using deep 3.6 and 4.5 μm observations in subarray mode, we found one object in the field of 47 UMa with [3.6]–[4.5] color similar to a T5 dwarf, which is, however, unlikely to share common proper motion with 47 UMa. We also found three objects with brown-dwarf-like [3.6]–[4.5] color limits in the fields of GJ 86, HD 160691, and GJ 581, as well as another in the field of HD 69830 for which we have excluded common proper motion. We provide model-based upper mass limits for unseen objects around all stars in our sample, with typical sensitivity to 10 M {sub J} objects from a projected separation of 50-300 AU from the parent star. We also discuss our data analysis methods for point-spread-function subtraction, image co-alignment, and artifact subtraction of IRAC subarray images.
We describe the operation and performance of the difference imaging pipeline (DiffImg) used to detect transients in deep images from the Dark Energy Survey Supernova program (DES-SN) in its first observing season from 2013 August through 2014 February. DES-SN is a search for transients in which ten 3 deg{sup 2} fields are repeatedly observed in the g, r, i, z passbands with a cadence of about 1 week. The observing strategy has been optimized to measure high-quality light curves and redshifts for thousands of Type Ia supernovae (SNe Ia) with the goal of measuring dark energy parameters. The essential DiffImg functions are to align each search image to a deep reference image, do a pixel-by-pixel subtraction, and then examine the subtracted image for significant positive detections of point-source objects. The vast majority of detections are subtraction artifacts, but after selection requirements and image filtering with an automated scanning program, there are ∼130 detections per deg{sup 2} per observation in each band, of which only ∼25% are artifacts. Of the ∼7500 transients discovered by DES-SN in its first observing season, each requiring a detection on at least two separate nights, Monte Carlo (MC) simulations predict that 27% are expected to be SNe Ia or core-collapse SNe. Another ∼30% of the transients are artifacts in which a small number of observations satisfy the selection criteria for a single-epoch detection. Spectroscopic analysis shows that most of the remaining transients are AGNs and variable stars. Fake SNe Ia are overlaid onto the images to rigorously evaluate detection efficiencies and to understand the DiffImg performance. The DiffImg efficiency measured with fake SNe agrees well with expectations from a MC simulation that uses analytical calculations of the fluxes and their uncertainties. In our 8 “shallow” fields with single-epoch 50% completeness depth ∼23.5, the SN Ia efficiency falls to 1/2 at redshift z ≈ 0.7; in our 2 “deep” fields with mag-depth ∼24.5, the efficiency falls to 1/2 at z ≈ 1.1. A remaining performance issue is that the measured fluxes have additional scatter (beyond Poisson fluctuations) that increases with the host galaxy surface brightness at the transient location. This bright-galaxy issue has minimal impact on the SNe Ia program, but it may lower the efficiency for finding fainter transients on bright galaxies.
Purpose: Stereotactic lung radiotherapy is complicated by tumor motion from patient respiration. Four-dimensional CT (4DCT) imaging is a motion compensation method used in treatment planning to generate a maximum intensity projection (MIP) internal target volume (ITV). Image guided radiotherapy during treatment may involve acquiring a volumetric cone-beam CT (CBCT) image and visually aligning the tumor to the planning 4DCT MIP ITV contour. Moving targets imaged with CBCT can appear blurred and currently there are no studies reporting on the effect that irregular breathing patterns have on CBCT volumes and their alignment to 4DCT MIP ITV contours. The objective of this work was therefore to image a phantom moving with irregular breathing patterns to determine whether any configurations resulted in errors in volume contouring or alignment. Methods: A Perspex thorax phantom was used to simulate a patient. Three wooden 'lung' inserts with embedded Perspex 'lesions' were moved up to 4 cm with computer-generated motion patterns, and up to 1 cm with patient-specific breathing patterns. The phantom was imaged on 4DCT and CBCT with the same acquisition settings used for stereotactic lung patients in the clinic and the volumes on all phantom images were contoured. This project assessed the volumes for qualitative and quantitative changes including volume, length of the volume, and errors in alignment between CBCT volumes and 4DCT MIP ITV contours. Results: When motion was introduced 4DCT and CBCT volumes were reduced by up to 20% and 30% and shortened by up to 7 and 11 mm, respectively, indicating that volume was being under-represented at the extremes of motion. Banding artifacts were present in 4DCT MIP images, while CBCT volumes were largely reduced in contrast. When variable amplitudes from patient traces were used and CBCT ITVs were compared to 4DCT MIP ITVs there was a distinct trend in reduced ITV with increasing amplitude that was not seen when compared to true ITVs. Breathing patterns with a rest period following expiration resulted in well-defined superior edges and were better aligned using an edge-to-edge alignment technique. In most cases, sinusoidal motion patterns resulted in the closest agreements to true values and the smallest misalignments. Conclusions: Strategies are needed to compensate for volume losses at the extremes of motion for both 4DCT MIP and CBCT images for larger and varied amplitudes, and for patterns with rest periods following expiration. Lesions moving greater than 2 cm would warrant larger treatment margins added to the 4DCT MIP ITV to account for the volume being under-represented at the extremes of motion. Lesions moving with a rest period following expiration would be better aligned using an edge-to-edge alignment technique. Sinusoidal patterns represented the ideal clinical scenario, reinforcing the importance of investigating clinically relevant motions and their effects on 4DCT MIP and CBCT volumes. Since most patients do not breathe sinusoidally this may lead to misinterpretation of previous studies using only sinusoidal motion.
A method of evaluating the initial assumptions and uncertainties of the physical connection between Airborne Imaging Spectrometer (AIS) image data and laboratory/field spectrometer data was tested. The Tuscon AIS-2 image connects to lab reference spectra by an alignment to the image spectral endmembers through a system gain and offset for each band. Images were calibrated to reflectance so as to transform the image into a measure that is independent of the solar radiant flux. This transformation also makes the image spectra directly comparable to data from lab and field spectrometers. A method was tested for calibrating AIS images using the surface as a reference. The surface heterogeneity is defined by lab/field spectral measurements. It was found that the Tuscon AIS-2 image is consistent with each of the initial hypotheses: (1) that the AIS-2 instrument calibration is nearly linear; (2) the spectral variance is caused by sub-pixel mixtures of spectrally distinct materials and shade, and (3) that sub-pixel mixtures can be treated as linear mixtures of pure endmembers. It was also found that the image can be characterized by relatively few endmembers using the AIS-2 spectra.
The objective of this project was to operate ground based Fabry-Perot Interferometers at several points under the auroral zone to analyze and quantify the vertical wind in the thermosphere. These measurements were made in conjunction with TIMED, especially GUVI data, to relate the observed wind to the resulting mixing and compositional changes in the thermosphere. The ground based wind measurements were obtained from a scanning Doppler imager (SDI) in Poker Flat, and a vertically aligned Fabry Perot Imager (FPI) in Inuvik. A third FPI at Eagle, Alaska, was operated for a brief overlapping period as well. The SDI at Poker Flat had been in operation for several years, and was continued to run with little support from this grant. The much more expensive operation, maintenance, and data acquisition of the remote Inuvik FPI was made possible with funds from this project. During the 2003/2004 and 2004/2005 seasons, we operated the Inuvik FPI from September to April during hours of darkness. Two trips to service the instrument were required per year, and a local caretaker was funded to help keep the instrument going during the winter seasons. The data were transfered via modem and phone line to Poker Flat and were then analyzed to obtain wind and temperature at the altitude of the auroral green line OI(557.7 nm). The final data product was archived and transferred to the GEDDS system at Poker Flat were it is available on the web: http://gedds.pfrr.alaska.edu/. The data set is also available from the CEDAR data base: http://cedarweb.hao.ucar.edu/.
Purpose: To investigate if pelvic nodal coverage for prostate patients undergoing intensity modulated radiotherapy (IMRT) can be predicted using mutual image information computed between planning and cone-beam CTs (CBCTs). Methods: Four patients with high-risk prostate adenocarcinoma were treated with IMRT on a Varian TrueBeam. Plans were designed such that 95% of the nodal planning target volume (PTV) received the prescription dose of 45 Gy (N=1) or 50.4 Gy (N=3). Weekly CBCTs (N=25) were acquired and the nodal clinical target volumes and organs at risk were contoured by a physician. The percent nodal volume receiving prescription dose was recorded as a ground truth. Using the recorded shifts performed by the radiation therapists at the time of image acquisition, CBCTs were aligned with the planning kVCT. Mutual image information (MI) was calculated between the CBCT and the aligned planning CT within the contour of the nodal PTV. Due to variable CBCT fields-of-view, CBCT images covering less than 90% of the nodal volume were excluded from the analysis, resulting in the removal of eight CBCTs. Results: A correlation coefficient of 0.40 was observed between the MI metric and the percent of the nodal target volume receiving the prescription dose. One patient's CBCTs had clear outliers from the rest of the patients. Upon further investigation, we discovered image artifacts that were present only in that patient's images. When those four images were excluded, the correlation improved to 0.81. Conclusion: This pilot study shows the potential of predicting pelvic nodal dosimetry by computing the mutual image information between planning CTs and patient setup CBCTs. Importantly, this technique does not involve manual or automatic contouring of the CBCT images. Additional patients and more robust exclusion criteria will help validate our findings.
Purpose: Accurate alignment of thoracic CT images is essential for dose tracking and to safely implement adaptive radiotherapy in lung cancers. At the same time it is challenging given the highly elastic nature of lung tissue deformations. The objective of this study was to assess the performances of three state-of-art intensity-based algorithms in terms of their ability to register thoracic CT images subject to affine, barrel, and sinusoid transformation. Methods: Intensity similarity measures of the evaluated algorithms contained sum-of-squared difference (SSD), local mutual information (LMI), and residual complexity (RC). Five thoracic CT scans obtained from the EMPIRE10 challenge database were included and served as reference images. Each CT dataset was distorted by realistic affine, barrel, and sinusoid transformations. Registration performances of the three algorithms were evaluated for each distortion type in terms of intensity root mean square error (IRMSE) between the reference and registered images in the lung regions. Results: For affine distortions, the three algorithms differed significantly in registration of thoracic images both visually and nominally in terms of IRMSE with a mean of 0.011 for SSD, 0.039 for RC, and 0.026 for LMI (p<0.01; Kruskal-Wallis test). For barrel distortion, the three algorithms showed nominally no significant difference in terms of IRMSE with a mean of 0.026 for SSD, 0.086 for RC, and 0.054 for LMI (p=0.16) . A significant difference was seen for sinusoid distorted thoracic CT data with mean lung IRMSE of 0.039 for SSD, 0.092 for RC, and 0.035 for LMI (p=0.02). Conclusion: Pulmonary deformations might vary to a large extent in nature in a daily clinical setting due to factors ranging from anatomy variations to respiratory motion to image quality. It can be appreciated from the results of the present study that the suitability of application of a particular algorithm for pulmonary image registration is deformation-dependent.
Purpose: To determine whether maximum or average intensity projection (MIP or AIP, respectively) reconstructed from 4-dimensional computed tomography (4DCT) is preferred for alignment to cone beam CT (CBCT) images in lung stereotactic body radiation therapy. Methods and Materials: Stationary CT and 4DCT images were acquired with a target phantom at the center of motion and moving along the superior–inferior (SI) direction, respectively. Motion profiles were asymmetrical waveforms with amplitudes of 10, 15, and 20 mm and a 4-second cycle. Stationary CBCT and dynamic CBCT images were acquired in the same manner as stationary CT and 4DCT images. Stationary CBCT was aligned to stationary CT, and the couch position was used as the baseline. Dynamic CBCT was aligned to the MIP and AIP of corresponding amplitudes. Registration error was defined as the SI deviation of the couch position from the baseline. In 16 patients with isolated lung lesions, free-breathing CBCT (FBCBCT) was registered to AIP and MIP (64 sessions in total), and the difference in couch shifts was calculated. Results: In the phantom study, registration errors were within 0.1 mm for AIP and 1.5 to 1.8 mm toward the inferior direction for MIP. In the patient study, the difference in the couch shifts (mean, range) was insignificant in the right-left (0.0 mm, ≤1.0 mm) and anterior–posterior (0.0 mm, ≤2.1 mm) directions. In the SI direction, however, the couch position significantly shifted in the inferior direction after MIP registration compared with after AIP registration (mean, −0.6 mm; ranging 1.7 mm to the superior side and 3.5 mm to the inferior side, P=.02). Conclusions: AIP is recommended as the reference image for registration to FBCBCT when target alignment is performed in the presence of asymmetrical respiratory motion, whereas MIP causes systematic target positioning error.
The Infrared Multi-Object Spectrograph (IRMOS) is a facility instrument for the Kitt Peak National Observatory Mayall Telescope (3.8 meter). IRMOS is a near-IR (0.8 - 2.5 micron) spectrograph with low to mid resolution (R=lambda/delta, lambda = 300 - 3800). The IRMOS spectrograph produces simultaneous spectra of - 100 objects in its 2.8 x 2.0 arc-min field of view using a commercial MEMS multi-mirror array device (MMA). The IRMOS optical design consists of two imaging systems, or "stages." The focal reducer, stage one, images the focal plane of the telescope onto the MMA. The spectrograph, stage two, images the MMA onto the detector. We describe the breadboard alignment method and imaging and scattered light performance for both the focal reducer and spectrograph. This testing provides verification of the optomechanical alignment method, and a measurement of the contribution of scattered light in the system due to mirror small scale surface error. After the stage I and 2 optics are integrated with the instrument, our test results will make it possible to distinguish between scattered light from the mirrors and the MMA. Image testing will be done at four wavelengths in the visible and near-IR. A mercury-argon pencil lamp will provide spectral lines at 546.1 and 1012 nm, and a blackbody radiation source lines at 1600 and 2200 nm. A CCD camera will be used as a detector for the visible wavelengths, and an IR photodiode will be used for the IR wavelengths. We compare our data with a theoretical analysis using a commercial software package. Mirror surface error is modeled by treating each surface as a superposition of various gratings (e.g., diamond turning tool marks, features due to the impurities of Al 6061, and periodic mid-frequency errors due to drift during machining).
Purpose: The scanning beam digital x-ray system (SBDX) is an inverse geometry fluoroscopic system with high dose efficiency and the ability to perform continuous real-time tomosynthesis in multiple planes. This system could be used for image guidance during lung nodule biopsy. However, the reconstructed images suffer from strong out-of-plane artifact due to the small tomographic angle of the system. Methods: The authors propose an out-of-plane artifact subtraction tomosynthesis (OPAST) algorithm that utilizes a prior CT volume to augment the run-time image processing. A blur-and-add (BAA) analytical model, derived from the project-to-backproject physical model, permits the generation of tomosynthesis images that are a good approximation to the shift-and-add (SAA) reconstructed image. A computationally practical algorithm is proposed to simulate images and out-of-plane artifacts from patient-specific prior CT volumes using the BAA model. A 3D image registration algorithm to align the simulated and reconstructed images is described. The accuracy of the BAA analytical model and the OPAST algorithm was evaluated using three lung cancer patients’ CT data. The OPAST and image registration algorithms were also tested with added nonrigid respiratory motions. Results: Image similarity measurements, including the correlation coefficient, mean squared error, and structural similarity index, indicated that the BAA model is very accurate in simulating the SAA images from the prior CT for the SBDX system. The shift-variant effect of the BAA model can be ignored when the shifts between SBDX images and CT volumes are within ±10 mm in the x and y directions. The nodule visibility and depth resolution are improved by subtracting simulated artifacts from the reconstructions. The image registration and OPAST are robust in the presence of added respiratory motions. The dominant artifacts in the subtraction images are caused by the mismatches between the real object and the prior CT volume. Conclusions: Their proposed prior CT-augmented OPAST reconstruction algorithm improves lung nodule visibility and depth resolution for the SBDX system.
Whether a macroscopic assembly of carbon nanotubes can exhibit the one-dimensional properties expected from individual nanotubes critically depends on how well the nanotubes are aligned inside the assembly. Therefore, a simple and accurate method for assessing the degree of alignment is desired for the rapid characterization of carbon nanotube films and fibers. Here, we present an end-to-end solution for determining the global and local spatial orientation of carbon nanotubes in films within a short amount of time using a fast, precise, and economical approach based on an image processing method applied to scanning electron microscopy images. Further, we first use Laplacian edge enhancement filtering for improving the appearance of edge regions, which is followed by image partitioning into multiple blocks to capture the nanoscale orientation characteristics and total variation-based image decomposition of these image blocks. We then perform a 2D-fast Fourier transform on the image decomposed textural components of these edge-enhanced image blocks to determine the orientation distribution, which is utilized to estimate the 2D nematic order parameter. To show the effectiveness of our method, we corroborated our results against results obtained with other state-of-the-art image processing and experimental techniques.
The quest for a B-mode imprint from primordial gravity waves on the polarization of the cosmic microwave background (CMB) requires the characterization of foreground polarization from Galactic dust. In this paper, we present a statistical study of the filamentary structure of the 353 GHz Planck Stokes maps at high Galactic latitude, relevant to the study of dust emission as a polarized foreground to the CMB. We filter the intensity and polarization maps to isolate filaments in the range of angular scales where the power asymmetry between E-modes and B-modes is observed. Using the Smoothed Hessian Major Axis Filament Finder (SMAFF), we identify 259 filaments at high Galactic latitude, with lengths larger or equal to 2° (corresponding to 3.5 pc in length for a typical distance of 100 pc). Thesefilaments show a preferred orientation parallel to the magnetic field projected onto the plane of the sky, derived from their polarization angles. We present mean maps of the filaments in Stokes I, Q, U, E, and B, computed by stacking individual images rotated to align the orientations of the filaments. Combining the stacked images and the histogram of relative orientations, we estimate the mean polarization fraction of the filaments to be 11%. Furthermore, we show that the correlation between the filaments and the magnetic field orientations may account for the E and B asymmetry and the C ℓ TE /C ℓ EE ratio, reported in the power spectra analysis of the Planck353 GHz polarization maps. Finally, future models of the dust foreground for CMB polarization studies will need to take into account the observed correlation between the dust polarization and the structure of interstellar matter.
Purpose: Average or maximum intensity projection (AIP or MIP) images derived from 4DCT images are often used as a reference image for target alignment when free breathing Cone-beam CT (FBCBCT) is used for positioning a moving target at treatment. This method can be highly accurate if the patient has stable respiratory motion. However, a patient’s breathing pattern often varies irregularly. The purpose of this study is to investigate the effect of irregular respiration on the positioning accuracy of a moving target with FBCBCT. Methods: Eight patients’ respiratory motion curves were selected to drive a Quasar phantom with embedded cubic and spherical targets. A 4DCT of the moving phantom was acquired on a CT scanner (Philips Brilliance 16) equipped with a Varian RPM system. The phase binned 4DCT images and the corresponding MIP and AIP images were transferred into Eclipse for analysis. CBCTs of the phantom driven by the same breathing curves were acquired on a Varian TrueBeam and fused such that the zero positions of moving targets are the same on both CBCT and AIP images. The sphere and cube volumes and centrioid differences (alignment error) determined by MIP, AIP and FBCBCT images were compared. Results: Compared to the volume determined by FBCBCT, the volumes of cube and sphere in MIP images were 22.4%±8.8% and 34.2%±6.2% larger while the volumes in AIP images were 7.1%±6.2% and 2.7%±15.3% larger, respectively. The alignment errors for the cube and sphere with center-center matches between MIP and FBCBCT were 3.5±3.1mm and 3.2±2.3mm, and the alignment errors between AIP and FBCBCT were 2.1±2.6mm and 2.1±1.7mm, respectively. Conclusion: AIP images appear to be superior reference images than MIP images. However, irregular respiratory motions could compromise the positioning accuracy of a moving target if the target center-center match is used to align FBCBCT and AIP images.