Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Optical flow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS2006

Attenuating Stereo Pixel-Locking via Affine Window Adaptation

For real-time stereo vision systems, the standard method for estimating sub-pixel stereo disparity given an initial integer disparity map involves fitting parabolas to a matching cost function aggregated over rectangular windows. This results in a phenomenon known as 'pixel-locking,' which produces artificially-peaked histograms of sub-pixel disparity. These peaks correspond to the introduction of erroneous ripples or waves in the 3D reconstruction of truly Rat surfaces. Since stereo vision is a common input modality for autonomous vehicles, these inaccuracies can pose a problem for safe, reliable navigation. This paper proposes a new method for sub-pixel stereo disparity estimation, based on ideas from Lucas-Kanade tracking and optical flow, which substantially reduces the pixel-locking effect. In addition, it has the ability to correct much larger initial disparity errors than previous approaches and is more general as it applies not only to the ground plane.

pixel locking effect↗
NASA NTRS2011

Algorithm for Autonomous Landing

Because of their small size, high maneuverability, and easy deployment, micro aerial vehicles (MAVs) are used for a wide variety of both civilian and military missions. One of their current drawbacks is the vast array of sensors (such as GPS, altimeter, radar, and the like) required to make a landing. Due to the MAV s small payload size, this is a major concern. Replacing the imaging sensors with a single monocular camera is sufficient to land a MAV. By applying optical flow algorithms to images obtained from the camera, time-to-collision can be measured. This is a measurement of position and velocity (but not of absolute distance), and can avoid obstacles as well as facilitate a landing on a flat surface given a set of initial conditions. The key to this approach is to calculate time-to-collision based on some image on the ground. By holding the angular velocity constant, horizontal speed decreases linearly with the height, resulting in a smooth landing. Mathematical proofs show that even with actuator saturation or modeling/ measurement uncertainties, MAVs can land safely. Landings of this nature may have a higher velocity than is desirable, but this can be compensated for by a cushioning or dampening system, or by using a system of legs to grab onto a surface. Such a monocular camera system can increase vehicle payload size (or correspondingly reduce vehicle size), increase speed of descent, and guarantee a safe landing by directly correlating speed to height from the ground.

Kuwata, Yoshiaki↗
NASA NTRS2012

Localization Using Visual Odometry and a Single Downward-Pointing Camera

Stereo imaging is a technique commonly employed for vision-based navigation. For such applications, two images are acquired from different vantage points and then compared using transformations to extract depth information. The technique is commonly used in robotics for obstacle avoidance or for Simultaneous Localization And Mapping, (SLAM). Yet, the process requires a number of image processing steps and therefore tends to be CPU-intensive, which limits the real-time data rate and use in power-limited applications. Evaluated here is a technique where a monocular camera is used for vision-based odometry. In this work, an optical flow technique with feature recognition is performed to generate odometry measurements. The visual odometry sensor measurements are intended to be used as control inputs or measurements in a sensor fusion algorithm using low-cost MEMS based inertial sensors to provide improved localization information. Presented here are visual odometry results which demonstrate the challenges associated with using ground-pointing cameras for visual odometry. The focus is for rover-based robotic applications for localization within GPS-denied environments.

Swank, Aaron J.↗
NASA NTRS2011

Benefit of "Push-pull" Locomotion for Planetary Rover Mobility

As NASAs exploration missions on planetary terrains become more aggressive, a focus on alternative modes of locomotion for rovers is necessary. In addition to climbing steep slopes, the terrain in these extreme environments is often unknown and can be extremely hard to traverse, increasing the likelihood of a vehicle or robot becoming damaged or immobilized. The conventional driving mode in which all wheels are either driven or free-rolling is very efficient on flat hard ground, but does not always provide enough traction to propel the vehicle through soft or steep terrain. This paper presents an alternative mode of travel and investigates the fundamental differences between these locomotion modes. The methods of push-pull locomotion discussed can be used with articulated wheeled vehicles and are identified as walking or inchinginch-worming. In both cases, the braked non-rolling wheels provide increased thrust. An in-depth study of how soil reacts under a rolling wheel vs. a braked wheel was performed by visually observing the motion of particles beneath the surface. This novel technique consists of driving or dragging a wheel in a soil bin against a transparent wall while high resolution, high-rate photographs are taken. Optical flow software was then used to determine shearing patterns in the soil. Different failure modes were observed for the rolling and braked wheel cases. A quantitative comparison of inching vs. conventional driving was also performed on a full-scale vehicle through a series of drawbar pull tests in the Lunar terrain strength simulant, GRC-1. The effect of tire stiffness was also compared; typically compliant tires provide better traction when driving in soft soil, however its been observed that rigid wheels may provide better thrust when non-rolling. Initial tests indicate up to a possible 40 increase in pull force capability at high slip when inching vs. rolling.

robotics↗
NASA NTRS2014

Improving Sensorimotor Adaptation Following Long Duration Space Flight by Enhancing Vestibular Information Transfer

Crewmember adapted to the microgravity state may need to egress the vehicle within a few minutes for safety and operational reasons after gravitational transitions. The transition from one sensorimotor state to another consists of two main mechanisms: strategic and plastic-adaptive and have been demonstrated in astronauts returning after long duration space flight. Strategic modifications represent "early adaptation" - immediate and transitory changes in control that are employed to deal with short-term changes in the environment. If these modifications are prolonged then plastic-adaptive changes are evoked that modify central nervous system function, automating new behavioral responses. More importantly, this longer term adaptive recovery mechanism was significantly associated with their strategic ability to recover on the first day after return to Earth G. We are developing a method based on stochastic resonance to enhance information transfer by improving the brain's ability to detect vestibular signals (Vestibular Stochastic Resonance, VSR) especially when combined with balance training exercises such as sensorimotor adaptability (SA) training for rapid improvement in functional skill, for standing and mobility. This countermeasure to improve detection of vestibular signals is a stimulus delivery system that is wearable/portable providing low imperceptible levels of white noise based binaural bipolar electrical stimulation of the vestibular system (stochastic vestibular stimulation). To determine efficacy of vestibular stimulation on physiological and perceptual responses during otolith-canal conflicts and dynamic perturbations we have conducted a series of studies: We have shown that imperceptible binaural bipolar electrical stimulation of the vestibular system across the mastoids enhances balance performance in the mediolateral (ML) plane while standing on an unstable surface. We have followed up on the previous study showing VSR stimulation improved balance performance in both ML and anteroposterior planes while stimulating in the ML axis only. We have shown the efficacy of VSR stimulations on enhancing physiological and perceptual responses of whole-body orientation during low frequency perturbations (0.1 Hz) on the ocular motor system using a variable radius centrifuge on both physiological (using eye movements) and perceptual responses (using a joystick) to track imposed oscillations. The variable radius centrifuge provides a selective tilting sensation that is detectable only by the otolith organs providing conflicting information from the canal organs of the vestibular system (intra-vestibular conflict). These results indicate that VSR can improve performance in sensory conflict scenarios like that experienced during space flight. We have showed the efficacy of VSR stimulation to improve balance and locomotor control on subjects exposed to continuous, sinusoidal lateral motion of the support surface while walking on a treadmill while viewing perceptually matched linear optic flow. We have shown the safety of short term continuous use of up to 4 hours of VSR stimulation and its efficacy in improving balance and locomotor function in Parkinson's Disease patients. This technique for improving vestibular signal detection may thus provide additional information to improve strategic abilities. We hypothesize that VSR stimulation will act synergistically with SA training to improve adaptability by increased utilization of vestibular information and therefore serve to optimize and personalize the SA countermeasure prescription. This forms the basis of its usefulness both as a training modality and further help in significantly reducing the number of days required to recover functional performance to preflight levels after long duration space flight.

Mulavara, A. P.↗
NASA NTRS2014

Push-Pull Locomotion for Vehicle Extrication

For applications in which unmanned vehicles must traverse unfamiliar terrain, there often exists the risk of vehicle entrapment. Typically, this risk can be reduced by using feedback from on-board sensors that assess the terrain. This work addressed the situations where a vehicle has already become immobilized or the desired route cannot be traversed using conventional rolling. Specifically, the focus was on using push-pull locomotion in high sinkage granular material. Push-pull locomotion is an alternative mode of travel that generates thrust through articulated motion, using vehicle components as anchors to push or pull against. It has been revealed through previous research that push-pull locomotion has the capacity for generating higher net traction forces than rolling, and a unique optical flow technique indicated that this is the result of a more efficient soil shearing method. It has now been found that pushpull locomotion results in less sinkage, lower travel reduction, and better power efficiency in high sinkage material as compared to rolling. Even when starting from an "entrapped" condition, push-pull locomotion was able to extricate the test vehicle. It is the authors' recommendation that push-pull locomotion be considered as a reliable back-up mode of travel for applications where terrain entrapment is a possibility.

extrication↗
NASA NTRS2009

Robust Landing Using Time-to-Collision Measurement with Actuator Saturation

This paper considers a landing problem for an MAV that uses only a monocular camera for guidance. Although this sensor cannot measure the absolute distance to the target, by using optical flow algorithms, time-to-collision to the target is obtained. Existing work has applied a simple proportional feedback control to simple dynamics and demonstrated its potential. However, due to the singularity in the time-to-collision measurement around the target, this feedback could require an infinite control action. This paper extends the approach into nonlinear dynamics. In particular, we explicitly consider the saturation of the actuator and include the effect of the aerial drag. It is shown that the convergence to the target is guaranteed from a set of initial conditions, and the boundaries of such initial conditions in the state space are numerically obtained. The paper then introduces parametric uncertainties in the vehicle model and in the time-to-collision measurements. Using an argument similar to the nominal case, the robust convergence to the target is proven, but the region of attraction is shown to shrink due to the existence of uncertainties. The numerical simulation validates these theoretical results.

parametric uncertainty↗
NASA NTRS2015

Advanced Bayesian Method for Planetary Surface Navigation

Autonomous Exploration, Inc., has developed an advanced Bayesian statistical inference method that leverages current computing technology to produce a highly accurate surface navigation system. The method combines dense stereo vision and high-speed optical flow to implement visual odometry (VO) to track faster rover movements. The Bayesian VO technique improves performance by using all image information rather than corner features only. The method determines what can be learned from each image pixel and weighs the information accordingly. This capability improves performance in shadowed areas that yield only low-contrast images. The error characteristics of the visual processing are complementary to those of a low-cost inertial measurement unit (IMU), so the combination of the two capabilities provides highly accurate navigation. The method increases NASA mission productivity by enabling faster rover speed and accuracy. On Earth, the technology will permit operation of robots and autonomous vehicles in areas where the Global Positioning System (GPS) is degraded or unavailable.

Center, Julian↗
NASA NTRS2016

Localization from Visual Landmarks on a Free-Flying Robot

We present the localization approach for Astrobee, a new free-flying robot designed to navigate autonomously on the International Space Station (ISS). Astrobee will accommodate a variety of payloads and enable guest scientists to run experiments in zero-g, as well as assist astronauts and ground controllers. Astrobee will replace the SPHERES robots which currently operate on the ISS, whose use of fixed ultrasonic beacons for localization limits them to work in a 2 meter cube. Astrobee localizes with monocular vision and an IMU, without any environmental modifications. Visual features detected on a pre-built map, optical flow information, and IMU readings are all integrated into an extended Kalman filter (EKF) to estimate the robot pose. We introduce several modifications to the filter to make it more robust to noise, and extensively evaluate the localization algorithm.

Coltin, Brian↗
NASA NTRS2016

Localization from Visual Landmarks on a Free-Flying Robot

We present the localization approach for Astrobee,a new free-flying robot designed to navigate autonomously on board the International Space Station (ISS). Astrobee will conduct experiments in microgravity, as well as assisst astronauts and ground controllers. Astrobee replaces the SPHERES robots which currently operate on the ISS, which were limited to operating in a small cube since their localization system relied on triangulation from ultrasonic transmitters. Astrobee localizes with only monocular vision and an IMU, enabling it to traverse the entire US segment of the station. Features detected on a previously-built map, optical flow information,and IMU readings are all integrated into an extended Kalman filter (EKF) to estimate the robot pose. We introduce several modifications to the filter to make it more robust to noise.Finally, we extensively evaluate the behavior of the filter on atwo-dimensional testing surface.

Coltin, Brian↗
NASA NTRS2017

Design Considerations for Attitude State Awareness and Prevention of Entry into Unusual Attitudes

Loss of control - inflight (LOC-I) has historically represented the largest category of commercial aviation fatal accidents. A review of the worldwide transport airplane accidents (2001-2010) evinced that loss of attitude or energy state awareness was responsible for a large majority of the LOC-I events. A Commercial Aviation Safety Team (CAST) study of 18 worldwide loss-of-control accidents and incidents determined that flight crew loss of attitude awareness or energy state awareness due to lack of external visual reference cues was a significant causal factor in 17 of the 18 reviewed flights. CAST recommended that "Virtual Day-Visual Meteorological Condition" (Virtual Day-VMC) displays be developed to provide the visual cues necessary to prevent loss-of-control resulting from flight crew spatial disorientation and loss of energy state awareness. Synthetic vision or equivalent systems (SVS) were identified for a design "safety enhancement" (SE-200). Part of this SE involves the conduct of research for developing minimum aviation system performance standards (MASPS) for these flight deck display technologies to aid flight crew attitude and energy state awareness similar to that of a virtual day-VMC-like environment. This paper will describe a novel experimental approach to evaluating a flight crew's ability to maintain attitude awareness and to prevent entry into unusual attitudes across several SVS optical flow design considerations. Flight crews were subjected to compound-event scenarios designed to elicit channelized attention and startle/surprise within the crew. These high-fidelity scenarios, designed from real-world events, enable evaluation of the efficacy of SVS at improving flight crew attitude awareness to reduce the occurrence of LOC-I incidents in commercial flight operations.

Ellis, Kyle K. E.↗
NASA NTRS2018

Applications of Phase-Based Motion Processing

Image pyramids provide useful information in determining structural response at low cost using commercially available cameras. The current effort applies previous work on the complex steerable pyramid to analyze and identify imperceptible linear motions in video. Instead of implicitly computing motion spectra through phase analysis of the complex steerable pyramid and magnifying the associated motions, instead present a visual technique and the necessary software to display the phase changes of high frequency signals within video. The present technique quickly identifies regions of largest motion within a video with a single phase visualization and without the artifacts of motion magnification, but requires use of the computationally intensive Fourier transform. While Riesz pyramids present an alternative to the computationally intensive complex steerable pyramid for motion magnification, the Riesz formulation contains significant noise, and motion magnification still presents large amounts of data that cannot be quickly assessed by the human eye. Thus, user-friendly software is presented for quickly identifying structural response through optical flow and phase visualization in both Python and MATLAB.

Branch, Nicholas A.↗
NASA NTRSDate not supplied

Monitoring the Calibration of the DSCOVR EPIC Instrument’s Visible Cchannels Using MODIS and VIIRS as a Reference

The Earth Polychromatic Imaging Camera (EPIC) instrument aboard the Deep Space Climate Observatory (DSCOVR) satellite has a constant unique view of the sunlit disk of the Earth from the Lagrange-1 (L1) point nearly a million miles away from the Earth. Due to EPIC not having any on-board calibration systems, the ten spectral channels of EPIC must be inter-calibrated using vicarious on-orbit methods, such as ray-matching with well-calibrated low Earth orbiting satellites like the Aqua-MODIS and SNPP-VIIRS radiometers. The recently released EPIC version 3 L1B data has greatly enhanced the residual navigation errors found in prior versions, thereby decreasing the calibration uncertainty in the aforementioned ray-matching calibration techniques. The automated navigation correction used by these ray-matching methods for further improving the EPIC geolocation will be compared with a new navigation correction method utilizing optical flow between the EPIC and MODIS/VIIRS images. The DSCOVR satellite went into safe mode on June 27, 2019 due to an anomaly with the satellite’s attitude control system, but operations recommenced on March 2, 2020. The calibration of the EPIC instrument before it went into safe mode and after it resumed will be analyzed in order to investigate any potential calibration shifts or discontinuities caused by the spacecraft anomaly.

Conor Haney↗
NASA NTRSDate not supplied

GeoNEX-ML: A Machine Learning System for Geostationary Satellite Imagery

Improved capabilities of earth monitoring satellites are enabling a wide range of studies on the environmental effects of climate change, often leveraging the recent advancements in machine learning. At the same time, the new capabilities, including higher spatial resolution and temporal frequency, are expanding the amount of data generated at exponential rates. Further, a large majority of archived datasets generated by scientific processing is never used. This motivates the development of an efficient machine learning system for end-to-end processing of multi-level satellite datasets, from level 1 top of atmosphere observations to user friendly environmental variables of interest. Using current generation geostationary satellites GOES-16/17 (NOAA/NASA), Himawari-8/9 (JAXA), and GK-2A (Korea), we present an interchangeable set of machine models to perform spectral adjustment, physical model emulation, LEO-GEO emulation, and optical flow in a high performance computing environment. We use these tools to generate consistent virtual observations across sensors, perform atmospheric correction and cloud detection, and estimate land surface temperature and atmospheric winds. This approach aims to improve the robustness of remotely sensed data processing by learning from diverse sets of observations while enabling near real-time and on-demand capabilities.

Geostationary satellites↗
NASA NTRSDate not supplied

GeoNEX-ML: A Machine Learning System for Earth Observations

Improved capabilities of earth monitoring satellites are enabling a wide range of studies on the environmental effects of climate change, often leveraging the recent advancements in machine learning. At the same time, the new capabilities, including higher spatial resolution and temporal frequency, are expanding the amount of data generated at exponential rates. At the NASA Earth eXchange (NEX), we build deep learning methods to learn from cross sensor satellite-based Earth observations for generating new datasets with efficient processing techniques. Using current generation geostationary satellites on NEX, we present an interchangeable set of machine models to perform spectral adjustment, physical model emulation, LEO-GEO emulation, and optical flow. These tools are used to generate consistent virtual observations across sensors, perform atmospheric correction and cloud detection, and estimate land surface temperature and atmospheric winds. This approach aims to improve the robustness of remotely sensed data processing by learning from diverse sets of observations while enabling near real-time and on-demand capabilities.

Geostationary↗
NASA NTRSDate not supplied

Deep Learning System for Efficient Processing of Geostationary Satellite Imagery

Improved capabilities of Earth monitoring satellites are enabling a wide range of studies on the environmental effects of climate change, often leveraging the recent advancements in machine learning. At the same time, the new capabilities, including higher spatial resolution and temporal frequency, are expanding the amount of data generated at exponential rates. Further, a large majority of archived datasets generated by scientific processing is never used. This motivates the development of an efficient machine learning system for end-to-end processing of multi-level satellite datasets, from level 1 top of atmosphere observations to user friendly environmental variables of interest. Using current generation geostationary satellites GOES-16/17 (NOAA/NASA), and Himawari-8/9 (JAXA), we present an interchangeable set of machine models to perform spectral adjustment among sensors, physical model emulation, LEO-GEO emulation, and optical flow in a high performance computing environment. We use these tools on the NASA Earth eXchange (NEX) to generate consistent virtual observations across sensors, perform atmospheric correction and cloud detection, and estimate surface reflectance, surface temperature and atmospheric winds. This approach aims to improve the robustness of remotely sensed data processing by learning from diverse sets of observations while enabling near real-time and on-demand capabilities.

Thomas Vandal↗
NASA NTRS2020

Automatic Feature Tracking on Small Bodies for Autonomous Approach

Abstract—The autonomous approach of a spacecraft to an asteroid or comet (a small body) relies heavily on visual feature tracking to aid in estimating relative trajectories and the properties of the small body. Feature tracking for small bodies brings several challenges, including changing lighting, poor visual texture, and a concentration of features in a small part of an image. Six existing, open-source algorithms for feature tracking were tested on a simulated dataset and compared to the ground truth in the path of features. The main finding is that none of the algorithms provide all of the desired characteristics of long feature tracks with low errors and few outliers. Instead, there is a trade-off between long feature tracks and low error. The feature-matching algorithms SIFT, and BRISK provide good error characteristics, but short feature tracks, whereas the optical flow algorithm KLT provides long feature tracks, but with many features of large error. Given the challenges in feature tracking, it is recommended to focus development on each component of a feature tracking system: detection, description, and outlier rejection.

Morrell, Benjamin J↗
DOE OSTI2015

Frontal view reconstruction for iris recognition

Iris recognition can be accomplished for a wide variety of eye images by correcting input images with an off-angle gaze. A variety of techniques, from limbus modeling, corneal refraction modeling, optical flows, and genetic algorithms can be used. A variety of techniques, including aspherical eye modeling, corneal refraction modeling, ray tracing, and the like can be employed. Precomputed transforms can enhance performance for use in commercial applications. With application of the technologies, images with significantly unfavorable gaze angles can be successfully recognized.

97 MATHEMATICS AND COMPUTING↗