Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “visual stereo”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Current and future use of TOPSAR digital topographic data for volcanological research

In several investigations of volcanoes, high quality digital elevation models (DEM's) are required to study either the geometry of the volcano or to investigate temporal changes in relief due to eruptions. Examples include the analysis of volume changes of a volcanic dome, the prediction of flow paths for pyroclastic flows, and the quantitative investigation of the geometry of valleys carved by volcanic mudflows. Additionally, to provide input data for models of lava flow emplacement, accurate measurements are needed of the thickness of lava flows as a function of distance from the vent and local slope. Visualization of volcano morphology is also aided by the ability to view a DEM from oblique perspectives. Until recently, the generation of these DEM's has required either high resolution stereo air photographs or extensive field surveying using the Global Positioning System (GPS) and other field techniques. Through the use of data collected by the NASA/JPL TOPSAR system, it is now possible to remotely measure the topography of volcanoes using airborne radar interferometry. TOPSAR data can be collected day or night under any weather conditions, thereby avoiding the problems associated with the derivation of DEM's from air photographs that may often contain clouds. Here we describe some of our initial work on volcanoes using TOPSAR data for Mt. Hekla (Iceland) and Vesuvius (Italy). We also outline various TOPSAR topographic studies of volcanoes in the Galapagos and Hawaii that will be conducted in the near future, describe how TOPSAR complements the volcanology investigations to be conducted with orbital radars (SIR-C/X-SAR, JERS-1 and ERS-1), and place these studies into the broader context of NASA's Global Change Program.

Mouginis-Mark, Peter J.↗

Solar System Visualizations

Solar System Visualization products enable scientists to compare models and measurements in new ways that enhance the scientific discovery process, enhance the information content and understanding of the science results for both science colleagues and the public, and create.visually appealing and intellectually stimulating visualization products. Missions supported include MER, MRO, and Cassini. Image products produced include pan and zoom animations of large mosaics to reveal the details of surface features and topography, animations into registered multi-resolution mosaics to provide context for microscopic images, 3D anaglyphs from left and right stereo pairs, and screen captures from video footage. Specific products include a three-part context animation of the Cassini Enceladus encounter highlighting images from 350 to 4 meter per pixel resolution; Mars Reconnaissance Orbiter screen captures illustrating various instruments during assembly and testing at the Payload Hazardous Servicing Facility at Kennedy Space Center; and an animation of Mars Exploration Rover Opportunity's 'Rub al Khali' panorama where the rover was stuck in the deep fine sand for more than a month. This task creates new visualization products that enable new science results and enhance the public's understanding of the Solar System and NASA's missions of exploration.

Brown, Alison M.↗

The New Realm of 3-D Vision

Dimension Technologies Inc., developed a line of 2-D/3-D Liquid Crystal Display (LCD) screens, including a 15-inch model priced at consumer levels. DTI's family of flat panel LCD displays, called the Virtual Window(TM), provide real-time 3-D images without the use of glasses, head trackers, helmets, or other viewing aids. Most of the company initial 3-D display research was funded through NASA's Small Business Innovation Research (SBIR) program. The images on DTI's displays appear to leap off the screen and hang in space. The display accepts input from computers or stereo video sources, and can be switched from 3-D to full-resolution 2-D viewing with the push of a button. The Virtual Window displays have applications in data visualization, medicine, architecture, business, real estate, entertainment, and other research, design, military, and consumer applications. Displays are currently used for computer games, protein analysis, and surgical imaging. The technology greatly benefits the medical field, as surgical simulators are helping to increase the skills of surgical residents. Virtual Window(TM) is a trademark of Dimension Technologies Inc.

Source record↗

Machine Vision for Autonomous Small Body Navigation

This paper describes machine vision algorithms that enable precision guidance and hazard avoidance during small body exploration through onboard visual feature tracking and landmark recognition.

Estimation Landmark Recognition↗

Autonomous Ocean World Exploration: Advancement of a Virtual Testbed

The search for life (extinct or extant) and potentially habitable bodies in our solar system and beyond is one of the 12 priority science questions outlined in the National Acadamies’ 2022 decadal survey [5]. Extraterrestrial destinations containing liquid water present an opportunity to search for life as we know it, and in recent years an increasing number of such locations have been discovered within our solar system. Several Jovian moons—Europa, Ganymede, and Callisto [10]—and the Saturnian moons Enceladus [8] and Titan [9] are known or suspected to harbor massive subsurface oceans. Of these "ocean worlds", Europa is the focus of at least one planned NASA orbiter mission, Europa Clipper [4], and an early lander mission concept, the Europa Lander [2, 3]. Whereas most robotic missions to the Moon and Mars (e.g. orbiters, rovers, landers) to date have had ground controllers on Earth tightly involved in mission operations, missions to more distant worlds will require a high degree of onboard autonomy due to long communication lags and blackouts, harsh environments (radiation, cold), and more limited battery and hardware life. The past decade has seen great advances in both AI technologies and computing scalability and performance that offer promising solutions for spacecraft autonomy and motivate the software system and research programs described in this paper. The Ocean Worlds Autonomy Testbed for Exploration, Research, and Simulation (OceanWATERS) [1], which has been in development at the NASA Ames Research Center since 2018, is a virtual environment for testing lander autonomy solutions. It is built on the Robot Operating System (ROS), runs on consumer-grade Linux workstations, and was released as open source in 2020. OceanWATERS provides a physical and visual simulation of a prototypical lander in a Europa-like environment (Figure 1). The lander was modeled after requirements and specifications made in JPL’s Europa Lander Study of 2016 [3]. Simulated lander systems include stereo cameras and spotlights mounted on an antenna mast that pans and tilts, a 6 degrees of freedom (DoF) robotic arm with a force-torque sensor and two interchangeable end effectors, and a battery pack power system. The environment consists of multiple terrain models including a highly detailed model sourced from the FROST dataset [11], simulation of surrounding planetary bodies based on an ephemeris model, and lighting from the sun with associated surface illumination, reflectance, and shadows. Operations supported by OceanWATERS include panoramic and directed imaging of the environment and lander workspace, Cartesian and joint-level arm commanding, grinding of the terrain surface (e.g. digging a trench), and scooping of ground material (Figure 2) which can be discarded or collected as science samples in a receptacle that can be emptied (science operations themselves are not simulated). These operations are realized as ROS Actions and are complimented by a wide selection of telemetry that is continually produced by each lander subsystem. The power system model is driven by the open-source Generic Software Architecture for Prognostics (GSAP) [11] that predicts the battery’s remaining useful life and other characteristics. As a testbed for high-level autonomy, OceanWATERS provides an execution framework based on PLEXIL [12], an open-source plan specification language and execution engine developed largely at Ames. NASA's initial development of OceanWATERS, as well the Ocean Worlds Lander Autonomy Testbed (OWLAT) [6], a complimentary physical testbed developed at JPL, was the first step in a plan for realizing candidate onboard autonomy solutions for such planetary landers. In 2020 NASA solicited applications for its Autonomous Robotics Research for Ocean Worlds (ARROW) program, and in 2021 the similar Concepts for Ocean worlds Life Detection Technology (COLDTech) program. Collectively six research teams, based in universities and companies across the United States, were awarded grants to develop and demonstrate autonomy solutions on OceanWATERS and OWLAT. These 1–2-year projects have now finished or are nearing completion, and a wide variety of autonomy challenges in ocean world surface missions were addressed. Prototyped and demonstrated solutions have included autonomous discovery, response and adaptation to system faults and unexpected environmental events, world model synthesis through perception, plan synthesis using learned models, methods to optimize sample target selection and prioritize science data transmission, extension of PLEXIL for stochastic decision-making, and an integration of a model of JPL’s mission-ready COLDArm [7]. Technologies used in these projects include many forms of machine learning, causal reasoning, automated planning, Markov decision processes, formal methods, and other advanced techniques. A more detailed summary of the ARROW and COLDTech projects is given herein. OceanWATERS has had significant enhancements since its open-source release in 2020. Many of its new features were driven or shaped by feedback from the ARROW and COLDTech teams and requirements of their projects. In support of enabling autonomous adaptation to spacecraft faults (a specific capability solicited by both programs), a fault injection and detection framework was developed that supports a wide and growing range of fault types such as locked joints, image loss, and battery failures. The power system model was completed and integrated into the simulator, starting as a single-cell battery model and later upgraded to a multi-cell model with associated faults such as cell disconnection. Arm/terrain interaction was improved by adding a force-torque sensor and associated faults, and an analytic dig force model based on the Balovnev bucket force equations. Environment fidelity was increased by modeling terrain deformation resulting from digging and scooping; visual improvements were made in textures, lighting, and shadows. To facilitate interoperation with OWLAT, a unified command and telemetry interface between the testbeds was developed at the ROS level, along with a PLEXIL interface. The number of lander operations was greatly expanded (e.g. with Cartesian-based arm and antenna movement), and a framework was designed for users to build their own lander actions. A GUI for PLEXIL plan selection was created (Figure 3), and an expansive set of plans were added, such as those that illustrate patterns for fault handling. This paper provides a self-contained high-level description of OceanWATERS, focusing on more detailed coverage of the aforementioned enhancements. It provides a high-level summary of the projects undertaken by participants in the ARROW and COLDTech programs and how these efforts have helped shape OceanWATERS. Finally, potential future work and directions for the testbed are listed, as likely informed by the recent planetary science decadal survey [5].

K Michael Dalal↗

Deciphering the Precision of Stereo IKONOS Canopy Height Models for U.S. Forests with G-LiHT Airborne LiDAR

Few studies have evaluated the precision of IKONOS stereo data for measuring forest canopy height. The high cost of airborne light detection and ranging (LiDAR) data collection for large area studies and the present lack of a spaceborne instrument lead to the need to explore other low cost options. The US Government currently has access to a large archive of commercial high-resolution imagery, which could be quite valuable to forest structure studies. At 1 m resolution, we here compared canopy height models (CHMs) and height data derived from Goddard's airborne LiDAR Hyper-spectral and Thermal Imager (G-LiHT) with three types of IKONOS stereo derived digital surface models (DSMs) that estimate CHMs by subtracting National Elevation Data (NED) digital terrain models (DTMs). We found the following in three different forested regions of the US after excluding heterogeneous and disturbed forest samples: (1) G-LiHT DTMs were highly correlated with NED DTMs with R (sup 2) greater than 0.98 and root mean square errors (RMSEs) less than 2.96 m; (2) when using one visually identifiable ground control point (GCP) from NED, G-LiHT DSMs and IKONOS DSMs had R (sup 2) greater than 0.84 and RMSEs of 2.7 to 4.1 m; and (3) one GCP CHMs for two study sites had R (sup 2) greater than 0.7 and RMSEs of 2.6 to 3 m where data were collected less than four years apart. Our results suggest that IKONOS stereo data are a useful LiDAR alternative where high-quality DTMs are available.

IKONOS↗

A computational model for dynamic vision

This paper describes a novel computational model for dynamic vision which promises to be both powerful and robust. Furthermore the paradigm is ideal for an active vision system where camera vergence changes dynamically. Its basis is the retinotopically indexed object-centered encoding of the early visual information. Specifically, the relative distances of objects to a set of referents is encoded in image registered maps. To illustrate the efficacy of the method, it is applied to the problem of dynamic stereo vision. Integration of depth information over multiple frames obtained by a moving robot generally requires precise information about the relative camera position from frame to frame. Usually, this information can only be approximated. The method facilitates the integration of depth information without direct use or knowledge of camera motion.

Moezzi, Saied↗

Vision Based Localization in Urban Environments

As part of DARPA's MARS2020 program, the Jet Propulsion Laboratory developed a vision-based system for localization in urban environments that requires neither GPS nor active sensors. System hardware consists of a pair of small FireWire cameras and a standard Pentium-based computer. The inputs to the software system consist of: 1) a crude grid-based map describing the positions of buildings, 2) an initial estimate of robot location and 3) the video streams produced by each camera. At each step during the traverse the system: captures new image data, finds image features hypothesized to lie on the outside of a building, computes the range to those features, determines an estimate of the robot's motion since the previous step and combines that data with the map to update a probabilistic representation of the robot's location. This probabilistic representation allows the system to simultaneously represent multiple possible locations, For our testing, we have derived the a priori map manually using non-orthorectified overhead imagery, although this process could be automated. The software system consists of two primary components. The first is the vision system which uses binocular stereo ranging together with a set of heuristics to identify features likely to be part of building exteriors and to compute an estimate of the robot's motion since the previous step. The resulting visual features and the associated range measurements are software component, a particle-filter based localization system. This system uses the map and the then fed to the second primary most recent results from the vision system to update the estimate of the robot's location. This report summarizes the design of both the hardware and software and will include the results of applying the system to the global localization of a robot over an approximately half-kilometer traverse across JPL'S Pasadena campus.

unmanned-ground vehicles↗

Topography from shading and stereo

Methods exploiting photometric information in images that have been developed in machine vision can be applied to planetary imagery. Integrating shape from shading, binocular stereo, and photometric stereo yields a robust system for recovering detailed surface shape and surface reflectance information. Such a system is useful in producing quantitative information from the vast volume of imagery being received, as well as in helping visualize the underlying surface.

Horn, Berthold K. P.↗

A virtual reality browser for Space Station models

The Graphics Analysis Facility at NASA/JSC has created a visualization and learning tool by merging its database of detailed geometric models with a virtual reality system. The system allows an interactive walk-through of models of the Space Station and other structures, providing detailed realistic stereo images. The user can activate audio messages describing the function and connectivity of selected components within his field of view. This paper presents the issues and trade-offs involved in the implementation of the VR system and discusses its suitability for its intended purposes.

Goldsby, Michael↗

Rover Slip Validation and Prediction Algorithm

A physical-based simulation has been developed for the Mars Exploration Rover (MER) mission that applies a slope-induced wheel-slippage to the rover location estimator. Using the digital elevation map from the stereo images, the computational method resolves the quasi-dynamic equations of motion that incorporate the actual wheel-terrain speed to estimate the gross velocity of the vehicle. Based on the empirical slippage measured by the Visual Odometry software of the rover, this algorithm computes two factors for the slip model by minimizing the distance of the predicted and actual vehicle location, and then uses the model to predict the next drives. This technique, which has been deployed to operate the MER rovers in the extended mission periods, can accurately predict the rover position and attitude, mitigating the risk and uncertainties in the path planning on high-slope areas.

Yen, Jeng↗

3D Stereo Immersive Environment Utilizing Mobile Devices / Peripherals

This project expands upon some of the FY23 IRAD funded "Full Lunar Surface Simulation Platform" (FLS-SP) derived technologies. The results have demonstrated a low-cost, compact, immersive technology that supports training, interaction, mission planning, and other man-in-loop activities in simulated environments called “3D Stereo Immersion Environment Utilizing Mobile Devices” (3D SIEUMD). These activities will become crucial for safety and mission, planning, and assurance in the Artemis program, as well as future commercial crew programs. The FLS-SP enabled the visualization of simulated lunar surface operations for rovers, landers, habitation, and EVA. The 3D SIEUMD technology allows for NASA S&MA experts, tasked with defining safety protocols and procedures, to interact in an immersive environment without expensive immersion gear. Reduction of the overall safety risk by immersing safety concepts into the design and engineering phase for very small cost is accomplished even before hardware is deployed to the lunar surface.

Scott Johnson↗

The CAVE (TM) automatic virtual environment: Characteristics and applications

Virtual reality may best be defined as the wide-field presentation of computer-generated, multi-sensory information that tracks a user in real time. In addition to the more well-known modes of virtual reality -- head-mounted displays and boom-mounted displays -- the Electronic Visualization Laboratory at the University of Illinois at Chicago recently introduced a third mode: a room constructed from large screens on which the graphics are projected on to three walls and the floor. The CAVE is a multi-person, room sized, high resolution, 3D video and audio environment. Graphics are rear projected in stereo onto three walls and the floor, and viewed with stereo glasses. As a viewer wearing a location sensor moves within its display boundaries, the correct perspective and stereo projections of the environment are updated, and the image moves with and surrounds the viewer. The other viewers in the CAVE are like passengers in a bus, along for the ride. 'CAVE,' the name selected for the virtual reality theater, is both a recursive acronym (Cave Automatic Virtual Environment) and a reference to 'The Simile of the Cave' found in Plato's 'Republic,' in which the philosopher explores the ideas of perception, reality, and illusion. Plato used the analogy of a person facing the back of a cave alive with shadows that are his/her only basis for ideas of what real objects are. Rather than having evolved from video games or flight simulation, the CAVE has its motivation rooted in scientific visualization and the SIGGRAPH 92 Showcase effort. The CAVE was designed to be a useful tool for scientific visualization. The Showcase event was an experiment; the Showcase chair and committee advocated an environment for computational scientists to interactively present their research at a major professional conference in a one-to-many format on high-end workstations attached to large projection screens. The CAVE was developed as a 'virtual reality theater' with scientific content and projection that met the criteria of Showcase.

Kenyon, Robert V.↗

In-simulator assessment of trade-offs arising from mixture of color cuing and monocular, binoptic, and stereopsis cuing information

The use of monochrome Helmet Mounted Display (HMD) systems is becoming prevalent in today's complex flight mission environment. These HMD systems can provide stereopsis cueing as an almost natural byproduct for binocular helmet systems of an additional image generation source is provided. The addition of color cueing capability is much more difficult. The application of stereopsis cueing to advanced HMD and heads-down flight display concepts has demonstrated gains in pilot situation awareness and improved task performance. To provide stereopsis, binocular HMD systems must trade some of the total field-of-view (FOV) available from their two monocular fields to obtain a partial overlap region. The visual field then provides a mixture of cues, with monocular regions on both peripheries, and, in the overlapped center, a binoptic (the same image to both eyes) or, if lateral disparity is introduced to produce two images, a stereo region. The goal of this research was to assess the trade-offs arising from the mixture of color cueing and monocular, biocular, and stereopsis cueing information in peripheral monitoring displays as encountered in HMD systems. The accompanying effect of stereopsis cueing in the foveal display of tracking information was also assessed.

Williams, Steven P.↗

Evaluating the Usability of Pinchigator, a system for Navigating Virtual Worlds using Pinch Gloves

Appropriate design of two dimensional user interfaces (2D U/I) utilizing the well known WIMP (Window, Icon, Menu, Pointing device) environment for computer software is well studied and guidance can be found in several standards. Three-dimensional U/I design is not nearly so mature as 2D U/I, and standards bodies have not reached consensus on what makes a usable interface. This is especially true when the tools for interacting with the virtual environment may include stereo viewing, real time trackers and pinch gloves instead of just a mouse & keyboard. Over the last several years the authors have created a 3D U/I system dubbed Pinchigator for navigating virtual worlds based on the dVise dV/Mockup visualization software, Fakespace Pinch Gloves and Pohlemus trackers. The current work is to test the usability of the system on several virtual worlds, suggest improvements to increase Pinchigator s usability, and then to generalize about what was learned and how those lessons might be applied to improve other 3D U/I systems.

Hamilton, George S.↗

Photo-realistic Terrain Modeling and Visualization for Mars Exploration Rover Science Operations

Modern NASA planetary exploration missions employ complex systems of hardware and software managed by large teams of. engineers and scientists in order to study remote environments. The most complex and successful of these recent projects is the Mars Exploration Rover mission. The Computational Sciences Division at NASA Ames Research Center delivered a 30 visualization program, Viz, to the MER mission that provides an immersive, interactive environment for science analysis of the remote planetary surface. In addition, Ames provided the Athena Science Team with high-quality terrain reconstructions generated with the Ames Stereo-pipeline. The on-site support team for these software systems responded to unanticipated opportunities to generate 30 terrain models during the primary MER mission. This paper describes Viz, the Stereo-pipeline, and the experiences of the on-site team supporting the scientists at JPL during the primary MER mission.

Edwards, Laurence↗

Robot Sequencing and Visualization Program (RSVP)

The Robot Sequencing and Visualization Program (RSVP) is being used in the Mars Science Laboratory (MSL) mission for downlink data visualization and command sequence generation. RSVP reads and writes downlink data products from the operations data server (ODS) and writes uplink data products to the ODS. The primary users of RSVP are members of the Rover Planner team (part of the Integrated Planning and Execution Team (IPE)), who use it to perform traversability/articulation analyses, take activity plan input from the Science and Mission Planning teams, and create a set of rover sequences to be sent to the rover every sol. The primary inputs to RSVP are downlink data products and activity plans in the ODS database. The primary outputs are command sequences to be placed in the ODS for further processing prior to uplink to each rover. RSVP is composed of two main subsystems. The first, called the Robot Sequence Editor (RoSE), understands the MSL activity and command dictionaries and takes care of converting incoming activity level inputs into command sequences. The Rover Planners use the RoSE component of RSVP to put together command sequences and to view and manage command level resources like time, power, temperature, etc. (via a transparent realtime connection to SEQGEN). The second component of RSVP is called HyperDrive, a set of high-fidelity computer graphics displays of the Martian surface in 3D and in stereo. The Rover Planners can explore the environment around the rover, create commands related to motion of all kinds, and see the simulated result of those commands via its underlying tight coupling with flight navigation, motor, and arm software. This software is the evolutionary replacement for the Rover Sequencing and Visualization software used to create command sequences (and visualize the Martian surface) for the Mars Exploration Rover mission.

Cooper, Brian K.↗

Recent developments at JPL in the application of image processing to astronomy

Four applications of image processing to astronomy, automated location and analysis of star and galaxy images, geometric and radiometric decalibration of vidicon spectra, display of multiband radio images, and generation of high resolution polarization direction and magnitude maps from images are presented with illustrative examples. The technique by which a digital image can be analyzed automatically to locate and segregate between stars and galaxies and the steps performed by the classifier to determine the nature of each object are outlined. The classification program executed on a 48 inch Schmidt plate of the cluster of galaxies 655 is described. The calibration and decalibration steps to remove geometric and radiometric distortions from a silicon vidicon camera digital spectra are discussed. Three methods of displaying multispectral radio data, generating a mosaic of each image, producing a color coded image to depict radio velocity, and producing a stereo pair with radial velocity as depth are described. The generation of polarization information from images obtained through linear polarizing filters is illustrated, and it is concluded that in each case information was displayed using digital techniques which could not readily have been provided visually.

Lorre, J. J.↗