Telerobotic remote presence: Achievements and challenges
The topics discussed include the following: telepresence, telerobotics, teleoperators, man machine systems, robot control, and remote control.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The topics discussed include the following: telepresence, telerobotics, teleoperators, man machine systems, robot control, and remote control.
The Stanford/VA Interactive Robotics Laboratory set out in 1978 to test the hypothesis that industrial robotics technology could be applied to serve the manipulation needs of severely impaired individuals. Five generations of hardware, three generations of system software, and over 125 experimental subjects later, we believe that genuine utility is achievable. The experience includes development of over 65 task applications using voiced command, joystick control, natural language command and 3D object designation technology. A brief foray into virtual environments, using flight simulator technology, was instructive. If reality and virtuality come for comparable prices, you cannot beat reality. A detailed review of assistive robot anatomy and the performance specifications needed to achieve cost/beneficial utility will be used to support discussion of the future of rehabilitation telerobotics. Poised on the threshold of commercial viability, but constrained by the high cost of technically adequate manipulators, this worthy application domain flounders temporarily. In the long run, it will be the user interface that governs utility.
The technology described in this paper was designed to reduce latency to user interactions in immersive virtual reality environments. It is also ideally suited to telerobotic applications such as interaction with remote robotic manipulators in space or in deep sea operations. in such circumstances the significant latency is observed response to user stimulus which is due to communications delays, and the disturbing jerkiness due to low and unpredictable frame rates on compressed video user feedback or computationally limited virtual worlds, can be masked by our techniques. The user is provided with highly responsive visual feedback independent of communication or computational delays in providing physical video feedback or in rendering virtual world images. Virtual and physical environments can be combined seamlessly using these techniques.
A cooperative research on super long distance space telerobotics is now in progress both in Japan and USA. In this program. several key features will be tested, which can be applicable to the control of space robots as well as to terrestrial robots. Local (control) and remote (work) sites will be shared between Electrotechnical Lab (ETL) of MITI in Japan and Jet Propulsion Lab (JPL) in USA. The details of a test bed for this international program are discussed in this report.
Method of automated telerobotic inspection of surfaces undergoing development. Apparatus implementing method includes video camera that scans over surfaces to be inspected, in manner of mine detector. Images of surfaces compared with reference images to detect flaws. Developed for inspecting external structures of Space Station Freedom for damage from micrometeorites and debris from prior artificial satellites. On Earth, applied to inspection for damage, missing parts, contamination, and/or corrosion on interior surfaces of pipes or exterior surfaces of bridges, towers, aircraft, and ships.
This paper describes a telerobotics display system, the Multi-mode Manipulator Display System (MMDS), that has applications for a variety of remotely controlled tasks. Designed primarily to assist astronauts with the control of space robotics systems, the MMDS has applications for ground control of space robotics as well as for toxic waste cleanup, undersea, remotely operated vehicles, and other environments which require remote operations. The MMDS has three modes: (1) Manipulator Position Display (MPD) mode, (2) Joint Angle Display (JAD) mode, and (3) Sensory Substitution (SS) mode. These three modes are discussed in the paper.
The invention relates generally to systems for determining the range of an object from a reference point and, in one embodiment, to laser-directed ranging systems useful in telerobotics applications. Digital processing techniques are employed which minimize the complexity and cost of the hardware and software for processing range calculations, thereby enhancing the commercial attractiveness of the system for use in relatively low-cost robotic systems. The system includes a video camera for generating images of the target, image digitizing circuitry, and an associated frame grabber circuit. The circuit first captures one of the pairs of stereo video images of the target, and then captures a second video image of the target as it is partly illuminated by the light beam, suitably generated by a laser. The two video images, taken sufficiently close together in time to minimize camera and scene motion, are converted to digital images and then compared. Common pixels are eliminated, leaving only a digital image of the laser-illuminated spot on the target. Mw centroid of the laser illuminated spot is dm obtained and compared with a predetermined reference point, predetermined by design or calibration, which represents the coordinate at the focal plane of the laser illumination at infinite range. Preferably, the laser and camera are mounted on a servo-driven platform which can be oriented to direct the camera and the laser toward the target. In one embodiment the platform is positioned in response to movement of the operator's head. Position and orientation sensors are used to monitor head movement. The disparity between the digital image of the laser spot and the reference point is calculated for determining range to the target. Commercial applications for the system relate to active range-determination systems, such as those used with robotic systems in which it is necessary to determine the, range to a workpiece or object to be grasped or acted upon by a robot arm end-effector in response to commands generated by an operator. In one embodiment, the system provides a real-time image of the target for the operator as the robot approaches the object. The system is also adapted for use in virtual reality systems in which a remote object or workpiece is to be acted upon by a remote robot arm or other mechanism controlled by an operator.
A highly effective predictive/preview display technique for telerobotic servicing in space under several seconds communication time delay has been demonstrated on a large laboratory scale in May 1993, involving the Jet Propulsion Laboratory as the simulated ground control station and, 2500 miles away, the Goddard Space Flight Center as the simulated satellite servicing set-up. The technique is based on a high-fidelity calibration procedure that enables a high-fidelity overlay of 3-D graphics robot arm and object models over given 2-D TV camera images of robot arm and objects. To generate robot arm motions, the operator can confidently interact in real time with the graphics models of the robot arm and objects overlaid on an actual camera view of the remote work site. The technique also enables the operator to generate high-fidelity synthetic TV camera views showing motion events that are hidden in a given TV camera view or for which no TV camera views are available. The positioning accuracy achieved by this technique for a zoomed-in camera setting was about +/-5 mm, well within the allowable +/-12 mm error margin at the insertion of a 45 cm long tool in the servicing task.
The Hydraulic Manipulator Testbed (HMTB) is the kinematic replica of the Flight Telerobotic Servicer (FTS). One use of the HMTB is to evaluate advanced control techniques for accomplishing robotic maintenance tasks on board the Space Station. Most maintenance tasks involve the direct manipulation of the robot by a human operator when high-quality visual feedback is important for precise control. An experiment was conducted in the Systems Integration Branch at the Langley Research Center to compare several configurations of the manipulator wrist camera for providing visual feedback during an Orbital Replaceable Unit changeout task. Several variables were considered such as wrist camera angle, camera focal length, target location, lighting. Each study participant performed the maintenance task by using eight combinations of the variables based on a Latin square design. The results of this experiment and conclusions based on data collected are presented.
A laser-directed ranging system has utility for use in various fields, such as telerobotics applications and other applications involving physically handicapped individuals. The ranging system includes a single video camera and a directional light source such as a laser mounted on a camera platform, and a remotely positioned operator. In one embodiment, the position of the camera platform is controlled by three servo motors to orient the roll axis, pitch axis and yaw axis of the video cameras, based upon an operator input such as head motion. The laser is offset vertically and horizontally from the camera, and the laser/camera platform is directed by the user to point the laser and the camera toward a target device. The image produced by the video camera is processed to eliminate all background images except for the spot created by the laser. This processing is performed by creating a digital image of the target prior to illumination by the laser, and then eliminating common pixels from the subsequent digital image which includes the laser spot. A reference point is defined at a point in the video frame, which may be located outside of the image area of the camera. The disparity between the digital image of the laser spot and the reference point is calculated for use in a ranging analysis to determine range to the target.
The work reported in this paper is the result, of a collaboration between researchers at the Jet Propulsion Laboratory and Steve Charles, MD, a vitreo-retinal surgeon. The Robot Assisted MicroSurgery (RAMS) telerobotic workstation developed at JPL is a prototype of a system that will be completely under the manual control of a surgeon. The system has a slave robot that will hold surgical instruments. The slave robot motions replicate in six degrees of freedom those of tile. surgeon's hand measured using a master input device with a surgical instrument, shaped handle. The surgeon commands motions for the instrument by moving the handle in the desired trajectories. The trajectories are measured, filtered, and scaled down then used to drive the slave robot.
A tool was developed that assists surgeons in manipulating surgical instruments more precisely than is possible manually. The tool is a telemanipulator that scales down the surgeon's hand motion and filters tremor in the motion. The signals measured from the surgeon's hand are transformed and used to drive a six-degrees-of-freedom robot to position the surgical instrument mounted on its tip. A pilot study comparing the performance of the telemanipulator system against manual instrument positioning was conducted at the University of Southern California School of Medicine. The results show that a telerobotic tool can improve the performance of a microsurgeon by increasing the precision with which he can position surgical instruments, but this is achieved at the cost of increased time in performing the task. We believe that this technology will extend the capabilities of microsurgeons and allow more surgeons to perform highly skilled procedures currently performed only by the best surgeons. It will also enable performance of new surgical procedures that are beyond the capabilities of even the most skilled surgeons. Copyright 1999 Wiley-Liss, Inc.
This paper describes the approach and algorithms developed for model-based obstacle detection and distance computation for the NASA Ranger Telerobotic Flight Experiment. Objects of interest, such as manipulator arms, the Ranger vehicle and solar arrays, are modeled using a small set of component types.
The kinematic design of a telerobotic mechanism for tending a plant growth space science experiment chamber is described. Ground based control of tending mechanisms internal to space science experiments will allow ground based principal investigators to interact directly with their space science experiments.
Operator interface has recently emerged as an important element for efficient and safe operatorinteractions with the telerobotic system. Recent advances in graphical user interface (GUI) andgraphics/video merging technologies enable development of more efficient, flexible operatorinterfaces. This paper describes an advanced graphical operator interface newly developed for aremote surface inspection system at Jet Propulsion Laboratory. The interface has been designed sothat remote surface inspection can be performed by a single operator with an integrated robot controland image inspection capability. It supports three inspection strategies of teleoperated human visual inspection, human visual inspection with automated scanning, and machine-vision-based automated inspection.
Overview of the Human Exploration Telerobotics (HET) project.
On medium- to long-duration human spaceflight missions, latency in communications from Earth could reduce efficiency or hinder local operations, control, and monitoring of the various mission vehicles and other elements. Regardless of the degree of autonomy of any one particular element, a means of monitoring and controlling the elements in real time based on mission needs would increase efficiency and response times for their operation. Since human crews would be present locally, a local means for monitoring and controlling all the various mission elements is needed, particularly for robotic elements where response to interesting scientific features in the environment might need near- instantaneous manipulation and control. One of the elements proposed for medium- and long-duration human spaceflight missions, the Deep Space Habitat (DSH), is intended to be used as a remote residence and working volume for human crews. The proposed solution for local monitoring and control would be to provide a workstation within the DSH where local crews can operate local vehicles and robotic elements with little to no latency. The Telerobotics Workstation (TRWS) is a multi-display computer workstation mounted in a dedicated location within the DSH that can be adjusted for a variety of configurations as required. From an Intra-Vehicular Activity (IVA) location, the TRWS uses the Robot Application Programming Interface Delegate (RAPID) control environment through the local network to remotely monitor and control vehicles and robotic assets located outside the pressurized volume in the immediate vicinity or at low-latency distances from the habitat. The multiple display area of the TRWS allows the crew to have numerous windows open with live video feeds, control windows, and data browsers, as well as local monitoring and control of the DSH and associated systems.
This paper presents a concept for a human mission to Mars orbit that features direct robotic exploration of the planet s surface via teleoperation from orbit. This mission is a good example of Human Exploration using Real-time Robotic Operations (HERRO), an exploration strategy that refrains from sending humans to the surfaces of planets with large gravity wells. HERRO avoids the need for complex and expensive man-rated lander/ascent vehicles and surface systems. Additionally, the humans are close enough to the surface to effectively eliminate the two-way communication latency that constrains typical robotic space missions, thus allowing real-time command and control of surface operations and experiments by the crew. Through use of state-of-the-art telecommunications and robotics, HERRO provides the cognitive and decision-making advantages of having humans at the site of study for only a fraction of the cost of conventional human surface missions. It is very similar to how oceanographers and oil companies use telerobotic submersibles to work in inaccessible areas of the ocean, and represents a more expedient, near-term step prior to landing humans on Mars and other large planetary bodies. Results suggest that a single HERRO mission with six crew members could achieve the same exploratory and scientific return as three conventional crewed missions to the Mars surface.