Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS2006

Generating CAHV and CAHVOR Images with Shadows in ROAMS

Part of the Rover Analysis, Modeling and Simulation (ROAMS) software that synthesizes images of terrain has been augmented to make the images more realistic. [ROAMS was described in "Simulating Operation of a Planetary Rover" (NPO-30722), NASA Tech Briefs, Vol. 28, No. 9 (September 2004), page 52. ROAMS simulates the operation of a robotic vehicle (rover) exploring terrain on a remote planet.] The images are needed for modeling responses of rover cameras that provide sensory inputs for machine-vision-based algorithms for controlling the motion of the rover. The augmented image-synthesizing part of the ROAMS software supports terrain geometry and texture specifiable by the user, CAHV and CAHVOR camera models, and more-realistic shadowing (see figure). (The letters in "CAHV" represent vectors in a standard photogrammetric model of a pinhole camera. Letters O and R in "CAHVOR" represent vectors used to model distortions.) A contemplated future version of ROAMS would support the CAHVORE model, which represents more-general cameras, including those having fish-eye or other wide-field-of-view lenses. (Letter E in "CAHVORE" represents a vector used to model apparent motion of a camera entrance pupil.)

Madison, Richard↗
NASA NTRS2004

Hybrid Image-Plane/Stereo Manipulation

Hybrid Image-Plane/Stereo (HIPS) manipulation is a method of processing image data, and of controlling a robotic manipulator arm in response to the data, that enables the manipulator arm to place an end-effector (an instrument or tool) precisely with respect to a target (see figure). Unlike other stereoscopic machine-vision-based methods of controlling robots, this method is robust in the face of calibration errors and changes in calibration during operation. In this method, a stereoscopic pair of cameras on the robot first acquires images of the manipulator at a set of predefined poses. The image data are processed to obtain image-plane coordinates of known visible features of the end-effector. Next, there is computed an initial calibration in the form of a mapping between (1) the image-plane coordinates and (2) the nominal three-dimensional coordinates of the noted end-effector features in a reference frame fixed to the main robot body at the base of the manipulator. The nominal three-dimensional coordinates are obtained by use of the nominal forward kinematics of the manipulator arm that is, calculated by use of the currently measured manipulator joint angles and previously measured lengths of manipulator arm segments under the assumption that the arm segments are rigid, that the arm lengths are constant, and that there is no backlash. It is understood from the outset that these nominal three-dimensional coordinates are likely to contain possibly significant calibration errors, but the effects of the errors are progressively reduced, as described next. As the end-effector is moved toward the target, the calibration is updated repeatedly by use of data from newly acquired images of the end-effector and of the corresponding nominal coordinates in the manipulator reference frame. By use of the updated calibration, the coordinates of the target are computed in manipulator-reference-frame coordinates and then used to the necessary manipulator joint angles to position and orient the end-effector at the target with respect to the same kinematic model from the calibration step. As the end-effector/target distance decreases, the computed coordinates of the end-effector and target become more nearly affected by the same errors, so that the differences between their coordinates become increasingly precise. When the end-effector reaches the target, the remaining effective position error is the distance that corresponds to more than about one pixel in the stereoscopic images of the target.

Baumgartner, Eric↗
NASA NTRS2004

Second-Generation Six-Limbed Experimental Robot

The figure shows the LEMUR II - the second generation of the Limbed Excursion Mechanical Utility Robot (LEMUR), which was described in "Six-Legged Experimental Robot" (NPO-20897), NASA Tech Briefs, Vol. 25, No. 12 (December 2001), page 58. The LEMUR II incorporates a number of improvements, including new features, that extend its capabilities beyond those of its predecessor, which is now denoted the LEMUR I. To recapitulate: the LEMUR I was a six-limbed robot for demonstrating robotic capabilities for assembly, maintenance, and inspection. The LEMUR I was designed to be capable of walking autonomously along a truss structure toward a mechanical assembly at a prescribed location and to perform other operations. The LEMUR I was equipped with stereoscopic video cameras and image-data-processing circuitry for navigation and mechanical operations. It was also equipped with a wireless modem, through which it could be commanded remotely. Upon arrival at a mechanical assembly, the LEMUR I would perform simple mechanical operations with one or both of its front limbs. It could also transmit images to a host computer. Each of the six limbs of the LEMUR I was operated independently. Each of the four rear limbs had three degrees of freedom (DOFs), while each of the front two limbs had four DOFs. The front two limbs were designed to hold, operate, and/or be integrated with tools. The LEMUR I included an onboard computer equipped with an assortment of digital control circuits, digital input/output circuits, analog-to-digital converters for input, and digital-to-analog (D/A) converters for output. Feedback from optical encoders in the limb actuators was utilized for closed-loop microcomputer control of the positions and velocities of the actuators. The LEMUR II incorporates the following improvements over the LEMUR I: a) The drive trains for the joints of the LEMUR II are more sophisticated, providing greater torque and accuracy. b) The six limbs are arranged symmetrically about a hexagonal body platform instead of in straight lines along the sides. This symmetrical arrangement is more conducive to omnidirectional movement in a plane. c) The number of degrees of freedom of each of the rear four limbs has been increased by one. Now, every limb has four degrees of freedom: three at the hip (or shoulder, depending on one s perspective) and one at the knee (or elbow, depending on one s perspective). d) Now every limb (instead of only the two front limbs) can perform operations. For this purpose, each limb is tipped with an improved quick-release mechanism for swapping of end-effector tools. e) New end-effector tools have been developed. These include an instrumented rotary driver that accepts all tool bits that have 0.125-in. (3.175-mm)-diameter shanks, a charge-coupled-device video camera, a super bright light-emitting diode for illuminating the work area of the robot, and a generic collet tool that can be quickly and inexpensively modified to accept any cylindrical object up to 0.5 in. (12.7 mm) in diameter. f) The stereoscopic cameras are mounted on a carriage that moves along a circular track, thereby providing for omnidirectional machine vision. g) The control software has been augmented with software that implements innovations reported in two prior NASA Tech Briefs articles: the HIPS algorithm ["Hybrid Image-Plane/Stereo Manipulation" (NPO-30492), Vol. 28, No. 7 (July 2004), page 55] and the CAMPOUT architecture ["An Architecture for Controlling Multiple Robots" (NPO-30345), Vol. 28, No. 10 (October 2004), page 65].

Kennedy, Brett↗
NASA NTRS2003

Device for Automated Cutting and Transfer of Plant Shoots

A device that enables the automated cutting and transfer of plant shoots is undergoing development for use in the propagation of plants in a nursery or laboratory. At present, it is standard practice for a human technician to use a knife and forceps to cut, separate, and grasp a plant shoot. The great advantage offered by the present device is that its design and operation are simpler than would be those of a device based on the manual cutting/separation/grasping procedure. [The present device should not be confused with a prior device developed for partly the same purpose and described in Compliant Gripper for a Robotic Manipulator (NPO-21104), NASA Tech Briefs, Vol. 27, No. 3 (March 2003), page 59.]. The device (see figure) includes a circular tube sharpened at its open (lower) end and mounted on a robotic manipulator at its closed (upper) end. The robotic manipulator simply pushes the sharpened open end of the tube down onto a bed of plants and rotates a few degrees clockwise then counterclockwise about the vertical axis, causing the tube to cut a cylindrical plug of plant material. Exploiting the natural friction between the tube and plug, the tube retains the plug, without need for a gripping mechanism and control. The robotic manipulator then retracts the tube, translates it to a new location over a plant-growth tray, and inserts the tube part way into the growth medium at this location in the tray. A short burst of compressed air is admitted to the upper end of the tube to eject the plug of plant material and drive it into the growth medium. A prototype has been tested and verified to function substantially as intended. It is projected that in the fully developed robotic plant-propagation system, the robot control system would include a machine- vision subsystem that would automatically guide the robotic manipulator in choosing the positions from which to cut plugs of plant material. Planned further development efforts also include more testing and refinement of the design and operation described above.

Cipra, Raymond↗
NASA NTRS2012

Ground Simulation of an Autonomous Satellite Rendezvous and Tracking System Using Dual Robotic Systems

A hardware-in-the-loop ground system was developed for simulating a robotic servicer spacecraft tracking a target satellite at short range. A relative navigation sensor package "Argon" is mounted on the end-effector of a Fanuc 430 manipulator, which functions as the base platform of the robotic spacecraft servicer. Machine vision algorithms estimate the pose of the target spacecraft, mounted on a Rotopod R-2000 platform, relay the solution to a simulation of the servicer spacecraft running in "Freespace", which performs guidance, navigation and control functions, integrates dynamics, and issues motion commands to a Fanuc platform controller so that it tracks the simulated servicer spacecraft. Results will be reviewed for several satellite motion scenarios at different ranges. Key words: robotics, satellite, servicing, guidance, navigation, tracking, control, docking.

Trube, Matthew J.↗
NASA NTRS2016

Towards Autonomous Operations of the Robonaut 2 Humanoid Robotic Testbed

The Robonaut project has been conducting research in robotics technology on board the International Space Station (ISS) since 2012. Recently, the original upper body humanoid robot was upgraded by the addition of two climbing manipulators ("legs"), more capable processors, and new sensors, as shown in Figure 1. While Robonaut 2 (R2) has been working through checkout exercises on orbit following the upgrade, technology development on the ground has continued to advance. Through the Active Reduced Gravity Offload System (ARGOS), the Robonaut team has been able to develop technologies that will enable full operation of the robotic testbed on orbit using similar robots located at the Johnson Space Center. Once these technologies have been vetted in this way, they will be implemented and tested on the R2 unit on board the ISS. The goal of this work is to create a fully-featured robotics research platform on board the ISS to increase the technology readiness level of technologies that will aid in future exploration missions. Technology development has thus far followed two main paths, autonomous climbing and efficient tool manipulation. Central to both technologies has been the incorporation of a human robotic interaction paradigm that involves the visualization of sensory and pre-planned command data with models of the robot and its environment. Figure 2 shows screenshots of these interactive tools, built in rviz, that are used to develop and implement these technologies on R2. Robonaut 2 is designed to move along the handrails and seat track around the US lab inside the ISS. This is difficult for many reasons, namely the environment is cluttered and constrained, the robot has many degrees of freedom (DOF) it can utilize for climbing, and remote commanding for precision tasks such as grasping handrails is time-consuming and difficult. Because of this, it is important to develop the technologies needed to allow the robot to reach operator-specified positions as autonomously as possible. The most important progress in this area has been the work towards efficient path planning for high DOF, highly constrained systems. Other advances include machine vision algorithms for localizing and automatically docking with handrails, the ability of the operator to place obstacles in the robot's virtual environment, autonomous obstacle avoidance techniques, and constraint management.

Badger, Julia↗
NASA NTRS2016

Systems and Methods for Automated Water Detection Using Visible Sensors

Systems and methods are disclosed that include automated machine vision that can utilize images of scenes captured by a 3D imaging system configured to image light within the visible light spectrum to detect water. One embodiment includes autonomously detecting water bodies within a scene including capturing at least one 3D image of a scene using a sensor system configured to detect visible light and to measure distance from points within the scene to the sensor system, and detecting water within the scene using a processor configured to detect regions within each of the at least one 3D images that possess at least one characteristic indicative of the presence of water.

Rankin, Arturo L.↗
NASA NTRS1993

CMOS Active Pixel Image Sensor

A new CMOS active pixel image sensor is reported. The sensor uses a 2.0mue double-poly, double-metal foundry CMOS process and is realized as a 28 x 28 array of 40 mue x 40 mue pixels. The Sensor features TTL compatible voltages, low noise and large dynamic range, and will be useful in machine vision and smart sensor applications.

CMOS↗
NASA NTRS2021

Detection and Tracking of Aircraft in the Far-Field from Small Unmanned Aerial Systems

Onboard far-field aircraft detection is needed for safe non-cooperative traffic mitigation in autonomous small Unmanned Aerial System (sUAS) operations. Machine vision systems, based on standard optics and visible light detectors, possess the ideal size, weight, and power (SWaP) requirements for sUAS. This work presents the design and analysis of a novel aircraft detection and tracking pipeline based on optical sensing alone. Key contributions of the work include a refined range inequality model based on sensing and detection with FAA well-clear separation assurance distances between aircraft in mind, a detector fusion method to maximize the benefit of two image detectors, and a comparative analysis of Linear Kalman-filtering and Extended Kalman-filtering to seek optimal tracking performance. The pipeline is evaluated offline against multiple intruder platforms, using two types of flight encounters: multirotor sUAS vs. fixed-wing sUAS and multirotor sUAS vs. general aviation(GA)plane. Analysis is restricted to the rate-limiting head-on and departing collision volume cases vertically separated for safety. Results indicate that it is feasible to use the proposed optical spatial-temporal tracking algorithm to provide adequate alerting time to prevent penetration of well-clear separation volumes for both sUAS and GA aircraft.

Unmanned Aerial System↗
NASA NTRS2021

Towards the Development of a Multi-Agent Cognitive Networking System for the Lunar Environment

This paper details the development of a multi-agent cognitive system intended to optimize networking performance in the lunar environment. NASA’s current concept of the future of lunar communication, LunaNet [1], outlines a complex network of networks. Challenges such as scalability, interoperability and reliability must first be addressed to successfully fulfill this vision. Machine intelligence can greatly reduce the reliance on human operators and enable efficient operations for tasks such as scheduling and network management. The application of machine learning, artificial intelligence, and other automated decision-making techniques can be used to allow network nodes to intelligently sense and adapt to changes in the environment such as link disruptions, new nodes joining the network, and support for a diverse range of protocols. Cognitive networking seeks to evolve these technologies into an autonomous system with improved science data return, reliability, and scalability. In this paper, we study three main areas a means to further develop cognitive networking capabilities: networking and flight software development, analysis of wireless data for modeling and simulation, and development of algorithms for a multi-agent system.

Cognitive Networking↗
NASA NTRSDate not supplied

Towards the Development of a Multi-Agent Cognitive Networking System for the Lunar Environment

This paper details the development of a multi-agent cognitive system intended to optimize networking performance in the lunar environment. NASA’s current concept of the future of lunar communication, LunaNet, outlines a complex network of networks. Challenges such as scalability, interoperability and reliability must first be addressed to successfully fulfill this vision. Machine intelligence can greatly reduce the reliance on human operators and enable efficient operations for tasks such as scheduling and network management. The application of machine learning, artificial intelligence, and other automated decision-making techniques can be used to allow network nodes to intelligently sense and adapt to changes in the environment such as link disruptions, new nodes joining the network, and support for a diverse range of protocols. Cognitive networking seeks to evolve these technologies into an autonomous system with improved science data return, reliability, and scalability. In this paper, we study three main areas a means to further develop cognitive networking capabilities: networking and flight software development, analysis of wireless data for modeling and simulation, and development of algorithms for a multi-agent system.

Rachel Dudukovich↗
NASA NTRS2022

Towards the Development of a Multi-Agent Cognitive Networking System for the Lunar Environment

This paper details the development of a multi-agent cognitive system intended to optimize networking performance in the lunar environment. One concept of the future of lunar communication, LunaNet, outlines a complex network of networks. Challenges such as scalability, interoperability, and reliability must first be addressed to successfully fulfill this vision. Machine intelligence can greatly reduce the reliance on human operators and enable efficient operations for tasks such as scheduling and network management. Machine learning, artificial intelligence, and other automated decision-making techniques can be used to allow network nodes to intelligently sense and adapt to changes in the environment such as link disruptions, new nodes joining the network, and support for a diverse range of protocols. Cognitive networking seeks to evolve these technologies into an autonomous system with improved science data return, reliability, and scalability. In this paper, we study four main areas as a means to further develop cognitive networking capabilities: networking protocol development, analysis of wireless data for modeling and simulation, development of algorithms for a multi-agent system, and spectrum sensing technology.

cognitive networking↗
NASA NTRS2021

AI4MARS: A Dataset for Terrain-Aware Autonomy on Mars

Deep learning has quickly become a necessity for selfdriving vehicles on Earth. In contrast, the self-driving vehicles on Mars, including NASA’s latest rover, Perseverance, which is planned to land on Mars in February 2021, are still driven by classical machine vision systems. Deep learning capabilities, such as semantic segmentation and object recognition, would substantially benefit the safety and productivity of ongoing and future missions to the red planet. To this end, we created the first large-scale dataset, AI4Mars, for training and validating terrain classification models for Mars, consisting of ~326K semantic segmentation full image labels on 35K images from Curiosity, Opportunity, and Spirit rovers, collected through crowdsourcing. Each image was labeled by ~10 people to ensure greater quality and agreement of the crowdsourced labels. It also includes ~1.5K validation labels annotated by the rover planners and scientists from NASA’s MSL (Mars Science Laboratory) mission, which operates the Curiosity rover, and MER (Mars Exploration Rovers) mission, which operated the Spirit and Opportunity rovers. We trained a DeepLabv3 model on the AI4Mars training dataset and achieved over 96% overall classification accuracy on the test set. The dataset is made publicly available.1

Ono, Hiro↗
NASA NTRS2022

Benchmarking and Testing of Qualcomm Snapdragon System-on-Chip for JPL Space Applications and Missions

As some space missions become more challenging due to new environments, greater distances, or more limited size, weight, and power (SWaP) constraints, spacecraft avionics must adapt to allow the spacecraft to be more autonomous and agile---eliminating the Spacecraft-Earth-Spacecraft feedback loop whenever possible. Prime examples of such missions include Aerobots (such as Ingenuity with extremely low SWaP constraints and demanding signal/image processing during flight) and landers in possibly hostile environments (such as a Europa lander mission, with limited communication capacity, high latency, and constrained power budget). To address these challenges, JPL worked with Qualcomm to demonstrate the use of their Snapdragon 801 system-on-chip (SoC) onboard the Ingenuity Helicopter on Mars. The Qualcomm Snapdragon SoC contains various subsystems, including an ARM cluster, a Graphics processing unit, a Digital Signal Processing subsystem, a Neural Processing Engine, Image Signal Processing subsystem, among others. Since the success of Ingenuity, JPL is continuing to work with Qualcomm to address other applications of the Snapdragon SoC technology. This includes the deployment of two 855 Snapdragon development boards onboard the International Space Station (ISS) for successful in-situ benchmarking of applications in space (beyond those tested on Ingenuity). In this paper, we will examine the performance of various applications that have been identified to benefit from greater onboard computational capability. These applications include (among others): machine vision algorithms that are expected to be critical in autonomous entry-descent-and-landing scenarios and real-time Aerobot flight navigation; Hyperspectral compression algorithms; Synthetic Aperture Radar Processing along with various instrument processing algorithms. We discuss how the infusion of Qualcomm's Snapdragon SoC is capable of enabling missions that may not have been able to achieve their goals with traditional flight computing. In addition, we also show that for some algorithms, the software implementation on the Snapdragon SoC outperforms traditional FPGA implementations.

Cretu, Vlad↗
NASA NTRS2015

Who's Got the Bridge? - Towards Safe, Robust Autonomous Operations at NASA Langley's Autonomy Incubator

NASA aeronautics research has made decades of contributions to aviation. Both aircraft and air traffic management (ATM) systems in use today contain NASA-developed and NASA sponsored technologies that improve safety and efficiency. Recent innovations in robotics and autonomy for automobiles and unmanned systems point to a future with increased personal mobility and access to transportation, including aviation. Automation and autonomous operations will transform the way we move people and goods. Achieving this mobility will require safe, robust, reliable operations for both the vehicle and the airspace and challenges to this inevitable future are being addressed now in government labs, universities, and industry. These challenges are the focus of NASA Langley Research Center's Autonomy Incubator whose R&D portfolio includes mission planning, trajectory and path planning, object detection and avoidance, object classification, sensor fusion, controls, machine learning, computer vision, human-machine teaming, geo-containment, open architecture design and development, as well as the test and evaluation environment that will be critical to prove system reliability and support certification. Safe autonomous operations will be enabled via onboard sensing and perception systems in both data-rich and data-deprived environments. Applied autonomy will enable safety, efficiency and unprecedented mobility as people and goods take to the skies tomorrow just as we do on the road today.

Allen, B. Danette↗
NASA NTRS2006

Acceleration of Stereo Correlation in Verilog

To speed up vision processing in low speed, low power devices, embedding FPGA hardware is becoming an effective way to add processing capability. FPGAs offer the ability to flexibly add parallel and/or deeply pipelined computation to embedded processors without adding significantly to the mass and power requirements of an embedded system. This paper will discuss the JPL stereo vision system, and describe how a portion of that system was accelerated by using custom FPGA hardware to process the computationally intensive portions of JPL stereo. The architecture described takes full advantage of the ability of an FPGA to use many small computation elements in parallel. This resulted in a 16 times speedup in real hardware over using a simple linear processor to compute image correlation and disparity.

machine vision↗
NASA NTRS2011

FPGA Implementation of Stereo Disparity with High Throughput for Mobility Applications

High speed stereo vision can allow unmanned robotic systems to navigate safely in unstructured terrain, but the computational cost can exceed the capacity of typical embedded CPUs. In this paper, we describe an end-to-end stereo computation co-processing system optimized for fast throughput that has been implemented on a single Virtex 4 LX160 FPGA. This system is capable of operating on images from a 1024 x 768 3CCD (true RGB) camera pair at 15 Hz. Data enters the FPGA directly from the cameras via Camera Link and is rectified, pre-filtered and converted into a disparity image all within the FPGA, incurring no CPU load. Once complete, a rectified image and the final disparity image are read out over the PCI bus, for a bandwidth cost of 68 MB/sec. Within the FPGA there are 4 distinct algorithms: Camera Link capture, Bilinear rectification, Bilateral subtraction pre-filtering and the Sum of Absolute Difference (SAD) disparity. Each module will be described in brief along with the data flow and control logic for the system. The system has been successfully fielded upon the Carnegie Mellon University's National Robotics Engineering Center (NREC) Crusher system during extensive field trials in 2007 and 2008 and is being implemented for other surface mobility systems at JPL.

Random access memory↗