SEARCH · Engineering Papers

Results for “computer vision techniques”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

NASA NTRS2012

Automatic Estimation of Volcanic Ash Plume Height using WorldView-2 Imagery

We explore the use of machine learning, computer vision, and pattern recognition techniques to automatically identify volcanic ash plumes and plume shadows, in WorldView-2 imagery. Using information of the relative position of the sun and spacecraft and terrain information in the form of a digital elevation map, classification, the height of the ash plume can also be inferred. We present the results from applying this approach to six scenes acquired on two separate days in April and May of 2010 of the Eyjafjallajokull eruption in Iceland. These results show rough agreement with ash plume height estimates from visual and radar based measurements.

pattern recognition↗

NASA NTRS1991

The Space Station Freedom Flight Telerobotic Servicer: the design and evolution of a dexterous space robot

The Flight Telerobotic Servicer (FTS) Project at the Goddard Space Flight Center is developing an advanced telerobotic system to assist in and reduce crew extravehicular activity (EVA) for Space Station) Freedom (SSF). The FTS will provide a telerobotic capability to the Freedom Station in the early assembly phases of the program and will be employed for assembly, maintenance, and inspection applications throughout the lifetime of the space station. Appropriately configured elements of the FTS will also be employed for robotic manipulation in remote satellite servicing applications and possibly the Lunar/Mars Program. In mid-1989, the FTS entered the flight system design and implementation phase (Phase C/D) of development with the signing of the FTS prime contract with Martin Marietta Astronautics Group in Denver, Colorado. The basic FTS design is now established and can be reported on in some detail. This paper will describe the FTS flight system design and the rationale for the specific design approaches and component selections. The current state of space technology and the nature of the FTS task dictate that the FTS be designed with sophisticated teleoperation capabilities for its initial primary operating mode. However, there are technologies, such as advanced computer vision and autonomous planning techniques currently in research and advanced development phases which would greatly enhance the FTS capabilities to perform autonomously in less structured work environments. Therefore, a specific requirement on the initial FTS design is that it has the capability to evolve as new technology becomes available. This paper will describe the FTS design approach for evolution to more autonomous capabilities. Some specific task applications of the FTS and partial automation approaches of these tasks will also be discussed in this paper.

Task Performance and Analysis↗

DOE OSTI2025

Automated Programmable Logic Controller Memory Forensics Using RGB Image Analysis and Deep Learning

The introduction of Industry 4.0 and Internet-based technologies has enhanced industrial control system operations but have inadvertently increased their vulnerabilities to cyber attacks. When an industrial control system is compromised, security analysts need to identify the root cause quickly to start the recovery process and develop mitigation strategies. Memory forensics is critical in the incident analysis process to ascertain what occurred. Approaches for analyzing the persistent memory in industrial control devices are limited and almost nonexistent for volatile memory. This chapter proposes an automated methodology for programmable logic controller memory dump analysis using computer vision and deep learning techniques. The methodology converts the sequences of bytes in a programmable logic controller memory dump to red-green-blue pixels and employs a deep learning model that learns the underlying patterns and features of pre-labeled forensic artifacts in images and segments them into distinct regions. The trained model is employed to automatically segment new memory images and identify forensic artifacts. Evaluation of the methodology on a Schneider Electric Modicon M221 programmable logic controller under code injection and code modification attacks demonstrates its ability to detect attack artifacts in memory dumps.

Asmar Awad, Rima [ORNL] (ORCID:0000000233407742)↗

DOE OSTI2025

Automated nuclear cloud feature extraction from film

Chemical, biological, radiological, nuclear, and explosives incidents require rapid detection and characterization for appropriate response. For a nuclear detonation, visible-light cameras may be used to locate the cloud and characterize fallout deposition when coupled with numerical models. Films from the United States’ nuclear testing era compose the only sizeable collection of imagery depicting high-yield detonations. These films offer unique insights into characteristics of flows involving scales that are difficult to replicate experimentally, and they are a valuable source of data for the validation of models for nuclear fallout transport, either as part of emergency response or forensic activities. In this work, we implement modern computer vision and machine learning techniques to identify and track the cloud automatically and subsequently determine the time dependence of some of its features. We trained a ResNet-18 image classifier on hundreds of images to categorize nuclear cloud morphology. Each category or cloud regime is determined by early cloud evolution and is associated to constitutive properties of the flow, such as distribution of vorticity. Next, we identified keypoint features using the KAZE algorithm and tracked these keypoints in the images, allowing us to determine the dimensions and velocities of the cloud across film frames. These measurements converted to real-world units provide valuable experimental data that can be used in the development and validation of nuclear cloud models. We compared the results of this method against manual cloud rise measurements from two different films. In one, our automated method accelerated the feature extraction process without sacrificing measurement accuracy.

Khristy, Joel [ORNL] (ORCID:0000000209963060)↗

NASA NTRS1991

The flight telerobotic servicer and technology transfer

The Flight Telerobotic Servicer (FTS) project at the Goddard Space Flight Center is developing an advanced telerobotic system to assist in and reduce crew extravehicular activity (EVA) for Space Station Freedom (SSF). The FTS will provide a telerobotic capability in the early phases of the SSF program and will be employed for assembly, maintenance, and inspection applications. The current state of space technology and the general nature of the FTS tasks dictate that the FTS be designed with sophisticated teleoperational capabilities for its internal primary operating mode. However, technologies such as advanced computer vision and autonomous planning techniques would greatly enhance the FTS capabilities to perform autonomously in less structured work environments. Another objective of the FTS program is to accelerate technology transfer from research to U.S. industry.

Andary, James F.↗

NASA NTRS2003

Terrain Portrayal for Head-Down Displays Flight Test

The Synthetic Vision Systems General Aviation (SVS-GA) element of NASA's Aviation Safety Program is developing technology to eliminate low visibility induced General Aviation (GA) accidents through the application of synthetic vision techniques. SVS displays present computer generated 3-dimensional imagery of the surrounding terrain to greatly enhance pilot's situation awareness (SA), reducing or eliminating Controlled Flight into Terrain (CFIT), as well as Low-Visibility Loss of Control (LVLOC) accidents. In addition to substantial safety benefits, SVS displays have many potential operational benefits that can lead to flight in instrument meteorological conditions (IMC) resembling those conducted in visual meteorological conditions (VMC). Potential benefits could include lower landing minimums, more approach options, reduced training time, etc. SVS conducted research will develop display concepts providing the pilot with an unobstructed view of the outside terrain, regardless of weather conditions and time of day. A critical component of SVS displays is the appropriate presentation of terrain to the pilot. The relationship between the realism of the terrain presentation and resulting enhancements of pilot SA and pilot performance has been largely undefined. Comprised of coordinated simulation and flight test efforts, the terrain portrayal for head-down displays (TP-HDD) test series examined the effects of two primary elements of terrain portrayal: variations of digital elevation model (DEM) resolution and terrain texturing. Variations in DEM resolution ranged from sparsely spaced (30 arc-sec/2,953ft) to very closely spaced data (1 arc-sec/98 ft). Variations in texture involved three primary methods: constant color, elevation-based generic, and photo-realistic, along with a secondary depth cue enhancer in the form of a fishnet grid overlay. The TP-HDD test series was designed to provide comprehensive data to enable design trades to optimize all SVS applications, as well as develop requirements and recommendations to facilitate the implementation and certification of SVS displays. The TP-HDD flight experiment utilized the NASA LaRC Cessna 206 Stationaire and evaluated eight terrain portrayal concepts in an effort to confirm and extend results from the previously conducted TP-HDD simulation experiment. A total of 15 evaluation pilots, of various qualifications, accumulated over 75 hours of dedicated research flight time at Newport News (PHF) and Roanoke (ROA), VA, airports from August through October, 2002. This report will present results from the portion of testing conducted at Roanoke, VA.

Hughes, Monica F.↗

NASA NTRS2014

Vision Based Autonomous Robotic Control for Advanced Inspection and Repair

The advanced inspection system is an autonomous control and analysis system that improves the inspection and remediation operations for ground and surface systems. It uses optical imaging technology with intelligent computer vision algorithms to analyze physical features of the real-world environment to make decisions and learn from experience. The advanced inspection system plans to control a robotic manipulator arm, an unmanned ground vehicle and cameras remotely, automatically and autonomously. There are many computer vision, image processing and machine learning techniques available as open source for using vision as a sensory feedback in decision-making and autonomous robotic movement. My responsibilities for the advanced inspection system are to create a software architecture that integrates and provides a framework for all the different subsystem components; identify open-source algorithms and techniques; and integrate robot hardware.

sensory feedback↗

NASA NTRS1993

Perceptual organization in computer vision - A review and a proposal for a classificatory structure

The evolution of perceptual organization in biological vision, and its necessity in advanced computer vision systems, arises from the characteristic that perception, the extraction of meaning from sensory input, is an intelligent process. This is particularly so for high order organisms and, analogically, for more sophisticated computational models. The role of perceptual organization in computer vision systems is explored. This is done from four vantage points. First, a brief history of perceptual organization research in both humans and computer vision is offered. Next, a classificatory structure in which to cast perceptual organization research to clarify both the nomenclature and the relationships among the many contributions is proposed. Thirdly, the perceptual organization work in computer vision in the context of this classificatory structure is reviewed. Finally, the array of computational techniques applied to perceptual organization problems in computer vision is surveyed.

Sarkar, Sudeep↗

DOE OSTI2024

A Method for Producing Hierarchical and Statistically Calibrated Predictions of Nuclear Material Properties from Existing Models

Computer vision-based analysis of micrographs of nuclear materials is an emerging technique for property prediction, synthetic route identification, and other material analysis tasks. These analysis tasks play a pivotal role in many material characterization applications such as signature development for treaty verification, process optimization, etc. The backbone in many of the recent computer vision-based techniques is a deep learning model, which takes a fixed-size set of pixels and provides a class prediction for that set of pixels. For example, previous work developed a deep convolutional neural network (CNN) to predict the synthetic route from a 256 px x 256 px patch taken from a larger image of uranium ore concentrates. In this work, we present several methods for first calibrating these models in a manner that they can provide accurate probabilities of their predictions’ veracity, and several methods of combining these probabilities. Overall, the combination of these two steps into a pipeline allows for full-image and even full-sample (where a sample has many images) predictions with associated confidence values. Finally, we show that one can also use the patch predictions and confidence to produce a visualization to map predicted constituents through the image. Results and examples for predicting and mapping uranium ore concentrates’ synthetic process from imagery will be presented.

artificial intelligence↗

DOE OSTI2025

Artificial Intelligence-Enhanced CMIP6 Climate Projections Across the Conterminous United States

This dataset comprises high-resolution climate projections at 1/24 degree grid (~4km) over the conterminous United States (CONUS) based on ten Global Climate Models (GCMs) that are part of the Coupled Models Intercomparison Project phase 6 (CMIP6). The CMIP6 GCMs are downscaled using two artificial intelligence (AI) techniques, primarily based on the computer vision approach called super-resolution. We train two separate networks: super-resolution convolutional neural network (SRCNN) and super-resolution generative adversarial network (SRGAN). The networks are trained using Daymet observations, originally available at a 1 km resolution. For training purposes, the Daymet data is interpolated to 1/24 degree (~4km), 0.25 degree and 1 degree, which serve as high, intermediate and low-resolution inputs respectively. For each of the SRCNN and SRGAN network, we use a two-step resolution enhancement, the first step generates 4x refinement from 1 degree to 0.25 degree and the second step generates 6x refinement from 0.25 degree to 1/24 degree (~4km). We downscale daily scale precipitation, maximum temperature and minimum temperature for the six CMIP6 GCMs for 1980 to 2019 in the historical period and 2020 to 2059 in the near-term future under the shared socioeconomic pathway 585 and 245 (SSP585 and SSP245) emission scenarios. We also perform double bias-correction with Daymet observations using a quantile mapping approach, first for GCMs prior to making predictions at 1 degree grid and second after making final predictions at ~4km.

13 HYDRO ENERGY↗

DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

MicroBooNE, a Liquid Argon Time Projection Chamber (LArTPC) located in the $\nu_{\mu}$-dominated Booster Neutrino Beam at Fermilab, has been studying $\nu_{e}$ charged-current (CC) interaction rates to shed light on the MiniBooNE low energy excess. The LArTPC technology employed by MicroBooNE provides the capability to image neutrino interactions with mm-scale precision. Computer vision and other machine learning techniques are promising tools for image processing that could boost efficiencies for selecting $\nu_{e}$-CC and other rare signals, reduce cosmic and beam-induced backgrounds, and improve the reconstruction of neutrino energies. The MicroBooNE experiment has been at the forefront of developing and testing such techniques for use in physics analyses. In this poster we overview deep-learning based reconstruction methods. We will showcase the use of a recurrent neural network to estimate neutrino energies and present a new reconstruction framework that uses convolutional neural networks to locate neutrino interaction vertices, tag pixels with track and shower labels, and perform particle identification on reconstructed clusters. We will present studies characterizing the performance of these new tools and demonstrate their effectiveness through their use in an inclusive $\nu_{e}$-CC event selection.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRSDate not supplied

Developing Methods for Exercise System Kinematics Tracking

BACKGROUND How to quantify the load and forces produced by exercise equipment and their Vibration Isolation and Stabilization (VIS) platforms in-flight is an active area of investigation. Kinematic tracking paired with system modeling can provide insights as well as verification and validation of simulations used for system design and development. Traditional motion capture methods can require significant cost in equipment procurement and crew-time, but newer lessons learned can be leveraged [1]. The VIS systems of current and future exercise hardware on the International Space Station (ISS) such as the Cycle Ergometer with Vibration Isolation System (CEVIS) and the European Enhanced Exploration Exercise Device (E4D) are not currently outfitted with IMUs or similar measurement devices. Video-based methods would enable use of multi-purpose, crew-familiar flight equipment. An initial exploration of video-based solutions was performed utilizing 2-camera video from crew cycling on Teal-CEVIS on the ISS. METHODS AND RESULTS Our group has scoped a variety of video-based object tracking methods. To date, we have primarily investigated computer vision toolkits such as openCV. Techniques explored include key-point detection, background subtraction, region-of interest tracking, color-based tracking, tag masking and tracking, and corner detection. Although object-tracking and 6D pose estimation is a rich field, space applications are a unique problem that are challenging for existing software and toolkits. The majority of the existing object-tracking applications involve vehicles/pedestrians and household objects with simple backgrounds. We have identified the following features which pose particular challenges for on-station exercise equipment tracking: Busy and visually cluttered background Low-textured tracking object with relatively small motions Occlusions and motion by human subject and loose, floating objects Limited number of video cameras with no fixed global references Limited ability to add tags, markers, or visual references to the tracking object Lack of training data for Machine Learning (ML) algorithms CONCLUSION We will summarize the efficacy of techniques tested for a ground mock-trial and the on-station exercise trial. It is likely that human-in-loop feedback or a conglomerate of methods is required. ML-based methods, like those implemented for human body tracking [2], may still be a viable option, but more training data and validation is needed.

L B Nilsson↗

DOE OSTI2024

Identifying Neutrino Final States and Energies in MicroBooNE with New Deep-Learning Based LArTPC Reconstruction Frameworks

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NASA NTRSDate not supplied

Automated Registration of Multi-Mode Nondestructive Evaluation Data

Registration techniques play a central role in applications of image processing to computer vision, medical imaging, and automatic target tracking. Feature-based techniques such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF) are commonly used to register images derived from a single modality. However, SIFT and SURF struggle to register images from different modalities because the features tend to manifest rather differently and at sometimes very different length-scales. The most successful methods that have been developed to register multi-modal data use information-theoretic approaches. These methods play a key part in nondestructive evaluation scenarios where data that is collected by sensors of different modalities must be registered to be fused. In this paper, automated registration based on normalized mutual information is applied to align data derived from ultrasonic and radiographic inspections of (i) additively manufactured titanium alloy test coupons, and (ii) thin, lithium metal pouch-cell batteries. The quality of the registration is quantified in terms of computational resources and spatial accuracy. In the first case the X-ray computed tomography (XCT) data is captured on a region corresponding to a small subset of the ultrasonic data, while in the case of the lithium batteries the digital radiography (DR) captures a larger region of interest than the ultrasonic data. In both cases the radiographic data resolution is much higher than for ultrasound, but interestingly, in both cases the accuracy of the registration is approximately equal to two-to-three-pixel lengths in the ultrasonic images.

Nondestructive Evaluation↗

NASA NTRS1989

Load balancing and task decomposition techniques for parallel implementation of integrated vision systems algorithms

Several techniques are presented to perform static and dynamic load balancing schemes for integrated vision systems. These techniques are novel in the sense that they capture the computational requirements of a task by examining the data when they are produced. Furthermore, they can be applied to many integrated vision systems because many algorithms in different systems are either the same or have similar computational characteristics. These techniques are evaluated by applying them to the algorithms in a motion estimation system. It is shown that the performance gains when these techniques are used are significant and the overhead of using these techniques is minimal. The performance is evaluated by implementing the algorithms using the presented techniques on a hypercube multiprocessor system.

Choudhary, Alok N.↗

NASA NTRS2020

Deep Learning Models for Planetary Seismicity Detection

Research in planetary seismology is fundamentally constrained by a lack of data. Seismo-logical science products of future missions can typically only be informed by theoretical signal/noise characteristics of the environment or likely Earth-analogues. Although objectives can be re-assessed after some initial data-collection upon lander arrival, transfer of high-resolution data back to Earth is costly on lander power usage. Over the last several years, development of GPU computing techniques and open-source high-level APIs have led to rapid advances in deep learning within the fields of computer vision, natural language processing, and collaborative filtering. These techniques are actively being adapted in seismology for a variety of tasks, including: earthquake detection, seismic phase discrimination, and ground-motion prediction. Until the recent detection of mars quakes during the Mars InSight mission, the only other measurements of seismicity recorded outside of Earth was on the Moon during the Apollo missions between 1969 to 1977. These unique data sets have been periodically revisited using new seismological methods, including ambient noise interferometry and Hidden Markov Models. Our objective is to develop a deep learning seismic detector and use it to catalog moonquakes from the Apollo 17 Lunar Seismic Profiling Experiment (LSPE) and compare the results with those obtained by other methods. Additionally, we will assess the accuracy tradeoff between using a training set of lunar data and one composed of Earth seismicity. In this document, we present preliminary results using a prototype classifier trained on a small set of earthquakes that was able to obtain detections for LSPE moonquakes with a greater accuracy than a recent study using Hidden Markov Models.

Civilini, F.↗

NASA NTRS2022

Velocimeter LIDAR Based Bulk Velocity Estimation for Terrain Relative Navigation Applications

A novel batch state estimation approach to estimate the translational and angular velocity states of a vehicle using a state-of-the-art velocimeter Light Detection and Ranging (LIDAR) sensor for use in Terrain and Hazard Relative Navigation (TRN/HRN) applications is presented in this paper. The velocimeter LIDAR is capable of measuring three dimensional position and line-of-sight (LOS) velocity associated with every pixel in the field-of-view (FOV). This new batch state estimation methodology is shown to provide accurate and statistically consistent velocity estimates with no a priori information. In contrast to traditional computer vision approaches for TRN, the proposed technique is not dependent on image features. This feature alleviates the need for accurate feature detection and correspondence to a predefined map making it suitable for unknown operating environments. Following a detailed development of the mathematical details associated with the batch state estimation methodology, the efficacy and utility of the proposed algorithms are evaluated through emulation robotics experiments performed at Texas A&M’s Land, Air, and Space Robotics (LASR) laboratory.

Skulsky, Eli↗

NASA NTRSDate not supplied

Developing Methods for Exercise System Kinematic Tracking

BACKGROUND How to quantify the load and forces produced by exercise equipment and their Vibration Isolation and Stabilization (VIS) platforms in-flight is an active area of investigation. Kinematic tracking paired with system modeling can provide insights as well as verification and validation of simulations used for system design and development. Traditional motion capture methods can require significant cost in equipment procurement and crew-time, but newer lessons learned can be leveraged [1]. The VIS systems of current and future exercise hardware on the International Space Station (ISS) such as the Cycle Ergometer with Vibration Isolation System (CEVIS) and the European Enhanced Exploration Exercise Device (E4D) are not currently outfitted with IMUs or similar measurement devices. Video-based methods would enable use of multi-purpose, crew-familiar flight equipment. An initial exploration of video-based solutions was performed utilizing 2-camera video from crew cycling on Teal-CEVIS on the ISS. METHODS AND RESULTS Our group has scoped a variety of video-based object tracking methods. To date, we have primarily investigated computer vision toolkits such as open CV. Techniques explored include key-point detection, background subtraction, region-of interest tracking, color-based tracking, tag masking and tracking, and corner detection. Although object-tracking and 6D pose estimation is a rich field, space applications are a unique problem that are challenging for existing software and toolkits. The majority of the existing object-tracking applications involve vehicles/pedestrians and household objects with simple backgrounds. We have identified the following features which pose particular challenges for on-station exercise equipment tracking: 1. Busy and visually cluttered background 2. Low-textured tracking object with relatively small motions 3. Occlusions and motion by human subject and loose, floating objects 4. Limited number of video cameras with no fixed global references 5. Limited ability to add tags, markers, or visual references to the tracking object 6. Lack of training data for Machine Learning (ML) algorithms CONCLUSION We will summarize the efficacy of techniques tested for a ground mock-trial and the on-station exercise trial. It is likely that human-in-loop feedback or a conglomerate of methods is required. ML-based methods, like those implemented for human body tracking [2], may still be a viable option, but more training data and validation is needed.

L Nilsson↗