Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computer vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Fast Solution in Sparse LDA for Binary Classification

An algorithm that performs sparse linear discriminant analysis (Sparse-LDA) finds near-optimal solutions in far less time than the prior art when specialized to binary classification (of 2 classes). Sparse-LDA is a type of feature- or variable- selection problem with numerous applications in statistics, machine learning, computer vision, computational finance, operations research, and bio-informatics. Because of its combinatorial nature, feature- or variable-selection problems are NP-hard or computationally intractable in cases involving more than 30 variables or features. Therefore, one typically seeks approximate solutions by means of greedy search algorithms. The prior Sparse-LDA algorithm was a greedy algorithm that considered the best variable or feature to add/ delete to/ from its subsets in order to maximally discriminate between multiple classes of data. The present algorithm is designed for the special but prevalent case of 2-class or binary classification (e.g. 1 vs. 0, functioning vs. malfunctioning, or change versus no change). The present algorithm provides near-optimal solutions on large real-world datasets having hundreds or even thousands of variables or features (e.g. selecting the fewest wavelength bands in a hyperspectral sensor to do terrain classification) and does so in typical computation times of minutes as compared to days or weeks as taken by the prior art. Sparse LDA requires solving generalized eigenvalue problems for a large number of variable subsets (represented by the submatrices of the input within-class and between-class covariance matrices). In the general (fullrank) case, the amount of computation scales at least cubically with the number of variables and thus the size of the problems that can be solved is limited accordingly. However, in binary classification, the principal eigenvalues can be found using a special analytic formula, without resorting to costly iterative techniques. The present algorithm exploits this analytic form along with the inherent sequential nature of greedy search itself. Together this enables the use of highly-efficient partitioned-matrix-inverse techniques that result in large speedups of computation in both the forward-selection and backward-elimination stages of greedy algorithms in general.

Moghaddam, Baback↗

Parallel Architectures and Parallel Algorithms for Integrated Vision Systems

Computer vision is regarded as one of the most complex and computationally intensive problems. An integrated vision system (IVS) is a system that uses vision algorithms from all levels of processing to perform for a high level application (e.g., object recognition). An IVS normally involves algorithms from low level, intermediate level, and high level vision. Designing parallel architectures for vision systems is of tremendous interest to researchers. Several issues are addressed in parallel architectures and parallel algorithms for integrated vision systems.

Choudhary, Alok Nidhi↗

Location generalizability of image-based air quality models

This paper is to be submitted at the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Computer Vision for Earth Observation workshop. The full paper abstract is below: The ability to rapidly quantify atmospheric pollutants is important both for global emissions monitoring and for mitigating the adverse effects that follow a hazardous chemical release. In the aftermath of a chemical release, imagery is often the only available resource to assess local conditions. Recent work has demonstrated initial success in predicting particulate matter pollution from imagery; however, these results are tied to a specific site and do not generalize to new geographic locations. In this work, we seek to understand how easily deep learning models generalize to new locations in the context of image-based air quality assessments, targeting two distinct tasks: (1) broad measures of particulate matter pollution, and (2) the mass of a given chemical released in hazardous plumes. For the latter, we focus on sulfur dioxide, a toxic aerosol and a major component of particulate matter pollution caused by industrial fossil fuel consumption. To develop a model that operates in the widest possible range of environments, we test different training strategies, including the use of new geolocation foundation models. The best performing models achieve >80% accuracy when evaluating unseen imagery at previously seen sites, but we find significant drops in performance when evaluating imagery from unseen sites, at best 65%. Additionally, we present the public release of the National Parks Air Quality Index Dataset, a new medium-sized dataset that pairs imagery with sensor-based air quality measurements at 15 different national parks.

Byler, Eleanor B. [BATTELLE (PACIFIC NW LAB)]↗

Computer Vision-Enabled Smart Traffic Monitoring for Sustainable Transportation Management

Transportation accounts for a significant portion of total global energy consumption. Excessive energy consumption usually occurs in urban traffic environments with congestion and travel delays. With the advancement of remote sensing and computer vision technologies, real-time traffic conditions can be monitored. Therefore, sustainable transportation management strategies can be developed to optimize the overall energy and environment performance and reduce congestion and emissions. This work presents a smart traffic monitoring system based on remote camera sensors. Real-time and historical traffic conditions at the US Department of Energy’s Oak Ridge National Laboratory (ORNL) were monitored and analyzed to develop optimal transportation management strategies for sustainability. Computer vision algorithms were developed and applied to process the real-time camera data to obtain complete traffic information across the ORNL campus. Weeks of historical data were collected and processed to analyze the traffic and identify bottlenecks. The proposed traffic monitoring and management approach can be applied and extended to benefit other campuses or urban areas.

Shao, Yunli↗

Data Validation Experiments with a Computer-Generated Imagery Dataset for International Nuclear Safeguards

Computer vision models have great potential as tools for international nuclear safeguards verification activities, but off-the-shelf models require fine-tuning through transfer learning to detect relevant objects. Because open-source examples of safeguards-relevant objects are rare, and to evaluate the potential of synthetic training data for computer vision, we present the Limbo dataset. Limbo includes both real and computer-generated images of uranium hexafluoride containers for training computer vision models. Here, we generated these images iteratively based on results from data validation experiments that are detailed here. The findings from these experiments are applicable both for the safeguards community and the broader community of computer vision research using synthetic data.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Crowdsourced reliable labeling of safety-rule violations on images of complex construction scenes for advanced vision-based workplace safety

Construction workplace hazard detection requires engineers to analyze scenes manually against many safety rules, which is time-consuming, labor-intensive, and error-prone. Computer vision algorithms are yet to achieve reliable discrimination of anomalous and benign object relations underpinning safety violation detections. Recently developed deep learning-based computer vision algorithms need tens of thousands of images, including labels of the safety rules violated, in order to train deep-learning networks for acquiring spatiotemporal reasoning capacity in complex workplaces. Such training processes need human experts to label images and indicate whether the relationship between the worker, resource, and equipment in the scenes violate spatiotemporal arrangement rules for safe and productive operations. False alarms in those manual labels (labeling no-violation images as having violations) can significantly mislead the machine learning process and result in computer vision models that produce inaccurate hazard detections. Compared with false alarms, another type of mislabels, false negatives (labeling images having violations as “no violations”), seem to have fewer impacts on the reliability of the trained computer vision models.

97 MATHEMATICS AND COMPUTING↗

Remote Sensor Design for Visual Recognition with Convolutional Neural Networks

While deep learning technologies for computer vision have developed rapidly since 2012, modeling of remote sensing systems has remained focused around human vision. In particular, remote sensing systems are usually constructed to optimize sensing cost-quality trade-offs with respect to human image interpretability. While some recent studies have explored remote sensing system design as a function of simple computer vision algorithm performance, there has been little work relating this design to the state-of-the-art in computer vision: deep learning with convolutional neural networks. We develop experimental systems to conduct this analysis, showing results with modern deep learning algorithms and recent overhead image data. Our results are compared to standard image quality measurements based on human visual perception, and we conclude not only that machine and human interpretability differ significantly, but that computer vision performance is largely self-consistent across a range of disparate conditions. This repository contains all the code required to replicate the results of our research paper, and the corresponding docker environment in which that code can be executed. All code is written in Python3 and utilizes the PyTorch library (v1.0) for neural network model training and evaluation. Jupyter notebooks are used to visualize images transformed with our code, in addition to plotting experimental results. We also include the specific parameter files used to execute our experiments, with the intent that these experiments can be replicated.

Zelinski, michaelE.↗

Sensor Control of Robot Arc Welding

The potential for using computer vision as sensory feedback for robot gas-tungsten arc welding is investigated. The basic parameters that must be controlled while directing the movement of an arc welding torch are defined. The actions of a human welder are examined to aid in determining the sensory information that would permit a robot to make reproducible high strength welds. Special constraints imposed by both robot hardware and software are considered. Several sensory modalities that would potentially improve weld quality are examined. Special emphasis is directed to the use of computer vision for controlling gas-tungsten arc welding. Vendors of available automated seam tracking arc welding systems and of computer vision systems are surveyed. An assessment is made of the state of the art and the problems that must be solved in order to apply computer vision to robot controlled arc welding on the Space Shuttle Main Engine.

Sias, F. R., Jr.↗

Automated Assembly Progress Monitoring in Modular Construction Factories Using Computer Vision-Based Instance Segmentation

Modular construction has recently gained interest as a transformative construction method. In this method, a large portion of the construction is performed inside factories, where processes are fast-paced and interdependent; therefore, any deviation from the schedule can delay the production. Such deviations are frequent in modular factories due to the labor-intensive nature of the tasks. This propagation of delays can be mitigated by continuously monitoring each process; however, current manual monitoring methods are laborious, and recently proposed contact sensor-based methods are intrusive to the work. In addition, recent computer vision-based monitoring methods inside factories are limited to detection algorithms that fail to provide the pixel-level accuracy required for assembly progress monitoring in highly occluded factory scenes, and they require a large number of manual annotations. Therefore, this paper proposes a method to monitor the installation of subassemblies in modular construction factories using mask R-CNN instance segmentation and improves the data efficiency of the model using a copy-paste augmentation method. This method was validated on the CCTV videos captured from a modular construction factory in the US, resulting in a 9% mAP improvement in segmentation.

computer vision↗

Uncrewed Aerial Systems for Emergency Medical First Response: A Market Research Report

This report presents the findings from market research conducted for NASA’s Aerial Aid Convergent Aeronautics Solutions (CAS) exploration project, which aims to assess the current state of the market and technological readiness for Uncrewed Aerial Systems (UAS) for medical emergency first response. The research reveals a robust and rapidly growing market for UAS, with a notable emerging sector for Drones as First Responders (DFR). Despite this growth, DFR applications are currently limited by regulatory, technical, and other challenges, which restrict their use primarily to manned remote video surveillance, and therefore are primarily employed by police units. To our knowledge, there is no evidence of UAS being utilized by medical first responders for scene assessment. Limited evidence exists for closely related applications; however, these are mostly confined to pilot programs for the delivery of medical supplies or equipment. Although there has been discussion around fully autonomous DFR applications for medical purposes such as UAS ambulances or patient transport drones, these applications are generally not yet operational in practice. The technology for full autonomy, especially in guidance and control, has seen significant advancements, and recent Federal Aviation Administration (FAA)regulations are likely to accelerate adoption. Computer vision algorithms for fully autonomous medical emergency response scene surveillance are primed for advancement and deployment. A notable gap likely exists between advancements in computer vision research and what is being integrated in the commercial DFR sector. This gap is primarily due to challenges such as quality assurance for autonomous systems, the availability of application-specific training datasets for computer vision algorithms, regulatory constraints, and public perception and privacy concerns.

Joshua M Fody↗

Intelligent Process Visualization through Nuclear Operation Process Modeling, Reasoning, and Object Detection from Field Videos (Final Report)

This report is a deliverable for the “Final Report” task of DOE NEET Project 19-16790, "Context-Aware Safety Information Display for Nuclear Field Workers." This project's overall goal is to test the hypothesis that integrating computer vision and process reasoning methods will enable proactive visualization of the safe operation and maintenance processes of Nuclear Power Plants (NPP) for field workers. Augmented Reality (AR) glasses adopting such proactive safety information visualization techniques can significantly increase personnel safety and reduce the NPP’s operating costs. The current practice of monitoring NPPs requires workers to switch between digital models, data, and physical workspaces in identifying relevant but potentially occluded objects and in assessing the risks of operation and maintenance processes. On the other hand, frequently changed field conditions require field workers to report to supervisors for real-time guidance. Such guidance is essential to ensure that changing conditions will not invalidate or endanger the work order and other ongoing processes that may jeopardize NPP operations. Additionally, incorrect recognition of equipment objects can result in communication errors and safety problems. AR techniques can assist engineers in viewing the physical workspaces with objects labeled with detailed operation procedures and safety reminders during field operations. The project team developed an “Intelligent Context-Aware Safety Information Display” (ICAD) for supporting Nuclear Power Plant (NPP) field workers in achieving safe and efficient execution of a series of operational tasks in uncertain and changing workspaces of an NPP. Before designing the ICAD-AR prototype, the project team synthesized NPP operational knowledge models through literature review studies, surveys, interviews with domain experts, and knowledge modeling. The project team conducted an extensive study of the operational procedures of various NPPs, and digital technologies that can support the safe and efficient execution of those procedures in different NPP operational contexts. This literature review helped the project team conduct surveys and interviews with nuclear engineers and field workers to identify three categories of information. The NPP knowledge modeling efforts reveal that the three categories of information identified have different levels of importance in a typical procedure of carrying out a series of tasks to achieve a specific NPP operation goal (e.g., shutdown, mode changes). These three categories of information include 1) Workspace dynamics – the changing spatial arrangements of workspaces, tools, protection equipment, and supporting materials, 2) Workflow prognostics – the dynamic dependencies between different parts of an NPP that functionally support and influence each other in terms of safety and efficiency, and 3) Hazards – objects and spaces that contain hazardous materials or physical conditions that can pose risks to workers or mechanical systems. The project team has profiled the importance levels of these categories of information into a knowledge model. This knowledge model specifies what types of information are more critical for a given task in a given workspace so that computers can automatically identify critical objects and sensors in a scene for delivering context-ware safety information to field workers through AR devices. Significant research development of this project results in technical research outcomes and a prototyping system that illustrates the technical feasibility of establishing an ICAD-AR system supporting the proactive safety information display for nuclear field workers. This final report summarizes the project team’s technological achievements in the past three years. Overall, the project team completed the development and integration of five techniques into a prototype ICAD Augmented Reality (ICAD-AR) system and demonstrated the developed system’s real-time execution in a mechanical room. The project team completed the analysis of using this prototype in other types of workspaces based on 3D image data and digital design models collected from two additional workspaces (a water treatment plant and a flow loop training facility). The integrated techniques include 1) Natural Language Processing (NLP) algorithms supporting the generation and updates of nuclear fieldwork process models based on text analysis of work packages and operation manuals; 2) sensor log analysis for predicting control actions in given sensor reading contexts; 3) computer vision algorithms for automatic localization and navigation of workers; 4) object detection algorithms for identifying task-related objects and correlated sensors for safety checking; 5) AR technique as a platform for supporting the integration. The testing results of these five techniques have shown that 1) the sensor log analysis model can predict the next control action with an accuracy of 0.883; 2) the trained natural language processing model can extract more than 80% of the critical information from paper-based procedures (PBPs); 3) the navigation algorithm with the integration of Visual Inertial Odometry (VIO) and Non-Recursive Bayesian Filter methods make operator’s trajectory estimation resilient to drift error; 4) the computer vision algorithm can detect task-specific and safety-critical objects with an average accuracy of 95.3%. The project team used work procedures collected from a flow loop training facility and two datasets collected from two mechanical rooms simulating the workspaces of NPPs to demonstrate the technical capabilities of the developed ICAD-AR prototype. The demonstration validated the technical feasibility of establishing the ICAD-AR system for nuclear field workers and identified the challenges in 1) automatic text analysis of work packages; 2) use of limited samples of sensor logs for predicting the proper timings of control actions; 3) reliably tracking workers and their task progress in mechanical rooms with many similar objects.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Satellite Embedding-Based Population Imputation for Areas with Missing Building Footprint Data: A Computer Vision-Based Approach

High-resolution population modeling is important for supporting effective decision-making across diverse sectors. LandScan Mosaic generates population estimates at the level of individual buildings and aggregates them to 3 arc-second grids, and this approach performs well in regions where building footprint data are comprehensive and reliable. However, large portions of the globe still suffer from incomplete, sparse, or entirely missing building stock datasets, creating a structural limitation for strictly building-based population models. To address this research gap, this study proposes a computer vision-based framework that employs Google Earth Engine satellite embeddings and UNet, which allows us to directly impute grid-level population estimates in building-data-deficient areas. Applied to Taiwan as a case study, the framework achieved strong predictive performance with R$^{2}$ of 0.89, RMSE of 18.70, and MAE of 8.41, outperforming traditional machine learning approaches. Notably, the proposed framework effectively addressed building false-positive errors inherent in Global Human Settlement Layer (GHSL) data, correctly identifying uninhabited areas that were erroneously classified as populated. The framework also offers significant advantages for global population mapping, particularly in terms of scalability and temporal consistency, thereby extending the coverage and accuracy of high-resolution population products in data-scarce regions worldwide. Urban planners, decision makers, and related stakeholders can obtain granular population distributions to support more accurate and targeted infrastructure investment, service delivery, resource allocation, and risk assessment decisions.

97 MATHEMATICS AND COMPUTING↗

A framework of computer vision-enhanced microfluidic approach for automated assessment of the transient sickling kinetics in sickle red blood cells

The occurrence of vaso-occlusive crisis greatly depends on the competition between the sickling delay time and the transit time of individual sickle cells, i.e., red blood cells from sickle cell disease (SCD) patients, while they are traversing the circulatory system. Many drugs for treating SCD work by inhibiting the polymerization of sickle hemoglobin (HbS), effectively delaying the sickling process in sickle cells (SS RBCs). Most previous studies on screening anti-sickling drugs, such as voxelotor, rely on in vitro testing of sickling characteristics, often conducted under prolonged deoxygenation for up to 1 hour. However, since the microcirculation of RBCs typically takes less than 1 minute, the results of these studies may be less accurate and less relevant for in vitro-in vivo correlation. In our current study, we introduce a computer vision-enhanced microfluidic framework designed to automatically capture the transient sickling kinetics of SS RBCs within a 1-min timeframe. Our study has successfully detected differences in the transient sickling kinetics between vehicle control and voxelotor-treated SS RBCs. This approach has the potential for broader applications in screening anti-sickling therapies.

59 BASIC BIOLOGICAL SCIENCES↗

Recent Advances in Soft Matter Characterization Capabilities Developed at NASA GRC

In 1991, famous French scientist Pierre-Gilles de Genes was awarded Nobel prize for his impactful research in soft matter, more specifically polymers. He is defined as the founding father of soft matter. In his Nobel lecture (https://www.nobelprize.org/uploads/2018/06/gennes-lecture.pdf ) he described soft matter aka complex fluids as materials with two primary features – (a) complexity and (b) flexibility. The sub-categories of soft matter (e.g.- granular materials, polymers, foams, colloids etc.) are defined on the basis of Pierre-Gilles de Gennes’ definition. At NASA GRC, we are pushing the boundaries for fundamental study of soft matter on Lunar Surface. With regard to Lunar surface science, we are focusing on developing capabilities pertaining to granular materials and bio-soft/active matter to facilitate future efforts in ISRU and bio-ISRU capabilities. In order to achieve fundamental goals of soft matter research within the limitations of Lunar environment, the scientific capabilities need to be small, flexible, modular, off the shelf and the focus needs to be more on developing an interdisciplinary capability that leverages the recent growth in AI/ML and Computer Vision to augment our understanding of fundamental science. This strategy would allow us to reduce our resource requirement during launch, installation, and occupied real estate footprint on Lunar surface In this talk, we will go over 3 different capabilities that we have developed in house and in close collaboration – (a) Differential Dynamic Microscopy (DDM), (b) Portable In-situ Chemical Spectroscopy (PICS) and (c) Computer Vision Enabled Observation. At very high level, Differential Dynamic Microscopy (DDM) allows us to study the structure-property-process relation (microrheology) of bio-soft/active matter using optical microscope and improved image analysis capabilities. PICS uses AI/ML-based advanced signal deconvolution and analysis technique that can work with existing portable spectroscopy tools to perform materials analysis (e.g.- granular materials and bio-soft/active matter) inspection on the go. Finally, computer vision enabled analysis allows us to use simple camera images for 3D reconstruction of experimental process and tracking of objects of interest in an experiment. We expect that this detailed process will allow us reach a thorough understanding of soft matter in Lunar environment. The capabilities developed by us will help to validate and establish fundamental understanding in Lunar environment. This will, in turn, allow us to guide future space exploration missions and expand the knowledge base of the scientific and engineering communities.

Suman Sinha Ray↗

Ultrafast processing of pixel detector data with machine learning frameworks

Modern photon science performed at high repetition rate free-electron laser (FEL) facilities and beyond relies on 2D pixel detectors operating at increasing frequencies (towards 100 kHz at LCLS-II) and producing rapidly increasing amounts of data (towards TB/s). This data must be rapidly stored for offline analysis and summarized in real time for online feedback to the scientists. While at LCLS all raw data has been stored, at LCLS-II this would lead to a prohibitive cost; instead, enabling real time processing of pixel detector data (dark, gain, common mode, background, charge summing, subpixel position, photon counting, data summarization) allows reducing the size and cost of online processing, offline processing and storage by orders of magnitude while preserving full photon information. This could be achieved by taking advantage of the compressibility of sparse data typical for LCLS-II applications. Faced with a similar big data challenge a decade ago, computer vision stimulated revolutionary advances in machine learning hardware and software. We investigated if these developments are useful in the field of data processing for high speed pixel detectors and found that typical deep learning models and autoencoder architectures failed to yield useful noise reduction while preserving full photon information, presumably because of the very different statistics and feature sets in computer vision and radiation imaging. However, the raw performance of modern frameworks like Tensorflow inspired us to redesign in Tensorflow mathematically equivalent versions of the state-of-the-art, “classical” algorithms used at LCLS. The novel Tensorflow models resulted in elegant, compact and hardware agnostic code, gaining 1 to 2 orders of magnitude faster processing on an inexpensive consumer GPU, reducing by 3 orders of magnitude the projected cost of online analysis and compression without photon loss at LCLS-II. The novel Tensorflow models also enabled ongoing development of a pipelined hardware system expected to yield an additional 3 to 4 orders of magnitude speedup, necessary for meeting the data acquisition and storage requirements at LCLS-II, potentially enabling acquiring every single FEL pulse at full speed. Computer vision a decade ago was dominated by hand-crafted filters; their structure inspired the deep learning revolution resulting in modern deep convolutional networks; similarly, our novel Tensorflow filters provide inspiration for designing future deep learning models for ultrafast and efficient processing and classification of pixel detector images at FEL facilities.

47 OTHER INSTRUMENTATION↗

Computational approaches to vision

Vision is examined in terms of a computational process, and the competence, structure, and control of computer vision systems are analyzed. Theoretical and experimental data on the formation of a computer vision system are discussed. Consideration is given to early vision, the recovery of intrinsic surface characteristics, higher levels of interpretation, and system integration and control. A computational visual processing model is proposed and its architecture and operation are described. Examples of state-of-the-art vision systems, which include some of the levels of representation and processing mechanisms, are presented.

Barrow, H. G.↗

Block segmentation in feature space for realtime object detection in high granularity images

Computer vision has applications in object detection, image recognition and classification, and object tracking. One of the challenges of computer vision is the presence of useful information at multiple distance scales. Filtering techniques may sacrifice details at small scales in order to prioritize the analysis of large-scale features of the image. We present a strategy for coarse-graining multidimensional data while maintaining fine-grained detail for subsequent analysis. The algorithm is based on fixed-size block segmentation in the feature space. We apply this strategy to solve the long-standing challenge of detecting particle trajectories at the Large Hadron Collider in real time.

Computer vision↗

Flight data acquisition methodology for validation of passive ranging algorithms for obstacle avoidance

The automation of low-altitude rotorcraft flight depends on the ability to detect, locate, and navigate around obstacles lying in the rotorcraft's intended flightpath. Computer vision techniques provide a passive method of obstacle detection and range estimation, for obstacle avoidance. Several algorithms based on computer vision methods have been developed for this purpose using laboratory data; however, further development and validation of candidate algorithms require data collected from rotorcraft flight. A data base containing low-altitude imagery augmented with the rotorcraft and sensor parameters required for passive range estimation is not readily available. Here, the emphasis is on the methodology used to develop such a data base from flight-test data consisting of imagery, rotorcraft and sensor parameters, and ground-truth range measurements. As part of the data preparation, a technique for obtaining the sensor calibration parameters is described. The data base will enable the further development of algorithms for computer vision-based obstacle detection and passive range estimation, as well as provide a benchmark for verification of range estimates against ground-truth measurements.

Smith, Phillip N.↗