Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computer vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Advancements in reflected target nonintrusive assessment (ReTNA) for large optical surface measurement

Reflected computer vision targets are a powerful tool for measurement of mirror surface shape, with several important advantages over traditional fringe deflectometry methods. This method was first presented in 2021 and has undergone significant improvement and demonstration since. We describe a new baseline system using reflected computer vision targets, and present results from a large-scale measurement campaign conducted on both commercial heliostats and test mirrors in the laboratory. Calibration of the measurement system with photogrammetry allows for accurate measurement without careful control of target shape or camera position. Overall, the results show that a baseline setup using this method achieves measurement uncertainties in the slope error root-mean-square less than ±0.11 milliradian due to a series of repeatability conditions, varying sample position, rotation, lighting, camera settings, and system rebuild and recalibration. We present a detailed description of the setup, the results generated by this measurement tool, repeated measurement results, and the strengths and limitations of this metrology system.

14 SOLAR ENERGY↗
DOE OSTI2024

Real World Use Case Evaluation of Radar Retro-reflectors for Autonomous Vehicle Lane Detection Applications

Lane detection plays a critical role in autonomous vehicles for safe and reliable navigation. Lane detection is traditionally accomplished using a camera sensor and computer vision processing. The downside of this traditional technique is that it can be computationally intensive when high quality images at a fast frame rate are used and has reliability issues from occlusion such as, glare, shadows, active road construction, and more. This study addresses these issues by exploring alternative methods for lane detection in specific scenarios caused from road construction-induced lane shift and sun glare. Specifically, a U-Net, a convolutional network used for image segmentation, camera-based lane detection method is compared with a radar-based approach using a new type of sensor previously unused in the autonomous vehicle space: radar retro-reflectors. This evaluation is performed using ground truth data, obtained by measuring the lane positions and transforming them into pixel coordinates. The performance of each method is assessed using the statistical R2 score, indicating the correlation between the detected lane lines and the ground truth. The results show that the U-Net camera-based method exhibits limitations in accurately detecting and aligning the lane lines, particularly in challenging scenarios. However, the radar-based lane detection method demonstrates a strong correlation with the ground truth which implies that the use of this sensor may improve current reliability issues from conventional camera lane detection approach. Furthermore, the study highlights the limitations of the U-Net model for camera lane detection, especially in scenarios with sun glare. This study shows that infrastructure-based radar retro-reflectors can improve autonomous vehicle lane detection reliability. The integration of different sensor modalities and the development of advanced computer vision algorithms are crucial for improving the accuracy, reliability, and energy efficiency of lane detection systems. Addressing these challenges contributes to the advancement of autonomous vehicles and the realization of safer and more efficient transportation systems.

Brown, Nicolas E.↗
DOE OSTI2014

Using computer-extracted image features for modeling of error-making patterns in detection of mammographic masses among radiology residents

Purpose: Mammography is the most widely accepted and utilized screening modality for early breast cancer detection. Providing high quality mammography education to radiology trainees is essential, since excellent interpretation skills are needed to ensure the highest benefit of screening mammography for patients. The authors have previously proposed a computer-aided education system based on trainee models. Those models relate human-assessed image characteristics to trainee error. In this study, the authors propose to build trainee models that utilize features automatically extracted from images using computer vision algorithms to predict likelihood of missing each mass by the trainee. This computer vision-based approach to trainee modeling will allow for automatically searching large databases of mammograms in order to identify challenging cases for each trainee. Methods: The authors’ algorithm for predicting the likelihood of missing a mass consists of three steps. First, a mammogram is segmented into air, pectoral muscle, fatty tissue, dense tissue, and mass using automated segmentation algorithms. Second, 43 features are extracted using computer vision algorithms for each abnormality identified by experts. Third, error-making models (classifiers) are applied to predict the likelihood of trainees missing the abnormality based on the extracted features. The models are developed individually for each trainee using his/her previous reading data. The authors evaluated the predictive performance of the proposed algorithm using data from a reader study in which 10 subjects (7 residents and 3 novices) and 3 experts read 100 mammographic cases. Receiver operating characteristic (ROC) methodology was applied for the evaluation. Results: The average area under the ROC curve (AUC) of the error-making models for the task of predicting which masses will be detected and which will be missed was 0.607 (95% CI,0.564-0.650). This value was statistically significantly different from 0.5 (p < 0.0001). For the 7 residents only, the AUC performance of the models was 0.590 (95% CI,0.537-0.642) and was also significantly higher than 0.5 (p = 0.0009). Therefore, generally the authors’ models were able to predict which masses were detected and which were missed better than chance. Conclusions: The authors proposed an algorithm that was able to predict which masses will be detected and which will be missed by each individual trainee. This confirms existence of error-making patterns in the detection of masses among radiology trainees. Furthermore, the proposed methodology will allow for the optimized selection of difficult cases for the trainees in an automatic and efficient manner.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Diffractive optical computing in free space

Abstract Structured optical materials create new computing paradigms using photons, with transformative impact on various fields, including machine learning, computer vision, imaging, telecommunications, and sensing. This Perspective sheds light on the potential of free-space optical systems based on engineered surfaces for advancing optical computing. Manipulating light in unprecedented ways, emerging structured surfaces enable all-optical implementation of various mathematical functions and machine learning tasks. Diffractive networks, in particular, bring deep-learning principles into the design and operation of free-space optical systems to create new functionalities. Metasurfaces consisting of deeply subwavelength units are achieving exotic optical responses that provide independent control over different properties of light and can bring major advances in computational throughput and data-transfer bandwidth of free-space optical processors. Unlike integrated photonics-based optoelectronic systems that demand preprocessed inputs, free-space optical processors have direct access to all the optical degrees of freedom that carry information about an input scene/object without needing digital recovery or preprocessing of information. To realize the full potential of free-space optical computing architectures, diffractive surfaces and metasurfaces need to advance symbiotically and co-evolve in their designs, 3D fabrication/integration, cascadability, and computing accuracy to serve the needs of next-generation machine vision, computational imaging, mathematical computing, and telecommunication technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Vision Transformers Explained Series

Since their introduction in 2017 with Attention is All You Need¹, transformers have established themselves as the state of the art for natural language processing (NLP). In 2021, An Image is Worth 16x16 Words successfully adapted transformers for computer vision tasks. Since then, numerous transformer-based architectures have been proposed for computer vision. This article walks through the Vision Transformer (ViT) as laid out in An Image is Worth 16x16 Words.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

PlantCV v2: Image analysis software for high-throughput plant phenotyping

Systems for collecting image data in conjunction with computer vision techniques are a powerful tool for increasing the temporal resolution at which plant phenotypes can be measured non-destructively. Computational tools that are flexible and extendable are needed to address the diversity of plant phenotyping problems. We previously described the Plant Computer Vision (PlantCV) software package, which is an image processing toolkit for plant phenotyping analysis. The goal of the PlantCV project is to develop a set of modular, reusable, and repurposable tools for plant image analysis that are open-source and community-developed. Here in this paper we present the details and rationale for major developments in the second major release of PlantCV. In addition to overall improvements in the organization of the PlantCV project, new functionality includes a set of new image processing and normalization tools, support for analyzing images that include multiple plants, leaf segmentation, landmark identification tools for morphometrics, and modules for machine learning.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

YOLO11 to SAM2 pipeline for feature extraction from nuclear test films

The response to the effects of nuclear detonations is supported by models that describe the evolution of the nuclear fireball and cloud and the associated transport of active debris. Validation of those descriptions relies on data from the nuclear test operations. Video records of those events offer a rich source of information that was exploited to a limited extent in historic analyses. Computer vision and machine learning techniques are powerful tools that can be used to increase the number of measurements that can be obtained from those films. In this work, we apply computer vision techniques to automatically track the temporal evolution of the nuclear fireball. In particular, we apply You Only Look Once 11 (YOLO11) and Segment Anything Model 2 (SAM2) in combination with minimal human intervention to digitized versions of the original nuclear test films. As part of the proposed workflow, the YOLO11 model is applied to films to determine bounding boxes for the fireball within each frame. These are then used as inputs to SAM2, which uses image segmentation to determine the fireball boundaries and their temporal evolution. We assess the accuracy of our approach by using it to determine the energy released during the Trinity nuclear test and comparing the results with previous analyses based on manual measurements.

Van Exel, Kimberly [ORNL] (ORCID:0009000877463894)↗
DOE OSTI2026

Pre-training Vision Models for the Classification of Alerts from Wide-field Time-domain Surveys

Modern wide-field time-domain surveys facilitate the study of transient, variable and moving phenomena by conducting image differencing and relaying alerts to their communities. Machine learning tools have been used on data from these surveys and their precursors for more than a decade, and convolutional neural networks (CNNs), which make predictions directly from input images, saw particularly broad adoption through the 2010s. Since then, continually rapid advances in computer vision have transformed the standard practices around using such models. It is now commonplace to use standardized architectures pre-trained on large corpora of everyday images (e.g., ImageNet). In contrast, time-domain astronomy studies still typically design custom CNN architectures and train them from scratch. Here, we explore the effects of adopting various pre-training regimens and standardized model architectures on the performance of alert classification. We find that the resulting models match or outperform a custom, specialized CNN like what is typically used for filtering alerts. Moreover, our results show that pre-training on galaxy images from Galaxy Zoo tends to yield better performance than pre-training on ImageNet or training from scratch. We observe that the design of standardized architectures are much better optimized than the custom CNN baseline, requiring significantly less time and memory for inference despite having more trainable parameters. On the eve of the Legacy Survey of Space and Time and other image-differencing surveys, these findings advocate for a paradigm shift in the creation of vision models for alerts, demonstrating that greater performance and efficiency, in time and in data, can be achieved by adopting the latest practices from the computer vision field.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Introducing SpaceNet 9 - Cross-Modal Satellite Imagery Registration for Natural Disaster Responses

Computer vision algorithms are increasingly leveraged to accelerate geospatial analysis for disaster response and recovery. As the diversity of remote sensing imagery grows with optical, SAR, and other modalities, a perquisite for analytics is cross-modal image registration. There is a high potential to harness computer vision for this pre-processing requirement toward enabling downstream analytics such as heterogeneous change detection, automated feature extraction, and data fusion. Advancement in these areas has the potential to simplify data wrangling tasks and further accelerate disaster response timelines. The SpaceNet 9 challenge (launching in mid-2024) focuses on addressing the cross-modal image registration problem and demonstrating the utility of such modules on earthquake impacted scenarios. This paper describes the motivation for the SpaceNet 9 and provides a first overview of the dataset, the baseline algorithm, and implications for seeking cross-modal image registration in Earth observation. Code is available at https://github.com/SpaceNetChallenge/SpaceNet9.

Hansch, Ronny↗
DOE OSTI2019

Robotic Non-destructive Test of Concrete Structures with GPR, Impact Echo and 3D Vision - 19540

Structural members come across with various internal defects that cannot be evaluated visually. Instead, hand-held nondestructive sensors are used to detect such defects. In many situations, the hand-held NDT is not possible in high-risk structures because of limited access or safety issues. Therefore, Robotic NDT plays major role in assessing such structures. Robot Operating Systems (ROS) is a powerful open source tool, which is used to program and control robots and other devices enabling one to use sensors, devices and new functionalities such as autonomous navigation, visual perception. For the research presented, ROS is used to program the Sawyer robot arm manufactured by Rethink Robotics. The control system for the robot ensures the ability to adapt to different weights of integrated measuring devices and measured force applied against the wall with a wheeled measuring device while making a remote sweep. Twelve reinforced concrete (RC) beams built with characterized defects are evaluated with the robotic arm integrated with the measuring devices. The RC beams are characterized by voids, corrosion and debonding problems of rebars. The Impact Echo and Ground Penetrating Radar (GPR) devices attached to the robot arm are used to measure the material properties of the concrete beams. The robot arm ensures consistency and uniformity of measurement even in obscure locations. Infrared cameras are used to obtain 3D images of the surfaces of the concrete structures. Roughness is analyzed using computer vision techniques prior to the use of other sensors that are in touch with the surfaces. The computer vision is also used to handle the images collected from GPR sensor. Hand-held NDE is also performed to verify the results obtained by robotic examination. (authors)

42 ENGINEERING↗
DOE OSTI2018

Automatic Imagery Data Analysis for Proactive Computer-Based Workflow Management during Nuclear Power Plant Outages

This report is being submitted for the task “Final Report” of DOE NEUP Project 15-8121 “Automatic Imagery Data Analysis for Proactive Computer-Based Workflow Management during Nuclear Power Plant Outages.” Typical nuclear power plant (NPP) outages always involve thousands of maintenance and refueling activities and a large number of workers in limited workspaces, while having tight schedules and zero-tolerance for accidents. During an outage, thousands of workers will be working in various workspaces across the NPP. High outage costs and expensive delays (approximately 1.5 million dollars of loss per day of delay) in NPP maintenance demand tight outage schedules. In packed workspaces, an automatic system that monitors human behaviors in real-time and provides insights about current and pending schedule deviations from the plan is critical for ensuring: 1) effective collaboration among workers and worker teams from different trades; 2) less waste of time and resources due to the lack of situational awareness; and 3) proactive outage project control. The overall goal of this project is to test the hypothesis that real-time imagery-based object tracking and spatial analysis, as well as human behavior modeling of outage participants, will significantly improve the efficiency of outage control while lowering the rates of accidents and incidents. Three objectives of this project are: 1) Establish real-time object tracking and spatiotemporal analysis methods that automatically assess the productivity of field activities and detect anomalous spatiotemporal relationships among activities that cause inefficiencies and risks; 2) Establish realtime human tracking and human factor modeling methods for automatically diagnosing unexpected actions of and interactions between outage participants, those which cause inefficient collaborations between Advanced Outage Control Center (AOCC), satellite outage centers, NPP workers, and maintenance service providers; and 3) Test the proposed automated object tracking, human behavior modeling, and spatiotemporal analysis methods in outage control case studies in order to characterize the effectiveness of automated imagery-data-driven methods in proactively improving the efficiency and safety of workflows in outage coordination and risk management. Recent studies of detailed human behavior monitoring on construction sites have examined the potential of applying advanced computer vision algorithms in detecting and tracking anomalous workers (i.e., workers who do not wear hard hats or safety vests) for ensuring job site safety. Some studies of human factor studies revealed the importance of modeling detailed interactions between individuals within and across teams in better understanding the impact of human in proactive project control. Other studies in the construction domain have developed computational simulation frameworks that formalize detailed spatiotemporal interactions between tasks in order to simulate the impacts of individual tasks on the performance of workflows. While integrated analyses that combine human factor assessment, as well as image processing and simulation, are in demand for effective decision-making, limited studies have examined the potential of such integrated analyses in NPP outage control. This research project has examined an automatic outage monitoring and control system that integrates human factor analysis, computer vision techniques, and simulation methods in order to enable engineers to better understand the interactions between humans, resources, and workflows during outage processes. The project aims at providing NPP maintenance agencies insights into more efficient use of limited resources in extending the life of a nuclear plant, as well as reducing waste while ensuring sufficient generation of electricity. This study is significant for the safety of nuclear plants, sustainable electricity generation for livable communities, and cost savings for maintaining electricity infrastructures in the United States.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Multisource Mobile Transfer Learning Algorithm Based on Dynamic Model Compression

With the development of the Internet of Things, the application of computer vision on mobile phones is becoming more and more extensive and people have higher and higher requirements for the timeliness of the recognition results returned and the processing capabilities of the mobile phone for image recognition. However, the processing capability and storage capability of the user terminal equipment cannot meet the needs of identifying and storing a large number of pictures, and the data transmission process will cause high energy consumption of the terminal equipment. At the same time, multisource deep transfer learning has outstanding performance in computer vision and image classification. However, due to the huge amount of calculation of the deep network model, it is impossible to use the existing excellent network model to realize image recognition and classification on the mobile terminal. In order to solve the abovementioned problems, we propose a multisource mobile transfer learning algorithm based on dynamic model compression, this algorithm considers the realization of multisource transfer learning computing in the case of multiple mobile device computing source domains, and the method also guarantees data privacy and security for each device (origin domain). Meanwhile, extensive experiments show that our method can achieve remarkable results in popular image classification datasets.

Gao, Peng↗
DOE OSTI2022

XRIPL

XRIPL is a library of tools for processing x-ray radiographs and extracting contours of interest within the image using computer vision techniques. Features include spatial calibration, denoising, background and attenuation correction through pseudo-flatfielding, watershed segmentation, contour processing, and visualization. The details of the XRIPL analysis pipeline are published in the Proceedings of the 23rd Topical Conference on High-Temperature Plasma Diagnostics Proceedings (LA-UR-20-30133) [1]. [1] P. M. Kozlowski, Y. Kim, B. M. Haines, H. F. Robey, T. J. Murphy, H. M. Johns, and T. S. Perry. Use of Computer Vision for analysis of image data sets from high temperature plasma experiments. Review of Scientific Instruments 92, 033532 (2021) https://doi.org/10.1063/5.0040285

Kozlowski, Pawel↗
DOE OSTI2016

Four-dimensional (4D) tracking of high-temperature microparticles

High-speed tracking of hot and molten microparticles in motion provides rich information about burning plasmas in magnetic fusion. An exploding-wire apparatus is used to produce moving high-temperature metallic microparticles and to develop four-dimensional (4D) or time-resolved 3D particle tracking techniques. The pinhole camera model and algorithms developed for computer vision are used for scene calibration and 4D reconstructions. 3D positions and velocities are then derived for different microparticles. Velocity resolution approaches 0.1 m/s by using the local constant velocity approximation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Enhancing Nanoparticle Detection in Interferometric Scattering (iSCAT) Microscopy Using a Mask R-CNN

Interferometric scattering microscopy (iSCAT) is a label-free optical microscopy technique that enables imaging of individual nano-objects such as nanoparticles, viruses, and proteins. Essential to this technique is the suppression of background scattering and identification of signals from nano-objects. In the presence of substrates with high roughness, scattering heterogeneities in the background, when coupled with tiny stage movements, cause features in the background to be manifested in background-suppressed iSCAT images. Traditional computer vision algorithms detect these background features as particles, limiting the accuracy of object detection in iSCAT experiments. Here, in this paper, we present a pathway to improve particle detection in such situations using supervised machine learning via a mask region-based convolutional neural network (mask R-CNN). Using a model iSCAT experiment of 19.2 nm gold nanoparticles adsorbing to a rough layer-by-layer polyelectrolyte film, we develop a method to generate labeled datasets using experimental background images and simulated particle signals and train the mask R-CNN using limited computational resources via transfer learning. We then compare the performance of the mask R-CNN trained with and without inclusion of experimental backgrounds in the dataset against that of a traditional computer vision object detection algorithm, Haar-like feature detection, by analyzing data from the model experiment. Results demonstrate that including representative backgrounds in training datasets improved the mask R-CNN in differentiating between background and particle signals and elevated performance by markedly reducing false positives. The methodology for creating a labeled dataset with representative experimental backgrounds and simulated signals facilitates the application of machine learning in iSCAT experiments with strong background scattering and thus provides a useful workflow for future researchers to improve their image processing capabilities.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Toward Large-Scale Image Segmentation on Summit

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (~ 10 2 × 10 2 ), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (~10 4 × 10 4 ). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ~ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified and mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.

Seal, Sudip↗
DOE OSTI2021

Safeguards-Informed Hybrid Imagery Dataset [Poster]

Deep Learning computer vision models require many thousands of properly labelled images for training, which is especially challenging for safeguards and nonproliferation, given that safeguards-relevant images are typically rare due to the sensitivity and limited availability of the technologies. Creating relevant images through real-world staging is costly and limiting in scope. Expert-labeling is expensive, time consuming, and error prone. We aim to develop a data set of both realworld and synthetic images that are relevant to the nuclear safeguards domain that can be used to support multiple data science research questions. In the process of developing this data, we aim to develop a novel workflow to validate synthetic images using machine learning explainability methods, testing among multiple computer vision algorithms, and iterative synthetic data rendering. We will deliver one million images – both real-world and synthetically rendered – of two types uranium storage and transportation containers with labelled ground truth and associated adversarial examples.

97 MATHEMATICS AND COMPUTING↗