Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

Vision System Measures Motions of Robot and External Objects

A prototype of an advanced robotic vision system both (1) measures its own motion with respect to a stationary background and (2) detects other moving objects and estimates their motions, all by use of visual cues. Like some prior robotic and other optoelectronic vision systems, this system is based partly on concepts of optical flow and visual odometry. Whereas prior optoelectronic visual-odometry systems have been limited to frame rates of no more than 1 Hz, a visual-odometry subsystem that is part of this system operates at a frame rate of 60 to 200 Hz, given optical-flow estimates. The overall system operates at an effective frame rate of 12 Hz. Moreover, unlike prior machine-vision systems for detecting motions of external objects, this system need not remain stationary: it can detect such motions while it is moving (even vibrating). The system includes a stereoscopic pair of cameras mounted on a moving robot. The outputs of the cameras are digitized, then processed to extract positions and velocities. The initial image-data-processing functions of this system are the same as those of some prior systems: Stereoscopy is used to compute three-dimensional (3D) positions for all pixels in the camera images. For each pixel of each image, optical flow between successive image frames is used to compute the two-dimensional (2D) apparent relative translational motion of the point transverse to the line of sight of the camera. The challenge in designing this system was to provide for utilization of the 3D information from stereoscopy in conjunction with the 2D information from optical flow to distinguish between motion of the camera pair and motions of external objects, compute the motion of the camera pair in all six degrees of translational and rotational freedom, and robustly estimate the motions of external objects, all in real time. To meet this challenge, the system is designed to perform the following image-data-processing functions: The visual-odometry subsystem (the subsystem that estimates the motion of the camera pair relative to the stationary background) utilizes the 3D information from stereoscopy and the 2D information from optical flow. It computes the relationship between the 3D and 2D motions and uses a least-mean-squares technique to estimate motion parameters. The least-mean-squares technique is suitable for real-time implementation when the number of external-moving-object pixels is smaller than the number of stationary-background pixels.

Talukder, Ashit↗
NASA NTRS2020

A ROS-based Simulator for Testing the Enhanced Autonomous Navigation of the Mars 2020 Rover

In order to achieve the ambitious objectives of the Mars 2020 (M2020) mission, in particular the ability to autonomously traverse more challenging terrains more efficiently, new surface mobility software was developed for Enhanced Navigation (ENav). That decision was made early in the project, before most of the new surface flight software (FSW) existed, which created a need for a separate framework where the new navigation algorithms could be quickly prototyped and tested, before more realistic FSW-based testbeds became available. The JPL robotics team chose the Robot Operating System [1] (ROS) as the environment in which to test the new ENav algorithms. This made it possible to write the algorithms in the C language required by the FSW, so they could be directly ported over to the flight module later on, while leveraging all the C++ libraries and tools provided by ROS for simulation and testing. The ENav algorithms were developed as a separate C library, and stubs were used to replace any FSW-specific code, such as Event Reporting (EVRs) and data products (DPs). A ROS simulator was developed to generate a rich set of varied 3D terrains representative of the candidate Mars landing sites and simulate the physics of the rover motion, the point cloud perceived by the rover’s stereo vision system, and the new thinking-while-driving (TWD) navigation logic which directs the rover to drive autonomously to user-specified waypoints. To simulate the rover motion and perception, a ROS node was developed that uses a software library called HyperDrive Sim (HDSim), which is a wrapper for the Rover Sequencing and Visualization Program [2] (RSVP). That library provides roverterrain settling, realistic slip modelling, and camera rendering capability based on the rover’s NavCam machine vision models. To simulate the navigation logic, a ROS node was created that initializes and runs the ENav algorithms in a way that mimics the FSW execution, while also providing the capability to load and replay data products, including re-running the recorded inputs through the ENav algorithms for testing. An engineering Graphical User Interface (GUI) was also developed to visualize various elements, such as the rover pose during the drive, the simulated and perceived terrain, the selected local and global paths to the goal, the evaluated candidate paths and the reasons why they were rejected, the keep-in and keep-out zones (KIOZs), etc. Finally, an advanced Monte Carlo (MC) framework that can run many simulations in parallel on the Cloud and automatically generate reports that capture the key ENav performance metrics was developed to evaluate the system in a statisticallymeaningful way. This paper provides an overview of the ROSbased simulator used for testing the M2020 ENav algorithms.

Toupet, Olivier↗
NASA NTRS1989

The 3D model control of image processing

Telerobotics studies remote control of distant robots by a human operator using supervisory or direct control. Even if the robot manipulators has vision or other senses, problems arise involving control, communications, and delay. The communication delays that may be expected with telerobots working in space stations while being controlled from an Earth lab have led to a number of experiments attempting to circumvent the problem. This delay in communication is a main motivating factor in moving from well understood instantaneous hands-on manual control to less well understood supervisory control; the ultimate step would be the realization of a fully autonomous robot. The 3-D model control plays a crucial role in resolving many conflicting image processing problems that are inherent in resolving in the bottom-up approach of most current machine vision processes. The 3-D model control approach is also capable of providing the necessary visual feedback information for both the control algorithms and for the human operator.

Nguyen, An H.↗
NASA NTRS2012

Metrology of Large Parts

As discussed in the first chapter of this book, there are many different methods to measure a part using optical technology. Chapter 2 discussed the use of machine vision to measure macroscopic features such as length and position, which was extended to the use of interferometry as a linear measurement tool in chapter 3, and laser or other trackers to find the relation of key points on large parts in chapter 4. This chapter looks at measuring large parts to optical tolerances in the sub-micron range using interferometry, ranging, and optical tools discussed in the previous chapters. The purpose of this chapter is not to discuss specific metrology tools (such as interferometers or gauges), but to describe a systems engineering approach to testing large parts. Issues such as material warpage and temperature drifts that may be insignificant when measuring a part to micron levels under a microscope, as will be discussed in later chapters, can prove to be very important when making the same measurement over a larger part. In this chapter, we will define a set of guiding principles for successfully overcoming these challenges and illustrate the application of these principles with real world examples. While these examples are drawn from specific large optical testing applications, they inform the problems associated with testing any large part to optical tolerances. Manufacturing today relies on micrometer level part performance. Fields such as energy and transportation are demanding higher tolerances to provide increased efficiencies and fuel savings. By looking at how the optics industry approaches sub-micrometer metrology, one can gain a better understanding of the metrology challenges for any larger part specified to micrometer tolerances. Testing large parts, whether optical components or precision structures, to optical tolerances is just like testing small parts, only harder. Identical with what one does for small parts, a metrologist tests large parts and optics in particular to quantify their mechanical properties (such as dimensions, mass, etc); their optical prescription or design (i.e. radius of curvature, conic constant, vertex location, size); and their full part shape. And, just as with small parts, a metrologist accomplishes these tests using distance measuring instruments such as tape measures, inside micrometers, coordinate measuring machines, distance measuring interferometers; angle measuring instruments such as theodolites, autocollimators; and surface measuring instruments including interferometers, stylus profilers, interference microscopes, photogrammetric cameras, or other tools. However, while the methodology may be similar, it is more difficult to test a large object for the simple reason that most metrologists do not have the necessary intuition. The skills used to test small parts or optics in a laboratory do not extrapolate to testing large parts in an industrial setting any more than a backyard gardener might successfully operate a farm. But first, what is a large part? A simple definition might be the part's size or diameter. For optics and diffuse surface parts alike, the driving constraint is ability to illuminate the part's surface. For reflective convex mirrors, large is typically anything greater than 1 meter. But, for refractive optics, flats or convex mirrors, large is typically greater than 0.5 meter. While a size definition is simple, it may be less than universal. A more nuanced definition might be that a large part is any component which cannot be easily tested in a standard laboratory environment, on a standard vibration isolated table using standard laboratory infrastructure. A micro-switch or a precision lens might be easily measured to nanometer levels under a microscope in a lab, but a power turbine spline or a larger telescope mirror will not fit under that microscope and may not even fit on the table.

Stahl, H. Philip↗
DOE OSTI2023

Universal Polarization Transformations: Spatial Programming of Polarization Scattering Matrices Using a Deep Learning‐Designed Diffractive Polarization Transformer

Abstract Controlled synthesis of optical fields having nonuniform polarization distributions presents a challenging task. Here, a universal polarization transformer is demonstrated that can synthesize a large set of arbitrarily‐selected, complex‐valued polarization scattering matrices between the polarization states at different positions within its input and output field‐of‐views (FOVs). This framework comprises 2D arrays of linear polarizers positioned between isotropic diffractive layers, each containing tens of thousands of diffractive features with optimizable transmission coefficients. After its deep learning‐based training, this diffractive polarization transformer can successfully implement N i N o = 10 000 different spatially‐encoded polarization scattering matrices with negligible error, where N i and N o represent the number of pixels in the input and output FOVs, respectively. This universal polarization transformation framework is experimentally validated in the terahertz spectrum by fabricating wire‐grid polarizers and integrating them with 3D‐printed diffractive layers to form a physical polarization transformer. Through this set‐up, an all‐optical polarization permutation operation of spatially‐varying polarization fields is demonstrated, and distinct spatially‐encoded polarization scattering matrices are simultaneously implemented between the input and output FOVs of a compact diffractive processor. This framework opens up new avenues for developing novel devices for universal polarization control and may find applications in, e.g., remote sensing, medical imaging, security, material inspection, and machine vision.

Optical neural networks↗
DOE OSTI2021

Multi-spatio-temporal scales PIV in a turbulent buoyant jet discharging in a linearly stratified environment

Time-resolved particle image velocimetry is implemented with a camera array and several laser sheets; this results in a multi-spatio-temporal scale system that is modular and flexible. The setup is optimized to capture global flow features, while locally resolving in space and time near the Kolmogorov scale. The apparatus relies extensively on machine vision CMOS cameras; they are inexpensive and stream to computer hard drive with virtually continuous time-resolved records (up to one hour for the current system). This leads to statistically converged data and also helps in minimizing down time between experimental runs. Details of the implementation and design of experiment are reported. The system instruments a vertical buoyant jet discharging in a linearly stratified environment. Refractive index matched aqueous solutions enable precise optical deployment. The density difference is 3% and the fluids have similar dynamic viscosity. The jet Reynolds number is 2.00 x 10 4 and is above the mixing transition. Such flows are typically challenging to instrument and few velocity field data are available to date. The velocity statistics are reported as well as first insights gained from the campaign.

42 ENGINEERING↗
DOE OSTI2022

Multi-pitch nano-accuracy surface profiler for strongly curved X-ray mirror metrology

We present our recent research and development effort on the Multi-Pitch Nano-accuracy Surface Profiler (MPNSP). This metrology instrument is developed to characterize strongly curved X-ray mirrors proposed to achieve diffraction-limited soft X-ray focusing for scientific applications at the synchrotron beamlines. The measurement process consists of forward-and-backward scans on the test mirror surface along its tangential direction at multiple pitch angles. Our research and development in this work aim to take the challenge of measuring strongly curved X-ray mirrors with a typical total slope range ≳10 mrad, while maintaining the Root Mean Square (RMS) value of measurement repeatability and self-consistency at 50 nrad RMS level. We first introduce the mechanical design, followed by a brief review of the mathematical model and the optimization algorithm for the MPNSP technique. By detecting the rotation axis of the mirror pitch with a machine vision approach, we can determine and then reduce the vertical distance between the pitch rotation axis and the test mirror surface to a sub-mm level. In addition, we describe a practical data acquisition procedure for strongly curved X-ray mirrors with a total slope range larger than the slope measuring range of the autocollimator used in the instrument. As a result, the proposed MPNSP measurements with different mirror orientations achieve remarkable self-consistency and reproducibility of < 50 nrad RMS in slope and < 0.5 nm RMS in height.

42 ENGINEERING↗
DOE OSTI2023

Rapid sensing of hidden objects and defects using a single-pixel diffractive terahertz sensor

Abstract Terahertz waves offer advantages for nondestructive detection of hidden objects/defects in materials, as they can penetrate most optically-opaque materials. However, existing terahertz inspection systems face throughput and accuracy restrictions due to their limited imaging speed and resolution. Furthermore, machine-vision-based systems using large-pixel-count imaging encounter bottlenecks due to their data storage, transmission and processing requirements. Here, we report a diffractive sensor that rapidly detects hidden defects/objects within a 3D sample using a single-pixel terahertz detector, eliminating sample scanning or image formation/processing. Leveraging deep-learning-optimized diffractive layers, this diffractive sensor can all-optically probe the 3D structural information of samples by outputting a spectrum, directly indicating the presence/absence of hidden structures or defects. We experimentally validated this framework using a single-pixel terahertz time-domain spectroscopy set-up and 3D-printed diffractive layers, successfully detecting unknown hidden defects inside silicon samples. This technique is valuable for applications including security screening, biomedical sensing and industrial quality control.

Li, Jingxi (ORCID:0000000165958680)↗
DOE OSTI2024

Thermal deflection in neutron scattering sample environments at Oak Ridge National Laboratory

The neutron sources at Oak Ridge National Laboratory use a wide suite of sample environment equipment to deliver extreme conditions for a number of experiments. Much of this instrumentation focuses on extremes of temperature, such as cryostats, closed-cycle refrigerators in both low and high temperature configurations, and radiant heating furnaces. When the temperature is controlled across a large range, thermal deflection effects can notably move the sample and affect its alignment in the beam. Here, we combine these sample environments with neutron imaging and machine vision to determine the motion of a representative sample with respect to the neutron beam. We find vertical sample displacement on the order of 1–2 mm and horizontal displacement that varies from near-negligible to 1.2 mm. While these deflections are not relevant for some of the beamlines at the first target station at the spallation neutron source and the high flux isotope reactor, they will become critical for upcoming instrumentation at the second target station, as well as any instruments targeting sub-mm samples, as neutron sources and optics evolve to smaller and more focused beams. We discuss mitigation protocols and potential modifications to the environment to minimize the effect of misalignment due to thermal deflection.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Advances in phosphor two-color ratio method thermography for full-field surface temperature measurements

We report thermographic phosphors can be employed for optical sensing of surface, gas phase, and bulk material temperatures through different strategies including the time-decay method, time-integrated method, and frequency-domain method. We focus on the time-integrated method, also known as the ratio method, as it can be more practical in many situations. This work advances the ratio method using two machine vision cameras with CMOS detectors for full-field temperature measurements of a solid surface. A phosphor calibration coupon is fabricated using aerosol deposition and employed for in situ determination of the temperature-versus-intensity ratio relationship. Algorithms from digital image correlation are employed to determine the stereoscopic imaging system intrinsic and extrinsic parameters, and accurately register material points on the sample to subpixel locations in each image with 0.07 px or better accuracy. Detector nonlinearity is carefully characterized and corrected. Temperature-dependent, spatial non-uniformity of the full-field intensity ratio—posited to be caused by a blue-shift effect of the bandpass filter for non-collimated light and/or a wavelength-dependent transmission efficiency of the lens—is assessed and treated for cases where a standard flat-field correction fails to correct the non-uniformity. In sum, pixel-wise calibration curves relating the computed intensity ratio to temperature in the range of T = 300–430 K are generated, with an embedded error of less than 3 K. This work offers a full calibration methodology and several improvements on two-color phosphor thermography, opening the door for full-field temperature measurements in dynamic tests with deforming test articles.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

GWAS supported by computer vision identifies large numbers of candidate regulators of in planta regeneration in Populus trichocarpa

Plant regeneration is an important dimension of plant propagation and a key step in the production of transgenic plants. However, regeneration capacity varies widely among genotypes and species, the molecular basis of which is largely unknown. Association mapping methods such as genome-wide association studies (GWAS) have long demonstrated abilities to help uncover the genetic basis of trait variation in plants; however, the performance of these methods depends on the accuracy and scale of phenotyping. To enable a large-scale GWAS of in planta callus and shoot regeneration in the model tree Populus, we developed a phenomics workflow involving semantic segmentation to quantify regenerating plant tissues over time. We found that the resulting statistics were of highly non-normal distributions, and thus employed transformations or permutations to avoid violating assumptions of linear models used in GWAS. We report over 200 statistically supported quantitative trait loci (QTLs), with genes encompassing or near to top QTLs including regulators of cell adhesion, stress signaling, and hormone signaling pathways, as well as other diverse functions. Our results encourage models of hormonal signaling during plant regeneration to consider keystone roles of stress-related signaling (e.g. involving jasmonates and salicylic acid), in addition to the auxin and cytokinin pathways commonly considered. The putative regulatory genes and biological processes we identified provide new insights into the biological complexity of plant regeneration, and may serve as new reagents for improving regeneration and transformation of recalcitrant genotypes and species.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Robotic sample changers for macromolecular X-ray crystallography and biological small-angle X-ray scattering at the National Synchrotron Light Source II

In this work, we present two robotic sample changers integrated into the experimental stations for the macromolecular crystallography (MX) beamlines AMX and FMX, and the biological small-angle scattering (bioSAXS) beamline LiX. They enable fully automated unattended data collection and remote access to the beamlines. The system designs incorporate high-throughput, versatility, high-capacity, resource sharing and robustness. All systems are centered around a six-axis industrial robotic arm coupled with a force torque sensor and in-house end effectors (grippers). They have the same software architecture and the facility standard EPICS-based BEAST alarm system. The MX system is compatible with SPINE bases and Unipucks. It comprises a liquid nitrogen dewar holding 384 samples (24 Unipucks) and a stay-cold gripper, and utilizes machine vision software to track the sample during operations and to calculate the final mount position on the goniometer. The bioSAXS system has an in-house engineered sample storage unit that can hold up to 360 samples (20 sample holders) which keeps samples at a user-set temperature (277 K to 300 K). The MX systems were deployed in early 2017 and the bioSAXS system in early 2019.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Meta-optic accelerators for object classifiers

Rapid advances in deep learning have led to paradigm shifts in a number of fields, from medical image analysis to autonomous systems. These advances, however, have resulted in digital neural networks with large computational requirements, resulting in high energy consumption and limitations in real-time decision-making when computation resources are limited. Here, we demonstrate a meta-optic–based neural network accelerator that can off-load computationally expensive convolution operations into high-speed and low-power optics. In this architecture, metasurfaces enable both spatial multiplexing and additional information channels, such as polarization, in object classification. End-to-end design is used to co-optimize the optical and digital systems, resulting in a robust classifier that achieves 93.1% accurate classification of handwriting digits and 93.8% accuracy in classifying both the digit and its polarization state. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine vision and artificial intelligence.

42 ENGINEERING↗
DOE OSTI2022

To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects

Abstract Privacy protection is a growing concern in the digital era, with machine vision techniques widely used throughout public and private settings. Existing methods address this growing problem by, e.g., encrypting camera images or obscuring/blurring the imaged information through digital algorithms. Here, we demonstrate a camera design that performs class-specific imaging of target objects with instantaneous all-optical erasure of other classes of objects. This diffractive camera consists of transmissive surfaces structured using deep learning to perform selective imaging of target classes of objects positioned at its input field-of-view. After their fabrication, the thin diffractive layers collectively perform optical mode filtering to accurately form images of the objects that belong to a target data class or group of classes, while instantaneously erasing objects of the other data classes at the output field-of-view. Using the same framework, we also demonstrate the design of class-specific permutation and class-specific linear transformation cameras, where the objects of a target data class are pixel-wise permuted or linearly transformed following an arbitrarily selected transformation matrix for all-optical class-specific encryption, while the other classes of objects are irreversibly erased from the output image. The success of class-specific diffractive cameras was experimentally demonstrated using terahertz (THz) waves and 3D-printed diffractive layers that selectively imaged only one class of the MNIST handwritten digit dataset, all-optically erasing the other handwritten digits. This diffractive camera design can be scaled to different parts of the electromagnetic spectrum, including, e.g., the visible and infrared wavelengths, to provide transformative opportunities for privacy-preserving digital cameras and task-specific data-efficient imaging.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Exploiting universal nonlocal dispersion in optically active materials for spectro-polarimetric computational imaging

Recent years have seen significant advancements in exploring novel light-matter interactions such as hyperbolic dispersion within natural crystals. However, current studies have predominantly concentrated on local optical response of materials characterized by a dielectric tensor without spatial dispersion. Here, we investigate the nonlocal response in optically-active crystals with screw symmetries, revealing their lossless, super-dispersive properties compared to traditional optical response functions. We leverage this universal nonlocal dispersion, i.e. the dispersion of optical rotatory power, to explore a novel spectral de-multiplexing scheme compared to conventional gratings, prisms and metasurfaces. We design and demonstrate an ‘Nonlocal-Cam’ - a camera that exploits nonlocal dispersion through sampling of polarized spectral states and the application of computational spectral reconstruction algorithms. The Nonlocal-Cam captures information in both laboratory and outdoor field experiments which is unavailable to traditional intensity cameras - the spectral texture of polarization. Merging the fields of nonlocal electrodynamics and computational imaging, our work paves the way for exploiting nonlocal optics of optically active materials in a variety of applications, from biological microscopy to physics-driven machine vision and remote sensing.

Wang, Xueji [Purdue Univ., West Lafayette, IN (Uni↗
DOE OSTI2022

Radiographic Imaging and Tomography

Radiographic imaging and tomography (RadIT) come in many flavors such as X-ray imaging and tomography (IT), proton IT, neutron IT, muon IT, neutrino IT, etc. We identify five RadIT themes: Physics, Sources, Detectors, Methods, and Data Science, which are integral parts of image interpretation and three-dimensional (3D) tomographic reconstruction. Traditionally, RadIT have been driven by medicine, non-destructive testing, material sciences, and security applications. The latest thrusts of growth come from automation, machine vision, additive manufacturing and virtual reality (‘metaverse’). The five RadIT themes parallel their counterparts in optical IT. Synergies among different forms of RadIT and with optical IT motivate further advances towards multi-modal IT and quantum IT.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Porosity Determination and Classification of Laser Powder Bed Fusion AlSi10Mg Dogbones Using Machine Learning [Slides]

Angular pores (lack of fusion) pores are more detrimental to the health of the part due to induced stress concentrations. Three types of porosity found in AM parts: keyhole porosity, gas porosity, and lack of fusion porosity. In this research, we address the question, "Which machine learning and/or machine vision techniques work best to segment porosity and determine the type of porosity observed in the sample?"

42 ENGINEERING↗