SEARCH · Engineering Papers
Results for “3D RGB”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Quantifying Boreal Forest Structure and Composition Using UAV Structure from Motion
The vast extent and inaccessibility of boreal forest ecosystems are barriers to routine monitoring of forest structure and composition. In this research, we bridge the scale gap between intensive but sparse plot measurements and extensive remote sensing studies by collecting forest inventory variables at the plot scale using an unmanned aerial vehicle (UAV) and a structure from motion (SfM) approach. At 20 Forest Inventory and Analysis (FIA) subplots in interior Alaska, we acquired overlapping imagery and generated dense, 3D, RGB (red, green, blue) point clouds. We used these data to model forest type at the individual crown scale as well as subplot-scale tree density (TD), basal area (BA), and aboveground biomass (AGB). We achieved 85% cross-validation accuracy for five species at the crown level. Classification accuracy was maximized using three variables representing crown height, form, and color. Consistent with previous UAV-based studies, SfM point cloud data generated robust models of TD (r(sup 2) = 0.91), BA (r(sup 2) = 0.79), and AGB (r(sup 2) = 0.92), using a mix of plot- and crown-scale information. Precise estimation of TD required either segment counts or species information to differentiate black spruce from mixed white spruce plots. The accuracy of species-specific estimates of TD, BA, and AGB at the plot scale was somewhat variable, ranging from accurate estimates of black spruce TD (+/−1%) and aspen BA (−2%) to misallocation of aspen AGB (+118%) and white spruce AGB (−50%). These results convey the potential utility of SfM data for forest type discrimination in FIA plots and the remaining challenges to develop classification approaches for species-specific estimates at the plot scale that are more robust to segmentation error.
A Real2Sim Digital Twin Pipeline for Photorealistic Robot Simulation: Evaluating VLA Policy Deployment on a Bimanual Mobile Robot
Digital twins that are automatically constructed from robot sensor data offer a promising pathway for scalable Real2Sim and Sim2Real transfer. However, it remains an open question whether photorealistic reconstruction alone is sufficient to support reliable deployment of vision-language-action (VLA) policies. We present a generative-AI-assisted Real2Sim pipeline that generates simulation-ready digital twins from real-world RGB observations with minimal manual intervention. The pipeline uses prompted segmentation to isolate scene components and a generative 3D model to directly produce simulation assets, eliminating the need for traditional multi-view reconstruction or manual 3D modeling.\r\nTo evaluate simulation fidelity, we deploy and compare policies from two VLA models in both the real robot and the reconstructed\r\nsimulation under identical tasks and initial conditions. We compare joint-level action trajectories and analyze how divergence evolves over time in closed-loop execution. Although the reconstructed environments are visually accurate, we observe increasing trajectory divergence during closedloop operation. These results indicate that photorealistic reconstruction alone is insufficient to preserve closed-loop control behavior\r\nin VLA policies, particularly in contact-rich manipulation settings where small perceptual errors compound over time.
Autonomous elemental characterization enabled by a low cost robotic platform built upon a generalized software architecture
Despite the rapidly growing applications of robots in industry, the use of robots to automate tasks in scientific laboratories is less prolific due to the lack of generalized methodologies and the high cost of hardware. This paper focuses on the automation of characterization tasks necessary for reducing cost while maintaining generalization and proposes a software architecture for building robotic systems in scientific laboratory environments. A dual-layer (Socket.IO and ROS) action server design is the basic building block, which facilitates the implementation of a web-based front end for user-friendly operation and the use of ROS Behavior Trees for convenient task planning and execution. A robotic platform for automating mineral and material sample characterization is built upon the architecture, with an open-source, low-cost three-axis computer numerical control gantry system serving as the main robot. A handheld laser induced breakdown spectroscopy (LIBS) analyzer is integrated with a 3D printed adapter, enabling (1) automated 2D chemical mapping and (2) autonomous sample measurement (with the support of an RGB-Depth camera). We demonstrate the utility of automated chemical mapping by scanning the surface of a spodumene-bearing pegmatite core sample with a 1071-point dense hyperspectral map acquired at a rate of 1520 bits per second. Furthermore, we showcase the autonomy of the platform in terms of perception, dynamic decision-making, and execution, through a case study of LIBS measurement of multiple mineral samples. The platform enables controlled and autonomous chemical quantification in the laboratory that complements field-based measurements acquired with the same handheld device, linking resource exploration and processing steps in the supply chain for lithium-based battery materials.
MISR Instrument Data Visualization
The MISR Interactive eXplorer (MINX) software functions both as a general-purpose tool to visualize Multiangle Imaging SpectroRadiometer (MISR) instrument data, and as a specialized tool to analyze properties of smoke, dust, and volcanic plumes. It includes high-level options to create map views of MISR orbit locations; scrollable, single-camera RGB (red-greenblue) images of MISR level 1B2 (L1B2) radiance data; and animations of the nine MISR camera images that provide a 3D perspective of the scenes that MISR has acquired. NASA Tech Briefs, September 2008 55 The plume height capability provides an accurate estimate of the injection height of plumes that is needed by air quality and climate modelers. MISR provides global high-quality stereo height information, and this program uses that information to perform detailed height retrievals of aerosol plumes. Users can interactively digitize smoke, dust, or volcanic plumes and automatically retrieve heights and winds, and can also archive MISR albedos and aerosol properties, as well as fire power and brightness temperatures associated with smoke plumes derived from Moderate Resolution Imaging Spectroradiometer (MODIS) data. Some of the specialized options in MINX enable the user to do other tasks. Users can display plots of top-of-atmosphere bidirectional reflectance factors (BRFs) versus camera-angle for selected pixels. Images and animations can be saved to disk in various formats. Also, users can apply a geometric registration correction to warp camera images when the standard processing correction is inadequate. It is possible to difference the images of two MISR orbits that share a path (identical ground track), as well as to construct pseudo-color images by assigning different combinations of MISR channels (angle or spectral band) to the RGB display channels. This software is an interactive application written in IDL and compiled into an IDL Virtual Machine (VM) ".sav" file.
Command and Data Handling Branch Internship
Modular Integrated Stackable Layers (MISL) is a computer system designed for simple, fast, and cost effective flexible reconfiguration in space environments such as the ISS and Orion projects for various uses. Existing applications include wireless and wired communications, data acquisition and instrumentation, and camera systems, and potential applications include bus protocol converters and subsystem control. MISL is based on Texas Instruments (TI)' MSP430 16-bit ultra-low-power microcontroller device. The purpose of my project was to integrate the MISL system with a liquid crystal display (LCD) touchscreen. The LCD, manufactured by Crystalfontz and part number CFAF320240F-035T-TS, is a 320 by 240 RGB resistive color screen including an optional carrier board. The vast majority of the project was done with Altium Designer, a tool for printed circuit board (PCB) schematic capture, 3D design, and FPGA (Field Programmable Gate Array) development. The new PCB was to allow the LCD to directly stack to the rest of MISL. Research was done with datasheets for the TI microcontroller and touchscreen display in order to meet desired hardware specifications. Documentation on prior MISL projects was also utilized. The initial step was to create a schematic for the LCD, power bus, and data bus connections between components. A layout was then designed with the required physical dimensions, routed traces and vias, power and ground planes, layer stacks, and other specified design rules such as plane clearance and hole size. Multiple consultation sessions were held with Hester Yim, the technical discipline lead for the Command and Data Handling Branch, and Christy Herring, the lead PCB layout designer in the Electronic Design and Manufacturing Branch in order to ensure proper configuration. At the moment, the PCB is awaiting revision by the latter-mentioned branch. Afterwards, the board will begin to undergo the manufacturing and testing process. Throughout the internship at Johnson Space Center, I gained several technical and professional skills. I gained proficiency in Altium Designer and experience using subversion clients, as well as knowledge in PSpice with OrCAD and battery design for spaceflight from on-site. I also gained networking, organization, and communication skills throughout meetings with coworkers and other interns. This internship at Johnson Space Center has impacted my future aspirations by further inspiring me to follow a career path into space rated engineering technology and human spaceflight applications. After graduation, I plan to attend graduate Modular Integrated Stackable Layers (MISL) is a computer system designed for simple, fast, and cost effective flexible reconfiguration in space environments such as the ISS and Orion projects for various uses. Existing applications include wireless and wired communications, data acquisition and instrumentation, and camera systems, and potential applications include bus protocol converters and subsystem control. MISL is based on Texas Instruments’ MSP430 16 bit ultra-low power microcontroller device. The purpose of my project was to integrate the MISL system with a liquid crystal display touchscreen. The LCD, manufactured by Crystalfontz and part number CFAF320240F-035T-TS, is a 320x240 RGB resistive color screen including an optional carrier board.The vast majority of the project was done with Altium Designer, a tool for printed circuit board (PCB) schematic capture, 3D design, and FPGA development. The new PCB was to allow the LCD to directly stack to the rest of MISL. Research was done with datasheets for the TI microcontroller and touchscreen display in order to meet desired hardware specifications. Documentation on prior MISL projects was also utilized. The initial step was to create a schematic for the LCD, power bus, and data bus connections between components. A layout was then designed with the required physical dimensions, routed traces and vias, power and ground planes, layer stacks, and other specified design rules such as plane clearance and hole size. Multiple consultation sessions were held with Hester Yim, the technical discipline lead for the Command and Data Handling Branch, and Christy Herring, the lead PCB layout designer in the Electronic Design and Manufacturing Branch in order to ensure proper configuration. At themoment, the PCB is awaiting revision by the latter-mentioned branch. Afterwards, the board will begin to undergo the manufacturing and testing process.Throughout the internship at Johnson Space Center, I gained several technical and professional skills. I gained proficiency in Altium Designer and experience using subversion clients, as well as knowledge in PSpice with OrCAD and battery design for spaceflight from on-site. I also gained networking, organization, and communication skills throughout meetings with coworkers and other interns. This internship at Johnson Space Center has impacted my future aspirations by further inspiring me to follow a career path into space rated engineering technology and human spaceflight applications. After graduation, I plan to attend graduate school for a master's or doctorate degree in electrical or computer engineering.
Improving Building Footprint Extraction Using NAIP and 3DEP Lidar Derived Features with Deep Learning
Accurate building footprint extraction is critical for applications ranging from population estimation to disaster management. Although optical imagery provides detailed spectral information, it often struggles with shadows, occlusions, and background clutter in dense urban environments. Lidar data, by contrast, offer precise elevation and structural attributes but face challenges such as variable point density and noise. This study integrates multispectral imagery from the U.S. Department of Agriculture (USDA) National Agriculture Imagery Program (NAIP) with lidar-derived feature height and intensity from the U.S. Geological Survey (USGS) 3D Elevation Program (3DEP) to improve footprint extraction using a U-Net–based deep learning model. A six-band input stack (RGB, near-infrared, height, intensity) was developed, normalized, and tiled for training and evaluation against Microsoft Global Building Footprints (GBF). Results from the Houston, TX test site show that the six-band model achieved a precision of 0.86, recall of 0.88, F1 score of 0.87, and Intersection-over-Union (IoU) of 0.76, consistently outperforming four-band baselines by reducing false positives while maintaining sensitivity. Predictions on withheld Houston tiles confirmed strong within-region generalization, yielded a precision of 0.78, recall of 0.81, F1 score of 0.79, and IoU of 0.66. Qualitative analysis further revealed limitations stemming from both training label quality and vegetation–building confusion. These findings demonstrate the complementary value of integrating spectral and structural information for robust building footprint extraction and how domain adaptation strategies can be used to enhance cross-regional transferability.
Bi-axial Strain Measurement of Woven-Webbing Inflatable Structures Using Digital Image Correlation with RGB Filtering
Crewed inflatable softgoods structures can significantly reduce launch volume for habitable pressure vessels. To ensure the operational safety of these structures, NASA has developed an extensive testing program to evaluate the structural capability on ground test articles. This paper will discuss a current challenge faced by test engineers evaluating the strain of large-scale inflatable space structures during ground testing. Namely, the difficulty that arises using monochrome digital image correlation (DIC) to evaluate bi-axial strain for woven-webbing restraint layers. This report provides insight into a novel method of photogrammetry using red-green-blue (RGB) filtering in conjunction with DIC to evaluate the strain in each direction on a test article experiencing bi-directional strain. To evaluate this concept, a uniaxial tensile test was performed that represented a biaxial woven architecture using orthogonally interfaced straps. The resulting analysis demonstrates promise for this novel approach for isolating a specific region and direction of strain, reducing the effects of straps not in the area of interest, while allowing for a larger region of analysis. Moreover, it was shown that the application of a white basecoat prior to applying photogrammetry patterns significantly reduced data loss while performing RGB filtered DIC.
Single Lens Dual-Aperture 3D Imaging System: Color Modeling
In an effort to miniaturize a 3D imaging system, we created two viewpoints in a single objective lens camera. This was accomplished by placing a pair of Complementary Multi-band Bandpass Filters (CMBFs) in the aperture area. Two key characteristics about the CMBFs are that the passbands are staggered so only one viewpoint is opened at a time when a light band matched to that passband is illuminated, and the passbands are positioned throughout the visible spectrum, so each viewpoint can render color by taking RGB spectral images. Each viewpoint takes a different spectral image from the other viewpoint hence yielding a different color image relative to the other. This color mismatch in the two viewpoints could lead to color rivalry, where the human vision system fails to resolve two different colors. The difference will be closer if the number of passbands in a CMBF increases. (However, the number of passbands is constrained by cost and fabrication technique.) In this paper, simulation predicting the color mismatch is reported.
Cool, Evolved Stars: Results, Challenges, and Promises for the Next Decade
Cool, evolved stars are the main source of chemical enrichment of the interstellar medium (ISM), and understanding their mass loss and structure offers a unique opportunity to study the cycle of matter in the Universe. Pulsation, convection, and other dynamic processes in cool evolved stars create an atmosphere where molecules and dust can form, including those necessary to the formation of life (e.g. Carbon-bearing molecules). Understanding the structure and composition of these stars is thus vital to several aspects of stellar astrophysics, ranging from ISM studies to modeling young galaxies and to exoplanet research. Recent modeling efforts and increasingly precise observations now reveal that our understanding of cool stars photospheric, chromospheric, and atmospheric structures is limited by inadequate knowledge of the dynamic and chemical processes at work. Here we outline promising scientific opportunities for the next decade that can provide essential constraints on stellar photospheres, chromospheres, and circumstellar envelopes (CSE), and tie together analyses of the spectra and interferometric and imaging observations of evolved stars. We identify and discuss the following main opportunities: (1) identify and model the physical processes that must be included in current 1D and 3D atmosphere models of cool, evolved stars; (2) refine our understanding of photospheric, chromospheric, and outer atmospheric regions of cool evolved stars, their properties and parameters, through high-resolution spectroscopic observations, and interferometric observations at high angular resolution; (3) include the neglected role of chromospheric activity in the mass loss process of red giant branch (RGB) and red super giant (RSG) stars and understand the role played by their magnetic fields; (4) identify the important shaping mechanisms for planetary nebulae (PNe) and their relation with the parent Asymptotic Giant Branch (AGB) stars.
Advanced Interactive 3D Visualization Tool for Customizable Analyses of Tomography Datasets in Material Science
Current methods for visualizing and analyzing 3D tomography datasets in materials science often lack the interactivity and depth required for detailed structural insights. This limitation restricts a researchers' ability to accurately interpret complex data, which is critical for advancing material innovations and understanding structural properties. To address this issue, we have developed a novel, web-based interactive 3D visualization and analysis tool from the Trame framework that offers customizable features to enhance data interpretability. The tool allows users to adjust parameters such as visible range, slice planes, data rotation, and layering, providing a more detailed and dynamic view of complex structures. Its user-friendly web interface increases the accessibility and ease of use for both novice and experienced researchers, to visualize large volumetric datasets. The tool supports a diverse range of data formats, making it versatile for various research applications. Unique capabilities include real-time data manipulation, automated feature detection, context-sensitive feedback, and real-time volume calculations and distributions per sliced region or layer, alongside the ability to quickly generate high-quality screenshots and videos for presentations and reports. These advancements offer a comprehensive solution for enhanced 3D data exploration, significantly improving the analysis process and communication of results in materials science.
Unsupervised Change Detection for Space Habitats Using 3D Point Clouds
This work presents an algorithm for scene change detection from point clouds to enable autonomous robotic caretaking in future space habitats. Autonomous robotic systems will help maintain future deep-space habitats, such as the Gateway space station, which will be uncrewed for extended periods. Existing scene analysis software used on the International Space Station (ISS) relies on manually-labeled images for detecting changes. In contrast, the algorithm presented in this work uses raw, unlabeled point clouds as inputs. The algorithm first applies modified Expectation-Maximization Gaussian Mixture Model (GMM) clustering to two input point clouds. It then performs change detection by comparing the GMMs using the Earth Mover’s Distance. The algorithm is validated quantitatively and qualitatively using a test dataset collected by an Astrobee robot in the NASA Ames Granite Lab comprising single frame depth images taken directly by Astrobee and full-scene reconstructed maps built with RGB-D and pose data from Astrobee. The runtimes of the approach are also analyzed in depth. The source code is publicly released to promote further development.
Deep Point Cloud Building Envelope Segmentation (DeeP-CuBES) using Deep Learning
Building Information Modeling (BIM) plays an important role in building design and construction, particularly for achieving energy-efficient retrofits. Building envelope retrofits using panelized prefabricated system, such as those popularized by the Energiesprong program, need accurate as-built dimensions of facade features (windows, doors, etc.) to achieve the desired thermal and air tightness. Traditionally, building surveying is done manually, resulting in a time-consuming and labor-intensive process. Recently, 3D point clouds from terrestrial LiDAR have been used to automate the generation of as-built dimensions of existing buildings. However, automated BIM using LiDAR relies on solving the point cloud semantic segmentation (PCSS) problem. In this work, we propose a robust pipeline for solving the PCSS problem using deep neural networks, focusing on overcoming challenges posed by imbalanced datasets and complex architectural features. We introduce the first high-density, labeled, and validated building envelope point cloud dataset derived from multiple building scans, specifically curated to tackle challenges in facade-level segmentation. Results from the trained neural networks show that advanced attention-based architectures and incorporating radiometry (light intensity and RGB) features significantly boost segmentation accuracy for windows and doors.
NH11B-1726: FrankenRaven: A New Platform for Remote Sensing
Small, modular aircraft are an emerging technology with a goal to maximize flexibility and enable multi-mission support. This reports the progress of an unmanned aerial system (UAS) project conducted at the NASA Ames Research Center (ARC) in 2016. This interdisciplinary effort builds upon the success of the 2014 FrankenEye project to apply rapid prototyping techniques to UAS, to develop a variety of platforms to host remote sensing instruments. In 2016, ARC received AeroVironment RQ-11A and RQ-11B Raven UAS from the US Department of the Interior, Office of Aviation Services. These aircraft have electric propulsion, a wingspan of roughly 1.3m, and have demonstrated reliability in challenging environments. The Raven airframe is an ideal foundation to construct more complex aircraft, and student interns using 3D printing were able to graft multiple Raven wings and fuselages into FrankenRaven aircraft. Aeronautical analysis shows that the new configuration has enhanced flight time, payload capacity, and distance compared to the original Raven. The FrankenRaven avionics architecture replaces the mil-spec avionics with COTS technology based upon the 3DR Pixhawk PX4 autopilot with a safety multiplexer for failsafe handoff to 2.4 GHz RC control and 915 MHz telemetry. This project demonstrates how design reuse, rapid prototyping, and modular subcomponents can be leveraged into flexible airborne platforms that can host a variety of remote sensing payloads and even multiple payloads. Modularity advances a new paradigm: mass-customization of aircraft around given payload(s). Multi-fuselage designs are currently under development to host a wide variety of payloads including a zenith-pointing spectrometer, a magnetometer, a multi-spectral camera, and a RGB camera. After airworthiness certification, flight readiness review, and test flights are performed at Crows Landing airfield in central California, field data will be taken at Kilauea volcano in Hawaii and other locations.
Tools for Analysis and Visualization of Large Time-Varying CFD Data Sets
In the second year, we continued to built upon and improve our scanline-based direct volume renderer that we developed in the first year of this grant. This extremely general rendering approach can handle regular or irregular grids, including overlapping multiple grids, and polygon mesh surfaces. It runs in parallel on multi-processors. It can also be used in conjunction with a k-d tree hierarchy, where approximate models and error terms are stored in the nodes of the tree, and approximate fast renderings can be created. We have extended our software to handle time-varying data where the data changes but the grid does not. We are now working on extending it to handle more general time-varying data. We have also developed a new extension of our direct volume renderer that uses automatic decimation of the 3D grid, as opposed to an explicit hierarchy. We explored this alternative approach as being more appropriate for very large data sets, where the extra expense of a tree may be unacceptable. We also describe a new approach to direct volume rendering using hardware 3D textures and incorporates lighting effects. Volume rendering using hardware 3D textures is extremely fast, and machines capable of using this technique are becoming more moderately priced. While this technique, at present, is limited to use with regular grids, we are pursuing possible algorithms extending the approach to more general grid types. We have also begun to explore a new method for determining the accuracy of approximate models based on the light field method described at ACM SIGGRAPH '96. In our initial implementation, we automatically image the volume from 32 equi-distant positions on the surface of an enclosing tessellated sphere. We then calculate differences between these images under different conditions of volume approximation or decimation. We are studying whether this will give a quantitative measure of the effects of approximation. We have created new tools for exploring the differences between images produced by various rendering methods. Images created by our software can be stored in the SGI RGB format. Our idtools software reads in pair of images and compares them using various metrics. The differences of the images using the RGB, HSV, and HSL color models can be calculated and shown. We can also calculate the auto-correlation function and the Fourier transform of the image and image differences. We will explore how these image differences compare in order to find useful metrics for quantifying the success of various visualization approaches. In general, progress was consistent with our research plan for the second year of the grant.