Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Computer vision”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65

VSLAM and Coplanar POSIT for Advanced Air Mobility Approach and Landing

Advanced Air Mobility (AAM) aircraft have many challenges for landing accurately and safely in urban, suburban, and rural environments. Localization in large and open rural environments could utilize GPS, but AAM aircraft in urban environments will encounter GPS degradation. Another challenge involves flight operation time, i.e., flying during the day or at night. There are different types of guidelines and landmarks at runways, heliports, and vertiports for daytime and nighttime applications. Tailoring feature detection methods for AAM approach and landing during the day and night pose different issues and challenges. It is easier to detect edges, lines, and other runway markers during the day than at night. Conversely, it is easier to detect landing light configurations and patterns at nighttime than daytime. Consequently, utilizing the same feature detector for daytime and nighttime operations may not be feasible. This paper focuses on vision-based precision approach and landing (PAL) and builds on previous work by comparing ORB SLAM, custom VSLAM, and coplanar pose from orthography and scaling with iterations (POSIT) for daytime and nighttime operations. Implementing high fidelity simulations with computer graphics and extracting video frames from flight test data provide promising results for AAM PAL applications.

Evan Kawamura↗

Selective Amnesia using Contrastive Subnet Erasure for Class Level Unlearning in Vision Models

We study concept-level forgetting in pretrained vision models: removing an entire semantic category so the system no longer recognizes that object in unseen images and contexts, rather than merely forgetting specific training examples. Prior work either applies blunt global projections or fine-tunes parameters, which can introduce collateral damage to unrelated features, add compute, and become unstable as forgetting strength increases. We introduce Contrastive Subnet Erasure (CSE), a training-free, encoder-centric edit that targets a compact set of channels most responsible for the class and attenuates them in a calibrated manner. The modification is algebraically folded into the subsequent layer, yielding no inference-time overhead and leaving task heads unchanged. To evaluate whether forgetting generalizes beyond the data used to specify the class, we introduce a cross dataset protocol in which the class is defined on a source dataset and performance is measured on a disjoint target dataset drawn from a different distribution with no shared images. This setup tests whether the model still fails to recognize the object when it looks different or appears in new scenes, and it helps avoid overfitting to patterns in the source dataset. Across CIFAR 10, CIFAR 100, and ImageNet under this protocol, CSE achieves stronger forgetting of the target class while better preserving non target utility than existing baselines in both single class and multi class settings. Overall, CSE provides a simple, stable, and deployment-ready mechanism for class-level unlearning in vision.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗

Synthetic Vision for Lunar and Planetary Landing Vehicles

The Crew Vehicle Interface (CVI) group of the Integrated Intelligent Flight Deck Technologies (IIFDT) has done extensive research in the area of Synthetic Vision (SV), and has shown that SV technology can substantially enhance flight crew situation awareness, reduce pilot workload, promote flight path control precision and improve aviation safety. SV technology is being extended to evaluate its utility for lunar and planetary exploration vehicles. SV may hold significant potential for many lunar and planetary missions since the SV presentation provides a computer-generated view of the terrain and other significant environment characteristics independent of the outside visibility conditions, window locations, or vehicle attributes. SV allows unconstrained control of the computer-generated scene lighting, terrain coloring, and virtual camera angles which may provide invaluable visual cues to pilots/astronauts and in addition, important vehicle state information may be conformally displayed on the view such as forward and down velocities, altitude, and fuel remaining to enhance trajectory control and vehicle system status. This paper discusses preliminary SV concepts for tactical and strategic displays for a lunar landing vehicle. The technical challenges and potential solutions to SV applications for the lunar landing mission are explored, including the requirements for high resolution terrain lunar maps and an accurate position and orientation of the vehicle that is essential in providing lunar Synthetic Vision System (SVS) cockpit displays. The paper also discusses the technical challenge of creating an accurate synthetic terrain portrayal using an ellipsoid lunar digital elevation model which eliminates projection errors and can be efficiently rendered in real-time.

Williams, Steven P.↗

Model-based vision for space applications

This paper describes a method for tracking moving image features by combining spatial and temporal edge information with model based feature information. The algorithm updates the two-dimensional position of object features by correlating predicted model features with current image data. The results of the correlation process are used to compute an updated model. The algorithm makes use of a high temporal sampling rate with respect to spatial changes of the image features and operates in a real-time multiprocessing environment. Preliminary results demonstrate successful tracking for image feature velocities between 1.1 and 4.5 pixels every image frame. This work has applications for docking, assembly, retrieval of floating objects and a host of other space-related tasks.

Chaconas, Karen↗

Lunar Applications in Reconfigurable Computing

NASA s Constellation Program is developing a lunar surface outpost in which reconfigurable computing will play a significant role. Reconfigurable systems provide a number of benefits over conventional software-based implementations including performance and power efficiency, while the use of standardized reconfigurable hardware provides opportunities to reduce logistical overhead. The current vision for the lunar surface architecture includes habitation, mobility, and communications systems, each of which greatly benefit from reconfigurable hardware in applications including video processing, natural feature recognition, data formatting, IP offload processing, and embedded control systems. In deploying reprogrammable hardware, considerations similar to those of software systems must be managed. There needs to be a mechanism for discovery enabling applications to locate and utilize the available resources. Also, application interfaces are needed to provide for both configuring the resources as well as transferring data between the application and the reconfigurable hardware. Each of these topics are explored in the context of deploying reconfigurable resources as an integral aspect of the lunar exploration architecture.

Somervill, Kevin↗

Mission Use of the SpaceCube Hybrid Data Processing System

The award-winning SpaceCube v2.0 system is a high performance, reconfigurable, hybrid data processing system that can be used in a multitude of applications including those that require a radiation hardened and reliable solution. This presentation provides an overview of the design architecture, flexibility, and the advantages of the modular SpaceCube v2.0 high performance data processing system for space applications. The current state of the proven SpaceCube technology is based on 11 years of engineering and operations. Eight systems have been successfully operated in space starting in 2008 with eight more to be delivered for payload integration in 2018 in support of various missions. This presentation will highlight how this multipurpose system is currently being used to solve design challenges of a variety of independent applications. The SpaceCube hardware adapts to new system requirements by allowing for application-unique interface cards that are utilized by reconfiguring the underlying programmable elements on the core processor card. We will show how this system is being used to improve on a heritage NASA GPS technology, enable a cutting-edge LiDAR instrument, and serve as a typical command and data handling (CDH) computer for a space robotics technology demonstration.Finally, this presentation will highlight the use of the SpaceCube v2.0 system on the Restore-L robotic satellite servicing mission. SpaceCube v2.0 is the central avionics responsible for the real-time vision system and autonomous robotic control necessary to find, capture, and service a national asset weather satellite.

Petrick, Dave↗

GIScience in the era of Artificial Intelligence: a research agenda towards Autonomous GIS

The advent of generative AI exemplified by large language models (LLMs) opens new ways to represent and compute geographic information and transcends the process of geographic knowledge production, driving geographic information systems (GIS) towards autonomous GIS. Leveraging LLMs as the decision core, autonomous GIS can independently generate and execute geoprocessing workflows to perform spatial analysis. In this vision paper, we further elaborate on the concept of autonomous GIS and present a conceptual framework that defines its five autonomous goals, five levels of autonomy, five core functions, and three operational scales. We demonstrate how autonomous GIS could perform geospatial data retrieval, spatial analysis, and map making with four proof-of-concept GIS agents. We conclude by identifying critical challenges and future research directions, including fine-tuning and self-growing decision-cores, autonomous modelling, and examining the societal and practical implications of autonomous GIS. By establishing the groundwork for a paradigm shift in GIScience, this paper envisions a future where GIS moves beyond traditional workflows to autonomously reason, derive, innovate, and advance geospatial solutions to pressing global challenges. Meanwhile, we emphasize that as we design and deploy increasingly intelligent geospatial systems, we carry a responsibility to ensure they are developed in socially responsible ways, serve the public good, and support the continued value of human geographic insight in an AI-augmented future.

Autonomous GI↗

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Lyngaas, Isaac↗

Visions of our Planet's Atmosphere, Land & Oceans

The NASA/NOAA Electronic Theater presents Earth science observations and visualizations in a historical perspective. Fly in from outer space to South Africa, Cape Town and Johannesburg using NASA Terra MODIS data, Landsat data and 1m IKONOS "Spy Satellite" data. Zoom in to any place South Africa using Earth Viewer 3D from Keyhole Inc. and Landsat data at 30 m resolution. Go back to the early weather satellite images from the 1960s and see them contrasted with the latest US and international global satellite weather movies including hurricanes & "tornadoes". See the latest visualizations of spectacular images from NASA/NOAA remote sensing missions like Terra, GOES, TRMM, SeaWiFS, Landsat 7 including 1 - min GOES rapid scan image sequences of Nov 9th 2001 Midwest tornadic thunderstorms and have them explained. See how High-Definition Television (HDTV) is revolutionizing the way we present science to the public. See dust storms and flooding in Africa and smoke plumes from fires in Mexico. See visualizations featured on the covers of Newsweek, TIME, National Geographic, Popular Science & on National & International Network TV. New computer software tools allow us to roam & zoom through massive global images e.g. Landsat tours of the US, and Africa, showing desert and mountain geology as well as seasonal changes in vegetation. See animations of the north and south polar ice packs and with icebergs on the coasts of Greenland and off the coast of Antarctica. Spectacular new visualizations of the global land, atmosphere & oceans are shown. Listen to the pulse of our planet. See how land vegetation, ocean plankton, clouds and temperatures respond to the sun & seasons. See vortexes and currents in the global oceans that bring up the nutrients to feed tiny algae and draw the fish, whales and fisherman. See the how the ocean blooms in response to these currents and El Nino/La Nina climate changes. See the city lights, fishing fleets, gas flares and biomass burning of the Earth at night observed by the "night-vision" DMSP military satellite. The presentation will be made using the latest HDTV and video projection technology that is now done from a laptop computer through an entirely digital path.

Hasler, Arthur F.↗

NASA's Small Spacecraft and Distributed Systems: Development and Demonstration of Technologies Enabling Swarms and New Spacecraft Platforms with AI and Edge Computing

NASA’s Small Spacecraft & Distributed Systems (SSDS) within the Research and Technology Mission Directorate (RTMD) expands U.S. capability to execute unique missions through targeted investment, rapid development, and flight demonstration of small spacecraft technologies applicable to exploration, science and the commercial space sector. SSDS strategically invests in technology development and on-orbit demonstrations executed across NASA, other government agencies, industry, and academia. The program’s University SmallSat Technology Partnerships initiative awards academic researchers with the opportunity to collaborate with NASA to mature innovative technology. Capabilities aligned with RTMD’s technology shortfalls and interests - power, processing, propulsion, sensors, communications, autonomous navigation, architectures, and advanced applications like artificial intelligence (AI), machine learning, and edge computing - are prioritized in SSDS investments. These investments enable distributed, autonomous, and cooperative small spacecraft systems that support swarm missions extending beyond low Earth orbit into cislunar and deep space. This paper highlights representative SSDS flight demonstrations that mature these capabilities to enable a future operational infrastructure needed to support sustained exploration of the Moon and beyond. SSDS’s investment strategy emphasizes rapid development and on-orbit demonstration to validate spacecraft technologies required for swarms and distributed mission architectures. The Starling swarm technology demonstration mission exemplifies this approach by advancing distributed spacecraft autonomy, cooperative operations, and space situational awareness. Extended flight testing and ongoing studies of next generation swarm configurations and on-orbit space traffic monitoring and management continue to inform future swarm designs. DiskSat’s four-spacecraft demonstration mission represents SSDS’s strategic vision to expand the design space for future small spacecraft through its commitment to advance novel platform concepts that can impact how science is performed on orbit. Continuing to invest in future platforms, the notional PY12 concept is a 12-spacecraft swarm hosting neuromorphic processors and is envisioned as an on-orbit testbed for AI, edge computing, and positioning, navigation and timing technologies. SSDS also invests in single-spacecraft technology demonstrations that underpin the success of future swarm missions and accelerate the availability of validated technologies across the small spacecraft ecosystem. Examples of such demonstrations include Pathfinder Technology Demonstrator-3 (PTD-3), which performed high-rate optical communications; PTD-R, which demonstrated a camera capable of simultaneous ultraviolet and short-wave infrared optical sensing; and CAPSTONE, the Cislunar Autonomous Positioning System Technology and Operations Navigation Experiment, which validated autonomous navigation in cislunar space. Collectively, SSDS-funded demonstrations advance capabilities across swarms and illustrate a coordinated investment strategy to mature high-impact technologies required for autonomous, distributed, and cooperative small spacecraft systems for low Earth orbit, cislunar, and deep space applications. Technology demonstrations strengthen SSDS partnerships with industry, academia, and other government agencies, and promote small spacecraft community adoption of capabilities required to close technical gaps for swarm missions.

Jan Stupl↗

NASA's Small Spacecraft and Distributed Systems: Development and Demonstration of Technologies Enabling Swarms and New Spacecraft Platforms with AI and Edge Computing

NASA’s Small Spacecraft & Distributed Systems (SSDS) within the Research and Technology Mission Directorate (RTMD) expands U.S. capability to execute unique missions through targeted investment, rapid development, and flight demonstration of small spacecraft technologies applicable to exploration, science and the commercial space sector. SSDS strategically invests in technology development and on-orbit demonstrations executed across NASA, other government agencies, industry, and academia. The program’s University SmallSat Technology Partnerships initiative awards academic researchers with the opportunity to collaborate with NASA to mature innovative technology. Capabilities aligned with RTMD’s technology shortfalls and interests - power, processing, propulsion, sensors, communications, autonomous navigation, architectures, and advanced applications like artificial intelligence (AI), machine learning, and edge computing - are prioritized in SSDS investments. These investments enable distributed, autonomous, and cooperative small spacecraft systems that support swarm missions extending beyond low Earth orbit into cislunar and deep space. This paper highlights representative SSDS flight demonstrations that mature these capabilities to enable a future operational infrastructure needed to support sustained exploration of the Moon and beyond. SSDS’s investment strategy emphasizes rapid development and on-orbit demonstration to validate spacecraft technologies required for swarms and distributed mission architectures. The Starling swarm technology demonstration mission exemplifies this approach by advancing distributed spacecraft autonomy, cooperative operations, and space situational awareness. Extended flight testing and ongoing studies of next generation swarm configurations and on-orbit space traffic monitoring and management continue to inform future swarm designs. DiskSat’s four-spacecraft demonstration mission represents SSDS’s strategic vision to expand the design space for future small spacecraft through its commitment to advance novel platform concepts that can impact how science is performed on orbit. Continuing to invest in future platforms, the notional PY12 concept is a 12-spacecraft swarm hosting neuromorphic processors and is envisioned as an on-orbit testbed for AI, edge computing, and positioning, navigation and timing technologies. SSDS also invests in single-spacecraft technology demonstrations that underpin the success of future swarm missions and accelerate the availability of validated technologies across the small spacecraft ecosystem. Examples of such demonstrations include Pathfinder Technology Demonstrator-3 (PTD-3), which performed high-rate optical communications; PTD-R, which demonstrated a camera capable of simultaneous ultraviolet and short-wave infrared optical sensing; and CAPSTONE, the Cislunar Autonomous Positioning System Technology and Operations Navigation Experiment, which validated autonomous navigation in cislunar space. Collectively, SSDS-funded demonstrations advance capabilities across swarms and illustrate a coordinated investment strategy to mature high-impact technologies required for autonomous, distributed, and cooperative small spacecraft systems for low Earth orbit, cislunar, and deep space applications. Technology demonstrations strengthen SSDS partnerships with industry, academia, and other government agencies, and promote small spacecraft community adoption of capabilities required to close technical gaps for swarm missions.

Jan Stupl↗

A vision system for a Mars rover

A Mars rover must be able to sense its local environment with sufficient resolution and accuracy to avoid local obstacles and hazards while moving a significant distance each day. Power efficiency and reliability are extremely important considerations, making stereo correlation an attractive method of range sensing compared to laser scanning, if the computational load and correspondence errors can be handled. Techniques for treatment of these problems, including the use of more than two cameras to reduce correspondence errors and possibly to limit the computational burden of stereo processing, have been tested at JPL. Once a reliable range map is obtained, it must be transformed to a plan view and compared to a stored terrain database, in order to refine the estimated position of the rover and to improve the database. The slope and roughness of each terrain region are computed, which form the basis for a traversability map allowing local path planning. Ongoing research and field testing of such a system is described.

Wilcox, Brian H.↗

Machine vision guided sensor positioning system for leaf temperature assessment

A sensor positioning system was developed for monitoring plants' well-being using a non-contact sensor. Image processing algorithms were developed to identify a target region on a plant leaf. A novel algorithm to recover view depth was developed by using a camera equipped with a computer-controlled zoom lens. The methodology has improved depth recovery resolution over a conventional monocular imaging technique. An algorithm was also developed to find a maximum enclosed circle on a leaf surface so the conical field-of-view of an infrared temperature sensor could be filled by the target without peripheral noise. The center of the enclosed circle and the estimated depth were used to define the sensor 3-D location for accurate plant temperature measurement.

NASA Discipline Life Support Systems↗

Real-Time Omnidirectional Stereo Rendering: Generating 360° Surround-View Panoramic Images for Comfortable Immersive Viewing

Surround-view panoramic images and videos have become a popular form of media for interactive viewing on mobile devices and virtual reality headsets. Viewing such media provides a sense of immersion by allowing users to control their view direction and experience an entire environment. When using a virtual reality headset, the level of immersion can be improved by leveraging stereoscopic capabilities. Stereoscopic images are generated in pairs, one for the left eye and one for the right eye, and result in providing an important depth cue for the human visual system. For computer generated imagery, rendering proper stereo pairs is well known for a fixed view. However, it is much more difficult to create omnidirectional stereo pairs for a surround-view projection that work well when looking in any direction. One major drawback of traditional omnidirectional stereo images is that they suffer from binocular misalignment in the peripheral vision as a user's view direction approaches the zenith / nadir (north / south pole) of the projection sphere. This paper presents a real-time geometry-based approach for omnidirectional stereo rendering that fits into the standard rendering pipeline. Our approach includes tunable parameters that enable pole merging - a reduction in the stereo effect near the poles that can minimize binocular misalignment. Results from a user study indicate that pole merging reduces visual fatigue and discomfort associated with binocular misalignment without inhibiting depth perception.

360° panorama↗

Visioning a Global Carbon Monitoring System That Can Quantify and Attribute Ocean Carbon Dioxide Removal: Where Are We Now and How Might We Get There?

As atmospheric greenhouse gas concentrations and the resulting social costs rise, it is increasingly important to conduct research on technical approaches to remove carbon dioxide from the atmosphere at the gigaton scale. One highly uncertain but perhaps plausible solution is ocean carbon dioxide (CO2) removal (CDR), which encompasses a suite of proposed techniques for increasing the net flux of CO2 from the atmosphere to the ocean. As reflected in a 2022 National Academy of Sciences report, interest in ocean CDR is rapidly growing, and new stakeholders from a wide range of fields want access to air-sea CO2 flux and ocean carbon information to understand the potential risks and benefits of ocean CDR. Moreover, there is an urgent need for research to develop a scientific basis to support these societal needs. At the same time, there is a growing need to vision a global carbon monitoring system that can quantify natural and anthropogenic perturbations in the air-sea CO2 flux to quantify and attribute the impacts of ocean CDR efforts on the ocean carbon sink. Here, we propose to present nascent efforts at the Ames Research Center to leverage NASA's unique remote sensing and computational capabilities and expertise in basic and applied Earth science to meet the needs of the ocean CDR stakeholder community as represented by the OceanVisions network. Specifically we aim to develop a global ocean CDR scenario explorer with two objectives: 1) to make quantitative estimates of the air-sea CO2 flux and its uncertainty more accessible to a wide range of users, and 2) to enable these diverse users to interactively explore the uncertain impacts on air-sea CO2 fluxes in a wide range of ocean CDR scenarios. The information would be provided to users via a public graphical user interface, and the results would be based on a new synthesis of scientific observations and knowledge in a global ocean mixed layer inverse model run on supercomputers. The proposed tool aims to fill a unique and valuable niche for users and the scientific community by balancing tradeoffs between user interests, scientific knowledge, and technical capabilities. We will conclude by visioning the potential implications for a global carbon monitoring system in a world with gigaton scale ocean CDR.

Global↗

Essay: Quantum Sensing with Atomic, Molecular, and Optical Platforms for Fundamental Physics

Atomic, molecular, and optical (AMO) physics has been at the forefront of the development of quantum science while laying the foundation for modern technology. With the growing capabilities of quantum control of many atoms for engineered many-body states and quantum entanglement, a key question emerges: what critical impact will the second quantum revolution with ubiquitous applications of entanglement bring to bear on fundamental physics? In this Essay, we argue that a compelling long-term vision for fundamental physics and novel applications is to harness the rapid development of quantum information science to define and advance the frontiers of measurement physics, with strong potential for fundamental discoveries. As quantum technologies, such as fault-tolerant quantum computing and entangled quantum sensor networks, become much more advanced than today’s realization, we wonder what doors of basic science can these tools unlock. We anticipate that some of the most intriguing and challenging problems, such as quantum aspects of gravity, fundamental symmetries, or new physics beyond the minimal standard model, will be tackled at the emerging quantum measurement frontier. . Published by the American Physical Society 2024

Physics↗

LANL Critical Benchmark Comparison Study and Subsequent Revision

As part of an international collaboration within the DOE Nuclear Criticality Safety Program (NCSP), LANL is involved in a comparison study to quantify differences in k-effective results from neutron transport simulations of critical benchmark experiments. The DOE NCSP Mission and Vision details the activity in which the French Institut De Radioprotection et De Sûreté Nucléaire (IRSN) leads the study with LANL and in conjunction with ORNL and LLNL to compare results of various neutron transport codes and nuclear data libraries to compute k-effective for ICSBEP benchmarks held in common by the entities. This report documents results obtained through partial completion of the overall effort with a focus on the changes made to LANL benchmarks modeled with MCNP6 using ENDF/B-VII.1 nuclear data that appeared to have discrepant results when compared with results of other codes. The feedback received through participation in the comparison collaboration has prompted an effort to review particular input files for benchmarks and revise when necessary. This report documents the results of review and revision of specific benchmarks highlighted as possibly discrepant in the comparison study. In addition, this effort prompted a new collaboration between LANL XCP and NCS Divisions in the development of a shared review/revision procedure and use of a new benchmark repository.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Analysis of Performance of Stereoscopic-Vision Software

A team of JPL researchers has analyzed stereoscopic vision software and produced a document describing its performance. This software is of the type used in maneuvering exploratory robotic vehicles on Martian terrain. The software in question utilizes correlations between portions of the images recorded by two electronic cameras to compute stereoscopic disparities, which, in conjunction with camera models, are used in computing distances to terrain points to be included in constructing a three-dimensional model of the terrain. The analysis included effects of correlation- window size, a pyramidal image down-sampling scheme, vertical misalignment, focus, maximum disparity, stereo baseline, and range ripples. Contributions of sub-pixel interpolation, vertical misalignment, and foreshortening to stereo correlation error were examined theoretically and experimentally. It was found that camera-calibration inaccuracy contributes to both down-range and cross-range error but stereo correlation error affects only the down-range error. Experimental data for quantifying the stereo disparity error were obtained by use of reflective metrological targets taped to corners of bricks placed at known positions relative to the cameras. For the particular 1,024-by-768-pixel cameras of the system analyzed, the standard deviation of the down-range disparity error was found to be 0.32 pixel.

Kim, Won↗