Results for “Reality” — Page 5

NASA NTRSDate not supplied

Lunar Instrument Data Integration Into the Virtual Reality Mission Simulation System for Decision Communication and Situational Awareness

In situ resource utilization (ISRU) technologies are a key advancement required to make human habitation on the Moon and Mars viable. The upcoming Volatiles Investigating Polar Exploration Rover (VIPER) mission will provide crucial correlations between volatiles and lunar geology to understand the water content available for ISRU on the moon. The mission will require the coordination of multi-disciplinary teams across the country making real time decisions based on rover instrument data. The virtual reality Mission Simulation System (vMSS) is a virtual reality platform designed at MIT by the Resource Exploration and Science of our Cosmic Environment (RESOURCE) team to provide teams with a collaboration interface for planetary missions like VIPER. Herein we determine the integration pathway for analog based datasets that are examples of VIPER's two main instruments, the near-infrared volatile spectrometer subsystem (NIRVSS) and the neutron spectrometer subsystem (NSS), into vMSS to provide the most valuable visualization tools. Focusing on improving situational awareness, decision making, reducing task load and incorporating comments from scientists working previous analogs and on the current VIPER mission, we recommend critical elements to implement into vMSS and the best approaches for data visualization. We present a review of relevant analogs and state of the art mission software. We have developed a design concept and path to flight of analysed instrument data integrated with data maps that allow for virtual manipulation and annotation between non-co-located team members. We focus on pre-mission mapping of a priori data for improved situational awareness, layering of analysed instrument data, correlative mapping and interactive capabilities for in-mission decision making, as well as archiving and annotation tools for post-mission analysis. Finally, we lay out the roadmap for the future development of immersive sample site visualization capabilities and the use of integrated instrument data in vMSS with automated temporal and geospatial planning.

Cody Alison Paige↗

NASA NTRS2023

Trades, Architecture, and Design of the Joint Augmented Reality Visual Informatics System (Joint AR) Product

Future expeditions will enable exploration and study of the planetary surfaces of the Moon and Mars by performing extravehicular activity (EVA) operations. Present-day International Space Station (ISS) EVA operations require an intricate choreography of crew, space suits, tools, systems, and flight teams to plan, train, and execute with limited advanced informatics. In this paper, the Joint Augmented Reality Visual Informatics System (Joint AR) project team at NASA Johnson Space Center (JSC) characterizes the design space for developing a modular augmented reality (AR) device for a spacesuit form factor that can support crew decision-making for EVA. The Joint AR product was defined via trade studies and market analysis of previous EVA display efforts, various AR components such as optics, commercial AR systems, light engines, data interfaces, and graphics engine software. This paper outlines the defining architectural design decisions, including safety criticality considerations, interfaces, and computer architectures. The outcomes of these studies result in a prototype design which is defined here as the Joint AR product. This work aims to enable a community-wide discussion toward realizing necessary suit-compatible AR features and capabilities for future missions.

Paromita Mitra↗

NASA NTRS2023

Trades, Architecture, and Design of the Joint Augmented Reality Visual Informatics System (Joint AR) Product

Future expeditions will enable exploration and study of the planetary surfaces of the Moon and Mars by performing extravehicular activity (EVA) operations. Present-day International Space Station (ISS) EVA operations require an intricate choreography of crew, space suits, tools, systems, and flight teams to plan, train, and execute with limited advanced informatics. In this paper, the Joint Augmented Reality Visual Informatics System (Joint AR) project team at NASA Johnson Space Center (JSC) characterizes the design space for developing a modular augmented reality (AR) device for a spacesuit form factor that can support crew decision-making for EVA. The Joint AR product was defined via trade studies and market analysis of previous EVA display efforts, various AR components such as optics, commercial AR systems, light engines, data interfaces, and graphics engine software. This paper outlines the defining architectural design decisions, including safety criticality considerations, interfaces, and computer architectures. The outcomes of these studies result in a prototype design which is defined here as the Joint AR product. This work aims to enable a community-wide discussion toward realizing necessary suit-compatible AR features and capabilities for future missions.

Paromita Mitra↗

NASA NTRS2023

Joint Augmented Reality Visual Informatics System: Concept of Operations

NASA proposed requirements for a digital display for an EVA spacesuit to provide relevant information to the crew member. The Joint Augmented Reality Visual Informatics System (Joint AR) project pursued four years of research and development towards a suit-display system in a near-eye, AR form factor. The project was responsible for developing software (custom graphics engine and core flight software), physical hardware prototyping (controls, projection display optics, suited display platform), virtual prototyping platform (a virtual reality testbed), and human-in-the-loop (HITL) operational testing informed by EVA flight controllers, crew members, and human factors engineers for con-ops definition. This document contains substantial updates to CTSD-ADV-1788 Rev. Basic. This revision was produced by the project to summarize the use-cases and and user experiences developed throughout the project, and refine the Basic revision originally drafted at the beginning of the project life cycle. The primary purpose of this document is to summarize and make available the scenario development efforts that have been pursued and explored within the Joint AR project. This includes descriptions of the scenarios themselves as well as corresponding potential of advanced informatics displays to support those specified scenarios. In doing so, this document provides a variety of approaches to deconstruct and hypothesize how future technological capabilities so that with future EVA work demands can be satisfied within future human planetary spaceflight missions.

Matthew Miller↗

NASA NTRSDate not supplied

Development of an Extended Reality (XR) Tool for Earth Science Visualization

In this presentation, we will discuss our work in adapting the NASA open source XR software, the Mixed Reality Exploration Toolkit (MRET), to an earth science domain. MRET is a NASA open source XR software for rapidly building extended reality (XR) environments for NASA domain problems, e.g., pulling in CAD models of thermal vac chamber and Roman Space Telescope to do fit checks. Primarily used for hardware integration & test, we have been adapting and extending MRET for science problems. Traditionally, scientists view and analyze the result of calculated or measured observables with static 1-D, 2-D or 3-D plots. It can be difficult to identify, track and understand the evolution of key features due to poor viewing angles and the nature of flat computer screens. Additionally, numerical models, such as the NASA GEOS climate model, are almost exclusively formulated and analyzed on Eulerian grids with points fixed in space and time. However, atmospheric phenomena such as convective clouds, hurricanes and wildfire smoke plumes move with the 3-D flow field, and it is often difficult and unnatural to understand these phenomena in an Eulerian reference frame as opposed to the Lagrangian reference frame in which nature operates. As part of an Earth Science Technology Office (ESTO) proposal, we have been adapting MRET to be a scientific exploration and analysis XR tool with integrated Lagrangian Dynamics (LD) for the Goddard Earth Observing System (GEOS) numerical weather prediction model. We believe this will help scientists identify, track, and understand the evolution of Earth Science phenomena. This presentation will discuss current results in our work in developing this XR tool for Earth Science.

Thomas G Grubb↗

NASA NTRSDate not supplied

Development of an Extended Reality (XR) Tool for Earth Science Visualization

In this presentation, we will discuss our work in adapting the NASA open source XR software, the Mixed Reality Exploration Toolkit (MRET), to an earth science domain. MRET is a NASA open source XR software for rapidly building extended reality (XR) environments for NASA domain problems, e.g., pulling in CAD models of thermal vac chamber and Roman Space Telescope to do fit checks. Primarily used for hardware integration & test, we have been adapting and extending MRET for science problems. Traditionally, scientists view and analyze the result of calculated or measured observables with static 1-D, 2-D or 3-D plots. It can be difficult to identify, track and understand the evolution of key features due to poor viewing angles and the nature of flat computer screens. Additionally, numerical models, such as the NASA GEOS climate model, are almost exclusively formulated and analyzed on Eulerian grids with points fixed in space and time. However, atmospheric phenomena such as convective clouds, hurricanes and wildfire smoke plumes move with the 3-D flow field, and it is often difficult and unnatural to understand these phenomena in an Eulerian reference frame as opposed to the Lagrangian reference frame in which nature operates. As part of an Earth Science Technology Office (ESTO) proposal, we have been adapting MRET to be a scientific exploration and analysis XR tool with integrated Lagrangian Dynamics (LD) for the Goddard Earth Observing System (GEOS) numerical weather prediction model. We believe this will help scientists identify, track, and understand the evolution of Earth Science phenomena. This presentation will discuss current results in our work in developing this XR tool for Earth Science.

Thomas Grubb↗

NASA NTRS1999

Computer Vision Assisted Virtual Reality Calibration

A computer vision assisted semi-automatic virtual reality (VR) calibration technology has been developed that can accurately match a virtual environment of graphically simulated three-dimensional (3-D) models to the video images of the real task environment.

Computer Vision Virtual Reality Calibration camera↗

NASA NTRS1995

A Commercial Implementation of the JPL Virtual Reality Calibration Technology for Telerobotics

There is a growing interest in the virtual reality (VR) calibration technique of matching graphically.

JPL Virtual Reality↗

NASA NTRS1994

Virtual Reality Calibration for Telerobotic Servicing

A virtual reality calibration technique of matching a virtual environment of simulated graphics models in 3-D geometry and perspective with actual camera views of the remote site task environment has been developed to enable high-fidelity preview/predictive displays with calibrated graphics overlay on live video.

error measurements↗

NASA NTRS2017

Holographic Rovers: Augmented Reality and the Microsoft HoloLens

Augmented Reality is an emerging field in technology, and encompasses Head Mounted Displays, smartphone apps, and even projected images. HMDs include the Meta 2, Magic Leap, Avegant Light Field, and the Microsoft HoloLens, which is evaluated specifically. The Microsoft HoloLens is designed to be used as an AR personal computer, and is being optimized with that goal in mind. Microsoft allied with the Unity3D game engine to create an SDK for interested application developers that can be used in the Unity environment.

augmented reality↗

NASA NTRS2017

Object and Facial Recognition in Augmented and Virtual Reality: Investigation into Software, Hardware and Potential Uses

As augmented and virtual reality grows in popularity, and more researchers focus on its development, other fields of technology have grown in the hopes of integrating with the up-and-coming hardware currently on the market. Namely, there has been a focus on how to make an intuitive, hands-free human-computer interaction (HCI) utilizing AR and VR that allows users to control their technology with little to no physical interaction with hardware. Computer vision, which is utilized in devices such as the Microsoft Kinect, webcams and other similar hardware has shown potential in assisting with the development of a HCI system that requires next to no human interaction with computing hardware and software. Object and facial recognition are two subsets of computer vision, both of which can be applied to HCI systems in the fields of medicine, security, industrial development and other similar areas.

object recognition↗

NASA NTRS2019

NASA KSC Intern Final Report - Virtual Reality in STEM Engagement

For this internship, I was a part of the multi-center NextGen STEM (Science, Technology, Engineering, and Mathematics) pilot. Specifically, I worked with the Developing Commercial Crew Program (CCP) Capabilities team that focuses on human spaceflight to the space station with NASA’s commercial partners. Composed of NASA employees and contractors (who are connected to KSC, JSC, and LaRC), they work to ensure that students will experience traditional classroom content and be immersed in emerging technologies in order to explore Commercial Crew missions and launch facilities. This group is in the process of creating educational products and resources to inspire the next generation to pursue STEM and to allow educators to have access to unique and enticing STEM activities. These products include: age-appropriate classroom lessons (including activity sheets and challenges) for kindergarten through twelfth grade; a CCP app and corresponding guide; learning experiences for students as well as educators; and virtual field trips and tours using NASA-created Virtual Reality (VR) videos and a custom VR app. My particular assignment was managing the VR equipment, developing an understanding of how to maximize the use of the equipment, creating documents that explain and convey the operations and procedures concerning the equipment (including lessons learned from equipment use), and training others to utilize the equipment effectively and correctly for conference events and group demonstrations. Over the course of this internship, I helped my team outline and learn essential procedures of operation for the VR equipment, solved technology problems that arose (due to the nature of creating unique products), and encouraged audiences at KSC and various conferences to consider using the CCP VR videos in their classrooms as an opportunity to engage and challenge the next generation to pursue STEM careers.

STEM↗

NASA NTRS1995

Virtual Reality Calibration and Preview/Predictive Displays for Telerobotics

A virtual reality (VR) calibration technique of matching a virtual environment of simulated 3-D graphic models with actual camera views of the remote site task environment has been developed. This VR calibration enables high-fidelity preview/predictive displays with calibrated graphics overlay on live video. Reliable and accurate calibration is achieved by operator-interactive camera calibration and object localization procedures based on new linear/nonlinear least-squares algorithms that can handle multiple-camera views.

Virtual↗

NASA NTRS1994

Using Virtual Reality for Science Mission Planning: A Mars Pathfinder Case

The Mars Pathfinder Project GDS Team has designed software that integrates virtual reality technology with existing JPL Navigational Ancillary Information Facilities (NAIF) and image processing capabilities. The result is an interactive, workstation-based application software that provides a high resolution 3-dimensional stereo display of Mars as if it were viewed through the lander camera. This paper describes the architecture and characteristics of this science mission planning software and discusses possible uses of this software by other planetary missions.

virtual↗

NASA NTRS1994

Commercialization of JPL Virtual Reality Calibration and Redundant Manipulator Control Technologies

Within NASA's recent thrust for industrial collaboration, JPL has recently established two technology cooperation agreements in the robotics area: one on virtual reality calibration with Deneb Robotics, Inc., and the other on redundant manipulator control with Robotics Research Corporation (RRC). These technology transfer tasks will enable both Deneb and RRC to commercialize an upgraded version of their products that will greatly benefit both space and terrestrial telerobotic applications.

Robotics↗

NASA NTRS1997

NASA's Wireless Augmented Reality Prototype (WARP)

The objective of Wireless Augmented Reality Prototype (WARP) effort is to develop and integrate advanced technologies for real-time personal display of information relevant to the health and safety of space station/shuttle personnel.

Wireless↗

NASA NTRSDate not supplied

Arm DevSummit Keynote: Environment for Data Engineering in Virtual Reality: Ethical Considerations

Currently, the US Government is going through a large-scale data Transformation effort, where the GSA playbook is guiding all agencies to make their processes data-driven with the help of emerging technologies. To address concerns about algorithm sharing, AI adoption, and vendor lock-in, the NASA Langley Research Center Digital Transformation Group has developed the Environment for Data Engineering in Virtual Reality (EnDEVR), a data science ecosystem that allows users to command and investigate customizable data analyses from a VR environment. The system has been evaluated in the Oculus Rift S and Quest environments, two popular VR systems powered by the ARM architecture. Current and future development will employ several AI capabilities to guide research and automation within the environment and to facilitate algorithm sharing. In this talk, we will discuss the results of an initial ethical investigation and recommended considerations for the use of AI within the system.

Artificial Intelligence↗

NASA NTRS2024

Enhancing Metal Additive Manufacturing Training with the Advanced Vision Language Model: A Pathway to Immersive Augmented Reality Training for Non-Experts

This paper introduces an innovative training system for the Renishaw AM400 metal printer, leveraging the synergy of the advanced Vision Language Model (VLM) with Augmented Reality (AR) within the Digital Twins (DT) framework. Aimed at overcoming the limitations of conventional training methods in metal additive manufacturing (AM), our system integrates AR to provide an immersive learning environment, enhancing the real-world experience with interactive digital overlays. The core of the system lies in its use of VLM, which, pre-trained on diverse datasets, excels in processing multi-modal data, thereby offering nuanced and contextually relevant guidance for trainees. Key experiments demonstrate the system’s effectiveness, particularly highlighting the usage of VLM as an Artificial Intelligence (AI) agent to integrate external tools like YOLO-v7 for valve state classification and CRAFT for control panel text recognition. This approach significantly improves recognition accuracy, operational understanding, and human–machine interaction, especially for non-expert users, making complex metal AM operations more accessible. The research not only showcases the potential of AR and VLM in industrial training but also sets a new standard for smart manufacturing practices, indicating broader applications in various industrial domains.

Metal additive manufacturing↗