Augmented Reality Tool for Operationally Relevant 3D Sensorimotor Assessments
Explore the source record for details and available documents.
Sarah Moudy↗
SEARCH · Engineering Papers
Results for “augmented reality”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Inside the ice shelf: using augmented reality to visualise 3D lidar and radar data of Antarctica
Explore the source record for details and available documents.
Alexandra L. Boghosian↗
Immersive Analytics in Critical Spatial Domains: From Materials to Energy Systems
Immersive analytics (IA) leverages virtual reality, augmented reality, and mixed reality to transform how users interact with complex datasets across domains such as science, industry, and education. These immersive technologies offer spatial and multimodal environments that foster intuitive exploration, but they also introduce challenges related to cognitive load, interface design, and system performance. Here, this article presents a comprehensive review of visualization techniques, interaction models, and multimodal inputs utilized in IA. Drawing on case studies in scientific visualization, industrial training, and educational communication, we examine both the potential and limitations of current systems. Finally, we propose future research directions, focusing on real‐time collaboration, adaptive user interfaces, and scalable data exploration strategies to advance the field.
99 - GENERAL AND MISCELLANEOUS↗
Developing Extended Reality Projects in Support of Design, Fabrication and Procedure
The goal of this internship was to improve and create virtual reality simulations and visualizations for use in parallel with the design, fabrication, and analysis of flight ready hardware for areas like the Environmental Control and Life Support Systems (ECLSS) and also Space Systems. Specifically, my work was done in the XRSpace lab at Marshall Space Flight Center (MSFC) with assistance directly and indirectly from workers at KSC, JSC and LaRC. Led by David Reynolds, the XRSpace lab develops products for various entities at NASA. The work done in the XRSpace lab focuses on Extended Reality (XR) solutions for both simulations and visualization capabilities. The goal of the lab is to support the larger systems of NASA and to help find ways that XR technologies can streamline and optimize the design process. Extended Reality is an umbrella term that encompasses Mixed Reality, Augmented Reality, and Virtual Reality. In this capacity, I was able to complete several elements in the design, building, testing, and deployment for a variety of immersive experiences, including a VR procedure simulation, visualization aids, and a 360-image capture tool.
Joseph Chambers↗
Human Systems Intergration & Engineering: HSI & E Design Influence
The design of space systems is a long and complicated process where changes to the design increase cost and schedule as the design progresses into implementation. Human Systems Integration uses tools such as mockups, Virtual Reality, Augmented Reality and human simulations early in the design process to evaluate designs for usability, operability, and maintainability. This early design influence provides cost and schedule savings during manufacturing and operations. Examples of how MSFC EV74 Human Systems Integration Team have used these tools to evaluate hardware designs will be discussed.
Tanya Andrews↗
An Evaluation of Extended Reality Technologies for Use in Verification Testing at NASA 2024 HRP IWS Abstract
BACKGROUND At NASA, verification testing is the formal process of ensuring that a product conforms to requirements set by a project or program. Some verification methods, such as Demonstrations and Test, require either the end product or a mockup of the product with sufficient fidelity to stand-in for the product during the test. Traditionally, these mockups have been physical (e.g., foam-core and wood) but there is growing interest in exploring new methods for testing with these mockups. These methods include virtual reality (VR), mixed reality, and augmented reality which are collectively referred to as eXtended Reality (XR) technologies. VR has already been adopted and used by many in the aerospace industry as a tool for use in early design phases (e.g., developmental testing) and may have the most potential for use in verification tests. Benefits of using VR mockups offer cost effectiveness, ease of iteration, simulation of hazardous conditions (e.g., an egress through a hatch with smoke obscuring vision), and the ability to simulate microgravity conditions, which are challenging to do with physical mockups. However, the validity of test results obtained from VR mockup demonstrations or testing, compared to the current gold standard of physical mockups, remains uncertain. It is unlikely that there is one clean answer as there are many different types of verification outcomes and each XR technology must be evaluated on its own merits. This is not an issue during developmental testing as the design is still in flux and the total success of the design is not dependent upon the results of a developmental test. Verification tests, however, only happen once, assuming no change to the design, and the results are used to certify the product. Therefore, establishing the validity of XR mockup-based verification outcomes is essential before considering them for any use in verification tests. OBJECTIVE AND METHOD To address this concern, the Human Research Program has funded a project to explore and qualify how XR technologies might be used in verification demonstration and testing at NASA. Currently, we are conducting a review of the literature on the utilization of XR mockups for design activities, prototyping, and user testing. We are employing the Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis method to identify the pros, cons, and barriers to adoption of XR technologies for verification testing at NASA. Additionally, we are developing a framework to guide the deployment of XR mockups for verification tests. Building upon available evidence from the literature and subject-matter expert feedback, our goal for the framework is to provide guidelines for which forms of XR mockups are suitable for a given verification test, when only physical mockups should be employed and to highlight areas for which more evidence is needed. To further refine our framework and to contribute to the body of evidence, we are planning a lab-based experiment comparing a VR mockup to a physical twin for a set of select verification outcomes. ANTICIPATED RESULTS In this presentation, we will present the work we conducted to evaluate XR technologies for use in verification tests at NASA. We will summarize and report our findings from the SWOT analysis and our lab-based study, and we will present the current state of the XR Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project. Technologies for Verification Testing framework. We will conclude by summarizing remaining work and future directions for the project.
Extended Reality↗
Seeing the Future of Spaceflight: Applications of Extended Reality (XR) Technologies
INTRODUCTION Advances in Earth Independent Medical Operations (EIMO) focused capabilities will be crucial for completing successful beyond low-earth-orbit (LEO) missions, where traditional Earth-based medical support and near real-time communication are limited, intermittent, or absent. In such restricted or limited communication environments, emerging extended reality (ER or XR) technologies may play a vital role in ensuring crew health and safety. Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) are common terms used to describe technologies that "generate or modify" while “extended reality” (ER or XR) is often used as an umbrella term for these technologies. This review explores how such cutting-edge “extended reality” information delivery systems are currently applied across various disciplines within NASA. The review focuses on their capabilities, limitations, and prospective future applications as they pertain to Earth-Independent Medical Operations and its components of pre-mission planning, medical decision making, resource management, and task load management. METHODS/DESIGN The review began with a literature search to identify relevant disciplines (e.g., medical education, austere environments) where XR technologies have been studied to highlight areas where evidence is lacking. Previous and current NASA XR projects and applications, as well as commercial partnerships, such as those with Small Business Innovation Research (SBIR), were identified. Discussions with technical experts involved in the design, development, and application of these technologies allowed for classification based on their respective spaceflight applications. RESULTS AND CONCLUSION This review highlights current and emerging XR technologies that have been developed, tested, and implemented both terrestrially and in spaceflight by NASA, its international partners, the U.S. Armed Forces, and various relevant commercial entities. While these novel information delivery systems vary in their respective technology readiness levels (TRL), they have the potential to be implemented on Earth, in LEO, and during deep space exploration. These systems have the potential for impacting and optimizing many aspects of EIMO.
Yevgeniy Zhivotovskiy↗
Augmented Virtual Reality Laboratory
Real time motion tracking hardware has for the most part been cost prohibitive for research to regularly take place until recently. With the release of the Microsoft Kinect in November 2010, researchers now have access to a device that for a few hundred dollars is capable of providing redgreenblue (RGB), depth, and skeleton data. It is also capable of tracking multiple people in real time. For its original intended purposes, i.e. gaming, being used with the Xbox 360 and eventually Xbox One, it performs quite well. However, researchers soon found that although the sensor is versatile, it has limitations in real world applications. I was brought aboard this summer by William Little in the Augmented Virtual Reality (AVR) Lab at Kennedy Space Center to find solutions to these limitations.
Resolved Skeleton↗
RMIT AR Android Application Abstract
This summer, I've worked as an intern for IT-C1's Rapid Model Import Tool (RMIT) project under Bill Little. The goal of this project was to develop a mobile application for IT-C1's larger project, RMIT, so that it could be deployed in a convenient way. The current function of RMIT is to import file types that are not normally compatible with the Unity 3D software development engine and make them into files that can be placed into Unity for the specific purpose of viewing CAD models in virtual reality (VR). The mobile application that I worked on uses augmented reality (AR), using the camera on a mobile device, to show virtual models in the real world for the purpose of visualization and understanding. In addition to single-player capabilities, this application will include a multi-player mode so that collaborating engineers can see each other's work on their own devices.Working on this application required many steps. First, I hard to learn a lot about how mobile development is different from desktop development, and what the limitations of augmented reality are at its current state. From there, I was assigned new tasks and goals every two weeks, which included addingground plane and air plane functionality, designing a user interface, adding multi-player capability and allowing the user to manipulate models placed in a scene.
Dorian Thompson↗
The Reality of Space: Exploring Virtual, Augmented, and Mixed Realities and the Future of Spaceflight
Explore the source record for details and available documents.
Yevgeniy Zhivotovskiy↗
Internship in Augmented and Virtual Reality - Rapid Model Import Tool
The integration of virtual and augmented reality, sometimes called mixed reality, is an emerging technology which will likely skyrocket overnight much in the way smartphones did a decade ago. Kennedy Space Center's Augmented and Virtual Reality (AVR) Lab is developing a Rapid Model Import Tool (RMIT) to create a quick and efficient way to bring NASA's complex engineering 3D models into virtual and augmented environments. The long-term objective is to create a tool that will ultimately benefit KSC engineers. Its various uses within NASA can potentially span from astronaut training, to marketing, to public outreach, to name a few. Unity is a prolific cross-platform game engine that allows users to build high quality 2D and 3D games for desktop, mobile, web, and game console platforms. It is perhaps also the most widely used software for virtual reality game development. At the AVR lab, we are looking at alternative uses of Unity to build tools for NASA engineers to perform design, development, testing, and training on spacecraft, rocket delivery systems, ground support equipment, and facilities at KSC. As an intern for the RMIT project, I am charged with the task of performing research on Unity-compatible file types to develop an efficient, affordable, preservative process to bring models from CATIA 3D engineering software into the Unity environment. With a tool called the NASA Enterprise Visualization Application (NEVA), developed by the Boeing Design Visualization group at KSC, we are able to easily convert CATIA's design models to. DAE (also known as COLLADA) and .OBJ file formats. I first reduce the polygon count of the model within CATIA itself, make any necessary tweaks to reduce the model further, and then export using NEVA. The .OBJ or. DAE files that I am left with are then converted by another intern to a Unity-compatible file format using a custom Python script. I have generated extensive documentation of this process in a NEVA User Guide. By the end of this semester, we will have built a solid framework for RMIT based on a thorough understanding of virtual reality specifications and file requirements, allowing future software development teams to go forward with development on the custom tool.
Leap Motion↗
Human-Robot Interaction
Human-robot interaction (HRI) is a discipline investigating the factors affecting the interactions between humans and robots. It is important to evaluate how the design of interfaces affect the human's ability to perform tasks effectively and efficiently when working with a robot. By understanding the effects of interface design on human performance, workload, and situation awareness, interfaces can be developed to appropriately support the human in performing tasks with minimal errors and with appropriate interaction time and effort. Thus, the results of research on human-robot interfaces have direct implications for the design of robotic systems. For efficient and effective remote navigation of a rover, a human operator needs to be aware of the robot's environment. However, during teleoperation, operators may get information about the environment only through a robot's front-mounted camera causing a keyhole effect. The keyhole effect reduces situation awareness which may manifest in navigation issues such as higher number of collisions, missing critical aspects of the environment, or reduced speed. One way to compensate for the keyhole effect and the ambiguities operators experience when they teleoperate a robot is adding multiple cameras and including the robot chassis in the camera view. Augmented reality, such as overlays, can also enhance the way a person sees objects in the environment or in camera views by making them more visible. Scenes can be augmented with integrated telemetry, procedures, or map information. Furthermore, the addition of an exocentric (i.e., third-person) field of view from a camera placed in the robot's environment may provide operators with the additional information needed to gain spatial awareness of the robot. Two research studies investigated possible mitigation approaches to address the keyhole effect: 1) combining the inclusion of the robot chassis in the camera view with augmented reality overlays, and 2) modifying the camera frame of reference. The first study investigated the effects of inclusion and exclusion of the robot chassis along with superimposing a simple arrow overlay onto the video feed of operator task performance during teleoperation of a mobile robot in a driving task. In this study, the front half of the robot chassis was made visible through the use of three cameras, two side-facing and one forward-facing. The purpose of the second study was to compare operator performance when teleoperating a robot from an egocentric-only and combined (egocentric plus exocentric camera) view. Camera view parameters that are found to be beneficial in these laboratory experiments can be implemented on NASA rovers and tested in a real-world driving and navigation scenario on-site at the Johnson Space Center.
Sandor, Aniko↗
The Next Wave: Humans, Computers, and Redefining Reality
The Augmented/Virtual Reality (AVR) Lab at KSC is dedicated to " exploration into the growing computer fields of Extended Reality and the Natural User Interface (it is) a proving ground for new technologies that can be integrated into future NASA projects and programs." The topics of Human Computer Interface, Human Computer Interaction, Augmented Reality, Virtual Reality, and Mixed Reality are defined; examples of work being done in these fields in the AVR Lab are given. Current new and future work in Computer Vision, Speech Recognition, and Artificial Intelligence are also outlined.
Extended Reality↗
Visual Stability of Objects and Environments Viewed through Head-Mounted Displays
Virtual Environments (aka Virtual Reality) is again catching the public imagination and a number of startups (e.g. Oculus) and even not-so-startup companies (e.g. Microsoft) are trying to develop display systems to capitalize on this renewed interest. All acknowledge that this time they will get it right by providing the required dynamic fidelity, visual quality, and interesting content for the concept of VR to take off and change the world in ways it failed to do so in past incarnations. Some of the surprisingly long historical background of the technology that the form of direct simulation that underlies virtual environment and augmented reality displays will be briefly reviewed. An example of a mid 1990's augmented reality display system with good dynamic performance from our lab will be used to illustrate some of the underlying phenomena and technology concerning visual stability of virtual environments and objects during movement. In conclusion some idealized performance characteristics for a reference system will be proposed. Interestingly, many systems more or less on the market now may actually meet many of these proposed technical requirements. This observation leads to the conclusion that the current success of the IT firms trying to commercialize the technology will depend on the hidden costs of using the systems as well as the development of interesting and compelling content.
virtual environment↗
Usability Evaluation of a Flight-Deck Airflow Hazard Visualization System
Many aircraft accidents each year are caused by encounters with unseen airflow hazards near the ground, such as vortices, downdrafts, low level wind shear, microbursts, or turbulence from surrounding vegetation or structures near the landing site. These hazards can be dangerous even to airliners; there have been hundreds of fatalities in the United States in the last two decades attributable to airliner encounters with microbursts and low level wind shear alone. However, helicopters are especially vulnerable to airflow hazards because they often have to operate in confined spaces and under operationally stressful conditions (such as emergency search and rescue, military or shipboard operations). Providing helicopter pilots with an augmented-reality display visualizing local airflow hazards may be of significant benefit. However, the form such a visualization might take, and whether it does indeed provide a benefit, had not been studied before our experiment. We recruited experienced military and civilian helicopter pilots for a preliminary usability study to evaluate a prototype augmented-reality visualization system. The study had two goals: first, to assess the efficacy of presenting airflow data in flight; and second, to obtain expert feedback on sample presentations of hazard indicators to refine our design choices. The study addressed the optimal way to provide critical safety information to the pilot, what level of detail to provide, whether to display specific aerodynamic causes or potential effects only, and how to safely and effectively shift the locus of attention during a high-workload task. Three-dimensional visual cues, with varying shape, color, transparency, texture, depth cueing, and use of motion, depicting regions of hazardous airflow, were developed and presented to the pilots. The study results indicated that such a visualization system could be of significant value in improving safety during critical takeoff and landing operations, and also gave clear indications of the best design choices in producing the hazard visual cues.
Aragon, Cecilia R.↗
Visualizing an Exascale Data Center Digital Twin: Considerations, Challenges and Opportunities
Digital twins are an excellent tool to model, visualize, and simulate complex systems, to understand and optimize their operation. In this work, we present the technical challenges of real-time visualization of a digital twin of the Frontier supercomputer.We show the initial prototype and current state of the twin and highlight technical design challenges of visualizing such a large High Performance Computing (HPC) system. The goal is to understand the use of augmented reality as a primary way to extract information and collaborate on digital twins of complex systems. This leverages the spatio-temporal aspect of a 3D representation of a digital twin, with the ability to view historical and real-time telemetry, triggering simulations of a system state and viewing the results, which can be augmented via dashboards for details. Finally, we discuss considerations and opportunities for augmented reality of digital twins of large-scale, parallel computers.
Maiterth, Matthias↗
Time Counts! Some Comments on System Latency in Head-Referenced Displays
System response latency is a prominent characteristic of human-computer interaction. Laggy systems are; however, not simply annoying but substantially reduce user productivity. The impact of latency on head referenced display systems, particularly head-mounted systems, is especially disturbing since not only can it interfere with dynamic registration in augmented reality displays but it also can in some cases indirectly contribute to motion sickness. We will summarize several experiments using standard psychophysical discrimination techniques that suggest what system latencies will be required to achieve perceptual stability for spatially referenced computer-generated imagery. In conclusion I will speculate about other system performance characteristics that I would hope to have for a dream augmented reality system.
human-computer interaction↗
Situated Visualization of Photovoltaic Module Performance for Workforce Development
The rapid growth of the solar energy industry requires advanced educational tools to train the next generation of engineers and technicians. We present a novel system for situated visualization of photovoltaic (PV) module performance, leveraging a combination of PV simulation, sun-sky position, and head-mounted augmented reality (AR). Our system is guided by four principles of development: simplicity, adaptability, collaboration, and maintainability, realized in six components. Users interactively manipulate a physical module's orientation and shading referents with immediate feedback on the module's performance.
augmented reality↗