Seeing The Future of Spaceflight: Applications of Extended Reality (XR) Technologies
Explore the source record for details and available documents.
Yevgeniy Zhivotovskiy↗
SEARCH · Engineering Papers
Results for “extended reality”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The Future of Visualization Technology Vis in Practice
Two decades ago, we witnessed an inflection point as graphics programming became commoditized. A comparable inflection point may be on the horizon, this time driven by the commoditization of immersive display technology with Augmented Reality (AR), Virtual Reality (VR), and Extended Reality (XR) technologies poised to redefine our interaction with data, environments, and collaboration. To drive the technology and its application, we should consider the lessons learned from successful large-scale immersive installations. Where these installations have thrived is by enhancing traditional workflows rather than entirely replacing them, supporting improved judgments of complex spaces, direct interaction in 3D, and embedding high-dimensional data.
AR/VR↗
Using XR for Improving Scientific Discovery With Numerical Weather Models
Our work explores the use of extended reality (XR) to im- prove scientific discovery with numerical weather/climate models that inform Earth science digital twins, specifically the NASA Goddard Earth Observing System (GEOS) global atmospheric model. The overall project is named the Vi- sualization And Lagrangian dynamics Immersive eXtended Reality Toolkit (VALIXR), which has two main areas of focus: (1) enhancing the understanding of and interaction with model output data through advanced visualizations in the XR environment, and (2) the integration of Lagrangian dynamics into the GEOS model, which allows a natural, feature-specific analysis of Earth science phenomena as op- posed to traditional, fixed-point Eulerian dynamics. Here, we report initial work on these focus areas.
Thomas Grubb↗
The Next Wave: Humans, Computers, and Redefining Reality
The Augmented/Virtual Reality (AVR) Lab at KSC is dedicated to " exploration into the growing computer fields of Extended Reality and the Natural User Interface (it is) a proving ground for new technologies that can be integrated into future NASA projects and programs." The topics of Human Computer Interface, Human Computer Interaction, Augmented Reality, Virtual Reality, and Mixed Reality are defined; examples of work being done in these fields in the AVR Lab are given. Current new and future work in Computer Vision, Speech Recognition, and Artificial Intelligence are also outlined.
Extended Reality↗
Immersive Digital Twin Laboratory for Engineering Education (CRADA Final Report)
This project aimed to create an immersive digital twin laboratory that incorporates advanced tracking and visualization capabilities. In collaboration with Fort Lewis College, NREL designed a state-of-the-art physical visualization laboratory, developed a software platform to enable interaction with tracked physical objects in the laboratory, and provided proof-of-concept curricula that included manipulating these tracked objects. The project was initiated to address the growing need for innovative educational tools in engineering education. As renewable energy systems, particularly solar installations, become more complex, there is a pressing need to bridge the gap between theoretical knowledge and practical application. Traditional methods of teaching solar engineering concepts often fall short of providing students with a comprehensive, hands-on understanding. This immersive digital twin laboratory was conceived to fill that gap by creating a safe, non-energized setting where students can interact with augmented solar installation objects, gaining valuable insights into system performance, design, and maintenance. The project utilized extended reality (XR) technologies, including head-mounted displays (HMDs) and a whole-room optical motion tracking system, to connect physical objects with their digital twins in real time. The laboratory was equipped with MagicLeap 2 HMDs, supported by a Vicon Vero 2.2 Optical Tracking System, which provided precise 6-degrees-of-freedom (6-DOF) tracking. We developed a software platform to manage the interaction between the tracked physical objects and their virtual counterparts, enabling real-time data synchronization, object recognition, and virtual overlays. We designed the system to be flexible and extendable, allowing for future integration of additional objects and curriculum. This research advances the field of engineering education by demonstrating the potential of immersive digital twin environments. The laboratory provides a dynamic learning space where students can experiment, collaborate, and learn without the risks associated with live experimentation. The ability to simulate and manipulate solar installation objects under various conditions has broad implications for workforce development, particularly in renewable energy. The project also highlights the economic feasibility of using XR technologies in educational settings, offering a cost-effective solution for institutions looking to enhance their curriculum. By fostering a deeper understanding of solar energy systems, this work contributes to the broader goal of supporting the global energy transition and preparing the next generation of engineers and technicians.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Interactive Quantum Chemistry Enabled by Machine Learning, Graphical Processing Units, and Cloud Computing
Modern quantum chemistry algorithms are increasingly able to accurately predict molecular properties that are useful for chemists in research and education. Despite this progress, performing such calculations is currently unattainable to the wider chemistry community, as they often require domain expertise, computer programming skills, and powerful computer hardware. In this review, we outline methods to eliminate these barriers using cutting-edge technologies. We discuss the ingredients needed to create accessible platforms that can compute quantum chemistry properties in real time, including graphical processing units–accelerated quantum chemistry in the cloud, artificial intelligence–driven natural molecule input methods, and extended reality visualization. We end by highlighting a series of exciting applications that assemble these components to create uniquely interactive platforms for computing and visualizing spectra, 3D structures, molecular orbitals, and many other chemical properties.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Early Fault Detection in Nuclear Systems: A Digital Engineering Approach
Nuclear energy systems present unique challenges in terms of ensuring safety, reliability, and efficiency during their design and operation. Early fault detection is critical for mitigating risks and fostering system resilience. However, current methods often fall short at identifying faults during early stages, potentially leading to costly delays and safety risks. The present work proposes a comprehensive digital engineering approach that leverages digital twins, digital threads, model-based systems engineering, artificial intelligence, and immersive extended reality to support early fault detection in nuclear systems. Through a series of case studies, we highlight specific gaps in the fault detection mechanisms of traditional nuclear design and operation processes, then demonstrate a suite of solutions we are working to implement to address these shortcomings in similar projects. Our findings suggest that a digital engineering approach to design and operation can significantly improve fault detection, ultimately leading to reductions in risk.
42 - ENGINEERING↗
Immersive Technologies for Human-in-the-Loop Lunar Surface Simulations
NASA, the National Aeronautics and Space Administration, continually seeks innovative solutions to enhance its operations, particularly in the realms of testing, evaluation, and training for future missions. Immersive technologies, such as virtual, augmented, and mixed reality have proven to be powerful tools for realistic, interactive, and engaging environments. This paper explores how the Simulation and Graphics Branch at NASA’s Johnson Space Center (JSC) leverages immersive technology, modern commercial rendering engines, and physics-based systems simulations to develop human-in-the-loop systems for humanity’s return to the Moon through the Artemis program. When NASA returns to the Moon, astronauts will travel to the Moon’s South Pole where lighting conditions will cause a more complex operational environment. Human-in-the-loop testing plays a crucial role in NASA's mission planning, spacecraft and space systems development, and evaluation of operational scenarios. The development of immersive environments such as a lunar rover mockup at a video wall enables engineers and astronauts to simulate and experience mission scenarios, integrated spacecraft systems, and operational procedures in a relevant environment before deployment. By integrating realistic virtual environments, immersive technology allows for the visualization and interaction with virtual spacecraft models, mission landscapes, and complex operational tasks. This approach helps identify potential design flaws, operational challenges, and safety considerations. It also provides valuable insights for risk reduction and helps improve mission efficiency and effectiveness. With advanced motion tracking systems and custom virtual environments data can be gathered and evaluated to help NASA refine training protocols, develop specialized training procedures and optimize human-robotic interactions for future space missions. Furthermore, immersive technology offers opportunities for future training initiatives at NASA. The Virtual Reality Laboratory at JSC has pioneered training with Virtual Reality (VR) since the Hubble Space Telescope repair missions in the early 1990’s. Extended Reality (XR) simulations enable astronauts to rehearse complex spacewalks, spacecraft maneuvers, and extravehicular activities in a safe and controlled environment. By replicating the physical and cognitive challenges of space missions, immersive training experiences enhance astronauts' situational awareness, decision-making abilities, and adaptability to unexpected scenarios. Additionally, immersive technology facilitates collaborative training, allowing geographically dispersed crew and mission control personnel to engage in synchronized simulations, fostering teamwork and effective communication. The adoption of immersive technology in NASA's testing, evaluation, and future training programs has yielded significant benefits. By incorporating human-in-the-loop testing for studies involving Extra Vehicular Activities (EVA), surface mobility and landing systems, NASA can identify and mitigate risks, optimize operational procedures, and enhance mission success. Ultimately, immersive training experiences can empower astronauts to better navigate the complexities of space missions, ensuring their safety, productivity, and success in the dynamic and challenging environments they will experience at the Lunar South Pole.
Simulation Modeling Virtual Reality Immersive Tech↗
Immersive Technologies for Human-in-the-Loop Lunar Surface Simulations
NASA, the National Aeronautics and Space Administration, continually seeks innovative solutions to enhance its operations, particularly in the realms of testing, evaluation, and training for future missions. Immersive technologies, such as virtual, augmented, and mixed reality have proven to be powerful tools for immersing users in realistic, interactive, and engaging environments. This paper explores how the Simulation and Graphics Branch at NASA’s Johnson Space Center (JSC) leverages immersive technology, modern commercial rendering engines, and physics-based systems simulations to develop human-in-the-loop systems for humanity’s return to the Moon through the Artemis program. When NASA returns to the Moon, astronauts will travel to the Moon’s South Pole where lighting conditions will cause a more complex operational environment. Human-in-the-loop simulations play a crucial role in NASA’s mission planning, spacecraft and space systems development, and evaluation of operational scenarios. The development of immersive environments such as a lunar rover mockup at a video wall enables engineers and astronauts to simulate and experience mission scenarios, integrated spacecraft systems, and operational procedures in a relevant environment before deployment. By integrating realistic virtual environments, immersive technology allows for the visualization and interaction with virtual spacecraft models, mission landscapes, and complex operational tasks. This approach helps identify potential design flaws, operational challenges, and safety considerations. It also provides valuable insights for risk reduction and helps improve mission efficiency and effectiveness. With advanced motion tracking systems and custom virtual environments, data can be gathered and evaluated to help NASA refine training protocols, develop specialized training procedures, and optimize human-robotic interactions for future space missions. Furthermore, immersive technology offers opportunities for future training initiatives at NASA. The Virtual Reality Laboratory at JSC has pioneered training with Virtual Reality (VR) since the Hubble Space Telescope repair missions in the early 1990’s. Extended Reality (XR) simulations enable astronauts to rehearse complex spacewalks, spacecraft maneuvers, and extravehicular activities in a safe and controlled environment. By replicating the physical and cognitive challenges of space missions, immersive training experiences enhance astronauts’ situational awareness, decision-making abilities, and adaptability to unexpected scenarios. Additionally, immersive technology facilitates collaborative training, allowing geographically dispersed crew and mission control personnel to engage in synchronized simulations, fostering teamwork and effective communication. The adoption of immersive technology in NASA’s testing, evaluation, and future training programs has yielded significant benefits. By incorporating human-in-the-loop simulations for studies involving Extra Vehicular Activities (EVA), surface mobility and landing systems, NASA can identify and mitigate risks, optimize operational procedures, and enhance mission success. Ultimately, immersive simulation experiences can empower astronauts to better navigate the complexities of space missions, ensuring their safety, productivity, and success in the dynamic and challenging environments they will experience at the Lunar South Pole.
HITL↗
Toward a Smart Metaverse City: Immersive Realism and 3D Visualization of Digital Twin Cities
Metaverse and its related extended reality technologies can enable immersive, realistic, and participatory visualization of 3D data, and their use and the potential within a smart city can be effective for supporting urban research and urban operations management. This book chapter describes a vision for prototyping a “Smart Metaverse City” to combine the unique advantage of the Metaverse technology with the two-way connectivity of a digital twin city application. This synergy aims to create a virtual environment for immersive geovisualization to help researchers and the public understand the complex urban system through science-based and data-driven approaches. This book chapter selectively reviews past technological and paradigm advancements for collecting, analyzing, and visualizing 3D urban big data. Then we present a prototyping Geographic Information System (GIS) framework, together with some relevant data sources and open-source web technologies, to help researchers create a smart Metaverse city. We demonstrate our vision and discuss its application opportunities through a real-world example, a digital twin city developed at the Oak Ridge National Laboratory, to facilitate participatory, smart, and sustainable campus management.
Xu, Haowen↗
Game Based Learning For Earth Science Applications Training
Current NASA Earth capacity development programs employ mechanisms ranging from online resource sharing, and virtual and in-person trainings to share knowledge. While these programs are highly successful at engaging individuals around the world – in 2018, over 8000 individuals and over 2000 institutions from all 50 US states and over 140 countries were engaged through over 150 projects and trainings – user feedback has highlighted the desire for expanded hands-on, practical experiences in incorporating NASA EO insights with localized data and actions. We aim to address this gap by leveraging the benefits of game-based learning to build user skills in integrating NASA and local EO data to guide decisions for climate resiliency and hazard planning. This project is being executed as a two-phase crowdsourced challenge: 1) Phase 1 will require a well-researched product concept that reflects an understanding of NASA’s Earth data and tools and user needs, and proposes an innovative and interactive game or extended reality experience to train users in identifying relevant NASA data and applying insights to their climate resiliency decisions; 2) Winners of Phase 1 will be provided seed funding to develop a working prototype of the product. We aim to award 1-3 final winners to support the development of more than one game, thereby ensuring that NASA's diverse audiences around the world can access training games that best suit their needs and capabilities. This EO training game project fits in the NASA Earth Science Applied Sciences Program’s Capacity Development Program, contributing to the program mission of “helping people around the world better understand [NASA’s Earth] data and find ways to use them” (https://appliedsciences.nasa.gov/what-we-do/capacity-building). The final training game will complement existing programmatic activities of workforce development, trainings, and collaborative projects, while providing the unique value of providing interactive experiences to users and collecting real-time data and feedback to improve NASA’s Earth applications’ products and services related to climate resilience.
Human centered design↗
Earth Independent Medical Operations (EIMO) Datascope: Challenges and Potential Solutions
Data flows and storage/retrieval capacity are severely constrained during missions in space and challenges will become even greater during exploration class missions. There is a need for an artificial intelligence (AI)-based clinical decision support system (CDSS) to monitor and analyze data to provide real-time consultative support for crew medical officer (CMO) decision-making. EIMO is defined as the gradual transition of medical care and decision making from terrestrial to space-based assets, enabling support of astronaut health and performance and reducing overall mission risk. While a hallmark of this paradigm shift from low-earth orbit is that on-board care will increasingly become the responsibility of the astronauts for primary management and decision making, terrestrial assets will continue to be paramount in pre-mission screening and planning, as well as prevention, health maintenance and long-term care contingencies. New capabilities and systems that enable progressively more robust and resilient systems and crews will be necessary to reduce risk and increase probability of deep space exploration mission success. An aspiration for EIMO is to develop AI-enhanced solutions for analysis of crew health & performance data and to facilitate clinical decision support for autonomous medical operations. A “system of systems” approach is envisioned whereby EIMO will deploy AI-supported natural language processing and machine learning (ML) techniques to utilize embedded reference databases and real-time data streams [input vectors] from multiple data sources. Constituent input vectors may include environmental controls, countermeasures data, behavioral data, physiologic wearables, point-of-care laboratory tests, personalized medical records, inventory trade space risk assessments, COTS medical databases, and ground support inputs. An ideal AI capability would possess trained fusion algorithms to cross reference input vectors with medical ‘knowledge’ [cultivated database] to stratify relevant data streams for predictive and actionable capabilities. In addition, EIMO will feature mobility, in that it can be accessed and can push/pull data within and between multiple vehicles/habitats. Large amounts and variable sources of data can be leveraged to diagnose, inform treatment strategies, and potentially predict medical events and performance decrements. Inclusion of advanced training tools using extended reality will enable increasingly autonomous medical care to aid a CMO when ground support is unavailable or time-delayed beyond required action window, e.g., emergent medical situations. EIMO CDSS would require very large datasets to train pre-flight and significant amounts of data are needed to support ML via in-flight CDSS operations. An additional challenge will be to find sufficient data to train a model relevant to astronaut demographics. The rapid, accelerating evolution of this field creates a propitious solution space to leverage multi-modal AI through public-private partnership(s). The status of multi-modal AI systems today would preclude their use for long duration missions as they remain unreliable and are subject to “digital hallucinations” and other errors that could pose operational risk. A federated labs structure is being considered to test and optimize data flow from the multiple input vectors leading to field testing in suitable ground/flight analogs. Critical to the success of an EIMO CDSS will be integration and interoperability and success will be defined by a system that can serve as an in-flight medical consult for the CMO providing critical support during medical contingencies. Benefits to terrestrial medicine may be significant as an outflow of the EIMO medical system, particularly for remote areas and communities lacking significant infrastructure, personnel and resources.
J Lemery↗
Earth Independent Medical Operations (EIMO) Training Technical Interchange Meeting, 26th October 2023: Background and Summary of Discussion
On 26th October 2023, ExMC convened a panel of Subject Matter Experts (SMEs) from NASA, broadly representing Headquarters, the Human Research Program, Medical Operations, the Human Health and Performance Directorate at JSC, the Health and Medical Technical Authority, the Flight Operations Directorate, and representation from other Centers including: Ames Research Center (ARC), and Glenn Research Center (GRC). Representatives from the Canadian Space Agency also participated in this TIM. In addition, SMEs from industry included representation from the following entities: Axiom Space, Space Exploration Technologies Corporation, Level Ex, Shiny Box Interactive. Additional SMEs from academia included: University of Houston, Touro University, Weill Cornell Medical College and the Translational Research Institute for Space Health at Baylor College of Medicine. This group of stakeholders discussed the training issues related to facilitating EIMO. Sub-topics for this discussion included the following: - Pre-launch medical curriculum development - Ground-based training on medical hardware - Dental health - Behavioral health - Telemedicine - Just-in-time training - Simulation-based training (extended reality)
CMO Training↗
Earth Independent Medical Operations (EIMO) Datascope: Challenges and Potential Solutions
Data flows and storage/retrieval capacity are severely constrained during missions in space and challenges will become even greater during exploration class missions. There is a need for an artificial intelligence (AI)-based clinical decision support system (CDSS) to monitor and analyze data to provide real-time consultative support for crew medical officer (CMO) decision-making. EIMO is defined as the gradual transition of medical care and decision making from terrestrial to space-based assets, enabling support of astronaut health and performance and reducing overall mission risk. While a hallmark of this paradigm shift from low-earth orbit is that on-board care will increasingly become the responsibility of the astronauts for primary management and decision making, terrestrial assets will continue to be paramount in pre-mission screening and planning, as well as prevention, health maintenance and long-term care contingencies. New capabilities and systems that enable progressively more robust and resilient systems and crews will be necessary to reduce risk and increase probability of deep space exploration mission success. An aspiration for EIMO is to develop AI-enhanced solutions for analysis of crew health & performance data and to facilitate clinical decision support for autonomous medical operations. A “system of systems” approach is envisioned whereby EIMO will deploy AI-supported natural language processing and machine learning (ML) techniques to utilize embedded reference databases and real-time data streams [input vectors] from multiple data sources. Constituent input vectors may include environmental controls, countermeasures data, behavioral data, physiologic wearables, point-of-care laboratory tests, personalized medical records, inventory trade space risk assessments, COTS medical databases, and ground support inputs. An ideal AI capability would possess trained fusion algorithms to cross reference input vectors with medical ‘knowledge’ [cultivated database] to stratify relevant data streams for predictive and actionable capabilities. In addition, EIMO will feature mobility, in that it can be accessed and can push/pull data within and between multiple vehicles/habitats. Large amounts and variable sources of data can be leveraged to diagnose, inform treatment strategies, and potentially predict medical events and performance decrements. Inclusion of advanced training tools using extended reality will enable increasingly autonomous medical care to aid a CMO when ground support is unavailable or time-delayed beyond required action window, e.g., emergent medical situations. EIMO CDSS would require very large datasets to train pre-flight and significant amounts of data are needed to support ML via in-flight CDSS operations. An additional challenge will be to find sufficient data to train a model relevant to astronaut demographics. The rapid, accelerating evolution of this field creates a propitious solution space to leverage multi-modal AI through public-private partnership(s). The status of multi-modal AI systems today would preclude their use for long duration missions as they remain unreliable and are subject to “digital hallucinations” and other errors that could pose operational risk. A federated labs structure is being considered to test and optimize data flow from the multiple input vectors leading to field testing in suitable ground/flight analogs. Critical to the success of an EIMO CDSS will be integration and interoperability and success will be defined by a system that can serve as an in-flight medical consult for the CMO providing critical support during medical contingencies. Benefits to terrestrial medicine may be significant as an outflow of the EIMO medical system, particularly for remote areas and communities lacking significant infrastructure, personnel and resources.
Medical Operations↗
Enterprise Digital Transformation: Transforming NASA's Workplace
NASA has a long history of serving the Nation and the world, by reaching for new heights and revealing the unknown for the benefit of all humankind. To ensure the agency will deliver on its mission in a hyper-connected, technology-enabled, globally-partnered, and rapidly-changing world, NASA launched a Digital Transformation initiative in late 2020 to modernize the way it works, the experience of its workforce and the agility of its workplace. At the core of its workplace transformation strategy, NASA is on a journey to leverage smart operational technologies, weaving together IoT, AI, intelligent automation, extended reality, and digital twins to build a network of reliable, adaptable and sustainable “Smart Centers” to execute its mission… on planet Earth and beyond. Come hear NASA’s digital strategy, the exciting progress being made, lessons learned from their journey, and how the AIMST community can enable NASA and its partners to serve humanity in a digital future.
Jill Marlowe↗
Summary of Technical Interchange Meetings (TIMs) Designed to Enable Earth Independent Medical Operations (EIMO)
The Exploration Medical Capability Element (ExMC) in NASA’s Human Research Program hosted a series of TIMs in 2023-2024 designed to stimulate discussion around specific topics with the goal of enabling EIMO. In context of the thematic constituent elements of EIMO, namely pre-mission planning, acute/emergent/prolonged medical decision making, supply/resource management and task load management, subject matter experts from industry, academia and government (NASA and other Agencies) provided valuable and actionable guidance and recommendations. Earth-based medical experts will remain indispensable for pre-mission planning, however, management of acute/emergent medical contingencies will require a gradual transition of medical care and decision making from terrestrial to space-based assets to enable support of astronaut health and performance and reduce overall mission risk. Key to achieving these enhancements is providing an integrated data system platform capable of utilizing multiple data streams in concert with a variety of on-board databases and passive monitoring of video and wearable sensors to enable a multi-modal, agentic AI-based clinical decision support system (CDSS) to support crew medical officer (CMO) medical decision-making. The EIMO series of TIMs (I-V) have proven to be instructive and portend a significant paradigm shift will be necessary to maintain crew health and performance on exploration class missions. Importantly, since the expected paradigm shift will be significantly different from the methods of operation that have been employed for the majority of missions from the inception of human spaceflight to date, any proposed methods must be deployed in the setting of ongoing operations early and be “tested, reviewed and practiced” while reliable back-up is available to facilitate an Enterprise-wide level of comfort and acceptance. Serious constraints on data transmission coupled with a large and expanding universe of on-board medical informatics data streams will necessitate implementation of a CDSS to supplant the current reliance on support provided by ground-based SMEs. Establishment of trust in the system by CMO/crew and the ground-based medical support team will be essential. Co-development of a CDSS with industry partners will assure that state of the art tools can be employed, and industry efficiencies can be leveraged. Training regimens, materials and tools must evolve to be responsive (just-in-time training) and facilitate autonomous execution of procedures. Proficiency metrics should be established and be based on validated competencies or milestones as opposed to a prescribed number of training hours. Training should be prioritized for broad, translatable skills that have universal application across a variety of medical conditions. Repetition was deemed to be the key to achieving proficiency and emphasis should lie in procedural training which is known to extinguish more rapidly than diagnostic skills. Advanced tools, e.g., extended reality, can provide more realistic and effective training. Use of advanced probabilistic risk assessment tools will be essential to optimize the medical system capability while carefully balancing risk relative to mass/power/volume limitations. Importance of factoring use-life of medical supplies and maintaining awareness of redundancy and opportunity to re-purpose under off nominal situations was emphasized. Consideration of adopting optimized performance standards vs. “good-enough” performance thresholds is warranted. The use of legacy systems as opposed to creating new systems may be preferable. Managing task load and associated cognitive load will be essential to maintain operational safety and behavioral health. ExMC aspires to create a shared EIMO paradigm and strategic vision for advancing medical system design through novel technologies, training, protocols, and support capabilities, built upon the spirit of successful strategies and innovations over the past six decades of space medicine operations.
Jay Lemery↗
Research Needs for Developing Qualification Standards for Flight Simulation Training Devices with Head Mounted Displays
This presentation discusses research needs to support evidence-based qualification standards for Flight Simulation Training Devices (FSTDs) with head mounted displays (HMDs). Currently, the FAA and EASA have no qualification standards specifically for FSTDs using HMDs and extended reality (XR) visuals, and existing qualification standards may not be sufficient. Special Conditions can be prescribed for the evaluation and qualification of individual devices, but these Special Conditions need to be assessed and adjusted on a case-by-case basis resulting in an elaborate process. Furthermore, test tolerances used in Special Conditions are often based on conventional visual systems and may not be adequate. This presentation first identifies gaps in the current knowledge of using HMDs in pilot training, and, more specifically, using HMDs in the training of manual aircraft control with motion cues. Next, research is proposed to help close these gaps, inform the development of qualification criteria and tolerances, and guide efforts to update regulations. The Vertical Motion Simulator (VMS) at NASA Ames Research Center, with its large motion capability, is currently undergoing significant upgrades including a state-of-the-art out-the-window visual system. In addition, an HMD capability will be introduced to use in conjunction with the conventional visual system. With its long history of setting standards in the simulation industry, the VMS is the ultimate tool to conduct research that could help accelerate the development of standards for the use of HMDs in FSTDs. The goal of the talk is ultimately to spark a discussion about the research needs and priorities to best serve the development of new qualification standards.
XR↗
Gigatraj: An Atmospheric Trajectory Model
Atmospheric trajectory models have a long history of success in tracking air motions in the lower stratosphere and upper troposphere over periods of up to a few days. Parcels have been traced backwards from observations to identify whatever phenomena (strong convection, volcanic eruptions, rocket launches, etc.) put their signature on them. Parcels have also been initialized at a known event and traced forward to examine their subsequent physical and chemical evolution. We describe a new trajectory model, "gigatraj," that aims to increase exibility by (a) making it straightforward to use new meteorological data sources, including those not based on regular latitude-longitude grids; (b) enabling a run-time choice of vertical coordinate system for kinematic and/or quasi-isentropic calculations; (c) allowing for the output of arbitrary meteorological products, selectable by the user and interpolated to the parcels' locations and times. The model can be run in a serial or parallel processing environment, so that large numbers of parcels can be traced in a reasonable time. Information is presented on model accuracy and performance. The former is demonstrated by runs using both test-pattern winds (comparing expected paths with actual output) and real-world winds (comparing forward and backward runs to characterize how well parcels retrace their paths). Sample cases are also shown, including a reverse domain lling (RDF) calculation illustrating a tropopause fold event. Model output can be displayed using the new Visualization And Lagrangian dynamics Immersive eXtended Reality (VALIXR) system, and an example will be shown. In addition, we describe work to incorporate a version of gigatraj into the Goddard Earth Observing System (GEOS) of NASA's Global Modeling and Assimilation O ce (GMAO) at Goddard Space Flight Center. This enables trajectory calculations to be performed within the running GEOS model at the latter's native time resolution, instead of using the winds from every few hours. It also provides access to all of GEOS's internal variables as they are calculated. This module may be useful, for example, for tracking rapid chemical changes in a Lagrangian framework.
dynamics↗