SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
NASA Ames SimLabs is home to unique simulation facilities capable of a wide range of aerospace systems research. This presentation will focus on the Vertical Motion Simulator with thoughts on future AR/VR applications and research.
Immersive environments are successfully being used to support mission operations at the Jet Propulsion Laboratory. This technology contributed to the Mars Pathfinder Mission in planning sorties for the Sojourner rover.
Explore the source record for details and available documents.
This archived content may contain outdated information or references that may not reflect current policy or programs.
Digital engineering has the power to transform organizations in profound ways, but digitizing legacy workflows and infusing new technologies into existing practices may not yield the intended results or address the underlying problems or bottlenecks. This paper describes the human-centered approaches used to understand stakeholder challenges and identify digital engineering needs during the design-build-test portion of the systems development lifecycle. Through a series of interactive stakeholder workshops and need-finding activities, key stakeholder needs were identified to help guide the integration of digital engineering technologies. The overarching need was for downstream system development steps to be considered further upstream in the project lifecycle in a manner that allows for proactive and interactive co-constructive idea exploration, problem resolution, and requirements considerations in a collective engagement with engineers and technicians from multiple disciplines to increase efficiency and reduce risks while offering opportunities to improve the final engineered system. Supporting this need was a wide-ranging desire from stakeholders for digital and non-digital support structures for efficient rapid iteration during system development. This work yielded several improvements for system development processes and for digital engineering infusion.
Ames Research Center granted Reality Capture Technologies (RCT), Inc., a license to further develop NASA's Mars Map software platform. The company incorporated NASA#s innovation into software that uses the Virtual Plant Model (VPM)(TM) to structure, modify, and implement the construction sites of industrial facilities, as well as develop, validate, and train operators on procedures. The VPM orchestrates the exchange of information between engineering, production, and business transaction systems. This enables users to simulate, control, and optimize work processes while increasing the reliability of critical business decisions. Engineers can complete the construction process and test various aspects of it in virtual reality before building the actual structure. With virtual access to and simulation of the construction site, project personnel can manage, access control, and respond to changes on complex constructions more effectively. Engineers can also create operating procedures, training, and documentation. Virtual Plant Model(TM) is a trademark of Reality Capture Technologies, Inc.
NASA is planning to return to the Moon in the mid-2020s as a steppingstone to Mars missions in the 2030s. Spacewalks, or extravehicular activities (EVAs), performed on the Moon and Mars will differ in a variety of ways from those that have been performed in decades past. NASA has identified multiple risks to human health and performance associated with a crewed mission to Mars, especially those associated with exploration EVAs which are expected to be a primary mission activity. Crew may be expected to conduct up to 24 hours of EVA per person per week, where the likelihood of injury and/or mental mistakes are increased compared to ground-based training or current microgravity EVAs and the consequences of which can be catastrophic. Current test environments for exploration EVA research and technology development are large, costly facilities that are limited in their availability or capabilities. Spacesuit testing in a reduced gravity environment such as NASA’s Neutral Buoyancy Laboratory, while a good representation of the crew’s physical workload during exploration EVAs, typically has small datasets and is difficult to integrate physiological sensors or other types of crew performance measures. Meanwhile, scientific field-based testing such as NASA’s Desert Research and Technology Studies offers an operationally relevant environment for exploration EVAs, particularly for cognitive workload, but is also limited by small datasets, lack of a pressurized spacesuit, and obtrusive measures. The limitations of current analogs for exploration EVAs identify a need for a new test environment that can approximate both the physical and cognitive demands associated with exploration EVAs to enable rapid, controlled, and repeatable evaluations of human health and performance risks of exploration missions. In response, the Human Physiology, Performance, Protection, and Operations Laboratory (H-3PO) at NASA Johnson Space Center has developed a hybrid reality exploration EVA analog named the Assessments of Physiology And Cognition in Hybrid-reality Environments (APACHE)[ to address these limitations using a combination of virtual, physical, and hybrid reality techniques. The APACHE facility resides at NASA Johnson Space Center and serves as a large “sandbox” for EVA research and simulation. At its center is a roughly 15x20ft space surrounded by a 14” tall sandbox partially filled with lunar regolith simulant to emulate the physical feeling of walking on a planetary surface and to allow for simulated geology operations. Nearby, a curved passive treadmill (Skillmill Connect, Technogym, Fairfield, NJ) and an omnidirectional treadmill (Infinadeck, Infinadeck, Rocklin, CA) are included to enable exploration of these large virtual environments while also imposing the physical demands, representative timelines, and cognitive burdens required to navigate and traverse these distances during exploration EVA. A 6DOF motion platform is used to simulate rover operations and supports various human performance evaluations and associated risks. Lastly, APACHE can support two extravehicular (EV) crewmembers working in tandem. A computer workstation is located nearby and also supports an intravehicular (IV) crewmember as part of a full mission simulation. The IV crewmember has direct video and audio communication with the EV crew in VR to provide operational and procedural support.
NASA is planning to return to the Moon in the mid-2020s as a stepping stone to Mars missions in the 2030s. Spacewalks, or extravehicular activities (EVAs), performed on the Moon and Mars will differ in a variety of ways from those that have been performed in decades past. NASA has identified multiple risks to human health and performance associated with a crewed mission to Mars, especially those associated with exploration EVAs which are expected to be a primary mission activity. Crew may be expected to conduct up to 24 hours of EVA per person per week, where the likelihood of injury and/or mental mistakes are increased compared to ground-based training or current microgravity EVAs and the consequences of which can be catastrophic. Current test environments for exploration EVA research and technology development are large, costly facilities that are limited in their availability or capabilities. Spacesuit testing in a reduced gravity environment such as NASA’s Neutral Buoyancy Laboratory, while a good representation of the crew’s physical workload during exploration EVAs, typically has small datasets and is difficult to integrate physiological sensors or other types of crew performance measures. Meanwhile, scientific field-based testing such as NASA’s Desert Research and Technology Studies offers an operationally relevant environment for exploration EVAs, particularly for cognitive workload, but is also limited by small datasets, lack of a pressurized spacesuit, and obtrusive measures. The limitations of current analogs for exploration EVAs identify a need for a new test environment that can approximate both the physical and cognitive demands associated with exploration EVAs to enable rapid, controlled, and repeatable evaluations of human health and performance risks of exploration missions. In response, the Human Physiology, Performance, Protection, and Operations Laboratory (H-3PO) at NASA Johnson Space Center has developed a hybrid reality exploration EVA analog named the Assessments of Physiology And Cognition in Hybrid-reality Environments (APACHE) to address these limitations using a combination of virtual, physical, and hybrid reality techniques. The APACHE facility resides at NASA Johnson Space Center and serves as a large “sandbox” for EVA research and simulation. At its center is a roughly 15x20ft space surrounded by a 14” tall sandbox partially filled with lunar regolith simulant to emulate the physical feeling of walking on a planetary surface and to allow for simulated geology operations. Nearby, a curved passive treadmill (Skillmill Connect, Technogym, Fairfield, NJ) and an omnidirectional treadmill (Infinadeck, Infinadeck, Rocklin, CA) are included to enable exploration of these large virtual environments while also imposing the physical demands, representative timelines, and cognitive burdens required to navigate and traverse these distances during exploration EVA. A 6DOF motion platform is used to simulate rover operations and supports various human performance evaluations and associated risks. Lastly, APACHE can support two extravehicular (EV) crewmembers working in tandem. A computer workstation is located nearby and also supports an intravehicular (IV) crewmember as part of a full mission simulation. The IV crewmember has direct video and audio communication with the EV crew in VR to provide operational and procedural support. The software used in APACHE was created by the JSC Engineering Directorate, in partnership with Buendea, powered by a custom Unreal Engine 5 (UE5.3, Epic Games) project. APACHE currently utilizes the HTC Vive Pro Eye in a wireless configuration for VR simulations. There are two virtual environments that subjects can explore within APACHE, a Lunar and Martian surface. The virtual Lunar surface was created from LIDAR data of the Lunar South Pole to create roughly 16 sq km of explorable terrain. The virtual Martian surface contains roughly 400 sq km of explorable terrain derived from Mars Reconnaissance Orbiter LIDAR data of the Jezero Crater. The immersion and related cognitive burdens of conducting a planetary EVA is simulated through a series of EVA-relevant tasks performed in the VR environment, using these high-fidelity visual representations. Additionally, APACHE includes biosensor driven informatics, such as real-time heart rate monitoring and/or derived values from model simulations, for active monitoring by the EV crew and added cognitive demand. A “Wizard of Oz” control panel enables test operators to activate contingency events such as simulated spacesuit malfunctions, loss of communications, and/or limited visibility. Embedded performance measures such as accuracy, completeness, and execution time have been developed for various exploration tasks to objectively quantify crew performance during an EVA and compare impacts to performance when different environmental stressors, both physical and cognitive, are added to or removed from the simulation. Additionally, validated cognitive and operational performance measures such as the Digit Symbol Substitution Task have been recreated and embedded in VR for direct and relatively unobtrusive measurement of motor perception. The APACHE environment currently supports multiple research studies at NASA. Examples include the CHAPEA project, a series of simulated year-long missions on Mars by a 4-person crew; and the CO2 Contingency Walk Back Study, an investigation of elevated CO2 exposure on crew performance during a contingency EVA scenario. APACHE also provides a test environment to support the development of the Crew State and Risk Model, which is a collection of individualized, mathematical models of crew physical and cognitive state; and the Personalized EVA Informatics and Decision Support system, an operational tool for flight controllers, and eventually a self-reliant Martian crew, to make biomedically-informed decisions in real-time to optimize the EVA planning and execution with respect to crew health and performance. Some technical challenges associated with developing the APACHE environment, as well as current limitations, include VR limitless natural walking with a hybrid spacesuit simulator, optimizing performance for wireless PC VR streaming while maintaining a high degree of visual fidelity, and the integration of various physiological (metabolic masks) and psychometric (eye tracking) sensors with the VR headset.
NASA is preparing to land the first woman and first person of color on the Moon within the next decade and establish a permanent sustainable human presence before sending humans onto Mars. To ensure the success of these missions, NASA has performed operational testing in terrestrial, aquatic, and laboratory analog environments that simulate Lunar and Martian environmental characteristics to evaluate exploration concepts of operations (ConOps), engineering design requirements, science support needs, mission operations techniques, and crew training. Terrestrial analogs include Desert Research and Technology Studies (D-RATS), Biologic Analog Science Associated with Lava Terrains (BASALT), and Next Space Technologies for Exploration Partnerships (NextSTEP)Habitat Ground Testing. Aquatic analogs include NASA Extreme Environment Mission Operations (NEEMO) and Pavilion Lake Research Project (PLRP).Laboratory analogs include the Neutral Buoyancy Laboratory (NBL), Active Response Gravity Offload System(ARGOS), rock yards, and virtual and hybrid reality simulation environments. While no single Earth-based analog environment is perfect for simulating all characteristics of other planetary surfaces, testing across multiple locations leverages the strengths of each to provide an integrated understanding of how to best conduct real spaceflight surface exploration missions.
Earth science (ES) digital twins will help us understand the complex interactions and interrelationships that make up our Earth system and the impacts of earth science phenomena on it. Our work addresses two underdeveloped areas in current ES digital twin work: improving the understanding and interaction with ES model outputs by using Virtual and Mixed Reality (XR) tools and improving the non-intuitive mapping of continuous ES natural phenomena to gridded reference frames in current numerical models. Traditionally, scientists working on ES view and analyze the results of calculated or measured observables with static 1-dimensional (1D), 2D or 3D plots displayed on flat computer screens or paper. Using such limited mediums, it can be very difficult to identify, track and understand the evolution of key features due to poor viewing angles and the nature of flat computer screens. In addition, numerical models, such as the NASA Goddard Earth Observing System (GEOS) ES model, are almost exclusively formulated, visualized and analyzed in an Eulerian reference frame with fixed grid points in space and time. However, ES phenomena such as convective clouds, hurricanes and wildfire smoke plumes are visualized and analyzed in a Lagrangian reference frame: therefore it is often difficult and unnatural to understand these phenomena in relation to each other, visualized either in an Eulerian or Lagrangian context. In 3D visualizations, data generally takes one of three forms: gridded (e.g., voxelized) data, where space is divided into regions; point clouds, where data is represented as a set of points; and meshes, where objects are rendered as surfaces composed of small polygons (usually triangles). A gridded, Eulerian reference frame has been the default representation for the 2D visual analysis of atmospheric data in part because the numerical methods used to generate atmospheric model data in the first place use a gridded approach, with equations defining the relationships between the physical variables in each of a grid's cells across successive timesteps. In our work, we are particularly interested in data from GEOS. Another reason why gridded representations tend to be used for visualizing data from such models is because trajectories are difficult to interpret from representations on 2D surfaces, due to line-of-sight ambiguity. Instead of a fixed grid from GEOS, we embed a trajectory model to simulate particles' movement throughout a GEOS run. We then ingest these particle trajectories as animated point clouds with a NASA open source XR toolkit, the Mixed Reality Exploration Toolkit (MRET), and merge GEOS data with ES phenomena data onto one combined visualization that the user can intuitively interact with. Efficient rendering of arbitrarily large point clouds is an ongoing challenge being addressed by the computer science community, with the GPU-based optimizations and efficient GPU memory utilization a common theme of recent advances, especially for XR, where sustained high frame rate is mandatory to save the user from suffering due to simulation sickness. In this work, we describe and evaluate our progress in choosing and implementing appropriate methods for rendering arbitrarily large point clouds within MRET for XR. While tracking the XR headset enables the immersion of a user within a 3D scene of a data visualization, tracking of XR handheld controllers or user’s hands enables us to implement intuitive user interactions with the visualized datasets. Conventional tools require a user working with an ES visualization to conduct many interactions to commit their intended selections or manipulations with a visualized dataset; for example to specify a set of points in 3D space. Doing so in a 2D flat screen interface has traditionally required specifying a set of points in three distinct 2D coordinate systems (XY, XZ, and YZ), which is cumbersome. In other scientific domains, it has been shown that specifying or selecting a location or volume in XR using handheld controllers or tracked hands allows for greater speed and accuracy. We anticipate the same will hold true for atmospheric data, and we will share initial results of measuring the utility of such an interface. Notably, as the data being visualized is generated by GEOS as a prediction based on initial conditions, an intended application of our tool is to serve as part of an iterative feedback loop. Through XR, a scientist will review and manipulate a GEOS model run, modifying the conditions as needed to do subsequent runs of GEOS. Thereby, XR-based improvements to speed and accuracy of 3D tagging of points minimizes the effort required by both the scientist and the computer cluster conducting the necessary calculations.
LIMITATIONS AND FEASIBILITY OF MINI X-RAY DEVICES IN SPACE ENVIRONMENTS As space exploration advances toward long-duration missions, reliable medical diagnostic tools become increasingly critical. The miniature x-ray (XR) technology demonstrations by the Exploration Medical Capability (ExMC) and the Exploration Medical Integrated Product Team (XMIPT) aim to assess the feasibility and utility of miniature XR devices in spaceflight. This abstract explores the limitations of current miniature XR systems, the challenges of training crew members, the potential role of clinical decision support systems (CDSS), and the feasibility of ground-based image interpretation. We also propose the integration of miniature XR into other ExMC efforts aimed at identifying the capabilities and resources needed for future exploration class missions. One of the primary challenges with miniature XR devices is the ability to achieve specific anatomical views, particularly in the confined and weightless conditions of a spacecraft. Operators may struggle to acquire diagnostic-quality images when space is limited for proper patient positioning and the volume of the imaging device. Since space radiation and detector limitations may further impact image quality, the flexibility of the operating procedures of these devices will be critical for their success in space applications. CHALLENGES IN TRAINING CREW TO OPERATE IMAGING DEVICES Training in the skills necessary to acquire diagnostic-quality scans may be a barrier for non-clinician crewmembers. The curriculum developed for crew medical officers (CMOs) will require simplification and adaptation to fit into the highly truncated pre-flight training period. Therefore, hands-on familiarization and simulation, both pre-flight and just-in-time training during missions, will be crucial to ensuring the crew can operate the devices in real-life situations. The ability to adjust acquisition parameters must be simplified or made automatic through exam selections on equipment user interfaces, and subject and operator positioning should be assisted with laser guidance and pictorial guides. POTENTIAL FOR CDSS OR ARTIFICIAL INTELLIGENCE (AI)-ASSISTED CDSS CDSS and AI-assisted CDSS offer significant promise in assisting crew members with limited medical training. These systems could provide real-time feedback on image quality and interpretation, helping to mitigate the risks of human error during space missions. Integrating procedural guidance tools, such as virtual and augmented reality, will support crewmembers in accurately positioning patients and obtaining high-quality images. However, the success of such systems will depend on the development of robust training datasets, integration with spaceflight-rated hardware, and the medical decision-making capabilities of operators. FEASIBILITY OF GROUND INTERPRETATION AND DATA TRANSMISSION Reliance on ground-based interpretation may prove difficult for acute care during exploration class-missions due to delays in transmission with increasing distance from Earth or complete communication blackout periods. In such instances where immediate interpretation for clinical intervention is required, crew must be able to interpret the images independently or utilize AI-based assistance to do so. File sizes for XR exams can also be large if numerous images are acquired and bandwidth constraints may limit data transmissions for both radiography and ultrasound exams. FUTURE WORK AND INTEGRATION INTO THE EVIDENCE LIBRARY Future work proposes integrating miniature XR devices into NASA’s Evidence Library to address medical conditions identified as significant contributors to crew morbidity and mortality. The possibility of combining miniature XR with other imaging modalities, such as ultrasound devices, is also under investigation. In conclusion, while miniature XR technology holds potential for extraterrestrial medical systems, there are significant challenges to overcome. Training, integration of AI tools, dedicated exam protocols for microgravity, and improved data transmission systems will be key to realizing the full benefits of miniature XR technology in space.
This paper investigates the effects of simulator motion fidelity on pilot performance using mixed-reality visuals. In NASA’s Vertical Motion Simulator, eleven pilots flew a lift-plus-cruise electric vertical takeoff and landing vehicle in a vertiport approach and landing task with four different motion conditions: no motion, small hexapod motion, large hexapod motion, and full motion. The simulator motion condition statistically significantly affected: 1) the severity of cybersickness, 2) motion ratings, 3) final approach and take-off boundary crossing groundspeed, 4) longitudinal touchdown point, and 5) battery time remaining after landing. The findings suggest that motion systems are essential in simulators with virtual- or mixed-reality visuals when it is required to have pilot flying performance similar to that in real flight.
This paper investigates the effects of simulator motion fidelity on pilot performance using mixed-reality visuals. In NASA’s Vertical Motion Simulator, eleven pilots flew a lift-plus-cruise electric vertical takeoff and landing vehicle in a vertiport approach and landing task under four different motion conditions: no motion, small hexapod motion, large hexapod motion, and full motion. The simulator motion condition statistically significantly affected: 1) the severity of cybersickness, 2) motion ratings, 3) final approach and take-off boundary crossing groundspeeds, 4) longitudinal touchdown point, and 5) battery time remaining after landing. These results suggest that the use of motion systems in simulators with virtual- or mixed-reality visuals is warranted.
Synthetic Vision System technology augments reality and creates a virtual visual meteorological condition that extends a pilot's cognitive and perceptual capabilities during flight operations when outside visibility is restricted. The paper describes the NASA Synthetic Vision System for commercial aviation with an emphasis on how the technology achieves Augmented Cognition objectives.
The Virtual Control Room (VCR) Proof of Concept (PoC) project is the result of an award given by the Fourth Annual NASA T&I Labs Challenge Project Call. This paper will outline the work done over the award period to build and enhance the capabilities of the Augmented/Virtual Reality (AVR) Lab at NASA's Kennedy Space Center (KSC) to create the VCR.
The National Aeronautics and Space Administration (NASA) has completed a flight test to evaluate the performance of an onboard prototype automation system operating in future high density urban airspace. The test was part of a research investigation of the Urban Air Mobility (UAM) concept, with a focus on a future environment having hundreds of simultaneous operations over a metropolitan area. The complexity of this future UAM airspace may require automation capable of replanning an aircraft’s path in the presence of traffic and other changing constraints. A live-virtual-constructive (LVC) approach was used to conduct the test. Prototype automation technology was integrated into one of the two live aircraft, which were combined with virtual traffic to create a mixed reality environment at the target airspace density. In-flight evaluation enabled verification of the automation’s functions and discovery of any unexpected behaviors resulting from its operation in an actual flight environment. The in-flight evaluation also provided data for validation of air traffic simulations. This paper discusses the design, methodology, and challenges overcome to conduct a successful flight test. Remaining challenges, future work, and recommendations to improve the flight test capability are also discussed.
This paper discusses configurations and test analogs toward the design of a virtual window capability in a Deep Space Habitat. Long-duration space missions will require crews to remain in the confines of a spacecraft for extended periods of time, with possible harmful effects if a crewmember cannot cope with the small habitable volume. Virtual windows expand perceived volume using a minimal amount of image projection equipment and computing resources, and allow a limited immersion in remote environments. Uses for the virtual window include: live or augmented reality views of the external environment; flight deck, piloting, observation, or other participation in remote missions through live transmission of cameras mounted to remote vehicles; pre-recorded background views of nature areas, seasonal occurrences, or cultural events; and pre-recorded events such as birthdays, anniversaries, and other meaningful events prepared by ground support and families of the crewmembers.