Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “augmented reality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Immersive Technologies for Human-in-the-Loop Lunar Surface Simulations

NASA, the National Aeronautics and Space Administration, continually seeks innovative solutions to enhance its operations, particularly in the realms of testing, evaluation, and training for future missions. Immersive technologies, such as virtual, augmented, and mixed reality have proven to be powerful tools for realistic, interactive, and engaging environments. This paper explores how the Simulation and Graphics Branch at NASA’s Johnson Space Center (JSC) leverages immersive technology, modern commercial rendering engines, and physics-based systems simulations to develop human-in-the-loop systems for humanity’s return to the Moon through the Artemis program. When NASA returns to the Moon, astronauts will travel to the Moon’s South Pole where lighting conditions will cause a more complex operational environment. Human-in-the-loop testing plays a crucial role in NASA's mission planning, spacecraft and space systems development, and evaluation of operational scenarios. The development of immersive environments such as a lunar rover mockup at a video wall enables engineers and astronauts to simulate and experience mission scenarios, integrated spacecraft systems, and operational procedures in a relevant environment before deployment. By integrating realistic virtual environments, immersive technology allows for the visualization and interaction with virtual spacecraft models, mission landscapes, and complex operational tasks. This approach helps identify potential design flaws, operational challenges, and safety considerations. It also provides valuable insights for risk reduction and helps improve mission efficiency and effectiveness. With advanced motion tracking systems and custom virtual environments data can be gathered and evaluated to help NASA refine training protocols, develop specialized training procedures and optimize human-robotic interactions for future space missions. Furthermore, immersive technology offers opportunities for future training initiatives at NASA. The Virtual Reality Laboratory at JSC has pioneered training with Virtual Reality (VR) since the Hubble Space Telescope repair missions in the early 1990’s. Extended Reality (XR) simulations enable astronauts to rehearse complex spacewalks, spacecraft maneuvers, and extravehicular activities in a safe and controlled environment. By replicating the physical and cognitive challenges of space missions, immersive training experiences enhance astronauts' situational awareness, decision-making abilities, and adaptability to unexpected scenarios. Additionally, immersive technology facilitates collaborative training, allowing geographically dispersed crew and mission control personnel to engage in synchronized simulations, fostering teamwork and effective communication. The adoption of immersive technology in NASA's testing, evaluation, and future training programs has yielded significant benefits. By incorporating human-in-the-loop testing for studies involving Extra Vehicular Activities (EVA), surface mobility and landing systems, NASA can identify and mitigate risks, optimize operational procedures, and enhance mission success. Ultimately, immersive training experiences can empower astronauts to better navigate the complexities of space missions, ensuring their safety, productivity, and success in the dynamic and challenging environments they will experience at the Lunar South Pole.

Simulation Modeling Virtual Reality Immersive Tech↗

Immersive Technologies for Human-in-the-Loop Lunar Surface Simulations

NASA, the National Aeronautics and Space Administration, continually seeks innovative solutions to enhance its operations, particularly in the realms of testing, evaluation, and training for future missions. Immersive technologies, such as virtual, augmented, and mixed reality have proven to be powerful tools for immersing users in realistic, interactive, and engaging environments. This paper explores how the Simulation and Graphics Branch at NASA’s Johnson Space Center (JSC) leverages immersive technology, modern commercial rendering engines, and physics-based systems simulations to develop human-in-the-loop systems for humanity’s return to the Moon through the Artemis program. When NASA returns to the Moon, astronauts will travel to the Moon’s South Pole where lighting conditions will cause a more complex operational environment. Human-in-the-loop simulations play a crucial role in NASA’s mission planning, spacecraft and space systems development, and evaluation of operational scenarios. The development of immersive environments such as a lunar rover mockup at a video wall enables engineers and astronauts to simulate and experience mission scenarios, integrated spacecraft systems, and operational procedures in a relevant environment before deployment. By integrating realistic virtual environments, immersive technology allows for the visualization and interaction with virtual spacecraft models, mission landscapes, and complex operational tasks. This approach helps identify potential design flaws, operational challenges, and safety considerations. It also provides valuable insights for risk reduction and helps improve mission efficiency and effectiveness. With advanced motion tracking systems and custom virtual environments, data can be gathered and evaluated to help NASA refine training protocols, develop specialized training procedures, and optimize human-robotic interactions for future space missions. Furthermore, immersive technology offers opportunities for future training initiatives at NASA. The Virtual Reality Laboratory at JSC has pioneered training with Virtual Reality (VR) since the Hubble Space Telescope repair missions in the early 1990’s. Extended Reality (XR) simulations enable astronauts to rehearse complex spacewalks, spacecraft maneuvers, and extravehicular activities in a safe and controlled environment. By replicating the physical and cognitive challenges of space missions, immersive training experiences enhance astronauts’ situational awareness, decision-making abilities, and adaptability to unexpected scenarios. Additionally, immersive technology facilitates collaborative training, allowing geographically dispersed crew and mission control personnel to engage in synchronized simulations, fostering teamwork and effective communication. The adoption of immersive technology in NASA’s testing, evaluation, and future training programs has yielded significant benefits. By incorporating human-in-the-loop simulations for studies involving Extra Vehicular Activities (EVA), surface mobility and landing systems, NASA can identify and mitigate risks, optimize operational procedures, and enhance mission success. Ultimately, immersive simulation experiences can empower astronauts to better navigate the complexities of space missions, ensuring their safety, productivity, and success in the dynamic and challenging environments they will experience at the Lunar South Pole.

HITL↗

Virtual reality and planetary exploration

NASA-Ames is intensively developing virtual-reality (VR) capabilities that can extend and augment computer-generated and remote spatial environments. VR is envisioned not only as a basis for improving human/machine interactions involved in planetary exploration, but also as a medium for the more widespread sharing of the experience of exploration, thereby broadening the support-base for the lunar and planetary-exploration endeavors. Imagery representative of Mars are being gathered for VR presentation at such terrestrial sites as Antarctica and Death Valley.

Mcgreevy, Michael W.↗

Virtual reality and planetary exploration

Exploring planetary environments is central to NASA's missions and goals. A new computing technology called Virtual Reality has much to offer in support of planetary exploration. This technology augments and extends human presence within computer-generated and remote spatial environments. Historically, NASA has been a leader in many of the fundamental concepts and technologies that comprise Virtual Reality. Indeed, Ames Research Center has a central role in the development of this rapidly emerging approach to using computers. This ground breaking work has inspired researchers in academia, industry, and the military. Further, NASA's leadership in this technology has spun off new businesses, has caught the attention of the international business community, and has generated several years of positive international media coverage. In the future, Virtual Reality technology will enable greatly improved human-machine interactions for more productive planetary surface exploration. Perhaps more importantly, Virtual Reality technology will democratize the experience of planetary exploration and thereby broaden understanding of, and support for, this historic enterprise.

Mcgreevy, Michael W.↗

Improving Sim-to-Real Transfer in Vision-Based Robot Navigation Via Instance-Level GAN-Based Data Augmentation

Achieving robust vision-based robotic tasks requires large amounts of data, which are often difficult to obtain in real-world scenarios. Simulators and synthetic data offer a cost-effective alternative, but the visual gap between simulation and reality hinders the performance of models when deployed in real-world environments. In this paper, we present a data augmentation pipeline that integrates a foundation model (Segment Anything Model) with an unsupervised image-to-image translation model (CycleGAN) for instance-level domain transfer from simulation to reality. This pipeline enables the generation of realistic labeled data from synthetic images for training supervised machine learning models in vision-based navigation tasks. We evaluate our approach on real-world data for ego-vehicle pose estimation, a critical autonomous navigation task involving the prediction of cross-track position and heading angle relative to road center line markings. The results of our tests show that our GAN-based data augmentation pipeline significantly outperforms models trained solely on simulation data or on data processed with standard image augmentation methods for sim-to-real transfer, enhancing model robustness and generalizability in real-world scenarios. Our method provides a scalable and flexible data augmentation tool for leveraging large synthetic datasets to enhance vision-based robotic navigation tasks.

artificial intelligence↗

Crew Medical Training to Progressively Enable EIMO

Background. Onboard medical capabilities have greatly expanded over the history of the US space program. Newly identified space-related medical conditions, technological advances, and longer mission durations have led to an increasing need for on-demand medical expertise. Lengthy communications delays, lack of resupply and evacuation opportunities on exploration-class missions place an ever-increasing burden on the crew to provide medical care. Having adequate knowledge, skills, and abilities (KSA) available is an essential component of successful Earth Independent Medical Operations (EIMO). Without appropriate crew training and KSA, cutting-edge medical equipment has little value. Presumably, the crew will include a qualified physician; however, if the physician is incapacitated, a non-physician crew medical officer (CMO) will be needed. While more crew time is needed for medical training, there will be concomitant increases in preflight training demands for vehicle system management, operations, science, and contingencies. In truly independent operations, onboard resources such as just-in-time training, mixed reality, decision support tools, and AI-enabled chatbot “consultants” will be needed to augment KSA. Overview. Because of crew time constraints, topical priorities must be determined for preflight training. Curricula should be developed that emphasize management of conditions with relatively high incidence and morbidity/mortality. Defining the required KSA levels to treat each condition is essential, but all crewmembers should have basic lifesaving skills. Procedural and diagnostic training on live patients and simulators should be prioritized over classroom lectures. Crews must be trained with onboard equipment, resources, mixed reality, and AI-based decision support tools. Mission simulations should include medical problems with/without ground support and with appropriate communication delays. Certification guidelines for each level of KSA must be established. Skills rapidly decay for non-physician CMO’s; both pre-flight and in-flight refresher training will be needed. During spaceflight just-in-time training, simulations, and onboard CME with crew physician can help retain skills. Discussion. Medical technology, simulation design, mixed reality, and AI are advancing at a dizzying rate. Recognizing the severe constraints on crew time, it is critical that astronaut training is highly efficient and adapted to keep pace with new innovations both pre-flight and during exploration missions. These challenges will be discussed during this panel session.

Jay Lemery↗

Effects of Multisensory Virtual Reality Nature on Affect, Presence, and Restorativeness

Virtual reality (VR) can provide access to restorative environments and sensory stimulation for individuals in isolated, confined, and extreme environments. We developed a nature-based VR intervention featuring three scenes (Garden, Forest, Beach) and examined the effects of multisensory augmentation with location-based olfactory, wind, and thermal stimuli (VR+OWT). Over a two-week period, fifty-one participants were assigned to a no-intervention control, standard audiovisual VR, or augmented VR (VR+OWT) condition. Data collection included weekly assessments of affect, stress, and cognitive performance; post-VR measures of affect, presence, and perceived restorativeness; and open-ended feedback. Both VR and VR+OWT groups produced immediate and sustained reductions in negative affect with no sustained systematic benefits on cognitive performance. The addition of olfactory, wind, and thermal stimuli enhanced presence for some scenes; however, feedback indicated that sensory mismatches and stimulus intensity occasionally detracted from the experience and may have contributed to lower coherence ratings in the VR+OWT group. In contrast, the perceived restorativeness of the VR environment was driven primarily by scene content (Garden vs Forest vs Beach) rather than the level of sensory stimulation. These findings support the potential of virtual nature to promote well-being and underscore the importance of coherence across sensory modalities for multisensory augmentation.

Virtual Reality Intervention↗

The assessment of virtual reality for human anatomy instruction

This research project seeks to meet the objective of science training by developing, assessing, and validating virtual reality as a human anatomy training medium. In ideal situations, anatomic models, computer-based instruction, and cadaver dissection are utilized to augment the traditional methods of instruction. At many institutions, lack of financial resources limits anatomy instruction to textbooks and lectures. However, human anatomy is three dimensional, unlike the one dimensional depiction found in textbooks and the two dimensional depiction found on the computer. Virtual reality is a breakthrough technology that allows one to step through the computer screen into a three dimensional world. This technology offers many opportunities to enhance science education. Therefore, a virtual testing environment of the abdominopelvic region of a human cadaver was created to study the placement of body parts within the nine anatomical divisions of the abdominopelvic region and the four abdominal quadrants.

Benn, Karen P.↗

Veridical data science

Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, composed of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and transparent results across the data science life cycle. The PCS workflow uses predictability as a reality check and considers the importance of computation in data collection/storage and algorithm design. It augments predictability and computability with an overarching stability principle. Stability expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model/algorithm perturbations. As part of the PCS workflow, we develop PCS inference procedures, namely PCS perturbation intervals and PCS hypothesis testing, to investigate the stability of data results relative to problem formulation, data cleaning, modeling decisions, and interpretations. We illustrate PCS inference through neuroscience and genomics projects of our own and others. Moreover, we demonstrate its favorable performance over existing methods in terms of receiver operating characteristic (ROC) curves in high-dimensional, sparse linear model simulations, including a wide range of misspecified models. Finally, we propose PCS documentation based on R Markdown or Jupyter Notebook, with publicly available, reproducible codes and narratives to back up human choices made throughout an analysis. The PCS workflow and documentation are demonstrated in a genomics case study available on Zenodo.

97 MATHEMATICS AND COMPUTING↗

A Virtual Reality Planning Environment for High-Risk, High-Latency Teleoperation

Teleoperation of robots in space is challenging due to high latency and limited workspace visibility. Previously, the Interactive Planning and Supervised Execution (IPSE) and Augmented Virtuality systems were developed to reduce failure risk. These tools were visualized on a 3D da Vinci surgical console and operated using the da Vinci manipulators or visualized on conventional monitors and operated with a keyboard and mouse. Experimental studies indicated operator preference for the latter. In this work, we develop a 3D virtual reality (VR) interface for IPSE, implemented on a Meta Quest 2 head-mounted display (HMD), and evaluate it against the prior 2D, keyboard-and-mouse-based interface. The results demonstrate improved operator load with the 3D VR interface, with no decrease in task performance, while also providing cost and portability benefits compared to the conventional 2D interface.

Will Pryor↗

Immersive Digital Twin Laboratory for Engineering Education (CRADA Final Report)

This project aimed to create an immersive digital twin laboratory that incorporates advanced tracking and visualization capabilities. In collaboration with Fort Lewis College, NREL designed a state-of-the-art physical visualization laboratory, developed a software platform to enable interaction with tracked physical objects in the laboratory, and provided proof-of-concept curricula that included manipulating these tracked objects. The project was initiated to address the growing need for innovative educational tools in engineering education. As renewable energy systems, particularly solar installations, become more complex, there is a pressing need to bridge the gap between theoretical knowledge and practical application. Traditional methods of teaching solar engineering concepts often fall short of providing students with a comprehensive, hands-on understanding. This immersive digital twin laboratory was conceived to fill that gap by creating a safe, non-energized setting where students can interact with augmented solar installation objects, gaining valuable insights into system performance, design, and maintenance. The project utilized extended reality (XR) technologies, including head-mounted displays (HMDs) and a whole-room optical motion tracking system, to connect physical objects with their digital twins in real time. The laboratory was equipped with MagicLeap 2 HMDs, supported by a Vicon Vero 2.2 Optical Tracking System, which provided precise 6-degrees-of-freedom (6-DOF) tracking. We developed a software platform to manage the interaction between the tracked physical objects and their virtual counterparts, enabling real-time data synchronization, object recognition, and virtual overlays. We designed the system to be flexible and extendable, allowing for future integration of additional objects and curriculum. This research advances the field of engineering education by demonstrating the potential of immersive digital twin environments. The laboratory provides a dynamic learning space where students can experiment, collaborate, and learn without the risks associated with live experimentation. The ability to simulate and manipulate solar installation objects under various conditions has broad implications for workforce development, particularly in renewable energy. The project also highlights the economic feasibility of using XR technologies in educational settings, offering a cost-effective solution for institutions looking to enhance their curriculum. By fostering a deeper understanding of solar energy systems, this work contributes to the broader goal of supporting the global energy transition and preparing the next generation of engineers and technicians.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA's Hybrid Reality Lab: One Giant Leap for Full Dive

This presentation demonstrates how NASA is using consumer VR headsets, game engine technology and NVIDIA's GPUs to create highly immersive future training systems augmented with extremely realistic haptic feedback, sound, additional sensory information, and how these can be used to improve the engineering workflow. Include in this presentation is an environment simulation of the ISS, where users can interact with virtual objects, handrails, and tracked physical objects while inside VR, integration of consumer VR headsets with the Active Response Gravity Offload System, and a space habitat architectural evaluation tool. Attendees will learn how the best elements of real and virtual worlds can be combined into a hybrid reality environment with tangible engineering and scientific applications.

Delgado, Francisco J.↗

Applied virtual reality at the Research Triangle Institute

Virtual Reality (VR) is a way for humans to use computers in visualizing, manipulating and interacting with large geometric data bases. This paper describes a VR infrastructure and its application to marketing, modeling, architectural walk through, and training problems. VR integration techniques used in these applications are based on a uniform approach which promotes portability and reusability of developed modules. For each problem, a 3D object data base is created using data captured by hand or electronically. The object's realism is enhanced through either procedural or photo textures. The virtual environment is created and populated with the data base using software tools which also support interactions with and immersivity in the environment. These capabilities are augmented by other sensory channels such as voice recognition, 3D sound, and tracking. Four applications are presented: a virtual furniture showroom, virtual reality models of the North Carolina Global TransPark, a walk through the Dresden Fraunenkirche, and the maintenance training simulator for the National Guard.

Montoya, R. Jorge↗

Biosystems Design by Machine Learning

Biosystems such as enzymes, pathways, and whole cells have been increasingly explored for biotechnological applications. Yet, the intricate connectivity and complexity of biosystems pose a major hurdle in designing biosystems with desired features. As -omics and other high throughput technologies have been rapidly developed, the promise of applying machine learning (ML) techniques in biosystems design has started to become a reality. ML models enable the identification of patterns within complicated biological data across multiple scales of analysis and can augment biosystems design applications by predicting new candidates for optimized performance. ML is being used at every stage of biosystems design to help find non-obvious engineering solutions with fewer design iterations. In this review, we first describe commonly used models and modeling paradigms within ML. We then discuss some applications of these models that have already shown success in biotechnological applications. Moreover, we discuss successful applications at all scales of biosystems design, including nucleic acids, genetic circuits, proteins, pathways, genomes, and bioprocess. Lastly, we discuss some limitations of these methods and potential solutions as well as prospects of the combination of ML and biosystems design.

59 BASIC BIOLOGICAL SCIENCES↗

The development, assessment and validation of virtual reality for human anatomy instruction

This research project seeks to meet the objective of science training by developing, assessing, validating and utilizing VR as a human anatomy training medium. Current anatomy instruction is primarily in the form of lectures and usage of textbooks. In ideal situations, anatomic models, computer-based instruction, and cadaver dissection are utilized to augment traditional methods of instruction. At many institutions, lack of financial resources limits anatomy instruction to textbooks and lectures. However, human anatomy is three-dimensional, unlike the one-dimensional depiction found in textbooks and the two-dimensional depiction found on the computer. Virtual reality allows one to step through the computer screen into a 3-D artificial world. The primary objective of this project is to produce a virtual reality application of the abdominopelvic region of a human cadaver that can be taken back to the classroom. The hypothesis is that an immersive learning environment affords quicker anatomic recognition and orientation and a greater level of retention in human anatomy instruction. The goal is to augment not replace traditional modes of instruction.

Marshall, Karen Benn↗

Fusion interfaces for tactical environments: An application of virtual reality technology

The term Fusion Interface is defined as a class of interface which integrally incorporates both virtual and nonvirtual concepts and devices across the visual, auditory, and haptic sensory modalities. A fusion interface is a multisensory virtually-augmented synthetic environment. A new facility has been developed within the Human Engineering Division of the Armstrong Laboratory dedicated to exploratory development of fusion interface concepts. This new facility, the Fusion Interfaces for Tactical Environments (FITE) Facility is a specialized flight simulator enabling efficient concept development through rapid prototyping and direct experience of new fusion concepts. The FITE Facility also supports evaluation of fusion concepts by operation fighter pilots in an air combat environment. The facility is utilized by a multidisciplinary design team composed of human factors engineers, electronics engineers, computer scientists, experimental psychologists, and oeprational pilots. The FITE computational architecture is composed of twenty-five 80486-based microcomputers operating in real-time. The microcomputers generate out-the-window visuals, in-cockpit and head-mounted visuals, localized auditory presentations, haptic displays on the stick and rudder pedals, as well as executing weapons models, aerodynamic models, and threat models.

Haas, Michael W.↗

Updated FY12 Ceramic Fuels Irradiation Test Plan

The Fuel Cycle Research and Development program is currently devoting resources to study of numerous fuel types with the aim of furthering understanding applicable to a range of reactors and fuel cycles. In FY11, effort within the ceramic fuels campaign focused on planning and preparation for a series of rabbit irradiations to be conducted at the High Flux Isotope Reactor located at Oak Ridge National Laboratory. The emphasis of these planned tests was to study the evolution of thermal conductivity in uranium dioxide and derivative compositions as a function of damage induced by neutron damage. Current fiscal realities have resulted in a scenario where completion of the planned rabbit irradiations is unlikely. Possibilities for execution of irradiation testing within the ceramic fuels campaign in the next several years will thus likely be restricted to avenues where strong synergies exist both within and outside the Fuel Cycle Research and Development program. Opportunities to augment the interests and needs of modeling, advanced characterization, and other campaigns present the most likely avenues for further work. These possibilities will be pursued with the hope of securing future funding. Utilization of synthetic microstructures prepared to better understand the most relevant actors encountered during irradiation of ceramic fuels thus represents the ceramic fuel campaign's most efficient means to enhance understanding of fuel response to burnup. This approach offers many of the favorable attributes embraced by the Separate Effects Testing paradigm, namely production of samples suitable to study specific, isolated phenomena. The recent success of xenon-imbedded thick films is representative of this approach. In the coming years, this strategy will be expanded to address a wider range of problems in conjunction with use of national user facilities novel characterization techniques to best utilize programmatic resources to support a science-based research program.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

A Flight Training Simulator for Instructing the Helicopter Autorotation Maneuver (Enhanced Version)

Autorotation is a maneuver that permits a safe helicopter landing when the engine loses power. A catastrophe may occur if the pilot's control inputs are incorrect, insufficient, excessive, or poorly timed. Due to the danger involved, full-touchdown autorotations are very rarely practiced. Because in-flight autorotation training is risky, time-consuming, and expensive, the objective of the project was to develop the first helicopter flight simulator expressly designed to train students in this critical maneuver. A central feature of the project was the inclusion of an enhanced version of the Pilot-Rotorcraft Intelligent Symbology Management Simulator (PRISMS), a virtual-reality system developed by Anacapa Sciences and Thought Wave. A task analysis was performed to identify the procedural steps in the autorotation, to inventory the information needed to support student task performance, to identify typical errors, and to structure the simulator's practice environment. The system provides immediate knowledge of results, extensive practice of perceptual-motor skills, part-task training, and augmented cueing in a realistic cockpit environment. Additional work, described in this report, extended the capabilities of the simulator in three areas: 1. Incorporation of visual training aids to assist the student in learning the proper appearance of the visual scene when the maneuver is being properly performed; 2. Introduction of the requirement to land at a particular spot, as opposed to the wide, flat open field initially used, and development of appropriate metrics of success; and 3. Inclusion of wind speed and wind direction settings (and random variability settings) to add a more realistic challenge in "hitting the spot."

Rogers, Steven P.↗