Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “augmented reality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Development and Validation of Sensorimotor Assessment Tasks Using Spaceflight Analogs

INTRODUCTION Exploration class missions will require a new level of crew autonomy to self-assess and treat their sensorimotor dysfunction around periods of gravitational transition, where sensorimotor disturbances are at their highest. There is a need to define sensorimotor assessment thresholds that indicate when performance in early extravehicular activities (EVAs) might be impacted or unsafe. The purpose of this study was to develop and validate a set of assessment tasks using sensorimotor spaceflight analogs including the Sensorimotor Disorientation Analog (SDA) and +3G x sustained centrifugation. METHODS For the SDA, forty-one healthy non-astronaut subjects (21 males, 20 females; Mean ± SD, Age: 34.0 ± 7.8 years) volunteered to participate in this study. Subjects attended two testing sessions: 1) familiarization to the tasks and baseline data collection, and 2) tasks completed under two levels of sensorimotor disorientation. Subjects performed a series of seven sensorimotor assessment tasks followed by two operationally relevant EVA analog tasks. The sensorimotor assessments tasks, ordered from least to most provocative of the sensorimotor system, included: postural stability, step test, prone/supine recovery from fall (RFF), four-square-step-test, obstacle negotiation with turns (OTT), kneel-and-reach, and augmented reality full-body coordination. The operational analog tasks included a top-hatch capsule egress and a 5-minute EVA traversal. Disorientation was applied via the SDA which utilized galvanic vestibular stimulation and weighted trunk, wrists, and ankles to target the vestibular and proprioceptive systems such that performance while wearing the SDA mimicked astronaut postflight performance at R+0 and R+1. Sixteen healthy non-astronaut subjects (10 males, 6 females; Age: 35.0 ± 9.5 years) completed +3G x 60-minute sustained centrifugation as an established spaceflight analog replicating the vestibular disruptions seen postflight. Subjects completed a pre-session which included familiarization to the tasks and a 5-minute centrifuge spin. The data collection session included, in order, a baseline data collection, 60-minute centrifuge spin, immediate post-centrifuge data collection, 45-minute seated recovery, and a final data collection. The same assessment and operational analog tasks were conducted as described above. RESULTS Pearsons’ correlation r-values were calculated between assessment and operational analog tasks. The two assessment tasks with the strongest correlations to the operational analog tasks were the RFF (SDA: r≥ 0.57, Centrifuge: r ≥0.55) and OTT (SDA: r≥ 0.66, Centrifuge: r ≥ 0.49). For brevity, repeated measures ANOVA with follow-up paired t-tests are discussed for these two assessment tasks. The SDA found significant differences between all three levels of disorientation for both RFF and OTT (p < 0.001). Significant differences were also found between baseline and immediate post centrifuge testing for both the RFF (p≤ 0.011) and OTT (p = 0.001). Additional results are forthcoming for both disorientation paradigms. DISCUSSION These validation studies provide evidence for down-selection of the assessment tasks that were able to distinguish between levels of sensorimotor disorientation and correlate to performance in operationally relevant tasks for use in upcoming Artemis missions.

S C Moudy↗

Limitations and Feasibility of Mini X-Ray Devices in Space Environments

LIMITATIONS AND FEASIBILITY OF MINI X-RAY DEVICES IN SPACE ENVIRONMENTS As space exploration advances toward long-duration missions, reliable medical diagnostic tools become increasingly critical. The miniature x-ray (XR) technology demonstrations by the Exploration Medical Capability (ExMC) and the Exploration Medical Integrated Product Team (XMIPT) aim to assess the feasibility and utility of miniature XR devices in spaceflight. This abstract explores the limitations of current miniature XR systems, the challenges of training crew members, the potential role of clinical decision support systems (CDSS), and the feasibility of ground-based image interpretation. We also propose the integration of miniature XR into other ExMC efforts aimed at identifying the capabilities and resources needed for future exploration class missions. One of the primary challenges with miniature XR devices is the ability to achieve specific anatomical views, particularly in the confined and weightless conditions of a spacecraft. Operators may struggle to acquire diagnostic-quality images when space is limited for proper patient positioning and the volume of the imaging device. Since space radiation and detector limitations may further impact image quality, the flexibility of the operating procedures of these devices will be critical for their success in space applications. CHALLENGES IN TRAINING CREW TO OPERATE IMAGING DEVICES Training in the skills necessary to acquire diagnostic-quality scans may be a barrier for non-clinician crewmembers. The curriculum developed for crew medical officers (CMOs) will require simplification and adaptation to fit into the highly truncated pre-flight training period. Therefore, hands-on familiarization and simulation, both pre-flight and just-in-time training during missions, will be crucial to ensuring the crew can operate the devices in real-life situations. The ability to adjust acquisition parameters must be simplified or made automatic through exam selections on equipment user interfaces, and subject and operator positioning should be assisted with laser guidance and pictorial guides. POTENTIAL FOR CDSS OR ARTIFICIAL INTELLIGENCE (AI)-ASSISTED CDSS CDSS and AI-assisted CDSS offer significant promise in assisting crew members with limited medical training. These systems could provide real-time feedback on image quality and interpretation, helping to mitigate the risks of human error during space missions. Integrating procedural guidance tools, such as virtual and augmented reality, will support crewmembers in accurately positioning patients and obtaining high-quality images. However, the success of such systems will depend on the development of robust training datasets, integration with spaceflight-rated hardware, and the medical decision-making capabilities of operators. FEASIBILITY OF GROUND INTERPRETATION AND DATA TRANSMISSION Reliance on ground-based interpretation may prove difficult for acute care during exploration class-missions due to delays in transmission with increasing distance from Earth or complete communication blackout periods. In such instances where immediate interpretation for clinical intervention is required, crew must be able to interpret the images independently or utilize AI-based assistance to do so. File sizes for XR exams can also be large if numerous images are acquired and bandwidth constraints may limit data transmissions for both radiography and ultrasound exams. FUTURE WORK AND INTEGRATION INTO THE EVIDENCE LIBRARY Future work proposes integrating miniature XR devices into NASA’s Evidence Library to address medical conditions identified as significant contributors to crew morbidity and mortality. The possibility of combining miniature XR with other imaging modalities, such as ultrasound devices, is also under investigation. In conclusion, while miniature XR technology holds potential for extraterrestrial medical systems, there are significant challenges to overcome. Training, integration of AI tools, dedicated exam protocols for microgravity, and improved data transmission systems will be key to realizing the full benefits of miniature XR technology in space.

A M Nelson↗

Discovering Digital Engineering Needs Through Human-Centered Approaches

Digital engineering has the power to transform organizations in profound ways, but digitizing legacy workflows and infusing new technologies into existing practices may not yield the intended results or address the underlying problems or bottlenecks. This paper describes the human-centered approaches used to understand stakeholder challenges and identify digital engineering needs during the design-build-test portion of the systems development lifecycle. Through a series of interactive stakeholder workshops and need-finding activities, key stakeholder needs were identified to help guide the integration of digital engineering technologies. The overarching need was for downstream system development steps to be considered further upstream in the project lifecycle in a manner that allows for proactive and interactive co-constructive idea exploration, problem resolution, and requirements considerations in a collective engagement with engineers and technicians from multiple disciplines to increase efficiency and reduce risks while offering opportunities to improve the final engineered system. Supporting this need was a wide-ranging desire from stakeholders for digital and non-digital support structures for efficient rapid iteration during system development. This work yielded several improvements for system development processes and for digital engineering infusion.

Design Science↗

Advancing Artificial Intelligence

Argonne researchers are accelerating scientific discovery with the latest virtual and augmented reality technologies. Artificial intelligence is playing a significant role in defining new research methods that bring us to resolution or discovery more quickly than ever before. Argonne is using or developing many of these technologies as AI extends to new branches of science and technology, sprouting new buds that represent various facets of human function and cognition — from learning to language processing to visual recognition. Read the story here www.anl.gov/article/welcome-to-the-machine

97 MATHEMATICS AND COMPUTING↗

The Future of Visualization Technology Vis in Practice

Two decades ago, we witnessed an inflection point as graphics programming became commoditized. A comparable inflection point may be on the horizon, this time driven by the commoditization of immersive display technology with Augmented Reality (AR), Virtual Reality (VR), and Extended Reality (XR) technologies poised to redefine our interaction with data, environments, and collaboration. To drive the technology and its application, we should consider the lessons learned from successful large-scale immersive installations. Where these installations have thrived is by enhancing traditional workflows rather than entirely replacing them, supporting improved judgments of complex spaces, direct interaction in 3D, and embedding high-dimensional data.

AR/VR↗

The Future of Robotics/Remote Handling Solutions for the Nuclear Industry - 17347

The nuclear industry is seeing a resurgence of development and utilization of advanced integrated systems; fused sensor arrays; autonomous, remote and hybrid operations; and augmented reality display and control system interfaces. A new age of 'robotics' and remote handling technologies. For many years the use of robotics in nuclear applications has been done sparingly due to technical limitations, perceived and real reliability issues, perceived and real risk concerns, and cost. Although robotics have been used successfully on a variety of projects globally, the underuse of this technology, particularly within the nuclear industry both public and private, is overwhelming. As the number of complex, high hazard projects in the nuclear industry increases, the use of robotics for maintenance, inspection, repair, decommissioning, and remediation activities will become increasingly vital in the future. The risks associated with direct human exposure are high and thus eliminating the risk, or managing it to acceptable levels by using robotics/remote handling as well as other technologies is critical to success as more extreme environment operations and tasks are undertaken. Recent and ongoing advances in technology now allows us to safely and effectively remove the direct / in-situ human element from these type of activities. All the while delivering the same or better outcomes when considering the alternatives when limited or no direct / in-situ human intervention would be allowed. The biggest obstacles the nuclear industry faces center on the cost, reliability, maintainability, effectiveness and appropriate uses of robotics/remote handling solutions in a nuclear (radiation and/or contamination) environment. In order to mitigate these concerns, a sound approach to the use of robotics needs to be addressed. This approach must include: 1. A better understanding of what technology is available and how it can be utilized across multiple sectors, e.g. Commercial Off-the-Shelf (COTS) technology modified / adapted for the nuclear sector; 2. How to implement technology in an integrated pragmatic approach that is cost effective by combining technological know-how / subject matter expertise (SME) with the equivalent subject matter expertise in regulatory, operational and maintenance requirements and limitations, such that a more complete set of functional and operational requirements can be jointly developed in the process. 3. The integration of requirements (project, site, regulatory, etc.), risk management (human factors) and technology, i.e. a marriage between political, programmatic, and technological factors. As the development of robotic/remote handling technology progresses and the successful implementation on projects is demonstrated, the better off the industry will be. This paper provides an overview of the fundamental issues facing the nuclear industry with regard to effective and successful technology selection, maturation, deployment, and integrated risk management, and offers suggestions on how we may overcome these persistent issues now and in the future. Robotic systems and solution experts that are focused on the integration of proven technologies - and that are an integrated part of the project team from its onset - are a necessity for successful planning during nuclear operations. The approach to nuclear operations should always be, 'what is the simplest, most effective way we can approach a particular project need?'. Understanding and implementing the right technologies is where the true value will be recognized. The integration of systems, which includes more than just robotics, is key when developing schemes for these types of complex operations. Often times during planning, the remote handling needs of the job are set aside to be evaluated at a later time due to risk concerns and lack of knowledge. The issue is, experts in the area of robotics or remote handling are rarely involved in the upfront planning. As a result, unnecessary constraints, risks, and obstacles are imposed on project teams to deliver. If done intelligently, this should be introduced at the forefront of planning. Using this type of sound approach leads to effective task organization for the end user. By developing an overall philosophy that leverages proven technologies with implementation expertise; i.e. solutions versus products, is a recipe for success. This is the area that is under-appreciated in the industry and one that needs to be recognized on a much broader scale. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Applying State-of-the-Art Tools for Safety-Optimized D and D - 17410

Decommissioning, Demolition and Disposal of 'hot' facilities (D3) is a relatively new science to the extent it has been part our public consciousness for ∼50-years. Due to the high hazard of uncontrolled exposure and the possibility of environmental release, the technologies applied to D3 have not embraced many of the recently developed engineering tools, especially those that leverage the digital and optical advances of the past 10-years. This summary paper provides an overview of how we are layering readily available technologies with sophisticated techniques to D3, ranging from agile program management to advanced modular construction to augmented reality. Using this approach, we are significantly improving safety and ALARA aspects while saving costs, optimizing schedules, maintaining quality and engaging the customer and the regulator in the D3 environment. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Implementation of Augmented Teleautonomy on a Hydraulic Manipulator for Nuclear Facility D and D - 19602

Robotic and remote system have been used in nuclear applications to keep the human workers from high radiation environment and enhance task performance. However, applications in the more unstructured nuclear environments present more formidable challenges to robotics and remote systems. In particular, remote operation tasks for nuclear facility D and D (or dismantling) requires dexterous manipulation of heavy objects with simple and rugged robotic equipment. To address such challenge, this paper presents a new teleoperation method of augmented teleautonomy for remote operation of tools for D and D tasks. Rather than imposing complex functionalities on the slave robot, focused innovation is directed to providing effective operator aids in such a way as to draw more performance out of human operators. In this regard, the development entails the implementation of an enhanced telerobotic operation method by incorporating multi-modal augmented reality operator aids and autonomous motion primitives. This paper presents the relevant technology development utilizing a mixed-reality telerobotic test bed, and system integration and demonstration with a rugged, hydraulic slave robot system. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Using Fiducial Markers for Pose Estimation of an OSWEC in a Wave Tank: Preprint

In this study, we consider a novel method of sensing the motion of a wave energy converter during testing in a wave flume under the influence of incoming waves. The wave energy converter considered in our research is an oscillating surge wave energy converter, which is a hinged paddle that responds to incoming waves. Motion sensing is normally done with inertial sensors, which can hinder the motion due to suspended cables that carry power and transmit signals. Our proposed method is contactless and can be implemented economically. A camera is used to record different marker patterns affixed to the moving paddle and the motion deduced by pose estimation algorithms. Fiducial markers are commonly used for robot localization and in augmented reality. There are many types of fiducial markers, including ArUco-type markers which are accurate, fast and robust. The system consists of markers attached to the paddle element and recorded using a machine vision camera. A pose estimation algorithm is then applied to the detected markers to estimate the tilt of the paddle. In this work, we examine the challenges of image acquisition and calibration for underwater targets, compare the motion obtained by this new system with a calibrated tilt sensor and identify areas where the new system may be superior.

computer vision↗

CommAwareNet: Towards Communication‐Aware Smart Facilities: Designing an Energy‐Efficient High‐Data‐Rate and Reliable Hybrid THz/VLC Comm. Arch. Reinforced with Intelligent Surfaces for Future Network

This project will develop foundations for energy-efficient, high-data-rate, and reliable communication links for next-generation advanced wireless technologies. The Internet of Everything is emerging as a major technology that can establish a massively connected network among both human-type users and machine-type devices. Particularly, massive machine-type communication is becoming the dominant communication paradigm for numerous emerging areas, including data center networking and wireless backhaul, healthcare (e-health), manufacturing (industry 4.0), utilities, transportation (connected cars and public safety), and virtual or augmented reality for human-machine interaction applications. The rising demand for high data rates, especially in indoor scenarios and densely populated outdoor environments, will eventually overload conventional RF-based technologies. This project aims to conduct fundamental research to design, optimize, and validate a communication-aware, energy-efficient, and reliable network (CommAwareNet) architecture for future smart facilities and environments that require unprecedented high throughput needs for massive numbers of the machine- and human-type users. The scientific and technical merit of this work lies in the research required to bring the proposed transformative CommAwareNet solution into fully functional and practical architecture.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The NASA Augmented/Virtual Reality Lab: The State of the Art at KSC

The NASA Augmented Virtual Reality (AVR) Lab at Kennedy Space Center is dedicated to the investigation of Augmented Reality (AR) and Virtual Reality (VR) technologies, with the goal of determining potential uses of these technologies as human-computer interaction (HCI) devices in an aerospace engineering context. Begun in 2012, the AVR Lab has concentrated on commercially available AR and VR devices that are gaining in popularity and use in a number of fields such as gaming, training, and telepresence. We are working with such devices as the Microsoft Kinect, the Oculus Rift, the Leap Motion, the HTC Vive, motion capture systems, and the Microsoft Hololens. The focus of our work has been on human interaction with the virtual environment, which in turn acts as a communications bridge to remote physical devices and environments which the operator cannot or should not control or experience directly. Particularly in reference to dealing with spacecraft and the oftentimes hazardous environments they inhabit, it is our hope that AR and VR technologies can be utilized to increase human safety and mission success by physically removing humans from those hazardous environments while virtually putting them right in the middle of those environments.

Little, William↗

Furthering Precision in Sentinel Node Navigational Surgery for Oral Cancer: a Novel Triple Targeting System

To describe an innovative sentinel lymph node (SLN) guidance approach using a radionuclide tracer, 3D augmented reality-guided imaging, and near infrared (NIR) fluorescence over-lay imaging with hand-held probes to optimize accuracy, efficiency, and precise navigation for sentinel node (SN) localization in head and neck cancer. In a cT1N0M0 squamous cell carcinoma of the tongue, pre-operative radionuclide lymphoscintigraphy was performed with a sentinel node-specific radiolabeled tracer. Intraoperatively, a 3D hand-held augmented reality (AR) scanning SPECT probe assessed concordance of the SN with pre-operative SPECT-CT images. The real-time optical video was linked to the SPECT-CT images for added precision. Final guidance to the SN was performed using ICG fluorescence imaging. Dynamic and SPECT-CT showed bilateral lymphatic drainage from the tumor. The 3D hand-held AR SPECT probe SN localization was concordant with pre-operative imaging. The optical video successfully demonstrated the lymphatic drainage in real-time through a unique overlay fluorescence image. The ICG localized to the same nodes identified by both the SPECT-CT and hand-held SPECT images. The use of dual radiation and fluorescence tracers improved SN detection, especially for SN close to the injection site. The hand-held probes allowed the surgeon to dissect continuously, without needing to change tools. The combination of augmented reality, nuclear medicine, and over-lay fluorescence imaging allowed greater accuracy for matching the preoperative imaging with intraoperative identification and precisely guiding the dissection. This method uniquely permitted the surgeon to efficiently dissect the SN with accurate visualization and optimal precision.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Incorporating Speech Recognition into a Natural User Interface

The Augmented/ Virtual Reality (AVR) Lab has been working to study the applicability of recent virtual and augmented reality hardware and software to KSC operations. This includes the Oculus Rift, HTC Vive, Microsoft HoloLens, and Unity game engine. My project in this lab is to integrate voice recognition and voice commands into an easy to modify system that can be added to an existing portion of a Natural User Interface (NUI). A NUI is an intuitive and simple to use interface incorporating visual, touch, and speech recognition. The inclusion of speech recognition capability will allow users to perform actions or make inquiries using only their voice. The simplicity of needing only to speak to control an on-screen object or enact some digital action means that any user can quickly become accustomed to using this system. Multiple programs were tested for use in a speech command and recognition system. Sphinx4 translates speech to text using a Hidden Markov Model (HMM) based Language Model, an Acoustic Model, and a word Dictionary running on Java. PocketSphinx had similar functionality to Sphinx4 but instead ran on C. However, neither of these programs were ideal as building a Java or C wrapper slowed performance. The most ideal speech recognition system tested was the Unity Engine Grammar Recognizer. A Context Free Grammar (CFG) structure is written in an XML file to specify the structure of phrases and words that will be recognized by Unity Grammar Recognizer. Using Speech Recognition Grammar Specification (SRGS) 1.0 makes modifying the recognized combinations of words and phrases very simple and quick to do. With SRGS 1.0, semantic information can also be added to the XML file, which allows for even more control over how spoken words and phrases are interpreted by Unity. Additionally, using a CFG with SRGS 1.0 produces a Finite State Machine (FSM) functionality limiting the potential for incorrectly heard words or phrases. The purpose of my project was to investigate options for a Speech Recognition System. To that end I attempted to integrate Sphinx4 into a user interface. Sphinx4 had great accuracy and is the only free program able to perform offline speech dictation. However it had a limited dictionary of words that could be recognized, single syllable words were almost impossible for it to hear, and since it ran on Java it could not be integrated into the Unity based NUI. PocketSphinx ran much faster than Sphinx4 which would've made it ideal as a plugin to the Unity NUI, unfortunately creating a C# wrapper for the C code made the program unusable with Unity due to the wrapper slowing code execution and class files becoming unreachable. Unity Grammar Recognizer is the ideal speech recognition interface, it is flexible in recognizing multiple variations of the same command. It is also the most accurate program in recognizing speech due to using an XML grammar to specify speech structure instead of relying solely on a Dictionary and Language model. The Unity Grammar Recognizer will be used with the NUI for these reasons as well as being written in C# which further simplifies the incorporation.

Chapa, Nicholas↗

Hybrid Reality Lab Capabilities - Video 2

Our Hybrid Reality and Advanced Operations Lab is developing incredibly realistic and immersive systems that could be used to provide training, support engineering analysis, and augment data collection for various human performance metrics at NASA. To get a better understanding of what Hybrid Reality is, let's go through the two most commonly known types of immersive realities: Virtual Reality, and Augmented Reality. Virtual Reality creates immersive scenes that are completely made up of digital information. This technology has been used to train astronauts at NASA, used during teleoperation of remote assets (arms, rovers, robots, etc.) and other activities. One challenge with Virtual Reality is that if you are using it for real time-applications (like landing an airplane) then the information used to create the virtual scenes can be old (i.e. visualized long after physical objects moved in the scene) and not accurate enough to land the airplane safely. This is where Augmented Reality comes in. Augmented Reality takes real-time environment information (from a camera, or see through window, and places digitally created information into the scene so that it matches with the video/glass information). Augmented Reality enhances real environment information collected with a live sensor or viewport (e.g. camera, window, etc.) with the information-rich visualization provided by Virtual Reality. Hybrid Reality takes Augmented Reality even further, by creating a higher level of immersion where interactivity can take place. Hybrid Reality takes Virtual Reality objects and a trackable, physical representation of those objects, places them in the same coordinate system, and allows people to interact with both objects' representations (virtual and physical) simultaneously. After a short period of adjustment, the individuals begin to interact with all the objects in the scene as if they were real-life objects. The ability to physically touch and interact with digitally created objects that have the same shape, size, location to their physical object counterpart in virtual reality environment can be a game changer when it comes to training, planning, engineering analysis, science, entertainment, etc. Our Project is developing such capabilities for various types of environments. The video outlined with this abstract is a representation of an ISS Hybrid Reality experience. In the video you can see various Hybrid Reality elements that provide immersion beyond just standard Virtual Reality or Augmented Reality.

Delgado, Francisco J.↗