Results for “cognitive capture” — Page 2

NASA NTRS2009

Spinoff 2009

Topics covered include: Image-Capture Devices Extend Medicine's Reach; Medical Devices Assess, Treat Balance Disorders; NASA Bioreactors Advance Disease Treatments; Robotics Algorithms Provide Nutritional Guidelines; "Anti-Gravity" Treadmills Speed Rehabilitation; Crew Management Processes Revitalize Patient Care; Hubble Systems Optimize Hospital Schedules; Web-based Programs Assess Cognitive Fitness; Electrolyte Concentrates Treat Dehydration; Tools Lighten Designs, Maintain Structural Integrity; Insulating Foams Save Money, Increase Safety; Polyimide Resins Resist Extreme Temperatures; Sensors Locate Radio Interference; Surface Operations Systems Improve Airport Efficiency; Nontoxic Resins Advance Aerospace Manufacturing; Sensors Provide Early Warning of Biological Threats; Robot Saves Soldier's Lives Overseas (MarcBot); Apollo-Era Life Raft Saves Hundreds of Sailors; Circuits Enhance Scientific Instruments and Safety Devices; Tough Textiles Protect Payloads and Public Safety Officers; Forecasting Tools Point to Fishing Hotspots; Air Purifiers Eliminate Pathogens, Preserve Food; Fabrics Protect Sensitive Skin from UV Rays; Phase Change Fabrics Control Temperature; Tiny Devices Project Sharp, Colorful Images; Star-Mapping Tools Enable Tracking of Endangered Animals; Nanofiber Filters Eliminate Contaminants; Modeling Innovations Advance Wind Energy Industry; Thermal Insulation Strips Conserve Energy; Satellite Respondent Buoys Identify Ocean Debris; Mobile Instruments Measure Atmospheric Pollutants; Cloud Imagers Offer New Details on Earth's Health; Antennas Lower Cost of Satellite Access; Feature Detection Systems Enhance Satellite Imagery; Chlorophyll Meters Aid Plant Nutrient Management; Telemetry Boards Interpret Rocket, Airplane Engine Data; Programs Automate Complex Operations Monitoring; Software Tools Streamline Project Management; Modeling Languages Refine Vehicle Design; Radio Relays Improve Wireless Products; Advanced Sensors Boost Optical Communication, Imaging; Tensile Fabrics Enhance Architecture Around the World; Robust Light Filters Support Powerful Imaging Devices; Thermoelectric Devices Cool, Power Electronics; Innovative Tools Advance Revolutionary Weld Technique; Methods Reduce Cost, Enhance Quality of Nanotubes; Gauging Systems Monitor Cryogenic Liquids; Voltage Sensors Monitor Harmful Static; and Compact Instruments Measure Heat Potential.

Source record↗

NASA NTRS1990

SHAPA: An interactive software tool for protocol analysis applied to aircrew communications and workload

As modern transport environments become increasingly complex, issues such as crew communication, interaction with automation, and workload management have become crucial. Much research is being focused on holistic aspects of social and cognitive behavior, such as the strategies used to handle workload, the flow of information, the scheduling of tasks, the verbal and non-verbal interactions between crew members. Traditional laboratory performance measures no longer sufficiently meet the needs of researchers addressing these issues. However observational techniques are better equipped to capture the type of data needed and to build models of the requisite level of sophistication. Presented here is SHAPA, an interactive software tool for performing both verbal and non-verbal protocol analysis. It has been developed with the idea of affording the researchers the closest possible degree of engagement with protocol data. The researcher can configure SHAPA to encode protocols using any theoretical framework or encoding vocabulary that is desired. SHAPA allows protocol analysis to be performed at any level of analysis, and it supplies a wide variety of tools for data aggregation, manipulation. The output generated by SHAPA can be used alone or in combination with other performance variables to get a rich picture of the influences on sequences of verbal or nonverbal behavior.

James, Jeffrey M.↗

NASA NTRS2017

Modeling Pilot Pulse Control

In this study, behavioral models are developed that closely reproduced pulsive control response of two pilots from the experimental pool using markedly different control techniques (styles) while conducting a tracking task. An intriguing find was that the pilots appeared to: 1) produce a continuous, internally-generated stick signal that they integrated in time; 2) integrate the actual stick position; and 3) compare the two integrations to issue and cease pulse commands. This suggests that the pilots utilized kinesthetic feedback in order to perceive and integrate stick position, supporting the hypothesis that pilots can access and employ the proprioceptive inner feedback loop proposed by Hess' pilot Structural Model. The Pulse Models used in conjunction with the pilot Structural Model closely recreated the pilot data both in the frequency and time domains during closed-loop simulation. This indicates that for the range of tasks and control styles encountered, the models captured the fundamental mechanisms governing pulsive and control processes. The pilot Pulse Models give important insight for the amount of remnant (stick output uncorrelated with the forcing function) that arises from nonlinear pilot technique, and for the remaining remnant arising from different sources unrelated to tracking control (i.e. neuromuscular tremor, reallocation of cognitive resources, etc.).

workload↗

NASA NTRSDate not supplied

GeoAI advances in specific landform mapping

Landform mapping (also referred to as geomorphology or geomorphometry) can be divided into two domains: general and specific (Evans 2012). Whereas general landform mapping categorizes all elements of the study area into landform classes, such as ridges, valleys, peaks, and depressions, the mapping of specific landforms requires the delineation (even if fuzzy) of individual landforms. The former is mainly driven by physical properties such as elevation, slope, and curvature. The latter, however, must consider the cognitive (human) reasoning that discriminates individual landforms in addition to these physical properties (Arundel and Sinha 2018). Both mapping forms are important. General geomorphometry is needed to understand geological and ecological processes and as boundary layer input to climate and environmental models. Specific geomorphometry supports such activities as disaster management and recovery, emergency response, transportation, and navigation. In the United States, individual landforms of interest are named in the U.S. Geological Survey (USGS) Geographic Names Information System, a point dataset captured specifically to digitize geographic names from the USGS Historical Topographic Map Collection (HTMC). Named landform extent is represented only by the name placement in the HTMC. Recent work has investigated CNN-based deep learning methods to capture these extents in machine-readable form. These studies first relied on physical properties (Arundel et al. 2020) and then included the HTMC as a band in RGB images in limited testing (Arundel et al. 2023). Results from the HTMC dataset surpassed those using just physical properties and using the HTMC alone performed best due to the hillshading and elevation (contour) data incorporated into the topographic maps. However, results fell short of an operational capacity to map all named landforms in the United States. Thus, our current work expands upon past research by focusing on the HTMC and physical information as inputs and the named landform label extents. Specifically, we propose to leverage pre-trained foundation models for segmentation and optical character recognition (OCR) models to jointly map landforms in the United States. Our approach aims to bridge the disparities among the independent information sources to facilitate informed decision-making. The modeling pipeline performs (1) segmentation using the physical information and (2) information extraction using OCR, in parallel. Then a computer vision approach merges the two branches into a labeled segmentation. References: Arundel, Samantha T., Wenwen Li, and Sizhe Wang. 2020. “GeoNat v1.0: A Dataset for Natural Feature Mapping with Artificial Intelligence and Supervised Learning.” Transactions in GIS 24 (3): 556–72. https://doi.org/10.1111/tgis.12633. Arundel, Samantha T, and Gaurav Sinha. 2018. “Validating GEOBIA Based Terrain Segmentation and Classification for Automated Delineation of Cognitively Salient Landforms BT - Proceedings of Workshops and Posters at the 13th International Conference on Spatial Information Theory (COSIT 2017).” In Proceedings of Workshops and Posters at the 13th International Conference on Spatial Information Theory (COSIT 2017), Lecture Notes in Geoinformation and Cartography, edited by Paolo Fogliaroni, Andrea Ballatore, and Eliseo Clementini, 9–14. Cham: Springer International Publishing. Arundel, Samantha T., Gaurav Sinha, Wenwen Li, David P. Martin, Kevin G. McKeehan, and Philip T. Thiem. 2023. “Historical Maps Inform Landform Cognition in Machine Learning.” Abstracts of the ICA 6 (August): 1–2. https://doi.org/10.5194/ica-abs-6-10-2023. Evans, Ian S. 2012. “Geomorphometry and Landform Mapping: What Is a Landform?” Geomorphology 137 (1): 94–106. https://doi.org/10.1016/j.geomorph.2010.09.029.

machine learning↗

DOE OSTI2025

Simulating nationwide coupled disease and fear spread in an agent-based model

Human cognitive responses, behavioral responses, and disease dynamics co-evolve over the course of any disease outbreak, and can result in complex feedbacks. We present a dynamic agent-based model that explicitly couples the spread of disease with the spread of fear surrounding the disease, implemented within the EpiCast simulation framework. EpiCast models transmission within a realistic synthetic population, capturing individual-level interactions. In our model, fear propagates through both in-person contact and broadcast media, prompting individuals to adopt protective behaviors that reduce disease spread. In order to better understand these coupled dynamics, we create and compare a range of compartmental models to ensure that introducing additional disease states does not prevent the emergence of multiple waves in these simpler models. Additionally, we compare a range of behavioral scenarios within EpiCast, varying the level and intensity of fear and behavior change. Our results show that the addition of asymptomatic, exposed, and pre-symptomatic disease states can impact both the rate at which an outbreak progresses and its overall trajectory in compartmental models. In EpiCast, the combination of non-local fear spread via broadcasters and strong behavioral responses by fearful individuals generally leads to multiple epidemic waves, an outcome that occurs only within a narrow parameter range when fear spreads purely through local contact. Accounting for the coupled spread of fear and disease is critical for understanding disease dynamics and designing timely, targeted responses to emerging infectious threats.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2024

Performance Year 1 Technical Report - OPEN COG Grid: Extendable Coherent Models-Datasets for Cognitive Power Grids

The OPEN COG Grid project is a collaborative effort between LLNL, NREL, and Texas A&M University (TAMU) to develop synthetic power system datasets that (i) contain all technical information that would be available in a real system, allowing to conduct studies ranging from dynamic simulation to long term planning studies; ii) are accessible to researchers from the broader data sciences community, as oppossed to power system experts only; and (iii) This report summarizes the work conducted during the first 15 months of execution of the project. These activities encompassed: 1. Conduct a survey of existing open data sets and open source power systems simulators, their supported use cases, and accessibility (Chapter 1). 2. Define a new extensible specification for power system data, covering all parameters necessary for most computational use cases (Chapter 2). 3. Collecting real technical system data to complete missing parameters in existing open source datasets (Chapter 3). 4. Develop models that capture the behavior of emergent actors in power grids, neglected by existing datasets; aggregated residential demand response (Chapter 4) and demand response of cryptocurrency miners (Chapter 5). 5. Collect detailed spatial information on distributed energy resources, particular, solar photovoltaic facilities (Chapter 6). The following chapters provide detailed descriptions of these tasks, the assumptions taken, and their findings. In conducting these tasks, the project team produced: two (accepted) conference papers; one journal paper under submission; one draft journal paper pending submission; released one repository with the developed power system data specification, with documentation and examples; and one extended dataset for the Texas power grid under review for release. The team hopes these contributions will enhance access to power system data and remove barriers to the development of new computational techniques for power systems, particularly, those inspired by cognitive sciences.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2023

Developing a Hybrid Spacesuit Simulator as a Research Tool for Assessing Extravehicular Activity Relevant Workload

Conducting human tests in a pressurized spacesuit is limited by availability, cost, and manpower; however, pressurized spacesuits are not always needed depending on the objectives of testing, including the development and testing of new informatics capabilities. The Human Physiology, Performance, Protection & Operations Laboratory (H-3PO) at NASA is developing a Hybrid Spacesuit Simulator (HS3) to support testing and characterization of human performance during analog planetary exploration extravehicular activities (EVAs). The goal of HS3 is to create a low-cost, modular, and unpressurized spacesuit simulator as a research tool that provides relevant physical and cognitive workload approximations with EVA-like immersion. HS3 consists of a soft outer suit, thermal control, gloves, boots, helmet, and integrated bioinformatics and communications. Baseline HS3 assessments were performed during 3-hour EVA simulations in two different subjects (DEMO1 and DEMO2) that included traverses at variable resistances and geological sampling activities. Liquid cooling garment (LCG) temperature, mean skin temperature, heart rate, motion capture, and metabolic rate were collected during each 3-hour simulated EVA. During DEMO1 and DEMO2, baseline metabolic rates at rest were 836 ± 327 BTU/hr and 869 ± 207 BTU/hr and increased to 2124 ± 548 BTU/hr and 2269 ± 559 BTU/hr, respectively, during 500m traverse. Average inlet LCG temperatures were 29.57 ± 6.62 °C and 25.63 ± 6.48 °C for DEMO1 and DEMO2 with increased outlet LCG temperatures of 33.53 ± 6.62 °C and 29.21 ± 4.79 °C, respectively. Overall, HS3 will enable future studies to characterize EVA tasks, human performance, and test future EVA capabilities in analog test environments without the need for pressurized suited environments.

Monica Hew↗

NASA NTRSDate not supplied

Developing A Hybrid Spacesuit Simulator as A Research Tool for Assessing Extravehicular Activity Relevant Workload

Conducting human tests in a pressurized spacesuit is limited by availability, cost, and manpower; however, pressurized spacesuits are not always needed depending on the objectives of testing, including the development and testing of new informatics capabilities. The Human Physiology, Performance, Protection & Operations Laboratory (H-3PO) at NASA is developing a Hybrid Spacesuit Simulator (HS3) to support testing and characterization of human performance during analog planetary exploration extravehicular activities (EVAs). The goal of HS3 is to create a low-cost, modular, and unpressurized spacesuit simulator as a research tool that provides relevant physical and cognitive workload approximations with EVA-like immersion. HS3 consists of a soft outer suit, thermal control, gloves, boots, helmet, and integrated bioinformatics and communications. Baseline HS3 assessments were performed during 3-hour EVA simulations in two different subjects (DEMO1 and DEMO2) that included traverses at variable resistances and geological sampling activities. Liquid cooling garment (LCG) temperature, mean skin temperature, heart rate, motion capture, and metabolic rate were collected during each 3-hour simulated EVA. During DEMO1 and DEMO2, baseline metabolic rates at rest were 836 ± 327 BTU/hr and 869 ± 207 BTU/hr and increased to 2124 ± 548 BTU/hr and 2269 ± 559 BTU/hr, respectively, during 500m traverse. Average inlet LCG temperatures were 29.57 ± 6.62 °C and 25.63 ± 6.48 °C for DEMO1 and DEMO2 with increased outlet LCG temperatures of 33.53 ± 6.62 °C and 29.21 ± 4.79 °C, respectively. Overall, HS3 will enable future studies to characterize EVA tasks, human performance, and test future EVA capabilities in analog test environments without the need for pressurized suited environments.

Suit simulator↗

NASA NTRS1997

Coupling of fingertip somatosensory information to head and body sway

Light touch contact of a fingertip with a stationary surface can provide orientation information that enhances control of upright stance. Slight changes in contact force at the fingertip provide sensory cues about the direction of body sway, allowing attenuation of sway. In the present study, we asked to which extent somatosensory cues are part of the postural control system, that is, which sensory signal supports this coupling? We investigated postural control not only when the contact surface was stationary, but also when it was moving rhythmically (from 0.1 to 0.5 Hz). In doing so, we brought somatosensory cues from the hand into conflict with other parts of the postural control system. Our focus was the temporal relationship between body sway and the contact surface. Postural sway was highly coherent with contact surface motion. Head and body sway assumed the frequency of the moving contact surface at all test frequencies. To account for these results, a simple model was formulated by approximating the postural control system as a second-order linear dynamical system. The influence of the touch stimulus was captured as the difference between the velocity of the contact surface and the velocity of body sway, multiplied by a coupling constant. Comparison of empirical results (relative phase, coherence, and gain) with model predictions supports the hypothesis of coupling between body sway and touch cues through the velocity of the somatosensory stimulus at the fingertip. One subject, who perceived movement of the touch surface, demonstrated weaker coupling than other subjects, suggesting that cognitive mechanisms introduce flexibility into the postural control scheme.

NASA Discipline Neuroscience↗

NASA NTRSDate not supplied

The Impact of Mood and Affect on Operationally Relevant Performance Among Astronaut-Like Individuals in Space Analog Settings

Crewmembers of future space exploration missions will face many challenges associated with living and working in extreme conditions for prolonged periods of time. Thus, it is imperative to understand how spaceflight hazards impact mood and affect, cognitive functioning, and ultimately, operational performance. The Human Factors and Behavioral Performance Exploration Measures (HFBP-EM) suite is a set of standardized measures used to assess behavioral health and performance related to future exploration class space missions. HFBP-EM includes the Profile of Mood States (POMS) as a self-report measure of psychological distress, and the Robotic On-Board Trainer (ROBoT) as an operationally relevant task that simulates the capture and grappling maneuvers required to operate the Canadarm2 on the International Space Station. Although previous space analog studies have demonstrated fluctuations in mood and affect ratings assessed by the POMS (Basner, 2014), no studies to date have examined impacts on operationally relevant tasks. Here we investigate the relationship between POMS ratings and performance on the ROBoT task in space analog settings.

S I Dev↗

NASA NTRS2004

Situation Awareness Implications of Adaptive Automation of Air Traffic Controller Information Processing Functions

The goal of this research was to define a measure of situation awareness (SA) in an air traffic control (ATC) task and to assess the influence of adaptive automation (AA) of various information processing functions on controller perception, comprehension and projection. The measure was also to serve as a basis for defining and developing an approach to triggering dynamic control allocations, as part of AA, based on controller SA. To achieve these objectives, an enhanced version of an ATC simulation (Multitask (copyright)) was developed for use in two human factors experiments. The simulation captured the basic functions of Terminal Radar Approach Control (TRACON) and was capable of presenting to operators four different modes of control, including information acquisition, information analysis, decision making and action implementation automation, as well as a completely manual control mode. The SA measure that was developed as part of the research was based on the Situation Awareness Global Assessment Technique (SAGAT), previous goal-directed task analyses of enroute control and TRACON, and a separate cognitive task analysis on the ATC simulation. The results of the analysis on Multitask were used as a basis for formulating SA queries as part of the SAGAT-based approach to measuring controller SA, which was used in the experiments. A total of 16 subjects were recruited for both experiments. Half the subjects were used in Experiment #1, which focused on assessing the sensitivity and reliability of the SA measurement approach in the ATC simulation. Comparisons were made of manual versus automated control. The remaining subjects were used in the second experiment, which was intended to more completely describe the SA implications of AA applied to specific controller information processing functions, and to describe how the measure could ultimately serve as a trigger of dynamic function allocations in the application of AA to ATC. Comparisons were made of the sensitivity of the SA measure to automation manipulations impacting both higher-order information processing functions, such as information analysis and decision making, versus lower-order functions, including information acquisition and action implementation. All subjects were exposed to all forms of AA of the ATC task and the manual control condition. The approach to AA used in both experiments was to match operator workload, assessed using a secondary task, to dynamic control allocations in the primary task. In total, the subjects in each experiment participated in 10 trials with each lasting between 45 minutes and 1 hour. In both experiments, ATC performance was measured in terms of aircraft cleared, conflicting, and collided. Secondary task (gauge monitoring) performance was assessed in terms of a hit-to-signal ratio. As part of the SA measure, three simulation freezes were conducted during each trial to administer queries on Level 1, 2, and 3 SA.

Kaber, David B.↗

NASA NTRSDate not supplied

GeoAI Advances in Specific Landform Mapping

Landform mapping (also referred to as geomorphology or geomorphometry) can be divided into two domains: general and specific (Evans 2012). Whereas general landform mapping categorizes all elements of the study area into landform classes, such as ridges, valleys, peaks, and depressions, the mapping of specific landforms requires the delineation (even if fuzzy) of individual landforms. The former is mainly driven by physical properties such as elevation, slope, and curvature. The latter, however, must consider the cognitive (human) reasoning that discriminates individual landforms in addition to these physical properties (Arundel and Sinha 2018). Both mapping forms are important. General geomorphometry is needed to understand geological and ecological processes and as boundary layer input to climate and environmental models. Specific geomorphometry supports such activities as disaster management and recovery, emergency response, transportation, and navigation. In the United States, individual landforms of interest are named in the U.S. Geological Survey (USGS) Geographic Names Information System, a point dataset captured specifically to digitize geographic names from the USGS Historical Topographic Map Collection (HTMC). Named landform extent is represented only by the name placement in the HTMC. Recent work has investigated CNN-based deep learning methods to capture these extents in machine-readable form. These studies first relied on physical properties (Arundel et al. 2020) and then included the HTMC as a band in RGB images in limited testing (Arundel et al. 2023). Results from the HTMC dataset surpassed those using just physical properties and using the HTMC alone performed best due to the hillshading and elevation (contour) data incorporated into the topographic maps. However, results fell short of an operational capacity to map all named landforms in the United States. Thus, our current work expands upon past research by focusing on the HTMC and physical information as inputs and the named landform label extents. Specifically, we propose to leverage pre-trained foundation models for segmentation and optical character recognition (OCR) models to jointly map landforms in the United States. Our approach aims to bridge the disparities among the independent information sources to facilitate informed decision-making. The modeling pipeline performs (1) segmentation using the physical information and (2) information extraction using OCR, in parallel. Then a computer vision approach merges the two branches into a labeled segmentation.

machine learning↗

NASA NTRSDate not supplied

Manual Crew Override of Vehicle Landings Following G-Transitions

BACKGROUND Manual control during exploration spaceflight consists of both planned automated supervisory control and unplanned crew override. This crew override capability is critical to enable overall mission success during landing contingencies. However, the introduction of manual override capabilities must be implemented to enable crews to mitigate risks introduced by human error. Adaptive changes in the sensorimotor system can manifest during g-transitions as spatial disorientation. While training and landing aids enable successful landing through disorientation, these adaptive changes may increase cognitive demand that needs to be accounted for in the manual control strategy. It is important to characterize these effects as soon as possible following the G-transition to develop appropriate countermeasures. METHODS In this study, we will examine operational piloting tasks following International Space Station (ISS) missions in a simulated lunar landing using a six-degree-of-freedom (6DOF) motion base. The motion-based simulation will be implemented in our laboratory at the Johnson Space Center (JSC) and will be available within hours following the return from commercial crew landings. The primary goals of this study are (1) to understand the impact of spaceflight on crew ability to perform manual crew override tasks, (2) to examine how adaptive changes in vestibular and cognitive function relate to changes in manual crew override proficiency, and (3) compare performance during late “just-in-time” on-board training with early post-flight crew performance. The impact of spaceflight on piloting capability will be assessed from pre- versus post-flight changes in crewmembers assigned to either short duration (< 30 day) or long duration (~6- month) missions to the International Space Station (ISS). Individual differences in post-flight vestibular and cognitive changes include motion sickness reports, measures of tilt motion perception accuracy and precision, and dual-task tracking. During the 6DOF lunar simulation, the crew will manually takeover attitude and rate-of-descent to the nominal or re-designated landing aim point during the approach phase. The outcome measures for the lunar crew override tasks will be the percent time maintaining actual vehicle states, e.g., attitude and rate-of-descent, within recommended guidance during the landing approach, number and maximum deviation outside limits, and root mean square error (RMSE). Given that “just-in-time” (JIT) training is an operational expectation for the Human Landing System (HLS) program, all participants will perform late inflight JIT training for each manual crew override task in which they will participate. Crew proficiency will be captured inflight during JIT training that will be implemented on a laptop with hand controllers to allow the crewmember to practice the landing task procedures like the approach implemented for JIT training with Shuttle landing and ISS telerobotic tasks. COUNTERMEASURE UPDATES The lunar lander simulation has been implemented onto a two-laptop display system where subjects select from alternative landing points, based upon avoiding hazards that the on-board system identifies during the landing. The subject makes inputs using a rotational hand controller (i.e., joystick) and a translational hand controller, which are processed by simulated vehicle dynamics to update the vehicle attitude and rate of descent. Subjects will utilize a combination of flight, situation, and status displays to monitor the state of the simulated vehicle. The simulation exports tilt and translation movements to a 6DOF motion base synced to visual movements and provides representative vestibular cues to the subject while performing the task inside an enclosed cabin featuring a simulated lunar out-the-window view and a heads-down lander cockpit view. RELEVANCE This project will deliver an operational demonstration of crew override capability following spaceflight and identify potential deficits that may require remediation. Comparison of individual vestibular and cognitive changes with crew override performance will help better characterize the manual control risks associated with sensorimotor alterations. The inclusion of “just-in-time” on-board training will ensure we are characterizing changes in override proficiency with this expected countermeasure in place. ACKNOWLEDGEMENTS: The authors acknowledge contributions from Draper, HLS Joint Test Panel, Crew HLS Interfaces for Piloting Working Group, and Dynamic Skills Trainer (DST) Lab toward the development of the lunar landing simulation. This project is funded by the Human Health Countermeasures Element.

Austin M Bollinger↗

NASA NTRSDate not supplied

How Autonomous Intelligent Systems Can Facilitate Earth-independent Medical Care: Going Beyond Telepresence

During the last decade, teleoperated robotic systems have extended humans’ sensorimotor competence to digitally fly beyond the physical barrier of distance and scale and thus transmit sensorimotor skills of the human through direct communication. Telepresence capabilities have enabled tele-physical remote access at small scales thanks to telerobotic mediums. Although the concept was initially motivated by space applications, such technologies quickly have expanded into the medical domain and resulted in teleoperated medical robots, including telerobotic surgical systems (such as the da Vinci surgical system). Effective telepresence fundamentally depends on an agile, reliable, and secure communication medium that can transmit real-time information between the operator and a remote device. However, direct telepresence may not be achievable for long-duration exploration spaceflight missions. Thus, autonomous systems and local intelligence represent potential solutions to the aforementioned issues. One example solution employs demonstration systems which enable learning from the pre-captured inputs of a skilled human operator. These will be computationally modeled and later probabilistically replicated toward the completion of remote physical tasks when direct telepresence is not viable - such as under communication blackout conditions. In other words, trained autonomous systems (e.g., robots) can perform remote operations that mimic the physical performance of experts during remote operations/training. Beyond learning the physics of the task, autonomous agents can also be used to conduct algorithmic decision-making that mimics the higher-level cognition of the expert. Thus, using an autonomous system, pre-trained cognitive and manipulation-based skills can be leveraged (acquired during pre-mission events) to produce digital twins of an intelligent operator. Such systems can be used for the real-time conduction of intricate tasks in complex and unstructured environments. Such systems will operationalize “cognitive digital twins” and can expand the reach of human cognition and manipulation through the power of data-driven learning from demonstration algorithms. This system category will be discussed as a fully autonomous operation in this talk. In addition to the above, we will also propose and discuss the possibility of partial-automation using remote intelligence and remote sensing. In contrast to full automation, partial automation can close the loop through a local operator equipped with augmented sensory awareness through wearable systems. Such technologies will allow the local operator to conduct delicate tasks while being guided using sensory augmentation and being monitored to gauge her/his level of cognitive focus and performance. The difference with the previous category is that a remote human will conduct the task. Further, rather than making a digital twin of human cognition, we will augment the control inputs of the local human to match those of the skilled expert operator who is not accessible in real-time. Going beyond classic telepresence and thus approaching intelligent telepresence, our vision is that autonomous agents will eventually enable the safe, consistent and efficient delivery of complex, remote and smart medical care during space exploration across operators in an Earth-independent fashion. We will discuss our collective vision from NASA and MERIIT@NYU lab in this talk.

Telepresence↗

NASA NTRS1998

BioSIGHT: Interactive Visualization Modules for Science Education

Redefining science education to harness emerging integrated media technologies with innovative pedagogical goals represents a unique challenge. The Integrated Media Systems Center (IMSC) is the only engineering research center in the area of multimedia and creative technologies sponsored by the National Science Foundation. The research program at IMSC is focused on developing advanced technologies that address human-computer interfaces, database management, and high-speed network capabilities. The BioSIGHT project at is a demonstration technology project in the area of education that seeks to address how such emerging multimedia technologies can make an impact on science education. The scope of this project will help solidify NASA's commitment for the development of innovative educational resources that promotes science literacy for our students and the general population as well. These issues must be addressed as NASA marches toward the goal of enabling human space exploration that requires an understanding of life sciences in space. The IMSC BioSIGHT lab was established with the purpose of developing a novel methodology that will map a high school biology curriculum into a series of interactive visualization modules that can be easily incorporated into a space biology curriculum. Fundamental concepts in general biology must be mastered in order to allow a better understanding and application for space biology. Interactive visualization is a powerful component that can capture the students' imagination, facilitate their assimilation of complex ideas, and help them develop integrated views of biology. These modules will augment the role of the teacher and will establish the value of student-centered interactivity, both in an individual setting as well as in a collaborative learning environment. Students will be able to interact with the content material, explore new challenges, and perform virtual laboratory simulations. The BioSIGHT effort is truly cross-disciplinary in nature and requires expertise from many areas including Biology, Computer Science Electrical Engineering, Education, and the Cognitive Sciences. The BioSIGHT team includes a scientific illustrator, educational software designer, computer programmers as well as IMSC graduate and undergraduate students.

Wong, Wee Ling↗

NASA NTRS1998

BioSIGHT: Interactive Visualization Modules for Science Education

Redefining science education to harness emerging integrated media technologies with innovative pedagogical goals represents a unique challenge. The Integrated Media Systems Center (IMSC) is the only engineering research center in the area of multimedia and creative technologies sponsored by the National Science Foundation. The research program at IMSC is focused on developing advanced technologies that address human-computer interfaces, database management, and high- speed network capabilities. The BioSIGHT project at IMSC is a demonstration technology project in the area of education that seeks to address how such emerging multimedia technologies can make an impact on science education. The scope of this project will help solidify NASA's commitment for the development of innovative educational resources that promotes science literacy for our students and the general population as well. These issues must be addressed as NASA marches towards the goal of enabling human space exploration that requires an understanding of life sciences in space. The IMSC BioSIGHT lab was established with the purpose of developing a novel methodology that will map a high school biology curriculum into a series of interactive visualization modules that can be easily incorporated into a space biology curriculum. Fundamental concepts in general biology must be mastered in order to allow a better understanding and application for space biology. Interactive visualization is a powerful component that can capture the students' imagination, facilitate their assimilation of complex ideas, and help them develop integrated views of biology. These modules will augment the role of the teacher and will establish the value of student-centered interactivity, both in an individual setting as well as in a collaborative learning environment. Students will be able to interact with the content material, explore new challenges, and perform virtual laboratory simulations. The BioSIGHT effort is truly cross-disciplinary in nature and requires expertise from many areas including Biology, Computer Science, Electrical Engineering, Education, and the Cognitive Sciences. The BioSIGHT team includes a scientific illustrator, educational software designer, computer programmers as well as IMSC graduate and undergraduate students. Our collaborators include TERC, a research and education organization with extensive k-12 math and science curricula development from Cambridge, MA.; SRI International of Menlo Park, CA.; teachers and students from local area high schools (Newbury Park High School, USC's Family of Five schools, Chadwick School, and Pasadena Polytechnic High School).

Wong, Wee Ling↗

NASA NTRS2016

Capturing Safety Requirements to Enable Effective Task Allocation Between Humans and Automaton in Increasingly Autonomous Systems

There is a current drive towards enabling the deployment of increasingly autonomous systems in the National Airspace System (NAS). However, shifting the traditional roles and responsibilities between humans and automation for safety critical tasks must be managed carefully, otherwise the current emergent safety properties of the NAS may be disrupted. In this paper, a verification activity to assess the emergent safety properties of a clearly defined, safety critical, operational scenario that possesses tasks that can be fluidly allocated between human and automated agents is conducted. Task allocation role sets were proposed for a human-automation team performing a contingency maneuver in a reduced crew context. A safety critical contingency procedure (engine out on takeoff) was modeled in the Soar cognitive architecture, then translated into the Hybrid Input Output formalism. Verification activities were then performed to determine whether or not the safety properties held over the increasingly autonomous system. The verification activities lead to the development of several key insights regarding the implicit assumptions on agent capability. It subsequently illustrated the usefulness of task annotations associated with specialized requirements (e.g., communication, timing etc.), and demonstrated the feasibility of this approach.

Neogi, Natasha A.↗

NASA NTRS1993

Reducing the cognitive workload - Trouble managing power systems

The complexity of space-based systems makes monitoring them and diagnosing their faults taxing for human beings. When a problem arises, immediate attention and quick resolution is mandatory. To aid humans in these endeavors we have developed an automated advisory system. Our advisory expert system, Trouble, incorporates the knowledge of the power system designers for Space Station Freedom. Trouble is designed to be a ground-based advisor for the mission controllers in the Control Center Complex at Johnson Space Center (JSC). It has been developed at NASA Lewis Research Center (LeRC) and tested in conjunction with prototype flight hardware contained in the Power Management and Distribution testbed and the Engineering Support Center, ESC, at LeRC. Our work will culminate with the adoption of these techniques by the mission controllers at JSC. This paper elucidates how we have captured power system failure knowledge, how we have built and tested our expert system, and what we believe its potential uses are.

Manner, David B.↗