Measuring and Modeling Shared Visual Attention

Mulligan, Jeffrey B.

Home/Archive/NASA NTRS

NASA NTRS · 20160008099

Measuring and Modeling Shared Visual Attention

Mulligan, Jeffrey B.

View original source ↗Export metadata ↓

Abstract

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval. We present two methods for comparing sequences of this sort, one based on computing the time-varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions.We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of activities, each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates.We have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Haslbeck, Gontar Schubert, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

Explore related subjects

attention eye movements cockpit

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Mulligan, Jeffrey B.. 2016-05-11. Measuring and Modeling Shared Visual Attention. https://ntrs.nasa.gov/citations/20160008099

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2025

Using Eye Tracking to Elucidate the Mechanisms Underlying Stimulation-Enhanced Visual Target Detection

Transcranial direct current stimulation (tDCS) is a noninvasive form of brain stimulation that involves passing a weak electrical current between electrodes on the scalp to modulate underlying neural tissue. TDCS has been shown to modulate cognition in a variety of domains, including memory, attention, and visual processing. Prior work from our laboratory has shown positive effects of tDCS on learning to detect target objects hidden in complex naturalistic visual scenes and learn rules for categorizing images, though the mechanism for these benefits remains unknown. One possibility is that tDCS optimizes visual search by modulating visual attention or via the reduction in search errors. One method of quantifying visual attention is to use eye tracking to record search patterns to determine if and how visual search is adjusted under verum stimulation conditions. Eye tracking data allows classification of errors into error types, including sampling errors (failing to look in the relevant region), recognition errors (looking at the critical portion of a scene, but failing to recognize it as such as evidenced by visual fixation), and decision-making errors (fixating on the relevant portion of a scene, but making the wrong determination). Our results indicate that the benefit tDCS confers on visual search for targets stems from the reduction in decision-making errors when targets are present (Cohen’s d = 0.86). Also reported is a replication of previous findings showing a tDCS-dependent improvement in learning this task, learning score (Cohen’s d = 0.88); d’ (Cohen’s d = 1.00). This provides support for moving tDCS into the application space by pairing it with analysts who are concerned with the type of search error that is corrected via stimulation.

attention↗

NASA NTRS2016

Attentional Modulation of Eye Torsion Responses

Eye movements generally have both reflexive and voluntary aspects, but torsional eye movements are usually thought of as a reflexive response to image rotation around the line of sight (torsional OKN) or to head roll (torsional VOR). In this study we asked whether torsional responses could be modulated by attention in a case where two stimuli rotated independently, and whether attention would influence the latency of responses. The display consisted of rear-projected radial "pinwheel" gratings, with an inner annulus segment extending from the center to 22 degrees eccentricity, and an outer annulus segment extending from 22 degrees out to 45 degrees eccentricity. The two segments rotated around the center in independent random walks, stepping randomly 4 degrees clockwise or counterclockwise at 60 Hz. Subjects were asked to attend to one or the other while keeping fixation steady at the center of the display. To encourage attention on one or the other segment of the display, subjects were asked to move a joystick in synchrony with the back and forth rotations of one part of the image while ignoring the other. Eye torsion was recorded with the scleral search coil technique, sampled at 500 Hz. All four subjects showed roughly 50% stronger torsion responses to the attended compared to unattended segments. Latency varied from 100 to 150 msec across subjects and was unchanged by attention. These findings suggest that attention can influence eye movement responses that are not typically under voluntary control.

attention↗

NASA NTRS2016

Measuring and Modeling Shared Visual Attention

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval. We present two methods for comparing sequences of this sort, one based on computing the time-varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions. We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of "activities," each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates. We have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Haslbeck, Gontar & Schubert, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

attention↗