The inter-relations between spatial and object factors during sustained visual attention to dynamic objects

Psicológica (2009), 30, 271-285. The inter-relations between spatial and object factors during sustained visual attention to dynamic objects Manuel J. Blanco 1 , Luz I. Leirós 1 , and David Soto 2 1 University of Santiago de Compostela, Spain 2 Imperial College London, England. The main goal of our research was to investigate attention to dynamic visual stimuli. Observers were instructed to attentively track a variable number of moving visual objects for a sustained period of several minutes. At random intervals, a target stimulus of brief duration was presented inside one of these objects and observers were required to identify it. The number of relevant objects to be tracked and the grouping strength among objects that had to be attended, operationalized as the size of relative area among those objects, were manipulated. The results indicated that only a single object could be attentively tracked, unless the objects formed a perceptual group. In this case, more than one dynamic object could be attended, but effectiveness of attention would be mediated by perception of the spatial relations among them. Previous research has shown that visual attention can operate in, at least, two ways: one space-based and another object-based (see Scholl, 2001; also Yantis and Serences, 2003, for comprehensive reviews). Visual attention operates on a spatial representation of the visual scene (e.g. Posner, 1980; Eriksen and Yeh, 1985) and also on objects or perceptual groups previously segmented on the basis of gestalt principles (e.g. Duncan, 1984; Yantis, 1992). Evidence for the spatial selection comes mainly from spatial cueing experiments (Posner, 1980; e.g. Blanco and Soto, 2002; Downing, 1988; Müller and Findlay, 1987) and from experiments using the flanker paradigm (e.g. Eriksen and Eriksen, 1974; Eriksen and St. James, 1986; Eriksen and Yeh, 1985). Evidence for object-based attention comes from a great variety of experimental procedures: selective looking (Neisser, 1 Acknowledgements: This research was supported by the grants SEJ2004-00541/PSIC and SEJ2007-61039/PSIC from the Ministerio de Educación y Ciencia of Spain to Manuel J. Blanco. Part of this work was presented at 5ª Reunión Científica sobre Atención (Murcia, Spain, 2005). Correspondence should be addressed to the first author. E-mail: [email protected] M.J. Blanco, et al. 272 1967), divided attention to object attributes (e.g. Duncan, 1984), cued detection and discrimination tasks (e.g. Brawn and Snowden, 2000; Soto and Blanco, 2004), visual search (Soto, Heinke, Humphreys, and Blanco, 2005; Yeshurun, Kimchi, Sha’shoua, and Carmel, in press), dissociations in neurological patients (e.g. Egly, Driver, and Rafal, 1994; see also Rafal, 1997, for a review), flanker or response-competition experiments (e.g. Kramer and Jacobson, 1991), inhibition of return (e.g. Tipper, Driver, and Weaver, 1991), negative priming (e.g. Tipper, Brehaut, and Driver, 1990), visual illusions (e.g. Hikosaka, Miyauchi, and Shimojo, 1993), attentional blink (e.g. Raymond, 2003), and binocular rivalry (Mitchell, Stoner, and Reynolds, 2004). Nowadays there is a general agreement about existence of objectbased effects on visual selection (see Scholl, 2001), but not regarding how many objects can be attended at a time. Results from well-known experiments on divided attention to object attributes of Duncan (1984) and others (Baylis and Driver, 1993; Behrmann, Zemel, and Mozer, 1998; Kim and Cave, 2001; Lavie and Driver, 1996; Watson and Kramer, 1999; though see Davis, Driver, Pavani, and Shepard, 2000, for contrary evidence) suggest that attention can select only one object at a time. However, a different conclusion comes from experiments with the multiple-object tracking (MOT) procedure developed by Pylyshyn and Storm (1988). In MOT experiments, observers must attentively track a number of identical objects moving independently and unpredictably among a larger set of identical moving objects. The results suggest that observers could track a maximum of about 3 or 4 moving objects (Pylyshyn and Storm, 1988; Pylyshyn, 1989; Scholl and Pylyshyn, 1999; Scholl, Pylyshyn, and Feldman, 2001; see also Yantis, 1992). In the present research, our aim was to investigate the characteristics of visual selection when dynamic stimuli have to be attended during prolonged and continuous periods of time. We were interested in finding out how many visual objects could be selected and maintained over time. Our observers were required to attend to a number of randomly moving visual objects for several minutes. We presented eight moving circles, differing in colour, with a white vertical line inside each one of them. The task was to identify an orientation change which took place at random temporal intervals in one of the lines. Attention to objects was accomplished by informing the observers about the probability of target presentation within each object. The target presentation (orientation change) was more likely to take place within objects that shared a common colour than within the others. Sustained visual attention to dynamic objects 273 Our experimental procedure resembles the MOT procedure developed by Pylyshyn and Storm (1988), but there are two significant differences between them. First, our task demanded attention to colour. We used objects that differed in colour, whereas in Pylyshyn’s procedure all the objects were identical. According to Pylyshyn (1989, 2001; see also Scholl, Pylyshyn, and Franconeri, 1999), changes in distinctive properties of tracked objects, like their colour or shape, are not encoded during MOT. However, that type of feature properties could be used to improve the tracking, because colour differences would reduce confusability among the stimuli, allowing that the selection of relevant objects was maintained longer than usual MOT. Besides, previous experiments in our laboratory have shown that attention can be object-based when colour is one of the defining features of the relevant objects in the display, both for stationary objects (Soto et al, 2005) and for moving objects (Soto and Blanco, 2004). Second, in the typical MOT experiments, trials are discrete and shorter than 30 seconds. It’s worth to mention that the seminal work of Pylyshyn and Storm (1988) tried to explain how mechanisms of early vision allow us to maintain a stable representation of a constantly changing world. It was believed that an observer can keep successfully keep tracked 3 or 4 objects over time by the temporal continuity of those objects and despite the changes in their locations (Scholl et al, 2001; Scholl et al., 1999). However, paradoxically these experiments make use of very short trials (from 7 to 15 s length), that represents a rather discontinuous and unstable world. In our experiments, the task lasted several minutes and it was performed continuously, without interruptions or breaks between trials. As we said above, we thought that it could not be ruled out the possibility that the number of objects that can be simultaneously tracked decreases when the task requires prolonged sustained attention and is performed uninterruptedly, relative to when the trials are discrete and only last for a few seconds. EXPERIME T 1 In Experiment 1, we investigated the number of moving objects that can be simultaneously attended. Observers were presented eight objects and they were told that a pre-specified subset of these ones would contain the target stimulus with highest probability. This subset was formed by one, two, or three objects sharing the same colour. If observers were able to simultaneously attend up to three objects, as previous evidence has suggested (Pylyshyn and Storm, 1988), then we should expect target identification to be improved for targets within any of the objects with the M.J. Blanco, et al. 274 highest probability of containing the target regardless of their number and relative to when the target appeared inside an irrelevant object. The use of a brief target duration (100 ms) ruled out the possibility that observers strategically switched attention very quickly between objects in order to achieve accurate performance, because that duration was shorter than the minimal time needed for identification and switching attention among locations (e.g. Duncan, Ward, and Shapiro, 1994). METHOD Participants. Thirty psychology students (22 female and 8 male) from the University of Santiago de Compostela (Spain) participated for course credits. Their ages ranged from 18 to 26 years, with a mean of 21 years. They were unaware of the purpose of the experiment. All participants had normal or corrected-to-normal colour vision and visual acuity. Experiments were approved by ethical committee of University of Santiago. Apparatus. The stimuli were presented on a 21’’ high-resolution colour monitor (IBM P275) (1024 x 768 pixels and 100 Hz) and the responses entered on a PST response box. A PC computer with a NVIDIA Pro TNT 64 MB graphics card controlled the experiment. The task was programmed using E-Prime V1.0 (Schneider, Eschman, and Zuccolotto, 2002). Presentation of the stimuli was synchronized with the refresh rate of the monitor. The displays were first drawn off-screen and then copied for visualization. Locations of the objects were computed successively across the experimental session. Electro-oculogram (EOG) recording was made with a Biopac MP 100 system. An adjustable chin rest helped to maintain head position. The luminance and colour of the stimuli were measured with a CS-100A Minolta photometer. Task. Figure 1 shows a schematic representation of the display. The entire task lasted seven minutes. The display was viewed binocularly from a distance of 90 cm. A big rectangle (13.30º x 19.54º of visual angle) and a fixation cross at its centre (0.53º x 0.53º) were initially displayed. The contour of the rectangle was white (CIE coordinates: x = 0.198, y = 0.175; luminance = 10 cd/m2, approximately) and its inner region black (luminance = 0 cd/m2). Eight circles, with a vertical line inside each one, were displayed within this rectangle at random locations with the constraint that the distance between the centres of each pair of moving objects was always higher than 1.12º of visual angle. The diameter of each circle was Sustained visual attention to dynamic objects 275 1.40º and its contour could be red (x = 0.615, y = 0.351), green (x = 0.291, y = 0.586), blue (x = 0.144, y =0.070), or yellow (x = 0.381, y = 0.519) (luminance = 5 cd/m2, approximately). A subset of the circles was printed in red and the remaining in different colours. During the entire task, the circles moved at a constant velocity of 3.9 degree/sec, with the trajectory of each circle being subjected to an independent and continuous Brownian-like motion inside the borders of the rectangle. As we said above, each moving circle contained a white vertical line (x=0.225, y=0.224; luminance=5.17 cd/m2) 0.56º in length. At several random times during the task, one of those lines was rotated 11º to the right or the left. The target line stayed with the new orientation for 100 ms and then returned to its original vertical orientation. Observers had to identify this orientation change (left or right), as fast and accurately as possible, by pressing a button of the response box (a different button for each orientation). The target probabilities being located at a red-circle or at a non-red circle were 0.80 and 0.20, respectively. The number of targets was 22, 28, and 34 for conditions of one, two, and three red-circles, respectively. Independently of the number of red circles, ratio of probabilities of target being located at a red circle or at a non-red circle was 4:1. The temporal interval between consecutive lines was random, but never lower than 2 or higher than 30 seconds. It is clear that retinal eccentricity can be a significant factor contributing to the results in this type of studies with moving stimuli. However, we decided not manipulate stimulus size because pilot studies had shown that size changes produced new expansion and contraction motions when the stimuli moved away or approached to fixation. We thought that these motion signals could affect negatively to target detection. In our experiments, to preclude a possible effect of eccentricity, the target was never presented inside the nearest area to fixation point. More specifically, the line could be showed in any place on the screen (inside one circle), but in a central area covering 3.2º x 2º from the fixation point. This area was determined from a previous pilot study, in which we analyzed the stimulus detection probability as a function of its eccentricity. In order to examine this issue, the rectangular frame for stimuli presentation was divided in three zones: One central -that coincided with the eliminated area afterwards- , one intermediate, and another more peripheral. Our results showed a clear effect of eccentricity in the central zone. Detection was better for targets in the central zone (69%) than in the peripheral zones (0.38 and 0.48), without differences along periphery. M.J. Blanco, et al. 276 Figure 1. Schematic depiction of the task in Experiment 1. At the beginning of the task eight circles in different colours, each one containing a vertical line, began to move independent and randomly on the screen. At several times during the task and at random intervals, one of the lines titled to right or left for a brief duration (100 ms), and observers were required to identify it. The black arrows (not actually presented) represent the motion of the objects, which only stopped at the end of the experiment. The actual display polarity was negative (dark background) and the real circle colours were red (solid line), yellow (dotted line), blue (dot-dashed line) and green (dashed line). Animations of the task are available for viewing over the internet at http://www.usc.es/hpcg/hpcg.html Procedure. Observers were assigned to one of three groups as a function of the number of red objects. They were informed about the probabilities of presentation of the target line inside each object and required to focus their gaze at the centre of the screen, without moving their eyes. In order to familiarize them with the task, observers performed Sustained visual attention to dynamic objects 277 training for a variable time (between 5 and 10 minutes) until his/her performance was stabilized. Then, they performed the task continuously for 7 minutes. RESULTS A D DISCUSSIO Trials on which blinks or eye movements greater than 2 degrees off the fixation point were detected (> 0.10 volts) concurrently with target presentation (within a time window of 200 ms), were excluded from data analysis. On average, less than 1% of the each observer’s total number of trials was rejected by this reason. The number of false alarms was also very low (< 1%), and, because of this, an analysis based on signal-detection theory (SDT) could not be carried out. Performance measure was the proportion of correct responses within a time window of 1.5 seconds after target presentation. The target location was categorized as high-probability (HP), for targets within the relevant objects (red circles), or low-probability (LP), for targets within distractor objects (non red circles). Proportion of correct responses was analyzed with an ANOVA 2(Target location: HP or LP) x 3(Group: 1, 2, or 3 HP objects). Figure 2 represents those results as a function of the group and target location. ANOVA showed a significant interaction between target location and group (F(1,27) = 5.58; p < .009): the proportion of correct responses was greater for targets within HP objects than for targets within LP objects but only when a single object had to be tracked (t(9) = 6.21; p < .03). Time on task was divided into four equivalent periods (105 seconds each one) and proportion of correct responses was calculated for each one. Analysis of variance did not show significant effects of period. EXPERIME T 2 It might occur that observers could simultaneously attend to more than one moving object, with enhanced sensitivity to targets appearing within them, if the objects were perceptually grouped. This possibility was examined in an experiment in which the relevant objects were grouped into a moving polygon (see Yantis, 1992, for similar experiments). In the second experiment, three HP objects moved in such a way that, at any time of the task, each one could be considered as a vertex of a triangle that varied its shape but with a constant area. Two processes could be involved in this experimental task (see Yantis, 1992): Formation of a M.J. Blanco, et al. 278 perceptual group according to Gestalt laws of perceptual organization, such as common fate and similarity (the three vertices of the virtual triangle were red), and maintenance of this representation during tracking. The critical issue was whether perceptual grouping could affect to sensitivity during tracking, and whether this effect was strictly object-based (that is, spatially invariant) or rather it was subserved by a spatial attention mechanism. In order to examine this issue, we varied the area of the virtual triangle formed by the relevant red objects. If object tracking was mediated by a spatially invariant attention mechanism that selects objects irrespective of their location, we should not expect any effect of the size of the triangle. However, if object tracking is mediated by location-based selection, that is, if what are tracked are not strictly the objects but their locations, we should expect the size of the rectangle modulating performance on the task. Figure 2. Data from Experiment 1. Proportion of correct detections as a function of the number of high-probability objects that had to be attentively tracked (Group variable) for each target-location condition (targets inside high-probability objects or targets inside low-probability objects). 50 60 70 80 90 100 123 Number of HP objects Proportion of correct responses High probability objects Low probability objects Sustained visual attention to dynamic objects 279 METHOD Participants. Eighteen psychology students (12 female and 6 male) from the University of Santiago de Compostela (Spain) participated for academic credits. Their ages ranged from 19 to 23 years, with a mean of 20 years. None of them had participated in the previous experiment. They were unaware of the purpose of the experiment and they had normal or corrected- to-normal colour vision and visual acuity. Task and procedure. The task was very similar to that of the Experiment 1 in the condition with three HP objects. Three red circles, three green circles, one blue circle, and one yellow circle, each one containing a vertical line, were displayed randomly inside a rectangular frame (13.30º x 19.54º). During all the time of the task, the trajectory of each non-red circle was subject to a random motion with the constraint that the minimum distance between the centres of each pair of objects had to be higher than 1.12º of visual angle. The motion of the red circles was similar to that of the non-red circles with constrains that, now, each of these circles was the vertex of a triangle of constant area but variable shape. Furthermore, LP circles were never displayed inside this virtual polygon. The area of that virtual triangle formed by the three red circles could be small (350 mm 2 ) or big (700 mm 2 ). Observers were not informed that the three red circles could be grouped into a virtual polygon. A comparative depiction of both conditions can be viewed at Figure 3. The procedure was identical to that of the previous experiment except that, in this case, observers were divided in two groups as a function of the area of the virtual polygon formed by the three red circles (small or large). RESULTS Figure 4 shows the correct detection percentage as a function of the target location and the area of the virtual polygon. As expected, an ANOVA 2 (target location) x 2 (area) indicated a trend for the effect of targetlocation to be significant (F(1,16) = 3.96; p < 0.06) and a significant interaction between the two factors (F(1,8) = 5.73; p < 0.03). Observers identified better the targets at HP objects than at LP objects when the area of the virtual triangle was small (t(8)= 6.03; p < .04), but not when that area was big (p > 0.05). Time on task was divided into four equivalent periods and proportion of correct responses was calculated for each one. Analysis of variance did not show effects of period.