Document [original]

The dynamics of attention in serial visual processing

Synopse zur Dissertation

zur Erlangung des akademischen Grades eines

Doktors der Philosophie (Dr. phil.)

der Fakultät für Kulturwissenschaften, Universität Paderborn

vorgelegt von

Dipl.-Psych. Frederic Hilkenmeier

Erstgutachterin: Prof. Dr. Ingrid Scharlau

Zweitgutachter: Prof. Dr. Werner Schneider

Disputation: 14.02.2012

Neben der Synopse besteht die Dissertation aus folgenden, in

Fachzeitschriften veröffentlichten Artikeln:

Hilkenmeier, F. & Scharlau, I. (2010). Rapid allocation of temporal

attention in the Attentional Blink Paradigm. European Journal of

Cognitive Psychology, 22, 1222 – 1234.

Hilkenmeier, F., Scharlau, I., Weiß, K., & Olivers, C. N. L. (2012).

The dynamics of prior entry in serial visual processing. Visual

Cognition, 20, 48 - 76.

Hilkenmeier, F., Olivers, C. N. L., & Scharlau, I. (2012). Prior

entry and temporal attention: Cueing affects order errors in RSVP.

Journal of Experimental Psychology: Human Perception and

Performance, 38, 180 - 190.

If the brain were so simple we could understand it, we would be so

simple we couldn't.

Lyall Watson

Table of Contents

1 Synopsis…………………………………………………………….….6

Footnotes……………..……………….…………………………..88

2 Experiments………………………………………………………......94

T2+1 Blank…………..…………………………………………....94

Explicit Order………..………………………………………….…98

Task-Set……………….………………………………………...100

SD-ISI Variation……………………......……………………….102

RSVP-Speed…………………………..………………………..104

Target-Colors…………………………..………………………..108

Single vs. Dual Stream………………..………………………..110

Cueing SOAs on a finer Scale………………..…………….....114

Monoptic / Dichoptic Blink………..…………………………....119

3 References……………...………………………...……...………....126

4 Summary in German………………………………….…..…….....150

The dynamics of attention in serial visual processing

Gathering, selecting and weighing information from our adjacencies

are fundamental parts of human perception and a necessary

precondition to successfully interact with our environment. And even

though the end-result of these processes, our everyday perception,

seems unambiguous and effortless to achieve (we just open our eyes

and there it is), a great deal of our cortical areas and cognitive

mechanisms are dedicated to produce this perception. One essential

aspect in this procedure is to prioritize relevant over irrelevant

information, a process commonly known as selective attention.

William James wrote: “my experience is what I agree to attend to”

(1890, p. 402). It becomes immediately clear from that quote that

attention is, at least to some degree, under voluntary control. This

means that we can choose whether we attend to a certain stimulus or

not. However, this is not the whole story. If we perceive a sudden

and unexpected movement in our peripheral visual field, this will

most likely interrupt our current task and will automatically turn our

head to see what is happening there. This means that the attentional

orienting process cannot only be elicited voluntarily in a goal-driven

manner, but also automatically by salient stimuli in our environment.

To what extent attention is elicited by bottom up stimulus saliency

and by the voluntarily top down task set, respectively, is still a matter

of debate (e.g. prominent: Folk, Remington, & Johnston, 1992;

Theeuwes, 1992, 1994, also see Belopolsky, Schreij, & Theeuwes,

2010; and Ansorge, Horstmann, & Scharlau, 2010, for more recent

studies). There is empirical evidence that suggests that the bottom-

up influence is initially strong, but vanishes over time (e.g. Donk &

van Zoest, 2008).

This orienting implies that attention has a limited spatial extent, much

like the fovea on the retina. In order to attend to another location,

attention must first disengage from the current location, then move,

and finally engage at the new location. The idea of such a “spotlight

of attention” dates at least back to Helmholtz (1867, p. 741, also see

James, 1890, p. 438) and is, as we will see below, still very popular

among psychologist (see e.g. Posner & Petersen, 1990).1

According to Ward (2008), orienting is one of the three main aspects

of attention, the other two being searching and filtering. Unlike in

orienting, in which we (automatically or voluntarily) react to the

appearance of a new stimulus, in search we are actively looking for a

certain stimulus. This search can be done very quickly and easily if

the stimulus we search for differs from the surrounding stimuli in a

single dimension like color, size, orientation or shape (for instance a

blue paperback among green paperbacks on a bookshelf). In fact,

this search can be completed in roughly the same time regardless of

the number of surrounding non-targets. If, however, the stimulus we

are looking for differs from the surrounding distractor-stimuli by a

conjunction of features (for instance a particular combination of color

and shape, like a green apple among red apples and green pears),

the number of surrounding non-targets has a large effect on search

Figure I:

spotlight of attention

times (e.g. Treisman & Gelade, 1980). It is hypothesized that the

simple, single feature search (the blue among green books; also

known as pop-out or parallel search) can be done without focusing

attention on each of the stimuli in turn (e.g. Woodman & Luck, 1999).

The feature-conjunction search (the green apple among red apples

and green pears; also known as serial search) on the other hand,

requires that the simple features (in this example shape and color)

are bound together into an integrated perceptual object (Treisman &

Gelade, 1980; but see Wolfe, Cave, & Franzel, 1989). In this model,

attention is the glue that pastes the different features together. This

feature integration must be performed for each stimulus in turn until

the target (the green apple) is found, explaining the higher search

times for more crowded scenes.

As with most psychological theories, this interpretation of search

times is not undisputed. Although it is widely accepted that

attentional processing of targets defined by conjunctions of features

is more demanding than processing of targets that are defined by a

single feature (e.g. Bundesen, 1990; Duncan & Humphreys, 1989), it

is still unclear whether this is due to integration or not. The observed

differences may also appear due to the difference between bottom-

up saliency and goal-driven attentional deployment in orienting as

discussed above (see e.g. Posner, 1980): If the target differs in only

one feature-dimension from the surrounding distractors (e.g. color),

this target automatically elicits an attentional orienting process,

because it is the most salient stimulus in the scene. If on the other

Figure II:

Schematic

representations and

idealized search

functions for single-

feature and feature-

conjuntion search

hand the target is defined as a combination of certain features, it is

not a priori the most bottom-up salient object. This means that

attention has to be voluntarily oriented to each object in turn until the

target is found. As Trick and Enns (1998, p. 371) point out, it is most

likely that feature binding and spatial orienting are both required in

any visual search task and should therefore be seen as

complementary and not as competing.

The third aspect of attention described by Ward (2008) is filtering.

Filtering means that we can quickly extract a great deal of

information from attended stimuli and at the same time suppress

information from unattended stimuli. The most impressive examples

of filtering are inattentional blindness and the cocktail-party effect.

Inattentional blindness can be characterized as the failure to detect

an unexpected, yet fully visible object, as attention is occupied with a

different task (for an overview, see e.g. Simons, 2007). In a highly

prominent study by Simons and Chabris (1999), participants had to

watch a short video-clip and count basketball passes by players

wearing white shirts while ignoring passes made by players wearing

black shirts. With this additional task, about half of the observers

failed to notice “when a person in a gorilla suit entered the display,

stopped and faced the camera, thumped its chest, and exited on the

far side of the display” (Simons, 2007). Although the term “blindness”

suggests that the missed stimulus (i.e. the gorilla) is not processed at

all, the related cocktail-party effect indicates otherwise. Early studies

by Cherry (1953), Treisman (1964), and Moray (1959) using a

Figure III:

Invisible gorilla test

dichotic-listening task showed that strong cues from the unattended

audio-stream (like one’s own name) are consciously perceived, and

that information from the unattended stream can be recalled when

task-demands on the attended stream are lowered, suggesting that

the information in the non-attended stream is at least processed up

to the semantic level.

Later on, the dichotic-listening task was converted to the visual

domain by Neisser and coworkers, using two distinct, but

superimposed videos instead of auditory streams (e.g. Neisser &

Becklen, 1975). Their results resemble the ones observed in the

auditory domain quite closely (also see Wolford & Morrison, 1980).

The dichotic-listening and inattentional-blindness experiments fueled

the debate about when exactly attentional filtering takes place and to

what degree unattended information are processed. This dispute

between early- (e.g. Broadbent, 1958) and late- (e.g. Duncan, 1980)

selection theories dominated the literature for quite a while. As

Pashler (2004, p.5) concludes, more recent evidence suggests that

perceptual selectivity is possible although it is often less than 100%

successful (e.g. Kahneman & Treisman, 1984; Yantis & Johnston,

1990). This “compromise” fits well with Treisman’s approach (1960),

suggesting that filtering attenuates rather than abolishes processing

of non-attended stimuli. This means that the filter in Treisman’s

model therefore allows unattended stimuli to pass trough, but only in

an attenuated form. This mechanism ensures that highly relevant but

unattended information can reach consciousness as well.

Figure V:

Frames from the

two films used by

Neisser & Becklen,

1976

Figure IV:

schematic

representation of a

dichotic-listening task

So what is attention? As yet, no exact definition has been agreed

upon, but an elegant summary of what attention characterizes was

given by James more than hundred years ago (1890, p. 403-404): "It

is the taking possession in the mind, in clear and vivid form, of one

out of several simultaneous possible objects or trains of thought.

Focalization, concentration of consciousness are of its essence. It

implies withdrawal from some things in order to deal effectively with

others". This quote illustrates that James especially emphasized the

filtering aspect of attention. We will return to the question of

attentional filtering later, when we discuss several theories of the

attentional-blink phenomenon.

In the last several decades the empirical focus of attention research

has shifted to the visual domain (see, e.g. Pashler, 2004, p.4). Since

then, researchers have addressed many fundamental questions

about the way in which visual information is selected. A number of

studies support Helmholtz’s claim of a spotlight of attention, most

prominently the cueing experiments done by Posner and colleagues

in the late 70s and early 80s (e.g. Posner, 1978, 1980, Posner,

Snyder, & Davidson, 1980; also see Prinzmetal, McCool, & Park,

2005). In these experiments, participants were asked to detect a light

appearing in one of several possible conditions around fixation. In the

majority of the trials one of the locations was precued, indicating at

what location the light most likely appeared. Posner’s results were

unambiguous: response times were fastest when the light actually

appeared at the precued location and slowest when the light was

Figure VI:

Spatial cueing

paradigm in the

style of Prinzmetal

and colleagues

presented at a different location than the precue indicated. Reaction

times of uncued trials lied almost exactly in between the valid- and

invalid-cued trials. Further evidence that attention can be directed to

a specific spatial location comes from LaBerge and coworkers

(LaBerge, 1983; LaBerge & Brown, 1986). LaBerge was interested in

the spatial extend of the spotlight. Therefore, he asked subjects to

perform two successive tasks on each trial. The second task was

always to detect a dot appearing at one of 5 possible positions. The

first task was either to determine whether a string of five letters

formed a word (the word task), or whether the middle letter in that

five-letter string was a vowel (the letter task). LaBerge hypothesized

and found that the word task required participants to attend to the

five-letter string as a whole, resulting in a wide attentional spotlight.

Therefore the reaction times in detecting the dot (the secondary task)

did not differ as a function of dot position. In the letter task on the

contrary, subjects focused their attention at the middle letter of the

string, since this was the only task relevant. As a result, reaction

times for the secondary task got slower and slower the larger the

distance between the dot and the middle position became. These

results furthermore indicate that the size of the focus of attention

depends of the task at hand (cf. the zoom-lens model described in

Footnote 1), and that the attentional spotlight does not have a distinct

range, but is rather distributed in a gradient fashion.

This latter claim is further supported by the illusory line motion

phenomenon (e.g. Hikosaka, Miyauchi, & Shimojo, 1993a, b, c). In

Figure VII:

Prototypical shape

of the attended

region found by

LaBerge

this phenomenon participants have to fixate on a dot at the middle of

the screen. Next, a stationary line is presented at one of the dot’s

ends. In the vast majority of participants this induces the impression

of motion: it seems as if the line grows from the attended dot towards

the unattended end. This perception can be explained as the result of

an attentional gradient: the dot is the attended object, the parts of the

line closest to the dot fall into the spotlight as well. With increasing

distance to the dot, the gradient and thus the attentional facilitation

get weaker, resulting in the impression of asynchrony in appearance

of the line (see Scharlau & Horstmann, 2006, p. 88; also see

Shimojo, Hikosaka, & Miyauchi, 1999; Bachmann, 1999).

One last aspect of the spotlight metaphor should be discussed here.

If attention shifts from one location (for instance the fixation point) to

another (e.g. the target-location), how does it move? Does it

illuminate intervening locations? Does it move at a fixed speed so

that locations farer away take longer to be reached? Whereas the

answer to the first question remains uncertain, the second one can

be negated (see Yantis, 2004, p. 236). Data from Remington and

Pierce (1984), as well as Kwak, Dagenbach, and Egeth (1991) and

Kröse and Julesz (1989) all indicate that spatial distance over which

attention has to travel does not influence the time until it gets there.

However, as Yantis (2004, p. 236) points out, these results do not

falsify the spotlight metaphor, as it is still possible that attention

moves at a variable velocity, for instance it could move faster the

further it has to go.

Figure VIII:

Illusory Line Motion

After looking at several spatial characteristics of attention, I will now

come back to the question what triggers an attentional shift. As

quoted at the beginning of this manuscript, James emphasized that

we can voluntarily direct our attention to specific stimuli (in his words:

active). He also noted that certain stimuli, “very intense, voluminous,

or sudden” (James, 1890, p. 416) immediately and involuntary draw

attention to themselves (i.e. passive). It wasn’t until a century later

before Jonides (1981) empirically tested whether one could resist this

automatic attentional capture. To this end, he employed two different

kinds of cues: peripheral, i.e. at the target-location, and central, i.e. a

cue at fixation that pointed at the target-location. Jonides found that

while it was easy to disregard the central cues, it was nearly

impossible to ignore the peripheral cue, even when it never appeared

at the correct target location (Remington, Johnston, & Yantis, 1992).

However, the attentional capture by irrelevant stimuli might not be as

absolute as originally thought. For instance Bacon and Egeth (1994)

demonstrated that a task-irrelevant singleton only captured attention

when the target was defined as a singleton as well (albeit in a

different dimension). When the target was defined in a more complex

way, the irrelevant singleton had no negative effect on task

performance. This led to the idea of an attentional control setting,

stating that only stimuli which are compatible with an a priori

determined feature set will tend to receive attention (e.g. Folk,

Remington, & Wright, 1994; Folk, Remington, Johnston, 1992). In

view of this theory, the incapability to ignore the non-predictive cue in

Figure IX:

peripheral and

central cueing

Jonides (1981) is a result of the fact that both the cue and the target

had a sudden onset, therefore the feature-dimension “onset” was

relevant, even though the cue itself was not. This hypothesis is not

undisputed. Whether and to what extend singletons that do not fit the

attentional set still capture attention is still the matter of a heated

debate (e.g. Schreij, Owens, & Theeuwes, 2008; Folk, Remington,

and Wu, 2009; Schreij, Theeuwes, & Olivers, 2010a, b).

Another difference between “active” and “passive” attentional

deployment is much less equivocal: Müller and Rabbitt (1989) found

that central and peripheral cues had distinctively different time

courses. Whereas “passive”, automatically elicited attention is

transient, i.e. is deployed rapidly and diminishes shortly afterwards,

“active”, voluntary attention, is sustained, i.e. it takes longer until it is

deployed, but it diminishes rather slowly. These results were later

corroborated in several different paradigms (e.g. Nakayama &

Mackeben, 1989; Cheal & Lyon, 1991; Carlson, Hogendoorn, &

Verstraten, 2006). These rapid and transient characteristics of

attention are not only relevant when attention is deployed in space,

but also when it is deployed in time, to which I will turn next.2

Much of the earlier work concentrated on understanding how humans

process information distributed across space. In the remaining part of

this manuscript, I will concentrate on the temporal aspects of

attention. Interest in this line of research has only risen in roughly the

last twenty five years. Since then researchers have addressed many

fundamental questions about the way in which visual information is

Figure X:

schematic functions

of transient and

sustained attentional

deployment

selected: How are items selected, when they compete for attention in

time rather than in space? Has attention to “dwell” on one object,

before it can turn to another? Does the processing of an earlier

stimulus impair the chances of a later object to reach

consciousness? Does it enhance them (e.g. Duncan, Ward, &

Shapiro, 1994; Bonnel & Prinzmetal, 1998; Broadbent & Broadbent,

1987; Bachmann, 1984)?

In the past two decades the “rapid serial visual presentation” design

has become a fruitful experimental paradigm for questions regarding

the temporal nature of attention (RSVP; Lawrence, 1971; Potter &

Levy, 1969). In RSVP visual stimuli, e.g. digits and letters, appear

sequentially at the same spatial location, each presented for a tenth

of a second or less and immediately replaced by the next one.

Observers are usually instructed to either report all items they have

seen (whole report), or to report pre-defined target items (for

example digits) and ignore the remaining distractor stimuli (for

instance letters; partial report; e.g. Nieuwenstein & Potter, 2006). By

manipulating the presentation speed, changing the amount of

information the observer has to report, and coupling the behavioral

measure with EEG recordings, the RSVP paradigm has provided

researchers with a versatile tool to study not only the time course of

attention and memory consolidation, but also how brain processes

contribute to conscious information processing (see Chun & Wolfe,

2001; Dux & Marois, 2009, p. 2).

Figure XI:

Rapid Serial Visual

Presentation

Especially the two-target version of the RSVP paradigm has become

increasingly popular to investigate an apparent limitation in visual

perception: Whereas observers have little to no difficulties in

reporting the first target stimulus (T1) in a sequence of distractors,

they often fail to identify the second target (T2) when it is presented

up to 500 ms after the first (Broadbent & Broadbent, 1987). It is as if

attention, analogous to the lid closure of an eye blink, briefly switches

off before new information can be processed. Hence, this

phenomenon was named the attentional blink (AB; Raymond,

Shapiro, & Arnell, 1992 p. 858). This second-target deficit is quite

robust: over the years, it was found in hundreds of studies, it is

reliable for a vast majority of subjects even after extensive training

(e.g. Taatgen, Juvina, Schipper, Borst, Martens, 2009) and it can be

found with a number of different types of stimuli like words (e.g.

Broadbent & Broadbent, 1987; Luck, Vogel, Shapiro, 1996),

alphanumerical stimuli (e.g. Hilkenmeier & Scharlau, 2010), symbols

(e.g. Chun & Potter, 1995), or pictures (e.g. Evans & Treisman,

2005). Moreover, the attentional blink can also be found within

different modalities, e.g. auditory (Duncan, Martens, & Ward, 1997)

or tactile (Hillstrom, Shapiro, Spence, 2002). All of this suggests that

the attentional blink reflects a fundamental mechanism of human

attention; and thus can give us insight into the basic components of

information selection and processing (see Martens & Wyble, 2010).

The basic empirical finding that the second of two task-relevant

stimuli in a stream of irrelevant distractors is often missed gave rise

to theories stating that when the first target-stimulus (T1) in the

stream is detected, a process is triggered to ensure T1’s encoding

and consolidation till it reaches consciousness. This processing not

only takes time, but momentarily consumes most of the attentional

resources available. As a result, when T2 is presented shortly

afterwards, there are little attentional resources left. Therefore, T2

cannot be processed properly and eventually its representation is

lost, i.e. the blink occurs (e.g. Ward, Duncan, & Shapiro, 1996;

Duncan, Ward, & Shapiro, 1994). When the temporal distance

between T1 and T2 increases, it is more likely that T1 has already

finished processing. Thus, on average there are more and more

attentional resources freed up; and successful encoding and

consolidation of T2 becomes more and more likely.

Interestingly, when T1 and T2 are presented right after each other

within 100 ms, T2 performance is virtually unimpaired, a finding

known as lag-1 sparing (Potter, Chun, Muckenhoupt, 1998, Visser,

Bischof, & Di Lollo, 1999).3 The lag-1 sparing result is especially

surprising for the afore mentioned theories of resource-depletion:

When T2 is presented in such close proximity to T1, all resources

should be devoted to T1. Hence, T2 performance at lag 1 should be

worst, not unimpaired.

Figure XII:

resource-depletion

model: blink

Figure 1: Typical time course of the attentional blink. Error bars

represent standard errors of the mean. The data is taken from the

100 ms / item condition of the “RSVP-Speed” experiment described

later in this manuscript.

To account for this intriguing result, many theories modified the basic

resource-limitation account by introducing two different stages of

processing and a sluggish attentional gate (most prominent: Chun &

Potter, 1995). In the first, capacity-free stage, multiple stimuli are

analyzed in parallel. If one of the stimuli shows task-relevant

features, this stimulus opens an attentional gate (also known as

“episode”, Chun & Potter, 1995; “batch”, Jolicœur, Tombu, Oriet,

Stevanovski, 2002; “window”, Visser et al, 1999; or “event”, Akyürek,

Riddell, Toffanin, & Hommel, 2007) and is transferred to the second

stage of processing. In this second stage, the stimulus is further

encoded and consolidated until it obtains a more durable

representation and becomes consciously reportable. In contrast to

stage 1, processing in the second stage is resource-heavy. This

means that stage 2 can only work in a serial manner, i.e. it can only

Figure XIII:

2-stage-model:

lag-1 sparing

process one “chunk” of information per time. However, the attentional

gate to the second stage does not close right after T1, but rather

“sluggishly” (e.g. Potter, 2006; Potter, Staub, and O’Connor, 2002).

The post-T1 item can slip through the gate as well and will most likely

be processed together with T1 in stage 2. If the post-T1 item is a

distractor, it will accidentally receive privileged processing. The

system will eventually realize that this stimulus is not task-relevant

and will discard it. In case of lag 1, the post-T1-item is T2, which

explains the high performance for both targets within 100 ms. This

joint processing of the two targets in stage 2 is commonly known as

episodic integration. With the auxiliary assumption of episodic

integration, resource-limitation theories can easily predict the time

course of T2 performance in RSVP as depicted in Figure 1: when

both targets are presented within 100 ms, they are processed in one

episode, hence T2 accuracy is unimpaired. When T2 is presented

about 200 ms after T1, the attentional gate to the second stage is

already shut down and therefore T2 has to wait until processing of T1

in stage 2 is finished: “When T2 appears before the second stage is

free, it will be detected by Stage 1 processing, but Stage 2

processing will be delayed. The longer the delay, the greater the

probability that T2 will have been lost…” (Chun & Potter, 1995,

p.122). This explains the steep decrease in T2 performance at lag 2.

With increasing temporal distance between T1 and T2, it becomes

more and more likely that T1 consolidation is complete and T2 can

enter the second stage of processing, explaining the gently inclining

Figure XIV:

2-stage-model:blink

performance for T2 between 300 ms and 500 ms. After about 500

ms, encoding of T1 should be completed and no longer interfere with

processing of T2 and T2 performance should again be unimpaired.

Theories that incorporate episodic integration and two stages of

processing are still widely popular in the attentional blink literature

(e.g. Bownman & Wyble, 2007; Jolicœur & Dell’Acqua, 1998; Dux &

Harris, 2007a; see Dux & Marois, 2009 for a recent review). Besides

the basic time course of the AB, episodic-integration theories can

account for a number of related findings as well: The claim that the

T2-deficit arises because T2 has to wait for T1 processing to be

completed and is therefore prone to decay and overwriting by

subsequent stimuli is well supported by data from Giesbrecht and Di

Lollo (1998). Their data show that the blink is strongly attenuated

when T2 is the last item in the stream and can therefore not be

overwritten by trailing distractors (also see Vogel & Luck, 2002; and

Sessa, Luria, Verleger, & Dell’Acqua, 2006).

In two of my own experiments, I could largely replicate this latter

result. When the distractor trailing T2 (the T2+1 item) was replaced

with a blank, the blink was nearly absent. However, when a later

distractor was replaced instead (the T2+2, or T2+3 item), the time

course of the attentional blink did not differ significantly. Interestingly,

replacing the T2+1 item had the same effect as T2 being the last item

in the stream (see Giesbrecht & Di Lollo, 1998). This means that the

additional 100 ms were sufficient to shield T2 from getting overwritten

by trailing distractors.

Figure XV:

summary of

experiment

„T2+1 blank“

Figure 2: results of the “T2+1 blank” experiment as a function of lag

and condition. Left: conditional T2 accuracy. Right (top): T1

accuracy, right (bottom): proportion of order reversals. Error bars

represent standard errors of the mean.

Episodic integration theories can also explain the controversial

finding that the blink gets stronger the more similar T1 and the T1+1

distractor are (Isaak, Shapiro, & Martin, 1999; Chun & Potter, 1995;

but see e.g. Maki, Bussard, Lopez & Digby, 2003). According to

episodic integration, T1 and the adjacent distractor are processed

together in the second stage. When both items are quite similar,

disentangling the target from the distractor takes longer. Therefore

processing of T2 is even more delayed and the probability of T2

getting overwritten increases. Likewise, the attentional blink is

strongly attenuated when the T1+1 distractor is replaced by a blank

(Chun & Potter, 1995, Raymond et al., 1992).

One empirical finding in particular made Potter and colleagues

rethink their original two-stage model (Potter, Staub and O’Connor,

2002): As can be seen in Figure 1, T1 performance at lag 1 is

impaired compared to later lags. It seems as if the high T2 accuracy

at lag 1 comes at the cost of decreased T1 performance. In fact, T2

often outperforms T1 if the two targets are presented in close

succession. This reliable finding challenges the basic two-stage

account. How can T2 performance be superior to T1 performance

when T1 enters the second stage first and therefore gets privileged

access to limited capacity processing resources in any case? Potter

and colleagues enhanced the original two-stage model to a “two-

stage competition model”, which postulates that T1 does not

automatically get access to the resources in stage 2, but that the

targets compete in stage 1. Depending on the respective

circumstances, the more salient target wins and will be processed.

This results in a tradeoff between the targets: the attentional

resources that one target wins (and which lead to that it is identified),

the other target automatically loses (Potter, 2006). Whichever target

wins this competition at stage 1 is transferred to the second stage

first. This does not mean that the two-stage completion model cannot

predict lag-1 sparing: When the temporal distance is short, and yet

both targets get identified at stage 1, they can both enter stage 2

together and are still processed as one episode. In case T2 wins the

competition at stage 1, it actually is T1 which is spared, not T2.

Figure XVI:

2-stage-competition

model: T1/T2 tradeoff

To summarize: at very short temporal intervals, the mechanisms of

the two-stage competition model differ from the mechanisms in the

basic two-stage model. In the former, at intervals below 100 ms T2

steals all the identification resources elicited by T1. Therefore T2 will

be identified easily and can enter stage 2. T1, however, has little

resources left and will not become identified at stage 1. Thus it

cannot join T2 in stage-2 processing and its representation is lost. At

stimulus onset asynchronies (SOAs) around 100 ms, T2 enters the

competition a little bit later, which gives T1 time to utilize some

resources for its own. T2 often gets the remaining ones, both targets

will be identified at stage 1 and therefore can both enter stage 2.

When the temporal interval between T1 and T2 increases to about

200 ms, T2 comes too late to compete with T1: T1 already used its

resources to become identified at stage 1 and is already in stage-2

processing. As in the original two-stage model, T2 will also be

identified at stage 1, but as processing at stage 2 is serial, T2 has to

wait at stage 1 and becomes prone to decay and overwriting (see

Potter et al., 2002, Potter, 2006, Dux & Marois, 2009). Interestingly,

the results of Potter et al. (2002) further suggest that competition is

not bound to a specific location, but overlaps to neighboring ones.

Moreover, by including direct competition between the two targets,

Potter and coworkers also incorporated the approach of Shapiro et

al.’s interference model (1994). Whereas Shapiro and colleagues

assume competition at a relatively late stage of processing in the

visual short term memory, Potter et al. (2002) state that the

competition appears before the items even enter the second stage of

processing.

Over the last years, several studies with three or more targets in a

single RSVP stream have seriously challenged the capacity-limitation

interpretation of the blink being a result of resource-heavy T1

processing: They showed that observers were able to identify several

proximal targets as long as they were presented immediately after

each other, a finding referred to as “spreading of the sparing” (e.g. Di

Lollo et al., 2005; Nieuwenstein & Potter, 2006; Olivers, Van Der

Stigchel & Hulleman 2007; Kawahara, Kumada & Di Lollo 2006). In

fact, Kawahara, Enns, and Di Lollo (2006) demonstrated that

performance for the third target in a stream of three consecutive

targets was significantly higher than performance of the first target,

which is clearly at odds with resource-depletion models. Of particular

interest in this context is the study of Nieuwenstein and Potter

(2006). In this study participants were able to report a string of up to

six consecutive colored items without showing an attentional blink.

When the task changed and participants were asked to report only

two items of a particular color, the same stimulus string produced the

standard blink pattern. This means that report- accuracy of the same

items at the same temporal distance is higher when all stimuli have

to be reported, in comparison to only two items having to be

reported. Put differently, observers are able to encode three targets

in a row in the same time as they otherwise fail to report the second

of two targets.

Another finding that does not sit easily with the limited-capacity idea

is the fact that T2 performance can be increased when the second

target is precued. Precueing T2 can be accomplished in several

ways. In some of our own work we used a temporal cue that provided

information about the temporal position of T2 (Hilkenmeier,

Tünnermann, & Scharlau, 2009; Hilkenmeier & Scharlau, 2010; “we”

is used when I am referring to work that was done with at least one

coauthor). In these studies the participants’ task was to report two

digits among distractor letters. In the cueing condition the identity of

the T1-digit was a valid cue for the lag T2 would appear in. If T1 was

a “1”, T2 would appear in lag 1, i.e. immediately after T1. If T1 was a

“3”, T2 would appear in lag 3, i.e. T1 and T2 were delineated by two

intermediate distractors. This temporal cue embedded in the identity

of T1 significantly increased T2 accuracy, even for extremely short

temporal distances between T1 and T2 of about 50 – 100 ms. This

means that within this quite short time span the identity of a target is

processed and the relevant information can rapidly be extracted and

used to adjust the task and substantially increase performance. This

finding goes against the notion that an extensive, time-consuming

identification process causes the blink.

Another way to increase the identification accuracy of the second

target is to insert an additional item with a task-relevant feature in the

RSVP stream just prior to T2. This was for instance done by

Nieuwenstein and colleagues (Nieuwenstein, 2006; Nieuwenstein,

Chun, Hooge & Van der Lubbe, 2005). In their studies the task was

Figure XVII:

temporal cue used by

Hilkenmeier and

colleagues

Figure XVIII:

cue used by

Nieuwenstein and

colleagues

to report two colored digits among black distractor-letters. Cueing T2

was achieved by coloring the distractor-letter preceding T2.

Importantly, even distractors that were presented in a different color

than the to-be-cued targets were highly effective cues, as long as

their color matched one of the possible target colors. This indicates

that cueing can occur in the absence of shared features between

cues and targets, as long as they both match the same attentional

set (also see Scharlau & Neumann, 2003; Folk, Remington, &

Johnson, 1992; Folk, Remington, & Wright, 1994). The “rapid

reversal” of the attentional blink found by Olivers et al. (2007; also

see Kawahara, Kumada et al., 2006; Olivers & Meeter, 2008) goes in

the same direction: Participants were confronted with a stream

containing T2 at lag 2 and additionally a third target (T3) at lag 3

(basically, instead of a cue preceding T2, they used T2 to cue the

immediately following T3). An attentional blink was found for T2 but

not for T3 which was almost completely spared; even though it was

presented in a temporal position relative to the first target which is

normally strongly impaired (see Figure 1). These results evidence

that the attentional blink is not, as Raymond et al. (1992, p. 858)

suggested, ballistic: It can be postponed as long as task-relevant

information is presented; it can be attenuated when the temporal

position of T2 is cued; and it can rapidly recover when consecutive

targets are shown during the blink-period.

All of this suggests that attention uses a more flexible mechanism for

mediating attentional deployment than simply deplete all of its

resources at once (Wyble et al., 2009).

To account for these more recent empirical findings, the theoretical

landscape shifted. Instead of emphasizing on resource-depletion or a

bottleneck as the source of the attentional blink, Di Lollo and

colleagues (2005) suggested to focus on the afore mentioned filtering

aspects of attention. In their “temporary loss of control” model (TLC)

they do not see the blink as a result of resource-depletion, but as the

temporary loss of an endogenously established task set. In the

typical attentional blink paradigm, this task set would be something

like “reject distractors and accept targets” (see, e.g. Kawahara et al.,

2006, p. 406). Importantly, this filter is volatile rather than static.

Following an earlier idea of William James (1890, p. 420), the

endogenous filter needs a periodic maintenance signal to be

sustained (see Di Lollo et al, 2005, p. 192). In the period leading to

the first target, this signal can easily be maintained, meaning that the

pre-T1 distractors can easily be inhibited. When T1 appears, the

endogenous maintenance signal is discontinued, since the system is

now busy with consolidating this first target (Olivers & Meeter, 2008,

p.4; Di Lollo et al, 2005, p. 198). Thus, the filter is no longer under

endogenous control, but becomes vulnerable to alteration by the

T1+1 stimulus. If this next item also belongs to the target-category,

the input filter remains unaltered and the T1+1 stimulus is processed

efficiently, i.e. sparing occurs. On the other hand, if the T1+1 stimulus

Figure XIX:

TLC model: blink

is a distractor, it will exogenously disrupt the input filter, prolonging

any processing of trailing target stimuli, which eventually leads to the

blink. As soon as consolidation of T1 finishes, the system gradually

regains control over the input filter again, reinstates the correct task

set and allows target processing to return to normal.

As Olivers and Meeter (2008, p. 4) point out, it is questionable

“whether TLC indeed manages to avoid the limited-capacity resource

depletion argument. Notably, it assumes that T1 occupies a central

executive for some time, during the course of which the system

is not ready for T2. It appears then that limited capacity

resources have entered through the back door”. To be clear, unlike

limited-capacity models, TLC states that the system can handle more

than one or two items before its resources deplete. The capacity of

the visual short term memory is the only limit here. In TLC the

cognitive system is limited in the way that it can only execute one

process at any given time, i.e. either maintaining the input filter or

consolidating T1 (see e.g. Di Lollo et al, 2005, p. 193).

Taken together, the TLC deserves credit for being one of the first

attentional-blink theories that “break the bottleneck” and shift the

focus to attentional filter settings as source of the attentional blink. In

subsequent years, more and more models followed this theoretical

shift and emphasized the role of attentional control and attentional

gating, for instance the “simultaneous type, serial token” model of

Bowman and Wyble (STST, 2007), the “boost and bounce” theory of

Olivers and Meeter (B&B, 2008), or the “episodic simultaneous type,

Figure XX:

TLC model:

spreading of the

sparing

serial token” model (eSTST, Wyble, Bowman, & Nieuwenstein,

2009). All of these theories share the idea that a spatiotemporally

constrained window of attentional enhancement is deployed in

response to detection of a potentially relevant stimulus (Wyble et

al., 2009, p. 3). The attentional facilitation is hypothesized in a rapid

and transient way: the enhancement should peak around 100 ms (or,

more roughly between 50 ms and 150 ms) and quickly decrease

afterwards (see Reeves & Sperling, 1986; Nakayama & Mackeben,

1989; also see the attentional cascade model, Shih, 2008, 2009). In

all of these theories the attentional facilitation (the “blaster” in

(e)STST, or the “boost” in boost & bounce) hits the T1+1 item as

well, allowing for lag 1 sparing. However, the theories differ in the

way they manage the transition from sparing to the attentional blink:

The models of Wyble and colleagues (STST; eSTST) as well as the

attentional cascade model (Shih, 2008) hold on to the idea of time-

consuming consolidation of T1 (and the T1+1 item) suppressing

further attention until T1 has been encoded. This means that

eventually T1 causes the blink. In the Boost and Bounce theory on

the other hand, time-consuming target processing plays no role in the

rise of the blink. Instead, this model assumes that it is the first

distractor after target information that shuts down further attentional

deployment (Olivers & Meeter, 2008).

Whether the blink is caused by T1 itself, or by the first post-target

distractor is still a matter of debate. The T2-cueing results of

Nieuwenstein and coworkers (Nieuwenstein, 2006; Nieuwenstein et

Figure XXI:

eSTST: spreading

of the sparing

al., 2005) and the rapid recovery results of Olivers et al., 2007

indicate that the blink deficit is caused by the first post-target

distractor, not by T1 itself: In these studies T2 performance increased

in lags at which T2 is usually blinked, but this increment did not come

at the cost of reduced T1 performance. If time consuming and

resource-heavy T1 consolidation causes the blink, one should expect

some effect on T1 performance since T2 identification interferes with

T1 consolidation. On the other hand, studies that show decreased T1

performance when T2 is presented in lag 2, as for instance Potter et

al. (2002; also see Figure 1) indicate that an early T2 indeed

interferes with T1 processing. Furthermore there is evidence about

the post-T1 distractor not being needed to induce the blink, as long

as T2 is sufficiently masked (Nieuwenstein et al., 2009). This is

clearly is at odds with the Boost and Bounce theory that emphasizes

the role of the post-target distractor.

The theoretical shift from limited capacity and resource depletion to

attentional control is highly controversial. A number of recent studies

report that the apparent spreading of the sparing with three targets in

a row really is due to a tradeoff between T1 and T3 (Dux, Asplund, &

Marois, 2008; 2009; but see Olivers, Spalek, Kawahara & Di Lollo,

2009), or that the apparent sparing of T3 really is caused by a

methodological artifact (Dell’Acqua, Joliceur, Luria, & Pluchino,

2009). Dell’Acqua et al. report that when T3 accuracy is analyzed

contingent on correct report of T1 and T2, T3 performance actually is

impaired. They conclude that the blink is still best explained by a

Figure XXII:

eSTST: blink

capacity-limited process in which T1 opens an attentional episode.

T2 can slip into this episode as well when it is presented within 120

ms of T1 and both targets are processed together. Even if T3 is the

next item in the row, there are little chances for it to enter the episode

as well and it will eventually get blinked (2009; p. 28f).

After conducting a thorough review of the attentional blink literature,

Dux and Marois (2009, p. 51) argue in a similar vein: they conclude

that it is possible for a common capacity-limited attentional resource

to underlie the deficit. According to this view “the process that is

responsible for the trade-off between T1 and T3 performance in

the serial target experiments of Dux et al. (2008; 2009) is the same

which underlies the AB impairment in the distractor-less design of

Nieuwenstein et al. (2009), or the attenuating effect of

distraction in the experiments of Olivers and Nieuwenhuis

(2005; 2006); namely, the deployment of selective attention. The

more attention that is deployed for T1, either because it is

more salient, more task relevant or requires more encoding into

working memory, then the less that is available to process

subsequent targets. Similarly drawing attention away from T1,

either by cuing a distractor prior to T2 (e.g., Nieuwenstein,

2006) or by including distracting tasks (see above), may

alleviate the T2 deficit. The neuroimaging evidence that AB

manipulations recruit the frontal-parietal attentional networks of the

brain (Hommel et al., 2006; Marois & Ivanoff, 2005) adds further

weight to the view that, first and foremost, the attentional blink

Figure XXIV:

B&B: sparing

Figure XXIII:

B&B: blink

represents a deficit of selective attention.” Similarly, Akyürek,

Toffanin, and Hommel (2008, p.575) point out that “lag 1 sparing is

presumably associated with two logically related but nevertheless

different processes: integration into the same episodic file and

competition within this file”.

As can be seen by this overview of the literature, there still is a

heated controversy on what underlying mechanisms cause the time-

course of the attentional blink. However, it seems that one critical –

and the most controversial – point is the explanation of lag-1 sparing.

Is it due to a sluggishly closing gate and episodic integration? Or is it

due to delayed attentional enhancement?

In the majority of our own research, we have created conditions for

which these different theoretical approaches on lag-1 sparing make

different predictions. To this end we did not concentrate on target-

identification accuracy at lag 1, but instead investigated the

accompanying phenomenon that the reported targets are often

perceived in the wrong order (Olivers, Hilkenmeier & Scharlau, 2010;

Hilkenmeier, Olivers, & Scharlau, 2011; Hilkenmeier, Scharlau, Weiß,

& Olivers, 2011). As can be seen in Figure 3, lag-1 sparing is indeed

accompanied by a substantial proportion of temporal order reversals

(Chun & Potter, 1995; Bowman & Wyble, 2007; Hommel & Akyürek,

2005; Akyürek & Hommel, 2005). These order reversals were

originally regarded as strong evidence in favor of episodic integration

in a second stage of processing: when T1 and T2 are presented in

close succession, they are processed as a single event – that is, they

Figure XXV:

B&B:

rapid recovery

are merged into a single representation or receive a single memory

trace, with a single time stamp (e.g. Bowman & Wyble, 2007;

Akyürek, Riddell, Toffanin, & Hommel, 2007; Hommel & Akyürek,

2005; Akyürek, Toffanin, & Hommel, 2008). Therefore the actual

temporal order between the targets is lost, leading to an increase in

order errors (e.g. Bowman & Wyble, 2007; Chun & Potter, 1995).

Figure 3: typical time course of order reversals found in RSVP. Error

bars represent standard errors of the mean. The data are taken from

the 100 ms / item condition of the “RSVP-Speed” experiment

described later in this manuscript.

Temporal order errors are consistent with theories of attentional

enhancement as well: one of the more intriguing effects of attentional

enhancement is that it can alter the perceived order of the stimuli

presented. That is, an attended stimulus can overtake a like, but

unattended stimulus in the race to awareness, an effect commonly

known as “prior entry” (Titchener, 1908). If lag-1 sparing in the

attentional blink is due to a somewhat delayed attentional

enhancement in a way that T2 benefits from attentional facilitation

originally triggered by T1, this should not only lead to increased

identification accuracy for T2, but, based on prior entry, to a

substantial amount of target-order reversals.

In sum, the basic finding of order reversals with both targets being

presented within about 100 ms is consistent both with an episodic

integration account as well as with an attentional enhancement

account. We have come up with a cueing design that can

differentiate between these two theoretical approaches.

Before I get to the prior-entry account and the related cueing

experiments in more detail, I will, in a classic psychophysical sense,

test the phenomenon of order reversals under a wide variety of

conditions, i.e., study “the effect on a subject's experience or

behavior of systematically varying the properties of a stimulus along

one or more physical dimensions” (Bruce, Green, & Georgeson,

1996, p.462). This means, I will discuss which factors do and do not

influence the proportion of order reversals and offer a theoretical

explanation for these results. Afterwards, I will explicate the prior

entry account in more detail, speculate what might open and close an

integration episode; and present empirical findings that strongly favor

the attentional over the integration explanation.

To start, I will discuss one objection that is essential for most of the

claims made in the rest of this manuscript: In the original AB

paradigm and in the vast majority of all published AB studies since,

the participants’ only task is to identify the two targets. To reliably

compute the proportion of order errors, it is necessary to modify this

task and ask participants to report the two targets in the perceived

order. One might object that this is a more demanding task or even

that it is a dual task: first identify the targets and then make a

judgment about their order of appearance. Therefore, it might be that

the task utilized in the experiments reported here is not strictly

equivalent to the standard attentional blink task and thus our results

will not be transferable to the AB paradigm as such (see e.g. Visser,

Bischof, & Di Lollo, 1999 for influences on the AB task). For two

reasons I am confident about this objection being invalid and that the

AB with explicit order task is equivalent to the AB without explicit

order task. For one, in Chun and Potter (1995) participants were

encouraged but not required to report the targets in the perceived

order (p. 118f). Nevertheless the time course of order reversals

reported by these authors looks strikingly similar to the time course

when participants are asked to report the targets in the correct order

(Compare Figure 8 in Chun & Potter, 1995 to Figure 1 in Hilkenmeier

& Scharlau, 2010; Figure 3 in Akyürek & Hommel, 2005a; Figure 2 in

Akyürek, Toffanin, & Hommel, 2008). This suggests that participants

report the targets in the order they perceived them in any case,

regardless of the fact whether they were explicitly asked to do so or

not. Secondly if the explicit order task differs from the basic

identification task, it must be more demanding, especially at short

inter-target intervals. This would suggest that identification

accuracies of the targets suffer in comparison to when the

participants’ only task is to identify the two targets. A visual

comparison of the time course of T1 performance and the conditional

T2 performance suggests that this is not the case (compare e.g.

Figure 2 in Chun & Potter, 1995 to Figure 1 in Hilkenmeier &

Scharlau, 2010; Figure 1 in Akyürek & Hommel, 2005; Figure 1 in

Akyürek, Toffanin, & Hommel, 2008).

Nevertheless I explicitly designed one control-experiment to test the

before raised objection. In two successive blocks participants were

asked to identify the two targets. In one block the instruction

emphasized that the targets had to be reported in the correct order.

In the other block the instruction stated that they could give their

report in any order they wanted. The order of the blocks was

counterbalanced between subjects.

Figure 4: Results of the “explicit order” control experiment as a

function of lag and condition. Left: conditional T2 accuracy. Right

(top): T1 accuracy. Right (bottom): proportion of order reversals.

Error bars represent standard errors of the mean.

Figure XXVI:

summary of

experiment

“explicit order”

As can be seen in Figure 4, there is no significant difference for the

conditional T2 accuracy or the proportion of order reversals between

the two conditions. Dealing with null-effects is not easy, especially

when, as in this case, the null-result is the desired one. Whether a

result reaches significance or not depends on a number of factors

including the population effect size and the number of subjects tested

(“Never use the unfortunate expression ‘accept the null hypothesis’”;

Wilkinson & the Task Force on Statistical Inference, 1999, p. 599).

When the number of subjects is small, it is less likely to reject the

null-hypothesis, even if there is a true difference in the underlying

population. Luckily, power analyses can provide us with an estimate

of the minimum sample-size required to detect an effect of an a priori

determined size with certain likelihood. For this experiment the a

priori determined likelihood to find an effect of at least medium size

exceeded 90% (Faul, Erdfelder, Buchner, & Lang; 2009; for effect

sizes see Cohen, 1988). Therefore, it is relatively safe to assume that

the additional order judgment does not influence the attentional blink

too much. A closer inspection of Figure 4 reveals that the

identification accuracies are higher in the “dual task” condition: There

is a significant main effect of condition for T1 accuracy, meaning that

T1 identification is better when subjects were asked to report both

targets in the correct order. It is hard to make perfect sense out of

this result, but the import message here is that the AB task with

additional order instruction is not more demanding than the AB task

without this instruction. In the following, I will work under this

premise.

After concluding that the additional order-judgment task has no major

effect on the primary target-detection task and therewith the time-

course of the attentional blink, I will look at that relationship the other

way round: Before participants can judge the order of events, they

first have to go through a demanding identification process. It is thus

possible that they have little resources left for the actual order

judgment. In other words, the high demands of the target-

identification task might have inflated the number of order reversals.

Participants might simply do not have the resources to deal with both

tasks at the same time. This would explain the relatively high amount

of order reversals after 100 ms. In other experimental paradigms as

for instance the temporal order judgment (TOJ), a delay of 100 ms

between the target stimuli is usually sufficient for a correct order

judgment (e.g. Scharlau, 2002; Scharlau, Ansorge, & Horstmann,

2006; Stelmach & Herdman, 1991; Weiß & Scharlau, 2011).

However, as pointed out in Hilkenmeier, Scharlau, Weiß, and Olivers

(2011) one of the main differences between a typical TOJ task and a

typical AB task are the higher task demands for the latter one. We

conducted several experiments to investigate the influence of task

demands on the proportion of order reversals, all with the same,

unexpected result: When the task-set is smaller and the identification

demands are therefore lessened, participants make more order

reversals, not less.

When subjects were asked to either report two target digits in the

correct order, out of all possible digits (2 out of 9), or to report

whether a “5” or a “7” appeared first (1 out of 2), they made less

order reversals in the former condition than in the latter.

Figure 5: order reversals as a function of lag and task-set size. Error

bars represent standard errors of the mean.

One could argue that this difference is due to guessing. In the

standard AB task (i.e., 2 out of 9) guessing has a rather limited

influence. Given a participant has only seen one target and has to

guess the other one, chances of a correct guess are one out of eight.

When the guess is wrong, this trial will not be taken into account for

the computation of order reversals. Even when the guess is correct,

there is still a “fifty-fifty chance” for the reported order being correct or

not. When the participant has not seen any target, the chances of

producing an order reversal are even lower. In the “1 out of 2”

condition the influence of guesses are much higher, as can be seen

in the following example:

Figure XXVII:

summary of

experiment

“task-set”

Imagine that correct order and incorrect order are guessed equally

often (which isn't even necessarily the case considering the small

number of repetitions we use). Case 1: a subject is only certain about

her order judgment in 20% off all cases. In the remaining trials she

has either seen only one target or no target at all. Of these 20%, the

order is perceived incorrectly in 35% (i.e. 7 percentage-points). In the

remaining 80% of trials, she guesses the order (50% correct, 50 %

incorrect). Since she can only choose between “5” and “7”, her guess

will in any case be taken into account for the proportion of order

reversals (either as a correct answer or as an order reversal). That

leaves her with 47 percentage-points order reversals.

Figure 6: Distribution of perceived order and guesses for two

hypothetical cases. Even though the proportion of perceived correct

order and perceived incorrect order is identical in both cases, the

measured proportion of order reversals is higher for the case with

more guesses.

Case 2: a subject is certain about his order judgment in 80% of all

cases. Of that 80%, the order is again incorrect in 35% (i.e. 28

percentage-points). In the remaining 20% of trials, he guesses the

order (50% correct, 50% incorrect). All of his guesses will be taken

into account as well. That leaves him with 38 percentage points order

reversals. Despite the proportion of actual perceived order-reversals

being equal (35%), the measured order reversals differ considerably

(47% to 38%).

To conclude: the more a participant guesses in the “1 out of 2”

condition, the higher the proportion of reported order reversals. Thus,

the results presented here might be due to a methodological artifact.

However, the results do not change even when we control for

guessing by using a ternary TOJ task and giving participants the

opportunity to refrain from their judgment (see Ulrich, 1987).

Participants rarely use the third “uncertain” judgment category, but

indicate that they perceive the targets in a distinct order.

Interestingly, they still make more order reversals in the easier task

with less target-alternatives (Hilkenmeier, Scharlau, Weiß, & Olivers,

2011, Experiment 1f). This leaves us with a very puzzling result,

though the important message here is that order reversals do not

increase when the task demands of the primary target-identification

task increase.

A further control experiment showed that order reversals do not vary

in respect of stimulus-duration / inter-stimulus-interval variations,

given the overall stimulus onset asynchrony remains constant.4

Thus, presenting the items, especially the targets, without any ISI

does not increase order reversals, even though one might assume

that the temporal separation is less clear (and therefore integration is

more likely to occur) when the targets are presented right next to

each other compared to when they are separated by an inter-

stimulus interval (Chua, personal communication).

Moreover, this result is well in line with Coltheart’s finding regarding

visible persistence (1980), showing that briefly presented stimuli are

perceived up to 100 ms when they are not overwritten by subsequent

masks (also see Sperling, 1960; Keysers & Perrett, 2002). According

to Bloch’s law, these “persistent” stimuli should have been perceived

rather gray than pure black. Anyway, it seems as if this had not

influenced identification accuracies either.

Figure 7: results of the “SD_ISI Variation” control experiment. Left:

Proportion of order reversals. Right: T1 and conditional T2 accuracy.

Error bars represent standard errors of the mean. As can be seen,

variations in stimulus duration and inter stimulus interval do not have

any major influence on target detection or perceived order.

Figure XXVIII:

summary of

experiment

“SD_ISI variation”

All in all, the experiments reported so far indicate that order reversals

are not the result of too high task demands and that the AB as such

is not influenced by the additional order-judgment task.

Next, I will tackle the question whether order reversals between T1

and T2 are due to T2 being the next item in the stream or due to the

temporal distance between the two targets. As Bowman & Wyble

(2007; also see Potter, Staub & O’Connor, 2002; Nieuwenhuis,

Gilzenrat, Holmes & Cohen, 2005) evidenced, the so-called “lag 1

sparing” is not constrained to the lag 1 position, but to about 100 ms

Target Onset Asynchrony (TOA). They tested this by speeding up the

RSVP stream to about 20 items / second. This means that when T2

was presented at lag 2, it was only presented 100 ms after T1.

Unfortunately, these authors did not report order reversals, hence it

is unclear whether this temporal misperception will spread to the

T1+2 position as well, provided the temporal distance between the

two targets is still around 100 ms. To test this, I chose a design close

to Bowman and Wyble (2007) and manipulated the speed of the

RSVP stream as well. In addition to the “50 ms” condition and the

“100 ms” condition utilized by Bowman and Wyble, I also employed

an intermediate “75 ms” condition as well as a “150 ms” condition. As

can be seen in Figure 8, I was able to replicate Bowman and Wyble’s

finding about lag-1 sparing spreading to lag 2 as long as it came

within about 100 ms of T1 onset. There are some discrepancies

between the result found in our lab and the ones reported by

Bowman and Wyble, but the important finding in the present context

is that temporal order reversals between T1 and T2 can indeed

spread to later lags, and are not bound to situations in which T1 and

T2 are presented right after each other. As with lag-1 sparing, the

important variable seems to be the temporal distance between T1

and T2, and not the number of intermediate distractors between

them.

Figure 8: Results of the “RSVP-Speed” experiment as a function of

RSVP speed and target onset asynchrony. Top: conditional T2

accuracy. Bottom: proportion of order reversals. Error bars represent

standard errors of the mean.

What is surprising in this context is the finding that at the same TOA

there are more order errors for faster presentation speeds. This

Figure XXIX:

summary of

experiment

“RSVP-Speed”

means that there are more order reversals even though the targets

are delineated by more intermediate distractors. For instance, at a

TOA of 200 ms, T1 and T2 are delineated by one distractor in the

standard 10 Hz stream condition, but delineated by three distractors

in the faster 20 Hz condition. Nevertheless, there are significantly

more order reversals in the latter compared to the former condition.

This brings us to our next aspect: the importance of backward

masking. The presentation speeds employed in this experiment do

not only differ in the number of distractors between the targets at a

given TOA, they also systematically vary in the strength of backward

masking. The different RSVP speeds were realized by varying the

inter stimulus interval. This means that the stimulus duration was set

to 50 ms in all conditions. In the “50 ms” condition the ISI was

therefore 0 ms, whereas it was 50 ms in the “100 ms” condition. As a

result, the distractor trailing the second target came much quicker in

the “50 ms” condition than it came in the “100 ms” condition,

impairing the visibility of T2. This impaired visibility indirectly causes

the higher proportion of order reversals: Note that for the

computation of order reversals we only take trials into account in

which both targets are identified. So in the trials that we consider,

backward masking did not hinder T2 on being identified. In the

relatively hard “50 ms” condition this might indicate that the identified

T2s are strongly activated (elsewise, they would not get identified in

the first place). These strongly activated T2s then (because they are

so strongly activated) often win the race to awareness against T1, i.e.

they are perceived as earlier. When backward masking is weaker, as

for instance in the “100 ms” condition, also less activated T2s get

identified. These T2s race against their respective T1s as well, but

they more often lose this race to awareness, i.e. T1 is perceived as

first and T2 is perceived as second, leading to a reduction in the

relative proportion of order reversals.

The order-error results of the already described “T2+1 blank”

experiment (see Figure 2) are consistent with this interpretation as

well: When the distractor trailing T2 is omitted, backward masking is

much weaker. This leads to better T2 performance (presumably

because less activated T2s get identified as well, since they have

more time to save themselves from becoming overwritten) and also

to a reduction in the proportion of order reversals (presumably

because these weakly activated T2s cannot make up for the

headstart T1 has in the race to awareness).

There are a number of other findings from our lab that can also be

interpreted in the light of T2 backward masking: For instance, the

proportion of order reversals at lag 1 is significantly lower when T1

and T2 are both colored in red, compared to when all stimuli are

black. This is possibly due to the fact that when T2 is red and the

T2+1 distractor is black, backward masking is weaker compared to

when both T2 and the T2+1 distractor are black. This difference

between colored targets and black targets is gone when target-color

and distractor-color are isoluminant and backward masking thus

does not differ between these conditions (also see Shih & Reeves,

Figure XXX:

summary of

experiment

“target_colors”

2007). Additionally, in these conditions forward masking for T2

remains nearly identical. In both cases the T1 stimulus has the same

color as T2. This does not mean that forward masking does not play

any role for order reversals; but, as also indicated by the already

discussed T2+1 experiment, it shows that manipulating backward

masking is sufficient to influence the proportion of order reversals.

Figure 9: Results of the “target colors” experiment as a function of

color condition. Left: proportion of order reversals. Right: T1 and

conditional T2 accuracy. Error bars represent standard errors of the

mean.

Another piece of evidence comes from an experiment in which we

compared conditions with targets in one location to targets in

different locations, both with and without surrounding distractors. In

both cases order reversals significantly increased when the target

were presented among distractors. In this experiment I cannot

disentangle the effects of forward masking and backward masking,

but the result can again be interpreted in light of the backward-

masking hypothesis. Later on, I will come back to the issue of targets

at the same location versus targets at different locations, but for the

moment let us consider another possible implication of this data:

Figure 10: Results of the “single vs. dual stream” experiment as a

function of condition and target onset asynchrony. Left: proportion of

order reversals. Right (top): T1 accuracy. Right (bottom): conditional

T2 accuracy. Error bars represent standard errors of the mean.

In Hilkenmeier, Scharlau, Weiß, and Olivers (2011) we argued that

one of the main differences between the temporal-order judgment

paradigm (TOJ) and the AB paradigm is the size of the task set.

Therefore, it seemed plausible that the higher identification demands

in the AB task lead to the significantly higher proportion of order

reversals compared to similar temporal distances in the TOJ task.

However, as was shown there as well as in the present manuscript,

the size of the task set did not influence the proportion of order

reversals in the hypothesized direction. On the contrary: order errors

increase with smaller task set size. Thus, this factor cannot explain

the difference between TOJ and AB. Another obvious difference is

the presence of distractors in the AB task and their absence in the

TOJ task: When the TOJ task is modified to include distractors as

well, order errors increase to a similar level as in the AB task. On the

Figure XXXI:

summary of

experiment

“single vs. dual

stream”

other hand, when the AB task is modified in a way that distractors are

omitted, order reversals decrease to a similar level as in the TOJ

task. Further research is needed, but it seems that the presence or

absence of distractors (and especially the T2+1 distractor) is one of

the key differentiators between the TOJ and the AB task.

Let us return to the aspect of same target location vs. different target

locations: As can be seen in Figure 10, the proportion of order

reversals is higher when the two targets are at different locations.

This is true both for conditions with surrounding distractors and for

conditions without distractors. In fact, the absolute amount of order

errors (i.e. not divided by the number of trials in which both targets

are identified, but divided by the absolute number of trials per

condition) does not differ between the single-stream with distractors

and the dual stream with distractors (t<1). Even though subjects have

additional information (location), they cannot use this information to

accurately determine the temporal order of events. On the contrary,

the actual order judgment is even worse. This finding seems odd in

light of Spence and colleagues’ results, which evidenced that

redundant spatial information can facilitate temporal discrimination

(Spence, Baddeley, Zampini, James & Shore, 2003; Zampini, Guest,

Shore, & Spence, 2005). However, at least the conditions with

distractors of the present experiment can again be explained in the

light of target strength. Consider the following example: at the

beginning of the trial attention is (or at least: should be) at fixation

between the two streams. When T1 is presented at the left stream,

Figure XXXII:

representation of

the 4 different

conditions in

“single vs. dual

stream”

attention shifts to that location. When T2 is presented at the right

stream shortly after (keeping in mind that in this experiment the

maximum TOA was 200 ms), attention again has to switch its

location. A weakly activated T2 is therefore often missed. When T2 is

strongly activated, it will get identified as well. And because it is so

strongly activated, it again has a better chance of overtaking T1 in

the race to awareness. On the other hand, when T1 and T2 are in the

same stream, also less activated T2s get identified (after all, attention

does not have to change locations). But again, these lesser activated

T2 more often lose the race to awareness against T1. The finding

that the absolute number of order errors does not differ between the

two conditions with distractors seems to corroborate this hypothesis:

in addition to the strongly activated T2s, more weakly activated T2s

get identified when T1 and T2 appear at the same location. These

weakly activated T2s increase the conditional T2 performance, but

they are too weak to overtake T1 in the race to awareness, i.e. the

absolute number of order reversals remains constant, but the relative

proportion of order reversals decreases. The same is true for the

already described RSVP-Speed experiment: for example, the

absolute number of order reversals at a TOA of 100 ms does not

significantly differ between the “50 ms condition” and the “100 ms

condition”. The relative proportion of order reversals in the latter one

is lower because the weaker masking of T2 leads to a higher T2

performance.

The idea of target strength is not a new one. It is incorporated in

several theories of the attentional blink, for instance the interference

model (Shapiro et al., 1994), the attention cascade model (Shih,

2008, 2009; also see Reeves & Sperling, 1986) or the eSTST model

(Wyble, Bowman, & Nieuwenstein, 2009, also see Bowman, Wyble,

Chennu, & Craston, 2008; Wyble & Bowman, 2005). The target

strength I propose here has a different twist than the definitions of

target strength before: In Wyble et al. (2009, p.9; Figure 11) replacing

the T2+1 distractor with a blank increases the target strength of T2,

leading not only to a reduced blink but also to an increment in the

proportion of order reversals (see Figure 11 for simulations of the

eSTST model).5, 6

Figure 11: Order error, T1 performance, and T1&T2 performance

simulations of the eSTST model as a function of target onset

asynchrony. Left (top): standard attentional blink with 10 items / sec.

Right (top): attentional blink with 20 items / sec. Left (bottom): 10

items / sec stream, but the T2+1 distractor is replaced by a blank.

This means that target strength is directly bound to visibility of a

target. Here, I suggest decoupling this relationship. The longer a

target is visible, the greater the chances of that target of becoming

processed. This is well in line with a number of findings reported here

(e.g. the T2+1 blanks experiment or the RSVP-Speed experiment) as

well as in the literature (e.g. Giesbrecht & Di Lollo, 1998; also see

Wyble et al., 2009, p. 9). This visibility, which can only be

determined after stimulus offset (or, more precisely, after the stimulus

is backward masked) does not necessarily translate to target

strength. Target strength might be determined within a shorter time-

span, even before stimulus offset. A similar approach was taken by

Olivers and Meeter (2008). Their computational model only takes the

sensory activity during the first 15 ms of presentation as a measure

of perceptual strength of a stimulus (in their model this strength is

used to determine the amplitude of the boost or the bounce,

respectively). Only this target strength, which is determined shortly

after onsets of the targets, is relevant for the perceived temporal

order of the targets. Variations in target length or post target masking

effects only influence the visibility of a target (and therefore have a

strong influence on target performance), but not its strength. In terms

of the attention cascade model (Shih, 2008; 2009, personal

communication) this means that the temporal order is determined by

the initial strength value and not, as proposed by Shih, by the

weighted strength at the end of the consolidation process. All

empirical data presented in this manuscript so far, especially the

order reversal data, are explainable within this definition of strength.

Later on, I will introduce a basic computational model that was drawn

up to account for the distribution of order reversals in Hilkenmeier,

Scharlau, Weiß, and Olivers (2011). This model can also account for

a number of order-reversal findings presented here and is largely

compatible with this definition of target strength. I will discuss the

computational model in more detail when I present the findings of

Hilkenmeier, Scharlau, Weiß, and Olivers (2011).

As just described, many AB theories that subscribe to the concept of

target strength hypothesize that this strength is determined at the

end of a consolidation process in which T1 and T2 are both

processed within the same batch (Shih, 2008, p. 214, p. 219; Shih,

2009; Akyürek, Toffanin, and Hommel, 2008 p. 575; Shapiro et al.,

1994; Bowman & Wyble, 2007). In the following, I will argue that it is

not necessary to assume such a late determination of perceived

order. In fact, it is not necessary to assume episodic integration

within a common batch at all.

The theoretical framework for this claim is the already outlined Boost

and Bounce theory of temporal attention (Olivers & Meeter, 2008). As

other recent theories about the attentional blink this model assumes

that a task-relevant event (e.g. a target among distractors) starts a

transient attentional enhancement (in the standard attentional blink

task that stimulus is T1). This attentional “boost” is delayed: it starts

about 25 ms after stimulus detection and peaks another 75 ms later

(i.e. at around 100 ms; also see Bowman & Wyble, 2007; Nakayama

& Mackeben, 1989; Wyble, Bowman, & Potter, 2009; Reeves and

Sperling, 1986 for earlier implementations of transient attentional

enhancement). Due to the temporal characteristics of the RSVP

paradigm, T1 is already overwritten by the next item when most of

the additional attention arrives. In case of Lag 1, this post-T1-item is

T2. Thus T2 receives even more attention than T1 and can therefore

easily outperform T1. Put differently: the attentional facilitation,

originally triggered to ensure T1 processing, “accidentally” enhances

the target strength of T2 and leads to higher identification accuracies

for the second target. In this sense, attention can manipulate target

strength.

As discussed previously, target strength determines the perceived

temporal order between the two proximal targets. This means that

the delayed attentional enhancement which accidentally hits T2 can

account for the substantial number of trials in which T2 wins the race

to awareness. The mechanics and timing of the gating mechanism

assumed in B&B theory (or, for that matter, in any other theory of

transient attentional enhancement) are thus ample to explain the

order errors and lag-1 sparing.

This explanation of order errors implies that a relative shift of

attention in favor of one of the targets should have an impact on the

amount of order errors as well, a hypothesis well in line with one of

the “fundamental laws of attention”: prior entry. As I briefly touched

previously in this manuscript, the law of prior entry states that “the

object of attention comes to consciousness more quickly than the

objects that we are not attending to“ (Titchener, 1908, p. 251). The

prior-entry effect was one of the initial topics of experimental

psychological research. When Titchener included it into his seven

fundamental laws of attention, he could already look back at more

than 50 years of experimental research on that topic (Scharlau, 2007;

Sternberg & Knoll, 1973). There are several theories explaining the

phenomenon of prior entry, for instance the asynchronous updating

model (Neumann & Scharlau, 2007a, b) or the temporal profile model

(Stelmach & Herdman, 1991; for an overview, see Scharlau, 2007).

Unfortunately, these theories as well as the models of Sternberg and

Knoll (1973) cannot be applied to the RSVP design used in our line

of research. The model of Sternberg and Knoll assumes that the two

to-be-judged stimuli are presented in two independent channels

(1973, p. 635, p. 659ff). These “channels” need not to be thought of

as different modalities (p. 637). In the visual domain for instance, it is

sufficient to assume that the two stimuli appear at two different

locations. In RSVP all items appear at the same location and the

targets usually belong to the same category. It is therefore

implausible to assume that T1 and T2 are presented in different

channels. Thus, the independent channel model cannot be utilized

here.

Likewise, the asynchronous updating model (Neumann & Scharlau,

2007a, b) requires that the two target-stimuli are presented at

different locations. Prior entry occurs because the precue presented

at one of the target-locations already directs attention towards its

location (e.g. Scharlau, 2007, p. 679f). Again, in RSVP distractors

and targets all appear at the same location, so the asynchronous

updating model should assume that attention is already directed to

that location before the first target even appears.

In Stelmach and Herdman’s model (1991) attention is allocated to

one of two locations by instruction. Thus, this model concentrates on

the temporal profiles of the two target stimuli and how attention

changes these two profiles (Stelmach & Herdman, 1991, Figure 10;

Weiß & Scharlau, 2010, Figures 2 & 3). It is unclear how the model

would deal with other forms of cueing. Would an additional cue get a

temporal profile as well? Would it still change the temporal profile of

the target in the same way? Would the temporal profile of the cue

interact with the temporal profile of the target at the same location?

Since these theoretical questions remain open, the model in its

current form cannot be employed to the RSVP paradigm.

Here, I focus on the perceptual retouch model (PRM, Bachmann,

1989) since PRM is most compatible with the RSVP design

employed in attentional blink studies (Bachmann & Hommuk, 2005).

Moreover, PRM comes with a plausible neurophysiological basis

which centers on the different nuclei of the thalamus. Later on, I will

speculate how this neurophysiological basis can be used to account

for the attentional blink as well.

The PRM originated as a theory to explain nonmonotonic backward

masking (Bachmann, 1984, p. 69), i.e. the phenomenon that when a

second stimulus (the "mask") is presented briefly (~ 30 – 80 ms) after

another stimulus (the “target”), visibility of the first stimulus is often

severely reduced (for an overview, see Breitmeyer & Öğmen, 2006,

2007). According to Bachmann’s theory, backward masking as well

as prior entry occur because of the asynchrony of two parallel

afferent processes: On the one hand, there is specific processing

(SP) which is fast, spatially precise and encodes the specific features

of an object like color or orientation. The neurophysiological

counterpart of the SP process for vision is the lateral geniculate

nucleus / corpus geniculatum laterale (LGN / CGL), the first relay

station for signals sent from the retina on their way to the visual

cortex (e.g. Nolte, 2002).

The other afferent process in Bachmann’s theory is the modulatory

nonspecific activation (NSP). NSP is necessary to modulate the SP

information, otherwise the SP information could not become

consciously available. NSP acts like a spotlight equipped with an

energy-saving lamp: As a spotlight, it not only illuminates a certain

stimulus, but the area around it as well, i.e. it is spatially imprecise.

As an energy-saving lamp, it also is not instantly on, but takes some

time until it reaches its full energy, i.e. it lags 50 – 80 ms behind the

SP. Although assumed to operate faster, this delayed modulation

has the same effect as the “boost” in Boost & Bounce theory or the

“blaster” in eSTST: It enhances stimulus information, but not

necessarily the ones (or at least not exclusively) that triggered the

modulation. If a second stimulus is presented in close spatiotemporal

proximity, it benefits from this modulation as well and its latency to

consciousness is shortened. On the neurophysiological side the NSP

is represented by the intralaminar nuclei, the reticular nuclei (TRN)

and the pulvinar. These nuclei do not participate directly in the

operations of encoding the contents of specific sensory information,

but modulate the level of activity in the LGN (Bachmann, Luiga,

Põder, & Kalev, 2003, p. 283). Surprisingly, more than 90 % of

synaptic inputs on the LGN are modulatory in nature (Van Horn,

Erisir, & Sherman, 2000), meaning that there are relatively few

synapses that get the basic visual information from the retina to relay

cells. These few specific synapses can be adjusted by many weak

modulatory synapses that can be combined in numerous ways to

allow for many forms of modulation. The drastic disparity between

synapses carrying actual content and synapses modulating that

content suggests that the major role of the thalamus is not only to

relay information, but to gate the flow of information to cortex

(Sherman, 2006; also see Crick, 1984). Moreover, this process is

highly efficient: For each relay cell in the LGN, there are roughly 160

neurons in primary visual cortex (at least in the cat; Sherman, 2006).

Thus, modifying the information flow at the level of the thalamus is

much more efficient than doing so after the information has reached

the cortex, making the thalamus an ideal starting-point for attentional

processes. As already mentioned, the spatial resolution of the non-

specific nuclei is quite poor. Not only the specific receptive field of the

SP is modulated (boosted) but also neighboring ones. As the LGN,

the primary visual cortex is organized in retinotopic maps, meaning

the NSP really modulates the actual neighboring receptive fields, i.e.

the retinal area around the stimulus that elicited the SP and the NSP

in the first place. This allows PRM to account for a number of spatial

phenomena like the illusory-line-motion (Figure VIII); the flash-lag-

effect, in which a briefly flashed object, which is aligned with a

moving object is typically perceived as lagging behind the moving

one; or the Fröhlich effect, in which a laterally moving object will

appear not at its first physical position, but shifted in the direction of

motion (see e.g. Bachmann, 1999, 2010, for the flash-lag effect also

see Priess & Scharlau, 2009).

Figure 12: Schematic view of the thalamus. Relevant nuclei for visual

information processing and their projections to and from the cortex.

All of the processes assumed in PRM can happen without attention.

In PRM any stimulus, not just an attended one, elicits both the SP

and the NSP process (Bachmann, personal communication); but

there are several ways to incorporate attention into the model: For

instance, attention could trigger extra NSP and thus facilitate the

target stimulus. Returning to the metaphor of a spotlight equipped

with an energy-saving lamp to describe the NSP-process, NSP +

attention would mean that this energy-saving lamp now has 60 W

instead of 40 W (NSP without attention). It is the same mechanism,

only stronger. In the following, I will describe the effects of the

additional attentional modulation on the perceived order of two target

stimuli in close temporal proximity.

To manipulate attention (and therewith target strength) in the RSVP

paradigm, we chose a design with colored stimuli close to the ones

used by Nieuwenstein and colleagues (Nieuwenstein, 2006;

Nieuwenstein et al., 2005) and Olivers and colleagues (Olivers, van

der Stigchel, & Hulleman, 2007; also see Olivers & Meeter, 2008, p.

24 “rapid reversal of the blink”). The two target stimuli were colored

letters, always presented right after each other in lag 1. Distractors

were black letters and digits, preceding and trailing the two targets.

In the crucial cueing condition, the distractor digit prior to T1 was

colored as well, i.e., it carried one target-defining property (color),

and one distractor-defining property (category).

Analogous to T1 starting the enhancement and T2 benefitting from it

in the standard AB, we hypothesized that the cue would start the

Figure XXXIII:

summary of the

experiments in

Olivers et al., 2010

enhancement and T1 should be the main profiteer from it. Thus, the

relative attentional weights should shift in favor of the first target,

resulting in less target-order reversals between T1 and T2. As

predicted, we found significantly less order reversals in the cueing

condition compared to a baseline condition in which the two targets

were not preceded by a cue (Olivers, Hilkenmeier, & Scharlau, 2010,

Experiment 1). The same was true for cue and T1 did not sharing the

same color, showing that cueing can occur in the absence of shared

features, as long as the cue carries a task-relevant property (Olivers,

Hilkenmeier, & Scharlau, 2010, Experiment 3). Moreover, we found a

substantial correlation between T2 accuracy and order reversals.

Participants who reported T2 more often than T1 also showed a

greater proportion of order reversals. In addition, the subjects that

showed the strongest reduction in order reversals due to the cue

also showed the strongest increase in T1 accuracy relative to T2

accuracy in the same condition. These results suggest that, in line

with the law of prior entry, order reversals at lag 1 are modulated by

the relative proportion of attention received by the two targets: Order

reversals occur when T2’s representation is strong enough to

overcome T1 in the race to awareness.

However, prior entry is not the only possible explanation for these

data patterns. The findings of Olivers et al. (2010) are largely

consistent with an episodic integration account: The colored precue

matches a task-relevant aspect of the target stimulus. Thus, it is

likely that the cue initiated an episode, particularly because the

Figure XXXIV:

Baseline and T1 cue

condition used in

Olivers et al., 2010

Figure XXXV:

Hypothesized

attentional

enhancement for the

baseline and T1 cue

condition

occurrence of a color singleton always was a valid predictor for the

first target (it either was T1 or it was at least signaling the very

imminent onset of T1). If we assume that episodes have a limited

duration, the cue and T1 are most likely processed in one episode,

but in most cases, T2 will come too late to be included as well. In that

case, T2 will have to start its own episode. Then, order errors are

less likely, as T1 and T2 are not part of the same event and thus not

very vulnerable to temporal confusion (Hommel, personal

communication). In this line of argument, T2 accuracy should

decrease when T1 is precued, a finding that is indeed present in the

data and cannot easily be explained by a straightforward prior-entry

account. The argumentation in favor of episodic integration relies on

a number of implications, though. For instance, as already

mentioned, it requires that the episode has a limited duration;

otherwise, the cue, T1, and T2 could all be part of the same episode,

which would result in no difference between the cue and the baseline

condition (a similar mechanism is assumed in the eSTST model of

Wyble, Bowman, & Nieuwenstein, 2009. In there, the attentional

episode remains open as long as task-relevant information, in this

case colored items, come in). Moreover, this argumentation assumes

that in a considerable number of trials T2 at lag 1 can trigger a

second episode, such that T2 can be given its own cognitive time

stamp. It is unclear why in this case T2 should be able to start a

second episode while T1 is being consolidated, but not in the

Figure XXXVI:

precue and T1 are

processed in one

episode, T2 at lag 1

manages to start a

second episode and

gets processed

separately

standard attentional blink task, in which processing of T1 is said to

cause the blink in the first place.

Furthermore, note that in the precue as well as in the baseline

condition the relevant T1 information occurs at exactly the same

temporal position. Even if the episode starts early in the precue

condition, it starts with an irrelevant distractor item. The T1 identity-

information is available only at the same moment as in the baseline

condition. Thus, T1 consolidation (which is assumed to be the cause

for the blink in most episodic integration theories) should not end

faster in the precue than in the baseline condition,7 unless we

assume that the precue somehow accelerates T1 processing. But

such an acceleration would come close to the prior entry account

championed here, namely the order of report being determined by

the relative amount of attention each target gains (for more

information regarding the episodic integration explanation see the

General Discussion in Olivers, Hilkenmeier, & Scharlau, 2010).

To summarize: the predictions of the prior entry account and the

predictions of the episodic integration account for T1 precueing data

are too similar and the results are too indecisive to exclude one of

the theories just yet. Nevertheless, at the very least this first set of

experiments shows that it is not necessary to assume episodic

integration to explain order reversals in RSVP. The results can at

least be equally well explained by prior entry and transient attentional

enhancement.

Next, I will present experimental conditions that will delineate the

prior entry predictions from the episodic integration predictions more

clearly.

To that end, we included conditions in which T1 and T2 are still

presented right after each other at lag 1, but instead of T1, T2 is

precued. In our interpretation of episodic integration this T2 cue

should not influence the proportion of order reversals. After all, the

T2 cue is presented after the episode already having started (elicited

by T1). Since T1 and T2 are presented at the same temporal

distance, episodic integration should occur to a similar degree,

regardless whether an additional cue is presented in between or not.

If, on the other hand, the temporal order of the two proximal targets is

determined by the relative distribution of attention between them (as

suggested by the law of prior entry), not only should precueing T1

lead to decreased order errors, but, by the same token, precueing T2

should lead to an increase in order reversals (see Hilkenmeier,

Olivers, & Scharlau, 2011, p. 7 lines. 111 - 114).

To integrate a cue in the temporal space between T1 and T2 at lag 1,

we decided to present each stimulus twice in succession for half the

usual presentation time. This allowed us to color each of the stimulus

“halves” individually. In specific, we colored only the first halves (the

first 50 ms) of each target stimuli. As in Experiment 3 of Olivers et al.

(2010), the two targets always had different colors. For instance the

first half of T1 was red, whereas the second half was black again.

Then the first half of the following T2 was green and the second half

of T2 again black. To precue T2, we colored the second half of T1 in

Figure XXXVII:

summary of

experiment 1 in

Hilkenmeier et al.,

2011

the same color as T2. We reasoned that, just like in a baseline

condition without any additional cues, the attentional enhancement

would be triggered by the first half of T1. But this enhancement

should be reinforced by the colored second half of T2, resulting in

more enhancement for the subsequent T2 and thus in a reduction of

order reversals.

As predicted by prior entry, the proportion of order reversals indeed

increased when T2 was precued (Hilkenmeier, Olivers, & Scharlau,

2011, Experiment 1). This result is again not limited to trials in which

the cue and T2 share the same color, indicating that at least part of

this effect must be attentional and not due to some kind of lower-level

sensory priming (Hilkenmeier et al., 2011, Experiment 3).8

The design employed in these experiments is vulnerable to one

critical objection: As already described, in the baseline condition only

the first half of T1 and the first half of T2 were colored. In between,

the second half of T1 was presented in distractor-black. In the T2

cueing condition on the other hand, the second half of T1 was

colored as well, resulting in an uninterrupted sequence of colored

(i.e. task-relevant) stimuli. One might object that in the baseline

condition the “distractor-like” second half of T1 might have caused an

early closure of the integration episode. Put differently, the

integration episode might be prolonged in the T2 cue condition as

long as task-relevant information was presented. In any case, this

argumentation leads to the claim that the T2 cue condition utilized in

Experiments 1 and 3 of Hilkenmeier et al. (2011) does not actually

Figure XXXVIII:

baseline and T2 cue

condition used in

Hilkenmeier et al.,

2011, Experiment 1

increase the proportion of order reversals, but that the baseline

condition employed artificially underestimates the proportion of order

errors.

To counter this objection, Experiment 2 in Hilkenmeier et al. (2011)

additionally included a modified baseline condition in which both

halves of T1 and T2 were colored. As in the T2 cue condition, this

modified baseline contained no distractor features between the two

targets. According to our interpretation of episodic integration, this

constant stream of “task-relevant information” should allow for a

single, prolonged episode, just as in the T2 cue condition. Therefore,

an episodic integration account should not predict any difference in

order errors between this modified baseline and the T2 cue condition.

These two conditions should significantly differ from the old baseline

in which the second half of T1 was colored in distractor-black.

Therefore the amount of order errors should be underestimated. Prior

entry on the other hand predicts that the T2 cue should still enhance

T2 processing relative to T1, and thus increase the number of order

reversals regardless to which baseline it is compared to.

The empirical data strongly support the prior entry hypothesis and

refute the objection of the cueing results being due to the specific

baseline condition utilized.

As described at the beginning of this manuscript, the mechanisms

underlying lag-1 sparing can be seen as key to understanding the

attentional blink. Unfortunately, most empirical results regarding lag-1

sparing can to some degree be interpreted both in light of episodic

Figure XLI:

old baseline

Figure XXXIX:

summary of

experiment 2 in

Hilkenmeier et al.,

2011

integration and in light of attentional enhancement. In our own

experiments we chose not to focus on the phenomenon of lag-1

sparing itself, but on the accompanying phenomenon of target order

reversals.

In our view, the present results cannot consistently be explained by

episodic integration. Even though the reduction in order reversals

when employing a T1 precue are fairly compatible with an integration

account, it is hard to imagine how an increment in order reversals

can be explained by episodic integration. Problematically, theories of

the attentional blink that promote the idea of resource depletion

cannot easily drop the assumption of episodic integration, since it is

not only used to explain order reversals, but lag 1 sparing itself.

When T2 at lag 1 cannot be processed together with T1, how can it

be spared when T1’s hunger for resources should be maximal?

The empirical evidence presented in Olivers et al. (2010) and

Hilkenmeier et al. (2011) first and foremost indicates that order

reversals in the RSVP paradigm are indeed best explained by the

law of prior entry: An attended stimulus enters consciousness prior to

an unattended one, i.e., attention alters the temporal features of the

perceived stimuli. In Titchener’s words: “the stimulus, for which we

are predisposed, requires less time than a like stimulus, to produce

its full conscious effect” (Titchener, 1908, p.251).9 This means, the

well known and reliable effect of prior entry does not only affect

experimental paradigms with distributed spatial locations in which a

cue draws attention to a certain location. It can equally well be

Figure XLI:

old baseline

Figure XL:

modified baseline

Figure XLII:

T2 cue

applied to temporal attention paradigms in which stimuli all appear at

the same location but at distinct moments in time. Importantly, the

effects of attention on the perceived order of events are not restricted

to experiments conducted in our own labs. After reanalyzing some of

the data of Akyürek, Abedian-Amiri, and Ostermeier (2011), the

same effect is visible in that data as well (Akyürek, personal

communication), even though that experiment was designed for a

different purpose and used a different kind of cue. This again

emphasizes the influence of cueing on the relative attentional

weights each target gets and thus the proportion of order reversals.

To summarize, models that assume transient attentional

enhancement offer straightforward explanation of key findings of the

attentional blink: lag-1 sparing, the actual blink, and (thanks to prior

entry) order reversals. All these aspects seem to be related to the

relative strength of the respective stimuli. This strength can be

manipulated in numerous ways, for instance, as done in the present

experiments, by the deployment of attention.

For the rest of this manuscript, I will work under the premise that

order reversals in the RSVP paradigm can indeed be manipulated by

attention, just as in paradigms with distributed locations.

Next, I will tackle the time course of attentional facilitation. The

reason for this is two-fold: first, theories of transient enhancement

predict a distinct time-course: the facilitation should rise rapidly and

reach its maximum somewhere around 100 ms. More precisely, the

PRM expects the maximum to coincide with the asynchrony between

specific and nonspecific processes, which is hypothesized between

50 – 80 ms. eSTST and B&B stay in line with earlier results of Müller

and Rabbitt (1989), Sperling and Weichselgartner (1995), and

Nakayama and Mackeben (1989; also see (Kristjansson, Mackeben,

& Nakayama, 2001; Kristjansson & Nakayama, 2003) and predict

the peak between 95 ms (Olivers & Meeter, 2008, p. 14) and 110 ms

(Wyble, personal communication) after cue detection. All theories

hypothesize that facilitation should quickly decrease and be

completely gone after a few hundred milliseconds.

The few studies that systematically investigated the temporal aspect

of prior entry (e.g. Scharlau, Ansorge, & Horstmann, 2006; Hikosaka,

Miyauchi, & Shimojo, 1993; also see Scharlau & Neumann, 2003 b)

found a more sustained time course. In Scharlau et al., (2006) the

size of the prior-entry effect rose with cueing SOAs up to about 130

ms, remained constant up to roughly 250 ms and then slowly

decreased with some residual effect even after 1000 ms.

However, all of these studies utilized different locations. Attention

was either exogenously or endogenously directed to one of these

locations. Therefore, their measure of the time course of prior entry

might be confounded by a spatial switching component: If the spatial

shift to the new location takes longer than the temporal facilitation

elicited by the cue, the peak of the measured facilitation is shifted to

a later point in time (see the GD in Hilkenmeier, Weiß, Scharlau, &

Olivers, 2011, for a extended discussion). Since in the present

Facilitation

time

cue

facilitation

location

switch

Facilitation

time

cue

facilitation

Facilitation

time

measured

Facilitation

time

cue facilitation

Facilitation

time

cue facilitation

location switch

Facilitation

time

measured

Figure XLIII:

two hypothetical

distributions of cue

facilitation and spatial

shift

paradigm all stimuli appear at the same location, we can measure

the time course in the absence of any spatial switching effects. Thus,

the present design may provide us with a purer estimate of the

dynamics of prior entry.

o study the time course of prior entry at one location, we again used

the cueing paradigm already utilized in Olivers et al. (2010) and

Hilkenmeier et al. (2011). While refining this paradigm for longer

cueing SOAs, we encountered some difficulties. For one, we cannot

employ this paradigm to measure the time course of any T2 cueing

effect. Since all stimuli are presented at the same location, the T2

cue has to be presented after the onset of T1. Otherwise, we cannot

ensure that it only facilitates T2 and not “accidentally” facilitates T1

as well. On the other hand, T1 and T2 have to be presented in close

succession; or else the temporal distance becomes too large,

resulting in hardly any reversals between the two targets. Thus, we

are restricted to precueing T1. Still, which kind of cue should be

used? Should the cue range over the complete cueing SOA, i.e. vary

in length? Or should it have a fixed duration? If so, should the

distractor items between cue and target be colored as well? Or

should they be eliminated? Should the colors between cue and target

change? Or stay the same? Should participants be able to refrain

from their judgment in case they are uncertain? Do higher task-

demands influence the cueing effect?

As it turns out, none of these factors (nor any tested combination)

significantly influenced the time course of prior entry at one location.

Figure XLIV:

two of the cueing

conditions used in

Hilkenmeier,

Scharlau et al., 2011

As can be seen in Experiment 1 of Hilkenmeier, Scharlau, Weiß, and

Olivers (2011), each type of cue led to qualitatively the same result:

The peak of facilitation was always in the 50 ms cue condition. At a

cueing SOA of 100 ms, there was virtually no facilitation left. A further

experiment measuring on a finer time scale confirmed that the ideal

cueing SOA seems to be quite early, somewhere between 30 and 50

ms.

Figure 13: proportion of order reversals for the “cueing on a finer

scale” experiment. Error bars represent standard errors of the mean.

Moreover, the facilitatory effect is rather short-lived, with no

significant reduction of order reversals for cueing SOAs longer than

100 milliseconds. Obviously, this time course of prior entry is

strikingly different from the one measured with distributed locations.

This indeed suggests that studies utilizing a paradigm with different

locations might overestimate the peak of prior entry facilitation.

Figure XLIV:

summary of

experiment “cueing

SOAs on a finer

scale”

To evaluate whether these findings are in line with theories of

transient enhancement, as opined by the author, we incorporated a

basic computational model that captures the gist of this class of

theories. More precisely, it is a simplified derivative of the recent

boost and bounce theory (Olivers & Meeter, 2008), but omits more

complex effects like masking and sustained activity. Basically, it

consists of two parts: bottom-up saliency of the cue and the targets,

which are modeled as gamma distributions peaking 40 ms after

target detection; and transient attentional responses, which are

modeled as gamma distributions peaking 90 ms after stimulus

detection.

The actual target evidence at a given point in time is operationalized

as the cumulative product of the target’s bottom-up saliency and its

transient response, multiplied by the transient responses of all

preceding cues and targets. Due to this multiplicative approach the

target evidence of T2 at lag 1 eventually overtakes the target

evidence of T1. Precueing T1 can postpone the point in time at which

target evidence of T2 surpasses target evidence of T1. The

underlying hypothesis here is that the longer it takes T2 to overcome

T1, the greater the chances of T1 entering working memory first.

This model is also compatible with the distinction between target

strength and target visibility. As stated earlier in this manuscript,

target strength, which is relevant for the perceived temporal order, is

determined earlier than target visibility, which is only determined after

the offset of a target and is relevant for the identification

performance. With this distinction we were able to explain a number

of empirical findings; for instance that omitting the T2+1 distractor

leads to higher T2 accuracy, but not to a higher amount of order

reversals (T2+1 blanks experiment, Figure 2, also see Figure 11 for

modulations of the eSTST model). Figure 14 shows how our model

would handle this data. The left panel shows a baseline condition in

which each item is presented for 100 ms and immediately replaced

by the following. The right panel shows the experimental condition in

which the T2+1 item is replaced by a blank. This was done by

extending the bottom-up saliency of T2 (bottom panels).10 As can be

seen in the top-panels of Figure 14, the cumulative target evidence

for T2 rises much higher when the distractor trailing T2 is replaced by

a blank. I would argue that, in line with the empirical results, this

higher target evidence leads to higher identification accuracies for

T2.11 Importantly, although the cumulative target evidence of T2

increases, the point in time at which T2 overtakes T1 does not

change between these conditions. This means that the proportion of

order reversals should not increase, which is also in line with the

empirical data, but contrary to the model-simulations of eSTST.

The simulations of our model also show that order reversals at lag 1

increase with faster streams, exactly as the empirical evidence of the

RSVP-Speed experiment suggests (Figure 8).12

Importantly, the model can explain strong facilitatory effects as soon

as 50 ms after cue onset, despite the attentional enhancement

reaching its peak only 90 ms after cue onset. The reason for this is

that the bulk of bottom-up activity occurs during the first 50 ms. It is

this bottom-up activity which interacts with the current attentional

activity. Even if attention is not quite optimal yet, the product of the

interaction is already some substantial activation. Thus, the early

drop in order errors is in fact predicted by a straightforward transient

attention model.

Figure 14: simulations of the computational model. Left: standard AB

with 10 items / sec. Right: The T2+1 distractor was replaced by a

blank. From bottom to top: Bottom-Up Activity, Top-Down Response,

Cumulative Target Evidence for T1 and T2, respectively.

The model predicts a substantial reduction of order errors also at 100

ms cueing SOA, almost on a par with the effect for 50 ms. Clearly,

this was not the case in Experiment 1 of Hilkenmeier, Scharlau,

Weiß, and Olivers (2011), nor in the experiment investigating the

cueing effect on a finer time-scale. The model predicts this time

course because it treats the cue as if it was a normal target, and thus

as if it triggered a full attentional response. Note that in the

experiments reported so far, the cue was a distractor even though it

carried the target color. It is therefore possible and perhaps even

likely that the cue initially triggers an attentional response, but that

upon detection of its distractor-like properties, either attention is

rapidly disengaged (Theeuwes, 2010), or even suppressed (Olivers

& Meeter, 2008).

To account for such inhibitory effects, we allowed that a single

stimulus cannot only trigger facilitation, but also delayed inhibition.

The inhibition was modeled as the enhancement, only with a

negative algebraic sign and 50 ms offset. With this additional

assumption we were able predict the pattern found in Experiment 1of

Hilkenmeier, Scharlau, et al. (2011), i.e. a peak of facilitation at a

cueing SOA of 50 ms.

In turn, this means that when we use a different and more task

relevant cue that does not trigger such inhibition, the facilitation

should extend to 100 ms. This is indeed what we found in

subsequent experiments, employing a third target. In these

Figure XLV:

representation of

the kinds of cues

used in Experiment

2 of Hilkenmeier,

Scharlau et al., 2011

experiments, T2 and T3 were always presented right after each other

(at lag 1, if you will), whereas the temporal distance between T1 and

the target-pair was varied in the same way as the cue-T1 distance

was varied in the previous experiments. T1 could be one of the digits

“2,3,4,6,8,9”, whereas T2 and T3 always consisted of the digit pair

“5,7”, or “7,5”. Since the cue-identity (T1) had to be reported, it was

now highly task relevant. At the same time, by reducing the

identification task of T2 and T3 to a temporal order judgment (“which

one came first, the 5 or the 7?”) we kept the overall task demands

relatively low (see the “Task-Set” experiment and the related

discussion). With this experimental setup, the facilitation sustained to

100 ms. There was no significant difference in order errors between

the 50 and 100 ms cueing SOA. This suggests that the more task-

relevant cue (the additional target) extended, but not amplified the

enhancement, just as predicted by the computational model.

An alternative view is that the equal facilitation at 50 and 100 ms is

the result of an overlay of two very different processes: a short-lived

sensory priming effect and a somewhat slower starting attentional

effect. In this view, the “distractor-like cue” just elicits the sensory

effect since it is colored and therefore quite salient. There might be

some developing attentional enhancement as well, but this

enhancement is again stopped as soon as the system realizes that it

is not dealing with a real target. The “target-like cues” on the other

hand profit from both the sensory priming (again, all targets are

colored whereas the distractors are black) and the developing

100

400

facilitation

cueing SOA in ms

non-specific

sensory

effect

100

400

facilitation

cueing SOA in ms

attention

sensory

100

400

facilitation

cueing SOA in ms

measured

Figure XLVI:

hypothetical

interaction between

sensory activity and

attention

attentional facilitation. Whereas the sensory effect again peaks at 50

ms, the attentional effect peaks at 100. This results in the measured

facilitation both at 50 and 100 ms cueing SOA.

However, this explanation is unlikely. A further control experiment in

which all stimuli were black (i.e. the targets are just defined by

category, Experiment 3b in Hilkenmeier, Scharlau, et al, 2011) led to

a qualitatively similar time course. Thus, a sensory priming effect

induced by a color change cannot be responsible for our data

pattern. In fact, we were unable to find any facilitatory effect of

additional color information, as can be seen by the nonsignificant

saliency x cueing SOA interaction in Experiment 3a of Hilkenmeier,

Scharlau, et al. (2011); however, this might be due to a power

problem.

To sum up, the experiments of Hilkenmeier, Scharlau et al (2011)

show a time course that is consistent with rapid and transient

facilitation. In general, such boosts are predicted by perceptual

retouch theory as well as recent theories of temporal attention in

RSVP processing (e.g. eSTST, Wyble et al., 2009; B&B, Olivers &

Meeter, 2008). The peak of facilitation was found at about 50 ms

when using a task-irrelevant cue. Facilitation sustained to 100 ms

with a task-relevant cue. Both of these results are consistent with a

basic computational model which assumes that evidence for a target

accumulates as a result of a rapid transient bottom-up signal, which

is gated by a slower, but still transient top-down signal. The only

additional assumption we have to make is that stimuli that carry a

100

400

facilitation

cueing SOA in ms

wide

attentional

boost

100

400

facilitation

cueing SOA in ms

wide boost

sensory effect

100

400

facilitation

cueing SOA in ms

hypothesized

Figure XLVII:

hypothetical

interaction between

sensory activity and

a wide attentional

boost

distractor-defining feature cannot only trigger facilitation, but also

inhibition.

To my understanding, such inhibitory effects are not part of

Bachmann’s perceptual retouch model, even though there is an

obvious neurophysiological counterpart: As already discussed,

Bachmann’s theory centers around different nuclei in the thalamus

region. One of these structures is the reticular nucleus (TRN). Each

information from the thalamus to higher-order structures in the cortex

must project through this thin sheet of inhibitory neurons that form a

capsule around the thalamus (see Figure 12). If some higher-order

cortex area realizes that instead of a target, it actually deals with a

colored distractor, it could very well project to the TRN to inhibit

further input (as can be seen in Figure 12, the TRN receives

projections from the cortex and in turn has inhibitory projection into

the LGN, the relay station for visual information coming from the

retina). In this way, the TRN would act like the bounce described in

B&B theory. Indeed, there is some empirical evidence that supports

this view: For instance O’Connor, Fukui, Pinsk, and Kastner (2002)

used fMRI to investigate attentional response modulation in the LGN.

As expected, LGN activity was enhanced when subjects attended to

the stimuli, but was also suppressed when they ignored them.

Unsurprisingly, the V1 activity mirrored this pattern, but interestingly,

the attentional effects in V1 were smaller than the ones in the LGN.

O’Connor et al. (2002) argued that this indicates that LGN

modulation must be influenced by factors other than cortico-thalamic

feedback from V1 to the LGN, and suggested a strong role of the

TRN in this process.

Another piece of evidence comes from McAlonan, Cavanaugh and

Wurtz (2008). As already described, TRN has an inhibitory influence

on the LGN. Thus, stronger LGN activity should be associated with

lower TRN activity. That was exactly what these authors found when

they recorded visually responsive neurons in the TRN and LGN of

awake macaque monkeys performing a simple spatial attention task.

Earlier results of the same authors (McAlonan et al., 2006) can also

be interpreted in light of the modulatory TRN role. In this earlier

study, monkeys had to attend to a tone while ignoring a visual

stimulus or vice versa. This task increased the firing of the inhibitory

TRN cells. I would argue that this increased TRN-firing was due to

inhibiting (bouncing) the non-relevant dimension, but for now, this

remains speculative. However, it would fit in well with the proposed

view that TRN could represent the neurophysiological counterpart of

the bounce in B&B theory: As long as no target-relevant stimuli are

presented (either nothing or distractors in RSVP) there is medium

TRN activity lightly inhibiting the visual information. When a target is

presented, the TRN activity (and therewith the inhibition) is lowered.

When a distractor is presented afterwards, TRN activity is amplified

to inhibit that distractor. This hypothesis is further corroborated by the

fact that the TRN modulation, just like the bounce, takes some time

until it reaches its full effect (see McAloan et al., 2008).

The integration of inhibitory effects into perceptual retouch (a retouch

& bounce theory, if you will) would have some distinct advantages

over the current PR model and B&B theory. By including a bounce,

the PR model would be able to explain a number of recent RSVP

findings, not only the time course of cueing presented here, but also

the standard attentional blink, the rapid recovery of the blink, and the

whole-report vs. partial report findings of Nieuwenstein and Potter

(2006). Moreover, this more inhibitory role of top-down attention

would fit in well with Belopolsky et al.’s (2010, p. 340) conclusion that

“the primary role of the top-down set is to control the disengagement

of attention from the features that do not match it.” By including the

spatial fuzziness of perceptual retouch, B&B theory on the other

hand would gain the ability to account for a number of spatial

distributed phenomena as well, thus extending from the RSVP

design to related paradigms as for instance the flash-lag-effect

(Bachmann & Põder, 2001) or illusory line motion (Bachmann, 1999).

Moreover, there is an upper limit of enhancement in PR. Once it is

reached, further enhancing stimuli can just maintain the level of

enhancement, but do not increase it any further. In my opinion, this is

an advantage over B&B, where (at least in the computational model)

every facilitatory stimulus just increases the level of enhancement,

making it difficult to compute the influence of cueing on order

reversals within this model.13

One might object that the processes assumed in perceptual retouch

are all relatively early. After all, the thalamus is the first relay station

of visual information coming from the retina. On the other hand,

biasing or gating information is often seen as a higher order process

that takes place in the prefrontal cortex (e.g. Miller, Erickson, &

Desimone, 1996; Miller & Cohen, 2001; Awh & Vogel, 2008; Olivers

& Meeter, 2008; but see Sherman, 2006; Crick, 1984). Maybe this

apparent discrepancy is not so hard to overcome after all: as can be

seen in Figure 4 of Hazy, Frank, and O’Reilly (2006), thalamus and

prefrontal cortex are connected via the basal ganglia in a complex

loop. Thus, any activity in the thalamus is mirrored in the prefrontal

cortex and vice versa, suggesting that early influences on the level of

the thalamus could fit in with current empirical evidence and theory.

One last experiment that can be seen as indication for such early

influences on the attentional blink shall be discussed here: In a

standard RSVP design, all stimuli are presented at the center of the

screen to both eyes. By employing shutter glasses, we are able to

present different visual information to each eye. We can present one

item to only one eye whereas the other eye only sees grey

background. This way, we can contrast conditions in which two

consecutive stimuli were presented to the same eye, to conditions in

which they are presented to different eyes. This means, we can

selectively change masking on eye level, leaving other factors like

stimulus duration, stimulus intensity, inter stimulus interval, and

location all unchanged. This use of the shutter technique is a

promising line of work since it enables us to disentangle different

aspects of stimulus presentation that were previously confounded.

Figure XLVIII:

summary of

experiment

“monoptic/dichoptic

blink”

As discussed earlier, masking has a strong influence on the

attentional blink. Thus, the original purpose of this experiment was to

explore the influence of early masking effects that occur before

binocular integration (e.g. Lumer, 1998). As hypothesized, during the

blink period (200 – 300 ms), the second target deficit was stronger

when masking for T2 was stronger, i.e. when the distractors

immediately preceding and trailing T2 were presented to the same

eye, compared to when they were presented to the other eye.

When the two targets were presented at lag 1 and both to the same

eye (i.e., strong masking), conditional T2 performance was

significantly higher than when the two targets were presented to

different eyes (i.e., weak masking). Likewise, there were significantly

more order errors when the two targets were presented to the same

eye.

Figure XLIX:

representation of

monoptic viewing

condition used in

“monoptic/dichoptic

blink”

Figure 15: Results of the “monoptic/dichoptic blink” experiment as a

function of viewing condition and lag. Left: conditional T2 accuracy.

Right (top): T1 accuracy. Right (bottom): proportion of order

reversals. Error bars represent standard errors of the mean.

As discussed previously, increased T2 accuracy combined with an

increased proportion of order reversals is indicative for transient

enhancement elicited by T1. Yet, the facilitation triggered by T1 only

seems to hit T2 when the second target is presented to the same

eye. When T2 is presented to the different eye, the facilitation does

not reach it, indicating that the boost primarily uses monocular

reentrant channels (Bachmann, personal communication).

In this line of argumentation, these findings can be seen as some of

the early thalamic effects described within Bachmann’s perceptual

retouch model: T1 is presented to the fovea of the left eye, and even

though the fovea has projections to both thalami, thalamo-cortical as

well as cortico-thalamic retouch effects are laterally biased to the eye

strong masking

weak masking

Figure XL:

representation of the

lags used inmonoptic

“monoptic/dichoptic

blink”

of the input (Bachmann, 2007, Bachmann, personal communication).

As a consequence, the nonspecific modularly effect triggered by T1

in the left eye is stronger for T2 when it is presented to the left eye as

well, compared to when T2 is presented to the right eye. Again, this

interpretation is not undisputed and further research in this direction

as well as further controls are necessary. Nevertheless, the results

indicate that early influences (occurring even before binocular

integration) do play a role in the attentional blink and in RSVP

processing as a whole. This suggests that merging the

neurophysiological assumptions of perceptual retouch with the

computational model of boost and bounce theory might be a

promising line of work.

To summarize: Selecting relevant over irrelevant information is one

of the crucial functions of our brain. It allows us to efficiently deal with

a limited number of objects while ignoring other information in our

environment. The mechanisms that allow for this selection are

collectively known as attention. How exactly attention works has

been one of the major topics of psychological research since the

days of Helmholtz and James. Much of the earlier work has

concentrated on how attention is distributed in space. Interest in the

temporal aspects of attention has only risen in the last 25 years or

so. To investigate the temporal dynamics of attention, the RSVP

design, and especially its two-target version, has become a fruitful

experimental paradigm.

In our own work we have taken a different approach. Instead of

concentrating on target-identification accuracy, we investigated the

proportion of temporal order reversals between the two targets.

These order reversals were originally seen as strong evidence for

episodic integration; however, we found that they can at least equally

well be explained by transient attention, via the “law of prior entry”,

thereby demonstrating that this law does not only apply to paradigms

utilizing different spatial locations, but also to the RSVP paradigm in

which all items are presented at one location but at distinct moments

in time. After testing the influences of different manipulations such as

task-demands, stimulus duration, presentation speed, or location on

the proportion of order reversals, we created precueing conditions

that lead to different predictions for resource-depletion/ episodic-

integration theories and transient-attention models.

The present results do not decide the argument between resource-

depletion / episodic-integration on the one, and transient attention

models on the other hand.14 However, they represent evidence that

can much more easily be explained by transient attention than by

episodic integration, indicating a strong role of the former one in the

dynamics of serial visual processing. Thus, the results presented

here can be seen as pieces of empirical evidence that inspired new

research and have therefore added to the scientific progress. And

this is really all one can aim for. To put it in Popper’s words (1945, p.

12): “In science, we never have sufficient reason for the belief that

we have attained the truth”.

Footnotes

An alternative to the spotlight model is the so-called zoom-lens

model (Erikson & St. James, 1986). In analogy to the zoom lens of a

photo camera, the size of the attentional focus can be adjusted.

Instead of moving the attentional focus from one location to another,

the system could simply “zoom out” to cover both spatial areas.

However, as with the photo camera, zooming out means losing

details, which in this context means that processing of an individual

object takes longer the larger the focus of attention is.

We will come back to rapid and transient deployment of attention in

more detail to explain lag-1 sparing and order reversals in the

attentional blink.

In attentional-blink studies, the temporal distance between the two

targets is usually specified in “lag”. Lag refers to the serial position

after T1. For instance, the stimulus presented at the T1+2 position is

presented in “lag 2”. Since all stimuli in the RSVP stream are

presented for the same duration (typically for 100 ms), lag can

without further ado be converted in target-onset interval. The term

“lag-1 sparing” suggests that it is really only the first item after T1 that

gets spared. However, studies that doubled the rate of presentation

from 100 to 50 ms per item evidenced that sparing extends out to

lag-2. That is, the second target is spared not because it is directly

adjacent to T1, but because it occurs within 100 ms (e.g. Bowman &

Wyble, 2007). Therefore, when I use the term “lag-1 sparing” I mean

the unimpaired T2 accuracy within the first 100 ms after T1

presentation.

The post hoc power analysis estimated a power of about 80% to find

a significant effect, assuming the effect size in our sample is equal to

the effect size in the underlying population.

The discrepancy between empirical and computational data is

especially troubling for the eSTST theory, since modeling the T2+1

blink data (or, more precisely the T2 end-of-the-stream data) was in

particular emphasized by the authors.

A similar mechanism is assumed in Shih (2008, p. 214, p. 219). In

there, the relative strength of two targets encoded in the same

consolidation process determines the perceived temporal order.

However, it is unclear whether T2 is processed in the same batch as

T1 when the T2+1 distractor is omitted (Shih, personal

communication). Therefore, it is possible that the attentional cascade

model would predict less order reversals in the T2+1 blank

experiment as well.

On the contrary: A straightforward resource competition model would

assume that the more difficult T1 is to detect, the stronger the

inhibition for T2 should be. Since in the precue condition, the item

immediately preceding T1 had the same color as T1. Thus, it was

more similar to T1. Therefore, it should take longer until T1 is read

out, resulting in a prolonged and deeper blink.

Please note that such a lower-level mechanism would not even be

problematic for the overall notion of target-strength. Brightness

summation or priming could as well strengthen the representation of

a target. The results at hand simply indicate that target strength can

be manipulated in absence of shared features between the cue and

the target, stressing the influence of attention (see Hilkenmeier et al.,

2011, lines 522-529; also see Nieuwenstein, 2006).

This does not necessarily mean that episodic integration plays no

role in RSVP order errors; the existence of prior entry does not

preclude the existence of integration.

10)

Omitting the T2+1 distractor could result in another bottom-up pattern

as the one shown in Figure 14. Yet, the modeling turns out in a

similar way for a number of different distributions as long as

extending the visibility only influences the latter part of the pattern

(the one where visibility actually changes) and not the overall

distribution.

11)

At this moment, target accuracies cannot be simulated within the

simple computational model. But since the long-term goal is to

integrate the order-error model back into the computational model of

boost and bounce, the T2+1 blank experiment data could proof

useful in testing that model and differentiate it from the latest

implementation of eSTST.

12)

Unfortunately, I was not able to simulate any other data than lag 1,

since the model does not account for items other than cues and

targets.

13)

The computational model presented here and in Hilkenmeier,

Scharlau et al. (2011) reduces the value of this statement. However,

as long as the current model is not implemented in boost and

bounce, this is still a valid point.

14)

“Scientists have thick skins. They do not abandon a theory merely

because facts contradict it. They normally either invent some rescue

hypothesis to explain what they then call a mere anomaly or, if they

cannot explain the anomaly, they ignore it, and direct their attention

to other problems.”Lakatos, 1978, p.3

Experiments

T2+1 blank Experiment

The purpose of this experiment was to test Bowman and Wyble’s

(2007, p. 36) claim that inserting a blank after the second target

would attenuate the blink. So far, empirical data for this specific

hypothesis was not available, only data about T2 being the last item

in the stream (Giesbrecht & Di Lollo, 1998), which basically abolished

the blink.

Method

Participants: Twelve students from Paderborn University, Germany

with (corrected-to-) normal vision participated for course credits or €6

an hour.

Stimulus, Design, and Procedure: Stimulus generation and response

recording were done using the Tscope programming library.

Backgrounds were gray. After a blank period of approximately 1000

ms, a 0.5 x 0.5˚ black fixation cross was presented for another 1000

ms in the center of the display and replaced by a rapid stream of 15

black digits and letters, presented in Courier New (approximately 0.8

x 0.8˚ in size). The letters I, O, Q, S, and Z were excluded, as was

the number 1. Each item was presented for 100 ms and then

immediately replaced by the following item, resulting in 10 different

items / sec. T1 was placed at position 4-9 in the stream. T2 followed

at lag 1, 2, 3, or 6. The two target-digits were always different. In half

of the trials the distractor-letter immediately trailing T2 was replaced

by a blank. Each lag was repeated 60 times: 30 times with a T2+1

distractor and 30 times without one. All trials were randomly

intermixed into a single block lasting about 40 minutes. The

participants’ task was to report the two target-digits in the correct

order at the end of the trial; unspeeded, and with feedback.

Results and Discussion

T1 accuracy, conditional T2 accuracy and proportion of order errors

as a function of lag can be seen in Figure 2, separately for the

baseline and the T2+1 blank condition. A repeated-measures

ANOVA showed no significant main effect of condition on T1

accuracy (F < 1), nor a significant interaction of condition with lag (F

< 1). As expected, the main effect of lag was significant (F[3,33] =

9.8, p < 0.05), showing the usual reduced T1 accuracy at lag 1. The

same analysis on T2|T1 accuracy revealed significant main effects

for condition (F[1,33] = 29.2, p < 0.05) and lag (F[3,33] = 3.9, p <

0.05), as well as a significant interaction (F[3,33] = 8.0, p < 0.05). As

can be seen in Figure 2, there was no blink when the T2+1 item was

replaced with a blank. This confirms Bowman and Wyble’s prediction.

However, the data do not fit that well into their computational model.

According to the model (standard settings), the blink should be

attenuated, but not abolished as when T2 is the last item in the

stream. Contrast analyses confirm this picture: when using the

values estimated by the computational model as contrast weights,

the t-value becomes negative (-2.1), indicating that the trend in the

observed pattern is opposite to the one suggested by (e)STST. In

fact, the values estimated when T2 is the last item in the stream fit

our empirical results better (competing contrast analysis tdiff[11] =

1.6, p < 0.05 one-tailed), but still far from perfect. Interestingly,

omitting the T2+1 distractor also affected the proportion of order

reversals. The repeated measures ANOVA showed significant main

effects for condition (F[1,33] = 22.5, p < 0.05) and lag (F[3,33] = 26.4,

p < 0.05), as well as a significant interaction (F[3,33] = 8.6, p < 0.05).

One could assume that T2, which persists longer when the trailing

distractor is omitted, is perceived as first more often since its visibility

increases. But contrary to that, there are less order reversals when

T2 is followed by a blank and is therefore more visible. This finding

might also shine light on the order-error results of the “RSVP-Speed”

Experiment. Order reversals increased when the RSVP stream was

presented with higher speed. This was realized by shortening the

inter-stimulus-interval, which in effect means that the T2+1 distractor

is presented more quickly.

To summarize, replacing the distractor immediately after T2 with a

blank basically abolishes the attentional blink. Thus, the effect is

much stronger than expected by Wyble and colleagues. It more

closely resembles the condition in which T2 is the last item in the

stream and not backward masked at all (Giesbrecht & Di Lollo,

1998). Omitting the T2+1 distractor also decreased the proportion of

order reversals. Maybe the underlying mechanism can also explain

the order-error results in the “RSVP-Speed” experiment. Here and

there, order errors decreased when the inter-stimulus-interval

between T2 and the following distractor increased, i.e. T2 backward-

masking was reduced.

Explicit Order Experiment

This experiment was designed to invalidate the objection that the

attentional blink task with the explicit instruction to report the two

targets in the correct order differs from the “classic” attentional blink

task without this order instruction. Since we were aiming for a null-

result, we conducted an a-priori power analysis to ensure we had

sufficient power to find an effect of at least medium size.

Method

Participants: Twenty one students from Paderborn University,

Germany with (corrected-to-) normal vision participated for course

credits or €6 an hour.

Stimulus and Procedure was identical to the previous experiment; the

Design differed in several ways: The experiment consisted of two

separate blocks. The order of these blocks was counterbalanced

between subjects. In both blocks participants had to report the two

target-digits embedded in the stream of distractor-letters. T2 could

appear at lag 1,2,3, or 6. In one block subjects were explicitly told to

report the targets in the perceived order. In the other block the

instruction emphasized that the order of report did not matter.

Results and Discussion

T1 accuracy, conditional T2 accuracy and proportion of order errors

as a function of lag can be seen in Figure 4, separately for the blocks

with and without order instruction. For T2|T1 accuracy and order

reversals, the main effect of block did not get significant (both F < 1).

The same was true for both interactions between block and lag (F <

1, and F[3,87] = 1.2, p = 0.33). The main effects of lag were of

course significant (F[3,87] = 23.3, p < 0.05, and F[3,87] = 165.4, p <

0.05, respectively). Surprisingly, for T1 accuracy the main effect of

the factor block did get significant (F[3,87] = 11.2, p < 0.05). So did

the main effect of lag (F[3,87] = 15.7, p < 0.05). However, T1

accuracy actually improved when participants had to report the two

targets in correct order. If anything, we had expected that the

addition of the judgment task would be more demanding and

therefore accuracies should be impaired compared to the condition

without order instruction. To summarize, the time course of the

attentional blink (i.e. the conditional T2 accuracy) and the time

course of order reversals is not influenced by the addition of the

explicit instruction to report the two targets in correct order. Subjects

seem to do that anyway.

100

Task-Set Experiment

The purpose of this experiment was to test whether the demands of

the target-identification task had an effect on the subsequent order

judgment task. We speculated that a more difficult identification task

could consume more resources. Therefore there would be fewer

resources left for the order judgment, leading to more order reversals

due to a higher proportion of order guesses. To realize different

demands for the identification task we manipulated the target-set

size. In the baseline condition all digits from 1 – 9 served as targets;

in the experimental condition the task set was reduced to the digits

“5” and “7”. This means “5” and “7” were presented in each trial and

the subject’s only task was to determine which of these two digits

came first.

Method

Participants: Twenty one students from Paderborn University,

Germany with (corrected-to-) normal vision participated for course

credits or €6 an hour.

Stimulus, Design, and Procedure was identical to the previous

experiment except for the following changes: Each item was

presented for 50 ms. After an ISI of 20 ms the following item was

shown, resulting in about 14 different items / sec. T1 was placed at

position 5-10 in the stream. T2 followed at lag 1, 2, or 3. The

experiment consisted of two separate blocks. The order of blocks

was counterbalanced between subjects. In the baseline block, the

101

target set consisted of the digits 1-9. The participants’ task was to

report the two target-digits in correct order (2 out of 9). In the

experimental block, the targets were always the “5” and the “7” in

random order. The participant’s task was to report which digit came

first (1 out of 2). Each lag in each block was repeated 40 times.

Results and Discussion

The proportion of order reversals can be seen in Figure 5. A

repeated measures ANOVA revealed significant main effects of

condition (“2 out of 9” vs. “1 out of 2”) and lag (F[1,40] = 26.4, p <

0.05 and F[2,40] = 14.3, p < 0.05, respectively). The interaction

between these factors was nonsignificant (F < 1). Unexpectedly,

order reversals increased when the task set was smaller and the task

therefore easier. This could be due to guessing since participants in

the “1 out of 2” condition could have simply guessed the order on

every trial without even paying attention to the stream. However,

additional experiments indicate that this effect holds true even when

participants have the opportunity to refrain from their judgment

(Hilkenmeier, Weiss, Olivers, Scharlau, 2011).

102

SD-ISI Variation Experiment

This experiment was originally motivated by Chua (personal

communication) who argued that the relatively high proportion of

order reversals found in our lab might be due to the fact that we

usually present the stimuli without inter stimulus interval. He

predicted that a reduced stimulus duration combined with longer ISI

should lead to a clearer separation of the two targets and thus less

order reversals.

Method

Participants: Twenty three students from Paderborn University,

Germany with (corrected-to-) normal vision participated for course

credits or €6 an hour.

Stimulus, Design, and Procedure was identical to the previous

experiment except for the following changes: The combination of

stimulus duration and inter stimulus interval was held constant to

about 68 ms / item. Within these 70 ms, we changed the SD/ISI in

four steps: SD 17 ms, ISI 51 ms; SD 34 ms, ISI 34 ms; SD 51 ms, ISI

17 ms; SD 68 ms, ISI 0 ms. These four possible conditions were

intermixed into a single session. T2 always followed T1 at lag 1 and

was repeated 30 times per condition.

Results and Discussion

The proportion of order reversals can be seen in Figure 7 (left). A

repeated measures ANOVA revealed no significant main effect of

103

SD / ISI variation (F<1). The same was true for T1 and conditional T2

accuracy (right side of Figure 5; F[3,66] = 1.3, p = 0.28 and F[3,66] =

1.5, p = 0.22, respectively). This indicates that a longer ISI and thus

an apparent easier separation between the targets does not

influence target performance or perceived order at all.

104

RSVP-Speed Experiment

The rationale of this experiment was to replicate and extend the

finding of Bowman and Wyble (2007) that lag-1 sparing is not bound

to the T1+1 position, but to the first 100 ms after T1 presentation,

regardless of the number of items presented within this time-span.

Here, we wanted to test whether this is true for temporal order

reversals between T1 and T2 as well.

Method

Participants: Twenty one students from Paderborn University,

Germany with (corrected-to-) normal vision participated for course

credits or €6 an hour.

Stimulus, Design, and Procedure: Stimulus generation and response

recording were again done using the Tscope programming library;

ancillary conditions like stimulus size and color were as in the original

experiment of Bowman and Wyble, 2007. The experiment consisted

of four separate blocks which varied the presentation speed of the

RSVP stream by manipulating the inter stimulus interval. In the

fastest condition the stimulus duration was 50 ms and the ISI 0 ms,

resulting in 20 different items / second. In the next condition SD was

again 50 ms and ISI was 25 ms, resulting in about 13.3 items /

second. In the third condition, SD was 50 ms and ISI was 50 ms as

well. Here, 10 items / second were presented, replicating a standard

attentional blink. In the last condition, SD was again held constant to

105

50 ms, but ISI was increased to 100 ms, resulting in about 6.7 items /

second. In each condition the TOAs up to 1000 ms were covered.

This means that in the “50 ms” condition, T2 could appear at a TOA

of 50 ms, 100 ms, 150 ms, 200 ms and so on, leading to 20 lags in

that condition. In the “75 ms” condition, T2 could appear with a TOA

of 75 ms, 150 ms, 225 ms and so on up to a TOA of 975 ms. The

equivalent was true for the “100 ms” and the “150 ms condition”.

Each lag in each condition was repeated 20 times, leading to 1000

experimental trials, divided in two sessions lasting about an hour

each.

Results and Discussion

The proportion of conditional T2 accuracy can be seen in Figure 8.

As is clear from a visual inspection, we could replicate Bowman and

Wyble’s first main finding that lag 1 sparing can spread to later lags

when the presentation speed increases. However, what is also clear

from a visual inspection is that we could not replicate their second

main finding, i.e., that the time course of the AB is independent from

presentation speed (Figure 19 in Bowman & Wyble, 2007). The

bottom of the curve seems to be wider and the slope less

pronounced, i.e. the second target does not recover as much. For

better comparison I conducted a repeated measures ANOVA with the

same data-points as Bowman and Wyble did, i.e. I focused on the

“50 ms” and “100 ms” condition and only compared the TOAs 100

ms, 200 ms, 300 ms, 400 ms, 500 ms, 600 ms, 700 ms, and 800 ms.

106

As expected, there was a significant main effect of lag (F[7,112] =

10.5, p < 0.05) and a significant main effect of presentation speed

(F[1,112] = 89.9, p < 0.05). The latter one simply means that,

unsurprisingly, the overall accuracy in the faster condition was lower.

However, unlike in Bowman and Wyble, the interaction between

speed and lag showed a significant effect as well (F[7,112] = 2.3, p <

0.05). The shape of the AB curve differed in the two presentation

speeds. It is yet unclear what causes the discrepancy between the

results in our own lab and the ones found by Bowman and Wyble

(2007).

The answer to the main question, however, can be seen in the lower

part of Figure 8. Order reversals seem to spread to later lags as

well. As with lag-1 sparing, the important variable seems to be the

temporal distance between T1 and T2, not the number of

intermediate distractors between them. More interestingly and more

surprisingly is the finding that at the same TOA there are more order

errors for faster presentation speeds. This means that there are more

order reversals, even though the targets are delineated by more

intermediate distractors. Holm-Bonferroni corrected t-tests show that

this is true for the whole spectrum in which order errors occur, i.e. up

to a TOA of 400 ms (all t[16] > 2.2, all p < 0.05). As speculated

previously, this might have to do with the effect that T2 backward

masking gets stronger with faster presentation times. However,

please note that we only take trials into account in which both targets

were identified. In the trials that consider, backward masking did not

107

hinder T2 on being identified. It just seems to selectively hinder a

correct order judgment.

108

Target Colors Experiment

In this experiment, I wanted to test the effect of target-color on the

proportion of order reversals.

Method

Participants: Sixteen students from Paderborn University, Germany

with (corrected-to-) normal vision participated for course credits or €6

an hour.

Stimulus, Design, and Procedure was identical to the T2+1 blank

experiment described earlier except for the following changes: In half

of the trials both target items were colored red, whereas they stayed

black in the remaining half. T1 and T2 always followed each other in

lag 1. Each condition was repeated 50 times, mixed into a single

session lasting less than twenty minutes.

Results and Discussion

T1 accuracy, conditional T2 accuracy and proportion of order errors

can be seen in Figure 9, separately for trials in which both targets

were black and for trials in which both targets were red.

Unsurprisingly, T1 accuracy improved when T1 and T2 differed in

color from the surrounding distractors (t[15] = 4.9, p < 0.05). T2|T1

performance did not improve significantly. This might be due to a

ceiling effect (t < 1). The proportion of order errors was also strongly

influenced by the color manipulation: when both targets were

colored, there were significantly less order reversals (t[15] = 3.8, p <

109

0.05). As discussed in the main text, this might be due to different

backward masking conditions of T2.

110

Single vs. Dual Stream Experiment

In this experiment I wanted to test the influence of distractor

presence and target location on the proportion of order reversals.

These two factors as well as task-set size are the main differences

between the TOJ paradigm and the AB paradigm. Since we already

rejected task-set size as the source for the different time courses of

order reversals in these two paradigms, I figured that one of the other

factors (or at least their interaction) should have a major influence on

order reversals.

Method

Participants: Participants were students from Paderborn University,

Germany. As evidenced by a simple visual test, all had normal or

corrected-to-normal vision and were paid 6€ / hour for participation.

Twenty participants took part in this experiment.

Apparatus: The experiment took place in a dimly lit room. The

participants sat at a distance of 57 cm – set by a chin rest – from a

19’’ CRT screen. The centre of the monitor was at eye level and its

resolution set to 800 x 600 pixels at 60 Hz. The experimental

program was written in MATLAB 7.7.0 including the PsychToolbox

(Brainard, 1997). The observers responded by pressing keys on the

keypad.

Stimuli: Stimuli were black on a medium grey background. The digits

1 to 9 provided the target set. Distractors were chosen from the

letters of the Roman alphabet (except for I, O, Q, B, S). All stimuli

111

subtended approximately 1° in visual angle. Each stimulus was

presented for 66 ms and immediately followed by the next item. This

yielded a presentation rate of 15 items/sec. The presentation rate

therefore is in between the speed of a typical attentional blink which

is about 10 items/sec and a standard TOJ paradigm, which usually

presents stimuli at about 33 ms. This was a compromise to gain

enough order errors in the TOJ task (which decrease with increasing

SOA between the two target-stimuli), and also to ensure for the AB

task not being too difficult.

Design: There were four separate blocks in this experiment, each

initiated by a 30 practice trials and a separate instruction. In the

standard AB block, both target digits appeared in a single stream at

the center of the screen among letter-distractors. The two targets

were shown immediately after each other (66 ms target-onset

asynchrony (TOA) / lag 1), with one intervening distractor (TOA 132

ms / lag 2) or with two distractors between them (TOA 198 ms / lag

3). The AB without distractors block was identical to the standard AB

condition, just without distractors. In the standard TOJ block the two

targets appeared at different locations without any distractors.

Fixation was marked by a “#” sign exactly between the two locations.

In the TOJ with distractors condition, there were two streams of

distractors. T1 was presented in one, T2 in the other stream. The

lags between the two targets were the same in all conditions. The

order of the four blocks was counterbalanced between subjects.

Each lag in each block was repeated 30 times resulting in 360

112

experimental trials in total. Both targets were selected randomly from

the digits 1-9 while never being identical. Distractors – if present –

were also selected randomly with the constrain that no single letter

was presented twice in succession within a trial.

Procedure: Participants initiated each trial by pressing the space bar.

After a delay of about 1000 ms, a fixation cross (a black “#”-sign) was

presented at the center of the screen for approximately another 1000

ms. Each stream began and ended with a # sign. In between these

signs there always were 2 targets as well as either 0 or 4 distractors,

depending on block. After each trial the observers identified the two

targets in order of appearance by pressing the corresponding keys

on the keypad. In case they had not recognized one or both targets,

they were encouraged to guess. The experiment lasted about an

hour and was conducted within a single session.

Results and Discussion

The lower right part of Figure 10 shows the conditional T2|T1

accuracy as a function of lag separately for each condition. As

expected, participants had no difficulties identifying both targets if

they were not embedded in a stream of distractors (“AB without

Distractors” and “TOJ without Distractors”, respectively). The T2|T1

accuracy for the standard AB with distractors also follows the usual

pattern: T2 is spared at lag 1. T2 accuracy then decreases at lag 2

and 3. The T2|T1 accuracy in the “TOJ with Distractors” condition is

lower since T2 is in a different stream and is therefore missed more

often. The “TOJ with Distractors” condition also shows lag-1 sparing,

113

but a weaker one than in the standard AB condition. This finding is in

line with Shih (2008) and Jefferies and colleages (Jefferies,

Ghorashi, Kawahara, & Di Lollo, 2007) who claim that lag-1 sparing

with targets in different streams can be found when observers have

no foreknowledge of T1’s location. The left part of Figure 10 shows

the proportion of order errors separately for each lag and condition. A

three-way repeated measures ANOVA of arc-sine transformed order

errors including Distractor Presence, Task (AB/TOJ) and Lag as

factors found a main effect of Lag (F[2, 38] = 46.6, p < .01), meaning

that order errors decreased with increasing temporal distance

between the targets. The ANOVA also showed a main effect of

distractor presence (F[1, 19] = 113, p < .01). As clear from Figure 10,

participants made much more order errors in conditions in which

distractors were present. The main effect of task was also significant

(F[1, 19] = 39.6, p < .01), indicating that it was more likely to reverse

the order of the two targets when they were shown at different

locations. More importantly, distractor and task did not interact (F <

1), indicating that the presence of distractors influences both tasks in

a similar vain. The interaction of distractor and lag just failed to reach

significance (F[2, 38] = 2.9, p = 0.065), the influence of lag on the

distractor effect is therefore not reliable. The two-way interaction

between task and lag (F[2, 38] = 3.9, p < .05) and the three way

interaction (F[2, 38] = 6.2, p < .01) both showed significant effects.

114

Cueing SOAs on a fine scale

To more precisely determine the peak of prior entry in RSVP we

looked at cueing SOAs between 0 and 100 ms in 10 ms steps. Since

all six subexperiments of Experiment 1 in Hilkenmeier, Scharlau,

Weiß, and Olivers (2011) led to qualitatively the same results, we

only ran one variation, namely the sustained cue of Experiment 1b.

Method

16 participants from Paderborn University took part in this

experiment.

Stimulus, Design, and Procedure: Stimulus generation and response

recording were programmed in C using the Tscope programming

library. After an approximately 1000 ms blank period, a 0.5 x 0.5˚

black fixation cross was presented for another 1000 ms in the center

of the display. It was replaced by a rapid serial visual presentation

(RSVP) of 18 digits and letters, presented in Courier New

(approximately 0.8 x 0.8˚ in size). The letters I, O, Q, S, and Z were

excluded, as was the number 1. Each item was presented ten times

in succession for 10 ms each, without any ISI. The item was then

immediately replaced by the next one, resulting in 10 different

items/sec. Splitting each item into ten pieces allowed us to color each

piece independently. T1 and T2 were letters, whose first halves (50

ms) were always colored, whereas the second halves were black

again. The two targets, following each other in lag 1, were embedded

in a stream of black distractor letters and digits. T1 was placed at

115

position 8-13 in the stream. The distractor preceding T1 was always

a digit. The participant’s task was to report the colored letters at the

end of the trial, unspeeded, and with feedback (for which order errors

were counted as correct).

There were eleven cueing SOAs: The “No Cue” condition in which

only the first halves of the two targets were colored red, a “10 ms”

condition in which the last tenth of the distractor-digit immediately

preceding T1 was colored red, a “20 ms” condition, in which the two

last tenth of the distractor-digit preceding T1 were colored, and so

on. The longest cueing SOA was 100 ms, i.e. the complete

distractor-digit preceding T1 was colored in red. Each of the eleven

different cueing conditions was repeated 40 times and randomly

mixed in a single session. The experiment lasted about 50 minutes.

Results & Discussion

Figure 16: Left: Proportion of order reversals as a function of cueing

SOA. Right: T1 accuracy, T2|T1 accuracy and T1 benefit over T2|T1

as a function of cueing SOA.

50 ms cue

0,5

0,6

0,7

0,8

0,9

p (correct)

T1i

T2|T1

-0,1

-0,05

0,05

0,1

No Cue

100

T1 benefit over T2|T1

cueing SOA in ms

T1 accuracy - T2|T1

accuracy

116

Figure 16 (left) shows the proportion of order reversals as a function

of cueing SOA. A repeated-measures ANOVA with the same factor

revealed a significant effect. (F[10, 150] = 4.2, p < 0.05). Since we

have to correct the pairwise comparisons for at least 10 t-tests, none

of them reached significance when using Holm-Bonferroni correction.

However, uncorrected, the 20 ms, 30 ms, 40 ms, and 50 ms cueing

SOAs all showed a significant reduction in order reversals compared

to the No cue condition (all t[15] > 2.3, all p < 0.05, uncorrected).

Since all of these data-points neighbor each other, we would argue

that the strongest reduction is indeed quite early, although not

statistically reliable. As can be seen in Figure 16, order reversals

monotonically decrease up to the cueing SOA of 50 ms and then

start to increase again. Thus, from a visual inspection of Figure 16,

the peak should be somewhere between the 30 and 50 ms cueing

SOA. T1 accuracy and T1 benefit over T2|T1 somewhat mirror the

time course of order-reversals. However, the strong T1 benefit at

longer cueing SOAs is due to a reduced T2 accuracy, not due to an

increased T1 performance.

In terms of modeling, the results are disillusioning at first. As can be

seen in Figure 17, a distractor like cue (i.e. eliciting facilitation and

inhibition with 50 ms offset) predicts the by far strongest cueing effect

for the earliest cueing SOAs of 10 and 20 ms.

117

Figure 17: estimated time before T2 overtakes T1 as a function of

cueing SOA. Blue: the cue elicits both a facilitatory and an inhibitory

effect. Red: cues < 50 ms trigger only facilitation.

But why should we assume that such a shortly presented cue elicits

inhibition at all? As already described, the system starts inhibiting as

soon as it realizes that it is dealing with a colored distractor and not

with a real target. This process is assumed to take about 50 ms.

Thus, the start of the inhibition is 50 ms after distractor-cue onset. In

turn, I would argue that cues < 50 ms do not trigger any inhibition.

Before the system realizes that it was tricked by a colored distractor,

this distractor is overwritten by a real target. Thus, the system has no

reason to start inhibition. The red bars of Figure 17 show the

predicted time course of facilitation when cues < 50 ms only trigger

facilitation. The estimated time course nearly perfectly matches the

empirical one (Figure 16). In both the simulated and the empirical

data, the strongest facilitation is at 50 ms. Moreover, in both cases

118

the increase in facilitation for the first 50 ms is less steep than the

decrease for cueing SOAs > 50 ms.

119

Monoptic/dichoptic blink

The original purpose of this experiment was to explore the influences

of early masking effects that occur before binocular integration (e.g.

Lumer, 1998) on the shape of the attentional blink.

Method

23 students from Paderborn University took part in this experiment.

Stimulus, Design, and Procedure:

The experiment was run on a 120 Hz TFT monitor. Participants wore

active shutter glasses (synchronized with the refresh rate of the

monitor) that opened and closed 60 times per second. Thus, a single

frame was presented to only one eye for 8.3 ms. For this time the

glasses of the other eye were closed. This way, we could present

different visual information to each eye. The RSVP stream consisted

of 10 items / sec. Each item was only presented to one eye. When a

stimulus was presented to the left eye, it appeared for 8.3 ms while

the left glass of the shutter glasses was opened. When the left glass

closed and the right glass opened, the stimulus disappeared for 8.3

ms. All that was presented to the right eye was the background color

(grey). When the right glass closed and the left glass opened again,

the stimulus was again presented, so that the left eye could see it.

Thus, each stimulus was presented for 6 x 8.3 ms (with 8.3 ms

between each repetition).

120

Figure 18: schematic representation of a single item presented in the

“monoptic/dichoptic blink” experiment

The experiment consisted of three conditions: a) all stimuli were

presented to one eye, b) the stimuli were presented alternating to

each eye, but both targets were always presented to the same eye,

c) the stimuli were presented alternating to each eye, but the targets

were always presented to different eyes.

In each condition, T2 could appear in lag 1, 2, 3, or 6. Each lag in

each condition was repeated 30 times. The participant’s task was to

report the two taget-letters at the end of the trial.

Results and Discussion

T1 accuracy, conditional T2 accuracy and proportion of order errors

as a function of lag can be seen in Figure 15, separately for the

baseline condition (all stimuli presented to one eye) and the two

121

experimental conditions (stimuli alternating between the eyes, targets

either both presented to the same eye or to different eyes).

To better understand the different masking conditions utilized in each

lag, Figure 19 shows a schematic representation of these trials. A

repeated measures one way analysis of variance showed a

significant main effect of condition on T1 accuracy (F[2,66]= 8.6, p <

0.001), a significant main effect of lag (F[3,66]= 26.4, p < 0.001), and

a significant interaction of condition with lag (F[6,66]= 4.1, p < 0.001).

The same analysis on T2|T1 accuracy revealed significant main

effects for condition (F[2,66]= 14.4, p < 0.001)and lag (F[3,66]= 17.1,

p < 0.001), as well as a significant interaction (F[6,66]= 6.0, p <

0.001). For order reversals, the main effect of condition showed no

significant effect (F<1) whereas lag as well as the interaction of lag

and condition were again significant (F[3,66]= 169, p < 0.001 and

F[6,66]= 6.5, p < 0.001, respectively). However, more important than

the mere analyses of variance between the conditions are planned t-

tests between two conditions at a given lag.

122

Figure 19: schematic representation of lag 1, 2, and 3 of the

“alternating stimuli, targets presented to the same eye” (left) and the

“alternating stimuli, targets presented to different eyes” (right)

conditions. The baseline condition in which all stimuli are presented

to one eye is not shown.

123

As can be seen in Figure 19, T1 performance at lag 1 is worse when

T2 is presented to the same eye compared to T2 being presented to

a different eye (t[22] = 4.2, p < 0.001 and t[22] = 3.5, p < 0.01 for all

stimuli presented to one eye and stimuli alternating but targets in one

eye, respectively). The reversed pattern is found for the conditional

T2 accuracy. Here, performance decreased when T2 was presented

at a different eye than T1 (t[22] = 6.4, p <0.001 and t[22] = 2.5, p <

0.05, respectively). These findings, in combination with the result that

there are more order reversals when the two targets are presented to

the same eye (t[22] = 4.3, p < 0.001 for alternating stimuli, targets in

the same eye) strongly suggests that interdependence between the

two targets is much stronger when T1 and T2 are presented to the

same eye. As already discussed in the main text, these findings were

not necessarily to be expected since the fovea (with which the

targets are most likely fixated) has projections in both hemifields.

However, as Bachmann (2007, personal communication) pointed out,

thalamo-cortical as well as cortico-thalamic retouch effects are most

likely laterally biased to the eye of the input, suggesting that

increased T2 accuracy and increased proportion of order reversals

are indeed due to a facilitatory modulation elicited by T1.

However, we also found evidence for the a priory assumed influence

of early masking: at lags 2 and 3, when the blink is most pronounced,

T2 accuracy was stronger impaired when masking for T2 (and

especially backward masking, as argued earlier in this manuscript)

was stronger: At lag 2, in the “alternating stimuli, targets in the same

124

eye” condition, the distractors immediately preceding and trailing T2

were presented to the other eye. Thus, masking was weak and T2

performance high. In the “alternating stimuli, targets in different eyes”

condition, the stimulus preceding T2 was presented to the other eye,

but the stimulus trailing T2 was presented to the same eye as T2.

Thus, forward masking of T2 was weak, whereas backward masking

was strong. Nevertheless, T2 performance in this condition was as

impaired as in the baseline condition in which all stimuli were

presented to one eye, i.e. forward and backward masking were

strong (t < 1; comparisons to the weak-masking “alternating stimuli,

targets in different eyes” condition: t[22] = 3.9, p < 0.001 and t[22] =

5.5, p < 0.001, respectively), again indicating that first and foremost

the distractor trailing T2 has a strong influence on the shape of the

blink. At lag 3 the picture is qualitatively the same: in both alternating

targets conditions the distractors immediately preceding and trailing

T2 are presented to the different eye. Thus masking is low and T2

performance in both conditions is relatively good (t[22] = 2.3,

nonsignificant after correction). When all stimuli are presented to the

same eye and thus masking is stronger, T2 performance

consequently decreases (t[22] = 4.8, p < 0.001 and t[22] = 3.0, p <

0.01, respectively).

125

126

References

Akyürek, E.G., Abedian-Amiri, A., & Ostermeier, S.M. (2011).

Content-Specific Working Memory Modulation of the

Attentional Blink. PloS one, 6, e16696.

Akyürek, E. G., & Hommel, B. (2005a). Short-term memory and the

attentional blink: Capacity versus content. Memory &

Cognition, 33, 654-663.

Akyürek, E. G., & Hommel, B. (2005b). Target integration and the

attentional blink. Acta Psychologica, 119, 305-314.

Akyürek E. G., Riddell P. M., Toffanin P., Hommel B. (2007).

Adaptive Control of event integration: evidence from event-

related potentials. Psychophysiology, 44, 383 - 391.

Akyürek, E. G., Toffanin, P., & Hommel, B. (2008). Adaptive control

of event integration. Journal of Experimental Psychology:

Human Perception and Performance, 34, 569–577.

Ansorge, U., Horstmann, G. & Scharlau, I. (2010). Top-down

contingent attentional capture during feed-forward visual

processing – A reply to Theeuwes. Acta Psychologica, 135,

123-126.

Awh, E., & Vogel, E. K. (2008). The bouncer in the brain. Nature

Neuroscience, 11, 5-6.

127

Bachmann, T. (1984). The process of perceptual retouch:

Nonspecific afferent activation dynamics in explaining visual

masking. Attention, Perception, & Psychophysics, 35, 69-84.

Bachmann, T. (1989). Microgenesis as traced by the transient

paired-forms paradigm. Acta Psychologica, 70, 3–17.

Bachmann, T. (1999). Twelve spatiotemporal phenomena, and one

explanation. In G. Aschersleben, T. Bachmann, & J. Müsseler

(Eds.), Cognitive contributions to the perception of spatial and

temporal events. (pp. 173-206). Amsterdam: Elsevier / North-

Holland. (Advances in Psychology, # 129).

Bachmann, T. (2010). Priming and retouch in flash-lag and other

phenomena of the streaming perceptual input. Chapter 30,

(536-557.) In R. Nijhawan, B. Khurana (Eds.), Space and Time

in Perception and Action. Cambridge: Cambridge University

Press.

Bachmann, T., & Hommuk, K. (2005) How Backward Masking

becomes Attentional Blink: Perception of Successive In-

Stream Targets. Psychological Science, 16, 740-742.

Bachmann, T., Luiga, I., Põder, E., & Kalev, K. (2003). Perceptual

acceleration of objects in stream: Evidence from flash-lag

displays. Consciousness and Cognition, 12, 279-297.

Bacon, W.F., & Egeth, H.E. (1994). Overriding stimulus-driven

attentional capture. Attention, Perception, & Psychophysics,

55, 485-496.

128

Belopolsky, A. V., Schreij, D., & Theeuwes, J. (2010). What is top-

down about contingent capture? . Attention, Perception, &

Psychophysics, 72, 326-341.

Bonnel, A.M., & Prinzmetal, W. (1998). Dividing attention between

the color and the shape of objects. Attention, Perception, &

Psychophysics, 60, 113-124.

Bowman, H., Wyble, B. (2007). The simultaneous type, serial token

model of temporal attention and working memory.

Psychological Review, 114, 38-70.

Brainard, D.H. (1997). The psychophysics toolbox. Spatial vision, 10,

433-436.

Breitmeyer, B. G., & Öğmen, H. (2006). The First Half Second: The

Microgenesis and Temporal Dynamics of Unconscious and

Conscious Visual Processes. Cambridge, MA: MIT Press.

Breitmeyer, B. G., & Öğmen, H. (2007). Visual Masking.

Scholarpedia, 2, 3330. Scholarpedia, 3, 1538. doi:10.4249/

scholarpedia.1538

Broadbent, D.E. (1958). Perception and communication. Oxford

University Press; Elmsford, NY. doi: 10.1037/10037-000

Broadbent, D.E., & Broadbent, M.H.P. (1987). From detection to

identification: Response to multiple targets in rapid serial

visual presentation. Attention, Perception, & Psychophysics,

42, 105-113.

129

Bruce, V., Green, P. and Georgeson, M. (1996). Visual Perception:

Physiology, Psychology, and Ecology. Psychology Press.

Bundesen, C. (1990). A theory of visual attention. Psychological

review, 97, 523-547.

Cheal, M.L., & Lyon, D.R. (1991). Central and peripheral precuing of

forced-choice discrimination. The Quarterly Journal of

Experimental Psychology, 43, 859-880.

Cherry, E.C. (1953). Some experiments on the recognition of speech,

with one and with two ears. Journal of the acoustical society of

America, 25, 975-979.

Chun, M. M., & Potter, M. C. (1995). A two-stage model for multiple

target detection in rapid serial visual presentation. Journal of

Experimental Psychology: Human Perception and

Performance, 21, 109-127.

Chun, M. M., Wolfe, J. M. (2001). Visual attention. In: Goldstein, E.B.

(ed). Blackwell Handbook of Sensation and Perception. Wiley-

Blackwell, 272-310.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral

Sciences (second ed.). Lawrence Erlbaum Associates.

Coltheart, M. (1980). Iconic memory and visible persistence.

Perception & Psychophysics, 27, 183-228.

130

Craston P, Wyble B, Chennu S, & Bowman H. (2008). The

attentional blink reveals serial working memory encoding:

Evidence from virtual & human event-related potentials.

Journal of Cognitive Neuroscience, 21, 550-566.

Crick, F. (1984). Function of the thalamic reticular complex: the

searchlight hypothesis. Proceedings of the National Academy

of Sciences of the United States of America, 81, 4586-4590.

Dell’Acqua, R., Jolicœur, P., Luria, R., & Pluchino, P. (2009). Re-

evaluating encoding-capacity limitations as a cause of the

attentional blink. Journal of Experimental Psychology: Human

Perception & Performance, 35, 338-351.

Donk, M., & van Zoest, W. (2008). Effects of Salience Are Short-

Lived. Psychological Science, 19, 733 -739.

Duncan, J. (1980). The demonstration of capacity limitation.

Cognitive Psychology, 12, 75–96.

Duncan, J., & Humphreys, G.W. (1989). Visual search and stimulus

similarity. Psychological review, 96, 433-458.

Duncan, J., Martens, S., & Ward, R. (1997). Restricted attentional

capacity within but not between sensory modalities. Nature,

387, 808-810.

Duncan, J., Ward, R., & Shapiro, K. (1994). Direct measurement of

attentional dwell time in human vision. Nature, 369, 313-315.

131

Dux, P. E., Asplund, C. L., & Marois, R. (2008). An attentional blink

for sequentially presented targets: Evidence in favor of

resource depletion accounts. Psychonomic Bulletin & Review,

15, 809-813.

Dux, P. E., Asplund, C. L., & Marois, R. (2009). Both exogenous and

endogenous target salience manipulations support resource

depletion accounts of the attentional blink: A reply to Olivers,

Spalek, Kawahara, and Di Lollo (2009). Psychonomic Bulletin

& Review, 16, 219-224.

Dux, P. E., & Harris, I. M. (2007a). On the failure of distractor

inhibition in the attentional blink. Psychonomic Bulletin &

Review, 14, 723–728.

Dux, P.E., & Marois, R. (2009). The attentional blink: A review of data

and theory. Attention, Perception, & Psychophysics, 71, 1683-

1700.

Erikson, C. W., & St. James, J. D. (1986). Visual attention within

and around the field of focal attention: a zoom lens model.

Perception & Psychophysics, 40, 225 – 240.

Evans, K. K., & Treisman, A. (2005). Perception of Objects in Natural

Scenes: Is It Really Attention Free? Journal of Experimental

Psychology: Human Perception and Performance, 31, 1476-

1492.

132

Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical

power analyses using G*Power 3.1: Tests for correlation and

regression analyses. Behavior Research Methods, 41, 1149-

1160.

Folk, C.L., Remington, R.W., & Johnston, J.C. (1992). Involuntary

covert orienting is contingent on attentional control settings.

Journal of experimental psychology: Human perception and

performance, 18, 1030-1044.

Folk, C.L., Remington, R.W., & Wright, J.H. (1994). The structure of

attentional control: Contingent attentional capture by apparent

motion, abrupt onset, and color. Journal of experimental

psychology: Human perception and performance, 20, 317-329.

Folk, C.L., Remington, R.W., & Wu, S.C. (2009). Additivity of abrupt

onset effects supports nonspatial distraction, not the capture

of spatial attention. Attention, Perception, & Psychophysics,

71, 308-313.

Giesbrecht, B., & Di Lollo, V. (1998). Beyond the attentional blink:

Visual masking by object substitution. Journal of Experimental

Psychology: Human Perception and Performance, 24, 1454-

1466.

Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2006). Banishing the

homunculus: making working memory work. Neuroscience,

139, 105-118.

133

Helmholtz, H. (1867). Handbuch der physiologischen Optik. Voss.

Fulltext online available at http://books.google.de/books?id=

Ih85AAAAcAAJ&printsec=frontcover&source=gbs_ge_summa

ry_r&cad=0#v=onepage&q&f=false

Hikosaka, O., Miyauchi, S., Shimojo, S. (1993a). Focal visual

attention produces illusory temporal order and motion

sensation. Vision Research, 33, 1219-1240.

Hikosaka, O., Miyauchi, S., Shimojo, S. (1993b). Voluntary and

stimulus-induced attention detected as motion sensation.

Perception, 22, 517-517.

Hikosaka, O., Miyauchi, S., Shimojo, S. (1993 c). Visual attention

revealed by an illusion of motion. Neuroscience Research, 18,

11-18.

Hilkenmeier, F., Olivers, C. N. L., & Scharlau, I. (2011). Prior entry

and temporal attention: Cueing affects order errors in RSVP.

In press at Journal of Experimental Psychology: Human

Perception and Performance.

Hilkenmeier, F., & Scharlau, I. (2010). Rapid allocation of temporal

attention in the attentional blink paradigm. European Journal

of Cognitive Psychology, 22, 1222-1234.

Hilkenmeier, F., & Scharlau, I., Weiß, K., & Olivers, C. N. L. (2011).

The dynamics of prior entry in serial visual processing. in

press at Visual Cognition.

134

Hilkenmeier, F., Tünnermann, J., Scharlau, I. (2009). Early Top-

Down Influences in Control of Attention: Evidence from the

Attentional Blink. KI 2009: Advances in Artificial Intelligence.

2009, Volume 5803/2009, 680-686.

Hillstrom, A.P., Shapiro, K.L., & Spence, C. (2002). Attentional

limitations in processing sequentially presented vibrotactile

targets. Attention, Perception, & Psychophysics, 64, 1068-

1082.

Hommel, B., & Akyürek, E. G. (2005). Lag-1 sparing in the attentional

blink: Benefits and costs of integrating two events into a single

episode. Quarterly Journal of Experimental Psychology, 58,

1415-1433.

Hommel, B., Kessler, K., Schmitz, F., Gross, J., Akyürek, E.,

Shapiro, K., & Schnitzler, A. (2006). How the brain blinks:

Towards a neurocognitive model of the attentional blink.

Psychological Research, 70, 425-435.

Isaak, Matthew I.; Shapiro, Kimron L.; & Martin, J. (1999). The

attentional blink reflects retrieval competition among multiple

rapid serial visual presentation items: Tests of an interference

model. Journal of Experimental Psychology: Human

Perception and Performance, 25, 1774-1792.

135

James, W. (1890) The principles of psychology, Vol I. Henry Holt and

Co. Fulltext online available at http://psychclassics.

yorku.ca/James/Principles/

Jefferies, L.N., Ghorashi, S., Kawahara, J.I., & Di Lollo, V. (2007).

Ignorance is bliss: The role of observer expectation in dynamic

spatial tuning of the attentional focus. Attention, Perception, &

Psychophysics, 69, 1162-1174.

Jolicœur, P., & Dell'Acqua, R (1998). The demonstration of short-

term consolidation. Cognitive Psychology, 36, 138–202.

Jolicœur, P., Tombu, M., Oriet, C., & Stevanovski, B. (2002). From

perception to action: Making the connection. In W. Prinz & B.

Hommel (Eds.), Attention and performance XIX: Common

mechanisms in per- ception and action (pp. 558–586). Oxford,

England: Oxford University Press.

Jonides, J. (1981). Voluntary versus automatic control over the

mind’s eye’s movement. Attention and performance IX, 9, 187-

203.

Kahneman, D., & Treisman, A., 1984. Changing views of attention

and automaticity. In R. Parasuraman & R. Davies (Eds.)

Varieties of Attention. New York: Academic Press, 29- 61.

Kawahara, J. I., Enns, J. T., Di Lollo, V. (2006). The attentional blink

is not a unitary phenomenon. Psychological Research, 70,

405-413.

136

Kawahara, J. I., Kumada, T., & Di Lollo, V. (2006). The attentional

blink is governed by a temporary loss of control. Psychonomic

Bulletin & Review, 13, 886-890.

Keysers, C. and Perrett, D.I. (2002) Visual masking and RSVP reveal

neural competition. Trends in Cognitive Sciences, 6, 120–125.

Kristjansson, A., Mackeben, M., & Nakayama, K. (2001). Rapid,

object-based learning in the deployment of transient attention.

Perception, 30, 1375-1387.

Kristjansson, A., & Nakayama, K. (2003). A primitive memory system

for the deployment of transient attention. Perception and

Psychophysics, 65, 711-724.

Kröse, B.J.A., & Julesz, B. (1989). The control and speed of shifts of

attention. Vision Research, 29, 1607-1619.

Kwak, H., Dagenbach, D., & Egeth, H. (1991). Further evidence for a

time-independent shift of the focus of attention. Attention,

Perception, & Psychophysics, 49, 473-480.

LaBerge, D. (1983). Spatial extent of attention to letters and words.

Journal of Experimental Psychology: Human Perception and

Performance, 9, :371-379.

LaBerge, D., Brown, V. (1986). Variations in size of the visual field in

which targets are presented: An attentional range effect.

Attention, Perception, & Psychophysics, 40, 188-200.

137

Lakatos, I. (1978). Philosophical papers volume I: The methodology

of scientific research programmes. Cambridge University

Press, Cambridge.

Lawrence, D.H. (1971). Two studies of visual search for word targets

with controlled rates of presentation. Attention, Perception, &

Psychophysics, 10, 85-89.

Luck, S.J., Vogel, E.K., & Shapiro, K.L. (1996). Word meanings can

be accessed but not reported during the attentional blink.

Nature, 383, 616-618.

Lumer, E.D. (1998). A neural model of binocular integration and

rivalry based on the coordination of action-potential timing in

primary visual cortex. Cerebral Cortex, 8, 553-561.

Maki, W.S., Bussard, G., Lopez, K., & Digby, B. (2003). Sources of

interference in the attentional blink: Target-distractor similarity

revisited. Perception & Psychophysics, 65, 188-201.

Martens, S., & Wyble, B. (2010). The attentional blink: Past, present,

and future of a blind spot in perceptual awareness.

Neuroscience & Biobehavioral Reviews, 34, 947-957.

Marois, R., & Ivanoff, J. (2005). Capacity limits of information

processing in the brain. Trends in Cognitive Sciences, 9, 296-

305.

138

McAlonan, K., Cavanaugh, J., & Wurtz, R.H. (2008). Guarding the

gateway to cortex: attention in visual thalamus. Nature, 456,

391-394.

McAlonan, K., Cavanaugh, J., & Wurtz, R.H. (2006). Attentional

Modulation of Thalamic Reticular Neurons. Journal of

Neuroscience, 26, 4444-4450.

Miller, E.K., & Cohen, J.D. (2001). An integrative theory of prefrontal

cortex function. Annual review of neuroscience, 24, 167-202.

Miller, E.K., Erickson, C.A., & Desimone, R. (1996). Neural

mechanisms of visual working memory in prefrontal cortex of

the macaque. The Journal of Neuroscience, 16, 5154-5167.

Moray, N. (1959). Attention in dichotic listening: Affective cues and

the influence of instructions. The Quarterly Journal of

Experimental Psychology, 11, 56-60.

Müller, H.J., & Rabbitt, P.M. (1989). Reflexive and voluntary orienting

of visual attention: Time course of activation and resistance to

interruption. Journal of Experimental Psychology: Human

Perception and Performance, 15, 315-30.

Nakayama, K., & Mackeben, M. (1989). Sustained and transient

components of focal visual attention. Vision Research, 29,

1631-1647.

Neisser, U., Becklen, R. (1976). Selective looking: Attending to

visually specified events. Cognitive Psychology, 7, 480-494.

139

Neumann, O., & Scharlau, I. (2007a). Experiments on the Fehrer-

Raab effect and the ‘Weather Station Model’ of visual

backward masking. Psychological Research, 71, 667-677.

Neumann, O., & Scharlau, I. (2007b). Visual attention and the

mechanism of metacontrast. Psychological Research, 71, 626-

633.

Nieuwenhuis, S., Gilzenrat, M. S., Holmes, B. D., & Cohen, J. D.

(2005). The Role of the Locus Coeruleus in Mediating the

Attentional Blink: A Neurocomputational Theory. Journal of

Experimental Psychology: General, 134, 291-307.

Nieuwenstein, M. R. (2006). Top-down controlled, delayed selection

in the attentional blink. Journal of Experimental Psychology:

Human Perception & Performance, 32, 973-985.

Nieuwenstein, M. R., Chun, M. M., van der Lubbe, R. H. J., & Hooge,

I. T. C. (2005). Delayed Attentional Engagement in the

Attentional Blink. Journal of Experimental Psychology: Human

Perception and Performance, 31, 1463-1475.

Nieuwenstein, M.R., & Potter, M.C. (2006). Temporal limits of

selection and memory encoding. Psychological Science, 17,

471-475.

Nieuwenstein, M. R., Potter, M. C., & Theeuwes, J. (2009).

Unmasking the attentional blink. Journal of Experimental

140

Psychology: Human Perception and Performance, 35,

159-169.

Nolte, J (2002). The Human Brain: An Introduction to Its Functional

Anatomy . Mosby

O'Connor, D. H., Fukui, M. M., Pinsk, M. A., & Kastner, S. (2002).

Attention modulates responses in the human lateral geniculate

nucleus. Nature Neuroscience, 5, 1203-1209.

Olivers, C. N. L., Hilkenmeier, F., & Scharlau, I. (2010). Prior entry

explains order reversals in the attentional blink. Attention,

Perception, & Psychophysics, 73, 53-67.

Olivers, C. N. L., & Meeter, M. (2008). A boost and bounce theory of

temporal attention. Psychological Review, 115, 836-863.

Olivers, C. N. L., & Nieuwenhuis, S. (2005). The beneficial effect of

concurrent task-irrelevant mental activity on temporal

attention. Psychological Science, 16, 265-269.

Olivers, C. N. L., & Nieuwenhuis, S. (2006). The beneficial effects of

additional task load, positive affect, and instruction on

the attentional blink. Journal of Experimental Psychology:

Human Perception and Performance, 32, 364-379.

Olivers, C. N. L., Spalek, T. M., Kawahara, J.-I., & Di Lollo, V. (2009).

The attentional blink: Increasing target salience provides no

evidence for resource depletion. A commentary on Dux,

141

Asplund, and Marois (2008). Psychonomic Bulletin & Review,

16, 214- 218.

Olivers, C. N., van der Stigchel, S., & Hulleman, J. (2007). Spreading

the sparing: against a limited-capacity account of the

attentional blink. Psychological Research, 71, 126-139.

Pashler, H. (Ed.) (2004). Attention. Philadelphia: Taylor & Francis

Press.

Popper, K. R. (1945). The open society and its enemies II: the high

tide of prophecy: Hegel & Marx. Routledge & Kegan Paul Ltd.,

London.

Posner, M.I. (1978). Chronometric explorations of mind. Lawrence

Erlbaum, Hillsdale, N.J.

Posner, M.I. (1980). Orienting of attention. The Quarterly journal of

experimental psychology, 32, 3-25.

Posner, M., & Petersen, S.E. (1990). The attention system of the

brain. Annual Review of Neuroscience, 13, 25-42.

Posner, M.I., Snyder, C.R., & Davidson, B.J. (1980). Attention and

the detection of signals. Journal of Experimental Psychology:

General, 109, 160-174.

Potter, M. C. (2006). Competition for attention in space and time: The

first 200 ms. In H. Öğmen & B. G. Breitmeyer (Eds.), The first

half second: The microgenesis and temporal dynamics of

142

unconscious and conscious visual processes. Cambridge, MA:

MIT Press.

Potter, M. C., Chun, M. M., Banks, B. S., & Muckenhoupt, M. (1998).

Two attentional deficits in serial target search: The visual

attentional blink and an amodal task-switch deficit. Journal of

Experimental Psychology: Learning, Memory, and Cognition,

24, 979-992.

Potter, M.C., & Levy, E.I. (1969). Recognition memory for a rapid

sequence of pictures. Journal of Experimental Psychology, 81,

10-15.

Potter, M. C., Staub, A., & O'Connor, D. H. (2002). The time course

of competition for attention: Attention is initially labile.

Journal of Experimental Psychology: Human Perception

and Performance, 28, 1149-1162.

Prinzmetal, W., McCool, C., & Park, S. (2005). Attention: reaction

time and accuracy reveal different mechanisms. Journal of

Experimental Psychology: General, 134, 73- 92.

Priess, H.-W., & Scharlau, I. (2009). What You See Is What You

Set – The Position of Moving Objects. In: B. Mertsching

(Ed.). KI 2009: Advances in Artificial Intelligence.

Proceedings of the 32nd Annual Conference on Artificial

Intelligence.

Raymond, J.E., Shapiro, K.L., & Arnell, K.M. (1992). Temporary

suppression of visual processing in an RSVP task: an

143

attentional blink? Journal of Experimental Psychology: Human

Perception and Performance, 18, 849-860.

Reeves, A., & Sperling, G. (1986). Attention gating in short-term

visual memory. Psychological Review, 93, 180-206.

Remington, R.W., Johnston, J.C., & Yantis, S. (1992). Involuntary

attentional capture by abrupt onsets. Attention, Perception, &

Psychophysics, 51, 279-290.

Remington, R., & Pierce, L. (1984). Moving attention: Evidence for

time-invariant shifts of visual selective attention. Attention,

Perception, & Psychophysics, 35, 393-399.

Scharlau, I. (2002). Leading, but not trailing, primes influence

temporal order perception: Further evidence for an attentional

account of perceptual latency priming. Perception &

Psychophysics, 64, 1346-1360.

Scharlau, I. (2007). Perceptual latency priming: A measure of

attentional facilitation. Psychological Research, 71, 678-686.

Scharlau, I., Ansorge, U., & Horstmann, G. (2006). Latency

facilitation in temporal-order judgments: Time course of

facilitation as a function of judgment type. Acta Psychologica,

122, 129-159.

Scharlau, I., Horstmann, G. (2006). Perceptual latency priming and

illusory line motion: Facilitation by gradients of attention?

Advances in Cognitive Psychology, 2, 87-97.

144

Scharlau, I., & Neumann, O. (2003a). Perceptual latency priming by

masked and unmasked stimuli: Evidence for an attentional

interpretation. Psychological Research, 67, 184-196.

Scharlau, I., & Neumann, O. (2003b). Temporal parameters and time

course of perceptual latency priming. Acta Psychologica, 113,

185-203.

Schreij, D., Owens, C., & Theeuwes, J. (2008). Abrupt onsets

capture attention independent of top-down control settings.

Attention, Perception, & Psychophysics, 70, 208-218.

Schreij, D., Theeuwes, J., & Olivers, C.N.L. (2010a). Abrupt onsets

capture attention independent of top-down control settings II:

Additivity is no evidence for filtering. Attention, Perception, &

Psychophysics, 72, 672-682.

Schreij, D., Theeuwes, J., & Olivers, C.N.L. (2010b). Irrelevant

onsets cause inhibition of return regardless of attentional set.

Attention, Perception, & Psychophysics, 72, 1725-1729.

Sessa, P., Luria, R., Verleger, R., & Dell'Acqua, R. (2006). P3 latency

shifts in the attentional blink: Further evidence for second

target processing postponement. Brain Research, 1137, 131-

139.

Sherman, S. M. (2006). Thalamus. Scholarpedia, 1, 1583.

doi:10.4249/scholarpedia.1583

145

Shih, S. I. (2008). The attention cascade model and the attentional

blink. Cognitive Psychology, 56, 210-236.

Shih, S. I. (2009). Using the attention cascade model to probe

cognitive aging. Psychology and Aging, 24, 550 – 562.

Shih, S., & Reeves, A. (2007). Attentional capture in rapid serial

visual presentation. Spatial Vision, 20, 301–315.

Shimojo, S., Hikosaka, O., Miyauchi, S. (1999). Automatic and

controlled attention detected by the line motion effect. In:

Gopher, D., Koriat, A. (eds). Attention and performance XVII:

cognitive regulation of performance: interaction of theory and

application. MIT Press, Cambridge; 1999. 145-163.

Simons, D.J. (2007). Inattentional blindness. Scholarpedia, 2, 3244.

doi:10.4249/scholarpedia.3244

Simons, D.J., Chabris, C.F. (1999). Gorillas in our midst: Sustained

inattentional blindness for dynamic events. Perception, 28,

1059-1074.

Spence, C., Baddeley, R., Zampini, M., James, R., & Shore, D. I.

(2003). Multisensory temporal order judgments: When two

locations are better than one. Perception & Psychophysics,

65, 318-328.

Sperling, G. (1960). The information available in brief visual

presentations. Psychological Monographs, 74, 1-29.

146

Stelmach, L. B., & Herdman, C. M. (1991). Directed attention and

perception of temporal order, Journal of Experimental

Psychology: Human Perception and Performance, 17, 539–

550.

Sternberg, S. and Knoll, R. L. (1973). The perception of temporal

order: fundamental issues and a general model. In: Attention

and Performance. Edited by S. Kornblum. New York:

Academic Press, No. 4, pp. 629-685.

Taatgen, N. A., Juvina, I., Schipper, M., Borst, J.P., Martens, S.

(2009). Too much control can hurt: A threaded cognition

model of the attentional blink. Cognitive Psychology, 59, 1-29.

Titchener, E.B. (1908). Lectures on the Elementary Psychology of

Feeling and Attention. New York, The Macmillan Co. Fulltext

online availabile at http://www.archive.org/details/elemen

tarypsycho00titc

Treisman, A.M. (1960). Contextual cues in selective listening. The

Quarterly Journal of Experimental Psychology, 1960, 12, 242-

248.

Treisman, A.M. (1964). Verbal cues, language, and meaning in

selective attention. The American Journal of Psychology, 77,

206-219.

Treisman, A.M., & Gelade, G. A.(1980). Feature-integration theory of

attention. Cognitive psychology, 12, 97-136.

147

Trick, L.M., & Enns, J.T. (1998). Lifespan changes in attention: The

visual search task. Cognitive Development, 13, 369--386.

Ulrich, R. (1987). Threshold models of temporal order judgments

evaluated by a ternary response task. Perception &

Psychophysics, 42, 224 – 239.

Van Horn, S. C., Erisir, A., & Sherman, S. M. (2000). Relative

distribution of synapses in the A-laminae of the lateral

geniculate nucleus of the cat. The Journal of Comparative

Neurology, 416, 509-520.

Visser, T., Bischof, W. F., and Di Lollo, V. (1999). Attentional

switching in spatial and non-spatial domains: evidence from

the attentional blink. Psychological Bulletin, 125, 458-469.

Vogel, E., & Luck, S.( 2002). Delayed working memory consolidation

during the attentional blink. Psychonomic Bulletin & Review,

9, 739-743.

Ward, L. M. (2008). Attention. Scholarpedia, 3, 1538.

doi:10.4249/scholarpedia.1538

Ward, R., Duncan, J., & Shapiro, K. (1996). The Slow Time-Course

of Visual Attention. Cognitive Psychology, 30, 79-109.

Weiß, K. & Scharlau, I. (2011) Simultaneity and temporal order

perception: Different sides of the same coin? Evidence from a

visual prior-entry study. Quarterly Journal of experimental

Psychology, 64, 394, 416.

148

Wilkinson L., & The Task Force on Statistical Inference (1999).

Statistical methods in psychology journals. American

Psychologist, 54, 594-604.

Wolfe, J.M., Cave, K.R., & Franzel, S.L. (1989). Guided search: An

alternative to the feature integration model for visual search.

Journal of Experimental Psychology: Human perception and

performance, 15, 419-433.

Wolford, G., & Morrison, F. (1980). Processing of unattended visual

information. Memory & Cognition, 8, 521-527.

Woodman, G.F., & Luck, S.J. (1999). Electrophysiological

measurement of rapid shifts of attention during visual search.

Nature, 400, 867-869.

Wyble B., & Bowman, H. (2005). Computational and experimental

evaluation of the attentional blink: testing the simultaneous

type serial token model. In: Bara BG, Barsalou LW, Bucciarelli

M (eds) CogSci 2005, XXVII Annual Conference of the

Cognitive Science Society. Cognitive Science Society,

Cognitive Science Society through Lawrence Erlbaum, Austin,

pp 2371–2376

Wyble, B., Bowman, H., & Nieuwenstein, M. (2009). The attentional

blink provides episodic distinctiveness: Sparing at a cost.

Journal of Experimental Psychology: Human Perception &

Performance, 35, 787 - 807.

149

Yantis, S. (2004). Control of visual attention. In: Pashler, H. (Ed.).

Attention. Philadelphia: Taylor & Francis Press.

Yantis, S., Johnston, J.C. (1990). On the locus of visual selection:

Evidence from focused attention tasks. Journal of

Experimental Psychology: Human Perception and

Performance, 16, 135 - 149.

Zampini, M., Guest, S., Shore, D., & Spence, C. (2005). Audio-visual

simultaneity judgments. Perception & Psychophysics, 67, 531

- 544.

150

The dynamics of attention in serial visual processing

Zusammenfassung der Dissertation auf Deutsch

vorgelegt von

Dipl.-Psych. Frederic Hilkenmeier

Oktober 2011

Informationen aus unserer Umwelt aufzunehmen, auszuwählen und

gegeneinander abzuwägen sind fundamentale Bestandteile der

menschlichen Wahrnehmung und eine notwenige Voraussetzung um

erfolgreich mit unserer Umgebung interagieren zu können. Obwohl

uns das Ergebnis dieser Prozesse, unsere alltägliche Wahrnehmung,

so mühelos erscheint (wir öffnen einfach unsere Augen, und schon

sehen wir ein scheinbar vollständiges, detailreiches, scharfes, und

farbiges Bild unserer Umwelt), so ist doch ein großer Anteil unseres

Gehirns damit beschäftigt, diesen Eindruck zu erzeugen, den unser

Sinnesapparat (unsere Augen) aufgrund seiner Physiologie gar nicht

liefern kann. Ein entscheidender Aspekt in der Erzeugung unserer

Sinneseindrücke ist, dass momentan wichtige Information von

momentan weniger Information getrennt wird. Dieser Prozess ist

allgemein bekannt als selektive Aufmerksamkeit. Wie und nach

welchen Kriterien Aufmerksamkeit arbeitet, ist eines der

meistbeforschten Themen der Psychologie und reicht zurück bis zu

151

den Anfängen unserer Disziplin zu Hermann von Helmholtz (1867)

und William James (1890). Der Großteil der frühen Arbeiten

beschäftigt sich mit der Frage, welche Reize Aufmerksamkeit an sich

ziehen und wie Aufmerksamkeit im Raum verteilt wird, wenn mehrere

Stimuli gleichsam um Aufmerksamkeit konkurrieren(z.B. Posner,

1980; oder Jonides, 1981). Das Interesse an zeitlichen Aspekten von

Aufmerksamkeit hat erst in den vergangenen 25 Jahren stark

zugenommen. Was passiert, wenn nicht alle Reize gleichzeitig,

sondern nacheinander dargeboten werden? Um die zeitliche

Dynamik von Aufmerksamkeit zu untersuchen, hat sich vor allem das

„schnelle, serielle visuelle Präsentation“-Paradigma (rapid serial

visual presentation; RSVP; Potter & Levy, 1969) etabliert. In diesem

Paradigma wird eine große Anzahl von Reizen (typischerweise

zwischen 15 und 25) sequenziell hintereinander am selben Ort

präsentiert. Jeder Reiz wird für ca. 100 Millisekunden (ms) gezeigt

und dann unmittelbar vom nächsten Stimulus überschrieben. Dieses

Paradigma kann mit einer Vielzahl von Reizklassen verwendet

werden, beispielsweise alphanumerischen Zeichen, Bildern oder

Wörtern, aber auch Tönen oder taktilen Reizen (für einen Überblick,

siehe z.B. Martens & Wyble, 2010). Ein weitverbreitetes

Versuchsdesign ist es, die Versuchsperson zwei zuvor spezifizierte

Zielreize in einer Reihe von Ablenkerreizen (Distraktoren) berichten

zu lassen, beispielsweise zwei farbige Buchstaben in einer Reihe

von schwarzen Zahlen. Während die Versuchsteilnehmer mühelos

den ersten Zielreiz erkennen können, berichten sie oft den zweiten

152

Zielreiz nicht gesehen zu haben, wenn er dem ersten in einem

Abstand von ca. 200 – 600 ms folgt. Dieses Phänomen wird als

„Aufmerksamkeitsblinzeln“ (Attentional Blink; AB; Raymond, Shapiro,

& Arnell, 1992) bezeichnet.

Abbildung 1: Links: Schematischer Ablauf eines Versuchs-

durchgangs. Rechts: Identifikationsleistung für den ersten

(gepunktet) und den zweiten (durchgezogen) Zielreiz in Abhängigkeit

vom zeitlichen Abstand zwischen den beiden Reizen.

Ursprünglich wurde die schlechte Erkennensleistung des zweiten

Zielreizes dadurch erklärt, dass dem kognitiven System nicht

genügend Ressourcen zur Verfügung stünden, um beide Zielreize zu

verarbeiten: Wenn der zweite Zielreiz kurz nach dem ersten kommt,

hat der erste schon nahezu alle Ressourcen verbraucht, der zweite

geht folglich leer aus und kann nicht adäquat verarbeitet werden

(z.B. Ward, Duncan, & Shapiro, 1996; Duncan, Ward, & Shapiro,

1994).

153

Wie allerdings in Abbildung 1 zu sehen ist, ist die Erkennensleistung

des zweiten Zielreizes sehr gut, wenn dieser direkt nach dem ersten

gezeigt wird (das sogenannte „lag-1 sparing“, Potter, Chun,

Muckenhoupt, 1998; Visser, Bischof, & Di Lollo, 1999). Um diesen

reliablen Befund erklären zu können, wurden die Theorien, die

begrenzte kognitive Ressourcen als Ursache für den Attentional Blink

ansehen, um zusätzliche Annahmen erweitert: Die wichtigste

Annahme ist, dass die Verarbeitung in zwei Schritten, oder Stufen

abläuft. Auf einer ersten, kapazitätsfreien Stufe können alle

dargebotenen Reize parallel verarbeitet werden. Damit ein Zielreiz

aber berichtet werden kann, also bewusst wahrgenommen wird,

muss dieser erst in eine zweite Stufe überführt und dort konsolidiert

werden. In dieser zweiten Stufe sind die Ressourcen dann wieder,

wie im vorherigen Modell, stark beschränkt, d.h. die zweite Stufe

kann die eingehenden Informationen nur seriell bearbeiten. Wenn

nun der erste Zielreiz auf der ersten Stufe erkannt wird, öffnet er ein

„Aufmerksamkeitsfenster“ und wird zur weiteren Verarbeitung auf die

zweite Stufe transferiert (z.B. Chun & Potter, 1995; Jolicoeur, Tombu,

Oriet, Stevanovski, 2002; Visser et al, 1999; Akyürek, Riddell,

Toffanin, & Hommel, 2007). Solange der erste Zielreiz auf der

zweiten Stufe verarbeitet wird, muss der zweite Zielreiz auf der

ersten Stufe verharren und ist dort der Gefahr ausgesetzt,

überschrieben oder vergessen zu werden. Doch das Fenster,

welches den ersten Zielreiz auf die zweite Stufe transferiert, schließt

nicht direkt nach dem ersten Zielreiz. Der Stimulus, der direkt nach

154

dem ersten Zielreiz kommt, wird oft ebenfalls mit in die zweite Stufe

transferiert. Falls dies der zweite Zielreiz ist, wird dieser also mit dem

ersten Zielreiz zusammen verarbeitet. Diese gemeinsame

Verarbeitung, bekannt als „episodic integration“ kann den Zeitverlauf

des Attentional Blink, wie in Abbildung 1 dargestellt, ohne große

Schwierigkeiten erklären.

Abbildung 2: Bildliche Darstellung des Attentional Blink (oben) und

des lag-1 sparing (unten) in Modellen, die eine gemeinsame

Verarbeitung annehmen.

Eine Begleiterscheinung des „lag-1 sparing“ ist, dass die Reihenfolge

der beiden berichteten Zielreize von den Versuchsteilnehmern oft

vertauscht wird. Dieser Befund wird oft ebenfalls im Sinne der

gemeinsamen Verarbeitung interpretiert: Wenn beide Zielreize in

155

einer Episode verarbeitet werden, geht die Reihenfolgeinformation

notwendigerweise verloren (Chun & Potter, 1995; Bowman & Wyble,

2007; Hommel & Akyürek, 2005; Akyürek & Hommel, 2005).

Abbildung 3: Anteil der Reihenfolgefehler in Abhängigkeit vom

zeitlichen Abstand zwischen den beiden Reizen.

Dies ist allerdings nicht die einzig mögliche Erklärung: Anstatt

anzunehmen, dass die Reihenfolgeinformation schlicht verloren-

gegangen ist, ist es durchaus möglich, dass Versuchspersonen

einen klaren Reihenfolgeeindruck haben, nur eben oftmals den

falschen (siehe Caldwell-Harris & Morris, 2008). Dies wäre konsistent

mit Theorien, die anstelle einer gemeinsamen Verarbeitung einen

kurzzeitigen Aufmerksamkeitsschub vorhersagen (transient

attentional enhancement; z.B. Reeves & Sperling, 1986; Nakayama

& Mackeben, 1989). Einer der verblüffenderen Effekte von

Aufmerksamkeit ist, dass sie die wahrgenommenen zeitlichen

Eigenschaften der Reize verändern kann. Ein beachteter Reiz kann

156

also als früher wahrgenommen werden, selbst wenn er gleichzeitig

oder sogar etwas später dargeboten wird, als ein gleichartiger, aber

unbeachteter Reiz. Dieses Phänomen des „früheren Eintritts“ (prior

entry; Titchener, 1908) sagt folglich ebenfalls Reihenfolgefehler

voraus, jedoch über einen komplett anderen Mechanismus: Anstatt

davon auszugehen, dass die kognitiven Ressourcen stark limitiert

sind, und der zweite Zielreiz nur zufällig und unter dem Verlust der

zeitlichen Information mit dem ersten zusammen verarbeitet werden

kann, gehen Theorien des kurzzeitigen Aufmerksamkeitsschubs

davon aus, dass der erste Zielreiz erleichternd für den zweiten wirkt:

der erste Zielreiz löst den Aufmerksamkeitsschub aus, doch bevor

dieser seine Wirkung voll entfalten kann, ist der erste Zielreiz bereits

durch den zweiten überschrieben worden.

Abbildung 4: Bildliche Darstellung des kurzfristigen Aufmerksam-

keitsschubs und seines Einflusses auf die wahrgenommene

Reihenfolge.

Im hier vorgestellten empirischen Promotionsprojekt wurde anhand

der zeitlichen Reihenfolgefehler näher zwischen den oben

beschriebenen großen Theoriesträngen (begrenzte kognitive

Ressourcen auf der einen, kurzzeitiger Aufmerksamkeitsschub auf

157

der anderen Seite) unterschieden. Dazu wurden

Experimentalbedingungen kreiert, für welche die beiden

Theoriezweige unterschiedliche Vorhersagen machen.

Dabei wurde das Cueing-Paradigmas benutzt, das schon von

Nieuwenstein, Chun, van der Lubbe, und Hooge (2005),

Nieuwenstein (2006), und Olivers und Meeter (2008) eingesetzt

wurde. Theorien des kurzfristigen Aufmerksamkeitsschubs vermuten,

dass der erste Zielreiz einen Aufmerksamkeitsschub einleitet, vom

zweiten Zielreiz jedoch schon überschrieben wird, bevor sich der

Großteil der Erleichterung auswirken kann. Der zweite Zielreiz

profitiert von der gesteigerten Aufmerksamkeit, wird schneller

verarbeitet und daher in einer Reihe von Durchgängen als früher

wahrgenommen. Falls es zutrifft, dass das Ausmaß an

Reihenfolgefehlern also durch das relative Verhältnis von

Aufmerksamkeit zwischen den beiden Zielreizen bestimmt wird,

sollten Reihenfolgefehler abnehmen, wenn mehr Aufmerksamkeit auf

den ersten Zielreiz verlagert wird. Dies wurde erreicht, indem ein

Hinweisreiz zeitlich direkt vor dem ersten Zielreiz platziert wurde, um

Aufmerksamkeit auf diesen zu lenken. Von dieser Aufmerksamkeit

sollte vor allem der erste Zielreiz profitieren. Das relative Verhältnis

von Aufmerksamkeit sollte sich damit zu seinen Gunsten

verschieben, d.h. Reihenfolgefehler sollten seltener auftreten.

158

Abbildung 5: links: Schematischer Ablauf eines Versuchsdurchgangs

für die Standard Attentional-Blink Bedingung ohne Hinweisreiz und

für die Experimentalbedingung mit Hinweisreiz. Rechts:

Angenommene Aufmerksamkeits-Erleichterung für Durchgänge ohne

und mit Hinweisreiz.

Wie erwartet wurden in Durchgängen mit Hinweisreiz weniger

Reihenfolgefehler gefunden als in Durchgängen ohne einen solchen

Hinweisreiz (Olivers, Hilkenmeier, & Scharlau, 2010). Dies ist ein

klares Indiz dafür, dass die Reihenfolgefehler im Attentional Blink

tatsächlich durch einen kurzzeitigen Aufmerksamkeitsschub und

„prior entry“ erklärt werden können.

Allerdings können die Ergebnisse aus Olivers et al. (2010) in

gewisser Weise auch durch „episodic integration“ erklärt werden: Da

der Hinweisreiz gewisse Eigenschaften mit den Zielreizen teilt (in

diesem Fall die Farbe), ist es plausibel anzunehmen, dass dieser

Hinweisreiz ebenfalls ein Aufmerksamkeitsfenster öffnen kann, und

dass der Hinweisreiz gemeinsam mit dem ersten Zielreiz auf der

159

zweiten Stufe verarbeitet wird. Der zweite Zielreiz wird nicht mit auf

die zweite Stufe transferiert, sondern muss sein eigenes

Aufmerksamkeitsfenster öffnen. Falls dies gelingt, hat der zweite

Zielreiz einen anderen Zeitstempel als der erste. Gelingt es nicht,

kann er nicht berichtet werden. Die Befunde zeigen, dass die

Erkennensleistung des zweiten Zielreizes tatsächlich abnimmt, wenn

vor dem ersten Zielreiz ein Hinweisreiz eingeblendet wird (Olivers et

al., 2010). Dies könnte für eine gemeinsame Verarbeitung in einer

Episode sprechen, auch wenn für diese Vermutung noch einige

Zusatzannahmen nötig sind (siehe Olivers et al., 2010; Hilkenmeier,

Olivers, & Scharlau, 2011).

Abbildung 7: Hinweisreiz und erster Zielreiz werden in einer

gemeinsamen Episode verarbeitet. Obwohl der zweite Zielreiz direkt

hinter dem ersten kommt, gelingt es ihm, in einer neuen Episode

ebenfalls in die zweite Stufe zu gelangen und dort separat verarbeitet

zu werden.

In der ersten Studie konnte also „prior entry“ als Alternative zur weit

verbreiteten Annahme der „episodic integration“ etabliert werden. In

einem zweiten Schritt wurden Experimentalbedingungen

herangezogen, die klarer zwischen diesen beiden theoretischen

Annahmen unterscheiden können. In Hilkenmeier, Olivers und

Scharlau (2011) wurde einen Hinweisreiz direkt vor dem zweiten

160

Zielreiz dargeboten. Laut „episodic integration“ sollte sich diese

Manipulation nicht von einer Kontrollbedingung ohne Hinweisreiz

unterscheiden, da der Hinweisreiz erst nach dem ersten Zielreiz, also

wenn die Episode bereits begonnen hat, präsentiert wird. Die „prior

entry“ Erklärung hingegen sagt voraus, dass dieser Hinweisreiz dazu

führen sollte, dass der zweite Zielreiz mehr Aufmerksamkeit

bekommt, Reihenfolgefehler also zunehmen sollten.

Abbildung 6: links: Schematischer Ablauf eines Versuchsdurchgangs

für die Experimentalbedingung mit Hinweisreiz vor dem zweiten

Zielreiz. Mitte: Angenommene Aufmerksamkeits-Erleichterung für

Durchgänge mit Hinweisreiz vor dem zweiten Zielreiz. Rechts:

Angenommene episodische Verarbeitung. Beide Zielreize werden,

wie in der Kontrollbedingung ohne Hinweisreize, in einer

gemeinsamen Episode verarbeitet.

Die empirischen Daten zeigen eine klare Zunahme von

Reihenfolgefehlern, belegen also die Theorie der kurzfristigen

Aufmerksamkeitserleichterung. Dies hat weitreichende Folgen für die

theoretischen Erklärungen des Attentional Blink: Wie bereits

161

beschrieben wurde die Zusatzannahme der gemeinsamen

episodischen Verarbeitung getroffen, um „lag-1 sparing“ im Rahmen

begrenzter kognitiver Ressourcen erklären zu können. Die

begleitenden zeitlichen Reihenfolgefehler wurden als einer der

Hauptbelege für diese theoretische Erklärung herangezogen. In den

vorliegenden Studien wurde gezeigt, dass die Manipulation von

Reihenfolgefehlern im Attentional Blink nicht schlüssig durch

gemeinsame episodische Verarbeitung erklärt werden kann. Einer

der Hauptbefunde für „episodic integration“ fällt also weg. Ohne

diesen Mechanismus kann der komplette Zeitverlauf des Attentional

Blink allerdings nur noch schwerlich durch begrenzte Ressourcen

erklärt werden. Stattdessen erhärtet dieser Befund neuere Theorien,

die den Attentional Blink nicht als Beleg für begrenzte Ressourcen,

sondern als vorübergehenden Kontrollverlust (Di Lollo, Kawahara,

Ghorashi, & Enns, 2005), oder als Resultat der Distraktor-

Verarbeitung ansehen (Olivers & Meeter, 2008).

Nachdem nun ein Einfluss von „prior entry“ auf die zeitlichen

Reihenfolgefehler im Attentional Blink belegt ist, wurde in einer

weiteren Studie dem Zeitverlauf dieser Erleichterung untersucht

(Hilkenmeier, Scharlau, Weiß, & Olivers, 2011). Dabei konnte gezeigt

werden, dass Hinweisreize, die nicht nur Zielreiz-, sondern auch

Distraktoreingenschaften besitzen, eine nur kurzfristige Erleichterung

auslösen. Diese Erleichterung erreicht ihren Höhepunkt schon nach

ca. 50 ms. Falls der Hinweisreiz hingegen nur Zielreizeigenschaften

besitzt, verlängert sich die Erleichterung auf ca. 100 ms. Dieses

162

Befundmuster konnte durch ein einfaches computationales Modell

vorhergesagt werden, das auf bisherigen Theorien der kurzfristigen

Aufmerksamkeitsverlagerung aufbaut (Bachmann, 1984; Reeves &

Sperling, 1986; Nakayama & Mackeben, 1989; Olivers & Meeter,

2008, Wyble, Bowman, & Nieuwenstein, 2009).

Abbildung 7: links: Simulationen des computationalen Modells. Links:

Hinweisreiz mit Distraktoreigenschaften. Dieser Reiz löst nicht nur

eine Erleichterung, sondern auch eine verzögerte Inhibierung aus.

Rechts: Hinweisreiz nur mit Zielreizeingeschaften. Dieser Reiz löst

ausschließlich Erleichterung aus.

Zusammenfassend kann man sagen, dass es im hier vorliegenden

Promotionsprojekt gelungen ist, den Einfluss einer kurzfristigen

Aufmerksamkeitserleichterung auf Reihenfolgefehler im Attentional

Blink nachzuweisen. Dies hat nicht nur für Theorien des Attentional

163

Blink eine gewisse Bedeutung, sondern zeigt darüber hinaus auch,

dass sich verschiedene experimentelle Paradigmen auf gemeinsame

Aufmerksamkeitsmechanismen zurückführen lassen.

Literaturverzeichnis

Akyürek, E. G., & Hommel, B. (2005b). Target integration and the

attentional blink. Acta Psychologica, 119, 305-314.

Akyürek E. G., Riddell P. M., Toffanin P., Hommel B. (2007).

Adaptive Control of event integration: evidence from event-

related potentials. Psychophysiology, 44, 383 - 391.

Bachmann, T. (1984). The process of perceptual retouch:

Nonspecific afferent activation dynamics in explaining visual

masking. Attention, Perception, & Psychophysics, 35, 69-84.

Bowman, H., Wyble, B. (2007). The simultaneous type, serial token

model of temporal attention and working memory.

Psychological Review, 114, 38-70.

Caldwell-Harris, C. L., & Morris, A. L. (2008). Fast Pairs: A visual

word recognition paradigm for measuring entrenchment, top-

down effects, and subjective phenomenology. Consciousness

and Cognition, 17, 1063-1081.

Chun, M. M., & Potter, M. C. (1995). A two-stage model for multiple

target detection in rapid serial visual presentation. Journal of

Experimental Psychology: Human Perception and

Performance, 21, 109-127.

164

Di Lollo, V., Kawahara, J., Ghorashi, S.M., Enns, J.T., 2005. The

attentional blink: resource depletion or temporary loss of

control? Psychological Research, 69, 191–200.

Duncan, J., Ward, R., & Shapiro, K. (1994). Direct measurement of

attentional dwell time in human vision. Nature, 369, 313-315.

Helmholtz, H. (1867). Handbuch der physiologischen Optik. Voss.

Fulltext online available at http://books.google.de/books?id=

Ih85AAAAcAAJ&printsec=frontcover&source=gbs_ge_summa

ry_r&cad=0#v=onepage&q&f=false

Hilkenmeier, F., Olivers, C. N. L., & Scharlau, I. (2011). Prior entry

and temporal attention: Cueing affects order errors in RSVP.

In press at Journal of Experimental Psychology: Human

Perception and Performance.

Hilkenmeier, F., & Scharlau, I., Weiß, K., & Olivers, C. N. L. (2011).

The dynamics of prior entry in serial visual processing. in

press at Visual Cognition.

Hommel, B., & Akyürek, E. G. (2005). Lag-1 sparing in the attentional

blink: Benefits and costs of integrating two events into a single

episode. Quarterly Journal of Experimental Psychology, 58,

1415-1433.

James, W. (1890) The principles of psychology, Vol I. Henry Holt and

Co. Fulltext online available at http://psychclassics.yorku.

ca/James/Principles/

165

Jolicœur, P., Tombu, M., Oriet, C., & Stevanovski, B. (2002). From

perception to action: Making the connection. In W. Prinz & B.

Hommel (Eds.), Attention and performance XIX: Common

mechanisms in per- ception and action (pp. 558–586). Oxford,

England: Oxford University Press.

Jonides, J. (1981). Voluntary versus automatic control over the

mind’s eye’s movement. Attention and performance IX, 9, 187-

203.

Martens, S., & Wyble, B. (2010). The attentional blink: Past, present,

and future of a blind spot in perceptual awareness.

Neuroscience & Biobehavioral Reviews, 34, 947-957.

Nakayama, K., & Mackeben, M. (1989). Sustained and transient

components of focal visual attention. Vision Research, 29,

1631-1647.

Nieuwenstein, M. R. (2006). Top-down controlled, delayed selection

in the attentional blink. Journal of Experimental Psychology:

Human Perception & Performance, 32, 973-985.

Nieuwenstein, M. R., Chun, M. M., van der Lubbe, R. H. J., & Hooge,

I. T. C. (2005). Delayed Attentional Engagement in the

Attentional Blink. Journal of Experimental Psychology: Human

Perception and Performance, 31, 1463-1475.

166

Olivers, C. N. L., Hilkenmeier, F., & Scharlau, I. (2010). Prior entry

explains order reversals in the attentional blink. Attention,

Perception, & Psychophysics, 73, 53-67.

Olivers, C. N. L., & Meeter, M. (2008). A boost and bounce theory of

temporal attention. Psychological Review, 115, 836-863.

Posner, M.I. (1980). Orienting of attention. The Quarterly journal of

experimental psychology, 32, 3-25.

Potter, M. C., Chun, M. M., Banks, B. S., & Muckenhoupt, M. (1998).

Two attentional deficits in serial target search: The visual

attentional blink and an amodal task-switch deficit. Journal of

Experimental Psychology: Learning, Memory, and Cognition,

24, 979-992.

Potter, M.C., & Levy, E.I. (1969). Recognition memory for a rapid

sequence of pictures. Journal of Experimental Psychology, 81,

10-15.

Raymond, J.E., Shapiro, K.L., & Arnell, K.M. (1992). Temporary

suppression of visual processing in an RSVP task: an

attentional blink? Journal of Experimental Psychology: Human

Perception and Performance, 18, 849-860.

Reeves, A., & Sperling, G. (1986). Attention gating in short-term

visual memory. Psychological Review, 93, 180-206.

Titchener, E.B. (1908). Lectures on the Elementary Psychology of

Feeling and Attention. New York, The Macmillan Co. Fulltext

167

online available at http://www.archive.org/details/elemen

tarypsycho00titc

Visser, T., Bischof, W. F., and Di Lollo, V. (1999). Attentional

switching in spatial and non-spatial domains: evidence from

the attentional blink. Psychological Bulletin, 125, 458-469.

Ward, R., Duncan, J., & Shapiro, K. (1996). The Slow Time-Course

of Visual Attention. Cognitive Psychology, 30, 79-109.

Wyble, B., Bowman, H., & Nieuwenstein, M. (2009). The attentional

blink provides episodic distinctiveness: Sparing at a cost.

Journal of Experimental Psychology: Human Perception &

Performance, 35, 787 - 807.