Punishment-related memory-guided attention: Neural dynamics of perceptual modulation

PUNISHMENT-RELATED MEMORY-GUIDED ATTENTION: NEURAL DYNAMICS OF PERCEPTUAL MODULATION Authors: Samuel Suárez-Suárez, Socorro Rodríguez Holguín, Fernando Cadaveira, Anna Christina Nobre, Sonia Doallo This is the peer reviewed version of the following article: Suárez-Suárez S; Rodríguez Holguín S; Cadaveira F; Nobre AC; Doallo S (2019). Punishment-related memoryguided attention: Neural dynamics of perceptual modulation. Cortex, 115, 231-245. doi: 10.1016/j.cortex.2019.01.029. This article may be used for non-commercial purposes in accordance with Elsevier Terms and Conditions for Use of Self-Archived Versions. 2 Punishment-related memory-guided attention: Neural dynamics of perceptual modulation Samuel Suárez-Suáreza, Socorro Rodríguez Holguína, Fernando Cadaveiraa, Anna Christina Nobreb, Sonia Doalloa a Department of Clinical Psychology and Psychobiology, University of Santiago de Compostela, Santiago de Compostela, Spain b Department of Experimental Psychology and Oxford Centre for Human Brain Activity, Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, United Kingdom Corresponding author: Sonia Doallo Department of Clinical Psychology and Psychobiology University of Santiago de Compostela Campus Vida, s/n, 15782 Santiago de Compostela Galicia-Spain Telephone: +34 881813916 E-mail: [email protected] Declarations of interest: none. Post-print (final draft post-refereeing) 3 ABSTRACT Remembering the outcomes of past experiences allows us to generate future expectations and shape selection in the long-term. A growing number of studies has shown that learned positive reward values impact spatial memory-based attentional biases on perception. However, whether memory-driven attentional biases extend to punishment-related values has received comparatively less attention. Here, we manipulated whether recent spatial contextual memories became associated with successful avoidance of punishment (potential monetary loss). Behavioural and electrophysiological measures were collected from 27 participants during a subsequent memory-based attention task, in which we tested for the effect of punishment avoidance associations. Punishment avoidance significantly amplified effects of spatial contextual memories on visual search processes within natural scenes. Compared to non-associated scenes, contextual memories paired with punishment avoidance lead to faster responses to targets presented at remembered locations. Event-related potentials elicited by target stimuli revealed that acquired motivational value of specific spatial locations, by virtue of their association with past avoidance of punishment, dynamically affected neural signatures of early visual processing (indexed by larger P1 and earlier N1 potentials) and target selection (as indicated by reduced N2pc potentials). The present results extend our understanding of how memory, attention, and punishment-related mechanisms interact to optimize perceptual decision in real world environments. Keywords: Long-term memory, visuospatial attention, punishment, event-related potentials Post-print (final draft post-refereeing) 4 1. Introduction Previous research has demonstrated the ability of spatial contextual long-term memory (LTM) to guide attention within scenes, and to modulate neural signatures of early perceptual analysis (Summerfield, Rao, Garside, & Nobre, 2011) and selection (Patai, Doallo, & Nobre, 2012) of relevant objects presented at memorized locations. This memory-driven modulation of visual processing engages activity in the frontalparietal network for visual-spatial orienting in concert with activation in other brain regions implicated in retrieval of object locations within specific contexts (e.g., hippocampus) (Stokes, Atherton, Patai, & Nobre, 2012; Summerfield, Lepsien, Gitelman, Mesulam, & Nobre, 2006). More recent work by Rosen and colleagues shows that LTM-guided visuospatial attention recruits a network spanning parietal cortical areas (lateral intraparietal sulcus, posterior precuneus and posterior callosal sulcus) and subcortical regions (the caudate head, mediodorsal thalamus and lobule VI/Crus I of cerebellum) (Rosen, Stern, Devaney, & Somers, 2017; Rosen, Stern, Michalka, Devaney, & Somers, 2016). A growing number of studies have also started addressing the important question of whether and how learned reward values impact spatial memory-based attentional biases on perception. Recent work has shown that rewardoutcome associations boost memories and attentional priority of specific spatial locations (Chelazzi et al., 2014; Hickey, Chelazzi, & Theeuwes, 2014; see also Anderson, 2015; Pollmann, Estocinova, Sommer, Chelazzi, & Zinke, 2016), and dynamically impact different levels of visual cortical processing of targets presented at previously rewarded locations (Doallo, Patai, & Nobre, 2013). These effects of reward history on attentional priority of space add to accumulating evidence indicating that reward-associations modulate bottom-up and top-down attentional priority of stimuli features and object categories (e.g. Anderson, Laurent, & Yantis, 2011; Della Libera & Post-print (final draft post-refereeing) 5 Chelazzi, 2006, 2009; Donohue et al., 2016; Harris et al., 2016; Hickey, Chelazzi, & Theeuwes, 2010; Hickey, Kaiser, & Peelen, 2015; Hickey & Peelen, 2015; Kiss, Driver, & Eimer, 2009; for reviews see Anderson, 2013; Chelazzi, Perlato, Santandrea, & Della Libera, 2013; Failing & Theeuwes, 2018; Pessoa, 2015; Vuilleumier, 2015). An important remaining question is whether punishment-related associations also influence memory-driven spatial-attentional biases. In contrast to the increasing number of studies examining positive reward-related effects on hippocampus-dependent memory and visuospatial attention, less is known about whether similar effects are observed when spatial locations acquire motivational value through the association with an aversive outcome (or its avoidance). The complexity of this issue increases if we consider that the question of whether reward and punishment processing rely on a common or on distinct neural systems is still debated (Bissonette, Gentry, Padmala, Pessoa, & Roesch, 2014; Brooks & Berns, 2013; Liu, Hairston, Schrier, & Fan, 2011). Stimuli that undergo aversive conditioning have been shown to exhibit enhanced processing under challenging conditions (Padmala & Pessoa, 2008), to capture attention automatically during visual search (Schmidt, Belopolsky, & Theeuwes, 2015), and to counteract the attentional blink (Lim, Padmala, & Pessoa, 2009). Growing evidence suggests that stimulus features and objects associated with a monetary loss through associative learning also affect attentional and perceptual processes. Face stimuli associated to the receipt of a monetary loss are more likely to be recognized than other faces in a subsequent attentional blink task (Raymond & O’Brien, 2009) and receive enhanced visual processing in a masked recognition task (O’Brien & Raymond, 2012). Stronger reorienting processes for abrupt-onset (exogenous) colored cues linked to a monetary punishment have also been reported (Bucker & Theeuwes, 2016; but see Rutherford, O'Brien, & Raymond, 2010), as well as increased attentional capture by Post-print (final draft post-refereeing) 6 colored singleton distractors associated to evasion of a monetary loss in a visual search task (Wentura, Müller, & Rothermund, 2014). Recent data reveal that perceptual features associated to receipt of punishment boost primary visual cortex responses (as reflected by the higher amplitude of the C1 potential), relative to reward-associated and neutral ones (Rossi et al., 2017), although evidence for an enhanced representation in visual ventral areas for gainversus loss-associated stimuli has been found for symbol cues during an economic decision-making task (San Martín, Appelbaum, Huettel, & Woldorff, 2014) and for object categories presented in images of natural scenes (Barbaro, Peelen, & Hickey, 2017). Though evidence for modulatory effects of punishment associations in visual perceptual and attentional tasks, whether it can be observed for specific locations in cluttered naturalistic contexts remains an open question; furthermore, it is unclear whether it has a lasting effect that can modulate LTM-based attention. Here, we aimed to shed light on whether and how punishment-related mechanisms influence the ability of LTM to guide perception in naturalistic contexts. We used a similar experimental approach as in our previous study (Doallo et al., 2013) to disentangle whether spatial expectations from LTM incorporate value signals specifically related to past avoidance of an aversive outcome to magnify experiencebased biases upon perceptual decisions on relevant objects in cluttered scenes. Eventrelated potentials (ERPs) were recorded to reveal the time course over which punishment-associated memory-guided attention exerts its effects on the ongoing neural activity in visual cortical areas. We analysed the P1, N1 and N2pc components. These potentials reflect visual processing (P1 and N1) and target selection (N2pc). The P1 potential, a positive deflection peaking about 100 ms after stimulus presentation, is modulated by visuospatial attention and thought to reflect a sensory gain control Post-print (final draft post-refereeing) 7 mechanism within extrastriate visual cortex (Heinze et al., 1994; Hillyard, Vogel, & Luck, 1998; Martínez et al., 1999). The N1 potential is a negative deflection, subsequent to the P1, which is also modulated by attention. The N1 attention effect is believed to index a top-down modulation of discriminative processing in areas of the ventral visual stream (Hopf, Vogel, Woodman, Heinze, & Luck, 2002; Luck, 1995; Vogel & Luck, 2000). The N2pc is an enhanced negativity at posterior electrode sites contralateral to the location of an attended target, typically emerging 200-300 ms after target onset. It is thought to reflect attentional selection of a target among distracters (Eimer, 1996; Luck & Hillyard, 1994) and is generated in parietal and inferior occipitaltemporal cortical areas (Hopf et al., 2000). These ERP components have been previously shown to be modulated by memory-guided orienting (Doallo et al., 2013; Patai et al., 2012; Summerfield et al., 2011) and have also been used to investigate value-based modulation of perceptual processing and attentional selection in other types of tasks (Donohue et al., 2016; Harris et al., 2016; Hickey et al., 2010; Itthipuripat, Cha, Rangsipat, & Serences, 2015; McLean & Giesbrecht, 2015; Qi, Zeng, Ding, & Li, 2013). In the present experiment, participants first performed a learning task during which they learnt the spatial location of a predefined target (a small key) embedded within visual scenes. Punishment-related associations were manipulated by punishing bad performance during the last block of the learning task on a proportion of trials. Twenty-four hours later, they completed an LTM-cued visual search task in which they had to discriminate the presence or absence of target key stimuli within the previously studied scenes while ERPs were recorded. The initial presentation of the scene (without the target) served as a contextual memory-based cue to orient spatial attention in each trial. Contextual scene cues could either have been associated with avoidance of an Post-print (final draft post-refereeing) 8 aversive outcome (i.e. potential monetary loss) or have had no outcome association. Based on our prior work (Doallo et al., 2013), we hypothesized that if experiencedependent attentional biases on perception are influenced by punishment avoidance associations, this would lead, specifically, to (i) enhanced behavioural performance, as revealed by improved reaction times and accuracy, and (ii) modulation of neural signatures of early visual processing (expressed as enhanced amplitudes of the P1 potential and earlier latencies of N1) and target selection (indexed by reduced amplitudes of N2pc). 2. Materials and Methods 2.1. Participants Thirty-two healthy students from the University of Santiago de Compostela (Galicia, Spain) participated in this study for monetary compensation. Participants completed a questionnaire regarding personal history of neurological and/or psychiatric disease, existing chronic disease and/or current pharmacological treatment. All participants gave written consent. Data from five participants were discarded during EEG preprocessing because of excessive artifacts in their EEG recording. The remaining 27 participants (18 women, age range 19-27, mean age 20.89 ± 2.02) were all right-handed and had normal or corrected-to-normal vision. The study was approved by the Bioethics Committee of the University of Santiago de Compostela. 2.2. Experimental procedure The task used in this study was a modified version of the experimental design employed in Doallo et al. (2013). There were two phases to the experiment. Participants first performed a learning task (see Figure 1), followed on the second day by a memory-cued Post-print (final draft post-refereeing) 9 orienting task (in which EEG activity was recorded) (Figure 2A) and a spatial memory recall task (Figure 2B). 2.2.1. Stimuli Two hundred and four digital images of scenes were obtained from the image set used in our previous study (Doallo et al., 2013). A set of 192 scenes was used in experimental trials, and additional 12 scenes were used for familiarization and practice trials. Each scene was prepared, using Matlab (Mathworks, Natick, MA), in two different formats, one for the learning task and one for the orienting task. For counterbalancing purposes, four learning task versions were prepared for each scene with the key (15 x 29 pixels, equivalent to 0.3º x 0.7º) placed in one of each of the four visual quadrants, preferably in a hidden location. The assignment of scenes to different experimental conditions, key presence or absence (in the orienting task) and key location within scenes were counterbalanced across participants. For the orienting task, the scenes with keys were remade to include a larger and brighter key (25 x 49 pixels; 0.6º x 1.1º) in the location of the original key to make it visible within the briefly displayed target scene. Scene stimuli were presented using Presentation (Neurobehavioral Systems, Albany, CA) and subtended 22º x 17º of visual angle at a viewing distance of 100 cm. 2.2.2. Learning task During the learning task, participants viewed 192 naturalistic scenes repeated in random order over five blocks (Figure 1A). Participants explored the scenes overtly to search for a small gold key target (present in all the scenes). Once located, they activated the mouse cursor with a left-side mouse click and indicated the location of the key by positioning the cursor on the location of the key and making a second left-sided mouse click. After a response or after the available search time expired, visual written feedback Post-print (final draft post-refereeing) 16 were thus corrected for multiple comparisons. Additional analyses should be considered as exploratory and hypothesis-generating. For all p values reported throughout the paper, we provide nominal p values and, where appropriate, corrected p values using FDR, as well as effect sizes calculated as partial eta-squared (ŋ2p) values. 2.5. Mediation analysis Mediation analyses were conducted to examine whether modulation of waveform potentials mediated the effect of condition on performance (i.e. whether the benefits in avoidance vs. safe trials were mediated by modulations of visual cortical processing). We performed separate mediation analyses for repeated-measures designs using the MEMORE macro for SPSS (Montoya & Hayes, 2017), with condition (avoidance vs. safe) as the independent variable, ERPs (amplitude and/or latency parameters) as the mediators, and behaviour (RT, accuracy and d’) as the dependent variables. The procedure described by Montoya and Hayes (2017) conceptualizes mediation analysis as a path-analytic framework in which mediation is assessed by a single test of the indirect effect (a*b; i.e. the conjunction of the effect of condition on the potential mediator [path a] and the effect of the potential mediator on behavioral performance [path b]). The direct effect of condition on behavioral performance that does not operate through the mediator is also calculated (path c’). Mediation analyses were conducted using the percentile bootstrap method with 10,000 iterations. The indirect effect was considered statistically significant if the confidence interval (CI 95%) excluded zero. 3. Results 3.1. Formation of robust LTMs for target locations within natural scenes Post-print (final draft post-refereeing) 17 Behavioural analysis of the learning task confirmed that participants were able to establish robust memories for the spatial locations at which target stimuli were presented. Over the course of the learning blocks, participants located an increasing number of targets, with increasing speed (Block 1: mean accuracy ± SEM = 92.4 ± 1.1%, mean search times ± SEM = 4.2 ± 0.2 sec; Block 5: 97.8 ± 0.4%, 1.2 ± 0.1 sec) (Figure 1B). ANOVAs testing for linear decreases in search times revealed a significant linear contrast over the learning blocks, F(1, 26) = 386.64, p < .001, ŋ2p = .94. A significant linear increase in accuracy over the learning blocks was similarly revealed, F(1, 26) = 48.53, p < .001, ŋ2p = .65. No difference was found between avoidance and safe scenes in the final (sixth) block either in search time (avoidance: 1.1 ± 0.05; safe: 1.1 ± 0.05; F(1, 26) = .07, p = .79, ŋ2p = .003) or in accuracy (avoidance: 97.7 ± 0.5; safe: 96.9 ± 0.5; F(1, 26) = 3.34, p = .08, ŋ2p = .11). The recall task performed immediately after the orienting task confirmed that participants retained strong memories of the key locations within the learned scenes on the day after the learning task. On average, participants could explicitly recall the correct locations of targets on 75% of scenes (± 2.7 SEM). In addition, subjects' confidence ratings varied systematically with the response distance from actual target location, F(1.6, 41.6) = 64.96, p < .001, ɛ = .8, ŋ2p = .71, revealing that higher confidence ratings were associated with more accurate memories [mean distance in pixels ± SEM; Rate 1: 81.4 ± 3.9; Rate 2: 60.7 ± 3.5; Rate 3: 36.9 ± 1.6] (see Figure 2B). Recall did not differ between avoidance-associated and safe key locations (accuracy: 75.9 ± 2.6% vs. 73.8 ± 3%, F(1, 26) = 1.93, p = .18, ŋ2p = .07; overall distance in pixels: 58.4 ± 3 vs. 60.9 ± 2.9, F(1, 26) = .39, p = .54, ŋ2p = .02; confidence ratings: 2.67 ± .04 vs. 2.68 ± .04, F(1, 26) = .01, p = .92, ŋ2p < .001). Post-print (final draft post-refereeing) 18 3.2. One single punishment avoidance-related association enhances the ability of spatial memories to drive visual search in natural scenes RT and accuracy levels for target-present and target-absent trials in avoidance and safe conditions in the orienting task are summarized in Table 1. To test our main hypotheses regarding behavioural performance (improved RT and accuracy), we examined the main effect of condition on these two dependent variables. There was a significant effect on RT, F(1, 26) = 6.67, p = .016, p(FDR corrected) = .048, ŋ2p = .204, revealing that target discrimination was faster in avoidance (700.03 ± 31.13) versus safe (709.07 ± 31.22) trials. The main effect of condition on accuracy was not significant (avoidance: 0.91 ± 0.01, safe: 0.92 ± 0.01; F(1, 26) = .09, p = .77, p(FDR corrected) = .81, ŋ2p = .003). In addition, RT and accuracy ANOVAs showed the following results. Target discrimination was faster in target-present trials (present: 651.58 ± 32.3, absent: 757.53 ± 32.37; F(1, 26) = 36.59, p < .001, ŋ2p = .59), but no significant interaction was found between condition and target presence on RT, F(1, 26) = .58, p = .45, ŋ2p = .02. Analysis of accuracy revealed marginally significant effects for target presence (present: 0.91 ± 0.01, absent: 0.93 ± 0.01; F(1, 26) = 3.93, p = .058, ŋ2p = .13) and for the interaction between condition and target presence, F(1, 26) = 3.89, p = .059, ŋ2p = .13, which showed a trend for higher accuracy for safe vs. avoidance conditions in target absent trials. To test the possibility that RT effects reflected a speed-accuracy trade-off, Pearson’s correlation between RT and accuracy effects (calculated by subtracting safe trials from avoidance trials) was performed. A significant negative correlation (r = -.48, p = .006) indicated that faster responses in avoidance vs. safe scenes were associated with higher levels of accuracy. Post-print (final draft post-refereeing) 19 Finally, the analysis of perceptual sensitivity showed that the d’ measure was equivalent between avoidance and safe trials (avoidance: 3.27 ± 0.63, safe: 3.2 ± 0.61; t(26) = .58, p = .57). 3.3. Punishment avoidance-associated memories modulate target-related neural activity 3.3.1. Early visual processing (P1 and N1 potentials) Target-present scenes elicited the expected visual potentials P1 and N1 over parietooccipital scalp regions in all conditions (Figure 3). As explained in the Introduction section, drawing on our previous work (Doallo et al., 2013; see also Summerfield et al., 2011), we expected that punishment avoidanceassociated memory-based orienting would result in enhanced amplitudes of the P1 potential and earlier latencies of N1. As predicted, there was a significant main effect of condition on P1 amplitude, F(1, 26) = 4.31, p = .048, ŋ2p = .14, revealing that P1 was larger for targets appearing at avoidance-associated remembered locations relative to safe locations. After having identified that one voltage value was outside 3SD from the mean, we verified that the main effect of condition on P1 amplitude remained significant when the outlier was replaced by the series’ mean, F(1, 26) = 4.68, p = .040, ŋ2p = .15. This result, however, did not survive the FDR correction, p(FDR corrected) = .067. Regarding the N1 latency, there was no a significant main effect of condition, F(1, 26) = 2.45, p = .13, p(FDR corrected) = .81, ŋ2p = .09. The ANOVAs carried out on the P1 amplitude and N1 latency also revealed the following results. A significant main effect of electrode on P1 amplitude, F(2.03, 52.66) = 3.73, p = .03, ɛ = .338, ŋ2p = .13, indicated that it was maximal over electrodes P5/6 and PO7/8, but no other main effects or interactions reached significance. The analysis of N1 latency showed that it peaked Post-print (final draft post-refereeing) 20 earlier over the more posterior sites (O1/2, PO3/4; main effect of electrode, F(3.09, 80.27) = 10.77, p < .001, ɛ = .515, ŋ2p = .29), and over contralateral (149 ± 2 msec) versus ipsilateral (151 ± 2 msec) sites (main effect of hemisphere: F(1, 26) = 4.72, p = .039, ŋ2p = .15). There also was a significant hemisphere x condition interaction on N1 latency, F(1, 26) = 6.31, p = .019, ŋ2p = .195, which showed the latencies to be earliest for targets preceded by avoidance-associated memory cues (149 ± 2 msec) than for those preceded by safe memory cues (153 ± 3 msec) over the ipsilateral hemisphere (p = .030). The post-hoc comparisons also revealed the effect of hemisphere being significant for safe trials (contralateral vs. ipsilateral: 150 ± 3 vs. 153 ± 3 msec; p = 0.009) but not for avoidance trials (149 ± 2 vs. 149 ± 2 msec; p = 0.708). The analysis of P1 latency showed earlier latencies at posterior electrodes (O1/2, PO3/4; main effect of electrode: F(3.13, 81.29) = 10.28, p < .001, ɛ = .521, ŋ2p = .28), but no significant effects of hemisphere or condition; only subsidiary ANOVAs on a significant 3-way interaction between electrode, hemisphere and condition, F(4, 104) = 2.48, p = .049, ɛ = .667, ŋ2p = .09, revealed a significant hemisphere x condition interaction at PO9/10 electrodes, F(1, 26) = 5.43, p = .028, ŋ2p = .17, showing a trend for earlier P1 latencies in avoidance (95 ± 2 msec) versus safe (100 ± 3 msec) trials over ipsilateral sites (p = 0.053). The analysis of N1 amplitude showed that it was larger at posterior electrodes (O1/2, PO3/4; main effect of electrode, F(3.33, 86.59) = 12.83, p < .001, ɛ = .555, ŋ2p = .33), but neither a main effect nor an interaction involving the factor condition were found, indicating that the amplitude of the N1 component was unaffected by type of memory cue. Post-print (final draft post-refereeing) 21 ___________________________________________ Insert Figure 3 around here ___________________________________________ 3.3.2. Target selection (N2pc) The N2pc potential was elicited by target-present scenes at parieto-occipital electrodes contralateral to the side of the target (Figure 4). The reliability of the N2pc was confirmed by a main effect of hemisphere, F(1,26) = 32.28, p < .001, ŋ2p = .55, on mean amplitudes 190-270 msec after target onset. If as predicted, based on our previous research, N2pc amplitude is modulated by the motivational value of the preceding cue, this should result in an interaction between hemisphere and condition. The predicted two-way interaction was indeed significant, even after FDR correction, F(1,26) = 6.23, p = .019, p(FDR corrected) = .048, ŋ2p = .19, indicating that, as expected, the amplitude of N2pc became attenuated by punishment avoidance associated-memory cues. Posthoc analysis indicated that a significant N2pc was elicited in safe trials (p < .001) and, although smaller, was also present in avoidance trials (p = .001). To confirm that the N2pc was related to the selection of the target in its scene context and to rule out that it may have been driven simply by the orienting of spatial attention, we also tested for the presence of the N2pc in target-absent trials. In these cases, there was no significant main effect of hemisphere, F(1,26) = .73, p = .40, ŋ2p = .03, on mean amplitudes 190-270 after scene onset, confirming that N2pc was not observed when no target was present. ___________________________________________ Insert Figure 4 around here ___________________________________________ Post-print (final draft post-refereeing) 22 Visual inspection of the waveforms also revealed a later lateralized effect, following the N2pc with opposite polarity (see Figure 4), which has been also reported in our previous studies examining LTM-guided visual search in naturalistic scenes (labeled as posterior contralateral positivity, PCP; Doallo et al., 2013; Patai et al., 2012). This effect was an enhanced positivity over posterior contralateral (relative to ipsilateral) scalp locations to the target side in the latency window between 320 and 380 msec poststimulus. An ANOVA analyzing ERP mean amplitudes through this latency window over PO9/10, PO7/8, PO3/4 and O1/2 confirmed the presence of this lateralized effect (main effect of hemisphere: F(1,26) = 14.67, p < .001, ŋ2p = .36), but it was not differentially modulated by punishment avoidance-associated contextual memories (hemisphere x condition: F(1,26) = .45, p = .508, ŋ2p = .02). 3.4. Relationship between behavioral and electrophysiological measures Mediation analyses showed that the P1 amplitude mediated the effect of condition on RT, ab = -4.227, bootSE = 2.739, CI95% [-10.499, -.102], which was the only indirect effect significantly different from zero (see also Supplementary Figure 1). This result suggests that faster responses in avoidance relative to safe conditions may occur through changes in sensory processing reflected by the P1 potential (i.e. P1 amplitude seems to predict a significant proportion of the co-variation between condition and RT). The complete pattern of results from the mediation analyses is shown in Supplementary Tables 1, 2 and 3. 4. Discussion The findings of this experiment demonstrate the role of past avoidance of an aversive outcome in magnifying memory-driven attentional biases upon perceptual decisions on Post-print (final draft post-refereeing) 23 relevant objects embedded in cluttered natural scenes. Importantly, our findings revealed the ability of one single punishment-avoidance association in memory to enhance attention and visual search processes in scenes and to modulate ongoing processing in visual cortical areas. Behavioral results showed that spatial expectations from LTM associated to successful avoidance of punishment conferred behavioral benefits, as revealed by faster responses to targets placed in punishment avoidanceversus safe-related remembered locations. These findings replicate and extend our previous results (Doallo et al., 2013) by showing that punishment-related LTM, similarly to what is observed when positive reward is involved, allows attention to reach the target location more rapidly compared with when memories not associated to motivational values guide spatial orienting. The observed effects in behavior thus indicate that punishment avoidance can bias attention through associations in memory but in the absence of immediate potentially negative consequences (i.e. no punishment was at stake in the visual search task). The present data expand upon the growing behavioral evidence showing that stimuli associated through learning to punishment avoidance (Wentura et al., 2014) or to an actual monetary loss (Bucker & Theeuwes, 2016) can influence attentional deployment and receive facilitated processing (O'Brien & Raymond, 2012). Unlike our prior study manipulating positive rewards, we did not find significant effects on accuracy. Taking into account that there was also a trend for higher accuracy in safe relative to avoidance trials for target absent scenes, it is hard to say, from the present results, if this difference may be attributable to specifically manipulate avoidance-related values, leading to differential behavioral effects than when gains are involved. In this sense, RT effects in the absence of accuracy ones, related to punishment-associated stimuli, have been reported previously (Bucker & Theeuwes, 2016). Post-print (final draft post-refereeing) 24 The moment-by-moment record of target-related ERPs showed that the behavioral improvement driven by avoidance-related associations was accompanied by modulation of multiple stages of visual processing. Similarly to the pattern of results observed in our previous study, the earliest effect was observed on the P1 potential, an index of attentional gain in extrastriate visual cortex, which showed larger amplitudes at parieto-occipital sites for targets appearing at punishment avoidance-associated remembered locations (relative to safe-related locations), in the absence of effects on the P1 latency. These findings provide evidence that avoidance-associated memorydriven attention biases early perceptual analysis of relevant objects embedded in crowded real-world scenes. Although the main effect of condition on P1 amplitude, which passed a nominal alpha threshold of .05, did not survive FDR correction, the finding that the P1 amplitude modulation significantly mediated the relationship between punishment-related associations in memory and benefits in the reaction time further underscores the significance of this potential in the prioritized processing of stimuli at spatial locations linked to evasion of punishment in the past. Our findings seem to differ from some recent reports of ERP modulations by acquired positive and negative motivational salience. Bayer et al. (2017) investigated the interplay between reward prospect (i.e. performance-based monetary incentives) during a cued pattern discrimination task on the activity of the primary and extrastriate visual cortex. Motivational relevance conveyed by the cue (related to both reward approach and punishment avoidance) increased the amplitude of the C1 potential (reflecting early perceptual processing in V1) but had no effects at the P1 level. On the other hand, Hammerschmidt, Sennhenn-Reulen, and Schacht (2017) found that neutral faces previously associated with three different monetary outcomes (gain, loss or zero) impacted differently the amplitude of the P1 potential: the facilitated sensory processing Post-print (final draft post-refereeing) 25 of stimuli with associated motivational salience, as reflected by larger P1 amplitudes, was confined to reward-related faces (relative to the neutral, zero-outcome, ones). When addressing the differences between earlier studies and ours, it is important to take into account that findings such as those from Hammerschmidt et al. (2017) indicate a spatially unspecific effect (rather than limited to specific spatial positions) of acquired motivational salience. The differences between the results reported by Bayer et al. (2017) and those of the current study could reflect differential effects of visual spatial cues conveying information about performance-based monetary incentives and contextual memory cues which acquired value in a prior learning experience. Another contributing factor to these differences could be the challenging perceptual conditions in our task. Our target stimuli were embedded within an associated cluttered scene, which may favor prioritization effects (as reflected by higher amplitudes of P1) by which stimuli appearing at spatial locations associated with avoidance of an aversive outcome in past encounters win representation at the expense of other stimuli. Regarding the N1 potential, an electrophysiological marker of discriminative processes in areas of the ventral visual stream (Hopf et al., 2002; Luck, 1995; Vogel & Luck, 2000), our results showed a significant interaction between condition and hemisphere on its latency, indicating that orienting of spatial attention based on loss avoidance tend to shortened the latency of the N1 component over ipsilateral sites. Although our primary hypothesis testing did not reveal a main effect of condition on N1 latency, this interaction suggests that contextual memories associated to avoidance of punishment are also able to speed up the discrimination of targets appearing at these locations in subsequent encounters, in line with our prior findings (Doallo et al., 2013). This result, however, should be interpreted with caution because it was not subjected to multiple testing correction. Post-print (final draft post-refereeing) 32 this issue, it is important to take into account that the restricted statistical power, mainly due to the characteristics of this type of experimental design, makes necessary to balance the likelihood of Type II errors. We should note that the ecological validity achieved by the use of memories for specific target locations in real scenes comes at a cost. It limits the number of scenes we can use, and prevents us from repeating scenes during the perceptual discrimination task (without possibly compromising the state of the memories being investigated). This consequently reduces the number of trials available and limits the statistical power that can be achieved in other types of designs. To deal with this limitation and balance the likelihood of Type I and Type II errors, we corrected our primary hypotheses for multiple comparisons, as reported throughout this study. In conclusion, the present study provides new evidence that memory-dependent spatial attentional biases on perception are influenced by punishment avoidance associations, leading to enhanced behavioral performance and modulation of neural signatures of target processing. It also extends our understanding of the role of acquired motivational value in selectively prioritizing specific spatial locations when searching for objects in naturalistic contexts. Acknowledgments This research was supported by a Project Grant to S.D. from the Consellería de Educación e Ordenación Universitaria (Xunta de Galicia, Spain) (EM2012-017). S.S-S was supported by a grant for predoctoral contracts from the Spanish Ministry of Economy and Competitiveness (BES-2016-076298). A.C.N. is supported by Wellcome Post-print (final draft post-refereeing) 33 Trust Senior Investigator Award (104571/Z/14/Z). We thank Kenia Correa for assistance in EEG data collection and preprocessing. Data statement We report how we determined our sample size, all data exclusions, all inclusion/exclusion criteria, whether inclusion/exclusion criteria were established prior to data analysis, all manipulations, and all measures in the study. No part of the study procedures or analyses was preregistered prior to the research being undertaken. Due to policy of the research group, public archiving of data and digital study materials will be done at the time our institution makes available an institutional repository for that purpose. Until then, readers seeking access to the data should contact the corresponding author. REFERENCES Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. (2006). Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron, 50(3), 507-517. doi: 10.1016/j.neuron.2006.03.036 Anderson, B. A. (2013). A value-driven mechanism of attentional selection. Journal of Vision, 13(3), 7-7. doi:10.1167/13.3.7 Anderson, B. A. (2015). Value-driven attentional capture is modulated by spatial context. Visual Cognition, 23(1-2), 67-81. doi: 10.1080/13506285.2014.956851 Anderson, B. A. (2016). The attention habit: how reward learning shapes attentional selection. Annals of the New York Academy of Sciences, 1369(1), 24-39. doi: 10.1111/nyas.12957 Post-print (final draft post-refereeing) 34 Anderson, B. A., Kuwabara, H., Wong, D. F., Gean, E. G., Rahmim, A., Brašic, J. R., … Yantis, S. (2016). The role of dopamine in value-based attentional orienting. Current Biology, 26(4), 550-555. doi: 10.1016/j.cub.2015.12.062 Anderson, B. A., Kuwabara, H., Wong, D. F., Roberts, J., Rahmim, A., Brašic, J. R., & Courtney, S. M. (2017). Linking dopaminergic reward signals to the development of attentional bias: A positron emission tomographic study. Neuroimage, 157, 2733. doi: 10.1016/j.neuroimage.2017.05.062 Anderson, B. A., Laurent, P. A., & Yantis, S. (2011). Value-driven attentional capture. Proceedings of the National Academy of Sciences, U.S.A., 108, 10367-10371. doi: 10.1073/pnas.1104047108 Anderson, B. A., Laurent, P. A., & Yantis, S. (2014). Value-driven attentional priority signals in human basal ganglia and visual cortex. Brain Research, 1587, 88-96. doi: 10.1016/j.brainres.2014.08.062 Barbaro, L., Peelen, M. V., & Hickey, C. (2017). Valence, not utility, underlies rewarddriven prioritization in human vision. Journal of Neuroscience, 37(43), 1043810450. doi: 10.1523/JNEUROSCI.1128-17.2017 Bayer, M., Rossi, V., Vanlessen, N., Grass, A., Schacht, A., & Pourtois, G. (2017). Independent effects of motivation and spatial attention in the human visual cortex. Social Cognitive and Affective Neuroscience, 12(1), 146-156. doi:10.1093/scan/nsw162 Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B, 57, 289–300. http://www.jstor.org/stable/2346101. Bissonette, G. B., Gentry, R. N., Padmala, S., Pessoa, L., & Roesch, M. R. (2014). Impact of appetitive and aversive outcomes on brain responses: Linking the Post-print (final draft post-refereeing) 35 animal and human literatures. Frontiers in Systems Neuroscience, 8, 24. doi:10.3389/fnsys.2014.00024 Brignani, D., Lepsien, J., & Nobre, A. C. (2010). Purely endogenous capture of attention by task-defining features proceeds independently from spatial attention. NeuroImage, 51(2), 859-866. doi: 10.1016/j.neuroimage.2010.03.029 Brooks, A. M., & Berns, G. S. (2013). Aversive stimuli and loss in the mesocorticolimbic dopamine system. Trends in Cognitive Sciences, 17(6), 281286. doi:10.1016/j.tics.2013.04.001 Bucker, B., & Theeuwes, J. (2016). Appetitive and aversive outcome associations modulate exogenous cueing. Attention, Perception & Psychophysics, 78(7), 22532265. doi:10.3758/s13414-016-1107-6 Chelazzi, L., Eštočinová, J., Calletti, R., Lo Gerfo, E., Sani, I., Della Libera, C., & Santandrea, E. (2014). Altering spatial priority maps via reward-based learning. Journal of Neuroscience, 34(25), 8594-8604. doi:10.1523/JNEUROSCI.027714.2014 Chelazzi, L., Perlato, A., Santandrea, E., & Della Libera, C. (2013). Rewards teach visual selective attention. Vision Research, 85, 58-72. doi:10.1016/j.visres.2012.12.005 Della Libera, C., & Chelazzi, L. (2006). Visual selective attention and the effects of monetary rewards. Psychological Science, 17(3), 222-227. doi: 10.1111/j.14679280.2006.01689.x Della Libera, C., & Chelazzi, L. (2009). Learning to attend and to ignore is a matter of gains and losses. Psychological Science, 20(6), 778-784. doi: 10.1111/j.14679280.2009.02360.x Post-print (final draft post-refereeing) 36 Doallo, S., Patai, E. Z., & Nobre, A. C. (2013). Reward associations magnify memorybased biases on perception. Journal of Cognitive Neuroscience, 25(2), 245-257. doi:10.1162/jocn_a_00314 Donohue, S. E., Hopf, J-M., Bartsch, M. V., Schoenfeld, M. A., Heinze, H-J., & Woldorff, M. G. (2016). The rapid capture of attention by rewarded objects. Journal of Cognitive Neuroscience, 28(4), 529-541. doi: 10.1162/jocn_a_00917 Eimer, M. (1996). The N2pc component as an indicator of attentional selectivity. Electroencephalography and Clinical Neurophysiology, 99(3), 225-234. doi:S0921884X96957112 Engelmann, J. B., Damaraju, E., Padmala, S., & Pessoa, L. (2009). Combined effects of attention and motivation on visual task performance: transient and sustained motivational effects. Frontiers in Human Neuroscience, 3, 4. doi: 10.3389/neuro.09.004.2009 Failing, M., & Theeuwes, J. (2018). Selection history: How reward modulates selectivity of visual attention. Psychonomic Bulletin & Review, 25, 514-538. doi: 10.3758/s13423-017-1380-y Ghazizadeh A., Griggs, W., Leopold, D. A., & Hikosaka, O. (2018). Temporalprefrontal cortical network for discrimination of valuable objects in long-term memory. Proceedings of the National Academy of Sciences of the United States of America, 115(9), 2135-2144. doi: 10.1073/pnas.1707695115 Griggs, W. S., Amita, H., Gopal, A., & Hikosaka, O. (2018). Visual neurons in the superior colliculus discriminate many objects by their historical values. Frontiers in Neuroscience, 12, 396. doi: 10.3389/fnins.2018.00396 Post-print (final draft post-refereeing) 37 Hammerschmidt, W., Sennhenn-Reulen, H., & Schacht, A. (2017). Associated motivational salience impacts early sensory processing of human faces. NeuroImage, 156, 466-474. doi:S1053-8119(17)30325-7 Harris, J. A., Donohue, S. E., Schoenfeld, M. A., Hopf, J-M., Heinze, H-J., & Woldorff, M. G. (2016). Reward-associated features capture attention in the absence of awareness: Evidence from object-substitution masking. NeuroImage, 137, 116123. doi: 10.1016/j.neuroimage.2016.05.010 Heinze, H. J., Mangun, G. R., Burchert, W., Hinrichs, H., Scholz, M., Münte, T. F., … Hillyard, S. A. (1994). Combined spatial and temporal imaging of brain activity during visual selective attention in humans. Nature, 372(6506), 543-546. doi: 10.1038/372543a0 Hickey, C., Chelazzi, L., & Theeuwes, J. (2010). Reward changes salience in human vision via the anterior cingulate. Journal of Neuroscience, 30(33), 11096-11103. doi: 10.1523/JNEUROSCI.1026-10.2010 Hickey, C., Chelazzi, L., & Theeuwes, J. (2014). Reward-priming of location in visual search. PLoS One, 9(7), e103372. doi: 10.1371/journal.pone.0103372 Hickey, C., Di Lollo, V., & McDonald, J. J. (2009). Electrophysiological indices of target and distractor processing in visual search. Journal of Cognitive Neuroscience, 21(4), 760-775. doi:10.1162/jocn.2009.21039 Hickey, C., Kaiser, D., & Peelen, M. V. (2015). Reward guides attention to object categories in real-world scenes. Journal of Experimental Psychology: General, 144, 264-273. doi: 10.1037/a0038627 Hickey, C., & Peelen, M. (2015). Neural mechanisms of incentive salience in naturalistic human vision. Neuron, 85, 512-518. doi: 10.1016/j.neuron.2014.12.049 Post-print (final draft post-refereeing) 38 Hickey, C., & Peelen, M. V. (2017). Reward selectively modulates the lingering neural representation of recently attended objects in natural scenes. Journal of Neuroscience, 37(31), 7297-7304. doi: 10.1523/JNEUROSCI.0684-17.2017 Hilimire, M. R., Mounts, J. R., Parks, N. A., & Corballis, P. M. (2010). Event-related potentials dissociate effects of salience and space in biased competition for visual representation. PloS One, 5(9), e12677. doi: 10.1371/journal.pone.0012677 Hillyard, S. A., Vogel, E. K., & Luck, S. J. (1998). Sensory gain control (amplification) as a mechanism of selective attention: electrophysiological and neuroimaging evidence. Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences, 353(1373), 1257-1270. doi: 10.1098/rstb.1998.0281 Hopf, J. M., Luck, S. J., Girelli, M., Hagner, T., Mangun, G. R., Scheich, H., & Heinze, H. J. (2000). Neural sources of focused attention in visual search. Cerebral Cortex, 10(12), 1233-1241. Hopf, J. M., Vogel, E., Woodman, G., Heinze, H. J., & Luck, S. J. (2002). Localizing visual discrimination processes in time and space. Journal of Neurophysiology, 88(4), 2088-2095. doi:10.1152/jn.2002.88.4.2088 Itthipuripat, S., Cha, K., Rangsipat, N., & Serences, J. T. (2015). Value-based attentional capture influences context-dependent decision-making. Journal of Neurophysiology, 114(1), 560-569. doi: 10.1152/jn.00343.2015 Jerde, T. A., Merriam, E. P., Riggall, A. C., Hedges, J. H., & Curtis, C. E. (2012). Prioritized maps of space in human frontoparietal cortex. Journal of Neuroscience, 32(48), 17382-17390. doi: 10.1523/JNEUROSCI.3810-12.2012 Kahnt, T., Park, S. Q., Haynes, J. D., & Tobler, P. N. (2014). Disentangling neural representations of value and salience in the human brain. Proceedings of the Post-print (final draft post-refereeing) 39 National Academy of Sciences of the United States of America, 111(13), 50005005. doi: 10.1073/pnas.1320189111 Kim, H. F., Ghazizadeh, A., & Hikosaka, O. (2015). Dopamine neurons encoding longterm memory of object value for habitual behavior. Cell, 163(5), 1165-1175. doi: 10.1016/j.cell.2015.10.063 Kiss, M., Driver, J., & Eimer, M. (2009). Reward priority of visual target singletons modulates ERP signatures of attentional selection. Psychological Science, 20(2), 245-251. doi: 10.1111/j.1467-9280.2009.02281.x Kuo, B. C., Rao, A., Lepsien, J., & Nobre, A. C. (2009). Searching for targets within the spatial layout of visual short-term memory. Journal of Neuroscience, 29(25), 8032-8038. doi:10.1523/JNEUROSCI.0952-09.2009 LaBar, K. S., & Cabeza, R. (2006). Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience, 7(1), 54-64. doi: 10.1038/nrn1825 Lim, S. L., Padmala, S., & Pessoa, L. (2009). Segregating the significant from the mundane on a moment-to-moment basis via direct and indirect amygdala contributions. Proceedings of the National Academy of Sciences of the United States of America, 106(39), 16841-16846. doi:10.1073/pnas.0904551106 Liu, X., Hairston, J., Schrier, M., & Fan, J. (2011). Common and distinct networks underlying reward valence and processing stages: A meta-analysis of functional neuroimaging studies. Neuroscience & Biobehavioral Reviews, 35(5), 1219-1236. doi:10.1016/j.neubiorev.2010.12.012 Luck, S. J. (1995). Multiple mechanisms of visual-spatial attention: Recent evidence from human electrophysiology. Behavioural Brain Research, 71(1-2), 113-123. Post-print (final draft post-refereeing) 40 Luck, S. J., & Hillyard, S. A. (1994). Spatial filtering during visual search: Evidence from human electrophysiology. Journal of Experimental Psychology. Human Perception and Performance, 20(5), 1000-1014. MacLean, M. H., & Giesbrecht, B. (2015). Neural evidence reveals the rapid effects of reward history on selective attention. Brain Research, 1606, 86-94. doi: 10.1016/j.brainres.2015.02.016 Martínez, A., Anllo-Vento, L., Sereno, M. I., Frank, L. R., Buxton, R. B., Dubowitz, D. J., … Hillyard, S. A. (1999). Involvement of striate and extrastriate visual cortical areas in spatial attention. Nature Neuroscience, 2(4), 364-369. doi: 10.1038/7274 Mohanty, A., Gitelman, D. R., Small, D. M., & Mesulam, M. M. (2008). The spatial attention network interacts with limbic and monoaminergic systems to modulate motivation-induced attention shifts. Cerebral Cortex, 18(11), 2604-2613. doi: 10.1093/cercor/bhn021 Montoya, A. K., & Hayes, A. F. (2017). Two-condition within-participant statistical mediation analysis: A path-analytic framework. Psychological Methods, 22(1), 627. doi: 10.1037/met0000086 Murty, V. P., LaBar, K. S., & Adcock, R. A. (2012). Threat of punishment motivates memory encoding via amygdala, not midbrain, interactions with the medial temporal lobe. Journal of Neuroscience, 32(26), 8969-8976. doi: 10.1523/JNEUROSCI.0094-12.2012 Murty, V. P., Ritchey, M., Adcock, R. A., & LaBar, K. S. (2010). fMRI studies of successful emotional memory encoding: A quantitative meta-analysis. Neuropsychologia, 48(12), 3459-3469. doi: 10.1016/ j.neuropsychologia.2010.07.030 Post-print (final draft post-refereeing) 41 O'Brien, J. L., & Raymond, J. E. (2012). Learned predictiveness speeds visual processing. Psychological Science, 23(4), 359-363. doi:10.1177/0956797611429800 Padmala, S., & Pessoa, L. (2008). Affective learning enhances visual detection and responses in primary visual cortex. Journal of Neuroscience, 28(24), 6202-6210. doi:10.1523/JNEUROSCI.1233-08.2008 Patai, E. Z., Doallo, S., & Nobre, A. C. (2012). Long-term memories bias sensitivity and target selection in complex scenes. Journal of Cognitive Neuroscience, 24(12), 2281-2291. doi:10.1162/jocn_a_00294 Pessoa, L. (2015). Multiple influences of reward on perception and attention. Visual Cognition, 23(1-2), 272-290. doi:10.1080/13506285.2014.974729 Pollmann, S., Estocinova, J., Sommer, S., Chelazzi, L., & Zinke, W. (2016). Neural structures involved in visual search guidance by reward-enhanced contextual cueing of the target location. NeuroImage, 124(Pt A), 887-897. doi:10.1016/j.neuroimage.2015.09.040 Ptak, R. (2012). The frontoparietal attentional network of the human brain: action, saliency, and a priority map of the environment. The Neuroscientist, 18(5), 502515. doi: 10.1177/1073858411409051 Qi, S., Zeng, Q., Ding, C., & Li, H. (2013). Neural correlates of reward-driven attentional capture in visual search. Brain Research, 1532, 32-43. doi: 10.1016/j.brainres.2013.07.044 Raymond, J. E., & O'Brien, J. L. (2009). Selective visual attention and motivation: The consequences of value learning in an attentional blink task. Psychological Science, 20(8), 981-988. doi:10.1111/j.1467-9280.2009.02391.x Post-print (final draft post-refereeing) Figure1 Click here to download high resolution image Post-print (final draft post-refereeing) Figure 2 Click here to download high resolution image Post-print (final draft post-refereeing) Figure 3 Click here to download high resolution image Post-print (final draft post-refereeing) Figure 4 Click here to download high resolution image Post-print (final draft post-refereeing) Suppl. material for online publication only Click here to download Suppl. material for online publication only: Supplementary data.docx Post-print (final draft post-refereeing)