Behavioural and neurophysiological signatures in the retrieval of individual memories of recent and remote real-life routine episodic events

Research Report Behavioural and neurophysiological signatures in the retrieval of individual memories of recent and remote real-life routine episodic events Berta Nicol as a,b,c , Xiongbo Wu a,b,c , Josu e Garcı ´a-Arch a,b,c , Mariella Dimiccoli d , Joanna Sierpowska e,l , Cristina Saiz-Masvidal f,g , Carles Soriano-Mas f,h,i , Petia Radeva j,k and Lluı ´s Fuentemilla a,b,c,* a Cognition and Brain Plasticity Group, Bellvitge Biomedical Research Institute-IDIBELL, Hospitalet de Llobregat, Spain b Department of Cognition, Development and Educational Psychology, University of Barcelona, Barcelona, Spain c Institute of Neurosciences, University of Barcelona, Spain d Institut de Rob otica i Inform atica Industrial (CSIC-UPC), Barcelona, Spain e Radboud University, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, the Netherlands f Bellvitge Biomedical Research Institute-IDIBELL, Barcelona, Spain g Department of Clinical Sciences, University of Barcelona, Spain h CIBER Salud Mental (CIBERSAM), Spain i Department of Psychobiology and Methodology in Health Sciences, Universitat Aut onoma de Barcelona, Spain j Department of Mathematics and Computer Science, University of Barcelona, Spain k Computer Vision Center, Spain l Radboud University Medical Center, Donders Institute for Brain Cognition and Behaviour, Department of Medical Psychology, Nijmegen, the Netherlands article info Article history: Received 4 February 2021 Reviewed 12 March 2021 Revised 1 April 2021 Accepted 12 April 2021 Action editor Michael Kopelman Published online 30 April 2021 Keywords: Autobiographical memory Theta rhythm EEG ERPs Wearable camera abstract Autobiographical memory (AM) has been largely investigated as the ability to recollect specific events that belong to an individual's past. However, how we retrieve real-life routine episodes and how the retrieval of these episodes changes with the passage of time remain unclear. Here, we asked participants to use a wearable camera that automatically captured pictures to record instances during a week of their routine life and implemented a deep neural network-based algorithm to identify picture sequences that represented episodic events. We then asked each participant to return to the lab to retrieve AMs for single episodes cued by the selected pictures 1 week, 2 weeks and 6e14 months after encoding while scalp electroencephalographic (EEG) activity was recorded. We found that participants were more accurate in recognizing pictured scenes depicting their own past than pictured scenes encoded in the lab, and that memory recollection of personally experienced events rapidly decreased with the passing of time. We also found that the retrieval of real-life picture cues elicited a strong and positive ‘ERP old/new effect’ over frontal regions and that the magnitude of this ERP effect was similar throughout memory *Corresponding author. Department of Cognition, Development and Educational Psychology, University of Barcelona, Pg Vall Hebr on, 171, s/n, 08035, Barcelona, Spain. E-mail address: [email protected] (L. Fuentemilla). Available online at www.sciencedirect.com ScienceDirect Journal homepage: www.elsevier.com/locate/cortex cortex 141 (2021) 128e143 https://doi.org/10.1016/j.cortex.2021.04.006 0010-9452/©2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http:// creativecommons.org/licenses/by-nc-nd/4.0/). tests over time. However, we observed that recognition memory induced a frontal theta power decrease and that this effect was mostly seen when memories were tested after 1 and 2 weeks but not after 6e14 months from encoding. Altogether, we discuss the implications for neuroscientific accounts of episodic retrieval and the potential benefits of developing individual-based AM exploration strategies at the clinical level. ©2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). 1. Introduction Autobiographical memories (AMs) are specific individualized compilations of our personal past daily life episodic experiences. The ability to recollect detailed information about past autobiographical events is a hallmark of episodic memory (Tulving, 2002). However, the vast majority of behavioural and neuroimaging studies of episodic retrieval have used laboratory-encoded stimuli, such as words or pictures, as memory probes. While such stimuli provide researchers with tight experimental control over the perceptual qualities, exposure duration, and retention interval of the events being tested, laboratory stimuli lack the richness of most real-world experiences (Chow et al., 2018;Chow &Rissman, 2017; Diamond &Levine, 2020;Nielson et al., 2015;St. Jacques et al., 2011). Thus, it is not unsurprising that performance on standard laboratory-based memory tasks may be largely unrelated to one's autobiographical retrieval abilities, as demonstrated by individuals with “highly superior autobiographical memory”(LePort et al., 2012,2017;Patihis et al., 2013) and “severely deficient autobiographical memory'' Fuentemilla, Palombo, & Levine, 2018;Palombo et al., 2015). A hallmark in the advancement of our understating of how episodic memory serves to retrieve real-life autobiographical experiences would be to have methods that allowed the automatic recording of daily life episodes prospectively at the individual level. Through such an approach, researchers would have the opportunity to examine an exhaustive collection of realistic real-life experience material of an individual ahead of sampling control during encoding. Previous research efforts proved effective in cueing AMs sampled at the individual level. Most notably, the use of self-recorded audiotapes or videos documenting selected real-life event experiences has helped characterize the involvement of a core brain network supporting the retrieval of AMs including the medial temporal lobe and the frontal and parietal regions (Levine et al., 2004;Svoboda &Levine, 2009), coordinated via neural oscillatory mechanisms in the range of the theta band (4e8 Hz) (Fuentemilla et al., 2014). However, this approach requires individuals to actively record selected experiences during their daily life routine, and therefore the effectiveness of the retrieval cues may still be partially explained by additional processes engaged during encoding such as selection, organization, and rehearsal of the recorded material. The recent incorporation of portable technology, such as wearable cameras, to study the cognitive and neural basis of AM retrieval appeared to be a promising venue for addressing the previous concern. This technology allows the automatic capture (e.g., every 30 sec) of face-front sequence of pictures of daily life activity without the need for the participant to be actively engaged in the recording process. Researchers have already shown that the presentation of pictures acquired with a portable camera engaged the core AM retrieval network (Cabeza et al., 2004;Rissman et al., 2016) and elicited a strong sense of first-person retrieval in participants, even when they were confronted with others'pictures depicting the same content (St. Jacques et al., 2011). The use of pictures collected from a wearable camera has also been shown to be a valuable approach to enhance AM retrieval in healthy young and older adults (Chow &Rissman, 2017;Xu et al., 2020) and in patients with memory impairments (Alle et al., 2017), with Alzheimer's disease (Woodberry et al., 2014) and with limbic encephalitis (Berry et al., 2007,2009). While the specific neural mechanisms that make picture cues taken in the real world by participants such a powerful retrieval cue remains elusive, functional neuroimaging studies highlighted that this type of cue may engage neural correlates of processes that are difficult to study using laboratory stimuli, including complex constructive processes, recollective qualities of emotion and vividness, and remote memory retrieval (Cabeza &St Jacques, 2007). In the current study, we sought to use this technology to investigate the neurophysiological underpinnings supporting the retrieval of individual AMs over the passage of time. To this end, we asked healthy participants to retrieve their own AMs cued by pictures taken automatically (i.e., every 30 sec) by a wearable camera carried during one week of daily life routine, after 1e2weeks,and6e14 months from the encoding period. To ensure pictures presented during the test cued most of the episodic events that unfolded during the encoding week, we implemented a convolutional network-based algorithm (Dimiccoli et al., 2015) on the entire recorded picture set that automatically grouped together temporally adjacent images sharing contextual and semantic attributes, akin to how we conceive what underlies an event episode from a perception and memory perspective (Zacks &Swallow, 2007; see also Jeunehomme &D'Argembeau, 2018 and 2019). In doing so, the large picture set collected reflecting an entire day's life activity (e.g., ~400 pictures) is grouped into a workable number of picture subsets (e.g., ~20) depicting sequences of temporally adjacent episodic events (e.g., breakfast at home, commuting to work, buying oranges in the corner shop, eating a sandwich at the park). We reasoned that by picking a representative picture from each of the subsets, it would then be possible to investigate whether an individual is capable of retrieving information about a single past episodic event. Additionally, the same participants were asked to enrol in a separate study that required them to encode and retrieve, one week after the fact, cortex 141 (2021) 128e143 129 pictures depicting indoor and outdoor scenes. This task, akin to standard lab-based experimental scenarios commonly used in memory research, was thought to help delineate differences between retrieval processes for when participants'memory was cued by real-life autobiographical versus lab-based event experiences. Complementing the behavioural data, we also aimed to analyse well-known neural response activity widely studied in the context of recognition memory in humans, the EventRelated Potentials (ERPs) and neural oscillations. ERPs have been employed to study the neural correlates of successful retrieval since the early 1990s. These studies have consistently identified a retrieval-related effect that takes the form of more positive-going ERPs for correctly classified old recognition test items relative to new items. In general, ERPs are more positive 300 msec after presentation of correctly recognized studied items compared to correctly reject non-studied items. This difference has been termed the ‘ERP old/new effect’. Several groups have found spatially and temporally distinct ERP modulations within the old/new effect that appear to be specifically associated with the retrieval of items with contextual information (i.e., recollection) or with a lack of such contextual details (i.e., familiarity). An early (~300e500 msec) component, distributed over frontal electrodes, is correlated with familiarity while a later (~500e800 msec) component, distributed over parietal electrodes, appears modulated by recollection (Curran, 2000; Curran et al., 2001;Duzel et al., 1997;Wilding &Rugg, 1996). An additional later (~600e1600 msec) and positive sustained ERP modulation, in most prominent frontal scalp sites, has been described in recognition memory studies. This modulation has been observed in a number of studies and is thought to be related to post-retrieval processing (Allan &Rugg, 1997; Curran et al., 2001;Donaldson &Rugg, 1999;Ranganath & Paller, 2000;Wolk, 2006). In the context of the current study, we hypothesized that ERP signatures could provide information about the participants'ability to retrieve contextual information associated with individual real-life episodic events and how the retrieval of these memory events changed over time from encoding. Indeed, the notion that memory representations change over time has concerned psychologists and neuroscientists for decades. A widely accepted view is that initial encoding of experiences renders temporary, labile memory and that it may become transformed into a more stable, long-lasting form via systems consolidation mechanisms. Systems consolidation refers to the gradual reorganization of the brain systems that support memory (Dudai &Morris, 2013;Squire & Alvarez, 1995). By this process, the hippocampus gradually becomes less important for storage and retrieval, and a more permanent memory develops in distributed regions of the neocortex. This view has received support from studies showing that memory recollection is supported by the coordinated activity of hippocampus and neocortical regions via neural oscillations at the theta band (~3e8Hz)(Fuentemilla et al., 2014;Nyhus &Curran, 2010;Herweg et al., 2016). While this perspective aligns well with the findings that successful retrieval would be accompanied by an increase in theta power during retrieval (Burgess &Gruzelier, 1997; Gruber et al., 2008;Guderian &Du ¨zel, 2005;Klimesch et al., 2001;Osipova et al., 2006), recent studies suggested that this picture remains controversial (see, Herweg, Solomon, Kahana, &Kahana, 2020 for a recent discussion on this topic) as other studies have shown the opposite effect, that theta power decreased during memory retrieval (Hanslmayr et al., 2010;Khader &R€ osler, 2010;Staudigl et al., 2010). Thus, while the specificity of the theta rhythm in retrieving past memories is a prevalent one in the literature, many questions still remain unclear. In the current study, we sought to investigate the role of theta rhythm during retrieval by examining neural oscillatory power changes upon successful retrieval of individual AM from real life. If our experimental approach is suitable to examine AM retrieval, we expect that participant’s ability to recollect detailed and rich contextual information from specific encoded experienced episodes to decrease with the passage of time. To the extent that the hypothesized behavioural effects were related to neural mechanisms associated with memory retrieval, we expect they should be accompanied by changes at the ERPs and neural oscillations at the theta band level. 2. Material and methods 2.1. Participants Sixteen healthy participants (8 females) participated in experiments 1 and 2 (see Rissman et al., 2016 and Bonnici et al., 2012 for similar sample sizes). The range in age was 22-37 years old (mean ¼27.68, SD ¼4.22). All participants provided informed written consent for the protocol approved by the Ethics Committee of the University of Barcelona. Participants received financial compensation for their participation. Two of the participants could not complete the follow-up test in experiment 1 (see details below) and these participants were excluded from all analyses. We report how we determined our sample size, all data exclusions (if any), all inclusion/exclusion criteria, whether inclusion/exclusion criteria were established prior to data analysis, all manipulations, and all measures in the study. However, no part of the study procedures or analyses was pre-registered prior to the research being conducted. 2.2. Experiment 1: retrieval of real-life memories 2.2.1. Design overview Participants were asked to cometo the training session a few days before starting the study. They were informed about how the camera worked and they read, understood, and signed the informed consent. We took special care in providing details on privacy issues so that all participants were fully aware of them before the study began. In the current study, participants wore the wearable camera for a period of 5e7 consecutive days (mean ¼6.64 days, SD ¼.63), including both week and weekend days. The data collection period was established from morning to evening (between 12 and 14 h per day). Once participants finished data collection, they were requested to return the materials to be processed. Participants confirmed not having checked the cortex 141 (2021) 128e143130 Fig. 1 eExperimental design. (A) Example of a stream of pictures obtained from one participant after one week of data collection. The implementation of the SR-Clustering algorithm allows the automatic organization of picture sequences into a set of meaningful events by the identification of similar context and semantic features. (B) Experimental design. One representative photo from each event sequence of pictures was selected and distributed to each of the three memory tests that followed the picture collection week. A recognition memory test was implemented one week (T1), two weeks (T2) and from 6 to 14 months (Follow-up test (FU)) after the data collection. Events extracted from picture sequences were numbered consecutively and pictures related to even numbered events were used as memory cues in T1 test and odd numbered event pictures in T2 test. Different pictures from same events tested in T1 and T2 were selected as cues in FU test. (C) Recognition memory task. Pictures were presented on the screen for 3000 msec. Afterward, an ‘Old/New’ question appeared on the screen. If the pictures were seen as ‘Old’, a “Remember/Know/Guess”appeared on the screen. Next, participants were asked to rate their emotional response towards the event indicating whether it was positive or negative on a scale ranging from 2 (maximum positive) to ¡2 (maximum negative), with 0 being neutral emotion. If the picture was seen as ‘New’, a ‘Sure/Not sure’ judgment appeared on the screen. cortex 141 (2021) 128e143 131 pictures during the encoding week. Participants returned to the lab to be tested one week later (T1), two weeks later (T2), and from 6 to 14 months after the last day of data collection (Follow-up, hereafter FU; mean ¼10.87 months, SD ¼2.02 months). See Fig. 1 for a summary of the experimental design. 2.2.2. Wearable camera We used the wearable Narrative clip 2 camera®(http:// getnarrative.com/) with a camera sensor of 8 MP and a resolution of 3264 2448(4:3). The camera was programmed to automatically take images every thirty seconds and produced pictures with an egocentric viewpoint. Participants were instructed to wear the device on a lanyard around the neck. Narrative clip 2 incorporated a downloading app that allowed participants to download the pictures directly to a hard drive. Participants were instructed to not watch the pictures until the experiment finalized. None of the participants reported having done so at the end of the study. 2.2.3. Picture selection We implemented a deep neural network-based algorithm, SR-Clustering, to automatically organize the stream of each participant's pictures into a set of temporally evolving meaningful events (Dimiccoli et al., 2015). The algorithm segments picture sequences into discrete events (e.g., having breakfast in a kitchen, commuting to work, being in a meeting) based on its ability to identify similar contextual and semantic features from the picture stream. The implementation of the SR-Clustering algorithm provided boundaries for a variable number of discrete events for each participant per day, during which 8 to 20 pictures were taken. Each participant's events were then manually inspected and those which displayed non-meaningful episodes (e.g., all pictures were blurred, or when the camera was pointing to the roof or was blocked by clothes) were discarded from the study. Picture events that included faces from peer interactions were excluded from the study to avoid the use of personally relevant memory cues. Three independent experimenters rated and selected the set of event pictures for each participant on the basis of these criteria, and only those events that were consistently selected by the three raters were included in the final set of picture events in the study. Note that the consistency across experimenters was set to ensure that the events captured by the algorithm were meaningful and did not involve implementing a subjective inclusion/ exclusion selection criterion to which events should be included later in the memory test by the experimenters. The variability observed in the number of events between participants reflected the diversity of each participant's daily life activities (e.g., a person working indoors for 8 h results in fewer events compared with people working outdoors). Once the images were organized into discrete events, we selected a representative picture from each event; thereby ensuring most of the past episodic experience was brought into the test. We then numbered the sequence of event pictures and assigned even-numbered pictures to be used as memory cues for test T1 and odd-numbered pictures to be included in the T2 test (Table 1). FU included picture cues used in T1 and T2 in the same proportion. Pictures cues presented to one participant depicting her own past (Old) were also presented to another participant as New images. For each participant, the number of New and Old images was roughly similar. In cases where the number of New images exceeded the number of Old images, we randomly selected a subset of New images. This ensured that differences between Old and New pictures presented to each participant were only based on the image'sdirectlinktoones' past while preserving the rest of the characteristics intact during the test (e.g., angle of view, picture image features, description of routine daily life activities). Old and New images were presented in random order during the experiment. See Fig. 1B for a summary of the experimental design. By design, none of the participants were friends with each other, and we never encountered an instance where two concurrently enrolled participants came into direct contact with one another while wearing their cameras. 2.3. Recognition memory task In the test, pictures were presented on the screen for 3000 msec. Afterwards, when an “Old/New”question appeared on the screen, participants were required to judge whether the picture reflected an event from the participant's own daily life (Old) or was experimentally novel, signalling with the right index and middle fingers, respectively. Next, participants were asked to judge whether they were “Sure/Not sure”when indicated that an image was “New”and “Remember/Know/Guess”when images were seen as “Old”. Participants were instructed that “Guess”referred to when they had no contextual memory reference for what was depicted in the image, but they recognized the content as being from their own life (e.g., viewing one's living room). “Know”was the signal for when the visual content in the picture was highly familiar but the subject could not determine what unfolded in it, perhaps because the eventin the test was part of a routine (e.g., playing football on Thursdays), while “Remember”was the signal for when the picture elicited a vivid memory of that specific event and it could be located in time. Participants'ability to order each event depicted Table 1 eTotal number of photos presented for each participant and retrieval test (Experiment 1). Participant Total number of photos in Test 1 Total number of photos in Test 2 Total number of photos in Follow-up test 1 118 118 192 2 168 158 110 3 200 212 118 4 198 192 80 5 194 196 166 6 246 246 190 7 198 188 124 8 256 210 168 9 204 240 144 10 290 156 138 11 252 188 158 12 238 228 180 13 160 180 142 14 222 238 120 Mean (SD) 210 (45) 196 (36) 145 (33) cortex 141 (2021) 128e143132 in the pictures along the encoding week was tested afterward more concretely, when they were asked to indicate whether the pictures depicted an event that took place at the “beginning, middle or end”of the encoding week as the image appeared on the screen. Finally, to explore the data further, participants were asked to rate the degree to which each of the pictures elicited an emotional response and to indicate whether it was positive or negative on a scale that ranged from 2 (maximum positive) to 2 (maximum negative), with 0 being neutral emotionally. See Fig. 1C for a summary of the recognition memory task. Temporal ratings are not shown in the design overview. 2.4. Experiment 2: retrieval of lab-based memories The experimental design was similar to experiment 1, but differed in that images depicting real-life experiences were replaced by neutral images of indoor and outdoor scenes extracted from previous experiments (e.g., Bunzeck &Du ¨zel, 2006;Fuentemilla et al., 2010). The experimental design involved an encoding phase and a test phase administered after each participant finished the retrieval session at the FU session in experiment 1. In the encoding phase, participants were instructed to indicate whether scene pictures were indoor or outdoor images. There were 80 scenes (40 indoor and 40 outdoor, presented in random order). Each scene was presented for 2000 msec preceded by a 1500 msec fixation period and followed by the text “indoor/outdoor”that prompted participants’ response (responding with the index or middle finger of their right hand). A period of 10 min of rest separated the study from the test phase. In the test phase, a scene picture was presented on the screen for 3000 msec. Afterwards, when an “Old/New”question appeared on the screen, participants were required to judge whether the word was presented in the previous study phase (Old) or was experimentally novel (New) with the right index and middle finger, respectively. The test phase included 160 scene images in total (80 Old and 80 New, randomly presented). Thereafter, as in experiment 1, a confidence judgment task followed. Here, new judgments were followed by “Sure/Not sure”and old judgments were followed by “Remember/Know/ Guess”. Participants were instructed to make confidence judgments following old judgments with respect to their ability to vividly retrieve the contextually associated information related to the image during encoding. They were instructed to respond “Guess”when they were unsure about their previous Old judgment, “Know”when they recognized the scene image but could not retrieve any contextual feature linked to it, and “Remember”when the scene image brought a vivid recollection of the specific context that surrounded the encoding of that particular image during encoding. 2.5. Behavioural data analysis A repeated measures Analysis of Variance (ANOVA) for hits (i.e., Correct Old responses) and Correct Rejections (i.e., Correct New responses), including time of the test (T1, T2 and FU) as a within-subject factor, was implemented to assess for statistical differences. Paired t-test comparisons were used as a post-hoc test. Significance threshold was set at p<.05. Given the dependency between category responses in confidence judgment (Guess, Know, Remember) (Experiments 1 and 2) and emotionality (Experiment 1) measures, we performed a Bayesian ordinal regression fitted using Monte Carlo Markov Chains algorithm via the brms (Bayesian Regression Models using Stan) package in R (Bu ¨rkner, 2017) to assess for changes in the likelihood of participants’ responses as a function of time of the test. In order to deal with the dependency between observations from the same participants and to accommodate the repeated measures study design, we used a multilevel approach within the Bayesian ordinal regression analysis, where time was considered a constant effect and participant was considered a varying effect [commonly known as fixed and random effects but see recommendations about this terminology in Gelman and Hill (2006)]. In addition, we could not assume that our predictor (time) would have the same effect on all response categories (e.g., neutral emotional ratings could increase over time while positive emotional ratings decrease). This was explicitly modelled by allowing for category specific effects (CSE), which imply estimating as many regression coefficients per category specific predictor as possible thresholds (C 1¼2 in our case). However, given that it has been suggested that fitting CSE in cumulative models is problematic (Bu ¨rkner &Vuorre, 2019), we used an adjacent category model instead. For both models, the Multilevel ordinal Bayesian regression was computed with 4 chains [2000 iterations per chain (warmup ¼1000) and weakly informative priors] (Bu ¨rkner, 2017). Both models were compared to their respective nonmultilevel version (without including participants as varying effect) and with their respective null models (intercepts only) with an approximate leave-one-out cross-validation, in which interpretability is similar than the one used to interpret the Akaike's information criterion (Bu ¨rkner &Vuorre, 2019). 2.6. EEG recordings and preprocessing EEG was recorded at a 500 Hz sampling rate (High-pass filter at .016 Hz, notch filter at 50 Hz) from the scalp using a BrainAmp amplifier tin electrodes mounted in an electrocap (Electro-Cap International) located at 29 standard positions (Fp1/2, Fz, F7/8, F3/4, FCz, FC1/2, FC5/6, Cz, C3/4, T3/4, Cp1/2, Cp5/6, Pz, P3/4, T5/ 6, PO1/2, Oz) and at the left and right mastoids. An electrode placed at the lateral outer canthus of the right eye served as an online reference. EEG was re-referenced offline to the linked mastoids. Vertical eye movements were monitored with an electrode at the infraorbital ridge of the right eye (EOG channel). Electrode impedances were kept below 3 kU. EEG was band-pass filtered offline at .1 - 40 Hz. Independent Component Analysis (Delorme &Makeig, 2004) was applied to the continuous EEG data to remove blinks and eye movement artefacts. EEG data from two participants were lost due to technical problems and were not able to be included in the rest of the EEG analysis. 2.7. Event-related potentials (ERPs) analysis The continuous sample EEG data were then epoched into 3100 msec segments (0e3000 msec relative to trial onset), and cortex 141 (2021) 128e143 133 the pre-stimulus interval (100 to 0 msec) was used as the baseline for baseline removal procedure. Trials exceeding ±100 mV in EEG and/or EOG channels within 100 to 3000 msec time window from stimulus onset were rejected offline and not used in ERPs and time-frequency analysis (see details below). For each participant, we obtained trial epochs that were correctly classified by that participant as either Old or New. We were unable to analyse ERP data for “Remember/ Know/Guess”responses separately because of the low number of trials in some of the experimental conditions (please see results below). 2.8. Time-frequency analysis The power of neural oscillatory activity was calculated by means of the continuous complex Morlet wavelet. It is a biologically plausible wavelet modulated by a Gaussian function which depends on the number of cycles the sinusoidal wave segment comprises. In the current study, the cycles of the Morlet wavelets used for convolution ranged from 4 to 10, increasing logarithmically as frequency increased. We adopted this modified wavelet approach to optimize the trade-off between the temporal resolution at lower frequency band and the frequency resolution at the higher frequency band. For all conditions in experiment 1 and 2, time-frequency analysis was carried out on for each participant and at single trial basis for hits in the memory tests, with epochs of 3500 msec time-locked to the presentation of photo starting at 500 msec before its onset. The convolution with Morlet wavelet was conducted for each frequency value from 1 Hz to 40 Hz, with 50 steps increasing logarithmically. Power values for each frequency were averaged across trials for each channel and then baselinecorrected by decibel conversion. 2.9. Cluster-based statistics of the ERP and timefrequency data To account for ERP differences elicited by Old and New pictures, a cluster-based permutation test was used (Maris & Oostenveld, 2007) to identify clusters of significant points in the resulting spatiotemporal 2D matrix (time and electrodes) in a data-driven manner and addressing the multiple-comparison problem by employing a nonparametric statistical method based on cluster-level randomization testing to control for the family-wise error rate. Statistics were computed for each time point, and the spatiotemporal points whose statistical values were larger than a threshold (p<.05, two-tail) were selected and clustered into connected sets on the basis of x, y adjacency in the 2D matrix. The observed cluster-level statistics were calculated by taking the sum of the statistical values within a cluster. Then condition labels were permuted 1000 times to simulate the null hypothesis, and the maximum cluster statistic was chosen to construct a distribution of the cluster-level statistics under the null hypothesis. The nonparametric statistical test was obtained by calculating the proportion of randomized test statistics that exceeded the observed cluster-level statistics. To assess for differences between Old and New conditions at the time-frequency level a similar statistical approach was adopted. However, clusters (p<.05, two-tail) were determined by connected sets of data samples that were contiguous on the basis of temporal, frequency, or spatial adjacency in the 3D matrix. Cluster statistics and null distribution were created following the same approach as for the ERP statistical approach. 3. Results 3.1. Experiment 1 3.1.1. Behavioural results Participants were highly accurate in correctly distinguishing pictures that depicted their own past (Old pictures) from those that belonged to others'past (New pictures) (Fig. 2A). False Alarms [T1 test: Mean (M) ¼.05, Standard Deviation (SD) ¼.05; T2 test: M ¼.03, SD ¼.03; FU test: M ¼.03, SD ¼.02] and Omissions (T1 test: M ¼.08, SD ¼.06; T2 test: M ¼.09, SD ¼.06; FU test: M ¼.11, SD ¼.09) were very rare in all tests. However, a repeated measures ANOVA, including the three-memory test (T1, T2, and FU) as a within-subject factor in the analysis, revealed that hit rate differed significantly across them [F(2,26) ¼4.94, p¼.01] (Fig. 2A). A series of paired t-test comparisons showed that hit rate decreased as a function of time from encoding. Thus, significant differences were found when T1 and FU hit rate were compared [t(13) ¼2.68, p¼.02] but not for T1 and T2 [t(13) ¼1.37, p¼.19], nor T2 and FU [t(13) ¼1.93, p¼.07]. These differences cannot be accounted for by a general decrease in performance over time as participants’ ability to identify New images (i.e., correct rejections) was similar in the three tests [F(1.29, 16.85) ¼2.14, p¼.16]. Correct rejections: T1 test: M ¼.94, SD ¼.05; T2 test: M¼.97, SD ¼.03; FU test: M ¼.97, SD ¼.02. Participants'confidence judgements for Hits for each of the memory tests are displayed in Fig. 2B. The bestimates for the model predicting differences in participants'confidence judgements as a function of time of the memory test [T1 (reference), T2, FU] included the mean, the standard error (SE) and the 95% credible intervals (CrI) of the posterior distribution of each parameter of interest. Category-specific effects are reported for changes over time on participants'responses for “Know”versus “Guess”and for “Remember”versus “Know”. The results showed that the likelihood of “Know” versus “Guess”responses increased [b¼.44, SE ¼.11, CrI (.22, .66)] but that “Remember”versus “Know”responses decreased in T2 compared to T1 [b¼.32, SE ¼.07, CrI (-.45, .18)]. Similarly, the likelihood that participants judged their responses as “Know”versus “Guess”increased in FU test [b¼.27, SE ¼.11, CrI (.05, .48)] and so did decrease the likelihood of “Remember”versus “Know”when compared to T1 [b¼.67, SE ¼.07, CrI (-.82, .53)]. To assess the likelihood that participants confidence judgment responses changed between T2 and FU, we performed the same analysis but replacing the reference category to T2. This analysis showed that the likelihood that participants responded to “Remember”versus “Know”was lower in FU than in T2 [b¼.40, SE ¼.05, CrI (-.50, .31)] while the likelihood that participants responded cortex 141 (2021) 128e143134 “Know”over “Guess”did not change [b¼.12, SE ¼.08, CrI (-.09, .28)]. Altogether, the results indicate that participants’ tendency to rate their confidence judgements as “Know” increased over time. The indicators selected to assess model fit and performance showed that there were no problems in model convergence (all effective sample sizes >1000; all Rhat ¼1; all Pareto k estimates <.05; 0 of 4000 iterations ended with a divergence). We additionally compared this model with a null model (intercepts model) as well as with an alternative model without the inclusion of varying effects for participants ID. The results showed that the inclusion of time as an exogenous variable considerably improved the model fit from the null model (diff. Looic ¼212.2; SEdiff ¼20.9) and that the inclusion of the varying effect of participants ID resulted in a better model fit in comparison with the model without varying effects (diff. Looic ¼193.6; SEdiff ¼20.2). The inclusion of participants ID as a varying effect was also supported by the results on SD (intercept) which showed appreciable variability between participants [95% CrI (.31, .77)]. We next examined if participants'emotional ratings for Hits differed as a function of memory test (T1, T2 and FU). We observed that participants rarely indicated maximum negative (i.e., 2 in the scale) or positive (i.e., þ2) ratings in the tests (negative: M ¼1.14%, SD ¼1.94%; positive: M ¼14.35%, SD ¼13.61%, averaged over the three test). Therefore, we grouped participants ratings 2 and 1 as negative and þ1 and þ2 as positive, leaving 0 as indicating neutral emotion (Fig. 2C). The results showed that the likelihood that participants rated differently neutral and negative responses did not change in T2 compared to T1 [b¼.13, SE ¼.12, CrI (-.36, .10)] but that it decreased the likelihood they rated positive frequently than neutral their choices in T2 in comparison to T1 [b¼.24, SE ¼.12, CrI (-.37, .12)]. When we analyzed the likelihood that participants'emotional ratings changed as a function of memory test, we found that the likelihood of judging the images as neutral versus negative [b¼.19, SE ¼.13, CrI (-.06, .44)] increased while positive versus neutral [b¼.06, SE ¼.06, CrI (-.19, .06)] did not differ in T2 when compared to T1. When we ran the same analysis but replacing T2 instead of T1 as a reference, we found that participants’ likelihood to indicate positive versus neutral [b¼.05, SE ¼.05, CrI (-.13, .04)] and positive versus negative [b¼.05, SE ¼.09, CrI (-.13, .23)] did not change in the FU test when compared to T2. In sum, these results indicate that emotional ratings upon the retrieval of AMs did not change between memory tests. The indicators selected to assess model fit and performance showed that there were no problems in model convergence (all effective sample sizes >1000; all Rhat ¼1; all Pareto k estimates <.05; 0 of 4000 iterations ended with a divergence). We additionally compared this model with a null model (intercepts only) and an alternative model without the inclusion of varying effects for participants ID. The results showed that the inclusion of time as an exogenous variable considerably improved the model fit from the null model (diff. Looic ¼317.9 SEdiff ¼24.7) and that the inclusion of the varying effect of participants ID resulted in a better model fit in comparison with the model without varying effects (diff. Looic ¼332.7 SEdiff ¼25.4). The inclusion of participants’ ID as a varying effect was also supported by the results on SD (intercept) which showed appreciable variability between participants [95% CrI (.42, 1.00)]. Finally, participants'response accuracy to temporal order memory was random (M ¼.49, SD ¼.22), thereby indicating this test was not suitable to capture the participants’ ability to retrieve temporal representations from the tests. 3.1.2. ERPs results Our analytical strategy was to first assess whether Old and New images elicited different patterns of brain activity in the participants. To address this issue, we averaged, at the participant level, the ERPs elicited by Old and New image conditions across tests and ran a cluster-based permutation test between these two conditions. This analysis revealed a significant cluster showing that Old images elicited higher ERP positive amplitude from 400 msec at stimulus onset, which lasted over the rest of the temporal window in the analysis and comprised frontal and central electrodes at the scalp (Fig. 3A). To assess how the identified Old/New ERP effect varied as a function of memory test, we selected the averaged ERP activity from the cluster to a repeated-measures ANOVA including experimental condition (Old, New) and memory test (T1, T2, and FU) as a within-subject factor. As expected, this analysis revealed a main effect of condition [F(1,11) ¼60.24, Fig. 2 eBehavioural data in healthy young participants from Experiment 1. (A) Averaged participants'accuracy and correct rejections in selecting their own events (old) compared to others'events (new) over three time periods (T1: one week, T2: two weeks, FU: 6e14 months). (B) Confidence judgments over three time periods (T1, T2, and FU). (C) Emotional ratings over three time periods (T1, T2, and FU). *indicates p<.05. Error bars represent SEM. cortex 141 (2021) 128e143 135 p<.01] and a significant condition x memory test interaction [F(2,22) ¼6.45, p<.01] (Fig. 3B) but not a main effect of memory test [F(2,22) ¼.79, p¼.47]. However, separate repeatedmeasures ANOVAs for ERP values to Old and New images revealed that none of them showed a statistically significant effect as a function of memory test [Old: F(2,22) ¼.66, p¼.53; New: F(2,22) ¼2.58, p¼.09]. 3.1.3. Time-frequency results Following the ERP analytical strategy, we first implemented a cluster-based permutation test to identify, in a data-driven manner, the existence of a main Old/New difference pattern of neural oscillatory response along the temporal x spatial x spectral dimension. Thus, spectral power measures elicited at the onset of Old and New correct responses were averaged Fig. 3 eERP old/new effect in Experiment 1. (A) Across participants grand-average Event-Related Potentials (ERPs) for Experiment 1 for Old and New conditions at Fz. A cluster-based permutation analysis between the two conditions revealed that Old pictures elicited greater ERPs amplitude than did fronto-central scalp regions. Dashed line indicates the temporal window of significance (p<.05, corrected). (B) Cluster-averaged individual ERP data for each of the three recognition memory tests (T1, T2, and FU) and conditions. *indicates p<.05 (corrected). Error bars represent SEM. Fig. 4 eTheta oscillations in Experiment 1. (A) Group-averaged changes in spectral power over the three periods (T1: 1 week, T2: 2 weeks, and FU: 6e14 months after encoding) at a representative electrode (i.e., Fz) elicited by pictures related to a participant's own personal events (Old) compared to others'events (New). Difference between Old and New conditions is also displayed. The thick white line depicts the timing of the significant cluster between conditions (p<.05, cluster-based permutation test). Decreased theta power was observed in old responses over fronto-central regions of the scalp. (B) Average theta power over three periods of time (T1, T2, and FU) for each condition separately. Significant (p<.05) interaction effect is indicated with a grey star. Error bars represent SEM. (C) Group-averaged differences in spectral power changes elicited by Old stimuli between the three tests distributed over time at a representative electrode (i.e., Fz). cortex 141 (2021) 128e143136 neuroscience of memory encoding and retrieval. Hove, UK: Psychology Press. Rugg, M. D., Mark, R. E., Walla, P., Schloerscheidt, A. M., Birch, C. S., & Allan, K. (1998). Dissociation of the neural correlates of implicit and explicit memory. Nature, 392, 595e598. Squire, L. R., & Alvarez, P. (1995). Retrograde amnesia and memory consolidation: A neurobiological perspective. Current Opinion in Neurobiology, 5, 169e177. Staudigl, T., Hanslmayr, S., & B€ auml, K.-H. T. (2010). Theta oscillations reflect the dynamics of interference in episodic memory retrieval. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience, 30, 11356e11362. St Jacques, P. L., Kragel, P. A., & Rubin, D. C. (2011). Dynamic neural networks supporting memory retrieval. Neuroimage, 57(2), 608e616. Svoboda, E., & Levine, B. (2009). The effects of rehearsal on the functional neuroanatomy of episodic autobiographical and semantic remembering: A functional magnetic resonance imaging study. Journal of Neuroscience, 29(10). Tulving, E. (2002). Episodic memory: From mind to brain. Annual Review of Psychology, 53(1), 1e25. Urbanowitsch, N., Gorenc, L., Herold, C. J., & Schr€ oder, J. (2013). Autobiographical memory: A clinical perspective. Frontiers in Behavioral Neuroscience, 7, 194. Wilding, E. L., & Rugg, M. D. (1996). An event-related potential study of recognition memory with and without retrieval of source. Brain: a Journal of Neurology, 119, 889e905. Wilding, E. L., & Rugg, M. D. (1997a). An event-related potential study of memory for words spoken aloud or heard. Neuropsychologia, 35, 1185e1195. Wilding, E. L., & Rugg, M. D. (1997b). Event-related potentials and the recognition memory exclusion task. Neuropsychologia, 35, 119e128. Winocur, G., & Moscovitch, M. (2011). Memory transformation and systems consolidation. Journal of the International Neuropsychological Society, 17, 766e780. Winocur, G., Moscovitch, M., & Bontempi, B. (2010). Memory formation and long-term retention in humans and animals: Convergence towards a transformation account of hippocampal-neocortical interactions. Neuropsychologia, 48, 2339e2356. Wolk,D.A.,Schacter,D.L.,Lygizosc,M.,Senc,M., Holcombd, P. J., Daffnerc, K. R., & Budsonc, A. E. (2006). ERP correlates of recognition memory: Effects of retention interval and false alarms. Brain Research, 1096,148e162. Woodberry, E., Browne, G., Hodges, S., Watson, P., Kapur, N., & Woodberry, K. (2014). The use of a wearable camera improves autobiographical memory in patients with Alzheimer's Disease. Memory, 23, 340e349. Xu, Q., Zhang, J., Grandjean, J., Tan, C., Subbaraju, V., Li, L., Lee, K. J., Hsieh, P. J., & Lim, J. H. (2020). Neural correlates of retrieval-based enhancement of autobiographical memory in older adults. Scientific Reports, 10, 1447. Zacks, J. M., & Swallow, K. M. (2007). Event segmentation. Current Directions in Psychological Science, 16(2), 80e84. cortex 141 (2021) 128e143 143