What information should I look for again? : Attentional difficulties distracts reading of task assignments

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY-NC-ND 4.0 https://creativecommons.org/licenses/by-nc-nd/4.0/ What information should I look for again? : Attentional difficulties distracts reading of task assignments © 2019 The Authors. Published by Elsevier Inc. Published version Hautala, Jarkko; Loberg, Otto; Azaiez, Najla; Taskinen, Sara; Tiffin-Richards, Simon P.; Leppänen, Paavo H.T. Hautala, J., Loberg, O., Azaiez, N., Taskinen, S., Tiffin-Richards, S. P., & Leppänen, P. H. (2019). What information should I look for again? : Attentional difficulties distracts reading of task assignments. Learning and Individual Differences, 75, Article 101775. https://doi.org/10.1016/j.lindif.2019.101775 2019 Contents lists available at ScienceDirect Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif What information should I look for again? Attentional difficulties distracts reading of task assignments Jarkko Hautala a,⁎ , Otto Loberg b , Najla Azaiez b , Sara Taskinen c , Simon P. Tiffin-Richards d , Paavo H.T. Leppänen b a Niilo Mäki Institute, Jyväskylä, Finland b Department of Psychology, University of Jyväskylä, Jyväskylä, Finland c Department of Mathematics and Statistics, University of Jyväskylä, Jyväskylä, Finland d Department of Education and Psychology, Free University Berlin, Germany ARTICLE INFO Keywords: Reading comprehension Eye movements Dyslexia Attention deficit Latent change scores ABSTRACT This large-scale eye-movement study (N= 164) investigated how students read short task assignments to complete information search problems and how their cognitive resources are associated with this reading behavior. These cognitive resources include information searching subskills, prior knowledge, verbal memory, reading fluency, and attentional difficulties. In this study, the task assignments consisted of four sentences. The first and last sentences provided context, while the second or third sentence was the relevant or irrelevant sentence under investigation. The results of a linear mixed-model and latent change score analyses showed the ubiquitous influence of reading fluency on first-pass eye movement measures, and the effects of sentence relevancy on making more and longer reinspections and look-backs to the relevant than irrelevant sentence. In addition, the look-backs to the relevant sentence were associated with better information search subskills. Students with attentional difficulties made substantially fewer look-backs specifically to the relevant sentence. These results provide evidence that selective look-backs are used as an important index of comprehension monitoring independent of reading fluency. In this framework, slow reading fluency was found to be associated with laborious decoding but with intact comprehension monitoring, whereas attention difficulty was associated with intact decoding but with deficiency in comprehension monitoring. 1. Introduction In the school context, a purposeful reading activity usually starts with task assignment, the proper understanding of which is assumed to be crucial for the entire reading activity (Rouet et al., 2017). For instance, the information need (Taylor, 1968) derived from search prompts is assumed to guide readers to develop productive search queries, evaluate and select relevant search results, and locate and focus on relevant information on the corresponding webpages (Belkin, 2000; Bilal & Kirby, 2002;Leu, Kinzer, Coiro, & Cammack, 2004). While qualitative evidence has suggested that there are large individual differences in interpreting broadly defined essay task assignments (Nelson, 1990), no previous study has investigated how students read simple and unambiguous task assignments, such as Internet search prompts (Zawilinski et al., 2007), or the consequences it has for subsequent reading performance. Understanding task assignments may be especially challenging for students with reading and attention deficits, who have been found to also show subtle reading comprehension deficits (Ghelani, Sidhu, Jain, & Tannock, 2004;Martinussen & Mackenzie, 2015;Miller et al., 2013; Sesma, Mahone, Levine, Eason, & Cutting, 2009). However, little is known about how these students cope with purposeful reading in general and new digital information literacy demands in particular (Ben-Yehudah et al., 2018). Nonetheless, because of the executive challenges in reading non-linear texts, increased difficulty is to be expected (Coiro, 2011;Savolainen, 2002;Taboada & Guthrie, 2006). For these reasons, we investigated how students with and without reading and attentional difficulties read short task assignments containing an important sentence and an unimportant sentence in an informational task that simulated an information search on the Internet as well as the possible implications of this assignment on task performance. https://doi.org/10.1016/j.lindif.2019.101775 Received 29 June 2018; Received in revised form 16 August 2019; Accepted 21 August 2019 ⁎ Corresponding author at: Niilo Mäki Institute, Asemakatu 4, P.O. Box 35, FI-40014 University of Jyväskylä, Jyväskylä, Finland. E-mail addresses: jarkko.v.hautala@jyu.fi,jarkko.hautala@nmi.fi(J. Hautala). Learning and Individual Differences 75 (2019) 101775 1041-6080/ © 2019 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/BY-NC-ND/4.0/). T 1.1. Cognitive basis of reading comprehension and learning difficulties Reading comprehension is built on the proper interplay of several cognitive functions (Bohn-Gettler & Kendeou, 2014;Miller et al., 2013). Fluent decoding skills free an individual's attentional working memory resources for comprehension (Verhoeven & Van Leeuwe, 2008). Prior knowledge (Kendeou & van den Broek, 2007), vocabulary (Calvo, Estevez, & Dowens, 2003), and nonverbal reasoning abilities (Tiu Jr, Thompson, & Lewis, 2003) are important resources for making semantic interpretations and connections between different concepts in a text. The relevant information is stored and further processed in the working memory (Sesma et al., 2009;Swanson, Zheng, & Jerman, 2009). Executive functions regulate the individual's attention to several objects over a lengthy period (Locascio, Mahone, Eason, & Cutting, 2010;McVay & Kane, 2012). Developmental dyslexia affects 4% to 9% of students (see BenYehudah et al., 2018). It is characterized by inaccurate, slow, and dysfluent reading at the word level despite adequate reading instruction and normal intelligence (DSM-V, APA, 2013). Laborious decoding often impairs reading comprehension (Kirby & Savage, 2008). In addition, dyslexia may be associated with additional comorbid subtle deficiencies in working memory (Moll, Gobel, Gooch, Landerl, & Snowling, 2016), and executive function (Locascio et al., 2010). Attention deficit and hyperactivity disorcer (ADHD) is characterized by difficulties with inattention, impulsivity, and hyperactivity (DSM-V, APA, 2013). It affects an estimated 5% to 7% of the population (Willcutt et al., 2010). Children with ADHD often struggle in school, and their academic difficulties extend from childhood to college (Lewandowski, Gathje, Lovett, & Gordon, 2013). Miller et al. (2013) showed that after controlling for word reading abilities, attention deficit still predicted poorer recall of the central ideas of a text. However, this effect is mediated by working memory deficits (Friedman, Rapport, Raiker, Orban, & Eckrich, 2017;Sesma et al., 2009), especially in the verbal domain (Koefler, Rapport, Bolden, Sarver, & Raiker; 2009; Pimperton & Nation, 2010). New evidence has suggested that attention deficits are also associated with impairments in metacognition, especially in planning (Alvarado, Puente, Jiménez, & Arrebillaga, 2011; Pezzica, Vezzani, & Pinto, 2018) and comprehension monitoring (Berthiaume, Lorch, & Milich, 2010). The comorbidity between dyslexia and ADHD is around 30% (Germanò, Gagliano, & Curatolo, 2010;Landerl & Moll, 2010). This comorbidity is associated with a particularly poor prognosis for future academic and behavioral difficulties (DuPaul, Morgan, Farkas, Hillemeier, & Maczuga, 2016). Reading and attention disabilities also share common cognitive deficiencies in the working memory (TiffinRichards, Hasselhorn, Woerner, Rothenberger, & Banaschewski, 2008) and in the executive function (Locascio et al., 2010). 1.2. Purposeful reading Generally, the ability to identify and focus on the relevant part of text for the required purpose of reading (McCrudden & Schraw, 2007) depends on the reader's metacognitive comprehension monitoring abilities (Rouet, Britt, & Durik, 2017), and it takes place typically by the conscious rereading of important parts of text (Cerdán, Gilabert, & Vidal-Abarca, 2011;Hyönä & Nurminen, 2006). The purpose of reading also guides thinking during reading. Think-aloud studies have shown that a specific goal, such as in the present study, makes thinking during reading more focused (i.e., monitoring, repeating, and paraphrasing) and less inferential (i.e., elaborating and predicting) compared to reading for general comprehension (Tilstra & McMaster, 2013; see also van den Broek, Lorch, Linderholm, & Gustafson, 2001). At the level of cognitive schemas, according to the theory of purposeful reading (RESOLV model; Britt, Rouet, & Durik, 2017;Rouet et al., 2017), readers implicitly or partly consciously construct a context model, a task model, and a text model. The context is understood as a set of reader qualities and environmental cues that affect the construction of a highly specific task model, which is used to guide the actual reading strategies and behavior (i.e., to represent the text). The task model consists of goals, plans, and values. The construction of the task model is assumed to be affected by limited processing resources, feelings-of-knowledge (FOKE), benefit-cost analyses, and decision thresholds for different actions. These processes govern evaluations, such as the differentiation of relevant from irrelevant information, noticing the need to reread a certain part of the text, and realizing when the reading goal is satisfied. Crucially, RESOLV predicts that students make minimal yet sufficient task-elaboration (Martinez, Vidal-Abarca, Gil, and Gilabert, 2009) and that the task model constructed has implications for subsequent reading performance. Thus, in reading task assignments, the students' existing schema knowledge becomes activated through the situation (a fictional thought: “This is a research experiment, so I should do the best I can”) and specific parts of the text (e.g., “Find information about…”). Then readers go through several implicit evaluations, such as FOKE (e.g., “Coral reefs are sea environment full of life.”), benefit-cost analyses to calibrate their effort level (e.g., “These tasks will be many so I should work efficiently to avoid spending whole day in this lab.”), and strategic approaches to the task at hand (e.g., “I repeat the objective-sentence to remember it.”). Although the executive demands for reading simple task assignments are rather low, students with attention deficits may nonetheless show aberrant purposeful reading behavior (e.g., “I read instructions fast to get to the real task.”). Moreover, low reading fluency may alter the dynamics of purposeful reading, such as by leading to the increased focus on relevant sentences in order to minimize the overall amount of reading (e.g., “I'll search for the objective-sentence and read that only.”). 1.3. Eye movements reflect reading comprehension processes While the quality of the mental model of a text can be assessed using various outcome measures of reading comprehension, the only nonintrusive way to study the construction of a mental model is to record the readers' eye movements (Rayner, 1998). Sentence-level eye movements in particular have been found to be instrumental in studying reading comprehension processes (Hyönä & Nurminen, 2006). Progressive fixations mainly reflect the efficiency of decoding a novel text (Rayner, 1998), yet small effects of text relevancy or reading purpose have also been found in some studies. For example, Kaakinen, Lehtola, and Paattilammi (2015) found that second graders slowed their progressive fixation durations (approximately 6 ms per word) when they were reading to answer a question compared to reading for general comprehension. However, this task-effect was present only in the later measures of older students and adults. In another study, primary school students showed a text relevancy effect in first-pass measures, including a 5-ms prolongation of the first fixation duration per word for task-relevant vs. irrelevant nouns in the text (Schoot, Vasbinder, Horsley, & Lieshout, 2008). This effect, however, did not correlate with reading comprehension, suggesting that it was merely reflective of the detection of relevant information, not its complete processing. de Leeuw, Segers, and Verhoeven (2016a, 2016b) found that in Dutch children in the fifth grade, shorter gaze duration for all words was associated with better vocabulary and working memory, while longer gaze duration for words in paragraph headings and first sentences specifically was associated with better reading comprehension. Finally, a few studies in adults have documented shorter progressive fixation durations for readers with high prior knowledge on the topic (Calvo, 2005;Kaakinen, Hyönä, & Keenan, 2003). Reinspections are regressive eye movements made during the firstpass reading of a sentence. Slow readers make reinspections for decoding purposes, whereas high comprehenders make reinspections in order to construct a more coherent representation of the entire text (de Leeuw et al., 2016a, 2016b;Kaakinen et al., 2015;Kender & J. Hautala, et al. Learning and Individual Differences 75 (2019) 101775 2 Rubenstein, 1977;Schoot et al., 2008;Yeari, van den Broek, & Oudega, 2015). As a sign of more effortful processing, second-grade children made more reinspections (although of similar duration) in reading to answer a question than in reading for comprehension. In contrast, in adults, reading to answer questions facilitated reading, which led to fewer and shorter reinspections than in reading for comprehension (Kaakinen et al., 2015). Burton and Daneman (2007) reported that metacognitively proficient adult readers with low working memory capability increased their reinspection durations for task-relevant portions of text. In summary, reinspections seem to reflect broadly different types of processing challenges stemming either from difficulties in word recognition, or semantic integration between words, but also due to adoption of different reading strategies depending on the reading purpose. Rereadings or look-backs are thought to reflect a reader's conscious and strategic comprehension monitoring processes (for a review, see Hyönä & Nurminen, 2006;Schotter, Tran, & Rayner, 2014). This process is strongly modulated by the goal of the reading activity. For example, reading for study vs. entertainment (Yeari et al., 2015) or for questions vs. general comprehension (Kaakinen et al., 2015) increased the number of look-backs during reading. In addition, look-backs to a relevant part of the text were associated with better reading comprehension performance (Hyönä, Lorch, & Kaakinen, 2002;Hyönä & Nurminen, 2006). However, Kaakinen et al. (2003) showed that good working memory and prior knowledge of the topic enabled an individual to read a task-relevant portion of the text without devoting extra processing time to it. In a similar vein, Calvo (2005) showed that readers with high vocabulary, access speed, and working memory made fewer look-backs when they read inferential sentences. These findings point to the strategic “on-demand”nature of look-backs, which depends on the quality of both the text and the reader. Taken together, the decoding of linguistic information is reflected by progressive fixation duration and immediate responsiveness to initial comprehension or decoding challenges by reinspection. Late comprehension processes are indexed by look-backs, which reflect a strategic effort to monitor, resolve, or strengthen an individual's representation of certain parts of the text in relation to the purpose of reading. Reinspections and look-backs are typically beneficial for reading comprehension, yet they are initiated on an “on-demand”basis, depending on several reader and task-related factors. 1.4. How do reading and attention difficulties disturb the reading process? While sentence-level eye-movement measures have not been previously studied in learning-disabled populations, it is well known that developmental dyslexia is associated with longer fixation durations and higher numbers of fixations (e.g., Hautala, Hyönä, Aro, & Lyytinen, 2011), shorter saccadic amplitudes during reading, and more reinspections (de Leeuw et al., 2016a, 2016b). These are all manifestations of poor reading fluency and of word decoding problems in particular. Concerning strategic eye movements during reading, de Leeuw et al. (2016a, 2016b) reported that in fourth grade children, better decoding skills was associated with more regressions specifically from a sentence's final words, whereas slower decoders showed longer regression path durations from the final sentences of a paragraph. Although the authors did not interpret these complex interactions, they may have various causes. For example, on one hand, slow decoding speed allows more time for the semantic integration of the words in a sentence. On the other hand, longer amounts of time spent on reading may also result in the memory “leakage”of distant content, which may be recovered by look-backs. We were able to find only two studies on ADHD populations concerning eye movements during reading. Thaler et al. (2009) reported that students with comorbid attention deficit and dyslexia read words with fewer fixations than students with dyslexia only, which led to more errors in reading. In contrast, Deans, O'Laughlin, Brubaker, Gay, and Krug (2010) reported that children with attention deficit aged 6 to 12 years exhibited more regressions and vertical saccades during reading. In summary, there is some evidence of slightly altered eye movement dynamics in purposeful reading due to slow decoding speed, but no previous study has focused on how attentional problems may disturb high-level reading processes. 1.5. The present study To study how students with and without reading and attentional difficulties read simple four-sentence task assignments for an informational task, we recorded their eye movements. Based on this reading, the students needed to select an appropriate search query, a search result, and after repetition of a task objective, to find and report an answer from a static webpage. Although our tasks were not designed to stress prior knowledge and verbal memory, these known predictors of a reader's eye movements and literacy performance were also included. A self-report measure of prior knowledge will be used in order to reflect the theoretical notion that readers base their decisions on their feelings of knowing rather than their “real”prior knowledge (Britt et al., 2017). The following research questions with associated hypotheses were posed: RQ1. Which eye-movement measures reflect purposeful reading processes that are operationalized as the difference in reading taskrelevant vs. irrelevant sentences? The difference may already be present in progressive fixations, but the effect should be pronounced in the probability and duration of reinspections and look-backs (Kaakinen et al., 2015;Schoot et al., 2008), or it should appear only in look-backs (Yeari et al., 2015). RQ2. How do cognitive skills contribute to purposeful and basic reading processes? Reading fluency is expected to dominate first-pass reading measures. In addition, higher FOKE may facilitate the initial reading of sentences, resulting in overall faster progressive fixation durations and fewer and shorter reinspections (Calvo, 2005;Kaakinen et al., 2003). More frequent and longer reinspections and look-backs at the relevant sentence (Kaakinen et al., 2015:Schoot et al., 2008;Yeari et al., 2015) should lead to a more elaborate task model and thus to better information searching performance (Britt et al., 2017). Limited working memory should be associated with an increased number of look-backs to relevant sentence because the reader may rehearse the relevant sentence (Burton & Daneman, 2007). RQ3. How do attentional difficulties interfere with purposeful reading processes? Because of their difficulties in metacognitive comprehension monitoring (Miller et al., 2013), readers with attention difficulties are expected to make fewer look-backs to task-relevant sentences. Alternatively, if they have intact comprehension monitoring abilities but impaired verbal memory, they should make more look-backs and reinspections to relevant sentences. Finally, if they have intact comprehension monitoring and verbal memory but difficulties in relevancy detection, they may exhibit fewer or more reinspections and look-backs in general. 2. Methods 2.1. Participants The participants were 164 sixth-grade primary school students (age M= 12 years 4.2 months, SD = 3.7 months; 98 boys, 66 girls) in central Finland with normal or corrected-to-normal vision. The eye-tracking measurement failed in three additional students, whose data were J. Hautala, et al. Learning and Individual Differences 75 (2019) 101775 3 excluded from the analysis. The students were invited to participate in the study based on the following screening assessments, which were conducted during a large-scale classroom study: reading skill, attentional functioning, and nonverbal IQ performance. Reading skill was assessed using three separate tasks: a word identification task (Lindeman, 1998), a pseudo-word text reading (Eklund, Torppa, Aro, Leppänen, & Lyytinen, 2015), and a word-chain segmentation test (Holopainen, Kairaluoma, Nevala, Ahonen, & Aro, 2004). It should be noted that both lexical and word decoding processes are important in reading Finnish (Eklund et al., 2015). The reading fluency factor was extracted from these three tests by principal axis factoring with a promax rotation (SPSS). All students below the 15th percentile for reading fluency were invited to participate in the study. Attentional functioning was assessed using a 55-item attention deficit questionnaire that was designed to be completed by the teachers (Klenberg, Jämsä, Häyrinen, & Korkman, 2010). Higher scores indicated more attentional problems. All students who scored higher than the 75th standardized percentile were invited to participate in the study. Students with nonverbal reasoning performance (a 15-min, 30-item version of Raven matrices; Raven, Court, & Raven, 1992) below the seventh percentile in the classroom sample were not invited to participate in this study. However, randomly selected students without attentional or reading difficulties who exceeded this criterion were invited to participate (see Table 1 for the number of students who fulfilled the selection criteria). In accordance with the Declaration of Helsinki, we obtained written consent from all students and their caregivers before the study. Ethical approval for this study was received from the Ethical Board of the University of Jyväskylä. 2.2. Task and materials 2.2.1. Verbal memory The digit span test in the Wechsler Intelligence Scale for Children (WISC-IV; Wechsler, 2010) requires students to repeat a list of digits spoken by an instructor in the order or the reverse order that it was presented. The former mode stresses short-term memory, and the latter mode stresses working memory in children (Alloway, Gathercole, & Pickering, 2006;St Clair-Thompson, 2010). The list consists of two to nine items in increasing order with two trials for each stimulus number. The task was discontinued after a mistake of a given length in both trials. The score for the correct answers (maximum score of 32) was used as the outcome score. According to the test manual, Cronbach's alpha reliability was 0.63. 2.2.2. Prior knowledge The information search tasks that were designed were likely not in a sixth grader's prior knowledge, which was confirmed by the results of a self-evaluation questionnaire of feelings of knowing that was completed before the experiment. It included questions such as “How much do you know about the threats to coral reefs?”The answer choices were as follows: 1) I know nothing (38.1% of responses); 2) I know very little (30.5% of responses); 3) I know a little (20.8% of responses); 4) I know something (9.4% of responses); 5) I know a lot about the subject (1.3% of responses). Because of the small number of high prior-knowledge responses, categories 3 through 5 were combined to achieve an evenly distributed three-category scale. Reliability across the 10 tasks was α =0.68. In the analyses prior knowledge is handled as a trial-level predictor. 2.2.3. Information search experiment The students completed 10 tasks that simulated searching for information on the Internet without a time-limit. Each task consisted a sequence of subtasks: 1) reading a four-line text for the task assignment; 2) selecting a search query among five alternatives; 3) selecting a search result among four alternatives; 4) reading a static webpage in which the answer was located at either the beginning or the end of a relevantlytitled paragraph; 5) reporting the answer verbally to a research assistant after leaving the webpage screen. Short instruction screens guided the students through the sequence (e.g., “Good work. Next choose the most appropriate search query for the given information search problem.”). To ensure that even the lowest-performing students could complete the tasks, the critical task assignment sentence was shown again by an instruction screen immediately prior to entering the webpage. To provide some thematic continuation in the lengthy experiment, two successive tasks always shared a common theme (see Appendix A). Only the eye movements during the first subtask (i.e., the task assignment screen) are investigated in the present paper. 2.2.4. Information search subskills score For each of the information search problems, the students received one point for selecting the best search query term, one point for selecting the best search result, and one point for correctly reporting the answer verbally to the research assistant after each information search task. The verbal answers were recorded, transcripted to the text, and scored according to predefined criteria for accurate responses. For example, in the task assignment presented in Fig. 1, the students needed to express two ideas presented on the webpage: miners took Indians' land, and many Indians were killed. The interrater reliability of the scores of the verbal responses was high (α=0.950). Because of the small number of zero scores, these values were included in the score 1 category. Thus, the final scale had three levels: low (20% of responses), medium (43.5% of responses), and high (36.6% of responses). The reliability of this summary scale across the 10 tasks was α=0.707. In the analyses, the performance of the information search was handled as a trial-level predictor. 2.2.5. Task assignment subtask The task assignment screen contained four sentences that were presented on separate lines (see Fig. 1 and Appendix A for all the texts with their English translations). The first and fourth sentences provided a context while the relevant (task objective) and irrelevant yet contextappropriate sentences were presented in counterbalanced positions on lines two and three. The relevant and irrelevant sentences had uniform Table 1 Descriptive results of cognitive measures in groups of students fulfilling the inclusion criteria. Measure Control (C) N= 87 Attentional difficulty (AD) N=28 Reading difficulty (RD) N=23 Comorbid (CM) N= 24 Comparisons M SD M SD M SD M SD Reading fluency (factor score) 0.28 0.85 −0.10 0.77 −1.49 0.39 −1.67 0.49 C = AD > RD = CM Attention difficulty (max = 110) 3.54 4.68 33.8 11.0 4.74 4.69 39.0 39.0 AD = CM > C = RD Prior knowledge (max = 3) 1.94 0.38 1.97 0.45 2.03 0.40 1.79 0.43 n.s. Search query selection (max = 10) 8.00 2.28 7.46 2.15 7.70 1.40 7.75 2.03 n.s. Information search subskills (max = 3) 2.27 0.31 2.15 0.34 2.02 0.25 1.89 0.37 C = AD > RD = CM Verbal memory (max = 32) 15.4 2.45 14.8 2.34 13.0 2.16 13.3 2.08 C = AD > RD = CM J. Hautala, et al. Learning and Individual Differences 75 (2019) 101775 4 lengths measured in characters, t(18) = −0.355, p= .741, and in words, t(18) = −1.55, p= .137. They also had equal mean word frequency, t(18) = −0.874, p= .394 based on a Finnish newspaper corpus (Research Institute for the Languages of Finland, 2007). The text was presented in 24-point Calibri font with 1.5 line spacing (1.15), which is sparser than the minimum accuracy limits of the eye tracker's spatial accuracy of 0.5. After the reading, the students continued to the next subtask by clicking the continue (Jatka) button. 2.3. Apparatus The students' eye movements were measured using a table-mounted EyeLink 1000 eye tracker (SR Research). To achieve high spatial accuracy of the eye-movement recordings, each student's head was stabilized using a chinrest and a forehead rest. The stimuli were presented on a Dell Precision T5500 workstation with an Asus VG-236 (1920 × 1080, 120 Hz, 52 × 29 cm) monitor. The participants viewed the stimuli at a distance of 60 cm. Calibration was performed using a 13-point grid with 1 degree of visual angle as the acceptance criterion. The calibration was conducted before the experiment and then repeated between trials when visible head movements were made, when drift was observed on the researcher's screen, or when the calibration-validation error exceeded 0.30 visual degrees. 2.4. Procedure One research assistant worked with the students in the measurement room, while another assistant controlled the measurement devices in the control room. The students first completed the prior knowledge questionnaire on paper. The information search task then was introduced through a practice task on paper with a research assistant. Next, the table height and forehead rest of eye-tracking system were adjusted, the eye tracker was calibrated, and the students completed one practice task using a mouse. The students then completed the 10 experimental information search tasks, taking one or more short breaks depending on individual needs. The calibration was repeated after each break. The students completed all tasks using a mouse. The experimental session lasted 45 to 90 min, depending on the student. 2.5. Eye-movement data processing Fixations and saccades were identified according to the criterion of 30 degrees/s using the Data Viewer Program (SR Research Ltd., Canada). To reduce the amount of noise from the fixation detection algorithm and low-level saccadic behaviors, such as rapid corrections of landing position errors (glissades), the exclusion criterion for the fixation duration was < 80 ms (2.6%) and > 1200 ms (0.2%). This is the field standard (see Hawelka, Gagl, & Wimmer, 2010). The four predefined areas of interest (AOI) corresponded to the four one-line sentences (Fig. 1). Eye-movement data contain spatial errors (offset), which can be reliably reduced by human correction based on visual inspections (Cohen, 2013). The trained research personnel were unaware of the hypotheses of the study and the qualities of the students. They visually inspected the eye-tracking data to exclude screens with poor-quality eye-tracking data that were beyond repair (2.6% of the screens in the experiment). They also manually repaired systematic offsets in fixation locations on the vertical axis where fixations fall on the wrong side of the AOI boundary. This repair was conducted in 33% of the task assignment screens, which affected 22.1% of fixations. The interrater agreement of whether a trial ought to be corrected between two trained persons for this repair procedure was 91.2%, which was determined using a randomly selected sample of 25 subjects. A pass-size histogram (i.e., the number of fixations during each visit to sentence AOIs) indicated the presence of two distributions: 1) a large number of skimming passes (Campbell & Maglio, 2001) consisting of one or two fixations; 2) the main distribution of proper reading passes, which averaged on eight fixations. Because skimming passes would produce a large error in the identification of the first and subsequent reading passes of the sentence, all passes with only one or two fixations were excluded before the eye-movement measures were calculated using a custom computer program (Hyönä, Kaakinen, & Penttinen, 2000). 1 2.6. Eye-movement measures In this study, the progressive fixation duration is the sum of the firstpass fixations located farther on the x-axis than any of the previous fixations. Reinspections are regressive fixations that fall on a previously read part of a sentence during the first-pass reading. Gaze duration is the sum of the durations of progressive and regressive fixations during the first-pass reading. Look-back fixations are fixations that fall on previously read or skipped sentences. The total fixation duration is the sum of all fixation durations on a sentence. In addition to the fixationduration measures, the probabilities of reinspections and look-backs were calculated. 3. Results 3.1. Descriptive analyses The group comparison based on the inclusion criteria showed no differences between attention deficit and control groups in cognitive measures except on the attention difficulty scale (Table 1). The reading difficulty and comorbid groups had lower scores on information search and verbal memory. The partial correlations after controlling for reading fluency (Table 2) confirmed that attention difficulty scores were not associated with poorer verbal memory in our sample, but they were associated with slightly poorer information search performance (r=−0.212). Higher verbal memory was associated with slightly better information search subskill score (r= 0.175). 3.2. Linear mixed models Separate (generalized) linear mixed-effects models [(G)LMM; Breslow & Clayton, 1993] were fitted for each dependent variable. This approach is the field standard of analysis in the eye-movement research on reading (e.g. Hohenstein, Matuschek, & Kliegl, 2017). Analyses were conducted using an R statistical software engine (version 3.4.2; R Core Team, 2017) in the R Studio environment (version 1.0.136; RStudio Fig. 1. An example of the task assignment overlaid with automatically generated words-specific area-of-interests (AOIs). These provided the basis for calculation of sentence-specific AOIs. 1 The analysis of skimming probability and duration did not produce statistically significant effects except the reading fluency on the skimming duration. J. Hautala, et al. Learning and Individual Differences 75 (2019) 101775 5 Team, 2016). The modeling was conducted using the Analysis of Factorial Experiments (afex) package (Singmann, Bolker, Westfall, & Aust, 2015), which builds on lme4_1.1-14 (Bates et al., 2014) and lmerTest (Kuznetsova, Brockhoff, & Christensen, 2017). The emmeans package (Lenth, 2017) was used to calculate the means and confidence intervals as well as for plotting. The gamma-distributed fixation-duration variables were log-transformed to obtain normal distributions, which reduced the need to fit complex polynomial terms (Hohenstein et al., 2017). Because of the large variance in the dependent variables, log-transformed values smaller or larger than 2.5 standard deviations from the student mean were excluded (de Leeuw et al., 2016a, 2016b). Durations of a single fixation to a sentence and the lowand high-end tails of each dependent variable, which are specified in the results section for each measure, were also excluded. Highly skewed predictors are known to be problematic for LMM (Hohenstein et al., 2017). In our preliminary analyses, we found that the highly skewed attention difficulty measure produced spurious interactions. Therefore, this measure was dichotomized on the 75th standard percentile for LMM analyses (17 points; Klenberg et al., 2010). For continuous dependent variables, all three-way interactions between the within-subject factor of relevancy and between-subject covariates (i.e., information search, prior knowledge, reading fluency, attentional difficulty, and verbal memory) were first included in the maximum model. For the GLMMs with dichotomous dependent variables (i.e., the looking probabilities) to converge, the fixed part of the initial model included only two-way interactions with the sentence-type factor. The models were built according to principles recently suggested in the literature by first fitting a maximal model, reducing it, and then reporting the simplest model that had a fit that was not statistically different from the maximal model (Baayen, Davidson, & Bates, 2008; Barr, 2013;Bates, Kliegl, Vasishth, & Baayen, 2015;Singmann & Kellen, 2017). Type III sum of squares tests were applied, which means that lower-order effects were estimated while considering higher-order effects. Sum contrasts were set for the factor predictors by comparing each level to the overall mean of the factor. To obtain convergence of the maximal models, the correlation parameters between random effects were omitted. A random structure was reduced iteratively by first excluding all the highest-level random factors that had variances estimated at zero and refitting the model. Next, the fixed part of the model was reduced iteratively by dropping all the non-significant fixed effects and their corresponding random terms. In the presence of significant interactions, the model fit was compared to the main effects model, in which interactions were omitted. Finally, the correlation parameters of random effects were added, whenever converging. Table 3 shows the statistically significant parameter estimates in proportional or odd-rate values. Table 4 shows the variances in the included random effects of each model. As shown in Figs. 2 and 3, the results are reported in backtransformed original scales; that is, in milliseconds or probabilities. 3.3. Progressive fixation duration A total of 2870 of 3168 observations (88.6%) were analyzed. Two hundred and thirty-three cases were excluded because the sentence was not fixated at all, 13 were excluded as extreme values (ranging 6 to 9 in the log-transformed values), and the remaining 48 were excluded because they deviated > 2.5 SD from the subject mean. The final model contained only the main effect of reading fluency (Table 3) with one standard deviation increase in reading fluency, which reduced the progressive fixation duration by 19.7% (95% confidence interval [CI] = 15.8%–22.7%) when the progressive fixation duration for the average reader was 1700 ms (Fig. 2). The random variability in the progressive fixation duration was relatively small. The largest random effect was the intercept in individuals with 95% CI of 1633–1781 ms (Table 4). Regarding the first research question, which asked what eye movement measures reflect purposeful reading, the progressive fixation durations clearly did not. The second research question asked which cognitive skills modulate purposeful reading. As expected, progressive fixation durations strongly reflected reading fluency, but not FOKE as one might have expected based on previous studies in adults (Calvo, 2005;Kaakinen et al., 2003). The third research question asked which eye movement measures are sensitive to attentional difficulties. The results showed that the initial progression in reading was similar in students with and without attentional difficulties. 3.4. Reinspection probability To increase the robustness of the measure, two regressive fixations during the first-pass reading were set as the criteria for reinspection classification, which was the case in 1816 of 3168 observations (57.3%). The final model consisted of the main effects of relevancy and reading fluency. Table 3 present parameter estimates and the results of statistical tests, and Fig. 2 estimated marginal means. Relevancy and reading fluency had the following main effects: regressions were more likely when the students read the relevant sentences (69%) than when they read the irrelevant sentences (47%). One SD increase in reading fluency decreased the regression probability by an odds ratio of 0.73 (95% CI = 0.63–0.84) (see Table 3 and Fig. 2). The large random intercept for the students (0.63) suggests large individual differences in the probability of making regressions in general (95% CI = 34–87%), irrespective of the level of reading fluency or sentence relevancy (Table 4). These results indicate that the effect of relevancy first manifests in reinspections during the initial reading of relevant sentences. Expectedly, slow decoding ability lead to more reinspections. In contrast to previous studies investigating reading of expository text (e.g. Kaakinen et al., 2015), knowledge and performance measures used the present study (FOKE, information subskill) were not related to reinspection probability. Students with and without attentional difficulties made reinspections with equal probability. There remained substantial individual variability in the probability of making reinspections, which was not associated with reading fluency or sentence relevancy. 3.5. Reinspection duration A total of 2303 of 3168 observations that included at least a single regression (72.7%) were analyzed; only six values were excluded as outliers. The final model consisted of the statistically significant main effects of relevancy and reading fluency (Table 3). The estimated marginal means (Fig. 2) indicated that one SD increase in reading fluency reduced the reinspection duration by 12% (95% CI = 7.0%–15%) when the reinspection duration of the average reader was 650 ms. The irrelevant sentences were reinspected for 544 ms (95% CI = 459–624 ms) and the relevant sentences for 775 ms (95% CI = 624–962 ms), on average. The random variability in the average reinspection duration was relatively small (95% CI of 613–687 ms) (Table 4). In summary, similar to reinspection probability, the students made longer reinspections to relevant sentences and because of slower decoding ability. Table 2 Pearson correlation coefficients between the independent variables. 12345 1. Prior knowledge (selfevaluated) 1 0.016 0.009 −0.043 2. Info search subskills 0.062 1 −0.212* 0.175* 3. Reading fluency 0.112 0.421 ⁎⁎ 1. 4.Attentional difficulties −0.031 −0.326 ⁎⁎ −0.345 ⁎⁎ 1−0.032 5. Verbal memory 0.009 0.323 ⁎⁎ 0.427 ⁎⁎ −0.175 ⁎ 1 J. Hautala, et al. Learning and Individual Differences 75 (2019) 101775 6 3.6. Look-back probability In 1041 of 3168 sentence reads (33%), there was a look-back containing at least three fixations, which was our distribution-based criteria for a proper reading pass whereas one or two fixation passes were interpreted as skims. The final model consisted of the main effects of relevancy, attention deficit, the information subskills score, the interactions between relevancy and attention deficit, and the interactions between the relevancy and the information search score. The model produced a convergence warning, but all optimizers produced the same results, in which case the warning can be taken as a false positive (Bates et al., 2014). All but the main effect of the information score were statistically significant predictors in the model (Table 3). Fig. 3 illustrates the nature of the interactions. The effect of relevancy was larger in students with intact vs. deficient attentional functioning. While both groups looked back at the irrelevant sentence with equal low probability (11%–12%, 95% CIs: 7%–17%) the students with intact attentional functioning looked back at the relevant sentence with much higher probability (54%, 95% CI = 45%–63%) than the students with attentional difficulties did (30%, 95% CI = 22%–40%). The interaction between relevancy and the information search subskill, indicated that trials with lowest vs. highest scores were associated with the greater likelihood of looking-back at a relevant sentence. The probability of look-backs to relevant sentences was 31% (95% CI = 23%–41%) for trials with low information subskill score and 45% and 50% (95% CIs = 36%–59%) for medium and high information subskill score, respectively. The large random intercept for the students (Table 4) suggests huge unexplained variability in the probability of performing look-backs in general (95%, CI = 4–65%). In summary, look-back probability reflected higher cognitive processes than decoding aspects of reading. As predicted on the basis of RESOLV model, making more look-backs to the relevant sentence was associated with better scores in the informational task. Crucially, the attention-deficit group made less look-backs to relevant sentences, suggesting highly selective influence of attention deficit on comprehension monitoring processes. However, there remained huge individual in the probability of making look-backs. 3.7. Look-back duration In 1016 of 3168 observations (32%), there was a look-back containing at least three fixations (and therefore not considered a skimming pass), which was not an extreme value (29) outside a log-transformed value of six to nine. Again, the final model consisted of the statistically significant main effects of relevancy and reading fluency (Table 3). The estimated marginal means (Fig. 3) indicated that an increase of one standard deviation in reading fluency reduced the lookback duration by 12% (95% CI = 7%–17%) when the duration for the average reader was 1693 ms. The irrelevant sentences were viewed for 1594 ms (95% CI = 1407–1806 ms), and the relevant sentences were viewed for 1798 ms (95% CI = 1606–2014 ms) on average. The random variability of individuals in average look-back duration was modest (95%, CI = 1603–1785 ms (Table 4). In addition to being more frequent, look-backs were also longer in relevant sentences. Slower reading speed predicted longer durations in looking-back at both sentence types. Attention deficit had no influence on look-back duration. 3.8. Bivariate latent change score modeling The preceding results suggest that firstand second-pass viewing reflected partially different cognitive processes. However, in these analyses, the possible interdependency between firstand second-pass viewing was not considered. For example, during the first-pass viewing, the students might have searched only the relevant sentence and return to read it properly later, in which case short first-pass viewing times would predict long look-back times and vice versa. Bivariate latent change score modeling (Kievit et al., 2017) was used to study how sentence relevancy effects in gaze duration and look-back duration measures depend on each other. This modeling takes the two main cognitive predictors of reading fluency and attention difficulties into account. Table 3 Fixed-effects results of the linear mixed-model analyses in a temporally ordered list of eye movement measures. Model Effect OR Proportional change CI95%df F χ 2 p Progressive fix. dur. Reading fluency 0.81 0.77–0.84 161.8 94.1 < 0.001 Reinspection, prob. Relevancy 1.61 1.25–2.04 8 8.69 0.003 Reading fluency 0.73 0.63–0.84 8 17.6 < 0.001 Reinspection, dur. Relevancy 1.19 1.08–1.33 9.99 10.1 0.009 Reading fluency 0.88 0.83–0.93 155.9 19.6 < 0.001 Look-back, prob. Relevancy 2.27 2.04–2.5 15 19.3 < 0.001 Attention difficulty 0.78 0.65–0.93 15 8.16 0.004 Relevancy:attention difficulty 1.22 1.11–1.35 15 11.1 < 0.001 Relevancy:info search level 1 vs.21.26 1.09–1.47 14 9.22 0.01 Look-back, dur. Relevancy 1.06 1.01–1.12 12.8 4.98 0.04 Reading fluency 0.88 0.83–0.93 154.1 18.5 < 0.001 Note. The bold font indicates when each effect first emerged, such as the effect of reading fluency appeared during progressive fixations, whereas the effect of relevancy first appeared in reinspection probability. Abbreviations: OR = odds ratio, prop. = proportional, fix. = fixation, dur. = durations. Table 4 Variances of the included random intercept (id, item) and the random slopes (factor item/id ) in the linear mixed-model analyses. Model id Relev. id IS id item Relev. item IS item Residual Progressive fix. duration 0.07 0.003 0.006 0.002 0.07 Reinspection, probability 0.63 0.08 0.15 0.14 Reinspection, duration 0.08 0.02 0.04 0.03 0.66 Look-back, probability* 0.84 0.19 0.08 0.06 0.11 0.0008 Look-back, duration 0.07 0.01 0.01 0.002 0.27 Note. The values are given in logarithmic scale for continuous duration measures and log-odds for dichotomous probability measures. To concretize the random effects sizes, the confidence intervals for the individual intercept (id) are given in the text in original scale. Abbreviations: *uncorrelated random effects structure. Relev. = Relevancy, IS = Information search subskill score, fix. = fixation. J. Hautala, et al. Learning and Individual Differences 75 (2019) 101775 7 The model was constructed using Mplus 8.0 (Muthén & Muthén, 1998-2017). A full information maximum likelihood method with robust standard error estimates and scale corrected chi-square test value against non-normality (MLR estimator in Mplus) was used. The model was modified by adding covariances and regression paths with the help of modification indices. The model fit was evaluated using a chi-square test value and the standardized root mean square error (SRMR). In a well-fitting model, the chi-square test value is non-significant and the SRMR is lower than 0.08. Finally, bias-corrected bootstrapped 95% confidence intervals for parameter estimates were calculated. The analysis was begun by defining the difference factors (i.e., the latent change score models). The eye-movement variable values were transformed to a logarithmic scale, and the variables were standardized. The regression coefficient and factor loadings were fixed to one (marked as * in Fig. 4). The regression coefficients derived from the irrelevant sentences to the difference factors of the relevant sentences were freely estimated. Two independent variables—reading fluency and attention difficulty score—were added to the model. After that, the two paths of reading fluency were added based on large modification indices. The model fit increased clearly when one path from attentional difficulties was added with two residual covariances, which resulted in a good model fit: χ 2 (5) = 8.21, p= .145, SRMR = 0.037. In the model, better reading fluency was associated with shorter gaze duration for irrelevant sentences and a smaller difference between relevant and irrelevant sentences in gaze duration. A higher attention difficulty score was associated with a smaller difference in look-back duration for relevant and irrelevant sentences. These results confirmed the results of separate analyses of eye movement measures by showing that the second-pass fixation duration was relatively independent from the first-pass fixation duration and that attention difficulty was specifically associated with shorter look-back durations at relevant Fig. 2. Predicted estimated means and 95% confidence intervals for statistically significant effects on first-pass eye-movement measures. The scale for the duration measures is in milliseconds. Fig. 3. Predicted estimated means and 95% confidence intervals for statistically significant effects on second-pass eye-movement measures. The scale for the duration measures is in milliseconds. Fig. 4. Path diagram of the bivariate latent change score model with parameter estimates and 95% bias-corrected bootstrap confidence intervals in parentheses. J. Hautala, et al. Learning and Individual Differences 75 (2019) 101775 8