Assessing reading and online research comprehension : Do difficulties in attention and executive function matter?

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY-NC-ND 4.0 https://creativecommons.org/licenses/by-nc-nd/4.0/ Assessing reading and online research comprehension : Do difficulties in attention and executive function matter? © 2021 the Authors Published version Kanniainen, Laura; Kiili, Carita; Tolvanen, Asko; Aro, Mikko; Anmarkrud, Øistein; Leppänen, Paavo H.T. Kanniainen, L., Kiili, C., Tolvanen, A., Aro, M., Anmarkrud, Ø., & Leppänen, P. H. (2021). Assessing reading and online research comprehension : Do difficulties in attention and executive function matter?. Learning and Individual Differences, 87, Article 101985. https://doi.org/10.1016/j.lindif.2021.101985 2021 Learning and Individual Differences 87 (2021) 101985 Available online 1 March 2021 1041-6080/© 2021 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Assessing reading and online research comprehension: Do difficulties in attention and executive function matter? Laura Kanniainen a , * , Carita Kiili b , c , Asko Tolvanen d , Mikko Aro e , Øistein Anmarkrud f , Paavo H.T. Lepp¨ anen a a Department of Psychology, University of Jyv¨ askyl¨ a, P.O. Box 35, FI-40014, Finland b Faculty of Education and Culture, University of Tampere, Tampere University, FI-33014, Finland c Department of Education, University of Oslo, P.O. Box 1092, Blindern, 0317 Oslo, Norway d Methodology Centre for Human Sciences, University of Jyv¨ askyl¨ a, P.O. Box 35, FI-40014, Finland e Department of Education, University of Jyv¨ askyl¨ a, P.O. Box 35, FI-40014, Finland f Department of Special Needs Education, University of Oslo, P.O. Box 1040, Blindern, 0318 Oslo, Norway ARTICLE INFO Keywords: Digital literacy Online research and comprehension Reading comprehension Executive function Attention difficulties ABSTRACT This study evaluated the relation between sixth graders’ (N =426) teacher-rated difficulties in attention and executive function (EF) and their comprehension skills. Reading comprehension was assessed with a multiplechoice task and online research and comprehension (ORC) with a problem-solving task. The analyses were controlled for gender, reading fluency and nonverbal reasoning. To investigate differences in students’ performance between the tasks, comprehension skills in the multiple-choice task were also controlled for in the ORC task. Structural equation models showed that teacher-rated attention and EF difficulties were related to students’ performance more in the problem-solving task than in the multiple-choice task. After controlling for all the background variables, these difficulties explained 9% of the variance of ORC performance in girls and 4% in boys. These results indicate that for students with attention and EF difficulties the ORC task was more challenging than the reading comprehension task. 1. Introduction During recent years, the role of technology in reading and learning has increased: over 70% of 15-year-old students in the OECD countries reported using a computer at school (OECD, 2015). School-related work often requires students to complete research tasks on the Internet and to locate, evaluate, synthesize and communicate online information, thereby highlighting the kinds of reading skills that are becoming increasingly important in the age of digitalization (Leu, Kinzer, et al., 2013). This digital turn has resulted in new reading comprehension assessments, alongside paper-and-pencil tests, that target the comprehension skills that students need when accessing the Internet (Coiro et al., 2018; Leu et al., 2015; OECD, 2013). However, studies focusing on the reading skills needed on the Internet have mainly neglected learners with learning difficulties (see Anmarkrud et al., 2018). Little is known about how students with difficulties in attention and executive function manage to solve problems with online information, even though difficulties in these domains are common (American Psychiatric Association, 2018). Reading research has shown that poor attention (e.g., Cain & Bignell, 2014) and difficulties in EF (for a review, see Follmer, 2018) interfere with comprehension. In addition, low executive functions, such as low shifting and inhibition abilities, have been assumed to overstrain students when they need to switch attention between different online resources and inhibit irrelevant information from relevant information (Schwaighofer et al., 2017). However, as far as we know, it is not investigated whether reading to learn from online information shows the same pattern. The other unexplored issue is gender differences in online reading among students with attention and EF difficulties. This is an important aspect to investigate for two reasons. First, girls and boys generally differ in their reading performance. Girls significantly outperformed boys in reading in all countries that participated in PISA 2018 (OECD, 2019). Second, students with attention and EF difficulties do not form a homogeneous group, but girls and boys slightly differ in how these difficulties are manifested. Boys seem to exhibit hyperactivity more frequently than girls (Owens et al., 2015), whose difficulties are * Corresponding author. E-mail address: [email protected] (L. Kanniainen). Contents lists available at ScienceDirect Learning and Individual Differences journal homepage: www.elsevier.com/locate/lindif https://doi.org/10.1016/j.lindif.2021.101985 Received 25 November 2019; Received in revised form 25 January 2021; Accepted 13 February 2021 Learning and Individual Differences 87 (2021) 101985 2 primarily related to inattention (Staller & Faraone, 2006). However, we do not know if girls’ and boys’ differences in the manifestation of these difficulties associate differently in their performance when reading on the Internet. In this study, we investigated how attention and EF difficulties were related to students’ online research comprehension performance in an everyday, regular classroom context and, therefore, students’ attention and EF difficulties were measured with informant-based teacher ratings. In particular, we examined how teacher-rated difficulties in attention and EF affect girls’ and boys’ reading comprehension measured with a multiple-choice task, and online research and comprehension measured with a problem-solving task. The study aims to provide knowledge that helps teachers to better understand the obstacles students with attention and EF difficulties face when reading to learn from online information. 2. Theoretical frame 2.1. Reading comprehension Reading comprehension refers to the process where readers construct and integrate word meanings to comprehend a written text (Perfetti & Stafura, 2014). According to the construction-integration (CI) model (Kintsch, 1998; Kintsch & Rawson, 2005) meaning-making involves building a representation at three levels: 1) linguistic representation, 2) a textbase model, and 3) a situation model. On the first level, readers build a linguistic representation by decoding letters to understand single words. On the second level, readers construct a textbase model from the linguistic input by combining the word meanings and, further, interrelating these meanings of the text to form a microstructure. This kind of semantic analysis continues with recognition of the wider topics contained in the whole text that help the reader to build a macrostructure. These two structures form the textbase model, although at this point it constitutes only a shallow understanding of the text. Finally, to build a deeper understanding of the text, readers need to construct a situational model by integrating the textbase information with their prior knowledge (Kintsch, 1998). Whereas the CI-model focuses on mental processes of reading comprehension, other models focus more on “the identification of component skills” explaining reading comprehension (Kendeou et al., 2016). For example, according to the Simple View of Reading (Hoover & Gough, 1990), reading comprehension consists of two main component skills: decoding and language comprehension. The development of effective and automatized basic decoding skill increases reading fluency, i.e., readers’ ability to read text accurately and rapidly (National Reading Panel, National Institute of Child Health, & Human Development, 2000), which has been seen as a limiting factor on reading comprehension (Perfetti & Stafura, 2014). When reading fluency has become automatized, readers can shift their attention from word recognition to comprehending the text (Fuchs et al., 2001). These theoretical models of reading comprehension also specify inferential processes as an integral component of reading comprehension (Kendeou et al., 2016). Accordingly, the role of nonverbal reasoning in reading comprehension increases when students have reached a sufficient level of literacy skills (e.g., Adlof et al., 2010). Reasoning skills are especially highlighted when students are working with expository texts on less familiar topics (Kintsch & Rawson, 2005) or need to answer comprehension questions that require critical thinking and problem solving (Adlof et al., 2010). In addition, component skills, such as prior knowledge (referring to topical knowledge; Bråten et al., 2013; Tarchi, 2010) and working memory (Sesma et al., 2009) have also been an area of interest in literacy research. 2.2. Online research and comprehension Traditional literacy skills form a critical foundation for successful reading when solving problems with online information (Kanniainen et al., 2019). However, the Internet as a complex reading environment makes additional demands on the reader, as illustrated in the online research and comprehension model (Kiili et al., 2018b; Leu, Kinzer, et al., 2013). The online research and comprehension (ORC) model defines online reading as a self-directed text construction process when seeking answers to questions on the Internet. At least five processes occur when conducting online research: 1) identifying the question, 2) locating information, 3) evaluating information, 4) synthesizing information, and 5) communicating information. ORC is a problem-based reading process that starts with identifying meaningful questions that direct students’ subsequent engagement with online texts and tools (Owens et al., 2002). To answer questions, students need to locate relevant information by formulating appropriate search terms for entry into a search engine (Guinee et al., 2003). To ensure the appropriateness of the information yielded, students are required to evaluate its credibility (Flanagin & Metzger, 2008). A recent study found that the evaluation of a credible online resource requires different abilities than the evaluation of a questionable online resource (Kiili et al., 2018b); moreover, for students, questioning credibility is more difficult than confirming credibility (Kiili et al., 2018a; P´ erez et al., 2018). Before synthesizing information across different online texts (Cho & Afflerbach, 2015), students first need to make sense of single online texts by identifying main ideas (Kiili et al., 2018b). To produce a high-quality synthesis, students explore different viewpoints and compare and contrast texts (Rouet, 2006). Finally, at least in the school context, students are often instructed to communicate and interact with each other by sharing their learning outcomes via different kinds of communication tools, such as social networks, emails, or writing tools (Leu, Forzani, et al., 2013). The present study drew upon the ORC model (Leu, Kinzer, et al., 2013), and the ORC assessment used in this study was built on the construct of online research and comprehension. This study extends the previous research on ORC skills by focusing on the challenges students with attention and EF difficulties face online. 2.3. Attention and executive function difficulties The present study conceptualizes difficulties in the attention and executive function (i.e., executive functions, executive functioning) by drawing on the theories of attention (Mirsky et al., 1999) and EF (Barkley, 1997, 2012; Miyake et al., 2000). EF is an umbrella term for the coordination of cognitive processes that support goal-directed behavior (Barkley, 2012), whereas attention refers to focusing, sustaining, and shifting attention (Mirsky et al., 1999). The basic cognitive processes of EF also involve the previously mentioned ability of shifting attention and, in addition, abilities of inhibition and updating of working memory contents (Miyake et al., 2000). Difficulties in attentional processes, such as focusing and sustaining attention, may be seen as difficulties to focus on instructions (i.e., directing attention) and to work for a long time (i.e., sustaining attention; Klenberg et al., 2010). Difficulties in shifting attention involve difficulties to switch attention between tasks or sources of relevant information, and difficulties in inhibition involve reduced ability to purposefully prevent “dominant, automatic, or prepotent responses” (Miyake et al., 2000). Students with difficulties in inhibition are, for example, not able to wait for their turn (i.e., impulsivity) or to stay seated (i.e., hyperactivity) or to inhibit external distractions (i.e., distractibility; Klenberg et al., 2010). Updating information in working L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 3 memory enables operation at a higher level of EF, which is required in functions, such as planning and nonverbal reasoning that are built on basic cognitive processes (Diamond, 2013). Difficulties in higher level of EF may be seen as difficulties to start working without extra supervision (i.e., initiative), plan one’s actions in advance (i.e., planning), accomplish tasks efficiently (i.e., execution of action), or evaluate one’s own performance (i.e., evaluation; Klenberg et al., 2010). The aforementioned attention and EF processes are important for managing goal-directed behavior (Cirino et al., 2018). Previous research has used both performance-based measures and informant-based rating measures to assess various aspects of these processes (for a review, see Toplak et al., 2013). Performance-based measures are usually conducted under clinical conditions, whereas informant-based measures are conducted by evaluating one’s performance in everyday, problem-solving situations (Follmer, 2018; Toplak et al., 2013). Thus, these two varying measurement types assess somewhat different aspects of cognitive functioning, and seem to be weakly related to each other (Gerst et al., 2017; Toplak et al., 2013). In spite of these differences, both measurement types seem to predict students’ academic performance. For example, both types have been found to associate similarly to students’ reading comprehension (r = 0.32–0.55 versus 0.38–0.55; Gerst et al., 2017). Performance-based measures can provide important information about the efficiency of cognitive processes, such as working memory, in a highly structured environment, whereas rating measures can inform individuals’ success in achieving learning goals (Toplak et al., 2013). Teachers’ assessments of students’ behavior in classrooms can provide an ecologically valid way of evaluating students’ difficulties in attention and EF at school (Barkley, 2012). Previous studies have indicated that teacher ratings of difficulties in attention and EF usually form a unidimensional, general factor both in clinical samples (e.g., Toplak et al., 2012) and in nonclinical, community samples (e.g., Caci et al., 2016). In this study, students’ difficulties in attention and EF were evaluated with informantbased teacher ratings, as we were interested in how well students with these difficulties performed in everyday, school-related tasks that require them to engage in executive processes. 2.4. Difficulties in attention and executive function in reading and online research comprehension Difficulties in both attention and EF have been investigated in various reading-related studies. The results have shown that the risk for reading difficulties is high for students with difficulties in attention (Rommelse et al., 2009; Willcutt & Pennington, 2000). Previous research also shows that inattentiveness is a stronger predictor of reading comprehension difficulties than hyperactivity or impulsivity, although this association seems to be mediated by word reading ability, which in turn is a stronger predictor than inattentiveness (e.g., Cain & Bignell, 2014). However, it seems that students with difficulties in attention have challenges in building mental representations beyond word reading ability, since sustaining attention uses a high proportion of their cognitive resources, leaving a smaller proportion for higher-level comprehension (Miller et al., 2013). Furthermore, EF difficulties have also shown to be associated with reading comprehension at both the elementary and secondary levels (Cutting et al., 2009). Shifting (Kieffer et al., 2013) and planning (Sesma et al., 2009) skills are known to contribute to reading comprehension after controlling for literacy skills such as word reading ability or reading fluency. Inhibition also seems to be associated with reading comprehension (for a review, see Butterfuss & Kendeou, 2018), especially when the measure of inhibition is associated with the memory functions (Borella et al., 2010). Although the various types of EF processes relate to reading comprehension, the common factor of EF seems to predict comprehension over the subprocesses, such as shifting and inhibition (Cirino et al., 2018). Further, the common EF factor was found to be strongly associated with reading comprehension (Cirino et al., 2019). While the effects of attention and EF difficulties on reading comprehension have been extensively studied, little is known about their role when reading in complex online information spaces where critical evaluation and the use of multiple online resources are at a premium. Some research compares reading comprehension on paper to digital reading comprehension in individuals with attention difficulties (Ben-Yehudah & Brann, 2019; Stern & Shalev, 2013), but focuses only on media conditions. Previous findings also show that students who regularly watched TV and simultaneously surfed on the Internet (i.e. media multitasked) reported more difficulties in situations that required them to shift their attention between multiple tasks and inhibit distractions (Baumgartner et al., 2014). We also found one small-scale study showing that less successful online readers seem to experience more difficulties with planning and executing actions, when they ran into a gap in their knowledge, than successful online readers (Cho et al., 2018). In addition, a few studies have been done on the ORC of individuals with reading difficulties (Andresen et al., 2018; Kanniainen et al., 2019; Castek et al., 2011; Henry et al., 2012) and intellectual disabilities (Salmer´ on, Fajardo, & G´ omez-Puerta, 2018). However, to our knowledge, the present study is the first attempt to explore the associations of difficulties in attention and EF with students’ ORC performance. 2.5. Gender differences Historically, attention difficulties have been regarded as less common in girls than boys; however, the gender difference has been found to be smaller than expected (Owens et al., 2015). Girls often receive lower ratings for hyperactivity, impulsivity and inattention (for a review, see Gershon, 2002), although these differences seem mainly to concern hyperactivity and impulsivity (Owens et al., 2015). Rater background seems to play a role in the assessment of attention difficulties (Gershon, 2002). It might be that teachers pay more attention to disruptive behavior than inattentive behavior, which could explain why girls’ difficulties are more often under-identified (e.g., Meyer et al., 2017). The rating inventories tend to better identify difficulties in boys than in girls (Skogli et al., 2013). In addition, it has been shown that boys with attention difficulties are more likely to be identified as having reading difficulties than girls with attention difficulties (Biederman et al., 2002). However, research has shown that girls with attention difficulties have more difficulties in full-scale IQ performance (verbal and nonverbal) than boys with attention difficulties (Gershon, 2002). Differences between girls and boys also exist in the identification of reading difficulties: dysfluently reading boys are more frequently identified than dysfluently reading girls (for a review, see Quinn, 2018). While convincing evidence that girls outperform boys in reading fluency and reading comprehension has been presented (e.g., Logan & Johnston, 2009; Torppa et al., 2018), less research exists on the effects of gender on ORC skills. Only some evidence favoring girls in these skills has been found (Kanniainen et al., 2019; Salmer´ on, García, & Vidal-Abarca, 2018). 2.6. Present study This study explored 1) the association of teacher-rated difficulties in attention and executive function with students’ performance in reading comprehension assessed with a multiple-choice task, and with online research and comprehension assessed with a problem-solving task; and 2) whether the associations were similar for girls and boys. First, as a preliminary step, we examined the factor structure of the teacher ratings of girls’ and boys’ difficulties in attention and EF, since we expected to find gender differences. Second, teacher-rated attention and EF difficulties in girls and boys were investigated in relation to their performance in the multiple-choice task, controlling for the effect of gender, reading fluency and nonverbal L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 4 reasoning. Finally, these difficulties were studied in relation to students’ performance in the problem-solving task by gender. In addition to controlling for the above-mentioned background variables, comprehension skills in the multiple-choice task was also controlled for to determine the association of difficulties in attention and EF with what is unique to ORC. Previous literature (e.g., Butterfuss & Kendeou, 2018; Cirino et al., 2019; Cutting et al., 2009; Miller et al., 2013) shows that students’ difficulties in attention and EF associate with their reading comprehension, but lacks information of the effects of these difficulties on their ORC performance. It is shown that students with reading difficulties (Kanniainen et al., 2019) and intellectual disabilities (Salmer´ on et al., 2018) need support when reading online. Completing the problembased ORC assessment task requires coordination of multiple online reading processes, such as critical evaluation of information and synthesizing information within and across different online resources. These kinds of complex reading processes may be overloading for students with low executive functions (Schwaighofer et al., 2017), especially, when they need to navigate between different online resources and online reading processes. Reading comprehension measures typically assess comprehension of linear texts, but working with multiple online texts goes beyond processing a single linear text (Cho & Afflerbach, 2017). Saying this, we hypothesized that the difficulties in attention and EF would play a greater role in the ORC task than in the reading comprehension task (Hypothesis 1). As previous studies suggest that teacher ratings of difficulties in attention and EF usually form a unidimensional, general factor, we did not set any particular hypothesis on a role of the difficulties in subprocesses of EF, such as low shifting and inhibition abilities. Based on previous research, girls seem to outperform boys in reading skills (e.g., Kanniainen et al., 2019; Torppa et al., 2018), and reading difficulties are less frequently identified in girls than boys (e.g., Quinn, 2018). Thus, gender differences in difficulties in attention and EF were expected to contribute less to girls’ than boys’ performance in both of the comprehension tasks (Hypothesis 2). 3. Method 3.1. Participants Participants were 426 sixth-grade students (207 girls, 219 boys) recruited from eight elementary schools in Central Finland during the years 2014–2015. We first contacted principals of the schools who further forwarded the request to the teachers of sixth graders. Thus, all the school classes participated voluntarily. The schools were located in both rural and sub-urban areas. All participants were aged between 12 and 13 years (M =12.34, SD =0.32) and were taught in mainstream classrooms following the Finnish National Curriculum (The Finnish National Board of Education, 2004). All participating students spoke Finnish as their primary language. The ethical statement for the study was received from the university Ethical Committee, and guardians signed a written consent for their children’s participation in the study. 3.2. Comprehension tasks 3.2.1. Reading comprehension A group-administered reading comprehension subtest drawn from the standardized Finnish reading test battery ALLU (Lindeman, 1998) was used to assess students’ comprehension skills. In this paper-andpencil assessment, the students were required to silently read a twopage (557 words) expository text containing instructions for consumers. These instructions included information on, for example, consumer protection policy and returns and exchanges of consumer goods. The expository texts of the test battery ALLU are factual and descriptive texts, and are comparable to traditional textbook texts (Lindeman, 1998). The students answered 12 multiple-choice (four options) questions, which represented five categories: 1) detail or fact (one question), 2) cause-effect or structure (one question), 3) conclusion or interpretation (four questions), 4) concept or phrase (three questions), and 5) main idea or purpose (three questions). The text was available when responding to the questions. One point was given for each item correctly responded to, and thus the maximum score was 12 points. A reading comprehension factor was formed based on these twelve items (see Data Analyses section). Cronbach’s alpha reliability coefficient was 0.64, and Revelle’s omega reliability coefficient was 0.86. 3.2.2. Online research and comprehension A Finnish adaptation (ILA; see Kiili et al., 2018b) of the Online Research and Comprehension Assessment (ORCA; Leu, Forzani, et al., 2013) was used to measure students’ online research and comprehension skills. The ILA assessment consisted of a simulated Internet environment and tasks that measured four areas of ORC skills: 1) locating information, 2) evaluating information, 3) synthesizing information, and 4) communicating information. Neither the ILA nor the original assessment, the ORCA, included a task measuring students’ ability to identify important questions, since working on answering a same question ensured standardization of the task and thus reliable analyses and comparison of the students’ performance across the skill areas (Kiili et al., 2018b). The ILA assessment began with a fictitious email containing a task assignment sent to the students by the school principal. In the email, the principal instructed the students to explore the health effects of energy drinks and, further, to write a recommendation with justifications on whether or not an energy drink vending machine should be purchased for the school. To help them to form their recommendation, students were asked to examine four different online resources (two news pages [OR1, OR4], an academic online resource [OR2], and a commercial online resource [OR3]). The ILA assessment was completed in a simulated closed Internet environment where students were prompted through the tasks by two avatar students via a simulated social networking site and a chat message window. Students were asked to locate two of the online resources (OR2, OR4) by formulating a search query in a search engine. For example, students were asked to locate a web page that informs how energy drinks affect teeth (OR4). When students received the search engine result list, they were asked to distinguish the relevant online resource from the irrelevant ones. The avatar student gave a link to the correct online resource, if a student failed in the locating task. Thus, students could still read and take notes from the relevant resources, and receive credits in the next parts of the task. The two other resources (OR1, OR3) were given. The avatar prompted students to evaluate two online resources (OR2, OR3) by asking them the following questions: ‘Is the author expert on health issues related to energy drinks?; Is the information provided on the web page reliable?; Why do you think so?’. When reading the online resources (OR1–OR4), students were tasked to take notes with their own words with a note taking tool. After taking notes, students were asked to write a summary about what they have learned about the health effects of energy drinks. Students were able to utilize their notes while synthesizing their summary. Finally, students communicated with the principal by composing an email, in which they justified their opinion concerning the purchase of the energy drink vending machine. The overview of the stimulus materials of the ILA is presented in Fig. 1. The flow of the ILA assessment and the scoring rubric for the measured subskills are presented in more detail in the Appendix A. For a more specific description of the content of the online resources used in the assessment, see Kiili et al. (2018b). Validation of the ILA assessment was performed with confirmatory factor analysis, the results of which reflected the online research and comprehension model (Kiili et al., 2018b). The Kappa values for interrater reliability in the ILA assessment were 1.000 for locating, and varied across the items: for evaluating, the Kappa values ranged from L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 5 0.947 to 0.983, for synthesizing, from 0.784 to 1.000, and for communicating, from 0.722 to 0.939. Cronbach’s alpha reliability coefficient for the ILA total score was 0.74. Students’ prior knowledge on the topic of the ILA assessment was measured with a task consisting of seven multiple-choice (each with four response options) questions on energy drinks and their health effects. Since the relatively poor reliability of the prior knowledge measure, it was excluded in the final analysis of this study. 3.2.3. Differences of the comprehension tasks There are clear differences between these two aforementioned comprehension tasks. The first task assesses reading comprehension with a multiple-choice task, whereas the second task assesses online research and comprehension with a problem-solving task. In the reading comprehension task, students worked on a single text with multiplechoice questions, but in the ORC task students worked on multiple texts. In addition, the ORC task requires students to show their comprehension with written responses. Thus, compared to the multiplechoice task, ORC task is more complex requiring different types of skills (e.g., locating relevant information with a search engine, evaluation of information) and formulation of written responses of one’s understanding. Both task types are common in daily school life (OECD, 2015). 3.3. Other measures and materials 3.3.1. Teacher-rated difficulties in attention and executive function The Attention and Executive Function Rating Inventory (ATTEX; Klenberg et al., 2010) was used to evaluate students’ difficulties in attention and EF. Teachers (N =24) were asked, using the ATTEX, to evaluate students’ difficulties in the areas of inhibition, attentional control, and execution of action. This rating inventory consists of 55 items, each rated on a three-point response scale (0 =not a problem; 1 = Fig. 1. Screen shots of a) the search engine for locating, b) the commercial online resource and the chat message window for evaluation, c) the note taking tool and the simulated social networking site for synthesis, and d) the mailbox for communication. Table 1 Subscales of the Attention and Executive Function Rating Inventory (ATTEX) with example items. Subscale Example item 1) Distractibility 1. Activities are interrupted by even the smallest external distracter 2) Impulsivity 5. Is clearly impatient 3) Motor hyperactivity 14. Constantly needs manual activities 4) Directing attention 21. Has difficulties focusing attention on instructions given to the whole group 5) Sustaining attention 26. Has difficulties completing tasks 6) Shifting attention 32. Has difficulties noting two things at the same time 7) Initiative 36. Is not able to start on tasks without extra supervision 8) Planning 41. Starts working on tasks without planning 9) Execution of action 45. Needs additional, individual supervision to accomplish tasks 10) Evaluation 55. Has difficulties evaluating own performance, difficulty forming an opinion Note. Each item has a three-point response scale (0 =not a problem; 1 =sometimes a problem; 2 =often a problem). L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 6 sometimes a problem; 2 =often a problem). The 55 items form ten subscales: 1) distractibility (q1–4), 2) impulsivity (q5–q13), 3) motor hyperactivity (q14–q20), 4) directing attention (q21–q25), 5) sustaining attention (q26–q31), 6) shifting attention (q32–q35), 7) initiative (q36–q40), 8) planning (q41–q44), 9) execution of action (q45–q52), and 10) evaluation (q53–q55) (Klenberg et al., 2010). These subscales, with an example item from each, are presented in Table 1. In a normative sample of Finnish students (Klenberg et al., 2010), the correlations between the total scores of the ATTEX and the ADHD Rating Scale–IV: School Version (DuPaul et al., 1998) ranged from 0.76 to 0.95 showing a good criterion validity. In this study, Cronbach’s alpha reliability coefficient was 0.98. 3.3.2. Reading fluency Three tests, comprising 1) the word identification test (Lindeman, 1998), 2) the word chain test (Holopainen et al., 2004), and 3) the oral pseudoword text-reading test (Eklund et al., 2014) were used to measure students’ reading fluency performance. Based on these three tests, a reading fluency factor was formed (see Data Analyses section). The McDonald’s omega, i.e., a model based reliability, was 0.79 (cf. Zhang & Yuan, 2016). The Kuder-Richardson reliability coefficient for the original version of the word identification test was 0.97 (Lindeman, 1998). The testretest reliability coefficient for the original version of the word chain test varied between 0.70 and 0.84 (Holopainen et al., 2004). The interrater reliability coefficient for the original version of the oral pseudoword text-reading test was 0.95 (Eklund et al., 2014). 3.3.3. Nonverbal reasoning The Raven’s Standard Progressive Matrices (RSPM; Raven, 1998) was used to assess students’ ability at abstract reasoning in a visuospatial task appropriate for children over age 11. The full version of the RSPM has 60 items; here, to reduce the burden on students, we used a shortened 30-item version (every second item from the full version). Comparisons of shortened and full versions of the RSPM have shown that shortened versions also produce an adequate estimate of nonverbal reasoning (e.g., Wytek et al., 1984). The number of correct responses formed the participants’ total score. Cronbach’s alpha reliability coefficient was 0.77. 3.4. Procedure Data were collected by trained research staff from three group testing sessions, each 45 min long, and one 5-minute individual test session. During the first two group sessions (one reading fluency task was individually administered, see below), students completed the paper-andpencil tests of reading fluency, nonverbal reasoning, and reading comprehension. In the third group session, the students completed the ILA assessment on laptops after answering the prior knowledge questions. The researchers provided technical assistance with the test application when needed, and students’ performance was recorded with a screen capture program and saved as log files. While one half of the class was completing the ILA assessment, the other half completed the individual test session, in which pseudoword text reading was assessed. After the first half had completed the ILA, the groups switched tasks. 3.5. Data analysis 3.5.1. Pre-analyses and goodness-of-fit indices Descriptive and reliability analyses were performed with IBM SPSS Statistics 24. Multigroup confirmatory factor analyses and structural equation models were conducted using Mplus Version 8. The maximum likelihood robust (MLR) estimator was used both with the model for teacher-rated difficulties in attention and executive function and for reading comprehension assessed with the multiple-choice task, since the pre-analysis revealed some non-normality in the distributions of the observed variables. The weighted least square mean and variance adjusted (WLSMV) estimator was used with the online research and comprehension model, since the ORC variables were ordered categorical. MLR estimation is conducted with standard errors and a χ 2 -test statistic that are robust to non-normality (Muth´ en & Muth´ en, 1998–2017). WLSMV estimation is conducted with a diagonal weight matrix with robust standard errors and with a meanand varianceadjusted χ 2 -test statistic that uses a full weight matrix (Muth´ en & Muth´ en, 1998–2017). Irrespective of missing values (0%–12%, depending on the variable), such as sickness absences, model parameters were estimated using all the incomplete cases. MLR uses the standard missing-at-random (MAR) approach, which assumes missingness to be a function of the observed covariates and observed outcomes (Muth´ en & Muth´ en, 1998–2017). WLSMV also assumes missingness to be a function of the observed covariates but not of the observed outcomes (Asparouhov & Muth´ en, 2010; Muth´ en & Muth´ en, 1998–2017). To ensure acceptable model fit for all models, the following cutoff criteria were applied: χ 2 -test (p > .05), root mean square error of approximation (RMSEA) <0.06, Tucker–Lewis index (TLI) and comparative fit index (CFI) ≥0.95 and, with the MLR estimator, also the standardized root mean squared residual (SRMR) <0.08, and, with the WLSMV estimator, also the weighted root mean square residual (WRMR) ≤0.90 (Hu & Bentler, 1999; Yu, 2002). 3.5.2. Factorial invariance across gender To test the assumption of gender differences in teacher ratings of difficulties in attention and executive function, multigroup confirmatory factor analyses (MGCFA) were carried out to test factorial invariance across the groups. The factorial invariance tests were conducted on four levels: 1) configural invariance, 2) weak factorial invariance, 3) strong factorial invariance, and 4) strict factorial invariance (Meredith, 1993). Factorial invariance was achieved if the Satorra-Bentler scaled chisquare difference (SBS∆ χ 2) test (Satorra & Bentler, 2001) was not statistically significant (p >.05). First, on the level of configural invariance, the baseline model (M1) was constructed using the MGCFA of the ten sum scores of the teacher rating scales: 1) distractibility, 2) impulsivity, 3) motor hyperactivity, 4) directing attention, 5) sustaining attention, 6) shifting attention, 7) initiative, 8) planning, 9) execution of action, and 10) evaluation. Next, the MGCFA model of weak factorial invariance (M2) was estimated by constraining all the factor loadings of the ten sum scores to be equal across the gender groups. In the MGCFA model of strong factorial invariance (M3), we also evaluated whether the intercepts were equal across the gender groups. Finally, the MGCFA model of strict factorial invariance (M4) was estimated by constraining the item residuals (error variances) to be equal across the groups in addition to invariant factor loadings and invariant item intercepts. After determining the final structure of the MGCFA model, i.e., M3, the factor scores were saved and included in the structural equation models described below. 3.5.3. Reading comprehension model The first structural equation model (SEM1) was formed to investigate girls’ and boys’ teacher-rated difficulties in attention and executive function in relation to their performance in reading comprehension assessed with a multiple-choice task. The above-mentioned saved factor scores for girls’ and boys’ difficulties in attention and EF were included in SEM1 together with a confirmatory factor analysis (CFA) model, i.e., a measurement model, constructed from the twelve items of the reading comprehension task. The latent variable of reading fluency and the observed variables of gender and nonverbal reasoning were controlled L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 7 for in SEM1. The reading fluency factor was based on CFA constructed from the three reading fluency tests described earlier. Finally, we defined a latent interaction term for the gender variable, and entered it into the model. This latent interaction term was formed to find out if girls’ and boys’ difficulties in attention and EF were differently associated with their reading comprehension performance in the multiple-choice task. The latent interaction term represented the association for boys’ attention and EF difficulties, and a specific factor of the residual variance of this interaction term was formed to represent the possible additional association for girls. 3.5.4. Online research and comprehension model The second structural equation model (SEM2) was constructed to investigate girls’ and boys’ teacher-rated difficulties in attention and executive function in relation to their performance in online research and comprehension assessed with a problem-solving task. The ORC skills were divided into six factors based on 15 observed variables. The six ORC factors represented the abilities to 1) locate information with a search engine, 2) confirm the credibility of information, 3) question the credibility of information, 4) identify main ideas from a single online resource, 5) synthesize information across multiple online resources, and 6) communicate a well-justified and source-based position via email to a specific audience (see also Kiili et al., 2018b). These six ORC factors were highly correlated, and thus a secondorder factor was derived to capture the common variance across the first-order factors. This measurement model resembled the CFA model found in the study by Kanniainen et al. (2019) in which they evaluated the CFA model with the common second-order factor and the six first order factors against the less restrictive, CFA model with the six firstorder factors. When they compared these nested models, a negative correlation between the residuals of questioning credibility and synthesizing was found indicating an inverse relation. This negative correlation was also resembled in the present study. Next, the saved factor scores of girls’ and boys’ difficulties in attention and EF were again included in the model, after controlling for gender, reading fluency, and nonverbal reasoning. Comprehension skills in the multiple-choice task were also controlled for in SEM2 by using the saved factor scores of reading comprehension from SEM1. Finally, the latent interaction term representing the association for boys’ difficulties in attention and EF, and the specific factor of the residual variance of this interaction term representing the association for girls, were derived to reflect possible differences in their respective ORC performance. 4. Results 4.1. Descriptive statistics Table 2 shows the descriptive statistics for reading and online research comprehension, reading fluency, and nonverbal reasoning. Descriptive statistics for the teacher-rated difficulties in attention and executive function were calculated for girls and boys separately for all the ten sum scores of the rating scales (Table 3). Overall, teachers rated girls as having fewer difficulties than boys, especially in motor hyperactivity. The correlation matrices for all the variables used in the SEM analyses are presented in Appendices B and C. 4.2. Factorial invariance of teacher-rated difficulties in attention and EF across gender Factorial invariance across gender was implemented on four levels using the multigroup confirmatory factor analyses (MGCFA) for teacherrated difficulties in attention and executive function. The final models are presented in Fig. 2. At the level of configural invariance, MGCFA model M1 showed a similar factor structure of teacher-rated difficulties in attention and EF for both girls and boys. The ten sum scores of the teacher-rating scales formed a single general factor of teacher-rated attention and EF difficulties. Examination of the modification indices revealed that the residuals of distractibility and impulsivity, impulsivity and motor hyperactivity, and directing attention and sustaining attention correlated significantly in both gender groups. In addition, the residuals of initiative and evaluation correlated significantly in girls, as did distractibility and motor hyperactivity in boys. The fit indices for this baseline model were: χ 2 -test (60) =136.68, p <.001; RMSEA 0.08; CFI 0.96; TLI 0.93; and SRMR 0.05. At the second level, MGCFA model M2 did not fully display weak factorial invariance of the factor loadings. Based on the modification indices, the factor loadings of motor hyperactivity and sustaining attention were freed, and the model re-estimated. The factor loadings of these two parameters were noninvariant across girls and boys. The fit indices were: χ 2 -test (67) =145.83, p <.001; RMSEA 0.07; CFI 0.96; TLI 0.94; and SRMR 0.06. The SBS∆ χ 2 for these two nested models (M1 vs. M2) was 11.17, df =7, p >.05. Partial weak factorial invariance was observed. At the third level, strong factorial invariance of intercepts was not completely achieved in MGCFA model M3. Since the factor loadings of motor hyperactivity and sustaining attention were noninvariant, the intercepts of these variables were also freed. In addition, the intercepts of distractibility and impulsivity were freed based on the residual variances, and the model was re-estimated. The results showed that these intercepts were noninvariant across gender. The fit indices were: χ 2 -test (72) =155.92, p <.001; RMSEA.07; CFI 0.95; TLI.94; and SRMR 0.06. The SBS∆ χ 2 for these two nested models (M2 vs. M3) was 9.29, df =5, p >.05. Partial strong factorial invariance was found, meaning that the teacher ratings of the difficulties of the students in attention and EF showed slight gender differences. The final level showed that strict factorial invariance could not be established (M3 vs. M4; SBS ∆ χ 2 = 94.56, df =10, p >.000), and hence the M3 MGCFA models for both girls and boys were selected as the final models (Fig. 2). 4.3. Structural equation model for reading comprehension The confirmatory factor analyses (CFA) of reading comprehension performance in the multiple-choice task showed that the twelve items of the task formed a single general factor of reading comprehension. The Table 2 Descriptive statistics of reading and online research comprehension, reading fluency and nonverbal reasoning. M SD Min. Max. Reading comprehension (max. 12 points) 6.91 2.53 1 12 Online research and comprehension (max. 46 points) 22.61 6.97 6 39 Word identification test (max. 80 points) a 48.42 9.34 21 80 Word chain test (max. 100 points) a 42.81 14.50 11 85 Pseudoword text-reading test (correctly read words/s) a 0.70 0.21 0.19 1.36 Nonverbal reasoning (max. 30 points) 22.12 3.74 7 30 a Variables used to form a reading fluency factor score. L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 8 factor loadings of this general factor ranged between 4%–24% (0.21–0.49, p <.001). The above-mentioned saved factor scores of the final MGCFA model (Fig. 2) of teacher-rated difficulties in attention and executive function were estimated in relation to the factor of reading comprehension in the first structural equation model (SEM1). The main effect of gender (e.g., differences in means), reading fluency, and nonverbal reasoning were controlled for in the model. The fit indices were: χ 2 -test (141) =166.81, p =.068; RMSEA =0.02; CFI =0.98; TLI =0.97; and SRMR =0.04. Thus, SEM1 indicated a good model fit. The results of the reading comprehension model, SEM1 are presented Table 3 Descriptive statistics of teacher-rated attention and executive function difficulties for girls and boys separately. Subscale Girls (N =205) Boys (N =219) M SD Min. Max. M SD Min. Max. Inhibition 1) Distractibility (max. 8 points) 0.40 1.00 0 7 1.72 1.88 0 8 2) Impulsivity (max. 18 points) 0.54 1.74 0 14 2.92 4.26 0 18 3) Motor hyperactivity (max. 14 points) 0.10 0.59 0 7 1.46 2.54 0 14 Attentional control 4) Directing attention (max. 10 points) 0.67 1.36 0 7 1.95 2.42 0 10 5) Sustaining attention (max. 12 points) 0.27 0.85 0 6 1.98 2.73 0 12 6) Shifting attention (max. 8 points) 0.25 0.85 0 5 1.17 1.94 0 8 Execution of action 7) Initiative (max. 10 points) 0.41 0.98 0 8 1.82 2.46 0 10 8) Planning (max. 8 points) 0.23 0.75 0 5 1.21 1.93 0 8 9) Execution of action (max. 16 points) 0.68 1.57 0 13 2.46 3.04 0 14 10) Evaluation (max. 6 points) 0.20 0.61 0 5 0.82 1.39 0 6 Fig. 2. Multigroup confirmatory factor analysis models (M3) showing the partial factorial invariance of teacher-rated difficulties in attention and executive function (EF) across gender. Notes: Only standardized and statistically significant (p <.05–.001) values are included in the figure. a All the factor loadings, except those for motor hyperactivity and sustaining attention, and all the intercepts, except those for distractibility, impulsivity, motor hyperactivity and sustaining attention, were constrained to be equal between the two groups. L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 15 Journal of Educational Psychology, 107(1), 126–140. https://doi.org/10.1037/ a0037121. Flanagin, A. J., & Metzger, M. J. (2008). Digital media and youth: Unparalleled opportunity and unprecedented responsibility. In M. J. Metzger, & A. J. Flanagin (Eds.), Digital media, youth, and credibility (pp. 5–27). Cambridge, MA: MIT Press. https://doi.org/10.1162/dmal.9780262562324.005. Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387. https://doi.org/10.2307/356600. Follmer, D. J. (2018). Executive function and reading comprehension: A meta-analytic review. Educational Psychologist, 53(1), 42–60. https://doi.org/10.1080/ 00461520.2017.1309295. Fuchs, L. S., Fuchs, D., Hosp, M. K., & Jenkins, J. R. (2001). Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading, 5(3), 239–256. https://doi.org/10.1207/ S1532799XSSR0503_3. Gershon, J. (2002). A meta-analytic review of gender differences in ADHD. Journal of Attention Disorders, 5(3), 143–154. https://doi.org/10.1177/108705470200500302. Gerst, E. H., Cirino, P. T., Fletcher, J. M., & Yoshida, H. (2017). Cognitive and behavioral rating measures of executive function as predictors of academic outcomes in children. Child Neuropsychology, 23(4), 381–407. https://doi.org/10.1080/ 09297049.2015.1120860. Guinee, K., Eagleton, M. B., & Hall, T. E. (2003). Adolescents’ internet search strategies: Drawing upon familiar cognitive paradigms when accessing electronic information sources. Journal of Educational Computing Research, 29(3), 363–374. https://doi.org/ 10.2190/HD0A-N15L-RTFH-2DU8. Henry, L. A., Castek, J., O’Byrne, W. I., & Zawilinski, L. (2012). Using peer collaboration to support online reading, writing, and communication: An empowerment model for struggling readers. Reading & Writing Quarterly, 28(3), 279–306. https://doi.org/ 10.1080/10573569.2012.676431. Holopainen, L., Kairaluoma, L., Nevala, J., Ahonen, T., & Aro, M. (2004). Lukivaikeuksien seulontamenetelm¨ a nuorille ja aikuisille [Dyslexia screening test for youth and adults]. Jyv¨ askyl¨ a, Finland: Niilo M¨ aki Instituutti. Hoover, W. A., & Gough, P. B. (1990). The simple view of reading. Reading and Writing, 2 (2), 127–160. https://doi.org/10.1007/BF00401799. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/ 10705519909540118. Kanniainen, L., Kiili, C., Tolvanen, A., Aro, M., & Lepp¨ anen, P. H. (2019). Literacy skills and online research and comprehension: struggling readers face difficulties online. Reading and. Writing, 32(9), 2201–2222. https://doi.org/10.1007/s11145-01909944-9. Kendeou, P., McMaster, K. L., & Christ, T. J. (2016). Reading comprehension: Core components and processes. Policy Insights From the Behavioral and Brain Sciences, 3 (1), 62–69. https://doi.org/10.1177/2372732215624707. Kieffer, M. J., Vukovic, R. K., & Berry, D. (2013). Roles of attention shifting and inhibitory control in fourth-grade reading comprehension. Reading Research Quarterly, 48(4), 333–348. https://doi.org/10.1002/rrq.54. Kiili, C., Leu, D. J., Marttunen, M., Hautala, J., & Lepp¨ anen, P. H. (2018). Exploring early adolescents’ evaluation of academic and commercial online resources related to health. Reading and Writing, 31(3), 533–557. https://doi.org/10.1007/s11145-0179797-2. Kiili, C., Leu, D. J., Utriainen, J., Coiro, J., Kanniainen, L., Tolvanen, A., … Lepp¨ anen, P. H. (2018b). Reading to Learn From Online Information: Modeling the Factor Structure. Journal of Literacy Research, 50(3), 304–334. https://doi.org/ 10.1177/1086296X18784640. Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge Unviersity Press. Kintsch, W., & Rawson, K. (2005). Comprehension. In M. J. Snowling, & C. Hulme (Eds.), The science of reading: A handbook (pp. 209–226). Malden, MA: Blackwell Publishing. Klenberg, L., J¨ ams¨ a, S., H¨ ayrinen, T., Lahti-Nuuttila, P., & Korkman, M. (2010). The attention and executive function rating inventory (ATTEX): Psychometric properties and clinical utility in diagnosing ADHD subtypes. Scandinavian Journal of Psychology, 51(5), 439–448. https://doi.org/10.1111/j.1467-9450.2010.00812.x. Leu, D. J., Forzani, E., Burlingame, C., Kulikowich, J., Sedransk, N., Coiro, J., & Kennedy, C. (2013). The new literacies of online research and comprehension: Assessing and preparing students for the 21st century with common core state standards. In S. B. Newman, & L. B. Gambrell (Eds.), Quality reading instruction in the age of common core standards (pp. 219–236). Newark, DE: International Reading Association. Leu, D. J., Forzani, E., Rhoads, C., Maykel, C., Kennedy, C., & Timbrell, N. (2015). The new literacies of online research and comprehension: Rethinking the reading achievement gap. Reading Research Quarterly, 50(1), 37–59. https://doi.org/ 10.1002/rrq.85. Leu, D. J., Kinzer, C. K., Coiro, J., Castek, J., & Henry, L. A. (2013). New literacies and the new literacies of online reading comprehension: A dual level theory. In N. Unrau, & D. Alvermann (Eds.), Theoretical models and process of reading (6th ed., pp. 1150–1181). Newark, DE: IRA. Lindeman, J. (1998). Ala-asteen lukutesti ALLU [Reading test for primary school ALLU]. Turku: Center for Learning Research. Logan, S., & Johnston, R. (2009). Gender differences in reading ability and attitudes: Examining where these differences lie. Journal of Research in Reading, 32(2), 199–214. https://doi.org/10.1111/j.1467-9817.2008.01389.x. McNamara, D.S., Roscoe, R., Allen, L., & Balyan, R., McCarthy, K. S. (2019). Literacy: From the Perspective of Text and Discourse Theory. Journal of Language and Education, 5(3), 56–69. doi:10.17323/jle.2019.10196. McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412. https://doi.org/10.1037/met0000144. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. https://doi.org/10.1007/BF02294825. Meyer, B. J., Stevenson, J., & Sonuga-Barke, E. J. (2017). Sex differences in the meaning of parent and teacher ratings of ADHD behaviors: An observational study. Journal of Attention Disorders, 1–10. https://doi.org/10.1177/1087054717723988. Miller, A. C., Keenan, J. M., Betjemann, R. S., Willcutt, E. G., Pennington, B. F., & Olson, R. K. (2013). Reading comprehension in children with ADHD: Cognitive underpinnings of the centrality deficit. Journal of Abnormal Child Psychology, 41(3), 473–483. https://doi.org/10.1007/s10802-012-9686-8. Mirsky, A. F., Pascualvaca, D. M., Duncan, C. C., & French, L. M. (1999). A model of attention and its relation to ADHD. Mental Retardation and Developmental Disabilities Research Reviews, 5(3), 169–176. https://doi.org/10.1002/(SICI)1098-2779(1999)5: 3<169::AID-MRDD2>3.0.CO;2-K. Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734. Muth´ en, L. K., & Muth´ en, B. O. (1998–2017). Mplus user’s guide (8th ed.). Los Angeles, CA: Muth´ en & Muth´ en. National Reading Panel, National Institute of Child Health, & Human Development. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Washington, DC: National Institute of Child Health and Human Development. Organisation for Economic Co-operation and Development [OECD]. (2013). PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy. Paris: OECD Publishing. doi:https://doi.org/10.1787 /9789264190511-en. Organisation for Economic Co-operation and Development [OECD]. (2015). Students, Computers and Learning: Making the Connection. Paris: OECD Publishing. doi: https://doi.org/10.1787/9789264239555-en. Organisation for Economic Co-operation and Development [OECD]. (2019). PISA 2018 Results (Volume II): Where All Students Can Succeed. Paris: OECD Publishing. doi: https://doi.org/10.1787/b5fd1b8f-en. Owens, E. B., Cardoos, S. L., & Hinshaw, S. P. (2015). Developmental progression and gender differences among individuals with ADHD. In R. A. Barkley (Ed.), Attentiondeficit hyperactivity disorder: A handbook for diagnosis and treatment (pp. 223–255). New York, NY: Guilford Press. Owens, R. F., Hester, J. L., & Teale, W. H. (2002). Where do you want to go today? Inquiry-based learning and technology integration. The Reading Teacher, 55(7), 616–625. P´ erez, A., Potocki, A., Stadtler, M., Macedo-Rouet, M., Paul, J., Salmer´ on, L., & Rouet, J. F. (2018). Fostering teenagers’ assessment of information reliability: Effects of a classroom intervention focused on critical source dimensions. Learning and Instruction, 58, 53–64. https://doi.org/10.1016/j.learninstruc.2018.04.006. Perfetti, C., & Stafura, J. (2014). Word knowledge in a theory of reading comprehension. Scientific Studies of Reading, 18(1), 22–37. https://doi.org/10.1080/ 10888438.2013.827687. Quinn, J. M. (2018). Differential identification of females and males with reading difficulties: A meta-analysis. Reading and Writing, 31(5), 1039–1061. https://doi. org/10.1007/s11145-018-9827-8. Raggi, V. L., & Chronis, A. M. (2006). Interventions to address the academic impairment of children and adolescents with ADHD. Clinical Child and Family Psychology Review, 9(2), 85–111. https://doi.org/10.1007/s10567-006-0006-0. Raven, J. C. (1998). Raven’s progressive matrices. Oxford: Psychologists Press Oxford. Rommelse, N. N., Altink, M. E., Fliers, E. A., Martin, N. C., Buschgens, C. J., Hartman, C. A., … Oosterlaan, J. (2009). Comorbid problems in ADHD: Degree of association, shared endophenotypes, and formation of distinct subtypes. Implications for a future DSM. Journal of Abnormal Child Psychology, 37(6), 793–804. https://doi.org/10.1007/s10802-009-9312-6. Rouet, J. (2006). The skills of document use: From text comprehension to web-based learning. Mahwah, NJ: Lawrence Erlbaum. Salmer´ on, L., Fajardo, I., & G´ omez-Puerta, M. (2018). Selection and evaluation of Internet information by adults with intellectual disabilities. European Journal of Special Needs Education, 1–13. doi:https://doi.org/10.1080/08856257.2018.1468 634. Salmer´ on, L., García, A., & Vidal-Abarca, E. (2018). The development of adolescents’ comprehension-based internet reading skills. Learning and Individual Differences, 61, 31–39. https://doi.org/10.1016/j.lindif.2017.11.006. Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514. https://doi.org/ 10.1007/BF02296192. Savalei, V., Reise, S. P., Vazire, S., & Fried, E. (2019). Don’t forget the model in your model-based reliability coefficients: A reply to McNeish (2018). Collabra: Psychology, 5(1). https://doi.org/10.1525/collabra.247. Schwaighofer, M., Bühner, M., & Fischer, F. (2017). Executive Functions in the Context of Complex Learning: Malleable Moderators?. Frontline Learning Research, 5(1), 58–75. doi:10.14786/flr.v5i1.268. Sesma, H. W., Mahone, E. M., Levine, T., Eason, S. H., & Cutting, L. E. (2009). The contribution of executive skills to reading comprehension. Child Neuropsychology, 15 (3), 232–246. https://doi.org/10.1080/09297040802220029. Skogli, E. W., Teicher, M. H., Andersen, P. N., Hovik, K. T., & Øie, M. (2013). ADHD in girls and boys – gender differences in co-existing symptoms and executive function measures. BMC Psychiatry, 13(1), Article 298. doi:https://doi.org/10.1186/1471 -244X-13-298. L. Kanniainen et al. Learning and Individual Differences 87 (2021) 101985 16 Staller, J., & Faraone, S. V. (2006). Attention-deficit hyperactivity disorder in girls. CNS Drugs, 20(2), 107–123. https://doi.org/10.2165/00023210-200620020-00003. Stern, P., & Shalev, L. (2013). The role of sustained attention and display medium in reading comprehension among adolescents with ADHD and without it. Research in Developmental Disabilities, 34(1), 431–439. https://doi.org/10.1016/j. ridd.2012.08.021. Tarchi, C. (2010). Reading comprehension of informative texts in secondary school: A focus on direct and indirect effects of reader’s prior knowledge. Learning and Individual Differences, 20(5), 415–420. The Finnish National Board of Education. (2004). National core curriculum for basic education 2004. Vammala: The Finnish National Board of Education. Toplak, M. E., Sorge, G. B., Flora, D. B., Chen, W., Banaschewski, T., Buitelaar, J., … Miranda, A. (2012). The hierarchical factor model of ADHD: Invariant across age and national groupings? Journal of Child Psychology and Psychiatry, 53(3), 292–303. https://doi.org/10.1111/j.1469-7610.2011.02500.x. Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry, 54(2), 131–143. https://doi. org/10.1111/jcpp.12001. Torppa, M., Eklund, K., Sulkunen, S., Niemi, P., & Ahonen, T. (2018). Why do boys and girls perform differently on PISA reading in Finland? The effects of reading fluency, achievement behaviour, leisure reading and homework activity. Journal of Research in Reading, 41(1), 122–139. https://doi.org/10.1111/1467-9817.12103. Willcutt, E. G., & Pennington, B. F. (2000). Comorbidity of reading disability and attention-deficit/hyperactivity disorder: Differences by gender and subtype. Journal of Learning Disabilities, 33(2), 179–191. https://doi.org/10.1177/ 002221940003300206. Wylie, J., Thomson, J., Lepp¨ anen, P. H., Ackerman, R., Kanniainen, L., & Prieler, T. (2018). Cognitive processes and digital reading. In M. Barzilai, J. Thomson, & S. Schroeder (Eds.), Learning to Read in a Digital World (pp. 57–90). Amsterdam: John Benjamins. Wytek, R., Opgenoorth, E., & Presslich, O. (1984). Development of a new shortened version of Raven’s Matrices test for application rough assessment of present intellectual capacity within psychopathological investigation. Psychopathology, 17 (2), 49–58. https://doi.org/10.1159/000284003. Yu, C. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes (doctoral dissertation). Los Angeles: University of California. Retrieved January 20, 2017 from http://www.statmodel.com/download /Yudissertation.pdf. Zhang, Z., & Yuan, K. H. (2016). Robust coefficients alpha and omega and confidence intervals with outlying observations and missing data: Methods and software. Educational and Psychological Measurement, 76(3), 387–411. L. Kanniainen et al.