Reading Comprehension Skills and Prior Topic Knowledge Serve as Resources When Adolescents Justify the Credibility of Multiple Online Texts

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY 4.0 https://creativecommons.org/licenses/by/4.0/ Reading Comprehension Skills and Prior Topic Knowledge Serve as Resources When Adolescents Justify the Credibility of Multiple Online Texts © 2024 The Author(s). Published with license by Taylor & Francis Group, LLC Published version Kiili, Carita; Strømsø, Helge I.; Bråten, Ivar; Ruotsalainen, Jenni; Räikkönen, Eija Kiili, C., Strømsø, H. I., Bråten, I., Ruotsalainen, J., & Räikkönen, E. (2024). Reading Comprehension Skills and Prior Topic Knowledge Serve as Resources When Adolescents Justify the Credibility of Multiple Online Texts. Reading Psychology, Early online. https://doi.org/10.1080/02702711.2024.2351485 2024 Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=urpy20 Reading Psychology ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/urpy20 Reading Comprehension Skills and Prior Topic Knowledge Serve as Resources When Adolescents Justify the Credibility of Multiple Online Texts Carita Kiili, Helge I. Strømsø, Ivar Bråten, Jenni Ruotsalainen & Eija Räikkönen To cite this article: Carita Kiili, Helge I. Strømsø, Ivar Bråten, Jenni Ruotsalainen & Eija Räikkönen (15 May 2024): Reading Comprehension Skills and Prior Topic Knowledge Serve as Resources When Adolescents Justify the Credibility of Multiple Online Texts, Reading Psychology, DOI: 10.1080/02702711.2024.2351485 To link to this article: https://doi.org/10.1080/02702711.2024.2351485 © 2024 The Author(s). Published with license by Taylor & Francis Group, LLC. View supplementary material Published online: 15 May 2024. Submit your article to this journal View related articles View Crossmark data Reading Psychology Reading Comprehension Skills and Prior Topic Knowledge Serve as Resources When Adolescents Justify the Credibility of Multiple Online Texts Carita Kiilia , Helge I. Strømsøb , Ivar Bråtenb , Jenni Ruotsalainenc and Eija Räikkönend aFaculty of education and culture, Tampere University, Tampere, Finland; bdepartment of education, University of oslo, oslo, norway; cdepartment of Teacher education, University of Jyväskylä, Jyväskylä, Finland; dFaculty of education and Psychology, University of Jyväskylä, Jyväskylä, Finland ABSTRACT This study sought to understand how well students (n = 274; Mage = 12.45) were able to identify the author, the main claim, and the supporting evidence (identification performance) and to justify the author’s expertise, the author’s benevolence, and the quality of the evidence (justification performance) while reading multiple online texts. The study also examined the contribution of prior topic knowledge and basic reading skills (word recognition and reading comprehension) to students’ identification and justification performance. Students read two more and two less credible online texts about sugar effects on health. After reading each text, they responded to multiple-choice items that measured the identification and justification performance. Justifying credibility seemed more challenging for students than identifying the claim, evidence, and author. Word recognition and reading comprehension were statistically significant predictors of identification performance, whereas prior knowledge and reading comprehension were statistically significant predictors of https://doi.org/10.1080/02702711.2024.2351485 © 2024 The author(s). Published with license by Taylor & Francis group, llc. CONTACT carita Kiili [email protected] Faculty of education and culture, Tampere University, P.o. Box 700, 33014 Tampere, Finland. supplemental data for this article can be accessed online at https://doi.org/10.1080/02702711.20 24.2351485. This is an open access article distributed under the terms of the creative commons attribution license (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting of the accepted Manuscript in a repository by the author(s) or with their consent. ARTICLE HISTORY Received 20 December 2023 Accepted 1 May 2024 2 C. KIILI ETAL. justification performance. The findings offer new insights into the relationship between basic reading skills and credibility evaluation that can inform both theory and instruction. Introduction Navigating the current online textual landscape, characterized by the rapid spread of misinformation, algorithmic bias, and persuasive argumentation (Kozyreva, Lewandowsky, & Hertwig, 2020), requires critical reading. To be able to read online texts critically, readers need to identify elements crucial to credibility (viz., the main argument and the source of the text) and evaluate the quality of the argument and the trustworthiness of the source (e.g., Barzilai, Thomm, & Shlomi-Elooz, 2020). To make accurate evaluative judgments of text credibility, readers need to understand why specific types of evidence can or cannot support authors’ claims well and why the source can or cannot be regarded as trustworthy. In other words, critical readers can justify their evaluative judgments. While several previous studies have examined how adolescents justify their overall text evaluations (e.g., Forzani, Corrigan, & Kiili, 2022; Potocki et al., 2020), we aimed to gain a more nuanced understanding of adolescents’ justifications by examining how they justify the source’s expertise and benevolence as well as the quality of evidence supporting the text’s main claim. Further, we examined two essential cognitive resources - prior topic knowledge and basic reading skills - that may contribute to adolescents’ identification (i.e., identification of source, claim, and evidence) and justification performance (e.g., Forzani, 2018; Kanniainen, Kiili, Tolvanen, Aro, & Leppänen, 2019). Regarding reading comprehension, we also sought to gain a nuanced understanding by examining what kinds of reading comprehension processes might contribute to adolescents’ identification and justification performance. These examinations of adolescents’ justification skills and underlying cognitive resources can contribute to building a highly needed theoretical understanding of credibility evaluation, which, in turn, may help educators design instruction to support students’ critical reading skills in online contexts. Content-Based and Source-Based Evaluation of Information Several models or frameworks suggest that readers can engage in content- and source-based evaluation to determine whether they can trust offline or online information (Barzilai et al., 2020; Forzani et al., 2022; Stadtler READIng PSyCHoLogy 3 & Bromme, 2014). For example, the content-source-integration model by Stadtler and Bromme (2014) suggests that readers can make first- or second-hand credibility judgments when encountering conflicting information in multiple texts. When making first-hand judgments, readers compare text content to their prior knowledge to determine whether the claims stated in the text are valid, whereas second-hand judgments are based on source information (e.g., source’s expertise, intentions) that readers evaluate to determine whether they can trust the source of the text. Readers tend to rely on their prior knowledge or beliefs when the text content is more familiar to them, whereas in evaluating more unfamiliar topics, readers put more emphasis on source information (e.g., Bråten, McCrudden, Stang Lund, Brante, & Strømsø, 2018; McCrudden, Stenseth, Bråten, & Strømsø, 2016). Barzilai et al. (2020) have elaborated on the notion of first- and second-hand evaluation. Their bi-directional model of first and second-hand evaluation strategies specifies three types of first-hand evaluation strategies: knowledge-based validation, discourse-based evaluation, and corroboration. Consistent with the content-source-integration model (Stadtler & Bromme, 2014), knowledge-based validation refers to comparing text content to one’s prior knowledge and beliefs, which is usually a routine part of comprehension (Richter, Münchow, & Abendroth, 2020). In discourse-based evaluation, readers draw on discourse features to determine how well knowledge is justified or communicated. For example, readers can evaluate the strength of the argument (McCrudden & Barns, 2016; Münchow, Tiffin-Richards, Fleischmann, Pieschl, & Richter, 2023) or the quality of evidence (List, 2023; List, Du, & Lyu, 2021). Finally, readers can engage in corroboration (see Wineburg, 1991) to examine whether other texts verify or contradict the content of the currently processed text. Drawing on Stadtler and Bromme (2014), Barzilai et al. (2020) also specified different kinds of second-hand evaluation strategies targeting source trustworthiness. By using sourcing strategies, readers can evaluate, for example, the expertise or benevolence of the source of the text (e.g., Thomm & Bromme, 2016). Importantly, Barzilai et al. (2020) highlighted that first- and second-hand evaluation strategies are not used in isolation; instead, readers can employ them reciprocally, meaning that evaluation of information can affect evaluation of sources, and vice versa. Students’ Content-Based and Source-Based Justifications Identifying components of arguments (i.e., claim, reasons, and evidence) and sources of information is a prerequisite for content- and source-based evaluation. Readers may struggle to identify the main claim in naturalistic 4 C. KIILI ETAL. texts (Diakidoy, Ioannou, & Christodoulou, 2017; Larson, Britt, & Larson, 2004), although the simplicity and explicitness of the structure of the argument can facilitate the identification of the argument (Christodoulou & Diakidoy, 2020). And, while identifying source features in short, linear texts may be straightforward, it seems to depend on students’ decoding skills (Potocki et al., 2020). In online contexts, identifying the author might be more challenging, however, as the location of author information may vary, or students may confuse the author with other source information, such as affiliation (Coiro et al., 2015). For example, 17% of middle school students (n = 773) have been shown to have difficulties identifying the author of a web page (Coiro et al., 2015). Failing to identify the main claim, the supporting reasons, and the author may impede students in using sources to qualify the validity of the claims (Perfetti, Rouet, & Britt, 1999) and evaluate the quality of presented arguments (cf. Christodoulou & Diakidoy, 2020). Students’ skills in using content- and source-based evaluation strategies have been studied by employing think-aloud methodology (Barzilai, Tzadok, & Eshet-Alkalai, 2015; Mason, Boldrin, & Ariasi, 2010; McGrew, 2021), or by asking students to justify their credibility evaluations or credibility rankings in writing (Braasch, Bråten, Strømsø, & Anmarkrud, 2014; Coiro et al., 2015; Potocki et al., 2020). Think-aloud studies have suggested that students’ spontaneous credibility evaluation during online inquiry is rather limited (Barzilai & Zohar, 2012; Kammerer, Gottschling, & Bråten, 2021) or superficial, even when students have been prompted to evaluate online information (McGrew, 2021). For example, Barzilai and Zohar (2012) examined sixth graders’ (n = 38) evaluation strategies by means of think-alouds and retrospective interviews. In that study, students completed two tasks: a search task on the open internet and a reading task, including three pre-selected, conflicting websites. When reading websites, students typically considered the content or the form of the website. However, source trustworthiness was only evaluated in 39% of the websites, and attention to scientific evidence was even more scarce. Similar results have been found among high school students when examining students’ evaluation of online information about social and political topics (McGrew, 2021). McGrew (2021) found that students (n = 18) often based their credibility judgments on topical relevance or superficial features (e.g., date, absence of author, or hyperlinks) but hardly consider the trustworthiness of the sources. In addition, when students checked whether the author provided some evidence, they did not evaluate its quality. Although think-aloud methodology can reveal students’ epistemic thinking in action (Barzilai & Zohar, 2012; Ferguson, Bråten, & Strømsø, READIng PSyCHoLogy 5 2012), students may vary in how comfortable and capable they are thinking aloud (Pressley & Afflerbach, 1995). Because of the resource-intensive nature of this methodology, the sample sizes are usually relatively small compared to many studies analyzing students’ written justifications. Moreover, similar patterns have been found across studies on written justifications and think-aloud studies: many students lack efficient evaluation strategies (e.g., Breakstone et al., 2021; Coiro et al., 2015; Potocki et al., 2020). For example, Potocki et al. (2020), who examined evaluation skills among 5th-, 7th-, and 9th-graders and undergraduates (n = 245), found that especially in the lower grades, students emphasized more content than source information when justifying their credibility evaluations. Similarly, Coiro etal. (2015) found that many middle school students’ (n = 773) justifications for the author’s expertise, the author’s point of view, and the overall credibility of online texts were quite often unacceptable or superficial. It is worth noticing that previous studies also have found considerable inter-individual differences among students, indicating that some students are able to justify the credibility across various perspectives and also display thorough reasoning while evaluating the texts (e.g., Barzilai & Zohar, 2012; Forzani et al., 2022; Hämäläinen, Kiili, Räikkönen, & Marttunen, 2021). Forzani et al. (2022) highlighted qualitative differences between better and poorer evaluators (n = 410) by examining 7th graders’ responses when asked to justify the credibility of one web page during the completion of an online inquiry task. Interestingly, better evaluators were more likely to refer to the source (89% of the better evaluators) than poorer evaluators (14% of poorer evaluators). A similar portion of better (13% of better evaluators) and poorer evaluators (16% of poorer evaluators) evaluated the content beyond just referring to it. Notably, better evaluators seemed to display more sophisticated reasoning in their responses. Although students’ written justifications seem to be a fruitful method for capturing variations in students’ justification skills, there may be several factors that can affect the results. First, insufficient writing skills may hinder some students from expressing their thinking (cf. McCarthy et al., 2022). Accordingly, tasks requiring written explanations seem to be more challenging than tasks in which students do not need to explain their reasoning (Sparks, van Rijn, & Deane, 2021). Second, explaining one’s reasoning is a challenging literacy task that requires behavioral and cognitive engagement, and not all students are willing to invest their full effort (Goldhammer et al., 2014; List & Alexander, 2018). To overcome these potential issues, this study assessed students’ justification ability with multiple-choice items. We focused on understanding how students 6 C. KIILI ETAL. justify both the credibility of the source (expertise and benevolence) and the content (i.e., the quality of evidence). Prior Knowledge and Basic Reading Skills in Justifying the Credibility Previous research suggests that several cognitive skills or resources contribute to students’ credibility evaluation (see Anmarkrud, Bråten, Florit, & Mason, 2022 for a systematic review of individual differences in sourcing), including prior topic knowledge and reading skills. Prior topic knowledge and reading skills can be considered closely related, as the construction of a coherent text representation presumably relies heavily on readers’ prior knowledge (Kintsch, 1988). Kintsch’s (1988, 1998; see also McNamara & Magliano, 2009) construction–integration model of text comprehension describes how comprehension occurs through an interaction between textual content and readers’ prior knowledge. The construction and integration processes result in representations that are called a textbase and a situational model, respectively. The textbase refers to the underlying meaning of explicit information in the text, whereas the situation model refers to the meaning of the text that readers construct by drawing inferences that go beyond the concepts explicitly stated in the text and integrating the textbase with their prior knowledge. Consequently, prior knowledge also has been considered an important resource for credibility evaluation (Brand-Gruwel, Kammerer, van Meeuwen, & van Gog, 2017; Lucassen, Muilwijk, Noordzij, & Schraagen, 2013). Previous studies have found that students’ prior topic or domain knowledge is positively related to their credibility evaluations (e.g., Braasch etal., 2014; Forzani, 2018; Kammerer etal., 2021). However, not all studies have observed statistically significant associations between prior knowledge and credibility evaluation (Hämäläinen et al., 2021; Mason et al., 2018). Further, the role of basic reading skills in credibility evaluation has drawn researchers’ attention. Previous studies have examined associations between credibility evaluation and one or two aspects of basic reading skills. For example, Dyoniziak, Potocki, and Rouet (2023) did not find any association between word reading fluency and eight graders’ (n = 90) credibility evaluation performance measured with items that required discrimination between the most and least credible websites and justification of source credibility (see also, Braasch et al., 2014). However, a positive association between word reading fluency and students’ ability to infer sources’ intentions was observed. Hämäläinen et al. (2021) found a positive association between reading fluency and the ability to justify credibility among upper secondary school students. Also, learners with reading READIng PSyCHoLogy 7 difficulties seem to struggle more with credibility evaluation than do other learners (Kanniainen et al., 2022). Whereas the results are somewhat mixed with respect to lower-level reading skills, prior research suggests that reading comprehension is essential to students’ credibility evaluation, at least for younger students. A large-scale study (n = 1431 seventh graders) by Forzani (2018) showed that reading comprehension contributed to students’ credibility evaluation after controlling for prior knowledge and gender. Notably, when Kanniainen etal. (2019) examined the role of both reading fluency (measured with three tests) and reading comprehension in sixth graders’ (n = 426) evaluation of online texts, reading comprehension was the only statistically significant predictor after considering for students’ prior knowledge, gender, spelling, and nonverbal reasoning. These findings highlight that although lower-level reading skills may be a prerequisite for skillful credibility evaluation of texts, comprehension skills probably matter more. However, reading comprehension is a complex skill that includes different types of comprehension processes, such as understanding words, locating information, identifying main ideas and authors’ purposes, making inferences at various levels, integration, interpretation, and evaluation, not all of which are equally demanding for the reader (Afflerbach, Cho, & Kim, 2015). For example, different comprehension processes are required based on the difficulty of the text and the task (Afflerbach etal., 2015) or depending on the text genre (Gersten, Fuchs, Williams, & Baker, 2001). Reading comprehension is typically assessed with batteries that include a combination of test items aiming at covering many different comprehension processes (Afflerbach, 2017), with higher performance also reflecting student’s ability to use higher-level comprehension processes in reading (cf. OECD, 2019). While much is known about the different comprehension processes that underlie reading comprehension as a composite (e.g., Ahmed et al., 2016; LARCC & Logan, 2017), less is known about associations between the different processes. To the best our knowledge, no study has examined associations between different comprehension processes required in multiple-choice reading comprehension tests and justification skills involved in credibility evaluation. Therefore, in addition to examining how word recognition and reading comprehension were associated with adolescents’ identification (i.e., identification of the author, claim, and evidence) and justification performance, we also explored how different reading comprehension items might contribute to their identification and justification performance. 14 C. KIILI ETAL. In contrast, more than half of the students (57.0%) justified the researchers’ expertise with relevant source information. When justifying the mother’s and the CEO’s expertise, 54.3% of the students recognized that the mother was not a health expert, whereas the corresponding percentage for the CEO was only 33.6%. Many students justified the CEO’s expertise with his work related to sugar products. Justifications for the Author’s Benevolence Justifying the author’s benevolence required students to consider the author’s intentions. As shown in Table 3, students performed best in justifying the mother’s persuasive intention. Although the commercial online text included marketing statements, only 63.0% of the students justified their benevolence judgments with commercial intentions. Notably, more than half of the Table 3. Responses to the justification items concerning the author’s benevolence. Justification alternative f% Researcher in health sciences (Research Center website) her job is to share unbiased knowledge. (correct) 156 58.87 she only puts forward her own opinion. 64 24.15 she tries to change school meals. 29 10.94 she brags about her accurate knowledge. 16 6.04 Total 265 100 Journalist (Newspaper website) his job is to tell accurate information. (correct) 153 57.74 he tries to show that parents are wrong. 75 28.30 he has written about the issue on the internet. 24 9.06 he just tries to get a lot of readers. 13 4.91 Total 265 100 Mother (Blog platform) she tries to get parents to change their behavior. (correct) 215 81.13 she has written a text on the internet. 24 9.06 it is her obligation to provide accurate information. 21 7.92 she has a lot of readers. 5 1.89 Total 265 100 CEO (Company website) he tries to increase the sales of his products. (correct) 167 63.02 he tries to inform his customers. 54 20.38 he shows that he is good at his job. 23 8.68 he writes on his companys website. 21 7.92 Total 265 100 READIng PSyCHoLogy 15 students struggled to understand that the researcher and the journalist were in professions that ideally require the sharing of unbiased information. Justifications for the Quality of Evidence Justifying the quality of evidence required students to consider how well the author supports the main claim with evidence (Table 4). It was challenging for students to understand that the mother could not verify the causal claim with her own observation: only one-fourth of the students questioned the validity of her observation as evidence. In contrast, almost half of the students (47.9%) questioned the validity of the company’s customer survey as evidence. Further, half of the students understood the value of research studies as evidence. It is worth noticing that some students seemed to think that evidence should be produced by the Table 4. Responses to the justification items concerning the quality of evidence. Justification alternative f% Research evidence (Research Center website) she uses two published studies. (correct) 135 50.94 Research has been carried out by foreign researchers. 53 20.00 she has not conducted the studies that she reports. 46 17.36 The research results are contradictory. 31 11.70 Total  265 100 Expert statement (Newspaper website) The expert is a doctor. (correct) 101 38.26 The expert’s report seems accurate. 74 28.03 The expert does not share her own experiences. 60 22.73 The expert says that children eat too much sweets. 29 10.98 Total  264 100 Own experience (Blog platform) she saw the change with his own eyes. 120 45.28 she relies only on one observation. (correct) 67 25.28 she showed that only sugar could have caused the hyperactivity. 61 23.02 she describes the events precisely. 17 6.42 Total  265 100 Customer survey (Company website) one could have selected the results that are favorable to the company. (correct) 127 47.92 several students have responded to the survey. 79 29.81 Respondents have eaten candy before the exam. 40 15.09 The main results of the survey have been published on the company’s website. 19 7.17 Total  265 100 16 C. KIILI ETAL. author. That is, 22.7% selected the option that questioned the quality of the evidence because the interviewed expert did not rely on her own experiences. Additionally, 17.4% chose the option that questioned the research evidence because the author did not conduct the reported research. Contributions of Prior Knowledge and Basic Reading Skills to Students’ Identification and Justification Performance Table 5 presents descriptive statistics for the variables included in regression analyses 1 and 2. Table 6 presents the results of the regression analysis predicting students’ identification performance (identification of the author, claim, and evidence). In Step 1, prior topic knowledge explained 5% of the variance in identification performance, F(1, 245) = 13.69, p < .001. In Step 2, the model, including prior knowledge and word recognition, explained 16% of the variance, F(2, 244) = 23.82, p < .001. After including reading Table 6. Results of sequential regression analysis for variables predicting students’ identification (author, claim, and evidence) performance (N = 246). Variable β p R2ΔR2 step 1 .05*** Prior topic knowledge .23 <.001 step 2 .16*** .11*** Prior topic knowledge .17 .006 Word recognition .34 <.001 step 3 .34*** .17*** Prior topic knowledge .07 .189 Word recognition .16 .006 Reading comprehension .47 <.001 Note. ***p < .001 Table 5. descriptive statistics and inter-correlations for variables included in the regression analyses. Variable (observed range) 12345 1. identification performance (2–12) .51*** .23*** .37*** .56*** 2. Justification performance (1–12) .31*** .31*** .57*** 3. Prior topic knowledge (4–12) .19** .27*** 4. Word recognition (4–40) .42*** 5. Reading comprehension (0–12) N264 264 265 266 257 Mean 9.22 5.98 8.01 18.84 6.76 standard deviation 2.01 2.66 1.75 6.52 2.45 skewness −0.94 0.29 0.00 0.32 −0.09 Kurtosis −0.85 −0.64 −0.34 0.46 −0.31 note.**p < .01 ***p < .001 READIng PSyCHoLogy 17 comprehension in the final model in Step 3, 34% of the variance in identification performance were explained, F(3, 243) = 41.10, p < .001. Only word recognition and reading comprehension were statistically significant predictors in the final model. Thus, the better students’ basic reading skills were, the better was also their identification performance. Table 7 presents the results of the regression analysis predicting students’ justification performance. In Step 1, prior topic knowledge explained 9% of the variance in students’ justification performance, F(1, 245) = 25.13, p < .001. After adding word recognition to the model (Step 2), the model explained 16% of the variance, F(2, 244) = 22.62, p < .001, with both prior topic knowledge and word recognition being statistically significant predictors.When reading comprehension was included in the equation in Step 3, the final model explained 35% of the variance in justification performance, F(3, 243) = 43.75, p < .001. Only prior knowledge and reading comprehension were statistically significant predictors in the final model. This means that the more prior knowledge students had about the text topic and the better reading comprehenders they were, the better they also wherein justifying the credibility. Associations between Specific Reading Comprehension Items and Identification and Justification Performance All reading comprehension items were positively correlated with identification and justification performance (see Appendix for Spearman correlations). We ran two additional regression analyses to investigate which kinds of comprehension items predicted students’ identification and justification performance. In these analyses, Steps 1 and 2 were the same as in the regression analyses presented above (see Tables 6 and 7). However, instead of using the comprehension sum score in Step 3, we entered all twelve reading comprehension items into the model. Table 7. Results of sequential regression analysis for variables predicting students’ justification performance (N = 246). Variable β p R2ΔR2 step 1 .09*** Prior topic knowledge .31 <.001 step 2 .16*** .06*** Prior topic knowledge .26 <.001 Word recognition .26 <.001 step 3 .35*** .19*** Prior topic knowledge .16 .004 Word recognition .07 .228 Reading comprehension .50 <.001 Note. ***p < .001 18 C. KIILI ETAL. Regarding the identification performance, the final model was statistically significant, F(14, 233) = 10.16, p < .001, explaining 38% of the variance. Four reading comprehension items (Item 1: β = .20, p < .001; Item 4: β = .13, p = .024; Item 9: β = .16, p = .004; Item 11: β = .15, p = .014) were statistically significant predictors. These items asked students to identify the text genre (1) and the purpose of the text (9), and to locate a detail presented in one particular sentence (11), in addition to the eight sentences that students were asked to number in the order they appeared in the text (4). Items 1, 9, and 11 were relatively easy. Of all students, 91%, 84%, and 78%, respectively, provided a correct response. Item 4 was the most challenging in the test: only 32% of the students responded correctly. Regarding the justification performance, the final model was statistically significant, explaining 37% of the variance, F(14, 233) = 11.28, p < .001. In this model, prior knowledge (β = .13, p = .018) and five reading comprehension items (Item 3: β = .17, p = .005; Item 4: β = .18, p = .002; Item 5: β = .16, p = .003; Item 7: β = .24, p < .001; Item 8: β = .12, p = .026) were statistically significant predictors. Of these items, only Item 4 predicted both identification and justification performance. Two other items predicting justification performance required students to infer a word’s meaning using morphological knowledge (7) or the text context (8). One item required students to make a bridging inference across two consecutive sentences (3). Finally, one item (5) asked students to indicate which of four statements was not true. Students needed to locate the corresponding ideas across the text (all in different paragraphs) and compare the statements to the text content. The statements and corresponding ideas in the text used different expressions and successful completion of the item required inferencing. The items predicting justification performance were substantially more difficult for students than were items predicting identification performance (Item 3: 56%, Item 5: 42%, Item 7: 42%, and Item 8: 54%). Discussion This study examined adolescents’ credibility evaluation skills. In particular, we focused on their justification performance, specifically on how well sixth grade students can justify the author’s expertise, the author’s benevolence, and the quality of evidence. We also sought to understand how prior knowledge and basic reading skills (word recognition and reading comprehension) may contribute to students’ credibility evaluation, that is, to their identification of components (author, claim, and evidence) subject to credibility evaluation and to justification of the credibility. READIng PSyCHoLogy 19 Consequently, this study shed further light on the relationship between credibility evaluation and lower- and higher-level reading processes. As such, it may contribute to the building of a theoretical understanding of credibility evaluation and the design of instruction that can promote students’ critical reading development. We found that identifying the argument (i.e., claim and evidence) and the author was easy for most students, maybe because the argument structure was relatively simple (cf. Christodoulou & Diakidoy, 2020) and the author information was explicitly available. However, some students struggled to identify the correct author in online texts (see also, Coiro et al., 2015). These students confused the author with the publication venue and, for example, a person interviewed in the text. Further, justifying credibility seemed to be more challenging for students than identifying the argument and the author. Depending on the target of the justification and the text, the proportion of the students who chose the correct justification option varied from 26% to 63%. Students performed best in justifying the author’s benevolence, with the proportion of students who chose the correct justification option varying from 58% to 81%. It should be noted that the multiple-choice options likely scaffolded students’ responses. For example, in this study, 63% of the sixth graders identified the author’s commercial intentions. In contrast, in a study that measured sixth graders’ justification performance with openended responses, only 19% of the sixth graders recognized or were able to express the author’s commercial intentions (Kiili, Leu, Marttunen, Hautala, & Leppänen, 2018). These results suggest that expressive tasks are more challenging than multiple-choice tasks (Sparks et al., 2021), which should be considered when interpreting results across the studies. In line with previous results (e.g., Barzilai & Zohar, 2012; McGrew, 2021; Potocki et al., 2020), quite a few students did not attend to source features, or they did not consider whether the author’s expertise matched the topic of the text. Further, our results showed that a causal claim based on the author’s observations was particularly difficult to question, as only 25% of the students could do that. This is not surprising, given that understanding causality may be challenging even for undergraduates (List, 2023). Even though our results suggest that some credibility aspects are more difficult to justify than others, it should also be noted that on some items, the alternative justification options may have been more difficult than on others. We found that all examined independent variables (prior knowledge, word recognition, and reading comprehension) correlated positively with identification and justification performance. However, word recognition and reading comprehension were statistically significant predictors of 20 C. KIILI ETAL. identification performance, whereas prior knowledge and reading comprehension were statistically significant predictors of justification performance. Thus, successful justification performance seemed to require more demanding reading processes than did identification performance. Framing these findings with Kintsch’s (1998) construction-integration model, establishing a textbase was probably sufficient for a good performance regarding the identification of the author and the argument, whereas justifying the credibility likely required the building of a situational model. Presumably, students needed to make more inferences beyond the text to justify the credibility, with prior topic knowledge supporting inferencing in this process. Noteworthy, our study was limited in considering only prior topic knowledge. It remains for future research to examine whether the role of prior knowledge would be even more substantial if other types of knowledge, such as source knowledge, were included in the examination. Our findings are in accordance with previous studies (Forzani, 2018; Kanniainen et al., 2019) suggesting that basic reading skills, especially reading comprehension, are foundational in credibility evaluation of online texts. However, the contribution of reading comprehension to credibility evaluation may vary depending on how the former is measured (cf. Andreassen & Bråten, 2010; Keenan, Betjemann, & Olson, 2008). In addition, reading comprehension tests may include items that measure different types of reading processes, which also seemed to be the case in our study. We found that more difficult reading comprehension test items, such as inferencing beyond the text, predicted justification performance. In contrast, reading comprehension items that were relatively easy for students (with one exception) predicted identification performance. The results of this initial examination suggest that it would be valuable to examine further how different types of reading comprehension processes may contribute to credibility evaluation. Limitations We acknowledge that the present study has several limitations. First, the low reliability of the identification performance may be considered a limitation. This low reliability may be caused by the fact that identification performance included different types of identification (i.e., identification of the author, claim, and evidence) across four texts that represented different genres. Second, using multiple-choice items in capturing students’ justification performance also comes with some limitations. Students were forced to choose one of the four options to justify their credibility evaluation. The provided options might not have represented their READIng PSyCHoLogy 21 actual thinking. To mitigate this limitation, we used students’ authentic written justifications to inform the formulation of the justification options. This procedure ensured that the options would reflect, at least to some extent, students’ ways of thinking. Further, multiple-choice items may overestimate students’ justification ability because the options prompt students’ thinking. However, an advantage of multiple-choice questions in measuring credibility evaluation is that performance is not dependent on students’ writing skills (see McCarthy etal., 2022; Sparks et al., 2021). Third, the reading comprehension test used in this study seemingly measured various types of comprehension processes, with only one or a few items measuring a specific comprehension process. Therefore, we could not make any firm conclusions about the specific types of comprehension processes that could facilitate credibility justification. Future research could profitably use reading comprehension measures focusing on specific comprehension processes, such as global coherence inferences (Jensen & Elbro, 2022). Instructional Implications Teachers can support the development of their students’ credibility justification skills in various ways. As reading comprehension skills seem to build an important foundation for justifying credibility, at least in online text contexts, teachers need to ensure that students have adequate opportunities to continuously practice reading comprehension, especially with tasks that require inferencing at the paragraph and text levels or even across texts. As it has been shown that reading longer texts either offline or online support comprehension skills (Torppa et al., 2020) and online reading comprehension (Kanniainen etal., 2022), fostering students reading engagement at school and home also seems important when educating critical online readers. Because students’ prior knowledge also may facilitate credibility justification, teachers could choose text topics students already know about when practicing reasoning about credibility. Prior topic knowledge could be activated before, during, and after reading, for example by means of open-ended prompts, visualizations, or analogical reasoning (Hattan & Alexander, 2024). Especially, this may assist students who do not activate and utilize their prior knowledge spontaneously. Activation could go beyond topic knowledge, however, with teachers also activating students’ knowledge about sources. As shown in this study, some students may need support identifying the author, claim, and evidence so that they can successfully proceed to 22 C. KIILI ETAL. thinking about the credibility of the author and the content of the text. In classrooms, teachers could discuss with their students the roles and responsibilities of the author, the publisher, and the interviewee, that is, how they can be considered to contribute to the online text. When reading in an authentic online environment, students may also need help locating author information. Argument knowledge, such as the form and the function of arguments, has also been shown to facilitate the evaluation of arguments (Christodoulou & Diakidoy, 2020). Finally, students would benefit from explicit instruction about how author information can be used to justify the author’s expertise and benevolence, and why one type of evidence is stronger than another. Teachers could, for example, model thinking (see Coiro, 2011) about whether the authors’ expertise match the topic of the texts, and what ethical and/or journalistic principles (ideally) direct researchers and journalists in their work. It could be useful also to demonstrate why causal claims cannot be proved by one observation or by personal experiences. For example, students could brainstorm potential, competing explanations for what might have caused the phenomenon in question. Having students discuss and reason about credibility in small-groups and share their reasoning with the class could provide teachers with opportunities to nudge shallow reasoning toward a deeper one. As the accuracy of information cannot always be judged by examining one single text, teachers should also encourage students to confirm the accuracy of information by checking other resources (Wineburg, Breakstone, McGrew, Smith, & Ortega, 2022). Acknowledgement The authors would like to thank Jari Hämäläinen, Symcode Oy for the software development of CORRE and Riikka Anttonen for helping in the data collection. Ethical Approval The ethical statement has been given by the Ethics Committee of the Tampere Region. Author Contributions CRediT: Carita Kiili—Conceptualization, Methodology, Formal analysis, Resources, Writing - original draft, Project administration, Funding acquisition; Helge I. Strømsø—Conceptualization, Methodology, Resources, Writing -review and editing; Ivar Bråten—Conceptualization, Methodology, Resources, Writing -review and editing; Jenni Ruotsalainen—Writing -review and editing; Eija Räikkönen—Methodology, Writing -review and editing. READIng PSyCHoLogy 23 Disclosure Statement No potential conflict of interest was reported by the author(s). Funding The study was funded by the Research Council of Finland (No. 324524). The work of the third author was supported by Strategic Research Council (No. 335625, No. 335727). ORCID Carita Kiili http://orcid.org/0000-0001-9189-4094 Helge I. Strømsø http://orcid.org/0000-0003-1836-3339 Ivar Bråten http://orcid.org/0000-0002-9242-9087 Jenni Ruotsalainen http://orcid.org/0000-0002-9165-8638 Eija Räikkönen http://orcid.org/0000-0003-4450-9178 Data Availability Statement Data available on request from the authors References Afflerbach, P. (2017). Understanding and using reading assessment, K–12 (3rd ed.). Alexandria, VA: ASCD/International Literacy Association. Afflerbach, P., Cho, B.-Y., & Kim, J.-Y. (2015). Conceptualizing and assessing higher-order thinking in reading. Theory into Practice, 54(3), 203–212. https:// www.jstor.org/stable/43893984. doi:10.1080/00405841.2015.1044367 Ahmed, Y., Francis, D. J., York, M., Fletcher, J. M., Barnes, M., & Kulesz, P. (2016). Validation of the direct and inferential mediation (DIME) model of reading comprehension in grades 7 through 12. Contemporary Educational Psychology, 44-45, 68–82. doi:10.1016/j.cedpsych.2016.02.002 Andreassen, R., & Bråten, I. (2010). Examining the prediction of reading comprehension on different multiple-choice tests. Journal of Research in Reading, 33(3), 263–283. doi:10.1111/j.1467-9817.2009.01413.x Anmarkrud, Ø., Bråten, I., Florit, E., & Mason, L. (2022). The role of individual differences in sourcing: A systematic review. Educational Psychology Review, 34(2), 749–792. doi:10.1007/s10648-021-09640-7 Barzilai, S., Thomm, E., & Shlomi-Elooz, T. (2020). Dealing with disagreement: The roles of topic familiarity and disagreement explanation in evaluation of conflicting expert claims and sources. Learning and Instruction, 69, Article 101367. doi:10.1016/j.learninstruc.2020.101367 Barzilai, S., Tzadok, E., & Eshet-Alkalai, Y. (2015). Sourcing while reading divergent Expert accounts: Pathways from views of knowing to written argumentation. Instructional Science, 43(6), 737–766. doi:10.1007/s11251-015-9359-4