Predicting Online Search Success from Self- Reported Skills, Search Behavior, Domain Knowledge, and Task Evaluations

ISSN 1424-3636www.medienpaed.com Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien   This work is licensed under a Creative Commons Attribution 4.0 International License http://creativecommons.org/licenses/by/4.0/ Predicting Online Search Success from SelfReported Skills, Search Behavior, Domain Knowledge, and Task Evaluations Implications for Teaching Information Literacy Martin Hermida1 , Luca Botturi2 , Loredana Addiamando2, Marzia Galloni2, Chiara Beretta2, Felipe Cardoso2, Luca Luceri2, Azza Bouleimen2, Silvia Giordano2, and Mirna Saad2  1 Schwyz University of Teacher Education, Goldau, Switzerland 2 Scuola Universitaria Professionale della Svizzera Italiana, Locarno, Switzerland Abstract Searching for information is a key activity in everyday life. Understanding how selfreported skills, search behavior and task characteristics relate to search performance can inform educational practices for teaching information literacy. This study investigates these relationships by having 102 adolescents perform search tasks and assessing their performance in relation to their self-reported information literacy, navigation behavior and task evaluations. Our findings indicate that self-reported information self-efficacy alone is not a reliable predictor of search success. Query revisions, domain knowledge and task importance positively impact search success, while reliance on superficial cues for evaluating information quality correlates with lower search success. We suggest that information literacy educators encourage learners to revise their queries, ignore superficial cues for credibility assessment, and gain an overview of a topic before searching for specific facts. Additionally, we recommend designing search tasks with potentially important consequences when solved incorrectly to increase learners’ engagement. Hermida, Martin, Luca Botturi, Loredana Addiamando, Marzia Galloni, Chiara Beretta, Felipe Cardoso, Luca Luceri, Azza Bouleimen, Silvia Giordano, and Mirna Saad. 2025. «Predicting Online Search Success from Self-Reported Skills, Search Behavior, Domain Knowledge, and Task Evaluations. Implications for Teaching Information Literacy». MedienPädagogik (Occasional Papers): 269–294. https://doi. org/10.21240/mpaed/00/2025.11.07.X. 270 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Vorhersagen des Erfolgs beim Suchen im Internet anhand selbstberichteter Fähigkeiten, Suchverhalten, Vorwissen und Aufgabenbewertung. Implikationen für das Vermitteln von Informationskompetenz Zusammenfassung Informationen im Internet suchen und beurteilen ist eine Schlüsselkompetenz. Das Verständnis dafür, wie selbsteingeschätzte Fähigkeiten, Suchverhalten und Aufgabenmerkmale mit der Suchleistung zusammenhängen, kann zu einer Verbesserung der Bildungspraktiken beim Vermitteln von Informationskompetenz beitragen. Diese Studie untersucht diese Zusammenhänge bei 102 Jugendlichen, die Suchaufgaben ausgeführt haben und deren Leistung in Bezug zu ihrer selbstberichteten Informationskompetenz, ihrem Navigationsverhalten und ihrer Aufgabenbewertung gesetzt werden. Die Ergebnisse zeigen, dass die Selbstwirksamkeitseinschätzung allein kein verlässlicher Prädiktor für den Sucherfolg ist. Das Modifizieren von Suchanfragen, das Vorwissen und die eingeschätzte Wichtigkeit der erhaltenen Aufgabe wirken sich positiv auf den Sucherfolg aus, während die Orientierung an oberflächlichen Merkmalen von Quellen bei der Bewertung der Informationsqualität mit geringerem Sucherfolg korreliert. Wir schlagen vor, dass im Rahmen der Vermittlung von Informationskompetenz Lernende ermutigt werden, ihre Suchanfragen zu modifizieren, oberflächliche Hinweise für die Glaubwürdigkeitsbewertung zu ignorieren und sich einen Überblick über ein Thema zu verschaffen, bevor sie nach spezifischen Fakten suchen. Zusätzlich empfehlen wir, Suchaufgaben mit potenziell wichtigen Konsequenzen bei falscher Lösung zu gestalten, um das Engagement der Lernenden zu erhöhen. 1. Introduction Searching for information is a fundamental activity, essential for finding facts, facilitating decision-making, and providing enjoyment (White 2016, 1). Its importance is underscored by international declarations such as the Alexandria Proclamation (IFLA 2005), UNESCO’s White Paper (Ramalho Correia 2002) and the Paris Agenda for Media and Information Literacy (UNESCO 2007; UNESCO 2011). Information literacy has been integrated into compulsory education systems and European school curricula, recognized as a key component within the broader “media and technology” domain (Guitert et al. 2017). Prominent frameworks, including the European DigComp 2.1 (Carretero et al. 2017), the British JISC model (JISC [Joint Information Systems Committee] 2014) or the American White Book on Digital Media Literacy (Hobbs 2010) highlight the centrality of information literacy. 271 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 As students transition into secondary education, they increasingly make independent decisions about their personal lives. Upon reaching adulthood at the age of 18, individuals become responsible for their actions and decisions, gain the right to vote, and are accountable to the law. Consequently, access to information about the world becomes critical, and the ability to manage information needs significantly impacts the quality of their life choices. Adolescents prefer search engines, online encyclopedias, and news sites, with social media becoming the primary source. They want their sources to be up-to-date, verified and provide a quick overview of the most important aspects (Feierabend et al. 2018; Gebel et al. 2014; Reuters Institute 2023). Although interest shifts during adolescence (Hasebrink 2017), Feierabend et al. (2023) report that teenagers are primarily interested in socio-political topics, climate change, wars and conflicts, and diversity. Despite the importance of online information, studies have shown that adolescents struggle to assess its credibility (Masullo Chen et al. 2017; Nygren and Guath 2019; Stanford History Education Group 2016). The International Computer and Information Literacy Study (ICILS) reached similar conclusions (Fraillon et al. 2020), as did research conducted in Switzerland (Zampieri et al. 2018), where the present study was conducted. There is little evidence on how the overall search process can be modelled to predict search success. Most data on online search are collected in artificial settings such as laboratory studies or fabricated systems (e.g., Gao et al. 2022). We lack comprehensive understanding of how web navigation, information literacy, digital skills and task characteristics collectively influence real-life search performance. While prior research has identified both individual and combined factors associated with successful search behavior (see below), to the best of our knowledge, no comprehensive model currently integrates self-reported skills, navigational behavior, domain knowledge and perceived task characteristics within a single analytical framework. Such a model allows for the estimation of the unique contribution of each factor while controlling for the others. Insights derived from this approach may inform the development of instructional courses and materials aimed at fostering information literacy. 2. Searching for Information Searching for information on the Internet can be classified into three categories: (a) navigational tasks (reach a particular site), transactional tasks (perform a webmediated activity), and informational tasks (acquire information present on one or more web pages) (Broder 2002). This study focuses on the latter category. The models developed to describe the information search process differ in complexity and scope (Case and Given 2016). They typically adopt a skill-based view of information 272 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 literacy, structured to facilitate the teaching of information literacy (Sample 2020) and often present a simplified view of the process, omitting external factors as well as the iterative and simultaneous nature of the steps involved (Bodemer 2017). For this study, we adapt a model proposed by Brand-Gruwel et al. (2005), which builds on the work of Eisenberg and Berkowitz (1992). This model divides the process of solving an information problem into five steps: 1) defining the information problem, 2) selecting information sources, 3) scanning information, 4) processing information and 5) organizing and presenting information. This model allows us to define search tasks functionally, as a series of actions performed to achieve a specific goal (Vakkari 2005). Wilson (1997) emphasizes the importance of considering the context in which information tasks are solved while Byström and Järvelin (1995) highlight the influence of personal factors on various steps in the information problem-solving process, such as the cognitive abilities for analyzing the information needs or evaluating the information found. Accordingly, while we will follow the idealized step model for solving information problems, we will also account for the context in which the search takes place and the personal factors that may influence the process. 3. Factors influencing search performance Performing an online search is successful when the information need is satisfied with accurate information, and is unsuccessful when the information need is either not satisfied or satisfied with incorrect information that is mistakenly believed to be correct. Several factors have been found to influence the information search process and search performance. 3.1 Digital Skills and Digital Literacy Digital literacy is “the ability to understand and use information in multiple formats from a wide range of sources when it is presented via computers” (Gilster 1997, 33), particularly by using the Internet to find and evaluate information sources. While digital literacy refers to mastering concepts and ideas, the actual operations needed to use digital media are usually termed digital skills (Van Laar et al. 2017; Vuorikari, Kluzer, and Punie 2022). Research generally shows a positive relationship between digital literacy and information benefits for teenagers. For instance, digital literacy enhances teenagers’ ability to evaluate the credibility of information, improve their information skills and effectively search for health information (Livingstone et al. 2023). 273 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 3.2 Information Literacy and Information Self-Efficacy Information literacy refers to a set of skills and abilities needed for information-related tasks. These include skills and abilities to discover, access, interpret, analyse, manage, create, communicate, store, and share information (CILIP 2018). The skills often overlap with digital skills which can be considered necessary but not sufficient prerequisite for exercising information literacy in a digital context. Self-efficacy refers to the belief in one’s ability to execute behaviors necessary to achieve a desired goal (Bandura 1977). The literature provides various scales for self-assessment of information literacy or information self-efficacy (Kurbanoglu et al. 2006; Madden et al. 2018; Timmers and Glas 2010). Although these scales differ in scope and breadth, they generally involve participants providing a self-assessment of their ability to locate, find and use information according to their needs. Selfefficacy is not a direct measure of ability but rather a belief in one’s capability. Selfefficacy for online searching has been explored in various studies. For example, Tsai and Tsai (2003) demonstrated that adolescents with higher Internet self-efficacy were more successful in online searches. However, the impact of self-efficacy on search success may vary depending on the task type. Hong (2006) found that selfefficacy positively influenced the quality of health-related content found online for general (broad) tasks, but not for a specific search task. In an experimental study Wood et al. (2016) observed that adults with higher generic information search skills (trained librarians) performed more efficiently. Information literacy is often measured through self-assessment due to its costeffectiveness. However, a more reliable approach involves measuring individuals’ performance in solving information problems. This method has been employed in several studies, including those by Walraven et al. (2009), Stanford History Education Group (2016), and most notably in the large-scale ICILS testings (Fraillon et al. 2020). Previous research has reported inconclusive links between self-assessment and performance measurements, with findings ranging from no correlation to negative or positive correlations and observations (Mahmood 2016; Clark 2017; Brazier et al. 2019). 3.3 Assessment of Credibility The utility of information hinges on its accuracy, making assessment of credibility a crucial aspect of searchers’ activities. Research suggests that young people are becoming increasingly aware of sensitive topics such as fake news. They are developing new – albeit not always effective – criteria for assessing the credibility of information, such as the quality of the online video in which it is presented (Masullo Chen et al. 2017) or the number and nature of reader comments (Botturi and Negrini 2018; Franklin and Eldridge 2017; Waddell 2018). Walraven et al. (2009) demonstrated that 274 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 although students may be aware of the relevant criteria for evaluating information, they do not necessarily apply these criteria during online searches. They often rely too heavily on the design of a website to assess its credibility or only invest the cognitive effort when the topic at hand is exceptionally important (Tamboer et al. 2022). Lucassen et al. (2013) propose three categories to describe the assessment of the credibility of information: 1) semantic features of the content (e.g. accuracy, completeness, neutrality); 2) surface features of the presentation (e.g. length, images, design); and 3) source features (e.g. authority, website brand). Their research indicates that searchers who are familiar with a topic tend to rely more on semantic cues, whereas users with less knowledge are more influenced by surface features. 3.4 Online Navigation The primary means by which individuals interact with online information retrieval systems are query statements and result selection (White 2016). Lau and Horvitz (1999) identified different user behaviors when refining search queries, including specialization, generalization, and reformulation. A study on adolescents’ searches for health information found that many use a trial-and-error approach to formulate search strings (Hansen et al. 2003). When selecting results, teenagers tended to choose those appearing higher on the first search results page, assuming them to be more relevant and reliable. The difficulty of the task at hand also influences how searchers navigate online. Gwizdka and Spence (2006) found a strong correlation between students’ web search task metrics and the perceived difficulty of the search task. Liu et al. (2010) observed that difficult tasks increased the number of queries and time spent on search results pages. 3.5 Domain Knowledge Research on adults’ information retrieval has demonstrated that domain knowledge and strategic knowledge are key components of search success in text searches (Downing et al. 2005). Similarly, domain knowledge and search strategy have been shown to significantly influence online search outcomes (Ford et al. 2002; Willoughby et al. 2009). Users with prior knowledge about a search task’s domain are better equipped to use semantic features to evaluate the credibility of the information found online (Lucassen et al. 2013). In an educational contexts, domain knowledge can be conceptualized either as a user characteristic (available knowledge) or a task characteristic (required knowledge). The latter can be influenced through task design and by adjusting the timing of its assignment within the learning process. 275 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 3.6 Task Characteristics The complexity of an information search task on the web can be assessed from both objective and subjective perspectives. Objectively, searching for information online is inherently complex due to the multitude of potential paths to the desired outcome and the uncertainty introduced by not knowing in advance what results search queries will return (Campbell 1988). The difficulty of non-hypertext search tasks can be categorized by the type of information being sought (Mosenthal 1998), a classification also applied in online search studies (Walraven et al. 2009). Tasks that require users to identify persons, groups, and animals are generally the simplest. More challenging are tasks that require users to provide sequences (chronologically ordered steps of events or processes), verifications or alternatives. The most difficult tasks involve explaining causes, effects, results or providing evidence for them (Mosenthal 1998). Tasks also possess a subjective complexity determined by the characteristics of the person performing the task, such as their prior knowledge and cognitive ability (Campbell 1988). Domain knowledge reduces the perceived difficulty of tasks and enhances the performance of tasks based on search findings (Willoughby et al. 2009). Finally, the perceived importance of a task is also a crucial factor. Search tasks with significant consequences if incorrectly solved, due to inaccurate information, increase users’ motivation to evaluate the credibility of the information (Metzger 2007). 4. Analytical Model for Predicting Search Performance Drawing on findings from the literature and the model proposed by Brand-Gruwel et al. (2005), we developed an analytical model that integrates self-assessed digital skills, information literacy, practices for assessing information credibility, perceived task characteristics (including importance, domain knowledge, difficulty), user navigation, and search task performance to investigate how search performance can be predicted (Figure 1). This model allows for the assessment of the unique contribution of each factor while controlling for the influence of the others. 276 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Fig. 1: Define the information problem Select source of information Scan information Process information Organize and present information Individual factors Tasks provided by researchers Data collected: Browser Logs Data collected via Questionnaire: Answer to Task, Task Difficulty, Task Duration Data collected via questionnaire: Demographics, Information Literacy, Digital Skills, Assessment of Credibility Step model (Brand - Gruwel et al., 2005) Data collected via questionnaire: Task Importance, Domain Knowledge Collected for each task Collected once Modelled Search Process and Data Collection. The first research question (RQ1) we aim to address is: Do self-assessed information literacy scores correlate with search performance success? The second research question (RQ2) will examine this relationship within a broader model that also tests the effect of additional predictors: Can the modeling of users’ navigation, digital skills, information literacy, credibility assessment and task characteristics explain why some individuals are successful at solving information problems while others are not? 5. Method The target group of this study comprised young individuals aged between 16 and 20. Participants were recruited through schools in central and southern Switzerland. The schools were contacted via their principals who were requested to forward an invitation to their students to participate in the study. Participants were compensated with an Amazon voucher worth CHF 20. At the beginning of the study, participants completed a questionnaire collecting data on their demographics, digital skills, information literacy self-efficacy, and their assessment of the credibility of information. The scales used were primarily adapted from existing scales in the literature (see detailed description in the results section). Participants were required to install a browser plugin which directed them to the search tasks, additional questionnaires, and recorded their browser navigation data. Participants were assigned four search tasks, three of which were closed-ended, i.e., they had a single correct solution. Each task was preceded by a brief questionnaire assessing the perceived importance of providing a correct answer and their familiarity with the topic. Participants then commenced the search process. Upon deciding that they had completed their search, participants were 277 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 directed to a questionnaire where they had to provide the answer to the task. Following this, they were presented with a short questionnaire asking them to indicate which source they used to answer the task, how difficult they perceived the task to be, their satisfaction with their solution, and whether they perceived the search to be long or short. To ensure the searches performed by the participants were as natural as possible, we chose to use a browser plugin that allowed participants to complete the search tasks on their own devices at a time of their choosing. Incorporating realistic consequences can enhance the authenticity of tasks for performers (Kim and Soergel 2006). We accounted for this by asking participants to provide important advice to a hypothetical friend. An abbreviated version of the tasks is shown in Table 1 (https://loisresearch.org/search-tasks/). Task Text Correct Answer Source (examples) 1 Your friend wants to start eating vegan, but she is only 13 years old and has asthma. Is this a good idea? No, in this case a vegan diet is not recommended. The Federal Office of Public Health 2 Your friend has been invited to a homecooked pesto dinner. But he has heard that basil can be poisonous and needs advice on whether or not to go to the dinner. Go to the dinner, basil in this quantity is not toxic. The University of Genoa 3 Your friend has heard that climate change will facilitate the spread of tropical diseases and wants to move further north. What would you advise her to do: stay or move to her relatives in Scandinavia? There is no need to move further north to evade tropical diseases. The Lancet – Regional Health Tab. 1: Tasks provided by researchers and correct answers. 6. Results Of the 152 participants who completed the initial questionnaire, 148 individuals also completed at least one of the search tasks. The mean age of these 148 participants was 19.13 years (SD = 2.44). Half (50%) of them were female, who were slightly but not significantly younger (M = 18.99, SD = 2.79) than the male participants (M = 19.28, SD = 2.02). 284 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Model 1 Model 2 Model 3 Model 4 Task 1 (Diet) Intercept 4.82*** (2.28-10.85) 579.94 (4.91 – 145184.26) 11.56 (0.03 – 6897.92) 0.00° (0.00 – 65.82) Search Actions 0.95 (0.87-1.05) 0.96 (0.87 – 1.06) 0.95 (0.86 – 1.06) 0.92 (0.82 – 1.04) Query Revisions 1.42 (0.82 – 2.78) 1.49 (0.86 – 2.87) 1.55 (0.86 – 3.24) 1.34 (0.27 – 6.95) Digital Skills 0.94 (0.87 – 1.02) 0.96 (0.88 – 1.04) 1.49° (0.93 – 2.37) Self-Efficacy 1.00 (0.88 – 1.14) 0.98 (0.85 – 1.12) 1.68° (0.94 – 3.04) Assessment of Credibility 0.88° (0.76 – 1.00) 0.86* (0.73 – 0.98) 0.85* (0.73 – 0.98) Domain Knowledge 1.10 (0.71 – 1.75) 1.08 (0.61 – 1.95) Task Importance 2.23° (0.96 – 5.20) 2.50* (1.03 – 6.23) Task Difficulty 1.25 (0.76 – 2.18) 1.17 (0.71 – 2.05) Digital Skills * Self Efficacy 0.99° (0.98 – 1.00) Domain Knowledge * Query Revisions 1.08 (0.62 – 2.18) AIC 121.86 120.5 119.79 120.16 R2 Tjur 0.012 0.081 0.114 0.156 N121 120 119 119 Task 3 (Climate) Intercept 0.97 (0.55 – 1.72) 2.62 (0.07 – 102.09) 1.04 (0.02 – 61.22) 0.00 (0.00 – 53.87) Search Actions 0.95 (0.85 – 1.06) 0.96 (0.85 – 1.08) 0.96 (0.84 – 1.09) 0.94 (0.83 – 1.06) Query Revisions 1.61* (1.09 – 2.54) 1.56* (1.04 – 2.53) 1.57° (1.02 – 2.58) 3.64* (1.45 – 10.83) Digital Skills 0.93* (0.87 – 0.99) 0.93* (0.87 – 0.99) 1.30 (0.85 – 2.13) Self-Efficacy 1.14* (1.03 – 1.27) 1.11° (1.00 – 1.24) 1.67° (0.99 – 3.16) Assessment of Credibility 0.87* (0.77 – 0.96) 0.87* (0.78 – 0.97) 0.88* (0.78 – 0.98) Domain Knowledge 1.33 (0.91 – 1.96) 1.98* (1.19 – 3.50) Task Importance 1.19 (0.80 – 1.79) 1.17 (0.78 – 1.78) Task Difficulty 1.15 (0.82 – 1.62) 1.09 (0.76 – 1.57) Digital Skills * Self Efficacy 0.99 (0.98 – 1.00) Domain Knowledge * Query Revisions 0.66* (0.43 – 0.96) AIC 174.26 164.29 164.89 162.37 R2 Tjur 0.048 0.145 0.157 0.200 N127 124 121 121 Tab. 7: Binary Logistic Regression Analysis for task 1 and 3 (Odds Ratios, Confidence Intervals and Significance). 285 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Perceived task importance emerged as a highly relevant factor. Participants who perceived it as more important to find the correct answer were more likely to do so. Although the two predictors digital skills and self-efficacy did not reach conventional levels of significance, they might indicate effects that could become significant with a larger sample size, allowing for the detection of smaller effects. In our model, these two predictors were significant at the p < 0.1 level, suggesting potential relevance for successfully solving online search tasks. Task 2 For task 2, there were no significant predictors in the initial models. However, in the final model, the data suggest that assessments of credibility and self-efficacy may be potential predictors of search success, with both having p-values < 0.1. Task 3 For task 3, the number of query revisions was a significant predictor when only navigation data was used (Model 1). This significance remained when digital skills, self-efficacy, and assessment of credibility were added (Model 2), with these additional factors also emerging as significant predictors of search success. Specifically, digital skills negatively affected the likelihood of solving the task correctly, possibly because participant who reported very high digital skills may overestimate their abilities (Kruger and Dunning 1999). Superficial assessment of credibility also negatively affected the likelihood of solving the task correctly, whereas higher selfefficacy increased the likelihood of finding the correct answer. When controlling for task characteristics (Model 3), the data suggests query revisions and self-efficacy as significant predictors (p < 0.1), while task characteristics did not show predictive value. In the final mode (Model 4), which controlled for interactions, query revisions, assessment of credibility, and domain knowledge become significant predictors of search success. The interaction between domain knowledge and query revisions was also significant, indicating that the effect of query revisions on search success varied according to participants’ domain knowledge. For participants with low domain knowledge, the probability of providing the correct answer increased with additional query revisions. Conversely, for participants with high domain knowledge, the probability of providing the correct answer decreased as the number of query revisions increased (see Figure 2). 286 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Fig. 2: 0% 25% 50% 75% 100% 0 3 6 9 Query Revisions Probability of giving correct answer Task Knowledge l o w a v e r age high Interaction of Query Revisions and Domain Knowledge. To generalize the results across all tasks, it can be concluded that assessing credibility based on superficial cues has a significant negative effect on search success, as evidenced by the results for task 1 and 3, while task 2 shows a similar tendency. Task characteristics also predict search success, although the specific characteristics varies: for task 1, task importance had a significant effect, while for task 3, it was domain knowledge. Interestingly, task difficulty did not significantly affect search performance in any of the models. Self-efficacy only becomes a relevant predictor when the threshold for statistical significance is lowered to < 0.1. The fact that self-efficacy would be a significant predictor in the final models for all three tasks under this condition suggests that information literacy self-efficacy could be an important factor in determining search performance, particularly with a larger sample size to detect smaller effects. Although the confidence intervals include 0, the lower threshold in every model is ≥ 0.94, indicating at a possible boundary issue. Another reason for the distinct results observed in the climate task could be the better distribution of its dependent variable (correct answer), with 40% of participants providing an incorrect answer and 54% providing a correct answer. In contrast, the other tasks showed more skewed distributions (diet task: 19% incorrect vs. 81% correct; pesto task 9% incorrect and 87% correct), which can impact the predictive power of a model. 287 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 7. Discussion In this study, we investigated various factors that influence search performance within a single model, enabling us to estimate the effect of each predictor while controlling for the influence of the others. This strengthens the explanatory power of the predictors by accounting for interdependencies. We utilized navigation data, questionnaire data on digital skills, information literacy self-efficacy, assessment of credibility, domain knowledge, importance, and difficulty as predictors of search success. To avoid creating an artificial laboratory situation that might distort the data, we allowed participants to perform the searches on their own computers. The tasks were designed to mimic real-life information problems, enhancing the ecological validity of the study. First, we demonstrated that information literacy self-efficacy may serve as a predictor for search success. When examined in isolation (RQ1), self-efficacy did not significantly predict participants’ performance. However, when controlling for additional predictors in the model (RQ2), the results indicate a possible positive effect. The fact that this effect was only marginally significant (p < 0.1) may be attributable to the nature of the tasks employed. As Hong (2006) shows, self-efficacy may be more relevant for broader search tasks and less relevant for narrower search tasks such as those used in this study. Compared to previous research, our results imply that the relationship between information literacy self-efficacy and search success (Clark 2017) may not be absent but more nuanced than previously assumed. We were able to confirm that refining queries can have a measurable positive effect on search performance (Hassan et al. 2013) for some tasks (Task 3) but not others (Task 1). However, a higher number of search actions did not increase the likelihood of a successful search. We also found evidence that using superficial cues (see Table 5) to assess the credibility of information (Masullo Chen et al., 2017; Waddell and Bailey, 2017) has a measurable negative effect on search performance. Task importance and domain knowledge significantly increased search performance. However, for task 3, we found an interaction between query revisions and domain knowledge. For those who had high domain knowledge, search performance decreased with additional query revisions. This could be due to participants overestimating their domain knowledge and becoming increasingly confused with each additional search. Alternatively, the topic of the task (climate change) might lead to encountering a mix of reliable and dubious information, increasing uncertainty with each additional search and thus lowering search performance. Furthermore, the effects of the predictors were not consistent across all tasks. This variability may be due to the task topic, task difficulty, perceived importance, or the nature of the online information landscape. One can imagine that the 288 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 information about climate change is very different from the information about the toxicity of basil. The nature of the task may substantially affect the extent to which various factors influence search success. Our study has several limitations. First, the scope was limited to a small number of tasks, which restricts the generalizability of our findings. Future studies should include a broader set of tasks to reveal patterns based on the task type or topic. The potential contamination of the navigation data may also limit our findings. Previous research has shown that users often search for related or unrelated topics simultaneously (Spink et al. 2002), introducing noise and obscuring the true effects of our predictors. Additionally, due to a technical issue, the tasks were not administered in random order, potentially introducing order effects such as learning or fatigue. To deepen insights into the effects of credibility assessment, it may be beneficial to explore measuring deeper assessment of information. Although we found some evidence for such a scale, its internal consistency was too low to be useful as a predictor of search success. Our participants conducted their search on laptops or desktop computers, examining how results might differ when using mobile phones or tablet could be a direction for future research. Modelling search success across different tasks is a highly generalized approach to understanding online search. However, we propose integrating our findings into information literacy instruction, which is also typically generalized as well. Educators often focus on the basic rules of online information searching due to time and resources constraints, preventing them from exploring diverse scenarios in depth (Togia et al. 2015; Hermida 2021). Nevertheless, educators could emphasize the importance of understanding the nature and requirements of each individual search task and select examples and exercises that involve different tasks. We also suggest incorporating all the following behaviors into instruction, as they might have a positive effect on search performance. Although not every behavior may be beneficial for all search tasks, our results indicate that these behaviors either have a positive effect or no effect, and therefore no potential negative impact. To summarise our findings, we suggest the following: – Encourage learners to revise their queries. – Encourage learners not to use superficial cues to assess credibility. Instead, encourage them to evaluate the source. It is important to keep in mind that the less familiar learners are with a topic, the more tempted they will be to rely on superficial cues. – Point out to learners that limited knowledge on a subject may reduce their ability to find the right information they need. Suggest that they get an overview of a topic to increase their general knowledge of it before searching for a specific fact. – Educators should design search tasks where learners perceive it as very important to find the correct answer. 289 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 In conclusion, this study highlights the complex interplay of various factors influencing online search performance. By integrating our insights into educational practices, we can better prepare students to navigate the online information landscape successfully. Although we recently witnessed the integration of artificial intelligence (AI) into online search, we believe our insights remain valuable. AI features do not inherently assess the credibility of the information they provide and are susceptible to hallucinations (Ji et al. 2023). Therefore, for complex and important tasks, users still need to find and evaluate sources of information themselves. References Bandura, Albert. 1997. Self-Efficacy. The Exercise of Control. New York: W. H. Freeman Company. Behrens, Shirley J. 1994. “A Conceptual Analysis and Historical Overview of Information Literacy”. College & Research Libraries 55 (4): 309–22. https://doi.org/10.5860/crl_55_04_309. Bodemer, Brett B. 2017. “The Importance of Search as Intertextual Practice for Undergraduate Research”. College & Research Libraries 78 (4): 413–27. https://doi.org/10.5860/crl-245. Botturi, Luca, and Lucio Negrini. 2018. “Less Is More: Teenager Digital Information Literacy and Parental Rules”. In ICERI2018 Proceedings, 8633–42. Seville, Spain. https://doi. org/10.21125/iceri.2018.0589. Brand-Gruwel, Saskia, Iwan Wopereis, and Yvonne Vermetten. 2005. “Information Problem Solving by Experts and Novices: Analysis of a Complex Cognitive Skill”. Computers in Human Behavior, The Pedagogical Use of Information and Communication Technology in Education: A Dutch Perspective, 21 (3): 487–508. https://doi.org/10.1016/j.chb.2004.10.005. Broder, Andrei. 2002. “A Taxonomy of Web Search”. ACM Sigir Forum 36 (2): 3–10. https://doi. org/10.1145/792550.792552. Byström, Katrina, and Kalervo Järvelin. 1995. “Task Complexity Affects Information Seeking and Use”. Information Processing & Management 31 (2): 191–213. https://doi. org/10.1016/0306-4573(95)80035-R. Campbell, Donald J. 1988. “Task Complexity: A Review and Analysis”. Academy of Management Review 13 (1): 40–52. https://doi.org/10.2307/258353. Carretero, Stephanie, Riina Vuorikari, and Yves Punie. 2017. DigComp 2.1: The Digital Competence Framework for Citizens. Luxembourg: Publications Office of the European Union. https://doi.org/10.2760/38842. Case, Donald Owen, and Lisa M. Given. 2016. Looking for Information: A Survey of Research on Information Seeking, Needs, and Behavior. Fourth edition. Studies in Information. Bingley, UK: Emerald. Chen, Gina Masullo, Peter S. Chen, Chen-Wei Chang, and Zainul Abedin. 2017. “News Video Quality Affects Online Sites’ Credibility”. Newspaper Research Journal 38 (1): 19–31. https://doi.org/10.1177/0739532917696087. 290 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 CILIP, The Library and Information Association. 2018. ‘CILIP Definition of Information Literacy’. London: The Library and Information Association. Clark, Melissa. 2017. ”Imposed-Inquiry Information-Seeking Self-Efficacy and Performance of College Students: A Review of the Literature”. The Journal of Academic Librarianship 43 (5): 417–22. https://doi.org/10.1016/j.acalib.2017.05.001. Downing, Ricard E., Joi L. Moore, and Steven W. Brown. 2005. “The Effects and Interaction of Spatial Visualization and Domain Expertise on Information Seeking”. Computers in Human Behavior 21 (2): 195–209. https://doi.org/10.1016/j.chb.2004.03.040. Eisenberg, Michael B., and Robert E. Berkowitz. 1992. “Information Problem-Solving: The Big Six Skills Approach”. School Library Media Activities Monthly 8 (5): 27. Feierabend, Sabine, Thomas Rathgeb, Hediye Kheredmand, and Stephan Glöckler. 2023. “JIM-Studie 2023. Jugend, Informationen, Medien”. Stuttgart: Medienpädagogischer Forschungsverbund Südwest. https://mpfs.de/app/uploads/2024/10/JIM_2023_web_final_kor.pdf. Feierabend, Sabine, Thomas Rathgeb, and Theresa Reutter. 2018. “JIM 2018. Jugend, Informationen, Medien. Basisuntersuchung Zum Medienumgang 12Bis 19-Jähriger in Deutschland”. Stuttgart: Medienpädagogischer Forschungsverbund Südwest. https:// mpfs.de/app/uploads/2024/11/JIM2018_Gesamt.pdf. Ford, Nigel, David Miller, and Nicola Moss. 2002. “Web Search Strategies and Retrieval Effectiveness: An Empirical Study”. Journal of Documentation 58 (1): 30–48. https://doi. org/10.1108/00220410210425395. Fraillon, Julian, John Ainley, Wolfram Schulz, Tim Friedman, and Daniel Duckworth. 2020. PREPARING FOR LIFE IN A DIGITAL WORLD. IEA International Computer and Information Literacy Study 2018 International Report. Springer. https://doi.org/10.1007/978-3-030-387815. Franklin, Bob, and Scott A. Eldridge, eds. 2017. The Routledge Companion to Digital Journalism Studies. London ; New York: Routledge/Taylor & Francis Group. Gao, Yizhu, Ying Cui, Okan Bulut, Xiaoming Zhai, and Fu Chen. 2022. “Examining Adults’ Web Navigation Patterns in Multi-Layered Hypertext Environments”. Computers in Human Behavior 129 (April):107142. https://doi.org/10.1016/j.chb.2021.107142. Gebel, Christa, Nadine Jünger, and Ulrike Wagner. 2014. “Informations-Und Engagementbezogenes Medienhandeln von Jugendlichen”. In Jugendliche Und Die Aneignung Politischer Information in Online-Medien, 53–136. Springer. Gilster, Paul. 1997. Digital Literacy. Weinheim: Wiley Computer Publishing. Gross, Melissa, and Don Latham. 2007. ‘Attaining Information Literacy: An Investigation of the Relationship between Skill Level, Self-Estimates of Skill, and Library Anxiety’. Library & Information Science Research 29 (3): 332–53. https://doi.org/10.1016/j.lisr.2007.04.012. Guitert, M., T. Romeu, and P. Baztán. 2017. “CONCEPTUAL FRAMEWORK ON DIGITAL COMPETENCES IN PRIMARY AND SECONDARY SCHOOLS IN EUROPE”. ICERI2017 Proceedings, 5081–90. 291 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Gwizdka, Jacek, and Ian Spence. 2007. “What Can Searching Behavior Tell Us About the Difficulty of Information Tasks? A Study of Web Navigation”. Proceedings of the American Society for Information Science and Technology 43 (1): 1–22. https://doi.org/10.1002/ meet.14504301167. Hansen, Derek L, Holly A Derry, Paul J Resnick, and Caroline R Richardson. 2003. “Adolescents Searching for Health Information on the Internet: An Observational Study”. Journal of Medical Internet Research 5 (4): 1–10. https://doi.org/10.2196/jmir.5.4.e25. Hasebrink, Uwe. 2017. “Audiences and Information Repertoires”. In The Routledge Companion to Digital Journalism Studies, 364–74. London: Routledge. Hobbs, Renee. 2010. Digital and Media Literacy: A Plan of Action. A White Paper on the Digital and Media Literacy Recommendations of the Knight Commission on the Information Needs of Communities in a Democracy. Aspen Institute. Hermida, Martin. 2021. ‘Der Weg der Medienund Kommunikationswissenschaft in die Volksschule (3F-Modell) Und Die Messung Der Einstellungen Der Lehrpersonen Zur Medienbildung’. In Media Literacy, edited by Guido Keel and Wibke Weber, 95–110. Baden-Baden: Nomos. https://doi.org/10.5771/9783748920656-95. Hong, Traci. 2006. “The Internet and Tobacco Cessation: The Roles of Internet Self-Efficacy and Search Task on the Information-Seeking Process”. Journal of Computer-Mediated Communication 11 (2): 536–56. https://doi.org/10.1111/j.1083-6101.2006.00026.x. IFLA. 2005. “Alexandria Proclamation on Libraries, the Information Society in Action”. Alexandria: IFLA – International Federation of Library Associations and Institutions. https:// repository.ifla.org/bitstream/123456789/3147/1/alexandria-proclamation-on-librariesthe-information-society-in-action-en.pdf. Ji, Ziwei, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. “Survey of Hallucination in Natural Language Generation”. ACM Computing Surveys 55 (12): 1–38. https://doi.org/10.1145/3571730. JISC [Joint Information Systems Committee]. 2014. “Developing Digital Literacies”. 2014. https://www.jisc.ac.uk/guides/developing-digital-literacies. Kim, Soojung, and Dagobert Soergel. 2006. “Selecting and Measuring Task Characteristics as Independent Variables”. Proceedings of the American Society for Information Science and Technology 42 (1). https://doi.org/10.1002/meet.14504201111. Kruger, Justin, and David Dunning. 1999. ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’. Journal of Personality and Social Psychology 77 (6): 1121–34. Kurbanoglu, S. Serap, Buket Akkoyunlu, and Aysun Umay. 2006. “Developing the Information Literacy Self-efficacy Scale”. Journal of Documentation 62 (6): 730–43. https://doi. org/10.1108/00220410610714949. Lau, Tessa, and Eric Horvitz. 1999. “Patterns of Search: Analyzing and Modeling Web Query Refinement”. In UM99 User Modeling. Proceedings of the Seventh International Conference, 119–28. Vienna: Springer. 292 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Leys, Christophe, Marie Delacre, Youri L. Mora, Daniël Lakens, and Christophe Ley. 2019. “How to Classify, Detect, and Manage Univariate and Multivariate Outliers, With Emphasis on Pre-Registration”. International Review of Social Psychology 32 (1): 5. https://doi. org/10.5334/irsp.289. Liu, Jingjing, Jacek Gwizdka, Chang Liu, and Nicholas J. Belkin. 2010. “Predicting Task Difficulty for Different Task Types”. Proceedings of the American Society for Information Science and Technology 47 (1): 1–10. https://doi.org/10.1002/meet.14504701173. Livingstone, Sonia, Giovanna Mascheroni, and Mariya Stoilova. 2023. “The Outcomes of Gaining Digital Skills for Young People’s Lives and Wellbeing: A Systematic Evidence Review”. New Media & Society 25 (5): 1176–1202. https://doi.org/10.1177/14614448211043189. Lucassen, Teun, Rienco Muilwijk, Matthijs L. Noordzij, and Jan Maarten Schraagen. 2013. “Topic Familiarity and Information Skills in Online Credibility Evaluation”. Journal of the American Society for Information Science and Technology 64 (2): 254–64. https://doi. org/10.1002/asi.22743. Madden, Andrew D., Sheila Webber, Nigel Ford, and Mary Crowder. 2018. “The Relationship between Students’ Subject Preferences and Their Information Behaviour”. Journal of Documentation 74 (4): 692–721. https://doi.org/10.1108/JD-07-2017-0097. Metzger, Miriam J. 2007. “Making Sense of Credibility on the Web: Models for Evaluating Online Information and Recommendations for Future Research”. Journal of the American Society for Information Science and Technology 58 (13): 2078–91. https://doi.org/10.1002/ asi.20672. Mosenthal, Peter B. 1998. “Defining Prose Task Characteristics for Use in Computer-Adaptive Testing and Instruction”. American Educational Research Journal 35 (2): 269–307. Ng, Wan. 2012. “Can We Teach Digital Natives Digital Literacy?” Computers & Education 59 (3): 1065–78. Norden, Lars-Ake, Linda Mannila, and Arnold Pears. 2017. “Development of a Self-Efficacy Scale for Digital Competences in Schools”. In 2017 IEEE Frontiers in Education Conference (FIE), 1–7. Indianapolis, IN: IEEE. https://doi.org/10.1109/FIE.2017.8190673. Nygren, Thomas, and Mona Guath. 2019. “Swedish Teenagers’ Difficulties and Abilities to Determine Digital News Credibility”. Nordicom Review 40 (1): 23–42. Porat, Erez, Ina Blau, and Azy Barak. 2018. “Measuring Digital Literacies: Junior High-School Students’ Perceived Competencies versus Actual Performance”. Computers & Education 126 (November):23–36. https://doi.org/10.1016/j.compedu.2018.06.030. Ramalho Correia, Ana Maria. 2002. Information Literacy for an Active and Effective Citizenship. White Paper Prepared for UNESCO, the U.S. National Commission on Libraries and Information Science, and the National Forum on Information Literacy. Prague. Reuters Institute. 2023. “Digital News Report 2023”. Oxford: Reuters Institute for the Study of Journalism. 293 M. Hermida, L. Botturi, L. Addiamando, M. Galloni, C. Beretta, F. Cardoso, L. Luceri, A. Bouleimen, S. Giordano, and M. Saad Pädagogik Zeitschrift für Theorie und Praxis der Medienbildung Medien www.medienpaed.com > 07.11.2025 Rosman, Tom, Anne-Kathrin Mayer, and Günter Krampen. 2015. «Combining Self-Assessments and Achievement Tests in Information Literacy Assessment: Empirical Results and Recommendations for Practice». Assessment & Evaluation in Higher Education 40 (5): 740–54. https://doi.org/10.1080/02602938.2014.950554. Sample, Angela. 2020. “Historical Development of Definitions of Information Literacy: A Literature Review of Selected Resources”. The Journal of Academic Librarianship 46 (2): 102116. https://doi.org/10.1016/j.acalib.2020.102116. Spink, Amanda, H. Cenk Ozmutlu, and Seda Ozmutlu. 2002. “Multitasking Information Seeking and Searching Processes”. Journal of the American Society for Information Science and Technology 53 (8): 639–52. https://doi.org/10.1002/asi.10124. Stanford History Education Group. 2016. “Evaluation Information: The Cornerstone of Civic Online Reasoning”. Stanford. Tamboer, Sanne L, Mariska Kleemans, and Serena Daalmans. 2022. “‘We Are a Neeeew Generation’: Early Adolescents’ Views on News and News Literacy”. Journalism 23 (4): 806–22. https://doi.org/10.1177/1464884920924527. Timmers, Caroline F., and Cees A. W. Glas. 2010. “Developing Scales for Information-seeking Behaviour”. Journal of Documentation 66 (1): 46–69. https://doi. org/10.1108/00220411011016362. Togia, Aspasia, Stella Korobili, Aphrodite Malliari, and Ilias Nitsos. 2015. ”Teachers’ Views of Information Literacy Practices in Secondary Education: A Qualitative Study in the Greek Educational Setting”. Journal of Librarianship and Information Science 47 (3): 226–41. https://doi.org/10.1177/0961000614532485. Tsai, Meng-Jung, and Chin-Chung Tsai. 2003. “Information Searching Strategies in WebBased Science Learning: The Role of Internet Self-Efficacy”. Innovations in Education and Teaching International 40 (1): 43–50. https://doi.org/10.1080/1355800032000038822. UNESCO. 2007. “Paris Agenda for Media and Information Literacy”. 2007. https://www.diplomatie.gouv.fr/IMG/pdf/Parisagendafin_en.pdf. UNESCO. 2011. “Digital Literacy in Education”. Policy Brief, 1–12. Vakkari, Pertti. 2005. “Task-Based Information Searching”. Annual Review of Information Science and Technology 37 (1): 413–64. https://doi.org/10.1002/aris.1440370110. Van Laar, Ester, Alexander J.A.M. Van Deursen, Jan A.G.M. Van Dijk, and Jos De Haan. 2017. ‘The Relation between 21st-Century Skills and Digital Skills: A Systematic Literature Review’. Computers in Human Behavior 72 (July):577–88. https://doi.org/10.1016/j. chb.2017.03.010. Waddell, T. Franklin. 2018. “This Tweet Brought to You by a Journalist: How Comment Gatekeeping Influences Online News Credibility”. Electronic News 12 (4): 218–34. https://doi. org/10.1177/1931243117739946. Walraven, Amber, Saskia Brand-Gruwel, and Henny P. A. Boshuizen. 2009. “How Students Evaluate Information and Sources When Searching the World Wide Web for Information”. Computers & Education 52 (1): 234–46. https://doi.org/10.1016/j.compedu.2008.08.003.