Full text
Master Degree Program in Information Management The Influence of ChatGPT on Students’ Opportunistic Behavior and Reduced Effort Beatriz Guita Grilo Master Thesis presented as partial requirement for obtaining the Master Degree in Information Management NOVA Information Management School Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa MGI
NOVA Information Management School Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa THE INFLUENCE OF CHATGPT ON STUDENTS’ OPPORTUNISTIC BEHAVIOR AND REDUCED EFFORT by Beatriz Guita Grilo Master Thesis presented as partial requirement for obtaining the Master’s degree in Information Management, with a specialization in Knowledge Management and Business Intelligence Supervised by Mijail Juanovich Naranjo Zolotov, Ph.D., NOVA Information Management School (NOVA IMS) July, 2024
i STATEMENT OF INTEGRITY I hereby declare having conducted this academic work with integrity. I confirm that I have not used plagiarism or any form of undue use of information or falsification of results along the process leading to its elaboration. I further declare that I have fully acknowledged the Rules of Conduct and Code of Honor from the NOVA Information Management School. Beatriz Guita Grilo Lisbon, 2024
ii DEDICATION AND ACKNOWLEDGEMENTS This dissertation would not have been possible without the endless support and encouragement of several important people in my life. Firstly, I want to dedicate this dissertation to my parents, Elsa and Rui. Their unconditional love and belief in me have been the rock of my academic journey. They have consistently supported me through every challenge and obstacle, and provided me with the opportunities that have allowed me to reach this milestone. I am especially thankful for their support during these difficult years, as well as their constant faith and belief in my work. I would also like to express my gratitude to my sister, Maria. She has been a constant source of strength throughout this entire process. Supporting me by listening and giving me encouragement during moments where I was feeling down and doubted myself. I am also deeply grateful to Francisco, my source of strength during tough times and whose endless support has played a crucial role in my success. His patience and readiness to help whenever I faced doubts have meant the world to me. Finally, I want to thank my supervisor, Dr. Mijail Naranjo. His guidance and insightful feedback have been fundamental to the development of this dissertation. His fast and thoughtful responses to my questions and his constructive feedback on my drafts have been essential throughout this journey. I’m extremely grateful for the support and availability.
iii ABSTRACT This study challenges the assumption that students become less invested in tasks once they achieve an average grade using AI tools like ChatGPT, this leads to growing concerns about students overreliance on ChatGPT to complete their tasks leading to the decrease of their critical thinking. This study, draws data from 249 participants gathered through an online questionnaire and analyzes it using Partial Least Squares Structural Equation Modeling (PLSSEM), proving that while students are using ChatGPT for quick answers, they often do not engage deeply to enhance their learning or to achieve their full potential, they tend to care less and less about their tasks when they achieve an average grade by just using AI. This leads to the fact that there is a growth of students opportunistic behavior in academic settings, where students settle for average when they could excel. The study points out the need for educational strategies that not only promote responsible use of AI tools but also encourage deeper engagement with learning materials to prevent a decline in academic rigor, reduce their lack of responsibility and promote academic success. KEYWORDS Artificial Intelligence; ChatGPT; Education; Students Opportunism; Students Performance; AI Tools; TTF Sustainable Development Goals (SDG):
iv TABLE OF CONTENTS Statement of Integrity ........................................................................................................ i Dedication and Acknowledgements ................................................................................. ii Abstract ............................................................................................................................ iii List of Figures ..................................................................................................................... v List of Tables ..................................................................................................................... vi List of Abbreviations and Acronyms ................................................................................ vii 1. Introduction .................................................................................................................. 1 2. Theoretical Background ................................................................................................ 3 2.1. Evolution of Educational Technologies ................................................................. 3 2.1.1. Traditional Learning........................................................................................ 3 2.1.2. AI Tool Disruption in the Learning Process .................................................... 4 2.1.3. Opportunism and Self-efficacy in Students’ Learning .................................... 5 2.1.4. Effectiveness of Different AI Tools in Learning Tasks and What Influences Students’ Choices…………………………………………………………………………………………………………………………6 2.2. Individual Learning Styles ...................................................................................... 8 3. Model Building and Hypothesis Development ............................................................. 9 4. Methodological Approach .......................................................................................... 15 5. Results......................................................................................................................... 19 5.1. Measurement Model Evaluation ......................................................................... 19 5.1.1. Internal Consistency ..................................................................................... 19 5.1.2. Convergent Validity ...................................................................................... 19 5.1.3. Discriminant Validity .................................................................................... 20 5.2. Structural Model Evaluation ................................................................................ 22 6. Discussion ................................................................................................................... 24 6.1. Theoretical Implications ...................................................................................... 24 6.2. Practical Implications ........................................................................................... 27 7. Conclusions ................................................................................................................. 29 8. Limitations and Future Work ...................................................................................... 30 Bibliographical References .............................................................................................. 31 Appendix A – NOVA IMS Ethics Committee Approval .................................................... 37
v LIST OF FIGURES Figure 1 – Conceptual model ................................................................................................... 10 Figure 2 – Structural model ...................................................................................................... 22
vi LIST OF TABLES Table 1 - Benefits and Challenges of AI in student learning ............................................. 4 Table 2 - Different AI tools for learning and their impact on students' learning process effectiveness ............................................................................................................. 7 Table 3 - Meaning of each variable presented on the conceptual model ...................... 10 Table 4 - Adapted questions for each construct ............................................................. 15 Table 5 - Sample characteristics ...................................................................................... 18 Table 6 - Construct reliability and validity measures ...................................................... 20 Table 7 - Fornell-Larcker criterion ................................................................................... 21 Table 8 - Cross loadings ................................................................................................... 21 Table 9 - Heterotrait-Monotrait Ratio of correlations (HTMT) ....................................... 22
vii LIST OF ABBREVIATIONS AND ACRONYMS TEL Technology Enhanced Learning AI Artificial Intelligence LLMs Large Language Models TTF Task-technology fit PLS-SEM Partial Least Squares Structural Equation Modeling CA Cronbach’s Alpha CR Composite Reliability AVE Average Variance Extracted HTMT Heterotrait-Monotrait Ratio of correlations VIF Variance Inflation Factor
7 tools can be effectively used to support students’ performance on specific learning tasks, depending on the nature of the task and the specific learning objectives. Building on prior research that has demonstrated the potential of AI tools to enhance student learning (Kooli, 2023), the table 2 aims to enlighten the impacts of commonly used AI tools in education. To achieve this, and going from Osamor et al. (2023) study, where it was presented a wide and developing range of AI-powered educational tools. Table 2 presents a focused analysis of the most frequently used AI tools and their anticipated effects on student learning process effectiveness. Table 2 – Different AI tools for learning and their impact on students’ learning process effectiveness AI Tools Impact on learning process effectiveness ChatGPT ChatGPT provides students’ with personalized feedback, support, interactive learning experiences and has a big availability. Gemini (formerly Google Bard) Gemini also provides students’ with personalized feedback, support, interactive learning experiences and has a big availability. Grammarly Grammarly improves writing skills, identifies grammatical errors, and provides suggestions for improvement. QuillBot QuillBot is very effective at summarizing texts and rewriting sentences. Duolingo Duolingo delivers engaging exercises and interactive games for language learning. DeepL DeepL provides accurate and naturalsounding translations, helping students’ to read and understand foreign texts more effectively. SciSpace SciSpace can interpret a research paper and answer questions about it.
8 By assessing the effectiveness of various AI tools for learning tasks, it is crucial to recognize that students’ choices of learning tools are influenced by a variety of factors. These factors go from individual preferences, experiences, self-efficacy, the nature of the learning task, the availability of tools and their perceptions of the tool’s effectiveness. A study by Kember et al. (2004) validates the Revised Two-Factor Study Process Questionnaire (R-SPQ-2F), which can assess students’ preferences for deep and surface approaches to learning, which may influence their tool choices and validates the Revised Learning Process Questionnaire (R-LPQ-2F), which can provide insights into students’ motives for choosing specific tools. In Dunn L (2002) study it is discussed the role of cognitive theories in learning, emphasizing that students’ are more likely to choose tools that align with their cognitive preferences. And in Hu & Hui (2012) study it is examined the effects of technology-mediated learning, finding that students’ are more likely to engage with tools that are interactive and provide immediate feedback. Basically, students’ choices of learning tools are influenced by a combination of personal factors, task-related factors, and environmental factors. 2.2. INDIVIDUAL LEARNING STYLES As previously discussed, individuals have different learning styles and strategies that work best for them. To elaborate, we will look into the different types of learning styles. Starting off by the older learning style the reading method, which is where students’ read articles, research papers, textbooks, etc. that makes them internalize the information that they want to learn, another common type is the auditory which is when students’ prefer to learn by lectures, recorded audio, videos, etc. (Moussa, 2014). Subsequently, Shabiralyani et al. (2015), mentions that “1% of what is learned is from the sense of TASTE, 1.5% of what is learned is from the sense of TOUCH, 3.5% of what is learned is from the logic of SMELL, 11% of what is educated is from the logic of HEARING and 83% of what is learned is from the sense of SIGHT” (p. 1). Thus, most common are the visual aids, these are students’ that prefer visual aids such as, charts, diagrams, flashcards, etc. because these can illustrate the concepts in a clearer way and make learning more real and more accurate. Since visual aids in learning tend to be the most effective method, they are used in classrooms and other learning environments. This same study reveals that visual aids increase motivation, clarification, increase the vocabulary, saves time, avoids dullness, and promotes a direct experience.
9 3. MODEL BUILDING AND HYPOTHESIS DEVELOPMENT The main goal of this research is to investigate the impact of ChatGPT on students’ opportunistic strategies and check if they only use it exclusively to complete their educational tasks or whether they work more to get the best possible performance. Since AI tools, more specifically ChatGPT continue to evolve and affect various aspects of education, understanding its potential influence on student learning outcomes and critical thinking skills is crucial. Thus, we will be focusing on the development of a conceptual model and its hypothesis. In which it will be able to provide an understanding of the relationships between ChatGPT usage, students’ individual characteristics, and potential outcomes. The model incorporates theoretical constructs inspired by previous studies Goodhue & Thompson (1995), Hoehle & Huff (2012) and Shaw & Gribbins (2005) including self-efficacy, effort expectancy, task-technology fit, and habit. Along with the constructs, different hypotheses were also developed based on the conceptual model, and will guide the implementation of data collection instruments and statistical analyses. The model was created based on the task-technology fit (TTF) theory, which based on previous studies aligns best with the goals and research problems that this study intends to address. In Goodhue & Thompson (1995) study on task-technology fit established a theoretical framework that indicates the interaction between the alignment of information systems with specific tasks and the subsequent performance outcomes. With this, Shaw & Gribbins (2005) sum up in their study that “(1) task-technology fit is a relevant concept to predict information systems success (e.g., performance impacts), and that (2) fit is determined by an appropriate interplay between tasks, technology, and individual, context-related characteristics” (p. 4). In this study, task-technology fit is defined as the match between the demands of the tasks assigned to students’, the quality of AI technologies available like ChatGPT, and the individual characteristics in which students’ use these technologies. The characteristics of the tasks are characterized by their nature. The technology, represented by ChatGPT, is defined by its functionalities, which include communication, information access, and data processing capabilities, and its adaptability to students’ contexts. The individual use context is characterized by their self-efficacy, habits and effort expectancy. The conceptual model, as illustrated in the figure 1, proposes an ideal match among these three dimensions with them being task characteristics, technology characteristics, and individual characteristics (Shaw & Gribbins, 2005) which will facilitate a robust task-technology fit, leading to more effective use of ChatGPT in educational settings.
10 Figure 1 – Conceptual model In order to create a base for this research, it is important to investigate the aspects of every variable in the model and the possible hypotheses that are associated with them. Firstly, by approaching each variable which are displayed by their meaning on table 3: Table 3 – Meaning of each variable presented on the conceptual model Variables Meaning Task Characteristics The task characteristics encompass the attributes of the specific task or goal individuals are trying to achieve with ChatGPT. It considers factors like task complexity, ambiguity and required knowledge. Understanding these characteristics helps to assess how well the ChatGPT’s functionalities align with the demands of the task, ultimately influencing its effectiveness and student acceptance. Technology Characteristics This variable examines the features and functionalities of ChatGPT itself. It approaches aspects like ease of use, accessibility, reliability, flexibility, and the available information provided. Analyzing these characteristics helps to determine how well the tool matches the specific needs and capabilities of the students’, promoting potential success of ChatGPT integration and outcomes.
11 Self-efficacy This variable acknowledges individuals’ confidence in their ability to successfully use ChatGPT and achieve their expected outcomes. High self-efficacy can lead to increased effort, determination, and positive expectations, leading to a greater use of ChatGPT. Effort expectancy This variable focuses on the perceived ease or difficulty of using ChatGPT to achieve the expected results. Lower effort expectancy indicates that the tool is complex or timeconsuming, which can discourage its use and reduce its potential benefits. Habit This variable examines the frequency of using ChatGPT, often influenced by initial experiences and perceived value. Recurrent use can lead to increased student comfort and reliance on the tool, potentially promoting its impact on performance. Task-technology fit This variable combines many of the factors approached above, measuring the degree to which ChatGPT characteristics align with the demands of the specific task. High tasktechnology fit suggests a strong match between tool capabilities and student needs, which leads to improved performance and satisfaction. Use of an AI tool This variable measures the frequency of student engagement with ChatGPT, providing insights into the integration and potential impact of this tool. Effort opportunism This variable evaluates the level to which students’ do their academic tasks with the aid of just ChatGPT itself, further providing insights on students’ overreliance on the tool over their own understanding.
12 Additional effort This variable refers to the extent students’ are willing to improve their academic tasks, using their class materials and own understanding, over the use of ChatGPT. Performance impact This variable evaluates the effect of using the AI tool on individual performance measures, such as task completion time, accuracy, or overall productivity. Looking at this relationship helps to evaluate the effectiveness of ChatGPT. Moreover, it is important to understand what type of relationships exist between the variables. To better understand them the following hypothesis have been created: H1: The characteristics of a task are positively associated with the fit of the technology. According to Shaw & Gribbins (2005) study, a task’s compatibility with a given technology needs careful consideration of various key attributes, the complexity of the task, and the time constraint in which it must be executed. Basically, the effectiveness of the technology in question, ChatGPT, is significantly enhanced when there is a correspondent match with the task characteristics. Therefore, a bigger task-technology fit is achieved when the AI’s capabilities are adapted to accommodate the task’s complications, demands, and time constraints. This alignment is critical, as it directly influences the efficiency and efficacy with which the task can be completed, leveraging the full potential of ChatGPT. H2: The characteristics of the technology are positively associated with the fit of the technology to the task. The hypothesis H2 approaches the multifaceted concept of task-technology fit within the proposed model and acknowledges the positive association between technology characteristics and the task-technology fit. According to Hoehle & Huff (2012) study and the different concepts of fit. H2 takes a direct approach, hypothesizing a simple positive relationship between the features of the technology and the overall fit with the demands of a specific task. Thus, specific features like adaptability, interactivity, and ease of use contribute directly to a better "fit" by aligning with the requests of the task. In simpler terms, the more features a technology has that align with the demands of the task, the better suited it is for accomplishing that task. H3: Individual characteristics, including self-efficacy, effort expectancy and habit are positively associated with the task-technology fit.
13 Fundamentally, if students’ find the technology’s features, functionalities, and overall design to be favorable and aligned with their individual preferences and values, they are more likely to take in a good "task-technology fit" (D’Ambra & Wilson, 2004). This is anticipated to influence the effectiveness of the learning process. Following Al-Rahmi et al. (2023) study, where the alignment between technology and personal values leads to technology adoption. It is hypothesized that a positive perception of technology characteristics will translate to a better fit for the learning task, potentially leading to enhanced learning outcomes. This positive impact could result from the self-efficacy, effort expectancy and habits taken from using ChatGPT which is related to the students’ individual characteristics and preferences. H4: A higher task-technology fit is positively associated with the usage of the technology, including the frequency of use of an AI tool, effort opportunism and additional effort beliefs. Hypothesis H4 proposes a positive association between a higher task-technology fit and the increased use of an AI tool. In Howard & Rose (2019) study says that individuals are more likely to adopt and frequently use technologies that they perceive as valuable and well-suited to their needs. This hypothesis suggests that when an AI tool aligns with the demands and characteristics of a specific task, students’ are more likely to find it useful and effective. Consequently, they are more likely to integrate it into their workflow and engage with it frequently. Thus, H4 emphasizes the importance of matching the capabilities of an AI tool to the specific needs of the task at hand. This fit can also influence student beliefs about effort. Since these tools, ChatGPT, have features that make tasks easier and faster might lead students’ to believe they require less effort to complete the task. On the other hand, these tools can also promote critical thinking, which requires the students’ notes and class materials to believe that they need to make an additional effort to learn more effectively and get better results on their academic tasks. Thus, this fit can influence students’ beliefs about the effort required. Tools that make tasks easier might promote beliefs about reduced effort (effort opportunism), while some students’ might believe that additional effort will lead them to have a better performance. Basically, this "fit" acts as a key driver of student engagement and influences the adoption and use of the technology. H5: A higher task-technology fit is positively associated with better outcomes in terms of performance impact. H5 approaches the potential consequences of a strong task-technology fit. Taking up the understanding that this fit comes from the merge between task characteristics, technology characteristics, and individual characteristics (Al-Rahmi et al., 2023), it is hypothesized to have a positive association with performance impact. Basically, when students’ perceive the technology as well integrated with the learning task and aligned with their needs and preferences, it likely leads to better learning outcomes and increased performance on their assignments.
14 H6: The usage of an AI tool is positively associated with better outcomes in terms of performance impact. Hypothesis H6 approaches the potential benefits of using an AI tool, proposing a positive association between its use and improved outcomes in terms of performance impact. This aligns with the research surrounding technology-enhanced learning and performance, suggesting that effective integration of AI tools can positively impact students’ experiences and achievements, as performance is impacted by the tool’s ability to display the information with the least amount of errors possible (Aljukhadar et al., 2014). Firstly, by frequently using AI tools it can lead to increased cognitive abilities and optimize learning processes, providing capabilities like personalized feedback, automated data analysis, and adaptive content delivery. This can promote problem-solving skills, reduce the amount of information our memory can process at times, and ultimately lead to improved performance on tasks or assessments. Secondly, AI-powered tools can raise or decrease our performance depending on our reduced effort behaviors to just use the tool to pass on an academic task and depending on the extra effort we are willing to make to get the best grade possible. Basically, the impact of AI tools on performance could be positive if it promotes a deeper learning, but negative if it encourages superficial use.
15 4. METHODOLOGICAL APPROACH To assess the students’ willingness to engage in additional work and their opportunistic tendencies, we created an electronic questionnaire, the questions for these questionnaire constructs were designed in accordance with the Portuguese grading system. This system evaluates student tasks on a scale from 0 to 20, where 20 represents the highest possible score and 15 is considered a "good" average grade. This knowledge can then be used to inform educational institutions and guide the development of strategies to maximize the benefits of ChatGPT while reducing its potential negative effects. The research design employed in this study provides a rigorous and versatile approach to investigating the impact of ChatGPT on students’ potential reduced effort. The model also enables rigorous variable exploration, controlled experimentation, and replication of real-world scenarios. The primary tool used is a comprehensive electronic questionnaire incorporating a mix of selfreflective questions. This questionnaire is carefully designed to evaluate two distinct yet interconnected dimensions: student opportunism and potential additional effort. These questions are structured to bring out responses that reflect the students’ practices, behaviors, and self-assessment in relation to their academic environment. The constructs of the model are evaluated in the questionnaire through a numerical scale, each statement within the questionnaire is measured against a seven-point Likert scale, that ranges from 1 to 7, where 1 represents "Strongly Disagree" and 7 represents "Strongly Agree", this scaling mechanism helps students’ to precisely quantify their level of agreement with each statement, ensuring an accurate reflection of their perspectives, as detailed in table 4. Table 4 – Adapted questions for each construct Construct Source Adapted Questions Task characteristics (Sarker & Valacich, 2010) 1) Ideas for assignments to improve content. 2) Review key concepts for quiz preparation. 3) Summarize PDFs for faster understanding.
16 Technology characteristics (Yang et al., 2013) 1) Using ChatGPT aligns well with my learning style. 2) ChatGPT provides helpful feedback. 3) ChatGPT is a highly adaptable technology. Self-efficacy (Sun et al., 2016) 1) I can use ChatGPT to prepare for a task independently. 2) I can use ChatGPT without relying on guidance from others. 3) I can use ChatGPT effectively even without prior experience. 4) I can use ChatGPT for a task with just the online help available. Effort expectancy (Parthasarathy & Bhattacherjee, 1998; Wixom & Todd, 2005) 1) It is easy for me to get the information I need from ChatGPT. 2) I think mastering ChatGPT for tasks is simple. 3) I feel that operating ChatGPT effectively is easy. Habit (Bhattacherjee & Lin, 2015; Venkatesh et al., 2012) 1) Using ChatGPT has become a habit for me. 2) Using ChatGPT comes naturally to me. 3) I automatically think of using ChatGPT when I need to complete a task. Tasktechnology fit (Sun et al., 2016) 1) ChatGPT’s functionalities are compatible with my tasks. 2) ChatGPT makes task preparation easier. 3) ChatGPT facilitates the sequencing of tasks. Use of an AI tool (Kositanurit et al., 2006) 1) How frequently do I use ChatGPT for academic tasks. Effort opportunism (Chen et al., 2022) 1) I will accept to have 15 and not do the manual work. 2) I prefer the 15 and dedicate my time to something else. 3) It is okay to get 15 if only using ChatGPT.
23 relationships are truly meaningful (statistically significant). Additionally, we evaluated the presence of multicollinearity, which can distort results. Since the variance inflation factor (VIF) indicated no such issue, it confirms the independence of this study’s constructs. Our findings suggest that the model variation in student performance is 79.3%. This is because the construct "Performance impact" exhibits a strong relationship and is statistically significant (p-value < 0.05) with task-technology fit (β= 0.483, p-value= 0.000) and use of an AI tool (β= 0.455, p-value= 0.000). Moreover, the data reveals that the construct task-technology fit has a positive relationship and is significantly influenced by technology characteristics (β= 0.261, p-value= 0.000), and by the students’ individual characteristics being these self-efficacy (β= 0.139, p-value= 0.025), effort expectancy (β= 0.242, p-value= 0.000) and habit (β= 0.274, p-value= 0.000), supporting the hypotheses H2, H3a, H3b, H3c, H4a, H4b, H5, H6a and H6c. On the other hand, it was not possible to demonstrate that performance impact was directly impacted by effort opportunism (β= 0.048, p-value= 0.111) and that task-technology fit was impacted with additional effort (β= 0.124, p-value= 0.261). In the same way, due to insufficient statistical evidence task-technology fit was not able to demonstrate its impact on task characteristics (β= 0.115, p-value= 0.086).
24 6. DISCUSSION 6.1. THEORETICAL IMPLICATIONS Having established the results and findings from the questionnaires in the previous chapter, the discussion chapter analyzes the meaning and implications of the results data. With this, this study’s goal is to evaluate the influence of ChatGPT, a well-known and accessible AI tool these days, in students’ within an educational setting, using an adapted TTF model (Goodhue & Thompson, 1995). As the findings support most of the previous research on related constructs, we can conclude that 9 out of the 12 hypotheses are empirically supported. Hypothesis 1 (H1) is supported by the relationship between task characteristics and tasktechnology fit (β= 0.115, p-value= 0.086) it doesn’t suggest a statistically significant impact. This relationship indicates that the inherent characteristics of the tasks might play a role in determining how well the technology fits, but other factors could also be influencing this fit. These findings imply that task characteristics likely play a role in task-technology fit, meaning the cause of using ChatGPT for a task depends more on other factors like technology characteristics and individual characteristics than the nature of an educational task. Looking into Hypothesis 2 (H2), the relationship between the technology characteristics and the task-technology fit is significant and positive (β= 0.261, p-value = 0.000), supporting the task-technology fit model (D’Ambra & Wilson, 2004; Goodhue & Thompson, 1995; Howard & Rose, 2019), meaning that the effectiveness of a technology, in this case ChatGPT in an educational setting is directly related to how well it meets the specific needs of the task that is intended to support, so the more the characteristics of the technology align with the needs of the task, the better the technology is in terms of usability and functionality, meaning a better fit overall. As for the individual characteristics hypotheses of this model (H3a, H3b, H3c), which are selfefficacy (β= 0.139, p-value = 0.025), effort expectancy (β= 0.242, p-value = 0.000) and habit (β= 0.274, p-value = 0.000), they are all statistically significant and have a strong relationship with task-technology fit meaning that the more self-efficacy a student has, the more confidence in their ability to use ChatGPT they have, which significantly improves their perception of the ChatGPT fit for their tasks. This aligns with Bandura & Adams (1977) study on self-efficacy, which suggests that individuals who believe in their abilities are more likely to embrace and effectively use new technologies. It indicates that students’ with high selfefficacy will perceive a better fit between their needs and ChatGPT, thereby increasing the likelihood of adopting and effectively using it, proving H3a. Effort expectancy also has a strong positive relationship to task-technology fit (H3b), indicating that when students’ perceive the use of ChatGPT as easy and requiring minimal effort, they perceive a better alignment with their academic tasks, this aligns with Mitchell & Nebeker (1973) study. This finding means that reinforcing the perceived ease of use plays an important role in technology acceptance. It
25 implies that making ChatGPT more user-friendly and accessible will enhance its fit for educational tasks, thereby increasing its adoption and usage among students’. Lastly, the significant positive relationship between habit and task-technology fit demonstrates that students’ who have established habits of using ChatGPT or other similar technologies will be more willing to perceive a good fit between ChatGPT and their tasks. Hypothesis (H3c) aligns with Fiorella (2020) study, which emphasizes that habitual behavior is a strong predictor of future usage. Therefore, students’ accustomed to using AI tools in their studies will find it easier to integrate ChatGPT into their workflow, perceiving it as a suitable tool for their tasks. In the model, the hypothesis that relates task-technology fit and use of an AI tool (H4a) is significant, indicating a strong relationship (β = 0.686, p-value = 0.000), meaning that the more frequently and effectively an AI tool is used, the better the fit between the technology and the students’ tasks (Goodhue & Thompson, 1995; Howard & Rose, 2019), this relationship implies that the actual use of the AI tool reinforces the perception that the technology is well suited for the tasks it supports, this proves the hypothesis and concludes that practical experience and frequent use help students’ understand and optimize the tool for their specific needs. H4b, shows the relationship between task-technology fit and effort opportunism (β = 0.345, p-value = 0.000) which is statistically significant, effort opportunism, refers to exploiting ChatGPT where less effort is needed to achieve a desired outcome, this has significant impact on the perceived fit of the technology with the students’ tasks, suggesting that the fit between the technology and tasks relies on opportunistic attempts to minimize effort. H4c, illustrates the relationship between task-technology fit and additional effort (β = 0.124, p-value = 0.261) which is not statistically significant, indicating that additional effort, or the extra time invested beyond the typical to complete an educational task, does not significantly influence how well the technology aligns with the task requirements. This suggests that task-technology fit is more dependent on inherent technology-task alignment than on the extra effort students’ put into making the AI tool work for their needs. As for the hypothesis that lead to this study’s outcome, performance impact, which illustrates the overall productivity of students’, the relationship between this variable and tasktechnology fit (H5), use of an AI tool (H6a) and additional effort (H6c) are statistically significant, (β = 0.483, p-value = 0.000), (β = 0.455, p-value = 0.000) and (β = 0.082, p-value = 0.035), respectively. This means that when the technology is well matched to the tasks it supports, there is a positive effect on performance outcomes, ensuring that when the technology aligns with the requirements of the tasks leads to a better fit which improves performance and also the more an AI tool is used, the greater the positive impact on performance (Aljukhadar et al., 2014; Goodhue & Thompson, 1995), suggesting that students’ who frequently and effectively use AI tools experience improvements in performance, due to the AI tool’s ability to enhance productivity. Despite additional effort having a weaker effect it is still statistically significant, suggesting that students’ tend to do a minimal effort in
26 achieving higher grades than the ones ChatGPT can give them even though the extra effort has a slight positive impact on their tasks performance. On the other hand, H6b, is the hypothesis between performance impact and effort opportunism (β = 0.048, p-value = 0.111), leads us to conclude despite not being statistically significant, that minimizing the effort of doing a task through opportunistic strategies does not directly impact performance in a meaningful way, indicating that finding ways to minimize effort does not necessarily translate into significant performance improvements. Finally, the first research question (RQ1): "How does students’ opportunistic use of ChatGPT affect their performance on academic tasks?" is directly addressed by the significant positive relationship between task-technology fit and effort opportunism which illustrates that students’ by taking advantage of ChatGPT where less effort is needed significantly perceive a better fit of the technology with their tasks. This opportunism, where students’ use ChatGPT to minimize effort, significantly influences how they engage with their academic tasks, suggesting that while it may enhance task completion efficiency, it does not necessarily translate to improvements in deeper learning outcomes, as indicated by the non-significant impact on overall performance (H6b). This implies that finding shortcuts through technology does not reflect on better performance, raising questions about the depth of learning and understanding. As for the second research question (RQ2): "How does the additional effort performed by students’ to improve their grades influence their performance on academic tasks when using ChatGPT?". The non-significant relationship between task-technology fit and additional effort indicates that the extra time students’ invest beyond the typical to complete an educational task does not significantly influence how well the technology aligns with task requirements. However, despite its weak effect, H6c shows that even with a small increase in effort, performance is still positively impacted statistically. This implies that more work improves performance without significantly changing the perceived fit of technology, emphasizing the importance of cautiousness on over relying too much on AI tools. Overall this study can be justified with the following theories, the theory of planned behavior and the cognitive theory, which together provide a comprehensive framework for understanding the dynamics of using educational tools like ChatGPT. The theory of planned behavior helps explain the behavioral intentions behind using ChatGPT. It suggests that students’ decisions to use ChatGPT are influenced by their behavioral beliefs (perceptions of the outcomes of using ChatGPT), normative beliefs (social pressures and expectations regarding the use of ChatGPT), and control beliefs (perceived ease or difficulty of using ChatGPT). Collectively, these beliefs shape students’ intentions to adopt behaviors they perceive as beneficial for enhancing their learning experiences with ChatGPT (Bosnjak et al., 2020).
27 The cognitive load theory explains how educational tools like ChatGPT can optimize cognitive processing. This theory is concerned with the management of working memory capacity during learning tasks. It suggests that effective educational tools can help reduce unnecessary cognitive load, thereby freeing up cognitive resources for higher-order thinking. However, the theory also challenges the assumption that a high load is invariably beneficial. It points out that learning can occur even without additional demands on working memory, indicating that the integration and automation of knowledge can happen efficiently with well-designed educational interventions (Schnotz & Kürschner, 2007). These theories collectively point out the complexity of integrating technology like ChatGPT in educational settings, emphasizing the need for tools that are both easy to use and effective in reducing cognitive overload, while still promoting robust learning and retention. 6.2. PRACTICAL IMPLICATIONS The findings suggest that while additional effort can be beneficial, students’ generally do not tend to invest in it. Instead, they often take a more opportunistic approach, relying on AI tools like ChatGPT to complete their tasks to achieve a satisfactory grade rather than attempt for the best possible outcome, the lack of significant impact of additional effort indicates that students’ rely on what ChatGPT provides, and they’re typically content with those results, not showing a strong tendency to do extra work to maximize their grades, encouraging students’ to engage deeply with their coursework beyond using AI tools for quick answers can be a good solution for this, since this can be done by designing assignments that require critical thinking and personal input, pushing students’ to apply their understanding more fully. The research findings also offer several practical implications for educational settings to integrate AI tools. Task characteristics play a significant role in technology fit, but it’s equally important to consider the technology’s features and individual student characteristics, educational settings should prioritize AI tools that align with educational tasks, emphasizing usability, functionality, and features that directly address student needs, this focus will ensure a good fit and maximize the AI tools potential. Developing programs to enhance student self-efficacy and confidence in using AI tools significantly improves perceived task-technology fit, institutions must provide user-friendly interfaces and clear instructions to promote a positive student experience and encourage its adoption, training programs that build students’ confidence through workshops, hands on activities, and mentorships will likely improve AI tools adoption, effectiveness and productivity. Regular use of AI tools helps students’ develop habits and build familiarity, leading to a better perceived fit. Students’ with experience using similar technologies will find integrating tools
28 like ChatGPT easier, leveraging existing student expertise will facilitate a smoother transition, allowing students’ to personalize their use of the tool and optimize it for specific tasks, ultimately leading to more effective learning. Aligning technology with tasks and encouraging frequent, effective AI tool use positively impacts student performance, the emphasis should be on empowering students’ to use AI tools strategically to enhance their performance and learning outcomes.
29 7. CONCLUSIONS This dissertation aimed to study the impact of modern AI tools, particularly ChatGPT, in education. As these technologies are rapidly evolving, it’s important to understand their influence. The findings of this study point to the important role of task-technology fit in students educational performance and technology adoption. The study reveals that while students’ task characteristics modestly affect task-technology fit, and variables such as technology characteristics, self-efficacy, effort expectancy and habits, have a greater influence. These variables contribute to a perception of a strong alignment between ChatGPT and educational tasks, facilitating ease of use and habitual integration. Significantly, the frequent use of AI tools like ChatGPT optimizes their efficacy for students specific needs, although additional effort does not significantly enhance task-technology fit, emphasizing a tendency towards opportunistic behavior rather than striving for higher academic achievements. This suggests a tendency for students to prioritize minimal effort strategies using ChatGPT to achieve satisfactory grades, even though deeper engagement could lead to better learning outcomes. Educational approaches that encourage critical thinking and personal engagement with course materials, beyond simply relying on AI for quick answers, can be a valuable solution to promote deeper learning and motivate students to strive for excellence. Thus, this research offers important insights into how AI tools like ChatGPT impact education, and important insights for the need to promote effective technology usage methods that enhance students’ performance, maximize their learning outcomes and incentive the want for an in-depth knowledge absorption.
30 8. LIMITATIONS AND FUTURE WORK This study acknowledges several limitations. Firstly, one of the limitations this study had was the short number of samples and time we had to collect them. This data was collected from various universities, degrees and levels of study of students’ above 18, it would be interesting if this study’s findings were applied for example in students’ with different nationalities, cultures and traditions. Secondly, this questionnaire was created to be answered anonymously and in an honest way but this method can also introduce bias, as students’ might not accurately reflect their use and perception of AI tools. So further work to improve this would be by employing mixedmethod approaches, like interviews or direct observation, this way more deeper insights could be provided into how students’ actually use AI tools like ChatGPT beyond their questionnaires answers. Furthermore, focusing only on ChatGPT might limit the understanding of how the rest of the AI tools that aid students’ in their educational tasks, impact students’ performance within their opportunistic behaviors and will to do extra work to get better grades. Thus, comparative studies on multiple AI tools could give interesting findings not only on students’ opportunism but also evaluating if these tools are in fact more effective than problematic in educational settings. Lastly, this study may not deeply analyze underlying psychological or behavioral reasons for students’ overreliance on AI tools. So for future work incorporating psychological models could be beneficial for understanding what drives students’ to use AI tools.
31 BIBLIOGRAPHICAL REFERENCES Akkoyunlu, B., & Soylu, M. Y. (2008). International Forum of Educational Technology & Society A Study of Student’s Perceptions in a Blended Learning Environment Based on Different Learning Styles. Source: Journal of Educational Technology & Society, 11(1), 183–193. https://doi.org/10.2307/jeductechsoci.11.1.183 Aljukhadar, M., Senecal, S., & Nantel, J. (2014). Is more always better? Investigating the tasktechnology fit theory in an online user context. Information and Management, 51(4), 391–397. https://doi.org/10.1016/j.im.2013.10.003 Alonso, F., López, G., Manrique, D., & Viñes, J. M. (2005). An instructional model for webbased e-learning education with a blended learning process approach. British Journal of Educational Technology, 36(2), 217–235. https://doi.org/10.1111/j.14678535.2005.00454.x Al-Rahmi, W. M., Al-Adwan, A. S., Al-Maatouk, Q., Othman, M. S., Alsaud, A. R., Almogren, A. S., & Al-Rahmi, A. M. (2023). Integrating Communication and Task–Technology Fit Theories: The Adoption of Digital Media in Learning. Sustainability (Switzerland), 15(10). https://doi.org/10.3390/su15108144 Baartman, L. K. J., & De Bruijn, E. (2011). Integrating knowledge, skills and attitudes: Conceptualising learning processes towards vocational competence. In Educational Research Review (Vol. 6, Issue 2, pp. 125–134). https://doi.org/10.1016/j.edurev.2011.03.001 Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. In Journal of AI (Vol. 52, Issue 7). https://doi.org/10.61969/jai.1337500 Bandura, A., & Adams, N. E. (1977). Analysis of Self-Efficacy Theory of Behavioral Change’. In Cognitive Therapy and Research (Vol. 1, Issue 4). https://doi.org/10.1007/BF01663995 Basbeth, F., Manajemen, P., Asrul, M., & Ibrahim, H. (2018). Four Hours Basic PLS-SEM A Step by Step Guide With Video Clips For Student and Scholar. https://www.researchgate.net/publication/344954763 Bhattacherjee, A., & Lin, C. P. (2015). A unified model of IT continuance: Three complementary perspectives and crossover effects. In European Journal of Information Systems (Vol. 24, Issue 4, pp. 364–373). Palgrave Macmillan Ltd. https://doi.org/10.1057/ejis.2013.36 Bilad, M. R., Yaqin, L. N., & Zubaidah, S. (2023). Recent Progress in the Use of Artificial Intelligence Tools in Education. Jurnal Penelitian Dan Pengkajian Ilmu Pendidikan: ESaintika, 7(3), 279–315. https://doi.org/10.36312/esaintika.v7i3.1377
32 Bosnjak, M., Ajzen, I., & Schmidt, P. (2020). The theory of planned behavior: Selected recent advances and applications. In Europe’s Journal of Psychology (Vol. 16, Issue 3, pp. 352– 356). PsychOpen. https://doi.org/10.5964/ejop.v16i3.3107 Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language Models are Few-Shot Learners. https://commoncrawl.org/the-data/ Castronova, J. A. (2002). Discovery Learning for the 21 st Century: What is it and how does it compare to traditional learning in effectiveness in the 21 st Century? https://www.academia.edu/10871733/Discovery_Learning_for_the_21st_Century_Wh at_is_it_and_how_does_it_compare_to_traditional_learning_in_effectiveness_in_the_ 21st_Century Chen, J., Liu, Z., & Hu, R. (2022). Research on the Influence of Tolerance of Opportunistic Behaviors of Channel Boundaries on the Choice of Response Strategies. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.823416 D’Ambra, J., & Wilson, C. S. (2004). Use of the world wide web for international travel: Integrating the construct of uncertainty in information seeking and the Task-Technology Fit (TTF) model. Journal of the American Society for Information Science and Technology, 55(8), 731–742. https://doi.org/10.1002/asi.20017 Drnovšek, M., Wincent, J., & Cardon, M. S. (2010). Entrepreneurial self-efficacy and business start-up: Developing a multi-dimensional definition. International Journal of Entrepreneurial Behaviour and Research, 16(4), 329–348. https://doi.org/10.1108/13552551011054516 Dunn L. (2002). LEARNING AND TEACHING BRIEFING PAPERS SERIES Oxford Centre for Staff and Learning Development. www.brookes.ac.uk/services/ocsld/resources/briefing_papers Dutta, S., & Chakraborty, T. (2023). Thus Spake ChatGPT. Communications of the ACM, 66(12), 16–19. https://doi.org/10.1145/3616863 Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71. https://doi.org/10.1016/j.ijinfomgt.2023.102642