Chatbot as a learning assistant: Factors influencing adoption and recommendation

i CHATBOT AS A LEARNING ASSISTANT: FACTORS INFLUENCING ADOPTION AND RECOMMENDATION Caio Lemos Moraes Dissertation presented as partial requirement for obtaining the Master’s degree in Information Management ii NOVA Information Management School Instituto Superior de Estatística e Gestão de Informação Universidade Nova de Lisboa CHATBOT AS A LEARNING ASSISTANT: FACTORS INFLUENCING ADOPTION AND RECOMMENDATION by Caio Lemos Moraes Dissertation presented as partial requirement for obtaining the Master’s degree in Information Management, with a specialization in Marketing Intelligence. Advisor: Tiago Oliveira, PhD Co Advisor: Gonçalo Baptista July 2021 iii ACKNOWLEDGEMENTS To Professor Tiago Oliveira I would like to acknowledge with gratitude the support and guidance throughout this research. To Gonçalo Baptista I would like to express my sincere gratitude for his patience, motivation, and suggestions. To my family and friends for always being there when I needed it. I would not have made it this far without their support. My heartfelt thanks. iv ABSTRACT Soon, it is expected that artificial intelligence (AI) may replace many jobs whose work is based in repetitive tasks. Considering the role that this technology will play in our lives over the next few years, it would be interesting to take advantage of its potential now and use it as a transformation agent in the educational system. This study aims to evaluate the main drivers for adoption and recommendation of chatbots as a learning assistant to students in higher education. The research uses an innovative model based on gamification affordance, support construct from the students control model, and performance expectance, hedonic motivation, and behavioural intention to adopt constructs from the well-known UTAUT2 model. The model was empirically assessed using structural equation modelling (SEM) based on 302 responses from an online survey conducted in a South American country, Brazil. Support and hedonic motivation were found to be the most significant drivers for behaviour intention to adopt a chatbot. To explain the antecedents of the intention to recommend a chatbot, support and behavioural intention to adopt were the most important drivers found. For scholars, this research brings new material for further exploration of individual drivers for technology adoption and recommendation. For practitioners, knowing the main drivers for adoption and recommendation of a chatbot enables them to develop a technology with higher chances of absorption in the market. KEYWORDS Chatbot; Gamification; Mobile learning; Personal Learning Environment. v INDEX 1. Introduction .................................................................................................................. 1 2. Literature review .......................................................................................................... 2 2.1. Chatbots Applied in Education .............................................................................. 2 2.2. Students Control Model for Online Learning ........................................................ 2 2.3. Gamified Learning Environments .......................................................................... 3 3. Research Model ............................................................................................................ 5 3.1. Performance expectancy (PE) ............................................................................... 6 3.2. Hedonic motivation (HM) ...................................................................................... 6 3.3. Gamification affordance ........................................................................................ 7 3.4. Support (SP) ........................................................................................................... 8 3.5. Behavioural intention to adopt (BI)....................................................................... 8 4. research Methodology ................................................................................................. 9 4.1. Measurement ........................................................................................................ 9 4.2. Data collection ....................................................................................................... 9 5. analysis and results..................................................................................................... 11 5.1. Measurement model ........................................................................................... 11 5.2. Structural model and hypotheses testing ........................................................... 13 6. Discussion ................................................................................................................... 16 6.1. Theoretical implications ...................................................................................... 17 6.2. Practical implications........................................................................................... 18 6.3. Limitations and future research .......................................................................... 18 7. Conclusions ................................................................................................................. 20 8. References .................................................................................................................. 21 9. Appendix ..................................................................................................................... 28 vi LIST OF FIGURES Figure 1 - Research model.......................................................................................................... 5 Figure 2 - Research model with results .................................................................................... 13 Figure 3a - Moderation effect of performance expectancy on behavioral intention to adopt over intention to recommend.......................................................................................... 17 Figure 3b - Moderation effect of competition on hedonic motivation over intention to recommend ...................................................................................................................... 17 Figure 3c - Moderation effect of support on behavioral intention to adopt over intention to recommend ...................................................................................................................... 17 Figure 3d - Moderation effect of competition on performance expectancy over intention to recommend ...................................................................................................................... 17 vii LIST OF TABLES Table 1 – Demographic data of responses ............................................................................... 10 Table 2 – Descriptive statistics, correlation, composite reliability (CR), and average variance extracted (AVE). ............................................................................................................... 11 Table 3 – Loadings and cross-loadings. .................................................................................... 12 Table 4 – Heterotrait-Monotrait Ratio (HTMT)........................................................................ 13 Table 5 – Conclusions for hypothesis ....................................................................................... 15 viii LIST OF ABBREVIATIONS AND ACRONYMS AI Artificial Intelligence UTAUT2 Unified Theory of Acceptance and Use of Technology MOOC Massive Open Online Course PLE Personal Learning Environment SEM Structural equation modelling AVE Average variance extracted HTMT Heterotrait-monotrait ratio VIF Variance inflation factor 1 1. INTRODUCTION Artificial intelligence (AI) has the potential to drastically change our society in the near future. Eventually, AI will replace jobs that mainly involve repetitive tasks while creating opportunities in areas that require more social, interpersonal, and creativity skills (Makridakis, 2017). At the same time, AI demonstrates potential to be applied to education. It can enhance learners’ experiences to respond to the future needs that AI itself will create (Bates, Cobo, Mariño, & Wheeler, 2020). Learning involves emotional and personal traits that should be addressed when using technology to enhance learners experience (Garrison, 2007). Through chatbots, AI can play an important role supporting the communication and emotional aspects of the learning process, delivering a more personal experience to each individual (Bates et al., 2020). Chatbots are gaining more space in the educational sector. Fadhil and Villafiorita (2017) presented promising findings indicating its use as a framework for gamification of learning. As presented by Markopoulos et. al (2015), the use of gamification in education can have a positive impact because it can increase intrinsic motivation, collaboration, and engagement among learners. There are few studies done about chatbot adoption in the learning segment, but there are even fewer studies regarding the topic in a South American country, Brazil when compared to European countries and the United States. Moreover, the studies regarding chatbot adoption for education started very recently, such as Boeding (2020) and Sandu and Gide (2019). Nonetheless, most of the articles about gamification in education focus on different information systems, such as Massive Open Online Course (MOOC) and web 2.0 technologies. Hence, there is a lack of studies using chatbots as a framework for gamification of learning. Learning using a chatbot means that students are proactively defining their learning preferences, and to help in this end, this research will assess the support construct from a pedagogy-driven model proposed by E. Rahimi, Van Den Berg, and Veen (2015a). Combining it with constructs from UTAUT2 from Venkatesh, Thong, and Xu (2012a), and gamification affordance from Suh, Cheung, Ahuja, and Wagner (2017), the proposed model will propose a holistic way to explore the main factors that may influence chatbot adoption and recommendation. Consequently, this research contributes to the scarce literature in this field by offering a unique model. To the best of our knowledge, there is no research assessing the intention to recommend chatbots in the education sector. The construct is relevant to the education sector because users can influence the visibility of the technology. A greater audience would be reached, which could bring more customers. Finally, as the technology was recently adopted in the educational sector, there is still a lack of studies on chatbot as a learning assistant. Hence, this study may add valuable knowledge to chatbot developers and the educational industry to develop market strategies to achieve better results in the educational market. 8 H5c. Competition affordance (CO) moderates hedonic motivation (HM) and intention to recommend (REC) in such a way that the relationship will be stronger among people who are more competitive. H5d. Competition affordance (CO) moderates performance expectancy (PE) and intention to recommend (REC) in such a way that the relationship will be stronger among people who are more competitive. 3.4. SUPPORT (SP) The lack of support makes students feel unmotivated and isolated leading to problematic academic behaviours (Ford & Roby, 2013). Support can improve scholars help-seeking skills and enhance their engagement in the learning process improving their learning skills (Roll, Aleven, McLaren, & Koedinger, 2011). The learning outcomes can be tied to students engagement, which relates the quality and quantity of their involvement with learning activities (Krause & Coates, 2008). The engagement is shown to increase the number of users that write positive reviews that may influence others (J. Wu, Fan, & Zhao, 2018). Chatbots have the potential to create a supportive environment increasing student’s interest and motivation (Tarouco et al., 2018). Therefore, we hypothesize that: H6a. Support (SP) will positively affect behavioural intention to adopt (BI) a chatbot as a learning assistant. H6b. Support (SP) will positively affect behavioural intention to recommend (REC) a chatbot as a learning assistant. Since students will seek help during its learning process, autonomy also impacts support, but it does not necessary mean that they need to give up of their control (Garrison & Baynton, 1987). Therefore, we hypothesize that: H6c. Support (SP) moderates behavioural intention to adopt (BI) and behavioural intention to recommend (REC) in such a way that the relationship will be stronger among people who feel that have more support (SP). 3.5. BEHAVIOURAL INTENTION TO ADOPT (BI) The act of recommending a technology to others is considered a post-adoption behaviour, and it is repeatedly been ignored by researchers that prefer to emphasize their studies on use (Lancelot Miltgen, Popovič, & Oliveira, 2013). Users that demonstrate more interest to adopt a new technology also have shown a greater chance to become adopters (Leong, Hew, Tan, & Ooi, 2013), thereafter more willing to recommend the technology to others (Lancelot Miltgen et al., 2013). Students tend to recommend more when they see that they can also gain when more students join the technology (Greenacre, Freeman, Cong, & Chapman, 2014). Chatbots are a promising technology to be applied to improve students’ learning goals (Colace et al., 2018). Therefore, we hypothesize that: H7. Behavioural intention to adopt (BI) will positively affect behavioural intention to recommend (REC) a chatbot for educational purposes. 9 4. RESEARCH METHODOLOGY 4.1. MEASUREMENT Based on the research model, an English-language questionnaire was created and reviewed for content validity by a group of information systems academics. The questionnaire contains four sections: UTAUT2 data constructs (Performance Expectancy, Hedonic Motivation and Behavioural intention), Gamification Affordance constructs (Rewards, Competition and Status), Students Control construct (Support) questions and finally, general information and demographic characteristics. The items and scales for the UTAUT2 constructs were adapted from Venkatesh et al. (2003; 2012a), the Support construct from Rahimi et al. (2015a), and Gamification Affordance constructs from Suh et al. (2017). Each item was measured on a seven-point Likert scale whose answer choice ranges from “strongly disagree” (1) to “strongly agree” (7). Age was measured in years, and gender was coded using a 0 (women) or 1 (men). The items for all constructs are included in Appendix A. The initial questionnaire was translated into Portuguese because data collection takes place in Brazil. The questionnaire was revised by a Brazilian academic in order to adapt it to the characteristics of the local Portuguese language. Finally, the questionnaire was translated back into English, to ensure the consistency of its content (Brislin, 1970). 4.2. DATA COLLECTION As claimed by Venkatesh et al. (2003), studies of technology acceptance have been widely developed using survey research. Therefore, it was designed as an online survey instrument with the revised Portuguese version of the questionnaire hosted by, SurveyMonkey, one of the main service providers for research papers and data collection. The target population comprised individual adults that are students attending a higher education degree or that have attended one in the past 3 years. Due to the target population, an e-mail list of students from several Brazilian universities was collected and used exclusively for this purpose. The link to the online questionnaire was also shared on social networks specifically in university discussion group pages. The survey was pilot tested with 25 participants within the target population who were not included in the final sample. The pilot showed confirmed that scales were valid and reliable. After the period of 16 weeks that started in late May 2020, a total of 597 people has visited the survey, 302 replied to it, representing a 50,6% response rate. 55.3% of the respondents were female and 44.7% male, most of them aged between 18 and 34 years with and education level up to the master’s degree, while a small group of 3% with doctor degree, as illustrated by table 1. 10 Sample (n=302) Age Gender Education 18-24 98 32.5% Female 167 55.3% Bachelor’s degree 180 59.6% 25-34 124 41.1% Male 135 44.7% Professional degree 72 23.8% 35-44 46 15.2% Master’s degree 41 13.6% 45-54 24 7.9% Doctor degree 9 3.0% 55-64 10 3.3% Table 1 – Demographic data of responses 11 5. ANALYSIS AND RESULTS Structural equation modelling (SEM) was used to test and assess the theoretical causal relationships. SEM is a statistical method used in explanatory research to evaluate the qualitative causal relationship of a model (Byrne, 2013). The research model was estimated with partial least squares (PLS-SEM), which is a variance-based method, with SmartPLS 3 software (Ringle, C.M., Wend, S., Becker, 2015). This method presents some important advantages and it is capable to be applied in many research approaches (Henseler, Ringle, & Sinkovics, 2009), and for studying complex models with great number of constructs (Chin, 1998). The dimension of the sample is more than 10 times greater than the maximum number of paths directed to a construct (Gefen & Straub, 2005), hence PLS-SEM is suitable for estimation. This technique has minimal restrictions when it comes to residual distributions and sample sizes compared to other SEM such as covariance-based techniques (Chin, 1998). 5.1. MEASUREMENT MODEL The measurement model was estimated based on construct reliability, indicator reliability, convergent validity, and discriminant validity. Table 2 presents that all constructs have composite reliability above 0.7, which is a strong indicator that the constructs are reliable (Straub, 1989). The criteria for indicator reliability is that loading should be higher than 0.7 and loadings below 0.4 should be eliminated (Churchill, 1979). The loadings presented on this research are higher than 0.7 and are statistically significant at 0.01, suggesting a good indicator reliability of the instrument. Average variance extracted (AVE) was the method used to test the convergence validity. All the constructs had a value above the minimal acceptable value of 0.50, meaning the latent variable explains more than half of the variance of its indicators (Fornell & Larcker, 1981; Hair, Sarstedt, Ringle, & Mena, 2012; Henseler et al., 2009). The discriminant validity of the constructs was evaluated with Fornell-Larcker, cross-loadings, and heterotrait-monotrait ratio (HTMT) criteria. The first criterion states that the square root of AVE should be greater than the correlations between the construct (Fornell & Larcker, 1981). The second criterion requires that the loading of each indicator should be greater than all cross-loadings (Chin, 1998; Götz, Liehr-Gobbers, & Krafft, 2010; Grégoire & Fisher, 2006). The square roots of AVEs (diagonal elements) presented on table 2 are higher than the correlation between each pair of constructs (off-diagonal elements). The patterns of loading are greater than cross-loading as shown on table 3. Construct Mean SD CR CA PE HM RW ST CO AT SP BI REC PE 5.52 1.43 0.96 0.95 0.92 HM 5.49 1.67 0.97 0.96 0.80 0.96 Rewards 5.48 1.69 0.97 0.96 0.53 0.54 0.96 Status 3.82 1.79 0.96 0.93 0.35 0.38 0.55 0.94 Competition 4.89 1.72 0.92 0.87 0.36 0.42 0.64 0.68 0.89 Support 5.40 1.48 0.96 0.95 0.74 0.77 0.53 0.45 0.44 0.84 0.90 BI 4.63 1.89 0.97 0.96 0.71 0.74 0.59 0.51 0.45 0.78 0.72 0.93 Recommend 5.80 1.38 0.89 0.74 0.64 0.70 0.57 0.37 0.40 0.71 0.73 0.76 0.89 Table 2 – Descriptive statistics, correlation, composite reliability (CR), and average variance extracted (AVE). 12 Construct ITEM PE HM RW ST CO AT SP BI REC Performance Expectancy PE1 0.907 0.736 0.482 0.267 0.298 0.625 0.654 0.623 0.603 PE2 0.929 0.710 0.508 0.330 0.348 0.644 0.664 0.647 0.591 PE3 0.899 0.689 0.468 0.299 0.297 0.657 0.682 0.652 0.582 PE4 0.937 0.746 0.481 0.361 0.351 0.719 0.714 0.681 0.567 PE5 0.920 0.772 0.496 0.365 0.362 0.692 0.689 0.664 0.587 Hedonic Motivation HM1 0.747 0.952 0.523 0.347 0.399 0.740 0.748 0.695 0.678 HM2 0.767 0.971 0.512 0.381 0.411 0.710 0.731 0.728 0.673 HM3 0.777 0.955 0.528 0.375 0.398 0.720 0.739 0.703 0.668 Rewards RW1 0.495 0.510 0.962 0.575 0.599 0.580 0.494 0.553 0.524 RW2 0.525 0.526 0.972 0.528 0.619 0.614 0.518 0.565 0.547 RW3 0.507 0.528 0.949 0.494 0.626 0.623 0.522 0.577 0.570 Status ST1 0.348 0.382 0.581 0.935 0.682 0.508 0.438 0.489 0.363 ST2 0.332 0.367 0.456 0.936 0.602 0.441 0.425 0.492 0.337 ST3 0.311 0.326 0.516 0.938 0.618 0.427 0.407 0.459 0.351 Competition CO1 0.384 0.439 0.626 0.636 0.950 0.523 0.444 0.467 0.415 CO2 0.355 0.406 0.633 0.590 0.925 0.491 0.454 0.395 0.398 CO3 0.176 0.230 0.407 0.603 0.783 0.320 0.221 0.300 0.193 AT5 0.637 0.626 0.572 0.438 0.457 0.896 0.732 0.681 0.614 AT6 0.633 0.654 0.577 0.462 0.455 0.895 0.715 0.747 0.652 Support SP1 0.705 0.710 0.495 0.394 0.378 0.777 0.921 0.692 0.662 SP2 0.638 0.696 0.462 0.431 0.431 0.740 0.896 0.638 0.622 SP3 0.612 0.684 0.441 0.409 0.393 0.746 0.891 0.659 0.649 SP4 0.682 0.712 0.522 0.340 0.329 0.760 0.875 0.640 0.672 SP5 0.723 0.698 0.512 0.418 0.393 0.756 0.911 0.660 0.696 SP6 0.622 0.641 0.429 0.448 0.445 0.752 0.885 0.605 0.597 Behavioural Intention BI1 0.647 0.657 0.570 0.494 0.436 0.748 0.680 0.944 0.704 BI2 0.674 0.734 0.541 0.476 0.413 0.751 0.695 0.956 0.728 BI3 0.668 0.690 0.531 0.513 0.405 0.738 0.695 0.943 0.693 BI4 0.664 0.671 0.458 0.444 0.341 0.627 0.622 0.890 0.679 BI5 0.663 0.690 0.634 0.463 0.476 0.772 0.683 0.925 0.751 Intention to Recommend REC1 0.651 0.663 0.523 0.404 0.376 0.669 0.672 0.786 0.911 REC2 0.474 0.583 0.493 0.252 0.328 0.596 0.620 0.559 0.872 Table 3 – Loadings and cross-loadings. Finally, the discriminant validity criterion is accomplished if all HTMT ratios are below the threshold of 0.9 (Henseler et al., 2014). In Table 4, we see that all HTMT ratios scored below 0.9; Hence, the constructs’ discriminant validity is confirmed. The measurement model results for construct reliability, indicator reliability, convergent validity, and discriminant validity meet the criteria indicating that the constructs are statistically distinct and can be used to test the structural model. 13 Constructs PE HM RW ST CO SP BI REC Performance expectancy Hedonic motivation 0,833 Rewards 0,554 0,567 Status 0,374 0,406 0,587 Competition 0,376 0,441 0,683 0,762 Support 0,777 0,807 0,557 0,482 0,462 Behavioural intention to adopt 0,743 0,770 0,611 0,542 0,475 0,756 Intention to recommend 0,748 0,827 0,673 0,441 0,465 0,859 0,889 Table 4 – Heterotrait-Monotrait Ratio (HTMT). 5.2. STRUCTURAL MODEL AND HYPOTHESES TESTING The multicollinearity of all variables was tested using the variance inflation factor (VIF). All VIF are lower than the threshold of 5, meaning the model does not have a multicollinearity problem (Hair, Ringle, & Sarstedt, 2011). The structural model was estimated using R2 measures and path coefficients’ level of significance. The model results are displayed on fig. 2, as well as the path coefficients. The significance of the path coefficients was assessed using bootstrapping procedure with 5000 iterations of resampling (Chin, 1998). Note: *** p < 0.01; ** p < 0.05; p < 0.10 Figure 2 - Research model with results The model explains 67.3% of the variation in behavioural intention to adopt, with the following variables presenting a statistically significant relationship, namely performance expectancy (𝛽 󰆹 = 0.192; p < 0.01), hedonic motivation (𝛽 󰆹 = 0.293; p < 0.01), rewards (𝛽 󰆹 = 0.138; p < 0.05), status (𝛽 󰆹 = 0.187; p < 0.01), and support (𝛽 󰆹 = 0.210; p < 0.01). Respectively hypotheses H1a, H2a, H3a, H4a, H6a are confirmed. On the other hand, competition (𝛽 󰆹 = -0.056; p > 0.10) was found not statistically significant, therefore hypotheses H5a is not confirmed. The variation in intention to recommend is explained by 70.3% through hedonic motivation (𝛽 󰆹 = 0.220; p < 0.01), rewards (𝛽 󰆹 = 0.137; p < 0.05), status (𝛽 󰆹 = -0.108; p < 0.05), support (𝛽 󰆹 = 0.240; p < 0.01), and behavioural intention to adopt (𝛽 󰆹 = 0.439; p < 0.01). Thus, hypotheses H2b, H3b, H4b, H6b, H7 are confirmed. However, performance expectancy (𝛽 󰆹 = -0.076; p > 0. 10), competition (𝛽 󰆹 = 0.010; p > 0.10) are not statistically significant. Consequently, hypothesis H1b and H5b are not confirmed. R² = 67.3% R² = 70.3% 14 Finally, several statically significant moderation effect were found, namely performance in relationship between behavioural intention to adopt and intention to recommend (𝛽 󰆹 = 0.148; p < 0.05), competition in relationship between hedonic motivation and intention to recommend (𝛽 󰆹 = 0.210; p < 0.01), competition in relationship between performance expectancy and intention to recommend (𝛽 󰆹 = -0.243; p < 0.01), and support in relationship between behavioural intention to adopt and intention to recommend (𝛽 󰆹 = -0.182; p < 0.01). Thus, hypothesis H1c, H5c, H5d, and H6c are confirmed. The supported hypothesis are presented in table 5. 15 Independent variable Dependent Variable Moderator Conclusion H1a Performance Expectancy (PE) Behavioural intention to adopt (BI) n.a. supported H1b Performance Expectancy (PE) Behavioural Intention to Recommend (REC) n.a. not supported H1c Performance Expectancy (PE) * Behavioural intention to adopt (BI) Behavioural Intention to Recommend (REC) Performance Expectancy (PE) supported H2a Hedonic Motivation (HM) Behavioural intention to adopt (BI) n.a. supported H2b Hedonic Motivation (HM) Behavioural Intention to Recommend (REC) n.a. supported H3a Rewards (RW) Behavioural intention to adopt (BI) n.a. supported H3b Rewards (RW) Behavioural Intention to Recommend (REC) n.a. supported H4a Status (ST) Behavioural intention to adopt (BI) n.a. supported H4b Status (ST) Behavioural Intention to Recommend (REC) n.a. supported H5a Competition (CO) Behavioural intention to adopt (BI) n.a. not supported H5b Competition (CO) Behavioural Intention to Recommend (REC) n.a. not supported H5c Competition (CO) * Hedonic Motivation (HM) Behavioural Intention to Recommend (REC) Competition (CO) supported H5d Competition (CO) * Performance Expectancy (PE) Behavioural Intention to Recommend (REC) Competition (CO) supported H6a Support (SP) Behavioural intention to adopt (BI) n.a. supported H6b Support (SP) Behavioural Intention to Recommend (REC) n.a. supported H6c Support (SP) * Behavioural intention to adopt (BI) Behavioural Intention to Recommend (REC) Support (SP) supported H7 Behavioural Intention to Adopt (BI) Behavioural Intention to Recommend (REC) n.a. supported Table 5 – Conclusions for hypothesis 16 6. DISCUSSION This research corroborates the previous studies that show that when the students have more control over their learning strategies (Koehler & Mishra, 2006; E. Rahimi, Van den Berg, & Veen, 2013; Valtonen et al., 2012), and perceive that the technology might improve their performance and enable a more pleasant experience (Markopoulos et al., 2015; Tsay, Kofinas, & Luo, 2018) will increase the behavioural intention to adopt the technology (Almaiah et al., 2019; J. Wu et al., 2018). This also positively affects the behavioural intention to recommend the chatbot to others (Huang et al., 2017; Kuester & Benkenstein, 2014; Loureiro et al., 2018). The research supported the positive effect that performance expectancy has on the behavioural intention to adopt, as earlier studies have suggested (Almaiah et al., 2019), but it was not able to explain the intention to recommend the technology. Hedonic motivation was found to be an important driver for behavioural intention to adopt as well as for the intention to recommend, as expected according to previous studies (Moorthy et al., 2019; Oluwajana et al., 2019; S. J. Yoo & Han, 2013). The empirical results showed that rewards is significant to predict the behavioural intention to adopt, as previously presented by Ortega-Arranz et al. (2019). Rewards was also found valid to explain the intention to recommend, as previous research identified (Kuester & Benkenstein, 2014; Teixeira & Mendes, 2019). Similarly, status was valid to predict behavioural intention to adopt but acted as a negative driver when it comes to the intention to recommend. It implies that those who value status more, might not be willing to recommend as much those who do not value status that much. Status can be related with how unique a user perceives himself (Latter, Phau, & Marchegiani, 2012), hence recommending the chatbot could impact on their status. The study results failed to validate the direct role of competition to predict both behavioural intention to adopt and to recommend. The support construct was shown to be valid to predict behavioural intention to adopt as well as to predict intention to recommend, aligned with some previous studies (Krause & Coates, 2008; Roll et al., 2011). As displayed in Fig.3a the moderation effect of performance expectancy on the relationship between behavioural intention to adopt and intention to recommend has been shown to be stronger among those with high levels of performance expectancy than for people with low levels. Fig 3b shows that the competition moderator presents a stronger influence of high hedonic motivation on intention to recommend when the user is more competitive. The slope on Fig 3c implies that the relationship between behavioural intention to adopt and intention to recommend is weaker for learners with higher levels of support than for learners with lower levels of support. Finally, Fig 3d illustrates that the competition moderator presents a stronger impact of high performance expectancy among those learners with lower competition. This is interesting because comparing fig. 3b and 3d, we can interpret that students want to see competition as a fun moment to learn. They expect it to be a pleasant experience in their learning path. When students face competition as a way to improve their performance, it may be perceived as not so interesting. They do not seem to want to compete with the direct goal of performing better on learning, they want to compete believing that the activity will be an amusing way to learn. 17 Figure 3a - Moderation effect of performance expectancy on behavioral intention to adopt over intention to recommend Figure 3b - Moderation effect of competition on hedonic motivation over intention to recommend Figure 3c - Moderation effect of support on behavioral intention to adopt over intention to recommend Figure 3d - Moderation effect of competition on performance expectancy over intention to recommend Regarding possible particularities of the sample, in 2018 Brazil was the 8th country in the world on the number of websites with chatbot per 1000 persons (Goboomtown, 2019). It is likely that users from the sample have had some type of contact with one or more chatbots in their lives, and that the past experiences might play a role on their expectations with a chatbot as a learning assistant. 6.1. THEORETICAL IMPLICATIONS Our research provides several contributions for the literature. First, as mentioned in the literature review, the field of chatbot adoption for higher education is relatively new. Therefore, the research on the topic is still scarce, especially when it comes to Brazil. Second, this study combines three distinguish models introducing moderators to achieve a broader view of the main drivers regarding chatbot in higher education adoption and recommendation. Third, as previously discussed, researchers have focused their attention on 4 4,5 5 5,5 6 6,5 7 Low BI High BI Intention to recommend Performance Expectancy Low Performance expectancy High Performance expectancy 4 4,5 5 5,5 6 6,5 7 Low HM High HM Intention to recommend Competition Low Competition High Competition 4 4,5 5 5,5 6 6,5 7 Low BI High BI Intention to recommend Support Low Support High Support 4 4,5 5 5,5 6 6,5 7 Low PE High PE Intention to recommend Competition Low Competition High Competition 24 Kuester, M., & Benkenstein, M. (2014). Turning dissatisfied into satisfied customers: How referral reward programs affect the referrer[U+05F3]s attitude and loyalty toward the recommended service provider. Journal of Retailing and Consumer Services, 21(6), 897–904. https://doi.org/10.1016/j.jretconser.2014.07.005 Lancelot Miltgen, C., Popovič, A., & Oliveira, T. (2013). Determinants of end-user acceptance of biometrics: Integrating the “big 3” of technology acceptance with privacy context. Decision Support Systems, 56(1), 103–114. https://doi.org/10.1016/j.dss.2013.05.010 Latter, C., Phau, I., & Marchegiani, C. (2012). The roles of consumers need for uniqueness and status consumption in haute couture luxury brands. Journal of Global Fashion Marketing, 1(4), 206– 214. https://doi.org/10.1080/20932685.2010.10593072 Lee, C. L., & Yang, H. J. (2011). Organization structure, competition and performance measurement systems and their joint effects on performance. Management Accounting Research, 22(2), 84– 104. https://doi.org/10.1016/j.mar.2010.10.003 Leong, L. Y., Hew, T. S., Tan, G. W. H., & Ooi, K. B. (2013). Predicting the determinants of the NFCenabled mobile credit card acceptance: A neural networks approach. Expert Systems with Applications, 40(14), 5604–5620. https://doi.org/10.1016/j.eswa.2013.04.018 Loureiro, S. M. C., Cavallero, L., & Miranda, F. J. (2018). Fashion brands on retail websites: Customer performance expectancy and e-word-of-mouth. Journal of Retailing and Consumer Services, 41(June 2017), 131–141. https://doi.org/10.1016/j.jretconser.2017.12.005 Lung-Guang, N. (2019). Decision-making determinants of students participating in MOOCs: Merging the theory of planned behavior and self-regulated learning model. Computers and Education, 134(October 2018), 50–62. https://doi.org/10.1016/j.compedu.2019.02.004 Makridakis, S. (2017). The forthcoming Artificial Intelligence (AI) revolution: Its impact on society and firms. Futures, 90, 46–60. https://doi.org/10.1016/j.futures.2017.03.006 Markopoulos, A. P., Fragkou, A., Kasidiaris, P. D., & Davim, J. P. (2015). Gamification in engineering education and professional training. International Journal of Mechanical Engineering Education, 43(2), 118–131. https://doi.org/10.1177/0306419015591324 Moorthy, K., Yee, T. T., T’ing, L. C., & Kumaran, V. V. (2019). Habit and hedonic motivation are the strongest influences in mobile learning behaviours among higher education students in Malaysia. Australasian Journal of Educational Technology, 35(4), 174–191. https://doi.org/10.14742/ajet.4432 Oliveira, T., Thomas, M., Baptista, G., & Campos, F. (2016). Mobile payment: Understanding the determinants of customer adoption and intention to recommend the technology. Computers in Human Behavior, 61(2016), 404–414. https://doi.org/10.1016/j.chb.2016.03.030 Oluwajana, D., Idowu, A., Nat, M., Vanduhe, V., & Fadiya, S. (2019). The adoption of students’ hedonic motivation system model to gamified learning environment. Journal of Theoretical and Applied Electronic Commerce Research, 14(3), 156–167. https://doi.org/10.4067/S071818762019000300109 Ortega-Arranz, A., Bote-Lorenzo, M. L., Asensio-Pérez, J. I., Martínez-Monés, A., Gómez-Sánchez, E., & Dimitriadis, Y. (2019). To reward and beyond: Analyzing the effect of reward-based strategies in a MOOC. Computers and Education, 142(March), 103639. https://doi.org/10.1016/j.compedu.2019.103639 25 Pluralsight. (2015). Keep Your Players Engaged: Why Rewards and Medals Play a Key Factor in Your Game. Retrieved November 14, 2020, from https://www.pluralsight.com/blog/filmgames/keep-players-engaged-rewards-medals-play-key-factor-game Rahimi, E., Van den Berg, J., & Veen, W. (2013). Investigating teachers’ perception about the educational benefits of Web 2.0 personal learning environments. E-Learning Papers, (35), 2013, (November), 1–13. Rahimi, E., Van Den Berg, J., & Veen, W. (2015). Facilitating student-driven constructing of learning environments using Web 2.0 personal learning environments. Computers and Education, 81, 235–246. https://doi.org/10.1016/j.compedu.2014.10.012 Rahimi, Ebrahim, van den Berg, J., & Veen, W. (2015). A learning model for enhancing the student’s control in educational process using Web 2.0 personal learning environments. British Journal of Educational Technology, 46(4), 780–792. https://doi.org/10.1111/bjet.12170 Rahimi, Ebrahim, Van Den Berg, J., & Veen, W. (2015). A learning model for enhancing the student’s control in educational process using Web 2.0 personal learning environments. British Journal of Educational Technology, 46(4), 780–792. https://doi.org/10.1111/bjet.12170 Ringle, C.M., Wend, S., Becker, J. M. (2015). SmartPLS 3. Retrieved from http://www.smartpls.com/ Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2), 267–280. https://doi.org/10.1016/j.learninstruc.2010.07.004 Russell, S., & Norvig, P. (2009). Artificial Intelligence: A Modern Approach (3rd ed.). USA: Prentice Hall Press. Sandu, N., & Gide, E. (2019). Adoption of AI-chatbots to enhance student learning experience in higher education in india. 2019 18th International Conference on Information Technology Based Higher Education and Training, ITHET 2019, (January). https://doi.org/10.1109/ITHET46829.2019.8937382 Shawar, B., & Atwell, E. (2003). Using dialogue corpora to train a chatbot. (October 2014). https://doi.org/10.13140/2.1.1455.7122 Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers and Education, 151(June 2019), 103862. https://doi.org/10.1016/j.compedu.2020.103862 Straub, D. W. (1989). Validating Instruments in MIS Research. MIS Quarterly, 13(2), 147. https://doi.org/10.2307/248922 Subhash, S., & Cudney, E. A. (2018). Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior, 87(February), 192–206. https://doi.org/10.1016/j.chb.2018.05.028 Suh, A., Cheung, C. M. K., Ahuja, M., & Wagner, C. (2017). Gamification in the Workplace: The Central Role of the Aesthetic Experience. Journal of Management Information Systems, 34(1), 268–305. https://doi.org/10.1080/07421222.2017.1297642 Suh, A., Wagner, C., & Liu, L. (2015). The effects of game dynamics on user engagement in gamified systems. Proceedings of the Annual Hawaii International Conference on System Sciences, 2015March, 672–681. https://doi.org/10.1109/HICSS.2015.87 26 Tarouco, L. M. R., Silveira, C., & Krassmann, A. L. (2018). Collaborative learning with virtual entities. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10924 LNCS(January), 480–493. https://doi.org/10.1007/978-3-319-91743-6_35 Teixeira, T. S., & Mendes, R. (2019). How to Improve Your Company’s Net Promoter Score. Retrieved November 14, 2020, from Harvard Business Review website: https://hbr.org/2019/10/how-toimprove-your-companys-net-promoter-score?registration=success Tsay, C. H. H., Kofinas, A., & Luo, J. (2018). Enhancing student learning experience with technologymediated gamification: An empirical study. Computers and Education, 121(December 2017), 1– 17. https://doi.org/10.1016/j.compedu.2018.01.009 Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S., & Torous, J. B. (2019). Chatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape. The Canadian Journal of Psychiatry, 64(7), 456–464. https://doi.org/10.1177/0706743719828977 Valtonen, T., Hacklin, S., Dillon, P., Vesisenaho, M., Kukkonen, J., & Hietanen, A. (2012). Perspectives on personal learning environments held by vocational students. Computers and Education, 58(2), 732–739. https://doi.org/10.1016/j.compedu.2011.09.025 Venkatesh, V., Morris, M., Davis, G., & Davis, F. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425–478. Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012a). Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology. MIS Quarterly: Management Information Systems, 36(1), 157–178. https://doi.org/10.2307/41410412 Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012b). CONSUMER ACCEPTANCE AND USE OF INFORMATION TECHNOLOGY: EXTENDING THE UNIFIED THEORY OF ACCEPTANCE AND USE OF TECHNOLOGY. MIS Quarterly, 36(1), 157–178. Retrieved from https://pdfs.semanticscholar.org/6256/0e2001480fd1f22558ce4d34ac93776af3e6.pdf%0Ahttp s://pdfs.semanticscholar.org/6256/0e2001480fd1f22558ce4d34ac93776af3e6.pdf?_ga=2.1245 39978.1994179764.1540339706-2125081534.1540339706 Venkatesh, V., Thong, J. Y. L., & Xu, X. (2016). Unified theory of acceptance and use of technology: A synthesis and the road ahead. Journal of the Association for Information Systems, 17(5), 328– 376. https://doi.org/10.17705/1jais.00428 Villagrasa, S., Fonseca, D., Redondo, E., & Duran, J. (2014). Teaching case of gamification and visual technologies for education. Journal of Cases on Information Technology, 16(4), 38–57. https://doi.org/10.4018/jcit.2014100104 Wise, R. A. (1998). Drug-activation of brain reward pathways. Drug and Alcohol Dependence, 51(1–2), 13–22. https://doi.org/10.1016/S0376-8716(98)00063-5 Wu, J., Fan, S., & Zhao, J. L. (2018). Community engagement and online word of mouth: An empirical investigation. Information and Management, 55(2), 258–270. https://doi.org/10.1016/j.im.2017.07.002 Wu, W., Tzamos, C., Daskalakis, C., Weinberg, M., & Kaashoek, N. (2015). Game theory based peer grading mechanisms for MOOCs. L@S 2015 - 2nd ACM Conference on Learning at Scale, 281– 286. https://doi.org/10.1145/2724660.2728676 Yoo, K. H., & Gretzel, U. (2008). What Motivates Consumers to Write Online Travel Reviews? 27 Information Technology & Tourism, 10(4), 283–295. https://doi.org/10.3727/109830508788403114 Yoo, S. J., & Han, S. H. (2013). The effect of the attitude towards e-learning: The employees’ intention to use e-learning in the workplace. International Journal on E-Learning: Corporate, Government, Healthcare, and Higher Education, 12(4), 425–438. Youcheng, W., & Fesenmaier, D. R. (2003). Assessing Motivation of Contribution in Online Communities: An Empirical Investigation of an Online Travel Community. Electronic Markets, 13(1), 33–45. https://doi.org/10.1080/1019678032000052934 Zainuddin, Z., Shujahat, M., Haruna, H., & Chu, S. K. W. (2020). The role of gamified e-quizzes on student learning and engagement: An interactive gamification solution for a formative assessment system. Computers and Education, 145(October 2019), 103729. https://doi.org/10.1016/j.compedu.2019.103729 28 9. APPENDIX Appendix A Construct Item Adapted from Performance Expectancy (PE) PE1: I Find the chatbot useful in my daily life as student PE2: I think that using the chatbot would increase my chances of success as a student PE3: I think that using the chatbot would enable me to conduct my tasks more quickly PE4: I think that using the chatbot would increase my productivity PE5: I think that using the chatbot would improve my performance Venkatesh et al. (2012) Hedonic Motivation (HM) HM1: I think that using the chatbot would be fun HM2: I think that using the chatbot would be enjoyable HM3: I think that using the chatbot would be very entertaining Venkatesh et al. (2012) Status (ST) I think that the Chatbot would offer me the possibility to: ST1: Have a higher status than others ST2: Be regarded highly by others ST3: Try to increase my status (Suh et al., 2017; Youcheng & Fesenmaier, 2003) Rewards (RW) I think that the Chatbot would offer me the possibility to: RW1: Obtain points as a reward for my studying activities RW2: Accumulate points I have gained RW3: Obtain more points if I try harder (Kankanhalli, Tan, & Wei, 2005; Suh et al., 2017) Competition (CO) I think that the Chatbot would offer me the possibility to: CO1: Compete with others CO2: Compare my performance with others CO3: Threaten the status of others by my active participation (Lee & Yang, 2011; Suh et al., 2017) 29 Construct Item Adapted from Support (SP) SP1: The chatbot helps me collaborate with my classmates, and teachers. SP2: The chatbot helps me share web resources and other content related with learning SP3: The chatbot allows me to support/ help others on using the technology SP4: The chatbot supports new ways of learning and interaction with others SP5: The chatbot allows additional opportunities to analyse and discuss class content with other students and teachers SP6: The chatbot promotes communication about technology with other students outside my class and with my family members (E. Rahimi et al., 2015) Behavioural Intention to adopt (BI) BI1: I intent to use the chatbot in the next months BI2: I predict I would use the chatbot in the next months BI3: I plan to use the chatbot in the next months BI4: I will try to use the chatbot in my daily life BI5: Using the chatbot to help me with my studies is something that I would do Venkatesh et al. (2012) Intention to Recommend (REC) REC1: I will recommend my friends to use the chatbot, if it is available REC2: If I have a good experience with the chatbot I will recommend friends to use it. (Oliveira et al., 2016) Page | i