Opening the Black Box: Explaining the Process of Basing a Health Recommender System on the I-Change Behavioral Change Model

SPECIAL SECTION ON DATA-ENABLED INTELLIGENCE FOR DIGITAL HEALTH Received November 6, 2019, accepted November 25, 2019, date of publication December 4, 2019, date of current version December 23, 2019. Digital Object Identifier 10.1109/ACCESS.2019.2957696 Opening the Black Box: Explaining the Process of Basing a Health Recommender System on the I-Change Behavioral Change Model SANTIAGO HORS-FRAILE 1,2,3, SHWETAMBARA MALWADE 4, FRANCISCO LUNA-PEREJON 3, CLAUDIO AMAYA 3, ANTÓN CIVIT 3, FRANCINE SCHNEIDER 2, PANAGIOTIS BAMIDIS 7, SHABBIR SYED-ABDUL 4,5, YU-CHUAN LI 4,5,6, AND HEIN DE VRIES 2 1Salumedia Tecnologías, 41011 Sevilla, Spain 2CAPHRI Public Health, Maastricht University, Maastricht, The Netherlands 3Computer Architecture and Technology, Universidad de Sevilla, 41012 Sevilla, Spain 4International Center for Health Information Technology (ICHIT), Taipei Medical University, Taipei 11031, Taiwan 5Graduate Institute of Biomedical Informatics, Taipei Medical University, Taipei 11031, Taiwan 6TMU Research Center of Cancer Translational Medicine, Taipei Medical University, Taipei 11031, Taiwan 7Laboratory of Medical Physics, Group of Applied and Affective Neuroscience, Faculty of Health Sciences, Medical School, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece Corresponding author: Shabbir Syed-Abdul ([email protected]) This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Programme under Grant 681120, and in part by the Taipei Medical University was funded by the Ministry of Science and Technology, Taiwan, under Grant 106-2923-E-038-001-MY2. ABSTRACT Recommender systems are gaining traction in healthcare because they can tailor recommendations based on users’ feedback concerning their appreciation of previous health-related messages. However, recommender systems are often not grounded in behavioral change theories, which may further increase the effectiveness of their recommendations. This paper’s objective is to describe principles for designing and developing a health recommender system grounded in the I-Change behavioral change model that shall be implemented through a mobile app for a smoking cessation support clinical trial. We built upon an existing smoking cessation health recommender system that delivered motivational messages through a mobile app. A group of experts assessed how the system may be improved to address the behavioral change determinants of the I-Change behavioral change model. The resulting system features a hybrid recommender algorithm for computer tailoring smoking cessation messages. A total of 331 different motivational messages were designed using 10 health communication methods. The algorithm was designed to match 58 message characteristics to each user profile by following the principles of the I-Change model and maintaining the benefits of the recommender system algorithms. The mobile app resulted in a streamlined version that aimed to improve the user experience, and this system’s design bridges the gap between health recommender systems and the use of behavioral change theories. This article presents a novel approach integrating recommender system technology, health behavior technology, and computer-tailored technology. Future researchers will be able to build upon the principles applied in this case study. INDEX TERMS Artificial intelligence, behavioral sciences, context awareness, filtering algorithms, mobile applications, recommender systems, smoking cessation. I. INTRODUCTION Traditional computer-tailored interventions based on behavioral change models can yield highly personalized motivational messages to help an individual adopt and maintain healthy habits [1]–[4]. However, these interventions typically The associate editor coordinating the review of this manuscript and approving it for publication was Yongqiang Cheng . provide feedback on ‘static’ scores for each individual’s answers. In contrast, a health recommender system (HRS) can dynamically compute a list of recommended items for each user using artificial intelligence (AI). An HRS is a type of machine learning system that provides users with relevant items (i.e., messages) based on, for instance, their past behavior or similarities they share with other users. Combining HRSs with behavioral change models can yield an innovative VOLUME 7, 2019 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ 176525 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model approach for the further use and improvement of eHealth interventions [5]. A recent scoping review [6] demonstrates that very few studies that employ HRSs describe their domain, design methodology, procedures, theoretical health promotion factors, behavioral change theories, or technical details. This lack of transparency in the design of an HRS hampers the replication of successful experiments and the identification of factors that contribute to this success, thus limiting the systems’ growth potential when applied in healthcare. Also, patients and healthcare professionals may not understand what features are being used to generate the predictions and recommendations computed by AI-based healthcare systems [7]. This is may be due to the difficulty for understanding the nature of the processes and logic followed by AI algorithms. Further, the process for tracing back the origin and reasons for a specific AI-based health recommendation is complex, and sometimes, even impossible. This black-box perception may yield trust barriers for its adoption [8], and undesired ethical [9] and legal implications [10]. In order to open this black box regarding how HRSs are designed, developed, and used in combination with computer-tailored technology, we shall describe the creation process followed in one m-Health intervention employed by the European project titled ‘SmokeFreeBrain’ (www.smokefreebrain.eu) [11]. This intervention, in combination with the standard care provided at the smoking cessation units, aims to help patients stop smoking (i.e., pharmacological treatment or nicotine replacement therapy). This system, tested in the 3M4Chan [12] randomized, controlled trial in Taiwan—whose results will be published in the near future—features an algorithm based on the I-Change behavioral change model [13] and the user context that selects the most relevant motivational messages for each user. The I-Change behavioral change model was chosen among a wide variety of models because it has been extensively used to explain smoking cessation behaviors in previous studies [14]–[17]. The primary goal of this paper is to open the black box of our HRS by thoroughly explaining the design process followed to combine it with the I-Change model - although the same reasoning and procedures may be applied for other behavioral models, as well. We present the principles of the design process and its outcomes for the system’s key components: motivational messages, a recommendation algorithm, and a mobile app. The transparent description of our HRS is a novel effort in the AI-based healthcare solution domain to increase trustworthiness, fairness, and interpretability on its recommendations. Our secondary goal is to foster transparency concerning the development of our technology by facilitating their replication, re-use, adaptation, and evolvement for future research and implementation in both similar and different contexts. II. RELATED WORKS Although recommender systems have been studied since the 1990s, recent and increasing interest has been expressed in applying them in the health domain, as indicated in the study of Schäfer et al. [18]. The first literature review related to this topic was conducted by Sezgin et al. in 2013 [19]. The authors presented the basic types of recommender systems and the challenges they faced as well as identified the first study to discuss HRS, which was published in 2007. Merely seven other studies were considered for inclusion in their review, which reflects the limited number of studies in this field. Sadasivam et al. [20] performed another literature review as part of their discussion of how computer tailoring may be advanced via recommender systems. In 2016, the authors presented the differences between the traditional rule-based tailoring systems and the new trend in data-driven HRSs, although earlier studies recommend food and physical activity, which may be considered characteristics of HRSs. Such systems date back to 2000 [21] and 2006 [22], as identified by Tran et al. in 2018 [23]. One of the first studies to propose the use of recommender systems in the health domain was conducted by Fernandez-Luque et al. in 2009 [5]. The authors suggest that personalized recommendations be generated with the feedback (ratings) users offered to social web content as well as the users’ similarity parameters. Recently, Torkamaan et al. proposed a basic model for achieving user satisfaction with HRSs [24] wherein effectiveness, privacy, trust, and transparency are key determinants. The authors additionally analyzed the factors influencing users when rating recommendations generated by HRSs [25], which they determined to be effectiveness, emotional gain, enjoyment, liking, and interest. Tondello et al. suggest that such HRSs can be complemented through indirect methods, such as personalized gamification approaches that support behavior change and engagement [26]. HRSs have been applied to a wide variety of health conditions, such as diabetes, drug-related side effects, lower back pain, generic health promotion, and cardio-metabolic risk. A limited number of studies report effectiveness values following the application of an HRS. For instance, for weight loss, the study conducted by Hales et al. [1] concludes that, by using a mobile app based on an HRS for weight loss, the participants managed to double their weight loss and improve their BMI reduction more so than the participants in the control group who did not benefit from the HRS. Regarding smoking cessation, Ghorai et al. [27] published a study describing an HRS that sent motivational SMSs according to the patients’ sex, ethnicity, and craving patterns, the last of which was reported by the users themselves via SMS. Three message intensity categories (normal, moderate, and high intensity) were applied, and the message content from previously successful smoking cessation programs was re-used. This study did not present the results for the HRS efficacy, and no other publication referencing this study has done so thus far. However, a relevant study titled ‘PERSPeCT’ [28] applied a recommender system to select and send motivational messages to influence users to quit smoking using multiple behavioural theories. This 176526 VOLUME 7, 2019 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model thirty-day study demonstrated that the HRS approach was more influential on users’ smoking cessation behaviors than was a traditional rule-based computer-tailoring approach in daily mean ratings and self-reported intervention influence. The PERSPeCT HRS was also helpful in making smokers more ready to set a quit date or to quit altogether compared to the traditional tailoring system. Video messaging rather than text-based messaging was tested in another study [29] whose effects are however not statistically proven in this case. Despite the limited evidence, the results from Hales et al. [1] and Sadasivam et al. [28] demonstrate the added value of an HRS. The collective intelligence generated by the aggregated data in an HRS offers real-time adaptation to users’ evolving needs and subsequent feedback. The long-term performance of recommender systems in temporally evolving networks has been investigated with industrial data sets that conclude that adopting heterogenous models is necessary for improving the user experience [30]. This result evidences the need for novel approaches that are able to adapt and evolve over time to match user preferences. Another recent study concludes that combining various models in hybrid recommender systems may improve computer tailoring in digital health interventions [31]. Other digital programs for smoking cessation do no employ HRSs, but rather apply static rules that tailor messages using behavioral change theories. For instance, the Text2Quit program [32] implemented social cognitive theory in a six-month study using tailored SMSs to support university students who wanted to quit smoking and found positive, statistically significant results that favored the participants’ receipt of tailored messages in both biochemically confirmed abstinence (11.1% in the experimental group and 5% in the control group) and self-reported abstinence (19.9% in the experimental group and 10.0% in the control group). Xhale.dk [33] also applied social cognitive theory for a message-based smoking cessation program that resulted in no statistically significant differences in the thirty-day abstinence rate checked after twelve months of the delivery of the intervention, compared to the tailored and untailored text message groups. The cognitive behavioral therapy was also employed by Strecher et al. [34] in an randomized controlled trial of 1,866 smokers, wherein the authors proved that high personalized and tailored messages contributed to an increase in the smokers’ six-month abstinence rates. A more recent model titled ‘‘Health Action Process Approach’’ [35] was used in a digital smoking cessation study [36] and resulted in a lower cigarette consumption rate by participants of the experimental group but did not express significant effects for abstinence rates, number of smoking cessation attempts, or states of change. Another frequently used model for smoking cessation is the transtheoretical model for behavior change [37] as identified by Noar et al. [38], and one study to employ that model was conducted by Haugh et al. [39]. The authors aimed to assess a tailored SMS-based programme that supported smoking cessation. Although they did prove its feasibility and acceptance, they did not manage to prove differences in significant effects between the three study groups (control, one weekly SMS message, or three weekly SMS messages) because this pilot study was equipped with underpowered data for purposes beyond validating the feasibility and acceptation of such system. Cheung et al. [40] analyzed the success factors of Dutch online smoking cessation interventions; among the six studies identified in the literature, five based their tailoring in socio-cognitive models (e.g., I-Change). By using such tailored digital interventions, smokers were between 1.15 and 2.84 times more likely to quit smoking than smokers in the control groups. Therefore, tailoring messages to motivational characteristics has been demonstrated to result in smokers’ increased attention, increased information processing abilities, increased motivation to quit, and successful quitting that can be maintained after 24 months [41], [42]. In conclusion, the use of HRS technology can specifically add value to behavioral change interventions when combined with tailoring, although one pitfall of most HRS studies is that they lack transparency and thorough detail [6]. Consequently, a taxonomy was developed to assist the reporting and classification of HRSs [6]. Valdez et al. proposed a framework for developing an HRS in which three dimensions are considered and covered: domain, evaluation, and inception [43]. Nevertheless, no consensus yet exists on how to describe or design an HRS. HRS technology currently fails to incorporate insights of the successes achieved with behavior change theories for health behavior principles and principles of computer-tailored technology. Beyond traditional digital tailoring programs, grounding HRS in theoretical behavioral change models is not common, as Cheung et al. identified [31] where only 3 of the 19 analyzed articles mentioned the inclusion of a theoretical behavioral model. The novelty of this paper is that it comprehensively describes how HRSs, health behavior principles, and computer-tailored technology are combined in a single digital health solution for smoking cessation, bridging the gap between AI-based collaborative intelligence for healthcare, human behavior, and personalization.. We will describe: (1) the motivational messages’ design, (2) the recommender system’s design, (3) the mobile app’s design, and (4) a clear differentiation between the aspects that must be carefully considered when replicating or evolving this study in future research in addition to the tools required to do so (i.e., tailoring recommendations from the World Health Organization, behavioral change models, data analytics services). III. METHODS A. SETTING Our HRS comprehends firstly a server running an algorithm that selects and sends motivational messages to support patients and encourage them to stay smoke free and secondly a mobile app called Quit and Return (hereafter, ‘QaR’)— programmed in two versions for both Android and iOS VOLUME 7, 2019 176527 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model devices—which receives messages from the server – and such messages can then be rated by users according their perceived usefulness. The system was developed between January 2017 and August 2017. B. MOTIVATIONAL MESSAGES: DESIGN PROCESS A group of researchers specializing in behavioral change theories for smoking cessation designed the motivational messages in English. We followed the tailoring recommendations made by the World Health Organization (WHO) [44] and the I-Change model to capture specific behavior change determinants, such as attitude (the perceived advantages and disadvantages of quitting), social support (the support to quit offered by others), skills (the actual capacity of the smoker to manage situations where they are tempted to smoke), self-efficacy (how the smoker perceives his/her ability to successfully quit), and action planning (the various actions that are needed to quit [e.g., mentioning one’s desire to quit to others] and to successfully cope with the accompanying challenges), all of which have been identified as key factors for increasing awareness, raising motivation, and changing behaviors in previous studies [45]–[49]. We chose the WHO’s ‘Encouraging people to quit smoking’ guidelines to complement the I-Change model introduced above because these guidelines constitute a well-known behavioral science publication endorsed by one of the most prestigious health entities in the world that includes contributions from experts both inside and outside the WHO. These guidelines were referenced to design the motivational messages because they provide case scenarios, examples of personalized reasons to stop smoking, common excuses for not quitting—to which we created counterreactive sentences—and strategies and tips to effectively quit smoking. Before writing the messages, we defined the message meta-features, which are the details or characteristics related to each user profile (i.e., demographic data, smoking habits, and I-Change-related factors, such as attitudes and self-efficacy for quitting). The team ensured that at least one message covered each meta-feature’s value to ensure that participants were provided at least one message for each combination of meta-features, thus avoiding cases wherein the system does not recommend a message. In other words, the system should have a sufficient variety of messages that map all potential user types; for instance, the system should include personalized messages that cover both genders, different age ranges, users with different nicotine dependence levels, users who have the skills to quit smoking and those who do not, users who are supported by others to quit smoking, and those who are quitting without the support of their friends and family, among other groups (for an example of a personalized message, see Table 3 in Appendix). The messages were first written in English and then translated into Mandarin Chinese, which were then validated by two Taiwanese doctors specializing in smoking cessation. C. RECOMMENDER SYSTEM DESIGN PROCESS We analyzed the designs of previous behavioral change interventions [50]–[52] to become aware of how the authors designed their solutions and applied any conclusions or findings they may have reached. We also analyzed the HRS previously used in the Social Local and Mobile (SoLoMo) intervention [53], which included the users’ recommendation ratings after six months. We did so to build upon an existing developed system that would speed up the development process—that is, it would only require adaptations and not a complete from-scratch development—and reduce potential flaws and design pitfalls. In addition, we assessed the need to increase granularity in feedback options for the received motivational messages, as users were offered merely three options for their responses (positive, negative, or neutral). Although this setup made the rating process simple to understand, most votes were concentrated around the positive option (see Fig. 1), which hindered the potential of the HRS because it did not have information about which messages were neither useful nor liked. Therefore, determining which messages were most useful for each user was difficult as they provided their ratings because most messages were rated as useful regardless of their content. We analyzed which alternatives were used in the existing popular platforms to increase granularity in the feedback options without rendering the rating process too complex for the users. To do so, we checked how the rating process was set up in popular services that use recommender systems, such as Amazon, eBay, and Netflix. D. MOBILE APP DESIGN PROCESS The QaR mobile app was evolved from the SmokeFree app used for the SoLoMo study [54]. Our 3M4Chan study required that the QaR app be made available to all users in Taiwan, which resulted in the removal of user data that would be linked to a hospital’s electronic health records (as was the case for the SoLoMo study). To improve the user experience of the SmokeFree app, we analyzed its six-month usage statistics using Flurry Analytics, the results of which reveal that users were interested in the messaging section (23.24% of use), the benefits/statistics section (15.84% of use), and their personal profiles (9.95% of use). These three sections resulted in a combined use of 49.03% versus 50.97% of the sixteen other features of the SmokeFree app. Therefore, we simplified and streamlined the SmokeFree app by removing any section other than the messaging,benefits/statistics, and personal profile sections in the QaR app. We did not remove other sections with low usage because they were necessary for making the app work properly and complying with legal restrictions (i.e., identifying the app’s creators and giving credit to the funding sources); these sections were: configuration (allowing users to configure their language and notification preferences), tutorial (showing the app’s basic functions), and ‘about us’ (describing the app’s authorship). 176528 VOLUME 7, 2019 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model FIGURE 1. Six-month HRS message ratings evolution in the SoLoMo study. To make the QaR app available to anyone in Taiwan, we introduced initial questionnaires to complete each user’s profile. To keep a low entry barrier, only three questions were required to make the app work, eighteen were required to start a quit attempt, and 52 others were voluntary in an attempt to improve the personalized recommendations. To complement the SmokeFree app’s usage analysis, we conducted a usability report following the MUSiC Performance Measurement method [55] with fifteen participants. From this analysis, we concluded that the tutorial shown when users first used the app could not sufficiently assist them in remembering the ‘hamburger menu’—that is, the three horizontal lines in the top left of the blue menu bar that allowed a user to navigate across various sections of the app. Consequently, in the QaR app, we instead developed the main sections to be accessible from a single screen via tabs. IV. RESULTS A. MOTIVATIONAL MESSAGE RESULTS 1) NAME PERSONALIsATION Most messages were designed to include the receiver’s name with some greeting words. 2) META-FEATURES ADAPTATION Resulting from the message design process and the HRS definition, we divided the meta-features into two groups: 7 basic meta-features, and 51 extended meta-features. The basic meta-features included the most essential demographic information (gender, and age), and five other smoking-cessation indicators typically required in smoking cessation interventions to determine the patients’ smoking habits (quitting date, cigarette consumption, weekly average expenditure in cigarettes, standardized nicotine dependence, and standardized motivation to quit). These 7 variables contain the minimum information required for them to assess a smoking cessation patient. Although these variables were previously used in the SoLoMo intervention, they were also validated by Taiwanese smoking cessation experts coming from Taipei Medical University Hospital, and Wellcome Clinic in Taipei. The 51 other extended meta-features corresponded to patients’ comorbidities, living companions, physical activity frequency, and considerations regarding the I-Change model’s key factors. These meta-feature topics were selected because the information they provide may be included and referred to in many smoking cessation motivational messages contents. Thus, the information introduced by meta-features allows the personalization and tailoring of these motivational messages.’’ The users provided such information through questionnaires in the app. The questionnaire related to the 7 basic meta-features was mandatory to start a quitting attempt, whilst the questionnaire for the 51 others was optional. Thus, that extended information allows the system further personalization and tailoring of the motivational messages, and a wider range of relevant topics for smoking cessation. Consequently, users providing a few meta-features only would receive less specific, and probably less relevant tailored motivational messages than users who completed all their extended meta-features. The I-Change model determinants included attitudes (twelve meta-features), social support (six meta-features), self-efficacy (seventeen meta-features), action planning VOLUME 7, 2019 176529 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model (eight meta-features), and skills (four meta-features). Table 2 of the Appendix offers an example message for each determinant. We created a total of 131 motivational message categories, each of which deals with smoking cessation-specific aspects and is related to one or more user meta-features. As we wrote at least one message to cover all categories and metafeatures, the 131 categories unfolded as 331 different messages. All messages were enounced from a positive point of view and exemplified the benefits of quitting smoking. In Table 3 of the Appendix, we include a sample case of how a category unfolds through various tailored messages. For this example, we took the category ‘skin’, which is one of the organs that is negatively affected by smoking; in this case, this category is exclusively associated with the ‘user age’ meta-feature. Based on a user’s age, we can determine within which of the three meta-feature types he/she is categorized: <30 years of age, 30–60 years of age, or 60 years of age or older. Depending on this categorization, the message stresses the importance of maintaining young and healthy skin (<30), the importance of ceasing the ageing process now (30–60), or the importance of regaining some of the already lost appeal (60+). 3) HEALTH COMMUNICATION METHODS All messages were originally written in plain English (i.e., using active voice, including ‘you’ pronouns, keeping sentences short) such that educational level was not a factor in understanding them. When translating the messages into Chinese, we requested that the translator maintain that level of simplicity. For instance, we used active rather than passive voice and we did not include superfluous, irrelevant, or distracting information. We kept the length of each message short enough to read in less than one minute (a maximum of 200 words, with an average of 85.5 words per message). We also incorporated several behavior change techniques into the messages; specifically, we covered ten of the eleven groups proposed by Abraham and Kools [56]. We present an example for each group and underline where the techniques were reflected in Table 4 of the Appendix. Other techniques we employed include repeating an answer, creating empathy, adding new knowledge, and changing existing misconceptions, for which application examples are described in Table 5 of the Appendix. For the first technique, we both included the user’s answer in many messages (as seen in the examples) and allowed the system to repeat sending the same message up to three times, although a message was only repeated if no other user-compatible message existed with fewer repetitions. In this case, we prepended to each message the following piece of text: ‘We know we have sent this message already, but we think it is important you remember it’. We did not ask users to provide what they thought about some typical misconceptions about smoking, as doing so would have required that the initial questionnaire be even longer. Instead, we identified some common misconceptions FIGURE 2. Implemented HRS diagram for the message selection process. and included the real facts in some messages, as may be perceived in the above example. B. RECOMMENDER SYSTEM RESULTS 1) TAXONOMY The recommender system algorithm resulting from the design phase responds to the taxonomy proposed by Hors-Fraile et al. [6], as shown in Table 6 of the Appendix. 2) USER CONTEXT SELECTION The user context is a special type of message category because it is based not on any meta-feature, but rather on the day or moment the message shall be sent. We created thirteen messages based on various possible user contexts (i.e., different week-days or moments during the day). These messages were only sent if the user was inside the associated context; for instance, a message designed for a Monday could only be sent on Mondays, and Table 7 of the Appendix provides examples of these messages. 3) MESSAGE SELECTION Messages were selected by a hybrid HRS algorithm in cascade which works in two steps (see Fig. 2). Step 1 (a knowledge-based algorithm) filtered down all the messages that were not compatible with the user’s metafeatures; that is, it reduced the category possibilities such that any remaining messages were suitable for the user. Step 2 (demographic filtering algorithm) involved selecting from the remaining messages that which the user had not yet received, that which had been received less frequently, and that which was rated more favorably by other users. The impact of the other users’ opinions on the final selected message by the HRS to be sent was directly associated with their metafeatures’ similarity to those of the user to whom the message was sent. At 00:02 am Taiwan time, the HRS computed the algorithm in the database and prepared the messages to be sent 176530 VOLUME 7, 2019 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model FIGURE 3. Selection of the number of messages for each user decision tree. the following day for each user. This process consisted of selecting how many messages should be sent to each user (see Fig. 3), selecting the message according to Step 1, and passing the results to Step 2 (in cascade). a: HRS STEP 1 Step 1 involved checking which messages were potentially suitable for a user profile, which comprised the set of all meta-features associated with the user’s questionnaire responses. The algorithm iterated the comprehensive list of messages and discarded those whose meta-features did not match the user profile; this action can be considered the application of a filter to reduce messages that were not applicable for the user. Context-dependent messages introduced in the previous section were treated as if they had a special meta-feature that was compared not against the user profile, but rather to the actual user context. Following the previous example, a message that should be sent in the morning was solely selected as a candidate for sending if its previously calculated time fell within the morning time frame. As an illustrative example, Fig. 4 presents a simplified version of a user with only four meta-features: gender, age, cigarette consumption, and amount of time since quitting. If we had a pool of eight messages, only those messages that possessed the four meta-features compatible with the user would be selected to progress to Step 2; in our example, exclusively messages C and G would be candidates for sending. The HRS grouped the values of our meta-features to reduce the number of possible combinations as described in Table 1. The values for grouping were proposed, and as an initial approximation—to our knowledge—no previous attempts have been made to perform similar categorization. FIGURE 4. Simplified representation of Step 1 – Knowledge-based algorithm. TABLE 1. Meta-feature value grouping. b: HRS STEP 2 The messages that reached this step went through the collaborative filtering process, which relied on the hypothesis that one would like what other people with similar user profiles liked. Users who possessed similar profiles to the user under evaluation were called neighbors [57]. VOLUME 7, 2019 176531 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model TABLE 2. Examples of messages for each type of included I-change determinants. TABLE 3. Example of message category unfolding in several tailored messages upon meta-features. By the end of this step, each message was assigned a calculated score that determined its probable relevance to the user. A higher score indicated the given message was more relevant to the user’s neighbors and more likely relevant to the user him/herself. In this step, we aimed to determine the message with the highest score. In order to calculate this score, a matrix was built by the system with all users and their message ratings, excluding the user for whom we were selecting the message (see Table 8 of the Appendix). The message score was the sum of all ratings provided by all users and was scaled by the neighbor similarity score. The next action involved normalizing the message score; if a message had never been rated by a user, its relevance rating score defaulted to the midpoint value of 0.5. The neighbor similarity score was calculated using the equation illustrated below (1), which considers all meta-features defined by users in their profiles. rA,B=Pn i=1(δFA(i),FB(i))+Pm j=1δFA(j),FB(j) max(|FA(j)|,|FB(j)|) Pn+m k=1(δFA(k),FB(k))(1) where: Aand Bare two users; Furepresents all meta-features completed by user ‘u’; Fu(x) represents the value of meta-feature ‘x’ of user ‘u’; |Fu(x)|represents the number of values of meta-feature ‘x’ of user ‘u’; nis the total number of single-value meta-features; mis the total number of multiple-value meta-features; and δy,zis a function that sums the number of matching meta-features between ‘y’ and ‘z’. Equation (1) returned a value between 0 and 1 because it provided the quotient between the number of matching 176532 VOLUME 7, 2019 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model TABLE 4. Examples of applied behavior change techniques. meta-feature values between user A and B as well as the total number of meta-features they had in common. The numerator was split in two addends to consider the cases of single-value meta-features and multiple-value meta-features. If a message was rated more than once by a user either because it was sent two or three times or because the user re-opened the message and rated it again, the new rating value overwrote the previous one. If the user had no neighbors, the system then faced the ‘cold start’ problem [58]–[60] and picked the first message stored in the list that met the requirements. If the user restarted his/her quitting attempt, the number of times each message had been sent was cleared. VOLUME 7, 2019 176533 S. Hors-Fraile et al.: Opening the Black Box: Explaining the Process of Basing a HRS on the I-Change Behavioral Change Model SHABBIR SYED-ABDUL was born in Hyderabad, India, in 1975. He received the M.D. degree from Saint Petersburg Medical Academy Russia, India, in 2001, the M.S. degree in telemedicine and ehealth from the University of Tromso, Norway, in 2008, and the Ph.D. degree in health informatics from National Yang-Ming University, Taiwan, in 2013. Since 2015, he has been a Leading Researcher and a Principal Investigator with the International Center for Health Information Technology, Taipei, Taiwan. He currently holds a position as an Associate Professor with the Graduate Institute of Biomedical Informatics, Taipei Medical University. He is also an Educator for MOOCs on FutureLearn and his courses, including Artificial Intelligence for Healthcare, Internet of Things for Active Aging, Social Media in Healthcare, and Digital Health for Cancer Management. He has published about 70 SCI articles in last five years. Studies cover topics related with Information and communication technologies, such as the role of social networking in healthcare, health information sharing, management and flow of medical information and comparing electronic medical record, and paper-based records. His major research interests are artificial intelligence, Longterm care, telemedicine, mHealth, translational medicine, big data analytics and visualization, personal health records, digital epidemiology, social Network in healthcare, and hospital information System. Dr. Shabbir’s awards include New Investigator of Global Health by Global Health Council, in 2010. YU-CHUAN (JACK) LI was born in Taipei, Taiwan, in 1966. He received the M.D. degree in medicine from Taipei Medical University, Taipei, in 1991, and the Ph.D. degree in medical informatics from the University of Utah, USA, in 1994. He has been a Pioneer of Medical Informatics Research, Asia, and a Practicing Dermatologist, Taiwan. He served as the Vice President of Taipei Medical University (TMU), from 2009 to 2011. He has been the Dean of the College of Medical Science and Technology, since 2011. He has also been a Professor with the Graduate Institute of Biomedical Informatics, since 1998. He has also been the Principle Investigator of many national and international projects in the domain of Electronic Health Record, Patient Safety Informatics, and Medical e-learning. He is an author of 130 scientific articles and three college-level textbooks. He became an Elective Fellow of the American College of Medical Informatics (FACMI), in 2010, Australian College of Health Informatics (FACHI), in 2010. He was the President of Asia Pacific Association for Medical Informatics (APAMI), from 2006 to 2009. He is currently leading the International Center for Health Information Technology (ICHIT). He is also a part of the Graduate Institute of Biomedical Informatics, and Research Center of Cancer Translational Medicine, Taipei Medical University. His main areas of expertise are artificial intelligence in medicine, information engineering, medical decision support systems, patient safety information systems, and medical big data analytics. Prof. Li’s awards include Outstanding IT Elite award, Taipei Computer Association, in 2015, and Taiwan’s Top Ten Outstanding Youth, in 2001. HEIN DE VRIES was born in Thesinge, The Netherlands, in 1956. He studied clinical, developmental and social psychology at Groningen University. He received the Ph.D. degree in health sciences in 1989 from Maastricht University for which he received the Catherine Pijls Award. From 1988 until 1998, he directed an Expert Center on cancer prevention funded by the Dutch Cancer Society. Since 1990, he has been involved in developing effective and cost-effective computer tailored interventions that were first sent via postal mail and later via the Internet, addressing smoking, alcohol, nutrition, physical activity, obesity, early detection, COPD, type 2 diabetes and sexual violence at Maastricht University. As part of this research line several studies have been conducted to optimize dHealth interventions. In 1998, he became a Full Professor in health communication at Maastricht University. He served as a Visiting Professor in Rhode Island, Durban, Metz, Lisbon. He is currently with the University of Seville. Prof. de Vries is involved as a co-editor/editor assistant with Preventive Medicine, Psychology and Health, Health Education, and Tobacco Prevention and Cessation. 176540 VOLUME 7, 2019