A Mobile Health Solution Complementing Psychopharmacology-Supported Smoking Cessation: Randomized Controlled Trial

O iginal Pape A Mobile Heal h Solu ion Complemen ing Psychopha macology-Suppo ed Smoking Cessa ion:Randomized Con olled T ial Lau a Ca asco-He nandez1,2, D med; F ancisco Jóda -Sánchez3, PhD; F ancisco Núñez-Benjumea3, MSc; Jesús Mo eno Conde3, MSc; Ma co Mesa González1, MSc; An ón Ci i -Balcells4, P o D ; San iago Ho s-F aile5, MSc; Ca los Luis Pa a-Calde ón3, MSc; Panagio is D Bamidis6, P o D , PhD; F ancisco O ega-Ruiz1, D med, PhD 1Smoking Cessa ion Uni , Medical-Su gical Uni o Respi a o y Diseases, Vi gen del Rocío Uni e si y Hospi al, Se ille, Spain 2Cen o de In es igación Biomédica en Red de En e medades Respi a o ias, Ca los III Ins i u e o Heal h, Mad id, Spain 3Resea ch and Inno a ion G oup in Biomedical In o ma ics, Biomedical Enginee ing and Heal h Economy, Ins i u e o Biomedicine o Se ille, Vi gen del Rocío Uni e si y Hospi al, Spanish Na ional Resea ch Council, Uni e si y o Se ille, Se ille, Spain 4Depa men o A chi ec u e and Compu e Technology, School o Compu e Enginee ing, Uni e sidad de Se illa, Se ille, Spain 5Salumedia Labs, Se ille, Spain 6Medical Physics Labo a o y, School o Medicine, A is o le Uni e si y o Thessaloniki, Thessaloniki, G eece Co esponding Au ho : F ancisco Jóda -Sánchez, PhD Resea ch and Inno a ion G oup in Biomedical In o ma ics Biomedical Enginee ing and Heal h Economy, Ins i u e o Biomedicine o Se ille Vi gen del Rocío Uni e si y Hospi al, Spanish Na ional Resea ch Council, Uni e si y o Se ille A enida Manuel Siu o S/N Se ille, 41013 Spain Phone: 34 670943651 Email: [email p o ec ed] Abs ac Backg ound: Smoking cessa ion is a pe sis en leading public heal h challenge. Mobile heal h (mHeal h) solu ions a e eme ging o imp o e smoking cessa ion ea men s. P e ious app oaches ha e p oposed suppo ing cessa ion wi h ailo ed mo i a ional messages. Some managed o p o ide sho - e m imp o emen s in smoking cessa ion. Ye , hese app oaches we e ei he s a ic in e ms o pe sonaliza ion o human-based nonscalable solu ions. Addi ionally, long- e m e ec s we e nei he p esen ed no assessed in combina ion wi h exis ing psychopha macological he apies. Objec i e: This s udy aimed o analyze he long- e m e icacy o a mobile app suppo ing psychopha macological he apy o smoking cessa ion and complemen a ily assess he in ol ed inno a i e echnology. Me hods: A 12-mon h, andomized, open-label, pa allel-g oup ial compa ing smoking cessa ion a es was pe o med a Vi gen del Rocío Uni e si y Hospi al in Se ille (Spain). Smoke s we e andomly alloca ed o a con ol g oup (CG) ecei ing usual ca e (psychopha macological ea men , n=120) o an in e en ion g oup (IG) ecei ing psychopha macological ea men and using a mobile app p o iding a i icial in elligence–gene a ed and ailo ed smoking cessa ion suppo messages (n=120). The seconda y objec i es we e o analyze heal h- ela ed quali y o li e and moni o heal hy li es yle and physical exe cise habi s. Sa e y was assessed acco ding o he p esence o ad e se e en s ela ed o he pha macological he apy. Pe -p o ocol and in en ion- o- ea analyses we e pe o med. Incomple e da a and mul inomial eg ession analyses we e pe o med o assess he a iables in luencing pa icipan cessa ion p obabili y. The echnical solu ion was assessed acco ding o he p ecision o he ailo ed mo i a ional smoking cessa ion messages and use engagemen . Cessa ion and no cessa ion subg oups we e compa ed using es s. A olun a y sa is ac ion ques ionnai e was adminis e ed a he end o he in e en ion o all pa icipan s who comple ed he ial. Resul s: In he IG, abs inence was 2.75 imes highe (adjus ed OR 3.45, P=.01) in he pe -p o ocol analysis and 2.15 imes highe (adjus ed OR 3.13, P=.002) in he in en ion- o- ea analysis. Los da a analysis and mul inomial logis ic models showed di e en pa e ns in pa icipan s who d opped ou . Rega ding sa e y, 14 o 120 (11.7%) IG pa icipan s and 13 o 120 (10.8%) CG pa icipan s had 19 and 23 ad e se e en s, espec i ely (P=.84). None o he clinical seconda y objec i e measu es showed JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 1h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X ele an di e ences be ween he g oups. The sys em was able o lea n and ailo messages o imp o ed e ec i eness in suppo ing smoking cessa ion bu was unable o educe he ime be ween a message being sen and opened. In ei he case, he e was no ele an di e ence be ween he cessa ion and no cessa ion subg oups. Howe e , a signi ican di e ence was ound in sys em engagemen a 6 mon hs (P=.04) bu no in all subsequen mon hs. High sys em app ecia ion was epo ed a he end o he s udy. Conclusions: The p oposed mHeal h solu ion complemen ing psychopha macological he apy showed g ea e e icacy o achie ing 1-yea obacco abs inence as compa ed wi h psychopha macological he apy alone. I p o ides a basis o a i icial in elligence–based u u e app oaches. T ial Regis a ion: ClinicalT ials.go NCT03553173; h ps://clinical ials.go /c 2/show/NCT03553173 In e na ional Regis e ed Repo Iden i ie (IRRID): RR2-10.2196/12464 (JMIR Mheal h Uheal h 2020;8(4):e17530) doi: 10.2196/17530 KEYWORDS smoking cessa ion; beha io al change; heal h ecommende sys ems; mHeal h; andomized con olled ial In oduc ion Tobacco use p esen s a majo p e en able public heal h p oblem; i is he leading cause o heal h de e io a ion and p ema u e dea h. The Wo ld Heal h O ganiza ion ecognizes smoking as a ch onic sys emic disease among he addic ions capable o p oducing high physical damage ela ed o mul iple diseases [1]. Fu he , smoking kills o e 7 million people annually, wi h global cos s es ima ed a US $1.4 illion [2]. Valida ed app oaches o acili a e smoking cessa ion include nico ine eplacemen he apy, pha macological ea men (ie, bup opion and a enicline), and beha io al and psychological suppo . Combining beha io al and psychological suppo wi h pha macological ea men is cu en ly he mos e ec i e in e en ion o achie ing obacco abs inence [3]. Mo eo e , bup opion was shown o app oxima ely double he likelihood o long- e m obacco abs inence as compa ed wi h placebo [4]. Fu he mo e, a enicline (2 mg, o al daily dose) was shown o iple he likelihood o main aining long- e m obacco abs inence as compa ed wi h placebo [5]. Despi e he p o en e icacy o hese ea men s, a meaning ul numbe o smoke s ail o s op smoking, and many ac o s in luence hei success, including he mo i a ion o s op and con inuous suppo . Mobile and web-based in e en ions ha e been used o acili a e e ec i e beha io al changes in di e en heal h domains [6-9], including smoking cessa ion. Scien i ic e idence has indica ed ha mobile phone–based ex messages suppo ing smoking cessa ion we e nea ly 1.7 imes mo e success ul han a con ol app oach a 6 mon hs [10]. I has been p o en ha he beha io al change impac is highe when use s ecei e mul iple ailo ed heal h ecommenda ions [11,12]. Howe e , in p e ious s udies, a adi ional compu e - ailo ing app oach was used o gene a e such pe sonalized heal h ecommenda ions. Compu e ailo ing in ol es he gene a ion o pa ien -speci ic ecommenda ions, ypically in he o m o messages, by compu e s, and i is pe o med a e pe sonal assessmen o ma ch he cha ac e is ics, needs, and in e es s o he pa ien s [13-16]. Cupe ino e al [17] ecen ly pilo ed a 12-week smoking cessa ion p og am in ol ing ex messaging wi h pha maco he apy suppo , which showed p omising abs inence a es a 3 mon hs and high pa icipan sa is ac ion. Howe e , he e is no e idence o he long- e m abs inence e icacy o mobile-based ailo ed in e en ions combined wi h psychopha macological he apies. As pa o he SmokeF eeB ain H2020 Eu opean Commission p ojec [18], he Social-Local-Mobile (So-Lo-Mo) s udy in es iga ed mobile and a i icial in elligence (AI) echnologies (heal h ecommende sys em [HRS]) as a complemen a y aid o pha macological ea men s. The expe imen al in e en ion ocused on p o iding ubiqui ous ailo ed suppo o pa ien s willing o s op smoking h ough a digi al he apeu ic mobile app solu ion. Pa icipan s ecei ing he So-Lo-Mo in e en ion we e p o ided wi h access o a code o be in oduced in he app. The code ac i a ed he app, connec ing i wi h he hospi al pa ien da abase. The app au oma ically downloaded he necessa y da a o ini ialize he app p o ile, educing he bu den o ha ing o c ea e a use p o ile. The mobile app was connec ed wi h an AI sys em designed o lea n om pa ien in e es s h ough hei in e ac ions wi h he app o dynamically (1) de e mine, pe sonalize, and send mo i a ional messages o suppo smoking cessa ion; (2) schedule he message deli e y equency acco ding o he ans heo e ical model o beha io al change [19]; and (3) calcula e he mos con enien ime o send a mo i a ional message o suppo smoking cessa ion o each pa ien . Speci ically, he AI sys em used o ailo mo i a ional messages is a hyb id HRS, whose design has been p esen ed by Ho s-F aile e al [20]. The mo i a ional messages we e designed a e conduc ing semis uc u ed in e iews in ol ing wo smoking cessa ion expe s (a pulmonologis and a psychologis ). Based on hei commen s, we iden i ied he ollowing i e di e en opics, which we e he mos ele an o pa icipan s o succeed in hei smoking cessa ion p ocess: (1) gene al mo i a ion; (2) heal hy die ecommenda ions; (3) ecommenda ions o an ac i e li es yle; (4) posi i e ein o cemen messages o mee ac i i y goals; and (5) bene i s o being a nonsmoke . A o al o 150 di e en messages we e w i en o each opic by a heal h communica ion and heal h p omo ion PhD candida e, and hey we e alida ed by he wo smoking cessa ion expe s. This numbe o messages was chosen o ensu e ha , e en in he wo s case scena io whe e he AI sys em de e mines ha a use needs o ecei e messages o a single opic, no use would ge he same message wice du ing he in e en ion, as he maximum JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 2h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X numbe o messages ha could be ecei ed is 150 du ing he 1-yea s udy. This app oach minimizes any po en ial obo ic and s a ic eelings. Messages included heal h communica ion and heal h p omo ion s a egies, such as c ea ing empa hy, adding new knowledge, and changing exis ing misconcep ions. The messages we e sho (less han 100 wo ds), w i en in simple Spanish, w i en using a close and iendly one, and w i en wi h easy- o-unde s and e ms o acili a e comp ehension by smoke s o all educa ional le els. O he s udies ha e used s eps simila o he ones used by us o message c ea ion [21]. Howe e , in ou design, he message con en was no associa ed wi h any speci ic beha io al change model. Each message was checked o be sui able o all gende s. When a message was clea ly gende speci ic (ie, ela ion be ween smoking and e ec ile dys unc ion in men), an al e na e sui able message o he o he gende was also designed (ie, isks o smoking du ing p egnancy). HRSs in ol e sel -lea ning algo i hms ha can adap o he cons an ly e ol ing needs and in e es s o use s and o e a high le el o pe sonaliza ion o ecommenda ions, aking ad an age o he so-called “collec i e in elligence.” This is a new ecommenda ion gene a ion pa adigm ha con as s he adi ional ailo ing app oach, which p o ides s a ic ecommenda ions acco ding o use esponses o usually leng hy ques ionnai es. Thus, as his no el app oach o ailo ing is s ill in i s in ancy [22], i is o high in e es o de e mine he ela ionship be ween he clinical ou comes o he So-Lo-Mo s udy and his ype o echnology. The main objec i e o his So-Lo-Mo s udy was o compa e usual psychopha macological he apy (con ol g oup, CG) alone and alongside he a o emen ioned digi al solu ion (in e en ion g oup, IG) o smoking cessa ion. The seconda y objec i es we e o analyze heal h- ela ed quali y o li e (HRQoL) and moni o heal hy li es yle and physical exe cise habi s. Complemen a ily, we assessed he impac o he AI-gene a ed mo i a ional messages on smoking cessa ion ou comes in he IG. Me hods S udy Design This 12-mon h, andomized, open-label, pa allel-g oup ial was pe o med a he Smoking Cessa ion Uni o Vi gen del Rocío Uni e si y Hospi al in Se ille (Spain) be ween Oc obe 24, 2016, and Oc obe 24, 2018, and i complied wi h he Decla a ion o Helsinki and Good Clinical P ac ice Guidelines. The ec ui men pe iod closed on Oc obe 23, 2017. The local e hics commi ee app o ed he s udy p o ocol, and w i en in o med consen was ob ained om each pa icipan p io o inclusion. The clinical s udy design (NCT03553173) and echnical s udy design (NCT03206619) ha e been published p e iously [23,24]. Randomiza ion A echnician gene a ed a andom-g oup able (n=240) using compu e me hods and ollowing a 1:1 a io be ween g oups. Clinicians en olled he pa icipan s and assigned hem o he g oup men ioned in he able acco ding o hei en olmen sequence. Pa icipan s we e blinded o his alloca ion, as hose in he IG we e old ha he p o ided mobile app was pa o usual ca e. Pa icipan s in he CG we e no in o med abou he exis ence o he app and did no ha e access o i . S udy Popula ion Smoke s we e ec ui ed du ing ou ine isi s o ou ou pa ien clinic. The inclusion c i e ia we e as ollows: (1) age o e 18 yea s and desi e o s op smoking; (2) owning an And oid sma phone (as he mobile app was only a ailable o And oid de ices owing o ime and esou ce cons ain s in he de elopmen phase o his s udy and And oid phones we e mo e likely o be owned by he a ge popula ion owing o hei lowe en y p ice as compa ed wi h iPhones); and (3) abili y o in e ac wi h he sma phone. Sma phone li e acy was assessed by asking he pa icipan s i hey commonly use o he ex exchange sma phone apps, such as Mail, SMS, and Wha sApp. The only exclusion c i e ion was any p e ious ad e se e ec ela ed o he p esen pha macological ea men . Powe Calcula ion and Rec ui men A sample size o 236 was calcula ed du ing he s udy design phase, acco ding o he ollowing pa ame e s: CI, 95%; s a is ical powe , 80%; CG success a e, 35%; IG success a e, 55%; and expec ed d opou a e, 20%. A o al o 240 pa icipan s we e ec ui ed and andomized o he s a i ied analysis. In e en ion and Con ol G oups Usual ca e in he Smoking Cessa ion Uni o Vi gen del Rocío Uni e si y Hospi al consis s o pha macological he apy wi h bup opion (ie, Zyn abac 150 mg; GlaxoSmi hKline, B en o d, UK) o a enicline (ie, Champix 0.5 mg o 1 mg; P ize , New Yo k, New Yo k, USA) plus beha io al he apy. To acili a e ec ui men and a oid bias associa ed wi h ea men cos , he SmokeF eeB ain p ojec inanced he d ugs o usual ca e. Thus, all pa icipan s ecei ed hei assigned ea men s ee o cha ge. Beha io al he apy, which was p o ided du ing ace- o- ace ollow-up consul a ions, included a ious psychological echniques, including mo i a ional in e iews and cogni i e-beha io al he apy. This psychopha macological he apy was p o ided o bo h CG and IG pa icipan s. CG pa icipan s (n=120) ecei ed psychopha macological he apy alone, whe eas IG pa icipan s (n=120) ecei ed psychopha macological he apy and used he digi al he apeu ic solu ion. Fu he in o ma ion ega ding en ollmen , alloca ion, and he s udy analysis phases is p o ided in Figu e 1. The app u ilized beha io al echniques by sending pe sonalized mo i a ional messages gene a ed using AI wi h he in en o achie e be e smoking cessa ion a es by imp o ing p og am adhe ence and abs inence a es [20]. A e he pa icipan ead each message, he app asked he pa icipan o a e how ele an he message was o him o he , collec ing eedback o he AI. Complemen ing he co e messaging ea u e, he mobile app had o he mino sec ions. These included a use p o ile linked o he pe o med physical ac i i y le el collec ed by Google Fi (Google, Moun ain View, Cali o nia, USA), ou smoking cessa ion bene i indica o s (sa ings, smoke- ee days, egained li e hou s by no smoking, and numbe o nonsmoked ciga e es since qui ing), ex -based in o ma ion abou smoking cessa ion, JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 3h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X and a sec ion con aining elaxing and dis ac ing elemen s (b ea hing exe cises and minigames). Figu e 2 p esen s an o e iew o he app ea u es. Figu e 1. CONSORT diag am o he s udy. JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 4h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X Figu e 2. S uc u e o app ea u es. Measu emen s In o ma ion om all pa icipan s included demog aphic (age and sex) and socioeconomic da a (p o ession and employmen s a us), consump ion his o y (daily ciga e es smoked, li ing wi h smoke s, pa ne smoking s a us, smoking cessa ion a emp s, e c), clinical in o ma ion (weigh , heigh , blood p essu e, como bidi ies, e c), nico ine dependence measu ed using he Fage s öm es o nico ine dependence [25], and mo i a ion o s op smoking acco ding o he Richmond es [26]. Sa e y was measu ed as he numbe o ad e se e en s ela ed o pha macological he apy. The main clinical ou come was he 1-yea smoking abs inence a e measu ed by exhaled ca bon monoxide (CO), which was assessed using a CO es e (Mic o+ Smoke lyze ; CoVi a, San a Ba ba a, Cali o nia, USA) and u ine co inine es s. Pa icipan s wi h an exhaled CO le el g ea e han 6 ppm we e conside ed smoke s [27]. U ine co inine (SmokeSc een es ; Concep Smoke JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 5h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X Sc een L d, Lincolnshi e, UK) is a colo ime ic es ha measu es he main nico ine me aboli es, including co inine. Pa icipan s wi h a co inine concen a ion g ea e han 200 ng/ml we e conside ed smoke s [28]. Pa icipan s we e conside ed smoke s when a leas one o he a o emen ioned condi ions was me . The HRQoL was assessed using he 36-i em Sho -Fo m Heal h Su ey (SF-36) ha was alida ed in Spanish [29] and he Eu oQol 5-dimension 5-le el (Eu oQoL-5D-5L) ques ionnai e [30]. Physical ac i i y was measu ed using he In e na ional Physical Ac i i y Ques ionnai e (IPAQ) [31], and a heal hy li es yle was in e p e ed ia body mass index (BMI) a ia ions du ing ollow-up consul a ions. A case epo o m, buil upon he OpenClinica [32] ool, was de eloped o acili a e in o ma ion managemen in he s udy. In o ma ion subse s we e egis e ed acco ding o he ollowing schedule: basal and 15, 30, 60, 90, 120, 180, and 365 (±5) days a e he basal consul a ion [23]. The HRS impac on smoking cessa ion was assessed acco ding o he p ecision o he ecommenda ions sen by he sys em. P ecision was calcula ed o bo h he message con en ecommenda ions and he ime o open he messages, as he HRS aimed o op imize bo h a iables. Fu he , we conside ed he gene a ed engagemen o he use wi h he sys em and he use app ecia ion o he messages as pa o he echnical e alua ion o HRS impac . The s a ing poin o measu e he e olu ion o hese me ics was he i s day ha a pa icipan a ed a message (Decembe 5, 2016), and he assessmen con inued o he ollowing 18 mon hs o e alua e he p og ess. De ailed desc ip ions o hese me ic calcula ions a e p esen ed below. P ecision o he Sys em This measu emen ocused on he sys em’s e ec i eness in ecommending ele an messages. The mobile app equi ed he use o a e all ead messages a leas once; o he wise, he use could no longe in e ac wi h he app. The e we e h ee message a ing op ions (like, dislike, and neu al). The a ings we e hen coded as 1 o “like,” −1 o “dislike,” and 0 o “neu al.” The p ecision me ic was calcula ed by di iding he numbe o hi s (messages a ed as “like”) by he o al numbe o sen messages (p ecision_p). A modi ica ion o his measu emen was pe o med by conside ing “likes” and “neu als” as hi s (p ecision_p_n,). We used wo a ian s o p ecision o ob ain di e en pe spec i es on how he sys em pe o med in selec ing ele an messages. The second app oach ocused on minimizing he nega i ely a ed messages. Rega dless o he a ian , when we code he hi s as 1 and misses as 0, his me ic is in ac an a i hme ic mean (we sum he numbe o hi s and di ide hem by he o al numbe o sen messages). The alue o his me ic anges om 0 o 1. A highe alue is associa ed wi h mo e sys em p ecision. This is because on sending messages a andom, we expec 33% posi i e eedback, 33% neu al eedback, and 33% nega i e eedback. Howe e , when he sys em lea ns and becomes mo e “in elligen ,” he pe cen age o nega i e eedback will dec ease o e ime, inc easing he alue esul ing om his me ic. Time o Open Mo i a ional Messages This pe ained o he a i hme ic mean o he elapsed ime be ween he message being sen and opened in a 30-day pe iod. We assumed ha his me ic would dec ease o e ime because he sys em was expec ed o lea n and become mo e “in elligen ” in p edic ing he bes ime o he use o open and ead he sen message. Engagemen Wi h he Sys em To measu e sys em engagemen , we included a ime s amp in each sen message and compa ed i wi h he ime s amp sen by he app o he se e when he use opened he gi en message. Thus, he engagemen me ic was de e mined by he a io o a ed messages calcula ed as he o al o all a ed messages om one use di ided by he o al numbe o messages sen o ha use . As we coded each message a ed wi h a “1” alue and di ided he inding by he o al numbe o sen messages, his a io coincides wi h he a i hme ic mean. Thus, we assume ha a highe alue is associa ed wi h mo e use in e es in he message and, consequen ly, highe use engagemen . As his me ic measu ed he engagemen o he use wi h he sys em and no he engagemen o he sys em wi h he use , we conside ed he se da e o s op smoking o each use as he poin o s a coun ing each mon h. A e his calcula ion, we o se he s a da e o each use o he i s day he use a ed he i s message. This allowed us o compa e he a e age engagemen o each use ac oss ime. The e o e, he esul s a e p esen ed acco ding he ela i e mon hs o he use se da e o s op smoking (eg, M3 esul s a e he esul s o he hi d mon h a e a pa icipan s opped smoking). Howe e , o one pa icipan , M3 could be Ap il i he pa icipan s opped smoking in Janua y, bu o ano he pa icipan , i could be Decembe i he pa icipan s opped smoking in Sep embe . The measu emen o his ollow-up was pe o med o he 12-mon h pe iod ha each pa icipan in he IG was en olled in he s udy, as pa icipan s in he CG did no use he mobile app. Subjec i e Quali y o he Sys em Sys em quali y was de e mined by he answe s in an anonymized i e-le el Like - ype app ecia ion ques ionnai e [24], making i impossible o link he answe s o he indi idual pa icipan s. The ques ionnai e was comple ed by each pa icipan a he end o he 1-yea ollow-up pe iod. The ques ions conce ned he HRS acco ding o he Recommende Sys ems Ques ionnai e o Use Expe ience (ResQue) model [33] and he con en o he messages acco ding o he I-Change model [34]. O he 15 cons uc s o he ResQue model, 12 we e ep esen ed in he ques ions, as he emaining h ee did no apply o ou HRS (ie, pu chase in en ion). All he answe s e lec ed he le el o ag eemen o he pa icipan wi h he p oposed opics (1 indica ing “ o ally disag ee” and 5 indica ing “ o ally ag ee”). We assessed how pa icipan s pe cei ed he quali y o he sys em a e 12 mon hs o usage by obse ing hei ques ionnai e esponse dis ibu ions, means, and SDs. G oup compa isons we e no easible owing o he comple e anonymi y o he ques ionnai e. The aim was o measu e he deg ee o possible imp o emen o he sys em in he ollowing a eas included in he ques ionnai e: quali y o ecommended i ems, in e ac ion adequacy, in e ace adequacy, pe cei ed ease o use, pe cei ed JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 6h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X use ulness, con ol/ anspa ency, a i udes, beha io al in en ions, and message con en o in luence smoking isk pe cep ion, con idence, social suppo , and coping ac ion suppo . S a is ical Analysis Desc ip i e analyses o pa icipan cha ac e is ics acco ding o absolu e and ela i e equencies o quali a i e a iables and mean (SD) o quan i a i e a iables we e conduc ed. Fo he p ima y endpoin and seconda y ou comes (BMI, physical ac i i y, and HRQoL), analyses we e conduc ed on a pe -p o ocol basis. These analyses included all pa icipan s who adhe ed o he schedule o eigh consul a ions (basal h ough 365 days a e he basal consul a ion). Rega ding incomple e da a analyses, homoscedas ici y es s we e conduc ed among g oups o cases wi h iden ical missing da a pa e ns o e alua e whe he da a we e missing comple ely a andom [35]. A mul inomial eg ession analysis was pe o med o assess he a iables in luencing he p obabili y o a pa icipan d opping ou as compa ed wi h he p obabili y o he ea men being e ec i e. A “no e icacy” ca ego y was se as he e e ence ca ego y. As mul inomial model e ec s a e ela i e o he e e ence ca ego y, o assess whe he he explica i e a iable e ec s we e di e en be ween he “d opou ” and “e icacy” ca ego ies, he same model was adjus ed using he “d opou ” ca ego y as a e e ence. Consequen ly, whe he he a iable e ec s we e di e en be ween he “d opou ” and “e icacy” ca ego ies was assessed. The ela i e isk a io (RRR) wi h 95% CI was used o each model. Los da a analysis and mul inomial logis ic models showed di e en pa e ns in pa icipan s who d opped ou as compa ed wi h he pa e ns in hose who comple ed he s udy, ega dless o ea men e icacy. The e o e, smoking abs inence a 1 yea was analyzed using logis ic eg ession models on a pe -p o ocol basis and in en ion- o- ea basis, and he e ec measu es we e he OR and 95% CI. Fo he selec ion o logis ic eg ession and mul inomial logis ic models in pe -p o ocol (n=94) analysis and in en ion- o- ea (n=240) analysis, a wo-s age s a egy was adop ed. In he i s s ep, he a iable impo ance was quan i ied using Random Fo es [36] wi h he mean dec ease in accu acy as he sco e. Va iables wi h a sco e abo e 0.5 we e included in an Akaike in o ma ion c i e ion–based s epwise selec ion s a egy. The ollowing wo es ic ions we e applied o he inal models selec ed: (1) absence o a pa e n in model esiduals and (2) a iables wi h a gene alized a iance in la ion ac o [37] abo e 5 we e no allowed in o de o a oid collinea i y. To de e mine whe he he HRS me ics had an impac on he clinical ou comes, we di ided IG pa icipan s in cessa ion and no cessa ion subg oups a 12 mon hs o he in e en ion. The ea e , we conduc ed es s o he 12-mon h esul s o p ecision, ime o a e messages, and engagemen , analyzing each o hese wo subg oups. Resul s Cha ac e is ics Bo h g oups had simila baseline cha ac e is ics, excep o he maximum abs inence ime and smoking cessa ion a emp s (Table 1). JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 7h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X Table 1. Pa icipan baseline cha ac e is ics. P alueCon ol g oup (n=120)In e en ion g oup (n=120)Cha ac e is ic .0952 (43.3)65 (54.2)Female, n (%) .0550.93 (10.85)48.38 (9.49)Age (yea s), mean (SD) .5416.67 (3.60)16.94 (4.07)Age a smoking onse (yea s), mean (SD) .4420.75 (9.39)21.45 (8.97)Daily ciga e es, mean (SD) .6049 (40.8)45 (37.5)Li es wi h smoke s, n (%) .4365 (54.2)71 (59.2)Pa ne smokes, n (%) .0451.14 (1.07)0.88 (1.08)Smoking cessa ion a emp s, mean (SD) .00313.68 (25.08)10.45 (25.49)Maximum abs inence ime, mean (SD) .7327.03 (6.46)27.02 (4.91)Body mass index, mean (SD) .4838 (31.7)33 (27.5)Unemployed, n (%) .7944 (36.7)46 (38.3)P e ious ea men s, n (%) .6915 (12.5)13 (10.8)Va enicline .7016 (13.3)14 (11.7)Bup opion .8415 (12.5)13 (10.8)Nico ine .3111 (9.2)16 (13.3)O he s .1259 (49.2)46 (38.3)Como bidi ies, n (%) .061.08 (1.34)0.83 (1.25)Cha lson index, mean (SD) .385.60 (1.99)5.89 (1.91)Fage s öm sco e, mean (SD) .889.28 (0.88)9.32 (0.80)Richmond, mean (SD) E icacy (Unadjus ed): 1-Yea Smoking Abs inence Ra e IG and CG pa icipan s who comple ed he s udy (pe -p o ocol analysis) achie ed e icacy a es o 64.7% (CI 51.6%-77.8%) and 40.0% (CI 25.7%-54.3%), espec i ely (P=.02; OR 2.75, CI 1.20-6.29) and a numbe needed o ea o 4 (CI 2.0-19.0). In he in en ion- o- ea analysis, IG and CG pa icipan s achie ed e icacy a es o 27.5% (CI 19.5%-35.5%) and 15.0% (CI 8.6%-21.4%), espec i ely (P=.02; OR 2.15, CI 1.13-4.08) and a numbe needed o ea o 8 (CI 4.0-43.0). Figu e 3 shows he e icacy e olu ion du ing he 1-yea ollow-up in each s udy g oup acco ding o bo h analy ic app oaches. JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 8h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X Figu e 3. E icacy e olu ion du ing he 1-yea ollow-up. Body Mass Index and Physical Ac i i y BMI a iances we e simila o bo h g oups. In he IG and CG, he mean BMI a iances we e 1.01 (CI 045-1.57) and 1.10 (CI 0.72-1.48), espec i ely, a 6 mon hs and we e 1.47 (CI 0.90-2.03) and 1.22 (CI 0.67-1.75), espec i ely, a 12 mon hs. The be ween-g oup mean BMI di e ences a 6 and 12 mon hs we e −0.09 (CI −0.77 o 0.60; P=.80) and 0.25 (CI −0.53 o 1.03; P=.52), espec i ely. Acco ding o he obse ed IPAQ sco es, physical ac i i y e olu ion pa e ns we e simila in bo h s udy g oups. Mo eo e , in he IG, 12.8% (6/47) and 25.5% (13/51) o pa icipan s inc eased hei physical ac i i y a 6 and 12 mon hs, espec i ely, whe eas 14.9% (7/47) and 13.7% (7/51) o pa icipan s educed i a he co esponding poin s. On he o he hand, in he CG, 14.6% (6/41) and 24.4% (11/45) o pa icipan s inc eased hei physical ac i i y a 6 and 12 mon hs, espec i ely, whe eas 24.4% (10/41) and 8.9% (4/45) o pa icipan s educed i a he co esponding poin s. Table 2 summa izes he BMI and IPAQ esul s. JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 9h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X Technical Ou comes The complemen a y echnical esul s showed ha he HRS is able o lea n he pa icipan s’ in e es s ega ding he suppo message opics o smoking cessa ion. The sys em was mo e p ecise a he end o he in e en ion han a he beginning, as was expec ed o a ecommende sys em. Howe e , he minimum p ecision eached was e y high (o e 96%). We belie e ha he e may be di e en easons o his ex emely high alue. Fo ins ance, i could be due o he implemen ed hyb id HRS algo i hm ha mi iga es he cold s a p oblem and accu a ely ecommends messages om he beginning acco ding o a weigh ing o mula desc ibed in a p e ious publica ion [20]. Addi ionally, i could be due o pa icipan s belie ing ha hei a ings would be iewed by heal h ca e p o essionals; hence, hey modi ied hei a ings owa d a highe alue han hey o he wise would. This po en ial Haw ho ne e ec would be a limi a ion o digi al heal h no iden i ied in a ecen s udy [44], as HRSs a e only s a ing o be used in heal h ca e. Ano he eason may be ha he sys em did no allow su icien g anula i y o he use s o o e; he pa icipan s we e only a o ded h ee a ing op ions. A wide sp ead o a ing op ions may ha e con ibu ed o he di e en ia ion o message ele ance o a g ea e ex en ( o ins ance, sepa a ing ai , good, and en i ely accu a e messages). The s a is ical analysis did no ind a signi ican di e ence in he p ecision achie ed be ween pa icipan s who we e success ul in smoking cessa ion and hose who we e no success ul. Hence, he achie ed p ecision o e ime canno be conside ed a p edic o o smoking cessa ion, as he HRS ecommended mo i a ional messages equally well o all pa icipan s. The ime- o-open me ic esul s showed ha he HRS was no able o p edic he bes ime o send a message o a dec ease in he ime be ween he message being sen and he use opening he message. This may be due o se e al ac o s in luencing pa icipan beha io o he han he ime hey ecei e a message, which we e no conside ed in he HRS. To imp o e he ime, a iables, such as he posi ion o he phone, loca ion, las ac i i y, and cessa ion day, should be conside ed o inclusion as pa ame e s in he HRS. The gene a ed engagemen by he sys em showed s a is ical signi icance (P=.04) be ween he cessa ion and no cessa ion subg oups a e he i s 6 mon hs o he in e en ion, a o ing a highe engagemen o hose who managed o s op smoking. This is in line wi h he in ui i e assump ion ha pa icipan s who engaged wi h he sys em o a g ea e ex en ecei ed g ea e bene i and consequen ly we e less likely o elapse. Howe e , hese di e ences we e no signi ican a 9, 11, and 12 mon hs owing o a dec ease in he numbe o pa icipan s o such an ex en a hese poin s ha he s a is ical powe was limi ed. As no all pa icipan s in he IG comple ed he pe cei ed quali y ques ionnai e and he ques ionnai e was anonymous, i is impossible o ensu e ha any conclusion de i ed om he collec ed esul s is no biased. Howe e , he esul s showed a clea ly posi i e pe cep ion o he quali y o he sys em. Reasonably, only hose who managed o s op smoking and p esumably bene i ed om he sys em p o ided answe s o he ques ionnai e, as hey would ha e been mo e mo i a ed o espond. On he o he hand, pa icipan s who d opped ou , p obably due o elapse, would ha e no been in e es ed in comple ing he ques ionnai e. Howe e , i could also be a gued ha hose who did no manage o s op smoking would ha e been willing o p o ide nega i e eedback on he sys em in he anonymized ques ionnai e. Ne e heless, he posi i e end o he answe s a o s he i s scena io. Fo simila esea ch p ac ices in he u u e, we sugges ollowing a design ha o ces pa icipan s o espond o he ques ionnai e bu s ill p ese es hei anonymi y. This could be achie ed by including he ques ionnai e in a p e ious s age o he ial o a oid an e ec by he high d opou a e associa ed wi h digi al in e en ions o smoking cessa ion. The ea u es o he p esen ed AI-based digi al he apeu ic solu ion enabled he sys em o e ec i ely suppo smoking cessa ion by p o iding suppo and ad ice o acili a ing abs inence, enhance mo i a ion, and clea ly show a bene i [45]. The HRS was well pe cei ed by he pa icipan s. O e ime, i iden i ied he mos ele an mo i a ional messages o send o each pa icipan , and hose who we e engaged wi h he sys em o a g ea ex en managed o success ully s op smoking by he end o he in e en ion. The e o e, his digi al he apeu ic solu ion may alle ia e he known d awbacks o in ensi e complemen a y beha io al in e en ions o smoking cessa ion, which, despi e hei bene i s, equi e ex ensi e a ailabili y o well- ained p o essionals, leading o limi ed scalabili y, poo accessibili y, and smoke esigna ion owing o long wai ing imes [46]. Limi a ions In p e ious simila s udies, he d opou a e was usually e y high, e lec ing he di icul y o smoking cessa ion and high elapse a e [3]. In his s udy, he d opou a es we e 57.5% and 62.5% in he IG and CG, espec i ely. A high d opou a e may bias he esul s; howe e , he d opou a e was simila be ween he g oups, minimizing he possibili y o bias. The e icacy o he in e en ion was e i ied by pe -p o ocol and in en ion- o- ea analyses, p o iding a mo e ealis ic pic u e while educing he impo ance o d opou o ull ea men compliance. Mo eo e , an analysis o he cha ac e is ics o he d opou and no d opou ca ego ies wi hin each g oup was ca y ou , and he same pa e n was ob ained be ween bo h g oups (Mul imedia Appendix 3). These indings sugges he po en ial bene i o he in e en ion, as pa icipan s did no expe ience abno mal ad e se e ec s and we e mo e likely o be abs inen a e 1 yea . Ne e heless, u he explo a ion o i s e icacy is needed o alida ion in o he cul u es, and s udies should include a la ge sample size and eal-wo ld da a. We ound some andom inconsis en alues in he en ies o he egis e ed ime, including duplica e en ies and nega i e ime alues. The p esen ed esul s co espond o il e ed da a ( emo ing such alues). Remo al o hese alues educed da a quali y o he me ic, and he indings may no accu a ely ep esen he en i e pa icipan g oup. These inconsis en alues may ha e been egis e ed owing o gli ches in communica ion be ween he sma phone app and he acking sys em on he se e . JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 16h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X Me ics o engagemen a he agg ega ed le el could no be p o ided as an icipa ed in he echnical p o ocol because he so wa e se ice used o ack use da a is no able o e ie e in o ma ion om da a olde han 2 yea s. This in o ma ion was no lis ed in he ea u es o he se ice when i was selec ed o he pu poses o he s udy. We encou age u u e esea che s o use in-house use da a acking se ices o a oid elying on hi d-pa y so wa e. Compa ison Wi h P io Wo k P e ious s udies ha e explo ed he use o mobile apps suppo ing smoking cessa ion. The Sma Qui s udy es ed he easibili y, accep abili y, p elimina y e icacy, and mechanism o beha io al change o an inno a i e sma phone-deli e ed accep ance and commi men he apy app o smoking cessa ion in 196 adul pa icipan s and epo ed a success ul cessa ion a e o 13% a e 2 mon hs o ollow-up [47]. In ano he s udy, he au ho s assessed he e icacy o an in e ac i e smoking cessa ion decision–aid app in 684 adul s and epo ed a success ul cessa ion a e o 10.2% [48]. Addi ionally, an app-based mind ulness aining p og am o smoking cessa ion was assessed in 143 pa icipan s, and a success ul cessa ion a e o 11.1% was achie ed [49]. A pa allel, double-blind, andomized, con olled, wo-a m ial compa ed he e icacy o an e idence-in o med sma phone app o smoking cessa ion (C ush he C a e) o ha o an e idence-in o med sel -help guide (On he Road o Qui ing) in 1599 subjec s and epo ed success ul cessa ion a es o 7.8% and 9.2%, espec i ely [50]. In all hese s udies, he abs inence a e was epo ed a 6 mon hs om baseline. Howe e , hey did no include pha macological ea men as pa o he in e en ion, which is a clea di e ence om he app oach in ou s udy. I is ema kable ha he d opou a es o bo h he IG and CG iden i ied in his s udy a e consis en wi h hose p e iously epo ed in he li e a u e [50]. Resea ch has been pe o med on in e en ion e icacy o he use o an app speci ically designed o suppo he smoking cessa ion p ocess when added o pha macological he apy. The p elimina y e icacy o an app designed o p omp smoke s o engage in physical ac i i y was assessed a 6 mon hs o ollow-up in 44 egula smoke s who ecei ed he app in addi ion o beha io al smoking cessa ion counselling, physical ac i i y p omo ion, and pha macological suppo , and he o e all abs inence a es we e 36% in an in en ion- o- ea analysis and 53% in a comple e-case analysis [51]. In his p e ious s udy, he alloca ion o pha macological ea men ( a enicline and nico ine eplacemen he apy) was no con olled by he s udy p o ocol, which hinde s he compa abili y o he esul s wi h hose o ou s udy. Addi ionally, in a andomized pilo s udy, a 12-week cou se o a enicline was p esc ibed o bo h a ms o assess he easibili y and accep abili y o he My Mobile Ad ice P og am (MyMAP) smoking cessa ion app and es ima e i s e ec s on smoking cessa ion and medica ion adhe ence [52]; howe e , i s e icacy could no be e idenced owing o he small sample size (n=33). As s a ed in he conclusion sec ion o a ecen sys ema ic e iew [53] ha included 26 s udies (n=33,849) in ol ing ex messaging and app-based smoking cessa ion in e en ions, he e is mode a e ce ain y e idence ha au oma ed ex message–based smoking cessa ion in e en ions esul in g ea e smoking cessa ion a es as compa ed wi h minimal smoking cessa ion suppo and he e is mode a e ce ain y e idence o he bene i o ex messaging in e en ions in addi ion o o he ypes o smoking cessa ion suppo as compa ed wi h smoking cessa ion suppo alone. The e idence o he compa ison o sma phone apps wi h less in ensi e suppo had e y low ce ain y, and u he andomized con olled ials a e needed o es hese in e en ions. In his sense, he esul s o his s udy aim o con ibu e o an inc ease in he ce ain y le el o he a ailable e idence in his domain. When ocusing on he assessmen o ecommende sys ems, such as he one used in his s udy, he scien i ic communi y has ex ensi ely concen a ed on imp o ing he pe o mance o ecommende sys em algo i hms wi h di e en me ics [54-58], mainly in nonheal h con ex s, such as leisu e [59] and e-comme ce [60,61]. Among hese app oaches, common assessmen s o p edic ion accu acy a e as ollows: (1) mean absolu e e o and oo mean squa ed e o , as well as hei no malized and a e aged a ian s o p edic a ings ha use s would gi e o i ems when he ac ual ecommenda ion a ings o he i ems a e known o he whole es se ; (2) p ecision, ecall, and alse posi i e a e [58,62] o p edic ion o he usage o he ecommenda ions; and (3) no malized discoun ed cumula i e gain when he sys em p esen s a la ge lis o elemen s (simila o a sea ch in Google), whe e we expec ha he mos ele an elemen s a e shown a he op o he lis ; and (4) co e age o he ecommenda ion se [63]. O he au ho s ha e p oposed he use o su eys o assess ecommende sys ems, such as ResQue [33]. Rega ding he e alua ion o mobile Heal h (mHeal h) apps o beha io al change, such as he one included in his s udy, some au ho s ha e p oposed e alua ion p ocedu es, such as he Na ional Ins i u e o Heal h and Ca e Excellence adap a ion [64], he Mobile App Ra ing Scale amewo k [65], and he Applica ion Usage Fac o , which is de ined as he loga i hm o he p oduc o he numbe o ac i e use s o a mobile app and he median numbe o daily uses o he app [66]. Despi e hese a emp s o se a me hodological amewo k o assess mHeal h apps, hey s ill lack s anda diza ion and comp ehensi eness [67]. In a ecen s udy, McKay e al iden i ied ha he e was no a ailable ela ed bes p ac ice [68]. Ins ead, hey p oposed he ollowing gene ic guidelines o wha hese ypes o e alua ions should include: (1) assessmen o he quali y o heal h- ela ed con en ; (2) e iew o he usabili y and unc ionali y o he app; and (3) c i ique o he app po en ial wi h ega d o beha io al change p omo ion. Despi e he lack o consensus on speci ic e alua ion me hods, i is well accep ed ha engagemen is a key elemen o mHeal h solu ions o be success ul [69]. Engagemen has been conside ed a posi i e ea ly indica o o beha io al change [70], as he p ocess o beha io al change equi es ime. P e ious s udies showed ha echnologies suppo ing eaching a speci ic beha io heal h goal help people s ick o he desi ed goal [71,72]. Fu he , Sche e e al [73] ound a signi ican posi i e co ela ion be ween engagemen and ewe d opou s in an mHeal h in e en ion. A ecen app oach o p o ide an engagemen sco e JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 17h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X o a mHeal h app is he Engagemen Index p oposed by Taki e al [74]. To calcula e he sco e, he au ho s combined he numbe o pages in he app ha a pa icipan isi ed each day, numbe o app accesses du ing he p og am, numbe o push no i ica ions opened, elapsed ime be ween app accesses, and subjec i e answe s o a ques ionnai e. Howe e , achie ing use engagemen is di icul [75], and i s le el could be imp o ed [76-78]. The e o e, i seems easonable o conside engagemen as a key me ic o digi al heal h solu ions. Howe e , he de ini ion o engagemen a ies among s udies. Fo ins ance, Iaco iello e al [79] conside ed he numbe o imes use s opened an app, numbe o in e ac ions hey had, and numbe o weeks in which use s had a leas one in e ac ion wi h he p og am as indica o s o engagemen . Owen e al [80] de e mined engagemen by calcula ing se e al a iables, such as he numbe o downloads and numbe o sessions (a session being basically opening o he app). Ye , i is common o jus use opening he app as an indica o o engagemen , as adop ed in p e ious s udies [40,81]. All hese engagemen in e p e a ions all unde he ca ego iza ion o “sys em usage da a” p oposed by Sho e al [82], which is mos equen ly used. Howe e , o he engagemen measu emen app oaches ha e been p oposed, such as ecological momen a y assessmen s, psychophysiological measu emen s, and quali i e me hods. In ou So-Lo-Mo s udy, he HRS was in ui i ely ela ed o use engagemen , as some s udies ha e shown ha good and imely ecommenda ions o s op smoking mo i a e use s o ead mo e u u e ecommenda ions [74,83]. Hence, we p oposed a combina ion o quali a i e and quan i a i e me ics, which measu e he in ended pu pose o he sys em (quali y o he ecommenda ions, engagemen o he pa icipan s wi h he sys em, and pa icipan s’ pe cep ion o he sys em). To ou knowledge, his is he i s assessmen in ol ing a HRS. Howe e , in he case o smoking cessa ion, low engagemen may no necessa ily esul in low impac on beha io al change, as some s udies ha e p e iously shown he “ga eway e ec ” [84] and “happy abandonmen ” [85]. The e a e se e al easons o his, including he bu den o he equi ed in e ac ions as compa ed wi h pe cei ed ou comes o he in e naliza ion o heal h habi s deno ing ha suppo is pe cei ed as no longe necessa y [86]. Consequen ly, we needed o ca e ully analyze he engagemen o use s ac oss ime poin s keeping in mind all hese ac o s o ex ac any conclusions. Thus, ou sys em’s in ended usage [82] is expec ed o be highe du ing he i s weeks o he smoking cessa ion p ocess and o p og essi ely dec ease un il he e is no engagemen wi h he sys em, which will be e lec ed in engagemen measu emen s. Conclusions The So-Lo-Mo in e en ion o e s a p omising s a egy o smoking cessa ion. The use o his digi al he apeu ic solu ion alongside pha macological ea men was much mo e e icacious in achie ing obacco abs inence as compa ed wi h pha macological ea men alone a he 1-yea ollow-up. Howe e , his in e en ion did no imp o e pa icipan HRQoL and physical ac i i y le els. Analysis o he impac o he HRS showed ha pa icipan s bene i ed om he ecommenda ions, and hose who engaged wi h he sys em we e mo e likely o succeed in smoking cessa ion. The e o e, he p oposed HRS had a posi i e impac on he pa icipan s and o e ed hem pe sonalized and ele an messages, al hough i could no de e mine when suppo i e messages should be sen o minimize he ime un il hey a e ead. Heal h ca e p o ide s should conside inco po a ing his digi al he apeu ic solu ion in hei usual ca e, as i can acili a e posi i e ou comes o pa icipan s willing o s op smoking. Acknowledgmen s The digi al he apeu ic solu ion used in he So-Lo-Mo s udy was join ly de eloped by Salumedia Tecnologías SLU, he A is o le Uni e si y o Thessaloniki, Se icio Andaluz de Salud, and Uni e sidad de Se illa based on he SmokeF eeB ain p ojec . We acknowledge P o Hein de V ies and D F ancine Schneide who p o ided guidance in he da a analysis o he echnical ou comes. This esea ch was unded by he H2020 Eu opean Commission esea ch and inno a ion p og am (g an ag eemen 681120) as pa o he SmokeF eeB ain p ojec (www.smoke eeb ain.eu). Con lic s o In e es SHF is a p oduc manage a Salumedia Tecnologías SLU, he company wi h exploi a ion igh s o he digi al he apeu ic solu ion used in he So-Lo-Mo s udy (DigiQui ). He con ibu ed o echnical da a ex ac ion o he p ecision, ime- o-open he messages, and pe cei ed quali y me ics and hei analysis. He was no in ol ed in he clinical ial design, execu ion, o analysis o he esul ing clinical da a. Some ins i u ions o all he o he au ho s (Uni e si y o Se ille, he A is o le Uni e si y o Thessaloniki, and he Se icio Andaluz de Salud as a legal ep esen a i e ins i u ion o he Vi gen del Rocío Uni e si y Hospi al) signed an exploi a ion ag eemen wi h Salumedia Tecnologías SLU o bene i om DigiQui comme cializa ion. This ag eemen was elabo a ed and signed be o e publishing he p esen esul s bu a e he ial was inished and i s esul s we e gene a ed. Mul imedia Appendix 1 Incomple e da a analysis. [DOC File , 81 KB-Mul imedia Appendix 1] JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 18h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X Mul imedia Appendix 2 Heal h ecommende sys em pe o mance analysis, P ecision_p. [DOCX File , 42 KB-Mul imedia Appendix 2] Mul imedia Appendix 3 Cha ac e is ics o he d opou and non-d opou wi hin each g oup. [DOC File , 126 KB-Mul imedia Appendix 3] Mul imedia Appendix 4 CONSORT-EHEALTH checklis (V 1.6.1). [PDF File (Adobe PDF File), 3306 KB-Mul imedia Appendix 4] Re e ences 1. WHO. WHO Global Repo On T ends In P e alence O Tobacco Smoking, 2015. Gene a: Wo ld Heal h O ganiza ion; 2015. 2. Goodchild M, Na gis N, Tu san d'Espaigne E. Global economic cos o smoking-a ibu able diseases. Tob Con ol 2017 Jan 30;27(1):58-64. [doi: 10.1136/ obaccocon ol-2016-053305] 3. Cahill K, S e ens S, Pe e a R, Lancas e T. Pha macological in e en ions o smoking cessa ion: an o e iew and ne wo k me a-analysis. Coch ane Da abase Sys Re 2013 May 31(5):CD009329. [doi: 10.1002/14651858.CD009329.pub2] [Medline: 23728690] 4. Reid RD, P i cha d G, Walke K, Ai ken D, Mullen K, Pipe AL. Managing smoking cessa ion. CMAJ 2016 Dec 06;188(17-18):E484-E492 [FREE Full ex ] [doi: 10.1503/cmaj.151510] [Medline: 27698200] 5. Cahill K, Lindson-Hawley N, Thomas K, Fanshawe T, Lancas e T. Nico ine ecep o pa ial agonis s o smoking cessa ion. Coch ane Da abase Sys Re 2016 May 09(5):CD006103 [FREE Full ex ] [doi: 10.1002/14651858.CD006103.pub7] [Medline: 27158893] 6. Condon LA, Coulson NS. Chap e 2 - Designing and Deli e ing In e en ions o Heal h Beha io Change in Adolescen s Using Mul i echnology Sys ems: F om Iden i ica ion o Ta ge Beha io s o Implemen a ion. In: Beha io Change Resea ch and Theo y. Cambi dge, Massachuse s: Else ie ; 2017:27-45. 7. Elbe SP, Dijks a A, Oenema A. A Mobile Phone App In e en ion Ta ge ing F ui and Vege able Consump ion: The E icacy o Tex ual and Audi o y Tailo ed Heal h In o ma ion Tes ed in a Randomized Con olled T ial. J Med In e ne Res 2016 Jun 10;18(6):e147 [FREE Full ex ] [doi: 10.2196/jmi .5056] [Medline: 27287823] 8. Zhao J, F eeman B, Li M. Can Mobile Phone Apps In luence People's Heal h Beha io Change? An E idence Re iew. J Med In e ne Res 2016 Oc 31;18(11):e287 [FREE Full ex ] [doi: 10.2196/jmi .5692] [Medline: 27806926] 9. Teasdale N, Elhussein A, Bu che F, Pie nas C, Cowbu n G, Ha mann-Boyce J, e al. Sys ema ic e iew and me a-analysis o emo ely deli e ed in e en ions using sel -moni o ing o ailo ed eedback o change die a y beha io . Am J Clin Nu 2018 Feb 01;107(2):247-256 [FREE Full ex ] [doi: 10.1093/ajcn/nqx048] [Medline: 29529158] 10. Whi ake R, McRobbie H, Bullen C, Rodge s A, Gu Y. Mobile phone-based in e en ions o smoking cessa ion. Coch ane Da abase Sys Re 2016 Ap 10;4:CD006611 [FREE Full ex ] [doi: 10.1002/14651858.CD006611.pub4] [Medline: 27060875] 11. B ug J, Glanz K, Van Assema P, Kok G, an B eukelen GJ. The impac o compu e - ailo ed eedback and i e a i e eedback on a , ui , and ege able in ake. Heal h Educ Beha 1998 Aug 30;25(4):517-531. [doi: 10.1177/109019819802500409] [Medline: 9690108] 12. Dijks a A, De V ies H, Roijacke s J, an B eukelen G. Tailo ing in o ma ion o enhance qui ing in smoke s wi h low mo i a ion o qui : Th ee basic e icacy ques ions. Heal h Psychology 1998;17(6):513-519. [doi: 10.1037/0278-6133.17.6.513] 13. Dijks a A. The Psychology o Tailo ing-Ing edien s in Compu e -Tailo ed Pe suasion. Social Pe s Psych Compass 2008 Ma ;2(2):765-784. [doi: 10.1111/j.1751-9004.2008.00081.x] 14. Enwald HP, Huo a i MA. P e en ing he obesi y epidemic by second gene a ion ailo ed heal h communica ion: an in e disciplina y e iew. J Med In e ne Res 2010 Jun 28;12(2):e24 [FREE Full ex ] [doi: 10.2196/jmi .1409] [Medline: 20584698] 15. Kamel Ghaliba A, Naza i E, Gholian-A al M, Ta a M. Comp ehensi e o e iew o compu e -based heal h in o ma ion ailo ing: a sys ema ic scoping e iew. BMJ Open 2019 Jan 09;9(1):e021022 [FREE Full ex ] [doi: 10.1136/bmjopen-2017-021022] [Medline: 30782671] 16. K eu e M, Fa ell D, Ole i ch L, B ennan L. Cus omizing communica ion wi h compu e echnology. In: Rou ledge , edi o . Tailo ed heal h messages. Mahwah, New Je sey: Law ence E lbaum Associa es, Inc; 2000. 17. Cupe ino AP, Ca ujano-Ba e a F, Ramí ez M, Rod íguez-Bolaños R, Th ashe JF, Pé ez-Rubio G, e al. A Mobile Smoking Cessa ion In e en ion o Mexico (Vi e sin Tabaco... ¡Decíde e!): Single-A m Pilo S udy. JMIR Mheal h Uheal h 2019 Ap 25;7(4):e12482 [FREE Full ex ] [doi: 10.2196/12482] [Medline: 31021326] JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 19h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X 18. Bamidis P, Pa aske opoulos E, Kons an inidis E, Spachos D, Billis A. Mul imodal e-heal h se ices o smoking cessa ion and public heal h: he smoke eeb ain p ojec app oach. In: P oceedings o he 16 h Wo ld Cong ess on Medical and Heal h In o ma ics. 2017 P esen ed a : MEDINFO 2017: P ecision Heal hca e h ough In o ma ics; Augus 2017; Hangzhou p. 5-9. [doi: 10.3233/978-1-61499-830-3-5] 19. P ochaska JO, Velice WF. The ans heo e ical model o heal h beha io change. Am J Heal h P omo 1997 Aug 26;12(1):38-48. [doi: 10.4278/0890-1171-12.1.38] [Medline: 10170434] 20. Ho s-F aile S, Nuñes Benjumea FJ, Ca asco He nández L, Ruiz FO, Fe nandez-Luque L. Design o wo combined heal h ecommende sys ems o ailo ing messages in a smoking cessa ion app. In: a Xi p ep in . 2016 P esen ed a : In e na ional Wo kshop on Engende ing Heal h wi h RecSys co-loca ed wi h ACM RecSys; 2016; Bos on, Massachuse s p. 160807192 URL: h ps://a xi .o g/abs/1608.07192 21. Ho s-F aile S, de V ies H, Malwade S, Luna-Pe ejon F, Amaya C, Ci i A, e al. Opening he Black Box: Explaining he P ocess o Basing a Heal h Recommende Sys em on he I-Change Beha io al Change Model. IEEE Access 2019;7:176525-176540. [doi: 10.1109/access.2019.2957696] 22. Ho s-F aile S, Ri e a-Rome o O, Schneide F, Fe nandez-Luque L, Luna-Pe ejon F, Ci i -Balcells A, e al. Analyzing ecommende sys ems o heal h p omo ion using a mul idisciplina y axonomy: A scoping e iew. In J Med In o m 2018 Jun;114:143-155. [doi: 10.1016/j.ijmedin .2017.12.018] [Medline: 29331276] 23. Jóda -Sánchez F, Ca asco He nández L, Núñez-Benjumea FJ, Mesa González MA, Mo eno Conde J, Pa a Calde ón CL, e al. Using he Social-Local-Mobile App o Smoking Cessa ion in he SmokeF eeB ain P ojec : P o ocol o a Randomized Con olled T ial. JMIR Res P o oc 2018 Dec 06;7(12):e12464 [FREE Full ex ] [doi: 10.2196/12464] [Medline: 30522992] 24. Ho s-F aile S, Schneide F, Fe nandez-Luque L, Luna-Pe ejon F, Ci i A, Spachos D, e al. Tailo ing mo i a ional heal h messages o smoking cessa ion using an mHeal h ecommende sys em in eg a ed wi h an elec onic heal h eco d: a s udy p o ocol. BMC Public Heal h 2018 Jun 05;18(1):698 [FREE Full ex ] [doi: 10.1186/s12889-018-5612-5] [Medline: 29871595] 25. Hea he on TF, Kozlowski LT, F ecke RC, Fage s öm KO. The Fage s öm Tes o Nico ine Dependence: a e ision o he Fage s öm Tole ance Ques ionnai e. B J Addic 1991 Sep;86(9):1119-1127. [doi: 10.1111/j.1360-0443.1991. b01879.x] [Medline: 1932883] 26. Richmond RL, Kehoe LA, Webs e IW. Mul i a ia e models o p edic ing abs en ion ollowing in e en ion o s op smoking by gene al p ac i ione s. Addic ion 1993 Aug;88(8):1127-1135. [doi: 10.1111/j.1360-0443.1993. b02132.x] [Medline: 8401167] 27. Pea ce MS, Hayes L, Newcas le Hea P ojec , Newcas le Thousand Families S udy. Sel - epo ed smoking s a us and exhaled ca bon monoxide: esul s om wo popula ion-based epidemiologic s udies in he No h o England. Ches 2005 Sep;128(3):1233-1238. [doi: 10.1378/ches .128.3.1233] [Medline: 16162711] 28. Pé ez-T ullén A, Ba olomé CB, Banegas JR. [New pe spec i es in he diagnosis and e olu ion o he consump ion o obacco: ma ke s o suscep ibili y and damage]. Med Clin (Ba c) 2006 Ap 29;126(16):628-631. [doi: 10.1157/13087722] [Medline: 16759556] 29. Vilagu G, Fe e M, Rajmil L, Rebollo P, Pe manye -Mi alda G, Quin ana JM, e al. [The Spanish e sion o he Sho Fo m 36 Heal h Su ey: a decade o expe ience and new de elopmen s]. Gac Sani 2005 Ap ;19(2):135-150 [FREE Full ex ] [doi: 10.1157/13074369] [Medline: 15860162] 30. Ramos-Goñi JM, C aig BM, Oppe M, Ramallo-Fa iña Y, Pin o-P ades JL, Luo N, e al. Handling Da a Quali y Issues o Es ima e he Spanish EQ-5D-5L Value Se Using a Hyb id In e al Reg ession App oach. Value Heal h 2018 May;21(5):596-604 [FREE Full ex ] [doi: 10.1016/j.j al.2017.10.023] [Medline: 29753358] 31. Boo h M. Assessmen o physical ac i i y: an in e na ional pe spec i e. Res Q Exe c Spo 2000 Jun 13;71 Suppl 2(sup2):114-120. [doi: 10.1080/02701367.2000.11082794] [Medline: 25680021] 32. Ca elaa s M, Rousseau J, Pa layan C, de Ridde S, Ve bu g A, Ross R, e al. OpenClinica. J Clin Bioin o ma ics 2015;5(Suppl 1):S2. [doi: 10.1186/2043-9113-5-s1-s2] 33. Pu P, Chen L, Hu R, edi o s. A use -cen ic e alua ion amewo k o ecommende sys ems. : ACM Digi al Lib a y; 2011 Oc P esen ed a : RecSys '11: Fi h ACM Con e ence on Recommende Sys ems; Oc obe 2011; Chicago, Illinois p. 157-164 URL: h ps://dl.acm.o g/doi/10.1145/2043932.2043962 [doi: 10.1145/2043932.2043962] 34. de V ies H. An In eg a ed App oach o Unde s anding Heal h Beha io ; The I-Change Model as an Example. PBSIJ 2017 Ma 9;2(2). [doi: 10.19080/pbsij.2017.02.555585] 35. Jamshidian M, Jalal S. Tes s o homoscedas ici y, no mali y, and missing comple ely a andom o incomple e mul i a ia e da a. Psychome ika 2010 Dec 3;75(4):649-674 [FREE Full ex ] [doi: 10.1007/s11336-010-9175-3] [Medline: 21720450] 36. B eiman L. Random o es s. Machine lea ning 2001;45(1):5-32 [FREE Full ex ] [doi: 10.1023/A:1010933404324] 37. Fox J, Mone e G. Gene alized Collinea i y Diagnos ics. Jou nal o he Ame ican S a is ical Associa ion 1992 Ma ;87(417):178-183. [doi: 10.1080/01621459.1992.10475190] 38. Ga ison KA, Pal P, Rojiani R, Dalle y J, O'Malley SS, B ewe JA. A andomized con olled ial o sma phone-based mind ulness aining o smoking cessa ion: a s udy p o ocol. BMC Psychia y 2015 Ap 14;15(1):83 [FREE Full ex ] [doi: 10.1186/s12888-015-0468-z] [Medline: 25884648] JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 20h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X 39. He ne JL, Vila daga R, Me ce LD, Kien z JA, B icke JB. Fea u e-le el analysis o a no el sma phone applica ion o smoking cessa ion. Am J D ug Alcohol Abuse 2015 Jan 14;41(1):68-73 [FREE Full ex ] [doi: 10.3109/00952990.2014.977486] [Medline: 25397860] 40. Ubhi HK, Michie S, Ko z D, Wong WC, Wes R. A mobile app o aid smoking cessa ion: p elimina y e alua ion o SmokeF ee28. J Med In e ne Res 2015 Jan 16;17(1):e17 [FREE Full ex ] [doi: 10.2196/jmi .3479] [Medline: 25596170] 41. Ge me o h LJ, Le ine MD. Pos cessa ion weigh gain conce n as a ba ie o smoking cessa ion: Assessmen conside a ions and u u e di ec ions. Addic Beha 2018 Jan;76:250-257 [FREE Full ex ] [doi: 10.1016/j.addbeh.2017.08.022] [Medline: 28865363] 42. Goldenbe g M, Dano i ch I, IsHak WW. Quali y o li e and smoking. Am J Addic 2014 Sep 25;23(6):540-562. [doi: 10.1111/j.1521-0391.2014.12148.x] [Medline: 25255868] 43. Fu S, Dao M, Wong CK, Yu W. SF-6D u ili y sco es o smoke s and ex-smoke s wi h o wi hou espi a o y symp oms a ending p ima y ca e clinics. Heal h Qual Li e Ou comes 2019 Ma 15;17(1):48 [FREE Full ex ] [doi: 10.1186/s12955-019-1115-z] [Medline: 30876466] 44. Alma io CV. The E ec o Digi al Heal h Technology on Pa ien Ca e and Resea ch. Gas oen e ol Hepa ol (N Y) 2017 Jul;13(7):437-439 [FREE Full ex ] [Medline: 28867974] 45. Ka hleen FH, Young-Il K, Mei ang C, Rekha R, Ma ia P, Rajani SS, e al. Web-Based In e en ion o T ansi ioning Smoke s F om Inpa ien o Ou pa ien Ca e: An RCT. Am J P e Med 2016 Oc ;51(4):620-629. [doi: 10.1016/j.amep e.2016.04.008] [Medline: 27647062] 46. Taylo G, Dalili M, Semwal M, Ci ljak M, Sheikh A, Ca J. In e ne -based in e en ions o smoking cessa ion. Coch ane Da abase Sys Re 2017 Sep 04;9:CD007078 [FREE Full ex ] [doi: 10.1002/14651858.CD007078.pub5] [Medline: 28869775] 47. Bal unas L, Ludwig B, Pee S, Ricci F. Con ex ele ance assessmen and exploi a ion in mobile ecommende sys ems. Pe s Ubiqui Compu 2011 Jun 21;16(5):507-526. [doi: 10.1007/s00779-011-0417-x] 48. BinDhim NF, McGeechan K, T e ena L. Sma phone Smoking Cessa ion Applica ion (SSC App) ial: a mul icoun y double-blind au oma ed andomised con olled ial o a smoking cessa ion decision-aid 'app'. BMJ Open 2018 Jan 21;8(1):e017105 [FREE Full ex ] [doi: 10.1136/bmjopen-2017-017105] [Medline: 29358418] 49. Ga ison K, Pal P, O'Malley SS, Pi man B, Gueo guie a R, Rojiani R, e al. C a ing o Qui : A Randomized Con olled T ial o Sma phone App-Based Mind ulness T aining o Smoking Cessa ion. Nico ine Tob Res 2020 Ma 16;22(3):324-331. [doi: 10.1093/n /n y126] [Medline: 29917096] 50. Baske ille NB, S uik LL, Guindon GE, No man CD, Whi ake R, Bu ns C, e al. E ec o a Mobile Phone In e en ion on Qui ing Smoking in a Young Adul Popula ion o Smoke s: Randomized Con olled T ial. JMIR Mheal h Uheal h 2018 Oc 23;6(10):e10893 [FREE Full ex ] [doi: 10.2196/10893] [Medline: 30355563] 51. Hassand a M, Lin unen T, Hagge MS, Heikkinen R, Vanhala M, Ke unen T. An mHeal h App o Suppo ing Qui e s o Manage Ciga e e C a ings Wi h Sho Bou s o Physical Ac i i y: A Randomized Pilo Feasibili y and Accep abili y S udy. JMIR Mheal h Uheal h 2017 May 26;5(5):e74 [FREE Full ex ] [doi: 10.2196/mheal h.6252] [Medline: 28550004] 52. McClu e JB, Ande son ML, B adley K, An LC, Ca z SL. E alua ing an Adap i e and In e ac i e mHeal h Smoking Cessa ion and Medica ion Adhe ence P og am: A Randomized Pilo Feasibili y S udy. JMIR Mheal h Uheal h 2016 Aug 03;4(3):e94 [FREE Full ex ] [doi: 10.2196/mheal h.6002] [Medline: 27489247] 53. Whi ake R, McRobbie H, Bullen C, Rodge s A, Gu Y, Dobson R. Mobile phone ex messaging and app-based in e en ions o smoking cessa ion. Coch ane Da abase Sys Re 2019 Oc 22;10:CD006611. [doi: 10.1002/14651858.CD006611.pub5] [Medline: 31638271] 54. C emonesi P, Ko en Y, Tu in R, edi o s. Pe o mance o ecommende algo i hms on op-n ecommenda ion asks. : RecSys '10; 2010 Sep P esen ed a : Fou h ACM Con e ence on Recommende Sys ems; 2010; Ba celona, Spain p. 39-46 URL: h ps://dl.acm.o g/doi/10.1145/1864708.1864721 [doi: 10.1145/1864708.1864721] 55. He nández del Olmo F, Gaudioso E. E alua ion o ecommende sys ems: A new app oach. Expe Sys ems wi h Applica ions 2008 Oc ;35(3):790-804. [doi: 10.1016/j.eswa.2007.07.047] 56. Ricci F, Rokach L, Shapi a B, Kan o P. In oduc ion o ecommende sys ems handbook. In: Recommende Sys ems Handbook. Bos on, Massachuse s: Sp inge ; 2011:1-35. 57. Sch öde G, Thiele M, Lehne W. Se ing goals and choosing me ics o ecommende sys em e alua ions. In: The RecSys 2011 Wo kshops - Decisions@RecSys 2011 and UCERSTI-2. 2011 Oc 23 P esen ed a : UCERSTI2 Wo kshop a he 5 h ACM con e ence on ecommende sys ems; 2011; Chicago p. 78-85 URL: h p://ceu -ws.o g/Vol-811/pape 12.pd 58. Shani G, Gunawa dana A. E alua ing ecommenda ion sys ems. In: Recommende sys ems handbook. Bos on, Massachuse s: Sp inge ; 2011:257-297. 59. Benne J, Lanning S. The ne lix p ize. In: P oceedings o KDD Cup and Wo kshop 2007. 2007 Aug 12 P esen ed a : KDD Cup and Wo kshop; 2007; San Jose, Cali o nia URL: h ps://www.cs.uic.edu/~liub/KDD-cup-2007/Ne lixP ize-desc ip ion. pd 60. Li S, Ka ahanna E. Online Recommenda ion Sys ems in a B2C E-Comme ce Con ex : A Re iew and Fu u e Di ec ions. JAIS 2015 Feb;16(2):72-107. [doi: 10.17705/1jais.00389] JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 21h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X 61. Scha e J, Kons an J, Riedl J. E-comme ce ecommenda ion applica ions. Da a mining and knowledge disco e y 2001 Jan;5(1-2):115-153 [FREE Full ex ] [doi: 10.1007/978-1-4615-1627-9_6] 62. He locke JL, Kons an JA, Te een LG, Riedl JT. E alua ing collabo a i e il e ing ecommende sys ems. ACM T ans. In . Sys 2004 Jan;22(1):5-53. [doi: 10.1145/963770.963772] 63. Ge M, Delgado-Ba en eld C, Jannach D. Beyond accu acy: e alua ing ecommende sys ems by co e age and se endipi y. In: RecSys '10: P oceedings o he ou h ACM con e ence on Recommende sys ems. 2010 Sep P esen ed a : The ou h ACM con e ence on Recommende sys ems; 2010; Ba celona, Spain p. 257-260 URL: h ps://dl.acm.o g/doi/10.1145/ 1864708.1864761 [doi: 10.1145/1864708.1864761] 64. McMillan B, Hickey E, Pa el MG, Mi chell C. Quali y assessmen o a sample o mobile app-based heal h beha io change in e en ions using a ool based on he Na ional Ins i u e o Heal h and Ca e Excellence beha io change guidance. Pa ien Educ Couns 2016 Ma ;99(3):429-435. [doi: 10.1016/j.pec.2015.10.023] [Medline: 26607787] 65. S oyano SR, Hides L, Ka anagh DJ, Zelenko O, Tjond onego o D, Mani M. Mobile app a ing scale: a new ool o assessing he quali y o heal h mobile apps. JMIR Mheal h Uheal h 2015 Ma 11;3(1):e27 [FREE Full ex ] [doi: 10.2196/mheal h.3422] [Medline: 25760773] 66. Lewis TL, Wya JC. App Usage Fac o : A Simple Me ic o Compa e he Popula ion Impac o Mobile Medical Apps. J Med In e ne Res 2015 Aug 19;17(8):e200 [FREE Full ex ] [doi: 10.2196/jmi .4284] [Medline: 26290093] 67. G undy QH, Wang Z, Be o LA. Challenges in Assessing Mobile Heal h App Quali y: A Sys ema ic Re iew o P e alen and Inno a i e Me hods. Am J P e Med 2016 Dec;51(6):1051-1059. [doi: 10.1016/j.amep e.2016.07.009] [Medline: 27659122] 68. McKay FH, Cheng C, W igh A, Shill J, S ephens H, Uccellini M. E alua ing mobile phone applica ions o heal h beha iou change: A sys ema ic e iew. J Telemed Teleca e 2016 Oc 18;24(1):22-30. [doi: 10.1177/1357633x16673538] 69. Singh K, D ouin K, Newma k L, Rozenblum R, Lee J, Landman A. De eloping a amewo k o e alua ing he pa ien engagemen , quali y, and sa e y o mobile heal h applica ions. Issue B ie , Commonw Fund Pub 2016 Feb 18;5:11 [FREE Full ex ] [doi: 10.15868/socialsec o .25066] 70. Ya dley L, Sp ing BJ, Ripe H, Mo ison LG, C ane DH, Cu is K, e al. Unde s anding and P omo ing E ec i e Engagemen Wi h Digi al Beha io Change In e en ions. Am J P e Med 2016 No ;51(5):833-842. [doi: 10.1016/j.amep e.2016.06.015] [Medline: 27745683] 71. Klasnja P, Consol o S, P a W. How o e alua e echnologies o heal h beha io change in HCI esea ch. In: CHI '11: P oceedings o he SIGCHI Con e ence on Human Fac o s in Compu ing Sys ems. 2011 May P esen ed a : The SIGCHI con e ence on human ac o s in compu ing sys ems; 2011; Vancou e , Canada p. 3063-3072 URL: h ps://dl.acm.o g/doi/ 10.1145/1978942.1979396 [doi: 10.1145/1978942.1979396] 72. Smi h W, Plode e B, Wadley G, Webbe S, Bo land R. T ajec o ies o engagemen disengagemen wi h a s o y-based smoking cessa ion app. In: CHI '17: P oceedings o he 2017 CHI Con e ence on Human Fac o s in Compu ing Sys ems. 2017 May P esen ed a : CHI '17: CHI Con e ence on Human Fac o s in Compu ing Sys ems; May, 2017; Den e , Colo ado, USA p. 3045-3056 URL: h ps://dl.acm.o g/doi/10.1145/3025453.3026054 [doi: 10.1145/3025453.3026054] 73. Sche e EA, Ben-Zee D, Li Z, Kane JM. Analyzing mHeal h Engagemen : Join Models o In ensi ely Collec ed Use Engagemen Da a. JMIR Mheal h Uheal h 2017 Jan 12;5(1):e1 [FREE Full ex ] [doi: 10.2196/mheal h.6474] [Medline: 28082257] 74. Taki S, Lyme S, Russell CG, Campbell K, Laws R, Ong K, e al. Assessing Use Engagemen o an mHeal h In e en ion: De elopmen and Implemen a ion o he G owing Heal hy App Engagemen Index. JMIR Mheal h Uheal h 2017 Jun 29;5(6):e89 [FREE Full ex ] [doi: 10.2196/mheal h.7236] [Medline: 28663164] 75. O'Conno S, Hanlon P, O'Donnell CA, Ga cia S, Glan ille J, Mai FS. Unde s anding ac o s a ec ing pa ien and public engagemen and ec ui men o digi al heal h in e en ions: a sys ema ic e iew o quali a i e s udies. BMC Med In o m Decis Mak 2016 Sep 15;16(1):120 [FREE Full ex ] [doi: 10.1186/s12911-016-0359-3] [Medline: 27630020] 76. Eysenbach G. The law o a i ion. J Med In e ne Res 2005 Ma 31;7(1):e11 [FREE Full ex ] [doi: 10.2196/jmi .7.1.e11] [Medline: 15829473] 77. Kohl LF, C u zen R, de V ies NK. Online p e en ion aimed a li es yle beha io s: a sys ema ic e iew o e iews. J Med In e ne Res 2013 Jul 16;15(7):e146 [FREE Full ex ] [doi: 10.2196/jmi .2665] [Medline: 23859884] 78. Anscombe J, Baca di A. AT Kea ney Insigh s. Chicago, Illinois: AT Kea ney; 2011. Is Mobile he Cu e o he Rising Cos s o Heal hca e? URL: h ps://www.a kea ney.com/documen s/10192/422267/ EAXIV_1_Is_Mobile_a_Cu e_ o _ he_Rising_Cos s_o _Heal hca e.pd /7337673b-bd46-4ae5-aa85-1906 83 6a05[accessed 2016-02-20] [WebCi e Cache ID 6 RLcACdh] 79. Iaco iello BM, S eine man JR, Klein DB, Sil e TL, Be ge AG, Luo SX, e al. Clicko ine, A Pe sonalized Sma phone App o Smoking Cessa ion: Ini ial E alua ion. JMIR Mheal h Uheal h 2017 Ap 25;5(4):e56 [FREE Full ex ] [doi: 10.2196/mheal h.7226] [Medline: 28442453] 80. Owen JE, Jawo ski BK, Kuhn E, Makin-By d KN, Ramsey KM, Ho man JE. mHeal h in he Wild: Using No el Da a o Examine he Reach, Use, and Impac o PTSD Coach. JMIR Men Heal h 2015 Ma 25;2(1):e7 [FREE Full ex ] [doi: 10.2196/men al.3935] [Medline: 26543913] JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 22h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X 81. B icke JB, Copeland W, Mull KE, Zeng EY, Wa son NL, Akioka KJ, e al. Single-a m ial o he second e sion o an accep ance & commi men he apy sma phone applica ion o smoking cessa ion. D ug Alcohol Depend 2017 Jan 01;170:37-42 [FREE Full ex ] [doi: 10.1016/j.d ugalcdep.2016.10.029] [Medline: 27870987] 82. Sho CE, DeSme A, Woods C, Williams SL, Mahe C, Middelwee d A, e al. Measu ing Engagemen in eHeal h and mHeal h Beha io Change In e en ions: Viewpoin o Me hodologies. J Med In e ne Res 2018 No 16;20(11):e292 [FREE Full ex ] [doi: 10.2196/jmi .9397] [Medline: 30446482] 83. Okoshi T, Tsubouchi K, Taji M, Ichikawa T, Tokuda H. A en ion and engagemen -awa eness in he wild: A la ge-scale s udy wi h adap i e no i ica ions. 2017 Ma 17 P esen ed a : 2017 IEEE In e na ional Con e ence on Pe asi e Compu ing and Communica ions, Pe Com 2017; Ma ch, 2017; Big Island, Uni ed S a es p. 100-110 URL: h ps://ieeexplo e.ieee.o g/ documen /7917856 [doi: 10.1109/pe com.2017.7917856] 84. Schwanda V, Iba a S, Reynolds L, Cosley D. Side e ec s and ga eway ools: ad oca ing a b oade look a e alua ing pe suasi e sys ems. In: CHI '11: P oceedings o he SIGCHI Con e ence on Human Fac o s in Compu ing Sys ems. Side e ec sga eway ools: ad oca ing a b oade look a e alua ing pe suasi e sys ems. P oceedings o he SIGCHI Con e ence on Human Fac o s in Compu ing Sys ems; 2011 May P esen ed a : CHI '11: CHI Con e ence on Human Fac o s in Compu ing Sys ems; May, 2011; Vancou e , Canada p. 345-348 URL: h ps://dl.acm.o g/doi/10.1145/1978942.1978991 [doi: 10.1145/1978942.1978991] 85. Clawson J, Pa e J, Mille A, Myna E, Mamykina L. No longe wea ing: in es iga ing he abandonmen o pe sonal heal h- acking echnologies on c aigslis . In: UbiComp '15: P oceedings o he 2015 ACM In e na ional Join Con e ence on Pe asi e and Ubiqui ous Compu ing. 2015 Sep P esen ed a : UbiComp '15: The 2015 ACM In e na ional Join Con e ence on Pe asi e and Ubiqui ous Compu ing; Sep embe , 2015; Osaka, Japan p. 647-658 URL: h ps://dl.acm.o g/doi/10.1145/ 2750858.2807554 [doi: 10.1145/2750858.2807554] 86. Eps ein D, Ca away M, Johns on C, Ping A, Foga y J, Munson S. Beyond Abandonmen o Nex S eps: Unde s anding and Designing o Li e a e Pe sonal In o ma ics Tool Use. In: CHI '16: P oceedings o he 2016 CHI Con e ence on Human Fac o s in Compu ing Sys ems. 2016 May P esen ed a : CHI'16: CHI Con e ence on Human Fac o s in Compu ing Sys ems; May 2016; San Jose, Cali o nia, USA p. 1109-1113 URL: h ps://dl.acm.o g/doi/10.1145/2858036.2858045 [doi: 10.1145/2858036.2858045] Abb e ia ions AI: a i icial in elligence BMI: body mass index CG: con ol g oup Eu oQoL-5D-5L: Eu oQol 5-dimension 5-le el HRS: heal h ecommende sys em IG: in e en ion g oup IPAQ: In e na ional Physical Ac i i y Ques ionnai e ResQue: Recommende Sys ems Ques ionnai e o Use Expe ience RRR: ela i e isk a io SF-36: 36-i em Sho Fo m Edi ed by G Eysenbach; submi ed 18.12.19; pee - e iewed by C Hao, A Vallée; commen s o au ho 13.01.20; e ised e sion ecei ed 03.03.20; accep ed 21.03.20; published 27.04.20 Please ci e as: Ca asco-He nandez L, Jóda -Sánchez F, Núñez-Benjumea F, Mo eno Conde J, Mesa González M, Ci i -Balcells A, Ho s-F aile S, Pa a-Calde ón CL, Bamidis PD, O ega-Ruiz F A Mobile Heal h Solu ion Complemen ing Psychopha macology-Suppo ed Smoking Cessa ion: Randomized Con olled T ial JMIR Mheal h Uheal h 2020;8(4):e17530 URL: h p://mheal h.jmi .o g/2020/4/e17530/ doi: 10.2196/17530 PMID: ©Lau a Ca asco-He nandez, F ancisco Jóda -Sánchez, F ancisco Núñez-Benjumea, Jesús Mo eno Conde, Ma co Mesa González, An ón Ci i -Balcells, San iago Ho s-F aile, Ca los Luis Pa a-Calde ón, Panagio is D Bamidis, F ancisco O ega-Ruiz. O iginally published in JMIR mHeal h and uHeal h (h p://mheal h.jmi .o g), 27.04.2020. This is an open-access a icle dis ibu ed unde he e ms o he C ea i e Commons A ibu ion License (h ps://c ea i ecommons.o g/licenses/by/4.0/), which pe mi s un es ic ed use, dis ibu ion, and ep oduc ion in any medium, p o ided he o iginal wo k, i s published in JMIR mHeal h and uHeal h, is JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 23h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X p ope ly ci ed. The comple e bibliog aphic in o ma ion, a link o he o iginal publica ion on h p://mheal h.jmi .o g/, as well as his copy igh and license in o ma ion mus be included. JMIR Mheal h Uheal h 2020 | ol. 8 | iss. 4 | e17530 | p. 24h p://mheal h.jmi .o g/2020/4/e17530/ (page numbe no o ci a ion pu poses) Ca asco-He nandez e alJMIR MHEALTH AND UHEALTH XSL • FO Rende X