Help for insomnia from the app store? A standardized rating of mobile health applications claiming to target insomnia [original]

RESEARCH ARTICLE

Help for insomnia from the app store? A standardized rating

of mobile health applications claiming to target insomnia

Laura Simon

| Josephin Reimann

| Lena Sophia Steubl

| Michael Stach

Kai Spiegelhalder

| Lasse Bosse Sander

| Harald Baumeister

Eva-Maria Messner

| Yannik Terhorst

Institute of Psychology and Education,

Department of Clinical Psychology and

Psychotherapy, Ulm University, Ulm, Germany

Institute of Databases and Information

Systems, Ulm University, Ulm, Germany

Faculty of Medicine, Department of

Psychiatry and Psychotherapy, Albert-

Ludwigs-University of Freiburg, Freiburg,

Germany

Institute of Psychology, Department of

Rehabilitation Psychology and Psychotherapy,

Albert-Ludwigs-University of Freiburg,

Freiburg, Germany

Correspondence

Laura Simon, Ulm University, Department of

Clinical Psychology and Psychotherapy, Lise-

Meitner-Straße 16, 89081 Ulm, Germany.

Email: [email protected]

Summary

A large number of mobile health applications claiming to target insomnia are available in

commercial app stores. However, limited information on the quality of these mobile

health applications exists. The present study aimed to systematically search the European

Google Play and Apple App Store for mobile health applications targeting insomnia, and

evaluate the quality, content, evidence base and potential therapeutic benefit. Eligible

mobile health applications were evaluated by two independent reviewers using the

Mobile Application Rating Scale-German, which ranges from 1 –inadequate to 5 –excel-

lent.Of 2236 identified mobile health applications, 53 were included in this study. Most

mobile health applications (68%) had a moderate overall quality. Concerning the four main

subscales of the Mobile Application Rating Scale-German, functionality was rated highest

(M=4.01, SD =0.52), followed by information quality (M=3.49, SD =0.72), aesthetics

(M=3.31, SD =1.04) and engagement (M=3.02, SD =1.03). While scientific evidence

was identified for 10 mobile health applications (19%), only one study employed a ran-

domized controlled design. Fifty mobile health applications featured sleep hygiene/

psychoeducation (94%), 27 cognitive therapy (51%), 26 relaxation methods (49%), 24 stim-

ulus control (45%), 16 sleep restriction (30%) and 24 sleep diaries (45%). Mobile health

applications may have the potential to improve the care of insomnia. Yet, data on the

effectiveness of mobile health applications are scarce, and this study indicates a large vari-

ance in the quality of the mobile health applications. Thus, independent information plat-

forms are needed to provide healthcare seekers and providers with reliable information

on the quality and content of mobile health applications.

KEYWORDS

apps, mHealth, mobile health, sleep disorder, systematic investigation

1|INTRODUCTION

The treatment of insomnia is highly relevant, given the high preva-

lence and burden of disease, and as insomnia is a risk factor for other

mental health disorders and somatic diseases (Hertenstein

et al., 2019; Morin et al., 2015; Sofi et al., 2014). National and interna-

tional clinical guidelines recommend Cognitive Behavioural Therapy

for Insomnia (CBT-I) as the first-line treatment (Riemann et al., 2017).

Received: 16 August 2021 Revised: 27 March 2022 Accepted: 4 May 2022

DOI: 10.1111/jsr.13642

This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any

medium, provided the original work is properly cited and is not used for commercial purposes.

J Sleep Res. 2022;e13642. wileyonlinelibrary.com/journal/jsr 1of14

https://doi.org/10.1111/jsr.13642

Despite CBT-I being a highly effective treatment, it is only provided

to a small proportion of patients suffering from insomnia, given its lim-

ited availability and accessibility (Koffel et al., 2018a). Given the lim-

ited availability of CBT-I and the emerging market of mobile health

applications (MHAs), patients suffering from insomnia or their

healthcare providers may refer to the commercial app stores. Indeed,

MHAs seem like a promising low-threshold approach for providing

digitalized CBT-I, given the ubiquity of smartphones and the possibil-

ity to access MHAs independent of place and time (Hussain

et al., 2015; Uyumaz et al., 2021). Moreover, MHAs may help to over-

come shortcomings of traditional on-site therapy, particularly if long

waiting times for on-site therapy are to be expected or if patients fear

being stigmatized for seeking on-site therapy (Andrade et al., 2014;

Ebert et al., 2018; Hussain et al., 2015).

Notwithstanding the potential benefits of MHAs, the free avail-

ability of MHA also harbors potential risks. While many app store

descriptions make claims regarding the effectiveness of MHAs, the

majority of MHAs available on the app market yield no direct scientific

evidence (Larsen et al., 2019; Terhorst et al., 2020), and the effective-

ness of already examined MHAs seems less established compared

with the well-established effectiveness of Internet-based interven-

tions (Weisel et al., 2019). In the case of insomnia, the efficacy of

Internet-based interventions is well studied (Soh et al., 2020;

Zachariae et al., 2016), whereas the evidence for MHAs delivering

CBT-I is limited. As no standards for MHAs available in the app stores

exist, the content of MHAs may not be guideline compliant, and

MHAs of low quality may lead to the dissemination of false informa-

tion, mistreatment or side-effects (Albrecht, 2016; Huckvale

et al., 2020). Moreover, issues regarding data protection, privacy and

the quality of the content of MHAs are of major concern (Hussain

et al., 2015). Thus, selecting a suitable MHA may be a major challenge

for both healthcare seekers and providers.

To our knowledge, two evaluations of commercially available MHAs

targeting insomnia have been conducted (Leigh et al., 2017;Yu

et al., 2019). Leigh et al. (2017) evaluated data security, clinical effective-

ness and user engagement using the ORCHA-24 framework of

19 Android MHAs available in the British Google Play Store identified

by the single search term “insomnia”.Yuetal.(

2019) evaluated the

quality of 12 MHAs available in the American Apple App and Google

Play Store using the Mobile Application Rating Scale (MARS; Stoyanov

et al., 2015) identified by the three search terms “insomnia”,“insomnia

treatment”and “sleep treatment.”A more complex search strategy may

be necessary to identify all relevant MHAs available in the Apple App

and Google Play Stores, as MHAs may have been indexed with other

keywords in the app stores. Moreover, given the high volatility of the

app market (Larsen et al., 2016), the two aforementioned evaluations

might be already outdated. Therefore, the primary aim of the study was

to provide an updated overview of the quality domains that are likely to

influence the effectiveness of MHAs (assessed using the MARS-German

[MARS-G]) of MHAs targeting insomnia that are currently available in

the Google Play and Apple App Stores. The secondary aim of the study

was a description of the content, evidence base and potential therapeu-

tic benefit of these MHAs.

2|METHODS

2.1 |Search strategy and eligibility criteria

A web crawler (Stach et al., 2020) was used to systematically screen

the European Apple App and Google Play Store with insomnia-related

search terms. The validity of this procedure has been demonstrated in

previous studies (Portenhauser et al., 2021; Schultchen et al., 2020;

Terhorst et al., 2018,2021). Search terms and eligibility criteria were

chosen with the intention that included MHAs are likely to represent

the MHAs with which healthcare seekers and providers are

TABLE 1 Eligibility criteria for inclusion

Inclusion criteria Exclusion criteria

Level 1: Before

downloading the

MHA; using

titles and

descriptions in

the app stores

•Target group:

adults suffering

from symptoms of

insomnia

•Useful for the

psychotherapeutic

treatment of

insomnia

according to the

description

•Title or

description

includes the word

“insomnia”or

“sleep disorder”

•MHA is available

in English,

German or French

•Only intended for

healthcare

providers

•Only intended for

relatives of

persons suffering

from insomnia

•Primarily intended

for other

disorders than

insomnia

•Needs another

device (e.g.

smartwatch) to

function

•Only available for

tablets

•Only functional in

blended-care

models

Level 2: After

downloading the

MHA; using the

content of the

MHAs

•Features at least

one of the

following CBT-I

components:

sleep hygiene/

psycho-

education,

stimulus control,

sleep restriction,

cognitive therapy

•Or features a

sleep diary or an

assessment of

sleep disturbances

•Fully functional to

allow assessment

•Features only an

alarm clock or

sleep-tracking

functions that

measure sleep

through sensors

•Features only

relaxing music,

noise or bedtime

stories

•eBooks

•Duplicate

•Not functional to

a degree that

allows assessment

•Part of the

content is only

accessible via

other modalities

(e.g. browser-

based

intervention)

Sleep hygiene education and psychoeducation were evaluated as one

treatment component in this study.

CBT-I, Cognitive Behavioural Therapy for Insomnia; MHA, mobile health

application.

2of14 SIMON ET AL.

confronted with when they search the app stores for suitable MHAs.

Table S1 summarizes the used search terms. The search was con-

ducted from 18 September to 23 September 2020. In addition, system-

atic literature reviews (Aji et al., 2020; Weisel et al., 2019) as well as

published evaluations of MHAs targeting insomnia available in the app

stores (Leigh et al., 2017; Yu et al., 2019) were screened for other rele-

vant MHAs. All identified MHAs were listed in a central database and

duplicates were removed. Identified MHAs were systematically

screened in a two-level process using pre-defined criteria as outlined in

Table 1. While sleep hygiene educations/psychoeducation and sleep

diaries are not effective face-to-face standalone interventions, we

decided to still include MHAs that featured only these components, as

they may be particularly interesting for therapists in blended-care

models. MHAs were only included if they were functional to a degree

that assessment was possible. Eventual technical problems were verified

on two separate devices. For the evaluation of the Android MHAs a

Huawei P10 lite (Modell WAS-LX1A) was used, and for the iOS MHAs

an Apple iPhone 6s (Modell NN0X2ZD/A) was used.

2.2 |Data collection process

Each MHA was rated by two independent reviewers with a degree in

psychology (JR, LS, JW and KB) using the MARS-G (Messner

et al., 2020). Before the rating, the reviewers received standardized

publicly accessible online training (https://www.youtube.com/watch?

v=5vwMiCWC0Sc). Each MHA was explored and used for at least

15 min to examine functionality, content and quality. The quality rat-

ing was carried out on a database specifically developed for these rat-

ings (Stach et al., 2020). If an MHA was available for both operating

systems (i.e. iOS and Android), the MHA was rated for both operating

systems individually. Author LSS was consulted if ratings of an item

differed by 2 points or more, and these discrepancies were resolved

by discussion. The overall quality rating showed an excellent level of

interrater reliability between the two reviews (two-way mixed for

agreement intraclass correlation coefficient [ICC] =0.92; 95% confi-

dence interval 0.91–0.93), and the internal consistencies were esti-

mated to be good to excellent for the subscales (Omega =0.86–

0.97), and excellent (Omega =0.96) for the overall ratings.

2.3 |General characteristics

The following descriptive and technical information were extracted

using the classification page of the MARS-G: (1) MHA name; (2) plat-

form (i.e. Apple App or Google Play Store); (3) annual cost in €(i.e. if

the MHA included a monthly subscription the annual cost was calcu-

lated); (4) user star ratings; and (5) privacy and security features. The

assessment of privacy and security features occurred on a descriptive

level (e.g. availability of privacy policy, imprint, usage of passwords,

and logins). All features were assessed based on the downloaded

MHAs, and only information disclosed within the MHA, its website or

its description in the app stores was investigated.

2.4 |App quality rating using the MARS-G and

scientific evidence

While the mechanisms of change are not sufficiently studied for

MHAs, it can be assumed that additional factors besides the content

influence the effectiveness of an MHA. For example, user engage-

ment and persuasive design appear to have a major influence on the

retention of MHAs outside the research context (Baumel et al., 2019).

Thus, MHA quality is likely to be a multidimensional construct (Nouri

et al., 2018). The MARS is a validated and widely used multi-

dimensional measure for the quality assessment of MHAs (Stoyanov

et al., 2015; Terhorst et al., 2020), which has been developed by a

multidisciplinary expert team. Its psychometric properties have been

investigated in an international validation study that included over

1200 MHAs from 15 indication areas, and the objectivity (ICC

=0.82), reliability (Omega =0.79–0.93), construct validity (root mean

sqaure error of approximation =0.074, Tucker-Lewis index =0.922,

confirmatory fit index =0.940, standardized root mean square resid-

ual =0.059) and concurrent validity were all good to excellent

(Terhorst et al., 2020). The generic formulation of the MARS items

allows for an adaption of the ratings to the targeted indication area.

While the MARS has been previously applied to the domain of insom-

nia (Yu et al., 2019), it must be noted that the MARS has not been val-

idated for MHAs targeting insomnia. For this study, we have used the

German version of the MARS (MARS-G) (Messner et al., 2020). The

quality rating of the MARS-G is based on a 5-point scale (i.e. 1 –

inadequate, 2 –poor, 3 –acceptable, 4 –good and 5 –excel-

lent), and includes four main subscales: (a) engagement (5 items: enter-

tainment, interest, customization, interactivity, target group);

(b) functionality (4 items: performance, usability, navigation, gestural

design); (c) aesthetics (3 items: layout, graphics, visual appeal);

(d) information quality (7 items: accuracy of app description, goals,

quality of information, quantity of information, quality of visual infor-

mation, credibility, evidence base; Messner et al., 2020). The informa-

tion quality was evaluated regarding the goal that was defined in the

app store description, which may limit the comparability of MHAs

with varying goals. For example, if the goal in the app store descrip-

tion was to educate users about insomnia, the focus of the informa-

tion quality evaluation was the quality of the psychoeducation. The

item evidence base was used to assess whether the MHAs have been

scientifically evaluated. This item was investigated by searching the

MHA name in Google Scholar, the developers' or providers' websites,

as well as systematic literature reviews of MHAs (Aji et al., 2020;

Weisel et al., 2019) for existing studies.

2.5 |Treatment components, potential therapeutic

benefit and potential therapeutic safety

To examine the compliance with guideline recommendations, it was

assessed if the MHAs included the following treatment components:

(a) sleep hygiene/psychoeducation; (b) stimulus control; (c) sleep restric-

tion; (d) cognitive therapy; (e) relaxation methods. Moreover, it was

SIMON ET AL.3of14

assessed if MHAs featured sleep diaries. Assessment of the featured

treatment components occurred on a descriptive level. Potential thera-

peutic gain and potential therapeutic safety were rated using the addi-

tional subscale therapeutic gain oftheMARS-G(Messneretal.,2020).

Therapeutic gain evaluates the benefit for the patient (i.e. to which extent

could the MHA support the user in the treatment of his or her symptoms),

benefit to the therapist (i.e. to which extent may the MHA help to opti-

mize the therapy), the transferability into a routine setting (i.e. has the

MHA been tested on patients and under conditions that are representa-

tive of routine psychotherapy setting), and potential therapeutic safety

(i.e. is there a risk for adverse effects due to misleading or wrong informa-

tion or incorrect recommendations). We extended the criteria defined by

this item by evaluating if MHAs formulated suspected or definitive diag-

noses, and if diagnoses were paired with the recommendation to consult

healthcare providers. Moreover, for MHAs featuring sleep restriction, it

was assessed if MHAs included information on contraindications

(e.g. sleep-disordered breathing or epilepsy; Spielman et al., 2011)and

possible negative effects of sleep restriction. Additionally, it was assessed

if information on differential diagnoses was provided, if app store

descriptions included disclaimers that the MHA does not substitute treat-

ment, and if information on finding on-site help was provided.

2.6 |Data analyses

The ratings of the two independent reviews were averaged for all cal-

culations. For the four main subscales, the average of the respective

items was taken, and for the overall quality the total score was calcu-

lated from the four main subscales of the MARS-G (Messner

et al., 2020). Mean scores (M) and standard deviations (SD) were calcu-

lated for the overall quality and the MARS-G main subscales. More-

over, the MARS-G overall rating and the ratings of the MARS-G

subscales were categorized as low (i.e. rating of less than 2.5), moder-

ate (i.e. rating between 2.5 and 4) and high (i.e. rating of 4 and higher).

Visual inspection of the histograms and Shapiro–Wilk normality

tests indicated a non-normal distribution of the data. Hence, Wilcoxon

rank-sum tests with continuity correction were used to test whether the

MHAs from the Apple App and Google Play Store differed regarding

their MARS-G overall ratings, and if MHAs with a free or non-free basic

version differed in their MARS-G overall rating. For all analyses, an alpha

level of 5% was defined. All statistical analyses were performed using R.

For an illustration of details of potentially helpful MHAs targeting

insomnia, MHAs that received a rating of 4 or higher for both the

overall quality and the additional subscale therapeutic gain will be

described in detail in the supplemental material (Table S2).

3|RESULTS

3.1 |Selection process

Figure 1displays the MHA selection process and provides an overview

of the reasons for exclusion. From 2236 identified MHAs, 53 MHAs

(2%) were included in this study. Thirty-seven MHAs were available in

the Google Play Store and 16 MHAs in the Apple App Store.

3.2 |General characteristics

The general characteristics of the included MHAs are summarized in

Table 2. The majority of the MHAs (85%) included a free basic ver-

sion. The annual cost of the eight MHAs requiring payment for the

basic version ranged from 1.57€to 10.99€(M=5.24€,SD =3.89€).

Ten MHAs included an extended version (i.e. to access all content of

the MHAs), with the annual cost ranging between 5.49€and 1920.00

€(M=225.10€,SD =596.68€,Median =35.99€). The MHAs that

were rated by users in the app stores (n=20) had an average user

star rating of M=3.85 (SD =0.80). The MHAs included on average

four security and privacy measures (M=3.62, SD =1.97). Most fre-

quently, a contact or imprint was provided (96%), while only seven

MHAs (13%) included an emergency function. Tables 3and 4detail

the security and privacy measures per MHA.

3.3 |App quality rating using the MARS-G and

scientific evidence

The overall quality of the MHAs conceptualized as the mean of the

four subscales of the MARS-G was moderate (M=3.46, SD =0.71);

14 MHAs received a high-quality rating, 36 a rating of moderate qual-

ity, and three MHAs a rating of low quality. Concerning the four main

subscales of the MARS-G, functionality was rated highest (M=4.01,

SD =0.52), followed by information quality (M=3.49, SD =0.72), aes-

thetics (M=3.31, SD =1.04) and engagement (M=3.02, SD =1.03).

Table 5summarizes the results of the MARS-G ratings, Tables 3and 4

detail the MARS-G ratings per MHA, and Figure S1 provides a graphi-

cal representation of the overall quality ratings and the four main sub-

scales of the MARS-G.

Two-sided Wilcoxon rank-sum tests indicated no significant dif-

ference (W=388.5, p> 0.05) in the overall quality between MHAs of

the Apple App Store (M=3.72, SD =0.71) and MHAs of the Google

Play Store (M=3.34, SD =0.69), and no significant difference

(W=234.5, p> 0.05) in the overall quality of MHAs with a free

(M=3.53, SD =0.73) and non-free basic version (M=3.07,

SD =0.44).

We were able to identify scientific evidence for 10 MHAs (19%).

Yet, this evidence included only one randomized controlled pilot study

that compared regular on-site CBT-I with a blended-care model that

paired on-site CBT-I with the MHA “CBT-i Coach”, and results of this

study yielded non-significant differences for insomnia severity (Koffel

et al., 2018b). The other evidence included observational studies (Eyal

et al., 2020; Harbison et al., 2018), a survey of clinicians (Kuhn

et al., 2016), and a randomized controlled trial investigating the

browser version of an MHA (Lorenz et al., 2019). Table 6provides a

summary of the evidence and the corresponding ratings for the item

evidence base of the MARS-G.

4of14 SIMON ET AL.

3.4 |Treatment components, potential therapeutic

benefit and potential therapeutic safety

The majority of the MHAs included sleep hygiene/psychoeducation

(n=50, 94%). Almost half of the MHAs included a sleep diary

(n=24, 45%) and 26 MHAs (49%) included relaxation methods. Cog-

nitive therapy (n=27, 51%), stimulus control (n=24, 45%) and sleep

restriction (n=16, 30.2%) were commonly featured in the MHAs.

Moreover, 17 MHAs (32%) featured a combination of sleep hygiene/

psychoeducation, behavioural therapy (i.e. sleep restriction or stimulus

FIGURE 1 Flowchart of the mobile

health application (MHA) selection process.

The British and German app Stores (Apple

App Store and Google Play Store) were

searched

SIMON ET AL.5of14

control) and cognitive therapy. Tables 3and 4detail the featured

treatment components per MHA.

Twelve MHAs (23%) were categorized as potentially beneficial

(i.e. rating of 4 or higher) for patients, and seven MHAs (13%) as

potentially beneficial for therapists using the additional subscale ther-

apeutic gain of the MARS-G. The ease of implementation in the routine

care was rated as low (i.e. rating of less than 2.5) for 45 MHAs (85%).

Figure S2 provides a visualization of the additional MARS-G subscale

therapeutic gain.

The item risks and side-effects indicated that 33 MHAs (62%) may

be associated with risks (i.e. rating of less than 4). Fourteen MHAs

included questionnaires to assess sleep disturbances. None of these

MHAs assessing sleep disturbances provided a definitive diagnosis.

Instead, seven MHAs indicated suspected diagnoses and advised

users to consult healthcare providers. Information on contraindica-

tions of using the MHA (e.g. epilepsy, shift work) was provided by the

MHAs “Insomnia Coach”(both operating systems) and “somnio”(both

operating systems). Additionally, the MHAs “Insomnia Coach”(both

operating systems) and “somnio”(both operating systems) included

information on potential side-effects of using the MHA. Thus, only

the MHAs “Insomnia Coach”(both operating systems) and “somnio”

(both operating systems) of the 16 MHAs featuring sleep restriction

included information on contraindications and/or possible adverse

effects of sleep restriction. The app store descriptions of 10 MHAs

included disclaimers that the MHAs do not substitute regular treat-

ment, and six MHAs included information on finding on-site help.

Moreover, seven MHAs included information on potential differential

diagnoses (e.g. sleep apnea).

The MHAs “Insomnia Coach”(both operating systems) and

“somnio”(both operating systems) achieved a high rating

(i.e. rating > 4) for the overall quality and the additional subscale ther-

apeutic gain. Thus, these MHAs may be particularly relevant for the

treatment of insomnia. Table S2 provides a detailed overview of

these MHAs.

4|DISCUSSION

We systematically assessed the general characteristics, quality rating

based on the MARS-G, evidence base, treatment components, poten-

tial therapeutic benefits and potential therapeutic safety of MHAs

targeting insomnia available in the Google Play and Apple App stores.

The screening process revealed a plethora of MHAs in the app stores.

Given the large number of MHAs targeting insomnia that feature con-

tent that may not be considered therapeutic (e.g. alarm clocks, relaxa-

tion music), it may be difficult for healthcare seekers and providers to

choose a suitable MHA.

Engagement was rated lowest of the four main subscales of the

MARS-G (M=3.02, SD =1.03), and almost 40% of the rated MHAs

were categorized as having a low level of engagement. This is a pat-

tern that has been also found in investigations of MHAs targeting

other mental health domains (Terhorst et al., 2020). Yet, user engage-

ment might be an important countermeasure against low retention

rates, which in turn appear to be a problem of MHAs in real-world set-

tings (Baumel et al., 2019). Thus, it may be advisable to employ fea-

tures of smartphones that offer unique benefits to therapy

(e.g. reminding functions and responsive sleep diaries), and persuasive

design to foster user engagement and retention (Baumeister

et al., 2019; Uyumaz et al., 2021).

TABLE 2 General characteristics of the reviewed MHAs

MHAs (%) M(SD)

App store/operating system

Apple App Store/iOS 16 (30.2%)

Google Play Store/Android 37 (69.8%)

Annual costs

Number and annual cost of

MHAs requiring payment for

the basic version

8 (15.1%) 5.24€(3.90€)

Number and annual cost of

MHAs offering an extended

versions

10 (18.9%) 225.10€(596.68€)

User star ratings

Apple App Store

MHAs with rating and

respective user star rating

5 (31.3%) 4.34 (0.65)

Google Play Store

MHAs with rating and

respective user star rating

15 (40.5%) 3.68 (0.79)

Security and privacy

Allows password use 19 (35.8%)

Requires a login 14 (26.4%)

Has a privacy statement 40 (75.4%)

Requires active confirmation of

a consent form

20 (37.7%)

Information on how data are

handled

37 (69.8%)

Contact/contact person/

imprint

51 (96.2%)

Secure data transfer 17 (32.1%)

Emergency functions available 7 (13.2%)

Security strategies for mobile

phone loss

4 (7.5%)

Eight MHAs, thus 16 in total, were available for both app stores.

The MHA “somnio”was classified as free of charge, as German users do

not need to pay a fee if general care physicians or psychotherapists

prescribe the MHA. However, the statuary health insurances have to pay

464,00€per prescription.

Users have to pay a fee to access the full content of MHAs offering an

extended version.

Multiple naming of different data protection precautions for one MHA is

possible.

MHA, mobile health application.

6of14 SIMON ET AL.

TABLE 3 Overview of rated iOS MHAs targeting insomnia

MHA name (developer)

Available

on android

User star

rating

(nratings)

Annual cost

basic

(extended)

Security

features Engagement Functionality Aesthetics Information

Overall

quality

Treatment

components

Somnio (mementor DE GmbH) Yes 4.2 (43) Free (NA)

Pw,Log,Pri,IC,

Info,Con,Mo

5.00 4.50 5.00 4.36 4.71 SHE,SC,SR,CT,

RE,SD,MI

Sleep Cure: Smart Sleep

Coach (SleepCure AB)

Yes None Free (35.99€) Pw,Pri,IC,Info,Con 4.60 4.25 4.83 4.75 4.61 SHE,SC,CT,RE,

SD,MI

Insomnia Coach (US

Department

of Veterans Affairs)

Yes 4 (12) Free (NA) Pri,IC,Info,Con,Em 4.30 4.12 4.67 4.57 4.42 SHE,SC,SR,CT,

RE,SD,MI

Soutien psy avec Mon

Sherpa (doctoconsult)

Yes None Free (NA) Pw,Log,Pri,IC,

Info,Con,Em

4.50 3.62 4.67 4.33 4.28 SHE,CT,RE

Sleeprate. Balance

Your Sleep. (Sleeprate)

Yes None Free (131.88€) Pw,Log,Pri,IC,

Info,Con,Mo

4.20 4.25 4.50 4.00 4.24 SHE,SC,CT,RE,SD,

Sleep School for Professionals

(The Sleep School Ltd)

No None Free (NA) Pw,Log,Pri,Con 3.90 4.25 4.50 3.83 4.12 SHE,CT,RE,MI

Sleepedy (Sleepedy, Inc.) No None Free (NA) Pw,Pri,IC,Info,Con 3.30 4.25 4.00 4.08 3.91 SHE,SC,SR,CT,SD

Sleep: Better Sleep with

CBT (Learning 2 Sleep)

No None Free (NA) Pw,Pri,IC,Info,Con 3.50 4.50 4.00 3.58 3.90 SHE,CT,RE,MI

Sleep School for Insomnia

(The Sleep School Ltd)

Yes None Free (37.99€) Pw,Log,Pri,Info,

Con

3.70 4.12 3.83 3.67 3.83 SHE,CT,MI

CBT-i Coach (US Department

of Veterans Affairs)

Yes 3.5 (56) Free (NA) Pri,IC,Info,Con,Em 3.80 3.50 3.33 4.57 3.80 SHE,SC,SR,CT,

RE,SD,MI

Night Owl - Sleep Coach

(Mindware Consulting, Inc)

Yes None 10.99€(NA) Pw,Log,Pri,Con 3.00 3.75 3.17 3.57 3.37 SHE,SC,CT,RE,SD,

Mieux dormir –Psychologies

(Teach on Mars)

No None 5.49€(NA) Pri,Info,Con 2.90 3.62 3.83 2.92 3.32 SHE,SC,CT,RE,MI

Besser Ein- und Durchschlafen

(Audiio GmbH)

No None 2.29€(NA) Con 2.20 4.62 3.17 3.25 3.31 SHE,RE,MI

Sleep Solution: Insomnia help

(Tom McKay)

No 5 (2) 2.30€(NA) Con 2.30 3.25 2.33 3.50 2.85 SHE,SC,SR,CT,

RE,SD,MI

Sleep-Diary (Anel Pasic) No None Free (NA) Info 2.30 3.75 2.67 2.00 2.68 SD

Sleep Smart (MindApps) No 5 (1) 2.29€(NA) Con 2.60 2.12 2.17 2.17 2.26 SHE,SR,RE,SD

Note: Con, contact details; CT, cognitive therapy; Em, emergency functions; IC, informed consent; Info, information on dealing with data; MHA, mobile health application; MI, mindfulness; Mo, safety

measurements for mobile loss; Pri, privacy statement; Pw, password protection; Re, relaxation strategies; SC, stimulus control; SD, sleep diary; SHE, sleep hygiene education/psychoeducation; SR, sleep

restriction.

Pound was converted into Euro according to the exchange rate (1 GBP =1,15 €) on 03.03.2021, and dollar was converted into Euro according to the exchange rate (1$ =0.84€) on 10.03.2021.

The MHA “somnio”was classified as free of charge, as German users do not need to pay a fee if general care physicians or psychotherapists prescribe the MHA. However, the statuary health insurances have

to pay 464,00€per prescription.

SIMON ET AL.7of14

TABLE 4 Overview of rated Android MHAs targeting insomnia

MHA name (developer)

Available

on iOS

User star

rating

(nratings)

Annual cost basic

(extended)

Security

features Engagement Functionality Aesthetics Information

Overall

quality

Treatment

components

Somnio (mementor DE GmbH) Yes 3.8 (47) Free (NA)

Pw,Log,Pri,IC,

Info,Con,Mo

5.00 4.62 5.00 4.36 4.75 SHE,SC,SR,CT,RE,

SD,MI

Soutien psy avec Mon Sherpa (Qare –

consultations médicales)

Yes None Free (NA) Pw,Log,Pri,IC,

Info,Con,Em

4.90 4.38 4.83 4.25 4.59 SHE,CT,RE

Sleep Cure (Sleep Cure) No 1.7 (12) Free (52.68€) Pw,Log,Pri,Info,Con 4.50 4.25 5.00 4.25 4.50 SHE,SC,SR,CT,RE,

SD,MI

Insomnia Coach (US Department of

Veterans Affairs)

Yes 3.6 (13) Free (NA) Pri,IC,Info,Con,Em 4.30 4.12 4.50 4.79 4.43 SHE,SC,SR,CT,RE,

SD,MI

HALEO –Sleep better within 2 weeks

(HALEO Preventive Health Solutions

Inc.)

No 4.5 (38) Free (1920.00€) Pw,Log,Pri,IC,

Info,Con

4.10 4.25 4.83 4.25 4.36 SHE,SC,SR,CT,RE,

BedTime Helper (PsyNovigo Ltd) No 3.1 (23) Free (NA) Pri,IC,Info,Con 3.70 4.62 4.83 4.10 4.31 SHE,SC,RE

Sleeprate. Balance Your Sleep (Sleeprate) Yes 3.9 (29) Free (9.99€) Pw,Log,Pri,IC,

Info,Con,Mo

4.00 4.12 4.67 3.71 4.13 SHE,SC,CT,RE,SD,

Sleep School for Insomnia (Sleep School) Yes None Free (NA) Pw,Log,Pri,IC,

Info,Con

4.20 4.25 4.00 3.92 4.09 SHE,CT,MI

Sleep Theory –Sleep Aid & Smart Alarm

Clock (Nox Limited)

No None Free (35.99€) Pw,Pri,Info,Con 4.10 3.62 4.17 4.00 3.97 SHE,SC,SR,RE,SD

InsomniaFix (NOVOS Behavioral Health

Solutions, LLC)

No 4.1 (10) Free (NA) Pri,Info,Con 3.30 4.50 3.67 4.10 3.89 SHE,SR,SD

Circady Sleep Diary (Circady) No 2.4 (14) Free (NA) Pw,Log,Pri,IC,

Info,Con

3.40 4.38 3.83 3.75 3.84 SD

CBT-i Coach (US Department of

Veterans Affairs)

Yes 3.8 (150) Free (NA) Pri,IC,Info,Con,Em 3.70 4.00 2.83 4.57 3.78 SHE,SC,SR,CT,RE,

SD,MI

Night Owl –Sleep Coach (Mindware

Consulting, Inc)

Yes 3.5 (34) 10.99€(NA) Pw,Log,Pri,Info,Con 3.00 4.12 3.17 3.64 3.48 SHE,SC,CT,RE,SD,

Sleep Restore (Mark Grant) No None Free (9.99€) Pri,IC,Info,Con 2.70 3.88 3.33 3.50 3.35 SHE,RE

Sleep Log Pro: The CBT-I sleep diary for

insomnia (Mind and Body)

Yes None 5.99€(NA) Pri,Info,Con 3.00 4.00 3.33 3.00 3.33 SHE,SC,SR,SD

Sleep Disorders (Subject Mastery

Academy)

No None Free (NA) Pri,Con 2.30 4.38 3.17 3.40 3.31 SHE

Sleep Log Free: The CBT-I sleep diary for

insomnia (Mind and Body)

No None Free (NA) Pri,Info,Con 3.10 3.88 3.17 3.08 3.31 SHE,SC,SR,CT,SD

Phantasiereisen zum Einschlafen

(start2dream.de)

No 4.4 (778) Free (5.49€) Pri,Info,Con 2.50 4.38 2.67 3.20 3.19 SHE,CT,

ezeCBT –(CBT –Cognitive Behavioural

Therapy) (Smash Appz)

No 4.4 (17) Free (NA) Pw,Pri,Info 3.00 3.25 3.00 3.17 3.10 SHE,CT

8of14 SIMON ET AL.

TABLE 4 (Continued)

MHA name (developer)

Available

on iOS

User star

rating

(nratings)

Annual cost basic

(extended)

Security

features Engagement Functionality Aesthetics Information

Overall

quality

Treatment

components

How to Cure Insomnia (Amamiya Apps) No None Free (NA) Pri,Info,Con 2.00 4.50 3.17 2.70 3.09 SHE,CT

How to Cure Insomnia (NC Media

Solutions)

No None Free (NA) Con 2.10 4.75 2.17 3.12 3.04 SHE,RE

How To Treat Insomnia

(freeCreativity2019)

No None Free (NA) Pri,IC,Info,Con 2.00 4.50 2.33 3.25 3.02 SHE

sleep disorders (referencehunt) No None Free (NA) Con 1.70 4.12 2.50 3.60 2.98 SHE

Sleep Smarter –Fight insomnia &

improve sleeping (Boost Media)

No None Free (NA) Pri,Info,Con 2.00 4.00 2.67 3.20 2.97 SHE,SC

Insomnia (FREE) (Yamz Apps) No 3.4 (5) Free (NA) Con 2.00 4.50 2.17 3.00 2.92 SHE

Sleep Diary (Medon) No None Free (NA) Info,Con 2.20 4.38 2.33 2.75 2.91 SD

Bien dormir Mieux vivre (rifio) No None Free (NA) Con 2.10 3.88 2.33 3.25 2.89 SHE,SC,RE

Cognitive Behavioural Therapy

(AppMaster365)

No 4.3 (9) Free (NA) Con 2.10 4.12 2.33 3.00 2.89 SHE

Sleep Disorders And Problems (Twayesh

Projects)

No None Free (NA) Pri,Info,Con 1.90 4.25 2.17 2.75 2.77 SHE

Psychiatry Pro-Diagnosis,Info,Treatment,

CBT & DBT (BDR limited)

No None Free (NA) Pri,Info,Con,Em 2.40 3.25 2.00 3.33 2.75 SHE,SC,SR,CT

Sleep Aid App Free –Insomnia Cure App

(Beatrix)

No None Free (NA) Pri,Con 2.00 3.25 2.00 3.42 2.67 SHE

Insomnia(Yamz Apps) No None 1.57€(NA) Con 1.70 4.50 1.83 2.40 2.61 SHE,SC

Sleep Better Guide (Expert Health

Studio)

No None Free (NA) Pri,IC,Info,Con 1.80 3.12 2.17 3.30 2.60 SHE,SD

Sleep Hygiene Guide (Creative Writing

Apps)

No None Free (NA) Con 1.80 4.00 1.83 2.60 2.56 SHE

Sleep Yoga & Meditation –Cure

Insomnia & Snoring (Dr Zio –Yoga

Teacher)

No None Free (10.99€) Pri,Info,Con 2.50 2.75 2.33 2.67 2.56 SHE

Insomnia Treatment

Remedies(StatesApps)

No 4.3 (6) Free (NA) Con 1.50 3.88 2.67 1.83 2.47 SHE

Wie zu überwinden Schlaflosigkeit

(Naura shaki)

No None Free (NA) Pri,Con 1.50 3.50 1.83 2.50 2.33 SHE

Note: Con, contact details; CT, cognitive therapy; Em, emergency functions; IC, informed consent; Info, information on dealing with data; MHA, mobile health application; MI, mindfulness; Mo, safety measurements for mobile

loss; Pri, privacy statement; Pw, password protection; Re, relaxation strategies; SC, stimulus control; SD, sleep diary; SHE, sleep hygiene education/psychoeducation; SR, sleep restriction.

Pound was converted into Euro according to the exchange rate (1 GBP =1,15 €) on 03.03.2021, and dollar was converted into Euro according to the exchange rate (1$ =0.84€) on 10.03.2021.

per prescription.

SIMON ET AL.9of14

Inaccurate, lacking or misleading information may impact users'

safety (Albrecht, 2016; Huckvale et al., 2020). Our ratings indicated that

most rated MHAs had a moderate information quality (62%). While

16 MHAs (30%) were of high information quality, there were also four

MHAs (8%) that were of low information quality. Given the abundance

of available MHAs targeting insomnia and the high variance in quality,

the selection of a suitable MHA might be a particularly difficult task for

healthcare seekers. Moreover, app store descriptions and user star rat-

ings may be misleading in the selection process (Nicholas et al., 2015).

However, several independent information platforms (e.g. mhad.

science,mindapps.org) aim to provide reliable and publicly accessible

information on the quality, scope, functionality and security features of

MHAs. Yet, many healthcare seekers and providers are unaware of

these initiatives. Hence, it appears to be important to disseminate infor-

mation about these platforms (e.g. via primary care settings, social

media) to increase their impact. Ultimately, the healthcare seekers

themselves will decide on which MHA to use. Thus, it seems important

to educate healthcare seekers on how to select a suitable MHA

(e.g. evidence-based content, scientific evaluation).

While we were able to identify scientific evidence for 10 MHAs

(19%), this evidence included only one randomized controlled pilot

study, and results of this study yielded non-significant group differ-

ences between on-site CBT-I paired with a MHA and regular on-site

CBT-I on insomnia severity (Koffel et al., 2018b). The other identified

studies were observational (Eyal et al., 2020; Harbison et al., 2018),

surveyed clinicians' perception of the MHA (Kuhn et al., 2016), or

investigated only the browser version (Lorenz et al., 2019). It appears

that MHAs that have been scientifically evaluated are often not avail-

able in the app stores, whereas MHAs that are available in the app

stores have often not been scientifically evaluated. Thus, evidence for

the effectiveness of freely available MHAs is not sufficient. In fact, a

recently published review by Aji et al. (2020) found that of eight scien-

tifically evaluated MHAs targeting insomnia, only one MHA

(i.e. “CBT-I coach”) was available in the app stores. Besides digitalized

CBT-I programs that are exclusively available via MHAs, there is also

an emerging number of scientifically evaluated digitalized CBT-I pro-

grams that are available via other modalities or which offer parts of

the intervention via MHAs. Meta-analyses support the efficacy of

these programs (Soh et al., 2020; Zachariae et al., 2016). However,

the scientific evaluations of these programs focus on the browser-

based versions of the programs, and it is not clear if the corresponding

MHAs work the same way (Moshe et al., 2021). Consequently, scien-

tific evaluations should study if the efficacy of browser-based CBT-I

programs can be generalized to MHAs that deliver the same content.

Moreover, the scientific community and healthcare systems should

implement ways that facilitate the dissemination of MHAs that have

been scientifically evaluated and proven to be effective. For example,

Germany has established a billing model where scientifically evaluated

MHAs can be prescribed by healthcare providers. It should be

observed whether such approaches promote rigorous scientific evalu-

ations of MHAs and their subsequent dissemination.

It seems important to include information on rationale, possible

adverse effects and contraindications of sleep restriction, as sleep

restriction is associated with adverse effects (e.g. excessive sleepiness,

difficulty to concentrate; Kyle et al., 2011) and is contraindicated for

certain conditions (e.g. sleep-disordered breathing or epilepsy;

Spielman et al., 2011). Yet, while 16 MHAs featured sleep restriction,

only four MHAs included such information. In addition to the afore-

mentioned risks, the investigation showed that inadequate data pro-

tection measures may also pose risks for users.

Despite these named issues, it seems promising that almost a

third of the rated MHAs featured a combination of psychoeducational

content/sleep hygiene, behavioural therapy, and cognitive therapy. In

particular, the highest-rated MHAs, which are described in detail in

Table S2, seem to have the potential to improve the care of insomnia,

as they included high-quality content, precautions for user's safety,

and features to enhance user engagement. Nevertheless, to improve

the care of insomnia and address the existing treatment gap, strate-

gies to disseminate MHAs that have been scientifically proven to be

effective need to be implemented.

4.1 |Strengths and limitations

We followed a well-established systematic approach for the evalua-

tion of MHAs, including an extensive and systematic search, a

TABLE 5 Quality ratings per subscale using the MARS-G

Mean

Standard

deviation Minimum Maximum

nMHAs

categorized

as low (%)

nMHAs

categorized

as moderate (%)

nMHAs

categorized

as high (%)

Overall quality 3.46 0.71 2.26 4.75 3 (5.7%) 36 (67.9%) 14 (26.5%)

Engagement 3.02 1.03 1.5 5 21 (39.6%) 19 (35.8%) 13 (24.5%)

Functionality 4.01 0.52 2.12 4.75 1 (1.9%) 17 (32.1%) 35 (66.0%)

Aesthetics 3.31 1.04 1.83 5 16 (30.2%) 20 (37.7%) 17 (32.1%)

Information 3.49 0.72 1.83 4.79 4 (7.5%) 33 (62.3%) 16 (30.2%)

Potential

therapeutic gain

2.58 0.85 1.25 4.88 31 (58.5%) 18 (34.0%) 4 (7.5%)

Note: The categorization was based on following criteria: low rating: < 2.5; middle rating: ≥2.5 and < 4; high rating: ≥4.

MHA, mobile health application.

10 of 14 SIMON ET AL.

TABLE 6 Synthesis evidence base

MHA CBT-i coach/Insomnia Coach

Somnio Sleeprate NightOwl

Reference Koffel et al. (2018b) Kuhn et al. (2016) Lorenz et al. (2019) Eyal et al. (2020)

Harbison et al. (2018)

Study design Randomized controlled pilot study

comparing onsite CBT-I with

onsite CBT-I plus MHA

Survey of CBT-I clinicians (working

for the US Department of

Veterans Affairs clinics)

Randomized controlled study (for

browser version of the MHA; did

not evaluate the MHA itself)

Observational study (pre-post

without control group)

Observational study (pre-post

without control group) using

participant sleep log data of the

MHA

Sample size

18 176 52 192

157

Insomnia diagnosis

according to

Not specified (referrals for CBT-I) Not applicable DSM-V & ISI ≥8 No information WASO > 29 min and SOL > 29 min

Summary results Intention-to-treat analysis indicated

no significant group differences

for insomnia severity. Semi-

structured interviews indicated an

improvement in the accessibility

of therapeutic materials and all

participants reported that they

would recommend the MHA to

family and friends.

Surveyed clinicians indicated

positive perceptions of the MHA

and that the MHA may enhance

the delivery of CBT-I in blended-

care models.

Significant differences (both within-

group and between-group) on

insomnia severity and significant

within-group differences on all

measured sleep diary parameters.

Significant within-group differences

in SOL, WASO and SE.

Significant within-group differences

in SOL, WASO, TST, and SE.

Results -ISI (main effect for treatment

condition): d=0.21

-Results of semi-structured

interviews: All participants of the

app group (n=9, 100%) would

recommend the app to family or

friends.

-Relative advantage M=5.15

(SD =0.79)

-Compatibility M=5.48 (SD =0.89)

-Complexity M=5.74 (SD =1.22)

-Future use intention M=6.22

(SD =0.82)

Between group effect sizes:

-ISI: d=1.79

Within-group effect sizes:

-ISI: d=1.59

-SE: d=1.16

-SOL: d=0.5

-WASO: d=1.02

-TST: d=0.45

-Restfulness: d=1.48

-Daytime performance: d=0.99

-SOL (for participants who had

initial SOL > 30 min): M=53.9

(SD =20.8) to M=32.7

(SD =25.4, p< 0.001.

-WASO (for participants who had

initial WASO > 30 min): M=46.3

(SD =19.0) to M=35.8

(SD =21.4), p< 0.001.

-SE (for participants who had initial

SE < 85) increase of 7.1%

(p> 0.001)

-SOL: M=55.1 (SD =61.4) to

M=27.9 (SD =33.3), t

(69) =6.4, p< 0.001,

-WASO: M=68 (SD =80.9) to

M=39.1 (SD =45.6), t

(69) =5.9, p< 0.001,

-SE: M=75.9 (SD =85.8) to

M=85.8, (SD =12.7), t

(69) =9.1, p< 0.001

-TST from M=6.5 (SD =2.3) to

M=6.9 (SD =2.0), t(69) =2.9,

p<0.002

-NWAK from M=2.6 (SD =3.1) to

M=2.2, (SD =2.6), t

(69) =1.5, p< 0.07

OCEBM Level of

Evidence

2b 5 1b 4 4

MARS-G -Item

Evidence Base

Rating

4322

Note: CBT-I, Cognitive Behavioural Therapy for Insomnia; D, Cohens´D; DSM-V, Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; ISI, Insomnia Severity Index; M, mean; MHA, mobile health application;

NWAK, number of awakenings; SD, standard deviation; SE, sleep efficiency; SOL, sleep-onset latency; TST, total sleep time; WASO, wake after sleep onset.

The evidence from CBT-i Coach was counted as evidence for both “Insomnia Coach”and “CBT-i Coach”, as the MHA “Insomnia Coach”is an updated version of the MHA “CBT-i Coach”.

Published in conference proceedings.

Sample size of the here-presented results.

Analysed subsample that experienced symptoms of insomnia.

Analysed subsample that completed at least 50% of the program.

According to the Centre for Evidence-Based Medicine, Oxford (OCEBM Levels of Evidence Working Group, “The Oxford 2011 Levels of Evidence”, Oxford Centre for Evidence-Based Medicine: http://www.cebm.net/index.

aspx?o=5653).

SIMON ET AL.11 of 14

screening based on pre-defined criteria, and a quality evaluation using

an objective, reliable and valid scale (Messner et al., 2020; Stoyanov

et al., 2015;Terhorstetal.,2020). Moreover, we evaluated MHAs avail-

able in German, English and French, which are the most commonly spo-

ken languages in the European Union (European Commission, 2012).

Nonetheless, we are mindful of the limitations of this study.

Given the volatility of the app market (Larsen et al., 2016), the present

review must be understood as a snapshot at the time of the search

conducted in September 2020. Additionally, the quality evaluation

was conducted on a meta-level and not per treatment component.

Thus, future studies should also evaluate the quality of the individual

treatment components per MHA. Moreover, a limited MHA testing

time of at least 15 min does not allow for an in-depth analysis of each

app. Hence, while the MARS scale has shown its psychometric quality,

it cannot be excluded that in-depth analysis on each MHA would pro-

vide differentiating insights. Given that the psychometric validation of

the MARS did not include MHAs targeting insomnia (Terhorst

et al., 2020), future studies should investigate the construct validity,

concurrent validity and re-test reliability of the MARS in the domain

of insomnia. Sleep restriction for example can be done in several ways

and the details of the implementation matter. Therefore, ultimately

scientific evidence on the effectiveness of each MHA is needed to

conclude on its usefulness. Furthermore, the MARS ratings are based

on the goals defined in the app stores. Correspondingly, MHAs with

fewer goals (e.g. providing a sleep diary) may achieve higher ratings

than more complex MHAs (e.g. providing a full CBT-I) if they have

been evaluated to adequately achieve the defined goal, which may

lead to an inflated rating of some of the MHAs. Hence, the ratings of

MHAs with varying treatment components may not be comparable.

Therefore, it is important to not solely rely on the MARS rating but to

additionally consider the treatment components that are featured in

the MHA when selecting an MHA. Moreover, we only included MHAs

from the Apple App and Google Play Store that may have caused a

selection bias. However, as the Apple App and Google Play Store

compromise over 99% of the total market (StatCounter, 2021), the

number of missed MHAs should be low. Additionally, only the German

and British app stores were searched. Searches in app stores of other

countries may have led to more MHAs meeting our eligibility criteria.

According to our eligibility criteria, we only included MHAs featuring

at least one CBT-I component. Thus, we did not include all MHAs

targeting insomnia nor did we examine MHAs that feature cir-

cumscribed therapeutic supporting tools or other tools for healthcare

providers and seekers. Moreover, privacy and data security features

were only assessed descriptively in this study. Thus, for a full appraisal

of the quality of MHAs presented in this study, an additional assess-

ment of the technical quality would be necessary.

5|CONCLUSION

A plethora of MHAs claiming to target insomnia exists in commer-

cial app stores. Our rating of 53 MHAs available in the European

app stores indicated a large variance in the quality using the

MARS-G. Some of the rated MHAs achieved a high rating indicating

the potential of MHAs in the care of insomnia. Yet, the rating also

revealed shortcomings of some MHAs, and that the scientific evi-

dence for MHAs available in the app stores is only preliminary.

Given these findings, it seems important to provide healthcare

seekers and providers with reliable information on the quality and

content of the MHAs using independent information platforms. To

realize the full potential of MHAs in the treatment of insomnia, the

unique technical aspects of smartphones and persuasive design

should be considered, and strategies to disseminate effective MHAs

need to be developed.

AUTHOR CONTRIBUTIONS

Laura Simon and Lena Sophia Steubl initiated this study. Laura Simon,

Lena Sophia Steubl, Josephin Reimann, Yannik Terhorst, Eva-Maria

Messner and Harald Baumeister contributed to the study design and

conceptualized the current research question. Michael Stach helped

compile the mobile health application data. Laura Simon and Josephin

Reimann rated the mobile health applications. Throughout the assess-

ment, raters were supervised by Lena Sophia Steubl (psychotherapist

in training), Eva-Maria Messner (licensed psychotherapist) and Lasse

Bosse Sander (licensed psychotherapist). Laura Simon, Lena Sophia

Steubl, Josephin Reimann and Yannik Terhorst conducted the data

analyses. Laura Simon wrote the first draft of the manuscript. All

authors revised and approved the final version of the manuscript for

submission.

ACKNOWLEDGEMENTS

The authors would like to thank Robin Kraft and Rüder Pryss for their

support in the development of the search engine and their support in

the MHAD project. The authors also thank Julia Weresch for her

assistance in rating the mobile health applications; Katja Barck, Char-

lotte Dechmann and Chiara Ritter for their assistance in rating the

French mobile health applications; and Isabelle Keller and Bettina

Salger for their help with the manuscript.

FUNDING INFORMATION

This research received no specific grant from any funding agency,

commercial or not-for-profit sectors. Open Access funding enabled

and organized by Projekt DEAL.

CONFLICT OF INTEREST

The authors have no affiliation with any of the rated MHAs. Eva-

Maria Messner, Yannik Terhorst, Michael Stach, Lasse Bosse Sander

and Harald Baumeister developed and run the German Mobile Health

App Database project (MHAD). MHAD is a self-funded project at Ulm

University with no commercial interests. Harald Baumeister, Eva-

Maria Messner and Lasse Bosse Sander received payments for talks

and workshops in the context of e-mental-health. Harald Baumeister

and Kai Spiegelhalder are (principle) investigators of several third-

party funded projects on e/m-health interventions, amongst others

online-based sleep interventions. All other authors declare no conflict

of interest.

12 of 14 SIMON ET AL.

DATA AVAILABILITY STATEMENT

The primary data of the study can be provided by the corresponding

author on reasonable request. Data will only be shared for scientific

purposes. Data sharing agreements may have to be signed depending

on the request. Support from the corresponding author is depending

on available resources.

ORCID

Laura Simon https://orcid.org/0000-0001-5538-2593

Lena Sophia Steubl https://orcid.org/0000-0002-8508-5271

Michael Stach https://orcid.org/0000-0001-9422-5523

Kai Spiegelhalder https://orcid.org/0000-0002-4133-3464

Lasse Bosse Sander https://orcid.org/0000-0002-4222-9837

Harald Baumeister https://orcid.org/0000-0002-2040-661X

Eva-Maria Messner https://orcid.org/0000-0001-6100-8354

Yannik Terhorst https://orcid.org/0000-0003-4091-5048

REFERENCES

Aji, M., Gordon, C., Stratton, E., Calvo, R. A., Bartlett, D., Grunstein, R., &

Glozier, N. (2020). A framework for the design engineering and clinical

implementation and evaluation of mHealth apps for sleep disturbance:

A systematic review (preprint). Journal of Medical Internet Research,

23(2), e24607. https://doi.org/10.2196/24607

Albrecht, U.-V. (2016). Kurzfassung. In U.-V. Albrecht (Ed.), Chancen und

Risiken von Gesundheits-Apps (CHARISMHA). Peter L. Reichertz Institut

für Medizinische Informatik der TU Braunschweig und der Medi-

zinischen Hochschule Hannover. https://doi.org/10.24355/dbbs.084-

201210110913-55

Andrade, L. H., Alonso, J., Mneimneh, Z., Wells, J. E., Al-Hamzawi, A.,

Borges, G., Bromet, E., Bruffaerts, R., de Girolamo, G., de Graaf, R.,

Florescu, S., Gureje, O., Hinkov, H. R., Hu, C., Huang, Y., Hwang, I.,

Jin, R., Karam, E. G., Kovess-Masfety, V., …Kessler, R. C. (2014). Bar-

riers to mental health treatment: Results from the WHO world mental

health surveys. Psychological Medicine,44(6), 1303–1317. https://doi.

org/10.1017/S0033291713001943

Baumeister, H., Kraft, R., Baumel, A., Pryss, R., & Messner, E.-M. (2019).

Persuasive e-health design for behavior change. In Digital phenotyping

and Mobile sensing (pp. 261–276). Springer.

Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019). Objective user

engagement with mental health apps: Systematic search and panel-

based usage analysis. Journal of Medical Internet Research,21(9), 1–15.

https://doi.org/10.2196/14567

Ebert, D. D., Van Daele, T., Nordgreen, T., Karekla, M., Compare, A.,

Zarbo, C., Brugnera, A., Svein, O., Trebbi, G., Jensen, K. L., Karhlke, F., &

Baumeister, H. (2018). Internet- and Mobile-BasedPsychological inter-

ventions:Applications, efficacy, and Potentialfor improving mental health.

European Psychologist,23(2), 167–168. https://doi.org/10.1027/1016-

9040/a000318

European Commission. (2012). Europeans and their languages. Special

Eurobarometer,386 June, 19. http://ec.europa.eu/public_opinion/

archives/ebs/ebs_386_en.pdf

Eyal, S., Altman, Y., & Baharav, A. (2020). 0507 Mobile cognitive behavioral

therapy is efficient in improving sleep in students. Sleep,43-

(Supplement_1), A194–A194. https://doi.org/10.1093/sleep/

zsaa056.504

Harbison, B. R., Cole, J., Wetzler, R., & Gorzynski, A. (2018). 0408 evalua-

tion of a CBT-I self-help program administered by a Mobile app. Sleep,

41(suppl_1), A155–A155. https://doi.org/10.1093/sleep/zsy061.407

Hertenstein,E.,Feige,B.,Gmeiner,T.,Kienzler,C.,Spiegelhalder,K.,

Johann, A., Jansson-Fröjmark, M., Palagini, L., Rücker, G.,

Riemann,D.,&Baglioni,C.(2019).Insomniaasapredictorof

mental disorders: A systematic review and meta-analysis. Sleep

Medicine Reviews,43,96–105. https://doi.org/10.1016/j.smrv.

2018.10.006

Huckvale, K., Nicholas, J., Torous, J., & Larsen, M. E. (2020). Smartphone

apps for the treatment of mental health conditions: Status and consid-

erations. Current Opinion in Psychology,36,65–70. https://doi.org/10.

1016/j.copsyc.2020.04.008

Hussain, M., Al-Haiqi, A., Zaidan, A. A., Zaidan, B. B., Kiah, M. L. M.,

Anuar, N. B., & Abdulnabi, M. (2015). The landscape of research on

smartphone medical apps: Coherent taxonomy, motivations, open

challenges and recommendations. Computer Methods and Programs in

Biomedicine,122(3), 393–408. https://doi.org/10.1016/j.cmpb.2015.

08.015

Koffel, E., Bramoweth, A. D., & Ulmer, C. S. (2018a). Increasing access to

and utilization of cognitive behavioral therapy for insomnia (CBT-I): A

narrative review. Journal of General Internal Medicine,33(6), 955–962.

https://doi.org/10.1007/s11606-018-4390-1

Koffel, E., Kuhn, E., Petsoulis, N., Erbes, C. R., Anders, S., Hoffman, J. E.,

Ruzek, J. I., & Polusny, M. A. (2018b). A randomized controlled pilot

study of CBT-I coach: Feasibility, acceptability, and potential impact of

a mobile phone application for patients in cognitive behavioral therapy

for insomnia. Health Informatics Journal,24(1), 3–13. https://doi.org/

10.1177/1460458216656472

Kuhn, E., Weiss, B. J., Taylor, K. L., Hoffman, J. E., Ramsey, K. M.,

Manber, R., Gehrman, P., Crowley, J. J., Ruzek, J. I., & Trockel, M.

(2016). CBT-I coach: A description and clinician perceptions of a

mobile app for cognitive behavioral therapy for insomnia. Journal of

Clinical Sleep Medicine,12(4), 597–606. https://doi.org/10.5664/jcsm.

5700

Kyle, S. D., Morgan, K., Spiegelhalder, K., & Espie, C. A. (2011). No pain, no

gain: An exploratory within-subjects mixed-methods evaluation of the

patient experience of sleep restriction therapy (SRT) for insomnia.

Sleep Medicine,12(8), 735–747. https://doi.org/10.1016/j.sleep.2011.

03.016

Larsen, M. E., Huckvale, K., Nicholas, J., Torous, J., Birrell, L., Li, E., &

Reda, B. (2019). Using science to sell apps: Evaluation of mental health

app store quality claims. Npj Digital Medicine,2(1), 18. https://doi.org/

10.1038/s41746-019-0093-1

Larsen, M. E., Nicholas, J., & Christensen, H. (2016). Quantifying app store

dynamics: Longitudinal tracking of mental health apps. JMIR mHealth

and uHealth,4(3), 1–9. https://doi.org/10.2196/mhealth.6020

Leigh, S., Ouyang, J., & Mimnagh, C. (2017). Effective? Engaging? Secure?

Applying the orcha-24 framework to evaluate apps for chronic insom-

nia disorder. Evidence-Based Mental Health,20(4), e20. https://doi.org/

10.1136/eb-2017-102751

Lorenz, N., Heim, E., Roetger, A., Birrer, E., & Maercker, A. (2019). Ran-

domized controlled trial to test the efficacy of an unguided online

intervention with automated feedback for the treatment of insomnia.

Behavioural and Cognitive Psychotherapy,47(3), 287–302. https://doi.

org/10.1017/S1352465818000486

Messner, E.-M., Terhorst, Y., Barke, A., Baumeister, H., Stoyanov, S.,

Hides, L., Kavanagh, D., Pryss, R., Sander, L., & Probst, T. (2020). The

german version of the mobile app rating scale (MARS-G): Develop-

ment and validation study. Journal of Medical Internet Research,22(3),

1–9. https://doi.org/10.2196/14479

Morin, C. M., Drake, C. L., Harvey, A. G., Krystal, A. D., Manber, R.,

Riemann, D., & Spiegelhalder, K. (2015). Insomnia disorder. Nature

Reviews Disease Primers,1,1–18. https://doi.org/10.1038/nrdp.

2015.26

Moshe, I., Terhorst, Y., Philippi, P., Domhardt, D., Cuijpers, P., Cristea, I.,

Pulkki-Råback, L., Baumeister, H., & Sander, L. B. (2021). Digital inter-

ventions for the treatment of depression: A meta-analytic review.

Psychol.

Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015). Mobile

apps for bipolar disorder: A systematic review of features and content

SIMON ET AL.13 of 14

quality. Journal of Medical Internet Research,17(8), e198. https://doi.

org/10.2196/jmir.4581

Nouri, R., Kalhori, S. R. N., Ghazisaeedi, M., Marchand, G., & Yasini, M.

(2018). Criteria for assessing the quality of mHealth apps: A systematic

review. Journal of the American Medical Informatics Association,25(8),

1089–1098. https://doi.org/10.1093/jamia/ocy050

Portenhauser, A. A., Terhorst, Y., Schultchen, D., Sander, L. B.,

Denkinger, M. D., Stach, M., Waldherr, N., Dallmeier, D.,

Baumeister, H., & Messner, E. M. (2021). Mobile apps for older adults:

Systematic search and evaluation within online stores. JMIR Aging,

4(1), 1–16. https://doi.org/10.2196/23313

Riemann, D., Baglioni, C., Bassetti, C., Bjorvatn, B., Dolenc Groselj, L.,

Ellis, J. G., Espie, C. A., Garcia-Borreguero, D., Gjerstad, M.,

Gonçalves, M., Hertenstein, E., Jansson-Fröjmark, M., Jennum, P. J.,

Leger, D., Nissen, C., Parrino, L., Paunio, T., Pevernagie, D.,

Verbraecken, J., …Spiegelhalder, K. (2017). European guideline for the

diagnosis and treatment of insomnia. Journal of Sleep Research,26(6),

675–700. https://doi.org/10.1111/jsr.12594

Schultchen, D., Terhorst, Y., Holderied, T., Stach, M., Messner, E.- M.,

Baumeister, H., & Sander, L. B. (2020). Stay present with your phone:

A systematic review and standardized rating of mindfulness apps in

European app stores. International Journal of Behavioral Medicine.,28,

552–560. https://doi.org/10.1007/s12529-020-09944-y

Sofi, F., Cesari, F., Casini, A., Macchi, C., Abbate, R., & Gensini, G. F. (2014).

Insomnia and risk of cardiovascular disease: A meta-analysis. European

Journal of Preventive Cardiology,21(1), 57–64. https://doi.org/10.

1177/2047487312460020

Soh, H. L., Ho, R. C., Ho, C. S., & Tam, W. W. (2020). Efficacy of digital cog-

nitive behavioural therapy for insomnia: A meta-analysis of random-

ised controlled trials. Sleep Medicine,75, 315–325. https://doi.org/10.

1016/j.sleep.2020.08.020

Spielman, A. J., Yang, C. M., & Glovinsky, P. B. (2011). Sleep restriction

therapy. Behavioral Treatments for Sleep Disorders, 10. https://doi.org/

10.1016/B978-0-12-381522-4.00001-8

Stach, M., Kraft, R., Probst, T., Messner, E.-M., Terhorst, Y., Baumeister, H.,

Schickler, M., Reichert, M., Sander, L. B., & Pryss, R. (2020). Mobile

health app database - a repository for quality ratings of mhealth apps.

Proceedings - IEEE symposium on computer-based medical systems,

2020-July, 427–432. https://doi.org/10.1109/CBMS49503.2020.

00087

StatCounter. (2021). Mobile operating system market share worldwide.

Mobile Operating System Market Share Worldwide.

Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O.,

Tjondronegoro, D., & Mani, M. (2015). Mobile app rating scale: A new

tool for assessing the quality of health mobile apps. JMIR mHealth and

uHealth,3(1), 1–9. https://doi.org/10.2196/mhealth.3422

Terhorst, Y., Messner, E.-M., Schultchen, D., Paganini, S., Portenhauser, A.,

Eder, A. S., Bauer, M., Papenhoff, M., Baumeister, H., & Sander, L. B.

(2021). Systematic evaluation of content and quality of English and

German pain apps in European app stores. Internet Interventions,24-

(February), 100376. https://doi.org/10.1016/j.invent.2021.100376

Terhorst, Y., Philippi, P., Sander, L. B., Schultchen, D., Paganini, S.,

Bardus, M., Santo, K., Knitza, J., Machado, G. C., Schoeppe, S.,

Bauereiß, N., Portenhauser, A., Domhardt, M., Walter, B., Krusche, M.,

Baumeister, H., & Messner, E. M. (2020). Validation of the Mobile

application rating scale (MARS). PLoS One,15(11 November), 1–14.

https://doi.org/10.1371/journal.pone.0241480

Terhorst, Y., Rathner, E.-M., Baumeister, H., & Sander, L. (2018). Help from

the app store?’: A systematic review of depression apps in German

app stores. Verhaltenstherapie,28(2), 101–112.

Uyumaz, B. E., Feijs, L., & Hu, J. (2021). A review of digital cognitive behav-

ioral therapy for insomnia (Cbt-i apps): Are they designed for engage-

ment? International Journal of Environmental Research and Public

Health,18(6), 1–16. https://doi.org/10.3390/ijerph18062929

Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., &

Ebert, D. D. (2019). Standalone smartphone apps for mental health—A

systematic review and meta-analysis. Npj Digital Medicine,2(1), 1–10.

https://doi.org/10.1038/s41746-019-0188-8

Yu, J. S., Kuhn, E., Miller, K. E., & Taylor, K. (2019). Smartphone apps for

insomnia: Examining existing apps' usability and adherence to evidence-

based principles for insomnia management. Translational Behavioral Medi-

cine,9(1), 110–119. https://doi.org/10.1093/tbm/iby014

Zachariae, R., Lyby, M. S., Ritterband, L. M., & O'Toole, M. S. (2016). Efficacy of

internet-delivered cognitive-behavioral therapy for insomnia –Asystem-

atic review and meta-analysis of randomized controlled trials. Sleep Medi-

cine Reviews,30,1–10. https://doi.org/10.1016/j.smrv.2015.10.004

SUPPORTING INFORMATION

Additional supporting information may be found in the online version

of the article at the publisher's website.

How to cite this article: Simon, L., Reimann, J., Steubl, L. S.,

Stach, M., Spiegelhalder, K., Sander, L. B., Baumeister, H.,

Messner, E.-M., & Terhorst, Y. (2022). Help for insomnia from

the app store? A standardized rating of mobile health

applications claiming to target insomnia. Journal of Sleep

Research, e13642. https://doi.org/10.1111/jsr.13642

14 of 14 SIMON ET AL.