Usability Study on Mobile Processes Enabling Remote Therapeutic Interventions
Marc Schickler1, R¨
udiger Pryss1, Winfried Schlee2, Thomas Probst3, Berthold Langguth2,
Johannes Schobel1, Manfred Reichert1
1Institute of Databases and Information Systems, Ulm University, Germany
2Clinic and Policlinic for Psychiatry and Psychotherapy, University of Regensburg, Germany
3Department for Psychotherapy and Biopsychosocial Health, Danube University Krems, Austria
1{marc.schickler, ruediger.pryss, johannes.schobel, manfred.reichert}@uni-ulm.de
Abstract—Many studies have revealed that therapeutic
homework is beneficial for the efficacy of therapies. Interest-
ingly, the latter have been less supported by IT systems so
far and, hence, therapeutic opportunities have been neglected.
For example, mobile devices can be used to notify patients
about assigned homework and help them to accomplish it in a
timely manner. In general, the use of mobile devices as well as
their sensors seem to be promising for the support of remote
therapeutic interventions. In the Albatros project, we have been
developing a framework that enables domain experts to flexibly
define the homework required in the context of a remote
therapeutic intervention. More precisely, the various tasks of a
homework can be specified as a mobile process, which is then
run on the mobile device of the respective patient. To realize
this vision, a configurator component using a model-driven
approach was developed. In particular, the Albatros configura-
tor shall relieve domain experts from complex technical issues
when defining a homework. The study presented in this paper
investigates whether domain experts are actually able to use the
configurator component. In particular, the study revealed three
insights. First, basic interventions can be easily defined with
an acceptable number of errors. Second, for defining complex
interventions (e.g., using a sensor when performing an exercise)
several issues could be identified that will contribute to improve
the Albatros configurator. Third, additional studies are needed
to evaluate the overall mental effort of domain experts when
using the configurator. Altogether, the Albatros framework may
be a reasonable alley to empower domain experts in creating
homework in the context of remote therapeutic interventions.
Keywords-Therapeutic Interventions, Mobile Processes, Mo-
bile Assistance, Mobile Therapy, Usability Study, mHealth
I. INTRODUCTION
The use of therapeutic interventions constitutes a funda-
mental pillar for increasing the efficacy of therapies in many
cases. In psychotherapy, for example, therapeutic homework
increases the therapy efficacy [1], especially when patients
quantitatively and qualitatively comply with their assigned
homework [2]. Therapeutic homework helps patients to
transfer the content learned in a therapy to their daily life.
Currently, the management of homework is less supported by
IT systems. However, the latter offer promising perspectives
that should be exploited to address existing drawbacks [3].
To monitor whether or not a patient is actually performing
a homework, for example, is very difficult without any
Figure 1. Treatment Phases
support by IT systems. As a consequence, therapists usually
evaluate the outcome of homework only retrospectively
when meeting the patient in a personal session, but not
prospectively between two sessions. However, if the period
between two sessions is rather long, the outcome of a
homework is usually discussed too late. Consider Fig. 1:
In a personal treatment session, a psychotherapist assigns
three homework tasks to a patient and they jointly specify
the contexts (e.g., in the morning) for this tasks. Without
any monitoring or feedback during treatment sessions, the
psychotherapist can only review and - if necessary - adjust
the homework in the next personal treatment session. As a
drawback, this might lead to an unnecessary extension of
the overall duration of the therapy. As the waiting times
for receiving a psychotherapy are usually long, the goal
should be to optimize therapy durations. In this context,
the use of IT systems offers promising perspectives. For
example, when using smartphones to report in real time,
between subsequent sessions, whether a homework task can
be performed successfully or is perceived as too difficult,
the overall procedure can be managed in a more efficient
way. Recent studies have confirmed that therapists crave for
an IT support of remote therapeutic interventions [4]. In
the Albatros project, we developed a framework that copes
with drawbacks in the context of therapeutic interventions
as described.
Figure 2. Treatment Sessions and Work At Home
In particular, the framework provides the following fea-
tures:
•It enables domain experts (e.g., psychotherapists, physiother-
apists, or physicians) to flexibly create and adjust therapeutic
homework themselves.
•It enables domain experts to manage the homework with their
patients through the help of smartphones.
•It supports patients to share feedback with domain expert,
again supported by smartphones.
•It supports patients in properly adhering to the assigned
homework based on reminders as well as instructions on the
smartphone.
To enable these features in a robust and flexible manner,
Albatros relies on three fundamental pillars [5]:
•Ameta-model was developed that captures the therapeutic
procedure technically.
•Mobile processes specified with this meta-model are used
to provide the homework tasks on the patients smartphones.
On one hand, mobile processes are the core concept of the
aforementioned features. On the other, mobile processes allow
for the proper support of a variety of homework.
•Aconfigurator component was developed that empowers
domain experts to define and adjust mobile processes on an
abstract level.
The overall technical framework [6] and the concepts
we developed based on the mobile processes have been
already introduced in [5]. In this work, we present results
on a study that was conducted to evaluate whether the
configurator component (i.e., the component to create mobile
processes on an abstraction level) is appropriate to empower
the study participants to create therapeutic homework tasks
themselves. The study results indicate that the concept used
for the configurator is feasible in practice. Furthermore, the
revealed results indicate that mobile processes constitute a
proper technical concept in this context. The remainder of
this paper is structured as follows: Section II discusses the
structure of therapeutic interventions and Section III presents
the developed configurator component as well as the benefits
of mobile processes. In Section IV, the design of the study
is summarized, whereas Section V discusses study results.
Finally, Section VI discusses related work and Section VII
concludes the paper with a summary and an outlook.
II. TREATMENT SESSIONS AND WORK AT HOME
Usually, a therapy starts with a personal meeting between
therapist and patient. During this meeting, the therapist
assesses a comprehensive anamnesis and establishes a ther-
apeutic alliance. Then, the therapist compiles a therapy plan
that consists of therapeutic interventions and the required
sessions to achieve a positive effect for the patient. Concern-
ing the therapeutic interventions, two intervention types are
created by therapists – the ones applied during the personal
meetings and the interventions to be accomplished by the
patient between the personal meetings (“homework”). Fig. 2
summarizes the overall homework procedure. Patients need
to process the assigned homework tasks as well as memorize
and perform them. As many patients have difficulties to
concentrate or memorize, and show a reduction of energy as
well as a decrease in activity as symptoms, the homework
procedure is error-prone. Thus, proper IT support is crucial
in this context.
III. MOBILE PROCESSES AND THE CONFIGURATOR
The mobile processes we use as the basic artefacts to
support homework cover the requirements we elicited in
practical projects with therapists [6], [5]. As these require-
ments need to be covered by the configurator, we briefly
summarize them along three categories: First, a number of
requirements related to homework need to be addressed,
e.g., therapists should be able to assign media elements to a
homework, which are then presented to the respective patient
on his mobile device. Second, requirements related to the
context of a homework need to be considered. Particularly,
context allows coping with the demands of therapists on
one hand (e.g., a homework to be performed after getting
up) and enabling researchers to gather context-sensitive data
on the other. Third, requirements related to mobile devices
need to be covered, e.g., the mobile device shall provide
Mobile Process Visual Homework Configurator
Exercise LevelSub-Process Level
Sensor Activity
Parallel Execution
Patient Activity
Figure 3. Mobile Processes and Developed Configurator
media elements to assist the patient when performing a
homework. Altogether, 14 requirements [6] were identified
that are considered by the mobile process model.
Furthermore, the configurator we developed shall allow
for defining the logic of mobile processes at a rather abstract
level in order to relieve domain experts from complex
specification tasks being out of the scope of their therapeutic
work. Consider the left hand side of Fig. 3, which shows
how therapeutic interventions can be represented in terms
of a mobile process. To be more precise, the depicted
mobile process comprises several process fragments. Each
fragment, in turn, corresponds to a specific homework whose
execution order is specified by the mobile process. In
general, a homework encompasses three process steps, i.e., a
notification step, an exercise step, and a feedback step.
Thereby, the execution order of these steps is as follows:
Notification →Exercise →F eedback. For example, a
patient is notified about an exercise to be accomplished
and then needs to provide feedback to the therapist. In
practice, each of these steps is executed on a mobile device.
Note that the concrete realization on such a device depends
on the specific homework scenario on one hand and the used
mobile platform on the other. Finally, a context is assigned to
homework in order to meet practical execution requirements.
The analysis of the practical scenarios revealed that
exercises require a fine-grained itemization. Due to the
latter, the introduction of a guiding mobile process model
was required. More specifically, therapists might want to
create exercises based on a pre-specified set of activities.
For example, an exercise may contain a warm-up activity,
an effort activity, and a cool-down activity. In addition,
therapists may want to utilize the sensors of the mobile
devices in this context. Thus, we integrated a context-
based sensing activity with exercises. Note that the practical
insights further revealed that the notification step as well as
the feedback step need to be itemized in the same way as
the exercise step. As the notification and feedback steps are
similar to the exercise step, we omit a detailed discussion
for them. Moreover, practical insights further revealed that
a more flexible concept is needed for activities to meet the
requirements of therapists. Therefore, we developed simple
as well as complex activities for defining exercises. While
simple activities solely contain one activity being performed
by a patient, complex activities may be substituted by
subprocesses. The latter, in turn, may be characterized by
complex activity structures. For example, activity effort is
substituted by a subprocess comprising activity swimming,
followed by a decision on whether swimming shall be
followed by activity biking or activity running (cf. Fig. 3). If
activity biking is chosen, biking speed can be measured by
a context-based sensing activity. Note that the context value
needed for evaluating whether to choose biking or running
is specified by the therapist after discussing this with the
patient.
Fig. 3 summarizes the levels of the mobile process model,
which meets the demands of therapeutic interventions in the
context of homework. In particular, the levels of a mobile
process shall guide the therapists in performing the required
steps in the right order. For example, therapists must not
create decisions on exercise level. In turn, decisions are only
allowed when using complex activities. For each level of
the mobile process model, in turn, the configurator provides
both simple and complex activities. Their features cover
most of the scenarios required in the context of therapeutic
interventions. For example, the configurator provides activ-
ities measuring the heart rate or filling in a questionnaire.
Note that therapists can only create therapeutic interventions
based on the available activities. In practice, however, there
exist scenarios in which the given functionality is not suf-
ficient. For example, therapists may want to use an activity
to measure skin conductance, while filling in a question-
naire in parallel. In such cases, an application developer
needs to implement the respective feature. Afterwards, the
application developer releases the implementation to the
modeling component in order to enable therapists to use
the new feature. In this context, two aspects need to be
emphasized. First, the mobile process model has turned out
to be a powerful instrument for application developers to
implement new features. In particular, the complex activities
enable them to decide in what way missing features can be
quickly and robustly realized. Second, the mobile process
model has covered all practical scenarios we had identified.
Therapeutic interventions created with the configurator will
be then transformed to executable processes running on
mobile devices. In this context, an automatic transforma-
tion procedure ensures that resulting mobile processes are
correctly executed by a mobile process engine (e.g. [7]).
Therefore, we realized a transformation feature for the
configurator. In the final step, a mobile process will be
deployed to a mobile application that we implemented. Our
realized mobile application uses a mobile process engine to
execute the mobile processes. The explanations have shown
that mobile processes constitute a powerful instrument for
capturing homework. However, the coverage of all the as-
pects discussed in this section result in a high complexity to
create the respective mobile processes. Consequently, as we
aim to empower therapists to specify and create homework
themselves, a configurator became necessary that abstracts
from complex technical details. Consider the right hand
side of Fig. 3, which depicts an exemplary screenshot taken
from the developed configurator component. It shows how
the mobile process levels are reflected by the configurator.
Although the configurator relieves therapists from many
technical decisions, still a lot of operations have to be
accomplished. Therefore, we conducted a study to answer
the question whether the configurator is usable for study
participants.
IV. STUDY DESIGN
Fig. 5 shows the overall study design we applied. As only
few comparable studies exist [8], we had numerous discus-
sions with domain experts to figure out how to measure
the applicability of the configurator. Finally, we decided to
follow the concept called Chinese whispers [9]. Accordingly,
the overall study was conducted in two phases, i.e., the
study comprised a description as well as a modeling phase
(cf. Fig. 5). In the description phase, we selected four
homework tasks with different levels of complexity. The
latter means that the mobile processes required for the four
homework tasks have different levels of modeling complex-
ity, i.e., Homework 1 shows the lowest and Homework 4
the highest complexity. Note that we discussed the mobile
processes with process experts to ensure that they fit to
the considered scenario. These four models were presented
to 28 participants who had to create textual descriptions
of the four models (i.e., each participant created all four
model descriptions). The textual descriptions constitute the
whispers handed over to 28 other participants in the second
phase; i.e., the modeling phase. In the modeling phase, then,
each of the other 28 participants had to create the four
mobile process models based on the textual descriptions.
During the modeling phase, the configurator recorded the
following three aspects for each participant and each model:
First, it records the errors the participants made regarding
the number of activities used to represent the exercises of
the homework (e.g., using three activities where only one
is required). Second, it records the errors the participants
made regarding the used decision connections of a mobile
process (e.g., using an AND-split decision instead of the
required OR-split decision). Third, it records the errors the
participants made regarding the used activity types (e.g.,
using a sensor activity instead of a non-sensor activity). Fig.
6 gives insights into the demographic data of the participants.
As the latter had to create process models, even though
on an abstract level, we asked them about their experience
with creating process models (cf. Fig. 6). In this study, we
mainly involved students from Ulm University. However,
we also involved participants from industry. Altogether, the
Figure 4. Study Design
5,2 %
Figure 5. Demographic Data of the Participants
study design was positively perceived by the involved 56
participants. This perception can be observed by answers to
the questions we asked each participant afterwards. More-
over, by using this study design, we tried to reveal the
following aspects: First, we raise the question whether or
not it is appropriate to express homework in terms of mobile
processes. This is reflected by the first phase of the study.
Second, we raise the question whether or not there are
indicators based on the made errors that may show that
the mental effort is overall on a reasonable level using
the developed configurator. Third, we raise the question,
even though complex mobile processes cannot be properly
modeled by the study participants themselves, whether the
configurator is a suitable instrument to be used by IT experts
to create these complex models in an easy and practical
manner.
V. STUDY RESULTS
The following results rely on data that was gathered
during the study discussed in Section IV. All statistical tests
were performed with R using Version 3.4.4. Mixed model
analyses of variance (ANOVAs) with random intercepts were
calculated with the amount of errors of the three error
categories as dependent variables. To be more precise, we
calculated whether the made errors in each of the three error
categories (i.e., wrong activities/exercises, wrong decision
connections, wrong activity types) change significantly from
homework modeling task 1 to 4. The results of the ANOVAs
are summarized in Table I and the means of the errors are
displayed in Fig. 7. As can be obtained from the results,
2,5
2,0
Homework Process Model
Error per Process Model
1,5
1,0
0,5
0,0
#1 #2 #3 #4
Wrong Activities / Exercises Wrong Logical connections Wrong Activity Type
0,07
0,39
2,25
1,46
0,18
1,21
1,57
0,07
0,29
Figure 6. Errors Made for the Homework Modeling Tasks
F-value (degrees of freedom) P-value
Error Categories
Wrong Activities /
Exercises 105, 51 (1.83) <.0001
Wrong Logical
Decisions 135, 67 (1.83) <.0001
Wrong Activity
types 16, 81 (1.83) .0001
Table I
STUDY RESULTS
each error category increases significantly from homework
modeling task 1 to homework modeling task 4. The increase
of errors from homework modeling task 1 to 4 was expected
by the study design using different levels of complexity. The
three questions raised in the last section can be answered
positively. Concerning Question 1, as all participants were
able to model the mobile processes based on the whispers of
the first phase, it seems that mobile processes were basically
understood. Concerning Question 2, as all participants made
only few errors for homework tasks 1 and 2, which already
constitute powerful homework definitions. Therefore, the
overall mental effort seems to be on a reasonable level.
Concerning Question 3, homework tasks 3 and 4 were
modeled by the participants with increased errors. To model
such a more complex task, a domain expert would need to
train himself to be able to create such homework himself or
he hands over this task to an IT expert. Even if the latter
must create the homework with the configurator, a complex
programming procedure can be spared. Note that we do not
involve therapists in the study. Therefore, a further study
with therapists needs to be conducted. However, our first
results are very promising regarding the successful use of
the configurator in practice.
VI. RELATED WORK
Three categories of related work are relevant in the context
of this paper. First, we need to consider approaches dealing
with mobile therapies. [10] discusses various approaches
using smartphones in the context of personal healthcare.
Interestingly, the discussed approaches have revealed that
current solutions focus only little on mobile therapies.
However, the beneficial use of interventions is emphasized
by most of them. In line with [10], [4] underlines the
benefits of mobile technology for the efficacy of psychother-
apy in more detail. In turn, none of the latter approaches
presents technical solutions for therapeutic interventions as
we developed. Opposed to that, [11] presents three clinical
studies in which mobile technology has been used for mobile
interventions. For each study, a specifically tailored mobile
application is realized. Although [11] shows the usefulness
of mobile interventions, a more generic technical solution
is omitted here. Second, approaches dealing with model-
driven concepts in the context of mobile healthcare are
relevant. Recently, approaches have been introduced that use
model-driven technical solutions in the context of mobile
healthcare. For example, in [12], the QuestionSys approach
is presented in which mobile data collection applications
can be realized by healthcare experts without the help
of IT experts or application developers. Another related
model-driven approach is presented in [13]. It comprises
a framework, which automatically creates healthcare plans
based on a model-driven concept. The healthcare plans, in
turn, are then applied to the smartphones. Third, approaches
dealing with homework in the context of psychotherapy
need to be discussed. Numerous related works deal with
homework in the context of psychotherapy [14], [1], [15] .
However, they mainly address the results of the conducted
studies with no particular focus on technical solutions. The
potential of technical solutions in the context of therapeutic
homework has recently been highlighted [3]. In current
approaches, mobile technology is used to support exposure
exercises for patients with anxiety disorders [16], [17]. The
development of a generic technical solution that supports
homework in particular and therapeutic interventions in
general, in turn, is currently less considered. Moreover, study
results on configurators that enable domain experts to create
sophisticated homework do not exist so far.
VII. SUMMARY AND OUTLOOK
This paper investigated the mobile process configura-
tor component of the Albatros framework with respect
to its applicability. The configurator, in turn, shall enable
therapists to create remote therapeutic interventions based
on homework running on mobile devices themselves. To
address the applicability, an study with 56 participants was
conducted. For the study, the participants were separated into
two groups, based on a study design following a Chinese
whispers principle [9]. The study results revealed that the
developed configurator component is a proper instrument to
create homework based on mobile processes. However, as
this study did not involve therapists, more studies need to be
conducted in future work. These studies should also include
qualitative interviews to evaluate the acceptability of Alba-
tros by therapists and patients. Altogether, the two general
ideas pursued in Albatros are (1) to empower therapists in
creating sophisticated remote therapeutic interventions based
on homework and (2) to support patients in successfully
performing therapeutic homework tasks. Moreover, further
work on Albatros will address the 6 essential features that
have recently been identified to be important for mobile
apps to maximize homework compliance [3]: (1) congruency
to therapy, (2) fostering learning, (3) guiding therapy, (4)
building connections, (5) emphasizing completion, and (6)
population specificity.
REFERENCES
[1] N. Kazantzis et al., “Meta-analysis of homework effects in
cognitive and behavioral therapy: a replication and extension,”
Clinical Psychology: Science and Practice, vol. 17, no. 2, pp.
144–156, 2010.
[2] ——, “Quantity and quality of homework compliance: a
meta-analysis of relations with outcome in cognitive behavior
therapy,” Behavior Therapy, vol. 47, no. 5, pp. 755–772, 2016.
[3] W. Tang and D. Kreindler, “Supporting homework compli-
ance in cognitive behavioural therapy: essential features of
mobile apps,” JMIR mental health, vol. 4, no. 2, 2017.
[4] O. Lindhiem et al., “Mobile technology boosts the effective-
ness of psychotherapy and behavioral interventions: A meta-
analysis,” Behavior modification, vol. 39, no. 6, pp. 785–804,
2015.
[5] M. Schickler et al., “Supporting Remote Therapeutic Inter-
ventions with Mobile Processes,” in 6th IEEE Int’l Conf on
AI & Mobile Services. IEEE Computer Society Press, 2017.
[6] ——, “An IT Platform Enabling Remote Therapeutic Inter-
ventions,” in 30th IEEE Int’l Symposium on Computer-Based
Medical Systems. IEEE Computer Society Press, 2017.
[7] J. Schobel et al., “A Mobile Service Engine Enabling Com-
plex Data Collection Applications,” in 14th Int’l Conf on
Service Oriented Computing, ser. LNCS, 2016, pp. 626–633.
[8] ——, “Development of mobile data collection applications
by domain experts: Experimental results from a usability
study,” in 29th Int’l Conf on Advanced Information Systems
Engineering, ser. LNCS. Springer, 2017, pp. 60–75.
[9] R. Wardy, “Chinese whispers,” Proceedings of the Cambridge
Philological Society, vol. 38, p. 149170, 1993.
[10] K. Wac, “Smartphone as a personal, pervasive health infor-
matics services platform: literature review,” arXiv preprint
arXiv:1310.7965, 2013.
[11] C. Depp et al., “Mobile interventions for severe mental
illness: design and preliminary data from three approaches,”
The Journal of nervous and mental disease, vol. 198, no. 10,
p. 715, 2010.
[12] J. Schobel et al., “End-User Programming of Mobile Services:
Empowering Domain Experts to Implement Mobile Data
Collection Applications,” in 5th IEEE Int’l Conf on Mobile
Services, 2016, pp. 1–8.
[13] A. Khambati et al., “Model-driven development of mobile
personal health care applications,” in Proc of the 23rd
IEEE/ACM Int’l Conf on Automated Software Engineering.
IEEE Computer Society, 2008, pp. 467–470.
[14] D. Cox et al., “Increasing adherence to behavioral homework
assignments,” Journal of behavioral medicine, vol. 11, no. 5,
pp. 519–522, 1988.
[15] F. Leichsenring et al., “The empirical status of psychody-
namic psychotherapy-an update: Bambi’s alive and kicking,”
Psychotherapy and psychosomatics, vol. 84, no. 3, pp. 129–
148, 2015.
[16] J. Boettcher et al., “Adding a smartphone app to internet-
based self-help for social anxiety: a randomized controlled
trial,” Computers in Human Behavior, 2018.
[17] T. Neben, A.-M. Seeger, T. Kramer, and A. White, “Regaining
joy of life: Theory-driven development of mobile psychother-
apy support systems,” 12 2015.
