Progress Determination of a BPM Tool with
Ad-Hoc Changes: An Empirical Study
Lisa Arnold, Marius Breitmayer and Manfred Reichert
Institute of Databases and Information Systems, Ulm University, Germany
{lisa.arnold, marius.breitmayer, manfred.reichert}@uni-ulm.de
Abstract. One aspect of monitoring business processes in real-time is
to determine their current progress. For any real-time progress determi-
nation it is of utmost importance to accurately predict the remaining
share still to be executed in relation to the total process. At run-time,
however, this constitutes a particular challenge, as unexpected ad-hoc
changes of the ongoing business processes may occur at any time. To
properly consider such changes in the context of progress determination,
different progress variants may be suitable. In this paper, an empirical
study with 194 participants is presented that investigates user accep-
tance of different progress variants in various scenarios. The study aims
to identify which progress variant, each visualised by a progress bar, is
accepted best by users in case of dynamic process changes, which usu-
ally effect the current progress of the respective progress instance. The
results of this study allow for an implementation of the most suitable
variant in business process monitoring systems. In addition, the study
provides deeper insights into the general acceptance of different progress
measurements. As a key observation for most scenarios, the majority
of the participants give similar answers, e.g., progress jumps within a
progress bar are rejected by most participants. Consequently, it can be
assumed that a general understanding of progress exists. This underlines
the importance of comprehending the users’ intuitive understanding of
progress to implement the latter in the most suitable fashion.
Keywords: Business process monitoring, empirical study, progress de-
termination, progress visualisation, real-time monitoring
1 Introduction
The quest to monitor business processes is very much in demand. Monitoring
processes allows us to identify and address problems and errors at an early
stage of process execution. A crucial task of business process monitoring is to
determine the progress of running process. This is particularly challenging when
facing ad-hoc changes during run-time, which might affect progress.
The unexpected addition or deletion of process instances as well as direct
changes of the underlying process model itself are examples of such ad-hoc
changes, which have a direct impact on progress determination. As a result of
this process flexibility, further investigation of how to determine and visualise
the progress of a running business process is needed, and to understand which
variants exist to cope with unexpected progress.
In this paper, we investigate how to cope with dynamic progress changes of
flexible processes during run-time. In particular, we want to know whether there
exists a generally accepted notation of progress and which type of visualising
progress are accepted best by users. For this purpose, an empirical study to
investigate the acceptance of different progress variants for business processes
is conducted. Its aim is to determine which type of progress is accepted by
the users of a business process monitoring system when facing process changes
during run-time. The results of the study shall allow us to implement the most
suitable variants of progress changes.
The remainder of this paper is structured as follows. Section 2 gives back-
grounds on the process paradigm presumed by this paper, of object-aware busi-
ness processes. In Section 3, the applied research methodology, and the design
of the empirical study are described. Section 4 analyses, evaluates, and discusses
the answers of the 194 participants of the study. Related work is discusses in
Section 5, whereas Section 6 concludes the paper.
2 Backgrounds
The framework PHILharmonicFlows, which we have developed in recent years,
enables implementing object-aware business processes, which allow for a high
user flexibility as well allows ad-hoc process changes during run-time [1]. In
object-aware processes the collection of process data is accomplished with data-
driven form sheets. The latter are auto-generated from the process specification.
Forms usually have to be designed manually for each activity while form genera-
tion is a built-in feature of PHILharmonicFlows due to its data-centric approach.
Thereby, forms logic and enactment utilises the operational semantics of the dy-
namic aspects of the form, such as the next value that is required. The latter
also allows for a high degree of flexibility, for example, a user is not forced to fill
out the fields of a form in-order. User may choose their preferred order, e.g., by
filling out the form from bottom to the top. Additionally, jumping back within
a form allows users to check or correct previously filled fields [2].
Basic to our approach are business objects. Each object is defined by its at-
tributes as well as its lifecycle, which describes object behaviour (i.e., the process-
ing of object-related forms) during run-time [3]. Moreover, the relations between
the various objects of a business process are manifested by a relational process
structure. Possible cardinality constraints on these object relations are 1:n or n:m
[4]. An object lifecycle is defined in terms of states. Each lifecycle has one start
state and at least one end state, and an arbitrary number of intermediate states.
Each state is represented by a form sheet and maybe refined by steps, which
correspond to object attributes and hence represent the data input fields of the
respective form. Finally, a coordination process controls the interactions between
these multiple lifecycle processes making use the relation process structure [5].
In [6], we have defined four research questions to be investigated in the con-
text of determining the progress of an object-aware business process. Moreover,
in [7], we presented an approach that addresses Research Question 1 and 2. To
answer Research Question 3, the challenges of determining the progress of large,
dynamically evolving process structures, which consist of interacting loosely cou-
pled, smaller processes that may be also subject to ad-hoc process changes must
be investigated first. Regarding these challenges, a solution for progress deter-
mination with empirical evidence is provided.
Research Question 1 How can the progress of a single lifecycle process with
its state-based view form be determined?
Research Question 2 How can the progress of the processing of a single
state within a lifecycle process be measured?
Research Question 3 How can the progress of multiple, interacting (i.e.,
interrelated) lifecycles be determined?
Research Question 4 How does a coordination process affect the progress
of an object-aware business process?
3 Research Methodology and Study Design
This section presents our research methods and the design of the empirical study.
We combine the methodologies from [8], [9], and [10] to investigate a real-world
scenario for a process monitoring tool. Section 3.1 introduces the research ques-
tions addressed by this paper. Section 3.2 defines the data collection method.
Section 3.3 presents the study design and Section 3.4 describes the used data
analysis method.
3.1 Research Questions
For each business entity, a separate instance (with its own lifecycle instance) of
an object type is created at run-time. In addition, changes of a lifecycle instance
may be defined at run-time by adding new attributes, which are considered in
the dynamically generated form sheets, i.e, new form sheets or input fields may
be created at run-time. Moreover, form sheets and input fields may be deleted
at run-time. In the context of such dynamic lifecycle and form changes as well
needs to investigated how the progress of the overall process be adapted and
what adaptations are accepted by different users or user groups. This leads us
to the following research question:
Main Research Question: Does a generally suitable accepted understand-
ing of progress exist?
This includes the following sub-research questions:
Sub-Research Question 1 Which variants of determining progress are re-
jected by most users?
Sub-Research Question 2 Are progress jumps or progress speed adjust-
ments better accepted by users in the context
of run-time changes?
Sub-Research Question 3 Which progress variant is most suitable?
3.2 Data Collection Method
The study is performed with an anonymous online questionnaire1leveraging the
web-based tool Google Forms for data collection. Further, the study is avail-
able in German and English. The language options do not differ with respect
to content or structure. The participants can choose their preferred language
in the first step of the questionnaire. The questionnaire was available for the
participants over a period of more than eight months.
3.3 Study Design
Demographic questions: In the first section of the online questionnaire, de-
mographic data of the participants are collected. The latter shall provide us with
information about the participants. In addition, these data allow us to compare
the results of the remaining study for different groups. In detail, the demo-
graphic questions refer to the participants’ gender, age group, highest school or
university degree, professional field, experience with reading or creation process
models, experiences with process monitoring, and satisfaction with existing pro-
cess monitoring tools (on a 5 point Likert-scale, from 1 not at all satisfied to 3
(neutral) to 5 very satisfied), if the participant has worked with any monitoring
tools before.
Intuitive view of progress: Following the demographic questions, each
participant is presented an introductory video of a linear, continuous progress
bar course of a lifecycle process i.e., sequentially passing the states of the lifecycle
process without any deviations. However, such a progress course is generally
not feasible due to unpredictable changes during lifecycle process execution.
Therefore, alternatives need to be developed that reflect process changes in the
progress bar of this process best. For this purpose, to each participant additional
eight videos of progress bar courses are presented, which should be rated in terms
of whether progress determination is properly presented to users and intuitively
perceived by them. For these ratings, a 5 point Likert-scale ranging from 1 (not
at all true) to 3 (neutral) to 5 (completely true) is used. The aim of this section
of the questionnaire is to investigate whether there are progress variants which
are rejected by users and therefore should be not used in the context of process
monitoring.
1The questionnaire and the responses of the 194 participants are available via
the following Researchgate link: https://www.researchgate.net/publication/
358140443_Study_Results
Scenario view of progress: In this section of the study, five intralogistic
scenarios are described. Along these scenarios different challenges of determin-
ing the progress of a running business process are covered, e.g., when creating or
deleting business object instances. The results shall help us to find the most suit-
able progress measurement for a business process monitoring system. Addition-
ally, we want to evaluate our hypothesis that a generally accepted understanding
of progress exists.
The following introduction is given before presenting the different intralogis-
tic scenarios to the users: Robots can support the intralogistic processes in an
industrial environment, receiving goods. Robots can support human interactions
when unloading, recognising and aligning packages. During the enactment of the
process, its progress is displayed in an optimised way. Unexpected (i.e., dynamic)
changes may occur during process execution e.g., the technical failure of a robot
or the commissioning of a replacement robot. When a robot fails, the remaining
robots must compensate this failure and perform additional work. In turn, this
might lead to a delay of the total process. The situation is similar to the ad-hoc
emergence of additional work.
For each scenario, the participants are confronted with two videos describing
different options for courses of the progress. In all five scenarios, Option Ade-
scribes the progress with no jumps by speeding up or slowing down the progress
depending on the respective ad-hoc changes. In turn, Option Benables progress
with the same speed. If progress adjustments become necessary corresponding
backward or forward jumps are made in the progress bar. In the following, the
five scenarios are described.
Creation of business object instances. During run-time an unexpected work-
load may occur. For example, the unexpected absence of staff (illness) or robots
requires the redistribution of the workload among fewer staff or robots respec-
tively. In this case, progress adjustments can be made by either jumping back in
the progress bar or slowing down the remaining progress. Scenario 1 shall help us
to find the most suitable solution when creating instances of a business object.
Scenario 1: Reordering packages
Before the start of each working day, the total amount of work is deter-
mined. Due to unexpected hoarding of toilet paper, as much toilet paper
as possible is reordered and delivered during the working day. The ordered
toilet paper is delivered by an unknown number of trucks throughout the
afternoon. The current progress of the goods’ delivery is determined each
time a truck arrives by considering the additional work.
In Figure 1, the progress determination according to Option Aand Bare
shown. Both progress determinations are identical until 60%. Then the progress
according to Option Agrows slower until 80%, due to the unexpected delivery
of toilet paper from the trucks. For Option B, with this delivery the progress
determination drops immediately to 50% and is growing up to 80% with the same
speed as before. When the progress reaches 80%, another unexpected truck load
arrives in the intralogistic centre. Again, for Option Athe progress grows slower
until 100%, whereas for Option Bthe progress drops immediately to 60% and
is growing up to 100% with the same speed as before.
Fig. 1. Progress adjustments in Option A
and Bafter the unexpected delivery of ad-
ditional parcels (cf. Scenario 1).
Fig. 2. Progress adjustments in Option A
and Bafter the cancellation of parcels (cf.
Scenario 2).
Deletion of business object instances. Besides the unexpected increase of the
total workload, an unexpected decrease of the latter may occur as well, e.g., after
cancelling orders or work assignments. Scenario 2 shall help us to find the most
suitable solution for adjusting progress measures in such cases.
Scenario 2: Order cancellation
Due to a more favourable price development of a competitor’s product,
unexpected cancellations are made throughout the working day. Conse-
quently, the overall workload is reduced.
In Figure 2, for Option Aand Bthe respective progress with respect to
Scenario 2 is depicted. For both options the progress determination evolves the
same way until reaching 50%. Then, for Ait grows significantly faster up to
80%, due to a first unexpected order cancellation. For Option B, with the order
cancellation the progress jumps immediately from 50% to 70%, and is then
growing up to 80% with the same speed as before. When reaching 80%, another
unexpected cancellation occurs. For Option Athe progress determination is then
growing even faster up to 100%, whereas for Option Bwith the cancellation the
progress immediately jumps from 80% to 90%, and is growing up to 100% with
the same speed as before.
Replacement of business object instances. During process execution, multiple
unexpected events might occur, which affect the overall progress determination.
Scenario 3: Robot failure and replacement by another robot
Assume that a robot fails when reaching 60% of the calculated progress
due to technical reasons. Adding a replacement robot takes some time as
the failed robot has to be removed and the new one needs some time to
be ready for use. Assume further that is not possible to predict whether
the replacement robot will be ready for use before completing the entire
process.
In Figure 3, for both options the respective progress determination is shown.
The latter are the same until reaching 60%. Then, for Option Athe progress
is growing slower due to of the unexpected robot failure, whereas for Option B
the progress drops immediately to 45%. Afterwards, for Option Bthe progress
is growing from 45% to 70%, whereas for Option Athe progress is growing from
60% to 70% in the same time. When activating the backup robot, the progress
determination readopted adapts again. For Option A, the progress is growing
significantly faster, whereas for Option Bit jumps from 70% to 85% with the
same speed as before.
Fig. 3. Progress adjustments in Option A
and Bafter the unexpected robot failure
and provision of a backup robot (cf. Sce-
nario 3).
Fig. 4. Progress adjustments in Option A
and Bwhen jumping back to a previous
form (cf. Scenario 4).
Progress jumps within a form. As shown in Section 2, in the PHILharmon-
icFlows approach the end-users interact with forms to collect data. After filling
out the current form, it is possible to activate the next one. However, it is also
possible to jump back to a previous form to review and update the input added
previously with the respective form. The input data of all forms is preserved.
Scenario 4: Jumping back to a previous form
This scenario switches to the customer’s perspective. During an order
process, multiple forms are displayed to the customer one following the
other. These forms and their order are as follows: Shopping cart →Enter
address →Payment method →Confirm →Done (end of order process).
Assume that before confirming the customer jumps back to the Shopping
cart and re-checks it. In this context note that the input data of the
previously filled forms Enter address and Payment method persists.
In Figure 4, for both options the progress determination is shown. Both
progresses evolve the same way until 90%. For Option Athe progress is growing
slower due to the backward jump in the form, whereas for Option Bthe progress
immediately drops to 10%. After the backward jump, for Option Bthe progress
is growing from 10% to 100% in the same time as it is growing from 90% to
100% in Option A.
Time- or work-based. In general, there are two metrics that can be used
to determine the progress of a business process. First, the progress calculation
can be based on the total execution time. Assume, for example, that a business
process needs from its start five days upon completion. Accordingly, on the
second day at 6am the progress reaches 25% (presuming a duration of 5 days with
24 hours). Alternatively, progress calculation can be based on the completion of
work packages, which results in progress of 20% on the second day at 6am as
only the work of the first day has been completed so far (assuming working hours
from 8am to 5 pm).
Scenario 5: Time- or work-based
If there are planned breaks during process execution in which no active
work is done, this affects the progress determination of the process. As-
suming that the following sequence of work packages and breaks, together
with the time required for them, is given.
Work package A: 3 hours + 15 minutes break
Work package B: 1 hour + 15 minutes break
Work package C: 4 hours + 30 minutes break
Work package D: 2 hours
Figure 5 depicts the determination for progress time- and worked-based sce-
nario. For Option A, there is no progress growth in the three breaks, whereas
for Option Bthe progress is growing continuously from 0% to 100%.
Outlook Section of the Questionnaire: The last section of the question-
naire concludes the study by asking general questions about progress determi-
nation and presentation. The aim is to get insights into the accepted duration
for progress determination and further fundamental monitoring components.
Ideally, progress determination is accomplished in real-time. However, the
calculations take time, and the more comprehensive the process is, the greater
Fig. 5. Progress adjustments in Option Abased on work and Bbased on time (cf.
Scenario 5).
the delay due to the continuous progress calculation will be. For this reason, we
investigated what delay in progress determination is acceptable for the users of a
process monitoring tool. For this investigation, we defined four process duration:
one hour,one day,one week, and one month. For each process duration, the
participants gave the maximal acceptable delay of progress determination.
Besides the progress determination, many more monitoring components exist,
e.g., Risk Management,Resource Management,Process Performance Measure-
ment,Alarm Trigger and Error Warnings, and Numbers of running processes.
Therefore, the participants were asked which components shall be part of a pro-
cess monitoring tool and which of them is the most important.
3.4 Data Analysis Method
All collected data are analysed and evaluated in a structured way to answer
the research question defined in Section 3.1. For this purpose, the methodology
from [8] is used. In a first step, the answers of all participants are analysed. In
a second step, an expert group among the participants is defined based on the
answers given in the demographic section of the questionnaire, e.g., experience
in modelling processes. The answers of these experts are then compared with the
ones of the first step. Pearson and Spearman Correlation are used to evaluate
the correlation between the data of the questionnaire.
4 Evaluation
Demographic questions: Overall, more male (110 | 56.7%) than female (84 |
43.3%) participants took part in the study. The majority of the participants are
between 18 and 25 years old (152 | 78.4%); 36 participants are between 26 and 35
years old (36 | 18.6%). The number of participants with an age between 36 and 65
(6 | 3.0%) is negligible. Most participants work in economy and administration
(89 | 45.9%) or information technology (76 | 39.2%). More than two third (133
| 68.6%) of the participants have experiences with process models or process
modelling. Much fewer participants (28 | 14.4%) use process monitoring tools
in the context of their professional activities. Participants with both modelling
experience and professional familiarity with monitoring tools (24 | 12.8%) are
considered as the experts in our study. In the following, the study results of the
experts (E) are compared with the ones of the non-experts (NE) to enable a
profound analysis. The non-experts are composed of all participants expect the
experts. Finally, the satisfaction score of 3.3/5 rated by the experts is slightly
above a neutral rating, which indicates that existing monitoring tools are not
perceived as satisfaction by users.
Intuitive view of progress: This section of the study deals with the intu-
itive view users have on the progress determination of a process. For this purpose,
they are confronted with eight different progress variants, which they shall eval-
uate with respect to suitability and usability. Table 4 describes the considered
progress determinations of Videos 1-8 as well as the average rating received from
experts and non-experts, respectively. For Videos 1, 3, and 6 the scores of the
non-experts and experts are almost the same (≤0.10). For Videos 2, 5, 7, and
8, the scores are slightly different (≤0.38). The biggest difference (0.50) occurs
for Video 4.
In a nutshell, adjustments (cf. Videos 2 and 7) are perceived as more suitable
and usable compared to progress jumps (cf. Video 1 and 3). Concerning the
latter, forward progress jumps (cf. Video 1) are considered being more intuitive
compared to backward progress jumps (cf. Video 3). The non-experts and experts
rate slow and fast progress speed adjustments differently. Non-experts prefer
progress that, from a certain point during process execution, is growing faster (cf.
Video 2) as more intuitive than progress growing slower (cf. Video 7). The experts
evaluate this the other way round. Note that the same observations can be
made for the permanently increased (Video 6) and decreased (Video 4) progress.
In practice, an unexpected more workload is more common, e.g., due to staff
absence (illness) or defect tools. This might be one reason why experts perceive
this as more intuitive, as they are used to such scenarios. Unexpectedly, the 90%
problem is rated slightly above neutral (cf. Video 8). With the experts rating it
even better than the non-experts. This might be related to the aforementioned
habitual practice. Finally, the progress variant considered being the least usable
by both groups is illustrated by Video 5, where the progress finished at 80%.
Consequently, progress representations should utilise the full progress spectrum
between 0% and 100%.
Scenario view of progress: Due to the additional information provided
(cf. Section 3.3), participants get a better understanding of when and for what
reason process changes become necessary. In the following analysis, first of all,
Scenarios 1 to 3 are considered together, followed by an individual analysis of
Scenarios 4 and 5.
Scenarios 1 to 3. In a nutshell, in the context of Scenario 1 (creation of
instances), Scenario 2 (deletion of instances), and Scenario 3 (replacement of
instances) we investigated whether progress speed adjustments (Option A) or
Table 4. Average evaluation of intuitive progress view for each video (#) by non-
experts (NE) and experts (E).
# Description ØN E ØE
1 Recalculation of progress at 60% and forward jump in progress. 3.02 3.04
2 Adjustment of progress speed at 60% and faster growing progress. 4.05 3.79
3 Recalculation of progress at 60% and backward jump in progress. 2.14 2.21
4 Progress becoming slower and slower over the entire process duration. 3.21 3.71
5 Progress growing equally, but finishing at 80%. 1.92 2.30
6 Progress becomes faster and faster over the entire duration. 3.52 3.42
7 Adjustment of progress speed at 60% and slower growing progress. 3.53 3.88
8 Progress growing uniformly up to 90% and then becoming slower and 3.13 3.33
slower.
progress jumps (Option B) are preferred by users. In all three scenarios progress
speed adjustments without jumps are preferred more than progress jumps:
Scenario 1: overall (140 | 72.2%), experts (17 | 70.8%), non-experts (123 | 72.4%);
Scenario 2: overall (124 | 63.9%), experts (14 | 58.3%), non-experts (110 | 64.7%);
Scenario 3: overall (158 | 81.4%), experts (20 | 83.3%), non-experts (138 | 81.2%).
Moreover, for all three scenarios the answers are analysed in detail. Table 5 gives
an overview of the eight possible answer sequences for Scenarios 1 to 3. The
sequences constitute all possible combinations of preferences regarding progress
jumps and are grouped by their number: none, 1, 2, and 3. Half of the experts
(12 | 50.0%) and almost half of the non-experts (78 | 45.9%) vote for Option
Ain all three scenarios, i.e., they prefer progress speed adjustments (without
jumps). Furthermore, one third of the non-experts (57 | 33.5%) preferred Option
B(i.e., progress jumps) in one of the three scenarios as opposed to only 4 experts
(16.7%). Additionally, only 23 non-experts (13.5%) and 7 experts (29.1%) choose
Option Bin two of the three scenarios. Finally, 12 non-experts (7.1%) and
1 expert (4.2%) prefer progress jumps instead of progress speed adjustments
for all scenarios. Figure 6 depicts the distribution of the preferred options for
both non-experts and experts. In summary, progress speed adjustments (without
jumps) are significantly more preferred than progress jumps even thought there
are some opposing opinions.
Moreover, Scenario 3 reflects the real world most accurately. Usually, more
than one unexpected event happens during run-time. In this case, Option A(no
progress jumps) is preferred by 138 (81.8%) non-experts. The experts, in turn,
show similar results (20 | 83.3%). The latter can be further divided into two
subgroups: computer scientists who are developing process monitoring tools and
professionals from economy, administration, transport, or logistics who use these
tools in their daily business. Of the latter, 100% agreed that avoiding progress
jumps is the better option for Scenario 3. This confirms the previous result.
Table 5. Percentage of possible answer
sequences for Scenario 1-3 (S1-S3) are
given by non-experts and experts.
S 1 S 2 S 3 %N E %E
A A A 45.9 50
A A B 5.3 0
A B A 17.0 16.7
B A A 11.2 0
A B B 4.1 4.2
B A B 2.3 8.3
B B A 7.1 16.6
B B B 7.1 4.2
Non-Expert Expert
Fig. 6. Associated pie diagram for percent-
age distribution of Table 5.
Scenario 4. Previously entered data are still available when jumping back to
an already completed form. For Option A, progress is determined on the basis of
the already entered data. Therefore, a backward jump to a previous form sheet
does not affect the progress of the process. For Option B, however, the progress
is correlated with the currently edited form. Consequently, progress drops when
jumping back to an already completed form. The non-experts (102 | 60.0%) and
experts (14 | 58.3%) rate this scenario similarly and prefer the first progress
determination without dropping progress.
Scenario 5. Progress is determined based on the calculated time (Option A)
or work-based (Option B). Non-experts (101 | 59.4%) and experts (13 | 54.2%)
preferred the work-based approach for this scenario. Especially the results of the
experts, however, have revealed no clear preference. This scenario will therefore
be re-considered in a more detailed study in future.
Outlook Section of the Questionnaire: This part of the study helps us
to define the accepted delay due to the progress determination of process mon-
itoring tool and gives an overview of other possible monitoring components. In
the following, the maximum accepted delay due to progress determination is
investigated. In the first step, outlier detection is conducted. When a partici-
pant specifies a maximum accepted delay of the progress determination, which is
larger than total the process duration, all for possible answers of this participant
related to delay of the monitoring tool (i.e., hour, day week and month accepted
delays) are omitted. Additionally, if the accepted delay for a process lasting an
hour is bigger than for a process lasting a day (the same applies to: hour >day >
week >month) from one participant, it can be assumed that this participant has
misunderstood the given format for this question. For this reason, answers from
such participants are omitted as well. This results in a total of 186 participants
(164 non-experts and 22 experts) for the following analysis.
In the second step, a Pearson Correlation is performed, due to the ex-
pected linear dependency. Table 6 shows the results of this analysis. Note that
the result correlation may be between −1and +1. The larger the absolute value
of the coefficient is, the stronger the relationships between the variables are.
However, in all relations there is only a moderate correlation (0.40 −0.69)
or strong correlation (0.70 −0.89) and not, as expected, a very strong
correlation (0.90 −1.00) [10].
Table 6. Analysis of Pearson and Spearman correlation with N= 186.Moderate,
Strong, and Very strong.
Pearson Spearman
Relation NE E NE E
Hour & Day 0.748 S0.637 M0.816 S0.835 S
Hour & Week 0.631 M0.621 M0.708 S0.644 M
Hour & Month 0.646 M0.499 M0.687 M0.520 M
Day & Week 0.807 S0.622 M0.859 S0.846 S
Day & Month 0.729 S0.429 M0.835 S0.729 S
Week & Month 0.809 S0.849 S0.891 S0.890 S
Using the results of the Pearson Correlation, the following hypothesis can
be established and investigated in a third step:The longer the total duration
of a process is, the smaller the percentage share of accepted delay from the total
process duration is. Having a closer look at the average accepted delay due to
progress determination we perceive the hypothesis backed up with 9.1% for the
one-hour process, about 5% for both a one-day-process and a one-week-process,
and 0.6% for a one-month-process. Furthermore, for some participants it can be
observed that the longer the process duration is, the more similar the accepted
delays become. These results indicate the existence of an absolute upper limit for
the accepted delay caused by the determination of the progress. This assumption
is further supported by the different delays accepted by the experts and non-
experts. For this reason, an additional Spearman Correlation is performed (cf.
Table 6), which is suitable for monotonic functions. As opposed to the Pearson
Correlation, the Spearman Correlation results in significantly more strong
correlations as expected.
In the fourth step, the maximum accepted delay caused by progress deter-
mination is visualised with the boxplot diagrams as shown in Figure 7. For each
of the four given total process duration’s (hour, day, week, and month) a box-
plot, which represents the accepted delay by the quartiles and the average of the
experts and non-experts are depicted. All values are given in seconds (for better
visibility, the scale of time (y-axis) is individual for each process duration). In
summary, the smaller quartiles of the experts indicate that they have a very sim-
ilar understanding of progress delays. In contrast, the results of the non-experts
show significantly more outlier points and a larger span of the boxplot. Except
(a.) One hour
1800
1200
600
60
0
323
600
300
20
352
(b.) One day
43200
28800
21600
18000
14400
7200
900
1
4501
18000
8400
7200
4050
1800
60
3739
(c.) One week
302400
172800
144000
118800
100800
86400
43200
5400
2
35831
86400
43200
6300
600
26209
(d.) One month
362439
172800
28800
4
117561
345600
172800
34200
600
110564
Non-Experts Experts
Fig. 7. Boxplot diagram showing the accepted delay caused by progress calculation for
the different process duration’s in seconds for non-experts and experts.
for the one-hour process, the average of the accepted delays are smaller for the
experts in comparison to the non-experts.
The last question investigates which monitoring component should be covered
by a process monitoring tool (Question A) and which should be implemented
next (Question B). No distinction is made between experts and non-experts. For
Question A, five possible additional components are given and multiple answers
are allowed. Figure 8 (a) shows the answers to this question. Note that all five
options are equally distributed (about 20%). To conclude all components seem to
be relevant. Moreover, participants may name further components of a process
monitoring tool. Proposed components include Completed sub-processes,Early
warning through pattern recognition based on machine learning, and Heat and
failed tasks.
For Question B, the most important monitoring component besides progress
determination should be chosen. The distribution is shown in Figure 8 (b). This
indicates, that Alarm Trigger and Error Warnings is preferred by the majority
(38,7%) of the participants.
5 Related Work
To the best of our knowledge there exist no works dealing with progress deter-
mination in process monitoring tools. In participant, this applies to object-aware
BPM, as there exists no comparable approach that offers the same flexibility
as PHILharmonicFlows does (e.g. ad-hoc changes of running process instances)
[11]. Consequently, no monitoring rules for processes being subject to ad-hoc
changes in object-aware BPM approaches exist.
(a.) Possible components
of monitoring
14.3%
19.2%
22.0%
22.5% 22.0%
(b.) Next important
component of monitoring
11.8%
13.9%
21.7%
38.7% 13.9%
Resource Management
Risk Management
Number of running processes (robot, work packages, workers, ...)
Alarm Trigger and Error Warnings
Process Performance Measurement
Fig. 8. Evaluation of which components of a monitoring shall be part of a process
monitoring tool (a) and realised next (b).
Concerning data-centric BPM, only few approaches offer this flexibility dur-
ing process execution. In artifact-centric BPM, for example, process executions
are driven by business data [12]. Moreover, a tool for designing and modelling
artifact-centric processes exists. There is no intuitive tool support for executing
and monitoring corresponding instances in the large scale (as in the case of our
PHILharmonicFlows engine) [13].
A second approach for data-centric BPM is case handling, which is based on
the idea what can be done to achieve a business goal [14]. With Flowers a tool
exists, which supports the design, implementation, and execution of data-centric
processes, but does not provide comprehensive monitoring support [13]. Note
that other data-centric approaches to BPM are available. However, all of them
are limited compared to the artifact-centric BPM and case handling paradigms
[13]. In summary, no related works of dynamic process changes exist.
Furthermore, there exists research emphasising the importance of progress
indicators. The study presented in [15] shows that progress representation can
influence perception of a user concerning the duration of a task. This progress
representation influences the decision whether a task (e.g., in form of a online
questionnaire) is abandoned or continued. Thereby, progress representation is
displayed with different progress speed throughout the total questionnaire. As a
result, this study shows that the slower the progress is in an early stage of the
questionnaire the higher abandonment rates are [15]. Furthermore, the experi-
ment presented in [16] shows that progress representations are important and
useful for user-interaction tools. Additionally, [16] underlines that users prefer
tools with progress visualisation.
6 Summary and Outlook
All the defined research questions could be addressed. Progress determination
and visualisation should always end up with 100% progress upon process termi-
nation (SR1). When ad-hoc changes occur, progress speed adjustments without
jumps are considered being more suitable and usable by the majority of the users
(SR2). 100% of the experts with a profession in economy, administration, trans-
port, or logistics agreed that no progress jumps are the best option, particularity
when facing more than one ad-hoc change (SR3). The well-known 90%-problem
(i.e., progress needs significant more time for the last 10% of the work than for
the first 90%) is evaluated slightly more suitable than neutral. Previously en-
tered data are still available when jumping back to an already completed form
the majority of the participants preferred progress speed adjustments without
jumps (SR2). Additionally, work-based progress calculation is favoured compared
to time-based one by a small majority (SR3). Furthermore, the study evalua-
tion indicates, which components are essential for monitoring tools and should
therefore be considered in further tool releases.
The conducted study confirms a generally accepted understanding of process
progress (Main Research Question). However, further research is needed to gain
a deeper understanding of progress in data-driven processes aware information
systems and its perception by end-users. For example, a Delphie study with a
focus on experts might provide additional insight into prediction needs.
Acknowledgment
This work is part of the ZAFH intralogistic project, funded by the European
Regional Development Fund and the Ministry of Science, Research and Arts of
Baden-Württemberg, Germany (F.No. 32-7545.24-17/12/1).
References
1. Andrews, K., Steinau, S. & Reichert, M. Enabling runtime flexibility in
data-centric and data-driven process execution engines. Information Sys-
tems (2021).
2. Andrews, K., Steinau, S. & Reichert, M. Enabling ad-hoc changes to object-
aware processes in 2018 IEEE 22nd International Enterprise Distributed
Object Computing Conference (EDOC) (2018), 85–94.
3. Steinau, S., Andrews, K. & Reichert, M. Executing Lifecycle Processes in
Object-Aware Process Management in Int. Symp. on Data-Driven Process
Discovery and Analysis (2017), 25–44.
4. Steinau, S., Andrews, K. & Reichert, M. The relational process structure in
Int. Conf. on Advanced Information Systems Engineering (2018), 53–67.
5. Steinau, S., Künzle, V., Andrews, K. & Reichert, M. Coordinating business
processes using semantic relationships in Conf. on Business Informatics
(2017), 33–42.
6. Arnold, L., Breitmayer, M. & Reichert, M. Towards Real-Time Progress
Determination of Object-Aware Business Processes in Proceedings of the
13th European Workshop on Services and their Composition 2839 (2021),
14–18.
7. Arnold, L., Breitmayer, M. & Reichert, M. A One-Dimensional Kalman
Filter for Real-Time Progress Prediction in Object Lifecycle Processes in
2021 IEEE 25th International Enterprise Distributed Object Computing
Workshop (EDOCW) (2021), 176–185.
8. Wieringa, R. Design science methodology for information systems and soft-
ware engineering (Springer, 2014).
9. Yin, R. K. Case study research: Design and methods (sage, 2009).
10. Schober, P., Boer, C. & Schwarte, L. A. Correlation coefficients: appropriate
use and interpretation. Anesthesia & Analgesia 126, 1763–1768 (2018).
11. Andrews, K., Steinau, S. & Reichert, M. Enabling ad-hoc changes to object-
aware processes in 2018 IEEE 22nd International Enterprise Distributed
Object Computing Conference (EDOC) (2018), 85–94.
12. Cohn, D. & Hull, R. Business artifacts: A data-centric approach to modeling
business operations and processes. IEEE Data Eng. Bull. 32, 3–9 (2009).
13. Steinau, S. et al. DALEC: a framework for the systematic evaluation of
data-centric approaches to process management software. Software & Sys-
tems Modeling 18, 2679–2716 (2019).
14. Van der Aalst, W., Weske, M. & Grünbauer, D. Case handling: a new
paradigm for business process support. Data & Knowledge Engineering 53,
129–162 (2005).
15. Conrad, F., Couper, M., Tourangeau, R. & Peytchev, A. The impact of
progress indicators on task completion. Interacting with computers 22, 417–
427 (2010).
16. Myers, B. The importance of percent-done progress indicators for com-
puter-human interfaces. ACM SIGCHI Bulletin 16, 11–17 (1985).
