How Advanced Change Patterns Impact the Process of Process Modeling [original]

How Advanced Change Patterns Impact the

Process of Process Modeling?

Barbara Weber1, Sarah Zeitelhofer1, Jakob Pinggera1, Victoria Torres2, and

Manfred Reichert3

1University of Innsbruck, Austria

[email protected]|[email protected]

2Universitat Polit`ecnica de Val`encia, Spain

[email protected]

3University of Ulm, Germany

manfred.reichert@uni-ulm.de

Abstract. Process model quality has been an area of considerable re-

search efforts. In this context, correctness-by-construction as enabled by

change patterns provides promising perspectives. While the process of

process modeling (PPM) based on change primitives has been thoroughly

investigated, only little is known about the PPM based on change pat-

terns. In particular, it is unclear what set of change patterns should be

provided and how the available change pattern set impacts the PPM. To

obtain a better understanding of the latter as well as the (subjective)

perceptions of process modelers, the arising challenges, and the pros and

cons of different change pattern sets we conduct a controlled experiment.

Our results indicate that process modelers face similar challenges irre-

spective of the used change pattern set (core pattern set versus extended

pattern set, which adds two advanced change patterns to the core pat-

terns set). An extended change pattern set, however, is perceived as more

difficult to use, yielding a higher mental effort. Moreover, our results in-

dicate that more advanced patterns were only used to a limited extent

and frequently applied incorrectly, thus, lowering the potential benefits

of an extended pattern set.

Key words: Process Model Quality, Process of Process Modeling,

Change Patterns, Controlled Experiment, Problem Solving

1 Introduction

Due to the important role they play for process–aware information systems,

process models have become increasingly important for many years [1]. In this

context, it was shown that process model understandability has a measurable

impact on whether or not a process modeling initiative is successful [2]. Still,

process models exhibit a wide range of quality problems, which not only hamper

comprehensibility but also affect maintainability [3, 4]. For example, [3] reports

on error rates between 10% and 20% in collections of industrial process models.

?This research is supported by Austrian Science Fund (FWF): P23699-N23

2 Weber et al.

To improve process model quality, change patterns appear promising. They

combine change primitives, e.g., to add nodes or edges, to high-level change op-

erations [4]. In particular, change patterns enable correctness-by-construction [6]

by providing only those change patterns to the modeler, which ensure that pro-

cess models remain sound after applying model transformations.

Recently, the creation of process models based on change primitives has re-

ceived considerable attention resulting in research on the process of process mod-

eling (PPM) [7, 8, 9]. This research focuses on the formalization phase of process

model creation, i.e., the interactions of the process modeler with the modeling

environment. The PPM utilizing change patterns, in turn, is still hardly un-

derstood. In previous work we presented an exploratory study to investigate

re-occurring challenges when using change patterns for process modeling [10].

The study revealed that process modelers did not face major problems when

using change patterns for constructing simple process fragments. When being

confronted with more complex process fragments, however, difficulties increased

observably. Building respective structures efficiently (i.e., without detours in the

PPM) requires process modelers to look ahead, since patterns cannot be always

combined arbitrarily. This need for looking ahead is a fundamental difference

compared to process model creation based on change primitives and was per-

ceived as both challenging and restrictive by subjects. Further, [10] emphasizes

that the basic set of change patterns, which allows creating control flow struc-

tures like sequence, exclusive branchings, parallel branchings, and loops, is not

sufficient for efficient model creation. In particular, the study observed that pat-

terns for moving process fragments might help to resolve detours efficiently.

On one hand, an extended set of change patterns (including move patterns)

offers more flexibility. On the other, it increases complexity, especially when

mapping the mental model of the process to be created to the available pattern

set. As a result, the extended change pattern set might make the modeling

environment more difficult to use. This raises the question whether the expected

benefits of an extended pattern set can be materialized in a practical setting. To

obtain an in-depth understanding of the impact an extended pattern set has on

the PPM, we implement a modeling editor offering two different change pattern

sets based on Cheetah Experimental Platform (CEP) [11]. Using this editor, we

conduct a controlled experiment with 42 process modelers. Our results indicate

that an extended pattern set yields higher mental effort for modelers and is

perceived as more difficult to use. At the same time, the expected benefits in

terms of increased problem solving efficiency did not materialize, suggesting to

focus on a core pattern set. The results provide a contribution toward a better

understanding on how tool features (like change patterns) impact the PPM, but

also give advice on how effective tool support should be designed.

Sect. 2 introduces backgrounds. Sect. 3 describes the controlled experiment.

Sect. 4 presents the subjective perception of change pattern use. Sect. 5 deals

with the impact of change patterns on problem solving efficiency and Sect. 6

details on the actual and potential use of patterns. Limitations are presented in

Sect. 7. Related work is presented in Sect. 8. Sect. 9 concludes the paper.

How Advanced Change Patterns Impact the Process of Process Modeling 3

2 Process Model Creation Based on Change Patterns

Most environments for process model creation are based on change primitives,

e.g., add/delete activity or add/delete edge. Process model adaptations

(i.e., transformation of a model S into model S’) may require the joint applica-

tion of multiple change primitives. Imagine process model S1in Fig. 1 without

the colored fragment. To transform this model into S1 (including the colored

fragment) 19 change primitives are needed: deleting the edge between activity D

and the parallel gateway, adding D,E, and Fto the process model, adding the con-

ditional branch around C(including transition conditions), and adding the edges

connecting the newly added elements with the process model. When applying

change primitive, soundness of the resulting process model cannot be guaran-

teed and must be explicitly checked after every model transformation. In turn,

change patterns imply a different way of interacting with the modeling environ-

ment. Instead of applying a set of change primitives, high-level change operations

are used to realize the desired model transformation. Examples of change pat-

terns include the insertion of process fragments, their embedding in conditional

branches or loops, or the updating of transition conditions. A catalog of change

patterns can be found in [4], while their semantics of these patterns are described

in [5]. To conduct the described transformation with change patterns (i.e., obtain

S1from a model where the colored fragment is missing), 6 pattern applications

are needed (i.e., serial insert of activity E, parallel insert of activity F, serial

insert of activity C, embed activity Cin conditional branch, and two updates of

conditions). As opposed to change primitives, change pattern implementations

typically guarantee model correctness after each transformation [6] by associat-

ing pre-/post-conditions with high-level change operations. In process modeling

environments supporting the correctness–by–construction principle (e.g., [12]),

usually process modelers only have those change patterns available for use that

allow transforming a sound process model into another sound one. For this pur-

pose, structural restrictions on process models (e.g., block structuredness) are

imposed. This paper investigates the impact of two different change pattern sets

on the PPM.

3 Experiment

This section describes research questions and the design of the experiment.

Research Questions. Our goal is to obtain an in-depth PPM understand-

ing when using change patterns. More specifically, we want to understand how

modelers experience their interaction with the modeling environment depending

on the available change pattern set.

RQ1: What is the impact of the change pattern set available to process

modelers on their subjective perception during model creation?

In addition to the subjective perception of modelers, we are interested in the

challenges faced by process modelers during the PPM depending on the used

4 Weber et al.

change pattern set. Respective challenges can result in modeling errors that

persist in the final model, but also detours on the way to a complete process

model, negatively affecting problem solving efficiency.

RQ2: What is the impact of the change pattern set available to process

modelers on the challenges faced during model creation?

Finally, we want to understand how well the additional patterns of the extended

pattern set was adopted (i.e., in their actual use) as well as the potential benefits

that could have been achieved through proper pattern usage.

RQ3: What was the actual use of the additional change patterns com-

pared to the potential of using those patterns?

Factors and Factor Level. The experiment considers a single factor, i.e,

the pattern set used to conduct the modeling task with factor levels: core and

extended. The core pattern set comprises a minimum change pattern set (see [4]

for the full pattern set) that allows modelers to create basic control-flow struc-

tures (i.e., sequences, parallel, conditional branchings, and loops): patterns AP1

(Insert Process Fragment), AP2 (Delete Process Fragment), AP8 (Embed Frag-

ment in Loop), AP10 (Embed Process Fragment in Conditional Branch), and

AP13 (Update Condition). Concerning pattern AP1, two variants were provided:

Serial and Parallel Insert. In addition, process modelers could rename activities.

In turn, the extended pattern set comprises all patterns included in the core pat-

tern set plus an advanced pattern for moving process fragments (AP3). To be

able to trace back the impact to single change patterns, we intentionally decided

to only add one additional pattern from which we expect a considerable impact

on problem solving efficiency to the extended pattern set. Similar to AP1, two

variants are provided: Serial and Parallel Move. While the core pattern set is

complete in the sense that all control-flow structures can be created, it does

not allow for arbitrary model transformations. In particular, in [10] we observed

that detours could have been addressed more efficiently with an extended pat-

tern set. In particular, we observed that patterns for moving process fragments

would have helped with many of the detours. Frequently, process modelers had

to undo or delete considerable parts of the model, which could have been re-

solved with the application of a single move pattern. Consider, for example, the

two models in Fig. 1. When transforming S1to S2without move patterns, the

modeler must perform a detour of 7 steps to delete the colored parallel branch

and to re-insert it after activity B (cf. problem solving path P1,2). On the con-

trary, using move patterns, transforming S1into S2just requires the application

of one change pattern, i.e., Serial Move, saving a total of 6 pattern applications.

Modeling Tasks. The modeling task is a slight adaption of the task used

in [10] and describes a process of the “Task Force Earthquakes” of the German

Research Center for Geosciences [13] (cf. Fig. 2—labels are abstracted for read-

ability). The task comprises 15 activities; all main control–flow structures like

sequence, parallel and conditional branchings, and loops are present. The model

How Advanced Change Patterns Impact the Process of Process Modeling 5

Model S1

P1,2 = < Delete Parallel Fragment, Serial Insert (E), Serial Insert (C),Embed (C) in Cond. Branch,

Update Condition (c1), Update Condition (c2),Parallel insert F>

Model S2

Fig. 1. Detour to go from S1to S2when no Move Patterns are available

has a nesting depth of 4. Subjects received an informal requirements descrip-

tion as well as the solution of the modeling task (i.e., a process model). Their

task was to re–model the process using change patterns. To model the process a

minimum number of 28 change patterns are required with both the core and the

extended change pattern set. Since subjects had the correct solution available,

the challenge lies in determining the patterns for re-constructing the model and

in combining the available patterns effectively.

A B C D E

c10

Fig. 2. Solution Model SS

Subjects. Novices and experts differ in their problem solving strategies.

Considering that industrial process modelers are often not expert modelers, but

rather casual modelers with a basic amount of training [14], subjects participat-

ing in our experiment are not required to be experts. In previous research with

software engineering students it has been shown that students can provide an

adequate model for the professional population [15, 16, 17]. Thus, we relied on

students (instead of professionals) in our experiment. To avoid difficulties due

to unfamiliarity with the tool, rather than the modeling task, we require some

prior experience with process modeling as well as change patterns. To ensure

that the subjects are sufficiently literate in change pattern usage, subjects are

provided with theoretical backgrounds. Further the subjects obtain hands-on

experience in the creation of process models using change patterns in terms of

a familiarization task.

Experimental Setup. The experiment consists of four phases. (1) collecting

demographic data, (2) familiarization with the change patterns editor, and (3)

performing a modeling task. Subjects were divided into two groups. While Group

Areceives the core pattern set, Group B conducts the same task based on

the extended pattern set. During modeling, all interactions with the modeling

environment are recorded by CEP [11]. This allows us to replay the creation of

the process model step-by-step [11], addressing RQ2 and RQ3. After completing

the modeling tasks, (4) mental effort as well as Perceived Ease of Use (PEU) and

6 Weber et al.

Perceived Usefulness (PU) of the Technology Acceptance Model [18] are assessed,

addressing RQ1.

Experimental Execution. Prior to the experiment a pilot was conducted to

ensure usability of the tool and understandability of the task description. This

led to improvements of CEP and minor updates of the task description. The

experiment was conducted by 42 graduate and postgraduate students from the

Universities of Innsbruck, Ulm, and Valencia. Subjects were randomly assigned

to groups, with an equal number of subjects for each group.

Data Validation. To obtain a valid data set, we checked for completeness

of the created process models. Unfortunately, 8 of the participants had to be re-

moved due to incomplete models. As, a result, 34 subjects remained in the data

set, which were equally distributed over the two groups. Since we did not con-

sider process modeling knowledge and experience as a factor in our experiment,

we screened the participants for prior knowledge regarding BPMN and change

patterns. For this, a questionnaire similar to [19] was used to verify that subjects

were equally distributed to the two groups. (cf. Table 1). The questionnaire used

Likert scale ranging from strongly disagree (1) to strongly agree (7). To test for

differences between the two groups, t-tests were run for normally distributed

data. For non–normally distributed data, the Mann-Whitney test was used. No

significant differences were identified between the two groups. Consequently, we

conclude that no differences could be observed between the two groups.

Question Group Min Max M SD

Familiarity with BPMN A 2 7 5.12 0.99

B 2 7 5.53 1.28

Confidence in understanding BPMN A 3 7 5.53 1.33

B 4 7 6.24 0.75

Competence using BPMN A 3 7 5.06 1.14

B 3 7 5.59 1.06

Familiarity change patterns A 2 7 4.76 1.44

B 2 7 4.53 1.46

Competence using change patterns A 2 7 4.59 1.33

B 2 6 4.41 1.28

Table 1. Demographic Data

4 Subjective Perception of Model Creation

This section addresses research question RQ1 dealing with the subjective per-

ception of process modelers when using change patterns. In particular, it investi-

gates how the used change pattern set (core vs. extended) impacts mental effort.

Further, we investigate the perceived ease of use and perceived usefulness.

4.1 Mental Effort

Descriptive Statistics. The results related to mental efforts are displayed in

Table 2. Mental effort was measured using a 7-point Likert scale ranging from

’very low’ (1) to ’very high’ (7). For Group A the mean mental effort was 3.35,

How Advanced Change Patterns Impact the Process of Process Modeling 7

corresponding approx. to ’rather low’ (3). In turn, for Group B the mental effort

was higher with a mean of 4.25, corresponding to ’medium’ (4).

Scale Group Min Max M SD

Mental Effort A 2 5 3.35 1.06

B 3 7 4.25 1.00

Perceived Ease of Use A 5.18 6.06 5.81 0.33

B 4.53 5.82 5.25 0.47

Perceived Usefulness A 4.13 4.75 4.38 0.21

B 3.87 4.87 4.27 0.36

Table 2. Subjective Perception

Hypothesis Testing. When using change patterns for process modeling, plan

schemata on how to apply change patterns need to be developed in order to

create complex process fragments. In this context, we investigate how the men-

tal effort of modelers is affected by utilizing a larger change pattern set. While

the extended change pattern set allows modelers to recover faster from detours,

it also requires them to develop additional plan schemata on how to apply the

move change patterns. Therefore, an extended change pattern set might impose

higher demands on the modeler’s cognitive resources. Especially, move change

patterns require modelers to imagine how the process model looks like after ap-

plying change patterns. This might put additional burden on them, requiring to

manipulate an internal representation of the process model in working memory.

In the light of the cognitive background, we expect the extended pattern set to

yield a significantly higher mental effort compared to the core pattern set.

Hypothesis H1 The usage of an extended change pattern set significantly

increases the mental effort required to accomplish the modeling task.

Since the data was normally distributed, a t-test was used for testing the

differences between the two groups (t(31) = −2.50, p = 0.02). The result allows

us to accept hypothesis H1.

Descriptive Statistics. In order to assess how far process modelers with mod-

erate process modeling knowledge consider the change pattern editor as easy

to use and useful, we asked them to fill out the Perceived Ease of Use (PEU)

and the Perceived Usefulness (PU). Both scales consist of 7-point Likert items,

ranging from ’extremely unlikely’ (1) over ’neither likely nor unlikely’ (4) to ’ex-

tremely likely’ (7). Regarding the PEU, the mean value was 5.81 for Group A

(core pattern set), corresponding approx. to ’quite likely’ (6). In turn, for Group

B (extended pattern set) the mean value was 5.25, corresponding approx. to

’slightly likely’ (5). Finally, regarding the PU, the observed mean value was 4.38

for Group A and 4.27 for Group B, corresponding approx. to ’neither likely nor

unlikely’ (4) for both groups. Three participants indicated that they could not

answer the questions on PU. Hence, they were removed for the analysis of PU.

Hypothesis Testing. As stated for mental effort already, the extended pattern

set requires modelers to develop additional plan schemata in order to apply

the change patterns properly. Accordingly, one would expect that an extended

8 Weber et al.

change pattern set is more difficult to use. However, these should also be per-

ceived as more useful since the extended pattern set helps to resolve detours

quicker compared to the core pattern set, i.e., when allowing to move a mis-

placed process fragment based on a respective pattern.

Hypothesis H2 The usage of an extended change pattern set significantly

lowers the perceived ease of use.

Hypothesis H3 The usage of an extended change pattern set significantly

increases the perceived usefulness.

Since none of the groups are normally distributed, we apply the Mann-

Whitney U-Test to test for differences regarding PEU and PU. While significant

differences in terms of PEU (U= 4010.50, p = 0.00) allow us to accept hypothesis

H2, no statistically significant differences in terms of PU (U= 3639.00, p = 0.06)

were observed.

Discussion. Our results indicate that the core pattern set leads to a significantly

lower mental effort for modelers and its use is perceived as being significantly

easier compared to the extended pattern set. This seems reasonable since mod-

elers need to devote additional cognitive resources in order to use the move

change patterns. Regarding PU, against our expectations, we could not obtain

any statistically significant result. When looking at the descriptive statistics,

the participants of Group B tend to perceive change patterns as less useful com-

pared to Group A. We might conclude that the move change patterns provided

for Group B are not as useful as expected (at least for the task assigned to the

subjects). Alternatively, the subjects of Group B might have struggled with the

usage of change patterns due to the additional patterns. In turn, this might have

foiled potential positive effects of the additional patterns. The results presented

in Sec. 5 support the latter explanation suggesting that process modelers had

considerable problems with the use of the move patterns.

5 Challenges when Modeling with Change Patterns

This section addresses research question RQ2 aiming to obtain an in-depth un-

derstanding how the chosen pattern set impacts the challenges faced by modelers.

5.1 Data Analysis Procedure

Step 1: Determine Solution Model, Distance, and Optimal Problem Solving

Paths. First, we create a model representing the correct solution (i.e., SS) for the

modeling task. Subjects had to work on a re-modeling task as described in Sect.

3, i.e., in addition to an informal textual description they obtained the solution

to the modeling task in the form of a graphical model. Thus, the goal state of

the modeling task was clearly defined and unique, i.e., subjects should create a

graphical representation of the process that exactly looks like the solution model.

To be able to assess not only how closely subjects reached the goal state (i.e., how

similar their resulting model is to the solution model), but also how efficiently

How Advanced Change Patterns Impact the Process of Process Modeling 9

their problem solving process was, we determine the distance for transforming

an empty model S0to SS, i.e., the minimum number of change patterns required

for the respective model transformation. Generally, there are several options to

create the solution model SSby starting from S0and applying a sequence of

model transformations. From a cognitive perspective, each sequence of change

patterns that leads to SSwithout detours constitutes an optimal problem solving

path. Starting from S0the process fragment depicted in Fig. 3 can be created

with 6 change patterns; e.g., SScan be created by first inserting Aand then

B, next embedding Bin a conditional branch, then updating the two transition

conditions, and finally inserting C(P0in Fig. 3).

Step 2: Determine Deviations from Solution Model and Optimal Problem

Solving Path.

To quantify the efficiency of the problem solving strategy used by the subjects

to accomplish the re-modeling task, their problem solving path is analyzed. To be

more specific, using the replay feature of CEP we compare the subject’s problem

solving path Pwith the optimal one and capture deviations from it. For this,

every superfluous change pattern application a subject performs is counted as a

process deviation. To quantify how close subjects reached the goal state (i.e., how

similar their resulting model is to the solution model SS) we consider product

deviations that measure the number of incorrect change pattern applications

leading to deviations between the final models created by the subjects and the

solution model SS.

Fig. 3 shows the problem solving path P0of one modeler who managed to

model the depicted fragment correctly (i.e., 0 process deviations and 0 product

deviations). Problem solving path P2, in turn, leads to a correct goal state (i.e.,

0 product deviations). However, the modeler made a detour of 2 change pat-

terns before reaching the solution (i.e., solution path P2comprises 2 superfluous

change patterns summing up to 2 process deviations). Now assume that the

modeler, who took a detour when creating the process model, did not correct

the introduced error ending up with an incorrect process model (cf. path P1

in Fig. 3). The application of the Embed in Loop pattern (instead of Embed in

Conditional Branch) constitutes 1 product deviation (i.e., the modeler applied

one incorrect change pattern that led to an incorrect goal state).

Since not every subject reached the goal state (i.e., their models contain product

deviations), the direct comparison of process deviations might favor modelers

that left out parts that were difficult to model and where other subjects produced

a high number of process deviations. To decrease this bias we consider a second

measure for operationalizing problem solving efficiency. In addition to the process

deviations described above this measure considers the effort needed to correct

an incorrect process model (denoted as fixing steps), i.e., the steps needed to

transform the created model into SS. For example, to correct the model that

resulted from P1 in Fig. 3, 5 fixing steps are needed, irrespective of whether

or not the core or the extended change pattern set is used. First the fragment

embedded in the loop has to be deleted. Next, Bhas to be re-inserted and

embedded in a conditional branch, and then the two transition conditions must

10 Weber et al.

B is necessary

B is not necessary

P0 = < Serial Insert (A), Serial Insert (B), Embed (B) in Cond. Branch,

Update Condition (B is not necessary),

Update Condition (B is necessary), Serial Insert (C) >

P1 = < Serial Insert (A), Serial Insert (B), Embed (B) in Loop,

Update Condition (B is not necessary),

Update Condition (B is necessary), Serial Insert (C) >

P2 = < Serial Insert (A), Serial Insert (B), Embed (B) in Loop,

Undo Embed (B) in Loop, Embed (B) in Cond. Branch,

Update Condition (B is not necessary),

Update Condition (B is necessary), Serial Insert (C) >

Fig. 3. Process Deviations, Product Deviations, and Fixing Steps

be updated. Fixing steps and process deviations are then combined in a single

measure called total process deviations. This measure does not only consider

detours (i.e., process deviations), but also model transformations that would be

needed to correct the process model (i.e., resolving product deviations).

Step 3: Detection of Outliers. In order to limit the influence of modelers who

are experiencing severe difficulties during the creation of the process model, we

test for outliers regarding the number of process deviations. For this purpose, we

utilize the Median Absolute Deviation (MAD) to detect outliers. More specifi-

cally, we apply a rather conservative criterion for removing outliers by removing

values that differ at least 3 times the MAD from the median [20]. As a result,

one PPM instance was removed from Group B regarding further analysis.

5.2 Results

Descriptive Statistics. To create a correct solution model 28 operations are

needed. Overall, 123 process deviations (i.e., detours in the modeling process)

and 44 product deviations (i.e., deviations of the final models from the solution

model) were identified (cf. Table 3). From the 123 process deviations 60 can be

attributed to Group A (3.53 deviations per subject), while 63 were found for

Group B (3.94 deviations per subject). In terms of product deviations they were

equally distributed among the two groups, i.e., 22 product deviations per group

(1.29 deviations per subject in Group A and 1.38 deviations per subject in Group

B). In order to resolve the product deviations, 45 fixing steps are required for

the models of Group A and 29 fixing steps for Group B resulting in 105 and 92

total process deviations respectively.

Hypothesis Testing. We test for significant differences between the two groups

regarding process deviations and total process deviations. We expect that the

modelers using the extended pattern set have significantly fewer process devia-

tions, because the extended pattern set allows them to resolve detours with fewer

steps. Moreover, we expect an impact on the total process deviations, since the

How Advanced Change Patterns Impact the Process of Process Modeling 11

Scale Group A Group B

Process deviations 60 63

Average Process deviations per modeler 3.53 3.94

Product deviations 22 22

Average Product deviations per modeler 1.29 1.38

Fixing steps 45 29

Average fixing steps per modeler 2.65 1.81

Total process deviations 105 92

Average total process deviations per modeler 6.18 5.75

Table 3. Overview of Deviations

extended pattern set allows transforming the model created by the modelers

with fewer steps into the solution model.

Hypothesis H4 The usage of an extended change pattern set significantly

decreases the number of process deviations.

Hypothesis H5 The usage of an extended change pattern set significantly

decreases the number of total process deviations.

To test for differences in terms of process and total process deviations, we

apply the t-test since the data was normally distributed. No statistical difference

could be observed for process deviations (t(31) = −0.24, p = 0.82) or total

process deviations (t(31) = 0.25, p = 0.81).

Discussion. Our results did not yield statistically significant differences between

the two groups. This indicates that the usage of an extended change pattern set

might not have an impact on both process deviations and total process devia-

tions. An alternative explanation could be that process modelers did not use the

provided patterns frequently enough to obtain statistically significant differences

(i.e., pattern adoption was low). Another explanation could be that subjects did

not use the patterns effectively, canceling out a potential positive impact. To

investigate these alternative explanations in more detail Sect. 6 analyzes the

actual use of the move change patterns.

6 Actual and Potential Use of an Extended Pattern Set

This section addresses research question RQ3 which deals with the actual use of

the additional change patterns compared to the potential usage of those patterns.

The analysis of invocations of the move change patterns revealed that the serial

move pattern was only applied 3 times (by 3 different participants), whereas

the parallel move pattern was used 18 times (by 7 different participants). This

indicates that the subjects adopted the move patterns only to a limited extend.

Out of the 21 move pattern applications, 11 were correct; i.e., they led to correct

intermediate models, either directly through the application of the pattern or

the application of the pattern in combination with additional patterns. In turn,

10 applications of the parallel move pattern were incorrect and either led to an

undesired model or did not yield any changes of the model. This indicates that

subjects had difficulties when applying the move change patterns.

12 Weber et al.

Though the actual use of the move patterns was limited, we investigate their

theoretical potential. For this, we analyze the number of fixing steps required

to correct a model with product deviations (i.e., to transform it into SS). We

further analyze how the availability of an extended pattern set impacts this

measure. In a second step, we analyze the potential of an extended pattern set

for reducing process deviations, i.e., by enabling a faster resolution of detours.

Scale Group A Group B

Fixing steps with move 45 64

Fixing steps without move 25 29

Saved operations 20 35

Process Deviations 60 63

Unnecessary Operations - 15

Saved operations 9 0

Potential process deviations 51 48

Table 4. Potential Use of the Move Change Pattern

To show the potential of an extended pattern set for resolving product devi-

ations, Table 4 depicts the number of fixing steps, when using the core pattern

set and for the extended pattern set. For Group A, 45 fixing steps are required

to correct all product deviations that occurred. By making the extended pattern

set available to Group A, this number could be reduced to 25 (i.e., 20 fixing steps

could be saved). In turn, for Group B the number of observed fixing steps is 29.

Without the extended pattern set, however, 64 fixing steps would be needed.

This indicates the theoretical potential of the extended pattern set for reducing

the number of fixing steps and, thus, the number of total process deviations.

To investigate the potential for reducing process deviations for Group A, we

analyze whether process deviations could have been reduced when using move

change patterns. In turn, for Group B we focused on the number of operations

that would have been saved if move patterns were always applied correctly. As

illustrated in Table 4, 9 operations could be saved if the move pattern had been

available for Group A resulting in 51 potential process deviations. Regarding

Group B, 15 operations could have been saved through correct pattern applica-

tion resulting in 48 potential process deviations.

Discussion. These results suggest that a theoretical potential for using move

change patterns exists. However, the subjects used the move change patterns

only to a limited extent and had troubles with their correct application. As a

consequence the potential of the additional patterns could not be fully exploited.

Since mental effort and perceived ease of use is lower with the core pattern set it

might be more favorable to use the core pattern set for process modelers that are

only moderately familiar with process modeling and are no experts in the usage

of change patterns. We might speculate that the extended pattern set could be

promising for more experienced users (who are literate in pattern usage).

How Advanced Change Patterns Impact the Process of Process Modeling 13

7 Limitations

As with every other research, this work is subject to several limitations. Cer-

tainly, the relatively small sample size constitutes a threat regarding the gen-

eralization of our results. Using students instead of professionals poses another

threat regarding external validity. In previous research with software engineer-

ing students it has been shown that students may provide an adequate model

for the professional population [15, 16, 17]. Still, generalizations should be made

with care. Moreover, since we used subjects who were moderately familiar with

process modeling and change patterns results cannot be generalized to expert

modelers. It can be assumed that process modelers experienced with the usage

of change patterns will presumably face less problems during model creation and

will be able to apply patterns more effectively. Another limitation relates to the

fact that we used only one modeling task in our study. The potential benefit of

move patterns, however, depends on the structure of the process model to be

created. For more complex process models with higher nesting depth the poten-

tial usefulness might be higher. Thus, it is questionable in how far results may be

generalized to models with different characteristics. As a consequence, we plan

further experiments testing the impact of model structure on challenges regard-

ing change pattern usage. Moreover, this work compares two particular change

pattern sets. Using an extended change pattern set with different patterns (e.g.,

a pattern to change a conditional fragment into a parallel fragment or to change

a conditional fragment to a loop) might lead to different results. Another lim-

itation regarding the external validity relates to the process modeling notation

(i.e., BPMN) and the modeling tool used (i.e., CEP). Results might be different

when using other modeling languages or different modeling tools.

8 Related Work

The presented work relates to research developed in the context of the creation

of process models and process model creation patterns.

Research on the creation of process models builds on observations of modeling

practice and distills normative procedures for steering the process of modeling to-

ward successful completion. To do so, [21, 22] deal with structured discussions

among different parties (system analysts, domain experts). In this line of re-

search, [23] analyzes the procedure of developing process models in a team, while

[24] discusses participative modeling. Complementary to these works, whose fo-

cus is on the effective interaction between the involved stakeholders, our work

focuses is on the formalization of the process model.

Researchers have also focused on the interactions with the modeling environ-

ment, i.e., the PPM. [9] identified three distinct modeling styles, whereas [7, 25]

suggest different visualization techniques for obtaining an overview of the PPM;

[8] demonstrates that a structured modeling style leads to models of better qual-

ity. [26] investigates the PPM using eye movement analysis. While these works

14 Weber et al.

focus on interactions with the modeling environment based on change primitives,

this paper investigates the use of change patterns.

Change patterns for process model creation have been investigated as well;

e.g., AristaFlow allows modeling a sound process schema based on an extensible

set of change patterns [12]. [27] describes a set of pattern compounds, comparable

to change patterns, allowing for the context-sensitive selection and composition

of workflow patterns. Complementary to these works, which have a strong design

focus, this paper provides empirical insights into the usage of change patterns.

More precisely, it builds upon the results obtained in [10], which describes re-

curring challenges modelers face during the PPM using change patterns.

9 Summary

While recent research has contributed to a better understanding regarding the

PPM, little is known about this process when utilizing change patterns. In this

experiment we investigate the impact of the available patterns on the PPM and

the modeler’s perception. The results indicate that an extended change pattern

set puts an additional burden on modelers who perceive them as more difficult

to use. In addition, when using these patterns, subjects faced considerable dif-

ficulties. Therefore, (against our expectations) our data does not indicate an

increased problem solving efficiency, i.e., the expected benefits of using the ex-

tended change pattern set did not materialize. This indicates that the change

pattern set should be selected with care, especially for modelers with limited

experience. Future research should include investigations on new change pat-

tern sets having a (theoretical) potential for reducing process deviations, e.g., a

pattern to change a conditional fragment into a parallel or a loop fragment.

References

1. Becker, J., Rosemann, M., Uthmann, C.: Guidelines of Business Process Modeling.

In: Business Process Management. Springer-Verlag, London, UK (2000) 30–49

2. Kock, N., Verville, J., Danesh-Pajou, A., DeLuca, D.: Communication flow orien-

tation in business process modeling and its effect on redesign success: Results from

a field study. Decision Support Systems 46 (2009) 562–575

3. Mendling, J., Verbeek, H.M.W., van Dongen, B.F., van der Aalst, W.M.P., Neu-

mann, G.: Detection and prediction of errors in EPCs of the SAP reference model.

Data and Knowledge Engineering 64 (2008) 312–329

4. Weber, B., Reichert, M., Rinderle, S.: Change Patterns and Change Support Fea-

tures - Enhancing Flexibility in Process-Aware Information Systems. Data and

Knowledge Engineering 66 (2008) 438–466

5. Rinderle-Ma, S., Reichert, M., Weber, B.: On the formal semantics of change

patterns in process-aware information systems. In: Proc. ER ’08. (2008) 279–293

6. Casati, F.: Models, Semantics, and Formal Methods for the design of Workflows

and their Exceptions. PhD thesis, Milano (1998)

How Advanced Change Patterns Impact the Process of Process Modeling 15

7. Pinggera, J., Zugal, S., Weidlich, M., Fahland, D., Weber, B., Mendling, J., Reijers,

H.: Tracing the Process of Process Modeling with Modeling Phase Diagrams. In:

Proc. ER-BPM ’11. (2012) 370–382

8. Claes, J., Vanderfeesten, I., Reijers, H., Pinggera, J., Weidlich, M., Zugal, S.,

Fahland, D., Weber, B., Mendling, J., Poels, G.: Tying Process Model Quality

to the Modeling Process: The Impact of Structuring, Movement, and Speed. In:

Proc. BPM ’12. (2012) 33–48

9. Pinggera, J., Soffer, P., Fahland, D., Weidlich, M., Zugal, S., Weber, B., Reijers,

H., Mendling, J.: Styles in business process modeling: an exploration and a model.

SoSyM (2013)

10. Weber, B., Pinggera, J., Torres, V., Reichert, M.: Change Patterns in Use: A

Critical Evaluation. In: Proc. BPMDS ’13. (2013) 261–276

11. Pinggera, J., Zugal, S., Weber, B.: Investigating the Process of Process Modeling

with Cheetah Experimental Platform. In: Proc. ER-POIS ’10. (2010) 13–18

12. Dadam, P., Reichert, M.: The ADEPT project: a decade of research and develop-

ment for robust and flexible process support. Comp Scie - R&D 23 (2009) 81–97

13. Fahland, D., Woith, H.: Towards process models for disaster response. In: Proc.

PM4HDPS 2008. (2008) 254–265

14. Pinggera, J., Zugal, S., Weber, B., Fahland, D., Weidlich, M., Mendling, J., Reijers,

H.: How the Structuring of Domain Knowledge Can Help Casual Process Modelers.

In: Proc. ER ’10. (2010) 445–451

15. H¨ost, M., Regnell, B., Wohlin, C.: Using students as subjects-a comparative study

of students and professionals in lead-time impact assessment. Empirical Software

Engineering 5(2000) 201–214

16. Porter, A.A., Votta, L.G.: Comparing detection methods for software require-

ments inspections: A replication using professional subjects. Empirical Software

Engineering 3(1998) 355–379

17. Runeson, P.: Using students as experiment subjects—an analysis on graduate and

freshmen student data. In: Proc. EASE’03. (2003) 95–102

18. Davis, F.: Perceived usefulness, perceived ease of use, and user acceptance of

information technology. MIS Quarterly 13 (1989) 319–340

19. Mendling, J., Reijers, H.A., Cardoso, J.: What Makes Process Models Understand-

able? In: Proc. BPM ’07. (2007) 48–63

20. Leys, C., Ley, C., Klein, O., Bernard, P., Licata, L.: Detecting outliers: Do not use

standard deviation around the mean, use absolute deviation around the median.

Journal of Experimental Social Psychology 49 (2013) 764–766

21. Frederiks, P., Weide, T.: Information modeling: The process and the required

competencies of its participants. Data and Knowledge Engineering 58 (2006) 4–20

22. Hoppenbrouwers, S., Proper, H., Weide, T.: A fundamental view on the process of

conceptual modeling. In: Proc. ER ’05. (2005) 128–143

23. Rittgen, P.: Negotiating Models. In: Proc. CAiSE ’07. (2007) 561–573

24. Stirna, J., Persson, A., Sandkuhl, K.: Participative Enterprise Modeling: Experi-

ences and Recommendations. In: Proc. CAiSE ’07. (2007) 546–560

25. Claes, J., Vanderfeesten, I., Pinggera, J., Reijers, H., Weber, B., Poels, G.: Visual-

izing the Process of Process Modeling with PPMCharts. In: Proc. TAProViz ’12.

(2013) 744–755

26. Pinggera, J., Furtner, M., Martini, M., Sachse, P., Reiter, K., Zugal, S., Weber,

B.: Investigating the Process of Process Modeling with Eye Movement Analysis.

In: Proc. ER-BPM ’12. (2013) 438–450

27. Gschwind, T., Koehler, J., Wong, J.: Applying patterns during business process

modeling. In: Proc BPM’08. (2008) 4–19