Influence of Psychological Distance on Process Modeling: A Gamification Approach [original]

Ulm University | 89069 Ulm | Germany Faculty of

Engineering and

Computer Science

Institute of Databases and

Information Systems

Influence of Psychological Distance

on Process Modeling:

A Gamification Approach

Master Thesis at Ulm University

Submitted by:

Michael Zimoch

[email protected]

Reviewer:

Prof. Dr. Manfred Reichert

Dr. Vera Künzle

Supervisor:

Jens Kolb

2015

Version June 10, 2015

2015 Michael Zimoch

This work is licensed under the Creative Commons. Attribution-NonCommercial-ShareAlike 3.0

License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/de/

or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California,

94105, USA.

Satz: PDF-L

EX 2ε

Abstract

Nowadays, Business Process Management (BPM) has progressed significantly and

established itself as an important management concept for enterprises. For creating

efficient and effective business processes enterprises have given process models a high

priority. A well-documented business process is intended not only to describe a proce-

dure in detail, but serves as a foundation for further actions such as process automation,

improving process performance, and the identification of potential consequences as

well as the quickness to respond for changes. To this end, it is important to ensure

that process models represent the corresponding real world business processes as

accurately as possible. In turn, a not proper described business process may lead to

ineffectiveness, costs, and even losses. Hence, a focus is set on the quality, granularity

as well as structure of process models. By now, numerous guidelines exist for creating

correct and sound process models in respect to their quality, granularity, and resulting

structure. However, hardly research addresses cognitive aspects when creating process

models. Thereby, cognitive aspects are of particular importance for creating and under-

standing process models.

This thesis contributes insights from a controlled experiment investigating the influence

of psychological distance on the process of process modeling. More precisely, the effects

of social distance of a process designer to the modeled domain has on the creation

of process models are evaluated. In this context, the recent and emerging trend of

gamification is applied. Therefore, gamification in a 3D virtual world is used to enhance

the effects of social distance and for a better reflection of a real world problem.

The final results obtained from the experiment do not agree with the theory. In particular,

significant differences between low and high social distance with respect to process

model quality, granularity, and structure are observed but are contrary to the stated

goal of the experiment. Hence, the findings underline the importance of understanding

the effects of cognitive aspects on the process of process modeling. However, the

results may provide valuable incitements for enterprises to compose adequate teams for

creating or optimizing business process models.

iii

Acknowledgments

First and foremost, I share the credit of my work with my supervisor Jens Kolb for his

invaluable assistance, support, and guidance.

My sincere thanks goes to my first reviewer Prof. Dr. Manfred Reichert. Our cooperation

was truly an inspiring experience. I would also like to express my very great appreciation

to my second reviewer Dr. Vera Künzle.

My thanks go to all participants and assistants who helped in the development of this

work.

Special thanks goes to my friends Michael, Bernd, Wolfgang, Sebastian, Drazen,

Raphael, Christoph, Kevin, and Thinh for their advice and unaffordable moral support.

A very special thanks goes to the probably best bug tester in the world. My beloved

girlfriend Kristina.

Last, but not least, I would like to thank my parents Heinrich, Mariola, my siblings Martin

and Annemarie. Her encouragement and support was in the end what made this thesis

possible.

Contents

1 Introduction 1

1.1 Motivation.................................... 1

1.2 Contribution................................... 3

1.3 StructureoftheThesis............................. 4

2 Fundamentals of Construal Level Theory 5

2.1 LevelofConstrual ............................... 5

2.2 Psychological Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Introduction on Gamification, Virtual World, and 3D Warehouse Scenario 9

3.1 Gamification .................................. 9

3.2 VirtualWorld .................................. 10

3.3 3D Warehouse Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Experiment Planning and Definition 19

4.1 GoalDefinition ................................. 20

4.2 ContextSelection................................ 22

4.3 Hypotheses Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.4 ExperimentSetup ............................... 25

4.5 ExperimentDesign............................... 31

4.6 Risk Analysis and Mitigations . . . . . . . . . . . . . . . . . . . . . . . . . 34

5 Experiment Operation 37

5.1 Experiment Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

vii

Contents

5.2 ExperimentExecution ............................. 39

5.3 DataValidation................................. 40

6 Experiment Analysis and Interpretation 43

6.1 Analysis of Raw Data and Descriptive Statistics . . . . . . . . . . . . . . . 44

6.2 DataSetReduction .............................. 49

6.3 HypothesisTesting............................... 49

6.4 Summary and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

7 Related Work 55

8 Conclusion 59

A Evaluation and Task Sheets 73

B Demographic Questionnaire 77

C Game Questionnaire 85

D Raw Data 89

E Experimental Results 99

F Detailed Results of Hypothesis Testing 113

viii

Introduction

1.1 Motivation

In todays world,

process models

are crucial and have become indispensable for process-

oriented enterprises. Current enterprise repositories comprise large process model

collections [

]. Thereby, process models vary in respect to their

quality, granularity

as well as

process model structure

and, hence, their

usefulness

. A

high quality

process models, however, is crucial for any enterprise in order to guarantee their proper

use. Therefore, it is important to focus on well-designed process models for a better

comprehension of the complex business processes within an enterprise. Further, well-

designed process models serve to increase the transparency of business processes as

well as the efficient placement of functions, roles, and interfaces. As a prerequisite for

the later use, the respective process model should reflect its corresponding real world

1 Introduction

business process at the right level of granularity and in sufficient detail [63].

In this context, considerable work exists that presents criteria for suitable process

models and addresses also comprehensibility issues [

]. Modeling guidelines (e.g.,

Guidelines of Process Modeling (GoM)

Seven Process Modeling Guidelines (7PMG))

exist, which support process designers in creating process models of high quality

[

]. Thereby, hardly work exists evaluating the

process of process modeling

[

]

from a cognitive point of view and their effects on the resulting process models [

However, if we do not understand the cognitive aspects affecting process model quality,

granularity, and structure, process modeling projects might not deliver the required

results or even fail.

In this context, a fundamental factor presumably influencing the

process of process

modeling

is the

social distance

[

Social distance

is addressed by the

Construal

Level Theory (CLT)

and constitutes an important part of

psychological distance

[

]. In

particular, studies have shown that human thinking and acting are strongly influenced

by psychological distance [

]. According to CLT, we can only experience the

here and

now

, and, hence, we form an abstract mental construal of distant objects [

]; e.g.,

when thinking about a music festival we plan to visit, it is important to know which bands

will be playing, but details about the trip are not in our mind set yet. In turn, just before

the festival takes place, it will be important for us to know with whom to visit the festival

or how to get there, i.e., planning is done at a more fine-grained level.

Similarly, in the

process of process modeling

, various actors having different distance

to the modeled business process and its environment may be involved. While certain

process models are designed by people directly participating in the respective business

processes, others are modeled by external consultants or people from organizational

units (e.g., the quality assurance department) not involved in the process, i.e., people

having a

high social distance

to the process. A relevant question in this context is how

social distance influences the process of process modeling.

1.2 Contribution

Taking CLT as theoretical basis, previous experiment has shown that there exists a corre-

lation between the psychological distance (i.e., social, spatial, temporal, and hypothetical

distance) and their influence on the resulting process models [

]. In particular,

the results show a significant influence of psychological distance on the quality and

granularity of the resulting process models. Among all distances, the social distance

showed very significant results. For that reason and to get more adequate as well as

profound insights, the focus is set only on the social distance in the experiment.

The goal of the experiment is to investigate the influence of social distance on the process

of process modeling. Therefore, we adopt a new approach:

gamification

. The use of

gamification

is a recent and emerging trend in a non-game context [

]. Gamification

uses techniques from gaming to affect people improving their enthusiasm for "boring

activities". In conjunction with process modeling new opportunities are created; e.g.,

facilitating the introduction of new processes within an enterprise using game-based

thinking. The techniques of gamification are used to enthrall and engage process de-

signers for business processes in a more enjoyable way than a formal instruction. This,

in turn, has implications on the resulting process models with respect to their quality,

granularity, and structure [83].

For the experiment, the concept of gamification is used in a 3D virtual world to convey

the social distance to subjects (i.e., participants in an experiment) of the experiment.

For this intention, a warehouse scenario in a 3D virtual world environment is developed.

In this environment, one group of subjects are able to replay an

order processing in

a warehouse

. A second group watches passively the same warehouse scenario in a

video. Afterwards, both groups are asked to model the respective scenario based on

their own experience. Through conveying the social distance by means of playing and

watching the influence of the social distance on the process of process modeling can be

evaluated.

1 Introduction

1.3 Structure of the Thesis

The structure of this thesis is as follows:

Section 2 introduces the Construal Level Theory and the psychological distance in

detail. Gamification, virtual world, and the development of the 3D warehouse scenario

is described in Section 3. Section 4 presents the experiment planning and definition.

Experiment operation, which includes preparation, execution, and data validation is

introduced in Section 5. Analysis and interpretation of the results including descriptive

statistics and visualization of data, hypothesis testing as well as summary and discussion

of the results are presented in Section 6. Finally, Section 7 discusses related work and

Section 8 summarizes the thesis.

The general process of the experiment is illustrated in Figure 1.1.

Experiment

Planning

Definition

Section 4 Section 5 Section 6 Section 7 & 8

Experiment

Operation

Analysis

Interpretation

Work

Conclusion

Introduction

Psy. Distance

Gamification,

Virtual World

Section 2 & 3

Figure 1.1: Experiment Process

Fundamentals of Construal Level Theory

The main focus in this thesis is on the

social distance

. However, for a better under-

standing of the latter the basics of the

Construal Level Theory

and its properties are

presented. Section 2.1 describes the

level of construal

. Section 2.2 introduces the

psychological distance

and their related distances, i.e.,

social, spatial, temporal

, and

hypothetical distance.

2.1 Level of Construal

Construal Level Theory (CLT)

is a social-cognitive theory in

social psychology

intro-

duced by [

] describing the effects of

psychological distance

on objects or events. The

fundamental idea is, which is already proved empirically [

], increasing psychological

distance affects the mental representation of objects or events. This influence on the

2 Fundamentals of Construal Level Theory

perception has a strong impact on actions and thinkings of an individual. For example,

moving house to a distant location in a distant future evokes general thoughts and

actions; e.g. starting a new life, searching new friends. The same event happening in a

near location and near future evokes more detailed thoughts and actions; e.g. moving

box packs, register the residence [93].

Strangers, distant locations, past events - everything that is distant from us creates a

more abstract reflection. The reason behind this effect is the

level of construal

. The

level

of construal

describes how individuals interpret and perceive objects in surrounding [

Increasing psychological distance affects cognitive abilities of an individual and, thus,

leads to a change in perception of objects or events.

Therefore, CLT describes two different levels of thinking:

high-level construal

and

low-

level construal

High-level construals

are abstract, coherent, and superordinate repre-

sentations, compared to

low-level construals

. The further away an object or event is the

more we think in high-level construals and, on the other side, the smaller the distance

the more we think in low-level construals. For example, from a distance we see the

forest (i.e., high-level construal) and as we get closer, we see the trees (i.e., low-level

construal) [

]. These two aspects are influenced by

psychological distance

, which is

introduced in the following.

2.2 Psychological Distance

A basic aspect of CLT is the

psychological distance

. While, for example,

objective

distance

describes the quantitative and in real world existing spatial distance of an

object or event to someone, the psychological distance describes feelings, thinkings,

and emotions in relation to an object or event. If an individual shall estimate the distance

between two distant locations, then one location is perceived as further away.

2.2 Psychological Distance

For example, individuals shall estimate the distance between a city and four other cities.

Two of them are in the same federal state and the other two cities are in different federal

states. Distance of the four cities to the marked city is always the same. The results

show that cities in foreign federal states are perceived more distant and are consequently

estimated as further away [13].

An object or event is defined as

psychological distant

, when it is not experienced phys-

ically. Objects or events, which are not experienced in the

here and now

, must be

constructed mentally. Therefore, psychological distance is separated into several subdis-

tances, i.e.,

social, spatial, temporal

, and

hypothetical distance

are being considered as

most important and are explained in the following [47].

Social Distance:

Experiences and decisions which are not self-experienced as well

as the relation to other individuals are

social distant

. For example, choosing a more

distant seat from another individual is taken to reflect social distance [

]. The way

how an individual decides for himself or for others is also affected by social distance.

An example are results of [

]: An individual expects more negative activities from

others than from himself. The results are in accordance with CLT. With increasing social

distance evaluation for distant individuals takes place at a more abstract information

level [90].

Spatial Distance: Spatial distance

refers to objects and events happening at another

physical location. Events that take place at, for example, another country are described

more abstract by individuals as if they occur in the same country. Studies show that

individuals interactions between two or more individuals are described more detailed if it

takes place at a nearby location. On the other hand, descriptions are more abstract if

interactions are spatial distant [27].

Temporal Distance: Temporal distance

deals with events on a temporal perspective.

When an individual thinks about temporal distant objects or events they are perceived

more abstract. Studies have shown, how individuals deal with temporal distance [

]. In

this context, individuals have to categorize several items for an event happening in the

near or distant future. If the event takes place in the near future, more categories are

described in detail. A more distant event results in fewer, course-grained categories [

2 Fundamentals of Construal Level Theory

We retain the possibility of better planning to react against unexpected events in a distant

future. For this reason, our actions are specified more abstract. On the other hand, our

actions must be prepared more detailed for events happening in the near future.

Hypothetical Distance: Hypothetical distance

accrues when an individual thinks about

unreal or unlikely events but also worthwhile or elaborate situations. A study dealing

with hypothetical distance is the following: as part of a contest, several prices are offered

to individuals. These prices are either highly attractive but hard to win or less attractive

but easy to win. It was shown that for highly attractive prices individuals are willing to

take more effort to win. The other way around, for less attractive prices is the effort

correspondingly low [88].

For a better understanding, Figure 2.1 summarizes the concept of CLT and the different

distances (i.e., social, spatial, temporal, and hypothetical distance) up.

With increasing psychological distance perceived objects and events are more abstract

in our mental representation. Although this leads to the fact that our actions and thoughts

are more general but lacks on accuracy. On the contrary, lower psychological distance

wages to a sophisticated but limited scope.

Social

Spatial

Temporal

Hypothetical

?Workmate

Workplace

Tomorrow

Lucrative

Superior

Foreign Office

Elaborate

Psychological

Distance !

Next Year

Figure 2.1: Psychological Distance

Introduction on Gamification, Virtual

World, and 3D Warehouse Scenario

Section 3.1 introduces gamification with its essentials and principles. An introduction

to virtual world is presented in Section 3.2. The implementation and application of

gamification in the context of our experiment as well as the process and progressive

development from concept to realization of the 3D warehouse scenario is presented in

Section 3.3.

3.1 Gamification

Engaged and motivated employees are more productive and more valuable for enter-

prises [

]; e.g., improved operational efficiency, increased quality in work. A variety

3 Introduction on Gamification, Virtual World, and 3D Warehouse Scenario

of methods and possibilities exist to engage employees and boost their productivity

as well as creativity [

]. Currently, a promising trend for increasing motivation

gamification

[

Gamification

is a concept that uses techniques from gaming and

integrates them in a non-game context to engage and motivate people (i.e., employees)

in a rather enjoyable way. Studies have already shown and proven the effectiveness of

gamification in a non-game context [

]. Thereby, the formula for an effective use

of gamification is straight forward: take an existing set of activities (e.g., banking or

administrative work) and apply rather simple and inconspicuous game techniques or

game elements like a set of game rewards in form of points; e.g., leveling or badges.

As a consequence, the work will become automatically less boring and more enjoyable,

thus resulting in higher efficiency [

]. Among existing methods and approaches,

the use of gamification promises new possibilities to engage and increase the motivation

of employees [

]. Hence, more and more enterprises recognize the value and potential

of gamification and move towards a gamified direction for more effectiveness and higher

positive outcomes [

]. Further, process-oriented enterprises make use of gami-

fication and its benefits by identifying new modeling strategies for process modeling;

e.g., translate the enthusiasm for play into workplaces [

]. Aside the use in enterprises,

gamification is becoming widely used in education, politics, and healthcare [

3.2 Virtual World

virtual world

, also known as

virtual reality

, is a computer-based simulated environment,

using the metaphor of the real world but without its physical limitations [

]. Thereby, a

virtual world can reflect the reality or a fantasy world; e.g., a fable world. However, a

virtual world is restricted to the laws and possibilities a developer of a virtual world deter-

mines. Individuals interact in this simulated world as textual, two, or three-dimensional

avatars

(i.e., created character in a virtual world) with each other or the environment, thus

experience a

degree of telepresence

, i.e., experience of presence in a remote location

[

]. Among others, the best-known virtual worlds are the massive multiplayer online

role-playing game

World of Warcraft

and the massive multiplayer universe

Second Life

[

]. Nowadays, the use of virtual worlds has proven its worth and has become an

3.3 3D Warehouse Scenario

important tool in education, entertainment, a laboratory for collaborative work, and an

appropriate instrument to present the real world in a simplified form [

]. Moreover,

the use of virtual worlds has proved to be a effective means for process modeling. Hence,

virtual worlds increase collaboration and consensual development of process models

and provide easy to understand and rich visualizations for process model elicitation, thus

improving the mutual understanding between stakeholders [12].

3.3 3D Warehouse Scenario

For the experiment, we want to make use from the benefits of gamification in a virtual

world in the context of process modeling. Therefore, relative to a real world process,

a simple scenario about an

order processing in a warehouse

is contrived. The entire

process takes place in a full 3D virtual environment with aspects of gamification; e.g.,

exploring (i.e., learn more about the virtual construct) and puzzle elements (i.e., motivate

subjects to solve a problem).

Firstly, by means of using gamification in a virtual world, we want an adequate reflection

of the real world problem; secondly, an increase in the motivation of subjects; and thirdly,

an enhancement of the effects of social distance, thus leading to an increase in the

quality, granularity, and structure of resulting process models.

As described in Section 2.2, experiences and decisions which are not self-experienced

are social distant. In other words, procedures or decisions where one is not actively

participating are, according to CLT, psychological distant, and in this regard social distant.

Hence, the idea evolved to create a manner in which a low as well as high social distance

is experienced. Therefore, to create a low and a high social distance, one group is

actively involved in the process while another group only has a passive position (cf.

Section 4.4). As a consequence, since the passive group has no possibility to intervene

in the process, they are confronted with a higher social distance than the active group

because the procedures and decisions in the scenario are not self-experienced.

3 Introduction on Gamification, Virtual World, and 3D Warehouse Scenario

In the following, the order processing in the warehouse scenario is introduced.

The scenario starts in the office of the warehouse. First of all, an order must be taken

from the table which provides information about what and which items need to be

processed. According the order, the following six items must be processed by the

subjects as shown in Table 3.1.

Item Quantity

Headphone 3

Smartphone 8

Display 2

Playstation 4 Console 3

Drone 2

Supersized Teddy Bear 1

Table 3.1: Items

At the storage racks, the subjects have the choice to get the items either with the forklift

or the automatic picking system. Since the forklift can carry only one pallet at a time the

items must be collected in a sequential order. The automatic picking system comprises

several grappler allowing to collect all items separately or at once. Afterwards, items are

disclosed at the collection point and must be checked for completeness. The next step

is to pack the items in appropriate boxes and, thereafter, the boxes are palletized. After

placing each box on a respective pallet, the subjects have the choice how to transport the

pallets to the shipping area either by using the forklift or the automatic loading system.

As before, the forklift can transport the pallets only sequentially while the automatic

loading system takes care of everything automatically. The distinguish feature of the

automatic loading system is that the subjects can print the required delivery documents

(i.e., bill of delivery and pallet receipts) in parallel. The latter is not possible when using

the forklift. Thereafter, the pallets are labeled with the printed pallet receipts and are

loaded on the trailer with the forklift. At the end, the bill of delivery is placed in the trailer

and doors are closed.

Hereafter, the entire order processing in the warehouse scenario is presented and

summarized as corresponding process model in Figure 3.1 and 3.2.

3.3 3D Warehouse Scenario

Take Order

Check

Available

Picking

Possibilities

Get and

Deposit

Drones

Get and

Deposit

Supersized

Teddy Bear

Use Automatic Picking System

Choose

Picking

Method

Get Everything

at Once

Get

Headphones Get Displays

Get

Supersized

Teddy Bear

Get Drones

Get

Smartphones

Get

Playstation 4

Consoles

Check and

Count

Headphones

Check and

Count

Smartphones

Check and

Count

Displays

Check and

Count

Playstation 4

Consoles

Check and

Count Drones

Check and

Count

Supersized

Teddy Bear

Use Forklift

At Once

Separate

Get and

Deposit

Displays

Get and

Deposit

Headphones

Get and

Deposit

Playstation 4

Consoles

Get and

Deposit

Smartphones

Take Three

Medium Boxes

Take Two

Small Boxes

Take One

Large Box

Packaging of

Headphones

Packaging of

Smartphones

Packaging of

Drones

Packaging of

Supersized

Teddy Bear

Packaging of

Displays

Packaging of

Playstation 4

Consoles

+++ + + +

Figure 3.1: Process Model of the Warehouse Scenario - Part 1

3 Introduction on Gamification, Virtual World, and 3D Warehouse Scenario

Check

Available

Transport

Possibilities

Label Pallet 1

Label Pallet 2

Label Pallet 3

Label Pallet 4

Put Bill of

Delivery in

Trailer and

Close Doors

Palletize

Smartphones

Palletize

Playstation 4

Consoles

Palletize

Headphones

Palletize

Drones

Palletize

Displays

Palletize

Supersized

Teddy Bear

Start Loading

Sequence

Print Bill of

Delivery Print Pallet

Receipts

Transport

Pallet 2 to

Shipping Area

Transport

Pallet 1 to

Shipping Area

Transport

Pallet 4 to

Shipping Area

Print Bill of

Delivery Print Pallet

Receipts

Use Automatic Loading System

Use Forklift

Transport

Pallet 3 to

Shipping Area

Load Pallet 2

into Trailer

Load Pallet 1

into Trailer

Load Pallet 3

into Trailer

Load Pallet 4

into Trailer

++++ + +

Figure 3.2: Process Model of the Warehouse Scenario - Part 2

3.3 3D Warehouse Scenario

As introduced in Section 3.1, a fundamental component of the experiment is to simu-

late a social distance which the subjects can perceive. Therefore, for perceiving a low

social distance, a warehouse scenario in a 3D virtual world environment is developed.

For implementing the respective 3D scenario several game engines are considered;

e.g.,

CryEngine

Unreal Engine

[

]. Finally, we have decided that

Unity

meets all

necessary requirements; e.g., integration with native code, deployment, and documen-

tation.

Unity

is a game development platform including a game engine and integrated

development environment to create interactive 3D and 2D content [

]. Besides a wide

dissemination and a great community, it also offers a wide range of useful extensions.

After some considerations about the type and genre of the warehouse scenario, we have

come to the conclusion that a simple

point and click scenario

is suitable for the experi-

ment. We contend that a point and click scenario is appropriate for subjects because it

can be learned quickly and easily within minutes and without any foreknowledge.

Afterwards, initial concept drawings and documents are made which described the game-

play, features, and appearances. Based on the initial concepts, the first scenario objects

(e.g., pallets, crates) and a rough draft of the warehouse surroundings are created.

Figure 3.3 shows a part of the warehouse surrounding, i.e., storage racks.

Figure 3.3: Storage Racks

Most scenario objects and graphics are designed and created within Unity but the

possibilities are very limited. Therefore, the external tool

Blender

is used for the design

3 Introduction on Gamification, Virtual World, and 3D Warehouse Scenario

and creation of objects and graphics [

Blender

is a free and open-source 3D computer

graphics software for creating animated videos, visual effects, models, and interactive

applications.

For further assistance during the implementation, the extensions

Adventure Creator

and

Playmaker

are applied [

Adventure creator

is a feature-packed 2D and 3D

adventure scenario creation kit which provides useful features and functionalities for

creating a point and click scenario.

Playmaker

adds a powerful visual state machine

editor to Unity for the making of A.I. behaviors, animation graphs, and interactive objects.

The first playable prototype containing only a single scene (i.e., empty warehouse) acted

as a proof-of-concept for testing and adding of features.

For better control, test, and overview a modular programming approach is chosen.

Therefore, the whole implementation is separated into several independent modules and

each module represents an area of the scenario. In total, there are seven modules in the

respective scenario and for each module the necessary aspects (i.e., functions, items,

scenario objects) are implemented separately. Upon completion and successful testing

of each single module, all modules are combined. Table 3.2 and Figure 3.4 are showing

all seven modules and the final layout of the warehouse as well as the chronological

progress through the areas of the 3D scenario.

3 Introduction on Gamification, Virtual Worlds and 3D Warehouse Scenario

Table 3.2: Modules

For further assistance during the implementation, the extensions

Adventure Creator

and

Playmaker

are applied [

Adventure creator

is a feature-packed 2D and

3D adventure game creation kit which provides useful features and functionalities for

creating a point and click game.

Playmaker

adds a powerful visual state machine editor

to Unity for the making of A.I. behaviors, animation graphs and interactive objects. The

first playable prototype containing only a single scene (i.e., empty warehouse) acted as a

proof-of-concept

for testing and adding of features. For better control, test and overview

a modular programming approach is chosen. Therefore, the whole implementation is

separated into several independent modules and each module represents an area of

the scenario. In total, there are seven modules in the scenario and for each module the

necessary aspects (i.e., functions, items, game objects) are implemented separately.

Upon completion and successful testing of each single module, all modules are combined.

The final layout of the warehouse scenario with all seven modules is shown in Table 3.2

and Figure 3.4.

No. Module

1 Office 1

2 Storage Racks

3 Collection Point

4 Packaging Area

5 Palletizing Area

6 Office 2

7 Shipping Area

235

4 6

Figure 3.4: Modules and Layout

Figure 3.4: Layout

Module No.

Office 1 1

Storage Racks 2

Collection Point 3

Packaging Area 4

Palletizing Area 5

Office 2 6

Shipping Area 7

235

4 6

Afterwards, all dialogues and dialogue options are written. Dialogues serve as instruc-

tions for subjects and are pointing out what needs to be done next. For further assistance

and to make navigation easier, static cameras are already pointing in the direction the

subjects need to move for the next step, i.e., field of view is defined and cannot be

changed. For interactions (e.g., take order, use forklift) so-called interactions points are

3.3 3D Warehouse Scenario

implemented and highlighted while hovering with the mouse-cursor and are initiated

by pressing the left mouse button. Additionally, to counteract the language barrier and

other linguistic problems, all texts in the scenario are available in English and German

language. Before the scenario is going gold (i.e., final scenario build), an extensive test

phase is carried out to fix bugs and to remove uncertain instructions.

Figure 3.5 shows the graphical user interface of the final 3D warehouse scenario. The

graphical user interface is kept deliberately simple to avoid potential misunderstandings

and problems. All items collected during gameplay (e.g., order, items, pallets) are stored

and displayed in the inventory bar on the top of the screen (1). Additional accessories

and interior are placed in the warehouse to create and improve an immersive virtual en-

vironment, i.e., created perception in a non-real world (2). The avatar provides subjects

with informations and instructions about what needs to be done next and is controlled

with the mouse (3). As explained above, interactions are performed by clicking on

interaction points (4).

3.3 3D Warehouse Scenario Implementation

implemented and highlighted while hovering with the mouse-cursor and are initiated

by pressing the left mouse button. Additionally, to counteract the language barrier and

other linguistic problems all texts in the scenario are available in English and German

language. Before the game is going gold (i.e., final game build), an extensive test phase

is carried out to fix bugs and to remove uncertain instructions.

Figure 3.5 shows the graphical user interface of the final scenario. The graphical user

interface is kept deliberately simple to avoid potential misunderstandings and problems.

All items collected during gameplay (e.g., order, items, pallets) are stored and displayed

in the inventory bar on the top of the screen (1). Additional accessories and interior are

placed in the warehouse to create and improve an immersive virtual environment, i.e.,

created perception in a non-real world (2). The controllable avatar provides subjects with

informations and instructions about what needs to be done next (3). As explained above,

interactions are performed by clicking on interaction points (4).

Figure 3.5: Graphical User Interface

Experiment Planning and Definition

An experiment is conducted to investigate the effects of

psychological distance

(i.e.,

social distance) on the process of process modeling and resulting artifacts.

The implementation of an experiment is not trivial and requires a proper arrangement in

order to guarantee that data obtained is valid and risks are minimized. Therefore, the

experiment planning and definition strongly considers recommendations given in [

] to

guarantee the validity of the results.

First, the definition of

why

the experiment is carried out is given and thereupon follows

the instruction of how the experiment is performed.

Section 4.1 explains the context of the experiment and define its goal. Section 4.2

introduces the context selection. The hypotheses considered for testing are introduced

in Section 4.3. Experimental setup is described in Section 4.4. The experiment design is

explained in Section 4.5. Section 4.6 discusses factors threatening the validity of results.

4 Experiment Planning and Definition

Figure 4.1 gives an overview on the structure of this section.

Goal

Definition

Section 4.1

Context

Selection

Section 4.2

Hypotheses

Formulation

Section 4.3

Experiment

Setup

Section 4.4

Experiment

Design

Section 4.5

Validity

Evaluation

Section 4.6

Figure 4.1: Experiment Planning and Definition

4.1 Goal Definition

In enterprises, process models are either created by in-house teams or external consul-

tants. Respective process designers are responsible for interviewing process participants

and capturing gathered knowledge in process models. However, these process design-

ers are often not directly involved in the business process to be documented; e.g.,

they may be a staff member of the quality assurance department. In other cases, due

to limited resources, enterprises assign such modeling tasks to external consultants

[

]. For want of information, the risk involved here is that resulting process models

may differ a lot and the documentation of the real world business process is flawed and

inadequate. However, it is not well understood and still unclear how such an increased

psychological distance, in our context the social distance, affects the quality, granularity,

and structure of process models. To close this gap, this thesis investigates the following

fundamental research question:

Is the process of process modeling, i.e., the quality, granularity, and structure of

process models resulting from it, affected by the social distance process designers

have on respective business processes?

4.1 Goal Definition

Despite the large number of research on process model quality [

granularity [

], and structure [

] hardly research considering the cognitive

aspects in the context of process modeling exist [

]. In particular, it is not

well understood how and if cognitive aspects actually lead to minor process model quality

(i.e., deficiencies regarding syntactic, semantic, pragmatic, and perceived quality), and

how they impact granularity and structure of process models. Until now, it has been

shown that there exists a correlation between the psychological distance (i.e., social,

spatial, temporal, and hypothetical distance) and their influence on the resulting process

models [

]. Continuing the previous experiment, the emphasis is one aspect of

the psychological distance: the

social distance

. According to CLT, however, existing

social distance

to objects influences the way we act and, therefore, presumably also

impacts the way how process models are created. Based on a controlled software

experiment, this thesis investigates the influence social distance has on the

process of

process modeling

and its

outcomes

. The experiment varies social distance with the use

of gamification in a 3D virtual world to learn whether social distance has any influence

on the quality and granularity as well as structure of the resulting process models.

A proper experiment definition in information system research ensures a safe implemen-

tation of an experiment and minimizes or even eliminates potential risks threatening the

experiment. As a starting point, for goal definition of the experiment, we use the

Goal

Quality Metric (GQM) proposed in [5] defined as follows:

Object of Study:

The

objects of study

are resulting process models created by subjects

of the experiment while confronting different social distances.

Purpose:

The

purpose

of the experiment is to evaluate the resulting process models

with respect to the influence of social distance on the process of process modeling.

Quality Focus:

The main effect studied in the experiment is the

level of construal

. To

measure the level of construal the focus is set on the

quality, granularity

, and

structure

of each resulting process model.

Perspective:

The

perspective

is set from the point of view of researchers. We want to

find out if there are any differences in the process models confronting different social

distances.

4 Experiment Planning and Definition

Context:

The experiment is conducted at the Institute of Databases and Information

Systems at Ulm University. Students and research assistants with basic and advanced

knowledge in process modeling are invited. The study is conducted as a

single object

study

and can be judged as being a

randomized, blocked, and balanced single factor

experiment [95] (cf. Section 4.5).

The focus is set to the measurement of the level of construal of each process model and

is defined in Table 4.1 as goal definition template:

Analyze process models

for the purpose of evaluating

with respect to their level of construal

from the point of view of researchers

in the context of students and research assistants.

Table 4.1: Goal Definition Template

4.2 Context Selection

Obviously, the most significant results of an experiment are achieved in a

practical

environment

with trained and professional employees. However, it is not reasonable

to perform an experiment in a practical environment. A practical environment involves

unsuspected risks and, therefore, it is advisable to perform an experiment in a

controlled

environment

, which is comparable to a practical environment, i.e., enclosure in which

measures are taken to provide an environment that meets certain requirements. On

the one hand, this option reduces the risks of an experiment and, on the other hand, it

reduces the emerging costs performing an experiment [95].

Our experiment participate students and research assistants in a controlled environment

and, hence, is run

off-line

, i.e., not in a practical environment. Therefore, not much effort

is needed for creating and defining the environment in which the experiment is run. The

experiment provides an insight to the research question (cf. Section 4.1) and, thus, may

serve as a foundation for further experiments. In addition, the results can be transferred

to a practical environment and the experimental context provides other researchers with

excellent opportunities to replicate the experiment.

4.3 Hypotheses Formulation

The

hypothesis

describes in concrete terms what are the intentions of an experiment.

Therefore, a hypothesis has to be clearly and unambiguously stated. In this context, two

types of hypotheses have to be formulated: null hypothesis and alternative hypothesis.

Null Hypothesis H0

describes the assumption that no effects or differences between

an old (

µold

, i.e., expected value of the old approach) and new approach (

µnew

, i.e.,

expected value of the new approach) exist in the experimental setting. Initially, the null

hypothesis is assumed to be true and the experiment tries to reject or disprove it. The

only reason for differences in observations are coincidental, i.e., H0:µold =µnew

Alternative Hypothesis H1

is exactly the opposite of the null hypothesis and describes

the existence of an association between research question and obtained experimental

results. It is typically what the researcher wants to show, i.e., H1:µold < µnew

Based on the goal of our experiment (cf. Section 4.1), the following hypotheses are

derived: The experiment investigates whether

social distance

(i.e., low (

µ1

) and high

(

µ2

)) influences the

level of construal

during the

process of process modeling

and, thus,

the quality, granularity, and structure of the corresponding process model. In total, we

have derived seven hypotheses: four referring to the quality dimensions (i.e.,

syntactic,

semantic, perceived

, and

pragmatic quality

), one referring to the

level of granularity

, one

referring to the process model structure, and one referring to additional factors:

Does social distance lead to an increase of the

syntactic quality when modeling a process?

H0,1

: There are no significant differences in the syntactic quality when

modeling processes with low social distance. H0,1:µ2=µ1

H1,1

: There are significant differences in the syntactic quality when

modeling processes with low social distance. H1,1:µ2< µ1

4 Experiment Planning and Definition

Does social distance lead to an increase of the

semantic quality when modeling a process?

H0,2

: There are no significant differences in the semantic quality when

modeling processes with low social distance. H0,2:µ2=µ1

H1,2

: There are significant differences in the semantic quality when

modeling processes with low social distance. H1,2:µ2< µ1

Does social distance lead to an increase of the

perceived quality when modeling a process?

H0,3

: There are no significant differences in the perceived quality when

modeling processes with low social distance. H0,3:µ2=µ1

H1,3

: There are significant differences in the perceived quality when

modeling processes with low social distance. H1,3:µ2< µ1

Does social distance lead to an increase of the

pragmatic quality when modeling a process?

H0,4

: There are no significant differences in the pragmatic quality when

modeling processes with low social distance. H0,4:µ2=µ1

H1,4

: There are significant differences in the pragmatic quality when

modeling processes with low social distance. H1,4:µ2< µ1

Does social distance lead to an increase of the

level of granularity when modeling a process?

H0,5

: There are no significant differences in the level of granularity

when modeling processes with low social distance. H0,5:µ2=µ1

H1,5

: There are significant differences in the level of granularity when

modeling processes with low social distance. H1,5:µ2< µ1

Does social distance lead to an increase of the

process model structure when modeling a process?

H0,6

: There are no significant differences in the process model struc-

ture when modeling processes with low social distance.

H0,6

µ2=µ1

H1,6

: There are significant differences in the process model structure

when modeling processes with low social distance. H1,6:µ2< µ1

4.4 Experiment Setup

Does social distance lead to an increase of the

additional factors when modeling a process?

H0,7

: There are no significant differences in the additional factors when

modeling processes with low social distance. H0,7:µ2=µ1

H1,7

: There are significant differences in the additional factors when

modeling processes with low social distance. H1,7:µ2< µ1

Additionally to introduced hypotheses, the following factors need to be considered when

planning an experiment, i.e., type-I-error, type-II-error, and power.

Type-I-Error

: Rejecting the null hypothesis although it is in fact true is called a

type-I-

error. P(type-I-error) = P(reject H0|H0is true)

Type-II-Error: The second type of error is not to reject the null hypothesis although the

alternative hypothesis is true. This kind of error is called a type-II-error.

P(type-II-error) = P(not reject H0|H0is false)

Power

: The

power

of a statistical test is used to indicate the probability of rejecting the

null hypothesis and the alternative hypothesis is true.

Power = P(reject H0|H0is false) = 1 - P(type-II-error)

Therefore, it is generally advisable to choose a statistical test with a high power as

possible to reduce the probability of a wrong assumption (cf. Section 6.3).

4.4 Experiment Setup

Based on the goal definition template (cf. Table 4.1), this section describes

subjects

objects,factor and factor levels, and response variables of our experiment.

Selection of Subjects:

Since it is not possible to evaluate the entire desired population

it is required to select a representative sample group. This enables to reason for the

whole population. A sample group is also known as a defined collection of

subjects

(i.e.,

participants in an experiment) with similar properties [23]. Ideally, process designers in

enterprises are modeling experts. Typically they only obtain basic training and, hence,

4 Experiment Planning and Definition

have limited process modeling skills [96].

Therefore, the selected subjects are students and research assistants. Any student and

research assistant with basic and advanced knowledge of process modeling in general

and about Business Process Model and Notation (BPMN) is able to participate.

Selection of Objects:

After selecting subjects, the

objects of the study

have to be

selected. The objects are the entities that are studied in the experiment.

The object of study are the resulting process models of each subject using process

modeling language

Business Process Model and Notation 2.0 (BPMN 2.0)

[

] (cf.

Section 4.1). To ensure familiarity and competence of subjects and to ensure that

differences in quality, granularity, and structure of process models are not caused due to

a lack of familiarity, but rather due to differences in social distance, we choose an easy

and understandable scenario, i.e., order processing in a warehouse (cf. Section 3.3).

We created task descriptions in two versions reflecting different social distances (cf.

Appendix A). One group is directly involved in the process while the other group is only

indirectly involved. More precise, subjects dealing with low social distance are playing the

warehouse scenario and, on the other hand, subjects dealing with high social distance

are watching the warehouse scenario in a video. The description of the task is rather

abstract and short to give subjects the possibility to model it as detailed as they like.

Factor and Factor Levels: Factor

and

factor levels

of an experiment are an important

consideration since they manipulate and control effects in the experiment. Generally, a

factor is divided into two or more factor levels and these factor levels have an influence

on the response variables.

The factor considered in our experiment is the

social distance

. Factor levels are

low

social distance

and

high social distance

and can be manipulated and controlled by

varying the distance: i.e., modeling the order processing in a warehouse either based on

the playable 3D scenario (i.e., low social distance) or video (i.e., high social distance).

As described in Section 3.3, in the context of our experiment, aspects of gamification

in a 3D virtual world are used to emerge a low social distance. Therefore, in order to

ensure the effects of social distance (cf. Section 2.2), subjects have full control and

freedom of choice about the process and decisions. Subjects dealing with high social

distance are watching the warehouse scenario in a video. Therefore, they have no

4.4 Experiment Setup

possibility of intervening and need to adopt a passive position. In our view, a passive

participation without the option of intervening will emerge a high social distance as

opposed to an active participation. Unlike the playable scenario, subjects only perceive

one way through the scenario and therefore have no freedom of choice. Hence, in

the warehouse scenario there are two main decisions, i.e., forklift or automatic picking

system and forklift or automatic loading system (cf. Section 3.3). Thereby, in the video

only the same decisions are used. For getting the ordered items from the storage racks

the forklift is used and for transporting the palletized items to the shipping area the

automatic loading system is used. The other decisions are mentioned but never used.

Response Variables: Response variables

can only be measured or observed and

must depend on the factor levels. A change in the factor levels lead to a change in

the response variables. The choice of response variables determine the measurement

scales and range of variables (i.e., categorical or continuous) and are used later for

evaluation. The choice of the appropriate statistical test depends on the basis of the

chosen measurement scales and range of variables.

As response variable, we consider the

level of construal

which cannot be directly mea-

sured. Considering the level of construal, everything being distant from us is created

more abstract (cf. Section 2.1). Hence, we assume that the level of construal impacts

quality, granularity, and structure of the resulting process models. Therefore, statistical

methods as well as an established analysis framework to measure the quality of process

models are applied [

]. Additionally, we identify the level of granularity and analyze the

process model structure using process metrics proposed in [56].

We assume that high social distance may lead to course-grained, more abstract, and

imprecise process models (i.e., reflecting a low process model quality, level of granularity,

and process model structure), while low social distance may result in more fine-grained

and precise process models (i.e., reflecting a high process model quality, level of granu-

larity, and process model structure). An adapted

semiotic theory framework

is used to

determine

process model quality

[

]. The latter is characterized by the following four

dimensions: syntactic, semantic, pragmatic, and perceived quality.

4 Experiment Planning and Definition

The

syntactic quality

of a process model is measured by counting the number of syntac-

tical errors (i.e., syntactical rule violations) of the applied modeling language, i.e., BPMN

2.0 [66].

The

semantic quality

is subdivided into the aspects

correctness, completeness, rele-

vance

, and

authenticity

of a process model.

Correctness

expresses that all elements

in the process model are correct and relevant to the business process.

Completeness

implies that no correct and relevant elements are missing in the final process model.

Further,

relevance

connotes that all elements in the process model are relevant for

the business process. In contrast to completeness, superfluous elements are also

considered. Finally,

authenticity

expresses that the chosen representation gives a true

account of the domain. In general, semantic quality is determined on a 7-point Likert

scale ranging from strongly disagree (0) to strongly agree (6).

The

pragmatic quality

describes the process model comprehension and is measured by

the

level of understanding

. Therefore, a 7-point Likert scale ranging from very hard to

understand (0) to very easy to understand (6) is applied.

Finally,

perceived quality

depends on the degree to which a subject agrees with the

resulting process model [

]. Therefore, the following questions are used as proposed

in [77]:

1. Does the final process model agree with your view of business process?

2. Are there significant aspects that are missing in the final process model?

3. Does the final process model describe the business process accurately?

4. Are there any serious mistakes in the final process model?

5. Would you have done the final process model in a different way?

Derived from the questions above, perceived quality can be further subdivided into

agreement, missing aspects, accurate description, mistakes

, and

satisfaction

Agree-

ment

expresses to which degree the process model matches with the real world business

process. Missing aspects rates whether significant aspects are missing in the resulting

process model. In turn,

accurate description

expresses how accurate the process model

matches the real world business process.

Mistakes

corresponds to the subject rate

4.4 Experiment Setup

indicating whether there are serious mistakes in the resulting process model. Finally,

satisfaction

expresses the degree of satisfaction a subject has with his process model.

The questions are put after the modeling task to score each question on a 5-point Likert

scale ranging from strongly disagree (0) to strongly agree (4).

Level of granularity

is measured by the complexity of the resulting process models.

Therefore, the

number of activities, gateways, nodes, edges

, and

elements

as well as

the number of possible execution paths are counted.

Process model structure

is analyzed with a set of process metrics presented in [

]. These process metrics consists of four determinants, i.e.,

separability, sequentiality,

cyclicity

, and

diameter

. Therefore, we consider a process model to be a kind of graph G

= (N,E) with at least three node types

N=T∪S∪J

, i.e.,

tasks T

splits S

joins J

, and

edges E⊆N×N.

Separability: Π

is defined as the ratio of the number of cut-vertices (i.e., a node whose

deletion separates the process model into multiple components) to the total number

of nodes in the process model. An increase in

Π(G)

should imply a decrease in error

probability of the process model [56].

Π(G) = |n∈N|n is cut −vertex|

|N| − 2

Sequentiality:

The degree to which the model is constructed of pure sequence tasks.

Sequentiality

relates edges of a sequence to the total number of edges in a process

model. An increase in

Ξ(G)

should imply a decrease in error probability of the process

model [56].

Ξ(G) = |e∈E|e∈(T×T)|

|E|

4 Experiment Planning and Definition

Cyclicity: |NC|

gives the number of nodes on cycle and cyclicity

CY CG

relates it to

the total number of nodes in a process model. An increase in

CY CG

should imply an

increase in error probability of the process model [56].

CY CG=|NC|

|N|

Diameter:

Describes the length of the longest path from a start node to an end node in

the process model [75].

Moreover,

number of modeling steps

and

modeling duration

are taken into consideration.

Further, we determine

mental effort

for creating a process model and

naming

of each

process model activity. Mental effort is rated on a 7-point Likert scale ranging from

extreme low mental effort (0) to extreme high mental effort (6). Activity naming is

rated on a 3-point Likert scale ranging from normal detailed (0) to complex detailed (2).

Therefore, we consider each label of an activity of the process models and evaluate the

level of detail.

Summarizing all the above, each process model of the subjects is reviewed for their

respective quality level, level of granularity, structure, and the additional factors. These

criteria are used later to determine whether the two different social distances (i.e., low

and high) have an influence on the process of process modeling. Figure 4.2 provides a

brief overview of the factor, factor levels, and variables in a research model.

Psychological Distance

F: Syntactic Quality

O: No. of Syntactical Errors

Quality

F: Theoretical Factor

O: Operationalization of Factor

F: Semantic Quality

O: Correctness

Authenticity

Relevance

Completeness

F: Perceived Quality

O: Agreement

Missing Aspects

Accurate Description

Mistakes

Satisfaction

Granularity

F: Granularity

O: No. of Activities

No. of Gateways

No. of Nodes

No. of Edges

No. of Elements

No. of Execution Paths

F: Social Distance

O: Level of Social Distance Structure

F: Structure

O: Sequentiality

Separability

Cyclicity

Diameter

F: Pragmatic Quality

O: Level of Understanding

Additional Factors

F: Additional Factors

O: No. of Modeling Steps

Modeling Duration

Naming

Mental Effort

Legend:

Figure 4.2: Research Model

4.5 Experiment Design

After formulation of hypotheses and definition of the experimental setup an appropriate

experiment design has to be determined. An experiment design describes the structure

and progress of an experiment. The selection of an unsuitable experiment design could

cause erroneous data or lead to a failure of the experiment. There are three general

principles that must be guaranteed for a correct experiment design.

Randomization

is a principle based on chance by which subjects are

assigned. By randomization an uniform distribution between subject groups can be

achieved. In our experiment, we assign each subject into one of two groups, i.e., low or

high social distance. Assigning subjects to groups, randomization is used.

Blocking

: In each experiment, undesired effects may occur that probably have an effect

on subjects. Hence, if there are no interests in these effects, a principle called blocking

can be used. Therefore, the subjects are grouped into two blocks, i.e., one dealing with

low social distance and one dealing with high social distance.

Balancing

: When investigating differences between two subject groups, it is desired

to use a balanced design, i.e., each group has an equal number of subjects. Thus, we

avoid imbalance between groups for each social distance in our experiment.

Summarizing, we use a

randomized, blocked

, and

balanced single factor experiment

Further, the study is characterized as a

single object study

in terms of the number of

subjects and objects. In particular, among all subjects, a single object study is conducted

on a single subject and a single object [

]. Figure 4.3 illustrates the chosen experiment

design.

Scenario

(Low Social

Distance)

n Subjects Factor

Subject 1

Subject n/2

Task

Subject

Group 1

Process Model

Object

Video

(High Social

Distance)

Subject n/2

Subject n Process Model

Subject

Group 2

Subject n/2+1

Figure 4.3: Experiment Design

4 Experiment Planning and Definition

Instrumentation:

For measuring of response variables it is essential to apply an ade-

quate

instrumentation

to guarantee that collected and analyzed data is valid. Obviously,

instrumentations shall not influence the outcome of the experiment, but rather provide

means for performing and to monitor it, without affecting the control of the experiment.

For the 3D warehouse scenario (i.e. low social distance), the respective application built

in Unity is used (cf. Section 3.3). Hence, the entire scenario is realized in this application.

After the respective scenario, the latter shuts itself down and the subjects continue to

use the provided experimental platform.

For the other part of the experiment (i.e., high social distance), a video of the warehouse

scenario is captured. For video capturing the tool

ActivePresenter

is used [

ActiveP-

resenter

is a desktop-camcorder for capturing videos or creating presentations and

screencasts. To capture the respective scenario in a video, the scenario is played while

the tool is recording in the background. Likewise in the playable variant, to counteract

the language barrier and other linguistic problems, the video is recorded in English and

German language. Further, to ensure that subjects catch every action and text, the play

of the scenario is slowed down and sufficient pauses are integrated. For video playback

any media player is suitable. Therefore, we use the free and open-source

VLC Media

Player [92].

Afterwards, the

Cheetah Experimental Platform (CEP)

is used by both groups [

]. CEP

is developed to foster experimental research on business process modeling. CEP allows

creating process models as well as integrating questionnaires. In particular, CEP is able

to record every modeling step, i.e., timestamps, type of modeling action, and duration.

Before modeling any task, a

questionnaire

(cf. Table 4.2) must be filled out by subjects to

characterize them and the individual skill levels. Subsequently, a second

questionnaire

for the purpose of gathering information about the subjects gaming experience needs to

be answered (cf. Table 4.3).

The subjects use the modeling environment of CEP to resolve the modeling task of the

warehouse scenario. All modeling actions plus modeling duration and needed modeling

steps are logged and stored separately in a database. After modeling the task, subjects

which played the scenario need to answer which decisions they made while playing the

respective scenario, i.e., choosing forklift or automation systems (i.e., automatic picking

4.5 Experiment Design

system and automatic loading system) (cf. Table 4.4). For evaluation of experimental

results we use the admin environment of CEP with assistance of a self-developed evalu-

ation sheet (cf. Appendix A). The individual evaluation points are described in Section

4.4.

Question Possible Answers

Which description matches best your current work status? Student, Academic, Professional

What is your gender? Male, Female, Other

Course of studies User-Defined Text

Overall, I am very familiar with the BPMN. 7-point Likert scale1

I feel very confident in understanding process models created with the BPMN. 7-point Likert scale1

I feel very competent in using the BPMN for process modeling. 7-point Likert scale1

How many years ago did you start process modeling? User-Defined Text

How many process models have you analyzed or read within the last 12 months User-Defined Text

How many process models have you created or edited within the last 12 months? User-Defined Text

How many activities did all these models have on average? User-Defined Text

How many work days of formal training on process modeling User-Defined Text

have you received within the last 12 months?

How many work days of self education have you made within the last 12 months? User-Defined Text

How many months ago did you start using BPMN? User-Defined Text

Table 4.2: Demographic Questionnaire

Question Possible Answers

Are you familiar with video games? 7-point Likert scale1

Which platform do you prefer for playing video games? Platform2

What is your favorite video game genre? Genre3

How many hours per week do you spend playing video games? User-Defined Text

Table 4.3: Game Questionnaire

Question Possible Answers

At the first choice, have you chosen the Picking System or the Forklift? Picking System, Forklift

(If you watched the video, please choose the Forklift)

At the second choice, have you chosen the Loading System or the Forklift? Loading System, Forklift

(If you watched the video, please choose the Loading System)

Table 4.4: After Game Questionnaire

1Strongly Agree, Agree, Somewhat Agree, Neutral, Somewhat Disagree, Disagree, Strongly Disagree

2Computer, Nintendo, Playstation, Xbox, Phone or Tablet, Other

3Action, Action-Adventure, Adventure, Role-Playing, Simulation, Strategy, Sports, Other

4 Experiment Planning and Definition

4.6 Risk Analysis and Mitigations

In an experiment, certain adverse factors have to be taken into account. These factors

may affect the results of the experiment. Therefore, it is important to pose the question,

how valid are the obtained results?

In the experiment, we have four levels of validity on social distance to consider:

conclu-

sion validity

(Is there a relationship between the treatment and the outcome?),

internal

validity

(Are the effects caused by the treatment?),

construct validity

(Does the opera-

tional definition reflect the theoretical meaning?), and

external validity

(Can the results

be generalized?). The four levels of validity are presented in Figure 4.4 [14].

Cause

Construct Effect

Construct

Treatment Outcome

Theory

Observations

Experiment Objective

Experiment Operation

Response Variable

Factor & Factor Levels

Cause-Effect

Construct

Treatment-Outcome

Construct

External

Conclusion

Internal

Construct

Figure 4.4: Levels of Validity

Threats to conclusion validity are:

The major threat regarding the conclusion validity is the use of a statistical test with

low power. The power of statistical tests is the probability that correctly rejects the null

hypothesis and reveals a true pattern in the data (cf. Section 4.3). Therefore, we use a

low

significance level (i.e., p-value (p))

between 0.01

≤

0.05. This helps to ensure

a strong presumption against the null hypothesis.

Another threat concerning the conclusion validity is the violated assumption of statistical

tests. Specific statistical tests have certain requirements which must be met for suc-

cessful testing; e.g., gaussian distribution, independent sample. In order to avoid the

4.6 Risk Analysis and Mitigations

violation and if assumptions seem uncertain we use the

Wilcoxon-Mann-Whitney-U-Test

(cf. Section 6.3).

Finally, a standardized application and procedure of the treatment is another important

point that needs to be considered. Applying different procedures of the treatment be-

tween different appointments may affect the subjects positively or negatively. Hence, a

standardized procedure is applied to subjects of all appointments (cf. Section 5.2).

Threats to internal validity are:

The selected modeling task is one of the most critical threats for internal validity. A

completely unknown and hard to understand scenario may lead to a strong lack of

familiarity. To ensure familiarity of subjects and to ensure differences in quality, granularity,

and structure are not caused due to a lack of familiarity, but rather due to social distance,

we choose an intuitive scenario, i.e., order processing in a warehouse (cf. Section 3.3).

Badly designed instrumentation (e.g., forms, documents) may affect the experiment

as well. Therefore, good practices and approved methods from literature and previous

experiment are used to minimize respective effects (cf. Section 4.5).

Threats that might influence the modeling outcome include process modeling experience

of the subjects involved and uneven distributions of subjects over two groups. With a

sufficiently large group, however, the scope of experience varies. Furthermore, data

validation ensures that in both groups the subjects are at least moderately familiar

with process modeling (cf. Section 5.3). With randomization, an even distribution is

guaranteed and, thus, it is assured that familiarity with process modeling is approximately

the same in both groups (cf. Section 4.5). Particularly, this prevents faulty models due to

lack of domain knowledge.

Another threat for internal validity is the process of maturation. To ensure that subjects are

not negatively influenced by tiredness, boredom, or hunger, the experiment is conducted

at a time of the day for which the mentioned frame of mind can be excluded. Furthermore,

expected duration of the experiment is about 60 minutes. About 30 minutes are required

to play or watch the warehouse scenario and another 30 minutes are required for process

modeling and answering questionnaires. The composition of those two parts should

prevent faulty models due to lack of motivation. All subjects are recruited on a voluntary

basis combined with the prospect of a bonus (cf. Section 4.4).

4 Experiment Planning and Definition

Threats to construct validity are:

A major threat for construct validity is the inadequate preoperational explication of

constructs. A not sufficiently defined preoperational definition of the experiment results in

a poorly experiment execution and obtained results are invalid. For this purpose, planning

and definition strongly considers recommendations given in [

] to guarantee the validity.

Furthermore, from previous experiments a proper and approved preoperational design

is used [41, 99].

Another threat regarding construct validity is the interaction of testing and treatment. The

application of specific treatments (e.g., measuring the number of syntactical errors) may

make subjects more sensitive and raises awareness on specific parts of the treatment.

Therefore, instructions are rather abstract and simple to disclose no information about

measured response variables.

Threats to external validity are:

A high threat to the external validity is involving students and research assistants instead

of professionals, which might limit generalizability of results. However, subjects rather

have profound knowledge in process modeling and experiments have shown that such

kind of results are transferable to professionals [

]. Hence, we may consider them as

proxies for professionals who have only obtained basic training.

Another threat is the resulting quality of process models. The quality of resulting process

models always depends on quality of applied instrumentation. To mitigate this threat we

use an up-to-date tool and modeling language, i.e., BPMN 2.0 and CEP (cf. Section

4.5).

Finally, a potential threat to external validity is the measuring of social distance with one

modeling task. However, previous experiment has shown that psychological distance

has an impact on the process of process modeling [

]. Further, to mitigate and to

improve generalizability, experiments in different environments or conditions one may

conduct.

Experiment Operation

Section 5 summarizes the experiment operation (cf. Figure 5.1). Generally, the op-

erational phase of an experiment consists of three steps:

preparation, execution

, and

data validation

. Therefore, Section 5.1 describes all necessary arrangements of exper-

iment preparation. The progress of experiment execution is discussed in Section 5.2.

Section 5.3 provides data validation and presents insights of collected data during the

experiment.

Section 5.1 Section 5.2 Section 5.3

Experiment

Preparation Experiment

Execution Data

Validation

Figure 5.1: Experiment Operation

5 Experiment Operation

5.1 Experiment Preparation

As subjects for the experiment, students and research assistants with basic and ad-

vanced knowledge in process modeling are invited to join the experiment. A majority of

the subjects are participants of the

Business Process Management (BPM)

course. The

teaching objective of BPM is to introduce the principles and methods of business process

management, i.e., initial insights into process modeling and analysis. For motivation and

attraction to seriously take part in the experiment, subjects are offered bonus points for

the BPM exercise lesson. Other subjects are participants on a voluntary basis or out of

pure interest. However, none of the subjects are aware of the intention of the experiment.

They only know that they take part in an experiment in the context of a thesis about

process modeling. All subjects are guaranteed anonymity and discrete treatment of

obtained data.

Before conducting the experiment pilot studies are performed. For each distance (i.e.,

low and high social distance) two pilot studies are performed whose results are used

to eliminate ambiguities and misunderstandings as well as to improve task description

and to optimize the 3D warehouse scenario and the video. Additionally, it is checked

whether social distance between the modeling tasks is sufficiently large and perceived

adequately.

Further, an evaluation sheet (cf. Appendix A) is created to assess the level of con-

strual by analyzing the quality, granularity, and structure of resulting process models (cf.

Section 4.4). In order to perform the experiment, CEP is configured and provided for

all emerging data (cf. Section 4.5). The entire process of the experiment is planned

within CEP (cf. Section 5.2). In CEP, relative to BPMN, it is defined when and in which

sequence questionnaires and tasks (i.e., modeling tasks) appear. Changes can be made

quickly and easily by editing the correlate activity. In addition, a database is established

in which all emerging data is stored.

5.2 Experiment Execution

The experiment takes place in the computer lab of the Institute of Databases and In-

formation Systems at Ulm University. Due to the spatial limitation of this computer lab

only 10 subjects can participate the experiment at the same time. Therefore, several

appointments within a period of two weeks are offered to the subjects. Each experiment

session lasts about 60 minutes and is assigned to one distance, i.e., low or high social

distance. Each appointment is based on the following procedure:

At the beginning, an introduction to the experiment is offered and explains the pro-

cedure of the experiment. Further, questions arising from the subjects are answered.

Afterwards, worksheets with task description (cf. Appendix A) are handed out as well as

a blank piece of paper for notes in the execution. Then, subjects start playing or watching

the warehouse scenario. Afterwards, subjects are requested to fill out the questionnaire

capturing their actual modeling experience (cf. Table 4.2). Subsequently, subjects need

to answer another questionnaire on issues relating to their gaming experience (cf. Table

4.3). Finally, subjects are asked documenting the warehouse scenario based on their

own experience and in a way they think it is performed. After finishing the modeling task,

subjects assigned to low social distance (i.e., playing the scenario) need to indicate their

choices which they have made during play, i.e., choosing forklift or automation systems

(cf. Table 4.4). Then, all subjects fill out additional questions concerning perceived

quality (cf. Section 4.4). At the end, subjects are able to provide feedback. All results

(i.e., questionnaires, created process models) are collected with CEP and stored in a

established database.

5 Experiment Operation

5.3 Data Validation

After performing the experiment, data is collected from 95 subjects in 19 sessions.

Fortunately, only data from one subject is removed due to the following reason:

•

The resulting process model is very flawed and questionable, i.e., the process

model differ substantially from the task description. This may have negatively

affected the data.

After removing, data of 94 subjects are considered in data analysis (cf. Section 6).

The subjects consists of 84 students and 10 research assistants. The allocation of the

students is presented in Table 5.1.

Background Number

Computer Science (CS) 13

Media Computer Science (MCS) 6

Software Engineering (SE) 1

Economics (Eco) 63

Business Mathematic (BM) 1

Table 5.1: Allocation of Students

33 of them are female and 61 are male (cf. Figure 5.2). All subjects have stated that they

are already experienced in BPMN. Therefore, we screened the subjects for familiarity

with BPMN, since our research setup requires subjects to be familiar with BPMN. On a

7-point Likert scale the median value for familiarity with BPMN is 3, i.e., above average.

For confidence in understanding BPMN process models, a median value of 3 is obtained.

Perceived competence in creating BPMN models has a median value of 3. Prior to the

experiment, subjects analyzed 19 process models and created 7 in average. Since all

values range above average and subjects are familiar with process modeling, we may

conclude that the participating subjects fit to the targeted profile. The full data set can

be found in Appendix B.

5.3 Data Validation

Scenario Video

(a) Distribution

Academic Student

(b) Profession

CS MCS SE Eco BM

Male Female

(d) Gender

Figure 5.2: Demographic Questionnaire

Concerning gaming experience, based on a 7-point Likert scale ranging from strongly

disagree (0) to strongly agree (6) familiarity with video games from both groups is above

the average (median value of 5 for low and 4 for high social distance). The favorite video

game platform is a computer system and the preferred video game genres are action for

low and sports for high social distance. Subjects with low social distance spend 4.06

hours per week playing video games while subjects with high social distance spend 2.28

hours per week playing video games. As a result of the video game questionnaire, we

also conclude that participating subjects fit to the targeted profile. The full data set can

be found in Appendix C.

Experiment Analysis and Interpretation

Section 6 describes the

statistical analysis

and

interpretation

of the experiment. Section

6.1 characterizes obtained data with assistance of visualization. Section 6.2 deals with

data set reduction and Section 6.3 tests the hypotheses for validity. Finally, Section 6.4

provides a summary and discussion about the results (cf. Figure 6.1).

Section 6.1 Section 6.2 Section 6.3

Raw Data

Descriptive

Statistics

Data Set

Reduction Hypothesis

Testing

Section 6.4

Summary

Discussion

Figure 6.1: Experiment Analysis and Interpretation

6 Experiment Analysis and Interpretation

6.1 Analysis of Raw Data and Descriptive Statistics

Descriptive statistics visualizes collected data as tables or graphics to provide a better

comprehension and to gain first impressions of how data is distributed, concentrated, or

spread out. Particularly worth mentioning is that descriptive statistics gives no decisions

about the validity of the results and serves only as a provider of informations for a better

understanding of the data.

The following tables show median values (i.e., middle value separating the lower half of

the data from the higher half of the data) from collected data (cf. Appendix D). Therefore,

each table consists of three classes (i.e. low, high, total) and each class represents

median values for low and high social distance as well as the median value of both

groups together. Furthermore, obtained results are visualized as box plots, i.e., low and

high social distance. Box plots span a distance between the 25% percentile (i.e., values

that are lower than the median) and the 75 % percentile (i.e., values that are greater than

the median). Hence, the line in the box plot represents the median. Straight lines outside

a box plot are so-called whiskers representing data not within the span of percentile.

The end of the whisker represents the smallest and largest data set.

Table 6.1 and Figure 6.2 present the results of

syntactic

(i.e., number of syntactical

error),

pragmatic

(i.e., level of understanding), and

semantic quality

(i.e., validity and

completeness of process models).

Syntactic Pragmatic Semantic

Group No. Errors Understanding Correctness Relevance Completeness Authenticity

Low 2 4 4 4 3 3

High 3 3 5 5 4 4

Total 3 4 4 4 3 3.5

Table 6.1: Syntactic, Pragmatic, and Semantic Quality

6.1 Analysis of Raw Data and Descriptive Statistics

Low High Low High Low High Low HighLow High Low High

No. Errors

Syntactic Understanding

Pragmatic Correctness Relevance Completeness Authenticity

Semantic

8 8

Figure 6.2: Syntactic, Pragmatic, and Semantic Quality

As shown in Table 6.1 and Figure 6.2, subjects with low social distance make less

syntactical errors (median of 2) and the final process models reflect a better level of

understanding (median of 4). Reversely, subjects with high social distance make more

syntactical errors (median of 3) and the resulting process models reflect a smaller level

of understanding (median of 3). This may be explained by the fact that the larger and

more complex the process models are the greater the risk to lose the overview, thus

resulting in more syntactical errors [

]. Semantic quality reflects notable differences

and process models affected by high social distance seem to give a better account to

the domain than process models affected by low social distance.

The obtained results for

perceived quality

(i.e., process model agreement) are shown in

Table 6.2 and Figure 6.3.

Group Agreement Missing Aspects Description Mistakes Satisfaction

Low 3 2 2 2 2

High 3 3 2 2 2

Total 3 3 2 2 2

Table 6.2: Perceived Quality

For perceived quality (cf. Table 6.2 + Figure 6.3), the only differences can be found in

missing aspects (median of 2 for low and 3 for high social distance). However, there are

no clear differences between the subject groups.

6 Experiment Analysis and Interpretation

Low High Low High Low High Low HighLow High

Agreement Missing

Aspects Description Mistakes Satisfaction

Figure 6.3: Perceived Quality

Table 6.3 and Figure 6.4 present the results regarding the

level of granularity

showing

the

number of activities, gateways, nodes, edges, number of elements

, and the

number

of possible execution paths through a process model.

Group No. Activities No. Gateways No. Nodes No. Edges No. Elements No. Paths

Low 20 6 28 32 60 4

High 26 8 37.5 43.5 81 4

Total 22.5 7 32 36 68.5 4

Table 6.3: Level of Granularity

Regarding level of granularity (cf. Table 6.3 + Figure 6.4), process models affected

by high social distance represent the warehouse scenario in more detail than process

models affected by low social distance. This is especially evident in the overall number

of elements. For low social distance the median for number of elements has a value

of 60 while high social distance has a median of 81. In spite of the differences in the

number of elements, the number of possible execution paths does not differ between the

distances and has a median of 4.

6.1 Analysis of Raw Data and Descriptive Statistics

Low High Low High Low High Low HighLow High Low High

No. Activties No. Gateways No. Nodes No. Edges No. Elements No. Paths

100

120 177

Figure 6.4: Level of Granularity

Following, Table 6.4 and Figure 6.5 present the results for

process model structure

, i.e.,

sequentiality, separability, cyclicity, and diameter.

Group Sequentiality Separability Cyclicity Diameter

Low 0.366 0.610 0 23

High 0.362 0.577 0.031 30.5

Total 0.364 0.596 0 27

Table 6.4: Process Model Structure

Low High

Low High Low HighLow High

Sequentiality Separability Cyclicity Diameter

0,25

0,5

0,75

Figure 6.5: Process Model Structure

According Table 6.4 and Figure 6.5, there are only minimal differences in process model

structure between the process models. However, diameter shows a clear distinction

between the social distances. Again, process models affected by high social distance

containing notable longer paths, i.e., median of 23 for low and 30.5 for high social

distance.

6 Experiment Analysis and Interpretation

Finally, the results of

additional factors

containing the

number of modeling steps, mod-

eling duration (in seconds), naming of process model activities

, and

mental effort

for

creating a process model are shown in Table 6.5 and Figure 6.6.

Group No .Steps Duration (sec) Effort Naming

Low 250.5 993 3 0

High 253.5 1489.5 3 1

Total 253.5 1210 3 0

Table 6.5: Additional Factors

Low High Low High

Low HighLow High

No. Steps Duration (sec)Effort Naming

250

500

750

1000

1250

1932

3968

1500

Figure 6.6: Additional Factors

Considering the additional factors (cf. Table 6.5 and Figure 6.6), the only noticeable

aspect is the modeling duration. While the number of modeling steps slightly differ the

modeling duration is very differentiating. Subjects with high social distance needed

nearly 50% more time for modeling (median of 1489.5 sec) than subjects with low social

distance (median of 993 sec). Despite the huge difference in modeling duration the

mental effort for creating the process models is in both groups the same value (median

of 3).

To gain an even better understanding of the data, results are visualized as various graphs

(cf. Appendix E).

Our observations are merely based on descriptive statistics. For a more rigid investiga-

tion, hypotheses will be tested for statistical significance in Section 6.3.

6.2 Data Set Reduction

Incidentally, one interesting side effect, however, not further explained in this thesis,

regarding the granularity, quality, and structure, process models created by women tend

to reflect a higher level of granularity, quality as well as process model structure.

6.2 Data Set Reduction

Generally, results of statistical analysis depends on the quality of input data. Faulty data

may lead to an incorrect conclusion. Therefore, it is important to identify outliers and

decide how to deal with them, i.e., a data set reduction might become necessary. Data

set reduction is critical when analyzing data because removed data could modificate the

results and that may lead to a loss of information.

In the experiment, we identified several outliers. For example:

•One subject modeled a process model with 64 branches.

•One subject needed for process modeling 3968 seconds.

•One subject made at process modeling 1146 steps.

We decided not to remove these subjects since they seem to be correct values and not a

result of wrong process modeling or due to strange events that never will happen again.

Removing them would falsify obtained results.

6.3 Hypothesis Testing

Even if descriptive statistics shows differences, hypotheses have to be tested to proof

the assumptions. With support of test procedures null hypotheses need to be rejected.

Initially, it is important to choose an adequate test procedure. [

] offers a selection of

common methods. Thereby, each method has a critical threshold that must be observed

in order to reject the null hypothesis. When testing hypotheses, it has to be observed

whether results exceed the critical threshold or not. There are basically two outcomes:

6 Experiment Analysis and Interpretation

•

Result is significant: If the critical threshold is exceeded, results of the experiment

are significant. The null hypothesis

is refuted and the alternative hypothesis

H1is accepted.

•

Result is not significant: If the threshold is not exceeded, results of the experiment

are not significant. The null hypothesis

cannot be refuted and needs to be

accepted. This does not indicate a failure of the alternative hypothesis

, but no

difference could be found between the experimental results.

To test the hypotheses, we use the

One-Tailed Wilcoxon-Mann-Whitney-U-Test (U-Test)

[

]. The U-Test is a non-parametric test with greater efficiency on non-gaussian

distributions, such as a mixture of guassian-distributions. For testing the hypotheses,

several values need to be calculated and the procedure reads as follows:

Every single data set is assigned with a numeric rank beginning with 1. Initially, the ranks

of both groups are summed up separately, i.e.,

(i.e., low social distance) and

(i.e.,

high social distance). Afterwards, a test value for

(i.e., low social distance) and

(i.e., high social distance) is calculated using the following equations:

U1=n1n2+n1(n1+ 1)

2−R1U2=n1n2+n2(n2+ 1)

2−R2

Here,

represents the number of subjects of group one (i.e., low social distance) and

n2represents the number of subjects of group two (i.e., high social distance).

For a small sample size, it is sufficient to determine the smaller

u-value

and compare

the u-value with the critical values for the U-Test [51].

However, since we have a larger sample size one can assume that U is approximately

gaussian-distributed. Therefore, the

standardized value

(i.e.,

z-value

) must be deter-

mined using the following equation:

z=U−mu

σu

Whereas, U is the smallest calculated u-value from both groups and

and

σu

are the

mean and standard deviation of U calculated as follows:

6.3 Hypothesis Testing

mu=n1n2

σu=sn1n2(n1+n2+ 1)

The calculated z-value can be compared with the values in the standard normal table

and, thus, it can be determined the probability of significance, i.e.,

p-value

. A p-value

below 0.05 indicates a significant result and the null hypothesis can be rejected. The

calculated values of hypothesis testing can be found in Appendix E.

Table 6.6 and 6.7 are showing the results of hypothesis testing.

Syntactic Quality H1

Response Variable p-value Significant?

Number of Syntactical Errors 0.046 (< 0.05) Yes

Semantic Quality H2

Response Variable p-value Significant?

Correctness 0.186 (> 0.05) No

Relevance <0.01 (< 0.05) Yes

Completeness <0.01 (< 0.05) Yes

Authenticity <0.01 (< 0.05) Yes

Perceived Quality H3

Response Variable p-value Significant?

Agreement 0.936 (> 0.05) No

Missing Aspects 0.603 (> 0.05) No

Accurate Description 0.529 (> 0.05) No

Mistakes 0.424 (> 0.05) No

Result Satisfaction 0.368 (> 0.05) No

Pragmatic Quality H4

Response Variable p-value Significant?

Level of Understanding <0.01 (< 0.05) Yes

Level of Granularity H5

Response Variable p-value Significant?

Number of Activities <0.01 (< 0.05) Yes

Number of Gateways 0.039 (< 0.05) Yes

Number of Nodes <0.01 (< 0.05) Yes

Number of Edges <0.01 (< 0.05) Yes

Number of Elements <0.01 (< 0.05) Yes

Number of Paths 0.435 (> 0.05) No

Table 6.6: Results of Hypothesis Testing

6 Experiment Analysis and Interpretation

Process Model Structure H6

Response Variable p-value Significant?

Sequentiality 0.849 (> 0.05) No

Cyclicity 0.091 (> 0.05) No

Separability 0.617 (> 0.05) No

Diameter <0.01 (< 0.05) Yes

Additional Factors H7

Response Variable p-value Significant?

Number of Modeling Steps 0.395 (> 0.05) No

Modeling Duration <0.01 (< 0.05) Yes

Mental Effort 0.285 (> 0.05) No

Naming 0.384 (> 0.05) No

Table 6.7: Results of Hypothesis Testing

As already indicated in raw data and descriptive statistics (cf. Section 6.1), several

response variables are revealing significant differences. In particular,

and

show

high significant results, whereas

and

likewise indicate high differences but not in

a complete extent. However,

does not include any significant results and

as well

as the tested additional factors H7reveal a few significant results.

In spite of the fact that several of the tested results are showing significant differences,

the alternative hypotheses are neither fully nor partial supported. Quite the contrary,

the results are against all defined alternative hypotheses and are supporting only the

null hypotheses (cf. Section 4.3). Therefore, the null hypotheses cannot be rejected

and must be accepted while alternative hypotheses must be rejected with respect to the

results of hypothesis testing.

A summary of the experiment analysis and interpretation as well as a in-depth discussion

about possible factors which lead to that outcome can be found in the following section.

6.4 Summary and Discussion

Introduced in Section 4.1, the objective of the controlled experiment is to get new

insights of the effects of social distance on the process of process modeling. Therefore,

particularly focus is on the following fundamental research question:

6.4 Summary and Discussion

Is the process of process modeling, i.e., the quality, granularity, and structure of

process models resulting from it, affected by the social distance process designers

have on respective business processes?

Hence, this experiment provides preliminary empirical results of the effects of social

distance of a process designer to the modeled domain has on the creation of process

models. Significant results are obtained for several tested aspects but in the end they are

not in accordance with the defined alternative hypotheses. Surprisingly, obtained results

do not agree with the theory and are contrary to the stated goal of the experiment. It is

expected that a low social distance leads to fine-grained and precise process models

while high social distance results in more course-grained and imprecise process models.

However, the contrary effect occurred. Accordingly, process models influenced by high

social distance reflect a higher process quality, level of granularity, and process model

structure than process models influenced by low social distance.

What are the reasons for causing such an outcome contrary to the expectations and

assumptions? Considering the fact that currently no work exist, in the context of process

modeling, dealing with cognitive aspects, in particular with social distance, and gam-

ification in a virtual world. Therefore, without replication or further results only vague

assumptions about the reasons can be made.

First of all, despite the pilot studies and the necessary prearrangements, one reason

may be the gamification approach. According to CLT and with properties obtained from

the previous experiment, a similar experiment is conducted to investigate the effects of

social distance on the process of process modeling. The difference in this experiment

is that a gamification approach is used to enhance the effects of social distance, to

increase the motivation of subjects, and for a better reflection of the real world problem.

Hence, the gamification approach may has an opposite effect. Although, gamification

should have a positive effect it is nevertheless possible that the effects of social distance

and gamification cancel each other out.

It is also possible that gamification in a 3D virtual world has no effects on process

modeling. Reason can be that the full virtual depiction of the domain is too abstract and,

therefore, it is hard for subjects to create a corresponding concrete representation. It

is conceivable, however, only to use inconspicuous game techniques instead of a full

6 Experiment Analysis and Interpretation

virtual environment.

On the other side, the intended effects of social distance (i.e., experiences and decisions

which are not self-experienced) do not exert a strong influence as expected. Maybe, in

this context, this intention plays a relatively secondary role and the social distance refers

stronger to the relation to other individuals.

Further, it must be considered that the distance between low and high social distance

may be not large enough or that a social distance is not perceived.

Moreover, the handling of the 3D warehouse scenario (i.e. learn to play) may be another

reason. Due to the fact that none of the subjects played the scenario before and there is

no proper introduction to the scenario the subjects need to familiarize themselves first.

The learning process disturbs or hinders the focus of the subjects and, thus, affects the

process modeling later.

Further, it is conceivable that subjects playing the warehouse scenario are more easily

distracted and busy than subjects watching the video. This follows from the fact that

subjects watching the video just need to watch the scenario and are able to write down

personal notes. In turn, subjects playing the scenario need to focus at the scenario and,

thereby, making the notes fades into the background.

Another possibility is, regarding process capturing, that a passive observation is more

effective than an imitation of the corresponding domain. With an observation of the

process en bloc details attract more attention which are easily overlooked by an active

participation.

In conclusion, there are many reasons that may have an impact on the results and,

therefore, it is not possible to make a clear statement. On the contrary, especially after

the obtained outcome, more results are needed by additional experiments either through

replication, similar, or control experiments with different focal points. Only in this way it

can be guaranteed to get a better understanding from the many influencing factors and,

thereby, to define proper guidelines dealing with cognitive aspects as well as the use of

gamification and virtual worlds on the process of process modeling.

Related Work

This thesis investigates the impact of social distance on the quality, granularity, and

structure of process models. Accordingly, our work is related to these aspects.

By now, different frameworks and guidelines in respect to process model quality exist.

Among others, the

SEQUAL framework

uses

semiotic theory

for identifying various

aspects of process model quality [

], whereas

Guidelines of Process Modeling (GoM)

describe quality considerations for process models [

] and

Seven Process Modeling

Guidelines (7PMG) characterize desirable properties of a process model [59].

Moreover, significant research on factors affecting process model comprehensibility

and maintainability exists. The influence of model complexity on process model com-

prehensibility is investigated in [

]. In turn, [

] analyzes the effect of modularity on

process understanding. The influence of grammatical styles for labeling activities on

model understanding is discussed in [

], and an experiment investigating the impact

of secondary notations is presented in [

]. The impact of different quality metrics on

7 Related Work

error probability is discussed in [

]. [

] provides prediction models for true usability

and maintainability of process models. Effects of how and at which level of granularity a

process designer models a particular process is described in [33].

Notwithstanding, in the context of process modeling there exists little work looking at

cognitive aspects. [

] presents the effects of reducing cognitive load on end user

understanding of conceptual models. Understanding complex models quickly reach

cognitive limits and the investigation on the cognitive difficulty of understanding different

relations between model elements is described in [25].

Common to all these works is the focus on the resulting process model (i.e., the product

of process modeling), while little attention has been paid on the process of the process

modeling itself. The Nautilus project complements these approaches by taking a close-

up view on the process of process modeling for tracing model quality back to different

modeling strategies resulting in process models of different quality [69].

Considering the process model structure, [

] introduces a set of process metrics that

investigates the influence of errors on domain factors of process models. [

] presents

results from a questionnaire dealing with process model structure as a particular quality

aspect and the connection with model, content, and personal related factors. Further-

more, influence of model and personal factors on the process model structure and

understandability is studied in [75].

Aside of the impact of social distance on the quality, granularity, and structure of process

models, gamification in a 3D virtual world is used to enhance the influence of social

distance, for a better reflection of the real world problem, and to increase motivation of

subjects. Meanwhile, there exist several research considering the use of gamification as

well as virtual worlds and is related to our work.

[

] introduces gamification and the possibilities of use in a non-gaming context. Further,

a shared understanding and findings from gamification concerning information systems

in regard to collaboration and opportunities are presented in [

]. [

] discusses the

effectiveness of gamification based on a quality service model analyzing the social and

psychological motivations of participants. Thereby, a study investigating social factors

towards gamification and the intention to use gamified services are discussed in [

Agile and efficient responds to changing requirements and consequential amendments

to corresponding business processes are provided in [

] using a gamification and BPM

approach incorporated into a social network. In turn, [

] concerns itself with adaptive

case management and the improvement of process planning with the use of gamifica-

tion. Furthermore, [

] provides preliminary evidence that blending business process

management to gamification concepts can increase morale as well as the willingness for

learning. Considerable work involving conceptual modeling of business processes in

a 3D virtual world can be found in [

]. Thereby, a BPMN process editor is embedded

in a 3D virtual world for extra support during process modeling. In addition, [

] pro-

vides an approach for collaborative business process modeling using a 3D environment

technology. A similar use case in a 3D warehouse scenario to visualize storyboards

for business process models is proposed in [

]. Finally, [

] takes a step further and

combines collaborative process modeling with augmented reality, i.e., hybrid of perceived

and computer-based reality.

All these works and researches cover many aspects in respect to process model quality,

granularity, and structure as well as the use of gamification and virtual worlds. However,

none of them has taken the social distance and gamification in a virtual world concur-

rently into account. However, more quantitative data is needed to support enterprises

in the selection of appropriate process designers and to ensure ideal conditions for

creating correct and sound process models. With this thesis we want to induce more

experimental research in the BPM field to attain this endeavor.

Conclusion

This thesis investigates whether social distance affects the process of process modeling,

i.e., the quality, granularity, and structure of process models resulting from it. In particu-

lar, based on a previous experiment and with the use of gamification in a virtual world,

another controlled experiment with 95 participants is conducted to get further insights

about the effects the social distance of a process designer to the modeled domain has

on the creation of process models. Therefore, one group is perceiving a low social

distance with the use of gamification in a 3D virtual world scenario (i.e., order processing

in a warehouse) while another group is watching the same scenario in a video, thus

perceiving a high social distance. Afterwards, both groups need to create a process

model from the played or watched scenario based on their own experience.

The resulting process models are revealing in several aspects significant differences and

results but are not in accordance with the defined alternative hypotheses. Surprisingly,

and interesting at the same time, contrary to the expectations and assumptions and de-

8 Conclusion

spite the use of gamification in a virtual world, process models influenced by high social

distance tend to reflect a higher syntactic, semantic, and pragmatic quality compared with

the process models influenced by low social distance. Most notably, process designers

with high social distance create more fine-grained and detailed process models.

Due to these observed differences and results the null hypotheses must be assumed and

the defined alternative hypotheses need to be rejected. Nonetheless, the final results

are very interesting and further research is highly recommended dealing with cognitive

aspects affecting process modeling as well as the use of gamification and virtual worlds.

Finally, generalization of the results needs to be confirmed by additional experiment, i.e.,

in order to obtain more accurate results allowing such a generalization, additional studies

are needed either through replication, similar, or control experiments in other environ-

ments to investigate the influence of social distance as well as the use of gamification

and virtual worlds on the process of process modeling. Furthermore, experiments related

to other psychological distances (i.e., spatial, temporal, and hypothetical distance) will

be subject of future work. Combining results for all psychological distances enables

to extract guidelines on how modeling teams in enterprises should be put together for

creating or optimizing business process models.

Bibliography

[1]

Amir, Bilal and Ralph, Paul: Proposing a Theory of Gamification Effectiveness. In:

Proceedings of the 36th International Conference on Software Engineering (ICSE

2014), Hyderabad, India, pp. 626–627 (2014)

[2]

Aparicio, Andrés F. and Vela, Francisco L. G. and Sánchez, José L. G. and Montes,

José I. M.: Gamification: Analysis and Application. In: New Trends in Interaction,

Virtual Reality and Modeling, Human–Computer Interaction Series, pp. 113–126.

Springer (2013)

[3]

Atomi Systems: Activepresenter,

http://www.atomisystems.com/

activepresenter/, last accessed on 2015/03/28

[4] Bartle, Richard A.: Designing Virtual Worlds. New Riders (2004)

[5]

Basili, Victor R. and Caldiera, Gianluigi and Rombach, Dieter: The Goal Question

Metric Approach. In: Encyclopedia of Software Engineering, vol. 2, pp. 528–532.

Wiley-Blackwell (1994)

[6]

Becker, Jörg and Rosemann, Michael and Uthmann, Christoph: Guidelines of

Business Process Modeling. In: Business Process Management, Lecture Notes in

Computer Science, vol. 1806, pp. 30–49. Springer (2000)

[7]

Blender Foundation: Blender,

http://www.blender.org/

, last accessed on

2015/04/10

[8]

Blizzard Entertainment: World of Warcraft,

http://eu.battle.net/wow/en/

last accessed on 2015/04/10

Bibliography

[9]

Böckermann, Petri and Ilmakunnas, Pekka: The Job Satisfaction-Productivity Nexus:

A Study Using Matched Survey and Register Data. In: Industrial and Labor Relations

Review, vol. 65(2), pp. 244–262. SAGE Publications (2012)

[10]

Brito, Thaigo P. and Paes, Josias and Moura, Antao B.: Game-Based Learning

in IT Service Transition. In: Proceedings of the 6th International Conference on

Computer Supported Education (CSEDU 2014), Barcelona, Spain, pp. 110–116

(2014)

[11]

Brown, Ross A.: Conceptual Modelling in 3D Virtual Worlds for Process Communi-

cation. In: Proceedings of the 7th Asia-Pacific Conference on Conceptual Modelling

(APCCM 2010), Brisbane, Australia, pp. 25–32 (2010)

[12]

Brown, Ross A. and Recker, Jan C. and West, Stephen: Using Virtual Worlds

for Collaborative Business Process Modeling. In: Journal of Business Process

Management, vol. 17(3), pp. 546–564. Emerald Group (2011)

[13]

Burris, Christopher T. and Branscombe, Nyla R.: Distorted Distance Estimation

Induced by a Self-Relevant National Boundary. In: Journal of Experimental Social

Psychology, vol. 41(3), pp. 305–312. Elsevier (2005)

[14]

Cook, Thomas D. and Campbell, Donald T.: Quasi-Experimentation: Design and

Analysis Issues for Field Settings. Houghton Mifflin (1979)

[15]

Crytek GmbH: CryEngine,

http://cryengine.com/

, last accessed on

2015/04/10

[16]

Davis, Alanah and Khazanchi, Deepak and Murphy, John and Zigurs, Ilze and

Owens, Dawn: Avatars, People, and Virtual Worlds: Foundations for Research in

Metaverses. In: Journal of the Association for Information Systems, vol. 10(2), pp.

90–117. AIS (2009)

[17]

Day, Samuel B. and Bartels, Daniel M.: Representation Over Time: The Effects of

Temporal Distance on Similarity. In: Cognition, vol. 106, pp. 1504–1513. Elsevier

(2008)

[18]

Deterding, Sebastian and Dixon, Dan and Khaled, Rilla and Nacke, Lennart: From

Game Design Elements to Gamefulness: Defining Gamification. In: Proceedings of

Bibliography

the 15th International Academic MindTrek Conference: Envisioning Future Media

Environments (MindTrek 2011), Tampere, Finland, pp. 9–15 (2011)

[19]

Deterding, Sebastian and Sicart, Miguel and Nacke, Lennart and O’Hara, Kenton

and Dixon, Dan: Gamification. Using Game-Design Elements in Non-Gaming

Contexts. In: Proceedings of the 29th Annual Conference Extended Abstracts on

Human Factors in Computing Systems (CHI EA 2011), Vancouver, Canada, pp.

2425–2428 (2011)

[20]

Dorling, Alec and McCaffery, Fergal: The Gamification of SPICE. In: Software

Process Improvement and Capability Determination, Communications in Computer

and Information Science, vol. 290, pp. 295–301. Springer (2012)

[21]

Duncan, Ishbel and Miller, Alan and Jiang, Shangyi: A Taxonomy of Virtual Worlds

Usage in Education. In: British Journal of Educational Technology, vol. 43(8), pp.

949–964. British Educational Research Association (2012)

[22]

Epic Games: Unreal Engine,

https://www.unrealengine.com/

, last ac-

cessed on 2015/04/10

[23]

Explorable: A Guide to Basic Steps of Science,

http://www.explorable.com/

last accessed on 2015/04/10

[24]

Figl, Kathrin and Derntl, Michael: The Impact of Perceived Cognitive Effectiveness

on Perceived Usefulness of Visual Conceptual Modeling Languages. In: Proceed-

ings of the 30th International Conference on Conceptual Modeling (ER 2011),

Brussels, Belgium, pp. 78–91 (2011)

[25]

Figl, Kathrin and Laue, Ralf: Cognitive Complexity in Business Process Modeling.

In: Proceedings of the 23rd International Conference on Advanced Information

Systems Engineering (CAiSE 2011), London, United Kingdom, pp. 452–466 (2011)

[26]

Figl, Kathrin and Weber, Barbara: Individual Creativity in Designing Business

Processes. In: Proceedings of the 24th International Conference Workshop on

Advanced Information Systems Engineering (CAiSE 2012), Gda´

nsk, Poland, pp.

294–306 (2012)

Bibliography

[27]

Fujita, Kentaro and Henderson, Marlone D. and Eng, Juliana and Trope, Yaacov

and Liberman, Nira: Spatial Distance and Mental Construal of Social Events. In:

Psychological Science, vol. 17(4), pp. 278–282. SAGE Publications (2006)

[28]

Hamari, Juho and Koivisto, Jonna: Social Motivations To Use Gamification: An

Empirical Study of Gamifying Exercise. In: Proceedings of the 21st European

Conference on Information Systems (ECIS 2013), Utrecht, Netherlands, pp. 105–

116 (2013)

[29]

Hamari, Juho and Koivisto, Jonna and Sarsa, Harri: Does Gamification Work? -

A Literature Review of Empirical Studies on Gamification. In: Proceedings of the

47th Annual Hawaii International Conference on System Sciences (HICSS 2014),

Waikoloa, Hawaii, pp. 3025–3034 (2014)

[30]

Hay Group: Engage Employees and Boost Performance,

http://www.haygroup.

com/us/downloads/details.aspx?id=7343, last accessed on 2015/04/10

[31]

Herzberg, Frederick: One More Time: How Do You Motivate Employees? Harvard

Business Publishing (2008)

[32]

Herzig, Philipp and Ameling, Michael and Schill, Alexander: A Generic Platfrom

for Enterprise Gamification. In: Proceedings of the 2012 Joint Working IEEE/I-

FIP Conference on Software Architecture and European Conference on Software

Architecture (WICSA-ECSA 2012), Helsinki, Finland, pp. 219–233 (2012)

[33]

Holschke, Oliver and Rake, Jannis and Levina, Olga: Granularity as a Cognitive

Factor in the Effectiveness of Business Process Model Reuse. In: Proceedings of

the 7th International Conference on Business Process Management (BPM 2009),

Ulm, Germany, pp. 245–260 (2009)

[34]

Höst, Martin and Regnell, Björn and Wohlin, Claes: Using Students as Subjects — A

Comparative Study of Students and Professionals in Lead-Time Impact Assessment.

In: Empirical Software Engineering, vol. 5(3), pp. 201–214. Springer (2000)

[35]

Hutong Games: Playmaker,

https://www.assetstore.unity3d.com/en/

#!/content/368, last accessed on 2015/04/10

Bibliography

[36]

ICEBOX Studios: Adventure Creator,

https://www.assetstore.unity3d.

com/en/#!/content/11896, last accessed on 2015/04/10

[37]

Kane, Sean F. and Duranske, Benjamin T.: Virtual Worlds, Real World Issues. In:

Landslide, vol. 1(1), pp. 9–16. American Bar Association (2008)

[38]

Kapp, Karl M.: The Gamification of Learning and Instruction: Game-Based Methods

and Strategies for Training and Education. Wiley-Blackwell (2012)

[39]

Kathleen, Nardella and Brown, Ross A. and Kriglstein, Simone: Storyboard Aug-

mentation of Process Model Grammars for Stakeholder Communication. In: Pro-

ceedings of the 5th International Conference on Information Visualization Theory

and Applications (IVAPP 2014), Lisbon, Portugal, pp. 114–121 (2014)

[40]

King, Dominc and Greaves, Felix and Exeter, Christopher and Darzi, Ara: Gamifica-

tion: Influencing Health Behaviours with Games. In: Journal of the Royal Society of

Medicine, vol. 106(6), pp. 76–78. SAGE Publications (2013)

[41]

Kolb, Jens and Zimoch, Michael and Weber, Barbara and Reichert, Manfred: How

Social Distance of Process Designers Affects the Process of Process Modeling:

Insights from a Controlled Experiment. In: Proceedings of the 29th Annual ACM

Symposium on Applied Computing (SAC 2014), Enterprise Engineering Track,

Gyeongju, South Korea, pp. 1364–1370 (2014)

[42]

Krogstie, John: Model-Based Development and Evolution of Information Systems:

A Quality Approach. Springer (2012)

[43]

Krogstie, John and Lindland, Odd I. and Sindre, Guttorm: Towards a Deeper

Understanding of Quality in Requirements Engineering. In: Proceedings of the 7th

International Conference on Advanced Information Systems Engineering (CAiSE

1995), Jyväskylä, Finland (1995)

[44]

Landers, Richard N. and Callan, Rachel C.: Casual Social Games as Serious

Games: The Psychology of Gamification in Undergraduate Education and Employee

Training. In: Serious Games and Edutainment Applications, pp. 399–423. Springer

(2011)

Bibliography

[45]

Leopold, Henrik and Pittke, Fabian and Mendling, Jan: Towards Measuring Process

Model Granularity via Natural Language Analysis. In: Proceedings of the 11th

International Conference on Business Process Management (BPM 2013), Beijing,

China, pp. 417–429 (2013)

[46]

Liberman, Nira and Sagristano, Michael D. and Trope, Yaacov: The Effect of

Temporal Distance on Level of Mental Construal. In: Journal of Experimental Social

Psychology, vol. 38(6), pp. 523–534. Elsevier (2002)

[47]

Liberman, Nira and Trope, Yaacov and Stephan, Elena: Psychological Distance. In:

Social Psychology: Handbook of Basic Principles, vol. 2, pp. 353–381. The Guilford

Press (2007)

[48]

Linden Research: Second Life,

http://secondlife.com/

, last accessed on

2015/04/10

[49]

Lindland, Odd I. and Sindre, Guttorm and Solvberg, Arne: Understanding Quality in

Conceptual Modeling. In: IEEE Software, vol. 11(2), pp. 42–49. IEEE Press (1994)

[50]

Mahni

c, Nika: Gamification of Politics: Start a New Game. In: Teorija in Praksa, vol.

51(1), pp. 143–161. FDV Publications (2014)

[51]

Marascuilo, Leonard A. and Serlin, Ronald C.: Statistical Methods for the Social

and Behavioral Sciences. In: Journal of Educational Statistics, vol. 15(1), pp. 72–77.

American Educational Research Association (1990)

[52]

Marczweksi, Andrzej: Gamification: A Simple Introduction. Amazon Digital Services

(2013)

[53]

Mendling, Jan: Detection and Prediction of Errors in EPC Business Process Models.

Ph.D. thesis, Vienna University of Economics and Business Administration (2007)

[54]

Mendling, Jan: Metrics for Process Models: Empirical Foundations of Verification,

Error Prediction, and Guidelines for Correctness. Springer (2008)

[55]

Mendling, Jan and Dijkman, Remco and Friedrich, Fabian and La Rosa, Marcello

and Leopold, Henrik and Puhlmann, Frank and Recker, Jan C. and Reijers, Hajo A.

and Smirnov, Sergey and Strembeck, Mark and Weidlich, Matthias: What is in a

Bibliography

Process Model beyond Structure?,

http://http://www.bpm.scitech.qut.

edu.au/seminars/2011/2011_0301_QUT.pdf

, last accessed on 2015/04/10

[56]

Mendling, Jan and Neumann, Gustaf: Error Metrics for Business Process Models.

In: Proceedings of the 19th International Conference on Advanced Information

Systems Engineering (CAiSE 2007), Trondheim, Norway, pp. 53–56 (2007)

[57]

Mendling, Jan and Reijers, Hajo A. and Cardoso, Jorge: What Makes Process

Models Understandable. In: Proceedings of the 5th International Conference on

Business Process Management (BPM 2007), Brisbane, Australia, pp. 48–63 (2007)

[58]

Mendling, Jan and Reijers, Hajo A. and Recker, Jan C.: Activity Labeling in Process

Modeling: Empirical Insights and Recommendations. In: Information Systems, vol.

35(4), pp. 467–482. Elsevier (2010)

[59]

Mendling, Jan and Reijers, Hajo A. and van der Aalst, Wil M. P.: Seven Process

Modeling Guidelines (7PMG). In: Information and Software Technology, vol. 52(2),

pp. 127–136. Elsevier (2010)

[60]

Mendling, Jan and Strembeck, Mark: Influence Factors of Understanding Business

Process Models. In: Proceedings of the 11th International Conference on Business

Information System (BIS 2008), Innsbruck, Austria, pp. 142–153 (2008)

[61]

Mendling, Jan and Verbeek, Eric H. M. W. and van Dongen, Boudewijn F. and

van der Aalst, Wil M. P. and Neumann, Gustaf: Detection and Prediction of Errors

in EPCs of the SAP Reference Model. In: Data and Knowledge Engineering, vol.

64(1), pp. 312–329. Elsevier (2008)

[62]

Moody, Daniel L.: Cognitive Load Effects on End User Understanding of Conceptual

Models: An Experimental Analysis. In: Proceedings of the 8th East European

Conference on Advances in Databases and Information Systems (ADBIS 2004),

Budapest, Hungary, pp. 129–143 (2004)

[63]

Moody, Daniel L.: Theoretical and Practical Issues in Evaluating the Quality of

Conceptual Models: Current State and Future Directions. In: Data and Knowledge

Engineering, vol. 55(3), pp. 243–276. Elsevier (2005)

Bibliography

[64]

Moody, Daniel L.: The “Physics” of Notations: Toward a Scientific Basis for Con-

structing Visual Notations. In: Proceedings of the 32nd ACM/IEEE International

Conference on Software Engineering (ICSE 2010), Cape Town, South Africa, pp.

485–508 (2010)

[65]

Neil, Macrae C. and Bodenhausen, Galen V. and Milne, Alan B. and Jetten, Jolanda:

Out of Mind but Back in Sight: Stereotypes on the Rebound. In: Journal of Per-

sonality and Social Psychology, vol. 67(5), pp. 808–817. American Psychological

Association (1994)

[66]

Object Management Group: Business Process Management Notation (BPMN) 2.0,

http://www.bpmn.org, last accessed on 2015/04/10

[67]

Oldenhave, Danny and Hoppenbrouwers, Stijn and van der Weide, Theo and

Lagarde, Remco: Gamification to Support the Run Time Planning Process in

Adaptive Case Management. In: Proceedings of the 14th International Conference

on Business-Process and Information Systems Modeling (BPMDS 2013), Valencia,

Spain, pp. 385–394 (2013)

[68] Pingerra, Jakob and Zugal, Stefan and Weber, Barbara: Investigating the Process

of Process Modeling with Cheeath Experimental Platform. In: Proceedings of the

1st International Workshop on Empirical Research in Process-Oriented Information

Systems (ER-POIS 2010), Hammamet, Tunisia, pp. 13–18 (2010)

[69]

Pinggera, Jakob and Soffer, Pnina and Zugal, Stefan and Weber, Barbara and Wei-

dlich, Matthias and Fahland, Dirk and Reijers, Hajo A. and Mendling, Jan: Modeling

Styles in Business Process Modeling. In: Proceedings of the 13th International

Conference on Business-Process and Information Systems Modeling (BPMDS

2012), Gda´

nsk, Poland, pp. 151–166 (2012)

[70]

Polyvyanyy, Artem and Smirnov, Sergey and Weske, Mathias: On Application of

Structural Decomposition for Process Model Abstraction. In: Proceedings of the 2nd

International Conference on Business Process and Services Computing (BPSC

2009), Leipzig, Germany, pp. 110–122 (2009)

Bibliography

[71]

Poppe, Erik and Brown, Ross A. and Recker, Jan C. and Johnson, Daniel M.:

Preliminary Evaluation of an Augmented Reality Collaborative Process Modelling

System. In: Proceedings of the 14th International Conference on Cyberworlds (ICC

2012), Darmstadt, Germany, pp. 77–84 (2012)

[72]

Pronin, Emily and Oliviola, Christopher Y. and Kennedy, Kathleen A.: Doing unto

Future Selves as you would do unto Others: Psychological Distance and Decision

Making. In: Personality and Social Psychology Bulletin, vol. 34(2), pp. 224–236.

SAGE Publications (2008)

[73]

Recker, Jan C. and Indulska, Marta and Rosemann, Michael and Green, Peter:

Business Process Modeling - A Comparative Analysis. In: Journal of the Association

for Information Systems, vol. 10(4), pp. 333–363. AIS (2009)

[74]

Reijers, Hajo A. and Mendling, Jan: Modularity in Process Models: Review and

Effects. In: Proceedings of the 6th International Conference on Business Process

Management (BPM 2008), Milan, Italy, pp. 20–35 (2008)

[75]

Reijers, Hajo A. and Mendling, Jan: A Study into the Factors that Influence the

Understandability of Business Process Models. In: IEEE Transactions on Systems,

Man and Cybernetics - Part A, vol. 41(3), pp. 449–462. IEEE Press (2011)

[76]

Reijers, Hajo A. and Mendling, Jan and Weber, Barbara and Reichert, Manfred:

Refactoring Large Process Model Repositories. In: Computers in Industry, vol.

62(5), pp. 467–486. Elsevier (2011)

[77]

Rittgen, Peter: Collaborative Modeling of Business Processes - A Comparative

Case Study. In: Proceedings of the 24th Annual ACM Symposium on Applied

Computing (SAC 2009), pp. 225–230 (2009)

[78] Rittgen, Peter: Quality and Perceived Usefulness of Process Models. In: Proceed-

ings of the 25th Annual ACM Symposium on Applied Computing (SAC 2010), Sierre,

Switzerland, pp. 65–72 (2010)

[79]

Riva, Guiseppe: Virtual Reality as Communication Tool: A Sociocognitive Analysis.

In: Presence, vol. 8(4), pp. 462–468. MIT Press (1999)

Bibliography

[80]

Rolón, Elvira and Sánchez Laura and Garcia, Félix and Ruiz, Francisco and Piattini,

Mario and Caivano, Danilo and Visaggio, Giuseppe: Prediction Models for BPMN

Usability and Maintainability. In: Proceedings of the 11th IEEE Conference on

Commerce and Enterprise Computing (CEC 2009), Vienna, Austria, pp. 383–390

(2009)

[81]

Santorum, Marco and Front, Agnes and Rieu, Dominique: ISEAsy: A Social

Business Process Management Platform. In: Proceedings of the 11th International

Conference on Business Process Management (BPM 2013), Beijing, China, pp.

125–137 (2013)

[82]

Schrepfer, Matthias and Wolf, Johannes and Mendling, Jan and Reijers, Hajo A.:

The Impact of Secondary Notation on Process Model Understanding. In: Proceed-

ings of the 2nd IFIP WG 8.1 Working Conference on the Practice of Enterprise

Modeling (PoEM 2009), Stockholm, Sweden, pp. 161–175 (2009)

[83]

Shahri, Alimohammad and Hosseini, Mahmood and Phalp, Keith and Taylor, Jacquai

and Ali, Raian: Towards a Code of Ehtics for Gamification at Enterprise. In: Pro-

ceedings of the 7th IFIP WG 8.1 Working Conference on the Practice of Enterprise

Modeling (PoEM 2014), Manchester, United Kingdom, pp. 235–245 (2014)

[84]

Siau, Keng and Rossi, Matti: Evaluation Techniques for Systems Analysis and

Design Modelling Methods–A Review and Comparative Analysis. In: Information

System Journal, vol. 21(3), pp. 249–268. Wiley-Blackwell (2011)

[85]

Sirkin, Mark R.: Statistics for the Social Sciences, vol. 7. Sage Publications (2005)

[86]

Somers, Toni M. and Nelson, Klara: The Impact of Critical Success Factors Across

the Stages of Enterprise Resource Planning Implementations. In: Proceedings

of the 34th Annual Hawaii International Conference on System Sciences (HICSS

2001), Maui, Hawaii, pp. 8016–8026 (2001)

[87]

Techcrunch: Gamification Concept,

http://www.techcrunch.com/2014/10/

05/gamification-is-dead-long-live-games-for-learning/

, last ac-

cessed on 2015/03/28

Bibliography

[88]

Todorov, Alexander and Goren, Amir and Trope, Yaacov: Probability as a Psycho-

logical Distance: Construal and Preferences. In: Journal of Experimental Social

Psychology, vol. 43(3), pp. 473–482. Elsevier (2007)

[89]

Trope, Yaacov and Liberman, Nira: Construal-Level Theory of Psychological Dis-

tance. In: Psychological Review, vol. 117(2), pp. 440–463. American Psychological

Association (2010)

[90]

Trope, Yaacov and Liberman, Nira and Wakslak, Cheryl: Construal Levels and

Psychological Distance: Effects on Representation, Prediction, Evaluation, and

Behavior. In: Journal of Consumer Psychology, vol. 17(2), pp. 83–95. Elsevier

(2007)

[91]

Unity Technologies: Unity,

http://www.unity3d.com/

, last accessed on

2015/04/10

[92]

VideoLAN Organizazion: VLC Media Player,

http://www.vlc.de/

, last ac-

cessed on 2015/04/10

[93]

Welpe, Isabell and Tumasjan, Andranik and Strobel, Maria: Construal Level The-

ory - A Theory for the Boundaryless Organization? In: Zeitschrift für betrieb-

swirtschaftliche Forschung, vol. 62, pp. 84–105. Handelsblatt Fachmedien GmbH

(2010)

[94]

West, Stephen and Brown, Ross A. and Recker, Jan C.: Collaborative Business

Process Modeling Using 3D Virtual Environments. In: Proceedings of the 7th Asia-

Pacific Conference on Conceptual Modelling (APCCM 2010), Brisbane, Australia,

pp. 51–60 (2010)

[95]

Wohlin, Claes and Runeson, Per and Höst, Martin and Ohlsson, Magnus C. and

Regnell, Björn and Wesslen, Anders: Experimentation in Software Engineering -

An Introduction. Kluwer Academic Publishers (2000)

[96]

Wolf, Celia and Harmon, Paul: The State of Business Process Management 2012.

In: A BPTrends Report. BPTrends (2012)

Bibliography

[97]

Wollard, Karen K. and Shuck, Brad: Antecedents to Employee Engagement: A

Structured Review of the Literature. In: Advances in Developing Human Resources,

vol. 13(4), pp. 429–446. SAGE Publications (2011)

[98]

Zichermann, Gabe and Cunningham, Christopher: Gamification by Design: Imple-

menting Game Mechanics in Web and Mobile Apps. O’Reilly Media (2011)

[99]

Zimoch, Michael: Experiments on Influence of Construal Level During Process

Modeling. Bachelor Thesis, University Ulm (2012)

Evaluation and Task Sheets

A Evaluation and Task Sheets

Evaluation Sheet

Number of Activities: __ Number of Edges: __

Number of Gateways: __ Number of Nodes: __

Overall: __ Number of Branches: __

___________________________________________________________________________

Steps: __ Duration: __

___________________________________________________________________________

Error Metrics

Sequentiality __ Cyclicity __

Diameter __ Separability __

___________________________________________________________________________

Syntactic

Number of Rule Violations: __

Semantic (7-Point Scale: 0 to 6)

Indicator Definition Rating

Correctness

All statements in the representation

are correct.

Relevance

All statements in the representation

are relevant to the problem

Completeness

The representation contains all

statements about the domain that are

correct and relevant

Authenticity

The representation gives a true

account of the domain

Pragmatic (7-Point Scale: 0 to 6)

Understandable: __

___________________________________________________________________________

Perceived Model Quality (Mental Effort 7-Point Scale: 0 to 6; Others 5-Point Scale: 0 to 4)

Mental Effort __

Agreement __

Missing Aspects __

Accurate Description __

Mistakes __

Result Satisfaction __

___________________________________________________________________________

Naming (3-Point Scale: 0 to 2)

Points: __

Figure A.1: Evaluation Sheet

Code: 1111

Experiment

Play the 3D warehouse scenario. You may take notes during gameplay.

Afterwards, model the played process using BPMN 2.0. Model the process based

on your own experience and in the way you think it was performed. Furthermore,

consider all eventualities in your process model. After finishing the modeling task,

press “Finish Modeling”.

Thank you for participation!

Figure A.2: Task Sheet 1 - Low Social Distance

A Evaluation and Task Sheets

Code: 9999

Experiment

Watch the 3D warehouse scenario. You may take notes during playtime.

Afterwards, model the watched process using BPMN 2.0. Model the process

based on your own experience and in the way you think it was performed.

Furthermore, consider all eventualities in your process model. After finishing the

modeling task, press “Finish Modeling”.

Thank you for participation!

Figure A.3: Task Sheet 2 - High Social Distance

Demographic Questionnaire

Based on demographic questionnaire (cf. Table 4.2), Figure B.1-B.4 present the obtained

results. All questions refer to a period within the past 12 months. We only count work

days within a year and, therefore, we assume that a year has about 250 work days.

Familiar, competent, and confident are determined on a 7-point Likert scale ranging from

strongly disagree (0) to strongly agree (6). The last question relates to the release date

of BPMN. The first version of BPMN stems from May 2004.

B Demographic Questionnaire

1 Game Low Academic Male Academic

2 Game Low Academic Male Academic

3 Game Low Academic Female Academic

4 Game Low Academic Male Academic

5 Game Low Academic Male Academic

6 Game Low Academic Male Academic

7 Game Low Academic Male Academic

8 Game Low Academic Male Academic

9 Game Low Academic Male Academic

10 Game Low Academic Male Academic

26 Game Low Student Female Economics

27 Game Low Student Female Economics

28 Game Low Student Female Economics

29 Game Low Student Female Economics

30 Game Low Student Female Economics

31 Game Low Student Male Economics

32 Game Low Student Female Economics

33 Game Low Student Female Economics

34 Game Low Student Female Economics

35 Game Low Student Male Computer Science

42 Game Low Student Male Computer Science

43 Game Low Student Male Computer Science

44 Game Low Student Male Economics

45 Game Low Student Female Media Computer Science

Course of Study

Gender

Subject

Type

Distance

Profession

Figure B.1: Demographic Questionnaire - Low - Part 1

46 Game Low Student Male Economics

47 Game Low Student Female Economics

48 Game Low Student Female Business Mathematics

62 Game Low Student Male Economics

63 Game Low Student Male Economics

64 Game Low Student Female Computer Science

65 Game Low Student Male Economics

66 Game Low Student Male Computer Science

67 Game Low Student Male Media Computer Science

68 Game Low Student Male Media Computer Science

69 Game Low Student Female Media Computer Science

70 Game Low Student Female Software Engineering

71 Game Low Student Male Computer Science

72 Game Low Student Male Computer Science

73 Game Low Student Male Media Computer Science

82 Game Low Student Male Economics

83 Game Low Student Male Economics

84 Game Low Student Female Economics

85 Game Low Student Male Media Computer Science

86 Game Low Student Male Computer Science

87 Game Low Student Male Economics

93 Game Low Student Male Economics

94 Game Low Student Male Economics

95 Game Low Student Male Computer Science

Course of Study

Gender

Subject

Type

Distance

Profession

Figure B.2: Demographic Questionnaire - Low - Part 2

B Demographic Questionnaire

130 20 10 0200 5 6 6 36

2150 100 10 1 5 6 6 6 6

330 12 4 0 2 5 5 5 7

4 5 3 10 110 3 4 4 24

530 15 8 0 2 6 6 6 60

6120 30 715 180 6 6 6 80

7250 40 15 010 6 6 6 60

820 10 10 0 5 5 6 5 5

915 10 20 0 0 5 5 5 60

10 50 20 10 030 6 6 6 74

26 5 3 3 2 10 3 3 3 0

27 10 020 20 5 2 3 3 1

28 5 5 15 3 2 1 1 1 2

29 4 3 10 2 3 2 2 3 1

30 20 8 6 1 1 2 3 2 1

31 25 715 1 1 3 3 3 1

32 1 1 10 1 1 0 0 0 1

33 2 0 5 0 1 2 2 2 1

34 20 820 25 10 3 3 3 2

35 15 20 12 5 2 4 5 4 2

42 10 5 7 3 5 3 5 5 6

43 10 10 15 1 1 3 5 4 20

44 0 0 30 0 0 1 1 3 26

45 50 20 15 9 4 5 5 5 6

Subject

Competent

Confident

Start BPMN

(Months)

No. Process

Analyzed/Read

No. Process

Created/Edited

No. Estimated

Activites

No. Training

Days

No. Self Education

(Days)

Familiar

Figure B.3: Demographic Questionnaire - Low - Part 3

46 5 1 10 3 2 1 1 1 1

47 0 0 0 3 2 3 1 2 1

48 0 0 0 3 10 3 2 2 0

62 15 510 1 2 2 3 3 5

63 5 0 5 1 4 1 3 1 1

64 310 20 20 20 4 5 5 24

65 5 5 15 14 10 3 3 3 2

66 5 5 15 3 5 4 4 3 3

67 20 530 3 2 4 4 4 12

68 10 510 510 4 4 3 8

69 6 3 8 5 8 2 2 3 3

70 15 815 7 3 5 5 4 14

71 50 25 40 3 3 4 4 5 2

72 20 520 3 2 3 4 3 12

73 2 5 15 3 5 5 2 2 3

82 50 5 5 3 1 0 0 0 2

83 40 10 10 2 1 4 4 4 3

84 25 510 3 3 2 4 2 2

85 20 10 10 20 5 3 4 3 3

86 1 1 3 1 0 2 2 2 1

87 20 815 5 5 3 3 4 3

93 20 510 5 2 0 1 0 3

94 10 320 7 3 2 2 2 1

95 50 20 20 2 5 4 5 5 36

Subject

Competent

Confident

Start BPMN

(Months)

No. Process

Analyzed/Read

No. Process

Created/Edited

No. Estimated

Activites

No. Training

Days

No. Self Education

(Days)

Familiar

Figure B.4: Demographic Questionnaire - Low - Part 4

11 Video High Student Female Economics

12 Video High Student Male Economics

13 Video High Student Male Economics

14 Video High Student Male Economics

15 Video High Student Male Economics

16 Video High Student Male Economics

17 Video High Student Female Economics

18 Video High Student Male Economics

19 Video High Student Male Computer Science

20 Video High Student Male Computer Science

22 Video High Student Male Economics

23 Video High Student Female Economics

24 Video High Student Male Computer Science

25 Video High Student Male Computer Science

36 Video High Student Female Economics

37 Video High Student Male Economics

38 Video High Student Female Economics

39 Video High Student Female Economics

40 Video High Student Male Economics

41 Video High Student Male Economics

49 Video High Student Female Economics

50 Video High Student Female Economics

51 Video High Student Female Economics

Course of Study

Gender

Subject

Type

Distance

Profession

Figure B.5: Demographic Questionnaire - High - Part 1

B Demographic Questionnaire

52 Video High Student Female Economics

53 Video High Student Female Economics

54 Video High Student Male Economics

55 Video High Student Male Economics

56 Video High Student Female Economics

57 Video High Student Female Economics

58 Video High Student Male Economics

59 Video High Student Female Economics

60 Video High Student Male Economics

61 Video High Student Male Economics

74 Video High Student Male Economics

75 Video High Student Male Economics

76 Video High Student Male Economics

77 Video High Student Male Economics

78 Video High Student Male Economics

79 Video High Student Male Economics

80 Video High Student Male Economics

81 Video High Student Female Economics

88 Video High Student Male Economics

89 Video High Student Female Economics

90 Video High Student Female Economics

91 Video High Student Male Economics

92 Video High Student Male Economics

Course of Study

Gender

Subject

Type

Distance

Profession

Figure B.6: Demographic Questionnaire - High - Part 2

11 10 115 3 2 1 1 1 1

12 10 10 5 1 1 1 1 1 3

13 10 210 3 1 5 5 2 2

14 0 0 0 0 0 0 0 0 2

15 20 5300 6 3 3 3 3 2

16 50 15 30 3 3 4 4 4 2

17 6 3 5 1 1 3 3 3 1

18 2 3 2 6 6 3 3 2 1

19 10 1 5 2 1 4 4 3 24

20 10 215 1 1 4 5 5 24

22 20 210 3 0 2 2 2 4

23 20 2 8 3 0 1 2 1 3

24 40 20 30 8 1 3 5 4 24

25 20 510 7 1 3 4 2 8

36 10 515 2 1 3 3 3 2

37 10 010 2 1 1 2 2 2

38 40 10 20 15 10 3 3 3 12

39 10 2 6 3 0 3 2 2 2

40 5 5 20 2 1 3 2 2 2

41 5 2 8 10 2 4 4 4 2

49 5 1 2 30 5 2 2 2 12

50 20 10 15 30 5 4 4 4 48

51 10 210 3 2 4 5 3 2

Subject

Competent

Confident

Start BPMN

(Months)

No. Process

Analyzed/Read

No. Process

Created/Edited

No. Estimated

Activites

No. Training

Days

No. Self Education

(Days)

Familiar

Figure B.7: Demographic Questionnaire - High - Part 3

52 25 10 50 5 3 3 3 3 2

53 7 1 10 2 2 3 3 3 2

54 10 710 2 1 4 5 4 2

55 5 2 10 15 2 3 3 3 2

56 5 1 10 5 5 1 4 1 2

57 1 1 10 2 1 3 3 3 1

58 7 5 15 5 5 2 3 2 6

59 10 620 2 0 1 1 1 2

60 3 3 10 1 1 3 3 3 2

61 5 2 8 1 1 4 4 3 2

74 5 0 5 1 0 1 2 2 1

75 10 3 7 2 1 4 3 3 2

76 2 1 20 6 6 0 1 2 1

77 1 1 3 1 1 2 2 2 2

78 5 2 8 1 1 1 4 1 2

79 15 10 15 2 2 3 4 3 2

80 15 715 2 1 2 3 3 2

81 20 315 1 0 4 2 2 2

88 10 1 5 1 0 2 4 2 1

89 10 015 3 0 4 5 4 24

90 5 3 15 3 2 0 0 0 1

91 10 520 10 0 4 4 4 1

92 1 2 8 2 1 2 2 2 2

Subject

Competent

Confident

Start BPMN

(Months)

No. Process

Analyzed/Read

No. Process

Created/Edited

No. Estimated

Activites

No. Training

Days

No. Self Education

(Days)

Familiar

Figure B.8: Demographic Questionnaire - High - Part 4

Game Questionnaire

Based on game and after game questionnaire (cf. Table 4.3 + 4.4), Figure C.1-C.4

present the obtained results. Familiar is determined on a 7-point Likert scale ranging

from strongly disagree (0) to strongly agree (6).

C Game Questionnaire

1 6 Computer Role-Playing 60 Picking System Loading System

2 6 Computer Action 2 Forklift Loading System

3 6 Computer Simulation 10 Forklift Loading System

4 5 Computer Sports 0 Picking System Loading System

5 6 Computer Strategy 0 Forklift Forklift

6 6 Xbox Action 6 Picking System Forklift

7 6 Computer Action 1 Forklift Loading System

8 6 Computer Action 1 Forklift Forklift

9 5 Computer Other 0 Forklift Forklift

10 5 Computer Action 0 Forklift Loading System

26 1 Computer Action 0 Picking System Loading System

27 1 Other Other 0 Forklift Loading System

28 5 Computer Action-Adventure 10 Picking System Loading System

29 1 Phone or Tablet Other 0 Picking System Loading System

30 1 Phone or Tablet Role-Playing 0 Forklift Loading System

31 2 Phone or Tablet Simulation 1 Picking System Loading System

32 0 Phone or Tablet Strategy 1 Forklift Loading System

33 0 Other Sports 0 Picking System Loading System

34 1 Phone or Tablet Strategy 1 Forklift Loading System

35 3 Playstation Sports 1 Picking System Loading System

42 5 Xbox Sports 0 Picking System Forklift

43 5 Computer Strategy 1 Forklift Loading System

44 5 Computer Strategy 10 Forklift Loading System

45 4 Computer Strategy 2 Picking System Forklift

Subject

Second Choice

Familiar with

Video Games

Favorite Video

Game Platform

Favorite Video

Game Genre

Hours of Gameplay

(Week)

First Choice

Figure C.1: Game and After Game Questionnaire - Low - Part 1

46 4 Playstation Sports 2 Forklift Loading System

47 1 Phone or Tablet Other 1 Picking System Forklift

48 0 Computer Strategy 0 Picking System Forklift

62 2 Playstation Action 1 Picking System Loading System

63 5 Computer Action 0 Picking System Loading System

64 0 Playstation Strategy 0 Picking System Loading System

65 6 Playstation Sports 1 Picking System Loading System

66 6 Computer Strategy 12 Forklift Loading System

67 6 Computer Other 15 Forklift Forklift

68 6 Playstation Action-Adventure 4 Picking System Loading System

69 0 Computer Strategy 0 Forklift Loading System

70 6 Computer Action 4 Forklift Forklift

71 3 Playstation Sports 3 Forklift Forklift

72 4 Computer Action 1 Picking System Forklift

73 5 Computer Action-Adventure 6 Forklift Forklift

82 0 Computer Sports 0 Picking System Loading System

83 6 Computer Role-Playing 6 Picking System Loading System

84 0 Phone or Tablet Strategy 1 Picking System Loading System

85 6 Computer Action 20 Picking System Loading System

86 3 Computer Sports 1 Forklift Loading System

87 4 Computer Strategy 6 Picking System Loading System

93 2 Computer Action 0 Picking System Loading System

94 5 Xbox Sports 1 Forklift Loading System

95 6 Computer Action 3 Picking System Loading System

Subject

Second Choice

Familiar with

Video Games

Favorite Video

Game Platform

Favorite Video

Game Genre

Hours of Gameplay

(Week)

First Choice

Figure C.2: Game and After Game Questionnaire - Low - Part 2

11 4 Phone or Tablet Strategy 1 Forklift Loading System

12 3 Playstation Sports 1 Forklift Loading System

13 5 Playstation Sports 2 Forklift Loading System

14 4 Playstation Sports 1 Forklift Loading System

15 4 Computer Action 5 Forklift Loading System

16 1 Computer Strategy 0 Forklift Loading System

17 2 Nintendo Strategy 0 Forklift Loading System

18 4 Playstation Sports 2 Forklift Loading System

19 5 Computer Role-Playing 0 Forklift Loading System

20 5 Playstation Role-Playing 0 Forklift Loading System

22 6 Computer Strategy 7 Forklift Loading System

23 2 Computer Action-Adventure 3 Forklift Loading System

24 2 Playstation Action-Adventure 0 Forklift Loading System

25 5 Computer Role-Playing 10 Forklift Loading System

36 3 Playstation Sports 1 Forklift Loading System

37 4 Playstation Sports 1 Forklift Loading System

38 4 Other Sports 1 Forklift Loading System

39 2 Phone or Tablet Other 0 Forklift Loading System

40 6 Computer Role-Playing 5 Forklift Loading System

41 4 Playstation Sports 4 Forklift Loading System

49 2 Phone or Tablet Strategy 1 Forklift Loading System

50 2 Computer Strategy 1 Forklift Loading System

51 0 Phone or Tablet Other 0 Forklift Loading System

Subject

Second Choice

Familiar with

Video Games

Favorite Video

Game Platform

Favorite Video

Game Genre

Hours of Gameplay

(Week)

First Choice

Figure C.3: Game and After Game Questionnaire - High - Part 1

52 0 Other Other 0 Forklift Loading System

53 1 Other Other 0 Forklift Loading System

54 6 Computer Role-Playing 20 Forklift Loading System

55 5 Computer Role-Playing 5 Forklift Loading System

56 0 Other Other 0 Forklift Loading System

57 1 Computer Other 0 Forklift Loading System

58 3 Computer Sports 1 Forklift Loading System

59 5 Xbox Role-Playing 2 Forklift Loading System

60 2 Playstation Sports 0 Forklift Loading System

61 5 Computer Action 1 Forklift Loading System

74 5 Computer Strategy 1 Forklift Loading System

75 4 Xbox Sports 2 Forklift Loading System

76 6 Computer Other 2 Forklift Loading System

77 2 Computer Strategy 0 Forklift Loading System

78 5 Computer Simulation 0 Forklift Loading System

79 5 Playstation Sports 4 Forklift Loading System

80 6 Computer Action 15 Forklift Loading System

81 1 Computer Simulation 0 Forklift Loading System

88 3 Playstation Sports 1 Forklift Loading System

89 3 Phone or Tablet Simulation 1 Forklift Loading System

90 1 Phone or Tablet Strategy 1 Forklift Loading System

91 4 Computer Sports 1 Forklift Loading System

92 5 Nintendo Sports 2 Forklift Loading System

Subject

Second Choice

Familiar with

Video Games

Favorite Video

Game Platform

Favorite Video

Game Genre

Hours of Gameplay

(Week)

First Choice

Figure C.4: Game and After Game Questionnaire - High - Part 2

Raw Data

Figure D.1-D.16 present detailed evaluation results from each process model.

D Raw Data

Subject No. Activities No. Gateways No. Nodes No. Edges Overall No. Branches No. Steps Duration (in Sec)

117 625 27 52 4158 535

214 824 27 51 8422 520

324 834 37 71 4298 617

414 622 24 46 4105 515

513 419 20 39 4381 483

612 418 19 37 492 432

713 621 23 44 472 390

816 523 24 47 4210 536

912 216 16 32 150 428

10 15 825 28 53 8187 650

26 20 729 32 61 4177 1307

27 25 10 37 44 81 4189 1226

28 39 10 51 55 106 4416 1378

29 36 846 54 100 4447 1166

30 21 932 41 73 41093 995

31 20 10 32 36 68 4272 1195

32 13 419 20 39 4106 919

33 10 315 16 31 453 722

34 23 631 34 65 12 329 1014

35 18 626 28 54 4602 1379

42 22 10 34 39 73 8337 981

43 30 15 47 56 103 4687 1099

44 16 422 26 48 4244 853

45 11 215 16 31 2153 734

Figure D.1: Raw Data - Low - Part 1

Subject No. Activities No. Gateways No. Nodes No. Edges Overall No. Branches No. Steps Duration (in Sec)

46 25 835 44 79 4160 1211

47 23 530 36 66 4127 1210

48 13 318 19 37 2128 819

62 26 533 37 70 12 173 1932

63 25 10 37 41 78 16 228 1704

64 27 534 44 78 4583 1828

65 34 945 50 95 8188 1886

66 29 10 41 46 87 8514 802

67 35 16 53 62 115 4281 1173

68 13 419 19 38 4257 856

69 17 726 28 54 4468 1277

70 35 12 49 58 107 1343 1372

71 11 215 15 30 2101 478

72 24 632 34 66 4158 836

73 17 625 27 52 4312 445

82 11 316 17 33 276 895

83 17 827 32 59 4286 1264

84 13 419 20 39 4416 1311

85 23 833 36 69 4609 979

86 23 631 33 64 4547 991

87 36 12 50 58 108 4358 1590

93 22 10 34 42 76 4166 1296

94 20 426 26 52 4113 1318

95 18 626 28 54 4468 1074

Figure D.2: Raw Data - Low - Part 2

Syntactic

Subject Diameter Sequentiality Separability Cyclicity

No. Syntactical

Errors

122 0.37 0.6 0 0

220 0.185 0.182 0 2

329 0.405 0.471 0.118 2

418 0.364 0.45 0 2

517 0.421 0.665 0 2

616 0.389 0.65 0 2

718 0.286 0.684 0 0

820 0.391 0.614 0.13 6

915 0.625 0.757 0 2

10 21 0.36 0.501 0.28 2

26 23 0.3125 0.481 0 3

27 26 0.223 0.514 0 3

28 37 0.491 0.469 0 2

29 37 0.371 0.66 0 2

30 22 0.098 0.4 0 3

31 27 0.222 0.533 0 2

32 17 0.4 0.765 0 2

33 13 0.25 0.692 0 3

34 27 0.412 0.759 0 3

35 23 0.357 0.708 0 2

42 28 0.205 0.438 0.441 5

43 41 0.283 0.387 0.489 7

44 18 0.346 0.6 0.5 1

45 14 0.563 0.769 0 4

Process Metrics

Figure D.3: Raw Data - Low - Part 3

Syntactic

Subject Diameter Sequentiality Separability Cyclicity

No. Syntactical

Errors

46 26 0.204 0.545 0.00 2

47 23 0.4 0.607 0.00 3

48 13 0.474 0.5 0.00 3

62 27 0.405 0.613 0.333 4

63 32 0.268 0.686 0 4

64 28 0.25 0.687 0.412 4

65 34 0.4 0.581 0 4

66 34 0.37 0.538 0.488 6

67 32 0.226 0.333 0 2

68 16 0.368 0.706 0 3

69 20 0.286 0.209 0 2

70 39 0.241 0.532 0 0

71 14 0.6 0.846 0 1

72 29 0.559 0.733 0 2

73 21 0.333 0.696 0 2

82 15 0.529 0.786 0.25 0

83 22 0.313 0.48 0.259 0

84 17 0.4 0.765 0 2

85 29 0.333 0.677 0 0

86 27 0.455 0.759 0 2

87 37 0.293 0.396 0 3

93 25 0.238 0.5 0.529 8

94 23 0.538 0.792 0 4

95 23 0.464 0.667 0 1

Process Metrics

Figure D.4: Raw Data - Low - Part 4

D Raw Data

Pragmatic

Subject Correctness Relevance Completeness Authenticity Understandable Naming

1 5 4 2 3 6 1

2 4 4 2 3 5 2

3 5 5 5 5 4 1

4 5 4 4 4 3 2

5 4 4 2 3 6 2

6 5 4 2 3 6 2

7 5 3 2 3 6 1

8 4 3 3 3 5 1

9 5 4 2 2 5 0

10 5 4 4 4 3 2

26 3 4 3 3 4 1

27 3 3 3 3 1 2

28 6 6 6 6 5 1

29 6 5 5 5 5 3

30 2 3 3 2 5 2

31 3 2 3 3 5 0

32 4 4 1 2 6 0

33 2 3 1 1 6 0

34 4 3 2 2 5 0

35 4 4 2 2 6 0

42 3 4 3 3 3 0

43 6 6 6 6 4 0

44 5 4 2 2 4 0

45 3 3 0 1 4 0

Semantic

Figure D.5: Raw Data - Low - Part 5

Pragmatic

Subject Correctness Relevance Completeness Authenticity Understandable Naming

46 4 4 3 3 3 0

47 3 4 3 3 4 0

48 2 3 0 1 6 0

62 5 5 4 5 3 1

63 5 5 4 5 4 0

64 4 4 4 4 3 0

65 5 5 5 4 3 0

66 3 3 4 4 3 0

67 6 6 5 6 3 0

68 3 4 0 2 6 0

69 3 4 3 3 3 1

70 5 5 3 3 4 0

71 2 3 0 0 6 0

72 5 5 3 4 4 0

73 3 3 2 2 5 0

82 2 2 0 2 4 0

83 5 5 4 4 5 1

84 3 2 0 2 5 0

85 3 3 2 2 5 1

86 4 4 2 3 4 0

87 2 2 4 3 2 0

93 5 5 5 5 3 2

94 3 3 3 3 3 0

95 4 4 3 4 5 0

Semantic

Figure D.6: Raw Data - Low - Part 6

Subject Agreement

Missing

Aspects

Accurate

Description

Mistakes

Result

Satisfaction

Mental

Effort

1 3 1 4 0 0 0

2 3 1 3 1 1 3

3 2 3 1 3 3 3

4 3 2 3 2 2 2

5 3 1 3 1 1 2

6 1 4 2 2 3 3

7 3 1 3 1 1 2

8 3 1 3 1 2 2

9 3 2 3 2 1 1

10 3 2 2 1 1 2

26 2 2 1 3 2 3

27 1 3 1 3 3 4

28 3 2 1 3 3 2

29 2 2 1 2 2 3

30 3 1 2 3 2 5

31 3 1 2 1 3 3

32 2 3 3 2 2 2

33 3 2 2 2 2 3

34 3 1 3 1 0 5

35 1 3 1 2 3 2

42 3 1 1 1 4 3

43 2 3 3 1 3 3

44 3 3 3 1 2 3

45 2 2 2 2 2 3

Perceived

Figure D.7: Raw Data - Low - Part 7

Subject Agreement

Missing

Aspects

Accurate

Description

Mistakes

Result

Satisfaction

Mental

Effort

46 2 2 3 2 1 5

47 2 2 2 2 1 3

48 2 2 2 2 2 3

62 3 1 2 2 1 4

63 3 1 2 2 2 4

64 2 2 3 1 4 3

65 3 2 3 2 2 3

66 3 2 2 1 2 2

67 3 1 4 0 0 3

68 3 1 2 2 3 2

69 2 2 1 3 2 3

70 3 3 2 3 3 3

71 1 3 1 1 2 2

72 3 2 3 1 2 2

73 3 1 3 0 1 1

82 1 3 4 3 0 6

83 3 3 2 1 1 3

84 3 2 2 2 2 4

85 2 4 2 2 3 1

86 3 3 3 1 3 3

87 3 1 2 3 3 4

93 2 2 1 2 2 3

94 2 2 2 2 2 3

95 4 1 2 1 2 3

Perceived

Figure D.8: Raw Data - Low - Part 8

D Raw Data

Subject No. Activities No. Gateways No. Nodes No. Edges Overall No. Branches No. Steps Duration (in Sec)

11 26 634 38 72 4132 1042

12 18 424 23 47 4109 1166

13 29 839 44 83 12 171 1911

14 25 431 26 57 4145 979

15 31 841 48 89 4634 1713

16 39 10 51 60 111 4260 1965

17 32 11 45 55 100 6462 2294

18 21 629 31 60 4215 1497

19 21 932 36 68 4145 1159

20 21 730 34 64 4198 1166

22 54 965 75 140 51 489 1825

23 33 843 55 98 4231 1710

24 19 425 26 51 4186 1162

25 15 421 22 43 4156 1210

36 35 845 48 93 4443 1957

37 22 832 37 69 4175 1205

38 24 632 34 66 4174 1615

39 45 653 69 122 8276 1653

40 24 12 38 46 84 4404 1840

41 18 626 28 54 2124 1287

49 60 466 68 134 4474 1579

50 46 755 65 120 64 254 1446

51 21 528 30 58 4127 1988

Figure D.9: Raw Data - High - Part 1

Subject No. Activities No. Gateways No. Nodes No. Edges Overall No. Branches No. Steps Duration (in Sec)

52 27 837 43 80 8620 1510

53 17 625 27 52 4307 1454

54 15 623 25 48 1152 843

55 16 523 25 48 4160 1431

56 17 423 24 47 496 789

57 28 11 41 45 86 4191 1010

58 55 14 71 90 161 4809 2085

59 43 12 57 69 126 81049 1687

60 28 838 44 82 8172 1089

61 18 626 28 54 41146 1110

74 29 435 39 74 4507 1776

75 26 13 41 48 89 12 374 2112

76 26 11 39 44 83 4244 1001

77 41 12 55 69 124 4516 1815

78 19 14 35 42 77 16 963 1656

79 20 931 35 66 4171 951

80 22 14 38 44 82 4155 1384

81 36 10 48 58 106 1359 1482

88 58 14 74 85 159 4253 1882

89 27 837 43 80 4272 1387

90 16 725 28 53 4759 1115

91 66 11 79 98 177 4503 3968

92 64 14 80 89 169 4267 2956

Figure D.10: Raw Data - High - Part 2

Syntactic

Subject Diameter Sequentiality Separability Cyclicity

No. Syntactical

Errors

11 30 0.5 0.715 0.353 4

12 20 0.478 0.608 0 4

13 31 0.386 0.591 0.154 6

14 23 0.5 0.513 0 4

15 32 0.313 0.561 0 4

16 41 0.333 0.494 0 2

17 38 0.327 0.558 0.533 5

18 25 0.419 0.63 0 2

19 27 0.419 0.533 0.241 5

20 27 0.382 0.679 0.267 4

22 51 0.48 0.76 0.154 1

23 28 0.275 0.341 0 2

24 23 0.5 0.667 0 2

25 19 0.455 0.737 0 2

36 41 0.542 0.721 0 3

37 27 0.27 0.633 0 3

38 29 0.471 0.667 0 3

39 35 0.232 0.49 0 8

40 29 0.239 0.472 0.553 1

41 21 0.393 0.708 0 2

49 60 0.765 0.891 0.06 2

50 47 0.492 0.66 0.2 8

51 25 0.5 0.808 0.179 3

Process Metrics

Figure D.11: Raw Data - High - Part 3

Syntactic

Subject Diameter Sequentiality Separability Cyclicity

No. Syntactical

Errors

11 30 0.5 0.715 0.353 4

12 20 0.478 0.608 0 4

13 31 0.386 0.591 0.154 6

14 23 0.5 0.513 0 4

15 32 0.313 0.561 0 4

16 41 0.333 0.494 0 2

17 38 0.327 0.558 0.5333 5

18 25 0.419 0.63 0 2

19 27 0.419 0.533 0.241 5

20 27 0.382 0.679 0.267 4

22 51 0.48 0.76 0.154 1

23 28 0.275 0.341 0 2

24 23 0.5 0.667 0 2

25 19 0.455 0.737 0 2

36 41 0.542 0.721 0 3

37 27 0.27 0.633 0 3

38 29 0.471 0.667 0 3

39 35 0.232 0.49 0 8

40 29 0.239 0.472 0.553 1

41 21 0.393 0.708 0 2

49 60 0.765 0.891 0.06 2

50 47 0.492 0.66 0.2 8

51 25 0.5 0.808 0.179 3

Process Metrics

Figure D.12: Raw Data - High - Part 4

D Raw Data

Pragmatic

Subject Correctness Relevance Completeness Authenticity Understandable Naming

11 4 5 4 4 2 1

12 5 5 1 2 6 0

13 5 5 5 5 1 1

14 5 5 2 2 5 0

15 4 4 2 2 4 1

16 4 5 5 4 2 1

17 3 3 3 3 2 0

18 5 4 3 3 3 2

19 5 5 5 5 2 2

20 4 5 3 4 3 1

22 4 5 5 5 4 1

23 5 5 5 5 3 1

24 4 4 2 3 5 1

25 5 4 2 2 6 0

36 5 5 5 6 3 1

37 5 5 4 5 4 1

38 4 5 3 4 4 0

39 5 5 6 5 3 0

40 6 5 5 5 2 0

41 5 5 4 5 4 0

49 5 6 5 6 3 1

50 4 6 6 5 1 0

51 5 5 4 3 2 0

Semantic

Figure D.13: Raw Data - High - Part 5

Pragmatic

Subject Correctness Relevance Completeness Authenticity Understandable Naming

52 5 5 4 4 3 1

53 5 5 3 3 4 0

54 5 5 2 3 6 0

55 5 5 3 4 5 0

56 3 4 2 2 6 0

57 3 5 4 4 2 0

58 5 5 6 5 4 1

59 3 4 6 5 4 1

60 3 4 4 4 4 1

61 3 4 2 2 5 0

74 3 4 4 3 4 1

75 4 5 4 4 3 2

76 3 4 4 4 2 1

77 5 4 4 5 4 0

78 5 5 5 5 4 0

79 4 4 4 4 3 1

80 5 5 5 5 2 0

81 2 4 3 3 3 0

88 4 5 6 6 3 2

89 4 5 4 4 3 1

90 5 5 3 3 4 1

91 5 5 6 6 1 2

92 2 5 6 5 2 1

Semantic

Figure D.14: Raw Data - High - Part 6

Subject Agreement

Missing

Aspects

Accurate

Description

Mistakes

Result

Satisfaction

Mental

Effort

11 2 3 1 3 2 1

12 3 1 3 4 2 2

13 1 3 1 3 3 5

14 2 3 1 3 3 3

15 2 2 1 2 3 3

16 1 2 2 1 4 5

17 3 2 3 2 2 3

18 2 2 2 2 2 3

19 3 3 1 2 2 1

20 3 0 3 0 2 3

22 2 3 2 3 4 3

23 2 3 2 4 3 5

24 3 2 3 1 2 3

25 2 2 3 1 2 2

36 3 2 3 2 2 3

37 2 1 2 2 2 2

38 3 1 3 2 2 3

39 3 1 3 1 1 3

40 3 2 3 2 2 3

41 3 2 3 2 1 3

49 3 3 2 3 3 3

50 1 3 3 1 0 5

51 3 2 2 2 2 3

Perceived

Figure D.15: Raw Data - High - Part 7

Subject Agreement

Missing

Aspects

Accurate

Description

Mistakes

Result

Satisfaction

Mental

Effort

52 4 2 2 2 2 4

53 2 2 2 2 2 3

54 3 3 3 1 1 3

55 2 2 2 2 2 3

56 3 0 3 1 1 3

57 3 1 3 2 1 2

58 3 2 3 2 3 3

59 3 2 3 3 4 2

60 3 2 3 1 1 4

61 4 0 4 0 0 2

74 3 2 3 2 2 4

75 1 3 1 2 3 3

76 2 2 2 1 3 2

77 2 2 2 2 3 4

78 1 4 1 4 4 5

79 3 1 3 1 1 3

80 3 2 3 1 3 4

81 3 1 3 2 1 2

88 3 3 2 2 3 3

89 4 2 0 0 3 2

90 3 3 2 3 2 3

91 2 1 3 1 2 4

92 2 3 2 2 2 4

Perceived

Figure D.16: Raw Data - High - Part 8

Experimental Results

Figure E.1-E.3 visualize the results as bar charts. Therefore, each bar chart consists of

three classes (i.e. low, high, total) and each class represents median values for low and

high social distance as well as the median value of both groups together.

Figure E.4-E.12 visualize the results as scatter plots. The x-axis of scatter plots indicate

the competencies (i.e., competent, confident, and familiar) in BPMN and the y-axis of

scatter plots show individual results of subjects.

E Experimental Results

Low High Total

Number

Group

Number of Activities

Low High Total

Number

Group

Number of Gateways

Low High Total

Number

Group

Number of Nodes

Low High Total

Number

Group

Number of Edges

Low High Total

Number

Group

Number of Elements

0.5

1.5

2.5

3.5

4.5

Low High Total

Number

Group

Number of Execution Paths

100

150

200

250

300

Low High Total

Number

Group

Number of Modeling Steps

200

400

600

800

1000

1200

1400

1600

Low High Total

Sec

Group

Modeling Duration (in Sec)

Low High Total

Value

Group

Naming

Low High Total

Value

Group

Level of Understanding

Figure E.1: Bar Charts - Part 1

100

Low High Total

Value

Group

Mental Effort

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Low High Total

Value

Group

Separability

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Low High Total

Value

Group

Sequentiality

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Low High Total

Value

Group

Cyclicity

Low High Total

Number

Group

Diameter

0.5

1.5

2.5

3.5

Low High Total

Number

Group

Number of Syntactical Errors

Low High Total

Value

Group

Correctness

Low High Total

Value

Group

Relevance

Low High Total

Value

Group

Authenticity

Low High Total

Value

Group

Completeness

Figure E.2: Bar Charts - Part 2

101

E Experimental Results

Low High Total

Value

Group

Agreement

Low High Total

Value

Group

Accurate Description

Low High Total

Value

Group

Missing Aspects

Low High Total

Value

Group

Mistakes

Low High Total

Value

Group

Result Satisfaction

Figure E.3: Bar Charts - Part 3

102

010 20 30 40 50 60 70

Competent

Number of Activities

Low

High

0 5 10 15 20

Competent

Number of Gateways

Low

High

020 40 60 80 100

Competent

Number of Nodes

Low

High

020 40 60 80 100 120

Competent

Number of Edges

Low

High

050 100 150 200

Competent

Number of Elements

Low

High

010 20 30 40 50 60

Competent

Number of Execution Paths

Low

High

0 200 400 600 800 1000 1200 1400

Competent

Number of Modeling Steps

Low

High

0 1000 2000 3000 4000 5000

Competent

Modeling Duration (in Sec)

Low

High

0 1 2

Competent

Naming

Low

High

0 1 2 3 4 5 6

Competent

Level of Understanding

Game

High

Figure E.4: Scatter Plots - Part 1

103

E Experimental Results

0123456

Competent

Mental Effort

Low

High

0 0.2 0.4 0.6 0.8 1

Competent

Separability

Low

High

0 0.2 0.4 0.6 0.8 1

Competent

Sequentiality

Low

High

0 0.2 0.4 0.6 0.8 1

Competent

Cyclicity

Low

High

010 20 30 40 50 60 70

Competent

Diameter

Low

High

0 2 4 6 8 10

Competent

Number of Syntactical Errors

Low

High

0 1 2 3 4 5 6

Competent

Correctness

Low

High

0123456

Competent

Relevance

Low

High

0123456

Competent

Authenticity

Low

High

0 1 2 3 4 5 6

Competent

Completeness

Low

High

Figure E.5: Scatter Plots - Part 2

104

0 1 2 3 4

Competent

Agreement

Low

High

01234

Competent

Missing Aspects

Low

High

0 1 2 3 4

Competent

Result Satisfaction

Low

High

0 1 2 3 4

Competent

Mistakes

Low

High

0 1 2 3 4

Competent

Accurate Description

Low

High

Figure E.6: Scatter Plots - Part 3

105

E Experimental Results

010 20 30 40 50 60 70

Confident

Number of Activities

Low

High

0 5 10 15 20

Confident

Number of Gateways

Low

High

020 40 60 80 100

Confident

Number of Nodes

Low

High

020 40 60 80 100 120

Confident

Number of Edges

Low

High

050 100 150 200

Confident

Number of Elements

Low

High

010 20 30 40 50 60

Confident

Number of Execution Paths

Low

High

0 200 400 600 800 1000 1200 1400

Confident

Number of Modeling Steps

Low

High

0 1000 2000 3000 4000 5000

Confident

Modeling Duration (in Sec)

Low

High

0 1 2

Confident

Naming

Low

High

0 1 2 3 4 5 6

Confident

Level of Understanding

Low

High

Figure E.7: Scatter Plots - Part 4

106

0 1 2 3 4 5 6

Confident

Mental Effort

Low

High

0 0.2 0.4 0.6 0.8 1

Confident

Separability

Low

High

0 0.2 0.4 0.6 0.8 1

Confident

Sequentiality

Low

High

0 0.2 0.4 0.6 0.8 1

Confident

Cyclicity

Low

High

010 20 30 40 50 60 70

Confident

Diameter

Low

High

0246810

Confident

Number of Syntactical Errors

Low

High

0 1 2 3 4 5 6

Confident

Correctness

Low

High

0 1 2 3 4 5 6

Confident

Relevance

Low

High

0123456

Confident

Authenticity

Low

High

0123456

Confident

Completeness

Low

High

Figure E.8: Scatter Plots - Part 5

107

E Experimental Results

01234

Confident

Agreement

Low

High

0 1 2 3 4

Confident

Missing Aspects

Low

High

0 1 2 3 4

Confident

Result Satisfaction

Low

High

0 1 2 3 4

Confident

Mistakes

Low

High

0 1 2 3 4

Confident

Accurate Description

Low

High

Figure E.9: Scatter Plots - Part 6

108

010 20 30 40 50 60 70

Familiar

Number of Activities

Low

High

0 5 10 15 20

Familiar

Number of Gateways

Low

High

020 40 60 80 100

Familiar

Number of Nodes

Low

High

020 40 60 80 100 120

Familiar

Number of Edges

Low

High

050 100 150 200

Familiar

Number of Elements

Low

High

010 20 30 40 50 60

Familiar

Number of Execution Paths

Low

High

0 200 400 600 800 1000 1200 1400

Familiar

Number of Modeling Steps

Low

High

0 1000 2000 3000 4000 5000

Familiar

Modeling Duration (in Sec)

Low

High

0 1 2

Familiar

Naming

Low

High

0 1 2 3 4 5 6

Familiar

Level of Understanding

Low

High

Figure E.10: Scatter Plots - Part 7

109

E Experimental Results

0 1 2 3 4 5 6

Familiar

Mental Effort

Low

High

0 0.2 0.4 0.6 0.8 1

Familiar

Separability

Low

High

0 0.2 0.4 0.6 0.8 1

Familiar

Sequentiality

Low

High

0 0.2 0.4 0.6 0.8 1

Familiar

Cyclicity

Low

High

010 20 30 40 50 60 70

Familiar

Diameter

Low

High

0 2 4 6 8 10

Familiar

Number of Syntactical Errors

Low

High

0 1 2 3 4 5 6

Familiar

Correctness

Low

High

0 1 2 3 4 5 6

Familiar

Relevance

Low

High

0 1 2 3 4 5 6

Familiar

Authenticity

Low

High

0123456

Familiar

Completeness

Low

High

Figure E.11: Scatter Plots - Part 8

110

0 1 2 3 4

Familiar

Agreement

Low

High

0 1 2 3 4

Familiar

Missing Aspects

Low

High

0 1 2 3 4

Familiar

Result Satisfaction

Low

High

0 1 2 3 4

Familiar

Mistakes

Low

High

0 1 2 3 4

Familiar

Accurate Description

Low

High

Figure E.12: Scatter Plots - Part 9

111

Detailed Results of Hypothesis Testing

Table F.1-F.25 summarizes the calculated values of testing the hypotheses. The single

values are defined as followed:

•

Rank

(R1|R2)

: Sum of ranks from group one

(i.e., low social distance) and

group two R2(i.e., high social distance)

•

U-Value

(U1|U2)

: Specific u-value for group one

(i.e., low social distance) and

group two U1(i.e., high social distance)

•Mean of Ranks (mu): Calculated mean of U

•Standard Deviation (σu): Standard deviation of U

•Z-Value: Calculated standard score of U

•P-Value: Result of the significance test

113

F Detailed Results of Hypothesis Testing

Number of Activities

Rank (R1|R2)1783 2682

U-Value (U1|U2)1601 607

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -3.755

P-Value <0.01

Table F.1: Number of Activities

Number of Gateways

Rank (R1|R2)2006.5 2458.5

U-Value (U1|U2)1377.5 830.5

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -2.065

P-Value 0.039

Table F.2: Number of Gateways

Number of Nodes

Rank (R1|R2)1799 2666

U-Value (U1|U2)1585 623

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -3.634

P-Value <0.01

Table F.3: Number of Nodes

Number of Edges

Rank (R1|R2)1826 2639

U-Value (U1|U2)1558 650

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -3.430

P-Value <0.01

Table F.4: Number of Edges

114

Number of Elements

Rank (R1|R2)1809 2656

U-Value (U1|U2)1575 633

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -3.559

P-Value <0.01

Table F.5: Number of Elements

Number of Execution Paths

Rank (R1|R2)2177 2288

U-Value (U1|U2)1207 1001

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.755

P-Value 0.435

Table F.6: Number of Execution Paths

Number of Modeling Steps

Rank (R1|R2)2167 2298

U-Value (U1|U2)1217 991

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.851

P-Value 0.395

Table F.7: Number of Modeling Steps

Modeling Duration (in Sec)

Rank (R1|R2)1629 2836

U-Value (U1|U2)1755 453

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -4.920

P-Value <0.01

Table F.8: Modeling Duration (in Sec)

115

F Detailed Results of Hypothesis Testing

Number of Syntactical Errors

Rank (R1|R2)2014.5 2450.5

U-Value (U1|U2)1369.5 838.5

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -2.004

P-Value 0.045

Table F.9: Number of Syntactical Errors

Sequentiality

Rank (R1|R2)2254 2211

U-Value (U1|U2)1130 1078

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.163

P-Value 0.849

Table F.10: Sequentiality

Cyclicity

Rank (R1|R2)2056 2409

U-Value (U1|U2)1328 880

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -1.691

P-Value 0.091

Table F.11: Cyclicity

Diameter

Rank (R1|R2)1755 2710

U-Value (U1|U2)1629 579

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -3.967

P-Value <0.01

Table F.12: Diameter

116

Separability

Rank (R1|R2)2347 2118

U-Value (U1|U2)1037 1171

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value 0.503

P-Value 0.617

Table F.13: Separability

Correctness

Rank (R1|R2)2104.5 2360.5

U-Value (U1|U2)1279.5 928.5

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -1.324

P-Value 0.186

Table F.14: Correctness

Relevance

Rank (R1|R2)1721 2744

U-Value (U1|U2)1663 545

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -4.224

P-Value <0.01

Table F.15: Relevance

Completeness

Rank (R1|R2)1822 2643

U-Value (U1|U2)1562 646

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -3.460

P-Value <0.01

Table F.16: Completeness

117

F Detailed Results of Hypothesis Testing

Authenticity

Rank (R1|R2)1860 2605

U-Value (U1|U2)1524 654

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -3.173

P-Value <0.01

Table F.17: Authenticity

Level of Understanding

Rank (R1|R2)2720 1745

U-Value (U1|U2)664 1544

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value 3.324

P-Value <0.01

Table F.18: Level of Understanding

Detailed Naming

Rank (R1|R2)2165 2300

U-Value (U1|U2)1219 989

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.866

P-Value 0.384

Table F.19: Detailed Naming

Mental Effort

Rank (R1|R2)2138.5 2326.5

U-Value (U1|U2)1245.5 962.5

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -1.067

P-Value 0.285

Table F.20: Mental Effort

118

Agreement

Rank (R1|R2)2269 2196

U-Value (U1|U2)1115 1093

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.079

P-Value 0.936

Table F.21: Agreement

Missing Aspects

Rank (R1|R2)2211 2254

U-Value (U1|U2)1173 1035

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.518

P-Value 0.603

Table F.22: Missing Aspects

Accurate Description

Rank (R1|R2)2196 2269

U-Value (U1|U2)1188 1020

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.632

P-Value 0.529

Table F.23: Accurate Description

Mistakes

Rank (R1|R2)2173.5 2291.5

U-Value (U1|U2)1210.5 997.5

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.802

P-Value 0.424

Table F.24: Mistakes

119

F Detailed Results of Hypothesis Testing

Result Satisfaction

Rank (R1|R2)2160 2305

U-Value (U1|U2)1224 984

Mean of Ranks (mu)1104

Standard Deviation (σu)132.212

Z-Value -0.904

P-Value 0.368

Table F.25: Result Satisfaction

120

List of Figures

1.1 ExperimentProcess .............................. 4

2.1 Psychological Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.1 Process Model of the Warehouse Scenario - Part 1 . . . . . . . . . . . . . 13

3.2 Process Model of the Warehouse Scenario - Part 2 . . . . . . . . . . . . . 14

3.3 StorageRacks ................................. 15

4.1 Experiment Planning and Definition . . . . . . . . . . . . . . . . . . . . . . 20

4.2 ResearchModel ................................ 30

4.3 ExperimentDesign............................... 31

4.4 LevelsofValidity ................................ 34

5.1 ExperimentOperation ............................. 37

5.2 Demographic Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6.1 Experiment Analysis and Interpretation . . . . . . . . . . . . . . . . . . . 43

6.2 Syntactic, Pragmatic, and Semantic Quality . . . . . . . . . . . . . . . . . 45

6.3 PerceivedQuality................................ 46

6.4 LevelofGranularity............................... 47

6.5 Process Model Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6.6 AdditionalFactors ............................... 48

A.1 EvaluationSheet................................ 74

A.2 Task Sheet 1 - Low Social Distance . . . . . . . . . . . . . . . . . . . . . . 75

121

List of Figures

A.3 Task Sheet 2 - High Social Distance . . . . . . . . . . . . . . . . . . . . . 76

B.1 Demographic Questionnaire - Low - Part 1 . . . . . . . . . . . . . . . . . . 78

B.2 Demographic Questionnaire - Low - Part 2 . . . . . . . . . . . . . . . . . . 79

B.3 Demographic Questionnaire - Low - Part 3 . . . . . . . . . . . . . . . . . . 80

B.4 Demographic Questionnaire - Low - Part 4 . . . . . . . . . . . . . . . . . . 80

B.5 Demographic Questionnaire - High - Part 1 . . . . . . . . . . . . . . . . . 81

B.6 Demographic Questionnaire - High - Part 2 . . . . . . . . . . . . . . . . . 82

B.7 Demographic Questionnaire - High - Part 3 . . . . . . . . . . . . . . . . . 83

B.8 Demographic Questionnaire - High - Part 4 . . . . . . . . . . . . . . . . . 83

C.1 Game and After Game Questionnaire - Low - Part 1 . . . . . . . . . . . . 86

C.2 Game and After Game Questionnaire - Low - Part 2 . . . . . . . . . . . . 86

C.3 Game and After Game Questionnaire - High - Part 1 . . . . . . . . . . . . 87

C.4 Game and After Game Questionnaire - High - Part 2 . . . . . . . . . . . . 87

D.1 RawData-Low-Part1............................ 90

D.2 RawData-Low-Part2............................ 90

D.3 RawData-Low-Part3............................ 91

D.4 RawData-Low-Part4............................ 91

D.5 RawData-Low-Part5............................ 92

D.6 RawData-Low-Part6............................ 92

D.7 RawData-Low-Part7............................ 93

D.8 RawData-Low-Part8............................ 93

D.9 RawData-High-Part1............................ 94

D.10 Raw Data - High - Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

D.11 Raw Data - High - Part 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

D.12 Raw Data - High - Part 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

D.13 Raw Data - High - Part 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

D.14 Raw Data - High - Part 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

D.15 Raw Data - High - Part 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

D.16 Raw Data - High - Part 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

122

List of Figures

E.1 BarCharts-Part1...............................100

E.2 BarCharts-Part2...............................101

E.3 BarCharts-Part3...............................102

E.4 ScatterPlots-Part1..............................103

E.5 ScatterPlots-Part2..............................104

E.6 ScatterPlots-Part3..............................105

E.7 ScatterPlots-Part4..............................106

E.8 ScatterPlots-Part5..............................107

E.9 ScatterPlots-Part6..............................108

E.10ScatterPlots-Part7..............................109

E.11ScatterPlots-Part8..............................110

E.12ScatterPlots-Part9..............................111

123

List of Tables

3.1 Items ...................................... 12

4.1 Goal Definition Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.2 Demographic Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.3 GameQuestionnaire.............................. 33

4.4 After Game Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.1 Allocation of Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

6.1 Syntactic, Pragmatic, and Semantic Quality . . . . . . . . . . . . . . . . . 44

6.2 PerceivedQuality................................ 45

6.3 LevelofGranularity............................... 46

6.4 Process Model Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

6.5 AdditionalFactors ............................... 48

6.6 Results of Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . 51

6.7 Results of Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . 52

F.1 NumberofActivities ..............................114

F.2 NumberofGateways..............................114

F.3 NumberofNodes................................114

F.4 NumberofEdges................................114

F.5 NumberofElements..............................115

F.6 Number of Execution Paths . . . . . . . . . . . . . . . . . . . . . . . . . . 115

F.7 Number of Modeling Steps . . . . . . . . . . . . . . . . . . . . . . . . . . 115

125

List of Tables

F.8 Modeling Duration (in Sec) . . . . . . . . . . . . . . . . . . . . . . . . . . 115

F.9 Number of Syntactical Errors . . . . . . . . . . . . . . . . . . . . . . . . . 116

F.10Sequentiality ..................................116

F.11Cyclicity.....................................116

F.12Diameter ....................................116

F.13Separability...................................117

F.14Correctness...................................117

F.15Relevance....................................117

F.16Completeness .................................117

F.17Authenticity ...................................118

F.18 Level of Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

F.19DetailedNaming ................................118

F.20MentalEffort ..................................118

F.21Agreement ...................................119

F.22MissingAspects ................................119

F.23 Accurate Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

F.24Mistakes.....................................119

F.25ResultSatisfaction ...............................120

126

Name: Michael Zimoch Matrikelnummer: 699504

Erklärung

Ich erkläre, dass ich die Arbeit selbstständig verfasst und keine anderen als die angegebe-

nen Quellen und Hilfsmittel verwendet habe.

Ulm,den .............................................................................

Michael Zimoch