Demonstrating the Effectiveness of Process Improvement Patterns [original]

Demonstrating the Effectiveness of

Process Improvement Patterns

Matthias Lohrmann and Manfred Reichert

Ulm University,

Databases and Information Systems Institute,

{matthias.lohrmann,manfred.reichert}@uni-ulm.de

Abstract. Improving the operational efficiency of processes is an im-

portant goal of business process management (BPM). There exist many

proposals with regard to process improvement patterns (PIPs) as prac-

tices that aim at supporting this task. Nevertheless, there is still a gap

with respect to validating PIPs in terms of their actual business value for

a specific organization. Based on empirical research and experience from

consulting projects, this paper proposes a method to tackle this chal-

lenge. Our approach towards a-priori validation of process improvement

patterns considers real-world constraints such as the role of senior stake-

holders and opportunities such as process mining techniques. In the sense

of an experience report, our approach as well as results are illustrated on

the basis of a real-world business process from human resources manage-

ment, covering a transactional volume of about 29,000 process instances

over the period of one year. Overall, our proposal enables practitioners

and researchers to subject PIPs to a sound validation procedure before

taking any process implementation decision.

Key words: Business Process Governance, Business Process Design,

Business Process Optimization, Process Mining, Process Intelligence

1 Introduction

Research on business process management (BPM) and process-aware informa-

tion systems (PAIS) has resulted in a multitude of contributions discussing op-

tions to improve the quality, performance, and economic viability of business

processes. Examples range from individual “best practices” [1] to comprehensive

business process quality frameworks [2]. In this context, we refer to process im-

provement patterns (PIPs) as abstract concepts to enhance particular aspects of

processes. As an example, consider the “knock-out” re-arrangement of tasks to

reach a decision within an appraisal process as early as possible [3].

To realize the full potential of PIPs in terms of practical relevance, it is nec-

essary to demonstrate their actual business value to practitioners, thus enabling

corresponding implementation decisions. To address this challenge, there exist

many propositions to empirically establish the effectiveness of PIPs, including

rather anecdotal evidence [4] as well as case studies [5] and survey-based research

2 Matthias Lohrmann, Manfred Reichert

[6]. Commonly, these approaches are based on an ex-post appraisal of qualita-

tive evidence given by process managers or other stakeholders to obtain general

insights applicable to analogous cases.

However, there is still a notable gap with regard to a-priori validation of

PIPs considering a particular application context, which ranges from an orga-

nization’s strategy and goals to its existing business process and information

systems landscape. In particular, this gap should be addressed for three reasons:

1. Similar to design patterns in software engineering [7], PIPs constitute ab-

stract concepts which may or may not be useful in a particular context.

Experience from other scenarios is thus not sufficient to take sound organi-

zational or PAIS-related implementation decisions.

2. Ex-post evidence is usually obtained from persons involved in the correspond-

ing implementation projects, which leads to a source of bias. In addition,

a-priori evaluation allows addressing a far wider spectrum of PIPs since it

is not necessary to conclude implementation projects before a PIP can be

assessed.

3. Contemporary technology, combining PAISs with process intelligence and

process mining tools [8–10], provides novel opportunities to quantitatively

and qualitatively gauge actual business processes, which should be leveraged

for scenario-specific PIP validation.

Reflecting these considerations, this paper contributes an approach towards

a-priori evaluation of PIPs for specific application scenarios on the basis of to-

day’s technical means. This enables sound implementation decisions, and extends

the relevance of corresponding propositions from BPM and information systems

(IS) research. Our approach considers both scientific rigor and practical require-

ments, and is demonstrated through an experience report covering a substantial

real-world business process. In particular when discussing the validity of our re-

search design, execution and results with practitioners, we made a number of

general observations on what will and what won’t work in a-priori empirical PIP

evaluation. Since these may be helpful as lessons learned for practical application

as well as future research, they are highlighted as key principles.

The remainder of this paper is structured as follows: Sect. 2 describes the

sample process we use to illustrate our approach and results. Sect. 3 illustrates

our approach towards addressing the research issue. Sect. 4 describes the actual

evaluation of PIPs and sample results. Sect. 5 and Sect. 6 discuss related work

and conclude the paper.

2 Sample Case: Applications Management Process

The business process we use to illustrate the concepts presented in this paper

stems from the field of human resources management. It addresses the handling

of incoming job applications as implemented in a professional services firm.

On the basis of discussions with stakeholders and the results of process min-

ing, we can model the sample business process as presented in Fig. 1, using

Demonstrating the Effectiveness of Process Improvement Patterns 3

Business Process Model and Notation (BPMN) [11]. For the sake of brevity, we

slightly simplify the model. If an application is received, the organization has to

decide whether to offer a job contract to the applicant. To this end, documents

are checked first by the recruiting function, and then by the business unit. If

both checks are passed, an interview is planned and executed. If the business

unit confirms its interest in hiring the candidate after the interview, approval

by senior management is obtained before a job contract can finally be offered to

the candidate who may then accept or decline the job offer.

Application

Recruiting function

Business

unit

Receive

appli-

cation

Check

docu-

ments

Docu-

ments

ok?

Assign

contact

partner

Send

letter of

refusal

yes

Check

docu-

ments

Inter-

view? Plan

interview

Execute

interview

Job

offer?

yes Send

contract

Confirm

with-

drawal

Receive

signed

contract

Event sub-process

Withdrawal

by applicant

Obtain

approval

Validate

job offer

Appr-

oved?

yes yes

Symbols:

Tasks Split / join gateways

Start / intermediate / end events

Fig. 1. Sample Process: Handling Incoming Applications (BPMN Notation)

Fig. 2 provides an overview on the distribution of termination states for our

sample process covering a time period of one fiscal year, i.e., the frequency of

possible states an application process instance might terminate in. We will refer

to this overview when discussing our research execution in Sect. 4. A correspond-

ing data sample obtained from the log database tables of the corresponding PAIS

(in this case, a SAP ERP system) was used for our analyses. The data sample

includes 27,205 cases (i.e., process instances) traceable in the PAIS. The 1,972

cases not included comprise, for example, cases handled in the business units

without involvement of the human resources (HR) department.

Additional facts and empirical analyses generated in the process mining tool

Disco and the statistical tool Minitab for the data sample are available in [12].

Note that Disco and Minitab were selected as examples of tools available to

practitioners, alternatives might be employed as well.

3 Methodology

Like other IS artifacts, PIPs constitute goal-bound artificial constructs in the

sense of the design science paradigm [13] to be evaluated in terms of “value or

utility” [14]. In our context, this results in a particular challenge: while PIPs con-

stitute abstract concepts applicable to a broad range of scenarios, their business

4 Matthias Lohrmann, Manfred Reichert

29,177

17,081

5,101 2,129 388 1,541 2,937

100% 59%

17%

7% 1% 5% 10%

Incoming

applications Declined by

recruiting

department

Declined by

technical

department

Declined after

interview Job offer not

approved Withdrawal by

applicant Applicant hired

Critical cases

Note: analysis includes internships, but no back office applications

(e.g., personal assistants); withdrawal by applicant can occur at

Fig. 2. Termination States of the Application Process: One Fiscal Year Sample

value must be determined with regard to a particular case to enable actual imple-

mentation decisions. To this end, we employ an extended conceptual framework

summarized in Fig. 3.

ResearchDesign

Process improvement

objectives

Business process /

application scenario

Process improvement

measures

Organizational

objectives Process improvement

patterns

Specific Generic

“What?”

“How?”

Additional concepts

Original concepts

Process improvement

objectives

Business process /

application scenario

Process improvement

measures

Organizational

objectives

Process improvement

patterns

Generic Specific

“What?”

“How?”

Additional concepts

Original concepts

Fig. 3. Extended Conceptual Framework

Beyond the concepts of PIPs and business processes or application scenar-

ios, we introduce organizational objectives, process improvement objectives, and

process improvement measures:

1. Organizational objectives reflect strategic goals an organization

wants to achieve, like, for example, cost savings. In principle, respective

objectives are generic, but how they are prioritized against each other is

specific to an organization’s strategy.

2. Process improvement objectives (PIOs) comprise characteristics

that enhance a process considering organizational objectives. PIOs

may be evident, like lowering cost, or they may require additional validation.

Consider, for instance, short cycle times: while it is not necessarily a strate-

gic goal to enact processes as fast as possible, this may be a PIO if a link

Demonstrating the Effectiveness of Process Improvement Patterns 5

between cycle times and cost can be demonstrated. PIOs provide an addi-

tional layer of abstraction as a “shortcut” between improvement measures

and organizational objectives. For the above example, measures might be

validated by demonstrating a positive impact on cycle times instead of over-

all cost. Note that the concept of PIOs corresponds to the requirement to

identify stakeholders’ goals as demanded in [15].

3. Process improvement measures (PIMs) are bundles of actions con-

sidered jointly for implementation.1They reflect the application of PIPs

to a specific process to realize PIOs. Note that, in many practical cases, mul-

tiple PIPs are bundled into one PIM to be jointly implemented, depending

on the application context. Accordingly, a PIM is a set of one or more PIPs

associated with a specific business process to address one or more particular

PIOs. A-priori validation of PIPs for a particular application context thus

amounts to assessing the business value of the corresponding PIMs consid-

ering relevant PIOs.

Note that, following the arrows, Fig. 3 can also be read as a methodological

top-down approach for process improvement that has proven its value in practice,

and is applied in Sect. 4: General organizational objectives are refined to PIOs

specific to the business process or application scenario. Then, corresponding

PIPs are selected from a generic set to be bundled into concise PIMs, again

under consideration of specifics of the business process or application scenario.

Key Principle 1 (Top-down Process Improvement Methodology). We

found top-down methodologies aiming at process improvement to be the most

stable in terms of change management. In principle, this paradigm is based on

an early senior management agreement on the general “call for action” (see the

concept of organizational objectives), which is then refined into process-related

objectives (see the PIOs concept), and finally into individual improvement mea-

sures considering the specifics of the process in question. PIPs or industry-specific

“best practices” (e.g., [16]) can be used to identify measures. The advantage of

this top-down approach lies in focusing the discussion of individual measures

on how things are to be achieved instead of what to achieve in general, and in

facilitating change management through an early agreement on basic principles.

In the sense of this paper, business processes and PIMs as our unit of study

are implemented by means of PAISs. To maintain scientific rigor, their assess-

ment should take into account requirements posed towards the empirical eval-

uation of propositions in software engineering or IS research. In [15], Wieringa

et al. subsume methodological considerations on scientific evaluation in IS en-

gineering. Accordingly, this section proceeds by aligning the basic requirements

described in [15] with regard to problem statement and research design to the

conceptual framework of Fig. 3.

1In this context, the term “measure” is not to be understood as an means of measuring

something (e.g., a performance indicator) or as an unit of quantity.

6 Matthias Lohrmann, Manfred Reichert

3.1 Problem Statement

Our problem statement (cf. Fig. 4) is structured along the requirements towards

effective empirical research [15]. It refines general principles for empirical vali-

dation of PIPs as appropriate for our sample case, and describes relevant key

success factors. Note that the research question refers to PIMs instead of PIPs.

This characteristic reflects our goal of scenario-specific validation.

Problem Statement

Problem

Statement

Components of

the problem

statement

Unit of Study

“About what?”

Research Question

“What do we want to

know?”

Relevant Concepts

“What do we know in

advance?”

Research Goal

“Why do we want to

know?”

statement

Application to

empirical

validation of PIPs

>Should PIMs be

implemented to

improve on organi-

zational

objectives?

>The business process

in question

>Proposed PIMs

i i PIP

>Related work: con-

ceptual work on the

PIPs, case studies for

other processes etc

>Implementing PIMs

will result in cost and

risks incurred (e g

validation

PIPs

Sample Case-

specific

refinement

zational

objectives?

>Should PIMs be

implemented to

reduce cost per hire

(cf Section 4 1)?

compr

PIP

>Application manage-

ment process (cf.

Section 2)

>Selected PIMs

(

cf.

other

processes

etc

>Cost per hire bench-

mark (cf. Section 2)

>Available research on

“

knock

out

”

processes

risks

incurred

process disruptions).

Thus, implementation

decisions should be

based on sound

investigation

refinement

Key success

factors

(cf

Section

1)?

>Proper demonstration

of the relevance of

organizational objec-

tb dd d

(

Section 4.3)

>Proper selection of

relevant PIPs to be

bundled into scenario-

ifi PIM

knock

out

processes

>Proper research into

available literature

>Use of available orga-

> Proper consideration

of related invest-

ments, transactional

ves

e a

resse

spec

ifi

PIM

snizational knowledge vo

ume e

Fig. 4. Problem Statement: Structure and Application

3.2 Research Design

Proper research design is the second requirement towards any scientific evalua-

tion [15]. Fig. 5 summarizes our considerations in this regard.

Research Design

Research

Design

Unit of data

Unit

data

collection

Measurement

instruments

Data analysis

procedures

Measurement

procedures

> As-is process description

incl. flowchart

> Process instances sample

Environment of

data collection

instruments

procedures

>Example: cf. Section 2

> Environment of recon-

> Process mining

> Semi-structured

stakeholder interviews

>Example: process mining

with Disco, telephone and

> Process enactment tracing

capabilities, statistical

process control

> PIMs description for

stakeholder interviews

> Quantitative analysis of

process mining results

> Qualitative analysis of

stakeholder interview

results

ciliation with stakeholders

>Example: the field: head of

recruiting, business unit HR

partner, business unit

managers

with

Disco,

telephone

and

face-to-face interviews

based on PIMs description,

recruitment center site visit

>Example: application

management workflow log

data, PIMs as described in

Section 4.3

>Example: statistical

analysis with Minitab,

stakeholder reconciliation

as described in Section 4.3

Fig. 5. Research Design

As discussed, in practical scenarios, PIPs are bundled into PIMs addressing

PIOs. Accordingly, the scenario-specific business value of PIPs can be demon-

Demonstrating the Effectiveness of Process Improvement Patterns 7

strated by validating the respective PIMs’ positive impact on PIOs which are,

in turn, desirable with respect to organizational objectives. Our research design

seeks to establish these characteristics by combining quantitative analysis based

on, for example, process enactment logs, and qualitative discussion with stake-

holders. Selected methods must be properly aligned to the application context:

Key Principle 2 (Appropriate Qualitative or Quantitative Demon-

stration of Business Value). For each PIO to be addressed by PIMs, the

underlying business value must be empirically demonstrated based on proper

qualitative or quantitative analyses with respect to organizational objectives.

Likewise, the business value of PIMs must be made transparent through appro-

priate analyses. While applicable methods are summarized in Fig. 5, the specific

approach for individual PIOs and PIMs must consider the actual application

context in order to balance expected insights against analysis effort. For exam-

ple, the omission of obviously redundant tasks not contributing to the business

objective of the process can be justified by a short qualitative description. In

contrast, the introduction of additional control tasks to diminish defects later

on in the process will require careful quantitative weighing of pros and cons.

Note that, regardless of the appropriate method of demonstrating business

value, all types of PIOs and PIMs require the context of a concise description

and a well-understood business process, and the cooperation of knowledgeable

stakeholders in the field as unit and environment of data collection (cf. Fig. 5).

Key Principle 3 (Identifying Relevant Stakeholders as Interview

Partners). To ensure the validity of measurement procedures, proper selection

of interview partners is of particular relevance for PIOs and PIMs that should

be validated qualitatively. For BPM scenarios, it is important to interview ex-

perienced senior personnel overlooking the end-to-end business process, and to

represent both the “supplier” and the “customer” perspective to avoid lopsided

optimization. For our sample process, we interviewed the head of recruiting oper-

ations and the administrator of the application management workflow from the

“supplier” side, and the HR partner of a business unit as well as team managers

from the business unit from the “customer” side.

4 Sample Case: Process Improvement Patterns Validation

We now apply the conceptual framework from Fig. 3 and our research design

from Sect. 3.2 to the process described in Sect. 2.

4.1 Organizational Objectives

As discussed in Sect. 3.1, obtaining clarity about the content and business value

of organizational objectives is an important prerequisite to ensure relevance of

PIP validation. For our process, the following considerations apply:

8 Matthias Lohrmann, Manfred Reichert

Reducing Cost per Hire as Organizational Objective. In our sample

application field (i.e., recruiting), organizations aim at filling vacant positions

quickly, cost-effectively, and with suitable candidates. To achieve this goal, per-

sonnel marketing is tasked to generate a sufficient number of suitable candidates

while the purpose of our sample process, application management, is to convert

applications into actual hires. In this context, the cost per hire key performance

indicator captures the total cost of both personnel marketing and applications

management. While recruiting cost spent per application is proprietary data,

based on client projects experience we may assume an amount of about 400

Euros. In our sample scenario, generating and managing 29,000 applications per

year would thus sum up to 11.6m Euros total cost. Accordingly, cost per hire may

be assumed to be around 4,000 Euros. Since hiring cost for talent in professional

services will be higher than in, e.g., manufacturing, this value corresponds well

to the average of 4,285 USD reported as cost per hire for larger organizations

by a benchmarking organization [17], and seems rather conservative considering

that professional recruiting consultants commonly charge half a year’s salary

for successful hires, depending on industry. This calculation demonstrates the

relevance of reducing cost per hire through an improved application handling

process.

4.2 Process Improvement Objectives (PIOs)

PIOs pertain to characteristics of the business process that affect the organiza-

tional objectives we want to improve on. Thus, they serve as a “shortcut” to

facilitate the discussion of the business value of PIMs without reverting to fun-

damental objectives. In our sample case, cost per hire is driven by the general

efficiency of the application management process, but also by its effectiveness

with regard to avoiding the termination states marked as “critical” in Fig. 2:

–Not approving a job offer after a successful interview may be caused by

defective steering of capacities (i.e., job vacancies), defective communication

of terms to be offered, or defective review of application documents.

–Job offers declined by applicants means that the applicant does not approve

of conditions offered, did not have a good impression during the application

process, or has decided to take another job offer.

Since terminating the process in these states means that significant effort

has been incurred with no business value in return, organizational objectives are

clearly violated: On average, only one out of six applications will successfully pass

interviews. However, considering defective termination events (cf. Fig. 2), only

one out of ten applicants can be hired. In other words, if the process enactment

defects lined out could be fully eliminated, only about 18,000 applications would

have to be acquired and managed to cover demand. This would reduce total

hiring cost by about 4.6m Euros. Accordingly, we seek to identify PIOs as process

characteristics which are apt to reduce the probability of the defective cases

described. Table 1 describes our results in this respect.

Demonstrating the Effectiveness of Process Improvement Patterns 9

PIOs Rationale

Reducing processing cost Emerging potentials in terms of reducing process enact-

ment effort per instance should be addressed.

Reducing failed

approvals

Final approval of job offers by senior management fails if

there are issues regarding vacancy management, reconcili-

ation of terms, or checking of documents. The probability

of these “late defects” should be addressed.

Reducing cycle times The probability of applicants’ obtaining and taking al-

ternative job offers increases with time. Therefore, cycle

times between applications being received and job offers

made should be as short as possible.

Table 1. Sample Case: Process Improvement Objectives

Note that for the first PIO, Reducing processing cost, there is an evident link

to our organizational objective of reducing cost per hire. However, the second

and third PIOs, Reducing failed approvals and Reducing cycle times, are based

on hypotheses on how process enactment defects which affect the organizational

objective can be reduced. Accordingly, they require qualitative or quantitative

evidence with regard to their relevance in terms of reducing enactment defects

and thus, in the end, improving cost per hire.

For the second PIO, Reducing failed approvals, we obtained qualitative ev-

idence by interviewing responsible managers, which confirmed the underlying

topics described in Table 1. Since the reasons for failed approvals are not cap-

tured in the applications management PAIS, quantitative evidence is not avail-

able. For the third PIO, Reducing cycle times, the causal link to its underlying

defect of applications withdrawn by candidates is not as obvious. However, the

correlation can be quantitatively demonstrated:

Correlation between Job Offers Declined and Cycle Times. We want to

determine whether there is a significant influence of cycle time between applica-

tion receipt and job offer in weeks on the probability of an applicant accepting

or declining a job offer. Accordingly, we use a binary logistic regression test to

evaluate the influence of a metric independent variable on a binary dependent

variable. For the test, we use a sample of 2,721 job offers representing about

70% of the annual volume (cf. Fig. 2), and consisting of instances fully covered

in the PAIS (not all interviews and feedbacks are documented in the PAIS). The

sample contains 261 cases where the job offer was eventually declined by the

applicant. This is the latest point in the process where withdrawal by the appli-

cant is possible, and a significant amount of effort will have been spent on each

respective case. Both independent samples have a size of more than 100 cases.

Thus, the binary logistic regression can be applied. Fig. 6 shows an excerpt from

the output of the statistical software package we used (Minitab). The p-value of

10 Matthias Lohrmann, Manfred Reichert

less than 0.05 indicates sufficient evidence to assume a significant impact of cycle

time. According to the “Odds Ratio” column, a one week delay can be expected

to increase the probability of an applicant declining a job offer by 16%.

Minitabtest

Logistic Regression Table Odds 95% CI

Predictor Coef SE Coef Z P Ratio Lower Upper

Constant -2,88227 0.130509 -22.08 0,000

Duration Weeks 0.0484205 0.0202145 2.40 0.017 1.05 1.01 1.09

P-Value: Probability

of duration not being a

relevant factor

Odds Ratio: Lowering

duration by one week

expected to reduce

withdrawal risk by 5%

Logistic Regression Table Odds 95% CI

Predictor Coef SE Coef Z P Ratio Lower Upper

Constant -2.58986 0.169500 -15.28 0.000

Duration_weeks 0.144378 0.0635831 2.27 0.023 1.16 1.02 1.31

P-Value: Probability

of duration not being a

relevant factor

Odds Ratio: Lowering

duration by one week

expected to reduce

withdrawal risk by 16%

Fig. 6. Minitab Output Excerpt: Binary Regression Test

Key Principle 4 (Considering Labor Relations and Restrictions in

Quantitative Analyses). Quantitative analyses to support PIO and PIM val-

idation like, for example, the use of process mining requires the collection of

data on actual process enactment. However, it is not a research objective to

assess individual performance of employees. In Germany, for example, inten-

tions in this respect are subject to worker participation regulations according

to the Betriebsverfassungsgesetz (the federal code on co-determination). Thus,

researchers must be careful when designing quantitative analyses and the re-

spective data collection procedures. Otherwise, organizations may refrain from

providing relevant data.

4.3 Process Improvement Measures (PIMs)

To address the PIOs described in the previous section, relevant PIPs are bun-

dled into PIMs specific to the application scenario. In our case, PIPs have been

selected from a framework by Reijers & Liman Mansar [1] on process redesign

practices (these are marked with an asterisk “*”) as well as from our ongoing

research on improving business process quality [18, 19]. Accordingly, as a men-

tal technique to identify propositions applicable to our application scenario, we

reflect available results on PIPs against our PIOs to obtain PIMs (cf. Fig. 3).

Table 2 summarizes PIOs and corresponding PIMs as bundles of PIPs.

Key Principle 5 (Prospective and Retrospective Identification of

PIOs and Potential PIPs). Relevant PIOs and potentially applicable PIPs

should be identified not only by prospectively considering the process model,

but also by retrospectively analyzing empirical data on process enactment. This

is crucial to focus on topics of actual value potential. For example, consider the

selection of critical cases in Fig. 2 that is reflected in the PIOs for our sample

case.

Demonstrating the Effectiveness of Process Improvement Patterns 11

PIOs Applicable PIMs with Comprised PIPs

Reducing processing cost PIM 1 (Application Management Automation): Task au-

tomation*, routing automation

Reducing failed

approvals

PIM 2 (Utilization and Capacity Management): Empower-

ment*, knock-out*

PIM 3 (Standardized Terms and Conditions): Triage*,

buffering*

Reducing cycle times PIM 4 (Managing Interview Feedback Cycle Times for

Successful Applicants): Control addition*, routing automa-

tion, escalation procedure

PIM 5 (Improving Application Routing): Case manager*,

knock-out*, mitigation of repetitive loops

Table 2. Defining Process Improvement Measures for Process Improvement Objectives

In actual design and implementation projects, it is common to document and

track individual PIMs through measure cards. As an example of this practice,

we choose two PIMs from Table 2 and describe them in more detail. For each

PIM, we give a short content description – with PIPs involved marked as italic

– and required implementation effort. On that basis, we appraise the business

value considering the impact on PIOs as well as implementation effort. Results

of our appraisal are validated through interviews with respective stakeholders.

Key Principle 6 (Considering Implementation Effort in Business

Value Appraisal). When discussing the business value of particular PIMs for

a business process, the respective implementation effort, including measures re-

quired, cost, time, and change management issues (e.g., training personnel to

enact new activities), must be taken into account. Accordingly, a PIM will pro-

vide business value only if implementation effort is justified by realized process

improvement potentials. For example, an organization may demand that the re-

quired investment may not exceed three times the projected annual cost savings

when appraising operational cost optimization measures.

Note that, in addition to the scope presented here, actual measure cards

comprise additional information relevant to project management such as project

planning, project organization, key milestones with “traffic light” status, risks,

next steps, and decision requirements. Reporting on measure cards usually takes

place in steering committee meetings of senior management.

PIM Card 2 (Utilization and Capacity Management). Among other reasons,

senior managers refuse to approve job offers when the business unit wishing to hire a

candidate cannot fully utilize present capacity (this is a common performance indicator

in professional services). While refusal reasons are not tracked in the PAIS, stakeholder

12 Matthias Lohrmann, Manfred Reichert

interviews resulted in an estimate of about 30% of total refusals to be caused by this

issue. Since candidates’ qualifications, in particular in graduate recruiting, are mostly

not specific to particular business units, the recruiting department can be empowered

to route applications to more appropriate teams from the start on. This results in an

early knock out of applications that would, in the end, be declined because of low uti-

lization.

Implementation. To enable utilization-based routing decisions, a new report on uti-

lization per team must be integrated into the application management workflow. Since

relevant data is available, and is routinely checked at other spots, the corresponding

implementation effort has been estimated to be 25 consultant days or 27,500 Euros.

In addition, relevant utilization thresholds must be agreed and communicated. The

recruiting center routes about 12,000 applications per year. If the additional operating

effort for the utilization check can be assumed to be 10 minutes per application, this

results in an overall additional capacity requirement of about 1.2 full time equivalents

(FTEs), resulting in approximately 84,000 Euros annual cost.

Business value. The PIM is expected to reduce the “late refusal” rate by about 30%

or 120 cases per year. Assuming a rate of job offers declined by the applicant of 7% (cf.

Sect. 4.2), this would reduce the number of applications to be generated and managed

to achieve a constant volume of hires by about 1,200. As we assumed the cost per appli-

cation to be about 400 Euros, an annual savings potential of 480,000 Euros compares

to 27,500 Euros one-off cost and 84,000 Euros operating expenditure per year.

Stakeholder verification. When discussing the PIM with senior stakeholders, its

business value appeared as rather clear. However, the distribution of utilization data

emerged as a “political” issue. Considering present organizational culture, the PIM will

not be implemented, but the basic capability to add utilization control functionality to

the PAIS will be included with the requirements definition for the new PAIS solution

to be completed by early 2013.

The abbreviated measure card presented above exemplifies how PIM im-

plementation benefits can be projected and matched against expected imple-

mentation effort. However, beyond this quantitative reasoning, qualitative (or

“political”) topics can play a role in implementation decisions as well.

PIM Card 4 (Managing Interview Feedback Cycle Times for Successful Ap-

plicants). The time span between successful interviews and job offers can be reduced

by implementing a control addition, i.e., additional control flow elements to ensure the

correct enactment of the process. Triggered through routing automation, the recruiting

department will call the interviewer directly when feedback is not available five busi-

ness days after an interview. If the interviewer cannot be reached within two business

days, an escalation procedure will take place by calling the respective supervisor. If

no feedback can be obtained through these PIPs within ten business days, a letter of

refusal will be sent.

Implementation. To implement the PIM, comprehensive tracing of interview dates

and an additional workflow with corresponding triggering mechanisms must be imple-

mented in the PAIS. This results in an estimated cost of approx. 38,500 Euros for 35

consultant days. In the data sample used for the binary regression test in Sect. 4.2,

about 51% of cases would fall under the proposed regulations. Accordingly, a total

volume of 7,000 interviews conducted annually (cf. Fig. 2) would result in about 3,600

escalation procedures. On the one hand, this number can be expected to decline over

time. On the other hand, multiple phone calls might be necessary for one escalation

Demonstrating the Effectiveness of Process Improvement Patterns 13

case. Hence, we assume that 20 escalation procedures can be handled in one person

day. This means that an additional 0.9 FTEs are required, resulting in about 63,000

Euros annual cost.

Business value. Based on our binary logical regression analysis (cf. Sect. 4.2), we rec-

onciled with stakeholders that the maximum interview feedback time can be reduced

to two weeks based on an escalation process. Applying the corresponding odds ratio

(cf. Sect. 4.2) to all cases in our sample which exceed this timeframe results in a re-

duction of 39.2 cases of “late withdrawals” (cf. Fig. 2). This would reduce the number

of applications to be generated and managed by about 390 per year, corresponding

to 156,000 Euros in annual savings. Considering additional operating expenditures of

63,000 Euros results in a total annual cost reduction of 93,000 Euros versus a one-off

cost of 38,500 Euros.

Stakeholder verification. During stakeholder interviews, we validated implementa-

tion cost with the application workflow administrator, additional processing effort at

the recruiting center with the head of recruiting, and overall viability of the new process

with the head of recruiting and the business unit HR partner. The escalation procedure

to provide timely feedback was challenged by the business unit HR partner, but not

by team managers. Final consent on the positive business value of the PIM could be

achieved by discussing the quantitative analysis of the underlying PIO (cf. Sect. 4.2).

Note that the PIMs presented exhibit a fairly positive business case with

implementation cost to total annual savings ratios below two years. They con-

stitute good examples of a phenomenon often encountered in practice: in many

cases, it is interesting to first identify and resolve existing process defects within

the framework of available technology before additional process automation is

implemented at huge cost.

Key Principle 7 (Leveraging “Quick Win” Potentials). In many practi-

cal scenarios, it is possible to identify “quick win” PIMs that can be implemented

with limited effort and should thus be prioritized, in particular in comparison

to full-scale PAIS implementation measures which are typically very costly. Ex-

amples include the elimination of process defects caused by process participants’

behavior, interface issues between departments, or issues with respect to mas-

ter data quality. Note that these topics are often identified through empirical

analyses (e.g., using process mining).

5 Related Work

Approaches aiming at empirical validation of PIPs can be traced back to quality

management and business process reengineering approaches which have evolved

since the 1950s and the early 1990s, respectively.

In terms of quality management, Six Sigma [20] is of particular interest be-

cause it aims at eliminating errors in manufacturing processes. While the scope

of BPM usually lies in administrative processes instead, there are interesting

analogies since Six Sigma is based on step-by-step optimization of production

processes through a-priori experimental changes to parameters.

14 Matthias Lohrmann, Manfred Reichert

Business process reengineering, as exemplified in [4, 21], aims at optimizing

processes “in the large” instead of implementing incremental PIPs. Transferring

process enactment to an external supplier or customer constitutes a good exam-

ple of this paradigm. While the potentials of this disruptive approach may seem

tempting, later empirical research has shown that the risk of projects failing is

substantial [22]. Thus, incremental improvement may still be a valid approach.

In contemporary BPM, [6] proposes a framework to select and implement

redesign practices. As opposed to our research, this approach does not aim at

evaluating individual PIPs, but at efficiently appraising a broad framework of

practices. However, we use earlier results from the same authors as a source of

PIPs to be assessed in more detail [1]. Note that research on PIPs addresses the

quality of business processes in the sense of business content. In contrast, [23–25]

exemplify propositions on process model quality in terms of structure, i.e., the

proper partitioning of actual business content into model elements.

In IS research, there have been diverse propositions to ensure common stan-

dards of scientific rigor in empirical research such as field experiments or case

studies [26, 27]. As a basis of this paper, we chose the requirements summary by

Wieringa et al. [15] due to its concise, checklist-based character, which makes it

readily applicable to research as well as discussion with practitioners.

6 Conclusion

This paper described a methodology for a-priori, scenario-specific validation of

process improvement patterns based on organizational objectives, process im-

provement objectives, and process improvement measures. In our methodology,

we leveraged available work on generic requirements towards empirical research

in IS engineering [15], thus demonstrating how these principles can be applied

to practical cases while ensuring the general appeal of our approach.

We reported on the application of the methodology to a real-world business

process, including validation of the respective results with practitioners. The

organization hosting our sample application scenario is currently implementing

a new application management PAIS to be completed in 2013. The agreed PIMs

have been included in the requirements definition for this project.

In particular when discussing research design and execution with stakehold-

ers, we identified a number of key principles useful as lessons learned for subse-

quent work which we included at appropriate spots in this paper.

References

1. Reijers, H.A., Liman Mansar, S.: Best practices in business process redesign: an

overview and qualitative evaluation of successful redesign heuristics. Omega 33(4)

(2005) 283–306

2. Heravizadeh, M., Mendling, J., Rosemann, M.: Dimensions of business processes

quality (QoBP). In: Proc. BPM Workshops. LNBIP 17 (2009) 80–91

Demonstrating the Effectiveness of Process Improvement Patterns 15

3. van der Aalst, W.M.: Reengineering knock-out processes. Decision Support Sys-

tems 30 (2001) 451–468

4. Davenport, T.J., Short, J.E.: The new industrial engineering: Information tech-

nology and business process redesign. Sloan Mgmt. Rev. (4) (1990) 11–27

5. zur Muehlen, M., Ho, D.T.: Service process innovation: A case study of BPMN in

practice. In: Proc. 41st HICSS, IEEE Computer Society (2008) 372

6. Liman Mansar, S., Reijers, H.A.: Best practices in business process redesign: use

and impact. Business Process Mgmt. J. 13(2) (2007) 193–213

7. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design patterns: Abstraction and

reuse of object-oriented design. In: Proc. 7th ECOOP. LNCS 707 (1993) 406–431

8. IDS Scheer: Process intelligence white paper: What is process intelligence? (2009)

http://www.process-intelligence.com.

9. van der Aalst, W.M.P.: Process Mining: Discovery, Conformance and Enhancement

of Business Processes. Springer (2011)

10. Reichert, M., Weber, B.: Enabling Flexibility in Process-aware Information Sys-

tems: Challenges, Methods, Technologies. Springer (2012)

11. The Object Management Group: Business Process Model and Notation: Version

2.0 (January 2011) http://www.omg.org/spec/BPMN/2.0.

12. Lohrmann, M., Reichert, M.: Demonstrating the effectiveness of process improve-

ment patterns with mining results. Technical Report UIB-2013-03, Databases and

Information Systems Institute, Ulm University (2013)

13. Simon, H.A.: The Sciences of the Artificial. 3rd edn. MIT Press (1996)

14. March, S.T., Smith, G.F.: Design and natural science research on information

technology. Decis. Support Syst. 15(4) (1995) 251–266

15. Wieringa, R., Heerkens, H., Regnell, B.: How to write and read a scientific evalu-

ation paper. In: Proc. 17th RE, IEEE Computer Society (2009) 361–364

16. Best Management Practice: IT Infrastructure Library. 2011 Edition (2011)

17. Society for Human Resource Management: Executive brief: What factors influence

cost-per-hire? (2012) http://www.shrm.org/benchmarks.

18. Lohrmann, M., Reichert, M.: Understanding Business Process Quality. SCI 444.

In: Business Process Management: Theory and Applications. Springer (2013)

19. Lohrmann, M., Reichert, M.: Efficacy-aware business process modeling. In: Proc.

20th CoopIS. LNCS 7565 (2012) 38–55

20. Pyzdek, T., Keller, P.: The Six Sigma Handbook. McGraw-Hill (2009)

21. Hammer, M., Champy, J.: Reengineering the corporation. A manifesto for business

revolution. HarperBusiness (1993)

22. Cao, G., Clarke, S., Lehaney, B.: A critique of BPR from a holistic perspective.

Business Process Mgmt. J. 7(4) (2001)

23. Becker, J., Rosemann, M., von Uthmann, C.: Guidelines of Business Process Mod-

eling. LNCS 1806. In: Business Process Management: Models, Techniques, and

Empirical Studies. Springer (2000) 241–262

24. Vanderfeesten, I., Reijers, H., van der Aalst, W.M.: Evaluating workflow process

designs using cohesion and coupling metrics. Comp. in Ind. 59(5) (2008) 420–437

25. Weber, B., Reichert, M., Mendling, J., Reijers, H.: Refactoring large process model

repositories. Comp. in Ind. 62(5) (2011) 467–486

26. Benbasat, I., Zmud, R.W.: Empirical research in information systems: The practice

of relevance. MIS Quarterly 23(1) (1999) 3–16

27. Jedlitschka, A., Ciolkowski, M., Pfahl, D.: Reporting Experiments in Software

Engineering. In: Advanced Topics in Empirical Softw. Eng. Springer (2007)