Towards an Evaluation Framework for Business Process Integration and Management [original]

Towards an Evaluation Framework

for Business Process Integration and Management

Bela Mutschler, Johannes Bumiller

DaimlerChrysler Research & Technology

P.O. Box 2360, 89013 Ulm, Germany

{bela.mutschler; johannes.bumiller}@daimlerchrysler.com

Manfred Reichert

University of Twente

Information Systems Group

7500 AE Enschede, The Netherlands

[email protected]

Abstract

Process-awareness in enterprise computing is a must in

order to adequately support business processes. Particu-

larly the interoperability of the (process-oriented) business

information systems and the management of a company’s

process map are difficult to handle. Process-oriented ap-

proaches (like workflow systems and enterprise application

integration tools) offer promising perspectives in this re-

spect. However, a major problem for project managers is

the accomplishment of economic-oriented assessments of

such approaches. Currently, there exists no suitable eval-

uation framework. This position paper discusses important

issues related to the introduction of such a framework. Do-

ing so, we distinguish two evaluation areas: Business Pro-

cess Integration and Business Process Management. While

the former operates at the technical level of process and

application integration, the latter addresses organizational

process topics. Starting from those two perspectives we de-

scribe benefits, evaluation criteria and metrics that are rel-

evant to set up an evaluation framework.

1. Introduction

Today, enterprises are continuously undergoing changes

which are driven by various internal and external events

[9]. Thereby, the alignment of information systems to busi-

ness processes is crucial [8]. In the automotive industry,

for example, a broad spectrum of enterprise information

systems (e.g., supplier chain management systems, prod-

uct data management systems) have to be aligned to various

processes ranging from administrative financial services to

knowledge-intensive engineering processes. Only the tight

interweavement of both processes and IT systems assures

an effective interoperability.

Without explicit knowledge about business processes in-

formation systems can only provide little support. Regard-

ing the interoperability of enterprise information systems,

process-oriented software technologies, like workflow man-

agement systems, application integration suites, or process

portals offer promising perspectives. However, the intro-

duction of such technologies, first of all, causes high costs:

business processes have to be redesigned and existing in-

formation systems have to be aligned according to the opti-

mized processes. Due to the occurrence of these additional

costs, project managers must be able to assess the benefits

as well as the cost-effectiveness of respective concepts.

Empirical studies conducted by Kleiner [7], for example,

have shown that the effort to implement process-oriented

applications can be significantly reduced when using com-

mercial workflow management components. At least this

indicates that processes can be implemented more quickly

with process-oriented software technology when compared

to classical programming. However, a major obstacle in

this context is the unavailability of an evaluation frame-

work which provides methods for the economic-oriented

assessment of process-oriented software technologies. In-

deed, cost benefit evaluation approaches (e.g., the time sav-

ings times salary approach or the hedonic wage model) exist

[11], but none of them represents an evaluation approach

that matches for process-oriented software technologies.

Costs are emphasized, but benefits are neglected and risks

are ignored. Evaluation criteria for ”process-orientation”

in enterprise computing are not included at all. Neverthe-

less, criteria and methods for economic-oriented justifica-

tions are highly needed in practice. In fact, any manager

who must decide whether to use innovative software tech-

nologies or not will demand a business case summarizing

an investments’ costs, benefits and risks.

The construction of an evaluation framework for

process-oriented software technologies has to be based on

well-defined evaluation criteria describing costs and bene-

fits, metrics to quantify these criteria, and formal evalua-

tion methods. This position paper describes our activities

towards the development of an evaluation framework to as-

sess costs and benefits of process-oriented software tech-

nologies. Doing so, we distinguish two evaluation perspec-

tives: Business Process Integration and Business Process

Management with the former as the technical enabler of the

latter. Business process integration focuses on the techni-

cal integration and interoperability of processes and appli-

cation systems (e.g., by providing middleware connectors

and message brokers) to enable seamless business process

execution. Internal integration includes all integration as-

pects within one enterprise. In contrast, external integration

focuses on cross-organizational integration patterns. Busi-

ness process management refers to the alignment of busi-

ness processes with an organization’s strategic goals. As-

pects included are the design, implementation and manage-

ment of process-oriented architectures, and the establish-

ment of process performance measurement systems, and the

utilization of process engines to control the flowlogic, to

automatically analyse process runtime data, and to support

business process changes.

Our work is part of the PAISCoBe project conducted

at DaimlerChrysler. The overall objective is to systemat-

ically identify and estimate the factors that influence the

costs (Co) and benefits (Be) of Process-aware Information

Systems (PAIS). PAIS cope with both business process in-

tegration and business process management issues. In this

project case studies, surveys, experiments and tool compar-

isons are accomplished to analyse relevant factors and their

impact on costs and benefits.

Sections 2 and 3 present important benefits, evaluation

criteria and metrics regarding the quantification of business

process integration and management. Section 4 discusses

related work. The paper concludes with a summary and an

outlook.

2. Business Process Integration

One of the reasons enterprises are facing the challenges

of integration is the way they were organized in the past.

Applications were implemented in a function-oriented way

either without any process support or with the logic of pro-

cess fragments being hard-wired in the application code.

Each business function had defined its own business en-

tities (e.g., data structures) without considering how other

information systems had represented the same entities. Al-

together, enterprise information systems were neither de-

signed to interact with each other nor with the information

systems of other enterprises (e.g., suppliers).

2.1. Background Information

Today, business operations are provided by a multitude

of enterprise-wide application systems (cf. Fig. 1). Some of

these applications include extensions to integrate business

partners in order to realize the extended enterprise. These

application systems have to be tightly integrated to provide

business process support. Another motivation for business

process integration has aroused from the need to systemat-

ically connect legacy applications with newly developed or

bought-off-the-shelf software components.

CAD, Engineering

& Manufacturing

Enterprise Content

Management

Product Lifecycle

Management

Enterprise

Resource Planning

Supply Chain

Management

Customer Relation-

ship Management

Business Integration

Figure 1. Integrating enterprise-wide infor-

mation systems.

Business process integration allows the sharing of data

information and business processes among connected ap-

plications and data sources. It is typically implemented

through the use of application-to-application modules (inte-

gration connectors), object-oriented middleware or message

brokers, and multi-tier application server platforms. Com-

mercial integration suites combine these tools and concepts,

and they provide a scalable and open platform for devel-

oping integrated end-to-end business processes. Regarding

enterprise integration two major approaches can be distin-

guished:

•Application-to-Application Integration (A2Ai). A2Ai

(or internal integration) focuses on the alignment of

business processes and applications within one enter-

prise and therefore addresses both business and tech-

nical issues. A2Ai is also known as Enterprise Appli-

cation Integration (cf. Fig. 2).

•Business-to-Business Integration (B2Bi). B2Bi (or ex-

ternal integration) focuses on the alignment of busi-

ness processes and supporting software systems that

typically span across several enterprises or business

units. Thereby, B2Bi does not only requires the ex-

change of business events between distributed trading

partners, but also demands the integration of business

processes with back-end applications.

EAI

Organizational

Architecture

IT Architecture

Business

Processes

Information

System

Architecture

Enterprise Application Integration

Application

Integration

Data

Integration

Figure 2. Enterprise Application Integration

covering applications and data structures.

Altogether, business process intelligence provides an ap-

proach for technically connecting enterprise information

systems in order to enable business process automation.

The inclusion of external trading partners and their pro-

cesses becomes possible as well, but constitutes a more ad-

vanced scenario which is outside the scope of this paper.

2.2. Evaluating Business Process Integration

In the following, we describe evaluation criteria and suit-

able metrics that help to set up an evaluation framework:

•Legacy Integration. Legacy systems are not defined

by age, language, platform, or data structure. Follow-

ing [13] an application system can be considered as a

legacy system if it is functioning in a production en-

vironment. Legacy integration focuses on the integra-

tion of enterprise information systems, i.e., on the inte-

gration on the application level and not on the process

level. Issues determining the characteristics and com-

plexity of legacy integration are related to business re-

lationships (e.g. the number of interfaces), business

interactions (e.g., the frequency of interacting with an-

other system), and transaction duration (e.g., the time

to provide a function result). Logical data definitions,

physical data formats, and aspects regarding semanti-

cal data integrity can be subject to evaluation as well.

Metrics to evaluate legacy integration include the time

to implement a system connector, the time to adapt

Business Process Integration

Evaluation Criteria

Legacy

Integration

Process

Automation

Process

Deployment

Process

Transparency

Business Conti-

nuity Management

Figure 3. BPI evaluation criteria.

a system connector if the underlying information sys-

tems change, or the time to connect a legacy system.

•Process Automation. Process automation refers to the

use of information systems to automate business pro-

cesses. Motivations for process automation are to au-

tomate the flow of activities, to coordinate the as-

signment and distribution of work among individu-

als, and to manage the completion of activities. The

benefits of process automation could be a significant

reduction of process cycle times, a shorter time-to-

market, and fewer unexpected process delays. Two

types of automation can be distinguished: fully au-

tomated processes (with no human intervention re-

quired), and semi-automated processes (with some hu-

man intervention required).

Some metrics can be used to evaluate process automa-

tion. Examples are the total number of business pro-

cesses fully automated, the time to set up a fully auto-

mated business process, the amount of resources to set

up a fully automated business process or the number of

processes that can be upgraded to fully automatic exe-

cution using a process-oriented software technology.

•Process Deployment. Enterprises cannot afford to

slowly replace or deploy business processes. Instead,

enterprises request for business agility and real-time

connectivity between people, systems, and business

entities. Facing these challenges, business process in-

tegration can be useful, as it supports the rapid deploy-

ment of business processes while leveraging the exist-

ing IT infrastructure.

Metrics that can be used to measure process deploy-

ment efficiency are the time needed to implement a

new business process (time to implement a business

process) or the time needed to adapt an already exist-

ing business process to changed requirements (time to

change a business process). Another evaluation crite-

rion concerns error costs as integration promises to re-

duce errors made during process deployment (though

not yet empirically proved). To quantify occurring er-

rors, various error metrics like defect density (DD) or

mean time to failure (MTTF) can be used. MTTF mea-

sures the time between failures and DD measures de-

fects relative to the software size (e.g., measured in

lines of code or function points). Metrics that can be

used in this context are the number of defects made

during business process deployment, the number of de-

fects occurring after business process deployment, or

the time till failure after business process deployment.

By assigning error data to financial indicators (e.g., the

costs to remove a defect) the impact of business pro-

cess integration regarding process deployment can be

quantified as a cost factor.

•Process Transparency. In an integrated IT infras-

tructure applications can be provided with knowledge

about the entire business process background (e.g.,

knowledge about network technologies, application in-

terfaces, or process participants). Such knowledge can

be used to identify (and finally optimize) cost-intensive

process activities (e.g., the unnecessary allocation of

valuable resources).

Metrics to quantify process transparency can be the

Number of Fully Traceable Business Processes or the

degree of traceable activities of a business process.

To assign process activities with costs, Activity-Based

Costing (ABC) can be used. ABC is a method for

allocating costs to products and services, and consti-

tutes therefore a means for planning and control. It

can help enterprises to gain better insights into activi-

ties and business processes by formalizing their costs.

Altogether, ABC allows attributing costs to activities

and products more accurately than traditional cost ac-

counting methods.

•Business Continuity Management (BCM). Evaluating

a process-oriented software technology Business Con-

tinuity Management (BCM) can be relevant, too. As

BCM assures the technical continuity of business pro-

cesses in the event of a disruption (e.g., the break-

down of a supporting information system) it has to be

analysed in the context of business process integration.

Generally, an incident is any event that seriously im-

pairs, interrupts or halts essential business processes at

one or more locations. An effective Business Continu-

ity Plan can be helpful to handle the disruption of busi-

ness processes. Such a plan must address basic process

assets, i.e., process activities, process participants and

owners as well as supporting information systems.

Metrics to evaluate a technology’s impact on business

continuity can be the number of process suspensions

in a given period of time, the time to restart a busi-

ness process after interruption, or the time without a

business process suspension.

Following these criteria, the impacts of a process-

oriented software technology can be assessed on a techni-

cal level. Besides, a second evaluation area can be business

process management, which particularly addresses organi-

zational aspects.

Legacy

Integration

Process

Automation

Process

Deployment

Process

Transparency

Business

Continuity

Management

time to implement a system connector

time to adapt a system connector

time to connect a legacy system

total number of business processes

fully automated

time to set up a fully automated

business process

number of fully traceable business

processes

degree of traceable activities of a

business process

number of process suspensions in a

given period of time

amount of resources to set up a fully

automated business process

number of processes that can be up-

graded to fully automatic execution

using process-oriented sw technology

time to implement a business process

time to change a business process

number of defects made during

business process deployment

number of errors occurring after

business process deployment

time to restart a business process

after interruption

time without a business process

suspension

M01

M02

M03

M04

M05

M06

M07

M08

M09

M10

M13

M14

M15

M16

M17

M18

M11 number of defects occurring after

business process deployment

M12 time till failure after business

process deployment

Figure 4. BPI evaluation criteria and metrics.

3. Business Process Management

Business process management (BPM) aims at the sup-

port of business processes using process-oriented tech-

niques and software to design, enact, control, and analyse

business processes [14]. Process modeling and analysis is-

sues as well as the system-supported control and monitoring

of processes are addressed. Its basic goal is to adequately

handle an enterprise’s process map and its evolution.

3.1. Background Information

BPM enables a new type of software architecture, not

only based on business objects, but on business processes as

well. For this purpose, BPM delivers a set of process man-

agement technologies that enable the automated orchestra-

tion of business processes and the management of related

information. BPM tools, for example, typically provide

a build time component for graphically modeling business

processes in the ”as is” and ”to be” states. Furthermore,

BPM is usually supplemented by Business Process Mod-

eling and Analysis components to support a-priori process

analyses as well as by features to support posterior process

analyses based on real process data.

Process

Diagnosis

Process Design

System

Configuration

Process

Enactment

Workflow

Management

BPM

Figure 5. The BPM Lifecycle.

Van der Aalst [14] has introduced the BPM lifecycle (cf.

Fig. 5) to illustrate all stages of a business process’ life-

cycle. Every business process has to be (re)designed in a

first step by using business process modeling and analysis

tools (Design Phase). After this, business processes are im-

plemented in the Configuration Phase resulting in process-

aware information systems (e.g., enterprise resource plan-

ning systems or product data management applications). In-

stances of the implemented business processes are executed

in the Enactment Phase. Finally, processes are analysed in

the Diagnosis Phase to identify potentials for process im-

provement (e.g., resource allocation bottlenecks). It is the

availability of specific process intelligence concepts (e.g.,

process mining) that enhances traditional workflow man-

agement approaches to BPM (cf. Fig. 5).

Besides BPM, there are many other concepts, methods

and tools that focus on the management of business pro-

cesses in enterprises. Examples include Total Quality Man-

agement, Simultaneous Engineering, Balanced Scorecards,

Six Sigma, and Business Process Reengineering. Figure 6

puts BPM in correlation to other management approaches.

Many organizations expect benefits from investments in

BPM technologies. Nevertheless, there is often only little

or no direct link between what organizations do to gain pro-

cess improvements and how successful respective actions

are. To establish such a link, evaluation criteria and metrics

to quantify assumed benefits are needed.

Management Approach BPM Borderline

Strategic Management

Value-oriented Management

Refactoring

Total Quality Management

Customer Relationship Management

Knowledge Management

Asset Management

Lean Management

Simultaneous Engineering

Change Management

Benchmarking

Balanced Scorecard

Six Sigma

Business Process Re-Engineering

KAIZEN

Activity-Based Costing Part of BPM

Part of BPM

Important appendix of BPM

Supported from BPM

Compatible with BPM

Pre-Condition for BPM

Figure 6. BPM and other approaches [12].

3.2. Evaluating BPM

In the following, we describe evaluation criteria and suit-

able metrics that help to set up a BPM evaluation frame-

work:

•Process Alignment. The process-oriented alignment

of information systems must quickly adaptable to

changes. When business processes are spread across

multiple applications, this alignment can be difficult

to sustain. To enhance process alignment, it is impor-

tant to detect discrepancies between the modeled and

the observed execution behaviour of processes, and to

continuously adapt the process models. Process Min-

ing and Delta Analyses [15] can help to detect such dis-

crepancies between modeled and observed behaviour.

Process-oriented information systems are based on ex-

plicit process models. Creating such process models is

a complex, time-consuming task. Process Mining can

help to reduce the effort for designing new or chang-

ing existing process models. Starting from logged run-

time data (Audit Trails) the focus is set on the deriva-

tion of a more optimal process model. Process mining

is not restricted to performance data, but can also ex-

tract causal relations between process activities.

Metrics to quantify process alignment based on Pro-

cess Mining activities can be the time to derive a new

process model,time to implement a process model, or

the time to redesign a business process.

•Process Implementation. Today, business processes

are usually implemented with IT support (e.g., work-

Business Process Management

Evaluation Aspects

Process

Alignment

Process

Implementation

Process

Change

Process

Flexibility

Stakeholder

Balancing

Process

Quality

Figure 7. BPM Evaluation Criteria.

flow management systems). BPM promises to realize

a faster implementation of business processes as the

implemented information systems can be aligned in a

process-oriented way.

Metrics to quantify BPM impacts regarding process

implementation this can be the time to implement a

business process or the resources needed to implement

a business process, e.g., the number of person months.

•Process Change. Evolving enterprise environments

frequently require process adaptations very often.

BPM promises to support evolving business processes.

Metrics to quantify a technology’s process change ca-

pabilities can be the time to change a business process,

or the resources needed to change a business process.

•Process Flexibility. To derive process flexibility eval-

uation criteria, it is useful to introduce different flex-

ibility levels. These levels can then be analysed sep-

arately. In this context, the Goal Question Metric ap-

proach can be used to assess stakeholder success mod-

els and to further derive useful metrics [5]. GQM is

a method for the systematic definition, establishment,

and exploitation of measurement programs supporting

the quantitative evaluation of software processes and

products [10].

Metrics to estimate process flexibility can be the time a

business process is running without external interven-

tion or the degree of on-demand resource allocation of

a business process.

•Process Quality. Process quality is a key element to

achieve high product quality. It can be only improved,

if the respective processes are well controlled (coordi-

nated). This is the case, if process errors can be ex-

cluded from the very beginning.

Process quality can be quantified based on errors [12].

Metrics to measure process quality can be first pass

yield (FPY) or six sigma. FPY is the percentage of a

process’ output objects that are free of errors and that

do not require reoperation. The goal of six sigma is to

reduce process output variation so that on a long term

basis this will result in no more than a given number of

defect (e.g., defects per million opportunities).

•Stakeholder Balancing. BPM enables an easier balanc-

ing of the competing requirements of users, acquirers,

developers, and maintainers of a process. This is im-

portant as non-balanced stakeholder interests can lead

to business process delays. One approach is the Model

Clash Spiderweb [3].

Metrics to quantify the effects of BPM regarding stake-

holder balancing can be the total number of conflicts

between all stakeholders, the time to resolve a model

clash or the complexity of a model clash spiderweb.

Process

Alignment

Process

Imple-

mentation

Process

Change

Process

Flexibility

Stakeholder

Balancing

time to derive a new process model

time to implement a process model

time to redesign a business process

time to implement a business process

resources needed to implement a

business process

degree of on-demand resource

allocation of a business process

total number of conflicts between all

stakeholders

resources needed to change a

business process

the time a business process is running

without external intervention

time to resolve a model clash

complexity of a model clash spiderweb

M01

M02

M03

M04

M05

M07

M08

M09

M13

M14

M15

time to change a business processM06

Process

Quality M11 first pass yield (FPY)

defects per million opportunities (DPMO)M12

Figure 8. BPM evaluation criteria and metrics.

Following these evaluation aspects, the impacts of a

process-oriented software technology are addressed on the

level of business processes.

4. Related Work

There are other approaches in enterprise computing that

focus on IT evaluations and economic-driven software en-

gineering research. The overall goal of these approaches is

to develop fundamental knowledge and practical techniques

to increase the value created over time by software and

IT projects, products, and portfolios [2] or to identify pro-

cess improvement potentials by analysing (real-time) pro-

cess execution data. In the following, some approaches are

sketched and discussed in the context of this paper.

4.1 The GRAAL Framework

The GRAAL Framework [16] investigates the alignment

of an enterprise’s Information and Communication Tech-

nologies (ICT) to its business processes and services. De-

scriptive goals are to acquire knowledge about how the

mentioned alignment can be generally maintained. Pre-

scriptive goals are to develop new techniques (e.g., agile

development methods) that help to maintain the alignment.

To achieve these goals it is necessary to identify and eval-

uate suitable techniques. Therefore, this framework defines

different enterprise layers (cf. Fig. 9), each of them repre-

senting a separate evaluation baseline. In our opinion, the

GRAAL framework provides a broad enterprise computing

evaluation approach. However, process-oriented evaluation

criteria are not included. The GRAAL framework rather

provides a static perspective. To assess process-oriented

software technology, however, a more focussed approach

is needed including specialized evaluation criteria, i.e., to

describe a technology’s ability to enhance business process

and information system evolution.

Physical Infrastructure

Software Infrastructure

Business Systems

Business

Business Environment

Service Privisioning

Social

World

Software

World

Physical

World

Figure 9. GRAAL Evaluation Baselines.

4.2 The e3 Value Framework

The e3 Value Framework is a multi-viewpoint require-

ments engineering method that is based on analysing e-

commerce initiatives through stakeholder-based viewpoints

[1]. Its overall goal is to define, derive and analyse multi-

-enterprise relationships, business cases and requirements.

The framework defines three evaluation perspectives each

of them representing an evaluation baseline to evaluate

stakeholder interests and derive suitable requirements. The

business value viewpoint focuses on the way of economic

value creation, distribution and consumption in multi-actor

networks. It enables setting up a prediction of revenues and

expenses, based on exchanges of valuable goods and ser-

vices between multiple actors. The business process view-

point focuses on a way to put the value viewpoint into op-

eration in terms of business processes. It examines oper-

ational fulfilment of business processes. The information

system viewpoint focuses on the information systems that

enable and support processes. Regarding the evaluation of

process-oriented software technologies, the e3 value frame-

work is helpful, but not sufficient. In fact, a more holis-

tic approach is needed (e.g., including evaluation criteria to

evaluate not only stakeholder success models and require-

ments, but process-oriented aspects). Nevertheless, this ap-

proach introduces important issues concerning the deriva-

tion of requirements.

4.3 Value-based Software Engineering

Value-Based Software Engineering (VBSE) [2] inte-

grates value considerations into software engineering prin-

ciples and practices. Seven key elements (benefits real-

ization analysis, stakeholder value proposition elicitation

and reconciliation, business case analysis, continuous risk

and opportunity management, concurrent system and soft-

ware engineering, value-based monitoring and control, and

change as opportunity) are defined that represent the foun-

dations for VBSE. As can be seen, the analyses of costs,

benefits, risks, and stakeholder interests plays a significant

role. Altogether, VBSE is an approach that combines ex-

isting techniques and management approaches with a new

value-oriented focus. Due to the enormous number of inte-

grated concepts VBSE is still in a conceptual stage. There-

fore, as there exists no VBSE best practice, it is hardly pos-

sible to transform VBSE into praxis. As VBSE focuses on

software development in general, it lacks to adequately sup-

port the evaluation of process-oriented software technology,

though interesting evaluation concepts are included.

4.4 Business Process Intelligence (BPI)

Organizations more and more realize that gaining knowl-

edge about their processes may imply many benefits that

can justify the costs of respective solutions. Business pro-

cess intelligence (BPI) applies business intelligence con-

cepts (e.g., analytical applications) to business processes

[4]. It is implemented as a set of integrated tools providing

features for the analysis, mining, prediction, control, and

optimization of processes. In particular, it provides valuable

information (e.g., about the adequacy of provided business

functions [6]) for the alignment of information systems to

business processes. It can also be used to identify critical

scenarios that may occur during the execution of a business

process (e.g., resource over-allocations or bottlenecks, un-

necessary waiting and idle times). Process mining (as an

important feature of business process intelligence) allows

for the derivation of optimized process models. This, in

turn, reduces effort-intense manual process analyses. Par-

ticularly the increasing use of process-oriented information

systems (e.g., enterprise resource planinng systems or sup-

plier chain management systems) in enterprises has been an

enabling success factor in this respect (as more and more

data can be collected real-time from process-supporting in-

formation systems). BPI can be applied using contempo-

rary BPI tools. Examples include Websphere Business Inte-

gration Monitor,ARIS Process Performance Manager and

BizTalk Server Business Activity Monitoring Framework.

Business Processes

Process-oriented

Information Systems

(Alignment)

Collection of

Execution Logs

Business

Process

Execution

Business

Process

Optimization

Business

Process

Intelligence

1

2

3

4 Supporting

Information

Systems

Process

Monitoring

Process

Warehouse

Derivation of

Process

Knowledge

Information

System

Adaptation

Figure 10. The BPI Lifecycle.

5. Summary and Future Work

Executives that have to decide whether to use process-

oriented software technologies or not typically demand for

a business case outlining an investment’s costs and bene-

fits. However, currently there exists no evaluation frame-

work to support investment decisions regarding process-

oriented software technologies. This paper has discussed

relevant issues for the development of an evaluation frame-

work to assess costs and benefits of process-oriented soft-

ware technologies. Therefore, we distinguished between

the two evaluation perspectives Business Process Integra-

tion and Business Process Management with the former as

the technical enabler of the latter. Following this distinction

we identified success-critical evaluation criteria. To quan-

tify the impacts of process-oriented software technologies

we have additionally assigned first metrics to the described

evaluation criteria.

We are aware of the problem that to set up a detailed

evaluation framework, both the evaluation criteria and the

metrics have to be described more precisely. It is our goal

not only to develop a framework which can be used to de-

scribe and evaluate process-oriented software technologies,

but process-oriented information systems as well. There-

fore, we plan to accomplish case studies, surveys, experi-

ments and tool comparisons to analyse relevant approaches

towards their economic impacts, costs and benefits. The

development of suitable cost models to quantify economic

impacts of respective investments and the transfer of the

framework into the practice are major requirements.

References

[1] Z. Baida, H. de Bruin, and J. Gordijin. e-Business Cases

Assessment: From Business Value to System Feasibility. Intl.

Journal of Web Engineering and Technology, volume 1(1),

pp.127-144, 2003.

[2] B. Boehm. Value-Based Software Engineering. ACM SIG-

SOFT Software Engineering Notes, volume 28(2), 2003.

[3] B. Boehm and D. Port. Avoiding the Software Model-Clash

Spiderweb. IEEE Software, volume 33(11), pp.120-122,

2000.

[4] D. Grigori, F. Casati, M. Castellanos, U. Dayal, M. Sayal,

and M. Shan. Business Process Intelligence. Computers in

Industry, 53, pp. 321-343, 2004.

[5] O. Jaufman. Reusage Knowledge on Process Flexibility

for Developing Measurement Programs. Proceedings of the

IWSM/MetriKon 2004, pp.495-504, 2004.

[6] P. S. Katsma. Business Function Support. Pearson Custom

Publications, 2004.

[7] N. Kleiner. Can Business Process Changes Be Cheaper Im-

plemented with Workflow-Management-Systems? Proceed-

ings of the 2004 Information Resources Management Asso-

ciation Conference (IRMA 2004), pp.529-532.

[8] R. Lenz and K. A. Kuhn. Towards a continuous Evolution

and Adaptation of Information Systems in Healthcare. Inter-

national Journal of Medical Informatics, volume 74, pp.75-

89, 2004.

[9] B. Mutschler, M. Reichert, and J. Bumiller. Towards

Process-Aware Enterprise Software Environments. Proceed-

ings of the 7th International Conference on Enterprise Infor-

mation Systems (ICEIS 2005), 2005.

[10] R. Rombach. Practical Benefits of Goal-Oriented Measure-

ment. N. Fenton and B. Littlewood, Software Reliability and

Metrics, Elsevier Applied Science, London, 1991.

[11] P. G. Sasonne. Cost-Benefit Methodology for Office Sys-

tems. ACM Transactions on Office Information Systems,

volume5(3), July 1987, pp.273-289, 1987.

[12] H. J. Schmelzer and W. Sesselmann. Geschaeftsprozessman-

agementprozess in der Praxis (4.Auflage). Hanser, 2004.

[13] U. M. Ullrich. Legacy Systems Transformation Strategies.

Prentice Hall, New York, 2002.

[14] W. M. P. van der Aalst. Business Process Management -

A Personal View. Business Process Management Journal,

volume 10(2), pp.248-253, 2004.

[15] W. M. P. van der Aalst and A. J. M. M. Weijters. Process

Mining - A Research Agenda. Computers in Industry, vol-

ume 53(3), pp.231-244, 2004.

[16] P. van Eck, H. Blanken, and R. Wieringa. Project GRAAL

- Towards Operational Architecture Alignment. Interna-

tional Journal of Cooperative Information Systems, volume

13(13), pp.235-255, 2004.