On the Calculation of Process Performance Indicators

On the Calculation of Process Performance1 Indicators ? 2 Antonio Manuel Guti´errez–Fern´andez, Manuel Resinas, Adela del–R´ıo–Ortega,3 Antonio Ruiz–Cort´es4 School of Computer Engineering5 University of Seville6 {amgutierrez,resinas,adeladelrio,aruiz}@us.es7 Abstract. Performance calculation is a key factor to match corporate8 goals between different partners in process execution. However, although,9 a number of standards protocols and languages have recently emerged10 to support business process services in the industry, there is no stan-11 dard related to monitoring of performance indicators over processes in12 these systems. As a consequence, BPMS use propietary languages to13 define measures and calculate them over process execution. In this pa-14 per, we describe two different approaches to compute performance mea-15 sures on business process decoupled from specific Business Process Man-16 agement System (BPMS) with an existing BPMS-independent language17 (PPINOT) to define indicators over business processes. Finally, some op-18 timization techniques are described to increase calculation performance19 based on computing aggregated measures incrementally.20 Keywords: Business Process Management, Key Performance Indica-21 tors, Complex Event Processing22 1 Introduction23 Nowadays, Business Process Management Systems (BPMS) massively support24 enterprise task flows and human interactions. Tools facilitate different stages25 in Business Process lifecycle: modelling, deployment, execution or monitoring.26 Furthermore, standards such as BPMN [6] or UML [9] provide visual languages27 to define tasks, workflows and decision making points so we can describe and28 analyze them independently of technology or runtime aspects.29 However, certain kinds of problems (such as bottlenecks or resource under-30 utilization) can only be detected by monitoring and measuring process executions31 and, so far, no standard provides business process monitoring or performance32 measuring. On the contrary, each BPMS uses proprietary languages to define33 ?This work has been partially supported by the European Commission (FEDER), the Spanish and the Andalusian R&D&I programmes (grants TIN2012–32273 (TAPAS), TIC–5906 (THEOS) and COPAS (P12–TIC-1867)) 2 measures and calculate them over business process execution12 . Consequently,34 defining performance measures is locked to the BPMS.35 In this paper, we focus on the definition and development of BPMS inde-36 pendent operations to compute measures over business process executions. To37 achieve this, we use a set of tools and techniques for the definition and au-38 tomated analysis of Process Performance Indicators (PPIs) [8], which includes39 a metamodel and a visual notation. This language has been applied to pro-40 cesses described with BPMN in a number of previous works, although its se-41 mantics can be applied to generic workflow languages. Computing measures as42 non-intrusive methods depends on BPMS architecture and facilities to retrieve43 information from them. To avoid intrusiveness, this proposal provides two dif-44 ferent approaches as contribution: consuming API to get relevant data once and45 capturing events along the process instances execution. In both perspectives,46 complex calculations can be computationally expensive, so we introduce incre-47 mental techniques to avoid unnecessary processing and performance deteriora-48 tion. A number of Business Processes (BPs) execution tools, such as Camunda3 49 or BIMP4are analyzed to apply these approaches and validate our proposal.50 In the next section, languages to describe performance metrics over business51 process are analysed using an example scenario. In section 3, operations on52 business process and approaches to calculate measures are proposed. Results for53 the example scenario are presented in Section 4. In the section 5, a computing54 optimization is introduced. Finally, conclusions and possible work extensions are55 described in section 6.56 2 Defining Performance Indicators57 As automation of business processes has increased, a number of standards to58 define workflows, resources assignment or decision making points have emerged.59 These standards assist business process analysis independently of the specific sys-60 tem chosen to automate them (or even in the absence of such system). Similarly,61 defining performance indicators independently of a BPMS requires a language to62 do it at design time. There already exists a number of languages for describing63 process-related performance indicators defined in the literature with, in some64 cases, supporting tools: Pedrinaci et al. [7] present a metric ontology to allow65 the definition and computation of metrics integrated in SENTINEL, a Semantic66 Business Process Monitoring Tool; Castellanos et al.’s approach [2] proposes the67 use of templates provided by a graphical user interface (integrated in the iBOM68 platform) to define business measures related to process instances, processes, re-69 sources or of the overall business operations; Momm et al.’s approach to develop70 process monitoring systems [5] includes a metamodel for the specification of the71 performance indicators monitoring and an automated generation of the required72 1http://wso2.com/products/business-activity-monitor 2http://www8.hp.com/us/en/software-solutions/business-process-monitoring 3http://camunda.org/ 4http://qbp-simulator.cloudapp.net/ 3 instrumentation and monitoring infrastructure; Wetzstein et al. introduce in [10]73 a framework for BAM as part of the semantic business process management, they74 describe a KPI ontology using WSML to specify KPIs over semantic business75 processes; Chau et al. [3] propose measuring process goals through capturing76 process events and defining an ad-hoc algorithm per measure to calculate the77 goal value.78 Nevertheless, they present several issues: its expressiveness in the type of in-79 dicators that can be defined is limited [1]; the explicit connection or relationship80 between the performance indicators and the business process elements is par-81 tially or not addressed in them, hindering its computation on existing BPMSs;82 finally, all of them are coupled to specific platforms or systems.83 An approach that overcomes the aforementioned issues is PPINOT[8], which84 proposes a language to express PPIs at design time, either graphically or by85 means of a template-based textual notation. However, although PPINOT has86 tooling support to define PPIs graphically, it lacks an implementation to compute87 them in process engines. This is why we will use PPINOT as starting point to88 define our approaches to compute PPIs in a way that is decoupled from specific89 BPMS. Metric definition in PPINOT is classified into three main categories90 depending on the number of process instances involved and the nature of the91 measure: base measures, aggregated measures, and derived measures.92 Base measures: They are obtained directly from a single process instance and93 do not require any other measure to be computed. Aspects that can be mea-94 sured include: 1) the duration between two time instants (time measures); 2)95 the number of times something happens (count measures); 3) the fulfillment96 of certain condition in both running or finished process instances (condi-97 tion measures); and 4) the value of a certain part of a data object (data98 measures).99 Aggregated measures: Sometimes, it is interesting not only knowing the value100 of a measure for a single process instance (base measures) but an aggregation101 of the values corresponding to the multiple instances of a process. For these102 cases, aggregated measures are used, together with an aggregation function103 such as average, maximum, etc.104 Derived measures: They are defined as functions of other measures. Depend-105 ing on whether the derivation function is defined over single or multi-instance106 measures, derived measures are classified accordingly as derived single-instance107 measures or derived multi-instance measures.108 To illustrate the definition of a business process and its performance indica-109 tors, we consider a simple business process for registering appointments in public110 health system. The procedure to book an appointment with doctors starts when111 the user logs. Figure 1 depicts the described process in BPMN and the related112 PPIs using PPINOT. Once a user has been identified, the system displays all113 the available days for his family doctor in the next two weeks (all users in public114 health system have always a family doctor assigned). When the User chooses115 a desirable day, the system displays available time slots for that day (15 min-116 utes slots). If the user selects one of these slots and confirms the decision, the117 4 system registers the appointment and sends a notification to the user118 personal inbox (mobile and/or email). If the user disagrees with the available119 slots, he can choose a different day to see its available slots. There are other ac-120 tivities related, such as choosing a new doctor or modify a previous appointment121 that are not considered in this example process.122 According to Spanish official stats, there are an average of 6 visits/person-123 /year5, so this process executes several hundreds of thousands times per day124 (240 million per year). This number can be even higher, considering the process125 instances that start but finish without a fixed appointment. Therefore, measur-126 ing performance indicators to optimize this process will have an impact on the127 computational resources.128 In this process, the following performance indicators are of interest:129 –Average time per appointment process instance (from the user logging to130 appointment registration)131 –Average of day changes to review time slots per doctors132 First metric gives basic performance information. Second metric gives hints133 about bad performance, as reviewing different days indicates the user finds it134 difficult to match available time slots to his preferences (useless schedule times135 or assignment between patient and doctors that could be improved).136 These two metrics are easy to define over the business process. The average137 total time for an appointment is measured as the average of the time difference138 between an appointment registration (end of task ”Register and Notify Appoint-139 ment”) and a user logs (end of task ”Identify Patient”). Time slot reviews per140 doctor is measured as the average number of times a user reviews different days141 per doctor (i.e., task ”Choose day” is executed). Both measures are Aggregated142 Measures of Time Measures in PPINOT.143 Fig. 1. Appointment Management BPMN + PPI 5http://goo.gl/QWtqTn 5 3 Methods to Retrieve Performance Information144 Despite these indicators are simple, there is no standard in BPMSs to calcu-145 late them so they require to be interpreted and transformed in terms of the146 BAM facilities. So, even using a BPMS-independent definition language, such as147 PPINOT, for defining PPIs over BPMN business processes, indicator computa-148 tion depends on the specific BPMS chosen for BP execution.149 Regardless of the process engine, operation to compute PPIs in a generic150 interface is defined by:151 –List <MeasureValue>PPICompute(PPI currentPPI)152 That is, with the PPI properties, as the type of measure or the business process153 definition and task/s related to it, execution data are collected to get the appro-154 priate values for the measure. For a PPI with base measure -as base measures155 correspond a single measure value por each process instance-, this operation will156 return as many values as process instances. For a PPI with aggregated measure,157 the scope of aggregation determines the aggregation of values from single process158 instances. In Table 1, we trace example execution data for this process and the159 expected results for the measures indicated in the previous section (final aggre-160 gated values). The result of this operation relies on having access to execution161 data from the process engine. We have identified two different approaches to get162 this data:163 –Data capture164 –Event capture165 Once we have the required execution information, the indicator value is triv-166 ially processed with mathematical calculation.167 Table 1. Execution trace BPInstance ID Process Execution Data Measure Value 1Finished in 60 sec MediumRequestTime 60 sec 3 day changes, Doctor ”A” DayChangesPerDoctor ”A”:3 2Finished in 75 sec MediumRequestTime 65 sec 4 day changes, Doctor ”B” DayChangesPerDoctor ”A”:3, ”B”: 4 3Finished in 90 sec MediumRequestTime 70 sec 5 day changes, Doctor ”A” DayChangesPerDoctor ”A”:4, ”B”: 4 3.1 Data capture168 Process engines usually persist activities in an easy-to-understand database or169 even provide facilities to query them via an API. Both procedures, database170 query and API consumption, are depicted in Figure 2. PPICompute method171 6 Fig. 2. Data capture direct from Database 1) or using a provided API 2) directly queries required task information from persistance system (Diagram 1)172 or consumes API provided by BPMS (Diagram 2).173 In this approach, the data capture provides at once all the required data for174 activities related to an indicator. Once the data are retrieved, some measures175 require post-processing these data. For example, in average time per process indi-176 cator, after retrieving timestamps from initial and final tasks, we have to get the177 difference between them for each process instance and then compute the average178 value for all the instances. Depending on the persistence or API components179 deployment, data querying may impact on process engine performance.180 3.2 Event capture181 Although data-capture based models are easy to design and implement, some182 process engines are not suitable for this approach. For instance, if we neither can183 access to persistence model nor to any data query API. In this case, we propose184 to capture relevant events6in process execution and use them to compute in-185 dicators. This approach requires extending BPMS listeners mechanisms (if they186 exist) or modifying the BPMN model (i.e.: observer pattern) to generate the187 required events so all events that occur during process execution are sent as a188 stream of events to PPICompute using a method such as (depicted in Figure 3):189 void processEvent ( RuntimeEvent event )190 Fig. 3. Event capture 6when we talk about events here, we refer to any stage of interest in process flow, not BPMN events 7 RuntimeEvent object usually includes information about process instance191 identification, timestamp and event trigger (task starting, data change, ...), so192 we can check if it is relevant for the PPI and compute it (in positive case).193 Once we get the information, we always require event data post-processing. Al-194 though events are simple to process, a continuos capture has an impact on the195 communication channel, commonly internet and http.196 4 Prototype Implementation197 Following the proposed approaches, we have developed indicator calculation com-198 ponents in different business process tools. The method to choice depends on199 BPMS capabilities or performance criteria, as both approaches are similar in200 terms of expressiveness. Camunda is an open source BPMN system forked from201 Activiti which features a REST API to get history execution so we applied data202 capture approach. On the other hand, BIMP is a business process simulator with203 no data persistence so we use event capture approach. Both approaches are204 4.1 Calculating indicators in Camunda205 Camunda interface offers methods to retrieve any activity (task, gateway or206 event) information so we can easily get timestamps for time indicators or ac-207 count tasks executions with the Camunda History Service. The implementation208 of timestamp capture for time measure in Camunda is illustrated in code 1.1,209 where camundaHistory is the Camunda History Service object.210 Listing 1.1. Data processing List < PPIValue > computeMeasure ( PPIValue ppi ){211 // startTimes is a Map < String ,Long >212 String pName = ppi . getProcessName ();213 Tim eInst antCo nditi on startTime = ppi . getFrom ();214 String startTask = startTime . getTask ();215 State startState = startTime . getChangesToState ();216 HistoricProcessInstanceQuery processQuery =217 camundaHistory.createHistoricProcessInstanceQuery();218 HistoricActivityInstanceQuery activityQuery =219 camundaHistory . createHistoricActivityInstanceQuery ();220 // Getting and Iterating Process Instances221 List < HistoricProcessInstance > processInstances =222 process Que ry . proc es sDefi ni ti on Name ( pName ). list ();223 for ( HistoricProces sInst ance pInstance : processInstances ){224 String instanceId = pInstance . getId ();225 activityQuery = activityQuery . processInstanceId ( instanceId );226 // Retrieving Measure Start execution data227 HistoricActivityInstance startActivity =228 activityQuery . activityId ( startTask ). singleResult ();229 if ( startState . equals ( Ge nericState . START )){230 star tTimes . add ( instanceId , startActiv ity . getSt art Tim e ());231 }else {232 star tTimes . add ( instanceId , startActiv ity . ge tEndTime ());233 }234 // Similar for Time End Condition in Measure235 // After getting data , we calculate Average ...236 }237 }238 8 4.2 Calculating indicators in BIMP239 As BIMP does not provide any facility to retrieve information, the event capture240 approach is used. So we capture and filter the BIMP stream of activities for241 measures of interest. The processing of an event for time capture follows the242 algorithm depicted in code 1.2.243 Listing 1.2. Event processing // startTimes , endTimes are Map < String ,Long >244 void process ( Ru nti meE ven t entry ) {245 if ( entry . matches ( Timer . START )) {246 star tTimes . add ( entry );247 } else {248 endTimes . add ( entry );249 }250 if ( startTimes . size () == endTimes (). size ()){251 //updateAverage252 average = getAverageTime ();253 }254 }255 5 Optimizing measure processing256 In both approaches, complex indicators in massive process instances scenarios257 can degrade performance. In the query method, data-capture measures, such as258 aggregations, require computationally expensive views (groups by), so real-time259 monitoring can compromise process engine enforcement if the database that260 stores history data is the same that stores runtime information. And, in both261 methods, processing a number of calculations have an impact on performance.262 Nowadays, improving big data processing is a research goal so a number of tech-263 niques and tools have been developed focusing on this topic. We take advantage264 of these techniques to extend our proposal[4].265 On this regard, we use incremental calculation technique. This technique is266 implemented by a number of tools to provide scalable processing in big data267 scenarios (such as Apache DataFu over Apache Hadoop). For the introduced268 example, to calculate average time on 30 days window, first result requires pro-269 cessing full data for 30 days. However, if we store intermediate daily times, future270 30 days windows processing can be made faster if we consider only the different271 day intervals. So, after computing the first 30 days window, calculating average272 for that window on the 31st day only requires adding data for the new day and273 removing the data from the 1st day (it is out of the new window). Incremental274 calculation is described below.275 Let InstDayithe number of process instances in a day i and TimeDayithe276 accumulated process Time for all the process instance in a day i (from ”Identify277 User” Task to ”Register Appoinment” Task):278 InstDayi=N(1) 9 Total Process Time per Day:279 T imeDayi= N X j=0 P rocessInstanceT imej(2) Accumulated Process Instances in the last 30 days:280 AccumInstDayi= j X j=i−30 InstDayi(3) Accumulated Process Time in the last 30 days:281 AccumT imeDayi= i X j=i−30 T imeDayj(4) Average Process Time in the last 30 days:282 AverageTimeDayi=AccumT imeDayi AccumInstDayi (5) If we store intermediate accumulated and daily number of instances, and283 total process time, we incrementally calculate average for the following day:284 AccumInstDayi+1 =AccumInstDayi−InstDayi−30 +InstDayi+1 (6) AccumT imeDayi+1 =AccumT imeDayi−T imeDayi−30 +T imeDayi+1 (7) AverageTimeDayi+1 =AccumT imeDayi+1 AccumInstDayi+1 (8) This process is depicted in Figure 4, where a 3-day window time is computed285 for the first 3 days and intermediate results are stored (Intermediate Stage 1).286 In our example, these intermediate data are accumulated process time and ac-287 cumulated number of instances. On the fourth day, instead of computing full288 measures for day 2, 3 and 4, we reuse previous results to extract data from day289 1 and add day 4 execution data and get the new valid result. So, measure calcu-290 lation leverages this technique to simplify queries, and avoid unnecessary data291 processing.292 In the measure for average day changes per doctor, incremental calculation293 can also be applied in simpler form since there is no need to use sliding time294 window (and subtracting old data) but just adding new daily information to295 calculation.296