A management tool for indicator-supported systems: A public health service application

204 European Journal of Operational Research 61 (1992) 204-214 North-Holland A management tool for indicator-supported systems: A public health service application E. Carrizosa, E. Conde, F.R. Fernfindez and J. Puerto Departamento de Estad[stica e Investigaci6n Operativa, Universidad de Sevilla, Sevilla, Spain Received March 1991; revised September 1991 Abstract: We develop a decision-making methodology for hierarchical structures. It provides different decision makers with decision-oriented information based on obtained satisfaction levels. This is specially convenient for public services, where the main goal is user's satisfaction. Our methodology and its associated software (INDI) have been implemented in the Andalusian Health Service (SAS), supporting resource allocation decisions in order to reduce the complaints presented against such an institution. Keywords: Decision Support Systems; decision theory; multicriteria decision making 1. Introduction Any organization in a competitive market needs some tools to react against unexpectedly increasing costs or decreasing quality. This has given an important role to the information department: monitoring the production process through information systems gives the manager the tools to compare the outputs with preset targets and check the quality of them. Until the beginning of the 80s, all that was required were Management Information Systems (MIS), the decisions being reserved to the managers, with the only aid of the reports given by the MIS. However, there has been in the last years an increasing interest in the development of normative decision models, supporting the making of non-structured or semi-structured decisions in order to systematize the process of continuous improvement. Most of the efforts have been devoted to the development of methodologies for specific decision situations (Valadares et al., 1986; Boldy, 1987; Andreu and Corominas, 1989; Vla6i6, 1989). The development of our methodology HIDS (Hierarchical Information and Decision System) and its associated software had as a starting point a contract with the Servicio Andaluz de Salud (Andalusian Health Service), known as SAS. There, we had to develop an interactive system supporting decisions about the allocation of resources in order to reduce the number of complaints in public health centers. We have developed a unified decision-making environment for supporting decisions at different levels in any hierarchical structure. This environment can be seen as a shell that the decision makers (DMs) use in order to develop their own DSS profile inside the structure. Our aim is to give all the DMs extremely manageable and meaningful decision-oriented information. The methodology is based on the analysis of the satisfaction level obtained through the outputs of the system. This satisfaction is measured by means of observed trajectories, normalized using satisfaction functions. This procedure is especially convenient for public services, where the main goal is to obtain a high level of user's satisfaction. This paper is divided into four sections. In Section 2, we describe the model and its underlying hypotheses, the Information and Decision Processes. In Section 3 we present a real application of our methodology applied to public health 0377-2217/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved E. Carrizosa et al. / Management tool for indicator-supported systems 205 services. Section 4 is devoted to the description of the software architecture that has been developed for the model. The paper ends with two mathematical appendices dealing with some aspects of the Information and Decision Processes. 2. The Hierarchical Information and Decision System (HIDS) 2.1. The model The model we propose is based on a set of hypotheses we describe below. They do not correspond to theoretical conditions, but appear in almost any real large-scale information-decision system. Due to their nature, these hypotheses are classified into three groups: - Structure hypotheses (S). - System Parameter hypotheses (P). - Preset Targets hypotheses (T). S1. The system can be represented through a hierarchical structure. $2. Any element in the system (DM) occupies only one place in the structure. $3. No element acts isolated from the rest. $4. There are two preorders on the structure, called the Decision relation and Information relation. Each one is the inverse of the other. $5. There exists only one maximal element for the Decision relation: The Super Decision Maker (SDM). For any DM, the Decision relation induces exactly one chain from the SDM to him. PI. Any DM at the lowest level in the hierarchy (terminal DM) is responsible for a unique sereice center. The performance of such a service is controlled by a stochastic process that depends on a set of parameters. T1. The performance quality of a service center i at time t is a function of a set V~(t) of numerical features (which are periodically measured), and a set of preset targets. T2. The satisfaction of a terminal DM is the performance quality of his associated service center. The satisfaction of any non-terminal DM at time t is a function of the satisfaction of those DMs subordinated to him following the Decision relation. In a natural way, the hypotheses described above lead us to a representation of the system by means of a mathematical structure H = (N, F, B), where: 1) N is the set of all the DMs in the hierarchy 2) F is the Decision relation. If (n, m)~F, we say that m is subordinated to n. 3) B is the Information relation. If (n, m) ~ B, we say that n informs m. 4) (N, F) is an acyclic connected digraph, with only one DM with zero rank: the Super Decision Maker (SDM). 5) (N, B) is an acyclic connected digraph, with only one element with maximal rank: the SDM. 6) (n, m) ~ F if (m, n) ~ B. Note that the hypotheses we imposed are explicitly used in this formulation. In fact, $1 and $2 justify the acyclic graph structure, $3 the connectedness, and $4 the existence of two graphs, one for each direction of flow: whereas the arcs in (N, F) represent in direction in which the decisions are transmitted, the ones in (N, B)-state the direction of information flow. Hypothesis $5 is taken into account in (4) and (5); hypothesis P1 is needed to explain the nondeterministic nature of the systems we are modeling. Finally, the T1 and T2 hypotheses allow the evaluation of system performance, and constitute the cornerstone of the Information Process we describe below. 2.2. The Information Process (IP) Efficient management needs very sharp and believable knowledge about system performance. Due to the system's nature, only the DMs in the lowest level in the hierarchy have direct access to information, but such information gives only very partial knowledge about the real performance of the whole system. However, the DMs in the highest levels (those with a broader action field) not only lack direct access to information, but also need more general knowledge about the system: as general as the kind of the decisions they have to take. Hence, we have to face two problems: • How should the DMs collect information? • How should such information be processed and sent to DMs in higher levels in the hierarchy? The aim of ever improving the quality of a service advises against collecting information about the service through merely descriptive vari- 206 E. Carrizosa et al. / Management tool for indicator-supported systems Figure 1. Some satisfaction functions ables. A decision-oriented process is more suitable, evaluating the efficacy or efficiency of performance against some preset and periodically fixed targets. A powerful tool for this purpose is provided by Performance Indicators (PI) (Weston and Brothers, 1984; Fortuin, 1988). Hence, instead of considering features in isolation, we will use performance indicators, which are defined by a feature and its associated target. To get this goal, such PIs must be easy to understand, clearly defined and representative of the system's performance, which is reflected in hypothesis T1 imposed to our model. Formally, in order to measure at time t the quality of the service associated to terminal DM i, a set V/(t) of quantitative variables are taken. For any variable v ~ V/(t), we need a couple (O,,(t), Gv(t)), where Q(t) is the observed value for v, and Q(t) is the goal for it. This couple is added to previous observations and goals, giving us the stochastic process {(O~.(s), GL,(s):s ~ T,,}, where T~, is the set of instants when the variable v has been observed. More precisely, the quality should be measured through the stochastic process {O,,(s):s ~ T,,}, where D,~(s) =f~,(O,,(s)- Gv(s), G~(s)), and fv is a [0, 1]-valued function, indicating the level of agreement between the observed value and the goal. Hence, De(s)= 1 if at time s, the preset goal is reached, and Dc(s) = 0 if the goal is not reached at all. The DMs should have at hand a set of functions f~ to fit their real perception of agreement; some particular instances, taken from PromethEe methodology (Brans et al, 1984; Brans and Vincke, 1985) are shown in Figure 1. With this, we have processed the two-dimen- sional stochastic process {(Or(s), G~,(s): s ~ T~} into a one-dimensional process {D~,(s):s ~ T~.} representing the level of achievement through time (Step 1 in Figure 2). We are faced now with the problem of evaluating the evolution of such an achievement (Step 2 in Figure 2). This evaluation should not be done using only the most recent observed value (DL(t)), because trends of improvement or worsening would not be appreciated: the same evaluation would be obtained for very different behaviors (Figure 3). This is the reason why we distinguish two different periods of time for a variable v: we Stochastic Process Ov(t),Gv(t) Bidimensionll Stochastic Pro¢Isl -4 '4 By(t) D=vi=tlon$ Proc=ss t N umber in (0,1) Figure 2. Evaluating a variable by a number E. Carrizosa et al. / Management tool for indicator-supported systems 207 Io DMn,, then DMn receives the m-tuple (an, .... , anm). This m-tuple has also to be processed into a scalar value; a procedure that fits these needs very well is the Analytic Hierarchy Process (AHP) (Saaty, 1980; Vargas, 1990). This globalizing process is recursively executed and ends reaching the SDM. 2.3. The Decision Process (DP) Figure 3. Different trends with the same last observed value define a short period as the time unity for the observations (week, month, quarter .... ), and a long period as the group of the short periods to be considered in order to explain the evolution of the quality of v (month, year .... ). With the definitions above, just the latest long period should be considered. The decisive role played by the DM should be taken into account in the evaluating (globalizing) process through his attitude towards risk. As has been widely studied in decision theory, a neutral attitude towards risk is expressed by means of the average value, a pessimistic attitude by the minimum, and an optimistic attitude by the maximum (Milnor, 1954). Hence, the suitable and simple choice for the globalizing function could have the form qb(a(1), a, a(n)), with a(1 ) = min{al,..., an} , a = (l/n) E a i, atn ) = max {a t ..... an}, n is the number of short periods included in a long period, and a i is the level of achievement in the i-th short period in the long period. A fitting procedure when qb is assumed to be additive is described in Appendix 1. With that procedure we have a scalar evaluation of any variable v for the service center associated to the terminal DMi. The information that has been processed by the terminal DMs has to flow towards the SDM through the digraph (N, B) (see Figure 4). The DMs in the hierarchy receive meaningful but simplified information. Of course, if it does not suffice, a drilling process should be available in order to obtain more technical data. Hence, if {DMn¢..., DMn, .} is the set of nonterminal DMs subordinated to DM n and an, represents the level of achievement obtained by The performance measure we try to optimize is the global system satisfaction, obtained using the IP described above. We assume that the stochastic mechanism controlling the system can be modified by means of resources, and if an unlimited amount of resources were available, the total satisfaction would be obtained. However, this is an utopic situation, so a procedure supporting decisions concerning resource allocation is needed. In real systems, optimal resource allocation does not suffice, so an optimal management of existing resources is also necessary. As a consequence of the IP in the hierarchy, the DMs can detect and correct wrong resource management; hence, the procedure we propose deals with the other aspect of the problem: the allocation of new resources. The DP is developed through the digraph (N, F). An amount of available resources for the DM can be seen as a flow emanating from him, and having as sinks his subordinates in (N, F). The process is recursively executed, until reaching the lowest level in the hierarchy, whose DMs are the only ones with the capability of using resources in modifying the system performance. Figure 4. Aggregating satisfactions 208 E. Carrizosa et al. / Management tool for indicator-supported systems The process can be done in different ways, depending on the level of knowledge that DMs have about the satisfaction function. If there exists a functional dependence between resources and outputs in the system (Vla6i~, 1989), the resource allocation problem is solved through the following mathematical programming problem: P(F) max (Sij(Oj, Uj, Xj)){,i,j)~F:FF, j,=¢} s.t. xj= E Xij Vj/(i,j)~F:FF(j)=#, jEFF 1 R >1 ~ Xlj , j~FF(I) E xik <~ E Xkj i~FTl(k) j~FF(k) Vk~N, k :g 1, FF(k) 4: ~, O <xij < cij 'q(i,j)~F where si~(Oj, Bj, x j) is the function measuring the satisfaction of DM/~ N, whose performance depends on parameters Oj, has preset goals Bj and receives xj units of resource. The total amount of available resources are R, we suppose that there exists some capacity Cij in the arcs, and the decisions variables are 0 r and Xj. P(F) is a multiobjective problem, and if the knowledge of a globalizing function is assumed (which is a sensible assumption due to the AHP followed in the IP), problem P(F) becomes a scalar problem, whose optimal solution gives the values of resource allocating variable parameters and global satisfaction obtained for such an amount of resources. When the explicit forms of the satisfaction functions are not known, the procedure above is infeasible. For such cases, an interactive methodology is more suitable. The iterative process we propose consists of two phases (Backward and Forward), described below. In the backward phase, the terminal DMs demand an amount of resources they would like to receive, and associate to such demand the level of satisfaction to be obtained. This information flows until it reaches the SDM. In the forward phase, an offer of resources flows from the SDm until reaching the terminal DMs. These two phases are repeated until a certain equilibrium is obtained. Comparatively, whereas the backward phase looks for optimality in satisfaction, the forward phase looks for feasibility. The backward phase This phase consists of three steps: Step 1. Fixing upper bounds for any terminal DM and resource. In the first backward phase, these bounds are determined in a realistic way, maybe updating past allocations. In the following backward phases, the bounds are determined based on the offers done in the preceding forward phase. Step 2. Demands and satisfaction. Any DM, DMj, gives a pair (rj, sj), where rj is a resource vector demand, feasible with respect to the present upper bounds u j, and sj is the level of satisfaction that would be obtained with such an amount of resources. The process is different depending on the kind of DM. A terminal DMj's demand must be the consequence of a desired change in his vector parameter. Such a change may be supported by means of the procedure described in Appendix 2. A non-terminal DMj's demand for resources rj equals Y'-i ri, and its associated satisfaction sj is the aggregation of the satisfactions of his subordinates, following the IP: s= Y~i wisi. Obviously, this aggregation should be supervised by DMj in order to correct possible errors in it, due to the fact that we have admitted a linear globalizing function and the attributes may not be mutually independent (Fishburn, 1970). Step 3. Feasibility test. The process of aggregation of resources and satisfaction ends reaching the SDM; he receives a pair (r, s), where r is the amount of resources the system desires, and s is the satisfaction that such resources would induce. The SDM has an amount r o of resources. If r o >/r, the system reaches equilibrium, and the DP stops; if this were not the case, a forward phase starts, with the aim of reaching feasibility. The number of iterations needed to reach equilibrium strongly depends on how realistic the demands made by the terminal DMs are. Such realism can be controlled by the SDM through rationally fixing the bounds in Step 1. The forward phase The amount of resources r i that DM i has must be distributed among his subordinate DMs. E. Carrizosa et al. / Management tool for indicator-supported systems 209 PROPERTIES El rMENT5 REOURSIYITY GLOBAL UNDIREDT ELABORATED WEIGHTING BY A.H.P. PARTIAL DIREOT EAtiY TO EY/¢UATE WEIGHTIN6 BY A.H.P. lObs'toO v"ue¢ Ov (tl It °(}'1~' =(hi Figure 5. Information and Decision Processes Such a distribution should be a function of the couples (r i, sj), Vj ~ F(i), obtained in the preceding backward phase, and should be done involving DMi. A feasible choice for this purpose could be obtained by solving the following linear mathematical program: max ~ ~sjwi6kj j k s.t. Y'~ 6~j <~ 1 Vk = 1 ..... r, J O<~6~j Vk=l,...,r, j~F(i) where 8kj is the fraction of k-type resource assigned to DMj and w J is the weight that DMi gives to this subordinate DMj. The optimal values for the decision variables Ski give the fraction of resources assignable to every DMj. This process is done recursively, and stops when the lowest level is reached. Then, a new backward phase starts. In order to fasten the convergence to equilibrium, a pruning process could be used, removing those DMs whose demands in a stage were feasible. These two sections have been devoted to describe the HIDS. A summarizing diagram is shown in Figure 5. For practical applications, all that is needed it so identify the real structure with the elements in our model. This is what we do in the next section. 3. A case study When this development was contracted, the public health services in Andalusia (Spain) depended on a public institution called SAS (Servicio Andaluz de Salud). [ EAt. DIRECTOR 1 H.I LM. P-k 6. • Figure 6. SAS structure 210 E. Carrizosa et al. / Management tool for indicator-supported systems The SAS organization Table 1 A simplified organigram of the administrative structure of this institution is shown in Figure 6. The top manager is the SAS director. At the second level there are two general directors: The Hospital Assistance General Manger (HAGM) and the Primary Assistance General Manager (PAGM). Andalusia is administratively divided into eight provinces, each one with its own Provincial Manager (PM), subordinate to the two general directors. Every center has its own manager (Hospital or Primary Assistance Center Manager). All the centers in a province are under the responsibility of their Provincial Manager. The complaint system Instead of being focused on productivity, public systems are mainly interested in offering high quality in their services. This is why one of the goals for SAS managers has always been the continuous improvement of the level of obtained satisfaction, the latter being measured by means of the complaints the users make. Such complaints are classified according to the type of personnel, the area of activity, the reason of the complaint and the answer obtained to the complaint (Table 1). HIDS modeling The hierarchical and administrative SAS structures coincide, except for the Provincial Managers. Since these DMs join two clearly separable tasks (the management of the hospitals and the primary assistance centers), they have been split up according to the organigram shown in Figure 7. The main parameters (given by the managers) that control the performance of the different elements in the system are shown in Table 2. The set of observed variables (outputs) throughout time shows different types of presented complaints, classified following Table 1. The managers have suggested that data should be collected in the centers every month (a short period). In order to process such information, all that is needed is (i) the satisfaction functions associated to the different variables and (ii) the Reason: CA. Assistent quality TP. Politeness EM. Performance of instruments TA. Bureaucracy PP. Tests lost LI. Cleaning PU. Punctuality IN. Information SA. Medical examination cancelled LE. Waiting list AL. Food IA. Assistent information OT. Others Area: PL. Floor UR. Urgency CE. External examinations QU. Surgical area PT. Internal examination LA. Laboratory and analysis SG. General Services Staff: ME. Doctors EM. Nurses AA. Office workers CE. Attendants MA. Maintenance DI. Directors CL. Restaurant, cleaning employees IN. Others. Result: a. b. C. d. e. f. The user is right and his problem is solved. The user is right and his problem remains unsolved. The user is wrong. The complaint is not clear enough. The user needs additional information. The center is not responsible for the reason of the complaint. I ;*s' °zRscr°R 1 N.k E. kl. P.A.E.M. Figure 7. HIDS modeling E. Carrizosa et al. / Management tool for indicator-supported systems 211 Table 2 Hospitals Primary Assistance Centers Number of beds X Number of examination rooms X X Special examinations X X Assistential staff X X Administrative staff X X General service staff X X Number of operating rooms X Number of ambulances X X Number of urgency booths X Figure 8. A menu display globalizing functions for trajectories; we have given the managers a small library of satisfaction functions, taken from decision theory manuals. The globalization of trajectories, done according to Appendix 1, is now being validated. Following also manager's suggestions, the goals for the variables are to be fixed at the beginning of every natural (a long period). Due to the nature of this system, what managers have to determine is the highest number of complaints of every type that is going to be seen as acceptable. In the DP, the main resources used in order to improve the satisfaction level are essentially of economical nature, although in some cases the users assigned to every center were also used as resources. For this purpose, the terminal DMs (Hospital or Primary Assistance Center Managers) could change the parameters associated to their managed center (see Table 2). Such a change should be executed following the manager's own experience, and, if needed, supported by the procedure proposed in Appendix 2: given a Health Service center, those services with similar parameters are detected; by means of a decision process, which takes into account technical constraints (upper bounds for the number of beds, dependence between the number of operating rooms and assistent staff, etc.), a change in the vector of parameters is proposed. 4. Software architecture HIDS has been implemented in a software architecture called INDI. INDI is an object-ori- ented decision environment that allows the representation of decision situations in a very general context, and allows the analysis of information. One the main features of INDI is its evolutive nature: it permits the addition of new relevant elements and discarding of useless ones. The system is based on a set of relational data bases, managed by a knowledge base that determines the relations among different elements in the hierarchy. This set of rules describes the processes of information and decision. INDI is a multi-user system, that runs in a local area network with DOS compatible computers. Some displays are shown in Figures 8 and 9. INDI has two different modules: 1) User interface. • Maintenance. Allows the representation of any hierarchical system. The user can create new elements and its associated features, and define the relations among these elements and the rest of the hierarchy. These data are located in the knowledge-base. • Information process generator. Generates the reports for any DM in the hierarchy, and processes the information through the IP to obtain the level of satisfaction for the DMs. ! JUNTA 0E ANOkLUCIA ESTANOAR: ESTANCIA MEDIA I D.A.P:I0.O0 CONSEJERIA DE SALLIO y OE~GLOSE: H.O~ I D.A.N:IO.QO SERVICIOS SCCIALES U.I'IEDIOA: DIAS PflEDICCIQNES. PERI ENERO FEBRERO VAqZO ABRIL P gAS 5 5 5 5 E R ~ 67 0 0 0 I 0 DES -62,00 5.00 5.00 5.00 T BAS 5.00 5.OO S.O0 S.OO A RF~ 67.00 67.00 67.00 67.00 e i~s -~.oo *E,?.oo -62.oo -~.oo PERIO EX~E~O N(~4AL NOW~L NORNAL T.A.H EXTEF6qO EXTE~40 EXTE~6NO EXTEF~O PULSE CL~L~J[ER TECLA PARA CONT[NUAR Figure 9. A predictions display 212 E. Carrizosa et al. / Management tool for indicator-supported systems "FORWARD PHASE -,Ho,__ I I.OEOISIOH SU,,ORT/ / FREE-TExT NE'NTS "UTIL "HIERARCHY MANAGEME "FORECASTING INOI ATORS RELA ON A ONG DM8 • oN--INE "~1 I'.AIN~ENAHCE~ I ""J'.AIN;~NANC~ ' [ORR[I~RRATI ON I t 'MS[TI gRF[[T ION l " SPECIAL FUNCTIONS l • INTRODUCTION OF DATA Figure 10 Graphical and screen-editor reports are available. • Analyzer. This is the cornerstone for the DP. It allows interactive analysis (about the resources), predictive analysis about the behavior of the observed variables and statistical analysis (homogeneity in terminal elements, cluster analysis). • Interactive aid. 2) System interface. Allows the configuration of hardware parameters, and the creation of users and levels of responsibility and accessibility for the different modules of INDI. An architecture diagram is shown in Figure 10. Appendix 1 Our aim is the fitting of an additive globalizing function q~ in order to aggregate the achievement trajectory obtained through a long period by means of a real value q~(a(1), ~, a(n )) representing the satisfaction associated to the variable. By additivity we mean that @ is of the form q~(a(l ), ~, a(,)) = ota(1 ) +/3~ + 3"a(n ) for some a,/3, 3' >~ 0, a +/3 + 3' = 1, to be estimated. We assume that, if the achievement trajectory is displayed, the DM is able to score the global achievement (satisfaction) in an long period with a value in [0, 1]. Even under this assumption, the procedure we propose is very useful, because if avoids time-con- suming routinely evaluations for every variable. The parameters are fitted in a preliminary fitting process. For any short period i, let a(1)(i) , ~(i) and a(m(i) be the lowest, average and highest achievement, respectively, obtained in the long period ending in i. Let also v(i) be the score in [0, 1] that the DM proposes for such a long period. When this process is repeated during k consecutive short periods, we obtain a sample {(a(1)(i), -d(i), a(m(i) , v(i)), i= 1 .... , k}. We propose as parameter estimators, (a*,/3", 3'*), the optimal solution for the problem: k min E [om(1)(i) +/3~(i) +3"a(m(i ) - v(i)] 2 i=1 s.t. a+/3+3"=1, a,/3, 3' >/0. Once the DM's scores agree with the globalized values, the fitting stage ends. From then on, the globalization will be automatically done as a*ao) +/3"~ + 3"*a(m. Of course, if at any time the DM stops agreeing with this globalization, a new fitting stage should start. Appendix 2 The backward stage in the DP starts at the lowest level, where the satisfaction associated to resource demand must be determined. Let DM n be a terminal DM with demand r n. In order to obtain the associated satisfaction s n, we propose a procedure that shows a heuristic use of resources by modifying the d-dimensional parameter O n associated to DM,. The information that DM, has about system performance is summarized in the deviation vec-