Method for selecting expert groups and determining the importance of expertsjudgments for the purpose of managerial decision-making tasks in health system

57 2, XVIII, 2015 Business Adminis a ion and Managemen DOI: 10.15240/ ul/001/2015-2-005 In oduc ion Cu en ends in scien ifi c and echnological ad ances a e b inging a signifi can imp o emen s in heal h ca e as a esul o c ea ion new ools ool o suppo decision making p ocess o decision-make s (DM) [8], [11], [12], [23], [27]. The expe s g oup app oach is used in heal h sec o bo h clinical [28], [31] and nonclinical [5], [24], [29], [30], [32] decision-making. The goal o his pape is o de elop, es and analyse a me hodology o de e mining he quali a i e and quan i a i e composi ion o an expe g oup and i s applica ion on example o he heal h echnology decision making in he Czech Republic. Heal h p o ide s ace he p oblem o ying o make decisions in si ua ions whe e he e is insu fi cien in o ma ion and also whe e he e is an o e load o (o en con adic o y) in o ma ion [13]. Regula o y and eimbu semen au ho i ies ace unce ain choices when conside ing he adop ion o heal h-ca e echnologies [4]. As he la ges sha e o heal hca e expendi u e is paid om public sou ces, he e fi cien decisions a e no only pu ely echnical and fi nancial p oblem bu may be seen as an issue o public in e es in he b oade e ms. This ype o decision- making me hodology may ha e a e y wide po en ial also ou side he sec o o heal hca e [14], [16]. Un o una ely he decision-making in he heal hca e (and many o he pa o public sec o s) in e ms o la ge in es men he Czech Republic is ela i ely o en he objec o se ious economics as well as legal conce ns. To seek o e idence-based me hodology ha has a po en ial o e idence-based decision-making is e y i al as an opposi e o he decision- making infl uenced by pa ial indi idual and g oup in e es s. Many o he mos e ec i e models used o fi nd expe s a e mainly based on he language models [17], [33]. One o he p oblems wi h models based on language model amewo ks is ha hey can ake in o accoun ex ual simila i ies be ween he que y opics and documen s [1], [20], [26]. The pape [2] p o ides wo expe s fi nding sea ch s a egies modelled o inco po a e di e en ypes o e idence ex ac ed om he da a. The ad an age o he mode n app oaches is machine lea ning echniques [18], [34] o disc imina i e p obabilis ic models [7] a possibili y o agg ega ion o a la ge numbe o he e ogeneous in o ma ion. In addi ion o ha , he e a e he ollowing p oblems and di fi cul ies: i is di fi cul o exp ess quali a i e in o ma ion on expe s in he quan i a i e o m because in o ma ion abou he candida es a ies wi h ime; he expe ience o a candida e is always a ying h ough ime. [8] In candida e-cen ic p obabili y es ima ion app oaches o academic expe fi nding [1], he assessmen o an expe is made using gene a i e p obabilis ic models. In que y independen me hods [25], knowledge o he expe candida es is p esen ed as a mix u e o language models. The pe son-cen ic [26] app oach is inc easingly being used. This pape [19] is based on ex ual simila i ies, he au ho ’s p ofi le in o ma ion and he au ho ’s ci a ion pa e ns o y o fi nd academic expe s. The me hods o Condo ce Fuse [19], Ma ko chain models [6] and mul i-c i e ia decision-making me hods 10] a e ecognised as ep esen ing he mos ele an wo ks [3]. The mul isenso y Da a Fusion app oach using Demps e -Sha e heo y o E idence oge he wi h Shannon’s En opy [20] was used o academic expe s fi nding. METHOD FOR SELECTING EXPERT GROUPS AND DETERMINING THE IMPORTANCE OF EXPERTS’ JUDGMENTS FOR THE PURPOSE OF MANAGERIAL DECISION-MAKING TASKS IN HEALTH SYSTEM Ilya I le , Pe e Kneppo, Mi osla Ba ák EM_2_2015.indd 57EM_2_2015.indd 57 3.6.2015 13:08:573.6.2015 13:08:57 58 2015, XVIII, 2 Ekonomika a managemen Expe s fi nding is a di fi cul ask because he expe s and hei skills and knowledge a e a e, expensi e, cons an ly changing and a ying in dep h. When add essing di fi cul mul idisciplina y p oblems, a combina ion o knowledge om se e al expe s is o en equi ed, especially om expe s in a ious fi elds. Ou me hod p oposal is based on he esea ch o expe s’ weigh ing ac o de e mining ha efl ec s he o e all compe ence o he expe s when p oblem sol ing. 1. Examina ion o Expe s An examina ion needs o be pe o med o de e mine he weigh -coe fi cien s o impo ance in oduced by each c i e ion o choice. Depending on he scale o he p oblem, he examina ion is o ganised ei he by a DM in pe son o by an expe g oup appoin ed by he DM. Decisions abou he numbe and compe ence o expe s a e made wi h ega d o he scope o he ask, he e aci y o he e alua ions o expe s’ cha ac e is ics and he a ailable esou ces. The ollowing asks need o be sol ed while c ea ing he expe eam: 1) unde s and he ask o be sol ed by he expe s; 2) de e mine fi elds o ac i i y linked o he ask; 3) decide wha sha e in he eam shall be alloca ed o expe s ep esen ing each fi eld o ac i i y; 4) de e mine he numbe o expe s in he eam and d a a lis he eo ; 5) analyse expe s’ qualifi ca ions and edi he d a lis o expe s; 6) ob ain he expe s’ ag eemen o wo k on he eam; and 7) fi nalise he lis o expe s. Depending on he chosen o m o de e mining expe s’ p e e ences, he main equi emen s o he expe s a e as ollows: 1) compe ence ( eliabili y and alidi y o decisions, awa eness and ep oducible assessmen and a gumen a i eness eplies); 2) impa iali y; 3) c ea i i y; 4) con o mism; 5) eam spi i (dependen on qua e na y ype); 6) ela ion o he examina ion; 7) deg ee o pa icipa ion in he sol ing p oblem; and 8) communica ion skills (dependen on qua e na y ype) (Fig. 1). The expe s’ cha ac e is ics, as lis ed abo e, gi e a comp ehensi e pic u e o he quali ies ha infl uence he examina ion esul s mos s ongly [20] (p ope ies ha a e w i en on a black backg ound a e aken in o accoun in ou model). 2. Quan i y Lis o he Expe G oup To de e mine he su i cien numbe o expe s, we needed o i nd a numbe H, so ha he inequali y W > H is ue, whe e W is he dispe si e Kendall’s coe i cien o conco dance (coe i cien o conco dance o expe s’ opinions. Cons an H is selec ed om he ela ionship PW > H = α and is ully de i ned by he le el o signi i cance α. I is i a ional o selec a low le el o signi i cance because i s dec ease esul s in an inc ease o H, which, consequen ly, inc eases he con ingency o he e o o he second ype [13]. The eby, we shall de e mine he necessa y quan i y o expe s ha gua an ees, a a fi xed le el o signifi cance, he gi en c i ical alue o dispe si e coe fi cien o conco dance. Fo easons o simplifi ca ion, we shall conside he ollowing ela ionship ue:  2 1. 1 n PW H P Hm n             (1) As 2 1 1 lim 1 n n n      (n = quan i y o pa ame e s), he le el o signifi cance o a c i e ion is de e mined by he p oduc H*m (m = numbe o expe s), which, e en wi h a small numbe o expe s, can make α su fi cien ly small. The su ey o expe s included eco ding, in an in o ma i e and quan i a i e o m, he expe s’ opinions abou a gi en p oblem. The main modes used o su ey expe s include ques ionnai es and in e iews, discussions and b ains o ming. The Delphi me hod could be used o consensus-building by using a se ies o ques ionnai es deli e ed using mul iple i e a ions [19], [20]. 3. P ocessing o Expe s’ E alua ions P ocessing is needed o ob ain gene alised da a and new in o ma ion ha is concealed in he expe s’ e alua ions. I g oup e alua ions o objec s p o e doub ul when compa ed wi h calcula ed s a is ics, i is necessa y o de e mine he easons o he ailed examina ion. The mos ypical easons o ailed examina ions include he ollowing: 1. D awbacks in he selec ion o he expe g oup: expe s’ goals did no co espond wi h he goal o he esea ch (confl ic o in e es s), unsound examina ion. EM_2_2015.indd 58EM_2_2015.indd 58 3.6.2015 13:08:573.6.2015 13:08:57 59 2, XVIII, 2015 Business Adminis a ion and Managemen 2. A confl ic o opinion exis s. In o de o disco e di e en poin s o iew, he expe s need o be g ouped acco ding o how close hei e alua ions a e. I such g ouping p o es success ul, s a is ical p ocessing has o be pe o med o each g oup sepa a ely. 3. Mis akes in he ex o he ques ionnai e, e.g., ambiguous in e p e a ion o ques ions o use o specifi c wo ds. 4. In oduc ion in o he ques ionnai e byway objec s. Insu fi cien con o mi y o expe s’ e alua ions does no allow o he g oup e alua ions o all objec s o be conside ed as eliable. I his si ua ion occu s, hose e alua ions will need o be excluded and he esul s ep ocessed. Depending on he goals o he expe e alua ion, he ollowing main asks a e equi ed o p ocess he su ey esul s: 1) de e mine expe s’ compe ences and gene alised e alua ions o objec s, 2) ank he objec s, 3) de e mine con o mi y in he expe s’ opinions, Fig. 1: Expe p ope ies and me hods o assessmen Sou ce: own EM_2_2015.indd 59EM_2_2015.indd 59 3.6.2015 13:08:573.6.2015 13:08:57 60 2015, XVIII, 2 Ekonomika a managemen and 4) de e mine he ela ionships among he anged objec s. The ollowing sec ion will desc ibe he indi idual poin s ele an o he ask we a e sol ing. 3.1 E alua ion o Expe Compe ence in he Gene alised E alua ion o Objec s The fi s p e equisi e o ensu e eliabili y o he examina ion esul s is o in i e expe s who a e in e es ed in he esul s o he examina ion. Concu en ly, he goals o expe s ha e o co espond wi h he goal o he examina ion, in gene al. I is clea ha in p esen -day condi ions, o mal indica o s o expe s (job i le, science deg ee, wo k expe ience, numbe o publica ions, e c.) can be used only as seconda y c i e ia in iden i ying expe s’ o al compe ence ha efl ec s hei o e all p o essional skills and quali ies. When using he sel -e alua ion me hod, he expe p o ides in o ma ion abou he fi elds he o she is mos compe en in [22]. I ollows ha o e alua e expe s’ compe ences; se e al main me hods could be used: sel - e alua ion, e alua ion o a colleague’s compe ence, es ing o he expe s, and he e alua ion o hem by he o ganize s o he examina ion based on p e ious examina ions. The me hods lis ed abo e belong o so-called ex e nal me hods wi h espec o he conduc ed examina ion. Howe e , he me hod o sel - e alua ion o compe ence a es he deg ee o he expe ’s sel -confi dence a he han hei eal compe ence. Simila ly, du ing he e alua ion me hod, o he people’s compe ence and he g oup’s awa eness abou each o he ’s abili ies play a ole. As can be seen, each me hod has i s d awbacks. O he me hods o e alua ion o expe s’ compe ences use pos e io da a, o he esul s om he e alua ion o objec s. He e, expe s’ compe ences a e e alua ed by he deg ee o con o mi y o hei e alua ions o he g oup e alua ion o objec s [22]. The essence o his app oach lies in he ac ha expe s who ha e exp essed con adic o y opinions ecei e low g ades o compe ence, and consequen ly, hei e alua ions play a less impo an ole when de e mining he g oup e alua ion. When an expe ’s e alua ion is close o he g oup’s e alua ion, he compe ence o his expe is ea ed as highe [21] and his ac could be used as a way o expe s’ compe ence de e mining. I should be no ed, ha g ope opinion o expe s wi h close o e all compe ence le el would ha e highe con o mi y [27]. 3.1.1 E alua ion Based on Objec i e and Subjec i e Pa ame e Assessmen To p e en he esul s o he sel -e alua ion me hod om being a me e efl ec ion o an expe ’s sel -confi dence, i is possible o use app oaches [9] ha p o ide an objec i e cons i uen o knowledge abou he expe ’s compe ence (Tab. 1). I is ad isable o de e mine an index o ela i e sel -e alua ion by an expe based on he deg ee o hei pa icipa ion in elabo a ing he p oblem as a complex coe fi cien , which exp esses he ela ionship be ween he expe and he examina ion, hei pa icipa ion and hei in e es . Fo each ques ion o g oup o ques ions on which expe ’s compe ence should be e alua ed, he e is a co esponding scale called he “ ela i e sel -e alua ion o expe ” in he able o expe e alua ions. To p e en he g ades in he scale om infl uencing he sel -e alua ion, he ela i e sel -e alua ion o expe scale con ains a lis o expe compe ence p ope ies wi hou any g ades. Wi h his app oach, he expe has o unde line he p ope ies ha , in his o he opinion, de e mine he le el o his o he pe sonal compe ence. The g ades [9] a e added by he wo king g oup while analysing he collec ed ques ionnai es (Tab. 1). 3.1.2 E alua ion o Expe Awa eness and His o He Rele ance o Knowledge Ano he way (Tab. 2) o de e mine an expe ’s weigh ing ac o is ia he index o amilia i y wi h he ask. I is calcula ed on he basis o an expe ’s e alua ion o hei own amilia i y wi h he p oblem and indica ion o ypical sou ces o a gumen s o suppo hei opinions (index o a gumen a ion, esul s om summing up he g ades in he e e ence able, index o amilia i y wi h he p oblem and esul s om he expe ’s sel -e alua ion exp essed on a 10-g ade scale and mul iplied by 0.1 wi h he pu pose o b inging he alue o one). In gene al, he ela i e sel - e alua ion o expe index is designed o make he expe pe o m a sel -e alua ion o his o he own compe ence on he gi en ques ion. EM_2_2015.indd 60EM_2_2015.indd 60 3.6.2015 13:08:573.6.2015 13:08:57 61 2, XVIII, 2015 Business Adminis a ion and Managemen Objec i e e alua ion Subjec i e e alua ion w1w2w3w4w5 Job posi ion G ades Educa ion G ades To al wo k expe ience (yea s) G ades Wo k expe ience in he p oblem a ea G ades Le el o pa icipa ion in he p oblem G ades Head o o ganisa ion 1.0 Ph.D. 1.0 >10 1.0 >10 1.0 Expe specialises in he gi en issue 1.0 Depu y head 0.8 Highe educa ion (mas e ) 0.8 10-5 0.8 10-5 0.8 Expe pa icipa es in p ac ical wo k on sol ing he issue, bu he issue does no belong o he expe ’s indica ed specialisa ion 0.8 Head o depa men 0.6 Highe educa ion (bachelo ) 0.6 <5 0.6 <5 0.6 The issue belongs o he expe ’s specialisa ion 0.6 Depu y head o depa men 0.4 less 0 0 0 0 0 The issue does no belong o he expe ’s specialisa ion 0.3 Sou ce: [9] w6,j Sou ces o a gumen s Le el o sou ce’s infl uence on he expe ’s opinion Indica o s and hei weigh s I ead o en and egu- la ly I ead o en, bu no egu- la ly I ead seldom I do no ead a all 100% 75% 20% 0% w6,1 Summa ising pape s by local au ho s 0.250 0.187 0.050 0 w6,2 Summa ising pape s by o eign au ho s 0.250 0.187 0.050 0 w6,3 Pa en in o ma ion 0.250 0.187 0.050 0 w6,4 Companies’ epo s (ca alogues, b ochu es, ecommenda ions, e c.) 0.250 0.187 0.050 0 Sou ce: own Tab. 1: Ques ionnai e o e alua ion o expe ’s compe ence Tab. 2: Re e ence able o indices o a gumen a ion (w6,j) EM_2_2015.indd 61EM_2_2015.indd 61 3.6.2015 13:08:573.6.2015 13:08:57 62 2015, XVIII, 2 Ekonomika a managemen 4 66, 1 1 4j j WW   (2) P ac ical esea ch on expe polls show ha al hough sel -e alua ion me hods a e no su fi cien as he sole c i e ion o de e mining expe compe ence, hei applica ion p o ides a mo e well- ounded selec ion and e alua ion o he expe s [9]. To sol e he p oblem o se ing he weigh coe fi cien s o expe s and, hus, de e mining he p obabili y o ob aining eliable e alua ions, we p opose o c ea e a comp ehensi e me hod based on he app oach ha combines a ious compe ence e alua ion me hodologies: a sel -e alua ion o expe s abou hei own compe ence on he p oblem; an in oduc ion o g ades o objec i e da a and an app ecia ion o he index o a gumen a ion. The eby, he compe ence index o an expe can be ea ed as a p obabili y o he expe ’s gi ing a eliable e alua ion, whe e 0 ≤ We ≤ 1. 4. Resul s and Discussion 4.1 De e mining he Quali y o he Expe G oup The me hodology was used o he selec ion o expe s o he p opose o a ional selec ion o la ge medical equipmen such as, compu ed omog aphy (CT), mammog aphic digi al X- ay sys ems (MAM), magne ic esonance imaging (MRI), adiog aphic/fl uo oscopic sys ems (gene al pu pose) (RFS), ul asonic scanning sys ems (ca diac) (USC) and ul asonic scanning sys ems (gene al pu pose) (USG). The lis o mos heal h ca e acili ies o depa men s in he Czech Republic whe e CT, MAM, MRI, RFS, USC and USG a e loca ed was o med. Po en ial expe s we e asked o answe he web-based ques ionnai e. The pu pose o expe s g oups’ c ea ion was p esen ed in each ques ionnai e. Candida es o he expe s we e manage s (heads o depa men s, heads o clinics, e c.) and employees (physicians, biomedical enginee s, adiological assis an s, senio echnicians, e c.) om he depa men s o adiology, imaging me hods, adio diagnos ics, in e en ional adiology, depa men s o ca diology and he Ins i u e o adiology and he o he depa men s in hospi als o all le els o o ganisa ion and all egions o he Czech Republic. The selec ion o expe s was conduc ed wi h 872 employees om 422 heal h acili ies in he Czech Republic (Tab. 3). The me hod o selec ing he mos knowledgeable expe s o he ask o selec ing and p ocu ing medical equipmen o hospi als is based on 1) he expe ’s o e all wo k expe ience, 2) expe ience in sol ing asks, 3) le el o educa ion and scien ifi c eco d, 4) in e es in sol ing he pa icula ask, 5) cu en posi ion and 6) awa eness o how o sol e he ask. This s udy also conside ed he The ask The numbe o heal h ca e acili ies’ s a ed po en ial expe s The numbe o ques ionnai es sen The numbe o e- sponses Selec ion o MRI 34 60 19 (31.7%) Selec ion o mammog aphic digi al X- ay sys ems 68 125 18 (14.4%) Selec ion o USC 101 190 22 (11.6%) Selec ion o CT 89 162 15 (9.3%) Selec ion o USG 116 116 9 (7.0%) Selec ion o RFS 14 219 13 (6.0%) To al 422 872 96 (11.0%) Sou ce: own calcula ions Tab. 3: The pa icipan s o he su ey EM_2_2015.indd 62EM_2_2015.indd 62 3.6.2015 13:08:573.6.2015 13:08:57 63 2, XVIII, 2015 Business Adminis a ion and Managemen 7) ele ance o he expe ’s knowledge and 8) he o e all sel -e alua ion conce ning hei o al compe ence in sol ing he ask. The da a ob ained om he ques ionnai e o expe s did no ollow a no mal (Shapi o- Wilk es ) dis ibu ion (p < 0.001) (“Sou ce o a gumen a ion” [p = 0.029] and “Sel - anking” [p = 0.003]). An analysis o he esul s e ealed (Mann-Whi ney U- es ) ha he e was no eason o ejec he null hypo hesis ha he in e -g oup alues o he compa ed ( o al a ing wTR and sel - a ing wSR) cha ac e is ics we e homogeneous (p = 0.285). A medium- s eng h posi i e co ela ion (Spea man’s ank co ela ion) was ound be ween he wo measu es ( = 0.550, p < 0.001) (Fig. 5). Examina ions o he co ela ions be ween he componen s ha de e mine he o al weigh o he expe s we e pe o med. The co ela ions be ween “Wo k expe ience in he p oblem a ea” and “Educa ion” ( = 0.268; p = 0.01) and “Wo k expe ience in he p oblem a ea” and “Job posi ion” ( = 0.342; p = 0.001) sugges ha sol ing he ask o selec ing la ge medical equipmen pa icipa ing po en ial expe s wi h highe le el o educa ion and highe ope a ing posi ions. The ask o selec ing la ge medical equipmen is ca ego ised as a manage ial ask should be sol ed by he mos expe ienced p o essionals fi lling a manage ial ole. The mode a e co ela ion be ween “Wo k expe ience in he p oblem a ea” and “Sou ces o a gumen s” ( = 0.394, p < 0.001) and he weak co ela ion be ween “Wo k expe ience in he p oblem a ea” and “Le el o pa icipa ion in he p oblem” ( = 0.239, p < 0.022), as well as “Sou ces o a gumen s” and “Job posi ion” ( = 0.231, p = 0.027), indica e ha highe le els o expe expe ience equa e o highe le els o pa icipa ion in sol ing he ask, highe le els o heo e ical p epa a ion and highe le els o o e all compe ence. The wSR index was ob ained as a esul o expe s’ sel - ankings on a scale om ze o (I am NOT compe en in add essing he issue o selec ion) o 10 (I am compe en in add essing he issue o selec ion). The wSR mos closely co ela ed wi h “Wo k expe ience in he p oblem a ea” ( = 0.519, p < 0.001), ha is, 52% o he gene al weigh ing ac o consis s o he expe s’ «Wo k expe ience in he p oblem a ea». The nex pa ame e s mos closely co ela ed o he wSR we e he expe s’ le els o a gumen a i eness ( heo e ical p epa a ion, sou ce o a gumen s and awa eness) ( = 0.440, p < 0.001), “Job posi ion” ( = 0.319, p = 0.002) and “Educa ion” ( = 0.280, p = 0.007). The p esence o he abo e co ela ions indica es ha he index o expe s’ sel - ankings is dependen on “Wo k expe ience in he p oblem a ea”, “Sou ces o a gumen s”, “Job posi ion” and “Educa ion”. A s a is ically signifi can associa ion was no de ec ed be ween wST and he expe s’ o al wo k expe ience (p = 0.089) o wSR and le el o expe s’ pa icipa ion in he p oblem sol ing ask o selec ing medical equipmen (p = 0.200). Fig. 2 shows ha he leas sensi i e (less a ying, depending on he expe ) indica o s we e “To al wo k expe ience”, “Educa ion”, “Wo k expe ience in he p oblem a ea” and “Le el o pa icipa ion in he p oblem”. These esul s a e indica i e o he ac ha specialis s wi h a high le el o educa ion and high-quali y posi ions, whose wo k was ela ed o he p oblem o selec ion, we e p e-selec ed as expe s. EM_2_2015.indd 63EM_2_2015.indd 63 3.6.2015 13:08:583.6.2015 13:08:58 64 2015, XVIII, 2 Ekonomika a managemen Fig. 2: The lis o expe s in o de by hei comp ehensi e assessmen o compe ence ( alues ob ained om he e alua ion o expe s on he objec i e and subjec i e c i e ia and e alua ion o hei own opinions abou hei own compe ence) Sou ce: own calcula ions EM_2_2015.indd 64EM_2_2015.indd 64 3.6.2015 13:08:583.6.2015 13:08:58 65 2, XVIII, 2015 Business Adminis a ion and Managemen 4.2 The O e all Weigh o Expe s’ Compe ence Fou di e en calcula ion models (Fig. 3) o he o al compe ence weigh ing ac o we e in es iga ed o de e mine he fi nal model o calcula ing he o al weigh o compe ence o each expe . In he fi s me hod:  (1) 1, 2 eSTTR Www (3) whe e 1 1, n TR i i ww n    (4) he weigh o each o he coe fi cien s (wST and wTR [ o mula 4]) a e conside ed equal, i.e., he con ibu ion o each o he coe fi cien s o he o al weigh We (1) is he same (as in We (2) and We (3)) (Fig. 5). Since he a i hme ic mean is no a obus s a is ic (is subjec o s ong infl uence o la ge de ia ions), We (1) inc easingly elies on he unce ain y o wST (Fig. 6-B). The second me hod o calcula ion (2) 2 2 , eSTTR Www (5) is an adap a ion o he calcula ion o he A- ype unce ain y measu emen in a calib a ion, whe e he inpu alues a e co ela ed, as in his case ( = 0.55, p < 0.001) (Fig. 5). The wTR is iewed as unce ain y in he assessmen o compe ence ob ained on he basis o objec i e and subjec i e pa ame e s. The wST is iewed as he unce ain y con ibu ed by o he unaccoun ed ac o s. The expe s’ weigh ing ac o s ob ained by his me hod we e e y accu a e ep oduc ions o he es ima e We (1) ( = 0.998, p < 0.001) (Fig. 6-B, Fig. 6-C) and ai h ully ep oduced he es ima es o wST and wTR. Fig. 3: The equi y s uc u e o he o al weigh 1 – We (1), 2 – We (2), 3 – We (3) and 4 – We (4) Sou ce: own calcula ions EM_2_2015.indd 65EM_2_2015.indd 65 3.6.2015 13:08:583.6.2015 13:08:58 72 2015, XVIII, 2 Ekonomika a managemen Abs ac METHOD FOR SELECTING EXPERT GROUPS AND DETERMINING THE IMPORTANCE OF EXPERTS’ JUDGMENTS FOR THE PURPOSE OF MANAGERIAL DECISION-MAKING TASKS IN HEALTH SYSTEM Ilya I le , Pe e Kneppo, Mi osla Ba ák This wo k aims o de elop a me hodology o de e mining he quali a i e composi ion o an expe g oup and he weigh ing ac o ega ding he impo ance o expe ’s judgmen s o he pu pose o pa icipa ing in decision-making. I is based on he expe ’s o e all wo k expe ience, expe ience in sol ing asks, le el o educa ion and scien i i c eco d, in e es in sol ing he pa icula ask, cu en posi ion and awa eness o how o sol e he ask. This s udy also conside ed he ele ance o he expe ’s knowledge and he o e all sel -e alua ion conce ning hei o al compe ence in sol ing he ask. Fo he pu pose o alida ing he me hodology, 96 po en ial expe s (physicians, biomedical enginee s, adiological assis an s, medical physicis s, e c.) om 72 heal h acili ies in he Czech Republic we e in e iewed h ough a web-based ques ionnai e. The calcula ion model ha was selec ed was able o elimina e e o s in es ima ing he p opo ionali y o ex eme alues and educes he impac o unce ain y in he expe s’ o e all sel -e alua ions conce ning hei o al compe ence. A s a is ically signi i can co ela ion was ound be ween he complex weigh ing ac o and he ollowing cha ac e is ics: he expe ’s expe ience in dealing wi h simila asks ( = 0.512, p < 0.001), he expe ’s heo e ical backg ound (awa eness) and he ele ance o he expe ’s knowledge ( = 0.440, p < 0.001), he expe ’s cu en posi ion ( = 0.319, p = 0.002) and he le el o his o he educa ion and scien i i c eco d ( = 0.280, p = 0.007). The de eloped me hodology may be especially use ul in scien i i c and echnological o ecas ing, medical and manage ial decision- making, quali y assessmen and ope a ional esea ch. Key Wo ds: G oup decision making; expe ; expe selec ion c i e ia; medical equipmen ; sel - assessmen ; weigh ing ac o . JEL Classi i ca ion: C44, D81, I11. DOI: 10.15240/ ul/001/2015-2-005 EM_2_2015.indd 72EM_2_2015.indd 72 3.6.2015 13:08:593.6.2015 13:08:59