Cross-cultural adaptation and validation of the Portuguese version of the Physical Therapy Outpatient Satisfaction Survey

Resea ch A icle Po J Public Heal h C oss-Cul u al Adap a ion and Valida ion o he Po uguese Ve sion o he Physical The apy Ou pa ien Sa is ac ion Su ey Luís M. Ca alhei o a, b Jan M. Cab i c Ped o L. Fe ei a b, d a Depa men o Physical The apy, Poly echnic Ins i u e o Coimb a, College o Heal h Technology o Coimb a, Coimb a, Po ugal; b Uni e si y o Coimb a, Cen e o Heal h S udies and Resea ch, Coimb a, Po ugal; c Depa men o Physical Pe o mance, No wegian School o Spo Sciences, Oslo, No way; d Facul y o Economics, Uni e si y o Coimb a, Coimb a, Po ugal Recei ed: Augus 2, 2016 Accep ed: No embe 27, 2017 Published online: Ma ch 1, 2018 Luís M. Ca alhei o Escola Supe io de Tecnologia da Saúde de Coimb a Rua 5 de Ou ub o, S. Ma inho do Bispo, Apa ado 7006 PT–3040-162 Coimb a (Po ugal) E-Mail lmsca alhei o @ gmail.com © 2018 The Au ho (s) Published by S. Ka ge AG, Basel on behal o NOVA Na ional School o Public Heal h E-Mail ka ge @ka ge .com www.ka ge .com/pjp DOI: 10.1159/000486250 Keywo ds Ou pa ien sa is ac ion · Physical he apy · Measu es · Heal hca e quali y Abs ac The aim o his s udy was o adap and alida e he Physical The apy Ou pa ien Sa is ac ion Su ey (PTOPS) o he Po - uguese cul u e. This e sion was ob ained by a o wa d/ backwa d ansla ion, consensus panels, and p e- es . The Po uguese PTOPS was adminis e ed o 76 physical he apy ou pa ien s in 10 heal h se ices. The con en analysis (pan- els o expe s and lay people) and he ac o analysis esul ed in a educ ion o he o iginal 34 i ems o 28 i ems ha al- idly iden i y 3 cons uc s. The eliabili y was accep able o bo h in e nal consis ency (C onbach’s alpha = 0.73) and e- p oducibili y (ICC be ween 0.84 and 0.87), which ep esen accep able le els o alidi y and eliabili y. © 2018 The Au ho (s) Published by S. Ka ge AG, Basel on behal o NOVA Na ional School o Public Heal h Adap ação e alidação in e cul u al da e são po uguesa do Physical The apy Ou pa ien Sa is ac ion Su ey Pala as cha e Sa is ação dos u en es · Fisio e apia · Medidas de esul ado · Qualidade dos cuidados Resumo Foi obje i o des e es udo adap a e alida pa a a cul u a po uguesa o Physical The apy Ou pa ien Sa is ac ion Su ey (PTOPS). Es a e são esul ou do p ocesso de adução, e o e são, painéis de consenso e p é es e. A PTOPS oi adminis ada a 76 doen es ambula ó ios de i- sio e apia, em 10 ins i uições de saúde. Da análise de con- eúdo (painéis de pe i os e gen e comum) e da análise a o ial esul ou uma edução dos 34 i ens da e são o i- ginal pa a 28 i ens, que iden i icam alidamen e 3 con- s uc os. A iabilidade oi acei á el que na coe ência in- e na (α de C onbach = 0,73), que na ep odu ibilidade (ICC en e 0,84 e 0,87). E idenciando ní eis acei á eis de alidade e iabilidade. © 2018 The Au ho (s) Published by S. Ka ge AG, Basel on behal o NOVA Na ional School o Public Heal h This a icle is licensed unde he C ea i e Commons A ibu ion- NonComme cial-NoDe i a i es 4.0 In e na ional License (CC BY- NC-ND) (h p://www.ka ge .com/Se ices/OpenAccessLicense). Usage and dis ibu ion o comme cial pu poses as well as any dis- ibu ion o modi ied ma e ial equi es w i en pe mission. Ca alhei o/Cab i/Fe ei a Po J Public Heal h 2 DOI: 10.1159/000486250 In oduc ion The assessmen o pa ien sa is ac ion and he mea- su emen o heal hca e ou comes a e a p io i y o qual- i y managemen o heal h se ices [1–4]. In he con ex o physical he apy and i s own speci ici ies, he lack o ad- equa e sa is ac ion ins umen s p e en s ha deside a- um. A heal hca e p ocess cha ac e ized by longe p o- ide /pa ien in e ac ions, in ol ing mo e physical con- ac , o en equi ing he ac i e in ol emen o he pa ien , and mo e equen isi s a e some o he ea u es iden i- ied, which may in luence bo h he ou come o ea men and he iew o i s quali y [5–9]. Howe e , in he li e a u e some ins umen s o ien ed owa ds he sa is ac ion wi h he physical he apy ca e can be ound, including he Physical The apy Ou pa ien Sa is ac ion Su ey (PTOPS) [9], he Physical The apy Pa ien Sa is ac ion Ques ionnai e (PTPSQ) [10], he Pa ien Su ey Ins umen (PSI) [5], and he Pa ien Sa - is ac ion Ques ionnai e o Physical The apy (PSQ-PT) [7]. The PTOPS is a measu e o pa ien sa is ac ion wi h he ou pa ien physical he apy ca e, which was de el- oped in 3 phases ha led o he iden i ica ion o a s uc- u ed model in 4 dimensions ela ed o sa is ac ion and dissa is ac ion wi h he p o ision o such ca e [9]. The PTOPS is an adequa e ins umen o measu e he sa is ac ion o physical he apy pa ien s in bo h clinical p ac ice and esea ch. In o de o make i s use possible in he Po uguese cul u e, an adap a ion and in e cul u al alida ion p ocess is equi ed [11–14]. The aim o his s udy was o pe o m he ansla ion and adap a ion o PTOPS o he Po uguese language and cul u e and o examine i s alidi y and eliabili y. Me hods C oss-Cul u al Adap a ion The PTOPS cul u al adap a ion p ocess ollowed a sequen ial me hodology, which is he mos common p ocedu e o his pu - pose [11–15]. The p ocess began a e he au ho s’ pe mission [9]. The o igi- nal e sion was deli e ed o wo Po uguese ansla o s wi h high luency in English who, independen ly, made he English/Po u- guese ansla ions. Bo h e sions we e checked in he p esence o he ansla o s in o de o ob ain a consensus e sion ( i s p e- limina y e sion). La e , his e sion was ansla ed back o En- glish by wo o he ansla o s. The ansla ions and back- ansla- ions we e used in a second consensus panel, esul ing in a second p elimina y e sion. Fo he unde s andabili y analysis, he second p elimina y e - sion was submi ed o a panel o 6 expe s in he managemen and/ o heal hca e quali y ields and a panel o lay people, use s o ou - pa ien physical he apy ca e, in ol ing 10 subjec s. The ou come o he eached consensus was he p e- inal Po uguese e sion o he PTOPS. Valida ion S udy Sample The p e- inal Po uguese e sion o he PTOPS was adminis- e ed o a sample o pa ien s o heal h se ices in he cen al egion o Po ugal. The subjec s we e selec ed a e ob aining in o med consen . To be included in he s udy, subjec s we e equi ed o be inishing o ha e inished an ou pa ien physical he apy ca e in less han 1 week, o be li e a e, and o no ha e any cogni i e dis- o de ha would p e en he comple ion o he ques ionnai e. The pa icipa ion o he heal h se ices in he s udy was subjec o p i- o e hics au ho iza ion. Measu emen s The PTOPS is a measu e o pa ien sa is ac ion wi h he ou - pa ien physical he apy ca e designed o be sel -adminis e ed. The o iginal e sion p o ides 34 i ems and 4 dimensions: (i) En- hance – ela ed o he physical en i onmen and pe sonal in e ac- ions, which when p esen , en ich in a posi i e way he expe ience o he pa ien ; (ii) De ac o – ecogni ion o basic in e pe sonal and physical needs o a pa ien which, when absen , gene a e a nega i e eeling; (iii) Loca ion – ease o loca ing and a eling o a clinic; and (i ) Cos s – compa ibili y be ween he pe cei ed alue o se ice and cos . The sco es o he dimensions a e de i ed om he a i hme ic mean o hei i ems, wi h a minimum o 1 and a maximum o 5 poin s. In he Enhance and Loca ion dimensions, highe sco es co espond o g ea e sa is ac ion and in he De ac- o and Cos s dimensions, highe sco es indica e less sa is ac ion. S a is ical Analysis The cons uc alidi y was assessed h ough ac o analysis which, in essence, examines he co ela ions be ween di e en a iables o ind a se o ac o s/componen s ha , heo e ically, ep esen wha he examined a iables ha e in common [16]. Commonali ies we e analyzed, excluding he i ems wi h a weigh ac o o less han 0.4. Fo he de ini ion o he main com- ponen s, he exis ence o mul icollinea i ies o he inpu a iables was obse ed using he Ba le es o sphe ici y and he Kaise - Meye -Olkin measu e o sampling adequacy, ob aining alues o he o iginal co ela ed a iables g ea e han o equal o 0.5. The numbe o main componen s o e ain ook in o accoun he eigen- alues la ge han 1, also he obse a ion o i s g aphical ep esen- a ion h ough he sc een plo , and by he c i e ion o he pe cen - age o explained a iance. Fo ex ac ion o he main componen s he me hod o a imax o a ion was used (componen ma ix wi h o hogonal o a ion “T”) wi h he Kaise s anda ds c i e ion [17, 18]. The alidi y was u he analyzed by he s udy o ela ions ob- ained by he applica ion o he measu e and hose egis e ed using independen ques ions ha could be ela ed o he meaning o he esponses o PTOPS: (i) I would ecommend hese se ices o am- ily o iends; (ii) I would come back o his se ice i I needed physical he apy ea men again, (iii) In gene al I am sa is ied wi h my expe ience o physical he apy. Po uguese Valida ion o he PTOPS 3 Po J Public Heal h DOI: 10.1159/000486250 Fo each ques ion, a Like scale o 5 poin s was used (1 = s ongly disag ee, 2 = disag ee, 3 = unce ain, 4 = ag ee, 5 = s ong- ly ag ee). An iden ical p ocedu e was used and ad oca ed by Gold- s ein e al. [10]. The s udy o ela ions was pe o med wi h Spea man’s co ela- ion coe icien . Fo he in e p e a ion o co ela ion coe icien s, he c i e ia o Cohen and Holliday [19] we e used, which sugges he ollowing classi ica ion: e y low co ela ion o alues less han 0.20; low i be ween 0.20 and 0.39; mode a e i be ween 0.40 and 0.69; high i be ween 0.70 and 0.89; and e y high o alues equal o o g ea e han 0.90. The es - e es eliabili y was examined by he applica ion o he measu e wi h an in e al o 1 week, using he in aclass co e- la ion coe icien (ICC). A coe icien o ep oducibili y equal o o g ea e han 0.70 was conside ed accep able o compa e g oups [20]. The in e nal consis ency o homogenei y o PTOPS was es ed by C onbach’s alpha. Values be ween 0.70 and 0.95 we e consid- e ed indica i e o accep able eliabili y [18–21]. Values o p < 0.05 we e conside ed s a is ically signi ican . S a- is ical analysis was pe o med using he S a is ical Package o he Social Sciences – SPSS e sion 17.0 o Windows (SPSS Inc., Chi- cago, IL, USA). Resul s C oss-Cul u al Adap a ion A e ob aining he second p elimina y e sion, 2 mi- no changes we e iden i ied as necessa y by subsequen panels (lay people and expe s). (i) Adding he code “9 = no applicable” o he Like scale ha accompanies each i em. In ac , i was de ec ed ha some i ems a e no en- i ely sui ed o he Po uguese con ex . These i ems a e, especially, ela ed o cos s, since in he Po uguese heal h sys em, pa ien s do no pay o only pay a small sha e o he alue o he ea men ca ied ou , a pa being di ec - ly alloca ed o he heal h sys em. (ii) Adding a b ie in o- duc ion on how o comple e he measu e in o de o im- p o e i s unde s andabili y – “Please place an X in he app op ia e box o indica e you deg ee o sa is ac ion wi h each o he ollowing s a emen s (1 = s ongly dis- ag ee, 2 = disag ee, 3 = unce ain, 4 = ag ee, 5 = s ongly ag ee. I no applicable o you si ua ion, please check 9).” A e hese changes we e made, he unde s anding o he panels was he ollowing: he p e- inal Po uguese e sion o he PTOPS is a sho measu e ha is easy o answe , unde s andable, and use ul and he used lan- guage is simple, clea , and colloquial. Valida ion S udy Sample A e he abo emen ioned changes, he p e- inal Po - uguese e sion o he PTOPS was adminis e ed o a sam- ple o 76 subjec s (Table 1) who we e inishing o had al eady inished hei ou pa ien physical he apy ca e in 10 heal h se ices (4 public and 6 p i a e) in he cen al egion o he coun y. O he o al sample, 16 subjec s (21%) also pa icipa ed in he ep oducibili y assessmen . Validi y In he s udy o he p e- inal Po uguese e sion o he PTOPS, he ac o analysis con i med he p oblems ini- ially an icipa ed o he Cos s dimension. Indeed, he analysis iden i ied he i ems 4, 9, 17, 25, and 30 as inde- e mina e. I em 16 (Enhance – I ha e o wai oo long be ween appoin men s) was also conside ed inde e mi- na e. A e emo ing hese i ems, i was possible o iden- i y he e en ion o 3 ac o s (Fig.1), classi ied as Enhanc- e , De ac o , and Loca ion. The componen ma ix a e o hogonal o a ion, which allowed he ex ac ion o a iables associa ed wi h each o he ac o s, is shown in Table 2. I em 8 was e- ained in he componen Loca ion and no in he compo- nen De ac o , since he co ela ion alues a e iden ical and his was i s posi ion in he o iginal measu e. Thus, in he Po uguese e sion o he PTOPS, he 34 o iginal i ems we e educed o a e sion consis ing o 28 i ems dis ibu ed among 3 dimensions: Enhance s – 2, 6, 10, 14, 18, 20, 21, 24, 28; De ac o s – 1, 3, 5, 7, 9, 11, 13, 15, 16, 19, 22, 25, 27; Loca ion – 4, 8, 12, 17, 23, 26. The sco es o i ems 8, 17, and 23 a e subjec o e e sal (1 = 5, 2 = 4, 4 = 2, 5 = 1). All he emaining analyses o psycho- me ic p ope ies we e made wi h his e sion. The s udy o associa ions be ween he sco es o he PTOPS and he ones obse ed in he independen ques- Table 1. Cha ac e is ics o he subjec s (n = 76) Gende Female 40 (52.6) Educa ional le el ≤Basic educa ion 42 (55.3) Employmen s a us Employee 39 (52.0) Age, yea s 45.43±16.37 PTOPS Enhance 4.39±0.42 De ac o 2.07±0.85 Loca ion 4.03±0.75 Cos s 2.93±0.80 Ca ego ical a iables a e exp essed as n (%); quan i a i e a - iables a e exp essed as mean ± s anda d de ia ion. Ca alhei o/Cab i/Fe ei a Po J Public Heal h 4 DOI: 10.1159/000486250 ions demons a ed he exis ence o ela ions o signi i- can alue, which was posi i e wi h he Enhance and Lo- ca ion dimensions and nega i e wi h he De ac o di- mension (Table 3). Reliabili y The C onbach’s α coe icien o he o al o he i ems was 0.73. The alues o he subscales we e 0.66 in he En- hance dimension, 0.96 in De ac o dimension, and 0.84 in he Loca ion dimension. Rega ding he ep oducibili y o he PTOPS, he e- sul s ob ained h ough he ICC we e 0.84 o he Enhanc- e , 0.87 o he De ac o , and 0.86 o he Loca ion com- ponen . Discussion Ou inal sample was composed o 76 subjec s, mos - ly emale (52.6%), wi h quali ica ions equal o o less han basic educa ion (55.3%) and a mean age o 45.43 yea s, co e ing physical he apy ca e in bo h public and p i a e sec o s. These cha ac e is ics a e o e lapping he ones o he Po uguese popula ion ha no mally uses heal hca e in gene al and physical he apy in pa - icula [22, 23]. The me hodology ollowed in he ansla ion o he PTOPS o he Po uguese people and cul u e ollowed he mos o en ecommended c i e ia in he li e a u e, in- cluding hose desc ibed by Hun [14, 15], Meadows e al. [11, 12], and Bea on e al. [13]. By he me hodology adop ed and he esul s ob- ained, we belie e ha he i ems in he ansla ed e - sion ha e he iden ical meaning as he o iginal measu e, wi h exis ing cong uence in he meaning o concep s be ween he English and Po uguese cul u e, poin ing o he ac ha he Po uguese e sion o he PTOPS holds unc ional equi alence. In ou opinion, he use o panels, expe s, and common people o he analysis o Componen s, n 27252321191715131197531 Eigen alue 10 8 6 4 2 0 Fig. 1. Numbe o componen s iden i ied. Colo e sion a ailable online Table 2. Ma ix o he componen o a ion (con e gen o a ion in 5 in e ac ions) PTOPS Componen s 1 Enhance 2 Loca ion 3 De ac o I em 33 0.837 I em 13 0.835 –0.226 I em 23 0.820 I em 27 0.812 I em 15 0.802 –0.217 I em 11 0.797 I em 31 0.769 I em 6 0.713 –0.340 I em 19 0.631 –0.327 I em 8 0.575 –0.220 I em 10 –0.544 0.529 I em 3 0.493 –0.254 I em 1 0.449 I em 20 0.336 0.320 I em 32 0.808 0.254 I em 5 0.677 0.270 I em 28 –0.519 0.656 I em 14 0.643 0.460 I em 21 –0.577 0.632 I em 7 0.653 I em 26 0.628 I em 22 0.325 0.556 I em 2 0.532 I em 18 0.272 0.457 I em 34 0.262 0.419 I em 12 0.389 I em 24 0.385 I em 29 0.332 Me hod o ex ac ion: analysis o he main componen s. Me hod o o a ion: a imax wi h Kaise no maliza ion. Values ma ked in g ay indica e in which componen he i ems load. Po uguese Valida ion o he PTOPS 5 Po J Public Heal h DOI: 10.1159/000486250 accep ance and unde s anding is also a sui able me h- odology. Panelis s el ha he ins umen did no p esen any p oblems as o i s cla i y and ha he e we e no edun- dancies o lack o ele an ques ions. The only p oblems epo ed in he accep ance o he p e- inal e sion con- ce ned he i ems ela ed o he Cos s dimension o he measu e, as hey a e conside ed o be inapp op ia e o he Po uguese eali y. In ac , models o inancing ca e in which he use pays no hing o almos no hing make i - ele an he weigh ha he p ice ac o has on he sa is- ac ion wi h ca e. This issue was subsequen ly con i med in he alida- ion es s and he i ems in ques ion we e emo ed. Thus, he esul s sugges ha he Po uguese e sion o he PTOPS has an accep able le el o con en alidi y. All ec- ommended changes we e o wa ded o he au ho s o he o iginal measu e. Fo he s udy o cons uc alidi y, we used ac o anal- ysis. Thus, i ems 4, 9, 16, 17, 25, and 30 o he o iginal measu e we e iden i ied as inde e mina e ela ing mainly o he Cos s dimension. A e elimina ing hese i ems, he 28 emaining i ems we e na u ally dis ibu ed in a model o 3 componen s, co esponding, in essence, o he di- mensions Enhance , De ac o , and Loca ion, o iginally de ined in he model o Roush and Sons oem [9]. Thus, hese esul s a e qui e e ealing o accep able le els o cons uc alidi y o he Po uguese e sion o he PTOPS. On he o he hand, he s udy o he ela ions o he al- ues ob ained in he Po uguese e sion o he PTOPS wi h he independen ques ions (I would ecommend hese se ices o amily o iends; I would come back o his se ice i I needed physical he apy ea men again; In gene al I am sa is ied wi h my expe ience o physical he - apy), showed esul s as expec ed. In o he wo ds, di ec ela ions wi h he posi i e dimensions (Enhance and Lo- ca ion) and in e se ela ions wi h he nega i e dimension (De ac o ) we e ob ained. The ob ained alues o co ela ion, al hough weak o mode a e, a e ne e heless accep able. Ac ually, much highe alues we e no expec ed because, while hese ques ions a e conside ed some o he mos disc imina ing in e ms o sa is ac ion [10–24], hey do no co e all o he mul idimensionali y o he cons uc in ques ion. Thus, hese esul s suppo he in e ence wi hd awal in e ms o he alidi y o he Po uguese e sion o his ins umen o pa ien sa is ac ion wi h ambula o y phys- ical he apy ca e. The esul s also show ha he Po uguese e sion o he PTOPS p esen s equally accep able alues o eliabil- i y, bo h in ega d o in e nal consis ency and ep oduc- ibili y. Values o C onbach’s alpha highe han 0.70, o he en i e scale, a e sugges i e o accep able le els o he in- e nal consis ency o he Po uguese e sion o he PTOPS. Al hough he o iginal s udy does no e e o al- ues ela ed o he es - e es eliabili y, he ICC alues ound o he adap ed e sion o he PTOPS we e always g ea e han o equal o 0.7 [20], ep esen ing accep able le els o in e empo al s abili y. Some limi a ions o his s udy should be acknowl- edged. The sample used is no ep esen a i e o he en i e popula ion o Po uguese physical he apy ou pa ien s. Fu he alida ion in la ge samples is he e o e ad ised. In conclusion, in iew o he ob ained esul s, he PTOPS is seman ically equi alen o he o iginal measu e and has accep able le els o alidi y and eliabili y. The e- o e and because his measu e is use ul o physical he - apy, i s use is ecommended in bo h clinical p ac ice and esea ch. Acknowledgemen s The au ho s would like o hank he physical he apy s a om he heal hca e ins i u ions. Also, he pa ien s who pa icipa ed in his s udy dese e ou deep ecogni ion. Disclosu e S a emen The au ho s epo no conflic s o in e es . Table 3. PTOPS e sus independen ques ions PTOPS Ques ion (i) Ques ion (ii) Ques ion (iii) Enhance ho*0.549b0.495a0.532b De ac o ho –0.486 –0.497 –0.431 Loca ion ho 0.424 0.303 0.405 (i) I would ecommend hese se ices o amily o iends. (ii) I would come back o his se ice i I needed physical he apy ea men again. (iii) In gene al I am sa is ied wi h my expe ience o physical he apy. * Spea man’s co ela ion coe icien ; all p alues a e <0.05. an = 70. bn = 72. Ca alhei o/Cab i/Fe ei a Po J Public Heal h 6 DOI: 10.1159/000486250 Re e ences 1 Gassée JP, Dehon M, Mei esonne A: Les plain es de pa ien cons i uen -elles un indi- ca eu u ile pou l’amelio a ion de la quali é des soins? Ges ions Hospi aliè es 1996; 356: 389–392. 2 B own K, Sheehan E, Sawye M, Ra os J, Smy h V: Pa en sa is ac ion wi h se ices in an eme gency depa men loca ed a a paedi- a ic eaching hospi al. J Paedia Child Heal h 1995; 31: 435–439. 3 Schulmeis e L, Quie K, Maye K: Quali y o li e, quali y o ca e, and pa ien sa is ac ion: pe cep ions o pa ien s unde going ou pa- ien au ologous s em cell ansplan a ion. Oncol Nu s Fo um 2005; 32: 57–67. 4 Donabedian A: An In oduc ion o Quali y Assu ance in Heal h Ca e. Ox o d, Ox o d Uni e si y P ess, 2003. 5 Bea ie PF, Pin o MB, Nelson MK, Nelson R: Pa ien sa is ac ion wi h ou pa ien physical he apy: ins umen alida ion. Phys The 2002; 82: 557–565. 6 Kei h RA: Pa ien sa is ac ion and ehabili a- ion se ices. A ch Phys Med Rehabil 1998; 79: 1122–1128. 7 Monnin D, Pe nege TV: Scale o measu e pa- ien sa is ac ion wi h physical he apy. Phys The 2002; 82: 682–691. 8 A ne z JE, Almin I, Be gs öm K, F azén Y, Nilsson H: Ac i e pa ien in ol emen in he es ablishmen o physical he apy goals: e - ec s on ea men ou come and quali y o ca e. Ad Physio he 2004; 6: 50–69. 9 Roush SE, Sons oem RJ: De elopmen o he physical he apy ou pa ien sa is ac ion su - ey (PTOPS). Phys The 1999; 79: 159–170. 10 Golds ein MS, Ellio SD, Guccione AA: The de elopmen o an ins umen o measu e sa is ac ion wi h physical he apy. Phys The 2000; 80: 853–863. 11 Meadows K, Ben zen N, Touw-O en F: C oss-cul u al issues: an ou line o he impo - an p inciples in es ablishing c oss-cul u al alidi y in heal h ou come assessmen ; in Hu chinson A, Ben zen N, Konig-Zahn C (eds): C oss Cul u al Heal h Ou come Assess- men : A Use ’s Guide. Ruinne , ERGHO, 1997, pp 34–40. 12 Meadows K: C oss-cul u al issues in he de- elopmen and use o heal h ou come mea- su es; in Long AF, Bli ze E (eds): Heal h Ou - comes and E alua ion: Con ex , Concep s and Success ul Applica ions. Leeds, Uni e - si y P in Se ices, 1997, pp 67–78. 13 Bea on DE, Bomba die C, Guillemin F, Fe - az MB: Guidelines o he p ocess o c oss- cul u al adap a ion o sel - epo measu es. Spine 2000; 25: 3186–3191. 14 Hun SM, Alonso J, Bucque D, Nie o M, Wiklund I, McKenna S: C oss-cul u al adap- a ion o heal h measu es. Eu opean G oup o Heal h Managemen and Quali y o Li e Assessmen . Heal h Policy 1991; 19: 33–44. 15 Hun SM: C oss-cul u al issues in he use o socio-medical indica o s. Heal h Policy 1986; 6: 149–158. 16 Hill MM, Hill A: In es igação po ques ion- á io. Lisbon, Edições Sílabo, 2002. 17 B yman A, C ame D: Análise de dados em ciências sociais: in odução às écnicas u ili- zando o SPSS pa a Windows. Oei as, Cel a Edi o a, 2003. 18 Pes ana MH, Gagei o JN: Análise de dados pa a ciências sociais: a complemen a idade do SPSS. Lisbon, Edições Sílabo, 2005. 19 Cohen L, Holliday M: S a is ics o Social Sci- en is s. London, Ha pe and Row, 1982. 20 Aa onson N, Alonso J, Bu nam A, Loh KN, Pa ick DL, Pe in E, e al: Assessing heal h s a us and quali y-o -li e ins umen s: a i- bu es and e iew c i e ia. Qual Li e Res 2002; 11: 193–205. 21 Te wee CB, Bo SD, de Boe MR, an de Wind DA, Knol DL, Dekke J, e al: Quali y c i e ia we e p oposed o measu emen p ope ies o heal h s a us ques ionnai es. J Clin Epidemiol 2007; 60: 34–42. 22 Po ugal, Minis é io da Saúde, Ins i u o Na- cional de Saúde: Inqué i o Nacional de Saúde: Con inen e, dados ge ais – 98/99. Lisbon, Ob- se a ó io Nacional de Saúde, 1999. 23 Po ugal, Minis é io da Saúde, Di eção Ge al da Saúde: Elemen os es a ís icos: in o mação ge al: saúde 2002. Lisbon, Di eção Ge al da Saúde, 2005. 24 Lab an C: Quels indica eu s de quali é pou les é ablissemen s hospi alie s?: une é ude in e na ionale. Ges ions Hospi aliè es 1996; 356: 363–369.