O iginal Resea ch A icle
INQUIRY: The Jou nal o Heal h Ca e
O ganiza ion, P o ision, and Financing
Volume 59: 1–9
© The Au ho (s) 2022
A icle euse guidelines:
sagepub.com/jou nals-pe missions
DOI: 10.1177/00469580211060143
jou nals.sagepub.com/home/inq
C oss-Cul u al Adap a ion and
Psychome ic E alua ion o he Educa ional
Con en Valida ion Ins umen in Heal h
Se gio Cazo la-Calde ón, PhDc RN
1
, Jos´
e Manuel Rome o-S´
anchez, PhD RN
1,2
,
Elena Fe n´
andez-Ga c´
ıa, PhD RN
3
, and Olga Paloma-Cas o, PhD RN
1,2

Abs ac
Heal hca e p o essionals o en use mul imedia pa ien educa ion media, bu no all ha e he same con en quali y. This s udy aimed o
c oss-cul u ally adap he Educa ional Con en Valida ion Ins umen in Heal h o he Spanish se ing and assess i s psychome ic
p ope ies. A me hodological alida ion s udy was ca ied ou be ween Janua y and Sep embe 2020. Da a collec ion ook place om
May o June 2020. A ansla ion, back ansla ion, commi ee e iew, and p e- es ing was ca ied ou . Subsequen ly, eliabili y (in e nal
consis ency), and alidi y ( ac o ial and con e gen ) we e assessed by equi ing 210 Heal hca e P o essionals o comple e he
ins umen based on ideo ma e ial. In addi ion, a efinemen o he ins umen was conduc ed based on he modifica ion
indexes. The ins umen showed adequa e in e nal consis ency, al hough some edundancy in he i ems (α= .93). Ex-
plo a o y ac o analysis sugges ed a uni ac o ial s uc u e ha explained an adequa e a iance (47.37%). Con e gen alidi y
was poo ( = .11; P= .05). A e analysis, 6 i ems we e dele ed wi hou impai ing he alidi y esul s and elimina ing
edundancy. The e o e, a 12-i em e sion o he ins umen was c ea ed. I can help o assess mo e objec i ely he con en s
o he ma e ials p esc ibed, acili a ing he choice o hose o highe quali y and po en ially imp o ing hei pa ien s’heal h
ou comes. Fu he s udies a e needed o confi m he p e ious esul s and eassess some o he sho comings.
Keywo ds
pa ien educa ion, pa ien educa ion handou , mul imedia, ins umen de elopmen , psychome ics, ques ionnai es, eliabili y
and alidi y, Spain
Wha Do We Al eady Know abou This Topic?
Al hough se e al ins umen s ha e been de eloped in he o m o checklis s o assess a ious aspec s ela ed o he
con en o PEM, none o hese ins umen s ound in a comp ehensi e li e a u e sea ch has been c oss-cul u ally adap ed
and psychome ically es ed in he Spanish se ing.
How Does You Resea ch Con ibu e o he Field?
This s udy p o ides p elimina y e idence on he eliabili y and alidi y o a 12-i em c oss-cul u ally adap ed e sion o
he Educa ional Con en Valida ion Ins umen in Heal h o heal h p o essionals o e alua e he con en o ideo-based
PEM o i s use wi h pa ien s in he Spanish con ex .
Wha a e You Resea ch’s Implica ions Towa d Theo y, P ac ice, o Policy?
The a ailabili y o a 12-i em c oss-cul u ally adap ed o he Educa ional Con en Valida ion Ins umen in Heal h could
be help ul o Spanish heal h p o essionals assis ing hem in selec ing he PEM wi h he bes possible con en o o e
hei pa ien s.
C ea i e Commons Non Comme cial CC BY-NC: This a icle is dis ibu ed unde he e ms o he C ea i e Commons
A ibu ion-NonComme cial 4.0 License (h ps://c ea i ecommons.o g/licenses/by-nc/4.0/) which pe mi s non-comme cial use,
ep oduc ion and dis ibu ion o he wo k wi hou u he pe mission p o ided he o iginal wo k is a ibu ed as specified on he SAGE
and Open Access pages (h ps://us.sagepub.com/en-us/nam/open-access-a -sage).
Main Tex
In oduc ion
Pa ien s ecei e la ge amoun s o new in o ma ion a a single
consul a ion.
1
Heal hca e p o essionals (HCPs) cu en ly ely
hea ily on e bal communica ion and assume ha pa ien s
ha e unde s ood he in o ma ion hey p o ide ully.
2
How-
e e , hey o en use echnical ja gon and p o ide pa ien s wi h
e y complex in o ma ion in a sho ime, which makes i
di ficul o unde s and and memo ize.
2
This si ua ion leads o
40-80% o e bal in o ma ion ecei ed a e a o mal con-
sul a ion being immedia ely o go en,
3
and almos hal o he
in o ma ion pa ien s ecall is inco ec .
4
Gi en he di ficul ies in
he spoken communica ion o heal h in o ma ion, i is common
o use pa ien educa ion media (PEM) o complemen i .
Video-based PEM p o ides isual and audi o y in o ma ion
and is easie o abso b o pa ien s han o he pa ien educa-
ional me hods
2
and hus could o e come ba ie s ela ed o
limi ed li e acy and hea ing impai men , espec i ely.
5
Thus,
ideo-based PEM has become inc easingly commonly used
among heal hca e p ac i ione s.
6
Howe e , i is logical o hink
ha he con en o all o hem may no be o he same quali y.
PEM should mee ele an aspec s o educa ional con en ,
which a e essen ial o ulfill he pu poses o hese ma e ials and
p o ide educa ion as e ficien ly as possible.
7
Fo his, se e al ins umen s ha e been de eloped in he o m
o checklis s o assess a ious aspec s ela ed o he con en s o
PEM.
5,6,8,9
Among hese ools, The Pa ien Educa ion Ma e ials
Assessmen Tool (PEMAT)
10
uses a sys ema ic me hod o
e alua e and compa e he unde s andabili y and ac ionabili y o
pa ien educa ion ma e ials, o he Sui abili y Assessmen o
Ma e ials (SAM) ins umen o e s a sys ema ic me hod o
objec i ely assess he sui abili y o heal h in o ma ion ma e ials.
O he ins umen s we e de eloped o asse s PEM in a specific
o ma , such as he Tool o E alua e Ma e ials Used in Pa ien
Educa ion (TEMP Ed), es ablished o help HCPs e alua e and
selec PEM in p in ed o ma bu no in mul imedia o ma . The
Educa ional Con en Valida ion Ins umen in Heal h (ECVIH),
a ool c ea ed by a g oup o B azilian esea che s o assis HCPs
in assessing heal h educa ion con en in 3 con en - ela ed di-
mensions: objec i es, s uc u e and p esen a ion and ele ance.
7
I is also an ins umen wi h ew i ems and a simple adminis-
a ion and sco ing sys em.
Howe e , none o he ins umen s lis ed abo e o o he s ound
in he comp ehensi e li e a u e sea ch has been c oss-cul u ally
adap ed and psychome ically es ed in he Spanish se ing.
Gi en he cha ac e is ics o b e i y and ease o sco ing and
applica ion o ECVIH, we belie e i would be help ul o
Spanish heal h p o essionals adap he ins umen ha would
assis hem in selec ing he PEM wi h he bes possible
con en o o e hei pa ien s. The e o e, his s udy aimed o
c oss-cul u ally adap he Educa ional Con en Valida ion
Ins umen in Heal h o he Spanish se ing and assess i s
psychome ic p ope ies.
Me hods
Design
A me hodological alida ion s udy was ca ied ou .
C oss-Cul u al Adap a ion
The guidelines p oposed by Guillemin e al.
11
we e used,
sligh ly modified o adap hem o he pu poses o he s udy o
p ese e equi alence in he c oss-cul u al adap a ion o he
ins umen . The ollowing phases we e included:
(1) T ansla ion. Th ee independen ansla ions o Spanish
we e made o he o iginal English ins umen pub-
lished by Lei e e al.
7
Two ansla o s we e nu ses and
we e wa ned o he objec i es and ela i e concep s o
he ma e ial o be ansla ed. A hi d ansla o was a
psychologis who was no wa ned o elici unexpec ed
meanings om he o iginal ins umen . All o hem
we e bilingual, English, and Spanish; hei mo he
ongue is he la e . The ansla o s discussed he 3
e sions and s udied he disc epancies un il a
consensus e sion was eached.
(2) Re o- ansla ion. The consensus e sion was e o-
ansla ed in o English by a p o essional ansla o
wi h a s able esidence in Spain whose mo he ongue
is English and un amilia wi h he ma e ial.
(3) Re iew. A commi ee o expe s was se up o compa e
he e o- ansla ion wi h he o iginal e sion o he
ins umen . This commi ee consis ed o he p e ious
ansla o s oge he wi h he p incipal au ho o he
o iginal ins umen . The decen e ing echnique was
used in which he i ems wi h p oblems we e loca ed,
and he au ho p o ided a wo king e sion o hem,
main aining he concep . On he o he hand, gi en ha
he o iginal ool was designed o assess w i en PEM
and his s udy sough o alida e i using he ma e ial in
1
Resea ch G oup PAIDI CTS-1019 “MELES - Nu sing Me hods and S anda d Languages”, Uni e sidad de C´
adiz, C´
adiz, Spain
2
Facul y o Nu sing and Physio he apy, Nu sing and Physio he apy Depa men , Uni e sidad de C´
adiz, C´
adiz, Spain.
3
Facul y o Nu sing, Physio he apy and Podia y,. Nu sing Depa men , Uni e sidad de Se ila, Se illa, Spain.
Co esponding Au ho :
Jos´
e Manuel Rome o-S´
anchez, Uni e sidad de C´
adiz. Facul ad de En e me ´
ıa y Fisio e apia, A da. Ana de Viya, 52; 11009 C´
adiz (Spain).
Email: [email p o ec ed]
2INQUIRY
audio- isual o ma , i was ag eed wi h he au ho o
eplace i em 15, “The ex size is adequa e”wi h “The
na a i e/dialogues a e hea d clea ly.”Once he ex-
pe s app o ed he back- ansla ion, he o iginal and
he Spanish adap a ion we e gi en o 5 bilingual nu ses
o de ec disc epancies.
Pa icipan s
The sample comp ised HCPs mee ing he ollowing inclusion
c i e ia: (1) Hold a bachelo ’s deg ee in heal h science ha
qualifies hem o p esc ibe PEM o pa ien s and (2) p ac ice o
ha e p ac ised he p o ession o which hey a e qualified in
Spain. Visual and/o audi o y impai men s ha p e en he
iewing o mul imedia ma e ial we e only conside ed as an
exclusion c i e ion.
This con enience sample was selec ed h ough non-
p obabilis ic ne wo k sampling
12
o ob ain subjec s wi h
di e se p o essional p ofiles and se ings. The 10:1 a io ule,
en pa icipan s pe i em, was adop ed o de e mine a su fi-
cien sample size.
13–15
Conside ing ha he ins umen o be
psychome ically es ed con ains 18 i ems, he sample size
would be se a 180 subjec s. Howe e , cu en ends ec-
ommend a sample size o a leas 200 cases, e en unde
op imal condi ions o high communali ies and well-defined
ac o s.
16
The e o e, an a emp was made o maximize he
sample size and hus ob ain a highe numbe o subjec s han
ini ially p oposed.
A panel o expe s was also selec ed in en ionally o enable
he assessmen o he con e gen alidi y o c i e ia. I
consis ed o 9 people ollowing he ecommenda ion o Poli
and Ta ano.
17
To belong o he panel, hey had o mee he
eligibili y c i e ia o he abo e sample and ha e p o en
expe ience, ha ing de eloped a leas 5 PEM in he las
2 yea s.
Ins umen s
The ollowing ins umen s we e used
(1) Socio-demog aphic and p o essional da a ques-
ionnai e. Consis ed o ques ions o assess he ol-
lowing a iables: age (yea s), gende (male, emale),
heal hca e p o essional p ofile (assis ance, eaching,
and esea ch, managemen ), educa ional p ofile
(nu se, physician, psychologis ), p o essional expe-
ience (yea s), academic deg ee (diploma, bachelo o
g adua e deg ee, o ficial mas e ’s deg ee, doc o a e).
(2) Specific ques ions o expe ’s ques ionnai e. Applied
o expe s only oge he wi h he o he ins umen s. I
consis ed o ques ions o assess he ollowing a i-
ables: o ma o he PE ma e ial c ea ed by hem
(w i en, mul imedia), PEM c ea ed in he las 2
yea s in each o ma (numbe ), and assessmen o he
con en s o he pa ien educa ion ma e ial shown
assessed wi h a single ques ion o mula ed as, “Ra e
he o e all con en o he ma e ial displayed o ed-
uca e he pa ien on (subjec o he ideo, in his case,
Basic li e suppo (BLS) om 0 o 10”and wi h a 10-
poin Like - ype measu emen scale whe e 0 indi-
ca es ha he o e all con en is “Ve y bad”and 10
“Ve y good.”
(3) Spanish e sion o he Educa ional Con en Valida-
ion Ins umen in Heal h (ECVIH-SV). De eloped
o his s udy in he c oss-cul u al adap a ion p ocess
desc ibed abo e. The o iginal ins umen was de-
eloped based on a na a i e e iew o he li e a u e
and expe consensus.
7
I has 18 i ems alloca ed
heo e ically in o 3 domains: (1) objec i es (i ems 1-
5), ela ing o pu pose, goals, o a ge he use o
educa ional ma e ials; (2) s uc u e and p esen a ion
(i ems 6-15), ela ing o gene al o ganiza ion, s uc-
u e, s a egy, consis ency, and su ficiency o he
p esen a ions; and (3) ele ance (i ems 16-18), ela ing
o he significance deg ee o he educa ional con en
and i s abili y o cause impac , mo i a ion, and/o
in e es . The i ems we e applied o he PEM o be
assessed, using a h ee-poin Like esponse scale
whe e 0 = disag ee, 1 = pa ially ag ee, and 2 = o ally
ag ee. Acco ding o expe s, he i ems ob ained mo e
han 80% ag eemen and good in e nal consis ency in
hei domains in he con en alida ion. ECVIH i ems
a e p esen ed in Online Appendix 1.
Da a Collec ion
The au ho s designed 2 non- alida ed anonymous elec onic
su eys o collec he da a specified in he p e ious sec ion, one
o he expe s’sample and he o he o he gene al sample.
Pa icipan s o bo h expe s and gene al samples we e
con ac ed by mail, gi ing in o ma ion abou he pu pose o
he s udy, he iden i y o he in es iga o s, and he ime in-
es men o he su ey. To fill in he su ey, access o he
websi e was h ough a link o he su ey, passwo d-p o ec ed
o p e en malicious access. Passwo d was sen oge he wi h
he in i a ion email. The elec onic su ey sys em p e en ed
mul iple en ies om he same indi idual h ough email
add esses. The anonymi y o he esponses was ensu ed. No
ad e ising was used.
Bo h su eys we e designed using he comme cial web
su ey p o ide www.su eymonkey.com. The accoun was
only accessible o he leading in es iga o o da a p o ec ion
pu poses. The web su ey was es ed be o e he s a o he
s udy by colleagues. Ques ions asked o all pa icipan s we e
manda o y (an au oma ic op ion in he su ey p og am).
Responden s could e iew and change exis ing answe s un il
he su ey was finished h ough a “back bu on.”
PEM o be assessed is needed o apply he ins umen . In
his case, 3 PEM we e included in he su eys, p ecisely 3
di e en ideos on he same opic, “how o pe o m Basic
Li e Suppo wi h an au oma ic defib illa o ,”aimed a he
Cazo la-Calde ón e al. 3
gene al public. These ideos we e p e-selec ed by he e-
sea ch eam based on hei opinion and ega ding hei
quali y, looking o di e ences be ween hem. The au ho s o
he ideos ha we e no classified as ee use we e asked o
au ho iza ion.
Each pa icipan in he gene al sample was equi ed o
use he ECVIH ins umen o assess he con en o only one
o hese ideos assigned andomly by he Web applica ion.
Video 1 was assessed by 83 pa icipan s (39.5%), ideo 2 by 70
(33.3%), and ideo 3 by 57 (27.1%). In con as , expe s
assessed all 3 ideos since hei assessmen was necessa y o
assess con e gen alidi y. The ideos we e shown o expe s
in andom o de o p e en he lea ning e ec in he applica ion
o he ins umen om cons i u ing a bias in he sco es.
Da a collec ion was conduc ed om May o June 2020.
E hical Conside a ions
This s udy is pa o a la ge p ojec app o ed by he An-
dalusian Coo dina ing Commi ee on E hics in Biomedical
Resea ch (PI-0019-2017). Pa icipa ion was olun a y, and
he anonymi y o he pa icipan s was gua an eed, so he web
applica ion did no eco d any da a ha could be used o
iden i y hem. The pa icipan s we e in o med o he pu pose
o he s udy and he p ocessing o he da a. They also signed a
w i en consen o m be o e filling he su ey. W i en pe -
mission was ob ained om he au ho s o he ins umen o
i s c oss-cul u al adap a ion and alida ion.
Da a Analysis
The no mali y o he a iables was checked using he
Kolmogo o -Smi no -Lillie o s es .
18
Da a ela ing o he
dis ibu ion sco es and he sample cha ac e is ics we e
summa ized using he desc ip i e s a is ics, he mean and
s anda d de ia ion o he quan i a i e a iables, while e-
quency and pe cen age we e used o he ca ego ical ones.
The eliabili y assessmen included es s o in e nal con-
sis ency: (a) C onbach’salpha(α) o he comple e ins umen
and when each i em was emo ed. A alue o αo .70-.90 was
conside ed app op ia e and ha he e we e no subs an ial
changes in alue when he i em was excluded; (b) i em- o al
co ela ion co ec ed using he Pea son co ela ion coe ficien .
An i em- o al co ela ion o ≥.40 was conside ed app op ia e.
19
The con e gen alidi y o he ins umen was assessed using
he assessmen o he o e all con en s o he PEM by he expe s.
Gi en ha no ins umen alida ed in Spain ha measu es he
same cons uc was ound in he li e a u e sea ch, an ad hoc
ques ion was used o assess he con en in gene al e ms. The
o al sco es on he ins umen we e co ela ed wi h he a e ages
ob ained in he p e ious ques ion in each o he 3 ideos shown
o he expe s h ough Pea son’s es . A co ela ion coe ficien
anging om .30-.40 indica es a high con e gen alidi y.
13
The cons uc alidi y was assessedin e mso ac o ial
alidi y h ough explo a o y ac o analysis (EFA) using p in-
cipal componen analysis (PCA). An EFA was ca ied ou using
he main componen s analysis me hod. As well as concep ual
in e p e abili y, he ollowing c i e ia we e used o iden i y he
op imal numbe o ac o s: (a) Sc ee plo
20
de e mined he poin
whe e he slope o he cu e is le eling o ;
21
(b) explained
a iance o a leas 40%
22
and ha each ac o con ains a leas 3
i ems.
23
The ac o ial ma ix was examined, ex ac ing hose
i ems ha load unambiguously on heex ac ed ac o s,con-
side ing hose wi h ac o loading >.50 on exac ly 1 ac o and
<.40 on all o he ac o s.
24
Da a we e checked be o ehand o
sui abili y o ac o analysis by examining Kaise –Meye –
Olkin (KMO) indices es o sampling adequacy and Ba le ’s
es o sphe ici y (BTS). A alue o KMO <.50 and a BTS
significance le el o P< .05 was being conside ed app op ia e.
25
Once he op imal ac o solu ion acco ding o he EFA was
ob ained, a confi ma o y ac o analysis (CFA) was ca ied
ou o de e mine whe he he da a fi he p oposed model. The
ollowing adjus men indices we e used: (a) Significance o
chi-squa e (χ2), being desi able a χ2 no significan o he
model;
26
(b) compa a i e fi index (CFI) and Tucke –Lewis
Index (TLI) o which alues g ea e han .90 indica e a good
fi , while alues g ea e han .95 indica e an excellen fi
27,28
and (c) oo mean squa e e o o app oxima ion (RMSEA)
o which alues below .08 indica e an accep able model fi
and .06 an excellen fi , such as a i s uppe limi o he 90%
confidence in e al.
29
I he model did no fi he da a ob-
ained in he sample, efining he ins umen by s udying he
disc epancies be ween he model and he da a h ough he
modifica ion indexes (MIs) was conside ed.
30
The MIs in-
dica e he amoun o dec ease in he χ2 o he model wi h a
deg ee o eedom. A high MI alue indica es ha he ele an
fixed-pa ame e should be eleased o imp o e he model
fi .
26
To ee i , i was decided o elimina e he specificele-
men s wi h highe MIs, which mean an imp o emen in he
model fi , wi h a dec ease o χ2 and a subs an ial imp o emen
o he o he adjus men indices.
31
The elimina ion was ca ied
ou one by one, s a ing wi h he one wi h he highes MI and so
on successi ely since changing a single pa ame e in a model
could a ec o he pa s o he solu ion.
32
A e each elimi-
na ion, he CFA was epea ed on he emaining elemen s,
assessing he specific adjus men a es. These s eps we e e-
pea ed p og essi ely un il he indices showed a good model fi
acco ding o he c i e ia specified in he p e ious pa ag aph.
Once he i ems ha comp ised he defini i e model we e
e ained, he eliabili y and con e gen alidi y es s we e
epea ed o check i hey also ob ained simila o be e
psychome ic p ope ies in addi ion o hei dimensionali y.
The esul s we e conside ed s a is ically significan i he
P- alues we e <.05. Da a we e analyzed using IBM SPSS
S a is ics o Windows, e sion 25, and IBM AMOS, e sion
23 (IBM Co p, A monk, NY, USA).
4INQUIRY
Resul s
C oss-Cul u ally Adap ed Ins umen
The ansla ions made independen ly by he ansla o s ob-
ained comple e coincidence in 3 i ems, pa ial coincidence in
4, and disc epancy in ele en. Pa ial coincidences we e
sol ed by choosing he ansla ion o he 3 ha bes ep e-
sen ed he i em, wi h each o ing o he 1 ha seemed
co ec . T ansla ions o dissen ing i ems we e esol ed by
join discussion, ob aining a consensus ansla ion o each
i em. The e o- ansla ion was e ised by he ansla o s and
one o he au ho s o he o iginal ins umen . She only ad-
jus ed 3 i ems. The ansla ions o hese we e modified
conce ning hei con ibu ions and sen back o compa ison,
a e which he ansla ion was app o ed. The bilingual
nu ses who e iewed he Spanish and he o iginal ins umen s
endo sed he equi alence o bo h e sions. The Spanish
e sion o he ECVIH is gi en in Online Appendix 1.
Desc ip ion o he Pa icipan s
O he 600 ques ionnai es sen ou , 222 we e e u ned co ec ly
filled in, implying a esponse a e o 37%. O hese, 12 we e
elimina ed as hey did no mee he inclusion c i e ia (8 we e non-
uni e si y senio heal hca e echnicians, and 4 we e non-g adua e
nu sing s uden s). The e o e, he final sample consis ed o 210
pa icipan s, mos o whom we e women (55.5%) wi h an a e age
age o 40.8 yea s (SD = 10.41). Mos o he sample we e nu ses
(79%), wo king mainly in he heal hca e field (80%) wi h an
accumula ed p o essional expe ience o 16.53 yea s (SD = 9.95).
Conce ning aining, 52.6% o he pa icipan s had some pos -
g adua e uni e si y educa ion. The pa icipan s’demog aphic and
p o essional p ope ies a e shown in de ail in Table 1.
Rega ding he expe s, 8 we e nu ses (88.9%) and 1 was a
physician. 77.8% had pos g adua e aining. Mos o hem had
heal hca e unc ions (88.9%) and an accumula ed p o essional
expe ience o 18 yea s (SD = 6.46). Mos o hem had ex-
pe ience in c ea ing pa ien educa ion ma e ials in bo h w i en
and mul imedia o ma s (77.8%), he es in w i en o ma
only, wi h an a e age o 19.44 ma e ials in w i en o ma
(SD = 19.70) and 22.00 in mul imedia o ma (SD = 29.26).
Sco e Dis ibu ion
The a e age scale sco e o he sample was 29.88 (SD = 2.77)
ou o a maximum o 36 poin s. The i ems had a e age sco es
in he ange o 1.33-1.87. The means and s anda d de ia ions
o each i em a e shown in Table 2.
Reliabili y
In e nal consis ency. The α alue o he ins umen was .93.
No i em significan ly inc eased o dec eased he αwhen i
was emo ed. All i ems eached good i em- o al co ela ions
(.41-.76), excep i em 14, which ob ained a low co ela ion
(.28). These esul s a e shown in de ail in Table 2.
Validi y
Con e gen alidi y. A poo , hough significan , co ela ion ( =
.11; P= .05) was ound be ween he o al ins umen sco es and
he a e age o he sco es gi en by he expe s o each ideo.
Fac o ial cons uc alidi y. The Ba le sphe ici y es (χ
2
=
2125.66; d = 153; P< .001) eached s a is ical significance,
and he KMO alue was adequa e (KMO = .92). The p e-
limina y PCA e ealed he p esence o 3 componen s wi h
eigen alues g ea e han 1, which explained he 47.37%, 7.30%,
and 5.99% o he a iance, espec i ely. Howe e , an inspec ion
o he sc ee plo indica ed a significan slope a e he fi s
componen , ollowed by a clea pla eau. The ac o ial ma ix
was examined, de e mining ha all bu 3 i ems had high ac o ial
loadings in he fi s componen (.45-.86) and low ones in he es
(<.40).I ems16and18loadabo e.40onbo h hefi s and
second componen s, al hough he loadings on he fi s we e
significan ly highe han on he second (i em 16: .69 s .44; i em
18: .71 s .41). I em 14 loads high on he second (.58) and hi d
componen (.53). Conside ing he sc ee plo , he high ac o
loading o almos all i ems on he fi s componen , he accep able
a iance explained by i , and ha he o he 2 componen s did no
sa is y he condi ion, a leas 3 i ems load he highes on ha
ac o . I was decided o ca y ou a new PCA by o cing he
Table 1. Pa icipan s’Demog aphic and P o essional
Cha ac e is ics.
N% MSD
Sex
Women 116 55.5%
Men 91 43.5%
N/A 2 1%
Age (yea s) 40.8 10.41
Field o wo k
Heal h ca e 168 80%
Managing 13 6.2%
P o esso and/o esea che 29 13.8%
Occupa ion
Nu se 166 79%
Physician 37 17.6%
Psychologis 5 2.4%
O he 2 1%
P o essional expe ience (yea s) 16.53 9.95
Academic deg ee
Bachelo 99 47.4%
Mas e 87 41.6%
PhD 23 11%
M, mean; SD, s anda d de ia ion.
Cazo la-Calde ón e al. 5

ex ac ion o a single componen ha explained a a iance o
47.37%, wi h all i ems wi h high loadings on i (.46-.81) excep
i em 14 ha has a sligh ly lowe loading on he componen (.32)
(Table 2). The e o e, he esul s o his second analysis suppo
he one-dimensionali y o he ins umen .
The CFA ca ied ou , assuming a one- ac o model as
ex ac ed om he EFA, de e mined ha his model did no
fi well wi h he da a ob ained wi h poo adjus men indices:
X
2
= 474.89 (P< .0001); CFI = .83; TLI = .81; RMSEA =.11
(90% CI:0.10-.12).
Ins umen Refinemen
Based on hese esul s, i was decided o efine he ool by e-
es ima ing and eassessing i mul iple imes o adjus he da a
o he chosen model by s udying he MIs changes by pai s o
i ems. I ems 6, 18, 5, 3, 10, and 14 we e p og essi ely
elimina ed un il an app op ia e fi o he da a o he model was
achie ed, ob aining excellen fi indices wi h he emaining
12 i ems excep o he χ2(χ
2
=101.21(P< .0001); CFI =
.96; TLI =.95;RMSEA =.06(90%CI:0.04-.08). Figu e 1
shows he selec ed final model wi h i s ac o loadings. As
can be seen, all he i ems ha e loadings highe han .50
excep i em 15, which loadings abo e .40. A 12-i ems
ECVIH was ob ained om his efinemen p ocess (Online
Appendix 2).
Once he ins umen had been efined, he es o he
psychome ic p ope ies we e eassessed conside ing only he
12 i ems e ained, wi h he ollowing esul s. Rega ding eli-
abili y in e ms o in e nal consis ency, he α o he 12-i ems
ins umen was .89. No i em significan ly inc eased o de-
c eased he αwhen i was emo ed (.88-.90). Rega ding he
con e gen alidi y, a poo co ela ion was again ound be-
ween he o al sco e o he 12-i ems ins umen and he a e age
o he sco es gi en by he expe s o each ideo, al hough mo e
significan han o he 18-i ems ECVIH ( = .14; P=.04).
Discussion
This s udy aimed o c oss-cul u ally adap and alida e he
Educa ional Con en Valida ion Ins umen in Heal h (EC-
VIH). I is he fi s assessmen o he psychome ic p ope ies
o ECVIH on a sample o HCPs. The ins umen , de eloped
o he B azilian con ex , was only p e iously assessed o
con en alidi y by expe s in he same s udy ha de eloped
i .
7
This s udy, he e o e, p o ides e idence o he in e nal
p ope ies o ECVIH when applied by HCPs on PEM, en-
do sing i s use wi h a mo e excellen scien ific basis.
The esul s endo se he eliabili y o he 18-i ems in-
s umen in e ms o in e nal consis ency wi h an adequa e
psychome ic pe o mance o mos i ems; howe e , he α
alue abo e .90 o he en i e ins umen exceeds he alues
Table 2. I em Sco es, Reliabili y Resul s, and Fac o Loadings o ECVIH.
I ems
I em
Sco e To al Sco e i I em
Dele ed, M
Co ec ed I em-To al
Co ela ion
C onbach’s Alpha i
I em Dele ed
Fac o
LoadingsMSD
1. Con empla es he p oposed heme 1.84 0.41 28.05 0.54 0.93 0.59
2. Sui s he eaching-lea ning p ocess 1.59 0.57 28.30 0.70 0.93 0.74
3. Cla ifies doub s on he add essed heme 1.42 0.65 28.47 0.61 0.93 0.66
4. P o ides eflec ion on he heme 1.56 0.65 28.32 0.67 0.93 0.72
5. Encou ages beha io change 1.57 0.64 28.31 0.69 0.93 0.74
6. Language app op ia e o he a ge
audience
1.66 0.58 28.23 0.74 0.92 0.78
7. Language app op ia e o he educa ional
ma e ial
1.70 0.56 28.19 0.71 0.93 0.76
8. In e ac i e language, enabling ac i e
in ol emen in he educa ional p ocess
1.33 0.74 28.55 0.60 0.93 0.64
9. Co ec in o ma ion 1.59 0.61 28.30 0.63 0.93 0.68
10. Objec i e in o ma ion 1.72 0.54 28.17 0.69 0.93 0.74
11. Enligh ening in o ma ion 1.52 0.63 28.36 0.74 0.92 0.78
12. Necessa y in o ma ion 1.87 0.39 28.02 0.61 0.93 0.67
13. Logical sequence o ideas 1.71 0.55 28.17 0.69 0.93 0.73
14. Cu en heme 1.86 0.40 28.02 0.28 0.93 0.32
15. App op ia e ex size 1.78 0.46 28.11 0.41 0.93 0.46
16. Encou ages lea ning 1.70 0.50 28.18 0.65 0.93 0.69
17. Con ibu es o knowledge in he a ea 1.77 0.46 28.11 0.76 0.93 0.81
18. A ouses in e es in he heme 1.70 0.52 28.19 0.67 0.93 0.71
M, mean; SD, s anda d de ia ion.
6INQUIRY
se as accep able. Ex emely high alpha alues sugges e-
dundancy be ween some o he i ems;
33
some i ems measu e
he same aspec o he cons uc .
34
In his case, i is usual o
elimina e he edundancy by elimina ing i em.
17
In his s udy,
i ems we e elimina ed based on he ac o abili y o he in-
s umen . Howe e , e en hough he efinemen was ca ied
ou based on a alidi y p ope y, he e-assessmen unde aken
o he eliabili y o he 12-i em ECVIH, all i ems e ained o
imp o ed pe o mance, and he alue o alpha o he en i e
ins umen dec eased o op imal alues, hus elimina ing
edundancy.
Al hough he i ems we e alloca ed in o 3 dimensions
heo e ically o s uc u e he ins umen in i s c ea ion,
7
a he
empi ical le el, he scale seems o espond o a one- ac o
s uc u e sugges ed by he EFA esul s, which accumula ed
adequa e a iance and ac o loadings on he app op ia e
ac o excep o 1 i em in he 18-i ems o iginal ins umen .
No good fi was ob ained when he one- ac o s uc u e was
es ed h ough a CFA on he da a o he 18 i ems. In he
con ex o s uc u al equa ion models, MIs a e a measu e
o en used o educe disc epancies be ween model and da a,
30
soi wasdecided ouse hem o efine he ins umen . Based
on hese, a model 1 showing an excellen fi o all he
calcula ed indices was ob ained, excep o he χ2, which is a
pa ame e ha is ex emely sensi i e o he sample size.
35
A
educed o m was ob ained om he p ocess wi h a clea
s uc u e consis ing o 1 ac o and be e eliabili y o he 12
i ems e ained. I also educed he ins umen applica ion
imes.
None o he e sions, he o iginal 18 o he efined 12-
i ems e sion, ga e good esul s o con e gen alidi y.
Among he easons behind hese nega i e findings is ha he
expe s’opinion was collec ed wi h a single ques ion de-
eloped ad hoc, which may no cap u e he en i e cons uc .
The e a e a ious po en ial limi a ions o he s udy ha
need o be discussed. The in e en ion o a single ansla o in
he back- ansla ion p ocess o he ins umen made i im-
possible o implemen a double-check mechanism. The use o
a con enience sample limi s he gene aliza ion o he esul s.
No all aspec s o eliabili y o alidi y we e assessed.
Sensi i i y o change was also no e alua ed, a p ope y ha
mus be known i he ins umen is o be used as a ool by he
c ea o s o heal h educa ion ma e ial o assess imp o emen s
in ha ma e ial’s con en .
Mo eo e , in his s udy, he CFA was ca ied ou on he
same da a as he EFA, as was he de e mina ion o p ope ies
o he efined e sion o 12 i ems. Fo all he abo e easons, i
would be necessa y o ca y ou u u e s udies wi h mo e
significan and ep esen a i e samples, i possible, o mi -
iga e hese deficien aspec s. Simila ly, i would be neces-
sa y o de e mine whe he he sou ce e sion and o he
c oss-cul u al adap a ions o o he in e na ional con ex s
p o ide simila psychome ic esul s. Finally, he educa-
ional ma e ial on which he alida ion was ca ied ou was
only o a mul imedia ype, and i would be necessa y o
assess he ool’s pe o mance on he ma e ial in w i en
o ma i i is o be used on i .
Conclusion
The esul s o his s udy p o ide p elimina y e idence on he
eliabili y and alidi y o ECVIH o heal h p o essionals o
e alua e he con en o ideo-based PEM o i s use wi h
pa ien s in he Spanish con ex . A efined e sion o 12 i ems
is p oposed, which, in addi ion o imp o ing he ac o abili y
o he o iginal ins umen , allows i o be comple ed in less
ime, imp o ing o main aining he psychome ic p ope ies
o he o iginal 18-i ems e sion. The a ailabili y o an in-
s umen makes i easie o HCPs o assess he con en s o he
educa ional ma e ial hey p esc ibe o pa ien s wi h g ea e
objec i i y, acili a ing he choice o he highes quali y ones
and po en ially imp o ing hei pa ien s’heal h ou comes. I
Figu e 1. I ems s uc u e.
Cazo la-Calde ón e al. 7
also allows he c ea o s o PEM o assess he con en o he
ma e ial, allowing o imp o emen in hose a eas whe e
HCPs pe cei e deficiencies.
Acknowledgmen s
The au ho s g a e ully hank he au ho s o he o iginal ins umen ,
D . Lei e and he colleagues, o accep ing ou eques o c oss-
cul u ally adap and alida ion he ECVIH as well hei suppo in
his s udy. A special hanks o Ca olina C´
ano as Ma ´
ınez (RN) and
Jos´
e Manuel Salas Rod ´
ıguez (MD) ounde s o Jacin o & iends’
P ojec (www. hinking-heal h.com) o allow us o use one hei
ideos o ou s udy. We exp ess ou g a i ude o e e yone who
pa icipa ed by p o iding expe ise and all he esponden s o
making possible he s udy epo ed in his pape .
Au ho Con ibu ions
Se gio Cazo la-Calde on: Concep ualiza ion, in es iga ion, e-
sou ces, w i ing—o iginal d a .
Jose Manuel Rome o-Sanchez: Me hodology, o mal analysis, da a
cu a ion, w i ing—o iginal d a .
Olga Paloma-Cas o: Concep ualiza ion, esou ces, w i ing— e iew
and edi ing, p ojec adminis a ion, unding acquisi ion.
Elena Fe nandez-Ga cia: In es iga ion, w i ing— e iew and edi ing.
Decla a ion o Conflic ing In e es s
The au ho (s) decla ed no po en ial conflic s o in e es wi h espec
o he esea ch, au ho ship, and/o publica ion o his a icle.
Funding
The au ho (s) disclosed eceip o he ollowing financial suppo o
he esea ch, au ho ship, and/o publica ion o his a icle: This pape
is pa o he fieldwo k o he p ojec en i led “E ec i eness o
Mul imedia Con en Exposu e in Imp o ing he Expe ience and
Repo ed Pa ien Ou comes in Pa ien s Su e ing om AMI Du ing
he T ans e o Hospi al: a Clinical T ial.”(PI-0019-2017), unded
by Heal h Depa men o he Andalusian Go e nmen , Spain, in he
‘2017 ITI Call o Funding o Heal h Sciences & Biomedical
Resea ch G an s’a e a igo ously pee - e iewed unding p ocess.
ORCID iDs
Se gio Cazo la-Calde ón h ps://o cid.o g/0000-0003-4965-6011
Jos´
e Manuel Rome o-S´
anchez h ps://o cid.o g/0000-0001-8227-9161
Elena Fe n´
andez-Ga c´
ıah ps://o cid.o g/0000-0002-7922-2663
Supplemen al Ma e ial
Supplemen al ma e ial o his a icle is a ailable online.
Re e ences
1. Selic P, S ab I, Repolusk M, Gucek NK. Wha ac o s a ec
pa ien s’ ecall o gene al p ac i ione s’ad ice? BMC Fam
P ac . 2011;12(1):141. doi:10.1186/1471-2296-12-141.
2. Abu Abed M, Himmel W, Vo m elde S, Koschack J. Video-
assis ed pa ien educa ion o modi y beha io : A sys ema ic
e iew. Pa ien Educ Counsel. 2014;97(1):16-22. doi:10.1016/
j.pec.2014.06.015
3. Kessels RPC. Pa ien s’memo y o medical in o ma ion. JR
Soc Med. 2003;96(5):219-222. doi:10.1258/j sm.96.5.219.
4. Jansen J, an Wee J, an de Meulen N, an Dulmen S, Hee en
T, Bensing J. Recall in olde cance pa ien s: Measu ing
memo y o medical in o ma ion. Ge on ol. 2008;48(2):
149-157. doi:10.1093/ge on /48.2.149.
5. Fa wana R, She i A, Manza H, Fa wana M, Yusu A,
She i I. Wa ch his space: A sys ema ic e iew o he use o
ideo-based media as a pa ien educa ion ool in oph hal-
mology. Eye. 2020;34(9):1563-1569. doi:10.1038/s41433-
020-0798-z.
6. Klein-Fedyshin M, Bu da ML, Eps ein BA, Law ence B.
Collabo a ing o enhance pa ien educa ion and eco e y.
J Med Lib Assoc: JMLA. 2005;93(4):440-445.
7. Lei e SDS, ´
Afio ACE, Ca alho LVD, Sil a JMD, Almeida
PCD, Pagliuca LMF. Cons uc ion and alida ion o an edu-
ca ional con en alida ion ins umen in heal h. Re B as
En e m. 2018;71(suppl 4):1635-1641. doi:10.1590/0034-7167-
2017-0648.
8. Fe guson LA. Implemen ing a ideo educa ion p og am o
imp o e heal h li e acy. J Nu se P ac . 2012;8(8):e17-e22. doi:
10.1016/j.nu p a.2012.07.025.
9. F eda MC. Issues in pa ien educa ion. J Midwi e y Wom Heal h.
2004;49(3):203-209. doi:10.1016/j.jmwh.2004.01.003.
10. Shoemake JS, Wol MS, and B ach C. The Pa ien Educa ion
Ma e ials Assessmen Tool (PEMAT) and Use ’s Guide.
Rock ille: Agency o Heal hca e Resea ch and Quali y U.S.:
1-5. h ps://www.ah q.go /p o essionals/p e en ion-ch onic-
ca e/imp o e/sel -mgm /pema /index.h ml (2017).
11. Guillemin F, Bomba die C, Bea on D. C oss-cul u al adap-
a ion o heal h- ela ed quali y o li e measu es: li e a u e e-
iew and p oposed guidelines. J Clin Epidemiol. 1993;46(12):
1417-1432. doi:10.1016/0895-4356(93)90142-n.
12. Clay on LH. TEMP Ed: De elopmen and psychome ic
p ope ies o a ool o e alua e ma e ial used in pa ien edu-
ca ion. J Ad Nu s. 2009;65(10):2229-2238. doi:10.1111/j.
1365-2648.2009.05049.x.
13. Nunnally JC, Be ns ein IH. Psychome ic Theo y. New Yo k:
McG aw-Hill; 1994.
14. Tho ndike RL. Applied Psychome ics. Bos on: Hough on-
Mi flin; 1982.
15. MacCallum R, Widaman K, Zhang S, Hong S. Sample size in
ac o analysis. Psychological Me hods - Psychol Me hods.
1999;4:84-99. doi:10.1037/1082-989X.4.1.84.
16. Fe ando PJ, Lo enzo-Se a U. El an´
alisis ac o ial ex-
plo a o io de los ´
ı ems: algunas conside aciones adicionales.
An Psicolog 2014;30(3):1170-1175. doi:10.6018/analesps.30.
3.199991.
17. Poli DF, and Beck CT. Nu sing Resea ch: P inciples and
Me hods. Philadelphia: Lippinco Williams & Wilkins. h ps://
books.google.es/books?id=5g6V YWnjUC (2004).
8INQUIRY
18. Lillie o s HW. On he Kolmogo o -Smi no es o no mali y
wi h mean and a iance unknown. J Am S a Assoc. 1967;
62(318):399-402. doi:10.1080/01621459.1967.10482916.
19. No man GR, and S eine DL. Bios a is ics: The Ba e Es-
sen ials. Hamil on: B.C. Decke . h ps://books.google.es/
books?id=8 kqWa dpuoC (2008).
20. Ca el RB. The Sc ee Tes Fo The Numbe o Fac o s.
Mul i a ia e Beha io al Resea ch 1996;1(2):245-276. doi:10.
1207/s15327906mb 0102_10. In p ess.
21. Kim J, Muelle CW. Fac o Analysis: S a is ical Me hods and
P ac ical Issues. Thousand Oaks, CA, USA: Sage Publica ions;
1978.
22. Manza D, Zanna W, Mohammed H, e al. Dimensionali y o
he Pi sbu gh Sleep Quali y Index in young collegia e
adul s. Sp inge Plus. 2016;5. doi:10.1186/s40064-016-
3234-x.
23. Ne emeye R, Bea den W, Sha ma S. Scaling P ocedu es. Issues
and Applica ions. Sage; 2003. doi:10.4135/9781412985772.
24. Fa in E, Schmid E, G amm L. Pa ien communica ion com-
pe ence: De elopmen o a Ge man ques ionnai e and co e-
la es o compe en pa ien beha io . Pa ien Educ Counsel.
2014;94(3):342-350. doi:10.1016/j.pec.2013.11.005.
25. Denise F. Poli . S a is ics and Da a Analysis o Nu sing
Resea ch. 2nd ed.. Uppe Saddle Ri e : Pea son; 2010.
26. Wang J, He e z A, Libe man G. La aplicación del modelo de
ecuación es uc u al en las in es igaciones educa i as. Cul u a
y Educacion. 2017;29(3):563-618. doi:10.1080/11356405.
2017.1367907.
27. Ben le PM. Compa a i e fi indexes in s uc u al models.
Psychol Bull. 1990;107(2):238-246. doi:10.1037/0033-2909.
107.2.238.
28. Tucke LR, Lewis C. A eliabili y coe ficien o maximum
likelihood ac o analysis. Psychome ika. 1973;38(1):1-10.
doi:10.1007/BF02291170.
29. Hu L, Ben le PM. Cu o c i e ia o fi indexes in co a iance
s uc u e analysis: Con en ional c i e ia e sus new al e na-
i es. S uc Equ Model: A Mul idisciplina y Jou nal. 1999;
6(1):1-55. doi:10.1080/10705519909540118.
30. S¨
o bom D. Model modifica ion. Psychome ika. 1989;54(3):
371-384. doi:10.1007/BF02294623.
31. J¨
o eskog KG, Olsson UH, Wallen in F Y. Mul i a ia e Analysis
wi h LISREL. Cham: Sp inge In e na ional Publishing; 2016.
doi:10.1007/978-3-319-33153-9.
32. MacCallum RC, Roznowski M, Necowi z LB. Model modi-
fica ions in co a iance s uc u e analysis: The p oblem o
capi aliza ion on chance. Psychol Bull. 1992;111(3):490-504.
doi:10.1037/0033-2909.111.3.490.
33. DeVellis RF. Scale De elopmen : Theo y and Applica ions.
SAGE Publica ions; 2016. h ps://books.google.es/books?id=
48ACCwAAQBAJ.
34. Jaju A, C ask MR, Menon A, Sha ma A. The Pe ec Design:
Op imiza ion Be ween Reliabili y, Validi y, Redundancy in
Scale I ems and Response Ra es. In: Win e Educa o s
Con e ence. Vol. 10. Ame ican Ma ke ing Associa ion; 1999:
127-131.
35. Wang J, He e z A, Libe man G. Applying s uc u al equa ion
modelling in educa ional esea ch. Cul u a y Educacion. 2017;
29:563-618. doi:10.1080/11356405.2017.1367907.
36. Heli ze D, Hollis C, Co ne J, Oes eiche N. Heal h li e acy
demands o w i en heal h in o ma ion ma e ials: An assess-
men o ce ical cance p e en ion ma e ials. Cance Con ol.
2009;16(1):70-78. doi:10.1177/107327480901600111.
Cazo la-Calde ón e al. 9