ORIGINAL PAPER
Jou nal o P e en ion
h ps://doi.o g/10.1007/s10935-022-00701-3
Abs ac
S udies ocusing on he ele ance o impac o a p og am, no jus on i s e icacy o
e ec i eness, can make impo an con ibu ions o p e en ion science. This a icle
documen s he social alidi y (i.e., a cons uc encompassing easibili y, accep abili-
y, and u ili y) o a uni e sal subs ance use p e en ion p og am o ea ly adolescen s
in Spain. The Man en e REAL (keepin’i REAL) p og am was cul u ally adap ed
o he Spanish con ex , implemen ed, and e alua ed in six public middle schools
in wo egions o Spain. Pa icipa ing eache s (N = 15), s uden s (N = 354), and e-
sea ch eam membe s (N = 6) epo ed on he easibili y, accep abili y, and u ili y
o he p og am implemen ed wi h i s g ade seconda y school s uden s. Quali a i e
and quan i a i e da a abou he p og am, i s cu iculum, and he implemen a ion
p ocess we e collec ed h ough eache s’ ocus g oups, s uden s’ su eys, and ob-
se a ion o ms comple ed by membe s o he esea ch eam. Man en e REAL was
pe cei ed o be a p e en ion p og am ha was easible o implemen a ion in Span-
ish middle schools, al hough some logis ics ela ed o school s uc u al cons ain s
should be add essed in u u e implemen a ions. The opics and ac i i ies in he
cu iculum we e highly accep ed by eache s and s uden s, and hey epo ed ha
he p og am was use ul in eaching esis ance s a egies o cope wi h subs ance use
and o he isky si ua ions. The indings suppo he social alidi y o he cul u ally
adap ed Man en e REAL p og am o ea ly adolescen s in Spain, and highligh how
eedback om s akeholde s in ol ed in he implemen a ion can imp o e he dis-
semina ion o e ec i e p e en ion app oaches.
Keywo ds Subs ance use · Adolescence · Resis ance s a egies · P e en ion
p og am · Social alidi y · Spain
Accep ed: 21 July 2022
© The Au ho (s) 2022
Social Validi y in Spain o he Man en e REAL P e en ion
P og am o Ea ly Adolescen s
Social Validi y o Man en e Real in Spain
OlallaCu ín1· I. MacFadden2· F. F.Ma siglia3,4· S. S.Kulis3,5
Ex ended au ho in o ma ion a ailable on he las page o he a icle
1 3
Jou nal o P e en ion
The aim o designing and e alua ing e idence-based p e en ion p og ams is o assu e
ha you h ha e access o quali y, e icien and e ec i e in e en ions ha can p e en
o delay hei use o alcohol and o he d ugs and inc ease he wellbeing and o e -
all heal h o communi ies (Eu opean Moni o ing Cen e o D ugs and D ug Addic-
ion, EMCDDA, 2011; Na ional Ins i u e on D ug Abuse, 2021). The high subs ance
use a es among Eu opean you h, and mo e speci ically Spanish you h, highligh he
impo ance o implemen ing science-based and e icacious p e en ion p og ams.
The Na ional EDADES 2019/2020 S udy epo s high a es o d ug use and abuse
in Spain among adolescen s. Fo example, 15.4% o all esponden s epo ed binge
d inking in he las 30 days, 63% o hem epo ed d inking alcohol, and one ou o
h ee adolescen s epo ed ge ing d unk in he las 30 days (Obse a o io Español de
las D ogas y las Adicciones, 2021).
The EMCDDA (2011) es ablished quali y s anda ds o gua an ee e ec i e and
e icacious d ug abuse p e en ion. The p inciples encompass he en i e p e en ion
p ocess, om assessmen o needs and esou ces o he inal e alua ion and dis-
semina ion o e icacious in e en ions (Ba h e al., 2012). Dissemina ing cul u ally
adap ed p og ams, igo ously e alua ed in a con olled se ing and wi h demons a ed
e ec i eness, inc eases he p obabili y o cong uence be ween he key componen s o
a p og am, i s expec ed ou comes and he p obabili ies ha p og ams will be adop ed
and implemen ed (Ba e a e al., 2017). The implemen a ion o hose s anda ds is,
howe e , une en ac oss Eu opean Union membe coun ies (EMCDDA, 2021).
In Spain, 30 school p e en ion p og ams a e lis ed in he e idence-based p e en-
ion da abase (Sociedad Cien í ica Española de Es udios sob e el Alcohol, el Alco-
holismo y las o as Toxicomanías, 2021). The e idence conside ed by he da abase
ocuses on e icacy. Howe e , p e ious esea ch has indica ed ha uni e sal school-
based d ug p e en ion in e en ions usually show low le els o e icacy (Espada e
al., 2015; Tanne -Smi h e al., 2018). The e o e, o he indica o s, such as social alid-
i y, should be conside ed o e alua e uni e sal p e en ion p og ams. An e idence-
based in e en ion iden i ied as e icacious will ha e a g ea e p e en ion impac i
i is pe cei ed by he end use s as accep able, app op ia e, easible and sus ainable
(Mu a e al., 2021). These cons uc s ha e been commonly used as indica o s o
implemen a ion ou comes ha help measu e implemen a ion success and a e key
in e media e ou comes ha acili a e in e en ion e ec i eness (P oc o e al., 2011).
Mo eo e , in o ma ion abou he p ocess o implemen a ion and he adequacy o he
cu iculum is essen ial no only o a success ul implemen a ion, bu o long- e m
sus ainabili y (Ha hun e al., 2009; Soneson e al., 2020). To ad ance knowledge on
hese issues, he ield o p e en ion is ocusing on he social alidi y, ele ance o
impac o he p og am, in addi ion o i s e icacy o e ec i eness.
The concep ual axonomy o P oc o e al., (2011) p o ides a heo e ical back-
g ound o implemen a ion esea ch and a guide o a me hodological app oach,
add essing a sho coming in concep ual cla i y based on he psychome ic p ope -
ies o measu es. P e ious models indica ed he ele ance o h ee a iables in he
assessmen o implemen a ion ou comes: easibili y, accep abili y, and app op ia e-
ness (Weine e al., 2017). The la e wo a e highly o e lapping concep s and ecen
esea ch assessing he ac ual unc ioning o p e en ion p og ams in eal se ings
1 3
Jou nal o P e en ion
poin s o he impac o he p og am as a a iable o be conside ed as an ou come
indica o o implemen a ion (Mu a e al., 2021).
Acco ding o Mu a e al., (2021) “social alidi y s udies a e mean o analyze
he social ele ance o public heal h in e en ion goals, he app op ia eness o hei
s a egies, and he social signi icance o hei impac , as pe cei ed by he a ge audi-
ence” (p. 660). F om a social alidi y pe spec i e, a dynamic and cyclical p ocess
o assessing implemen a ion allows in e en ion esea ch o ede ine and op imize
p og ams and, subsequen ly, augmen he e ec i eness o he in e en ion in eal se -
ings and speci ic con ex s. In his p ocess, he in o ma ion p o ided by bo h he pa -
icipan s and he acili a o s o he in e en ion is use ul o e alua e he signi icance
o he p og am (Mu a e al., 2021).
Conside ing he p e ious heo e ical, me hodological, and social models, as well
as cul u al esponsi eness, his s udy in es iga ed social alidi y by assessing whe he
a pa icula p e en ion p og am is (1) easible o be implemen ed (in he speci ic con-
ex wi h he a ge popula ion and he a ailable esou ces), (2) accep able ega ding
con en s and ac i i ies, and unde s andable, and/o (3) use ul o gain knowledge and
impac pa icipan s’ li es.
Keepin’ i REALin Spain: Man en e REAL
keepin’ i REAL, a p og am o iginally designed o a mul icul u al popula ion o mid-
dle school s uden s in he US, was demons a ed e icacious in mul iple s udies in he
Uni ed S a es (Hech e al., 2003; Kulis e al., 2007; Ma siglia e al., 2011). The o igi-
nal e sion o keepin’ i REAL (Ma siglia & Hech , 2005), designed o be deli e ed
by egula class oom eache s in middle schools, has been cul u ally adap ed in e na-
ionally and has shown o be e ec i e in di e en social and cul u al con ex s, includ-
ing in Mexico, Gua emala, and U uguay (Kulis e al., 2019; Kulis, Ga cia-Pé ez e
al., 2021; Kulis, Ma siglia e al., 2021; Ma siglia e al., 2018). A ecen ial o he
mo e ecen DARE e sion o keepin’ i REAL in B azil epo ed mo e mixed e i-
dence, bu his e sion was deli e ed by police o ice s o elemen a y school s uden s
younge han he a ge popula ion o he o iginal e sion (Valen e & Sanchez, 2022).
The in e en ion eaches li e skills and ou speci ic s a egies o manage di e en
isky si ua ions, summa ized in he ac onym o REAL: Re use by saying no e bally
o non- e bally; Explain why you don’ wan o use subs ances; A oid si ua ions
whe e subs ances a e consumed o a ailable; and Lea e si ua ions whe e subs ances
a e being used. Pa icipan s lea n he REAL s a egies as a ool o empowe men and
o he li e skills o assess isks associa ed wi h subs ance use and o make decisions
abou i . The in e en ion alida es he you h’s belie s and a i udes ha con ibu e
o hei posi i e de elopmen . Because keepin’ i REAL is designed o in eg a e he
sociocul u al alues o he pa icipa ing adolescen s, he manuals and accompany-
ing ideos ha e been cul u ally adap ed when he p og am has been implemen ed in
coun ies ou side he Uni ed S a es (Ma siglia e al., 2019).
Man en e REAL is he Spanish language e sion o his uni e sal p e en ion
p og am o seconda y school adolescen s, consis ing o 12 sessions and 5 ideos
p oduced in each home coun y. Al hough he collabo a ion be ween a bina ional
1 3
Jou nal o P e en ion
Spanish-US eam da es back o 2005 (Luengo e al., 2008), keepin’ i REAL was cul-
u ally adap ed in Spain be ween 2015 and 2019 h ough h ee main phases by unded
s udies. Fi s , a pilo s udy in Se ille was conduc ed (2015–2017) o es he easibil-
i y and accep abili y o he Mexico-based Man en e REAL e sion in Spain (Ma -
siglia e al., 2019). This e sion was selec ed o be implemen ed in Spain because
i was he mos ecen ly p oduced Spanish language e sion o he Man en e REAL
cu iculum. This pilo also aimed o es he po en ial e ec sizes o he p og am o
he p e en ion o d ug use among adolescen s in he Spanish con ex and o iden i y
he socio-cul u al adap a ion needs. One o he main indings o he s udy poin ed o
he need o cul u ally adap he ideos and he ac i i ies (e.g., he examples p o ided,
he cha ac e s p esen ed, and he ype o Spanish e nacula used).
Phase wo consis ed o a cul u al adap a ion s udy conduc ed in Se ille (2017–
2018). S uden s and eache s om wo middle public schools in Se ille pa icipa ed
in he adap a ion p ocess. The s udy ocused on he adap a ion o he ma e ials o
he p og am (2 manuals and 5 ideos) o he Spanish con ex . The adap a ion o he
con en o he ideos in ol ed 32 s uden s be ween 11 and 12 yea s old pa icipa ing
in ou ocus g oups, who wo ked on sc ip de elopmen o he i e ideos. Mo e
han 60 s uden s om he same wo schools pa icipa ed in p oducing he ideos, wi h
he suppo and echnical assis ance o a p o essional ideog aphe . Based on he
eedback ecei ed om eache s, s uden s and obse e s, esea che s om he bina-
ional eam and 5 ex e nal expe s e ised he manuals. Once he cul u ally adap ed
cu iculum was inished and e ised, a new s udy was conduc ed o assess i s e icacy.
Phase h ee consis ed o a small Randomized Con ol T ial (RCT) s udy (2018–
2019) o es he e icacy o he adap ed e sion o he p og am in he wo si es
in ol ed in he adap a ion p ocess, Se ille and San iago de Compos ela. The s udy
was conduc ed in 12 public seconda y schools o medium socio-economic le el (6
om each si e). The o al sample included 755 i s g ade seconda y school s uden s
(equi alen o 7 h g ade in he US). The esul s sugges he p og am was e icacious
in cu bing he inc ease in excessi e alcohol consump ion and in oxica ion episodes
among s uden s who ecei ed he Man en e REAL in e en ion, ela i e o ea men -
as-usual con ols (Cu ín e al., 2020).
The cu en s udy
A e inding p elimina y e idence o i s e icacy in he small RCT s udy, he p esen
s udy assessed he social alidi y o Man en e REAL in he Spanish con ex , examin-
ing ac o s ha con ibu e o he quali y, impac and sus ainabili y o his e idence-
based and cul u ally adap ed p e en ion in e en ion. This s udy aimed o analyze
he easibili y, accep abili y, and u ili y o he Man en e REAL p og am in he i s
g ade o Seconda y Compulso y Educa ion in Spain, equi alen o 7 h g ade in he
Uni ed S a es. Because Man en e REAL was de eloped by you h and o you h, we
hypo hesized, om a social alidi y pe spec i e, ha he cul u ally adap ed e sion
o he in e en ion o Spain would be e alua ed as easible, accep able and use ul by
he pa icipan s and acili a o s. To p o ide he pe spec i es o di e en in o man s
in ol ed in he implemen a ion o Man en e REAL, om he wo di e en egions,
1 3
Jou nal o P e en ion
we analyzed da a e alua ing he in e en ion om he pe spec i e o pa icipa ing
s uden s, eache s (i.e., p og am acili a o s), and he esea ch eams’ obse a ions o
ideli y du ing he implemen a ion p ocess.
Me hod
Pa icipan s
The sample o his s udy came om he small clus e andomized ial es ing he e i-
cacy o he cul u ally adap ed Man en e REAL in Spain. The 12 selec ed s a e schools
om Se ille and San iago de Compos ela we e andomized in o wo condi ions:
Man en e REAL (schools implemen ing he p og am) o con ol ( ea men -as-usual
schools). Schools in bo h si es, Se ille and San iago, we e assigned o con ol and
in e en ion condi ions (3 schools pe si e and pe condi ion). All s uden s en olled
in he i s g ade o he selec ed schools we e in i ed o pa icipa e and 80.1% o ha
a ge popula ion pa icipa ed in he s udy (see Cu ín e al., 2020, o sampling and
implemen a ion de ails). Fo he p esen s udy, he analysis is limi ed o he Man en e
REAL condi ion, conside ing only s uden s a ending schools assigned o he expe i-
men al g oup (N = 354; 6 schools) and eache s implemen ing he p og am (N = 15),
as well as membe s o he ield esea ch eams se ing as obse e s (N = 6).
Pa icipa ing s uden s we e en olled in he ini ial g ade o seconda y school
(47.3% emale; aged 11 o 15, Mage = 12.3; SD = 0.59). The eache s implemen ing
Man en e REAL we e 53.3% emale, aged 24 o 50 (Mage = 38.64; SD = 7.41), and
had an a e age o 12 yea s eaching expe ience. Membe s on he esea ch eams
om he wo pa icipa ing uni e si ies included he ield coo dina o s o he p ojec
and g adua e s uden s who pe o med oles as liaisons o he schools and obse e s
o implemen a ion ideli y.
P ocedu e
Desc ip ion o he in e en ion
In he schools assigned o he in e en ion g oup, he p og am was implemen ed
by he egula u o s (home oom eache s) in hei class ooms du ing school hou s.
Be o e s a ing he cu iculum’s lessons, all 1s g ade (7 h g ade in he US) u o s
ecei ed p ac ical aining o e a wo-day long session o lea n how o deli e he
Man en e REAL cu iculum wi h ideli y. P og am implemen a ion was s uc u ed
and guided by a Teache ’s Manual and a S uden ’s Wo kbook, which de ail all he
p ocedu es o eache s o ollow and he ac i i ies o be comple ed by s uden s. Man-
en e REAL has a o al o 12 lessons and eache s we e gene ally able o deli e one
lesson pe week. Teache s began deli e y o he cu iculum lessons in Janua y o
2019 and comple ed i s implemen a ion in Ap il o he same yea .
1 3
Jou nal o P e en ion
Da a collec ion p ocedu es
E hical esea ch e iew commi ees o he Spain and US esea ch eams app o ed
he s udy’s esea ch design and all quali a i e and quan i a i e da a collec ion p oce-
du es. In o ma ion abou easibili y, accep abili y, and u ili y was collec ed h ough
ocus g oups wi h eache s, s uden s’ su eys, and he esea ch eam’s obse a ions
o ideli y du ing he implemen a ion p ocess.
Teache s ocus g oups. All eache s (N = 15) om he six schools ha imple-
men ed Man en e REAL we e in i ed o pa icipa e in ocus g oups abou a mon h
a e inishing he p og am implemen a ion. All o hem accep ed he in i a ion and
ga e hei assen o pa icipa e in abou a one-hou ocus g oup. In each si e, ocus
g oups we e conduc ed wi h eache s in each school (6 ocus g oups; a ound 2–4
eache s pe school) o ge in o ma ion abou he implemen a ion p ocess and opin-
ions abou he Man en e REAL cu iculum. All he ocus g oups we e eco ded and
ansc ibed in Spanish (in San iago de Compos ela some o he eache s used he
Galician language, bu hei commen s we e ansc ibed in o Spanish).
S uden pos - es su eys. S uden s (N = 354) en olled in he ini ial g ade le el o
he six seconda y schools assigned o he in e en ion condi ion comple ed a pos -
es su ey abou a mon h a e inishing he p og am implemen a ion. Pa en s p o-
ided in o med consen o hei child o pa icipa e in he su ey, and s uden s ga e
hei assen be o e comple ing he su ey. Quali ied membe s o he esea ch eams
in o med s uden s ha comple ing he su ey was olun a y and answe s would
emain con iden ial, and p o ided p ope ins uc ions o answe he sel - epo ed
su ey in a 50-minu e class oom session. Ques ions assessing di e en aspec s o
Man en e REAL and s uden s’ opinions abou he p og am we e included a he end
o he pos - es su ey.
Fideli y obse a ions. Quali ied membe s o he esea ch eams isi ed he class-
ooms o each implemen ing eache a sessions 1, 3, and 7 o a e ideli y o he
manual, class oom managemen , and s uden pa icipa ion. The obse e s comple ed
s anda dized o ms wi h closed-ended a ings o a ious aspec s o ideli y and pa -
icipan s’ in ol emen , and open-ended commen s abou implemen a ion issues and
eedback p o ided o eache s.
Measu emen s
Feasibili y
The deg ee o which he p og am was iable o be implemen ed was explo ed
h ough he ollowing measu es ha di ec ly a ec p og am implemen a ion (Bi d e
al., 2014; Lohan e al., 2018; Soneson e al., 2020).
Quali a i e measu es om eache s’ ocus g oups. (1) In e en ion i : pe cep-
ion o he p og am as ele an o add ess he p io i ized goals (subs ance use) and
app op ia e o he a ge popula ion (1s g ade seconda y school s uden s). (2) Imple-
men a ion cha ac e is ics: complexi y o he deli e y o he cu iculum, help ulness
o he manual, and ime adequacy (o ime conce ns) o implemen he p og am. (3)
P ac icali y: engagemen in deli e y by eache s and pa icipa ion by s uden s.
1 3
Jou nal o P e en ion
Quan i a i e measu es om s uden s’ pos - es su eys. 1) P ac icali y: s u-
den s epo ed on he pos - es su ey how many ideos hey wa ched (0–5) and how
much hey pa icipa ed in he p og am (0 = no a all, 1 = a li le, 2 = some, 3 = a lo ).
Quan i a i e measu es om esea che s’ obse a ion o ms. (1) P ac icali y:
obse e s a ed h ee closed-ended i ems o assess s uden in ol emen wi h he cu -
iculum (i.e., pa icipa ion, engagemen , and a en ion o he ideos). (2) Fideli y:
obse e s in o med abou ideli y using mul iple measu es. One was whe he he
eache comple ed each o 8 o 10 cu iculum i ems ha we e speci ic o he obse ed
lesson (e.g., in oduced a pa icula opic, comple ed an ac i i y in he s uden man-
ual). Two o he h ee sessions obse ed ha e an 8-i em cu iculum checklis ; he
o he session has a 10-i em checklis . These we e combined in o an index coun ing
he numbe comple ed, om 0 o 10. Addi ionally, single i ems assessed how well
he eache was p epa ed o he lesson, in o med abou i s con en , ollowed he
lesson plan, ga e clea ins uc ions, and mo i a ed s uden s o pa icipa e (1 = no
a all, 2 = somewha , 3 = mos ly, 4 = comple ely). Ano he wo i ems a ed he each-
e ’s class managemen o he g oup p ocesses in he cu iculum (1 = poo , 2 = well,
3 = excellen ) and whe he he pace o ins uc ion was app op ia e (1 = oo as , 2 = oo
slow, 3 = app op ia e). One mo e i em assessed whe he he eache added con en no
included in he manual (0 = no, 1 = yes).
Accep abili y
The deg ee o which he pa icipan s accep ed he p og am was explo ed h ough
he ollowing measu es (Lohan e al., 2018) ha assess he pe cei ed app op ia e-
ness, ai ness, and easonableness o an in e en ion o add essing an iden i ied need
(Gadke e al., 2021; Nas asi & T usco , 2000).
Quali a i e measu es om eache s’ ocus g oups. (1) Sa is ac ion: o e all sa -
is ac ion wi h he p og am (e.g., enjoyed implemen ing i , liked he p og am, liked
he expe ience). (2) Com o wi h opics and ac i i ies: g ade in which pa icipan s
and/o acili a o s liked opics and ac i i ies in he cu iculum (e.g., commen s abou
how much hey liked he cu iculum, wha opics/ac i i ies hey liked mo e o less).
(3) Unde s anding o con en : g ade in which pa icipan s and/o acili a o s unde -
s ood (o ha e p oblems unde s anding) concep s and ac i i ies in he cu iculum
(e.g., commen s abou p oblems unde s anding concep s o ac i i ies ins uc ions).
(4) Willingness o use he in e en ion in he u u e: exp essed willingness o con inue
implemen ing he in e en ion in schools.
Quan i a i e measu es om s uden s’ pos - es su eys. (1) Sa is ac ion: s u-
den s esponded o ou i ems on he pos - es su ey asking abou how much hey
liked Man en e REAL, including i s a ious componen s ( ideos, homewo k, class-
oom ac i i ies) and he p og am o e all (0 = did no like i a all, 1 = did no like i
much, 2 = liked i , 3 = liked i a lo ). (2) Com o wi h opics and ac i i ies: s uden s
epo ed in ou pos - es ques ions whe he he p og am was in e es ing, un, easy o
pay a en ion o, o bo ing (0 = s ongly disag ee, 1 = disag ee, 2 = ag ee, 3 = s ongly
ag ee).
1 3
Jou nal o P e en ion
U ili y
The deg ee o which he p og am is pe cei ed as use ul in p oducing indi idual and
social change (Mu a e al., 2021) was explo ed h ough he ollowing measu es.
Quali a i e measu es om eache s’ ocus g oups. (1) Knowledge: in o ma ion
acqui ed and pe cep ion ha con en s a e use ul. (2) Applicabili y: simila i y wi h
pa icipan s’ li e; au hen ici y o con en . (3) Impac : changes in he li es o pa ici-
pan s; changes in he wo k o acili a o s and ou ine o se ices.
Quan i a i e measu es om s uden s’ pos - es su eys. (1) Knowledge: s u-
den s epo ed on he pos - es su ey whe he he p og am ga e hem use ul in o -
ma ion (0 = s ongly disag ee o 3 = s ongly ag ee), and how much hey lea ned om
i (0 = no a all, 1 = a li le, 2 = some, 3 = a lo ). (2) Applicabili y: s uden s assessed
he au hen ici y o con en in h ee pos - es i ems, whe he i was “like my li e”,
“like you hs I know”, and like si ua ions ha hey know o he s uden s ge in o. Two
addi ional i ems assessed whe he si ua ions in he cu iculum and cha ac e s in he
ideo appea ed “ eal” (0 = s ongly disag ee o 3 = s ongly ag ee). (3) Impac : s u-
den s epo ed whe he hey alked abou he p og am wi h a ious people in hei
social ne wo k: pa en s, siblings, cousins, o he amily membe s, iends, and o he s.
These we e assessed sepa a ely (0 = no, 1 = yes) and as a coun o he numbe o di -
e en ca ego ies o people hey alked wi h (0 o 6).
Da a Analysis
Quali a i e analysis was conduc ed ollowing a hema ic and amewo k app oach o
iden i y codes and analyze pa e ns ( hemes) among he eache s’ expe iences wi h,
and belie s abou , he Man en e REAL p og am and o e ec i ely o ganize and in e -
p e he da a. Thema ic analysis was conduc ing ollowing B aun and Cla ke’s (2006)
ecommenda ions: (a) ead and become amilia ized wi h he da a, (b) gene a e ini ial
codes in a sys ema ic me hod, (c) iden i y hemes based on a ailable codes, (d) e iew
codes and hemes o accu a e da a ep esen a ion, (e) de ine and name hemes, and
( ) selec i id examples o illus a e indings. Addi ionally, Gibbs (2018) and Rabiee
(2004) ecommenda ions on hema ic coding and ca ego iza ion o quali a i e da a
(s eps b o d o B aun & Cla ke’s ecommenda ions) we e used as a e e ence du -
ing he da a analysis p ocess o inc ease consis ency in coding and dec ease mis ep-
esen a ion o he da a: e.g., c ea ing codes ha a e analy ical and heo e ical and
no me ely desc ip i e; o ganizing codes hie a chically and compa ing codes (which
may be in e p e ed as a cons an compa ison analysis; Onwuegbuzie e al., 2009); and
in e p e da a conside ing wo d meaning (keywo ds-in-con ex acco ding o Onwueg-
buzie e al., 2009), ocus g oup con ex , and in e nal consis ency h oughou he ocus
g oup. To ensu e ha a consis en amewo k on social alidi y o p e en ion p o-
g ams was e lec ed in he p ocess o coding, membe s o he esea ch eam con-
duc ed a li e a u e e iew o syn hesize exis ing concep s o assessing social alidi y,
which became he basis o he ini ial coding scheme (s ep b o B aun & Cla ke’s
ecommenda ions). The codes we e gene a ed om keywo ds ha de ine easibili y,
accep abili y, and u ili y a e he ag eemen o h ee esea che s based on he li e a-
u e e iewed. Focus g oup da a we e ansc ibed e ba im in Spanish by he local
1 3
Jou nal o P e en ion
eam esponsible o da a collec ion in each si e and wo independen esea che s
u ilized hese Spanish-language ansc ip s o conduc he hema ic analysis.
Quan i a i e analysis was conduc ed wi h SPSS 25 o analyze equencies, o he
desc ip i e s a is ics (means, s anda d de ia ions, ange), and - es s o assess di -
e ences in eache s’ implemen a ion and s uden s’ pe cep ions o Man en e REAL
be ween he wo si es (i.e., Se ille and San iago de Compos ela). Di e ences be ween
si es on he e alua ions o he p og am can be an indica o o he cul u al app op ia e-
ness o he p og am a e he cul u al adap a ion. Because i was adap ed in Se ille,
he p og am migh be iewed as mo e socially alid in ha si e.
Resul s
Quali a i e esul s
A summa y o quali a i e esul s ega ding he eache s’ opinions and expe iences
wi h he Man en e REAL p og am is displayed in Table 1.
Feasibili y
Teache s p o ided o e all posi i e eedback abou Man en e REAL, pe cei ing i o
be a easible p og am o implemen a ion in Spain and o be a well-designed p og am
o add ess subs ance use and pee p essu e a he beginning o adolescence.
“Es impo an ísima la p e ención, en es a edad an pequeña casi ningún p o-
g ama [se aplica] . . pa a mí debe ía se obliga o io” (Teache , Se ille). “Desde un
p incipio nos pa eció muy in e esan e es e p og ama y c eemos que es muy nece-
sa io empeza a a a la p esión de g upo en los p ime os años de la secunda ia”
(Teache , San iago de Compos ela). [“P e en ion is e y impo an , a his e y
young age almos no p og am [is being implemen ed]. . o me i should be manda-
o y” (Teache , Se ille). “F om he beginning we ound his p og am e y in e es ing
and we belie e ha i is e y necessa y o s a dealing wi h pee p essu e in he ea ly
yea s o middle school” (Teache , San iago de Compos ela).]
One o he cha ac e is ics o he p og am ha eache s liked he mos was he
manualiza ion o he in e en ion. I helped inc ease he eache s’ con idence in hei
abili y o deli e e ec i e p og am esul s, and educe conce ns ha acili a o s o
Man en e REAL should ha e p io expe ise in p e en ion. Teache s el com o -
able implemen ing he sessions due o he de ailed guidelines p esen ed in he manu-
als. This con ibu ed o he high pe cei ed p ac icali y o he p og am, as eache s
exp essed hei in ol emen in deli e ing he cu iculum and no ed posi i e s uden
pa icipa ion.
“Es uc u ado, eso es undamen al, eso es lo que más me ha gus ado el hecho que
odo es u ie a an es uc u ado”, “Yo he es ado a gus o y ambién he enido espues a
po pa e del alumnado” (Teache s, Se ille). “Es á odo muy explicado en los manu-
ales y eso es una ayuda cuando no sabes del ema”, “Mos a on mucho in e és en
las a eas y ac i idades que se lle a on a cabo” (Teache s, San iago de Compos ela).
[“I is a s uc u ed p og am, ha is undamen al, ha is wha I liked he mos , he ac
1 3
Jou nal o P e en ion
ing i s quali y and social impac . The indings o e all cap u ed posi i e pe cep ions
o eache s, s uden s, and obse e esea che s abou he Man en e REAL p og am.
They iewed i as a easible, accep able, and use ul p og am o p e en subs ance
use in ea ly adolescen s in Spain. Conside ing cul u ally app op ia eness, di e ences
be ween si es indica ed ha he p og am is pe cei ed mo e posi i ely by hose pa -
icipan s li ing in he speci ic cul u al con ex in which he cul u al adap a ion o he
cu iculum was conduc ed (i.e., Se ille).
The high le el o subs ance use among Eu opean and Spanish popula ions, and
young adolescen s in pa icula , highligh s he need o in e en ions ocused on ea ly
uni e sal p e en ion. Es ablishing he scien i ic and social alidi y o in e en ions is
needed in o de o change ac ual subs ance using beha io s by eenage s. These ind-
ings suppo he p emise ha a educ ion in isky beha io s among adolescen s can
happen by implemen ing p og ams ha a e no only e idence-based (Delegación del
Gobie no pa a el Plan Nacional sob e D ogas, 2019) and cul u ally adap ed (Holle an
e al., 2002), bu ha a e also socially alida ed.
Following his line o a gumen , i is essen ial o conside he deg ee o e ec i e
applica ion and he ecogni ion o he di e en ac o s in ol ed in he in e en ion. In
his s udy, social alidi y was measu ed wi h he le els o easibili y, accep abili y,
and u ili y by he wo main s akeholde s in he in e en ion p ocess: eache s and
s uden s, as well as obse e ield esea che s. In o ma ion p o ided by he ac o s
in ol ed in he in e en ion o his s udy is consis en wi h he widely a i med iew
ha pa icipa ion in he en i e p ocess o he design, cul u al adap a ion, implemen a-
ion and e alua ion o p e en ion in e en ions inc eases hei social alidi y (Gosin
e al., 2003).
One o he ou s anding p ocesses ha gua an ee he success o a p e en ion p o-
g am h ough he inclusion o he cul u e o he a ge popula ion is he co-design
o he con en o he in e en ion ma e ials, as shown by he highe posi i e eed-
back p o ided by s uden s in Se ille, he ci y in which he ma e ials and ideos we e
adap ed and eco ded. Cul u ally ailo ed p og ams a e based on he p emise ha
including you h cul u e in he con en and o ma o he p e en ion message will
educe adolescen d ug use (Kandel, 1995; Holle an e al., 2002). Indeed, he ind-
ings suppo he concep ha adap a ion models based on high le el o communi y
in ol emen and pa icipa ion p oduce s ong p e en ion in e en ions (Cas o e al.,
2004, 2010; Wingood & DiClemen e, 2008).
In ou s udy, elemen s ha s and ou as con ibu ing o he easibili y o he p o-
g am include key s a egies o consis en ly p o ec he co e elemen s o he cu icu-
lum while adap ing and ansmi ing cul u ally adap ed messages in he mos e ec i e
way. Included among hese a e he de ailed manualiza ion o he cu iculum and he
ex ensi e aining o egula class oom eache s (Ha hun e al., 2009). The aining
o eache s and he manualiza ion o he p og am o e a aluable oppo uni y o
ep oduce i s con en s ai h ully and implemen he p og am wi h ideli y.
This s udy, in compa ison o o he s (e.g., Mu a e al., 2021), addi ionally included
he ex e nal pe spec i e o esea che s by comple ing s anda dized obse a ion o ms.
O he s udies collec ed ield no es om esea che s (Lohan e al., 2018), bu did no
p esen speci ic indings in his ega d. Resea che s’ iews on easibili y ( ideli y and
p ac icali y-s uden s’ pa icipa ion) help alida e he pe sonal pe spec i e o each-
1 3
Jou nal o P e en ion
e s/ acili a o s and s uden s and p o ide ano he aluable esou ce o in o ma ion o
assess p og am implemen a ion.
On he o he hand, eache s iden i ied challenges o easibili y, such as he need
o adjus o school s uc u al cons ain s, such as hei sugges ions o inc ease he
numbe o sessions and/o hei du a ion, and o e he cu iculum in smalle g oups
o s uden s. The issue o class size may ela e back o he o iginal design o he cu -
iculum o middle school class ooms in he Uni ed S a es ha o en ha e 25 o ewe
s uden s. Wi h la ge class sizes, al e na i es migh be needed when implemen ing
highly in e ac i e class oom ac i i ies like ole-playings.
A key elemen o p og am accep abili y was sa is ac ion wi h i s con en and o -
ma . Teache s and s uden s s ongly endo sed Man en e REAL’s highly p ac ical and
pa icipa o y ac i i ies and ideos, in line wi h he iew ha p og ams based on pa -
icipa o y lea ning a e mo e widely accep ed (Gosin e al., 2003). The combina ion o
o al, w i en, and audio isual suppo ing ma e ials acili a ed accep abili y o con en
and ac i i ies and i inc eased he po en ial e ec i eness o he p og am (Espada e
al., 2015).
The abili y o a p og am o p omo e indi idual and social changes in daily li e is a
key elemen o p og am u ili y. Man en e REAL impac ed s uden s’ li es, no only in
es aining consump ion o alcohol and o he d ugs, bu mo e gene ally by impa ing
skills o na iga ing li e challenges (Cu ín e al., 2021). S uden s and eache s ecog-
nized ha hese skills we e made accessible h ough e ec i e cu iculum and ideo
messaging, and he pa icipa o y eaching and lea ning s yle. F om he eco-e olu i e
pe spec i e (Ma siglia & Holle an, 1999), a con ex ualized p e en i e in e en ion
ha de elops asse i e esponses and esilience in adolescen s in hei own ela ional
en i onmen s is essen ial o imp o ing he lea ning o social and beha io al skills.
In his ega d, a key con ibu ion o he social alidi y o Man en e REAL was ha
egula eache s we e able o implemen he in e en ion wi h ideli y, oo ed in he
es ablished social con ex s o hei school class ooms (Ho ne e al., 2005).
Finally, i is ele an o no e ha a ound 60% o s uden s epo ed “i was no
a all like my li e” and a ound 45% epo ed “i was no a all like you hs I know”.
Conside ing he o he posi i e indica o s o easibili y, accep abili y, and u ili y, hese
indings migh be an indica o o ea ly adolescence as he ideal age o p e en isky
si ua ions, as p e ious esea ch indica ed (Go edson & Wilson, 2003). De elop-
men ally, many s uden s did no eel he speci ic si ua ions implica ing subs ance
use o e s and consump ion we e si ua ions ha hey had ye expe ienced. The e o e,
an icipa ing hese si ua ions, eaching esis ance s a egies o subs ance o e s, and
p omo ing socioemo ional skills o ace isky si ua ions, can e ec i ely p e en la e
adolescen in ol emen in subs ance use and/o o he unheal hy beha io s (Sancas-
siani e al., 2015).
Limi a ions
The s udy’s assessmen s o he social alidi y o Man en e REAL we e limi ed by
aspec s o he esea ch design, a ailable da a sou ces, and challenges in measu ing
complex concep s. Fi s , he esul s e lec he iews o eache s and s uden s om a
1 3
Jou nal o P e en ion
con enience sample o schools om wo ci ies in di e en egions o Spain. The e is
a need o u u e op imiza ion esea ch ha examines he social alidi y o Man en e
REAL in di e en pa s o Spain. Second, a ailable in o ma ion on social alidi y
was es ic ed o sho - e m assessmen s, which make i di icul o es ablish whe he
e ec s ela ing o he u ili y and e ec i eness o he p og am will pe sis o e ime.
Thi d, concep ually complex aspec s o social alidi y, such as accep abili y and
applicabili y, a e di icul o gauge eliably and may ha e wide a ia ions ac oss sub-
popula ions and di e en social se ings. Fo example, di e ing depending on social
dynamics in pa icula class ooms and wi h di e en eache - acili a o s. Finally,
esul s showing di e ences be ween he wo ci ies in social alidi y assessmen s sug-
ges ha e lec ions o pa icula social con ex s in he cul u al p og am adap a ion
–such as he se ing o he ideos in Se ille and he use o language iden i ied wi h a
pa icula egion– may colo hese assessmen s.
Conclusions
The high p e alence o subs ance use and abuse in Spain, among you hs and adul s,
highligh s he need o dissemina e e ec i e p e en ion app oaches and conduc sys-
ema ic esea ch o imp o e he quali y, each, and social alidi y o p e en ion p o-
g ams. Teache s and s uden s p o ided s ong and no ably consis en e idence ha
he cul u ally adap ed e sion o Man en e REAL o Spain (shown as e icacious in
a small RCT) was iewed as: easible o implemen a ion, al hough wi h challenges
ela ed o school s uc u al cons ain s; accep able in e ms o sa is ying and com o -
able con en ; and o u ili y in p omp ing desi ed indi idual change.
Based on hese indings demons a ing he social alidi y o Man en e REAL, we
ecommend ha ce ain elemen s need o be inco po a ed in o he design and es ing
o p e en ion p og ams: (1) sys ema ic da a collec ion o es ablish an e idence base,
(2) igo ous cul u al adap a ion h ough a p ocess o co-design wi h he di e en
ac o s in ol ed in he in e en ion, (3) an unde s anding o he con ex and cul u al
speci ici y on he pa o he p og am acili a o s, (4) app op ia e aining o he acili-
a o s o deli e he p og am wi h ideli y, and (5) engage he ac i e pa icipa ion o
he a ge popula ion. The adap a ion, implemen a ion and es ing o Man en e REAL
in Spain ollowed hese s eps, which p oduced a e y p omising end esul .
Funding This s udy was unded by he Global Cen e o Applied Heal h Resea ch (A izona S a e Uni-
e si y) and suppo ed by he P og ama de Axudas ? e apa posdou o al da Xun a de Galicia (Conselle ?a
de Cul u a, Educaci?n e O denaci?n Uni e si a ia), by FEDER/Minis e io de Ciencia, Inno aci?n y
Uni e sidades ? Agencia Es a al de In es igaci?n (G an PSI2015-65766-R) ?, and by he Axuda pa a a
consolidaci?n e es u u aci?n de unidades de in es igaci?n compe i i as e ou as acci?ns de omen o nas
uni e sidades do SUG (GRC, 2018; Conselle ?a de Cul u a, Educaci?n e O denaci?n Uni e si a ia, Xun a
de Galicia).
Open Access unding p o ided hanks o he CRUE-CSIC ag eemen wi h Sp inge Na u e.
1 3
Jou nal o P e en ion
Decla a ions
Con lic o in e es The au ho s decla e ha hey ha e no con lic o in e es .
Resea ch in ol ing human pa icipan s and/o animals All p ocedu es pe o med in s udies in ol -
ing human pa icipan s we e in acco dance wi h he e hical s anda ds o he ins i u ional and/o na ional
esea ch commi ee and wi h he 1964 Helsinki decla a ion and i s la e amendmen s o compa able e hi-
cal s anda ds. This a icle does no con ain any s udies wi h animals pe o med by any o he au ho s. The
s udy was app o ed by he Bioe hics Commi ee o he A izona S a e Uni e si y and he Uni e sidade de
San iago de Compos ela.
In o med consen In o med consen was ob ained om he pa en s and assen was ob ained om all indi-
idual pa icipan s included in he s udy.
Open Access This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License,
which pe mi s use, sha ing, adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long
as you gi e app op ia e c edi o he o iginal au ho (s) and he sou ce, p o ide a link o he C ea i e
Commons licence, and indica e i changes we e made. The images o o he hi d pa y ma e ial in his
a icle a e included in he a icle’s C ea i e Commons licence, unless indica ed o he wise in a c edi line
o he ma e ial. I ma e ial is no included in he a icle’s C ea i e Commons licence and you in ended use
is no pe mi ed by s a u o y egula ion o exceeds he pe mi ed use, you will need o ob ain pe mission
di ec ly om he copy igh holde . To iew a copy o his licence, isi h p://c ea i ecommons.o g/
licenses/by/4.0/.
Re e ences
Ba e a, M., Be kel, C., & Cas o, F. G. (2017). Di ec ions o he ad ancemen o cul u ally adap ed p e-
en i e in e en ions: local adap a ions, engagemen , and sus ainabili y. P e en ion Science, 18(6),
640–648. h ps://doi.o g/10.1007/s11121-016-0705-9
Ba h, R. P., Lee, B. R., Lindsey, M. A., Collins, K. S., S iede , F., Cho pi a, B. F., Becke , K. D., &
Spa ks, J. A. (2012). E idence-based p ac ice a a c oss oads: he imely eme gence o common
elemen s and common ac o s. Resea ch on Social Wo k P ac ice, 22(1), 108–119. h ps://doi.
o g/10.1177/1049731511408440
Bi d, V. J., Le Bou illie , C., Leamy, M., Williams, J., B ads ee , S., & Slade, M. (2014). E alua ing
he easibili y o complex in e en ions in men al heal h se ices: s anda dized measu e and epo -
ing guidelines. The B i ish Jou nal o Psychia y, 204(4), 316–321. h ps://doi.o g/10.1192/bjp.
bp.113.128314
B aun, V., & Cla ke, V. (2006). Using hema ic analysis in psychology. Quali a i e Resea ch in Psychol-
ogy, 3(2), 77–101. h ps://doi.o g/10.1191/1478088706qp063oa
Cas o, F. G., Ba e a, M., & Ma inez, C. R. (2004). The cul u al adap a ion o p e en ion in e en ions:
esol ing ensions be ween ideli y and i . P e en ion Science, 5(1), 41–45. h ps://doi.o g/10.1023/
B:PREV.0000013980.12412.cd
Cas o, F. G., Ba e a, M., & Holle an, L. K. (2010). Issues and challenges in he design o cul u ally
adap ed e idence-based in e en ions. Annual Re iew o Clinical Psychology, 6, 213–239. h ps://
doi.o g/10.1146/annu e -clinpsy-033109-132032
Cu ín, O., Kulis, S. S., Manei o, L., MacFadden, I., Na as, M. P., Ala cón, D., Gómez-F aguela, J. A.,
Villalba, C., & Ma siglia, F. F. (2020). E ec i eness o he Man en e REAL p og am o p e en -
ing alcohol use in Spanish adolescen s. Psychosocial In e en ion, 30(3), 113–122. h ps://doi.
o g/10.5093/pi2020a19
Delegación del Gobie no pa a el Plan Nacional sob e D ogas (2019). Adicciones Compo amen ales
In o me 2019. Mad id: Minis e io de Sanidad. Re ie ed om h ps://pnsd.sanidad.gob.es/p o esio-
nales/sis emasIn o macion/home.h m
Espada, J. P., Gonzál ez, M. T., O gilés, M., Llo e , D., & Guillén-Riquelme, A. (2015). Me a-analysis o
he e ec i eness o school subs ance abuse p e en ion p og ams in Spain. Psico hema, 27(1), 5–12.
h ps://doi.o g/10.7334/psico hema2014.106
1 3
Jou nal o P e en ion
Eu opean Moni o ing Cen e o D ugs and D ug Addic ion [EMCDDA] (2011). Eu opean d ug p e-
en ion quali y s anda ds. A manual o p e en ion p o essionals. Luxembou g: Publica ions O ice
o he Eu opean Union. Re ie ed om h ps://www.emcdda.eu opa.eu/publica ions/manuals/
p e en ion-s anda ds_en
Eu opean Moni o ing Cen e o D ugs and D ug Addic ion [EMCDDA] (2021). Eu ope D ug Repo .
T ends and De elopmen . Luxembou g: Publica ions O ice o he Eu opean Union. Re ie ed om
h ps://www.emcdda.eu opa.eu/publica ions/ed / ends-de elopmen s/2021_en
Gadke, D. L., K a ochwill, T. R., & Ge inge , M. (2021). Inco po a ing easibili y p o ocols in in e en-
ion esea ch. Jou nal o School Psychology, 84, 1–18. h ps://doi.o g/10.1016/j.jsp.2020.11.004
Gibbs, G. R. (2018). Analyzing quali a i e da a (2nd ed.). London: SAGE. h ps://doi.
o g/10.4135/9781526441867
Gosin, M., Ma siglia, F. F., & Hech , M. L. (2003). Keepin ’i REAL. A cu iculum ailo ed o he s eng hs
and needs o p e-adolescen s o he Sou hwes . Jou nal o D ug Educa ion, 33(2), 119–142. h ps://
doi.o g/10.2190/DXB9-1V2P-C27J-V69V
Go edson, D. C., & Wilson, D. B. (2003). Cha ac e is ics o e ec i e school-based subs ance abuse
p e en ion. P e en ion Science, 4, 27–38. h ps://doi.o g/10.1023/A:1021782710278
Ha hun, M., Dus man, P., Ree es, L., Ma siglia, F. F., & Hech , M. L. (2009). Using communi y-based
pa icipa o y esea ch o adap Keepin’i REAL: C ea ing a socially, de elopmen al, and academi-
cally app op ia e p e en ion cu iculum o i h g ade s. Jou nal o Alcohol & D ug Educa ion,
53(3), 12–38
Hech , M. L., Ma siglia, F. F., Elek-Fisk, E., Wags a , D. A., Kulis, S., Dus man, P., & Mille -Day, M.
(2003). Cul u ally g ounded subs ance use p e en ion: An e alua ion o he keepin’ REAL cu icu-
lum. P e en ion Science, 4(4), 233–248. h ps://doi.o g/10.1023/A:1026016131401
Holle an, L. K., Ree es, L., Dus man, P., & Ma siglia, F. F. (2002). C ea ing cul u ally g ounded ideos o
subs ance abuse p e en ion. Jou nal o Social Wo k P ac ice in he Addic ions, 2(1), 55–78. h ps://
doi.o g/10.1300/J160 02n01_04
Ho ne , R. H., Ca , E. G., Halle, J., McGee, G., Odom, S., & Wole y, M. (2005). The use o single-subjec
esea ch o iden i y e idence-based p ac ice in special educa ion. Excep ional Child en, 71(2), 165–
179. h ps://doi.o g/10.1177/001440290507100203
Kandel, D. B. (1995). E hnic di e ences in d ug use: Pa e ns and pa adoxes. In G. J. Bo in, S. Schinke,
& M. A. O landi (Eds.), D ug abuse p e en ion wi h mul ie hnic you h (pp. 81–104). Thousand Oaks,
CA: Sage
Kulis, S. S., Ga cia-Pé ez, M. H., Ma siglia, F. F., & Aye s, S. L. (2021). Tes ing a cul u ally adap ed you h
subs ance use p e en ion p og am in a Mexican bo de ci y: Man en e REAL. Subs ance Use and
Misuse, 56(2), 245–257. h ps://doi.o g/10.1080/10826084.2020.1858103
Kulis, S. S., Ma siglia, F. F., Medina-Mo a, M. E., Nuño-Gu ié ez, B. L., Co ona, M. D., & Aye s, S. L.
(2021). Keepin’ i REAL – Man en e REAL in Mexico: A clus e andomized con olled ial o a
cul u ally adap ed subs ance use p e en ion cu iculum o ea ly adolescen s. P e en ion Science,
22(5), 645–657. h ps://doi.o g/10.1007/s11121-021-01217-8
Kulis, S. S., Ma siglia, F. F., Po a, M., A é alo A alos, M. R., & Aye s, S. L. (2019). Tes ing he keepin’ i
REAL subs ance use p e en ion cu iculum among ea ly adolescen s in Gua emala Ci y. P e en ion
Science, 20(4), 532–543. h ps://doi.o g/10.1007/s11121-018-0956-8
Kulis, S. S., Nie i, T. A., Yabiku, S., S omwall, L., & Ma siglia, F. F. (2007). P omo ing educed and dis-
con inued subs ance use among adolescen subs ance use s: E ec i eness o a uni e sal p e en ion
p og am. P e en ion Science, 8(1), 35–49. h ps://doi.o g/10.1007/s11121-006-0052-3
Lohan, M., A en in, Á., Cla ke, M., Cu an, R. M., McDowell, C., Agus, A., McDaid, L., Bonell, C., &
Young, H. (2018). Can eenage men be a ge ed o p e en eenage p egnancy? a easibili y clus e
andomised con olled in e en ion ial in schools. P e en ion Science, 19(18), 1079–1090. h ps://
doi.o g/10.1007/s11121-018-0928-z
Luengo, M. A., Kulis, S., Ma siglia, F. F., Rome o, E., Gómez-F aguela, J. A., Villa , P., & Nie i, T. (2008).
A c oss-na ional s udy o p eadolescen subs ance use: explo ing di e ences be ween you h in Spain
and A izona. Subs ance Use & Misuse, 43, 1571–1593. h ps://doi.o g/10.1080/10826080802241078
Ma siglia, F., & Hech , M. (2005). Keepin’ i REAL. D ug esis ance s a egies: Teache guide. San a
C uz, CA: ETR Associa es
Ma siglia, F. F., & Holle an, L. (1999). I’ e lea ned so much om my mo he : Na a i es om a g oup
o Chicana high school s uden s. Child en & Schools, 21(4), 220–237. h ps://doi.o g/10.1093/
cs/21.4.220
1 3
Jou nal o P e en ion
Ma siglia, F. F., Kulis, S. S., Kiehne, E., Aye s, S. L., Libisch, C. A., & Ba os, L. (2018). Adolescen
subs ance use p e en ion and legaliza ion o ma ijuana in U uguay: A easibili y ial o he keepin’
i REAL p e en ion p og am. Jou nal o Subs ance Use, 23(5), 457–465. h ps://doi.o g/10.1080/14
659891.2017.1358308
Ma siglia, F. F., Kulis, S. S., Yabiku, S. T., Nie i, T. A., & Coleman, E. (2011). When o in e ene: elemen-
a y school, middle school o bo h? E ec s o keepin’ i REAL on subs ance use ajec o ies o Mexi-
can he i age you h. P e en ion Science, 12(1), 48–62. h ps://doi.o g/10.1007/s11121-010-0189-y
Ma siglia, F. F., Medina-Mo a, M. E., Gonzal ez, A., Alde son, G., Ha hun, M. L., Aye s, S. L., Nuño-
Gu ié ez, B. L., Co ona, M. D., Mendoza-Meléndez, M. A., & Kulis, S. S. (2019). A bina ional cul-
u al adap a ion o he keepin’ i REAL subs ance use p e en ion p og am o adolescen s in Mexico.
P e en ion Science, 20(7), 1125–1135. h ps://doi.o g/10.1007/s11121-019-01034-0
Mu a, S. G., de Almeida Nob e-Sando al, L., San os Rocha, V. P., Vilela Mi anda, A. A., Duailibe, D.,
Fa ias, K. A., Lopes de Menezes, D., Abdala, J. C. G., Mendes Gomes, I., M. S., & do Ama al Vinha,
L. G. (2021). Social alidi y o he s eng hening amilies p og am in no heas e n B azil: he oices
o pa en s, adolescen s, and acili a o s. P e en ion Science, 22(5), 658–669. h ps://doi.o g/10.1007/
s11121-020-01173-9
Nas asi, B. K., & T usco , S. D. (2000). Accep abili y esea ch in school psychology: cu en ends and
u u e di ec ions. School Psychology Qua e ly, 15(2), 117–122. h ps://doi.o g/10.1037/h0088781
Na ional Ins i u e on D ug Abuse (2021). NIDA S a egic Plan. Na ional Ins i u es o Heal h. Re ie ed
om h ps://www.d ugabuse.go /abou -nida/s a egic-plan
Obse a o io Español de las D ogas y las Adicciones. (2021). In o me 2021. Alcohol, abaco y d ogas
ilegales en España. Mad id: Minis e io de Sanidad
Onwuegbuzie, A. J., Dickinson, W. B., Leech, N. L., & Zo an, A. G. (2009). A quali a i e amewo k o
collec ing and analyzing da a in ocus g oup esea ch. In e na ional Jou nal o Quali a i e Me hods,
8(3), 1–21. h ps://doi.o g/10.1177/160940690900800301
P oc o , E., Silme e, H., Ragha an, R., Ho mand, P., Aa ons, G., Bunge , A., G i ey, R., & Hensley, M.
(2011). Ou comes o implemen a ion esea ch: concep ual dis inc ions, measu emen challenges,
and esea ch agenda. Adminis a ion and Policy in Men al Heal h and Men al Heal h Se ices
Resea ch, 38(2), 65–76. h ps://doi.o g/10.1007/s10488-010-0319-7
Rabiee, F. (2004). Focus-g oup in e iew and da a analysis. The P oceedings o he Nu i ion Socie y,
63(4), 655–660. h ps://doi.o g/10.1079/pns2004399
Sancassiani, F., Pin us, E., Hol e, A., Paulus, P., Mo o, M. F., Cossu, G., Ange meye , M. C., Ca a, M.
G., & Linde , J. (2015). Enhancing he emo ional and social skills o he you h o p omo e hei
wellbeing and posi i e de elopmen : a sys ema ic e iew o uni e sal school-based andomized con-
olled ials. Clinical P ac ice & Epidemiology in Men al Heal h, 11(Suppl 1), 21–40. h ps://doi.
o g/10.2174/1745017901511010021
Sociedad Cien í ica Española de Es udios sob e el Alcohol, el Alcoholismo y las o as Toxicomanías
(2021). P e ención basada en la e idencia. In e enciones e aluadas. Re ie ed om h p://www.
p e encionbasadaenlae idencia.ne /
Soneson, E., Howa h, E., Fo d, T., Humph ey, A., Jones, P. B., Coon, J. T., Roge s, M., & Ande son, J. K.
(2020). Feasibili y o school-based iden i ica ion o child en and adolescen s expe iencing, o a - isk
o de eloping, men al heal h di icul ies: a sys ema ic e iew. P e en ion Science, 21(5), 581–603.
h ps://doi.o g/10.1007/s11121-020-01095-6
Tanne -Smi h, E. E., Du lak, J. A., & Ma x, R. A. (2018). Empi ically based mean e ec size dis ibu ions
o uni e sal p e en ion p og ams a ge ing school-aged you h: A e iew o me a-analyses. P e en-
ion Science, 19, 1091–1101. h ps://doi.o g/10.1007/s11121-018-0942-1
Valen e, J. Y., & Sanchez, Z. M. (2022). Sho - e m seconda y e ec s o a school-based d ug p e en ion
p og am: Clus e - andomized con olled ial o he B azilian e sion o DARE’s keepin’i REAL.
P e en ion Science, 23(1), 10–23. h ps://doi.o g/10.1007/s11121-021-01277-w
Weine , B. J., Lewis, C. C., S anick, C., Powell, B. J., Do sey, C. N., Cla y, A. S., Boyn on, M. H., &
Halko, H. (2017). Psychome ic assessmen o h ee newly de eloped implemen a ion ou come mea-
su es. Implemen a ion Science, 12, 108. h ps://doi.o g/10.1186/s13012-017-0635-3
Wingood, G. M., & DiClemen e, R. J. (2008). The ADAPT-ITT model: a no el me hod o adap ing e i-
dence-based HIV In e en ions. JAIDS Jou nal o Acqui ed Immune De iciency Synd omes, 47, S40–
S46. h ps://doi.o g/10.1097/QAI.0b013e3181605d 1
Publishe ’s No e Sp inge Na u e emains neu al wi h ega d o ju isdic ional claims in published maps
and ins i u ional a ilia ions.
1 3
Jou nal o P e en ion
Au ho s and A ilia ions
OlallaCu ín1· I. MacFadden2· F. F.Ma siglia3,4· S. S.Kulis3,5
Olalla Cu ín
[email p o ec ed]
1 Depa men o Psychology and Psychobiology, Uni e sidade de San iago de Compos ela,
Calle Xosé Ma ía Suá ez Núñez, s/n, 15782 A Co uña, San iago de Compos ela, Spain
2 Depa men o Sociology, Uni e sidad Pablo de Ola ide, Se illa, Spain
3 Global Cen e o Applied Heal h Resea ch, A izona S a e Uni e si y, Phoenix, AZ, Uni ed
S a es
4 School o Social Wo k, A izona S a e Uni e si y, Phoenix, AZ, Uni ed S a es
5 School o Social and Family Dynamics, A izona S a e Uni e si y, Tempe, AZ, Uni ed S a es
1 3
