scieee Science in your language
[en] (orig)

Development and validation of a framework for the assessment of school curricula on the presence of evolutionary concepts (FACE)

Sá-Pinto, Xana,Realdon, Giulia,Torkar, Gregor,Sousa, Bruno,Georgiou, Martha,Jeffries, Alex,Korfiatis, Konstantinos,Paolucci, Silvia,Pessoa, Patrícia,Rocha, Joana,Stasinakis, Panagiotis K.,Cavadas, Bento,Crottini, Angelica,Gnidovec, Tanja,Nogueira, Teresa · 2021

Abstract

Evolution is a key concept of biology, fundamental to understand the world and address important societal problems, but research studies show that it is still not widely understood and accepted. Several factors are known to influence evolution acceptance and understanding, but little information is available regarding the impacts of the curriculum on these aspects. Very few curricula have been examined to assess the coverage of biological evolution. The available studies do not allow comparative analyses, due to the different methodologies employed by the authors. However, such an analysis would be useful for research purposes and for the development of appropriate educational policies to address the problem of a lack of evolution acceptance in some countries. In this paper we describe the steps through which we developed a valid and reliable instrument for curricula analysis known as FACE: “Framework to Assess the Coverage of biological Evolution by school curricula.” This framework was developed based on the “Understanding Evolution Conceptual Framework” (UECF). After an initial pilot study, our framework was reformulated based on identified issues and experts’ opinions. To generate validity and reliability evidence in support of the framework, it was applied to four European countries’ curricula. For each country, a team of a minimum of two national and two foreign coders worked independently to assess the curriculum using this framework for content analysis. Reliability evidence was estimated using Krippendorf's alpha and resulted in appropriate values for coding the examined curricula. Some issues that coders faced during the analysis were discussed and, to ensure better reliability for future researchers, additional guidelines and one extra category were included in the framework. The final version of the framework includes six categories and 34 subcategories. FACE is a useful tool for the analysis and the comparison of curricula and school textbooks regarding the coverage of evolution, and such results can guide curricula development.

Read accessible full text

Development and validation of a framework for the assessment of school curricula on the presence of evolutionary concepts (FACE)

Author: Sá-Pinto, Xana,Realdon, Giulia,Torkar, Gregor,Sousa, Bruno,Georgiou, Martha,Jeffries, Alex,Korfiatis, Konstantinos,Paolucci, Silvia,Pessoa, Patrícia,Rocha, Joana,Stasinakis, Panagiotis K.,Cavadas, Bento,Crottini, Angelica,Gnidovec, Tanja,Nogueira, Teresa
Publisher: Springer Nature
Year: 2021
Source: https://repositorio.ulisboa.pt/bitstream/10451/47530/1/s12052-021-00142-2.pdf
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
h ps://doi.o g/10.1186/s12052‑021‑00142‑2
CURRICULUM AND EDUCATION
De elopmen and alida ion o a amewo k
o  heassessmen o school cu icula
on hep esence o e olu iona y concep s
(FACE)
Xana Sá‑Pin o1, Giulia Realdon2, G ego To ka 3, B uno Sousa4, Ma ha Geo giou5, Alex Je ies6,
Kons an inos Ko ia is7, Sil ia Paolucci8, Pa ícia Pessoa1, Joana Rocha9, Panagio is K. S asinakis10,
Ben o Ca adas11,12, Angelica C o ini13, Tanja Gnido ec3, Te esa Noguei a14,15, Penelope Papadopoulou16,
Cos anza Piccoli13, Johan Ba s ad17, Heloise D. Du ou 18, Milena Pejchino ska19, Alma Pob ic20,
D agana C e ko ić21 and E angelia Ma ikaki22*
Abs ac
E olu ion is a key concep o biology, undamen al o unde s and he wo ld and add ess impo an socie al p oblems,
bu esea ch s udies show ha i is s ill no widely unde s ood and accep ed. Se e al ac o s a e known o in luence
e olu ion accep ance and unde s anding, bu li le in o ma ion is a ailable ega ding he impac s o he cu iculum
on hese aspec s. Ve y ew cu icula ha e been examined o assess he co e age o biological e olu ion. The a ailable
s udies do no allow compa a i e analyses, due o he di e en me hodologies employed by he au ho s. Howe e ,
such an analysis would be use ul o esea ch pu poses and o he de elopmen o app op ia e educa ional poli‑
cies o add ess he p oblem o a lack o e olu ion accep ance in some coun ies. In his pape we desc ibe he s eps
h ough which we de eloped a alid and eliable ins umen o cu icula analysis known as FACE: “F amewo k o
Assess he Co e age o biological E olu ion by school cu icula.” This amewo k was de eloped based on he “Unde ‑
s anding E olu ion Concep ual F amewo k” (UECF). A e an ini ial pilo s udy, ou amewo k was e o mula ed based
on iden i ied issues and expe s’ opinions. To gene a e alidi y and eliabili y e idence in suppo o he amewo k, i
was applied o ou Eu opean coun ies’ cu icula. Fo each coun y, a eam o a minimum o wo na ional and wo o ‑
eign code s wo ked independen ly o assess he cu iculum using his amewo k o con en analysis. Reliabili y e i‑
dence was es ima ed using K ippendo ’s alpha and esul ed in app op ia e alues o coding he examined cu icula.
Some issues ha code s aced du ing he analysis we e discussed and, o ensu e be e eliabili y o u u e esea ch‑
e s, addi ional guidelines and one ex a ca ego y we e included in he amewo k. The inal e sion o he amewo k
includes six ca ego ies and 34 subca ego ies. FACE is a use ul ool o he analysis and he compa ison o cu icula and
school ex books ega ding he co e age o e olu ion, and such esul s can guide cu icula de elopmen .
Keywo ds: Cu icula analysis, E olu ion educa ion, Con en analysis, Lea ning goals
Open Access
E olu ion: Educa ion and Ou each
*Co espondence: ema ikak[email p o ec ed]
22 Facul y o P ima y Educa ion, Na ional and Kapodis ian Uni e si y
o A hens, Na a inou 13A, 10680 A hens, G eece
Full lis o au ho in o ma ion is a ailable a he end o he a icle
© The Au ho (s) 2021. This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License, which pe mi s use, sha ing,
adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long as you gi e app op ia e c edi o he o iginal au ho (s) and
he sou ce, p o ide a link o he C ea i e Commons licence, and indica e i changes we e made. The images o o he hi d pa y ma e ial
in his a icle a e included in he a icle’s C ea i e Commons licence, unless indica ed o he wise in a c edi line o he ma e ial. I ma e ial
is no included in he a icle’s C ea i e Commons licence and you in ended use is no pe mi ed by s a u o y egula ion o exceeds he
pe mi ed use, you will need o ob ain pe mission di ec ly om he copy igh holde . To iew a copy o his licence, isi h p://c ea i eco
mmons .o g/licen ses/by/4.0/. The C ea i e Commons Public Domain Dedica ion wai e (h p://c ea i eco mmons .o g/publi cdoma in/
ze o/1.0/) applies o he da a made a ailable in his a icle, unless o he wise s a ed in a c edi line o he da a.
In oduc ion
Science educa ion should con ibu e o inc ease s u-
den s’ scien i ic li e acy and imp o e he capaci y o
unde s anding science and he p ocesses o p oducing
Page 2 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
his knowledge, o ensu e mo e ci izens can apply hese
concep s in hei daily li es and pa icipa e in scien i ic
deba es and discussions (USA Na ional Resea ch Coun-
cil [NRC], 2007 and 2012). School cu icula should be
aligned wi h his goal. The school cu iculum ep e-
sen s “ he exp ession o educa ional ideas in p ac ice”
(P ideaux 2003, p.326). The e o e, he lea ning goals ha
a coun y wan s i s s uden s o achie e and he skills i
wan s hem o de elop a e exp essed and included in i s
school cu iculum. Di e en coun ies conside di e en
goals and skills o be mo e impo an han o he s, and
o his eason i is expec ed ha cu icula would a y
in bo h ype and s uc u e (Scholl 2012). The e is much
discussion in he li e a u e conce ning he de ini ion o
a school cu iculum (Young 2014; Bybee 2003) and i s
ole in educa ion (Bu ill e al. 2015). I a scien i ic heo y
needs o be widely augh , unde s ood, and accep ed by
he s uden s and u u e ci izens o a coun y, i should be
included in he na ional cu iculum. Cu iculum may be
conside ed a se o o icial policy documen s deli e ed o
eache s and ypically c ea ed by he ele an minis y o
educa ion and/o o he s a e au ho i ies ( o mal cu icu-
lum) (Sande s and Mako sa, 2016). These include all he
necessa y opics ha a eache should each and some
guidelines on how o do so. I a scien i ic heo y and
all i s associa ed concep s a e no included in he cu -
iculum, hen s uden s may no ha e he chance o lea n
abou i a school.
E olu ion is uni e sally acknowledged as one o he
mos impo an scien i ic concep s and as he uni ying
heme in biology. Since nume ous b oad hemes in he
ield o biology a e h eaded and held oge he by he he-
o y o biological e olu ion, se e al esea che s a gue ha
unde s anding his heo y is necessa y o scien i ic li e -
acy (Fowle and Zeidle 2016). Indica i e o his ele ance,
he U.S. Na ional Academy o Sciences (NAS) s a es ha
“ ew o he ideas in science ha e had such a a - eaching
impac on ou hinking abou ou sel es and how we ela e
o he wo ld” (NAS, 1998, p.21) and ha “ he eaching o
e olu ion should be an in eg al pa o science ins uc-
ion” (NAS, 1999, p.2). Acco ding o he USA NRC (NRC,
2012) e olu ion should be conside ed one o he ou key
concep s in biology o be explo ed om kinde ga en
onwa d, wi h inc easing complexi y. This is suppo ed by
se e al esea che s ha emphasize he need o de elop
lea ning p og essions o eaching e olu ion, which
should be e iden om he cu icula and ex books om
p ima y educa ion and ac oss biology opics (e.g., P inou
e al. 2011; Vaughn and Robbins 2017).
In ac , he s udy o e olu ion p omo es in e and
in adisciplina y links, allowing s uden s o in e ela e
concep s om biological, physical, and Ea h and space
sciences and use hem o achie e a be e unde s anding
o he wo ld a ound hem, as well as o add ess new p ob-
lema ic si ua ions (NRC 2012). E olu ion is ela ed wi h
se e al daily li e expe iences— om explaining biodi e -
si y, including he ecosys ems inside ou species, o d ug
esis ance by bac e ia, leas o mosqui os—and a basic
unde s anding o e olu iona y p ocesses is undamen-
al o add ess a numbe o key socie al p oblems such as
biodi e si y loss, clima e change, heal h o ood secu i y
(Ca oll e al. 2014), esis ance o an ibio ics and biocides
and pandemics (Lede be g 1988). In addi ion, he e is a
connec ion be ween unde s anding e olu ion and nego-
ia ing socie al p oblems. Sadle (2005), o example,
ound ha , while examining he in o mal easoning o
biology majo s on scena ios based on gene ic enginee ing
socio-scien i ic issues, hei unde s anding o e olu ion
s ongly in luenced hei decision-making. Fu he mo e,
a deepe engagemen wi h e olu ion and i s unde s and-
ing can de elop a g ea e knowledge o scien i ic and e i-
dence-based hinking (Heddy and Nadelson 2012) and i
also p o ides an e ec i e con ex o de eloping a deep
unde s anding o he Na u e o Science (NoS) (Nelson,
e al. 2019), which is impo an o p omo ing science li -
e acy (Holb ook and Rannikmae 2007).
Despi e i s cen al impo ance in unde s anding bio-
logical sys ems and add essing some indi iduals’ daily
li e and social p oblems, e olu ion is s ill no well unde -
s ood (o e en accep ed) by a la ge pa o socie y, a pa -
e n ha is obse ed ac oss di e en de elopmen al
s ages, coun ies, cul u al and eligious backg ounds
(Al e s and Nelson 2002; Asgha e al. 2007; A hanasiou
e al. 2012; A hanasiou and Ma ikaki 2013; A hanasiou
and Papadopoulou 2012; Blackwell e al. 2003; Eh linge
e al. 2008; K uge and Muelle 2002; Mille e al. 2006,
Nehm and Reilly 2007; Nehm e  al. 2009a, b; P inou
e al. 2008 and 2011; Sieckel and F ied ichsen 2013; To
e al. 2017; an Dijk and Reydon 2010). The e a e se e al
explana ions o his pe sis en and c oss-cul u al lack o
e olu iona y unde s anding including, among o he s:
• The p esence o “cogni i e bias” ha lead o e olu-
ion misconcep ions (Gelman 2003; Sh ulman 2006;
E ans 2008; Sina a e al 2008; Kelemen 1999; Kele-
men e al. 2013; Kelemen 2012; Ro man e al. 2017)
• The ac ha e olu iona y science in eg a es knowl-
edge, no ms and me hods om dis inc disciplines
such as geology, a chaeology, and subdisciplines
wi hin biology such as gene ics and ecology among
o he s (Gould 2002)
• The di e ence o meanings be ween common and
scien i ic language, such as “adap ”, “adap a ion”,
“p essu e” and “ i ness”, among o he wo ds, ha u -
he s eng hens misconcep ions (Al e s and Nelson
2002; Hull 1995; Rec o e al., 2013)
Page 3 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
• The pe cei ed con lic be ween e olu ion and eli-
gious, poli ical and pe sonal belie es (Asgha e al.
2007; Boujaoude e al. 2011; Chuang 2003; G i i h
and B em 2004; Golds on and Kyze 2000)
• Teache s’ lack o p epa edness o each abou his
subjec (P inou e al. 2011; Ya es and Ma ek 2014;
Vene is and Ma ikaki 2017; Be z e al. 2019; G esch
and Ma ens 2019; bu see Plu ze e al., 2020 o
encou aging esul s)
The ways ha educa ional esou ces, such as ex books
and school cu icula, a e p oduced may ha e u he
con ibu ed o his pa e n. In ac , in many ex books,
e e ences o e olu ion and e olu iona y concep s a e
agmen ed and limi ed o pa icula chap e s (Nehm
e al. 2009a, b; P inou e al. 2011) and some e en ein o ce
common misconcep ions (P inou e al. 2011). To s udy
he impac s o dis inc coun ies’ cu icula designs and
consequen unde s anding o e olu ion by s uden s, com-
pa a i e analyses a e needed. Al hough he accep ance
and li e acy abou e olu ion has shown o a y g ea ly
among coun ies (Mille e  al. 2006), ew s udies ha e
analysed he e ec o coun ies’ cu icula on public e olu-
ion li e acy. The s udy o Pinx en e al. (2020) suppo s
he hypo hesis ha an ea lie in oduc ion o e olu ion
in science cu icula, and a mo e in-dep h and ans e sal
explo a ion o e olu iona y ideas, may help o inc ease
bo h unde s anding and accep ance o e olu ion. Few
cu icula analyses ega ding he co e age o e olu iona y
concep s a e a ailable in he li e a u e, and hese mos ly
analyse he cu icula based on a gene al assessmen o
he p esence o absence o he opic o e olu ion (Ba be á
e al. 1999; Tidon and Lewon in 2004), o some special
opics (e.g. Quessada and Clemen 2011), o he ela-
ionship be ween eligious and scien i ic iews (Asgha
e  al. 2010). Howe e , none o hese examined which
majo ounda ional and key concep s equi ed o e o-
lu ion unde s anding we e p esen om he i s school
yea s onwa ds. Some esea che s pa ially add essed his
p oblem h ough he use o an induc i e con en analysis
me hod, ha is an analysis in which he coding scheme is
designed based on he analysis o he cu iculum and was
no p ede ined based on a ce ain heo e ical amewo k
(e.g. Kuschmie z e al. 2020) o based on mixed me hods
ha included induc i e and deduc i e analysis (Asgha
e al. 2015; Sande s and Mako sa 2016). Such esea ch,
al hough e y help ul, lacked a amewo k o compa a-
i e analysis. Indeed, compa able esea ch equi es a p e-
de ined coding scheme (o amewo k).
Skoog and Bilica (2002) de eloped such a amewo k
o analyze he science s anda ds o he s a es o USA, bu
hei ocus was on a limi ed se o o e a ching e olu ion-
a y concep s and no on hei ounda ional concep s,
hus limi ing hei applicabili y o lowe school g ades.
Some yea s la e , Asgha e al. (2015) p o ided some e y
use ul esul s ega ding he p esence o e olu iona y con-
cep s in he biology educa ion cu icula om dis inc
Canadian p o inces and e i o ies, basing hei empla e
o analysis on he “Unde s anding E olu ion Concep-
ual F amewo k” (UECF). The UECF, which was de el-
oped “by a eam o eache s and scien is s making use o
esou ces such as he A las o Science Li e acy, Bench-
ma ks o Science Li e acy, and he Na ional Science
educa ion S anda ds” (Sco chmoo and Thanukos 2007,
pp. 232–3), is he esul o a collabo a i e p ojec o he
Uni e si y o Cali o nia Museum o Paleon ology and he
Na ional Cen e o Science educa ion (Unde s anding
E olu ion, 2020). UECF includes he ounda ional as well
as he ad anced concep s needed o de elop a sophis i-
ca ed unde s anding o e olu iona y heo y (Asgha e al.
2015). I is di ided in o i e dimensions: His o y o li e,
E idence o e olu ion, Mechanisms o e olu ion, Na u e
o science, and S udying e olu ion. Each dimension is
u he de eloped in o co e ideas app op ia e o each
g ade (K-16). Finally, each co e idea is di ided in sub-
se s o ela ed e olu iona y ideas. UECF, acco ding o i s
c ea o s, is “a lis o concep ual unde s andings ega ding
e olu ion, aligned ac oss g ade le els o help ins uc o s
iden i y age-app op ia e lea ning goals o hei s uden s
and unde s and how concep s augh a one g ade le el
lay he g oundwo k o mo e sophis ica ed concep s la e
on” (Unde s anding E olu ion 2020). UECF indica es
which e olu ion concep s and mechanisms s uden s
should lea n abou . I is use ul as an analy ical amewo k
ha iden i ies he ounda ional e olu iona y ideas in ele-
men a y g ades, as well as speci ic concep s and mecha-
nisms conce ning e olu ion in la e g ades (Asgha e al.
2015). Al hough UECF canno be di ec ly used as a cu -
icula assessmen ool, i is a use ul heo e ical basis o
in o m he design o such ools. This was done by Asgha
e al., who de eloped hei own cu icula assessmen ool
based on UECF and on he Canadian Common F ame-
wo k using he concep s “ ela ed o ossils and deep ime,
na u al selec ion, and human e olu ion” (Asgha e  al.
2015, p.5). Un o una ely, his assessmen ool is ocused
only on a limi ed se o e olu ion concep s and he s udy
does no p esen much in o ma ion abou he assessmen
ool i sel . This p e en s o he esea che s om pe -
o ming simila analyses. Howe e , Asgha e al. (2015)
e ealed he use ulness o UECF as an ini ial basis o he
de elopmen o a amewo k ha could be used o ana-
lyze and compa e di e en coun ies’ cu icula.
In his pape we aim o de elop a amewo k ( empla e)
suppo ed by alidi y and eliabili y e idence ha could
be used o pe o m compa a i e analyses o coun ies’
cu icula.
Page 4 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
F amewo k de elopmen me hods
To de elop a amewo k o pe o m compa a i e cu -
icula analysis, we s a ed by iden i ying scien i ic s udies
ha analysed cu icula o hei co e age o e olu ion. A
non-sys ema ic sea ch allowed us o iden i y he s udies
o Skoog and Bilica (2002) and Asgha e al. (2015). To
iden i y addi ional s udies pe o ming cu icula analysis
ega ding he co e age o e olu ion we ha e made wo
sea ches in he Web o Science: one using he “e olu ion”
and “cu iculum analysis”; a second one wi h “e olu ion”
and “cu icula analysis”. F om hese sea ches we did no
e ie e any pape s ela ed wi h analysis o he cu icula
ega ding he co e age o e olu ion. Gi en he sca ci y
o pape s p o iding a me hodological amewo k o ana-
lyse cu icula ega ding hei co e age o e olu ion, we
ollowed he example o Asgha e al. (2015) and s a ed
de eloping ou F amewo k o Assess he Co e age o
biological E olu ion by school cu icula (FACE) based on
UECF.
Con en analysis (Bjø ns ud and Nilsen 2011; E doğan,
e al. 2009; Mkumbo 2009; Seke and Guney 2012) was
he selec ed me hod o analyze cu icula and speci ically
he “deduc i e con en analysis” as his is “guided by a
hal -s uc u ed o s uc u ed analysis ma ix” (Kyngäs
and Kaakinen 2020, p.23). Based on he UECF we buil a
sys em o ca ego ies and subca ego ies—a ibu ing code
numbe s o each ca ego y and subca ego y— ha we used
o p oceed wi h he con en analysis. The i e knowledge
dimensions ha UECF includes we e conside ed as he
ca ego ies o ou analysis: i) His o y o Li e, ii) E idence
o E olu ion, iii) Mechanisms o E olu ion, i ) Na u e o
Science (NoS) and ) S udying E olu ion. The main lea n-
ing goals ha , acco ding o UECF, suppo lea ning in
hese i e ca ego ies we e conside ed as subca ego ies.
Se e al s udies suppo he impo ance o hese i e ca -
ego ies and hei subca ego ies as we desc ibe below.
His o y o li e
Explo ing and unde s anding he His o y o Li e allows
s uden s o: i) explo e dis inc empo al scales, a h esh-
old concep ha is essen ial o e olu ion unde s and-
ing (Tibell and Ha ms 2017); ii) unde s and deep ime, a
p e equisi e o unde s and mac oe olu iona y p ocesses
ha has been p o en o be challenging o many s uden s
and o p edic s uden s’ accep ance o e olu ion (Ca ley
and No ick 2009; Co ne e al. 2010); iii) pe cei e he
his o ical pa e ns o empo al scales o na u al en i-
onmen al changes and i s co ela ion wi h ex inc ion
a es and compa e hose wi h p esen day pa e ns o
ully unde s and he human impac in he en i onmen
(Wyne and DeSalle 2020). Aligned wi h hese goals,
UECF included lea ning goals ha add ess dis inc ime
scales ( u ned in o he subca ego ies 1.1 o 1.5 and 1.7
see Appendix A) including deep ime (subca ego ies 1.1,
1.3), he geological and human induced changes and i s
impac s on e olu ion (subca ego ies 1.3, 1.4 and 1.6) as
well as he ex inc ion p ocess (subca ego y 1.5).
E idence o e olu ion
Recen wo k has shown ha s uden s’ posi ion on he
ela ionship be ween e olu ion and c ea ion can be
a ec ed, among o he ac o s, by hei unde s anding o
he scien i ic e idence suppo ing e olu ion (Yas i and
Mancy 2016). In ag eemen wi h his e idence, UECF
includes se e al lea ning goals ela ed wi h he e idence
o e olu ion (ca ego y 2 ha includes subca ego ies 2.1
o 2.6).
Mechanisms o e olu ion
Unde s anding he p ocesses ha cause e olu ion a e
essen ial no only o unde s anding he wo ld a ound
us bu also o be able o add ess cu en socioscien i ic
issues (Fowle and Zeidle , 2016; Peel e al. 2019). UECF
add esses he e olu iona y p ocesses in he dimension
“e olu iona y mechanisms” (ca ego y 3 om FACE),
which includes no only lea ning goals ha a e aligned
wi h he key and h eshold concep s p oposed by Tibell
and Ha ms (2017) o unde s and e olu ion by na u al
selec ion (subca ego ies 3.1 o 3.3 and 3.5 o 3.12; see
Appendix A) bu also lea ning goals ha speci ically
add ess o he e olu iona y p ocesses such as sexual
selec ion (subca ego y 3.7) and d i (subca ego y 3.8).
Al hough sexual selec ion and d i a e usually much
less o en add essed by e olu ion educa ion esea ch and
educa ional cu icula, hese play a e y impo an ole in
species e olu ion, being undamen al o he unde s and-
ing o na u al wo ld and popula ions, o he eaching o
e olu ion (P ice e al. 2014; Sá-Pin o e al. 2017), and in
he case o d i , o add ess p oblems such as biodi e si y
loss (P ice e al. 2014).
S udying e olu ion
In alignmen wi h ecommenda ions o science educa-
ioneduca ion (NRC 2012) UECF also includes lea ning
goals o s uden s o unde s and how esea che s s udy
e olu ion and how knowledge om e olu iona y biol-
ogy can be applied in daily li e con ex s. These lea ning
goals a e included in he dimension “S udying E olu ion”
which was u ned in o ou ca ego y 4 (wi h subca ego ies
4.1, 4.2 and 4.3).
Na u e o science
Finally, UECF also add esses s uden s’ unde s anding
abou he na u e o science (NoS), which has been con-
side ed e y impo an o e ec i e science educa ion,
and e olu ion educa ion in pa icula (e.g. F eeman e al.
Page 5 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
2014; Handelsman e al. 2006; Labo e al. 2009; Singe
e al. 2012; Wieman 2014). Se e al s udies (e.g. Rudolph
and S ewa d 1998; Lomb oso e al. 2008; Sina a e al.
2008; Scha mann 2018; Nelson e al. 2019) show a di ec
co ela ion be ween accep ing e olu ion and unde -
s anding NoS. This means ha i he cu icula a e made
wi h he pu pose o s uden s o no only know bu also
o accep e olu ion, hen paying a en ion o NoS gains
an ex a impo ance. This impo ance was ecognized
by he US Na ional Academy o Sciences, and he UECF
au ho s. NoS was u ned in o ou ca ego y 5 (wi h sub-
ca ego ies 5.1 o 5.5 aligned o dimensions o NoS p o-
posed in he Appendix H o NRC 2013).
This ini ial e sion o he F amewo k o he Assess-
men o school Cu icula on he p esence o E olu ion-
a y concep s (p e-FACE) was ini ially pilo ed in he
I alian cu iculum. This cu iculum was chosen because
i s lea ning goals a e ph ased in a complex, some imes
ambiguous wo ding, allowing di e en possible in e p e-
a ions. The e o e, i would be ideal o e ealing possible
gaps o weaknesses o he p e-FACE as a amewo k o
analyzing cu icula.
We a e awa e ha we analyze he “la en con en ” o
e olu ion concep s in he cu iculum, as “ he locus o
meaning is in he con en bu mus be in e ed by ec-
ognising a pa e n ac oss elemen s” (Po e and Le -
ine‐Donne s ein 1999, p. 261). In ou case he uni o
analysis was he “meaning uni ” – “ he cons ella ion o
wo ds o s a emen s ha ela e o he same cen al mean-
ing” (G aneheim and Lundman 2004, p. 106)–inside he
lea ning goals exp essed in a cu iculum. Each lea ning
goal exp essed in he cu iculum was conside ed as one
meaning uni , al hough in some a e cases a lea ning goal
could simul aneously add ess wo di e en lea ning goals
ega ding e olu ion lea ning. One example is a goal ha
is asking s uden s o “ ela e he consequences o an i-
bio ic misuse wi h inc eased bac e ial esis ance”. This
equi es s uden s o unde s and ha an h opogenic en i-
onmen al changes and biological e olu ion a e linked
(subca ego y 1.4), bu also ha e olu ion can be di ec ly
obse ed (subca ego y 2.2). So, a lea ning goal like his
includes wo meaning uni s and each meaning uni was
coded sepa a ely. Fo easons o ex economy om now
on when we e e o lea ning goals we a e in ac e e ing
o meaning uni s ega ding he con en analysis.
Validi y e idence was ga he ed ollowing he s eps p o-
posed by Po e and Le ine-Donne s ein (1999) (Table1)
and sugges ions we e made on he “app op ia eness,
meaning ulness, co ec ness, and use ulness” o ou
amewo k acco ding o ou esul s (F aenkel e al. 2012,
p. 147).
Th ee I alian code s analyzed he I alian cu iculum
and also ansla ed i s goals in o English. Two non-I alian
code s analyzed he ansla ed lea ning goals using he
p e-FACE. We chose o include in e na ional code s–
besides he I alian ones–so ha we would ensu e ha he
code s would see he cu iculum o he i s ime. Wo k-
ing along wi h he code s on he I alian cu iculum o
spo any inconsis encies in he amewo k o o e lapping
ca ego ies and aking unde conside a ion he c i ique o
Hanisch and Ei dosh (2020), we made some adjus men s
o he p e-FACE in o de o: i) join some subca ego ies
ha we e edundan and/o e u ned o e lapping esul s;
ii) include guidelines o cla i y he condi ions unde
which a lea ning goal should, o should no , be included
in a gi en subca ego y.
Table 1 S eps ollowed oensu e alidi y
* adjus ed om Po e and Le ine‑Donne s ein (1999, p.261 and 266)
S eps ensu ing he alidi y in hela en pa e n con en analysis* S eps ensu ing he alidi y inou esea ch
De elop a coding scheme ha guides code s in he analysis o con en .
I he scheme is ai h ul o he heo y in i s o ien ing code s o he
ocal concep s, i is ega ded as a alid coding scheme
Ou coding scheme was p e‑FACE which was de eloped based on he UECF.
As desc ibed abo e UECF co e s he majo e olu ion ideas (see Appendix
A) and has also been used by Asgha e al. (2015). The e o e, using his as a
basis enhances he alidi y o ou coding scheme
Code s ha e o ecognise pa e ns in he ex Code s had o ecognise pa e ns in he cu iculum = he p esence o he
concep s o he p e‑FACE in he cu iculum unde examina ion
Assess he decisions made by code s agains some s anda d (no m). I
he codes ma ch he s anda d o co ec decision making, hen he
coding is ega ded as p oducing alid da a. We look a he pa e n o
ag eemen ha shows a leas 80% o he code s making he same
coding. This is a high deg ee o ag eemen , and his se s a ai ly con‑
sis en no m. I means ha in ou analysis he codes we e e ec i e in
assessing wha i was in ended o assess ( alidi y) and his would be a
widely held judgmen ( eliabili y)
Code s (expe s wi h di e se p o iles and expe ise in he ield o biology and
educa ion –some a e expe s in e olu iona y biology, science educa ion
and science communica ion and some a e elemen a y/seconda y biology
eache s o elemen a y school/biology eache s’ aine s), some wo king
independen ly and some no , p o ided he coding. The independen cod‑
ing o he da a ensu ed ha all meaning uni s would be iden i ied and ha
none was le ou side, ha is, all lea ning goals e e ing o e olu ion a e
included. Codes p o ided by he code s we e compa ed and he in e a e s’
(in e code s’) ag eemen assessed by using K ippendo ’s alpha coe icien
(K ippendo 2011). Accep able esul s mean a widely held judgmen :
anyone who would ead he same ex ac o he cu iculum would be led
o he same esul s ega ding which e olu ion concep was co e ed

Page 6 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Ou amewo k, a his poin he “p e-FACE-2”, con-
sis ed o he same i e ca ego ies as he UECF, namely i)
His o y o Li e, ii) E idence o E olu ion, iii) Mechanisms
o E olu ion, i ) S udying E olu ion, ) Na u e o Science
(NoS), bu esul ed in ha ing ewe subca ego ies han in
he beginning o he analysis. Based on his amewo k
we pe o med he ollowing analysis. Each subca ego y
was assigned a numbe whe e he i s digi iden i ies
he main ca ego y o which an idea belongs, and he nex
digi (s) iden i ies he speci ic subca ego y (see Table2).
To cha ac e ize a lea ning goal, a code should conside
a i s he ca ego y in which i i s, hen decide abou he
speci ic subca ego y, and inally eco d e e y occu ence
in he analyzed cu iculum.
Da a used o  hede elopmen o  he amewo k
Wi hin he Eu opean con ex exis s a wide ange o cu -
icula designs and adi ions. In Scandina ian coun ies
and in he UK, school cu icula a e designed in a highly
gene al o m, only men ioning gene al opics o he
schools and eache s hemsel es o be he esponsible
Table 2 Concep ual amewo k o  heanalysis o school cu icula ega dinge olu ion (p e-FACE-2*)
* please no e ha he inal e sion o FACE is p esen ed in Table4
Ca ego y Subca eg o y
1. His o y o li e 1.1 Li e has been on Ea h o a long ime
1.2 P esen day li e o ms a e ela ed o pas li e o ms
1.3 La ge scale en i onmen al changes (caused by geological, geophysical, as onomical ac o s) and biological e olu ion
a e linked
1.4 An h opogenic en i onmen al changes and biological e olu ion a e linked
1.5 Many li e o ms ha once exis ed ha e gone ex inc
1.6 Ra es o e olu ion a y
1.7 Li e o ms/species/ change h ough ime
2. E idence o E olu ion 2.1 Simila i ies and/o di e ences among exis ing o ganisms (including mo phological, de elopmen al, and molecula
simila i ies) p o ide e idence o e olu ion
2.2 E olu ion can be di ec ly obse ed
2.3 The ossil eco d p o ides e idence o e olu ion
2.4 The geog aphic dis ibu ion o ex an species p o ides e idence o e olu ion
2.5 A i icial selec ion p o ides e idence o e olu ion
2.6 O ganisms’ ea u es, when analysed in ela ion o hei en i onmen p o ide e idence o e olu ion
3. Mechanisms o E olu ion 3.1 E olu ion is o en de ined as a change in allele equencies wi hin a popula ion
3.2 The e is a ia ion wi hin a popula ion
3.3 Li ing hings ha e o sp ing ha inhe i many ai s om hei pa en s bu a e no exac ly iden ical o hei pa en s
3.4. E olu ion occu s h ough mul iple mechanisms
3.5. Na u al selec ion ac s on he a ia ion ha exis s in a popula ion
3.6 Inhe i ed cha ac e is ics a ec he likelihood o an o ganism’s su i al and ep oduc ion
3.7 Sexual selec ion occu s when selec ion ac s on cha ac e is ics ha a ec he abili y o indi iduals o ob ain ma es
3.8 Gene ic d i ac s on he a ia ion ha exis s in a popula ion
3.9 Fi ness is ep oduc i e success— he numbe o iable o sp ing p oduced by an indi idual in compa ison o o he
indi iduals in a popula ion/species
3.10 Species can be de ined in many ways
3.11 Specia ion is he spli ing o one ances al lineage in o wo o mo e descendan lineages
3.12 E olu ion does no consis o p og ess in any pa icula di ec ion
4. S udying e olu ion 4.1 Scien is s s udy mul iple lines o e idence abou e olu ion
4.2 In e e yday li e we can ind applica ions o e olu iona y biology
4.3 Classi ica ion is based on e olu iona y ela ionships
5. Na u e o Science 5.1 Science is a human endea o (achie emen )
5.2 Science p o ides explana ions o he na u al wo ld
5.3 Science is based on empi ical e idence
5.4 Scien i ic ideas can change h ough ime
5.5 Scien i ic heo ies a e buil h ough a anspa en collec i e endea o
Page 7 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
execu o s o con en . Fo example, in he o icial No -
wegian cu iculum, e olu ion is ba ely men ioned, and
o icials a e us ing eache s on how he o mal con en
should be adap ed and deli e ed o he s uden s (Udi
2013 and 2020). I is sel -e iden ha his kind o cu -
icula we e no sui able o be analyzed wi h he p oposed
amewo k. Thus, in his s udy we chose among a speci ic
adi ion o cu iculum-de elopmen ha is cha ac e -
ized by a mo e de ailed p esc ip ion le el; his i s many
Eu opean coun ies bu does no aim o e lec he whole
ange o Eu opean cu icula adi ions. Fou Eu opean
coun ies’ cu icula o his kind (G eece, I aly, Po ugal
and Slo enia) we e used o es he de eloped amewo k.
Gi en he di e ences be ween coun ies’ school sys-
ems, we decided o analyze g ades 1–9 Biology o Sci-
ence cu icula, o o he subjec s in which Biology is
augh , i he la e did no exis as a sepa a e subjec in
he school cu iculum o a gi en school g ade/coun y.
An excep ion was made o I aly, o which we analyzed
g ades 1–10, as he o icial cu iculum conside s he 9 h
and 10 h g ades oge he . Al hough impo an e olu ion
lea ning goals may be add essed in Geog aphy, Geology,
o His o y, he lea ning goals o hese disciplina y ields
we e only analysed i hey we e augh in he same disci-
pline ha also add essed Biology lea ning goals.
In h ee ou o ou coun ies (i.e., Po ugal, Slo e-
nia and G eece) 9 h is he g ade un il which all s uden s
sha e he same compulso y subjec s and p og ams. A e
he 9 h g ade (a e 8 h g ade in I aly), s uden s a e usu-
ally allowed o choose dis inc educa ional b anches,
some o which do no include any biological discipline
(in o ma ion abou he o icial documen s analyzed and
he dis inc educa ional sys ems p o ided in Appendix
B).
In many coun ies, al hough e olu ion is explo ed
mo e in dep h in highe g ades, se e al o ganiza ions and
esea che s a gue o he inclusion o e olu iona y ideas
s a ing in he i s school yea s (Campos e  al. 2013;
Emmons e al. 2017; Kelemen e al. 2014; NRC 2012).
This pe spec i e mo i a ed de eloping a amewo k o
cu iculum analysis ha could be applied o lowe school
g ades o s udy and guide cu icula cons uc ion.
Reliabili y andcoding p ocess
To pe o m eliabili y es s in con en analysis (K ippen-
do 2004, p. 212, 219), i is impo an o use “se e al
esea che s wi h di e se pe sonali ies”—like he au ho s
o his pape who a e cha ac e ized by a ious p o es-
sional and educa ional p o iles and in many cases we e
code s. The code s wo ked in di e ing en i onmen s
(i.e. di e en o igins o code s in ou case) and demon-
s a ed ep oducibili y (in e code eliabili y; i.e. ‘ wo o
mo e indi iduals, wo king independen ly o each o he ,
applying he same eco ding ins uc ions o he same
uni s o analysis’; K ippendo 2004, p. 219). Mo e han
one code applied he same coding scheme o he same
uni s o analysis; a minimum o wo code s om each
coun y (local code s) independen ly ead he cu iculum
o hei coun y and iden i ied any e olu iona y goals
hey could ind in hese documen s. These code s gene -
a ed a able whe e each lea ning goal would occupy a cell
in a line (wi h e y ew excep ions whe e a lea ning goal
could include mo e han one meaning uni , as explained
abo e). In he cell igh nex o i hey we e asked o w i e
he ansla ion o his ex in English, which was checked
by he es o he na ional eam membe s o be consis -
en wi h he meaning o he ini ial ex . This p ocedu e
allowed in e na ional code s (one o wo o eign code s
who had access only o he lea ning goals bu no o he
coding) o con ibu e a "blind" analysis. A “na ional coo -
dina o ” om each coun y ga he ed all code s’ esul s in
one ile (p esen ed in Table3) and ga he ed eliabili y e i-
dence. A e ha , he/she (i) o ganized mee ings wi h his/
he coun y’s local code s o discuss esul s, (ii) iden i y
cases in which many disag eemen s occu ed, and (iii)
p opose possible changes o be included in he ame-
wo k o add ess hese p oblems. The p oblems and solu-
ions ound in each coun y we e hen discussed by he
eam o na ional coo dina o s who p oduced changes o
he FACE.
The phases o FACE de elopmen a e summa ized in
Fig.1. The eliabili y o all code s o each cu iculum
was es ed by K ippendo ’s alpha using IBM(c) SPSS 25
and he “syn ax kalpha” c ea ed by Hayes and K ippen-
do (2007).
Resul s anddiscussion
Ou goal was o de elop a alid and eliable amewo k
o esea che s o assess school cu icula acco ding o
whe he hey add ess he ideas, concep s, and mecha-
nisms ha a e necessa y o unde s and e olu ion. Ac oss
he ou coun ies included in his s udy we ound e i-
dence suppo ing he p esence o lea ning goals add ess-
ing 29 om he 33 subca ego ies ini ially included in his
analysis (see Table2 o he amewo k used and Table4
o he inal e sion o FACE). Reliabili y was calcula ed
based on K ippendo ’s alpha coe icien s (Table5) and
esul s con i med code eliabili y o each coun y (K ip-
pendo 2011).
Al hough K ippendo ’s alpha alue was always abo e
he lowes accep able le el o alpha (0.67, K ippendo
2011), se e al issues ha e been iden i ied du ing he p o-
cess o he de elopmen o he amewo k. To o e come
hese p oblems, ollowing he sugges ion o Po e and
Le ine-Donne s ein (1999, p. 267) o “p o ide o mu-
lae o weigh ing he di e en elemen s so ha [ u u e]
Page 8 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Table 3 An example o  he esul ing ile lis ing andcoding helea ning goal(s) inacoun y’s cu iculum
* o he meaning o he coded numbe s see Table2
Subjec Exac ex incoun y’s language Exac ex inenglish Code 1 Code 2 Code 3 Cycle/g ade
Biology […] iconosce e nei ossili indizi pe icos ui e nel empo le
as o mazioni dell’ambien e isico, la successione e l’e oluzione
delle specie
( he s uden ) Recognise ha ossil eco ds a e he clues o
econs uc en i onmen al changes o e ime, species suc‑
cession and e olu ion
23* 23 23 Middle school (g ades 6 h‑8 h)
Page 9 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
code s will know how o so h ough con lic ing se s o
cues as well as how o handle o he coding p oblems”, we
p o ided guidelines o be applied in speci ic cases. One
o hem conce ns classi ying lea ning goals ha ela e
biological s uc u e and unc ion. In FACE, he subca -
ego y 2.7 “O ganisms’ ea u es, when analyzed in ela-
ion o hei en i onmen p o ide e idence o e olu ion”
(Table2) was de i ed om he UECF which p o ided as
an example ha “Fo m is linked o unc ion.” Howe e ,
his subca ego y spa ked an in ense deba e in ou anal-
ysis o he school cu icula, mos ly when ying o code
lea ning goals ha would link s uc u e and unc ion o
in e nal o gans wi hou men ioning he o ganism’s li -
ing en i onmen . Fo example, in he Po uguese “Essen-
ial Lea ning Goals Guidelines” i is w i en: “Rela e he
o gans o he male and emale ep oduc i e sys em wi h
hei unc ion” (6 h g ade; Po uguese Go e nmen /Min-
is y o Educa ion, 2018 p10); “Iden i y he mo phol-
ogy and ana omy o he hea o a mammal, explaining
i s main cons i uen s and hei espec i e unc ions” (9 h
g ade, Po uguese Go e nmen /Minis y o Educa ion,
2018i p9); and in he I alian cu icula we ead: “The s u-
den can ecognize in he /his o ganism s uc u es and
unc ions a mac oscopic and mic oscopic le els” (6 h-
8 h g ades). To o e come he unce ain y o whe he one
should a ibu e subca ego y 2.7 in hese cases o no we
decided ha a lea ning goal i s in his subca ego y only
i i enables he connec ion o a pa icula ea u e o he
o ganism and i s ex e nal en i onmen (see Table4 o
he inal e sion o FACE).
Ano he p oblem iden i ied du ing he applica ion o
ou amewo k a ose wi h subca ego y 3.2—“The e is
a ia ion wi hin a popula ion”—and subca ego y 2.1—
Simila i ies and/o di e ences among exis ing o ganisms
(including mo phological, de elopmen al, and molecula
simila i ies) p o ide e idence o e olu ion”—as hese
we e some imes misused by some o ou membe s, who
would conside cases o in aspeci ic a iabili y belonging
o 2.1. To sol e his p oblem, we p opose ha : i) code s
should assess whe he he lea ning goal is ocusing on he
mechanisms o e olu ion o he e idence o e olu ion, as
hese wo subca ego ies a e pa o di e en ca ego ies;
and ii) he subca ego y 2.1 o be applied only o lea n-
ing goals ha men ion in e speci ic di e si y o di e si y
among highe axonomic le els (example p oka yo ic
e sus euka yo ic cells, see Table4 o he inal e sion
o FACE).
A simila p oblem was aised by he in e p e a ion o
he subca ego y 1.2 “P esen day li e o ms a e ela ed o
pas li e o ms”. When classi ying lea ning goals ela ed
wi h genealogical ees, some code s applied his subca -
ego y o ela ionships be ween indi iduals o he same
species. To a oid his, we included a guideline s a ing
ha subca ego y 1.2 should only be applied o lea ning
goals men ioning dis inc species and no dis inc indi-
iduals o he same species (see Table4 o he inal e -
sion o FACE).
Fig. 1 Desc ip ion o he p ocess ha led o he de elopmen o
FACE
Page 16 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Table 4 (con inued)
Ca ego y Subca ego y Guidelines Examples
3.5. Na u al selec ion ac s on he a ia ion ha
exis s in a popula ion A lea ning goal conside ed o ep esen subca ‑
ego y 3.5 may ask s uden s o unde s and ha
na u al selec ion can ac a mul iple hie a chical
le els such as genes, cells, indi iduals, popula‑
ions, species, and la ge clades
Do no apply o lea ning goals ha men ion mo e
han one e olu iona y p ocess like “Na u al selec‑
ion and gene ic d i ac on he a ia ion ha
exis s in a popula ion”. Fo hese lea ning goals
apply subca ego y 3.4
E olu ion esul s om selec ion ac ing upon gene ic
a ia ion wi hin a popula ion. (UECF)
Na u al selec ion ac s on pheno ype as an exp es‑
sion o geno ype. (UECF)
The amoun o gene ic a ia ion wi hin a popula ion
may a ec he likelihood o su i al o he popula‑
ion; he less he a ailable di e si y, he less likely
he popula ion will be able o su i e en i onmen‑
al change. (UECF)
Na u al selec ion some imes a o s he e ozygo es
o e homozygo es a a locus. He e ozygo e ad an‑
age p ese es gene ic a ia ion a ha locus (i.e.,
wi hin he popula ion, i main ains mul iple alleles
a ha locus). (UECF)
To de ine na u al selec ion and desc ibe he mecha‑
nism by which li ing o ganisms e ol e. (GR 9 h
g ade)
Explain he need o he in e en ion o sex cells in
he ep oduc ion o some li ing beings and hei
impo ance o he e olu ion o he species. (PT,
5 h g ade)
3.6 Inhe i ed cha ac e is ics a ec he likelihood o
an o ganism’s su i al and ep oduc ion A lea ning goal conside ed o ep esen subca ‑
ego y 3.6 may ask s uden s o unde s and ha
ad an ageous ai s o en pe sis in a popula ion.
Ad an ageous ai s depend on he en i onmen
and he selec i e p essu es i imposes
Depending upon he en i onmen , some li ing
hings will su i e be e han o he s. (UECF)
En i onmen al changes may a ec an o ganism’s
abili y o su i e. (UECF)
O e ime, he p opo ion o indi iduals wi h ad an‑
ageous cha ac e is ics may inc ease (and he
p opo ion wi h disad an ageous cha ac e is ics
may dec ease) due o hei likelihood o su i ing
and ep oducing. (UECF)
Popula ions, no indi iduals, e ol e. (UECF)
T ai s ha con e an ad an age may pe sis in he
popula ion and a e called adap a ions. (UECF)
3.7 Sexual selec ion occu s when selec ion ac s
on cha ac e is ics ha a ec he p obabili y o
ob aining a ma e
Sexual selec ion can lead o physical and beha io al
di e ences be ween he sexes (UECF)

Page 17 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Table 4 (con inued)
Ca ego y Subca ego y Guidelines Examples
3.8 Gene ic d i ac s on he a ia ion ha exis s in
a popula ion Do no apply o lea ning goals ha men ion mo e
han one e olu iona y p ocess like “Na u al selec‑
ion and gene ic d i ac on he a ia ion ha
exis s in a popula ion”. Fo hese lea ning goals
apply subca ego y 3.4
Smalle popula ions a e mo e s ongly a ec ed by
gene ic d i han a e la ge popula ions. (UECF)
Gene ic d i can cause loss o gene ic a ia ion in a
popula ion. (UECF)
Founde e ec s occu when a popula ion is ounded
om a small numbe o indi iduals. (UECF)
Bo lenecks occu when a popula ion’s size is g ea ly
educed. (UECF)
E olu ion esul s om gene ic d i ac ing upon
gene ic a ia ion wi hin a popula ion. (UECF)
3.9 Fi ness is ep oduc i e success — he numbe
o iable o sp ing p oduced by an indi idual in
compa ison o o he indi iduals in a popula ion/
species
An o ganism’s i ness depends on bo h i s su i al
and i s ep oduc ion. (UECF)
Fi ness is o en measu ed using p oxies like mass,
numbe o ma ings, and su i al because i is
di icul o measu e ep oduc i e success di ec ly.
(UECF)
3.10 Species can be de ined in many ways A lea ning goal conside ed o ep esen subca ‑
ego y 3.10 may ask s uden s o:
· p o ide one o mo e species de ini ion
· men ion ha hyb ids can occasionally esul om
ma ing be ween dis inc species o o m
The e a e many de ini ions o species. (UECF)
Some hyb ids ha e inc eased i ness ela i e o hei
pa en s. (UECF)
3.11 Specia ion esul s om he spli ing o one
ances al lineage in o wo o mo e descendan
lineages
Specia ion is o en he esul o geog aphic isola ion.
(UECF)
Specia ion equi es ep oduc i e isola ion. (UECF)
Occupying new en i onmen s can p o ide new
selec ion p essu es and new oppo uni ies, leading
o specia ion. (UECF)
3.12 E olu ion does no consis o p og ess in any
pa icula di ec ion
4. S udying e olu ion 4.1 Scien is s s udy mul iple lines o e idence
abou e olu ion A lea ning goal conside ed o ep esen subca ‑
ego y 4.1 may ask s uden s o unde s and ha :
· scien is s s udy li ing beings and how hese a e
ela ed o each o he
· ou knowledge o e olu ion is cons uc ed
and con inuously e ined by mul iple lines o
e idence
Scien is s use ossils o lea n abou pas li e. (UECF)
Scien is s use mul iple lines o e idence (including
mo phological, de elopmen al, and molecula
e idence) o in e he ela edness o axa. (UECF)
Scien is s use expe imen al e idence o s udy e olu‑
iona y p ocesses. (UECF)
Scien is s use a i icial selec ion as a model o lea n
abou na u al selec ion. (UECF)
E olu iona y ees (e.g., phylogenies o cladog ams)
a e buil om mul iple lines o e idence. (UECF)
Explain he con ibu ion o he s udy o ossils and
ossiliza ion p ocesses o he econs uc ion o he
his o y o li e on Ea h. (PT, 7 h g ade)
Page 18 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Table 4 (con inued)
Ca ego y Subca ego y Guidelines Examples
4.2 In e e yday li e we can ind applica ions o
e olu iona y biology To ecognise, by expe iencing plan cul i a ion and
animal aising, ha e e y o ganism’s li e is ela ed
o o he and di e en li e o ms. (IT, 4 h‑5 h g ades)
The a ie y o li ing beings and he complexi y o
hei s uc u es and unc ions a e subjec o he
s udy o e olu ion and sys ema ics, Mendelian
gene ics and o ganism‑en i onmen in e ac ions,
wi h he aim o alo isa ion and main enance o
biodi e si y. (IT, 9 h‑10 h g ades)
4.3 Classi ica ion is based on e olu iona y ela ion‑
ships Do no apply o lea ning goals ha explo e classi i‑
ca ions ha a e no clea ly based on e olu iona y
ela ionships
Apply o lea ning goals ha explo e classi ica ions
ha a e nes ed and biologically ele an (phylo‑
gene ically co ec )
To classi y hem (o ganisms) acco ding o speci ic
c i e ia, o iden i y s uc u al and unc ional simila i‑
ies and di e ences, and ela e hem o he needs
c ea ed by he en i onmen in which hey li e. (GR
gene al goals o 7 h‑9 h g ades)
To classi y ypical li ing o ganisms acco ding o clas‑
si ica ion ules. (GR 7 h g ade)
To explain he meaning o classi ica ion and he
pa ame e s mos commonly used o classi y o gan‑
isms. (IT 9 h‑10 h g ades Technical/Voca ional)
5. Na u e o Science 5.0 Unde s anding he NoS Goals ha men ion he impo ance o explo ing
NoS ha canno be a ibu ed o any o he
subca ego ies below should be conside ed as
belonging o subca ego y 5.0
Value he na u e o science, con inuing he de elop‑
men o scien i ic me hodology in i s di e en
s ages (P , om he 1s o he 4 h g ade)
5.1 Science is a human endea o (achie emen ) Recognise ha science is a human ac i i y, wi h
i s own goals, p ocedu es and ways o hinking,
h ough he explo a ion o cu en o his o ical
e en s ha documen i s na u e. (PT,5 h and 6 h
g ades)
5.2 Science p o ides explana ions o he na u al
wo ld Scien is s can es Ideas abou e en s and p ocesses
long pas , e y dis an , and no di ec ly obse able.
(UECF)
To use, as possible, he knowledge hey achie e
o in e p e , phenomena, p ocesses o p oblems
ha occu (…). (GR gene al biology goals 7 h‑9 h
g ades)
5.3 Science is based on empi ical e idence A lea ning goal conside ed o ep esen subca ‑
ego y 5.3 may ask s uden s o unde s and ha
science is based on e idence collec ed wi h ou
senses and ex ensions o ou senses
Scien is s use mul iple esea ch me hods (expe i‑
men s, obse a ional esea ch, compa a i e
esea ch, and modeling) o collec da a. (UECF)
Scien is s use mul iple esea ch me hods (expe i‑
men s, obse a ions, compa isons, and modeling)
o collec e idence. (UECF)
Page 19 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Table 4 (con inued)
Ca ego y Subca ego y Guidelines Examples
5.4 Scien i ic Ideas can change h ough ime A lea ning goal conside ed o ep esen subca ‑
ego y 5.4 may ask s uden s o unde s and ha
scien i ic knowledge is open o ques ion and
e ision depending on new ideas and/o new
e idence
5.5 Scien i ic heo ies a e buil h ough a anspa ‑
en collec i e endea o A lea ning goal conside ed o ep esen subca ‑
ego y 5.5 may ask s uden s o unde s and ha :
· science always exposes ideas o es ing
· scien is s may explo e di e en hypo heses o
explain obse a ions
· accep ed scien i ic heo ies mus su i e igo ous
es ing and be suppo ed by mul iple lines o
e idence o be accep ed
· scien i ic con o e sy and deba e wi hin he com‑
muni y con ibu e o scien i ic p og ess
Scien is s es hei ideas using mul iple lines o
e idence. (UECF)
6. De elopmen o scien i ic p ac ices 6.0 De elopmen o p ac ices ha scien is s
employ as hey in es iga e and build models and
heo ies abou he wo ld. ****
A lea ning goal conside ed o ep esen subca ‑
ego y 6.0 may ask o :
· s uden s’ engagemen and unde s anding scien‑
i ic me hodologies and p ac ices
· s uden s de eloping he scien i ic way o hinking
Recognise he impo ance o he scien i ic me hod
o he s udy o li e p ocesses. (GR 7 h g ade)
The s uden explo es phenomena wi h a scien i ic
app oach: […] obse es and desc ibes he un old‑
ing o e en s, asks ques ions based on pe sonal
hypo heses, p oposes and ealises simple expe i‑
men s. (IT g ades 1s ‑5 h)
Know how o ask ques ions, aise hypo heses, make
in e ences, p o e esul s and know how o com‑
munica e, ecognizing how knowledge is buil . (PT,
1s ‑4 h g ade)
Build scien i ic explana ions based on scien i ic
concep s and e idence, ob ained h ough he
pe o mance o di e si ied p ac ical ac i i ies—lab‑
o a o y, expe imen al, ield—and planned o y o
answe o mula ed p oblems. (PT, 5 h‑6 h g ade)
Implemen p ac ical in es iga ions, based on
sys ema ic obse a ion, modeling and labo a o y/
expe imen al wo k, o add ess p oblems ela ed
o e es ial ma e ials, di e si y o li ing beings
and hei in e ac ions wi h he en i onmen . (PT,
5 h‑6 h g ade)
Plan and implemen p ac ical in es iga ions based
on sys ema ic obse a ion, modeling and wo k
labo a o y/expe imen al, o espond o p oblems
ela ed o he dynamics o plane Ea h and o he
e idence ha helps ell i s s o y. (PT, 7 h‑9 h g ade)
* The examples p o ided a e ei he om he UECF (Unde s anding E olu ion Concep ual F amewo k) o hey a e goals om he analyzed cu icula
** Guidelines came up h ough his analysis and a e conside ed impo an o be aken unde conside a ion o u u e analyses
*** Lea ning goals such as his a e an example o he ac ha some lea ning goals in he cu iculum may be a ibu ed o achie ing mo e han one subca ego y
**** This subca ego y is ph ased acco ding o NRC (2012)
Page 20 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
All code s e e ed o goals hey iden i ied desc ibing
he need o s uden s o engage in scien i ic p ac ices o
ecognize he impo ance o scien i ic me hods, such as:
“Know how o ask ques ions, aise hypo hesis, make
in e ences, p o e esul s and know how o commu-
nica e, ecognizing how knowledge is buil ” (Po -
ugal, Essen ial Lea nings 1s o 4 h g ades; Po -
uguese Go e nmen /Minis y o 2018a p8; 2018b
p9; 2018c p9; 2018d p10 espec i ely); “implemen
p ac ical in es iga ions, based on sys ema ic obse -
a ion, modeling and labo a o y/expe imen al wo k,
o add ess p oblems ela ed o e es ial ma e ials,
di e si y o li ing beings and hei in e ac ions wi h
he en i onmen . (…) Build scien i ic explana ions
based on scien i ic concep s and e idence, ob ained
h ough he pe o mance o di e si ied p ac i-
cal ac i i ies—labo a o y, expe imen al, ield—and
planned o y o answe o mula ed p oblems.” (Po -
ugal, Essen ial Lea nings 5 h o 6 h g ades; Po u-
guese Go e nmen /Minis y o Educa ion, 2018e p4;
2018 p4).
“[The s uden ] Explo es phenomena wi h a scien-
i ic app oach”; “The pupil obse es and desc ibes
he un olding o e en s, asks ques ions based on
pe sonal hypo heses, p oposes and ealizes simple
expe imen s, wi h he help o he eache ” (I aly,
lea ning goals o g ades 1s o 5 h).
“Is able o collec quali a i e and quan i a i e da a by
obse ing and pe o ming measu emen s, o eco d
and p esen hem app op ia ely” (Slo enia, subjec
Science in 6 h and 7 h g ade).
Al hough engaging in scien i ic p ac ices is undamen-
al o os e ing he de elopmen o s uden s’ scien i ic
li e acy (NRC, 2012), goals like hese we e no di ec ly
desc ibed in ou amewo k o analysis. Bu mos o he
local code s ecognized hese and simila lea ning goals
as belonging o he subca ego ies e e ing o Na u e
o Science (NoS), mainly based on he assump ion ha
engaging in scien i ic p ac ices could p o ide a chance
o ge be e acquain ed wi h he NoS. Fu he mo e,
coding o hese lea ning goals was among he ones wi h
he leas consensus be ween he code s. The con usion
be ween scien i ic p ac ices, scien i ic inqui y and NoS
is common and longs anding ( e iewed by Lede man
2019). Howe e , esea ch esul s show ha s uden s will
only lea n abou he NoS i his is explici ly in eg a ed
in he ins uc ion ( e iewed by Lede man 2019). To
u he imp o e ou amewo k, we ecommended he
in oduc ion o a new and independen ca ego y, which
would allow esea che s o be mo e p ecise and include
in his ca ego y he lea ning goals ha a e ela ed wi h
he de elopmen o scien i ic p ac ices as de ined by NRC
(2012). Ano he issue ela ed wi h NoS was he ac ha ,
in some coun ies, some o icial documen s men ion he
impo ance o aluing he NoS bu hey do no di e en i-
a e he dimensions o he NoS o be lea ned by s uden s,
he e o e p ecluding i s assignmen o any subca ego y
o NoS p esen in ou amewo k. One example o such
s a emen s can be ound in Po uguese o icial docu-
men s om 1s o 4 h g ade ha men ion in i s in o-
duc o y ex , ha i is essen ial o “Value he na u e o
science, con inuing he de elopmen o scien i ic me h-
odology in i s di e en s ages” (Po uguese Go e nmen /
Minis y o 2018a p3, 2018b p3; 2018c p3; 2018d p4). To
o e come his issue, we p oposed a guideline sugges ing
ha such s a emen s should be coded as belonging o ca -
ego y 5 (NoS) wi hou de ailing he subca ego y, he e-
o e hey should be coded as subca ego y 5.0 (Table4).
FACE (Table 4) will p o ide esea che s a ool sup-
po ed by alidi y and eliabili y e idence o analyze di -
e en coun ies’ cu icula. Al hough coun ies di e
signi ican ly in se e al aspec s (e.g., educa ional policies,
cu icula, eache s’ educa ion, school ex books), ha ing
a ool suppo ed by e idence o assess co e age o e o-
lu iona y key concep s by school cu icula could help
esea che s ge a clea e pic u e o each coun y. When
compa ing, o example, he public’s accep ance o e olu-
ion in di e en coun ies (Mille e al. 2006), ha ing an
idea abou he educa ion ha hose people ha e ecei ed
based on hei o icial school cu icula can help o unde -
s and he impac o he cu icula in public’s e olu ion li -
e acy and accep ance and in o m app op ia e decisions
on educa ional policies o inc ease hese.
S udy limi a ions andsugges ions o  u he esea ch
We should highligh ha al hough impo an e olu ion
lea ning goals may be add essed in Geog aphy, Geology,
o His o y, he lea ning goals o hese disciplina y ields
we e only analyzed i hese we e augh in he same disci-
pline ha also add essed Biology lea ning goals.Fu he -
mo e, in ou analysis we did no co e he cu icula o
he school yea s du ing which e olu ion is s udied as a
majo opic o in mo e dep h (in Po ugal his is explo ed
in he 11 h g ade). This may explain why some o FACE’s
subca ego ies we e no iden i ied in any o he examined
cu icula (e.g. Ra es o e olu ion a y, E olu ion is o en
de ined as a change in allele equencies wi hin a popu-
la ion, Gene ic d i ac s on he a ia ion ha exis s in a
popula ion e c.). This did no su p ise us because UECF
sugges s ha some o hese subca ego ies should be
explo ed by olde s uden s. I is possible ha addi ional
p oblems could a ise when using he FACE in uppe edu-
ca ional le els. Gi en his, u u e wo k should expand he
Page 21 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
s udy o FACE’s use ulness in e alua ing highe g ades’
cu icula.
FACE (Table4) could also be used o assess whe he
he e olu iona y concep s a e p esen ed in o he school
subjec s’ cu icula, besides biology, and could also be
used o assess hei p esence in a con inui y o in a ag-
men ed ashion. The analysis o Scheuch and Rachbaue
(2019) ound a agmen a ion o e olu iona y concep s
in Aus ian school ex books, and Nehm e al. (2009a,
b) conside agmen a ion as a majo possible sou ce o
misconcep ions as i gi es s uden s a agmen a y pic-
u e o e olu ion (Sande s and Mako sa 2016). O cou se,
we should no unde es ima e he ole o a good eache
in o e coming any obs acles posed by he cu iculum.
The e o e, besides analyses o coun ies’ cu icula u -
he s udies ocusing on eache s, s uden s, ex books
and eaching p ac ices a e needed o imp o e ou unde -
s anding abou how e olu ion is augh in each coun y.
Abb e ia ions
FACE: F amewo k o he Assessmen o school Cu icula ega ding E olu ion;
FC: Fundação pa a a Ciência e a Tecnologia; GR: G eece; IT: I aly; K‑9: Kinde ‑
ga en o 9 h g ade; NoS: Na u e o Science; NRC: Na ional Resea ch Council
(USA); PT: Po ugal; SL: Slo enia; UECF: Unde s anding E olu ion Concep ual
F amewo k; UK: Uni ed Kingdom.
Acknowledgemen s
We would like o hank E hymia Niki a and Asimakis Talamagas o hei alu‑
able commen s and he e o e hei con ibu ion o his deli e able. We would
also like o hank he Eu oSci izen S ee ing Commi ee and specially o Tania
Jenkins o hei wo k and suppo o Eu oSci izen WG2.
Au ho con ibu ions
EM was esponsible o he in e na ional coo dina ion o he p ojec . EM,
XSP, GR and GT we e na ional coo dina o s o he p ojec . EM, XSP, GR, GT, BS,
PP, AJ, KK, PKS, TN, JB, HD, MG, DC, MP we e in ol ed in he de elopmen o
FACE. EM, XSP, GR, GT, AC, BS, BC, PP, CP, MP, TG, KK, PKS, TN, MG pe o med he
cu iculum analysis ei he as local o o eigne esea che s. EM led he da a
analysis. EM, XSP, GR led he w i ing o he pape and all au ho s con ibu ed
o discussions and o i s inal e sion.
Funding
Xana Sá‑Pin o is unded by Po uguese na ional unds h ough FCT –
Fundação pa a a Ciência e a Tecnologia, I.P., wi hin he scope o he ame‑
wo k con ac o eseen in he numbe s 4, 5 and 6 o he a icle 23, o he
Dec ee‑Law 57/2016, o Augus 29, changed by Law 57/2017, o July 19 and
h ough he p ojec UID/CED/00194/2019. Te esa Noguei a was suppo ed
by FCT, unde he p ojec UID/00329/202 and he con ac ALG‑01–0145‑
FEDER‑028824. Angelica C o ini is suppo ed by he Po uguese Na ional
Funds h ough FCT wi h he In es igado FCT g an (IF/00209/2014). Cos anza
Piccoli is suppo ed by FCT PhD S uden ship (SFRH/BD/144342/2019).
This pape is a esul o a collabo a i e wo k o esea che s pa icipa ing in he
Eu oSci izen CA17127, Eu opean Funded p ojec (Wo king G oup 2).
A ailabili y o da a and ma e ials
The da ase s gene a ed du ing he cu en s udy a e no publicly a ailable
due o he ac ha hey consis pa o a wide p ojec ha will p esen
he esul s o a compa a i e analysis o Eu opean coun ies’ school cu icula
ega ding e olu ion bu a e a ailable om he co esponding au ho on
easonable eques . O icial pape s e e ing o each coun y’s school cu ‑
iculum ha p o ided he o iginal da a used in his pape a e he ollowing:
G eek educa ional sys em Go e nmen ’s Gaze e Vol. B, No. 304/13–03‑03 [in
G eek] ΑΔΑ: 6ΥΧΙ4653ΠΣ‑ΧΨΕ (h ps ://www.al a i a.g /si es /de au l / iles
/a ac hmen s/didak ea_ili.pd ). Po uguese educa ional sys em. Po uguese
Assembly o Republic (2009). Law n.º 85/2009, om 27 o Augus . h ps ://d e.
p /appli ca io n/con e udo/48882 6. Po uguese Go e nmen /Dec ee‑Law n.º
55/2018, om 6 july. h ps ://d e.p /appli ca io n/con e udo/11565 2962. Po u‑
guese Go e nmen /Minis y o educa ion, Educa ion (2018a, b, c, d, e, , g, h,
i). I alian educa ional sys em. Indicazioni nazionali pe il cu icolo della scuola
dell’in anzia e del p imo ciclo d’is uzione (D.M. 254 del 16 no emb e 2012):
h p://www.indic azion inazi onali .i /2018/08/26/indic azion i‑2012/. Indicazioni
nazionali igua dan i gli obie i i speci ici di app endimen o conce nen i le
a i i à e gli insegnamen i comp esi nei piani degli s udi p e is i pe i pe co si
liceali (D.M. n 211 del 7/10/2010); Regolamen o ecan e no me conce nen i
il io dino degli is i u i ecnici ai sensi dell’a icolo 64, comma 4, del dec e o
legge 25 giugno 2008, n. 112, con e i o dalla legge 6 agos o 2008, n. 133;
Regolamen o ecan e no me conce nen i il io dino degli is i u i p o essionali
ai sensi dell’a icolo 64, comma 4, del dec e o legge 25 giugno 2008, n. 112,
con e i o dalla legge 6 agos o 2008, n. 133: h p://a chi io.pubbl ica.is u
zione .i / i o ma_supe io i/nuo e supe io i/index .h ml Slo enian educa ional
sys em Eu ydice (2019). Slo enia o e iew, h ps ://eacea .ec.eu op a.eu/na io
nal‑polic ies/eu yd ice/con e n /slo e nia_en. Go .si. Republic o Slo enia (2020).
P og ami in učni nač i osno ni šoli [P og ams and syllabuses in compulso y
basic school]. Re ie ed om h ps ://www.go .si/ eme/p og ami‑in‑ucni‑nac
i‑ ‑osno ni‑soli/.
E hics app o al and consen o pa icipa e
No applicable.
Consen o publica ion
No applicable.
Compe ing in e es s
The e a e no compe ing in e es s.
Au ho de ails
1 Resea ch Cen e on Didac ics and Technology in he Educa ion o T ain‑
e s, Depa men o Educa ion and Psychology, Uni e si y o A ei o, A ei o,
Po ugal. 2 UNICAMea h G oup, Geology Sec ion, Uni e si y o Came ino,
Came ino, I aly. 3 Uni e si y o Ljubljana Facul y o Educa ion, Ka delje a
ploščad 16, 1000 Ljubljana, Slo enia. 4 Alpoen e, Ag upamen o de Escolas
de Albu ei a Poen e, Albu ei a, Po ugal. 5 Depa men o Biology, Na ional
and Kapodis ian Uni e si y o A hens, A hens, G eece. 6 Milne Cen e
o E olu ion, Depa men o Biology and Biochemis y, Uni e si y o Ba h,
Ba h, UK. 7 Depa men o Educa ion, Uni e si y o Cyp us, Nicosia, Cyp us.
8 Labo a o io di Scienze Spe imen ali, Foligno, I aly. 9 Uni e si y o T ás‑Os‑
Mon es E Al o Dou o, Vila Real, Po ugal. 10 Ampelokipoi Labo a o y Cen e
o Na u al Sciences (EKFE), A hens, G eece. 11 IPsan a ém, Poly echnic Ins i u e
o San a ém‑School o Educa ion, San a ém, Po ugal. 12 CeiED, Lusó ona Uni‑
e si y, Lisboa, Po ugal. 13 CIBIO/InBio InBIO, Uni e sidade Do Po o, Cen o
de In es igação Em Biodi e sidade E Recu sos Gené icos, Campus Ag a io
de Vai ão, 4485‑661 Vai ão, Po ugal. 14 INIAV, Na ional Ins i u e o Ag a ian
and Ve e ina y Resea ch, Oei as and Vai ão, Po ugal. 15 cE3c, Cen e o Ecol‑
ogy, E olu ion and En i onmen al Changes, Uni e si y o Lisbon, Lisbon,
Po ugal. 16 Depa men o Ea ly Childhood Educa ion, Uni e si y o Wes e n
Macedonia, Kozani, G eece. 17 Uni e si y College o G een De elopmen ,
B yne, No way. 18 Le Ce cle FSER, Pa is, F ance. 19 Facul y o Educa ion, “S .
Klimen Oh idski” Uni e si y, Bi ola, No h Macedonia. 20 Facul y o Science,
Uni e si y o Sa aje o, Sa aje o, Bosnia and He zego ina. 21 Chai o Gene ‑
ics and E olu ion, Facul y o Biology, Uni e si y o Belg ade, Belg ade, Se bia.
22 Facul y o P ima y Educa ion, Na ional and Kapodis ian Uni e si y o A h‑
ens, Na a inou 13A, 10680 A hens, G eece.
Appendix A
See Table5.

Page 22 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Table 5 Th eshold andkey concep s o e olu ion byna u al selec ion (Tibel andHa ms, 2017) andexamples o how hese
a e add essed inFACE
Type o Concep Concep Examples o how hese concep s a e add essed inFACE
(subca ego ies a e iden i ied by hei codes anddesc ip ion;
guidelines p o ided whenneeded)
Th eshold concep Tempo al scale P ocesses aking place a e y long empo al scales:
1.1 Li e has been on Ea h o a long ime
1.3 La ge scale en i onmen al changes (caused by geological, geo‑
physical, as onomical ac o s) and biological e olu ion a e linked
2.3 The ossil eco d p o ides e idence o e olu ion
P ocesses aking place a ou species empo al scale:
1.4 An h opogenic en i onmen al changes and biological e olu ion
a e linked
2.6 A i icial selec ion p o ides e idence o e olu ion
P ocesses aking place in a gene a ion ime scale:
3.3 Li ing hings ha e o sp ing ha inhe i many ai s om hei
pa en s bu a e no exac ly iden ical o hei pa en s
P ocesses aking place a di e se ime scale:
1.5 Many li e o ms ha once exis ed ha e gone ex inc
1.7 Li e o ms/species/ change h ough ime
Spa ial scale La ge wo ldwide scale
1.3 La ge scale en i onmen al changes (caused by geological, geo‑
physical, as onomical ac o s) and biological e olu ion a e linked
2.4 The geog aphic dis ibu ion o ex an species p o ides e idence o
e olu ion
Ecosys em/popula ion scale
2.6 O ganisms’ ea u es, when analysed in ela ion o hei en i onmen
p o ide e idence o e olu ion
3.5 Na u al selec ion ac s on he a ia ion ha exis s in a popula ion
Indi iduals’ scale
3.6 Inhe i ed cha ac e is ics a ec he likelihood o an o ganism’s
su i al and ep oduc ion
Cell/molecula scales
3.3 Li ing hings ha e o sp ing ha inhe i many ai s om hei
pa en s bu a e no exac ly iden ical o hei pa en s (includes he ol‑
lowing guideline: ecombina ion and mu a ions in ep oduc i e cells
esul in new he i able ai s and a e sou ces o di e si y)
P obabili y 3.6 Inhe i ed cha ac e is ics a ec he likelihood o an o ganism’s
su i al and ep oduc ion
3.7 Sexual selec ion occu s when selec ion ac s on cha ac e is ics ha
a ec he p obabili y o indi iduals o ma e
Randomness 3.3.‑ Li ing hings ha e o sp ing ha inhe i many ai s om hei
pa en s bu a e no exac ly iden ical o hei pa en s (guideline: A
lea ning goal conside ed o ep esen subca ego y 3.3 may ask s u‑
den s o unde s and ha ecombina ion and mu a ions a e andom
p ocesses)
3.8 Gene ic d i ac s on he a ia ion ha exis s in a popula ion
Key concep s o na u al selec ion: P inciple Va ia ion O igin o a ia ion 3.3 Li ing hings ha e o sp ing ha inhe i many ai s om hei pa ‑
en s bu a e no exac ly iden ical o hei pa en s. (guideline: A lea n‑
ing goal conside ed o ep esen subca ego y 3.3 may ask s uden s
o unde s and ha ecombina ion and mu a ions in ep oduc i e
cells esul in new he i able ai s and a e sou ces o di e si y)
Indi idual a ia ion 3.2 The e is a ia ion wi hin a popula ion
3.5. Na u al selec ion ac s on he a ia ion ha exis s in a popula ion
3.8 Gene ic d i ac s on he a ia ion ha exis s in a popula ion
Di e en ial i ness 3.6 Inhe i ed cha ac e is ics a ec he likelihood o an o ganism’s
su i al and ep oduc ion
3.9 Fi ness is ep oduc i e success — he numbe o iable o sp ing
p oduced by an indi idual in compa ison o o he indi iduals in a
popula ion/species
Page 23 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
Table 5 (con inued)
Type o Concep Concep Examples o how hese concep s a e add essed inFACE
(subca ego ies a e iden i ied by hei codes anddesc ip ion;
guidelines p o ided whenneeded)
Key concep s o na u al selec ion: P inciple Inhe i ance He i able ai s 3.6 Inhe i ed cha ac e is ics a ec he likelihood o an o ganism’s
su i al and ep oduc ion
3.3 Li ing hings ha e o sp ing ha inhe i many ai s om hei
pa en s bu a e no exac ly iden ical o hei pa en s
Rep oduc ion 3.7 Sexual selec ion occu s when selec ion ac s on cha ac e is ics ha
a ec he p obabili y o indi iduals o ma e
3.9 Fi ness is ep oduc i e success — he numbe o iable o sp ing
p oduced by an indi idual in compa ison o o he indi iduals in a
popula ion/species
Key concep s o na u al selec ion: P inciple Selec ion Selec ion p essu e 3.6 Inhe i ed cha ac e is ics a ec he likelihood o an o ganism’s
su i al and ep oduc ion (Guideline: A lea ning goal conside ed
o ep esen subca ego y 3.6 may ask s uden s o unde s and ha
(…) ad an ageous ai s depend on he en i onmen and selec i e
p essu es i imposes)
Di e en ial su i al
and ep oduc ion 3.6 Inhe i ed cha ac e is ics a ec he likelihood o an o ganism’s
su i al and ep oduc ion
Change in popula ion 3.1 E olu ion is o en de ined as a change in allele equencies wi hin
a popula ion
Specia ion 3.11 Specia ion esul s om he spli ing o one ances al lineage in o
wo o mo e descendan lineages
Appendix B
Desc ip ion o each coun y’s school sys em
and heanalysed documen s
G eece
The G eek educa ional sys em consis s o p eschool edu-
ca ion (manda o y), p ima y educa ion (6 yea s), sec-
onda y educa ion (6yea s, om which he 3 o hem a e
manda o y—lowe seconda y educa ion—and he es
op ional and di ided o P o essional and Gene al s ud-
ies, which bo h can lead o exams o e ia y educa ion)
and e ia y educa ion ( he yea s depend on he ype o
s udies). The G eek cu icula a e o icial documen s and
eache s ha e o ollow hem—only a e y small de ia-
ion is allowed—and he e is only one school ex book
o each subjec , which is a ailable o ee o s uden s
o public schools and also can be publicly accessed on
he in e ne . The cu icula ha a e alid oday in G eece
we e published 17yea s ago and a e only being sligh ly
modi ied om ime o ime (Go e nmen ’s Gaze e Vol.
B, No. 304/13–03-03; ΑΔΑ: 6ΥΧΙ4653ΠΣ-ΧΨΕ). Biol-
ogy, as a sepa a e subjec , exis s only in seconda y edu-
ca ion, whe eas in p ima y Educa ion he e a e mo e
gene al cou ses and eache s wi h no specializa ion in
science each hese cou ses. P eschool s uden s a e con-
side ed o ge in ouch wi h na u e and hei en i on-
men and s uden s o p ima y educa ion ge in ouch
wi h some biological concep s du ing hei i s 4 g ades
in a cou se called “S udying he En i onmen ’’ ( e e ing
o manmade and na u al en i onmen ). Du ing hei
las wo yea s o p ima y educa ion a cou se called Sci-
ence dedica es abou 1/3 o i s con en o Biology. How-
e e , om 2018 an o icial pape ha was adminis e ed
o schools asked eache s no o each mos o hese bio-
logical concep s. As a esul , G eek s uden s who will
g adua e compulso y educa ion will ha e s udied biology
mainly du ing hei 3 las yea s and only o 1h/week.
Po ugal
In Po ugal, compulso y educa ion s a s a he age
o 6yea s old, p io o which p e-school educa ion is
a ailable in bo h p i a e and public schools, bu a end-
ance is no manda o y (Po uguese Republic Assem-
bly, 2009). Compulso y educa ion ex ends o 12yea s
which include basic educa ion and seconda y educa ion.
In basic educa ion, he subjec s a e common o all s u-
den s (un il 9 h g ade). In seconda y educa ion, s uden s
can a end ei he science o humani ies cou ses, which
ha e common and speci ic subjec s o each b anch, o
hey can a end p o essional cou ses (Po uguese Go -
e nmen /Dec ee-Law 55/2018). Basic Educa ion com-
p ises h ee cycles. The i s cycle is a ended by s uden s
om 6–10yea s-old and ex ends om he 1s o he 4 h
g ade. Du ing his cycle a single eache is esponsible
o each se e al subjec s ha include po uguese, ma h-
ema ics, s udy o he en i onmen , a is ic educa ion,
physical educa ion and ci izenship and de elopmen . 1s
Page 24 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
cycle s uden s also ha e o a end English lessons (Diá io
da Républica Ele ónico/Dec ee-Law 55/2018). Du ing
his cycle, biology educa ion is add essed in he s udy o
he en i onmen lea ning goals, which aim o de elop
con en knowledge and skills in biology, geology, phys-
ics, chemis y, geog aphy, his o y and echnology (Po -
uguese Go e nmen /Minis y o Educa ion, 2018a, b,
c, d). In he second cycle, om 5 o 6 h g ade (11-12y
s uden s), and hi d cycle, om 7 o 9 h g ade (13-15y
s uden s), each disciplina y ield is augh by specialized
eache s in ha ield. Biology is augh in he discipline
o Na u al Sciences, oge he wi h Geology. Fo all hese
cycles and disciplina y ields, he lea ning goals in e ms
o con en knowledge and skills a e mos ly de e mined
by he na ional o icial p og ams, wi h he school boa ds
being allowed o de e mine up o 25% o he annual
s anda ds. The essen ial lea ning goals documen s de ail
he con en lea ning and skills ha should be de eloped
by all s uden s du ing basic educa ion. Fo ha eason,
in he p esen s udy we analysed he essen ial lea ning
goals o s udy o he en i onmen ( om he 1s o he 4 h
g ade; Po uguese Go e nmen /Minis y o 2018a o d)
and na u al sciences ( om he 5 h o he 9 h g ade Po -
uguese Go e nmen /Minis y o Educa ion, 2018e o i).
I aly
The I alian School Sys em is o ganized in wo cycles. The
Fi s Ins uc ion Cycle is composed o Kinde ga en, (3
o 6yea s, non-compulso y educa ion), P ima y School
(1s o 5 h g ade, om 6 o 11yea s o age, compulso y
educa ion) and Lowe Seconda y School (6 h o 8 h
g ade, om 11 o 14yea s o age, compulso y educa ion).
The Fi s Ins uc ion Cycle o e s he same cu iculum o
all s uden s. The Second Ins uc ion Cycle is composed
o Uppe Seconda y School (9 h o 13 h g ade). I is com-
pulso y un il 16yea s o age (9 h and 10 h g ades) and is
di ided in o di e en school ypes: Liceo, Technical and
Voca ional High Schools, each one wi h u he speciali-
za ions. Liceo High School is a gene al high school p e-
pa ing o uni e si y, Technical High School p epa es o
employmen bu also gi es access o uni e si y, Voca-
ional High School, is mo e ocused on p ac ical subjec s
and wo k expe ience, bu gi es access o uni e si y oo.
In Kinde ga en and in P ima y School, a eache (gen-
e ally wi hou a speci ic academic backg ound) can each
all subjec a eas, including Biology, p esen in bo h seg-
men s wi h di e en names. In Lowe Seconda y School,
science and ma hema ics a e augh by he same eache ,
usually wi h a biology o a ma hema ics deg ee. Science
eaching ime is o icially 3h pe week. In Liceo High
School, Biology is augh wi hin Na u al Sciences (2–3h
pe week) by a eache wi h a deg ee in biology/na u al
sciences o ano he ela ed subjec . In Technical and
Voca ional High Schools, Biology is augh as in Liceo,
(wi hin Na u al Sciences, 2h pe week) bu only in he
i s wo g ades, wi h he excep ion o some specializa-
ions (e.g. ag icul u e). School cu icula a e na ionwide
and egula ed a he gene al le el by Minis y o Educa-
ion guidelines. Teache s a e gua an eed a good amoun
o libe y in eaching and in choosing among many di e -
en ex books.
Slo enia
In Slo enia, p e-school educa ion is op ional. Child en
can en ol as ea ly as a he age o 11mon hs and a end
i un il hey s a compulso y school. Nine-yea com-
pulso y school is di ided in o h ee h ee-yea cycles
( o s uden s om 6 o 14yea s old). I is manda o y,
99% public, and s a e- inanced. The i s six yea s can be
ecognised as he p ima y (ISCED 1) le el. G ades 7–9
a e in e na ionally ecognised as he lowe seconda y
school (ISCED 2). This is ollowed by he uppe second-
a y school sys em (ISCED 3) ( om wo up o i e yea s)
(Eu ydice, 2019). Biology lea ning objec i es a e al eady
included in he cu iculum o p e-school educa ion, in
one o six p og ams named Na u e. In he nine-yea com-
pulso y school biology educa ion is included in ou com-
pulso y school subjec s: Lea ning abou he en i onmen
(1s , 2nd and 3 d g ade), Science and Technology (4 h
and 5 h g ade), Science (6 h and 7 h g ade), and Biol-
ogy (8 h and 9 h g ade). The subjec “Science” dedica es
abou 2/3 o i s con en o biology. Biology educa ion is
also a pa o uppe seconda y educa ion in subjec s o
Biology, Science o Science and Socie y, depending on
he s udy p og am. P ima y school eache s a e each-
ing all school subjec s in he i s h ee-yea s cycle and
mos o he second h ee-yea cycle. In 5 h and 6 h g ade
specialized subjec eache s g adually ake o e eaching
subjec s. These o he s each he subjec s Science in 6 h
and 7 h g ade and Biology in he 8 h and 9 h g ade.
Recei ed: 22 Sep embe 2020 Accep ed: 5 Feb ua y 2021
Re e ences
Al e s B, Nelson CE. Teaching e olu ion in highe educa ion. E ol.
2002;56(10):1891–901.
Asgha A, Bean S, O’Neill W, Al e s B. Biological e olu ion in Canadian sci‑
ence cu icula. Repo s o he Na ional Cen e o Science Educa ion.
2015;35(5):1.1–1.21.
Asgha A, Wiles J, Al e s B. Disco e ing in e na ional pe spec i es on biologi‑
cal e olu ion ac oss eligions and cul u es. In Jou Di e s O ganiz.
2007;6(4):81–8. h ps ://doi.o g/10.18848 /1447‑9532/CGP/ 06i0 4/39200 .
Asgha A, Wiles J, Al e s B. The o igin and e olu ion o li e in Pakis ani
High School Biology. J Biol Educ. 2010;44(2):65–71. h ps ://doi.
o g/10.1080/00219 266.2010.96561 96.
A hanasiou K, Ka akos E, Papadopoulou P. Concep ual ecology o he
e olu ion accep ance among G eek educa ion s uden s: wha is he
Page 25 o 27
Sá‑Pin oe al. E o Edu Ou each (2021) 14:3
con ibu ion o knowledge inc ease? J Biol Educ. 2012;46(4):234–41.
h ps ://doi.o g/10.1080/00219 266.2012.71678 0.
A hanasiou K, Ma ikaki E. Concep ual in en o y o na u al selec ion as a
ool o measu ing G eek uni e si y s uden s’ e olu ion knowledge:
di e ences be ween no ice and ad anced s uden s. In J Sci Educ.
2013;36(8):1262–85. h ps ://doi.o g/10.1080/09500 693.2013.85652 9.
A hanasiou K, Papadopoulou P. Concep ual ecology o he e olu ion accep ‑
ance among G eek educa ion s uden s: knowledge, eligious p ac ices
and social in luences. In J Sci Educ. 2012;34(6):903–24. h ps ://doi.
o g/10.1080/09500 693.2011.58607 2.
Ba be a O, Zanon B, Pe ez‑Pla JF. Biology cu iculum in wen ie h‑cen u y
Spain. Sci Educ. 1999;83:97–111.
Be z N, Le e s JS, Tho EED, Fux M, de Nesne a K, Tanne KD, Coley JD. Cogni‑
i e cons ual‑consis en ins uc o language in he unde g adu‑
a e biology class oom. Li e Sci Educ. 2019. h ps ://doi.o g/10.1187/
cbe.19‑04‑0076.
Bjø ns ud H, Nilsen S. The de elopmen o in en ions o adap ed eaching
and inclusi e educa ion seen in ligh o cu iculum po en ial A con en
analysis o No wegian na ional cu icula pos 1980. Cu iculum J.
2011;22(4):549–66. h ps ://doi.o g/10.1080/09585 176.2011.62721 6.
Blackwell WH, Powell M, Dukes GH. The p oblem o s uden accep ance o
e olu ion. J Biol Educ. 2003;37(2):58–67.
Boujaoude S, Asgha A, Wiles JR, Jabe L, Sa ieddine D, Al e s B. Biology
p o esso s’ and eache s’ posi ions ega ding biological e olu ion
and e olu ion educa ion in a middle eas e n socie y. In J Sci Educ.
2011;33(7):979–1000.
Bu ill G, Lappan G, Gonula es F. Cu iculum and he Role o Resea ch. In: Cho
S, edi o . The P oceedings o he 12 h In e na ional Cong ess on Ma h‑
ema ical Educa ion. Sp inge , Cham 2015.
Bybee R. E olu ion in pe spec i e: The science eache ’s compendium. A ling‑
on, VA: NSTA P ess; 2003.
Campos R, Sá‑Pin o A. Ea ly e olu ion o e olu iona y hinking: eaching e olu‑
ion in elemen a y schools. E ol Educ Ou each. 2013;6(1):25. h ps ://
doi.o g/10.1186/1936‑6434‑6‑25.
Ca oll SP, Jø gensen PS, Kinnison MT, Be gs om CT, Denison RF, Gluckman
P, Smi h TB, S auss SY, Tabashnik BE. Applying e olu iona y biology o
add ess global challenges. Science. 2014;346(6207):1245993. h ps ://
doi.o g/10.1126/scien ce.12459 93.
Ca ley KM, No ick LR. Digging deep: Explo ing college s uden s’ knowledge o
mac oe olu iona y ime. J Res Sci Teach. 2009;46(3):311–32.
Chuang HC. Teaching e olu ion: A i udes and s a egies o educa o s in U ah.
Am Biol Teache . 2003;65(9):669–74.
Co ne S, B ooks DC, Moo e R. Is he age o he ea h one o ou “so es ou‑
bles?” s uden s’ pe cep ions abou deep ime a ec hei accep ance o
e olu iona y heo y. E olu ion. 2010;64(3):858–64. h ps ://doi.o g/10.11
11/j.1558‑5646.2009.00911 .x.
Eh linge J, Johnson K, Banne M, Dunning D. Why he unskilled a e unawa e:
u he explo a ions o (absen ) sel ‑insigh among he incompe en .
O gan Beha Hum Decis P ocess. 2008;105(1):98–121.
Emmons N, Lees K, Kelemen D. Young child en’s nea and a ans e o he
basic heo y o na u al selec ion: an analogical s o ybook in e en ion.
J Res Sci Teach. 2017;55(3):321–47. h ps ://doi.o g/10.1002/ ea.21421 .
E doğan M, Kos o a Z, Ma cinkowski T. Componen s o en i onmen al li e acy
in elemen a y science educa ion cu iculum in Bulga ia and Tu key.
EURASIA J Ma h Sci Tech Educ. 2009;5(1):15–26.
Eu ydice (2019). Slo enia o e iew, 2019. h ps ://eacea .ec.eu op a.eu/na io nal‑
polic ies/eu yd ice/con e n /slo e nia_en. Accessed 10/11/2019.
E ans EM. Concep ual change and e olu iona y biology: A de elop‑ men al
analysis. In: Vosniadou S, edi o . In e na ional handbook o esea ch on
concep ual change. New Yo k: Rou ledge; 2008. p. 263–94.
F ankel JR., Wallen NE, & Hyun HH. (2012). How o design and e alua e
esea ch in educa ion (8 h ed.). McG aw Hill.
F eeman S, Eddy SL, McDonough M, Smi h MK, Oko oa o N, Jo d H,
Wende o h MP. Ac i e lea ning inc eases s uden pe o mance in
science, enginee ing, and ma hema ics. P oc Na l Acad Sci USA.
2014;111:8410–5.
Fowle SR, Zeidle DL. Lack o e olu ion accep ance inhibi s s uden s’ nego ia‑
ion o biology‑based socioscien i ic issues. J Biol Edu. 2016;50(4):407–
24. h ps ://doi.o g/10.1080/00219 266.2016.11508 69.
Gelman SA. The essen ial child: O igins o essen ialism in e e yday hough .
New Yo k: Ox o d Uni e si y P ess; 2003.
Golds on MJD, Kyze P. Teaching e olu ion: Na a i es wi h a iew om
h ee sou he n biology eache s in he USA. J Res Sci Teach.
2000;46(7):762–90.
Gould SJ. The s uc u e o e olu iona y heo y. Camb idge, MA: Ha a d
Uni e si y P ess; 2002.
Go .si. Republic o Slo enia (2020). P og ami in učni nač i osno ni šoli [P o‑
g ams and syllabuses in compulso y basic school]. Re ie ed om h ps
://www.go .si/ eme/p og ami‑in‑ucni‑nac i‑ ‑osno ni‑soli/
Go e nmen ’s Gaze e Vol. B, No. 304/13–03–03 [in G eek] ΑΔΑ: 6ΥΧΙ4653ΠΣ‑
ΧΨΕ h ps ://www.al a i a.g /si es /de au l / iles /a ac hmen s/didak
ea_ili.pd . Accessed 10/11/2019.
G aneheim UH, Lundman B. Quali a i e con en analysis in nu sing esea ch:
concep s, p ocedu es and measu es o achie e us wo hiness.
Nu se Educ Today. 2004;24(2):105–12. h ps ://doi.o g/10.1016/j.
ned .2003.10.001.
G esch H, Ma ens M. Teleology as a aci dimension o eaching and lea ning
e olu ion: A sociological app oach o class oom in e ac ion in science
educa ion. J Res Sci Teach. 2019;56(3):243–69. h ps ://doi.o g/10.1002/
ea.21518 .
G i i h JA, B em SK. Teaching e olu iona y biology: P essu es, s ess, and cop‑
ing. J Res Sci Teaching. 2004;41(8):791–809.
Hanisch S, Ei dosh, D. Challenges wi h concep ualiza ions o e olu ion in
biology educa ion. 2020. h ps ://doi.o g/10.13140 /RG.2.2.26589 .64484 .
h ps ://www. esea chga e.ne /publi ca io n/34249 2432_
Handelsman J, Ebe ‑May D, Beichne R, B uns P, Chang A, DeHaan R, Gen ile
J, Lau e S, S ewa J, Tilghman SM, Wood WW. Scien i ic Teaching. Sci.
2004;304:521–2.
Ha len W. P inciples and big ideas o science educa ion. Ha ield: ASE ‑ Asso‑
cia ion o Science Educa ion College Lane; 2010. h ps ://www.ase.o g.
uk/bigid eas. Accessed 10/11/2019.
Hayes AF, K ippendo K. Answe ing he call o a s anda d eliabili y measu e
o coding da a. Communica ion Me hods Measu es. 2007;1:77–89.
Heddy BC, Nadelson LS. A global pe spec i e o he a iables associa ed wi h
accep ance o e olu ion. E ol Educ Ou each. 2012;5(3):412–8.
Holb ook J, Rannikmae M. The na u e o science educa ion o enhanc‑
ing scien i ic li e acy. In J Sci Edu. 2007;29(11):1347–62. h ps ://doi.
o g/10.1080/09500 69060 10075 49.
Hull D. Uni e sal Da winism. Na u e. 1995;377(6549):494. h ps ://doi.
o g/10.1038/37749 4a0.
Indicazioni nazionali pe il cu icolo della scuola dell’in anzia e del p imo ciclo
d’is uzione (D.M. 254 del 16 no emb e 2012): h p://www.indic azion
inazi onali .i /2018/08/26/indic azion i‑2012/ Accessed 10/11/2019.
Indicazioni nazionali igua dan i gli obie i i speci ici di app endimen o
conce nen i le a i i à e gli insegnamen i comp esi nei piani degli s udi
p e is i pe i pe co si liceali (D.M. n 211 del 7/10/2010).
Kelemen D. Why a e ocks poin y? Child en’s p e e ence o eleological expla‑
na ions o he na u al wo ld. De Psychol. 1999;35(6):1440–52.
Kelemen D. Teleological minds: How na u al in ui ions abou agency and
pu pose in luence lea ning abou e olu ion. In Roseng en KS, B em SK,
E ans EM, & Sina a GM. E olu ion Challenges: In eg a ing Resea ch and
P ac ice in Teaching and Lea ning abou E olu ion. Ox o d Uni e si y
P ess; 2012. p.66–92
Kelemen D, Emmons NA, Schillaci RS, Ganea PA. Young Child en Can Be Taugh
Basic Na u al Selec ion Using A Pic u e S o ybook In e en ion. Psychol
Sci. 2014;25(4):893–902. h ps ://doi.o g/10.1177/09567 97613 51600 9.
Kelemen D, Ro man J, Ses on R. P o essional physical scien is s display ena‑
cious eleological endencies: pu pose‑based easoning as a cogni i e
de aul . J Exp Psychol Gen. 2013;142(4):1074–83.
K ippendo K. Compu ing K ippendo ’s Alpha‑Reliabili y. 2011; Re ie ed
om h ps :// epos i o y .upenn .edu/asc_pape s/43. Accessed
20/11/2019.
K ippendo K. Con en analysis: An in oduc ion o i s me hodology. 2nd ed.
Thousand Oaks, Cali o nia: Sage Publica ions; 2004.
K uge J, Muelle RA. Unskilled, unawa e, o bo h? The be e ‑ han a e age
heu is ic and s a is ical eg ession p edic e o s in es ima es o own
pe o mance. J Pe s Soc Psychol. 2002;82(2):180–8.
Kuschmie z P, Benie mann A, G a D. De elopmen and e alua ion o he
knowledge abou e olu ion 2.0 ins umen (KAEVO 2.0). In J Sci Educ.
2020;42(15):2601–29. h ps ://doi.o g/10.1080/09500 693.2020.18225 61.
Kyngäs H, Kaakinen P. Deduc i e Con en Analysis. In: Kyngäs H, Mikkonen
K, Kää iäinen M, edi o s. The Applica ion o Con en Analysis in