Bezem, Pablo; Piezunka, Anne; Jacobsen, Rebecca
A icle — Accep ed Manusc ip (Pos p in )
School Inspec o s’ Decision-making: E idence om a
Compa a i e Pe spec i e
Leade ship and Policy in Schools
P o ided in Coope a ion wi h:
WZB Be lin Social Science Cen e
Sugges ed Ci a ion: Bezem, Pablo; Piezunka, Anne; Jacobsen, Rebecca (2024) : School Inspec o s’
Decision-making: E idence om a Compa a i e Pe spec i e, Leade ship and Policy in Schools, ISSN
1744-5043, Taylo & F ancis, London, Vol. 23, Iss. 2, pp. 253-274,
h ps://doi.o g/10.1080/15700763.2022.2129073
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/319279
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h ps://c ea i ecommons.o g/licenses/by-nc/4.0/
SCHOOL INSPECTORS’S DECISION-MAKING
1
School Inspec o s' Decision-making: E idence om a Compa a i e
Pe spec i e
Pablo Bezem a, Anne Piezunka b & Rebecca Jacobsen c
a Ma y Lou Ful on College o Teaching and Lea ning Inno a ion, A izona S a e Uni e si y
b WZB Be lin Social Science Cen e & Goe he-Uni e si ä F ank u
c College o Educa ion, Michigan S a e Uni e si y
ABSTRACT
In an e a o es -based accoun abili y, school inspec ions can o e a mo e nuanced unde s anding o why
schools ail. Ye , we ha e limi ed knowledge o how inspec o s a i e a hei decisions on school quali y.
Analyzing inspec o s' decision-making can e eal he unde lying iews ega ding school accoun abili y and
open oppo uni ies o school imp o emen . We use a compa a i e case s udy o con as ing inspec ion
sys ems in he Uni ed S a es, Ne he lands, and A gen ina. Based on in-dep h in e iews wi h inspec o s,
ou indings e eal ha inspec o s’ sensemaking and decisions a e s ongly in luenced by local cul u e,
p o essional adi ions, and iews on school accoun abili y. These con as ing p ocesses illus a e ade-o s
be ween igid and lexible app oaches o school inspec ion ha ha e consequences o school imp o emen .
Keywo ds: accoun abili y; school quali y; school imp o emen ; school inspec ion, sensemaking
This is an Accep ed Manusc ip o an a icle published by Taylo & F ancis in Leade ship and Policy in
Schools on 06 Oc 2022 (2024, Volume 23, Issue 2, pp. 253-274), a ailable a :
h ps://doi.o g/10.1080/15700763.2022.2129073.
I is deposi ed unde he e ms o he C ea i e Commons A ibu ion-NonComme cial License
(h ps://c ea i ecommons.o g/licenses/by-nc/4.0/), which pe mi s non-comme cial e-use, dis ibu ion, and
ep oduc ion in any medium, p o ided he o iginal wo k is p ope ly ci ed.
SCHOOL INSPECTORS’S DECISION-MAKING
2
In oduc ion
High-s akes es ing inc easingly plays a cen al ole in school accoun abili y and imp o emen in
many coun ies. This is d i en, in pa , by an audi cul u e ha emphasizes pe o mance
measu emen (Apple, 2005; Cla ke & Ozga, 2011). An appealing ea u e o es -based
accoun abili y is he pe cep ion ha i objec i ely measu es educa ional pe o mance and allows
compa isons ac oss dis ic s and ime (Bloem 2015). Ye , his ypically does no p o ide nuanced
in o ma ion o iden i y why ce ain schools all behind; i does no cap u e he my iad ac o s ha
in luence school quali y (e.g. Da ling-Hammond e al. 2016; Gagnon & Schneide 2019). As a
s and-alone policy, es -based accoun abili y o en incen i izes schools o na owly ocus on
es ed subjec s (e.g. Fi che and Hea ne 2010; Jacob 2005) and s a egies o boos sco es, which
migh no p omo e subs an i e lea ning (Au ho names, 2008).
School inspec ion (SI) is an al e na i e app oach o moni o ing and imp o ing school
quali y. Unlike es -based accoun abili y, school inspec ion is no limi ed o s anda dized es s o
e alua e s uden pe o mance. Inspec ion e alua ions equen ly assess a a ie y o school
a ibu es h ough obse a ion and di ec in e ac ion wi h school s akeholde s, such as eache s,
s uden s, and pa en s ( an B uggen, 2010). In his way, inspec ions can p o ide a summa i e
assessmen o o e all school quali y while also unco e ing ac o s ha help o hinde school
imp o emen . The eby, i se es as an al e na i e mechanism o achie e mo e nuanced
accoun abili y and insigh s o imp o emen s.
Despi e i s wide use a ound he wo ld, ou unde s anding o how inspec ion is ca ied ou
emains limi ed. Inspec ion p ac ices a y conside ably ac oss se ings ( an B uggen, 2010),
o en in ela ion o unde lying belie s and assump ions abou he ole o accoun abili y in school
imp o emen . These iews o school accoun abili y can in luence he wo k o inspec o s in ways
SCHOOL INSPECTORS’S DECISION-MAKING
3
ha we do no ye unde s and. Fo example, a manage ial app oach o accoun abili y would ocus
on he alidi y o e alua ions and school e ec i eness h ough high-s akes s anda dized
assessmen s (Sinclai , 1995). In his si ua ion, school inspec o s may eel cons ained o “s ick o
he sc ip ” and in e p e hei ole mo e na owly, a ge ing hei ocus while isi ing a school.
O he app oaches o accoun abili y emphasize insigh s om eaching p o essionals, wi h less
conce n abou a oiding bias in inspec ion esul s. In hese cases, inspec o s may ake on a mo e
induc i e app oach o hei wo k whe e hey use hei expe ise and backg ound o guide hei
inspec ions as hey look o eme gen issues. To unde s and how inspec ion wo ks, i is c i ical o
analyze inspec o s’ decision p ocess and how school quali y is judged. A high deg ee o
a ia ion exis s ac oss sys ems, ye limi ed wo k has sough o illumina e how con ex ual ea u es
shape how inspec o s wo k. Examining inspec o s’ decision-making can e eal iews ega ding
school accoun abili y and pe cei ed legi imacy o he inspec ion p ocess and eedback.
In ou s udy, we ocus on he indi idual pe spec i es o inspec o s. Unde s anding
inspec o s’ hinking and how pe sonal pe spec i es a e u ilized can shed ligh on his p ocess.
This s udy poses wo ques ions: (1) Wha a e he sou ces guiding inspec o s’ hinking du ing
inspec ions? and (2) How do he pe sonal pe spec i es o inspec o s in luence school
e alua ions? To add ess hese ques ions, we de elop a compa a i e case s udy o inspec ion in a
US school dis ic , a p o ince in A gen ina, and he Ne he lands. These cases di e ac oss h ee
dimensions: (i) he le el o s akes a ached o inspec ion, (ii) inspec ions’ ole wi hin he
accoun abili y sys em, and (iii) inspec o s’ p o essional backg ound. No p e ious s udy has
e alua ed wha d i es inspec o s’ decision-making om an in e na ional compa a i e
pe spec i e.
SCHOOL INSPECTORS’S DECISION-MAKING
4
Li e a u e Re iew: Inspec o s’ Decision-making
Despi e he ac ha inspec ion sys ems ha e long exis ed a ound he wo ld, mos p e ious
esea ch ocuses na owly on Eu opean sys ems. Fu he mo e, he wide a ie y o SI
a angemen s ha exis makes i challenging o build a cohe en body o li e a u e ha con e ges
on key indings (de Wol & Janssens, 2007; Eh en, 2016; Kle ks, 2012). The local na u e o
inspec ion has ein o ced a endency o ocusing inspec ion esea ch on local sys ems, which is
o en hen published in coun y-speci ic jou nals in he na i e language. SI esea ch published in
mo e widely ead jou nals, in he English language, has expanded du ing he las i e yea s. Ye ,
he li e a u e emains limi ed and agmen ed. The empi ical li e a u e ha does exis has ended
o ocus on he e ec s and side e ec s o SI (e.g. Al ich e & Keme ho e 2015; Kle ks 2012).
Wi hin his eme ging body o li e a u e, limi ed empi ical esea ch has cen e ed on school
inspec o s hemsel es and hei in luence on he e alua ion p ocess. Mos ea ly s udies published
in English we e conduc ed p ima ily in he UK. Despi e using a highly s anda dized inspec ion
p ocedu e and a eliable sys em o class oom obse a ion (Ma hews e al., 1998), a ious s udies
conclude ha he p o essional judgmen o OFSTED inspec o s played a key ole in hei
e alua ions (Gil oy & Wilcox, 1997; Lee & Fi z, 1997; Woods & Je ey, 1998). I was ound
ha inspec o s’ eedback o schools is in luenced by pe cei ed cons ain s ha he local con ex
imposes on eache s (Woods & Je ey, 1998). In addi ion, inspec o s’ p o essional backg ound
impac s hei judgmen . Fo example, p io expe ience se ing as a class oom eache can
inc ease empa hy and a sense o collegiali y wi h eache s (Bax e , 2013; Mille & Johnson,
1998).
Since p o essional judgmen plays a ole in inspec o s’ decision-making, i is ele an o
explo e how indi idual judgmen a ies. Silcock and Wyness (1998) shed some ligh on he
SCHOOL INSPECTORS’S DECISION-MAKING
5
issue, inding a wide di e si y in inspec o belie s abou educa ion and cu en e o ms as well as
hei empa hy wi h challenges aced by eache s. These di e ences we e appa en despi e
s anda dized aining and e alua ion ools. This ea ly esea ch demons a ed ha p o ound
di e ences in co e belie s ega ding educa ion can pe sis in a highly s anda dized sys em.
Recen s udies ha e ocused on he p ocess o judgmen o ma ion in SI sys ems, whe e
eedback is decided h ough consensus among a g oup o inspec o s (Dede ing & Sowada, 2017;
Lindg en, 2015; Ru z e al., 2017). Despi e he use o p o ocols and s anda ds, inspec o s ha e
some disc e ion, bo h as indi iduals and he o e all g oup, when wo king owa ds a consensus
and making decisions (Dede ing & Sowada, 2017; Ru z e al., 2017). Lindg en (2015)
demons a es ha in he highly s anda dized Swedish sys em, he e is a s a k con as be ween
how decisions a e o med du ing he inspec ion p ocess ( he “backs age” o inspec ion) e sus
how inal eedback is p esen ed o he school and communi y ( he “ on s age”). E en when
inspec o s p esen ha d e idence o jus i y decisions in he “ on s age,” he e is nego ia ion
among inspec o s in he “backs age,” whe e hei judgmen s encompass a mix o unce ain y,
adap a ion, and c ea i i y.
These indings demons a e ha he human elemen and p o essional judgmen emain
cen al in he inspec ion p ocess, ega dless o e o s o s anda dize p ocesses and p ocedu es.
Despi e knowing ha a ia ion does occu , i is no ye unde s ood how speci ic pe sonal aspec s
o he inspec o s and ins i u ional ea u es o he school sys em in luence he inspec ion p ocess.
This s udy aims o p o ide ini ial insigh s in o his c i ical aspec o inspec ion sys ems.
SCHOOL INSPECTORS’S DECISION-MAKING
6
Theo e ical F amewo k
Ou esea ch d aws on sensemaking heo y o analyze how inspec o s a i e a hei decisions on
school quali y (Weick, 1995). In addi ion, di e en app oaches o school accoun abili y help o
iden i y di e ences be ween ou h ee cases.
Sensemaking Theo y
Sensemaking is a way o unde s anding how indi idual ac o s comp ehend an unknown o
ambiguous si ua ion, make meaning o i , and hen ac based on his in e p e a ion (Weick,
1995). The eby, hey e e o a cogni i e amewo k ha consis s o cogni i e s uc u es—such
us belie s, knowledge, and a i udes— hei si ua ion, and policy signals (Spillane e al., 2002).
A g owing body o li e a u e in educa ion d aws on his heo y o unde s and eache s’
and adminis a o s’ in e p e a i e amewo ks when enac ing educa ional policies (e.g. Cobu n
2005; Hal e son e al 2004; Rigby 2015; Spillane e al 2002). Sensemaking is he p ocess in
which indi iduals e ospec i ely s uc u e he unknown, u ning ci cums ances in o si ua ions
hey comp ehended, o a ionalize hei ac ions (Weick e al., 2005). Educa ional s udies ha
d aw on his app oach ha e add essed how his p ocess is in luenced by p eexis ing wo ld iews,
p io knowledge, expe ience, o mal and in o mal ne wo ks, and he o ganiza ional and social
con ex wi hin which sense-make s wo k (Ball & Bowe, 1992; Cobu n, 2001; Hill, 2001; Po ac
e al., 1989; Spillane e al., 2002).
Sensemaking li e a u e ela ed o policy implemen a ion has ocused on how knowledge
s uc u es a e accessed and applied in p ac ical si ua ions. One inding is ha obse a ions made
by indi iduals who implemen policy can o en ocus on he supe icial aspec s o a si ua ion ha
hen igge a memo y o ano he si ua ion. This jeopa dizes he abili y o di e in o he deepe
signi icance o wha is obse ed (Spillane e al. 2006). This li e a u e has also ound ha
SCHOOL INSPECTORS’S DECISION-MAKING
7
indi idual easoning abou a complex judgmen ends o be biased owa d in e p e a ions ha a e
consis en wi h hei belie s and alues (Spillane e al., 2002). Sensemaking p ocesses a e
media ed by conside a ions abou o ganiza ional s uc u es (e.g. wo k en i onmen , no ms, and
ules), p o essional a ilia ion and ne wo ks, and adi ions (e.g. Cobu n 2001;Spillane e al.
2006). Policy implemen a ion s udies ha e shown he ele ance o socially media ed
sensemaking. Fo example, when eache s implemen ins uc ional policies, sensemaking is
media ed by school leade s’ pa icipa ion in he in e p e a ion o he policies (Cobu n, 2005) as
well as by in e ac ions wi h o he eache s (Cobu n 2001; Hill 2001).
A sepa a e body o li e a u e in o ganiza ional s udies ocuses on sensemaking wi hin
o ganiza ions. Sensemaking heo y has been used in his ield o unde s and con using o
ambiguous e en s wi hin o ganiza ions (Mai lis & Ch is ianson, 2014; Sandbe g & Tsoukas,
2015; Weick, 1995). Simila o sensemaking esea ch on educa ion policy implemen a ion,
o ganiza ional s udies highligh he impo ance o cons uc ing in e subjec i e meaning, which
occu s when a ious ac o s wi hin an o ganiza ion, such as manage s and pee s, shape each
o he ’s unde s anding (Gioia e al., 1994).
Using hese pe spec i es and building upon pas esea ch, ou s udy d aws on
sensemaking heo y o unde s and how inspec o s in e p e si ua ions ha hey obse e in schools
and a i e a judgmen s ega ding school quali y. School inspec o s mus econcile go e nmen
guidelines, bes p ac ices, and inspec ion p o ocols wi h he si ua ions hey ind in he schools.
The e o e, sensemaking heo y p o ides use ul lens o unde s and his p ocess. Sensemaking is
likely media ed by inspec o s’ own expe ience and belie s abou educa ion, he in e ac ion wi h
o he inspec o s, and o ganiza ional cul u e. The sensemaking li e a u e p o ides use ul
cons uc s o cap u e he a ie y o ac o s ha in luence how inspec o s econcile policies wi h
SCHOOL INSPECTORS’S DECISION-MAKING
8
he p ac ice (e.g. Cobu n 2005; Hill 2001). While p o ocols do exis , he e is lexibili y o
inspec o s o use p o essional judgmen (Dede ing & Sowada, 2017; Gil oy & Wilcox, 1997;
Lindg en, 2015). The e o e, we ely on he sensemaking li e a u e o analyze how inspec o s
in e p e complex si ua ions obse ed a schools and how hey a i e a decisions.
School Accoun abili y F amewo ks
Se e al concep ualiza ions o accoun abili y exis , which a e in o med by dis inc heo e ical
lenses, including manage ial, p o essional, poli ical & public, and ma ke accoun abili y (Klein,
2020). These concep ualiza ions di e ega ding how quali y is measu ed (e.g. he deg ee o
s anda diza ion), he objec i e o quali y assessmen , he powe balance be ween he ac o s
in ol ed, and o whom schools a e held accoun able.
In he las i y yea s, many coun ies ha e inc easingly inco po a ed ins umen s o
manage ial accoun abili y o public policy (Klein, 2020). This app oach e e s o he
esponsibili y o an agen o ins i u ion owa ds a supe io on a delega ed ask (Lindbe g, 2013).
In educa ion, he manage ial app oach ocuses on schools ende ing accoun o hei
e ec i eness based on ex e nal s anda ds measu ed by ou comes. This app oach led o he
adop ion o high-s akes accoun abili y e o ms wo ldwide. In some cases, his na ow de ini ion
o accoun abili y became dominan (e.g. Molle , 2009; Ranson, 2003; Apple, 2005).
O he ele an app oaches o accoun abili y ha e been ypi ied as p o essional, poli ical
& public, and ma ke (Sinclai , 1995; Knapp & Feldman, 2012. See also Klein, 2020). Fi s ,
p o essional accoun abili y e e s o he adhe ence o s anda ds de ined by he p o ession. In
educa ion, his has been a adi ional iew in which eache s uphold s anda ds o a success ul
eaching p ac ice wi hin he p o ession, place s uden s’ needs a he cen e o hei wo k, and
hold each o he accoun able (Molle , 2009, Klein, 2020). Second, poli ical & public
SCHOOL INSPECTORS’S DECISION-MAKING
15
ansla ed in o English. In e iews om he Ne he lands and he US si es we e ansc ibed and
checked o accu acy. Pa icipan s we e in o med ha in e iew esponses we e anonymous,
ansc ip s would no be sha ed, and a pseudonym would be used o ci e hem. In he Uni ed
S a es and A gen ina, inspec o s we e gi en a US$ 25 gi ca d a e pa icipa ion. In he
Ne he lands, we we e ad ised by SI esea che s no o o e incen i es. Pa icipan s in each s udy
si e e lec cha ac e is ics o inspec o s in hei loca ion wi h espec o yea s o expe ience and
demog aphics. Desc ip i e in o ma ion abou in e iewees is p esen ed in Table 1.
Table 1. Inspec o s’ Backg ound and Expe ience
Va iable
U.S. Case
n=8
Ne he lands Case
n=6
A gen ina Case
n=9
Indi idual inspec o s
Inspec o expe ience, in yea s
2.3
9.8
6.2
Educa ion expe ience, in yea s
14.3
22.8
31.0
Class oom eaching expe ience, in yea s1
8.9
10.7
13.1
Adminis a i e expe ience, in yea s1
2.7
7.0
15.2
% o inspec o s, wi hin case
% o me class oom eache s
100%
50%
100%
% o me adminis a o s
38%
20%
100%
% wi h g adua e deg ee
100%
75%
56%
1 Only hose inspec o s wi h expe ience as eache s/adminis a o s we e included in hese indica o s.
The in e iew ansc ip s we e coded using deduc i e and induc i e codes. Deduc i e
codes we e o mula ed based on ou heo e ical amewo k, mainly om concep s ela ed o
SCHOOL INSPECTORS’S DECISION-MAKING
16
sensemaking heo y (Cobu n, 2005; Mai lis & Ch is ianson, 2014; Spillane e al., 2002, 2006;
Weick, 1995). Induc i e codes s emmed om in e iews in he h ee si es. Responses we e
coded acco ding o Miles, Hube man, and Saldaña’s (2014) app oach o quali a i e analysis by
obse ing pa e ns and hemes wi hin and ac oss case s udies. We used DEDOOSE quali a i e
da a analysis so wa e o coding and analysis. To ensu e he eliabili y o codes, we used an
independen -code me hod. Fi s , in e iew ansc ip s we e independen ly coded by wo
esea che s, be o e hen compa ing he coding o ag eemen . We ollowed an i e a i e p ocess
un il a leas 75% ag eemen was achie ed.
We conduc ed wo ounds o coding. The i s ound ocused on: i) sou ces o
in o ma ion used du ing inspec ion, ii) use o local con ex in o ma ion, and iii) inspec o s’
de ini ion o good quali y educa ion. These codes we e de ined induc i ely. The second ound
elied mo e hea ily on deduc i e codes ha we e mo e abs ac and equi ed mo e in e p e a ion.
The main codes include i ) inspec o s’ pe cep ion o school adminis a o s, ) ype o
ecommenda ions, and i) sou ces guiding hinking. This la e code is emphasized in ou
analysis (Table 2). The sub-codes a e based on he inspec ion p ocedu es and sensemaking
heo y. The heo y was used o de ine he oci on he knowledge s uc u es accessed by
inspec o s when acing p ac ical si ua ions, especially when hey ha e o make sense o complex
si ua ions.
A e he coding p ocedu e was comple e, we iden i ied gene al and coun y-speci ic
pa e ns in he da a. The da a we e examined isually in he o m o c oss- abula ions, and cha s
showing he equency wi h which codes occu ed as well as he p esence o absence o codes
wi hin and ac oss in e iews, o each o he case s udies. To ensu e ha epo ed pa e ns a e an
SCHOOL INSPECTORS’S DECISION-MAKING
17
accu a e ep esen a ion o each s udy si e, we sha ed ini ial indings wi h in e iewees and
inco po a ed hei eedback (Miles e al., 2014).
Table 2. Sou ces Guiding Inspec o s’ Thinking
Sou ce
Desc ip ion
A
Indica o s o school quali y
S anda dized ub ics, indica o s, o me ics used o e alua e school
quali y du ing inspec ion
B
Mul i-in o man app oach
Simul aneous use o mul iple sou ces o in o ma ion - o he same
o di e en kind - o alida e e idence
C
In e ac ions among
inspec o s
In e ac ion among inspec o s o be ween inspec o s and echnical
pe sonal o discuss abou indings
D
Local con ex in o ma ion
Re e ences o local con ex in o ma ion, including s uden s'
demog aphics, cha ac e is ics o he neighbo hood, school his o y,
and change o s a
E
Inspec o s’ pe sonal
backg ounds
Re e ences pe sonal expe ience, belie s, o p o essional judgmen .
Findings
O e all, ou compa a i e analysis indica es ha pe sonal pe spec i es a e an impo an in luence
inspec o s’ e alua ions. The ex en o in luence is a ec ed by local cul u e, p o essional
adi ions, and iews on school accoun abili y. In pa icula , he case in he Uni ed S a es elies
on high-s akes es ing o accoun abili y and i s audi cul u e emphasizes pe o mance
measu emen and da a-d i en decision-making. The inspec o s’ hinking is in used wi h his
cul u e and hei decision-making is highly in luenced by a es -based accoun abili y mindse ha
leads o s ic ly adhe ing o p o ocols and educes inspec o s’ p o essional insigh s in an e o o
a oid bias. Despi e oppo uni ies p esen ed by SI o dig deepe and iden i y unique s eng hs and
weaknesses a schools, inspec o s ac i ely dis ega ded insigh s ha do no i wi hin he con ines
o he p o ocol. In con as , he Du ch and he A gen inean cases illus a e app oaches in which
inspec o s ha e mo e lexibili y and can ely on hei p o essional backg ound and judgmen .
SCHOOL INSPECTORS’S DECISION-MAKING
18
Inspec o s ou inely pu sued “su p ising” obse a ions e en when hese do no i nea ly in o a
p o ocol o he an icipa ed ocus a eas. As a esul , hese inspec o s adop a mo e holis ic
unde s anding o school quali y and a enues o imp o emen .
Applying he sensemaking-app oach, we p esen indings o he ca ego ies o sou ces
ha guide inspec o s’ hinking: (A) Indica o s o school quali y, (B) Mul i-in o man app oach
(C) In e ac ion among inspec o s, (D) Local con ex in o ma ion, and (E) Inspec o s’ pe sonal
backg ounds (see de ini ions in Table 2). Key indings ega ding he in luence o hese
ca ego ies on inspec o s’ hinking ac oss ou cases a e summa ized in Figu e 1. Fo quo ing
s udy pa icipan s, hei names include a subsc ip indica ing wha coun y hey ep esen : “-US”
o he Uni ed S a es, “-NETH” o he Ne he lands, “-AR” o A gen ina.
(A) Indica o s o School Quali y
In he U.S. case, indica o s o school quali y in he o m o a s anda dized ub ic a e he
co ne s one o he e alua ion p ocess and all o he inspec o s men ioned hem epea edly when
explaining hei hough p ocesses. Thei da a collec ion is s uc u ed a ound a ious aspec s o
school quali y, a se ies o me ics o measu e hem, guidelines o making obse a ions, and
ques ions o schools in o de o e alua e each indica o . We ound ha inspec o s in he US case
es ic da a collec ion almos exclusi ely o sou ces speci ied in he p o ocol: class oom
obse a ions, in e iews and ocus g oups, and school documen s. Only wo inspec o s
men ioned ha hey seek publicly a ailable school in o ma ion be o e hei isi (e.g. es esul s
and school websi e) o du ing he isi (e.g. eache s’ planning documen s, s uden s’ iles). This
s ands in s a k con as wi h he Du ch and A gen inean cases whe e de eloping a deep
knowledge o a school be o e he ac ual isi is a i al pa o he inspec ion p ocess.
SCHOOL INSPECTORS’S DECISION-MAKING
19
Figu e 1. Compa ison o Sou ces Guiding Inspec o s’ Thinking
U.S. Case
Ne he lands Case
A gen ina Case
Indica o s o school
quali y
Mul i-in o man
app oach
In e ac ions
inspec o s
Local con ex
in o ma ion
Inspec o s’ pe sonal
backg ounds
Legend: Deg ee o In luence on Inspec o s' Thinking
High
Low
No e: The ela i e in luence o each sou ce wi hin each case was de e mined by he numbe o men ions
in in e iews.
Mo eo e , in he US case, he p o ocol explici ly de e mines he s uc u e and na u e o
class oom obse a ions and he ques ions o be asked in in e iews and ocus g oups. Fo
example, when inspec o Amy-US was asked abou wha in o ma ion she pe sonally looks o o
asks o see apa om he equi ed da a, she emphasized he impo ance o adhe ing o he
p o ocol:
I jus ollowed he … p o ocol. I ask only he ques ions ha a e ou lined o eache s and only he
ques ions ha a e ou lined o s uden s. I a s uden [ esponse] needs o be elabo a ed, I would say
“could you ell me mo e …,” bu I don’ b ing my own ques ions o he p ocess o any hing like
ha . I jus ied o ollow wha is asked o me.
SCHOOL INSPECTORS’S DECISION-MAKING
20
We asked inspec o s abou he alue hey placed on di e en in o ma ion sou ces o e alua e
school quali y. We ound ha obse ing class oom ins uc ion is consis en ly he mos alued
sou ce o in o ma ion used by inspec o s o e alua e school quali y. In con as o he o he s udy
si es, mos inspec o s in he US case conside ed o he sou ces o in o ma ion o play a seconda y
ole, i conside ed a all. School planning documen s we e deemed he leas aluable sou ce o
in o ma ion o e alua e school quali y and we e used mos ly o iangula ion pu poses.
Simila ly, he use o school clima e obse a ions—such as cul u e, in e ac ions among s uden s
and eache s, and acili y condi ion–was also seconda y.
Mo e han in he o he s udy si es, when he inspec o s in he US case explained hei
hough p ocess, hey epea edly made di ec ema ks abou how hey y o a oid pe sonal bias
when comple ing he ub ic and ollowing how he p o ocol de ines good ins uc ion. This was
highligh ed by Sa ah-US and Donna-US:
Sa ah-US: We all come o he able wi h ou own expe ise, wi h ou own belie s and alues, wi h
ou own biases o p econcei ed no ions abou wha school should look like … he ub ic hen
helps people pu hose hings aside, unde s and hei in luence, and hen eally g ound hemsel es
in bo h e idence and he ub ic o ge o a sha ed unde s anding.
Donna-US: I 's eally impo an … as a e iewe , o no ha e a bias … I migh ha e a bias
owa ds wha good ins uc ion looks like. So ins ead o using he ub ic in on o me, I'm going
owa ds wha I hink is good. O I migh ha e a bias owa ds wha a unc ioning school
en i onmen looks like and sounds like. So ins ead o using he e idence in on o me, I'm jus
going owa ds wha I hink … I hink ha can be bo h posi i e and nega i e.
Ac oss all in e iews in he US case, we ound a igid emphasis on obse ing he p o ocol,
some imes in ways ha appea o impede impo an insigh s. This s ance seems consis en wi h
manage ial app oach o school accoun abili y which ocuses p ima ly on con olling schools
SCHOOL INSPECTORS’S DECISION-MAKING
21
h ough eliable measu e o school e ec i eness based on ex e nal s anda ds. In his si ua ion,
mos inspec o s s ill ecognized he ele ance o hei own expe ise and expe ience; hey
acknowledged which indica o s bes cap u e school quali y and hei p e e ed in o ma ion
sou ces. Ye , as shown below, in Sec ion E, inspec o s in he US case dismissed his weal h o
knowledge and equa ed d awing on hei pe sonal expe iences wi h bias. Inco po a ing such
in o ma ion in o e alua ions was iewed as a alidi y h ead o he inspec ion p ocess.
Inspec o s in he US case claim o ac i ely supp ess he in luence o hei educa ion knowledge
and expe ience du ing inspec ion by s a ing ha hey s ic ly adhe e o p o ocol. Fu he mo e,
unlike he o he esea ch si es, when explaining hei hinking in speci ic si ua ions, hey we e
a ely able o sha e conc e e examples based on hei own expe ience. Ins ead, hey epea edly
e e ed o “good p ac ices” ou lined in he ub ic.
Simila o he US case, Du ch inspec o s u ilize indica o s o school quali y in all hei
school isi s. Howe e , hese indica o s we e no he dominan in luence on inspec o s’
na a i es abou hei hinking p ocess. While he policies and p ocedu es used by he Du ch
sha es simila i ies wi h he US case, and i s QualiE p o ocol, Du ch inspec o s exp essed g ea e
lexibili y in he inspec ion p ocess, including s ages o da a collec ion, choosing which c i e ia
o ocus on and emphasize, and how o in e p e indica o s o school quali y. When collec ing
in o ma ion, Du ch inspec o s use s anda dized da a only as a s a ing poin . Ou in e iews
e ealed ha da a collec ion and usage a e guided by inspec o s’ choices o p obe mo e deeply
in o key a eas, as hei insigh s and unde s anding o a gi en school e ol e o e he cou se o he
isi . No consensus exis s among he Du ch inspec o s ega ding he mos aluable in o ma ion
sou ces. Se e al inspec o s ound his ques ion di icul o answe , in con as o he inspec o s in
he US case who p omp ly e e enced s anda dized p o ocols. Some Du ch inspec o s especially
SCHOOL INSPECTORS’S DECISION-MAKING
22
alued in e iews wi h eache s and adminis a o s as well as class oom obse a ions. Only a ew
men ioned school clima e o s uden in e iews. Du ch inspec o s we e mo e likely o iew he
use ulness o in o ma ion sou ces as highly con ex ual, based on hei expe ience, school
con ex , and speci ic issues ha eme ged du ing a school isi .
In con as o hese wo cases, in A gen ina, a s anda dized se o indica o s o school
quali y ha e almos no ole in he inspec ion p ocess. When indica o s a e collec ed, hey a e
mainly used o adminis a i e pu poses. Inspec o s do no use s anda dized me ics o e alua e
quali y sys ema ically. This is no o say ha he inspec o s lack s anda ds o e alua ing school
quali y. Ra he , hey ha e conside able eedom o decide wha in o ma ion o collec and how o
e alua e schools. Consis en wi h his eedom, we obse ed immense he e ogenei y in e ms o
in o ma ion sou ces used and which sou ces a e mos alued. Inspec o s emphasized he
impo ance o gaining an unde s anding o how he school unc ions, emphasizing he
impo ance o “being p esen ” in he school, “walking a ound,” and “li ing in he momen s o
he ins i u ional li e.” Being p esen allows inspec o s o be c i ical and p o ide suppo o he
school. Wi hou insigh s om school isi s, inspec o s do no eel hey could uly unde s and a
school. The e o e, hey would be unable o p o ide necessa y assis ance o help schools imp o e.
Planning documen s a e conside ed use ul o e alua ion and collabo a i e wo k be ween
inspec o s and schools.
Bias was a ely a conce n among inspec o s om A gen ina and he Ne he lands. In hese
wo cases, inspec o s did no hesi a e o make use o hei p o essional judgmen , expe ience as
ins uc ional expe s, and amilia i y wi h schools gained om mul iple isi s. Ra he han be a
cause o conce n, his was iewed as necessa y o be a good inspec o and exac ly wha enables
hem o be e ec i e. Fu he mo e, in he Ne he lands, u ilizing his ype o knowledge is iewed
SCHOOL INSPECTORS’S DECISION-MAKING
23
as necessa y o be a good inspec o . Pa o he inspec o aining p ocess includes an ex ensi e
shadowing o expe ience inspec o s, whe e hose who a e new ha e an oppo uni y o u he
de elop and o lea n o use hei expe ise. This is no o say inspec o s in A gen ina and he
Ne he lands do no e lec c i ically on hei own p ac ices and main ain a conce n o in eg i y in
he p ocess. In bo h coun ies, nea ly all inspec o s exp essed spon aneously in he in e iew a
conce n abou being ho ough in hei analysis and he impo ance o jus i ying hei conclusions
in hei eedback o schools. Bu a emp s o s anda dize he p ocess and a conce n o bias and
eliabili y we e less p ominen in ou con e sa ions.
(B) Mul i-In o man App oach
Once inspec o s in he US case collec da a, hey a e u ned o e o QualiE whe e s a conduc
a s anda dized a ing p ocess. The p ocessed da a a e hen used o assess o e all ends and
pa e ns in he school. This mul i-in o man app oach seeks e idence ha is con i med
epea edly using he same ype o in o ma ion and hen iangula ed wi h o he sou ces. Fo
example, e idence om only one o wo class ooms is insu icien o make a claim. A i ma ion
mus be ound in mul iple classes and hen iangula ed wi h e idence om addi ional sou ces,
such as in e iews and school documen s. Many inspec o s highligh ed ha his app oach
p o ides a holis ic iew, as Michelle-US explains:
[The mul i-in o man app oach] eally gi es you ha holis ic iew o , “Okay, his is wha we saw
in he class oom. This is wha he eache s and s uden s a e doing.” Bu hen, wha a e people
ac ually saying abou i ? Wha a e he pa en s saying, he s uden s, and he s a ? And how did
hose s o ies suppo one ano he , o how a e hey di e en ?
Impo an ly, mos inspec o s exp essed con idence in he ocus on he ends, as opposed o
ou lie s, as a eliable app oach o e alua e he o e all quali y o he schools. O e all, mos
SCHOOL INSPECTORS’S DECISION-MAKING
24
inspec o s seem o emb ace his app oach as an e ec i e way o e alua e he quali y o he
school. As Lau a-US explained:
Bo h imes ha I' e done i [( he inspec ion)], i 's been eally clea , e en a e he i s hal o one
day, wha he ends a e. I 's been kind o shocking, because you hink, “Oh, ob iously, hese
class ooms [a e] di e en han ha .” No, hey' e ne e di e en . I 's all he same. I 's always been
eally shocking how quickly you can come o wha he big p oblem is. Usually, i 's ac ually been
p e y easy o pick he op wo o h ee, and because he schools ha hey pick o do hese hings
a e ... li e ally on i e, so i 's p e y clea .
In e iews demons a ed how sys ema ically he mul i-in o man app oach is applied: E idence
is ga he ed, inspec o s ocus on majo ends, and disca d in o ma ion ha does no i wi hin
hese b oad ends. Donna-US emphasize ha hey “a e looking o ends and consis encies,
e sus an ou lie o some hing ha migh s ike you as w ong.” Con adic o y in o ma ion
obse ed while da a is being collec ed migh ise a ed lag and can help na owing he ocus.
When his happens, he e is some leeway o u he inqui ies as long as he sea ch and sou ces
o in o ma ion o be used a e pa o he p o ocol. Ye , in con as o he o he esea ch si es, we
did no ind examples o inspec o s pu suing p o essional hunches ha lead o addi ional
ques ion being asked, no ocusing on excep ional obse a ions, no conduc ing addi ional
in e iews ha migh lead o new disco e ies.
Inspec o s in he Ne he lands ac as in es iga o s. Each has conside able eedom in
deciding which ocus a eas o emphasize du ing school isi s. Se e al inspec o s explained ha
a e e iewing school in o ma ion, p io o he isi , hey y o an icipa e he main di icul ies a
he school. Hypo heses a e de eloped ha hey hen seek o e i y o disp o e du ing he
isi a ion day. Se e al inspec o s no ed how hei expe ise can assis in de eloping hese ini ial
assessmen s. They ac i ely d aw upon hei p io knowledge and as expe iences wi h a wide
SCHOOL INSPECTORS’S DECISION-MAKING
31
(D) Local Con ex In o ma ion
Conside a ion o he “local con ex ” plays a mino ole in US inspec o s’ na a i es abou hei
hinking. Local con ex includes s uden demog aphics, neighbo hood cha ac e is ics, o a
school’s his o y. Se e al inspec o s explained ha a e an ex ended ime wo king in he dis ic ,
hey had in e ac ed wi h mos schools a some poin and became amilia wi h he local con ex
o he schools. Some o hem men ioned ha hey ha e his knowledge, bu hey iew i only as
backg ound in o ma ion, as illus a ed by Aidan-US who said ha his con ex ual in o ma ion is
“in he back o hei head.” Howe e , inspec o s did no men ion his ype o in o ma ion
ac o ing in o hei unde s anding o a school’s unc ioning. The in e iews did no e eal many
speci ic examples o how his knowledge in luences hinking du ing he inspec ion.
Conside a ions o school con ex a e no explici ly desc ibed in he o mal p o ocol. The e o e,
mos inspec o s ac i ely s o e o exclude his in o ma ion, s a ing ha “i should no ma e ” in
hei e alua ions. When asked o u he explana ion, some inspec o s highligh ed ha hey mus
closely ollow he ub ic and a oid bias, hus assuming ha he objec i i y o he p ocess migh
be comp omised by conside ing he con ex . Se e al inspec o s go u he by a guing ha
inspec ed schools a e low-pe o ming and hus majo di e ences in a school con ex a e no
p esen .
The A gen inean and Du ch inspec o s conside con ex ual in o ma ion a c i ical piece o
in o ma ion o unde s anding a school’s unc ioning. A gen ina is he only esea ch si e in
which each inspec o is pe manen ly assigned o a g oup o schools. The e o e, inspec o s
become deeply acquain ed wi h he local con ex , s uden demog aphics, school his o y, and
school s a . In mos in e iews, inspec o s highligh ed ha he inspec ions a e “si ua ed,” locally
o ien ed based on he school eali y. As poin ed ou by Alejand a-AR: “E e y hing you do in he
SCHOOL INSPECTORS’S DECISION-MAKING
32
school in based on he con ex , in he si ua ional aspec , ha is wha I ask o , ha he pedagogic
p ojec depa om he e.” To a g ea e ex en han he o he case s udies, inspec o s in e iewed
in A gen ina con inuously e e ence hei knowledge o school con ex when in e p e ing
p oblems, p io i izing in o ma ion sou ces, in e p e ing s uden pe o mance indica o s, and
de e mining ecommenda ions.
In he Ne he lands, inspec o s also wo k wi h a ixed g oup o schools. Ye , a e se e al
yea s, inspec o s swi ch g oups as a way o ensu e objec i i y. Du ch inspec o s exhibi ed mo e
knowledge o s uden demog aphics s uden s compa ed o U.S inspec o s. They also o e ed
de ailed desc ip ions o challenges aced by schools, wi h speci ic con ex ual ci cums ances such
a la ge immig an popula ion whe e lea ning Du ch was c i ical o pa en s ha ing low
educa ional capi al o suppo lea ning a home. This local knowledge was used by inspec o s o
in e p e a ious sou ces o in o ma ion collec ed du ing inspec ion. Fo example, inspec o s
conside whe he a school wi h a high p opo ion o immig an s uden s should de elop
p o isions o language educa ion in hei planning documen s. As S en-NETH explained,
I you ha e a school wi h pa en s who speak a home ano he language, he schools ha e o in es
mo e in cu iculum in ocabula y o Du ch o hose child en. Then, he expec a ion abou he
quali y o cu iculum a e di e en … You canno pu ha in o s ic c i e ia. ... [Ano he
example,] i you a e in a small school ha has o pu child en o se e al [g ades oge he ] in one
g oup, …. you know i 's a e y ha d job o he eache o o ganize he lessons in a way ha he
challenges all he child en … so, his kind o si ua ion plays in o he way you judge he quali y o
ins uc ion.
In he US case, he p o ocols do no include con ex as pa o hei e alua ion. Explici
conside a ion abou how local con ex in luences e alua ion o school quali y played a e y
SCHOOL INSPECTORS’S DECISION-MAKING
33
mino ole in inspec o s’ na a i es. This con igu a ion downplays he ole o si ua ional ac o s
ha migh spa k sensemaking p ocesses in inspec o s (Sandbe g & Tsoukas, 2015).
(E) Inspec o s’ Pe sonal Backg ounds
We ind ha inspec o s in he US case p io i ize objec i i y and eliabili y; he e is an emphasis
on s anda dized ub ics, which cons ains he use o pe sonal and p o essional knowledge. Ye ,
nea ly all inspec o s in he US case belie e hei backg ound in educa ion p o ides a necessa y
quali ica ion o hei ole. All in e iewed inspec o s had expe ience as class oom eache s and
hei wo k a he dis ic Depa men o Educa ion in ol es e alua ion o class oom ins uc ion
(see Table 2). Fewe inspec o s (less han hal ) ha e expe ience as a school adminis a o ; among
hose who do, he a e age expe ience is less han h ee yea s. Some inspec o s no ed ha an
ins uc ional backg ound was necessa y because hey knew wha o look o du ing school isi s.
This was illus a ed by Sa ah-US:
I hink ha ha ing [an] ins uc ional backg ound is c i ically impo an … being an educa o ,
someone who is highly amilia wi h he ins uc ional aspec o educa ion … olks who ha e ha
ins uc ional-speci ic lens, who ca y wi h hem he lens o wha high-quali y eaching and
lea ning looks like. I you know wha i means o s and up in on o s uden s and deli e
ins uc ional con en and assess s uden s. I mean he e's a lo o inside language in he ub ics. …
You ha e o know wha you a e looking o , so you ha e o know wha eaching looks like.
In addi ion, h ee o he in e iewees had expe ience as adminis a o s and hey belie ed his was
impo an p epa a ion o hei ole as an inspec o . Lisa-US explained how he judgmen is
in o med by adminis a i e expe ience, which can p o ide a mo e sys emic iew:
I eally hink ha my yea s as assis an p incipal helped me because I'm able o see he school as
an en i e sys em and no jus as one speci ic pa , and so I hink ha 's a g ea quali ica ion [ o ]
…. unde s anding ... So i some class ooms a e ha ing ange managemen issues, i i 's no a
SCHOOL INSPECTORS’S DECISION-MAKING
34
end ac oss he en i e school, i he e's a bigge end a ising ha he ins uc ion isn' igo ous,
ha ha 's a bigge ocus o he school in ying o wo k wi h some indi idual class ooms.
Expe ience wo king in he dis ic o ice is addi ional ac o ha inspec o s eel p epa es hem.
Se e al inspec o s men ioned ha his expe ience allows hem o “ha e a sense o wha he
schools look like.”
We ound ha in mos cases, inspec o s ely on hei p o essional judgmen in ways ha
a e wi hin he scope o he p o ocol, ye he app oaches migh no be explici ly s a ed in he
p o ocol. Fu he mo e, in some cases, in e iews e ealed a ension be ween using p o essional
judgmen o comple e he ub ic and main aining an unbiased and uni o m p ocess. This ension
was illus a ed by Michelle-US who no ed how she mus econcile he e alua ion ub ic wi h he
wide a ie y o elemen s she pe sonally conside s du ing class oom obse a ion:
Michelle-US: When I'm in he class oom, I look o s uden engagemen , and com o , and
lis ening, and lea ning. … I hink, because oo o en we can ocus jus on he eache s o he
adul s. … we eally ha e o look a he kids. When you' e in a school en i onmen , i 's holis ic…
you' e using you senses, igh ? You' e looking, and you' e eeling, and you' e hea ing, and all o
hese di e en hings ha you ge when you' e in a place ha is no necessa ily on any ub ic, bu
you ge he ibe, and he eeling o i . And hen, you kind o couple ha wi h wha people a e
saying in he in e iews, and wha hei body language is, and hei emo ional le el, and how hey
espond o hings.
In e iewe : … how do you pu all o his oge he in he ub ic and he eedback o he schools?
Michelle-US: Well, hose hings I was jus sha ing, I do pe sonally. So, hose a en' necessa ily on
he ub ic. Bu I hink ha 's wha comes ou when you' e collabo a ing wi h a eam. You don'
wan o be biased o make commen s based on pe sonal opinion.
Finally, we ound ha when c a ing eedback o inspec ed schools, inspec o s in he US case
use hei judgmen mainly o diagnos ic pu poses. Unlike he o he si es, we did no ind many
SCHOOL INSPECTORS’S DECISION-MAKING
35
explici conside a ions ega ding how eedback and ou comes om he inspec ion a ec he
schools. This shows ha inspec o s’ sensemaking p ocess is delimi ed by he scope p esc ibed by
he p o ocols and is essen ially e ospec i e mo e han p ospec i e (Gioia e al., 1994; Sandbe g
& Tsoukas, 2015).
In di ec con as o he U.S. case, inspec o s o he Ne he lands and he A gen inean case
ely mo e hea ily on hei pe sonal pe spec i es. In he A gen inean case, inspec o s sha ed
openly he ways ha hei pe sonal expe ience, belie s and p o essional judgmen in luence
mul iple aspec s o inspec ion. The alue placed on his weal h o knowledge migh be due o
inspec o ’s posi ions being he highes s ep in he eaching p o essional ladde in A gen ina.
Mo e han in he o he esea ch si es, inspec o s equen ly made explici ema ks abou how
hey ely on hei expe ience o in o m decisions. This was illus a ed by Alejand a-AR explaining
ha he ecommenda ions o schools a e no only based on go e nmen no ms:
Based on wha he educa ional policy is posing, bu mixed wi h my pe spec i e and s ance, wha
I’ e lea ned all hese yea s. Ob iously, he educa ional policy gi es you a amewo k in many
ega ds. You can’ s ay om wha is s ipula ed. Bu wi hin hese limi s, my expe ience and
knowledge a e also impo an when he ime comes o make sugges ions.
Inspec o s in he A gen inean case did no shy away om equen ly explaining how hei
pe sonal pe spec i es in luence hei hough p ocess. Se e al inspec o s explained ha he
p ocess is in o med by hei iews on wha hey conside c i ical issues in educa ion. Inspec o s
see hei ole as mo e poli ical, as se e al explained how hey ac as a b idge be ween he mac o
and he mic o policies o schools.
Since inspec o s om A gen ina de e mine hei p ocedu es, in con as o he o he
cases, hey ha e conside ably mo e leeway o use hei pe sonal judgmen . As in he case o he
US dis ic , inspec o s ely on hei expe ience as eache s o judge eaching quali y in he
SCHOOL INSPECTORS’S DECISION-MAKING
36
class oom. This was exp essed, by Ma celo-AR who said ha “class oom p esence, [allows you]
o e i y he p ocesses…. A e so many yea s, you us you in ui i e knowledge. And you can
ealize e y quickly whe he he kid lea ned o no wha he should ha e.” Bu unlike he US
case, in he A gen inean case, inspec o s’ use o p o essional judgmen goes beyond class oom
ins uc ion and includes a wide ange o aspec s o ins i u ional li e, including obse a ions o he
clima e, in e ac ions among eache s, and he ela ionships o he schools wi h he amilies and
he communi y.
Also, inspec o s in he A gen inean case end o use hei judgmen o a wide ange o
issues. Since inspec o s make ecommenda ions o in e en ions in schools, hey a e obliga ed
o go a s ep u he and make ecommenda ions ega ding how o co ec he p oblems iden i ied.
They mus judge which p ac ices a e likely o be e ec i e a a gi en school. We ound ha
in e iew exce p s in he A gen inean case coded as “pe sonal expe ience and belie s” show high
co-occu ence wi h he pa en codes “ ecommenda ions o he schools” and “ esponses o
s uggling schools.” In his ega d, hey seem o ollow a p o essional app oach o accoun abili y,
in which “ eache s” a e keeping o he eache s accoun able and p o iding insigh s o
imp o emen , placing s uden needs a he cen e o hei wo k. This di e s om he US case,
whe e inspec o s only e alua e aspec s included in he p o ocol and es ic hei pe sonal
judgmen as eache s.
In he Ne he lands, inspec o s’ leeway o manage and di ec he inspec ion p ocess wi hin
hei amewo k, o e s oppo uni ies o elying on hei pe sonal p e e ences and using hei
p o essional judgmen . In his p ocess, se e al inspec o s explained how hey de e mine wha he
p oblems a e, elying on hei expe ise and “gu eeling.” Some inspec o s make a dis inc ion
SCHOOL INSPECTORS’S DECISION-MAKING
37
be ween “ha d da a” ound in school s a is ics and documen s and “so da a” ha is mo e elian
on hei judgmen . This was illus a ed by La s-NETH:
Some [documen s a e] jus esul s, like how much o he child en a e on he igh le el when
hey’ e le p ima y school and a e in seconda y school now. So you can’ a gue wi h ha … you
can a gue abou … “how did you come o hese esul s?” Tha ’s he ha d pa , bu he o he pa s,
he so pa s, like gi ing chances o child en… hose a e no always in he pape s, so you can
only see ha in [pe son], when you’ e a he school, and well some imes you can ge a eeling o
how i should be a he school… i ’s a bi o a gu eeling ...
All he Du ch inspec o s p o ided examples o how hey ely on hei p o essional judgmen o
in o m he p ocess and decide on inal eedback o schools. Howe e , Du ch inspec o s’
na a i es abou hei hinking p ocess did no explici ly e e o hei p io p o essional
expe ience as hea ily as he inspec o s in he A gen inean case.
We ound ha inspec o s in he A gen inean case and he Ne he lands end o p o ide
holis ic judgmen s o school quali y and a e mo e ocal abou hei pe sonal iews ega ding
highe -le el goals o educa ion, wha is good quali y educa ion, and how schools should
unc ion. In hei na a i es, hei hinking is media ed by holis ic judgmen s ocused on wha
hey belie e is impo an o a school. In he Ne he lands, o example, o e alua e he quali y o
he school, La s-NETH asks “wha is impo an o he kids?” and “wha is he school
adminis a ion doing o gi e he bes educa ion hey can?” And Lo e-NETH asks how she “would
eel i she had kids in he school.” In he A gen inean case, when he e is a speci ic con lic
si ua ion in a school, Ca los-AR lis ens o s uden s and ies o iew he si ua ion om hei
pe spec i e. In he US case, inspec o s a oid men ioning his ype o hinking, which hey ea
poses a isk o in oducing bias.
SCHOOL INSPECTORS’S DECISION-MAKING
38
Unlike he US case, A gen inean and Du ch inspec o s’ na a i es abou hei hinking
no only ocus on diagnosing he cu en si ua ion, bu also how he school has p og essed and
how he eedback migh a ec he school. The e o e, inspec o s’ sensemaking p ocess is bo h
e ospec i e and p ospec i e. Familia i y wi h schools om p e ious inspec ions acili a es he
p ospec i e emphasis (Kaplan & O likowski, 2013). Fo example, we ound ha inspec o s in
bo h coun ies use hei pe sonal knowledge o s akeholde s as an indica o o school quali y. In
A gen ina and he Ne he lands, nea ly hal o inspec o s belie e ha a key indica o o school
quali y is hei “con idence ha he school adminis a o s unde s and and add ess he main
p oblems aced by he school,” o mo e gene ally, “ us in he adminis a o s.” This was
illus a ed by S en-NETH when he was asked how he esponds when a school aces weakness, bu
is no ailing:
I hink ha has o do wi h us . Then you y o p edic he u u e. You look a he quali y o he
s a and he quali y o he managemen and you ask you sel he ques ion “i hey a e no a he
le el hey ha e o be a he momen , bu do I us imp o emen p ocess, do I hink he
imp o emen p ocess will go on and hey will eally imp o e, he quali y educa ion will imp o e
in one o wo yea s.”
The ac ha inspec o s om A gen ina and he Ne he lands highligh us in school
adminis a o s as a key indica o o school quali y migh be a ibu able o sus ained ela ionships
be ween inspec o s and school s akeholde s. In his way, hese coun ies di e om he US case,
whe e he inspec ion p ocess is designed o a oid epea in e ac ions o inspec o s wi h he same
schools, as a way o enabling an objec i e p ocess.
Discussion
This s udy sheds ligh on school inspec o s’ sense-making and decision-making. I examines he
sou ces guiding inspec o s’ hinking and how pe sonal pe spec i es in luence school e alua ions.
SCHOOL INSPECTORS’S DECISION-MAKING
39
We ind a adeo be ween igid and lexible inspec ion app oaches. Mo e igid p o ocols aim o
educe pe sonal bias and can achie e g ea e compa abili y o esul s ac oss schools. Ye ,
lexible app oaches can allow mo e de ailed unde s anding o indi idual schools – iden i ying
oo causes o conce ns.
The US case emphasizes ollowing p o ocols and achie ing g ea e compa abili y o
esul s ac oss schools. This logic guides inspec o hinking h oughou he inspec ion p ocess. In
con as , inspec ion in he Du ch and he A gen inean cases aims o add ess complexi y by
o e ing lexibili y o inspec o s o exe cise p o essional judgmen . As a esul , inspec o s can
examine oo causes o obse ed conce ns and p o ide schools wi h mo e speci ic guidance o
imp o emen . Ye , such lexibili y educes he compa abili y o esul s ac oss schools and isks
in oducing inspec o s’ biases in o he p ocess.
We ind ha sensemaking mechanisms shape inspec o s’ e alua ions in di e en ways a
each o he h ee s udy si es. Oppo uni ies o e e ing o indi idual knowledge and belie s a e
limi ed in he US case by s ic p o ocols and dis ega d o complexi y. P o ocols aim o a oid
pe sonal bias, a he expense o de ailed unde s anding o indi idual schools. Da a collec ion
emphasizes consis ency by adhe ing o an e alua ion ub ic. In o ma ion collec ed du ing
inspec ion o ms he basis o de e mining o e all “school ends.” Con lic ing o incomple e
e idence does no lead o u he inqui y. Inspec o s do no aim o indi idually cap u e a
comple e pic u e o schools. While inspec o s mus each a o mal consensus, ypically
inspec o s ag ee wi h o e a ching indings. They acknowledge ha hey ha e limi ed
in o ma ion o dispu e hese indings. The in e subjec i e cons uc ion o meaning among
inspec o s is limi ed. In con as , indi idual and socially-media ed sensemaking play a key ole
in he cases o A gen ina and he Ne he lands. The e alua ion p ocess elies hea ily on inspec o
SCHOOL INSPECTORS’S DECISION-MAKING
40
pe spec i e, expe ience, and in ui ion, as well as local con ex in o ma ion. They aim o o m a
holis ic iew o school quali y and conside how inspec ion eedback may in luence a school’s
ajec o y. Fu he mo e, complexi ies a e add essed h ough local con ex in o ma ion,
co obo a ing in o ma ion sou ces, and consul ing wi h colleagues. In es iga ing hese
complexi ies ul ima ely in luence he e alua ion ocus.
The in e na ional compa ison shows ha hese con as ing app oaches o inspec ion a e
in luenced by he local cul u e and p o essional adi ions, which a e associa ed wi h hei iews
on school accoun abili y. The h ee cases ep esen con as ing app oaches o school
accoun abili y. The Uni ed S a es has one o he mos de eloped sys ems o manage ial
accoun abili y in he wo ld (Figlio & Loeb, 2011). This sys em o es -based accoun abili y has
o ged a pa h dependence in educa ional ins i u ions, bo h wi hin adminis a i e s uc u es and
s ee -le el p ac ices (McDonnell, 2008, 2013; see also Spillane e al., 2011). Findings om his
s udy show ha eliance on manage ial app oaches has led o a mindse ha in luences how
inspec o s na iga e he e alua ion p ocess. In his con ex , he mission o inspec o s is o ill he
e alua ion ub ics as an ins umen o con ol, o keep schools accoun able. The Du ch case akes
a mo e eclec ic iew o school accoun abili y. On one hand, his educa ional sys em has
inco po a ed elemen s o manage ial accoun abili y: es -based accoun abili y, use o ex e nal
s anda ds, and con ol o e schools. On he o he hand, he coun y has a long adi ion o school
inspec ion ha elies on p o essional s anda ds, p o iding insigh s beyond he s anda ds and
pu ing child en a he cen e o he inspec ion p ac ice. In his con ex , he quali y c i e ia o
inspec ion a e a s a ing poin o del e in o complex issues and p o ide insigh s o
imp o emen . The A gen ina case poses a s a k con as o his manage ial app oach. He e,
iews on accoun abili y a e shaped by p o essional backg ound since all inspec o s a e eache s
SCHOOL INSPECTORS’S DECISION-MAKING
47
Miles, M. B., Hube man, A. M., & Saldaña, J. (2014). Quali a i e da a analysis: A me hods
sou cebook (3 h ed.). SAGE Publica ions.
Mille , A., & Johnson, D. C. (1998). Expe ise o “baggage”? Wha helps inspec o s o inspec
p ima y ma hema ics? B i ish Educa ional Resea ch Jou nal, 24(5), 503–518.
h ps://doi.o g/10.1080/0141192980240502
Mølle , J. (2009). School leade ship in an age o accoun abili y: Tensions be ween manage ial
and p o essional accoun abili y. Jou nal o Educa ional Change, 10(1), 37–46.
h ps://doi.o g/10.1007/s10833-008-9078-6
Nusche, D., B aun, H., Halász, G., & San iago, P. (2014). OECD Re iews o E alua ion and
Assessmen in Educa ion: Ne he lands 2014. OECD.
h ps://doi.o g/10.1787/9789264211940-en
OECD, O. o E. C. and D. (2015). Educa ion a a glance 2015 - OECD Indica o s.
h ps://doi.o g/10.1787/19991487
Phillips, D., & Schweis u h, M. (2014). Compa a i e and in e na ional educa ion - An
in oduc ion o heo y, me hods , and p ac ice (2nd ed.). G oup, Con inuum In e na ional
Publishing.
Po ac, J. F., Thomas, H., & Baden-Fulle , C. (1989). COMPETITIVE GROUPS AS
COGNITIVE COMMUNITIES: THE CASE OF SCOTTISH KNITWEAR
MANUFACTURERS. Jou nal o Managemen S udies, 26(4), 397–416.
h ps://doi.o g/10.1111/j.1467-6486.1989. b00736.x
Ranson, S. (2003). Public accoun abili y in he age o neo‐libe al go e nance. Jou nal o
Educa ion Policy, 18(5), 459–480. h ps://doi.o g/10.1080/0268093032000124848
Rigby, J. G. (2015). P incipals’ sensemaking and enac men o eache e alua ion. Jou nal o
SCHOOL INSPECTORS’S DECISION-MAKING
48
Educa ional Adminis a ion, 53(3), 374–392. h ps://doi.o g/10.1108/JEA-04-2014-0051
Rouleau, L. (2005). Mic o‐p ac ices o s a egic sensemaking and sensegi ing: How middle
manage s in e p e and sell change e e y day. Jou nal o Managemen S udies, 42(7), 1413–
1441.
Ru z, S., Ma hew, D., Robben, P., & Bon , A. (2017). Enhancing esponsi eness and
consis ency: Compa ing he collec i e use o disc e ion and disc e iona y oom a
inspec o a es in England and he Ne he lands. Regula ion & Go e nance, 11(1), 81–94.
h ps://doi.o g/10.1111/ ego.12101
Sandbe g, J., & Tsoukas, H. (2015). Making sense o he sensemaking pe spec i e: I s
cons i uen s, limi a ions, and oppo uni ies o u he de elopmen . Jou nal o
O ganiza ional Beha io , 36(S1), S6–S32. h ps://doi.o g/10.1002/job.1937
Schee ens, J., Eh en, M., Sleege s, P., & Leeuw, R. de. (2012). OECD Re iew on E alua ion and
Assessmen F amewo ks o Imp o ing School Ou comes.
Sinclai , A. (1995). The chameleon o accoun abili y: Fo ms and discou ses. Accoun ing
O ganiza ions and Socie y, 20(2), 219–237. h ps://doi.o g/10.1016/0361-3682(93)E0003-Y
Spillane, J. P., Pa ise, L. M., & She e , J. Z. (2011). O ganiza ional Rou ines as Coupling
Mechanisms. Ame ican Educa ional Resea ch Jou nal, 48(3), 586–619.
h ps://doi.o g/10.3102/0002831210385102
Spillane, J. P., Reise , B. J., & Gomez, L. M. (2006). Policy Implemen a ion and Cogni ion The
Role o Human, Social, and Dis ibu ed Cogni ion in F aming Policy Implemen a ion. In M.
I. Honig (Ed.), New di ec ions in educa ion policy implemen a ion (pp. 47–64). S a e
Uni e si y o New Yo k P ess, Albany.
Spillane, J. P., Reise , B. J., & Reime , T. (2002). Policy implemen a ion and cogni ion:
SCHOOL INSPECTORS’S DECISION-MAKING
49
Re aming and e ocusing implemen a ion esea ch. Re iew o Educa ional Resea ch, 72(3),
387–341. h ps://doi.o g/10.3102/00346543072003387
Teddlie, C., & Yu, F. (2007). Mixed me hods sampling : A ypology wi h examples. Jou nal o
Mixed Me hods Resea ch, 1(1), 77–100. h ps://doi.o g/10.1177/1558689806292430
an B uggen, J. C. (2010). Inspec o a es o Educa ion in Eu ope; some compa a i e ema ks
abou hei asks and wo k.
an de Sluis, M. E., Reezig , G. J., & Bo ghans, L. (2017). Implemen ing New Public
Managemen in Educa ional Policy. Educa ional Policy, 31(3), 303–329.
Va us, F. K., & Ba le , L. (2016). Re hinking case s udy esea ch: A compa a i e app oach
(1s ed.). Rou ledge.
Weick, K. E. (1995). Sensemaking in o ganiza ions. SAGE Publica ions.
Woods, P., & Je ey, B. (1998). Choosing Posi ions: Li ing he Con adic ions o OFSTED.
B i ish Jou nal o Sociology o Educa ion, 19(4), 547–570.
h ps://doi.o g/10.1080/0142569980190406
SCHOOL INSPECTORS’S DECISION-MAKING
50
No es
1: A he ime o he in e iews, he Du ch sys em was ansi ioning o School Boa d inspec ions in
addi ion o con inuing wi h he isk-o ien ed school inspec ions. In ou in e iews, we ocused on he on-
si e school inspec ions as implemen ed un il he academic yea 2016-17.
