Ilie a, Galina e al.
A icle
Fac o s in luencing use pe cep ion and adop ion o e-
go e nmen se ices
Adminis a i e Sciences
P o ided in Coope a ion wi h:
MDPI – Mul idisciplina y Digi al Publishing Ins i u e, Basel
Sugges ed Ci a ion: Ilie a, Galina e al. (2024) : Fac o s in luencing use pe cep ion and adop ion o e-
go e nmen se ices, Adminis a i e Sciences, ISSN 2076-3387, MDPI, Basel, Vol. 14, Iss. 3, pp. 1-30,
h ps://doi.o g/10.3390/admsci14030054
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/320877
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h ps://c ea i ecommons.o g/licenses/by/4.0/
Ci a ion: Ilie a, Galina, Tania
Yanko a, Ma ga i a Ruse a, Yulia
Dzhaba o a, Veselina Zheko a,
S anisla a Klisa o a-Belche a, Tanya
Mollo a, and Angel Dimi o . 2024.
Fac o s In luencing Use Pe cep ion
and Adop ion o E-Go e nmen
Se ices. Adminis a i e Sciences 14:
54. h ps://doi.o g/10.3390/
admsci14030054
Recei ed: 4 Feb ua y 2024
Re ised: 4 Ma ch 2024
Accep ed: 6 Ma ch 2024
Published: 12 Ma ch 2024
Copy igh : © 2024 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
adminis a i e
sciences
A icle
Fac o s In luencing Use Pe cep ion and Adop ion o
E-Go e nmen Se ices
Galina Ilie a 1,* , Tania Yanko a 1, Ma ga i a Ruse a 1, Yulia Dzhaba o a 2, Veselina Zheko a 1,
S anisla a Klisa o a-Belche a 1, Tanya Mollo a 1and Angel Dimi o 1
1Depa men o Managemen and Quan i a i e Me hods in Economics, Uni e si y o Plo di Paisii
Hilenda ski, 4000 Plo di , Bulga ia; s l@uni-plo di .bg (T.Y.); m_ use a@uni-plo di .bg (M.R.);
.zheko a@uni-plo di .bg (V.Z.); s anisla a.belche a@uni-plo di .bg (S.K.-B.);
anya.mollo a@uni-plo di .bg (T.M.); angel.dimi o @uni-plo di .bg (A.D.)
2Depa men o Ma ke ing and In e na ional Economic Rela ions, Uni e si y o Plo di Paisii Hilenda ski,
4000 Plo di , Bulga ia; jjaba o a@uni-plo di .bg
*Co espondence: galili@uni-plo di .bg
Abs ac : The objec i e o his s udy is o in es iga e and de e mine ac o s in luencing use pe cep ion
and accep ance o elec onic go e nmen se ices in he con ex o echnological ad ancemen s. The
esea ch ocuses on classi ying he main ea u es o e-adminis a i e sys ems wi h an emphasis on
use sa is ac ion by in eg a ing bo h adi ional and mode n da a analysis echniques. S uc u al
Equa ion Modelling (SEM), machine lea ning (ML) echniques, and mul i-c i e ia decision-making
(MCDM) me hods ha e been applied o su ey da a o unco e he in e dependencies be ween
a iables om he pe spec i e o online use s. The de eloped models disco e and explain he
unde lying ela ionships in use a i udes owa ds e-go e nmen se ices. As he pe cep ion o
cus ome sa is ac ion is subjec i e and dynamic, s akeholde s should conduc egula measu emen s
and da a analysis o ensu e con inuous imp o emen o e-public se ices.
Keywo ds: elec onic public se ices; elec onic go e nmen se ices; echnology adop ion; cus ome
sa is ac ion; beha iou in en ion; s uc u al equa ion modelling; PLS-SEM; machine lea ning
1. In oduc ion
Elec onic adminis a i e p ocesses s eamline adi ional ope a ions, educing cos s
associa ed wi h ou da ed manual p ac ices and con ibu ing o he de elopmen o a mo e
e icien , anspa en , and cus ome -cen ic adminis a i e ecosys em
(Do an e al. 2023)
.
Addi ionally, digi al go e nmen pla o ms enhance connec i i y ia in e ac ions o e
dis ance h ough cos -e ec i e communica ion sys ems. Digi aliza ion in he public sec o
no only imp o es he quali y and accessibili y o public se ices, i eshapes ci izen–
go e nmen , businesses–go e nmen , and go e nmen –go e nmen ela ionships. Mo e-
o e , his ans o ma ion in public o ganiza ions suppo s he ansi ion o cleane ene gy
sou ces by op imizing esou ce usage and minimizing he need o physical inpu s (Fi man-
dayu and El aki 2023;Gomez-T ujillo and Gonzalez-Pe ez 2021;Hochs e e e al. 2023). The
digi al e olu ion o public adminis a ion u he acili a es he de elopmen o inclusi e
ins i u ions and posi i ely in luences he o e all p og ess o socie y
(Toko ska e al. 2023).
Elec onic go e nmen (e-go e nmen ) e e s o he use o in o ma ion and communica ion
echnologies (ICT) in deli e ing public se ices. E-go e nmen ins umen s, including
digi al pla o ms, IT sys ems, and so wa e apps, ensu e dynamic in e ac ions emo ely
e en in imes o c isis, as demons a ed du ing he COVID-19 pandemic
(Hodzic e al. 2021)
.
Fu he mo e, e-go e nmen ools ha e he po en ial o o e se ices o ulne able so-
cial g oups, such as indi iduals li ing in po e y, olde adul s, people wi h disabili ies,
immig an s, and you h by add essing hei speci ic needs (Seljan e al. 2020).
Adm. Sci. 2024,14, 54. h ps://doi.o g/10.3390/admsci14030054 h ps://www.mdpi.com/jou nal/admsci
Adm. Sci. 2024,14, 54 2 o 30
A he global le el, in e na ional o ganiza ions such as Uni ed Na ions (UN), O ga-
niza ion o Economic Co-ope a ion and De elopmen (OECD), and Wo ld Bank (WB)
collabo a e wi h hei membe coun ies o acili a e he sp ead o e-go e nmen ech-
nologies and p ac ices. These coope a i e e o s aim o enhance openness and ci izen
engagemen in public adminis a ion ac i i ies. Fo example, he UN Sus ainable De elop-
men Goals (SDGs), as ou lined in hei 2030 Agenda (UN Gene al Assembly 2015), closely
align wi h he concep o e-go e nmen , emphasizing connec i i y and open da a a ail-
abili y as undamen al o a mo e sus ainable global communi y (O hman e al. 2020). As a
esul , an inc easing numbe o coun ies a e p io i izing he digi iza ion o public se ices.
Fo example, in he Eu opean Union (EU), he Digi al Decade p og amme (2022) seeks
o achie e 100% online p o ision o “key public se ices” by 2030, while he goal o he
In e ope able Eu ope Ac (2022) is o boos c oss-bo de in e ope abili y and co-ope a ion
in he public sec o . In Denma k, India, he Ne he lands, and o he coun ies, elec onic
public se ices a e e en manda o y o public adminis a ion (Tangi e al. 2021).
Elec onic public se ices (e-public se ices, elec onic adminis a i e se ices, e-
adminis a i e se ices) encompass a wide ange o ac i i ies a bo h cen al and e i o ial
le els o go e nmen . These se ices include digi al documen submission, elec onic ax
iling and paymen , online applica ions o licenses and pe mi s, and online communi-
ca ion wi h o ganiza ional s uc u es and en i ies. Thei p e alence can be signi ican ly
inc eased, d i en by echnological inno a ions and he demand o e icien and con enien
go e nmen in e ac ions (To es e al. 2005).
The COVID-19 pandemic and subsequen social dis ancing measu es accele a ed
he dissemina ion o e-go e nmen se ices. Wi h es ic ions on in-pe son con ac s, in-
di iduals, and businesses u ned o digi al channels o ul il hei adminis a i e needs
emo ely. This ansi ion has led o ele a ed u iliza ion o elec onic pla o ms o a i-
ous go e nmen - ela ed ansac ions e en among cus ome s who p e iously had limi ed
expe ience o engagemen wi h elec onic public se ices (Cha zopoulou e al. 2021). Un-
o una ely, as he heal h si ua ion no malizes, he e has been some dec ease in eques s o
e-adminis a i e se ices. Fo example, in Bulga ia, du ing he COVID-19 heal h c isis, he
numbe o p o ided e-adminis a i e se ices inc eased by 130% in 2021 compa ed o he
p e ious yea , while in he ollowing yea (2022) i dec eased by 18% (Council o Minis e s
2020;Council o Minis e s 2021;Council o Minis e s 2022).
Acco ding o UN Elec onic Go e nmen E alua ion Index (EGDI) epo s, Eu ope
has consis en ly held he highes a e age EGDI among he con inen s since he ini ia ion
o UN e-Go e nmen Su ey in 2003. Fu he mo e, he de elopmen o e-go e nmen
ac oss Eu ope is no ably mo e uni o m compa ed o o he con inen s (UN DESA 2022). The
majo i y o su eyed Eu opean coun ies all wi hin he highes EGDI g oup, wi h eigh o
hem anking among he global leade s in e-go e nmen de elopmen . Howe e , as pe he
EU e-Go e nmen Benchma k epo (Capgemini e al. 2023), he e a e signi ican a ia ions
among Eu opean coun ies in key indica o s such as anspa ency and c oss-bo de se ices.
Globally hese di e ences a e e en mo e p onounced, wi h A ican coun ies acing he
g ea es lag in e-go e nmen de elopmen .
Mo eo e , he deploymen o new e-adminis a i e se ices o en en ails a a ie y o
challenges ex ending beyond so wa e implemen a ion issues, such as in eg a ion, use
awa eness, and aining needs. These issues ha e a nega i e impac on pe cep ion o
e-go e nmen se ices. Howe e , he e is no uni ied amewo k o me hodology o
assessmen o e-public se ices adop ion. Examining ac o s in luencing hese a i udes
and p edic ing hei impac on he u iliza ion o elec onic public se ices poses a complex
challenge o he ollowing h ee p ima y easons:
1.
Recen ad ancemen s in ICT echnologies, including A i icial In elligence (AI),
blockchain, and he In e ne o Things (IoT) can enhance he me hods and channels o
e-go e nmen (I i´c e al. 2022).
2.
The dynamics, unce ain y, and complexi y o he economic landscape in luence
use s’ equi emen s, p e e ences, and habi s. As echnologies e ol e, expec a ions
Adm. Sci. 2024,14, 54 3 o 30
and demands o use s o he channels deli e ing e-public se ices a e inc easingly
shi ing online (Sol ak e al. 2019).
3.
The exis ing me hods o cus ome sa is ac ion esea ch can be expanded h ough he
inco po a ion o machine lea ning (ML) (AlHadid e al. 2022), uzzy logic, big da a,
and o he in elligen echniques o hei combina ions.
This en ichmen equi es he explo a ion o new dependencies in unde s anding use
sa is ac ion and p e e ences owa ds e-adminis a i e se ices.
The objec i e o his s udy is o examine he ac o s ha in luence use pe cep ion and
in en ion o use e-public se ices. By es ablishing a comp ehensi e unde s anding o hese
ac o s, we aim o de elop a heo e ical amewo k and empi ical models ha can guide
go e nmen agencies in designing and implemen ing e ec i e e-adminis a i e sys ems.
Addi ionally, we in es iga e he impac o demog aphic and socioeconomic ac o s, such as
gende , age, educa ion le el, esidence a ea, and mon hly income, on use accep ance and
adop ion o e-adminis a i e se ices.
The main asks o his esea ch a e as ollows:
•
P opose a me hodological amewo k ha acili a es he sys ema ic analysis o cus-
ome da a and can e eal hidden ela ionships be ween ac o s in luencing he adop-
ion o new in o ma ion echnologies (IT) in he public sec o ;
•
Collec and sys emize a cus ome da ase abou hei expe iences and p e e ences e-
ga ding online public se ices (gende , age, esiden ial a ea, mon hly income, a i udes
and opinions);
•
C ea e and alida e a S uc u al Equa ion Model (SEM) based on ac o s om he li e -
a u e e iew and assess hei in luence on cus ome a i ude owa d e-adminis a i e
se ices;
•
Iden i y he key ac o s a ec ing cus ome use and in en ion o use e-adminis a i e
se ices acco ding o he ob ained model;
•
C ea e and e alua e al e na i e ML and MCDA models o p edic ion o use pe cep-
ion and adop ion o e-adminis a i e se ices.
To explo e cus ome adop ion o e-go e nmen se ices, we di ide sa is ac ion ac o s
in o se en main g oups and employ he co esponding ma hema ical models o p edic ion.
The ob ained ac o s’ weigh s can be in eg a ed in o mul i-c i e ia assessmen sys ems
o e alua ion o e-adminis a i e se ices. The main con ibu ion o his pape is he
de elopmen o a new complex me hodology inco po a ing s uc u al equa ion and ML
models wi h MCDM o e alua ion, compa ison, and p edic ion o cus ome a i udes
owa ds e-public se ices.
The emainde o his pape is o ganised as ollows: Sec ion 2p o ides an o e iew o
e-go e nmen se ices and he indica o s o hei assessmen ; Sec ion 3 e iews ele an
li e a u e on use pe cep ion and accep ance o e-public se ices; Sec ion 4ou lines he
esea ch objec i es and me hodology; Sec ion 5p esen s he esul s ob ained om he
analysis o he collec ed da ase ; inally, in Sec ion 6we discuss he implica ions o he
s udy, highligh i s con ibu ions, and p o ide u u e esea ch di ec ions in he ield o
e-adminis a i e se ices.
2. S a e o he A Re iew o Digi al Adminis a i e Se ices
Digi al public se ices e olu ionize he way ci izens and businesses in e ac wi h
public adminis a ion, p o iding e icien ICT ins umen s o a ious ansac ions. These
inno a i e se ices enable hei use s o con enien ly pe o m e e yday asks such as doc-
umen submissions, ee paymen s, and applica ion p ocessing h ough digi al in e ac ions.
Simul aneously, hey empowe go e nmen agencies by ans o ming se ice deli e y me h-
ods, o e ing added alue and enhanced use expe ience h ough geog aphical bounda ies.
Addi ionally, hese ad ancemen s os e imp o emen s in adminis a ion- o-adminis a ion
ela ionships, o e ing g ea e collabo a ion and e iciency in in e go e nmen al in e ac-
ions.
Adm. Sci. 2024,14, 54 4 o 30
Howe e , a signi ican challenge a ises, as indi iduals may no be ully acquain ed
wi h he capabili ies o hese cu ing-edge adminis a i e applica ions. This lack o knowl-
edge o en esul s in a disc epancy be ween he po en ial bene i s o elec onic adminis-
a i e se ices and he needs o he ci izens hey se e. In his con ex , we explo e he
dis inc i e ea u es o e-adminis a i e se ices, emphasizing hei ans o ma i e po en ial.
2.1. Key Fea u es and Taxonomy o Elec onic Public Se ices
Digi al public se ices enclose a wide ange o online se ices p o ided by go e n-
men al bodies o ci izens, businesses, and o he s akeholde s. Recen ly, eme ging ICT such
as AI, blockchain, and IoT (In e ne o Things) ha e eshaped he landscape o deli e ing
such public se ices (I i´c e al. 2022).
A i icial In elligence plays a signi ican ole in au oma ing ou ine adminis a i e
asks, imp o ing decision-making p ocesses, and enhancing o e all e iciency. AI algo-
i hms analyse as da ase s o de i e insigh s, enabling go e nmen s o make da a-d i en
policy decisions and s eamline se ice deli e y. Cha bo s powe ed by AI me hods p o ide
ci izens wi h ins an suppo , enhancing accessibili y and esponsi eness in public se ices
(Al-Mushay 2019).
Blockchain echnology, known o i s decen alized and secu e na u e, has b ough
anspa ency and us o public se ice ansac ions. I ensu es he in eg i y and immu abil-
i y o eco ds, educing aud and co up ion isks. In a eas such as iden i y e i ica ion
and documen au hen ica ion, blockchain enhances secu i y and minimizes he isk o
da a manipula ion. This echnology is pa icula ly aluable in c ea ing accoun able and
anspa en sys ems o public eco ds, such as land egis ies and inancial ansac ions,
os e ing g ea e us be ween he go e nmen and i s s akeholde s (Lykidis e al. 2021).
The In e ne o Things is ano he ans o ma i e echnology ha con ibu es o he
deli e y o public se ices. IoT de ices anging om sma senso s o connec ed in as-
uc u e enable eal- ime moni o ing and da a collec ion. In public sa e y, o ins ance,
IoT de ices can enhance eme gency esponse sys ems by p o iding accu a e and imely
in o ma ion. Sma ci y ini ia i es le e age IoT o op imize esou ce managemen , imp o e
anspo a ion, and e ine en i onmen al moni o ing (Bansal e al. 2022).
In eg a ing inno a i e echnologies such as AI, blockchain, and IoT in deli e ing
public se ices c ea es a echnological syne gy ha imp o es e iciency while os e ing
mo e ci izen-cen ic, secu e, and esponsi e go e nance.
E-public se ices a e also impo an in a ac ing o eign in es men s by es ablishing
an accessible and anspa en business en i onmen . Coun ies wi h well-de eloped dig-
i al pla o ms o public se ices c ea e a posi i e business clima e, o e ing s eamlined
p ocesses o egula o y compliance, licensing, and o he adminis a i e p ocedu es. Con-
sequen ly, e-public se ice in as uc u e no only enhances he o e all a ac i eness o a
coun y o o eign in es men s bu e lec s i s commi men o mode niza ion and e ec i e
go e nance (Al-Sadiq 2021).
Gi en he inc easing dependence on digi al business- o-consume in e ac ions in a i-
ous aspec s o e e yday li e, ci izens expec he public sec o o mee ele a ed s anda ds sim-
ila o hose se by business so wa e. As echnologies e ol e, ci izens expec e-go e nmen
o p o ide in ui i e in e aces, imely se ices, and a use -cen ic app oach, aligning wi h
he benchma ks es ablished by success ul online business applica ions (Holze e al. 2019).
Fo ins ance, he in eg a ed se ices o he Chinese supe -applica ion WeCha , includ-
ing u ili y paymen s and no i ica ions, can be pe sonalized o some ex en based on use
p e e ences and loca ion (Pan 2020).
S a ing in he 1990s, nume ous public adminis a ion en i ies ha e implemen ed
echnological inno a ions by o e ing online al e na i es o adi ional se ice deli e y
me hods. Elec onic public se ices can be ca ego ized in a ious ways by di e en c i e ia.
Se ice model: E-public se ices can be spli by hei ecipien s (use s) and classi ied
in o h ee main ca ego ies: go e nmen - o-ci izen (G2C), go e nmen - o-business (G2B),
and go e nmen - o-go e nmen (G2G).
Adm. Sci. 2024,14, 54 5 o 30
Se ice ype: This c i e ion ca ego izes online public se ices based on hei p ima y
unc ion, dis inguishing be ween co e go e nmen unc ions, ci izen-cen ic se ices, and
hose ca e ing o businesses and indus ies.
Go e nmen le el: Acco ding o hei adminis a i e le el, public se ices can be
di ided in o h ee g oups—cen al, egional, o local go e nmen o e ings.
In e ac i i y le el: Di e en ia es be ween in o ma ional, ansac ional, and in e ac i e
se ices, e lec ing he ex en o use engagemen and unc ionali y.
Deli e y channel: E-public se ices can be classi ied acco ding o he medium h ough
which hey a e accessed—online pla o ms, sys ems, o mobile applica ions.
Se ice ma u i y: Acco ding o he s age model (Layne and Lee 2001), di e en s ages
ep esen he e olu iona y p og ession in he implemen a ion o elec onic public se -
ices. A common s age model o e-go e nmen includes he ollowing s ages: eme ging,
enhanced, ansac ional, in eg a ed, ne wo ked, and ubiqui ous se ices (Lemke e al. 2020).
The widesp ead use o digi al public se ices by ci izens, businesses, and public
adminis a ion elies on many ac o s, including he p esence o sui able elec onic in as-
uc u e, in e ne a ailabili y, and he p o iciency o design, implemen , manage, and u ilize
e-go e nmen sys ems.
2.2. Assessing Elec onic Public Se ices
When e alua ing elec onic go e nmen de elopmen , a ange o assessmen ools can
be employed o e alua e he quali y, secu i y, and o e all e ec i eness o public se ices.
These can be ca ego ized in i e main a eas: (1) indices; (2) s anda d speci ica ions; (3)
amewo ks; (4) heo e ical models; and (5) me ics (Table 1). These benchma ks can
complemen each o he in enhancing he unde s anding and imp o emen o he adop ion
and ma u i y o e-go e nmen ini ia i es.
Among he mos widely used indices o e-go e nmen e alua ion a e he ollowing:
EGDI: Published by he UN, EGDI is a composi e index ha measu es he s a e
o e-go e nmen de elopmen in coun ies based on h ee dimensions: online se ices,
elecommunica ions in as uc u e, and human capi al. I anks coun ies acco ding o hei
e-go e nmen eadiness (He nández e al. 2024).
e-Go e nmen Benchma k: This benchma k o he Eu opean Commission assesses
he digi al ma u i y o Eu opean coun ies in p o iding online public se ices. I e alu-
a es di e en dimensions, such as online se ices, online c oss-bo de se ices, elec onic
IDen i ica ion (eID), e-documen s, and p e- illed o ms (Majo 2023).
Digi al Go e nmen Index: The OECD de ines a se o indica o s o assess he ma u i y
o digi al go e nmen ac oss i s membe coun ies. I co e s aspec s such as digi al-by-
design, da a-d i en public sec o , go e nmen as a pla o m, open by de aul , use -d i en,
and p oac i eness (Ubaldi and Okubo 2020).
Go Tech Ma u i y Index: WB’s index assesses he eadiness o coun ies o pa icipa e
in he digi al economy. I conside s ac o s such as he egula o y en i onmen , echnology
in as uc u e, and digi al li e acy (Dene e al. 2021).
These indices a y in hei me hodologies, ocus a eas, and he dimensions ha hey
conside . Go e nmen s, policymake s, and esea che s o en use a combina ion o hese
indices o gain a comp ehensi e unde s anding o he s a e o e-go e nmen de elopmen
globally and egionally.
Adm. Sci. 2024,14, 54 6 o 30
Table 1. Compa ison o he mos widely used measu emen ools o e alua ing digi al go e nmen
se ices.
Assessmen
Tool Measu emen Goal(s) App aisal
Dimensions
E alua ion
Scope
EGDI (UN DESA 2022)
E-go e nmen
de elopmen
EGDI e alua es online se ices, elecommunica ion
in as uc u e, human capi al
Global and
egional
e-Go e nmen
Benchma k
(EC 2023)
E-go e nmen
de elopmen
The index consis s o assessmen s o online se ices, online
c oss-bo de se ices, eID, e-documen s, p e- illed o ms Eu opean
Digi al Go e nmen
Index (Ubaldi and
Okubo 2020)
E-go e nmen
de elopmen
This index comp ises six e-go e nmen measu es:
digi al-by-design, da a-d i en, pla o m-based, open,
use -d i en, and p oac i e
Global and
egional
Go Tech Ma u i y
Index
(Dene e al. 2021)
E-go e nmen
ma u i y
GTMI includes ou componen s: co e go e nmen sys ems,
public se ice deli e y, digi al ci izen engagemen , and
Go Tech enable s
Global and
egional
ISO 20000
(ISO/IEC 20000-1:2018
2018)
Pe o mance, Quali y
Managemen
S anda d o IT se ice managemen , ocusing on e iciency
and pe o mance O ganiza ional
ISO 27001
(ISO/IEC 27001:2022
2022)
Secu i y
S anda d o in o ma ion secu i y managemen sys ems,
ensu ing con iden iali y, in eg i y, and a ailabili y o
in o ma ion
O ganiza ional
EN 301549 (EN
301549:2018 2018)Accessibili y This s anda d con ains de ailed equi emen s o websi es,
web-deli e ed documen s, and mobile applica ions
Eu opean
o ganiza ional
CAF (P o ok 2020)
O ganiza ional
Pe o mance
Assessmen
The amewo k has nine c i e ia: leade ship, pe sonnel,
pa ne ships, budge , knowledge, IT, p ocesses, ci izens and
cus ome s, social esponsibili y, and key pe o mance
Eu opean
o ganiza ional
SERVQUAL
(Pa asu aman e al.
1991)
Se ice Quali y
This amewo k assesses se ice quali y based on i e
ac o s: angibles, eliabili y, esponsi eness, assu ance, and
empa hy map
O ganiza ional
TAM (Da is 1989) Use Accep ance
Model e alua ing use accep ance o echnology, ocusing
on ac o s in luencing use s’ willingness o adop and use
e-se ices
Use -cen ic
UTAUT (Venka esh
e al. 2003)Use Beha iou
Model in eg a ing a ious ac o s o p edic use accep ance
and beha iou owa d e-public se ices Use -cen ic
UX E alua ion * Usabili y, Sa is ac ion
Me ic assessing o e all use expe ience, encompassing
usabili y, accessibili y, and sa is ac ion wi h
e-adminis a i e se ices
Use -cen ic
Digi al Accessibili y
E alua ion * Accessibili y Me ic e alua ing he accessibili y o e-adminis a i e
se ices o ensu e usabili y o indi iduals wi h disabili ies Use -cen ic
E iciency and
Pe o mance Me ics * Pe o mance Me ics
Me ics assessing he e iciency and pe o mance o
e-adminis a i e se ices, including esponse ime,
h oughpu , se e up ime, and esou ce u iliza ion
Use -cen ic
Ci izen-Cen ic
E alua ion *
Use Sa is ac ion,
Expec a ions
Me ic e alua ing he ex en o which e-adminis a i e
se ices a e ci izen-cen ic and mee use needs and
expec a ions
Use -cen ic
Digi al Inclusion
Assessmen * Inclusi eness
Me ic e alua ing he inclusi eness o e-adminis a i e
se ices, ensu ing accessibili y o di e se use g oups based
on ac o s such as language di e si y and ou each e o s
Use -cen ic
No e: * deno es ha he concep con inually e ol es, in eg a ing di e se guidelines and me hodologies de eloped
by a ious o ganiza ions and expe g oups.
The ISO/IEC s anda ds con ibu e o he assessmen o e-go e nmen de elopmen
by o e ing common guidelines and measu es, acili a ing consis ency, anspa ency and
Adm. Sci. 2024,14, 54 7 o 30
in e ope abili y ac oss di e en sys ems o public se ices. They help go e nmen s o align
hei digi al s a egies wi h in e na ional bes p ac ices in a mo e e icien e-go e nmen
s uc u e. The ISO/IEC 20000 s anda d (ISO/IEC 20000-1:2018 2018) ou lines guidelines
o se ice managemen , including he planning, deli e y, and imp o emen o IT se ices.
The e o e, his s anda d can be applied o assess he quali y o elec onic adminis a i e
se ices (Sa wa e al. 2023). The ISO/IEC 27001 s anda d (ISO/IEC 27001:2022 2022)
ocuses on in o ma ion secu i y managemen sys ems and is applicable o e alua ing he
secu i y aspec s o elec onic adminis a i e se ices, ensu ing he con iden iali y, in eg i y,
and a ailabili y o in o ma ion. Al hough o iginally designed o quali y managemen
in gene al, ISO/IES 9001 can be applied o assess he quali y o IT se ices, including
elec onic adminis a i e se ices. I emphasizes cus ome sa is ac ion and con inuous
imp o emen . EN 301549 (EN 301549:2018 2018) is a Eu opean s anda d p o iding accessi-
bili y equi emen s o ICT p oduc s and se ices, ensu ing ha elec onic adminis a i e
se ices a e accessible o all use s, including hose wi h disabili ies.
While ISO/IEC 20000 guides IT se ice managemen , ISO/IEC 27001 ocuses on
in o ma ion secu i y, ISO/IES 9001 emphasizes quali y, and EN 301549 se s Eu opean
accessibili y equi emen s, hese s anda ds collec i ely p o ide a comp ehensi e amewo k
o assess and enhance elec onic adminis a i e se ices.
A di e se ange o amewo ks se es o p o ide sui able ools o benchma king
e-public se ices, o e ing s uc u ed ules o op imize IT se ice managemen and imp o e
he o e all e iciency o digi al se ice deli e y. In o ma ion Technology In as uc u e
Lib a y (ITIL) is a se o p ac ices o IT se ice managemen . I o e s a amewo k o
deli e ing high quali y IT se ices, aligning hem wi h he needs o he business. ITIL
can enhance he managemen o elec onic adminis a i e se ices (Ba me an e al. 2022).
In ecen yea s, he Common Assessmen F amewo k (CAF) (P o ok 2020) has been p i-
o i ized as a quali y managemen sys em in he adminis a ions o EU membe s a es. I
se es as a common amewo k o e alua ing he pe o mance o public sec o o ganiza-
ions. Ano he ins umen o e-public se ices assessmen is he Eu opean Founda ion
o Quali y Managemen (EFQM). EFQM is a amewo k o o ganiza ional managemen
and quali y imp o emen known as he EFQM Excellence Model. I is a holis ic app oach
o assessing and imp o ing o ganiza ional pe o mance, ocusing on leade ship, s a egy,
people, pa ne ships, esou ces, p ocesses, p oduc s/se ices, cus ome esul s, people
esul s, socie y esul s, and key pe o mance esul s (Rahma i and Jalil and 2023). The
Open Da a Readiness Assessmen e alua es he eadiness o e-adminis a i e se ices o
publish and use open da a. This amewo k p o ides a s uc u ed app oach o assessing
open da a eadiness (Kawashi a e al. 2020).
In he assessmen o e-public se ices, a ious heo e ical models encompass aspec s
such as quali y, ma u i y, use accep ance, and b oade echnology adop ion in he public
sec o . SERVQUAL compa es expec a ions wi h pe cep ions on di e en se ice quali y
aspec s. La e , his model was iden i ied wi h i e dimensions o se ice quali y: Tangibles,
Reliabili y, Responsi eness, Assu ance, and Empa hy (Pa asu aman e al. 1991). Ma u i y
models, such as he Capabili y Ma u i y Model In eg a ion (CMMI), can help o ganiza ions
o assess and imp o e hei p ocesses, capabili ies, and o e all ma u i y in deli e ing
se ices o solu ions. They o e a sys ema ic app oach o o ganiza ions in mo ing om
lowe ma u i y le els o highe ones while inco po a ing bes p ac ices and con inuous
imp o emen . Ma u i y models a e o en applied a he o ganiza ional le el o e alua e and
enhance he ma u i y o e-go e nance p ocesses, se ice deli e y, and o e all capabili ies
(Kawashi a e al. 2020;Huj an e al. 2023). The Technology Accep ance Model (TAM) (Da is
1989) and Uni ied Theo y o Accep ance and Use o Technology (UTAUT) (S e ano ic e al.
2021) a e use -cen ic models ha ocus on unde s anding and p edic ing use accep ance
and adop ion o echnology, including e-go e nmen se ices. TAM emphasizes Pe cei ed
Ease o Use and Pe cei ed Use ulness, while UTAUT inco po a es addi ional ac o s such
as Pe o mance Expec ancy, E o Expec ancy, Social In luence, and Facili a ing Condi ions
(Ped osa e al. 2020;Zeeba ee e al. 2022).
Adm. Sci. 2024,14, 54 8 o 30
The abo emen ioned models o e-public se ice e alua ion con ibu e dis inc in-
sigh s: SERVQUAL emphasizes se ice quali y, ma u i y models assess e olu iona y s ages,
TAM ocuses on use accep ance, and UTAUT p o ides a holis ic unde s anding o use
beha iou .
Me ics o e-public se ices co e a ious aspec s o use expe ience e alua ion,
ensu ing usabili y, accessibili y, and sa is ac ion; digi al accessibili y e alua ion o indi-
iduals wi h disabili ies; e iciency and pe o mance me ics measu ing esponse ime,
h oughpu , se e up ime, and esou ce u iliza ion; ci izen-cen ic e alua ion gauging
use sa is ac ion and expec a ions; and digi al inclusion assessmen conside ing language
di e si y, accessibili y ea u es, and ou each e o s (Holze e al. 2019).
Depending on hei main ea u es, he mos widely used measu emen ools o
e alua ing digi al go e nmen se ices, as ou lined in Table 1, can be classi ied based on
se e al c i e ia such as assessmen ype, ocus, me hodology and applica ion scope.
Type o Assessmen : S anda ds such as ISO/IEC 20000 and amewo ks such as CAF
de ine s uc u ed guidelines o quali y managemen and pe o mance assessmen , while
models such as SERVQUAL and me ics such as Use Expe ience (UX) E alua ion co e
di e se aspec s o gauging elec onic public se ice.
Focus o Assessmen : Use -cen ic app oaches a e add essed by models such as TAM
and UTAUT and me ics such as UX E alua ion and Ci izen-Cen ic E alua ion, s anda ds
such as ISO/IEC 27001 emphasize secu i y, and EN 301549 and he Digi al Accessibili y
E alua ion ocus on accessibili y.
Me hodology: Quan i a i e me hodologies a e employed in s anda ds such as ISO/IEC
20000 and me ics such as E iciency and Pe o mance, Digi al Accessibili y E alua ion,
and Digi al Inclusion Assessmen , while quali a i e me hods a e applied in amewo ks
such as CAF and models such as SERVQUAL and UX E alua ion. Quali a i e me hods
in ol e app oaches such as use su eys, ocus g oups, in-dep h in e iews, and usabili y
s udies o use pe cep ions and o e all expe iences wi h elec onic se ices.
Scope: Global s anda ds such as ISO/IEC 20000 and ISO/IEC 27001 ha e a wo ldwide
ocus, Eu opean s anda ds such as EN 301549 ca e o a egional con ex , o ganiza ional
assessmen s a e add essed by amewo ks such as CAF and models such as SERVQUAL,
and use -cen ic e alua ions a e co e ed by models such as TAM and UTAUT and me ics
such as UX E alua ion and Ci izen-Cen ic E alua ion.
The mul i ude o measu emen ools o he assessmen o e-public se ices p o ides a
mul i ace ed app oach, om use expe ience and e iciency o secu i y and accessibili y,
con ibu ing o con inuous imp o emen and in o med decision-making in digi al go e -
nance. When e alua ing elec onic adminis a i e se ices, o ganiza ions o en combine
ele an measu emen ools based on hei speci ic goals, egula o y equi emen s, and he
na u e o he se ices p o ided.
2.3. Challenges in E alua ing Elec onic Public Se ices
When assessing elec onic public se ices h ough di e en amewo ks, models, and
indices, se e al challenges may a ise. Fi s , di e en amewo ks and models employ
a ied a ibu es, making i di icul o compa e and s anda dize e alua ions ac oss indices.
Second, he apid e olu ion o echnology equi es cons an upda es. Fo example, UTAUT
con inues o p og ess and be en iched. Las ly, con ex ual a ia ions a e an issue, as he
e ec i eness o e alua ion amewo ks may a y based on he goals and cha ac e is ics o
each go e nmen o egion.
To o e come hese obs acles, con inuous e inemen and adap a ion o e alua ion
measu es a e impo an o ensu e alignmen wi h he changing landscape o elec onic
public se ices. Conside ing he p oblem o selec ing he op imal app oach, we p opose
a combina ion o assessmen me hods o e-public se ices o ensu e comp ehensi e and
eliable es ima es. The TAM and UTAUT models a e among he mos widely employed
o unde s anding and p edic ing cus ome sa is ac ion in echnology adop ion. Thei
simplici y and cla i y make hem accessible and applicable ac oss di e se echnological
Adm. Sci. 2024,14, 54 15 o 30
da a on model cons uc s ( om Ques ion #10 o Ques ion #20) showed ha he e we e eigh
duplica es o wo da ase ows, as ollows: (#40, #81, #139, #148, #150, #160), (#119, #124,
#128, #156) (see Figu e 1).
Adm. Sci. 2024, 14, x FOR PEER REVIEW 15 o 31
opinions. In elligen ML me hods can unco e unknown pa e ns and ela ionships be-
ween a iables ha may no be appa en when using classical s a is ical me hods.
5. Resul s
The me hodology ou lined in Sec ion 4 was employed s ep-by-s ep o add ess he
esea ch asks.
5.1. Cus ome Da a Collec ion
We sha ed a link o he online su ey h ough ou ins i u ional websi es, social media
(Facebook g oups), and emails. The su ey a ge ed Bulga ian use s o online public se -
ices and was comple ed on a olun a y basis. C ea ed using Google Fo ms, he su ey
consis ed o 21 ques ions designed o measu e cus ome s’ pe cep ions owa d e-public
se ices (Ilie a e al. 2024). The esponden s’ da a we e collec ed om 18 Janua y 2023 o
22 May 2023. A o al o 258 pa icipan s comple ed he ques ionnai e, o whom 64 indi-
ca ed ha hey do no use online adminis a i e se ices (Ques ion #7). A duplica e check
was pe o med, and he e we e no iden ical alues ound in he da ase ows. Howe e ,
he da a on model cons uc s ( om Ques ion #10 o Ques ion #20) showed ha he e we e
eigh duplica es o wo da ase ows, as ollows: (#40, #81, #139, #148, #150, #160), (#119,
#124, #128, #156) (see Figu e 1).
Figu e 1 illus a es he deg ee o simila i y be ween he esponden s’ answe s, wi h
close dis ances indica ing smalle diffe ences. The deg ee o simila i y is ep esen ed by
diffe en colou s, anging om ull coincidence (0—whi e) o maximum diffe ence (20—
ligh g een). Because he da ase did no con ain iden ical eco ds, all obse a ions a e
included in he analysis.
Figu e 1. The ma ix o dis ances (o de ed dissimila i y ma ix) be ween esponden s’ answe s.
Da a s o age
The ques ionnai e and esponden s’ da ase a e a ailable online (Ilie a e al. 2024).
Da a encoding
Figu e 1. The ma ix o dis ances (o de ed dissimila i y ma ix) be ween esponden s’ answe s.
Figu e 1illus a es he deg ee o simila i y be ween he esponden s’ answe s, wi h
close dis ances indica ing smalle di e ences. The deg ee o simila i y is ep esen ed
by di e en colou s, anging om ull coincidence (0—whi e) o maximum di e ence
(20—ligh g een). Because he da ase did no con ain iden ical eco ds, all obse a ions a e
included in he analysis.
Da a s o age
The ques ionnai e and esponden s’ da ase a e a ailable online (Ilie a e al. 2024).
Da a encoding
The ules o coding and coded da a a e also a ailable online. Ou o all 21 esponses,
19 ha e been coded (Ilie a e al. 2024). The emaining wo open- ex answe s (municipali y
and opinions) ha e been addi ionally p ocessed.
Da a p ep ocessing
P ep ocessing was ca ied ou and he da ase quali y was examined o accu acy and
consis ency.
5.2. S a is ical Analysis
To cla i y he p o ile o he pa icipan s in he su ey, a classical s a is ical analysis
(pe cen age dis ibu ion o esponses, desc ip i e s a is ics, and co ela ion analysis) was
pe o med.
Main Cha ac e is ics o he Sample
Table 3illus a es he demog aphics o he ques ionnai e esponden s. A signi ican
majo i y o he pa icipan s we e emale, accoun ing o 74% o he o al numbe o pa ici-
Adm. Sci. 2024,14, 54 16 o 30
pan s (Ques ion #1). Mo e han h ee qua e s (77%) o he esponden s we e unde he
age o 40 (Ques ion #2). The sample was domina ed by indi iduals wi h a mos a high
school deg ee, comp ising 59.3% o he pa icipan s (Ques ion #6). The su ey was p i-
ma ily conduc ed in u ban a eas, wi h 95.7% o he esponden s esiding in such loca ions
(Ques ion #3).
Table 3. Use p o ile o he sample (n= 258).
Va iables
o he Sample No. o Responden s Pe cen age (%)
1. Gende Male 66 25.6
Female 192 74.4
2. Age
Unde 20 77 29.8
Be ween 21 and 30 87 33.7
Be ween 31 and 40 35 13.6
Be ween 41 and 50 43 16.7
O e 50 16 6.2
3. Place o esidence
Ci y 161 62.4
Town 86 33.3
Village 11 4.3
4. Municipali y/P o ince - - -
5. Mon hly income pe
household membe
Less han BGN 1320 140 54.3
Mo e han BGN 1320 118 45.7
6. Educa ion
High school 153 59.3
Bachelo 59 22.9
Mas e 42 16.3
PhD 4 1.6
7. Do you use elec onic
adminis a i e se ices?
No 64 24.8
Yes 194 75.2
The geospa ial dis ibu ion o pa icipan s (Ques ion #4) e eals ha he majo i y we e
om he Plo di dis ic (71.6% o he su ey pa icipan s). The second highes p opo ion
o esponden s we e om he Paza dzhik dis ic (6.2% o he o al), ollowed by he So ia
ci y dis ic wi h 5.2% o sample size. The su ey p ima ily a ge ed he Sou he n Cen al
egion, comp ising 83.5% o pa icipan s, ollowed by he Sou hwes e n and Sou heas e n
egions, each ep esen ing 6.2%.
Fo 75% o he pa icipan s, online adminis a i e se ices we e he p e e ed o m o
communica ion wi h public au ho i ies (Ques ion #7). This pe cen age is ela i ely close o
he adop ion a e o digi al public se ices as epo ed in a na ional su ey conduc ed a
he beginning o 2023 (NCPS 2023).
The mos widely u ilized e-public se ices a he e i o ial le el encompass hose asso-
cia ed wi h local axes and ees and ci il s a us, ep esen ing 40.4% and 26.0%, espec i ely
(Ques ion #8). A he na ional le el, he mos commonly used e-se ices include social
secu i y o employees, co po a e axes, and alue-added ax, cons i u ing 23.3%, 22.0%,
and 19.8%, espec i ely (Ques ion #9).
Fea u e Selec ion
The colou dep h o he hea maps in Figu es 2and 3 ep esen s s anda dized alues,
anging om a minimum o
−
2.92 (whi e) o a maximum o 1.95 (g een). The dend og am
a he op o Figu e 2illus a es he g ouping o esponden s based on simila i ies in hei
e-go e nmen a i udes. Addi ionally, he a iable s uc u e in Figu e 3( igh ) unde sco es
hei simila i ies. Bo h hea maps isually display clus e s o obse a ions and a iables
sha ing simila cha ac e is ics wi h no unusual o unexpec ed pa e ns.
Adm. Sci. 2024,14, 54 17 o 30
Adm. Sci. 2024, 14, x FOR PEER REVIEW 17 o 31
o he adop ion a e o digi al public se ices as epo ed in a na ional su ey conduc ed
a he beginning o 2023 (NCPS 2023).
The mos widely u ilized e-public se ices a he e i o ial le el encompass hose
associa ed wi h local axes and ees and ci il s a us, ep esen ing 40.4% and 26.0%, espec-
i ely (Ques ion #8). A he na ional le el, he mos commonly used e-se ices include so-
cial secu i y o employees, co po a e axes, and alue-added ax, cons i u ing 23.3%,
22.0%, and 19.8%, espec i ely (Ques ion #9).
Fea u e Selec ion
The colou dep h o he hea maps in Figu es 2 and 3 ep esen s s anda dized alues,
anging om a minimum o −2.92 (whi e) o a maximum o 1.95 (g een). The dend og am
a he op o Figu e 2 illus a es he g ouping o esponden s based on simila i ies in hei
e-go e nmen a i udes. Addi ionally, he a iable s uc u e in Figu e 3 ( igh ) unde -
sco es hei simila i ies. Bo h hea maps isually display clus e s o obse a ions and a -
iables sha ing simila cha ac e is ics wi h no unusual o unexpec ed pa e ns.
Figu e 2. Hie a chical g oup hea map by esponden s.
Figu e 2. Hie a chical g oup hea map by esponden s.
Clus e ing
To iden i y he g oups o use s wi h simila cha ac e is ics and he a iables ha ha e
a compa able e ec on a i udes owa d e-go e nmen , we employed he k-means me hod
o clus e analysis. The op imal numbe o clus e s was de e mined using he Elbow and
Silhoue e me hods, wi h he esul s indica ing ha his numbe is 2 (Figu e 4). The wo
clus e s consis ed o 62 and 132 esponden s, espec i ely.
The smalle clus e (Clus e #1) comp ises dissa is ied e-go e nmen use s wi h a
less posi i e a i ude owa ds e-public se ice adop ion, as e lec ed by lowe a ings in
A i ude (Ques ion #14), In en ion o Use (Ques ion #15), and In en ion o Recommend
(Ques ion #16) e-go e nmen se ices (Table 4). The indica o s wi h he mos signi ican
in luence on o e all dissa is ac ion a e T us (Ques ion #17), Se ice Quali y (Ques ion
#19–Ques ion #20), and Pe cei ed Use ulness (Ques ion #10). In con as , he second clus e ,
consis ing o he majo i y o use s, demons a es a highe le el o sa is ac ion wi h digi al
public se ices. Pe cei ed Ease o Use (Ques ion #11) and Pe cei ed Risk (Ques ion #18)
a e he mos signi ican ac o s con ibu ing o he posi i e a i ude o his second g oup o
use s. Table 4p esen s he a e age alues o he indica o s o he wo clus e s along wi h
he di e ences be ween hese es ima es.
Adm. Sci. 2024,14, 54 18 o 30
Adm. Sci. 2024, 14, x FOR PEER REVIEW 18 o 31
Figu e 3. Hie a chical g oup hea map by ac o s.
Clus e ing
To iden i y he g oups o use s wi h simila cha ac e is ics and he a iables ha ha e
a compa able effec on a i udes owa d e-go e nmen , we employed he k-means me hod
o clus e analysis. The op imal numbe o clus e s was de e mined using he Elbow and
Silhoue e me hods, wi h he esul s indica ing ha his numbe is 2 (Figu e 4). The wo
clus e s consis ed o 62 and 132 esponden s, espec i ely.
Figu e 3. Hie a chical g oup hea map by ac o s.
Sen imen Analysis
The open-ended ques ion (Ques ion #21) ecei ed 54 ex eplies. A e p ep ocessing
and il e ing, 35 esponses conce ning e-public se ices we e kep . The a e age sen imen
sco es o esponses we e as ollows: posi i e—22 (63%, a e age: 0.712), neu al—5 (14%,
a e age: 0.519), and nega i e—8 (23%, a e age: 0.218). These esul s indica e ha espon-
den s gene ally suppo e-go e nmen se ices as a con enien means o in e ac ing wi h
public adminis a ion, highligh ing some o he ad an ages. Those exp essing a nega i e
a i ude p ima ily had conce ns abou complex websi e na iga ion and po en ial secu-
i y issues. Neu al opinions suppo ed he usage o e-public se ices bu highligh ed
weaknesses in online da a p ocessing.
Adm. Sci. 2024,14, 54 19 o 30
Adm. Sci. 2024, 14, x FOR PEER REVIEW 19 o 31
Figu e 4. Use s’ clus e s by k-means (k = 2, 3, 4, 5) using 34 inpu indica o s.
The smalle clus e (Clus e #1) comp ises dissa is ied e-go e nmen use s wi h a less
posi i e a i ude owa ds e-public se ice adop ion, as e lec ed by lowe a ings in A i-
ude (Ques ion #14), In en ion o Use (Ques ion #15), and In en ion o Recommend (Ques-
ion #16) e-go e nmen se ices (Table 4). The indica o s wi h he mos signi ican in lu-
ence on o e all dissa is ac ion a e T us (Ques ion #17), Se ice Quali y (Ques ion #19–
Ques ion #20), and Pe cei ed Use ulness (Ques ion #10). In con as , he second clus e ,
consis ing o he majo i y o use s, demons a es a highe le el o sa is ac ion wi h digi al
public se ices. Pe cei ed Ease o Use (Ques ion #11) and Pe cei ed Risk (Ques ion #18)
a e he mos signi ican ac o s con ibu ing o he posi i e a i ude o his second g oup
o use s. Table 4 p esen s he a e age alues o he indica o s o he wo clus e s along
wi h he diffe ences be ween hese es ima es.
Figu e 4. Use s’ clus e s by k-means (k= 2, 3, 4, 5) using 34 inpu indica o s.
The esponden s p oposed he ollowing ecommenda ions o e-se ice imp o e-
men s:
•
Online sys ems could p o ide echnical suppo o use s and esponse o use que ies
in eal ime;
•The s uc u e and na iga ion sys em o websi es could be op imised;
•
The ci izens’ easy access o e-se ices could be ensu ed wi hou addi ional equi e-
men s, such as elec onic signa u es o aining in accoun ing;
•Cybe secu i y measu es and da a p o ec ion could be s eng hened.
The ecommended ac ions aim o acili a e use s’ accep ance and u iliza ion o e-
go e nmen se ices. Enhancing he bene i s o online sys ems and imp o ing esponsi e-
ness aligns wi h he p inciple o Ease o Use by p omo ing a mo e use - iendly expe ience.
Imp o ing websi e o ganiza ion and na iga ion co esponds o E o Expec ancy by sim-
pli ying use in e ac ions. The sugges ion o ensu e easy accessibili y wi hou complex
Adm. Sci. 2024,14, 54 20 o 30
p e equisi es aligns wi h Facili a ing Condi ions, speci ically educing ba ie s o en y.
S eng hening cybe secu i y add esses conce ns abou Pe cei ed Risk, mi iga ing use s’
app ehensions abou po en ial secu i y issues. Collec i ely, hese ecommenda ions con-
ibu e o a posi i e use expe ience and accep ance o e-go e nmen se ices based on
UTAUT p inciples.
Table 4. A e age alues by clus e and absolu e di e ences be ween clus e s by indica o s.
Q10.1 Q10.2 Q10.3 Q11.1 Q11.2 Q11.3 Q11.4 Q12.1 Q12.2
Clus e #1 2.968 2.903 2.371 2.661 2.629 2.823 3.097 2.274 2.145
Clus e #2 4.144 4.114 3.568 3.424 3.614 3.985 4.136 3.402 3.364
Di e ence
−1.176 −1.211 −1.197 −0.763 −0.985 −1.162 −1.039 −1.128 −1.219
Q12.3 Q13.1 Q13.2 Q13.3 Q14.1 Q14.2 Q14.3 Q14.4 Q14.5
Clus e #1 2.355 3.387 3.387 3.129 2.903 2.919 2.968 2.935 2.274
Clus e #2 3.333 4.576 4.417 4.220 3.947 4.205 4.129 4.212 3.530
Di e ence
−0.978 −1.189 −1.030 −1.091 −1.044 −1.286 −1.161 −1.277 −1.256
Q15.1 Q15.2 Q15.3 Q16.1 Q16.2 Q16.3 Q17.1 Q17.2 Q17.3
Clus e #1 3.081 2.613 2.710 2.500 2.500 2.177 2.371 2.500 2.371
Clus e #2 3.947 3.848 3.955 4.076 4.091 3.432 3.788 3.879 3.735
Di e ence
−0.866 −1.235 −1.245 −1.576 −1.591 −1.255 −1.417 −1.379 −1.364
Q18.1 Q18.2 Q19.1 Q19.2 Q19.3 Q20.1 Q20.2
Clus e #1 3.097 2.839 2.823 2.565 2.419 2.629 2.452
Clus e #2 3.114 3.068 3.977 3.856 3.742 3.697 3.765
Di e ence
−0.017 −0.229 −1.154 −1.291 −1.323 −1.068 −1.313
5.3. SEM Model o Cus ome A i udes owa ds E-Go e nmen Se ices
The assessmen o consume a i udes owa ds adop ion o e-public se ices lacks
consensus on de ining inpu s and ou pu s, as highligh ed in he e iew o p e ious simila
esea ch (Sec ion 3). To add ess his issue, we i e a i ely employed he SEM me hod using
Sma PLS 3 so wa e (Ringe e al. 2015).
The algo i hm used o s uc u al eg ession modelling in ol ed he ollowing i e
s eps:
1. Fo mula e hypo heses abou cons uc s and hei in e ela ionships.
2. Iden i y indica o s o each cons uc .
3. Execu e he modelling p ocedu e and assess he model i .
4.
E alua e he quali y o he model; i sa is ac o y, p oceed o S ep 5, o he wise e u n
o S ep 3 o enhance he model.
5. Discuss he ob ained esul s.
In S ep 1, hypo heses abou model cons uc s and hei in e ela ionships a e o mu-
la ed, guided by he compa ison o exis ing models o use a i udes owa ds e-public
se ices (Table 2) (AlHadid e al. 2022).
H
1
:The e is a signi ican impac o Pe cei ed Use ulness on A i ude owa ds e-go e nmen
se ices.
H
2
:The e is a signi ican impac o Pe cei ed Ease o Use on A i ude owa ds e-go e nmen
se ices.
H3:The e is a signi ican impac o Social In luence on A i ude owa ds e-go e nmen se ices.
H
4
:The e is a signi ican impac o Facili a ing Condi ions on A i ude owa ds e-go e nmen
se ices.
H5:The e is a signi ican impac o Pe cei ed T us on A i ude owa ds e-go e nmen se ices.
H6:The e is a signi ican impac o Pe cei ed Risk on A i ude owa ds e-go e nmen se ices.
Adm. Sci. 2024,14, 54 21 o 30
H7:The e is a signi ican impac o Se ice Quali y on A i ude owa ds e-go e nmen se ices.
H
8
:Demog aphic cha ac e is ics ha e a s a is ically signi ican impac on cus ome sa is ac ion
wi h e-go e nmen se ices (AlHadid e al. 2022). The demog aphic cha ac e is ics include Gende ,
Age, Residence, Income and Educa ion le el.)
S ep 2. Iden i y indica o s o each cons uc .
The indica o s o la en a iables (8 cons uc s and 28 a iables) we e de i ed om
he su ey ques ionnai e (Ilie a e al. 2024). The measu emen model consis ed o 23 inpu
indica o s: PU1, PU2, and PU3 om he Pe cei ed Use ulness (PU); PE1, PE2, PE3, and PE4
om Pe cei ed Ease o Use (PE); SI1, SI2, and SI3 om Social In luence (SI); FC1, FC2, and
FC3 om Facili a ing Condi ions (FC); PT1, PT2, and PT3 om Pe cei ed T us (PT); PR1
and PR2 om Pe cei ed Risk (PR); SQT1, SQT2, SQT2, SQR1, SQR2, and SQR3 om Se ice
Quali y (SQ); and i e ou pu indica o s, ATT1, ATT2, ATT3, ATT4, and ATT5, om he
ou pu cons uc A i ude owa ds e-Go e nmen Se ices (ATT), ep esen ed in Figu e 5.
Adm. Sci. 2024, 14, x FOR PEER REVIEW 22 o 31
o he new model a e accep able, and he model examina ion con inues o es ablish he
alidi y and eliabili y o he cons uc s (S ep 4).
Cons uc Validi y and Reliabili y
The ini ial phase o he alidi y check equi es assessing bo h he measu emen
model and he s uc u al model. The measu emen model e alua es he alidi y and eli-
abili y o he cons uc s; his e alua ion encompasses e alua ing he eliabili y o he con-
s uc s and indica o s along wi h he con e gen alidi y and disc iminan alidi y o he
la en a iables. The s uc u al model is essen ial o de e mining he signi icance o he
p oposed hypo heses.
Figu e 5. Measu emen model wi h six la en a iables along wi h hei pa h coefficien s and p-
alues.
Fac o Loadings
Fac o loadings measu e he ex en o which each i em in he co ela ion ma ix is
linked o he speci ied p incipal componen . In ou model, all i ems demons a e ac o
loadings su passing he ecommended h eshold o 0.5 sugges ed by Hai e al. (Hai e
al. 2014). Figu e 6 and Table 5 display he model’s ac o loadings.
Figu e 6. SEM p ocedu e esul s, showing he eg ession coefficien o each cons uc and he coe -
icien o de e mina ion.
Figu e 5. Measu emen model wi h six la en a iables along wi h hei pa h coe icien s and p- alues.
S ep 3. Execu e he modelling p ocedu e and assess he model i .
The PLS algo i hm has been employed and model pa ame e s ha e been ob ained.
S ep 4. E alua e he quali y o he model. I sa is ac o y, p oceed o S ep 5; o he wise,
e u n o S ep 3 o enhance he model.
Based on e alua ion o he pa h coe icien s, he model does no align well wi h he
da ase . This disc epancy a ises om he p- alues o Pe cei ed Use ulness, Pe cei ed
Ease o Use, and Social In luence (0.437, 0.680, and 0.251, espec i ely), which exceed he
accep able h eshold (Figu e 5). Consequen ly, he p ocess e u ns o S ep 3 and modi ies
he model se ings by elimina ing ce ain ac o s. Now, he p- alues o he pa h coe icien s
o he new model a e accep able, and he model examina ion con inues o es ablish he
alidi y and eliabili y o he cons uc s (S ep 4).
Cons uc Validi y and Reliabili y
The ini ial phase o he alidi y check equi es assessing bo h he measu emen model
and he s uc u al model. The measu emen model e alua es he alidi y and eliabili y
o he cons uc s; his e alua ion encompasses e alua ing he eliabili y o he cons uc s
and indica o s along wi h he con e gen alidi y and disc iminan alidi y o he la en
Adm. Sci. 2024,14, 54 22 o 30
a iables. The s uc u al model is essen ial o de e mining he signi icance o he p oposed
hypo heses.
Fac o Loadings
Fac o loadings measu e he ex en o which each i em in he co ela ion ma ix is
linked o he speci ied p incipal componen . In ou model, all i ems demons a e ac o load-
ings su passing he ecommended h eshold o 0.5 sugges ed by Hai e al.
(Hai e al. 2014).
Figu e 6and Table 5display he model’s ac o loadings.
Adm. Sci. 2024, 14, x FOR PEER REVIEW 22 o 31
o he new model a e accep able, and he model examina ion con inues o es ablish he
alidi y and eliabili y o he cons uc s (S ep 4).
Cons uc Validi y and Reliabili y
The ini ial phase o he alidi y check equi es assessing bo h he measu emen
model and he s uc u al model. The measu emen model e alua es he alidi y and eli-
abili y o he cons uc s; his e alua ion encompasses e alua ing he eliabili y o he con-
s uc s and indica o s along wi h he con e gen alidi y and disc iminan alidi y o he
la en a iables. The s uc u al model is essen ial o de e mining he signi icance o he
p oposed hypo heses.
Figu e 5. Measu emen model wi h six la en a iables along wi h hei pa h coefficien s and p-
alues.
Fac o Loadings
Fac o loadings measu e he ex en o which each i em in he co ela ion ma ix is
linked o he speci ied p incipal componen . In ou model, all i ems demons a e ac o
loadings su passing he ecommended h eshold o 0.5 sugges ed by Hai e al. (Hai e
al. 2014). Figu e 6 and Table 5 display he model’s ac o loadings.
Figu e 6. SEM p ocedu e esul s, showing he eg ession coefficien o each cons uc and he coe -
icien o de e mina ion.
Figu e 6. SEM p ocedu e esul s, showing he eg ession coe icien o each cons uc and he
coe icien o de e mina ion.
Table 5. Fac o loadings o indica o s.
Indica o
Va iable
Fac o
Loading
Indica o
Va iable
Fac o
Loading
Indica o
Va iable
Fac o
Loading
Indica o
Va iable
Fac o
Loading
ATT1 0.834 FC1 0.926 PT1 0.946 SQT1 0.852
ATT2 0.861 FC2 0.965 PT2 0.956 SQT2 0.827
ATT3 0.831 FC3 0.893 PT3 0.96 SQT3 0.814
ATT4 0.914 PR1 0.959 SQR1 0.854
ATT5 0.740 PR2 0.962 SQR2 0.879
Indica o Mul icollinea i y
To assess he mul icollinea i y among indica o s, he Va iance In la ion Fac o (VIF)
s a is ic was calcula ed. A VIF alue below i e is conside ed accep able, indica ing ac-
cep able mul icollinea i y (Fo nell and Books ein 1982). Table 6p esen s he VIF alues,
demons a ing ha each indica o has a VIF below he ecommended h eshold.
Table 6. Cons uc eliabili y (DG ho and CR), con e gen alidi y (AVE), and mul icollinea i y
(VIF).
Fac o DG ho CR AVE VIF
Pe cei ed Risk 0.917 0.960 0.922 1.079
Pe cei ed T us 0.951 0.968 0.910 2.391
Facili a ing
Condi ions 0.924 0.949 0.862 1.341
Se ice Quali y 0.914 0.926 0.715 2.217
A i ude 0.898 0.921 0.702
DG ho: Dillon–Golds ein’s ho (>0.7), CR: Composi e Reliabili y (>0.6); AVE: A e age Va iance Ex ac ed (>0.5);
VIF: Va iance In la ion Fac o s (<5).
Reliabili y Analysis
To assess cons uc eliabili y, wo main me hods—Dillon–Golds ein’s ho (DG ho)
and Composi e Reliabili y (CR)—we e applied o measu ing epea abili y. To ensu e
adequa e eliabili y, bo h he DG ho and CR alues should su pass 0.7 (Fo nell and
Books ein 1982). The DG ho alues anged om 0.898 o 0.951, while he CR alues
Adm. Sci. 2024,14, 54 23 o 30
anged om 0.921 o 0.968 (Table 6). Consequen ly, he DG ho and CR alues o all la en
a iables in he model a e accep able, indica ing eliable coe icien s o all cons uc s.
Cons uc alidi y equi es wo ypes o alidi y assessmen : con e gen alidi y and
disc iminan alidi y.
Con e gen Validi y
Con e gen alidi y e e s o he le el o consis ency among mul iple measu es o
he same concep . To assess he con e gen alidi y o he cons uc , he a e age a iance
ex ac ed (AVE) was calcula ed, wi h a minimum h eshold o 0.5 (Fo nell and Books ein
1982). The AVE sco es o all cons uc s we e s a is ically signi ican , indica ing he s ong
con e gen alidi y o ou model (Table 6).
Disc iminan Validi y
Disc iminan alidi y e e s o he deg ee o which measu es o dis inc concep s can
be dis inguished om each o he .
He e o ai –Mono ai Ra io (HTMT)
To assess disc iminan alidi y, he HTMT (He e o ai –Mono ai ) a io calcula es he
co ela ion be ween cons uc s. The h eshold o HTMT a ies in he li e a u e, ypically
alling be ween 0.85 and 0.9. The esul s o ou model, ou lined in Table 7, show ha he
HTMT a ios o he cons uc s a e below he speci ied h eshold o 0.85 and a e s a is ically
signi ican .
Table 7. Disc iminan alidi y—HTMT.
Fac o ATT FC PR PT SQ
A i ude
Facili a ing Condi ions
0.669
Pe cei ed Risk 0.193 0.033
Pe cei ed T us 0.623 0.520 0.145
Se ice Quali y 0.67 0.473 0.082 0.774
Assessmen o S uc u al Model
The p- alues o he model cons uc s indica e a signi ican impac on use a i udes
owa ds e-public se ices, wi h alues below 1% o Facili a ing Condi ions, Pe cei ed Risk,
and Se ice Quali y and below 5% o Pe cei ed T us , as shown in Figu e 7and Table 8.
These indings align wi h hypo heses H
4
–H
7
and p e ious simila esea ch. The eg ession
coe icien s o all inpu ac o s a e posi i e.
Adm. Sci. 2024, 14, x FOR PEER REVIEW 24 o 31
Table 7. Disc iminan alidi y—HTMT.
Fac o ATT FC PR PT SQ
A i ude
Facili a ing Con-
di ions 0.669
Pe cei ed Risk 0.193 0.033
Pe cei ed T us 0.623 0.520 0.145
Se ice Quali y 0.67 0.473 0.082 0.774
Assessmen o S uc u al Model
The p- alues o he model cons uc s indica e a signi ican impac on use a i udes
owa ds e-public se ices, wi h alues below 1% o Facili a ing Condi ions, Pe cei ed
Risk, and Se ice Quali y and below 5% o Pe cei ed T us , as shown in Figu e 7 and
Table 8. These indings align wi h hypo heses H4–H7 and p e ious simila esea ch. The
eg ession coefficien s o all inpu ac o s a e posi i e.
Figu e 7. Pa h coefficien s and p- alues—inne and ou e model.
Table 8. The pa h coefficien o he ela ionship be ween la en a iables.
Hypo hesis 𝜷 Sample
Mean Mean SD S a is ics p Values R2
2 Q
2
H4 Facili a ing Condi ions
→ A i ude 0.389 0.389 0.387 0.078 4.972 0.000
0.559
0.256
0.385
H5 Pe cei ed Risk → A i-
ude 0.185 0.185 0.182 0.051 3.628 0.000 0.072
H6 Pe cei ed T us → A i-
ude 0.210 0.21 0.204 0.085 2.461 0.014 0.042
H7 Se ice Quali y → A i-
ude 0.276 0.276 0.284 0.094 2.921 0.004 0.078
Rega ding he ob ained s uc u al model, he pa hways FC → ATT and SQ → ATT
exp ess ela i ely la ge effec s, while he PT → ATT and PR → ATT ela ionships demon-
s a e weak in luences.
S ep 5. Discuss he ob ained esul s.
The easons o ejec ing he effec s o Pe cei ed Use ulness (H1), Pe cei ed Ease o
Use (H2), and Social In luence (H3) on Use A i ude owa ds e-go e nmen se ices (Fig-
u e 5) can be explained based on he speci ic con ex o ou s udy. The new po al o he
NRA has been de eloped using he la es IT echnologies, which has changed supposed
use pe cep ions. Use s do no conside Pe cei ed Use ulness as a c i ical ac o due o he
high le el o ma u i y o he new po al. Addi ionally, he elec onic equi alen s o NRA
public se ices ha e been a ailable since 2012 and a e well-es ablished, leading o he e-
jec ion o Pe cei ed Ease o Use as an in luence on Use A i ude. Fu he mo e, as use s
ha e accumula ed sufficien expe ience wi h he po al, he in luence o social ne wo ks
Figu e 7. Pa h coe icien s and p- alues—inne and ou e model.
Adm. Sci. 2024,14, 54 24 o 30
Table 8. The pa h coe icien o he ela ionship be ween la en a iables.
Hypo hesis βSample
Mean Mean
SD
S a is ics pValues R2 2Q2
H4Facili a ing Condi ions →A i ude
0.389
0.389 0.387
0.078
4.972 0.000
0.559
0.256
0.385
H5Pe cei ed Risk →A i ude
0.185
0.185 0.182
0.051
3.628 0.000
0.072
H6Pe cei ed T us →A i ude
0.210
0.21 0.204
0.085
2.461 0.014
0.042
H7Se ice Quali y →A i ude
0.276
0.276 0.284
0.094
2.921 0.004
0.078
Rega ding he ob ained s uc u al model, he pa hways FC
→
ATT and SQ
→
ATT ex-
p ess ela i ely la ge e ec s, while he PT
→
ATT and PR
→
ATT ela ionships demons a e
weak in luences.
S ep 5. Discuss he ob ained esul s.
The easons o ejec ing he e ec s o Pe cei ed Use ulness (H
1
), Pe cei ed Ease
o Use (H
2
), and Social In luence (H
3
) on Use A i ude owa ds e-go e nmen se ices
(Figu e 5) can be explained based on he speci ic con ex o ou s udy. The new po al o he
NRA has been de eloped using he la es IT echnologies, which has changed supposed
use pe cep ions. Use s do no conside Pe cei ed Use ulness as a c i ical ac o due o
he high le el o ma u i y o he new po al. Addi ionally, he elec onic equi alen s o
NRA public se ices ha e been a ailable since 2012 and a e well-es ablished, leading o he
ejec ion o Pe cei ed Ease o Use as an in luence on Use A i ude. Fu he mo e, as use s
ha e accumula ed su icien expe ience wi h he po al, he in luence o social ne wo ks
has dec eased. Consequen ly, H
3
, which pe ains o Social In luence, is no suppo ed in
ou indings.
Ou esul s a e consis en wi h hose ob ained by Mensah e al. (2020) in hei s udy
on he adop ion o Chinese e-go e nmen se ices. Bo h s udies ejec he signi icance o
he same h ee ac o s—Pe cei ed Use ulness, Pe cei ed Ease o Use, and Social In luence
(Table 2). One possible eason o his his alignmen is he ac ha bo h Bulga ia and
China ha e de eloped e-go e nmen sys ems, anked in he VHEGDI g oup acco ding
o he las UN su ey (UN DESA 2022). Howe e , esul s om a p e ious s udy by Xie
e al. (Xie e al. 2017), conduc ed in China i e yea s ea lie , show a di e en ou come, wi h
hese h ee ac o s signi ican ly in luencing use a i udes owa ds e-go e nmen se ices.
This di e ence suppo s ou assump ion ega ding he impac o IT ma u i y. Addi ion-
ally, Table 2indica es ha in some coun ies less de eloped in in o ma ion echnologies
hese h ee ac o s only pa ially in luence use a i udes owa ds e-go e nmen se ices
(ElKheshin and Saleeb 2020;Nug oho e al. 2022).
In he i ed SEM model, Facili a ing Condi ions (H
4
) demons a ed he s onges
posi i e ela ionship (Figu e 7,
β
= 0.389, p< 0.001) wi h a i ude owa ds e-public se ices.
Fo ci izens, Facili a ing Condi ions encompass access o echnological esou ces along wi h
he p o ision o echnical suppo , aiding hem du ing ansac ions wi h e-go e nmen .
The obus impac o Facili a ing Condi ions indica es ha a ailable elecommunica ion
se ices, so wa e quali y, a mobile- iendly in e ace, and accessible suppo sys ems can
enhance cus ome sa is ac ion. This inding aligns wi h he esul s o p e ious simila
s udies by Ku da i e al., Camille i, AlHadid e al., Nug oho e al., and Ga cia-Rio e al.
(Ku ali e al. 2017;Camille i 2020;AlHadid e al. 2022;Nug oho e al. 2022;Ga cia-Rio
e al. 2023).
The esul o es ing H
5
, he e ec o Pe cei ed T us , shows ha con idence measu es
o e-public se ices can inc ease he in en ion o use s o adop new e-go e nmen pla -
o ms (
β
= 0.210, p
≤
0.05). Ci izens need o us ha he in o ma ion a ailable on hese
pla o ms is accu a e and up o da e in o de o ely on and use hese e-se ices. When
ci izens us hese online pla o ms, hey a e mo e likely o ac i ely pa icipa e, p o ide
eedback, and engage in a ious go e nmen ini ia i es, leading o a mo e in e ac i e and
esponsi e go e nance model. This ou come aligns wi h esea ch ha iden i ies his a i-
