Measuring efficiency and satisfaction in the context of digital transformation

Krejnus, Matej et al. Article Measuring efficiency and satisfaction in the context of digital transformation Administrative Sciences Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Krejnus, Matej et al. (2024) : Measuring efficiency and satisfaction in the context of digital transformation, Administrative Sciences, ISSN 2076-3387, MDPI, Basel, Vol. 14, Iss. 9, pp. 1-26, https://doi.org/10.3390/admsci14090217 This Version is available at: https://hdl.handle.net/10419/321031 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ Citation: Krejnus, Matej, Katarína RepkováŠtofková, Jana Štofková, Zuzana Štofková, Erika Louˇcanová, Adela Poliaková, and Lucia Šujanská. 2024. Measuring Efficiency and Satisfaction in the Context of Digital Transformation. Administrative Sciences 14: 217. https://doi.org/ 10.3390/admsci14090217 Received: 6 August 2024 Revised: 6 September 2024 Accepted: 9 September 2024 Published: 12 September 2024 Copyright: © 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). administrative sciences Article Measuring Efficiency and Satisfaction in the Context of Digital Transformation Matej Krejnus 1, Katarína RepkováŠtofková1, Jana Štofková1, Zuzana Štofková2, Erika Louˇcanová3,* , Adela Poliaková2and Lucia Šujanská2 1Department of Communication, University of Zilina, 010 26 Zilina, Slovakia 2Department of Economics, University of Zilina, 010 26 Zilina, Slovakia; [email protected] (A.P.) 3Department of Marketing, Trade and World Forestry, Technical University of Zvolen, T. G. Masaryka 24, 960 53 Zvolen, Slovakia *Correspondence: [email protected] Abstract: Currently, much attention is paid to digital transformation in all areas, including the public sphere. The latest studies show that it is necessary for the public sector to monitor the efficiency and satisfaction with the services provided. However, there are significant gaps in research in this area, including in Slovakia. This research proposes and applies the measurement of efficiency using the DEA method in the context of e-Government, provides a comparison of the roles of states in the use of public electronic services in the EU, and applies the method of measuring satisfaction using the American Customer Satisfaction Index, focused on the central state portal in Slovakia. The main methods that were used to fulfil the objectives of the work were data envelopment analysis, “DEA”, and the American Customer Satisfaction Index, “ACSI”. Other methods used include the Mann–Whitney U test, the chi-squared test, and Sperm correlation analysis. From the results of the work, it is possible to conclude that ACSI can be applied within Slovakia. Furthermore, the results show a strong correlation between perceived quality and satisfaction, which is 0.855. Overall satisfaction with the central state portal of public electronic services reached 61.7%. We conclude that it would be appropriate and possible to use ACSI as part of DEA measurement. Keywords: DEA; efficiency; American Customer Satisfaction Index; ACSI 1. Introduction Currently, a period of digital transformation is underway, in which almost all information and knowledge are transformed into a digital form, which allows us to store a large amount of data. In addition, it allows us to efficiently process, search, archive and categorize data. Humans are developing ever more advanced systems that extend into the field of artificial intelligence, which is based only on binary computer notation (Harari 2015). Citizens of countries constantly communicate with public administration institutions. Therefore, it is important to improve and simplify communication using information and communication technologies (ICT) that contribute to more effective interaction between the given subjects. Knowledge is becoming increasingly important in our society, and its proper use is a key aspect of success in today’s digital world. Efficiency represents the use of economic resources that bring the maximum level of satisfaction achievable with given inputs and technologies (Beˇnováet al. 2006). The efficiency of any phenomenon or process is the result of the relationship between the size of the inputs put into the realization of this phenomenon or process and the size of the outputs that result from the realization of this phenomenon. It follows that the effective unit reaches the value 1 and the other units reach values in the interval (0.1) (Beˇnováet al. 2006). The goal is to approach the value of 1 in efficiency. In this context, an important question is how to measure inputs and outputs when it comes to the public sector and the Adm. Sci. 2024,14, 217. https://doi.org/10.3390/admsci14090217 https://www.mdpi.com/journal/admsci Adm. Sci. 2024,14, 217 2 of 26 field of e-Government (Beˇnováet al. 2006). Economy, efficiency and effectiveness represent the 3Es. The 3E principles refer to the following: 1. Economy: performing activities at the lowest possible cost. 2. Efficiency: maximizing output with the minimum input. 3. Effectiveness: achieving the desired objectives or producing the intended effect (Greenlaw et al. 2022). To ensure the principle of purchasing value for money is applied, it is optimal for the fulfillment of objectives not to assess compliance with the 3E principles individually, but as a whole (see Figure 1) (Sborník Dobré Praxe-Mpvs.Cz 2020). Adm. Sci. 2023, 13, x FOR PEER REVIEW 2 of 27 (Beňová et al. 2006). The goal is to approach the value of 1 in efficiency. In this context, an important question is how to measure inputs and outputs when it comes to the public sector and the field of e-Government (Beňová et al. 2006). Economy, efficiency and effectiveness represent the 3Es. The 3E principles refer to the following: 1. Economy: performing activities at the lowest possible cost. 2. Efficiency: maximizing output with the minimum input. 3. Effectiveness: achieving the desired objectives or producing the intended effect (Greenlaw et al. 2022). To ensure the principle of purchasing value for money is applied, it is optimal for the fulfillment of objectives not to assess compliance with the 3E principles individually, but as a whole (see Figure 1) (Sborník Dobré Praxe-Mpvs.Cz 2020). Figure 1. Principle of 3E. The public sector provides services and brings some new possibilities, so it is important to design new input and output factors. Djellal and Gallouj (2008) divide measurement methods into two categories: index-based methods and threshold value methods. Index-based methods are based on pointers (Djellal and Gallouj 2008). Digitization can be described as an important technological trend that is transforming society, business and the functioning of the state. Although there are many benefits of digitization, it requires investment and associated costs (What Is Digitalization 2018). Digitization is the technical process of transforming analogue information flows into digital data that have discontinuous values, which are based on two separate states (Feldman 2003). Digital technology takes information and decomposes it into its smallest components. By transforming an analog signal into distinct pieces, digitization enables the manipulation of information, text, graphics, software code, sound and video, i.e., its informing, transforming ability (Gorenšek and Kohont 2018; What Is Digitalization, 2018). The extent and impact of the digital economy are determined by the ways in which individuals adopt information and communication technologies (Feliciano-Cestero et al. 2023). As a result of the ongoing globalization and informatization projects, information and communication technologies are gradually being integrated into the processes of egovernment. It is increasingly recognized that e-government is moving towards a holistic approach and that governance in the field of sustainable development requires strategic national planning (UN E-Government Survey 2022). The authors Fine and Johnson pointed out that public sector companies face many similar challenges to private sector companies, especially in the area of digitization (Fine and Johnson 2005). Breaugh and colleagues argue that public administration digitization projects require collaborative approaches for successful development and implementation (Breaugh et al. 2023). In the Figure 1. Principle of 3E. The public sector provides services and brings some new possibilities, so it is important to design new input and output factors. Djellal and Gallouj (2008) divide measurement methods into two categories: index-based methods and threshold value methods. Indexbased methods are based on pointers (Djellal and Gallouj 2008). Digitization can be described as an important technological trend that is transforming society, business and the functioning of the state. Although there are many benefits of digitization, it requires investment and associated costs (What Is Digitalization 2018). Digitization is the technical process of transforming analogue information flows into digital data that have discontinuous values, which are based on two separate states (Feldman 2003). Digital technology takes information and decomposes it into its smallest components. By transforming an analog signal into distinct pieces, digitization enables the manipulation of information, text, graphics, software code, sound and video, i.e., its informing, transforming ability (Gorenšek and Kohont 2018;What Is Digitalization 2018). The extent and impact of the digital economy are determined by the ways in which individuals adopt information and communication technologies (Feliciano-Cestero et al. 2023). As a result of the ongoing globalization and informatization projects, information and communication technologies are gradually being integrated into the processes of egovernment. It is increasingly recognized that e-government is moving towards a holistic approach and that governance in the field of sustainable development requires strategic national planning (UN E-Government Survey 2022). The authors Fine and Johnson pointed out that public sector companies face many similar challenges to private sector companies, especially in the area of digitization (Fine and Johnson 2005). Breaugh and colleagues argue that public administration digitization projects require collaborative approaches for successful development and implementation (Breaugh et al. 2023). In the business sphere, measuring the efficiency of processes is essential, as well as measuring customer satisfaction. Therefore, this article focuses on the importance of measuring efficiency and satisfaction in the context of digital transformation within the public sector. Adm. Sci. 2024,14, 217 3 of 26 It is also necessary to mention that in the Slovak Republic, there is no regular measurement of satisfaction with new or existing services. The paper fills this gap and suggests a way in which it could be implemented and possibly adjusted. Similarly, this paper fills a gap in measuring the efficiency of data envelopment analysis (DEA analysis), since similar measurements using several available indicators were not identified within the European Union (EU). Overall, this work presents an overview for satisfaction assessment in e-government services, measuring the efficiency of the state of e-Government and providing a basis for future research in this area. Citizens outside big cities benefit from using e-Government services. E-Government services can be much more useful for residents who live in rural areas (Roy et al. 2015; Seo and Bernsen 2016). Urban residents use the Internet twice as often as rural residents (Local E-Government Development 2022). The benefits of online communication in rural areas may be greater than in urban areas due to transportation costs, time constraints, or traffic congestion (Roy et al. 2015). The usefulness of online services is especially perceived in terms of flexibility, speed, availability, mobility and education. It is important that the content is expressed in a comprehensible and readable form, in addition to the correct organization of information and security of communication, i.e., perceived trust, which ensures more reliable information when using e-Government services. Government information provided by government institutions on websites can generally be considered and perceived as trustworthy; some requests or information may be misunderstood if not provided verbally or personally (Roy et al. 2015). E-Government is divided into several levels—e-Government 1.0 to 3.0. E-Government services are constantly evolving, becoming more sophisticated and complex and, thus, increasing their costs for operation and development (Štandardy Vlastností Elektronických Služieb Verejnej Správy 2014). E-Government 1.0 focuses on the provision of transactional public administration services for citizens and businesses. E-Government 2.0 is aimed at increasing citizen participation, as well as openness and accountability. E-Government 3.0 is considered as a strategic response to the growing problems and challenges that modern society has to cope with, i.e., the flood of data from the second generation of e- Government. The goal is to provide support in policy making and solving social problems for the well-being of citizens using new technologies (Vrabie 2023;Charalabidis et al. 2019). The National Agency for the Information Society considers the concept of e-Government 3.0 as a set of measures that provides low-cost, high-quality services designed to ensure the satisfaction of citizens. In addition, it adapts to the needs of citizens, promotes entrepreneurship and increases efficiency and better access to information and services. Consequently, it adapts services to citizens using new technologies (Nam 2013). With the arrival of new technologies, new levels of e-Government services can be defined. It is important to ensure that web portals do not comprise technical problems that can negatively affect the opinions of citizens. In addition, it is important to secure functionality at all levels, so that there are no malfunctions, page slowdowns, etc. (Muhammad and Hromada 2023;Paul and Paul 2023). The development of the levels of e-Government is shown in Table 1. The efficiency of e-Government is considered by several authors to be one of the most important drivers of progress in e-Government, and the efficiency of electronic public administration includes various disciplinary perspectives (Chan et al. 2008;Purón-Cid 2014). The efficiency of e-Government can also be measured by measuring user satisfaction, process efficiency, security and trust, innovation and adaptability, etc. In addition, it is possible to identify other areas that are closely related to e-Government (Purón-Cid 2014). The improvement of quality in public services is an important issue. The OECD has identified that improving service quality is also a measure of success. In addition, several IT tools were found to improve services, such as online portals, targeted customization, email communication, authentication, corruption reduction, transparency, etc. (West 2004; von Haldenwang 2004;Purón-Cid 2014). Adm. Sci. 2024,14, 217 4 of 26 Table 1. Development stages of e-Government. e-Government 1.0 e-Government 2.0 e-Government 3.0 Main intention Better service Openness and cooperation Solving societal problems, ensuring citizen welfare, optimizing resources Main method Online public administration Open and coordinated governance Intelligent management Level of use National National and local From local to international Tools used and service delivery Web portal (personal visit—one-way communication) Web portal, social media (two-way communication) Ubiquitous smart services, smartphones, apps ICT area Infrastructure and organization People and data Artificial intelligence (AI) technologies and Internet of Things (IoT) infrastructure An overview of efficiency measurement models was carried out in the area of e- Government. DEA has proven to be a useful tool for measuring performance, as well as the efficiency of electronic public administration and digital development in various countries, such as China, South Korea, Mexico, Taiwan and the European Union (see Table 2). Table 2. DEA models for measuring efficiency in the field of e-Government. Authors Input Data Output Data State and Level Models e- Government (Wu and Guo 2015) GDP index, Internet penetration index, operational maintenance index Information disclosure index, interaction index, education services index, employment services index Country Regional DEA— J-SBM, S-SBM (Afonso et al. 2010) Technical quality of governance, democratic quality of governance, government expenditure Higher education attainment, child survival rate (HDI), life expectancy at birth, income equality (Gini index), median income, deflation, GDP/capita, employment rate, highway density, share of renewable energy European Union States DEA— BCC (Luna et al. 2013) Number of Internet users, number of computer users, number of smartphone users, efficiency of public administration, infrastructure index Level of information, interactions, transactions, level of integration, citizen participation Mexico DEA— CCR, BCC (Seo et al. 2018) IT budget (in 42 central administrations and demand for information systems from 2014 to 2017), number of employees Number of public services (korea.go.kr), number of open public data (public portal for open data (data.go.kr), Government 3.0 activities (e.g., achievement reports) States DEA—CCR, BCC (Liu and Tang 2009) Number of personal computers per hundred inhabitants, ratio of computers relative to the Internet, number of Internet servers per ten thousand inhabitants. share of population on websites, GDP per capita, level of education in the region, level of security of web services. Information provision, interaction (communication) and information processing on the authorities’ websites South Korea Resorts DEA—C2R (Hsieh et al. 2013) Number of computers, number of servers., systems development budgets, budgets for maintenance of systems, payments to IT staff. Annual revenue of government units, number of completed transactions from e-commerce, number of malicious attacks by outsiders, website capacity, time taken to upload/download information. China Regional DEA— CCR, BCC (Yalçın 2021) Capital investment (share of public expenditure), investment in human resources (share of civil servants). Government websites and services (level of online services and their integration in public administration), impact of new media channels in public administration (Government Weibo, competitiveness index) China Regional DEA— BCC a Malmquist index Adm. Sci. 2024,14, 217 5 of 26 The factors that are most commonly used in the construction of input and output models are the economic factor, the technological factor and the human factor. The main variables include, for example, the level of digital services provided, gross domestic product (GDP), the employment rate in public administration, the information and communication infrastructure variables themselves, etc. A fundamental element in measuring efficiency is data homogeneity, which means that each datum has to be the same within the comparison of DMU units. The main measurement model is DEA, using CCR (Charnes–Cooper– Rhodes) and BCC (Banker–Charnes–Cooper) models. The authors Wu and Guo (2015) consider the measurement of e-Government performance within provincial governments in China to be an important topic and conclude that the authorities are quick to respond to citizens’ demands. Afonso et al. (2010) recommend that government officials focus on improving efficiency in new EU member states and that they meet the EU’s goals for raising living standards. Seo et al. (2018) state that the Korean government’s 3.0 initiative has improved the efficiency of public service delivery (Wu and Guo 2015;Seo et al. 2018). There are various indices that are relevant and evaluate the levels of digital society and e- Government. These indicators are important in assessing the level of digitization, efficiency and involvement of government digital services institutions in different countries. These indices and assessments are policy-making tools that help countries and regions to identify areas where they can improve their digital level. Individual authors used various international indicators from databases such as Eurostat, The Organisation for Economic Co-operation and Development (OECD), the European Commission, the World Bank, etc., when measuring efficiency. The United Nations created a database in which data on the development of e-Government in 193 countries of the world are recorded and stored. An important indicator in the development and progress of e-Government is the e-Government Development Index (EGDI). The E-Government development index measures the willingness and ability of certain governments to use information and communication technologies to provide public services. The index can reach values in the range from 0 to 1, where 1 represents the maximum value and 0 the minimum value (E-Government Development Index 2022). Table 3presents the results. Table 3. Values of the e-Government Development Index (EDGI). Ranking 2022 Country EGDI 2022 EGDI 2020 EGDI 2018 Comparison in 2022 in [%] k 2020 in (%) k 2018 Order to 2018 Numbers to 2018 TOP 10 Denmark 0.9717 0.9758 0.9150 −0.41 5.67 / 57 ×10−3 Finland 0.9533 0.9452 0.8815 0.81 7.18 +4 72 ×10−3 J. Republic of Korea 0.9529 0.956 0.9010 −0.31 5.19 / 52 ×10−3 New Zealand 0.9432 0.9339 0.8806 0.93 6.26 +4 63 ×10−3 Iceland 0.9410 0.9101 0.8316 3.09 10.94 +14 109 ×10−3 Sweden 0.9410 0.9365 0.8882 0.45 5.28 / 53 ×10−3 Australia 0.9405 0.9432 0.9053 −0.27 3.52 −535 ×10−3 Estonia 0.9393 0.9473 0.8486 −0.80 9.07 +8 91 ×10−3 Netherlands 0.9384 0.9228 0.8757 1.56 6.27 +4 63 ×10−3 United States of America 0.9151 0.9297 0.8769 −1.46 3.82 +1 38 ×10−3 12. Singapore 0.9133 0.9762 0.9663 −0.17 3.21 −532 ×10−3 20. Austria 0.8801 0.8914 0.8301 −1.13 5.00 / 50 ×10−3 29. Latvia 0.8599 0.7798 0.6996 8.01 16.03 +28 160 ×10−3 34. Poland 0.8437 0.8255 0.7926 0.18 5.11 −151 ×10−3 45. Czech Republic 0.8088 0.8531 0.7084 −0.94 10.04 +9 100 ×10−3 47. Slovakia 0.8008 0.8135 0.7155 −0.47 8.53 +2 85 ×10−3 Adm. Sci. 2024,14, 217 6 of 26 The table shows that e-governance is a dynamic process that is influenced to a certain extent by various factors such as policy, the level of investment, implementation and technological development. The biggest improvement compared to 2018 was achieved by Iceland and Latvia, which means that these countries have recently emphasized the modernization of e-Government. Denmark maintained its leadership compared to 2018, which means stability in the area of e-Government, with constant improvement of services. Even a country with a lower level of economic power can have a high level of e-Government. Other commonly offered online services include applying for government vacancies and business licenses, applying for certificates and paying utility bills. The evaluation of e-Government is carried out by the European Commission and compares the levels of e-Government in individual countries. The comparison is made in four areas, according to which the indicators for the government are measured (see Figure 2) (E-Government Benchmark 2021). Individual areas are focus on the user, transparency, login and eID and cross-border services. The results achieved by Malta and Estonia indicate that their e-Governments are most focused on users, transparency, technological equipment and services, and they are open to users from other European countries as well. These countries were followed by Luxembourg (87%), Iceland (86%), the Netherlands (85%), Finland (85%), Denmark (84%), Lithuania (83%), Latvia (80%), Norway (79%), Spain (79%) and Portugal (81%). Adm. Sci. 2023, 13, x FOR PEER REVIEW 7 of 27 Figure 2. Total e-Government benchmark scores for EU countries (OECD Better Life Index 2022). From Figure 2, it can be concluded that Denmark and Estonia are significant leaders in the digitalization of public administration and that they achieved above-standard results for most indicators. By contrast, Slovakia achieved lower values in selected indicators, or decreases in some areas. Slovakia should pay more attention to building e- Government and target areas such as education, the better participation of citizens in public sector and involvement in collaborations in the area of e-Government, such as with Estonia and Finland. When carrying out the analysis in Slovakia and abroad, we also focused on the evaluation of the central portals of the public administrations of individual countries. The central portals in various countries are different. Denmark, Estonia and Finland are also leaders in terms of the results achieved within the EU27 (27 states of the European Union). In some areas, Singapore achieves a better or equal level. In Denmark, Finland and Singapore, citizens mainly access e-Government services via smartphones. In Singapore, this adoption of this approach is as high as 83%. In Slovakia, there is still a large share of citizens who do not communicate with institutions electronically, at approximately 30% less than in other countries. From the performed analysis, we identified a gap in research on the measurement of efficiency and satisfaction in the context of digital transformation in Slovakia. Efficiency measurement models within the European Union are focused more on EU digitization and not on EU e-Government, which forms part of digitization. For this reason, we decided to measure and identify efficiency within the European Union using available data and indices. 2. Results The Results section is divided into two parts. The first part identifies the level of portal usage. The testing of hypotheses and comparisons between urban and rural residents were carried out. Furthermore, the second part is the calculation of the American Customer Satisfaction Index (ACSI). The second part is the results of the DEA measurement. 1. Identification of the use of electronic public administration services and measurement of satisfaction with the state portal (Slovensko.sk)—ACSI. 2. Identification of the effectiveness of e-Government in Slovakia—DEA. 2.1. The First Part—Primary Research in Slovakia The rate of the use of public services through the state portal in Slovakia is 74%, and 26% of citizens do not use this portal. Rural portals are used by 88% and other services are used by 87% of the respondents (see Table 4). Other services include electronic vignettes, e-prescriptions, ePN, the online land registry, etc. The basic access points include the Figure 2. Total e-Government benchmark scores for EU countries (OECD Better Life Index 2022). From Figure 2, it can be concluded that Denmark and Estonia are significant leaders in the digitalization of public administration and that they achieved above-standard results for most indicators. By contrast, Slovakia achieved lower values in selected indicators, or decreases in some areas. Slovakia should pay more attention to building e-Government and target areas such as education, the better participation of citizens in public sector and involvement in collaborations in the area of e-Government, such as with Estonia and Finland. When carrying out the analysis in Slovakia and abroad, we also focused on the evaluation of the central portals of the public administrations of individual countries. The central portals in various countries are different. Denmark, Estonia and Finland are also leaders in terms of the results achieved within the EU27 (27 states of the European Union). In some areas, Singapore achieves a better or equal level. In Denmark, Finland and Singapore, citizens mainly access e-Government services via smartphones. In Singapore, this adoption of this approach is as high as 83%. In Slovakia, there is still a large share of citizens who do not communicate with institutions electronically, at approximately 30% less than in other countries. From the performed analysis, we identified a gap in research Adm. Sci. 2024,14, 217 7 of 26 on the measurement of efficiency and satisfaction in the context of digital transformation in Slovakia. Efficiency measurement models within the European Union are focused more on EU digitization and not on EU e-Government, which forms part of digitization. For this reason, we decided to measure and identify efficiency within the European Union using available data and indices. 2. Results The Results section is divided into two parts. The first part identifies the level of portal usage. The testing of hypotheses and comparisons between urban and rural residents were carried out. Furthermore, the second part is the calculation of the American Customer Satisfaction Index (ACSI). The second part is the results of the DEA measurement. 1. Identification of the use of electronic public administration services and measurement of satisfaction with the state portal (Slovensko.sk)—ACSI. 2. Identification of the effectiveness of e-Government in Slovakia—DEA. 2.1. The First Part—Primary Research in Slovakia The rate of the use of public services through the state portal in Slovakia is 74%, and 26% of citizens do not use this portal. Rural portals are used by 88% and other services are used by 87% of the respondents (see Table 4). Other services include electronic vignettes, e-prescriptions, ePN, the online land registry, etc. The basic access points include the central state portal, Slovensko.sk, the rural portals of municipalities and other electronic services. The skewness may indicate that respondents are less satisfied or have a different experience, which is reflected in the lower-scale values. Table 4. Descriptive analysis of satisfaction questions from the primary questionnaire. Other Electronic Services State Portal (Slovensko.sk) Rural Portal Rate of use of basic access points in Slovakia number of respondents in % yes no yes no yes no 87% 13% 74% 26% 88% 12% Min 1 1 1 Max 10 10 10 Average 6.986 6.621 6.741 Modus 8 8 5 Median 7 7 7 Standard deviation 1.900 1.976 2.355 Peakiness −0.050 −0.414 −0.392 Skewness −0.512 −0.325 −0.543 N 368 272 174 Based on the analysis, hypotheses were established. The hypotheses were tested using the chi-squared test between the city and rural areas. The aim was to find out the difference in use between citizens living in cities and those in rural areas. The research question (RQ) was as follows: Does the type of permanent residence, i.e., city, influence the central portal for public electronic services or electronic state services? Hypothesis 0AB (H0A). There is no significant difference in the level of use of the central state portal for public electronic services (Slovensko.sk) between citizens living in cities and in rural areas. Hypothesis 1A (H1A). There is a significant difference in the level of use of the central state portal (Slovensko.sk) between citizens living in cities and in rural areas. Adm. Sci. 2024,14, 217 8 of 26 Research question 1 (RQ 1): Do you currently live in a city or a rural area? Research question 2 (RQ 2): How often do you visit the portal (slovensko.sk) (see Figure 3)? Adm. Sci. 2023, 13, x FOR PEER REVIEW 8 of 27 central state portal, Slovensko.sk, the rural portals of municipalities and other electronic services. The skewness may indicate that respondents are less satisfied or have a different experience, which is reflected in the lower-scale values. Table 4. Descriptive analysis of satisfaction questions from the primary questionnaire. Other Electronic Services State portal (Slovensko.sk) Rural Portal Rate of use of basic access points in Slovakia number of respondents in % yes no yes no yes no 87% 13% 74% 26% 88% 12% Min 1 1 1 Max 10 10 10 Average 6.986 6.621 6.741 Modus 8 8 5 Median 7 7 7 Standard deviation 1.900 1.976 2.355 Peakiness −0.050 −0.414 −0.392 Skewness −0.512 −0.325 −0.543 N 368 272 174 Based on the analysis, hypotheses were established. The hypotheses were tested using the chi-squared test between the city and rural areas. The aim was to find out the difference in use between citizens living in cities and those in rural areas. The research question (RQ) was as follows: Does the type of permanent residence, i.e., city, influence the central portal for public electronic services or electronic state services? Hypothesis 0AB (H0A). There is no significant difference in the level of use of the central state portal for public electronic services (Slovensko.sk) between citizens living in cities and in rural areas. Hypothesis 1A (H1A). There is a significant difference in the level of use of the central state portal (Slovensko.sk) between citizens living in cities and in rural areas. Research question 1 (RQ 1): Do you currently live in a city or a rural area? Research question 2 (RQ 2): How often do you visit the portal (slovensko.sk) (see Figure 3)? Figure 3. Use of public electronic services (slovensko.sk) for respondents rural and city areas. Figure 3. Use of public electronic services (slovensko.sk) for respondents rural and city areas. The positive answers that were used to represent a “yes” answer were always, often, sometimes, and occasionally. A negative answer was represented by “no”—we do not use it. If ChíX2-test < ChíX2-critical, we accepted hypothesis H0B. In this example, based on calculations, with ChíX2-test > ChíX2-critical, we accept H0B (See Table 5). Table 5. Chi-squared test results for cities and municipalities in rural areas. Observed Data City Rural Area Total Yes 134 138 272 Well 57 39 96 Total 191 177 N = 368 Expected data City Rural area Total Yes 141.2 130.8 272 Well 49.8 46.2 96 Total 191 177 N = 368 Indicator Degree of freedom Chi X2-test Chi X2-Critical Significance level p-value Decision rule Values 2 2.905 5.991 5×10−28.8 ×10−2Accepted H0B From the results, it can be concluded that there are no significant differences in the use of public electronic services (slovensko.sk) in between the two areas, nor within regions or between cities and municipalities. Respondents from rural areas use the portal to a greater extent and more frequently than respondents from cities. 2.2. Evaluation of the American Customer Satisfaction Index The American Customer Satisfaction Index questionnaire contains 15 questions from the Government model. The individual questions are focused on information, process, customer service and websites. The mentioned areas represent perceived quality. Furthermore, there are questions about the customer’s expectations of the service. In addition to these areas, there are questions on overall customer satisfaction, customer complaints and user trust. The individual questions on the questionnaire use a Likert scale from 1 to 10, where, for example, 1 is a negative attitude—“do not recommend, difficult, etc.”, and 10 is a positive answer—“recommend, easy, etc.” By calculating satisfaction and using the American Adm. Sci. 2024,14, 217 15 of 26 tropical weighing provides us with a more objective view when determining weights and is more suitable for determining the weight of a complex system. Each method has its disadvantages, so it would be advisable to use the AHP method as well, in order to take advantage of the advantages of the methods of objectivity and subjectivity. For this reason, when calculating the complex weight, the proportion of AHP could be set at 50% and the weight of the entropy method at 50% in future research (Yang and He 2022). Furthermore, the time aspect of the service was not taken into account during the primary research. How likely is it that there would be a difference in satisfaction if the service were available 1 year or 10 years after release, or if it were a newly launched service, or a portal? It is possible to assume that a certain group of users will adopt the service and that there may be a difference between a user who is using the service for the first time and one who has used it several times already. Future research could also address the question of whether there is a difference in ACSI satisfaction between citizens living in a city and in a rural area. In addition, a suitable solution for when to use the service is to use it at a regular time interval (1 per year), with major system changes or minor updates that could affect the customer experience. It would also be possible to compare (Morgeson and Mithas 2009) who measured the levels of satisfaction with central state portals and private sectors. In one study, the authors found that federal portals do not provide the same level of quality as other electronic services (Morgeson and Mithas 2009). When modernizing and updating the portal, Slovensko.sk, it would be appropriate to carry out such a comparison, as it would be possible to find out whether the updated central portal for public services is comparable in terms of quality with the most modern portals in the field of the electricity trade. The ACSI questionnaire can be used at several levels, whether at the local level or with other portals for public services. Tan et al. (2008) state in their research that electronic public administration is increasingly becoming a well-known part of virtual countries. However, the lack of public trust caused by the novelty and uncertainty of online transactions hinders the widespread adoption of public electronic services. Taking into account the perspective of technology as a social actor with whom the customer communicates and transacts, we propose a research model that emphasizes the key role of e-Government service quality as the main driver of citizens’ beliefs about the trustworthiness of e-government websites, which, in turn, promotes the corresponding adoption of e-Government website services. The findings of this study demonstrate that the quality of e-government websites is important in building citizens’ trust in public electronic services (Tan et al. 2008). The results of the study by Li and Shang (2023) reveal that the use of e-government by citizens revives their trust in government indirectly, through the influence on citizens’ evaluation of the integrity of public administration, as well as its performance and responsiveness based on the experience of using e-Government, while the mediating effects of citizens’ evaluations of the government on the relationship between the use of electronic public administration and public trust are reduced by citizens’ expectations of their government. The findings highlight multifaceted strategies to accelerate the adoption of e-Government. Increasing citizens’ trust in e-Government systems through increased reliability, security and transparency remains essential. At the same time, it is necessary to take initiatives to cultivate digital access, skills and abilities within segments of the population (AbdulKareem and Oladimeji 2024). The quality of electronic services has a significant impact on creating public trust. The quality of electronic services also significantly affects public satisfaction. Public trust directly contributes to public satisfaction with public services, but other unmeasured factors also play a role in shaping public perception and satisfaction (Taufiqurokhman et al. 2024). DEA is the measure of efficiency in the European Union. It follows from the results of the work that the countries in the northern part of the European Union are more effective in the field of digitization than the others. The resources they spend on building e-Government infrastructure and services effectively correspond to the outputs. Moreover, when measuring the efficiency of the Mann–Whitney U test, we statistically confirmed that there is a difference between the levels of efficiency of e-Government in selected areas of the European Union. In addition, the ACSI results could be added to the model (in model B) if Adm. Sci. 2024,14, 217 16 of 26 such a measurement were implemented across the European Union, and the two output indicators could be the level of online communication with the public administration and satisfaction (ACSI) with the public administration‘s central state portal or another service. If it were possible to increase satisfaction by “10%”, individual states could take a closer look at this index and, thus, increase the required parameters of perceived quality. When measuring efficiency, countries that are leaders in the field of efficiency were included. These countries include Estonia, Latvia, Finland and Denmark. The inputs of these countries are spent efficiently in relation to the outputs. Even countries with a low budget can achieve a high level of e-Government if the spending of resources is efficient in relation to their outputs, as in Estonia or Latvia. The cooperation of some countries with others, such as Estonia, Finland and others, can also be considered a key factor (Finland and Estonia Deepen Cross-Border Digital Partnerships 2023). The Recovery and Resilience Plan is intended to assist selected or all countries in their digital transformation (Recovery and Resilience Facility (Country Pages) 2023). These investments and their results will only be visible in a few years. Therefore, it is necessary to monitor how the efficiency of the use of these financial resources was affected by the results of e-Government, either by the increased interaction between the citizen and the public administration, or by the increased number of online services. Digital technologies increasingly contribute to increasing productivity, efficiency, accessibility and, above all, the overall well-being of the population. Among the limitations we encountered while writing the research, for example, is the low number of articles using the DEA method in the field of e-Government. Other restrictions may influence the cultural and economic nature of the development of this part of Europe. Education in digital skills among the population (digital culture), economic opportunities and investments are needed for the development of not only e-Government, but also the economic and knowledge availability of these services (Taipale 2013). Other limitations include the limited comparability of data for measuring efficiency across the world and the necessary normalization of data. This would make it possible to compare EU countries with Asian countries. Furthermore, countries do not record investments or resources used in e-Government for a certain period to a uniform standard so that these data can be examined for use in the DEA method. The determination of the weights using the AHP method when determining the ACSI area was also an important issue. There was a problem with finding relevant experts who understand indexes, the ACSI index and the AHP method. Next, a summary of further suggestions for future research is presented. The use of the ACSI, mainly when introducing new services, is recommended. The creation of several ACSI questionnaires that take into account the levels of given countries or measure satisfaction with the local portals of cities and municipalities is also suggested. Furthermore, the use ACSI for new services, such as “mID” and others, is proposed. To use the analytic hierarchy process (AHP method) and the entropy method when determining the weights, in order to use the advantages of both methods of objectivity and subjectivity, is suggested. The use of the DEA method when comparing local regions in Slovakia is indicated. A use for cities and municipalities that provides and most efficiently manages particular budgets to provide the best services to citizens is suggested. When measuring efficiency, it is also important to take into account education levels in mathematics. Future research could address AI solutions within AI countries. The European Union is becoming increasingly interconnected, with individual countries’ economies becoming integrated into a unified digital market. This fact should be taken into account in European Union politics, with a focus on the unification of services provided at the level of citizen identification and the creation of shared services that can be utilized by any EU member state. It would be beneficial for individual nations to engage in more collaborative efforts, both within their own borders and with countries outside the European Union. The practical implications of e-Government, particularly in terms of policy, warrant further discussion to ensure effective implementation and optimization. Moreover, the search for long-term strategic solutions that are viable for a period exceeding five years is of Adm. Sci. 2024,14, 217 17 of 26 paramount importance. The 3E principle was described in this article. The efficiency principle is the main part of the 3E principle, which is the focus of this paper. The DEA method was employed to ascertain the efficiency of each country. The outcome of the DEA model is directly correlated with e-Government efficiency. The cost-effectiveness of the countries in the DEA model is represented by their respective GDPs. The measurement results assist in identifying which countries utilize resources to produce quality e-Government services. Examples of countries that have achieved high levels of e-Government despite low economic strength (GDP) include Estonia, Hungary and Latvia. The DEA model incorporates a temporal dimension, enabling the identification of cost-effectiveness over time. The efficacy of the countries in question can be represented by the ACSI score, which can be measured in the European Union. It is important to determine whether the individual results have met the expectations of the population, which should be reflected in their satisfaction (Krejnus et al. 2023;Kumar et al. 2021). 4. Materials and Methods This article contains both qualitative and quantitative data collection. Within the article, mathematical–statistical methods, such as multicorrelation methods and entropic method of determining weights were used. The mathematical–statistical methods of DEA and the American Customer Satisfaction index (ACSI) were empleyed as the primary analytical tools. The procedure for using DEA is presented below. In the framework of primary research, quantitative method in the form of questionnaire survey was used. A research gap was found in identifying the efficiency of e-Government within EU countries using various indices. This gap was identified by analyzing articles in an international context; the specific countries are mentioned in Table 2. For measuring efficiency within public administration, or e-Government, the DEA method is used using the CCR model. Subsequently, the primary research, aimed at measuring satisfaction in the selected area, was designed using the American Customer Satisfaction Index, Government model, and other questions. Within the research, following research questions were posed: • Is it possible to measure the impact of digitization on the economy, society, management or the environment? • Is it possible to apply ACSI for central state portal, Slovensko.sk? What will be the overall satisfaction with central state portal? • Are there indicators that can be used to measure efficiency in the European Union, and if so, can they be used in the measurement? •What models were used to measure the efficiency of e-Government? • Is there a difference between the use of e-Government services among citizens living in the city and in the countryside? In order to address the research questions, it was necessary to employ a number of different methods. Methods of analysis, collection and processing of information, excerption and compilation methods were used. In addition, the CCR efficiency measurement model, which is output-oriented, was employed in the measurement process. Within this method, the query method and chi-squared method were used. The research questions are answered in the Discussion and in the Conclusion (see Table 11). Table 11. Research methods used. American Customer Satisfaction Index—ACSI Efficiency Measurements of e-Government—DEA Within the framework of ACSI, the following main methods were used: the inquery method, the calculation of the ACSI index itself, the correlation analysis between elements, descriptive analysis, the entropic method of determining the weights and the chi-squared test. Within DEA, the following methods were used: output-oriented CCR measurement model, super-efficiency, correlation analysis, descriptive analysis and Mann–Whitney U test. Adm. Sci. 2024,14, 217 18 of 26 Customer Satisfaction Index The most widely used customer satisfaction indexes in the world are The American Customer Satisfaction Index—Government and the European Customer Satisfaction Index (The American Customer Satisfaction Index—Government 2024). The ACSI government model is a critical control point for evaluating the success of a government’s projects and online initiatives. It is one of the more complex and representative reflections of citizens’ experiences with government websites. ACSI is used to track user satisfaction with product quality over time. The results can be compared with the results of organizations in both the private and public sectors. The ACSI government model was first used in 1994 (The American Customer Satisfaction Index Science and Methodology 2019). ACSI provides cause-and- effect analysis to better target resources to where improvement will have the greatest impact on product improvement. The government’s ACSI model consists of satisfaction factors, satisfaction outcomes and overall satisfaction. Satisfaction factors are grouped into four broad categories that are used as inputs to measure quality (information, process, customer service and website) on the left, overall satisfaction (ACSI) in the middle and satisfaction outcomes on the right (see Figure 5). As one of the solutions, the ACSI model—Government questionnaire was utilized, and it was subsequently used to evaluate satisfaction with e-Government services aimed at the central state portal of the Slovak Republic. Adm. Sci. 2023, 13, x FOR PEER REVIEW 19 of 27 Figure 5. The government ACSI model. The process of selecting participants was carried out by random selection among citizens of the Slovak Republic. The questionnaire was created in electronic form using the Google Docs tool. Electronic inquiry took place in the time interval from 19 February 2022 to 21 February 2022. The target group was represented by the citizens of Slovak Republic aged 18 to 65 years, with residence in the Slovak Republic and experience using e- Government services. The criteria for inclusion in the sample of respondents were age (18– 65 years), residence (Slovakia) and use of e-Government services. Participants under the age of 18, older than 66, respondents residing outside the Slovak Republic and participants who do not use e-Government services were excluded from the survey. The survey involved 423 respondents. The target sample for filling out the questionnaire was 386, which was calculated on the basis of the base set, which is greater than 100,000. The variability of the base set represents a value of 0.5. The confidence interval is 95% and the maximum accessible margin of error is 5%. Of the participants, 301 were women and 122 were men. There were 225 respondents with current permanent residence in cities and 198 in rural areas. Of the total number, 162 respondents live in the West Slovakia region (38.3%), 106 respondents (25.1%) in East Slovakia and 155 (36.6%) in Central Slovakia. The largest group of respondents was between 41 and 50 years old, with a 2nd degree of higher education and in employment. The methodology of the article is described in Table 8. The DEA method ranks among the most important management methods. It allows users to evaluate efficiency based on selected inputs and outputs. DEA was first used in 1978 by Charnes, Cooper and Rhodes as a CCR model. In 1984, Bunker, Charnes and Cooper introduced a variant of BCC that evaluates the efficiency of decision-making units under the assumption of variable returns to scale (Omrani et al. 2020; Charnes et al. 1978). The basic goal of the DEA method is to compare organizational units, which are also referred to as decision-making units (DMUs). Each decision-making unit uses a certain number of inputs for its activity, and the activity results in certain outputs. Input quantities are those units that are consumed in the given activity, and outputs represent the resulting products. In general, smaller input values and larger output values are preferred. The meaning and purpose of the analysis depends on the selected inputs and outputs in the model. Inputs and outputs should be logically linked, as they are in a production process. In order for inputs and outputs to be chosen correctly, we use correlation analysis. With the help of correlation analysis, we select the relationship between the variables and thus eliminate the variables with very strong and very weak correlation. In addition to a suitable correlation coefficient between the indicators, the inputs and outputs must be adapted to the number of decision-making units (DMUs). The rule used is the sum of the number of inputs and outputs 1/3 or 1/5 of the total number of decision-making units. The CCR model calculates the weights of inputs and outputs, the so-called optimization calculation, so that they are as accurate as possible for the DMU Figure 5. The government ACSI model. The process of selecting participants was carried out by random selection among citizens of the Slovak Republic. The questionnaire was created in electronic form using the Google Docs tool. Electronic inquiry took place in the time interval from 19 February 2022 to 21 February 2022. The target group was represented by the citizens of Slovak Republic aged 18 to 65 years, with residence in the Slovak Republic and experience using e-Government services. The criteria for inclusion in the sample of respondents were age (18–65 years), residence (Slovakia) and use of e-Government services. Participants under the age of 18, older than 66, respondents residing outside the Slovak Republic and participants who do not use e-Government services were excluded from the survey. The survey involved 423 respondents. The target sample for filling out the questionnaire was 386, which was calculated on the basis of the base set, which is greater than 100,000. The variability of the base set represents a value of 0.5. The confidence interval is 95% and the maximum accessible margin of error is 5%. Of the participants, 301 were women and 122 were men. There were 225 respondents with current permanent residence in cities and 198 in rural areas. Of the total number, 162 respondents live in the West Slovakia region (38.3%), 106 respondents (25.1%) in East Slovakia and 155 (36.6%) in Central Slovakia. The largest group of respondents was between 41 and 50 years old, with a 2nd degree of higher education and in employment. The methodology of the article is described in Table 8. The DEA method ranks among the most important management methods. It allows users to evaluate efficiency based on selected inputs and outputs. DEA was first used Adm. Sci. 2024,14, 217 19 of 26 in 1978 by Charnes, Cooper and Rhodes as a CCR model. In 1984, Bunker, Charnes and Cooper introduced a variant of BCC that evaluates the efficiency of decision-making units under the assumption of variable returns to scale (Omrani et al. 2020;Charnes et al. 1978). The basic goal of the DEA method is to compare organizational units, which are also referred to as decision-making units (DMUs). Each decision-making unit uses a certain number of inputs for its activity, and the activity results in certain outputs. Input quantities are those units that are consumed in the given activity, and outputs represent the resulting products. In general, smaller input values and larger output values are preferred. The meaning and purpose of the analysis depends on the selected inputs and outputs in the model. Inputs and outputs should be logically linked, as they are in a production process. In order for inputs and outputs to be chosen correctly, we use correlation analysis. With the help of correlation analysis, we select the relationship between the variables and thus eliminate the variables with very strong and very weak correlation. In addition to a suitable correlation coefficient between the indicators, the inputs and outputs must be adapted to the number of decision-making units (DMUs). The rule used is the sum of the number of inputs and outputs ≤ 1/3 or 1/5 of the total number of decision-making units. The CCR model calculates the weights of inputs and outputs, the so-called optimization calculation, so that they are as accurate as possible for the DMU in terms of its efficiency while observing the conditions of the maximum efficiency units of all other units (DEA Modely a Meranie Eko-Efektívnosti 2022). The model evaluates the electronic public services in a given time period (see Table 9). As in the previous measurement, the maximum number of inputs is calculated based on m+s<n 5 , where n represents the number of EU countries. The model has a maximum number of indicators for measurement, which are divided into 4 input and 2 output indicators. The indicators meet the criterion that their data are available from 2014 to 2022. Each input indicator was chosen on the basis that a larger input value in a given country would be more likely to provide more effective e-Government services. Interactions with public administration online, which is one of the most important indicators, were chosen as the output indicators. The output indicators provide us with a picture of the use of e- Government services for input taxes, which take into account the performance of countries in key areas of the current economy and the level of e-Government (see Table 12). Table 12. Input and output DMUs for time model—measurement model B. Index Used Attributes from the Index Index Attribute Index Direction Use of the Internet Human factor (number of Internet users) Access - Time availability of data From 2014 to 2022 EGDI Technological factor (level of e-Government) Access - GDP per capita Economic factor Access - GBARD share of total government expenditure Economic factor (share of R&D expenditure) Access - Submission of completed forms when communicating with the public administration Human factor (level of communication) Output + Interaction with public administration online Human factor (e-Government—e- Government user communication) Output + The statistical relationship between individual quantities can be determined using the Spearman correlation coefficient. The correlation coefficient ris defined by the following relation: r=1−6×∑N n=1D2 N×(N2−1)(1) Adm. Sci. 2024,14, 217 20 of 26 N= number of elements; D= difference between xnand yni.e., in two rows; r= correlation coefficient. The correlation coefficient can take the values 〈− 1; 1 〉 . The correlation coefficient takes values from the interval 〈− 1; 1 〉 and expresses the degree of linear correlation between variables. Minus 1 means absolute indirect dependence, 0 means non-existent linear dependence, and 1 means absolute direct dependence between two variables. Within DEA analysis, it is advisable that the correlation coefficient is not higher than 0.8, otherwise the efficiency result may be distorted. The ideal correlation coefficient is between 0.3 and 0.8 (Schober et al. 2018). A research gap was found in identifying the efficiency of e-Government within the EU countries using various indices. For measuring efficiency within public administration, or e-Government, the DEA method is applied using the CCR model. Subsequently, the primary research, aimed at measuring satisfaction in the selected are, a was designed using the American Customer Satisfaction Index, Government model, and other questions. The correlation between individual indicators was determined for the year 2022. The average value of all correlations for the year 2022 was 0.570, which is in the range <0.3 to 0.8>. Thus, we also confirmed the appropriateness of our inputs and outputs. Correlations between individual elements are presented in Table 13. Table 13. Identification of the baseline population of interest. Year 2022 A B C D E F A 1 B 0.669 1 C 0.446 0.717 1 D 0.654 0.425 0.376 1 E 0.798 0.591 0.311 0.420 1 F 0.714 0.677 0.473 0.442 0.845 1 Correlation analysis was performed for all years, but we report only one. The appropriate values of the correlation for individual years were (2020—0.541), (2018—0.596), (2016—0.613), and (2014—0.629). The methodology and procedure of the paper are presented in Table 14. Table 14. Methodology of the article. Paper Information Method Research Phase Main data collection The data were focused on digitization, e-Government, efficiency Analysis, method determination I. Analysis Books, scholarly articles on digitization, e-Government, DEA models that have been used to measure e-Government within Asia. Key findings from DEA measurement models. Analysis, method of collecting and processing information, extraction and compilation methods, method of abstraction II. User identification and satisfaction measurement using ACSI Data were obtained from respondents. Primary results of entropic data measurement method, chi-squared test, multi-correlation method III. Adm. Sci. 2024,14, 217 21 of 26 Table 14. Cont. Paper Information Method Research Phase DEA method Output-oriented CCR model - IV. From the analysis, none of the indices that can be used have been filled in. The article was targeted at e-Government, meaning that it was necessary to establish input and output data. - Determination of the number of inputs and outputs, verification of appropriate inputs and outputs. Multi-correlation method, Mann–Whitney U test Determination of input and output values. Descriptive data analysis. Dividing selected countries into regions and EU areas and determination of efficiency in 2014, 2016, 2018, 2020, 2022. Identification of effective and ineffective countries. Conclusion, Discussion and results The Conclusion determines the level of satisfaction determined by ACSI within Slovakia and the efficiency levels of countries for individual years. Synthesis method, deduction method, induction method, generalization method, DEA method, comparisons with other studies V. Source: own processing. The article comprises both qualitative and quantitative data collection. In addition, the article uses several methods, such as multi-correlation methods, entropic methods for determining weights, etc. The procedure for using DEA is presented in Figure 6. Adm. Sci. 2023, 13, x FOR PEER REVIEW 22 of 27 Slovakia and the efficiency levels of countries for individual years. generalization method, DEA method, comparisons with other studies Source: own processing. The article comprises both qualitative and quantitative data collection. In addition, the article uses several methods, such as multi-correlation methods, entropic methods for determining weights, etc. The procedure for using DEA is presented in Figure 6. Figure 6. The methodological procedure uses DEA methods. The graph presents a diagram showing the calculation procedure for the DEA method. The graph can be divided into several parts. The diagram begins with the identification of existing efficiency measurement models in the examined area and, subsequently, it presents the selection of appropriate indicators and data, with the help of which the goal of DEA efficiency measurement can be achieved. These indicators and data were used for the calculation from which the efficiency ratings were obtained. If there were insufficient data on the examined issue, it was not possible to carry out such a measurement, or it was necessary to look for other indicators and data. This step is one of the most important, as a wider spectrum of information gives us greater possibilities when combining data during measurement. After this step, it was necessary to move from A1-A12-A2 to select a suitable DEA model. There are several models, which differ in how they calculate efficiency. Therefore, it is necessary to choose the correct direction of the input- or output-oriented model, to know what the number of decision-making units (DMUs) will be and, based on this, to calculate the maximum number of inputs and outputs. Subsequently, it is important to verify, if necessary, the time point of data availability, whether the data are available for Figure 6. The methodological procedure uses DEA methods. Adm. Sci. 2024,14, 217 22 of 26 The graph presents a diagram showing the calculation procedure for the DEA method. The graph can be divided into several parts. The diagram begins with the identification of existing efficiency measurement models in the examined area and, subsequently, it presents the selection of appropriate indicators and data, with the help of which the goal of DEA efficiency measurement can be achieved. These indicators and data were used for the calculation from which the efficiency ratings were obtained. If there were insufficient data on the examined issue, it was not possible to carry out such a measurement, or it was necessary to look for other indicators and data. This step is one of the most important, as a wider spectrum of information gives us greater possibilities when combining data during measurement. After this step, it was necessary to move from A1-A12-A2 to select a suitable DEA model. There are several models, which differ in how they calculate efficiency. Therefore, it is necessary to choose the correct direction of the input- or output-oriented model, to know what the number of decision-making units (DMUs) will be and, based on this, to calculate the maximum number of inputs and outputs. Subsequently, it is important to verify, if necessary, the time point of data availability, whether the data are available for a longer period or only one year, etc. In this step, it is necessary to divide the indicators and data. Furthermore, it is necessary to test the correctness of inputs and outputs based on correlation analysis. A moderate correlation between indicators and data is most appropriate. If the input and output data have inappropriate correlations, it is necessary to find other indicators and try to repeat the correlation analysis. This is also necessary in the case of a longer period of time. If the data are suitable, we can perform descriptive statistics for the data after the correlation analysis. Subsequently, it is necessary to calculate the efficiency itself using the selected model, determine the order of efficiency, etc. For measuring efficiency over time, it is suitable, for example, to calculate the average efficiency for a certain period. In our case, it is appropriate to divide the countries into regions, or EU regions. In individual steps, it is possible to divide this diagram into 10 steps. In addition, this diagram also serves as a tool for understanding the DEA procedure (Krejnus et al. 2023). 5. Conclusions This research was aimed at verifying the use of the American Customer Satisfaction Index of the e-Government model in Slovakia and the use of the efficiency measurement method (DEA). For summary of answers to the research questions, see Table 15. Table 15. Answers to the research questions. Type of Question Research Question Short Answer First Main Is it possible to apply ACSI for central state portal, Slovensko.sk? What will be the overall satisfaction with central state portal? Yes, it is possible. Satisfaction is 67.8%. Second Main Are there indicators that can be used to measure efficiency in the European Union, and if so, can they be used in the measurement? Yes, there are several indicators and they can be used in the DEA method. - Which models were used to measure the efficiency of e-Government? The most commonly used models are CCR and BCC. - Is there a difference between the use of e-Government services between citizens living in the city and in the countryside? Within Slovakia, the difference is not statistically significant. Both methods were applied, and their applicability was verified. During primary research, the verification of the use of the ACSI model was carried out. From the results, it can be concluded that it is possible to use this measurement model. The research revealed that the overall results of satisfaction with the central state portal for public services was 61.7% (Slovensko.sk). Furthermore, the results of the model showed that quality is the most important, as it has the greatest impact on customer satisfaction and, subsequently, this Adm. Sci. 2024,14, 217 23 of 26 satisfaction affects customer trust. Of the qualitative elements of central portal for public services, it is necessary to focus on optimization and improvement of processes. Central portal for public services has several shortcomings, such as outdated design, reliability problems, etc. In the case of DEA, it was necessary to propose a measurement procedure and then indicators and measurement models were used in the field of e-Government. The conclusions can be drawn from the output that measuring efficiency for the European Union area is possible, and two measurement proposals were introduced, focused on the technological area. This research showed Estonia, Denmark, the Netherlands and Hungary as the effective countries. The above results show that governments should focus on service system connectivity, as well as processes and quality, to achieve utility for users. In addition, of the European Union areas, the western European Union is the most effective compared to the other areas, which was also confirmed by the Mann–Whitney U test. The article showed the applicability of the methods. When measuring DEA, further research would be appropriate to determine the effects between inputs on the resulting efficiency. At ACSI, based on these conclusions, experts could consider e-Government 3.0, reworking questions focusing on the local area of e-Government, or new services. It is also necessary to mention that within Slovakia, there is no regular measurement of satisfaction with new or existing e-Government services. Thus, the research filled this gap and suggested a way in which it could be implemented and, possibly, adjusted. A similar gap was identified in measuring the efficiency of DEA analysis, in which a similar measurement using several available indicators was not identified within the European Union. Overall, this article provides an understanding for satisfaction assessments, measuring the efficiency of the state of e-Government and a basis for future research in this area. The government should focus on service system connectivity, as well as processes and quality, to achieve utility for users. The combination of the ACSI and DEA methods would provide a more comprehensive view of ACSI areas with the necessity for improvement. The final output model could include the interaction with online public administrations and the ACSI index. If it is necessary to increase the ACSI index, a more detailed examination could be carried out of those elements within the index that do not currently reach a satisfactory level. This would allow for a more focused approach to be taken with regard to quality, customer satisfaction and any areas requiring improvement. With these two indicators, we would determine exactly the levels of satisfaction and interaction. It would be appropriate to measure ACSI in all EU countries in order to compare these results among individual EU member countries. We consider this to be the other limit of this study, but the scope of the research did not allow us to process these data in the study. Future research should therefore focus on the final output model, including the interaction with online public administration and the ACSI index. If it were to turn out that it is necessary to increase the ACSI index, it would be possible to take a closer look at which elements from that index do not reach a high level and, thus, to focus on the areas of quality, satisfaction or complaints. With these two indicators, we would determine exactly the levels of satisfaction and interaction. It would be appropriate to measure ACSI values in all EU countries so that we could compare these results among individual EU member countries. Author Contributions: Conceptualization, M.K., L.Š., E.L., Z.Š. and K.R.Š., methodology, M.K. and L.Š.; software, M.K. and K.R.Š.; validation, J.Š. and K.R.Š.; formal analysis, M.K., K.R.Š., L.Š and A.P.; investigation, M.K., J.Š., Z.Š., K.R.Š. and L.Š.; resources, M.K., K.R.Š., J.Š., Z.Š., E.L. and L.Š.; data curation, M.K.; writing— M.K., K.R.Š., J.Š. and L.Š.; writing—review and editing, M.K., K.R.Š., Z.Š., E.L., J.Š., L.Š. and A.P.; visualization, E.L., M.K. and L.Š.; supervision, K.R.Š. and Z.Š.; project administration, K.R.Š., E.L. and J.Š.; funding acquisition, Z.Š. and J.Š. All authors have read and agreed to the published version of the manuscript. Funding: This paper was supported by Ministry of Education, Research, Development and Youth of the Slovak, grant number KEGA 048ŽU-4/2022 and VEGA 1/0460/22. Adm. Sci. 2024,14, 217 24 of 26 Institutional Review Board Statement: The survey in this paper was non-interventional, anonymous and focused on exploring impulses, attitudes, etc. The research was not conducted on patients, nor did it involve human material or human tissues. All respondents had the right not to participate in the survey and by filling out the questionnaire, they expressed their consent to participate in the research. Ethical clearance from our home institutions is not required for this type of research. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study . Data Availability Statement: The data presented are available on request from the corresponding author. Acknowledgments: This paper was supported by projects KEGA 048ŽU-4/2022. VEGA 1/0460/22. Conflicts of Interest: The authors declare no conflicts of interest. References AbdulKareem, Abdulrazaq Kayode, and Kazeem Adebayo Oladimeji. 2024. Cultivating the digital citizen: Trust, digital literacy and e-government adoption. Transforming Government: People, Process and Policy 18: 270–86. [CrossRef] Afonso, António, Ludger Schuknecht, and Vito Tanzi. 2010. Public Sector Efficiency: Evidence for New EU Member States and Emerging Markets. Applied Economics 42: 2147–64. [CrossRef] Beˇnová, Elena, CibákováViera, and Peter Plavˇcan. 2006. In Efektívnost verejného sektora. In Úvod Do Teorie Verejného Sektora, 1st ed. Bratislava: Vysokáškola ekonómie a manažmentu verejnej správy, pp. 20–21. Breaugh, Jessica, Maike Rackwitz, and Gerhard Hammerschmid. 2023. Leadership and Institutional Design in Collaborative Government Digitalisation: Evidence from Belgium, Denmark, Estonia, Germany, and the UK. Government Information Quarterly 40: 101–788. [CrossRef] Chan, Calvin M. L., Yi Meng Lau, and Shan L. Pan. 2008. E-Government Implementation: A Macro Analysis of Singapore’s e-Government Initiatives. Government Information Quarterly 25: 239–55. [CrossRef] Charalabidis, Yannis, Euripidis Loukis, Charalampos Alexopoulos, and Zoi Lachana. 2019. The Three Generations of Electronic Government: From Service Provision to Open Data and to Policy Analytics. In Lecture Notes in Computer Science. Berlin and Heidelberg: Springer, vol. 11685, pp. 3–17. Charnes, Abraham, William W. Cooper, and Edward Rhodes. 1978. Measuring the Efficiency of Decision Making Units. European Journal of Operational Research 2: 429–44. [CrossRef] Countries’ Performance in Digitisation. 2022. Available online: https://digital-strategy.ec.europa.eu/en/policies/countriesdigitisation-performance (accessed on 19 October 2022). DEA Modely a Meranie Eko-Efektívnosti. 2022. Available online: https://lnk.sk/ua67 (accessed on 19 October 2022). Digital Around the World. 2022. Available online: https://datareportal.com/global-digital-overview (accessed on 19 October 2022). Djellal, Faridah, and Faïz Gallouj. 2008. Measuring and Improving Productivity in Services: Issues, Strategies and Challenges. Società Editrice Il Mulino 2: 1–246. E-Government Benchmark. 2021. Available online: https://digital-strategy.ec.europa.eu/en/library/egovernment-benchmark-2021 (accessed on 12 October 2023). E-Government Development Index. 2022. Available online: https://publicadministration.un.org/egovkb/en-us/About/Overview/ -E-Government-Development-Index (accessed on 19 October 2022). Feldman, Tony. 2003. An Introduction to Digital Media. London: Routledge. Feliciano-Cestero, María M., Nisreen Ameen, Masaaki Kotabe, Justin Paul, and Mario Signoret. 2023. Is Digital Transformation Threatened? A Systematic Literature Review of the Factors Influencing Firms’ Digital Transformation and Internationalization. Journal of Business Research 157: 113–546. [CrossRef] Fine, Derek, and Tamarie Johnson. 2005. Digital Asset Management in the Public Sector. Journal of Digital Asset Management 1: 25–31. [CrossRef] Finland and Estonia Deepen Cross-Border Digital Partnerships. 2023. Available online: https://www.computerweekly.com/news/36 5535367/Finland-and-Estonia-deepen-cross-border-digital-partnerships (accessed on 22 November 2023). Gorenšek, Tilen, and Andrej Kohont. 2018. Conceptualization of the Digitalization. Computer Science 11: 1–23. Greenlaw, Steven A., David Shapiro, and Daniel MacDonald. 2022. Principles of Economics 3e. Milton Keynes: Lightning Source. GSMA Intelligence. 2022. Available online: https://www.gsmaintelligence.com/data/ (accessed on 19 October 2022). Harari, Yuval. 2015. Scientific revolution. In Sapiens: A Brief History of Humankind, 1st ed. New York: HarperCollins, pp. 243–46. Hsieh, P. H., C. S. Huang, and David C. Yen. 2013. Assessing Web Services of Emerging Economies in an Eastern Country—Taiwan’s e-Government. Government Information Quarterly 30: 267–76. [CrossRef] Krejnus, Matej, Jana Stofkova, Katarina Repkova Stofkova, and Vladimira Binasova. 2023. The Use of the DEA Method for Measuring the Efficiency of Electronic Public Administration as Part of the Digitization of the Economy and Society. Applied Sciences 13: 3672. [CrossRef] Kumar, Raman, Sehijpal Singh, Paramjit Singh Bilga, Jatin, Jasveer Singh, Sunpreet Singh, Maria Luminita Scutaru, and Catalin Iulian Pruncu. 2021. Revealing the Benefits of Entropy Weights Method for Multi-Objective Optimization in Machining Operations: A Critical Review. Journal of Materials Research and Technology 10: 1471–92. [CrossRef]