An efficiency measurement of e-government performance for network readiness: Non-parametric frontier approach

Nam, Hyundong; Nam, Taewoo; Oh, Minjeong; Choi, Sungyong Article An efficiency measurement of e-government performance for network readiness: Non-parametric frontier approach Journal of Open Innovation: Technology, Market, and Complexity Provided in Cooperation with: Society of Open Innovation: Technology, Market, and Complexity (SOItmC) Suggested Citation: Nam, Hyundong; Nam, Taewoo; Oh, Minjeong; Choi, Sungyong (2022) : An efficiency measurement of e-government performance for network readiness: Non-parametric frontier approach, Journal of Open Innovation: Technology, Market, and Complexity, ISSN 2199-8531, MDPI, Basel, Vol. 8, Iss. 1, pp. 1-19, https://doi.org/10.3390/joitmc8010010 This Version is available at: https://hdl.handle.net/10419/274315 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/   Citation: Nam, H.; Nam, T.; Oh, M.; Choi, S. An Efficiency Measurement of E-Government Performance for Network Readiness: Non-Parametric Frontier Approach. J. Open Innov. Technol. Mark. Complex. 2022,8, 10. https://doi.org/10.3390/ joitmc8010010 Received: 5 October 2021 Accepted: 23 December 2021 Published: 6 January 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). Journal of Open Innovation: Technology, Market, and Complexity Article An Efficiency Measurement of E-Government Performance for Network Readiness: Non-Parametric Frontier Approach Hyundong Nam 1, Taewoo Nam 2, Minjeong Oh 3and Sungyong Choi 4,* 1Graduate School of Governance, Sungkyunkwan University, Seoul 03063, Korea; [email protected] 2 Department of Public Administration, Sungkyunkwan University, Seoul 03063, Korea; [email protected] 3Global Elite Division, Yonsei University, Wonju 26493, Korea; [email protected] 4Division of Operations and Supply Chain Management, School of Business, Hanyang University, Seoul 04763, Korea *Correspondence: [email protected] Abstract: Information and Communications Technology (ICT) network readiness competency improves service quality and provides efficient service in implementing successful e-governments. By confirming ICT network readiness of e-governments, it must be redesigned using limited resources effectively to achieve realistic goals. When ICT investment and economic performance are featured, e-government’s network readiness competency improves potential demand, supply, and service maturity. It reflects information technology (IT) development competency on performance effectively. In this study, we propose the Data Envelope Analysis (DEA) method to present a method of improving ICT network readiness between countries. We derived the ICT network’s readiness competency level and strategic plan by comparing each country for efficient ICT operation of e-governments. If we make rankings in a non-traditional and efficient manner, it will become a successful strategy for ICT in the future. This effort provides guidance for each government and a solution for the growth delay problem, which is required for advancement in ICT investment and productivity. It also guides each government to overcome marginal products. Keywords: e-government; informatization; efficiency strategy; user-centric ICT innovation; ICT competitiveness; network readiness 1. Introduction E-government has used information technology in the field of the economy, breaking away from the traditional methods of government operation and bringing a paradigm of integrating functions between governments [ 1 ]. Currently, society demands innovative digital services and advanced Information and Communication Technologies (ICT) for e-governments in order to solve social problems and respond to changes in the political, economic, and social environment [ 2 ]. Information technology contributes to enhancing the efficiency of government functions. It also strengthens the government’s capability of systemizing knowledge and information of the government organizations electronically and of managing government organizations efficiently [ 3 ]. In order to improve the capabilities of e-government, ICT investment and technology development have become the center of attention in making information technology develop and in enhancing national competitiveness [4]. ICT has the effect of facilitating technology spreading throughout our society, affecting economic activity in various ways and performing a significant role [ 5 , 6 ]. Jorgenson and Stiroh [ 7 ] and Oliner and Sichel [ 8 ] asserted that technological progress and innovations of ICT increase productivity and create added value, thereby bringing rapid economic growth forward. Innovation promotes national competitiveness and sustainable economic development. Countries with advanced development in ICT-driven innovation show better competency. This study demonstrates that ICT industry, infrastructure, and environment J. Open Innov. Technol. Mark. Complex. 2022,8, 10. https://doi.org/10.3390/joitmc8010010 https://www.mdpi.com/journal/joitmc J. Open Innov. Technol. Mark. Complex. 2022,8, 10 2 of 19 are important factors for innovation competency. Innovation reflects high-quality innovation performances, including R&D and patents of ICT industry in general [ 9 ]. It is essential to explore new strategies by sharing these efficient innovation performances. By utilizing shared resources, efforts must be made to efficiently decrease the uncertainty of investment into futuristic technology. In other words, “Open Innovation” is required to solve ICT-related problems and to search for sustainable ICT strategies. The positive effects of ICT progress on industries and national economies have been proven empirically [ 10 , 11 ]. Fukugawa [ 12 ] indicates that the ICT environment is required to be preceded in creating economic added value, and Koh and Magee [ 13 ] perceived the significance of information technology progress by verifying that competitiveness and productivity are enhanced through information technology progress. Furthermore, the added value of the ICT industry announced by OECD would increase the growth of the related industry at a very high speed [ 14 ]. ICT in the fourth industrial revolution is being used as a significant indicator remarkably in evaluating national competitiveness because of its significance and ripple effect. Many countries have been making great efforts on ICT progress through cooperative work in accordance with a rapidly changing environment of information-oriented generation based on ICT progress [ 15 ]. International organizations and civilian departments have been continuously discussing and understanding the degree of informatization by the country for its progress. This kind of discussion provides us with guidelines for analyzing the level of informatization objectively by country and proposes a conceptual model for measuring information society. International Telecommunication Union (ITU), as a representative of international comparison index, guides the ICT development index, e-government readiness index of United Nations (UN), and the online participation index. In addition, the World Economic Forum (WEF) announced the Network Readiness Index (NRI), which indicated the ICT environment and informatization development level that is used as a continuous index in terms of reinforcing competitive advantages [16,17]. Relative comparison applying the international comparison index is used as a key measure to understand the degree and change of informatization progress and competitiveness level in a globally competitive environment by analyzing the level of ICT informatization objectively [ 18 – 20 ]. This is effective in tracking comparison and changes, which does not establish absolute national rank or standard. Each country uses a multiplication about the index to inquire the availability and influence of ICT accessibility and technology. Again, it contributes to measuring various aspects of the information society and is reflected in the countries with ICT policies by understanding national ICT policies [ 21 ]. At the point when economic results are highlighted by ICT technology progress, research about the effect of ICT on various industrial fields has been constantly conducted [ 22 – 24 ]. On the other hand, research about the effective operational output of ICT is insufficient. These issues [ 25 ] suggest a need to address the problem of technological advances in economy or growth retardation in production factors and a need to examine marginal products. In other words, the more efficient the ICT environment, the better the ICT influence. If ICT achievement is inefficiently operated in a successful ICT environment, it will require efforts to raise efficient productivity. Therefore, there is a need to redefine plans for raising efficient operation check-ups and ICT environment efficiency. In the method of measuring efficiency, Data Envelopment Analysis (DEA) is effective in comparing Decision Making Units (DMU) placed in a similar environment. Efficiency signifies that relatively similar resources are input to produce higher performance or fewer resources are needed to achieve the same performance [ 26 ]. It is necessary to find the cause of inefficiency and improve the input and output by measuring relative effectiveness between countries. Ineffective operating countries are required to benchmark efficient countries in order to find measures of operating potential improvement. In addition, the relations between external environmental factors and ICT production efficiency should be investigated. DEA is suggested for efficient ICT operation. “The network readiness index (NRI)” announced by WEF should be used, which allows relative comparisons of ICT levels by J. Open Innov. Technol. Mark. Complex. 2022,8, 10 3 of 19 country. This study compares each country’s ICT development capability level with that of a relative country to derive the ICT strategy of Korea. It shows what efforts are required for e-government innovation by improving ICT operational efficiencies and reducing inefficiencies. In terms of achievement, national informatization is significant in planning a strategy that may strengthen the national economy and national competitiveness. Most of the preceding studies performed research on current capabilities by examining other aspects of e-government; thus, empirical studies that researched on ICT network readiness level are lacking. Those studies have been limited to ICT investment or economic performance, and studies that applied the index were not suitable for relative comparison because they measured countries’ indices simultaneously. It will be an empirical study that can measure the new efficiency of ICT if the efficiency is measured by collecting the NRI of each country to provide ICT performances of e-governments. In addition, the relative comparison is expected to be of great significance by objectively deriving efficiency and environmental factors and classifying the income ranges of each country by using the index, which improves the downside of the information index. This study emphasizes the importance of network readiness. Network readiness is required for greater improvements in each country’s efficient ICT influence—that is, sustainable economic growth and increased productivity. A higher level of network readiness is effective in overcoming limitation on productivity and environmental factors. However, most of the studies related to ICT focuses only on technology and performances [ 27 , 28 ]. This study not only measures efficient ICT performance and degree of information but also inspects inefficient network readiness and discovers unproductive external environmental factors. This is a measure for solving growth delay problems or limitation on productivity. These efforts provide guidelines for sustainable development [ 29 ]. It also means understanding environmental factors for ICT network readiness of each country and acceptance of problems [ 21 ]. It is necessary to perform theoretical and empirical research on how to change this network’s readiness environment. This study presents questions as follows. First, what is the difference between an efficient country and an inefficient country? Second, what are the causes of inefficiencies in constant returns to scale? Third, what are the external environmental factors that mediate efficiency? We reviewed the preceding studies to develop a study method. Specifically, this study provides an approach and direction for the development of an efficient level of network readiness. The Network Readiness Index is the result of a government’s effort to evaluate the performances of that government. As a result, it determines the ratings of efficiency that differs from traditional ratings. The remainder of the paper is composed as follows. In Section 2, we examine the theoretical background of e-governments, ICT and informatization, and international comparison index. In Section 3, we introduce the methodology and present the research model and data. In Section 4, we show the results of efficiency analysis by country and expxlain the results. Finally, in Section 5, we provide our discussion with the results of the analysis. 2. Literature Review 2.1. The Future Direction of Innovative E-Government Currently, the government perceives a barrier in early process, information security issues, and limitations caused by current government operation methods [ 30 ]. In this regard, e-government has a paradigm of using information technology across the economy to provide services and connect intergovernmental functions [ 1 ]. In the past, e-government focused on performing administrative tasks efficiently by using ICT technologies such as computers and the Internet. However, as e-government has become more advanced, it has overcome the limitations of supplier-based service and strengthened its approach toward all aspects of government by using ICT technology and the Internet [ 31 ]. Many countries have been focusing on ICT investment and technology development to advance information technology of the e-government and to enhance national competitiveness [4]. J. Open Innov. Technol. Mark. Complex. 2022,8, 10 4 of 19 The use of information technology provides many possibilities to enhance the efficiency of government function [ 3 , 32 , 33 ]. It has improved the quality of e-government services [ 34 , 35 ], reduced corruption, and strengthened government power [ 2 , 36 ]. Furthermore, it has contributed to rapid change across various fields, including politics and government at the macro-level and reengineering management at the micro level [37–39] . In addition, e-government has been working as an important factor in economic growth, focusing on government policy innovation and making efforts to strengthen national competitiveness [ 4 , 40 ]. It is impossible to define a country’s economy only by GDP or productivity. Batterbury and Fernando [ 41 ] explained that an environment with effective structures, institutions, and policies should be accompanied. Kharlamova and Vertelieva [ 42 ] emphasizes that national competitiveness will provide great potentialities of the industrial economy. To solve environmental pollution and social difficulties and to cope with changes in the political, economic, and social environment, the public sector demands high quality e-government innovation through the introduction and use of digital technology [43,44]. The government has focused on ICT readiness level. The government also focused on citizen’s participation in planning and implementing ICT-based services prior to efficient government operation [ 45 , 46 ]. ICT’s readiness capabilities may enhance the potential demand, supply, and maturity of e-government services and realize efficient e-government services according to high development capabilities. High quality readiness for technology development and implementation is a prerequisite for the public sector where e-government services are innovative. It makes a citizen’s life more comfortable. Enterprises will be able to contribute to production in various economic sectors and enhance competitiveness according to the level of readiness of e-government. In other words, for e-governments to succeed, evaluation and measurement of e-government readiness are essential [47,48]. However, according to the ICT readiness of e-government, many countries fail to provide effective technology and performance. Moreover, poor readiness may reduce the capability and bring a bad environment that could take advantage of ICT opportunities. Koh et al. [ 48 ] distinguished e-government components by covering the planning of goals, policies, processes, and technologies to evaluate the readiness of the integrated e-government services. This study proposes the need to validate critical issues and confirm readiness before implementing services for e-government to be successful. Chanyagorn and Kungwannarongkun [ 49 ], Azman and Salman [ 50 ], and Wielicki and Arendt [ 51 ] mentioned that ICT measurement is necessary for a multi-dimensional aspect of a country. It is useful for understanding the impact of ICT readiness level from a macro perspective and for establishing international policies. This study tracked other aspects of e-governments. As a result, we highlight readiness that opposes current capabilities. However, there are relatively few empirical studies that evaluated the readiness level of e-governments as it is limited to analyses of specific part [52,53] . For efficient operation, an approach toward changing the production environment or ICT influence power should be taken according to ICT readiness of e-governments. Therefore, it is necessary to conduct various evaluation systems and empirical studies to prove that ICT readiness environments promote efficient supply and improve the ICT effect by using government resources. 2.2. Informatization and International Comparison Index The world is advancing into an information society and is influencing general information society structure, new technologies, environment, and economy. The progress in informatization is not limited to individuals or companies. However, it is a key factor that determines national competitiveness [ 54 ]. In order to prepare for a rapidly growing information society, accurately analyzing the level of informatization and predicting the development of changes are necessary [ 55 ]. Therefore, tools that may analyze the information objectively and systematically are necessary, which are various indicators related to informatization [ 56 , 57 ]. In line with the rapidly developing environment of information J. Open Innov. Technol. Mark. Complex. 2022,8, 10 5 of 19 generation, many countries contribute much effort to raise the level of ICT. International organizations and civilian departments also make efforts to understand the degree of informatization by a country and develop indicators to measure the level of informatization as informatization advances. The International Ranking (Index) is a tool evaluating the development of Information and Communication Technologies (ICT). In particular, the most influential tools are the ICT Development Index (IDI) made by ITU, Network Readiness Index (NRI) made by WEF, and e-Government Development Index (EGDI) of the United Nations. The international comparative index of major informatization is shown in Table 1. Table 1. International comparative index of major informatization. Organizations International Comparative Index of Informatization International organizations ITU ICT Development index, IDI ICT Opportunity Index, ICT-OI Digital Opportunity Index, DOI UN e-Government Development index, EGDI e-Participation Index, EPI WB Knowledge Economy Index, KEI Civilian departments WEF Networked Readiness Index, NRI Global Competitiveness Index, GCI IMD World Competitiveness Index, WCS EIU Digital Economy Rankings CIW Global Innovation Index, GII The informatization index used so far has difficulties in reflecting the changed level of national informatization due to the rapid progress of informatization. Therefore, developing appropriate informatization indicators is required, according to the information generation, in order to objectively understand the progress and level of national informatization. However, national informatization level measurement by the informatization index has difficulties with respect to expressing it as an index. This is because all kinds of national and social structure and informatization levels related to informatization in a country’s society must be included. Moreover, incorrect statistical indicators of each article in several informatization-related integrations may cause problems in interpretation [ 58 ]. In other words, new integrated methods and statistical index development that may reflect new information technology should be the subject to research. Additionally, informatization influence variables that may consider various environments should also be subject to research. As mentioned above, the international comparison index for a country covers a wide range of information society categories. Therefore, there is a limit to preparing detailed statistical indicators. Since the information gap is a relative concept, measurement value fluctuates according to the figures of other countries. Thus, national index calculation is considered a difficult method. Cuervo and Menéndez [ 59 ] points out that the development of statistical indicators for digital society changes into the availability of data and the quality and reliability of information sources. In other words, the international comparative index, which is used in the analysis, should be carefully considered. The international comparative index, mentioned above, and e-Government Development Index (EGDI) must correct statistical flaws when calculating index [ 18 ]. Global efforts are still insufficient in providing a proper measurement for achieving SDG [ 60 ]. Individual indicators and sub-index, weighted by country, are subjectively estimated figures or indexed based on inadequate statistical measurement models. As a result, ICT Development Index (IDI) cannot solve the problem of predicting specific result criteria connected to socio-economic performance [ 58 ]. Currently, ITU has been expanding its efforts to draw details of national indicators that may be analyzed objectively. In addition, the DEA approach is not suitable for indexes such as IMD, EIU, and CIW. They have ambiguous boundaries between input and output and lack thorough subfactors. Therefore, the effort to develop a new index of J. Open Innov. Technol. Mark. Complex. 2022,8, 10 6 of 19 evaluation is necessary. A careful approach is needed for seeking availability and quality of data in the measurement of ICT efficiency. On the other hand, WEF introduces a new NRI developed jointly with the Harvard Institute for International Development to supplement inherent problems in recent data quality and credibility of ICT. NRI, revised in 2019, consists of NRI experts, is measured based on more than 30 technology-related review indicators, and surveys the development process. Sitnicki and Netreba [ 20 ] proved that the algorithm developed by WEF improved discrimination between indices of network readiness values. Many economies applied NRI for ICT plan strategies, often cited by leaders, in public and private institutions [ 17 ]. In more than 120 countries, NRI has been used to include necessary contents to find reliable index development with a broad global scope and strengthened competency. NRI guides us to prepare for more changes of new technological innovation in the future by solving current ICT problems. Network readiness ensures governance mechanisms and efficiencies for achieving a broader goal. Furthermore, it influences the economy and quality of life to gain insights into achieving SDGs [61,62]. 3. Materials and Methods 3.1. Selection of Data and Variables For a relative comparison, each country must have an objective basis for the effectiveness of ICT technologies and usage efficiency compared to accessibility for relative comparison. By performing this, we will see how efficiency inputs required for production in a country have been operated and produced results. Therefore, this study applied NRI provided by WEF for data analysis. Many countries apply the ICT informatization index and reflect it in their national informatization strategy. The operational definition of variables is shown in Table 2. Table 2. Operational definition of variables. Factors Measuring Indicator Sub-Indicator Contents Input Technology Access Index Content Future Technologies People Individuals Businesses Governments Governance Trust Regulation Inclusion Output Impact Economy Quality of Life SDG Contribution As a strategy to increase effectiveness and productivity of ICT, this study proposes a Data Envelopment Analysis that can measure efficiency by applying Technology, People, Government, and Impact indices found by the ICT environment of each country. After analyzing DEA and comparing the relative efficiency of each country, the point of projection is derived, and potential improvement value is deduced that may convert inefficient countries into efficient countries. In addition, external influencing factors are used as independent variables to analyze environmental factors by reflecting the efficiency value, which is the dependent variable. ICT environment in the country is the result of government efforts. It discovers rankings of efficiency that differ from traditional rankings. Therefore, it is a methodology from the point of view of efficient operation and production. This study applied DEA to measure the relative effectiveness of the ICT environment. The data used for analysis in this study include NRI announced by the WEF. The 2020 NRI is the second version of the renewed NRI model, which measures the economies of a total of 134 countries based on ICT results across 60 variables. The second version was redesigned to J. Open Innov. Technol. Mark. Complex. 2022,8, 10 7 of 19 reflect how it should be integrated within governance structure for technology and people to effectively impact the economy and society. NRI comprises rankings and subindices for digital innovation of a total of 134 countries. Examining how ICT impact is exerted effectively is required through relative comparisons with the countries concerned in applying input resources, ICT technology, people, and government environments. For a relative comparison, each country must have an objective basis for ICT impact efficiency compared to ICT input resources. By performing this, we will observe how the ICT environment has operated the inputs efficiently needed for production in a country to produce results. 3.2. Collection of Analysis Targets The Decision-Making Unit (DMU), which is the target of efficiency analysis, must be determined first in order to measure efficiency. DEA should premise homogeneity for relative comparisons between DMUs. Homogeneity between DMUs means that DMUs must be comparable to each other by securing similar corporate activities, similar kinds and ranges of resources, and similar external environmental factors [ 63 ]. Therefore, homogeneity is applied to the criteria for selecting the subjects of analysis in this study. The NRI 2020 result index is closely related to income levels (high-income countries, upper-middle-income countries, lower-middle-income countries, low-income countries). Considering each country’s income level, WEF should divide them into four types of incomes (1st–4th quartile) and use DMU with homogeneity. For DEA efficiency analysis, it is desirable to have a large population of DMUs. If the number of DMU units is small, most DMUs correspond to the efficiency frontier, resulting in incorrect measurements. This is because the relative efficiency of DMU may be close to 1. As an important factor in achieving the goal of DMU, the number of input and output variables may affect the same decision factors as much as the number of DMUs. Banker et al. [ 64 ] argues that DMU units must be satisfied more than three times the sum of inputs and outputs (calculation variables) to improve this. This study applies technology, people, and governance corresponding to the ICT environment as input variables. The impact is selected as an output that may be measured because of the ICT environment. Countries divided by four types of income and measurement variables have no problems in analyzing efficiency. 3.3. Data Envelopement Analysis In terms of various methods of measuring efficiency, DEA measures efficiency by several input and output indicators. This has the advantage of considering input and output indicators, and there is no need to assume a production function. First proposed by [ 65 ], the DEA measurement model is based on the [ 26 ] concept of relative efficiency. Considering several criteria of relative effectiveness of an organization that operates with the same goal through DEA, the “Linear Programming” model has been developed to systematically evaluate and reflect decision making. This model determines empirical efficiency limits on the set constructed by the observed value of the input and output factors of DMU. It also measures the relative efficiency of the set that is evaluated by comparing the value of DMU. This has been applied in various fields to measure input and output index for production [ 66 ]. Sutopo et al. [ 67 ] explained that it is a useful method for assessing and describing performance effectiveness, which is meaningful in modeling the production frontier to provide a level of efficiency. An environment that may produce a specific level of output through a specific level of input from a measurable value is defined as “producible.” These associations are called a “producible set.” The producible set is defined as Constant Returns to Scale (CRS) and Variable Returns to Scale (VRS). The viewpoint expands or contracts equally if a CRS exists. On the other hand, VRS signifies that Increasing Returns to Scale (IRS) or Decreasing Returns to Scale (DRS) exist if Constant Returns to Scale is not established. In this respect, DEA is a method of analysis applying a producible set. According to the study of [ 26 ], J. Open Innov. Technol. Mark. Complex. 2022,8, 10 8 of 19 efficiency is about reducing inputs while producing the same output from an economic perspective or increasing output while using the same inputs. In other words, technical efficiency is considered when improving at a higher rate than inputs. The following are the advantages of measuring efficiency: First, it is possible to derive the production possibility frontier or efficiency frontier by applying various figures as a nonparametric statistical technique that considers input and output simultaneously without assuming a production function. Second, it can grade weights that are advantageous to DMUs without subjectively weighting the number of inputs and outputs and analyzing efficiency by securing objectivity. Third, DMUs are feasible for relative comparison for measuring how efficient they are. Finally, inefficient DMUs are moved into efficient DMUs with potential improvements that enable efficient operation. DEA introduces different models depending on the purpose of the study and the data character. Typical models are Charnes, Cooper, and Rodes (CCR) and Banker, Charnes, and Cooper (BCC) models. The CCR model is designed to evaluate other DMUs based on the most efficient DMU without considering economies of scale for DMU. Therefore, the CCR model is assumed to be a Constant Returns to Scale (CRS) model because there is no change in revenue depending on the scale. On the other hand, the BCC model considers economies of scale or non-economics. This Variable Return to Scale (VRS) model assumes that “there is a change” in revenue depending on the scale. The VRS model defines scale inefficiency that may occur when the scale is not optimal. The calculation standard of the mathematical model for a producible set that satisfies this is as follows. VRS model defines scale inefficiencies when the scale is not optimal. The following expressions describe a standard of computation of a mathematical model for a product set that satisfies the definition. φk∗=max φ θ,λ k subject to xk m≥ J ∑ j=1 xjλj(m=1, 2, . . . , M); (1) φkyk n≤ J ∑ j=1 yjλj(n=1, 2, . . . , N); λj≥0(j=1, 2, . . . , J) When deriving a virtual price for VRS model, it may obtain predictable results through the observed value. φk∗=max φ θ,λ k subject to xk m≥ J ∑ j=1 yj mλj(m=1, 2, . . . , M); (2) φkyk n≤ J ∑ j=1 yj nλj(n=1, 2, . . . , N); J ∑ j=1 λj=1; λj≥0(j=1, 2, . . . , J) J. Open Innov. Technol. Mark. Complex. 2022,8, 10 15 of 19 5. Conclusions 5.1. Discusion ICT promotes technology diffusion throughout society and influences economic activity from various angles. In a successful ICT environment, inefficient operating methods and marginal products are not guaranteed for effective growth. In other words, we need to recognize the limited resources pursued by economics and the efficiency of making the most efficient use of those resources. Optimal application of ICT resources overcomes marginal productivity and contributes to efficient economic growth. Countries with similar environmental levels should be compared to increase ICT- oriented innovation competency. Relative comparison of countries with similar environment level is a significant factor for enhancing national competitiveness. Therefore, countries must confirm national ICT efficiency by comparing the results achieved with input resources. Governments must accurately assess whether their organizations are operating efficiently and recognize inefficiency problems. In addition, it has been confirmed that ICT operational efficiency is affected by factors necessary for global innovation. Governments should benchmark other countries that operate efficiently in earning. Countries with similar earnings should manage their ICT operations by referring to each other’s ICT environment and accessibility. According to the results of this study, it was confirmed that country requirements for benchmarking by earning were different. Additionally, external environmental factors affecting efficiency were also different. In other words, they must gain insights from efficiency analysis by applying DEA methodology for problem-solving. The implications of this study are summarized as follows. First, 12 countries, including Singapore, showed an efficiency value of 1 in the CCR model. Twenty-one countries, including Italy, showed an efficiency value of 1 in the BCC model. Ireland, Malaysia, Poland, and Singapore, which are the first quartile high-income countries, show efficient ICT performances among countries with high incomes. Comparative countries should benchmark countries with efficient ICT operation management skills. They should promote ICT efficiency by considering factors that affect efficiency such as GDP growth, GDP environmental, institutions, skill, and innovation capability. For second quartile uppermiddle-income countries, Mexico and Vietnam were identified as inefficient countries. To increase efficiency, environmental factors such as institutions, skills, and innovation capability should be considered. For third quartile lower-middle-income countries, El Salvador and Lao PDR were confirmed to be more efficient than other comparative countries. Countries that require advancements in efficiency should improve by comparing El Salvador and Lao PDR. Furthermore, environmental factors such as population, institutions, infrastructure, skills, financial system, market size, business dynamism, innovation capability, etc., are important. Input and production were carried out efficiently for Chad, DR Congo, Ethiopia, Yemen, which are fourth quartile low-income countries. The selected countries indicate that ICT impact, which corresponds to ICT performances, environment, and accessibility, operates efficiently. Countries with low income should benchmark Chad, DR Congo, Ethiopia, and Yemen. External environments that affect efficiency were identified as GDP per capita, institutions, infrastructure, ICT adoption, macro-economic stability, skills, product market, etc. About 9% of the countries among all the countries showed the optimal state of economy scale in terms of efficiency of scale. However, about 78% of countries showed DRS. This study identified problems that ICT environment was the excessive, or ICT impact was not efficiently operated. If the production rate is low, it is necessary to reduce input factors or improve excessive input environment. The relations between the input environment and ICT impact should be carefully examined and ICT adequacy and the environment should be established. Non-homogeneous ICT construction negatively impacted the purpose of use and the environmental industry, and active system construction must be arranged to increase effectiveness and productivity. IRS showed that about 13% of countries need J. Open Innov. Technol. Mark. Complex. 2022,8, 10 16 of 19 to increase their inputs because their output scale is larger than their inputs. Countries with IRS may operate ICT impact efficiently if the ICT environment is increased or the investment in the ICT environment is increased. Second, this study conducted a Tobit test for 12 external environmental variables, including 12 Global Competency Indices that may influence ICT efficiency, the population of the country, GDP per capita, 10-year average annual GDP growth, and 5-year average FDI inward flow (GDP Environmental) as well as 16 external environment variables including the four above. The analysis results showed the effect on all brackets of institution and skill factors. Specifically, in the group of high-income countries, five external factors, including 10-year average annual GDP growth (%), were found to affect ICT efficiency. As 10-year average annual GDP growth (%) and 5-year average FDI inward flow (GDP Environmental) increased, ICT efficiency more affected. Moreover, if countries with the highest income showed negative estimate values on institutions, skills, and business dynamics factors, it is effective to establish an ICT development strategy for those factors. In the group of lower-middle-income countries, as the population decreases, ICT efficiency was influenced more. It may be interpreted as a result of problems or impacts that the ICT use in third-percentile income countries has not expanded, or partially developed environments do not meet Networked Readiness efficiency. In addition, it was found that many external factors in global competitiveness have more influences in the lower-middle- income countries and low-income countries than in other groups. The results, therefore, suggest that a country should consider the external environment to increase ICT efficiency in the future. It requires a country to build an efficient ICT environment by applying each country’s national competitiveness factors and planning a strategic ICT influence suitable for the relevant field. We, therefore, found that more efforts are needed for an ICT environment that has not yet been fully established in the group of lower-middle- income and low-income countries. The group of these countries may have a successful ICT influence only by applying competitive strategies and ICT environments suitable for their own environments considering the matters. Each government should plan a business model that improves ICT operationality and efficiency. It is necessary to seek ICT-oriented innovation strategies by selecting external environmental factor that will promote ICT efficiency. Considerations for the state of ICT operation, environment, accessibility, and infrastructure of efficient country must be conducted as well. In conclusion, we recognized that the improvements for the inefficiency problems must be incorporated. At the same time, an informatization strategy that considers the external environment must be preceded. This study proposes relative efficiency values for the NRI issued by WEF in 2021. Furthermore, we verified external environmental factors affecting ICT efficiency. The study is significant because it proposes a method to improve ICT impact effectiveness that is relatively inefficient. Many countries have made persistent efforts to improve ICT development’s positive effects on industry and the national economy. However, technological advances in the economy or delays in growth of factors of production factors must be addressed to ensure successful effectiveness and productivity. The ICT environment is very significant for national competitiveness and competitive advantage in terms of results. These issues indicate that there is a need to examine resources and marginal products that are being operated inefficiently. The findings led us to find a measure that may solve growth delay and increase the marginal product. It is important to understand the level through relative comparison with external environmental factors to improve ICT application effectively. The differentiation of this study is analyzing efficiency and deriving impact factors by considering income levels in 134 countries. 5.2. Further Research and Limitations This study compared effective operational performance with the NRI level announced by WEF. Therefore, it shows insights for planning future strategies to solve problems. Nevertheless, some limitations may arise. First, detailed grouping is required for objective J. Open Innov. Technol. Mark. Complex. 2022,8, 10 17 of 19 and relative comparisons. Efforts are needed for relative comparison with competitive countries based on population, income, and specific environments. Second, standardized indicators that may measure the external ICT environment should be developed. If objective indicator development is preceded, many studies should find the cause of the problem by using an objective approach. Based on the results of the study, we sought strategies to make an efficient country by considering the potential improvement level of countries with insufficient impact efficiency. This paper may be used as an important reference for ICT policy managers of the government to increase ICT efficiency and improve ICT impact quality performance that is necessary for national competitiveness. The result of the efforts will become the foundation for national growth along with the development of science technology through the integration of ICT technology, environment, governments, and people. Author Contributions: All authors significantly contributed to the scientific study and writing. H.N. contributed to the overall idea, model formulation and analysis, and writing of the manuscript; T.N. contributed the process of refining research questions. M.O. contributed the process of refining research ideas. S.C. contributed to the process of detailed writing and discussions on open innovation of e-government performance, as well as preparation for publishing the paper. All authors have read and agreed to the published version of the manuscript. Funding: This study was supported by Ministry of Education of Republic of Korea and National Research Foundation of Korea (BK21FOUR Toward Empathic Innovation: Through Platform Governance Education & Research Programs: #4199990114294). Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: Not applicable. Conflicts of Interest: The authors declare no conflict of interest. References 1. Ayanso, A.; Chatterjee, D.; Cho, D.I. E-Government readiness index: A methodology and analysis. Gov. Inf. Q. 2011 ,28, 522–532. [CrossRef] 2. Hazlett, S.A.; Hill, F. E-government: The realities of using IT to transform the public sector. J. Serv. Theory Pract. 2003 ,13, 445–452. [CrossRef] 3. Moon, M.J. The evolution of e-government among municipalities: Rhetoric or reality? Public Adm. Rev. 2002,62, 424–433. [CrossRef] 4. Menna, A.; Walsh, P.R.; Ekhtari, H. Identifying enablers of innovation in developed economies: A National Innovation Systems approach. J. Innov. Manag. 2019,7, 108–128. [CrossRef] 5. Bosworth, B.P.; Triplett, J.E. The early 21st century US productivity expansion is still in services. Int. Product. Monit. 2007,14, 3. 6. Kretschmer, T. Information and Communication Technologies and Productivity Growth: A Survey of the Literature. OECD Digit. Econ. Pap. 2012,195, 1–37. [CrossRef] 7. Jorgenson, D.W.; Stiroh, K.J. US economic growth at the industry level. Am. Econ. Rev. 2000,90, 161–167. [CrossRef] 8. Oliner, S.D.; Sichel, D.E. The resurgence of growth in the late 1990s: Is information technology the story? J. Econ. Per. 2020 ,14, 3–22. 9. Zhang, C.; Wang, X. The influence of ICT-driven innovation: A comparative study on national innovation efficiency between developed and emerging countries. Behav. Inf. Technol. 2019,38, 876–886. [CrossRef] 10. Barua, A.; Kriebel, C.H.; Mukhopadhyay, T. Information technologies and business value: An analytic and empirical investigation. Inf. Syst. Res. 1995,6, 3–23. [CrossRef] 11. Dewan, S.; Kraemer, K.L. Information technology and productivity: Evidence from country-level data. Manag. Sci. 2000 ,46, 548–562. [CrossRef] 12. Fukugawa, N. Science parks in Japan and their value-added contributions to new technology-based firms. Int. J. Ind. Organ. 2006 , 24, 381–400. [CrossRef] 13. Koh, H.; Magee, C.L. A functional approach for studying technological progress: Application to information technology. Technol. Forecast. Soc. Change 2006,73, 1061–1083. [CrossRef] 14. Guellec, D.; Wunsch-Vincent, S. Policy responses to the economic crisis: Investing in innovation for long-term growth. OECD Digit. Econ. Pap. 2009,6, 1–37. [CrossRef] 15. Dobrota, M.; Jeremic, V.; Markovic, A. A new perspective on the ICT Development Index. Inf. Dev. 2012 ,28, 271–280. [CrossRef] 16. Leyli, A. Why to increase country’s network readiness level? Am. Sci. 2021,1, 56–59. J. Open Innov. Technol. Mark. Complex. 2022,8, 10 18 of 19 17. Dutta, S.; Lanvin, B. The Network Readiness Index 2020: Accelerating Digital Transformation in a Post-COVID Global Economy; Portulans Institute, WITSA: Washington, DC, USA, 2020. 18. Máchová, R.; Lnˇeniˇcka, M. Evaluating the quality of open data portals on the national level. J. Theor. Appl. Electron. Commer. Res. 2017,12, 21–41. [CrossRef] 19. Nevado-Peña, D.; López-Ruiz, V.R.; Alfaro-Navarro, J.L. Improving quality of life perception with ICT use and technological capacity in Europe. Technol. Forecast. Socl. Change 2019,148, 119734. [CrossRef] 20. Sitnicki, M.; Netreba, I. Interdependence Assessing for Networked Readiness Index Economic and Social Informative Factors*. Balt. J. Econ. 2020,6, 47–53. [CrossRef] 21. Emrouznejad, A.; Cabanda, E.; Gholami, R. An alternative measure of the ICT-Opportunity Index. Inf. Manag. 2010 ,47, 246–254. [CrossRef] 22. Stare, M.; Jakliˇc, A.; Kotnik, P. Exploiting ICT potential in service firms in transition economies. Serv. Ind. J. 2006 ,26, 287–302. [CrossRef] 23. Franklin, M.; Stam, P.; Clayton, T. ICT impact assessment by linking data. Econ. Lab. Market. Rev. 2009,3, 18–27. [CrossRef] 24. Alderete, M.V. Examining the ICT access effect on socioeconomic development: The moderating role of ICT use and skills. Inf. Technol. Dev. 2017,23, 42–58. [CrossRef] 25. Prasanna, R.P.I.R.; Jayasundara, J.M.S.B.; Naradda Gamage, S.K.; Ekanayake, E.M.S.; Rajapakshe, P.S.K.; Abeyrathne, G.A.K.N.J. Sustainability of smes in the competition: A systemic review on technological challenges and sme performance. J. Open Innov. Technol. Mark. Complex. 2019,5, 100. [CrossRef] 26. Farrell, M.J. The measurement of productive efficiency. J. R Stat. Soc. Ser. A Stat. Soc. 1957,120, 253–281. [CrossRef] 27. Jorgenson, D.W.; Vu, K. Information technology and the world economy. Scand. J. Econ. 2005,107, 631–650. [CrossRef] 28. Okazaki, S. What do we know about mobile Internet adopters? A cluster analysis. Inf. Manag. 2006,43, 127–141. [CrossRef] 29. Matteo, C.; Simone, G.; Paolo, M. ICT and Growth: A DEA Analysis of 14 OECD Countries. L’industria 2011,3, 435–454. 30. Marche, S.; McNiven, J.D. E-government and e-governance: The future isn’t what it used to be. Can. J. Adm. Sci. 2003 ,20, 74–86. [CrossRef] 31. Barnes, S.J.; Vidgen, R.T. Data triangulation and web quality metrics: A case study in e-government. Inf. Manag. 2006 ,43, 767–777. [CrossRef] 32. Yildiz, M. E-government research: Reviewing the literature, limitations, and ways forward. Gov. Inf. Q. 2007,24, 646–665. [CrossRef] 33. Abu-Shanab, E.; Al-Azzam, A. Trust Dimensions and the adoption of E-government in Jordan. Int. J. Inf. Commun. Technol. Hum. Dev. 2012,4, 39–51. [CrossRef] 34. Al-Kibsi, G.; De Boer, K.; Mourshed, M.; Rea, N.P. Putting Citizens On-Line, Not in Line. McKinsey Q. 2001 , 64–73. Available online: https://www.proquest.com/openview/7e4ecc1c1e64e587745ff765694d975b/1?pq-origsite=gscholar&cbl=30375 (accessed on 1 October 2021). 35. West, D.M. E-government and the transformation of service delivery and citizen attitudes. Public Adm. Rev. 2004 ,64, 15–27. [CrossRef] 36. Von Haldenwang, C. Electronic government (e-government) and development. Eur. J. Dev. Res. 2004,16, 417–432. [CrossRef] 37. Norris, P. Critical Citizens: Global Support for Democratic Government; Oxford University Press Inc.: New York, NY, USA, 1999. 38. Anderson, P. Perspective: Complexity theory and organization science. Organ. Sci. 1999,10, 216–232. [CrossRef] 39. Fountain, J.E. Paradoxes of public sector customer service. Governance 2001,14, 55–73. [CrossRef] 40. Pillania, R.K. Competitiveness and emerging markets. Bus. Strategy Ser. 2009,10, 90–95. [CrossRef] 41. Batterbury, S.P.; Fernando, J.L. Rescaling governance and the impacts of political and environmental decentralization: An introduction. World Dev. 2006,34, 1851–1863. [CrossRef] 42. Kharlamova, G.; Vertelieva, O. The international competitiveness of countries: Economic-mathematical approach. Econ. Soc. 2013 , 6, 39–52. [CrossRef] 43. Asogwa, B.E. Electronic government as a paradigm shift for efficient public services: Opportunities and challenges for Nigerian government. Libr. Hi Tech 2013,31, 141–159. [CrossRef] 44. Waller, L.; Genius, A. Barriers to transforming government in Jamaica: Challenges to implementing initiatives to enhance the efficiency, effectiveness, and service delivery of government through ICTs (e-Government). Transform. Gov. People Process Policy 2015,9, 480–497. [CrossRef] 45. Prybutok, V.R.; Zhang, X.; Ryan, S.D. Evaluating leadership, IT quality, and net benefits in an e-government environment. Inf. Manag. 2008,45, 143–152. [CrossRef] 46. Irani, Z.; Love, P.E.; Montazemi, A. E-government: Past, present, and future. Eur. J. Inf. Syst. 2007,16, 103–105. [CrossRef] 47. Mohammed, F.; Ibrahim, O. Refining e-government readiness index by cloud computing. J. Teknol. 2013,65, 23–34. [CrossRef] 48. Koh, C.E.; Prybutok, V.R.; Zhang, X. Measuring e-government readiness. Inf. Manag. 2008,45, 540–546. [CrossRef] 49. Chanyagorn, P.; Kungwannarongkun, B. ICT readiness assessment model for public and private organizations in developing country. Int. J. Inf. Educ. Technol. 2011,1, 99–106. [CrossRef] 50. Azman, H.; Salman, A.; Razak, N.A.; Hussin, S.; Hasim, M.S.; Sidin, S.M. Determining critical success factors for ICT readiness in a digital economy: A study from user perspective. Adv. Sci. Lett. 2015,21, 1367–1369. [CrossRef] 51. Wielicki, T.; Arendt, L. A knowledge-driven shift in perception of ICT implementation barriers: Comparative study of US and European SMEs. J. Inf. Sci. 2010,36, 162–174. [CrossRef] 52. Kaylor, C.; Deshazo, R.; Van Eck, D. Gauging e-government: A report on implementing services among American cities. Gov. Inf. Q. 2001,18, 293–307. [CrossRef] J. Open Innov. Technol. Mark. Complex. 2022,8, 10 19 of 19 53. Norris, D.F.; Moon, M.J. Advancing e-government at the grassroots: Tortoise or hare? Public Adm. Rev. 2005 ,65, 64–75. [CrossRef] 54. Taranenko, I. Strategic Analysis of Innovation-Based Competitiveness in The Global Economy. Montenegrin J. Econ. 2013 ,9, 127–134. 55. Nasab, E.H.; Aghaei, M. The effect of ICT on economic growth: Further evidence. IBBA 2009,5, 46–56. 56. Babenko, V.; Perepelytsa, A. Research of the informatization process development of the countries of the World. Technol. Audit Prod. Res. 2018,6, 22–29. [CrossRef] 57. Babenko, V.; Perevozova, I.; Mandych, O.; Kvyatko, T.; Maliy, O.; Mykolenko, I. World informatization in conditions of international globalization: Factors of influence. GJESM 2019,5, 172–179. 58. James, J. The ICT Development Index and the digital divide: How are they related? Technol. Forecast. Soc. Change 2012 ,79, 587–594. [CrossRef] 59. Cuervo, M.R.V.; Menéndez, A.J.L. A multivariate framework for the analysis of the digital divide: Evidence for the European Union-15. Inf. Manag. 2006,43, 756–766. [CrossRef] 60. Osman, I.H.; Zablith, F. Re-evaluating electronic government development index to monitor the transformation toward achieving sustainable development goals. J. Bus. Res. 2021,131, 426–440. [CrossRef] 61. Wu, J.; Guo, S.; Huang, H.; Liu, W.; Xiang, Y. Information, and communications technologies for sustainable development goals: State-of-the-art, needs and perspectives. IEEE Commun. Surv. Tutor. 2018,20, 2389–2406. [CrossRef] 62. Gupta, N.; Benson, C.C. Sustainability, and competitive advantage: An empirical study of value creation. Compet. Forum 2011 ,9, 121–136. 63. Dyson, R.G.; Allen, R.; Camanho, A.S.; Podinovski, V.V.; Sarrico, C.S.; Shale, E.A. Pitfalls and protocols in DEA. Eur. J. Oper. Res. 2001,132, 245–259. [CrossRef] 64. Banker, R.D.; Cooper, W.W.; Seiford, L.M.; Thrall, R.M.; Zhu, J. Returns to scale in different DEA models. Eur. J. Oper. Res. 2004 , 154, 345–362. [CrossRef] 65. Charnes, A.; Cooper, W.W.; Rhodes, E. Measuring the efficiency of decision-making units. Eur. J. Oper. Res. 1978 ,2, 429–444. [CrossRef] 66. Emrouznejad, A.; Yang, G.L. A survey and analysis of the first 40 years of scholarly literature in DEA: 1978–2016. Socio-Econ. Plan. Sci. 2018,61, 4–8. [CrossRef] 67. Sutopo, W.; Astuti, R.W.; Suryandari, R.T. Accelerating a technology commercialization; with a discussion on the relation between technology transfer efficiency and open innovation. J. Open Innov. Technol. Mark. Complex. 2019,5, 95. [CrossRef] 68. Becchetti, L.; andres londono Bedoya, D.; Paganetto, L. ICT investment, productivity, and efficiency: Evidence at firm level using a stochastic frontier approach. J. Product. Anal. 2003,20, 143–167. [CrossRef] 69. Lee, B.; Sohn, E.; Won, D.; Yeo, W.; Kim, K.; Kim, S. The efficiency analysis of national R&D planning for the field of precision medicine in korea. J. Open Innov. Technol. Mark. Complex. 2018,4, 39. 70. Aristovnik, A. The impact of ICT on educational performance and its efficiency in selected EU and OECD countries: A nonparametric analysis. Turkish Online J. Educ. Technol. 2012,11, 144–152. [CrossRef] 71. Hwang, B.Y.; Jun, H.J.; Chang, M.H.; Kim, D.C. Efficiency Analysis of the Royalty System from the Perspective of Open Innovation. J. Open Innov. Technol. Mark. Complex. 2018,4, 22. [CrossRef] 72. Milana, C.; Zeli, A. The contribution of ICT to production efficiency in Italy: Firm-level evidence using data envelopment analysis and econometric estimations. OECD Sci. Technol. Ind. Work. Pap. 2002,2002, 1–60. 73. Susiluoto, I. Effects of ICT on Regional Economic Efficiency; Helsinki City Urban Facts Office Web Publications: Helsinki, Finland, 2003. 74. Thanassoulis, E. A data envelopment analysis approach to clustering operating units for resource allocation purposes. Omega 1996,24, 463–476. [CrossRef] 75. Tobin, J. Estimation of relationships for limited dependent variables. Econometrica 1958,26, 24–36. [CrossRef] 76. Samut, P.K.; Cafrı, R. Analysis of the efficiency determinants of health systems in OECD countries by DEA and panel tobit. Soc. Indic. Res. 2016,129, 113–132. [CrossRef] 77. Afzal, M.N.I. An empirical investigation of the National Innovation System (NIS) using Data Envelopment Analysis (DEA) and the TOBIT model. Int. Rev. Appl. Econ. 2014,28, 507–523. [CrossRef] 78. McDonald, J. Using least squares and tobit in second stage DEA efficiency analyses. Eur. J. Oper. Res. 2009,197, 792–798. [CrossRef] 79. Hsu, F.M.; Hsueh, C.C. Measuring relative efficiency of government-sponsored R&D projects: A three-stage approach. Eval. Program Plan 2009,32, 178–186. 80. Schwab, K. The Global Competitiveness Report 2019; World Economic Forum: Geneva, Switzerland, 2019.