EDMS Use in Local E-Government: Extent of Use and Overall Performance Mediated by Routinization and Infusion

Afonso et al. EDMS Use in Local E-Government Twentieth Americas Conference on Information Systems, Savannah, 2014 1 EDMS Use in Local E-Government: Extent of Use and Overall Performance Mediated by Routinization and Infusion Completed Research Paper Carlos Miguel Afonso University of the Algarve [email protected] Andrew Schwarz Louisiana State University [email protected] José L. Roldán Universidad de Sevilla [email protected] Manuel J. Sánchez - Franco Universidad de Sevilla [email protected] Abstract This study analyzes the effects of several post-adoption behaviors (extent of use, routinization and infusion) on overall performance in using an Electronic Document Management System (EDMS). Furthermore, we test whether the routinization and infusion variables mediate the influence of the extent of use on overall performance. This research collects data from a survey answered by 2,175 employees (EDMS users) of Portuguese municipalities. The Partial Least Squares technique is applied to test the model. The results showed that routinization is directly predicted by the extent of use, whereas infusion is directly affected by the extent of use and also by routinization. Consequently, such post-adoptive behaviors are interrelated not only in a sequential process, but also in parallel. In addition, overall performance is directly influenced by routinization and infusion. Finally, an indirect effects analysis shows that routinization and infusion mediate the relationship between extent of use and overall performance. Keywords Post-adoption behaviors, overall performance, partial least squares, edms, e-government. Introduction Electronic Document Management Systems (EDMS) are organizational information systems and technology (IST), which emerged from the 90s to manage digital documents for organizational needs (Sprague, 1995). EDMS allow organizations to carry out documental records management, to improve management processes, the organizational communication of concepts and ideas, to manage knowledge, and to enhance the efficiency of activities, and they have an important role in organizational memory (Borglund & Sundqvist, 2007; Hjelt & Björk, 2007; Sprague, 1995). Furthermore, EDMS enable governmental organizations and municipal councils to secure document transfers and to comply with legislative requirements (Wilkins et al., 2009; Sprehe, 2005). Despite all the potential benefits resulting from implementing EDMS in organizations, its success depends on the use that the great majority of employees make of the system (Johnston & Bowen, 2005; Gunnlaugsdottir, 2008). Therefore, a low use of this IST produces a low level of productivity (Venkatesh & Davis, 2000) and a high level of use potentially leads to better performances (Sundaram et al., 2007). In this way, to obtain benefits with a better performance, the use of EDMS (extent of use) is a necessary but not sufficient condition. The use of EDMS by employees must be active and thorough. Though the research stream related to the benefits resulting from EDMS use is important, it has been scantly expressed (e.g., Borglund & Sundqvist, 2007; Cho, 2007; Bhattacherjee et al., 2008; Gunnlaugsdottir, 2008; vom Brocke et al., 2011). This is why it is appropriate to develop more research to Afonso et al. Adoption and Diffusion of Information Technology (SIGADIT) 2 Twentieth Americas Conference on Information Systems, Savannah, 2014 address more insights about the link between EDMS uses and their results. Therefore, we intend in our work to study various types of use of EDMS, beyond the traditional measure of usage based on the frequency and amount of use (extent of use). These different types of post-adoption behaviors are a use integrated into the work routine (routinization) and a use that enhances individuals’ work productivity (infusion) (Sundaram et al., 2007). We also want to study how these various types of use relate to each other and how they affect the results of the use of an EDMS by a user. Furthermore, an analysis of the indirect relations is performed with the aim of assessing whether the routinization and infusion variables mediate in the relationship between extent of use and overall performance. In order to accomplish the research objectives proposed, this work is organized as follows. At the beginning, we present a literature review related to overall performance and the three types of postadoption behaviors that we intend to explore and also connect to the various relationships between them. As a result of this literature review, some hypotheses are proposed. Next, the research method is shown, followed by the results. A discussion of these results is addressed, based on an analysis of the data collected from a survey answered by 2,175 employees. The last section includes the conclusion, with the practical implications of the results for the work for managers and professionals. The limitations identified in this study are also offered. Literature Review and Research Hypotheses Overall Performance The use of IST plays a central role in IS (information systems) research (DeLone & McLean, 1992). This use is necessary for the performance improvement of individuals, groups and organizations (Myers et al., 1996). We can find some research related to the benefits of EDMS resulting from features (e.g., Sprague, 1995; Zantout & Marir,1999), implementation (e.g., Sprehe, 2000), and adoption (e.g., Bhattacherjee et al., 2008; Borglund & Sundqvist, 2007; vom Brocke et al., 2011; Cho, 2007; Gunnlaugsdottir, 2008). Research explicitly related to the benefits of EDMs has also been identified (e.g., Johnston & Bowen, 2005; Smyth, 2005; Sprehe, 2005; Wandryk, 1995; Wilkins et al., 2009). In the well-known IS success model, with the aim of keeping the research model simple, only net benefits were used to refer to individual, organizational, and societal impacts (DeLone & McLean, 2003). This approach was possible because these three impact variables are extraordinarily correlated. Overall performance will here be conceptualized as a broad concept that includes individual, workgroup and organizational performance. Individual performance is understood as being the effect of EDMS use on an employee’s work performance, usually measured by the increase in individual productivity and also by improving the decision capacity (Ifinedo et al., 2010). Group performance is the impact of EDMS use on subunits or departments of an organization, often measured by improving coordination, interdepartmental communication and productivity (Ifinedo et al., 2010). Organizational performance is usually measured by improvements in customer service, decision-making processes , among others (Ifinedo et al., 2010). To sum up, when the resulting effect of EDMS on an individual performance is strong, it is likely that the performance impact of the working group to which it belongs will also be strong. Furthermore, it is expected that the entire organization will get positive results when its units are positively affected (Ifinedo et al., 2010). The Role of Extent of Use Traditionally, usage has been measured as the amount or frequency of use (Schwarz & Chin, 2007), and this is the traditional surrogate measure of the acceptance of an IS (Saga & Zmud, 1994). We then conceptualize extent of use as a combination of the duration and frequency of use. On the other hand, routinization is defined as the extent to which the use of technology has been integrated into the normal work routine of a user, so the EDMS use is no longer perceived as out-of-the-ordinary but actually becomes institutionalized (Saga & Zmud, 1994). Routinization means an efficient use of the EDMS. Research results suggest an interrelationship between extent of use and routinization (Sundaram et al., 2007). It is expected that a greater extent of EDMS use could achieve and support a routinization level having the consequences of the EDMS use integrated into the employee work routine. From the above, the following hypothesis is presented: H1: Extent of use has a positive influence on routinization Afonso et al. EDMS Use in Local E-Government Twentieth Americas Conference on Information Systems, Savannah, 2014 3 The full degree of potential offered by an EDMS goes beyond the routinization stage (Sundaram et al., 2007). Therefore, we define infusion as the extent to which an EDMS user utilizes this system to its fullest degree to maximize its potential (Jones et al., 2002). Infusion involves the idea of effective use. Hall and Loucks (1977) observe that a continued interaction with an IS (i.e., extent of use) can lead to achieving a higher level of use, that is, infusion. It is expected that a higher extent of EDMS use could achieve and support an infusion level having the consequences of higher productivity, but less strong than if it were integrated into the employee work routine (Jones et al., 2002; Sundaram et al., 2007). Thus, we hypothesize: H2: Extent of use has a positive influence on infusion The role of Routinization As EDMS users become familiar with this IS, they might not be satisfied with the current use situation and may, exploring, find more useful functionalities to support their work tasks (Saga & Zmud, 1994). Therefore, through direct experience and learning processes accumulated in prior stages, users who attain the routinization level have the potential to use an EDMS in a more comprehensive and sophisticated manner, allowing them to attain the infusion stage (Po-An Hsieh & Wang, 2007). For instance, the digitalization of work leads to a decline in the substitutable routine labor and an increase in non-routine cognitive labor (Bhansali & Brynjolfsson, 2007). We therefore present the following hypothesis: H3: Routinization has a positive influence on infusion Routinization is a consequence of a learning process known as exploitation, which can improve performance outcomes (Po-An Hsieh & Zmud, 2006). Exploitation is linked to the routine execution of knowledge (March 1991). That is, the improvement of existing competencies by using what has already been learnt; namely, by adaptation. Its results are predictable, achieved quickly and positive, leading to foreseeable and immediate benefits for EDMS users in the short run (Burton-Jones & Straub, 2006). Therefore, it is expected that this level of EDMS use should enable an increase in overall performance, with a stronger effect than simple use and with a less significant effect than infusion. Therefore we hypothesize: H4: Routinization has a positive influence on overall performance The role of Infusion Infusion is achieved through the direct experience of users with the EDMS and an associated learning process known as exploration (March, 1991). This enables EDMS users to use this system to its full potential. Particularly, exploration allows the search for novel or innovative ways of doing things with an EDMS (Burton-Jones & Straub, 2006). Although, its results are not clearly certain in the short-term, longrun performance depends on sustaining a reasonable level of exploration (March, 1991). Indeed, when EDMS users achieve a higher level of usage this may allow them to exploit the fullest potential of an SNS, resulting in more positive overall performance (Cooper & Zmud, 1990). For instance, the increase in nonroutine cognitive labor and in value-adding communication activities leads to performance improvements and less stressed-out, happier and more productive employees (Bhansali & Brynjolfsson, 2007; Sundaram et al., 2007). From the above, the following proposition is presented: H5: Infusion has a positive influence on overall performance Method In terms of epistemology, this study uses a positivist approach generally followed in investigations applying structural equation modeling (Roldán & Sánchez-Franco, 2012). Regarding the methodology, the study pursues a quantitative research methodology, using the following methods: literature review, empirical work with primary data collection through a structured on-line questionnaire with closed questions, descriptive statistical techniques, and, to evaluate the model of the study, a variance-based structural equation modeling (Partial Least Squares - PLS). As the methods selected for this investigation are consistent with IS research recommendations (Straub et al., 2004) of the guidelines for the validation of positivist research in information systems, rigorous research is provided. Afonso et al. Adoption and Diffusion of Information Technology (SIGADIT) 4 Twentieth Americas Conference on Information Systems, Savannah, 2014 Participants An on-line survey was developed to gather information from 2,715 valid responses, allowing the measuring of the post-adoption behaviors and overall performance from using an EDMS in Portuguese municipalities. We applied a pre-questionnaire to validate the understanding of the meaning of the items by the EDMS users. This was given to employees of municipalities who are users of the system under study and members of a focus group. A simulation of sending the invitation message was also carried out. The development of this process resulted in some changes in some statements, formats and also in the correction of some errors. This pre-test process was repeated until we realized that the research tool was ready to be applied. We sent to all the 308 Portuguese municipalities an e-mail message of invitation: "Request for collaboration in a research project". Nearly 10% responded within a period of 15 days. Given the small number of responses, there was a follow-up by telephone (Urbach & Ahlemann, 2010). This approach resulted in a total of 96 municipalities agreeing to participate. From this total, 25 municipalities with the system used to record correspondence or with the system still being installed were not used in this study. The participating municipalities had to have a working EDMS and have started the process of the dematerialization of documents and processes. The Presidents of the 71 municipalities involved in the research indicated an employee to send the on-line survey address to the registered EDMS users. The sending of these electronic mail messages to "Request for the participation of EDMS users" started 15 days after the municipalities were contacted; reminder messages were sent at intervals of about 15 days (Urbach & Ahlemann, 2010). The power analysis approach was used for the analysis of the necessary number of responses. The required 599 (Green, 1991) responses for an alpha value of 0.05, a high power of 0.80, a small effect size and four predictors were largely exceeded. Detailed descriptive statistics relating to the respondents' characteristics are shown in Table 1. Measure (responses = 2175) Items Frequency Percent Gender Male 748 34% Female 1427 66% Age < 24 12 0.6% 25 - 34 634 29.1% 35 - 44 921 42.3% 45 - 54 501 23.0% >55 107 4.9% Education Less than Secondary school 112 5.2% Secondary school 802 36.9% University 1089 50.1% Master 167 7.7% PhD 5 0.2% EDMS Experien ce < 2 years 546 25.1% 2 - 5 years 1281 58.9% > 5 years 348 16.0% Internet Experience <=7 years 364 16.7% 8 - 10 years 502 23.1% 11 - 13 years 577 26.5% 14 - 16 years 467 21.5% >=17 years 265 12.2% Municipality Region Littoral 1953 89.8% Interior 2 22 10.2% Table 1. Descriptive statistics of the respondents' characteristics. Afonso et al. EDMS Use in Local E-Government Twentieth Americas Conference on Information Systems, Savannah, 2014 5 Measures The questionnaire was designed from a comprehensive literature review. Validated instruments adapted from prior studies have been selected to measure use, routinization, infusion and overall performance. All the selected scales have been modified to make them relevant to the EDMS-based context (all items appear in Appendix A). In particular, the variable extent of use was self-reported and was measured by three items adapted from Moon and Kim (2001). First, respondents were asked how many hours they had used the EDMS in the last week. Second, respondents were asked to rate their EDMS use intensity on a seven-point scale ranging from extremely infrequent to extremely frequent. Third, respondents were asked to rate their EDMS usage on an eight-point scale ranging from not at all to several times a day. For the items of the rest of the constructs, a seven-point Likert scale was applied, ranging from “completely disagree” (1) to “completely agree” (7). In this vein, the routinization measure has been adapted from Sundaram et al. (2007). The infusion scale has been taken from the study of Jones et al. (2002). These three previous variables have been modeled as first order reflective constructs. On the other hand, overall performance has been designed as a superordinate multidimensional construct (Polites et al., 2012) that is reflected in three reflective dimensions: individual impact (II), workgroup impact (WI) and organizational impact (OI). These dimensions have been measured with items proposed by Ifinedo et al. (2010). In order to achieve the content validity, a pilot test was carried out with five IS and business professors. Data Analysis The research model proposed has been assessed using Partial Least Squares (PLS), a variance-based structural equation modeling (Reinartz et al., 2009). PLS has been chosen based on the following grounds (Roldán & Sánchez-Franco, 2012): (1) This research is focused on the prediction of the dependent variables; (2) the phenomenon under research has not yet been studied in depth; (3) the research model is complex in terms of the number of indicators, the types of relationships (direct and indirect), and the levels of dimensionality; (4) we use latent variables scores in subsequent analysis for a predictive relevance. Overall performance, as a superordinate multidimensional construct, has been operationalized applying a two-stage approach (Wright et al., 2012). We have used SmartPLS software (Ringle et al., 2005). Results Measurement Model The results show that the measurement model satisfies all common requirements (Roldán & Sánchez- Franco, 2012). Individual item reliability is adequate when an item has a factor loading that is greater than 0.7 for its construct or dimension. In our study, reflective indicators and dimensions meet this requirement (Table 2). Afonso et al. Adoption and Diffusion of Information Technology (SIGADIT) 6 Twentieth Americas Conference on Information Systems, Savannah, 2014 Extent of use Routinization Infusion Overall performance eu1 0.868 0.697 0.458 0.254 eu2 0.861 0.644 0.484 0.321 eu3 0.734 0.459 0.386 0 .190 rot1 0.633 0.902 0.565 0.413 rot2 0.718 0.953 0.636 0.436 rot3 0.705 0.927 0.595 0.416 inf1 0.551 0.657 0.910 0.511 inf2 0.507 0.599 0.917 0.506 inf3 0.269 0.339 0.730 0.403 inf4 0.491 0.587 0.895 0.535 II 0.318 0.435 0.574 0.925 WI 0.285 0.4 20 0.508 0.938 OI 0.274 0.413 0.496 0.929 Table 2. Loadings and cross-loadings. Since all composite reliabilities and Cronbach´s alphas are greater than 0.7 (Table 3), the model presents reliable constructs and dimensions. Convergent validity is assessed using the average variance extracted (AVE). All reflective constructs and dimensions attain convergent validity because their AVE figures surpass the 0.5 level (Table 3). Finally, all variables achieve discriminant validity. Confirmation of this validity comes, on the one hand, from the cross loadings analysis (Table 2) and, on the other hand, from the comparison of the square root of AVE versus correlations (Table 3). Composite reliability Cronbach´s alpha AVE EU ROT INF OP 0.863 0.762 0.678 EU 0.823 0.949 0.918 0.860 ROT 0.740 0.927 0.923 0.888 0.751 INF 0.540 0.646 0.867 0.951 0.923 0.866 OP 0.315 0.455 0.567 0.931 Notes: 1) Diagonal elements (bold) are the square roots of the variance shared between the constructs and their measures (AVE). Off-diagonal elements are the correlations between constructs. For discriminant validity, diagonal elements should be larger than off-diagonal elements. 2) EU: Extent of Use; ROT: Routinization; INF: Infusion; OP: Overall Performance. Table 3. Construct reliability, convergent validity and discriminant validity Structural Model The assessment of the structural model is based on the algebraic sign, magnitude and significance of the structural path coefficients, the R 2 values, and the Q 2 test for predictive relevance. Bootstrapping (5000 resamples) is used to generate standard errors and t-statistics (Hair et al., 2011), which allow the evaluating of the statistical significance of path coefficients. Concurrently, percentile bootstrapping confidence intervals of standardized regression coefficients are given. In this manner, all hypotheses are supported since the five direct relationships depicted in the research model are positive and significant (Figure 1, Table 4). In accordance with Chin (1998), we achieve moderate values of variance explained in the dependent constructs since R 2 values are above 0.33 (Table 4). In addition, cross-validated redundancy measures confirm that the structural model has satisfactory predictive relevance for the three endogenous constructs (Q 2 > 0). Finally, Table 4 shows the amount of variance that each antecedent construct explains on each dependent variable. In this way, we would like to highlight that while routinization explains only 6.87% of the variance of overall performance, the infusion variable manages to explain 26.65%. Afonso et al. EDMS Use in Local E-Government Twentieth Americas Conference on Information Systems, Savannah, 2014 7 Figure 1. Structural model results Direct effect t-value Explained variance Percentile 95% confidence intervals Routinization (R 2 = 0.548 / Q 2 = 0.470) EU ( a 1 ) 0.7 40 *** 66.841 54.77% 0.718 0.761 Sig. Infusion (R 2 = 0.426 / Q 2 = 0.300) EU ( a 2 ) 0.136 *** 5.256 7.37% 0.084 0.188 Sig. ROT ( a 3 ) 0.545 *** 21.469 35.25% 0.495 0.597 Sig. Overall Performance (R 2 = 0.335 / Q 2 = 0.269) ROT ( b 1 ) 0.151 *** 5 .746 6.87% 0.099 0.201 Sig. INF ( b 2 ) 0.470 *** 19.633 26.65% 0.423 0.515 Sig. Notes: 1) EU: Extent of Use; ROT: Routinization; INF: Infusion; OP: Overall Performance. 2) *** p < 0.001, (based on t(4999), one-tailed test), t(0.001, 4999) = 3.09. Sig. denotes a significant direct effect at 0.05 Table 4. Effects on endogenous variables. Finally, a post-hoc indirect effect analysis is presented. In this way, we have followed the analytical approach proposed by Preacher and Hayes (2008) to test the following mediated relationships: a 1 b 1 , a 2 b 2 , and a 1 a 3 b 2 . Chin (2010) proposes a two-step procedure for testing mediation in PLS: (1) use the specific model in question with both direct and indirect paths included and perform N bootstrap resampling and explicitly calculate the product of the direct paths that form the indirect path being assessed. (2) Estimate the significance using percentile bootstrap (Williams & MacKinnon, 2008). This generates a 95% confidence interval (CI) for the indirect relationships. As Table 5 shows, all the indirect relations are significant. This means that the routinization and infusion dimensions mediate the influence of use on overall performance. Afonso et al. Adoption and Diffusion of Information Technology (SIGADIT) 8 Twentieth Americas Conference on Information Systems, Savannah, 2014 Indirect effect Point estimate Percentile bootstrap 95% confidence interval Lower Uppe r a 1 b 1 = EU  ROT  OP 0.149 0.073 0.149 a 2 b 2 = EU  INF  OP 0.089 0.040 0.089 a 1 a 3 b 2 = EU  ROT  INF  OP 0.218 0.162 0.218 Total 0.396 0.334 0.396 Note: 1) EU: Extent of Use; ROT: Routinization; INF: Infusion; OP: Overall Performance. Table 5. Indirect effects of extent of use on overall performance Discussion Theoretical Implications This study addresses the behavior and interrelationships of different types of use – extent of use, routinization and infusion – contributing to a growing research interest in understanding different types of IST use (e.g., Burton-Jones & Straub, 2006; Gharavi & Carter, 2009; Jones et al., 2002;McLean et al., 2011; Sundaram et al., 2007; Schawartz & Chin, 2007; Tennant et al. 2011; Venkatesh et al., 2008). The results of the study conform with the work of Sundaram et al. (2007), demonstrating that the higher the level of use of the EDMS by a user is, the better the results obtained with its use are – such as an increase in overall performance. In fact, infusion represents the best predictor of overall performance (explaining 26.65% of its explained variance) compared to routinization. The possibility – mentioned by Cooper and Zmud (1990) and Saga and Zmud (1994) – of verifying the parallel relationships between extent of use and infusion was empirically supported, as well as the indirect relationship between extent of use and infusion via routinization. Furthermore, the research results showed that routinization and infusion mediate the influence of extent of use on overall performance. This study also presents significant support for the relations in the model of success of DeLone and McLean (2003) by verifying the relationships between post-adoption behaviors and the net benefit of use (overall performance). This contributes to the consolidation of work in this area of Information Systems Success studies. It is also noted that the values obtained for the measures used to study the use of EDMS in the municipalities have proved to be adequate. Practical Implications This research also provides implications for information systems and management professionals in charge of EDMS. These professionals have the important aim of increasing the use of EDMS and the resulting value. Despite the high investments in IST, the expectations of use are often inconsistent with the rate of actual use. In this sense, these professionals need to understand the uses and results of employees using the EDMS, producing the creation of measures to be applied to the development of new EDMS or to correct existing EDMS. With this in mind, this work provides the professionals with a model that can be applied within an organization and is therefore useful toward having a better understanding regarding EDMS use. The results suggest that managers should promote the increasing of EDMS use in order for users to integrate the EDMS into their work routine. For this purpose, managers should employ training in the workplace tailored to users’ specific needs. Also, after EDMS integration into the work routine, we recommend monitoring (using a log system to measure actual use) the EDMS use by developing use encouragement actions. With these actions, employees should use EDMS more frequently and efficiently, thus achieving and supporting an infusion level, which brings about a higher productivity (Jones et al., 2002; Sundaram et al., 2007). The level of infusion may also be achieved by training which is specially designed for this purpose. Limitations and Future Research The first limitation is related to the object of study, which does not differentiate between the various EDMS software solutions. Second, the present study is limited to not being longitudinal, which does not allow having values that reflect the dynamics of the variables and the relationships between them over Afonso et al. EDMS Use in Local E-Government Twentieth Americas Conference on Information Systems, Savannah, 2014 9 time – perceptions vary over time and users will also gain experience. Third, extent of use, which is measured in a self-reported way, is also a limiting feature of many studies of Acceptance. Finally, derived from the object of study, the EDMS can be a tool to be used to perform various tasks, and this study is vague at this task level. The study of the EDMS associated with a specific task and not the tool itself is recommended. Acknowledgements This research was supported by the Junta de Andalucía (Consejería de Economía, Innovación y Ciencia) Spain (Proyecto de investigación de excelencia SEJ-6081). REFERENCES Bhansali, S., and Brynjolfsson, E. 2007. “Digitizing Work: Driving and Measuring Changes in Information Worker Time Use and Performance Via a Longitudinal Quasi-Experiment,” in ICIS 2007 Proceedings, , December 31, p. Paper 26. Bhattacherjee, A., Perols, J., and Sanford, C. 2008. “Information technology continuance: A theoretical extension and empirical test,” Journal of Computer Information Systems (49:1), pp. 17–26. Borglund, E. A. M., and Sundqvist, A. 2007. “The role of EDM in information management within SMEs,” in 2nd International Conference on Digital Information Management, 2007. ICDIM ’07, (Vol. 1) Presented at the 2nd International Conference on Digital Information Management, 2007. ICDIM ’07, , October, pp. 31–36. Burton-Jones, A., and Straub, D. W. 2006. “Reconceptualizing System Usage: An Approach and Empirical Test,” Information Systems Research (17:3), pp. 228–246. Chin, W. 2010. “How to Write Up and Report PLS Analyses,” in Handbook of Partial Least Squares, Springer Handbooks of Computational Statistics, V. Esposito Vinzi, W. W. Chin, J. Henseler, and H. Wang (eds.), Springer Berlin Heidelberg, pp. 655–690. Chin, W. W. 1998. “The partial least squares approach for structural equation modeling,” in Modern methods for business research, Methodology for business and management, Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers, pp. 295–336. Cho, V. 2007. “A Study of the Impact of Organizational Learning On Information System Effectiveness,” International Journal of Business and Information (2:1), pp. 127–158. Cooper, R. B., and Zmud, R. W. 1990. “Information Technology Implementation Research: A Technological Diffusion Approach,” Management Science (36:2), pp. 123–139. DeLone, W. H., and McLean, E. R. 1992. “Information systems success: the quest for the dependent variable,” Information Systems Research (3:1), pp. 60–95. Delone, W. H., and McLean, E. R. 2003. “The DeLone and McLean model of information systems success: a ten-year update,” Journal of Management Information Systems (19:4), pp. 9–30. Gharavi, H., and Carter, L. 2009. “A Reflection on the Use of Structured Case Analysis: Infusion of Information Systems in a Regulated Sector,” in AMCIS 2009 Proceedings, January 1, Paper 475. Green, S. B. 1991. “How many subjects does it take to do a regression analysis?” Multivariate Behavioral Research (26:3), pp. 499–510. Gunnlaugsdottir, J. 2008. “Registering and searching for records in electronic records management systems,” International Journal of Information Management (28:4), pp. 293–304. Hair, J. F., Ringle, C. M., and Sarstedt, M. 2011. “PLS-SEM: Indeed a Silver Bullet,” The Journal of Marketing Theory and Practice (19:2), pp. 139–152. Hall, G., and Loucks, S. 1977. “A developmental model for determining whether the treatment is actually implemented,” American Educational Research Journal (14), pp. 263-276. Hjelt, M., and Björk, B. 2007. “End-User Attitudes toward EDM Use in Construction Project Work: Case Study,” Journal of Computing in Civil Engineering (21:4), pp. 289–300. Ifinedo, P., Rapp, B., Ifinedo, A., and Sundberg, K. 2010. “Relationships among ERP post-implementation success constructs: An analysis at the organizational level,” Advancing Educational Research on Computer-supported Collaborative Learning (CSCL) through the use of gStudy CSCL Tools (26:5), pp. 1136–1148.