Towards an instrument for surveying knowledge management practices

Botha, D. F. Article Towards an instrument for surveying knowledge management practices South African Journal of Business Management Provided in Cooperation with: University of Stellenbosch Business School (USB), Bellville, South Africa Suggested Citation: Botha, D. F. (2005) : Towards an instrument for surveying knowledge management practices, South African Journal of Business Management, ISSN 2078-5976, African Online Scientific Information Systems (AOSIS), Cape Town, Vol. 36, Iss. 1, pp. 1-6, https://doi.org/10.4102/sajbm.v36i1.615 This Version is available at: https://hdl.handle.net/10419/218316 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/ S.Afr.J.Bus.Manage.2005,36(1) 1 Towards an instrument for surveying knowledge management practices D.F. Botha Centre for Knowledge Dynamics and Decision-making, Department of Information Science, University of Stellenbosch Stellenbosch 7600, Republic of South Africa [email protected] Received July 2004 The research was conducted to develop and stabilize a data collection and analysis instrument for an annual survey of knowledge management practices in the South African business sector. From a literature study it was deduced that six factors could be identified with necessary and sufficient reason to be used as main parameters for the assessment instrument. Synthesized a priori and posteriori judgmental knowledge was used to construct a number of aggregational indicators for each factor. Each indicator posited as a statement being a recognized knowledge management practice. A modified six point Likert scale was created to score/indicate gradual progression towards full implementation of a specific practice. Regression factor analysis and one-way analysis of variance was used on the collected data to evaluate the instrument. These analyses indicated that the instrument could detect the expected differences on the demographics and on the KM practices of the survey and could therefore be declared sound and verified. The analyses also revealed several modifications that could be used to improve the instrument. Introduction This paper outlines the research methodology used for the ‘Development of a SA Knowledge Management Audit Instrument’ and includes important statistical outputs from the analyses. The purpose of the paper is to provide interested readers with more background on how the research for the development of the instrument was conducted. It is organized in the following subsections: (1) research objective; (2) literature review and model development; (3) survey instrument development; (4) data collection and sample description; (5) survey data analysis; (6) instrument evaluation/verification; (7) conclusion. Research objective The objective of the study was to develop and stabilize a methodology, including a data collection and analysis instrument for an annual audit of knowledge management (KM) practices in the SA business sector. The aim of this paper is to report on the development and evaluation of the audit instrument. Literature review and model development By careful examination of the literature on topics related to measures of knowledge management and knowledge management practices, a conceptual framework for the measurement model was construed (Botha & Fouche, 2002). Primary factors representing key organizational variables were identified by using deductive/inductive reasoning, judgement based on grounded management theory, frequency of discussion by notable academics and practitioners, focus of research topics, and qualitative and quantitative emphasis placed on the main factors – the dependent variables. The factors finally selected and judged to be representative of an organisational knowledge ecology were knowledge leadership, organizational culture, structures, processes and routines, information and communication technology and measures. A comparative analysis of similar assessment models, found during the literature examination, was done. The most important in this category was the work done by David Skyrme Associates (Skyrme, 1999) and a joint development by Arthur Andersen (AA) and The American Productivity & Quality Center (RSA, 1996). Skyrme used ten key dimensions of which five, namely, leadership, culture/structure, processes, technology and measures corresponded with the selections for this study. The AA model on the other hand is based on four knowledge management enablers, leadership, culture, technology and measurements, which act in a dynamic relationship with the so called KM processes, namely, create, identify, collect, adapt, organize, apply, and share. The emphasis in the model, called the knowledge management reference model (KMRM) used for this study is however, on the interrelationship between the four primary factors, culture, structure, processes and technology, which are in continual alignment with leadership and monitored by measures. The description and validation of this model – of which the data model is depicted in Figure 1 below – was presented as a paper at an international conference in Pretoria and subsequently published in the relevant proceedings of the conference (Botha & Fouche, 2002).The rationale for using this model as basis for the development of the instrument and the comparative and definitive discussion of the six constructs (primary factors) are covered in this paper. 2 S.Afr.J.Bus.Manage.2005,36(1) Figure 1: Knowledge Management Reference Model (KMRM) Audit instrument development KMRM was used as basis for the development of a KM practice audit instrument suitable for the intended survey. For each of the six main factors discussed above a number of knowledge management practices deemed to be aggregated indicators – the independent variables – of these factors were selected. Criteria used for this selection were the quantitative and qualitative mention and discussion of these practices by notable practitioners and researchers as contributors and indicators of sustainable KM practices (Allee, 1997; Badaracco, 1991; Skyrme, 2000; Cortada & Woods, 2001; Davenport & Prusak, 1998; Marchand, Davenport & Dickson, 2000; McDermott, 1999; Myers, 1996; Sveiby, 1997; Zack, 1999). Although it was suggested by the literature, the supposition that a causal relationships between the main factors exist was accepted. Similarly a supposition that a specific knowledge management practice can contribute to more than one of the main factors was also postulated. This reasoning was then used to construct the Knowledge Management practices survey instrument. The set of knowledge management practices for each of the six factors was formulated in the form of statements. Each statement attempts to describe a knowledge management practice, employed by a world class organization in pursuit of sustainable competitive advantage – deemed to be a best practice (Allee, 1997; Badaracco, 1991; Cortada & Woods, 2001; Davenport & Prusak, 1998; Marchand et al., 2000; Myers, 1996; Skyrme, 1999 & 2000; Sveiby, 1997; Zack, 1999). The contents of the statements are analyzedsynthesized a priori judgements made on information from the extant literature and put forward as the most plausible (Kant, 1929). Twenty-four statements – best practices – were formulated in this way. A modified Likert – a composite measure with an internal intensity order – ordinal scoring scale was designed to indicate progressive degrees in the state of the implementation of knowledge management practices. This scale guides individuals to assess the present status of a specific KM practice in their organization. The modified Likert scale conforms to the criteria as proposed by Sedlack and Stanley (1992). The twenty-four statements constituting the knowledge management practices and the selected Likert scoring scale were composed into an audit instrument. The data-model, therefore consist of the six main factors – the dependent variables, each dependent on three or more observable variables – the independent variables, counting as 24 indicators in total. Respondents could score these 24 indicators over a range of six intervals counting from zero to five, where a score of five indicates the highest state of implementation of a KM practice. Pilot-testing of the model and the audit instrument were conducted at several South African based companies that were reputedly well advanced in the practice of knowledge management, some with considerable global experience. Data collection and sample description Data was collected by doing several presentations on the knowledge management reference model (KMRM) to groups of South African company representatives at venues in Cape Town, Johannesburg and Pretoria. After the presentations the attendees were requested to complete the audit instrument. Usable data was received from 83 respondents. At a separate session data were collected from 12 additional respondents from a single company. These KM practice performance Leadership (L) Culture (C) Structure (S) Processes (P) Technology (T) Measures (M) 6x Indicators 3x Indicators 5x Indicators 3x Indicators 3x Indicators 4x Indicators S.Afr.J.Bus.Manage.2005,36(1) 3 targeted respondents (Marchand et al., 2001) were largely from the upper echelons of the companies they represented i.e. from the executive and managerial level. Survey data analysis Aim The aim of the data analysis was twofold. The primary aim was to verify the audit instrument. The secondary aim was to give a first order presentation of the status of KM practices in the SA industry (Botha & Fouche, 2002). Principal component analysis Principal Component Analysis (PCA) is but one method of classical linear Multivariate Analysis (MVA) (Van de Geer, 1993:17), but it is probably the oldest and best known of the techniques of multivariate analysis and extensively used in the behavioural sciences. Although the term ‘principal component analysis’ is in common usage, other terminology may be encountered for the same technique, for example, ‘empirical orthogonal functions’, ‘factor analysis’, ‘eigenvector analysis’ and ‘latent vector analysis’ may camouflage PCA. The central idea of PCA is to reduce the dimensionality of a data set in which there are a large number of interrelated variables, while retaining as much as possible of the variation present in the data set. This reduction is achieved by transforming to a new set of variables, the principal components, which are ordered so that the first few retain most of the variation present in all of the original variables (Jolliffe, 1986). The basic idea of the technique is to describe the set of multivariate data in terms of a set of uncorrelated variables each of which is a particular linear combination of the original variables. The new variables are derived in decreasing order of importance so that, for example, the first principal component accounts for as much as possible of the variation in the original data (Everitt & Dunn, 1991:45). Generally if a set of p (>2) variables has substantial correlation among them, then the first few principal components (PCs) will account for most of the variation in the original variables, but the total variation in the original p variables is only accounted for by all p principal components. Algebraically, the PCA involves finding the eigenvalues and eigenvectors of the sample correlation matrix (Gnanadesikan, 1997: 8). PCA can be based on either, sample correlation or population covariance matrices. For this study the former was used Combination of scores When several scores aimed at assessing the same characteristic are available it may be advantageous both for parsimony, and to aid interpretation to combine them into a single score that captures the essential information. Therefore the importance of the first principal component is that it can be used to calculate a single score on any of the six main factors – dependent variables – for a single company, using data collected with the audit instrument on that company. In the following sections the results of applying this technique to certain groups of scores are presented. Evaluating/Verifying the KM audit instrument Leadership The three Leadership (L) scores are quite highly correlated, as can be seen from the correlation coefficients in Table 1. Table 1: Correlation coefficients for Leadership (L) L 1 L 2 L 2 0,804 L 3 0,523 0,606 A Principal Component Analysis of these scores revealed that 77% of total variation is explained by the first principal component (LPr1); it is expressed as a linear function of the three scores: LPr1=0.591*L1+0.612*L2+0.525*L3 where L1, L2 and L3 are standardized versions of the original scores. LPr1 can be re-written in terms of the original scores, but for the purpose of this analysis the relative weights of the scores are more meaningful, as shown in Figure 2. 0.53 0.61 0.59 L1-Vision L3-Org Learn Leadership (L) L2-Strategy Figure 2: Relative weights of the Leadership indicators Due to a very convincing LPr1 the relative contributions of L1, L2 and L3 are sufficient to use their mean score as a single score for Leadership, i.e. for example from the surveyed sample the mean score for Leadership was L = 2,63. Because of the high correlation between L1 and L2 it could be considered to use only one of the scores in a revised instrument. Culture The correlation between the scores on Culture (C) is as depicted in Table 2. 4 S.Afr.J.Bus.Manage.2005,36(1) Table 2: Correlation coefficients for Culture (C) C4 C5 C6 C7 C8 C5 0 ,558 C6 0 ,463 0 ,368 C7 0 ,585 0 ,542 0 ,351 C8 0 ,717 0 ,516 0 ,392 0 ,554 C9 0 ,648 0 ,372 0 ,428 0 ,418 0 ,744 The highest correlation in this case is between C4, C8 and C9. The aggregation of the six C scores into a single score is not as convincing as the previous case but because the principal component, CPr1 explains 61% of the total variation, the mean of the six scores could be used as a single score for C as expressed by the linear function: CPr1=0.464*C4+0.370*C5+0.330*C6+0.385*C7+0.461*C 8+0.423*C9 This is graphically depicted in Figure 3. 0.42 0.46 0.39 0.33 0.37 0.46 C4-Comm C5-Cus Orient C6-Collabor C7-Workplace C8-Sharing C9-Contribute Culture (C) Figure 3: Relative weights of the Culture indicators The high correlation between C4, C8 and C9 indicates possibilities of aggregation into a single indicator. Structure The correlation between independent variables for Structure is shown in Table 3. Table 3: Correlation coefficients for Structure (S) S10 S11 S12 S13 S11 0 ,434 S12 0 ,482 0 ,771 S13 0 ,336 0 ,542 0 ,498 S14 0 ,303 0 ,388 0 ,399 0 ,320 The high correlation between S11 and S12 indicates a possible single indicator. The first principal component, SPr1 explains 55% of the total variation, the second 16% and the third 14%. Aggregating the five S scores into a single score is not convincing, but because of a convincing first principal, SPr1 the mean of the five S scores can be used as a single score. SPr1=0.409*S10+0.516*S11+0.517*S12+0.442*S13+0.323 *S14 The relative weights of the five S indicators are shown in Figure 4. 0.41 0.52 0.44 0.52 0.32 S10-Teams S11-KM Roles S12-Mancoms S13-Incentives S14 Alliances Structure ( S ) Figure 4: Relative weights of the Structure indicators Note: The importance of the first principal component is that it can be used to calculate a single score on any of the six main factors for a single company, using data collected with the audit instrument on that company. Processes The correlation between independent variables (indicators) for the dependent variable Processes (P) is shown in Table 4: Table 4: Correlation coefficients for Processes (P) P15 P16 P16 0,523 P17 0,520 0,704 Again the correlation between P16 and P17 is such that a single indicator could be construed to collect data for this score. The first principal component, PPr1 explains 72% of the total variance and therefore highly convincing. Thus the three scores of the factor P can be aggregated to present as a single score as shown in Figure 5. PPr1+0.533*P15+0.599*P16+0.598*P17 0.53 0.60 0.60 P15 K Integrat P16-Info Man P17-Buss Intel Processes (P) Figure 5: Relative weights of Processes indicators Technology The independent variables correlation for Technology (T) is shown in Table 5. S.Afr.J.Bus.Manage.2005,36(1) 5 Table 5: Correlation coefficients for Technology (T) T18 T19 T19 0,839 T20 0,352 0,357 With the very high correlation between T18 and T19 sufficient evidence is revealed to combine the two into one indicator. Therefore it might not be necessary to ask respondents to score on both IS architecture (T18) and IT infrastructure (T19), a single indication on one of the two will be sufficient to score Technology (T). The first primary component explains 69% of the total variance, proving this measure on technology to be a sound and useful entity of the instrument. TPr1=0.640*T18+0.641*T19+0.423*T20 0.64 0.42 T18 IS Architec T19 IT Infrastr T20-KM Apps Technology Figure 6: Relative weights of Technology indicators The three scores, T18-T19 can again be aggregated to produce a single score for the factor T. Note that from the formula for TPr1 it can be deducted that the contribution of T20 is far less than that of T18 and T19 but still sufficient to warrant its retention as an indicator. Measures The correlations: Table 6: Correlation coefficients for Measures (M) M21 M22 M23 M22 0 ,660 M23 0 ,637 0 ,960 M24 0 ,551 0 ,434 0 ,446 Because of the very high correlation between M22 and M23, one of the two can be left out of a revised instrument. The first principal component explains 72% of the total variance, and the second 18%. Again the significant contribution – due to the relative weights – of M21-M24 supports their aggregation into a single score for the factor M. MPr1=0.495*M21+0.546*M22+0.544*M23+0.401*M24 0.50 0 55 0.54 0.40 M21-Perf Ind M22 KM Tools M23 Prog Rep M24-Alignment Measures Figure 7: Relative weights for Measures indicators Correlation between the six main factors The following correlations between the principal components of L&C, L&S, L&P, L&M, L&T, C&S, C&P, C&M, C&T, S&P, S&M, S&T, P&M, P&T, and T&M were calculated: Table 7: Correlation coefficients for principal components LPr1 CPr1 SPr1 PPr1 TPr1 CPr1 0 ,629 SPr1 0 ,707 0 ,724 PPr1 0 ,493 0 ,581 0 ,637 TPr1 0 ,350 0 ,478 0 ,493 0 ,639 MPr1 0 ,579 0 ,404 0 ,675 0 ,653 0 ,424 These correlations are in general relatively high; all are statistically significant on the 5% level. This signifies that strong causal relationships between the main factors were detected by the instrument as suggested by the KMRM. This provided further evidence to verify the usefulness of the instrument. The instrument was found to be sound. Some results of the testing pointed to possible modifications that could improve the data collection model. Conclusion There is ample evidence from the analyses above that the developed survey instrument is sound and that it could be verified as a useful instrument to conduct audits on knowledge management practices in companies. Subsequent to this research the South African Information and Knowledge Management Tool (SAIKMAT™) was developed and successfully applied by management consultants during KM strategy workshops for organizations like De Beers Industrial Diamonds, the Human Sciences Research Council and the Department of Home Affairs. The validation methodology made it possible to customize the audit instrument to suit specific organizations. Although much progress has been made during the past decade to develop a philosophy and conceptual frame of KM, the discipline still lacks proven practice. Rigorous measurement and assessment of KM practices should contribute to the development of an empirical base of best practice. Further research should be conducted to explore possible causal correlation between knowledge management practices and organizational performance. 6 S.Afr.J.Bus.Manage.2005,36(1) References Allee, V. 1997. The knowledge evolution – expanding organizational intelligence. Boston MA: Butterworth- Heinemann. Badaracco, Jr. J.L. 1991. The knowledge link – how firms compete through strategic alliances. Boston MA: Harvard Business School Press. Botha, D.F. & Fouche, B. 2002a. ‘The assessment of corporate knowledge management practices: the role of a reference model.’ Proceedings of the 2nd Biennial DISSAnet Conference, Pretoria, pp.279-293. Botha, D.F. & Fouche, B. 2002b. ‘Knowledge management practices in the South African business sector: Preliminary findings of a longitudinal study’, South African Journal of Business Management, 32(2):13-19. Cortada, J.W. & Woods, J.A. 2001. The Knowledge Management Yearbook 2000-2001. Boston MA: Butterworth-Heinemann. Davenport, T.H. & Prusak, L. 1998. Working knowledge – how organizations manage what they know. Boston: Harvard Business School Press. Everitt, B.S. & Dunn, G. 1991. Applied multivariate data analysis. London: Edward Arnold, a division of Hodder & Stoughton. Gnanadesikan, R. 1997. Methods for statistical analysis of multivariate observations. 2 nd Edition. New York: John Wiley & Sons. Jolliffe, I.T. 1986. Principal component analysis. New York: Springer-Verlag. Kant, I. 1929. Critique of pure reason. Tr. Norman Kemp Smith. London. Boston: Allyn and Bacon. Marchand, D.A., Davenport, T.H. & Dickson, T. 2000. Mastering information management – complete MBA companion in information management. London: Financial Times, Prentice Hall. Marchand, D.A., Kettinger, W.J. & Rollins, J.D. 2001. Information orientation:Tthe link to business performance. Oxford: Oxford University Press. McDermott, R. 1999. ‘Why information technology inspired but cannot deliver knowledge management’, California Management Review, 41(4):103-107. Myers, P.S. 1996. Knowledge management and organizational design. Boston MA: Butterworth- Heinemann. RSA, 1996. Tomorrow’s company inquiry: Final report. London: RSA. Sedlack, R.G. & Stanley, J. 1992. Social research theory and methods. Boston: Allyn and Bacon. Skyrme, D. 1999. Measuring the value of knowledge – metrics for the knowledge-based business. London: Business Intelligence Ltd. Skyrme, D.J. 2000. Knowledge networking – creating the collaborative enterprise. Oxford: Butterworth-Heinemann. Sveiby, K. 1997. The new organizational wealth – managing and measuring knowledge-based assets. San Francisco: Berrett-Koehler Publishers. Van de Geer, J.P. 1993. Multivariate analysis of categorical data: Theory. Newbury Park, CA: SAGE Publications. Zack, M.H. 1999. Knowledge and strategy. Boston, MA: Butterworth - Heinemann.