Analysis of utility and use of a web-based tool for digital signal processing teaching by means of a technological acceptance model

Analysis of utility and use of a web-based tool for digital signal processing teaching by means of a technological acceptance model S.L. Toral * , F. Barrero, M.R. Martı ´nez-Torres Department of Electronic Engineering, University of Seville, Avda. Camino de los, Descubrimientos s/n, Seville 41092, Spain Received 15 December 2005; accepted 15 December 2005 Abstract This paper presents an exploratory study about the development of a structural and measurement model for the technological acceptance (TAM) of a web-based educational tool. The aim consists of measuring not only the use of this tool, but also the external variables with a significant influence in its use for planning future improvements. The tool, designed with Shockwaveeand Macromedia Directore, is a web-based educational environment that has been applied as a teaching methodology in an undergraduate course involved with modern microprocessors architectures and applications. The proposed methodology may be extensible to similar tools and tries to fill the lack of scientific studies in the validation and acceptance of computer-based educational tools. Ó2005 Elsevier Ltd. All rights reserved. Keywords: Information systems theory; Technological acceptance model; Distributed learning environments; Interactive learning environments; Multimedia/hypermedia systems 0360-1315/$ - see front matter Ó2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2005.12.003 * Corresponding author. Tel.: +34 954 48 12 93; fax: +34 954 48 73 73. E-mail address: [email protected] (S.L. Toral). www.elsevier.com/locate/compedu Computers & Education 49 (2007) 957–975 1. Introduction In recent years there have been important changes in engineering education, especially in electrical and computer engineering (ECE) education, both in terms of content (curriculum) and what it is taught or how it is taught (delivery of material) (Carley, Khosla, & Unetich, 2000; Felder, Felder, & Dietz, 1998; Milliken & Barnes, 2002; Roppel, 2000; Taylor, Heer, & Fiez, 2003; Wilson & Jennings, 2000). On the one hand, emphasis in the electrical engineering field has shifted significantly to the design of digital systems. On the other hand, technology that is around everything we do, has taken a place in the classroom. Since the early days of computers, learning through computer-based environments (hypermedia tools, web-based educational support, simulation environments, etc.) has dramatically increased (Aedo, Dı ´az, Ferna ´ndez, Martı ´n, & Berlanga, 2000; Almeida, Piazzalunga, Ribeiro, Casemiro, & Moreno, 2003; Bagui, 1998; Christian & Belloni, 2001; Conole, Dyke, Oliver, & Seale, 2004; Metzger, Flanagina, & Zwarun, 2003; Pahl, 2003). The pedagogical value of computer and web-based educational tools has been demonstrated in a lot of situations and educational experiences. But, although many enhancements have been developed in this area over the years, not much attention has been paid to scientific analysis for validating the tools in order to improve the teaching process; once a computer-based educational tool has been designed, it is never validated or, at most, poorly validated. However, two kinds of approaches to the task of e-learning or web-based educational tools evaluation can be distinguished. The first one is based on a simple questionnaire followed by a basic statistical treatment. The majority of the provided assessment data in scientific studies correspond to this scheme. Sometimes, just cognitive outcomes are considered (Fuller & Moreno, 2004). Other times, several dimensions about user perceptions are included. In Abdel-Qader, Bazuin, Mousavinezhad, and Patrick (2003), overall satisfaction and quality of materials are the unique variables taken into account. More elaborated questionnaires considering new dimensions can be found in Chevalier, Craddock, Riley, and Trunk (2000), Avouris, Tselios, and Tatakis (2001) and Hurley and Lee (2005). But in all the cases, the data treatment is reduced to show frequency of user answers. The second kind of approaches consists of a more complex questionnaire followed by a more complex statistical study, usually a multivariate statistical analysis. There is a big difference respect to the previous approach: the focus of attention is not only the use, satisfaction or efficiency of the tool, but the variables with an incidence over this final result and how these variables are interrelated among them. Some attempts in this field are worth mentioning. In Cappel and Hayen (2004), a questionnaire is used to test the satisfaction, perceived effectiveness and quality of an e-learning tool. But this questionnaire includes also questions related to some specific learning dimensions, such as clarity of information, easy of use, usefulness, etc. Several dimensions are also used in Toral, Barrero, Martı ´nez-Torres, Gallardo, and Lillo (2005), although in this case a correlation analysis is performed to extract some conclusions about relationships among dimensions. Some other studies are based on a previous model taken from information systems theory: the technological acceptance model (TAM). Basically, TAM models how users use a technology. In Selim (2003), Ngai, Poon, and Chan (2007) Ong, Lai, and Wang (2004) TAM is applied to a web-based and to an asynchronous e-learning tool. In the first case, the original model has been used while in the next two cases an external dimension has been added. 958 S.L. Toral et al. / Computers & Education 49 (2007) 957–975 Following this trend, this paper proposes the application of the technological acceptance model (TAM) to validate the application of a web-based educational tool as a teaching methodology and to plan future improvements. The tool has been used in an advanced microprocessor technology course, included in the telecommunication engineering curriculum at the University of Seville (Spain), since academic year 2002–2003. As a difference to the previous studies based on TAM, this study considers the most frequently mentioned external variables in the literature as external dimensions with an incidence over TAM variables. A methodology to achieve the complete model to be applied is also detailed. The paper is organized as follows. First, the web-based educational tool and the course are introduced. Then, the methodology used to validate the utility and use of the educational tool is described. The first step of this methodology consists of validating a questionnaire using Cronbach’s alpha. Through this index, the main constructs or dimensions related to the TAM model are validated and a set of indicators are obtained. Due to the large number of dimensions and indicators, a principal component analysis (PCA) technique is then applied to reduce the number of variables. The principal component of each dimension is used to obtain both a small number of indicators and to measure all the correlations among the different detected dimensions. Using the results of PCA analysis as a starting point, a structural equation model technique (PLS, partial least square) is then applied to obtain the final structural and measurement TAM model. The correlations of PCA analysis are used as the hypothesis to be validated and the weigh of indicators in the first principal component is used to obtain the minimum number of indicators to be used in PLS analysis.The results allow both measuring the use of the tool and determining the external variables with a significant influence in this use. Future improvements and conclusions will be pointed out in the final section. 2. Web-based educational tool to improve DSP teaching for undergraduate Telecommunication engineering studies at the University of Seville, was first organized in 1991. An intermediate undergraduate digital electronic subject-based on the design of advanced embedded digital systems and DSPs has been taught since then. The lack of enough lab work in the original organization of studies motivated a change in 1998, searching for an increment in the practical work. This fact, together with the necessity of improving the teaching methodology in classrooms with more than 300 students per year, was the starting point to develop new teaching methods using computer-based educational tools. In this approach, it has also been taken into account the new European Area for Higher Education and ECTS credits, which will shortly modify the teaching and learning processes (Communique ´of the Conference of Ministers responsible for Higher Education, 2003; Joint declaration of the European Ministers of Education, 1999). The course has been used to teach advanced microprocessors architecture connecting hardware concepts with software and, even, theoretical knowledge related to digital signal processing. We have chosen a DSP, Texas Instruments C3x family, as an example of microprocessor architecture to fully accomplish the objectives of the course; the internal structure of C3x family is used to explain complex embedded systems and its utility in telecommunication area. The TMS320C3X family is a 32 bits floating point general-purpose DSP from Texas Instruments (TIe,http:// www.ti.com). The family architecture features present a central processing module that appears S.L. Toral et al. / Computers & Education 49 (2007) 957–975 959 in all family members and it has been designed to execute commonly-used digital signal processing benchmarks with a minimum time load for single-multiplier architecture. These DSPs optimize speed implementing functions in hardware whereas other processors implement them through software or microcodes, performing parallel multiply and ALU operations on integer or floating-point data in a single cycle. Each processor also possesses a general-purpose register file, a program cache, dedicated ARAUs, internal dual-access memories, one DMA channel supporting concurrent I/O, and a short machine-cycle time. General-purpose applications are greatly enhanced by the large addresses space, multiprocessor interface, internally and externally generated wait states, two external interface ports, two timers, serial ports, and multiple interrupt structure. The analyzed educational tool consists of a web-site that has been designed (http:// www.gte.us.es/~fbarrero/CSED/) to promote an asynchronous learning forum, a group work environment for collaborative learning and a multimedia learning tool. This tool has been developed taking into account traditional elements of the TAM models: usefulness and easy of use. We have designed this site to be used in different teaching scenarios (lab exercises, theoretical lessons, self-learning, etc.). Students can access teaching material using this tool, to chat and discuss about the concepts of the course and to download data files containing reference material and class notes in pdf format. Other capabilities, based on data bases programmed using PHP, Perl Hypertext Preprocessor, are provided (see Figs. 1–4); student auto-evaluation tool, doubt, notice and mark boards, quality inquiry to provide asynchronous feedback of the teaching process, chat service. Maintenance of the web site is easily done with administration protected pages programmed using PHP. The multimedia content of the web-site has been design using the MacromediaeDirector software (Figs. 5–10), and it can be used in traditional classrooms and for asynchronous and noneyewitness active self-learning to engage students in active reinforcement of the concepts acquired in class. Notice that Director allows the development of dynamic web pages due to Shockwave Fig. 1. Learning forum welcome screen. 960 S.L. Toral et al. / Computers & Education 49 (2007) 957–975 compression techniques and that a Shockwave site needs a browser with a Shockwave Player plug-in (this plug-in is self-detected and it is free-downloadable in case of necessity). The multimedia content is a tight coupling of text, tutorials, illustrations, video segments and animations that integrates theoretic concepts related to TMS320C3X internal architecture and practical experiences to guarantee a ‘‘learning by doing’’ philosophy. In fact, two learning units have been designed, one devoted to the theoretical teaching and the other one related to practical exercises and lab work carried out during the academic year. After a small visual introduction, each unit can be accessed. Fig. 2. Teaching material access (I): download and on-line access. Fig. 3. Teaching material access (II): on-line access. S.L. Toral et al. / Computers & Education 49 (2007) 957–975 961 On the one hand, the theoretical unit consists of four sections, plus a generic embedded systems introduction. These chapters consist of: An introduction to the DSP devices concepts and their main applications. Basic hardware concepts of TMS320C3X devices: CPU and internal bus operations, memory organization and external bus timing access. Fig. 4. Doubt board. Fig. 5. Embedded systems: the beginnings. 962 S.L. Toral et al. / Computers & Education 49 (2007) 957–975 Basic software concepts of TMS320C3X devices: data format, addressing modes and basic assembler instructions. Internal peripheral: timers, serial interfaces and DMA. On the other hand, the practical unit is based on nine chapters that include the necessary didactic material for the implementation of four lab exercises using a DSP starter kit (DSK30 platform). The content of this unit is organized in the following way: Fig. 6. TMS320C3X internal architecture description. Fig. 7. TMS320C3X status register. S.L. Toral et al. / Computers & Education 49 (2007) 957–975 963 Different DSP applications are presented; implementation of audio signal processing techniques, digital wave generators and FIR or IIR filters. Some tutorials have been included; assembler instructions and addressing modes, description of program architecture, DSK30 functionality and characteristics. Moreover, two databases have been included to allow easy access to CPU instruction and internal DSP registers description, and a student auto-evaluation tool. Fig. 8. Register addressing mode. Fig. 9. Lab exercise based on audio signal processing. 964 S.L. Toral et al. / Computers & Education 49 (2007) 957–975 3. Questionnaire validation To validate the tool previously described, a questionnaire has been prepared to achieve the following aims: Checking the use of the tool. Identifying the external variables with a sensible influence in the use of the tool. The questionnaire, distributed among 142 students of the course, consists of 62 questions using 1–7 Likert type scale (1, strongly disagree; 7, strongly agree). Table 1 summarizes the technical chart of the study carried out. The questionnaire (detailed in Table 2) has groups of questions to measure several dimensions frequently reported in the literature: learning goal orientation, application specific self-efficacy and enjoyment (Agarwal & Karahanna, 2000; Yi & Hwang, 2003), focused immersion, curiosity, playfulness and voluntariness (Agarwal & Karahanna, Fig. 10. One of the included databases: instruction database. Table 1 Technical chart of the study Universe Students with personal experience with the tool Information gathering method Personal survey Sample unit Students who has attended the DSP course Population census 155 Sample size 142 Sample error 3.15% Reliability level 99%, Z= 2.58, p=q= 0.5 Sample procedure The survey was directed to the totally of the student attending the DSP course Date of the field study The survey was carried out in May, 2004 S.L. Toral et al. / Computers & Education 49 (2007) 957–975 965 Table 5 PLS results Indicator Load Consistency Discriminant validity ASSE E C P EOU U UI Us ASSE ASSE6 0.7465 0.894 0.793 0.351 0.500 0.425 0.614 0.482 0.476 0.483 ASSE7 0.8248 ASSE8 0.8126 ASSE9 0.7841 ASSE10 0.7923 E E1 0.8884 0.907 0.351 0.875 0.516 0.421 0.432 0.490 0.463 0.397 E2 0.8752 E3 0.8606 C C1 0.7757 0.875 0.500 0.516 0.837 0.744 0.441 0.539 0.461 0.396 C2 0.9146 C3 0.8135 P P1 0.9473 0.965 0.425 0.421 0.744 0.949 0.347 0.406 0.289 0.295 P2 0.9622 P3 0.9381 EOU EOU1 0.7545 0.930 0.614 0.432 0.441 0.347 0.830 0.429 0.386 0.387 EOU2 0.8399 EOU3 0.8928 EOU4 0.8502 EOU5 0.8607 EOU6 0.7704 U U1 0.8571 0.956 0.482 0.490 0.539 0.406 0.429 0.885 0.524 0.384 U2 0.8949 U3 0.8970 U4 0.9111 U5 0.8853 U6 0.8662 UI UI1 0.9538 0.953 0.476 0.463 0.461 0.289 0.386 0.524 0.954 0.547 UI2 0.9534 Us Us1 0.9210 0.926 0.483 0.397 0.396 0.295 0.387 0.384 0.547 0.928 Us2 0.9358 Table 6 Path coefficients table (T-statistic) Us UI U EOU ASSE E C P Us 0.000 9.250 0.000 0.000 0.000 0.000 0.000 0.000 UI 0.000 0.000 5.292 2.592 0.000 0.000 0.000 0.000 U 0.000 0.000 0.000 3.127 0.000 0.000 6.474 0.000 EOU 0.000 0.000 0.000 0.000 6.165 3.204 0.000 0.000 ASSE 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 E 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 C 0.000 0.000 0.000 0.000 2.917 2.804 0.000 8.916 P 0.000 0.000 0.000 0.000 3.904 3.429 0.000 0.000 972 S.L. Toral et al. / Computers & Education 49 (2007) 957–975 5. Conclusion A web-based educational tool, designed with Shockwaveeand Macromedia Directorehas been presented as a teaching methodology in an undergraduate course involved with advanced microprocessors architectures and their applications. The tool has been designed taking into account the major variables of the TAM model. In order to measure its use and to improve the educational tool, a questionnaire has been developed and validated with the Cronbach’s alpha estimator of reliability. Then, principal component analysis is incorporated as a priori theoretical and measurement model to obtain the hypotheses and the indicators to measure each of the identified constructs or dimensions. Finally, PLS technique has been applied to analyze the measurement model and structural model concurrently in order to validate the previous set of hypotheses and indicators. Two types of conclusions have been obtained. First, the final use of the tool; more than 80% of the students have navigated through the 44% of the tool contents. Second, the external variables with a significant influence in this use: application specific self-efficacy (easy navigation through the web-site is guarantied due to the connection between classroom and web-site contents), enjoyment, playfulness and curiosity. These variables can be improved promoting interactivity and cooperative learning. Notice that the proposed study could be extended to any web-based educational tool, because the measurement of the acceptance and the identification of the external variables is the only way of knowing if an educational tool is properly working in a specific teaching methodology, playing the role it was designed for. References Abdel-Qader, I. M., Bazuin, B. J., Mousavinezhad, H. S., & Patrick, J. K. (2003). Real-time digital signal processing in the undergraduate curriculum. IEEE Transactions on Education, 46(1), 95–101. Aedo, P., Dı ´az, P., Ferna ´ndez, C., Martı ´n, G. M., & Berlanga, A. (2000). Assessing the utility of an interactive electronic book for learning the Pascal language. IEEE Transactions on Education, 43(3), 403–413. Agarwal, R., & Karahanna, E. (2000). Time flies when you’re having fun cognitive absorption and beliefs about information technology usage. MIS Quarterly, 24(4), 665–694. Almeida, S. F., Piazzalunga, R., Ribeiro, V. G., Casemiro, M. B., & Moreno, R. (2003). Combining interactivity and improved layout while creating educational software for the Web. Computers & Education, 40, 271–284. Avouris, N. M., Tselios, N., & Tatakis, E. C. (2001). Development and evaluation of a computer-based laboratory teaching tool. Computer Applications in Engineering Education, 9(1), 8–19. Bagui, S. (1998). Reasons for increased learning using multimedia. Journal of Educational Multimedia and Hypermedia, 7, 3–18. Cappel, J. J., & Hayen, R. L. (2004). Evaluating e-learning: a case study. Journal of Computer Information Systems, 44(4), 49–56. Carley, L., Khosla, P., & Unetich, R. (2000). Teaching ‘Introduction to Electrical and Computer Engineering’ in context. Proceedings of the IEEE, 88, 8–22. Chevalier, L. R., Craddock, J. N., Riley, P. C., & Trunk, B. J. (2000). Interactive multimedia labware for strength of materials laboratory. Computer Applications in Engineering Education, 8(1), 31–37. Chin, W. W. (1998). The partial least squares approach for structural equation modelling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295–336). Mahwah, NJ: Erlbaum. Chin, W. W. (2003). PLS-Graph (Version 3.00, Build 1058) [Computer software]. University of Houston. Christian, W., & Belloni, M. (2001). Physlets: Teaching physics with interactive curricular material. Englewood Cliffs, NJ: Prentice-Hall. S.L. Toral et al. / Computers & Education 49 (2007) 957–975 973 Communique ´of the Conference of Ministers responsible for Higher Education (2003). Realising the European Higher Education Area. Berlin. Conole, G., Dyke, M., Oliver, M., & Seale, J. (2004). Mapping pedagogy and tools for effective learning design. Computers & Education, 43, 17–33. Davis, F. D., & Venkatesh, V. (2004). Toward preprototype user acceptance testing of new information systems: implications for software project management. IEEE Transactions on Engineering Management, 51(1), 31–46. Felder, R. M., Felder, G. N., & Dietz, J. (1998). A longitudinal study of engineering students performance and retention V. Comparisons with traditionally-taught students. Journal of Engineering Education, 98(4), 469–480. Fornell, C., & Larcker, D. F. (1981). Structural equation models with unobservable variables and measurement errors. Journal of Marketing Research, 18(1), 39–50. Fornell, C. (1982). A second generation of multivariate analysis, methods: Volume 1. New York, NY: Praeger Publishers. Fuller, D. A., & Moreno, A. F. (2004). Experimenting with a computer-mediated collaborative interaction model to support engineering courses. Computer Applications in Engineering Education, 12(3), 175–188. Hubona, G. S., & Geitz, S. (1997). External variables, beliefs, attitudes and information technology usage behavior. Proceedings of the thirtieth Hawaii international conference on system sciences, 3, 21–28. Hurley, W. G., & Lee, C. K. (2005). Development, implementation, and assessment of a web-based power electronics laboratory. IEEE Transactions on Education, 48(4), 567–573. Joint declaration of the European Ministers of Education (1999). The European Higher Education Area. Bologna. Laitenberger, O., & Dreyer, H.M. (1998). Evaluating the usefulness and the easy of use of a web-based inspection data collection tool. In Proceedings 5th international software metrics symposium (pp. 122–132). Lee, Y., Kozar, K. A., & Larsen, K. R. T. (2003). The technology acceptance model: past, present, and future. Communications of the Association for Information Systems, 12, 752–780. Metzger, M. J., Flanagina, A. J., & Zwarun, L. (2003). College student web use, perceptions of information credibility, and verification behaviour. Computers & Education, 41, 271–290. Milliken, J., & Barnes, L. P. (2002). Teaching and technology in higher education: student perceptions and personal reflections. Computers & Education, 39, 223–235. Ngai, E. W. T., Poon, J. K. L., & Chan, Y. H. C. (2007). Empirical examination of the adoption of WebCT using TAM. Computers & Education, 48(2), 250–267. Ong, Chorng-Shyong, Lai, Jung-Yu, & Wang, Yi-Shun (2004). Factors affecting engineers’ acceptance of asynchronous e-learning systems in high-tech companies. Information and Management, 41(6), 795–804. Pahl, C. (2003). Managing evolution and change in web-based teaching and learning environments. Computers & Education, 40, 99–114. Roppel, T. (2000). An interdisciplinary laboratory sequence in electrical and computer engineering: curriculum design and assessment results. IEEE Transaction on Education, 43, 143–152. Selim, H. M. (2003). An empirical investigation of student acceptance of course websites. Computers & Education, 40, 343–360. Taylor, R. L., Heer, D., & Fiez, T. S. (2003). Using an integrated platform for learning to reinvent engineering education. IEEE Transaction on Education, 46(4), 409–419. Toral, S. L., Barrero, F., Martı ´nez-Torres, M. R., Gallardo, S., & Lillo, J. (2005). Implementation of a web-based educational tool for digital signal processing teaching using the technological acceptance model. IEEE Transactions on Education, 48(4), 632–641. Wilson, J., & Jennings, W. (2000). Studio courses: How information technology is changing the way we teach, on campus and off. Proceedings of the IEEE, 88, 72–80. Yi, M. Y., & Hwang, Y. (2003). Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. International Journal of Human–Computer Studies, 59, 431–449. Sergio L. Toral Marı ´nwas born in Rabat, Morocco, in 1972. He received the electrical engineering and Ph.D. degrees at University of Seville in 1995 and 1999, respectively. He is currently a full Professor at the Electronic Engineering Department of the University of Seville. His research interests include microprocessor and DSP devices systems, e-learning tools and their evaluation, stochastic processing and their industrial applications. 974 S.L. Toral et al. / Computers & Education 49 (2007) 957–975 Federico J. Barrero Garcı ´awas born in Seville, Spain, in 1967. He received the electrical engineering and Ph.D. degrees at University of Seville in 1992 and 1998, respectively. In 1992 he joined the Electronic Engineering Department at the University of Seville, where he presently maintains a full Professor position. His current interests include microprocessor and DSP devices systems, e-learning and multimedia educational tools, fuzzy logic-based system, and control of Electrical Drives and Power Electronic. Rocı ´o Martı ´nez Torres was born in Madrid, Spain, in 1973. She received the degree in Business Administration in 1996 and the Ph. D. degree from the University of Seville, Spain, in 2003. She is currently a full Professor at Business Administration and Marketing Department of the University of Seville. Her research interests include intellectual capital and knowledge management, and acceptance of e-learning tools. S.L. Toral et al. / Computers & Education 49 (2007) 957–975 975