Improving learning performance in laboratory instruction by means of SMS messaging

Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 Innovations in Education and Teaching International Vol. 44, No. 4, November 2007, pp. 409–422 ISSN 1470–3297 (print)/ISSN 1470–3300 (online)/07/040409–14 © 2007 Taylor & Francis DOI: 10.1080/14703290701602854 Improving learning performance in laboratory instruction by means of SMS messaging M. R. Martínez-Torres, S. L. Toral * , F. Barrero and S. Gallardo University of Seville, Spain Taylor and Francis LtdRIIE_A_260137.sgm 10.1080/14703290701602854Innovations in Education and Teaching International1470-3297 (print)/1470-3300 (online)Original Article2007Taylor & Francis444000000November 2007Dr SToral[email protected] The study described in this paper outlines an attempt to explore those factors that contribute to learning performance improvement in laboratory instruction. As a case study, the educational methodology involved in a basic microcontroller course was analyzed. Traditional lab sessions based on the control of peripherals with low interactivity have been replaced with new sessions based on mobile technology and the Short Message Service (SMS). This allows the development of greater interactivity and the provision of more motivating features. Using the key tenets of the three basic learning theories (behaviorist, cognitivist and constructivist) and the notion of interactivity as causal factors, the study described in this paper presents a performance learning model based on the theory of reasoned action. This learning model identifies the variables with a significant influence on the learning performance, allowing a statistical analysis to quantify their influence. The results obtained demonstrate the important roles of interactivity and motivating features in a laboratory instruction from both a qualitative and a quantitative point of view. Introduction Considerable research has been performed in social psychology about a person’s attitudes and how they may affect behavior. Specifically, the Theory of Reasoned Action (Ajzen & Fishbein, 1980) is a widely accepted model that has been used in predicting and explaining behavior across a wide variety of domains, e.g. in marketing management to understand consumer behavior or in information systems to determine user acceptance in computer technology (Ajzen & Fishbein, 1980; Sheppard et al ., 1988; Davis et al . 1989). This theory assumes that individuals behave rationally, and postulates that behavioral intention (and subsequently the final behavior) is a function of attitude (toward the behavior). Triandis (1977, 1980) brings up the role of facilitating conditions. He makes the case that the likelihood of the occurrence of behavior is affected by the presence or absence of facilitating conditions. In this study, these facilitating conditions * Corresponding author. E. S. Ingenieros, Dpto. Ingeniería Electrónica, Avda. Camino de los Descubrimientos s/n, 41092, Sevilla, Spain. Email: [email protected] Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 410 M. R. Martínez-Torres et al. are the various features that foster a favourable student attitude in laboratory instruction, including interactivity and learning styles. Behavioral intentions also contribute to a better learning performance putting forth a student’s best efforts to learn the material. According to Figure 1, it is necessary to nurture a suitable attitude toward the new learning environment if the new environment is to facilitate improved performance. In this sense, two causal factors, the interactivity of laboratory instruction systems and the learning styles, can fulfil this role in certain contexts (Jackson et al ., 1997). The influence of attitude on learning performance is rooted in the extensive work of Herzberg who establishes that employees’ job satisfaction (an expression of positive attitude) leads to superior employee performance, that is, satisfaction (Herzberg et al ., 1959; Schwab & Cummings, 1970). Figure 1. The Theory of Reasoned Action (TRA) for learning performance in a laboratory instruction In this paper, the model of Figure 1 is applied to derive a methodology for the evaluation of a technological innovation. The paper is structured as follows. First, a review of the causal factors and variables with a significant influence on the learning performances is detailed. The case study is then presented: an improvement in a lab instruction based on microcontrollers and their applications. Next, the traditional laboratory instruction is compared with the improved version, taking into account the variables obtained from the conceptual research model. A statistical analysis is then performed in order to identify the variables that have a significant influence on the expected behavior. Finally, we discuss the implications of our findings, outline some directions for future research, and present our concluding remarks. Causal factors and variables with a significant influence on learning performance Users who are given a higher degree of interactivity, making the best use of the three learning theories, will be expected to have a better attitude towards the learning experience and the material being learned. Furthermore, students’ favourable attitude will motivate them to excel in their learning performance. The external and intermediate variables which have a significant influence on learning performance are interactivity, learning styles and user attitude. Interactivity There have been many different descriptions and interpretations of interactivity as a concept. Traditionally, interactivity is generally assumed to be a face-to-face communication and learning in a classroom-type environment (Kettanurak et al ., 2001). Within the context of communication technologies, interactivity can be defined as the degree to which technology supports/ enables interaction resembling human conversation, and interpreted as the degree to which a system exhibits ‘first-personness’ (Laurel, 1986). The three interactive aspects of firstpersonness are: CAUSAL FACTORS ATTITUDE PERFORMANCE RELATED BEHAVIORS GAIN IN LEARNING Figure 1. The Theory of Reasoned Action (TRA) for learning performance in a laboratory instruction Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 Improving learning performance in laboratory instruction 411 ● The interactive frequency or the measure of how often a user input is enabled. ● The range of choices available to a user at a given moment during the interaction; for instance, binary choice (e.g. yes/no) has a very narrow range. ● The interactive significance or the measure of the impact of a user’s choices and actions upon the outcome/experience of an interactive session. It has been posited that an instructional system offering greater frequency of interaction and wider range of choices will more actively engage the learner during the session. Even small opportunities for choice can increase students’ sense of self-determination, above all, if there is a clear link between the performed activities and the obtained results. For instance, in the case study presented, it is not the same to control some digital inputs or outputs connected to some switches or LEDs than controlling the same inputs and outputs to manage the communication with a cell phone and sending or receiving an SMS message. Learning styles There are three primary learning styles (Jonassen, 1991), each with different emphasis on the design of instructional systems: ● Behaviorist learning theory is shaped by selective reinforcement in the form of motivational and/or correctional feedback to increase the likelihood of target behaviors. ● Cognitivist learning theory is concerned with what learners know and how they acquire it, in contrast to behaviorists’ concern of what learners do. The primary goal of instruction from a cognitive perspective is to transfer knowledge to learners in the most efficient and effective manner. ● Constructivist learning theory investigates the spontaneous and self-regulated learning found in nature. It is an untutored concept formation. According to constructivists, learning is a process where the learner constructs an internal representation of knowledge via personal interpretations of experiences. User’s attitude The user attitude links interactivity and learning styles with the user learning performances (Haseman et al ., 2002). There are six different dimensions of user attitude: attitude toward the knowledge content, format in which it is delivered, extent of user control afforded by the system/task, feedback offered by the program, user-friendliness or ease-of-use of the system/ task, and the degree of enthusiasm/motivation generated by interacting with the task (Haseman et al ., 2002). The first five of these are related with the experimental task itself. The last one, motivation, is based on the user’s expectation. The most comprehensive and widely accepted explanation of people motivation is based on expectancy theory (Vroom, 1964), improved by Porter and Lawler (1968). It states that an individual tends to act in a certain way based on the expectation that the act will be followed by a given outcome, and on the attractiveness of that outcome to the individual. An individual will exert a high level of effort if there is a strong relationship between effort and performance, performance and rewards, and rewards and satisfaction of personal goals. Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 412 M. R. Martínez-Torres et al. Using these variables, the full conceptual research model is shown in Figure 2. The causal factors, degree of interactivity and learning styles are applied over the target system, and they should lead to a positive attitude toward the system and the learning experience. Finally, the positive attitude facilitated by the interactivity combined with the system offering the opportunity for the subjects to further explore, discover and assimilate the nuances in the learning environment should enable them to perform better. This conceptual model will be applied to the case study described next. Figure 2. Conceptual research model Case study In recent years, engineering education has dramatically changed, especially in electrical and computer engineering education, both in terms of curriculum content and delivery of material (Felder et al ., 1998; Carley et al ., 2000; Roppel, 2000; Wilson & Jennings, 2000; Taylor et al ., 2003). Emphasis in the electrical engineering field has shifted significantly from emphasizing ‘continuous-time linear time-invariant systems’ to ‘discrete-time linear time-invariant systems’. The use of programmable digital systems and microprocessors is becoming more widespread because of this emphasis on discrete-time systems. To incorporate microprocessor, microcontroller and digital signal processor courses is common at the sophomore higher educational levels. Multiple computer-based educational tools have been recently developed for improving the teaching methodology on microprocessor courses (Hughes, 2000; Martínez-Torres et al. , 2005). Despite the tremendous increase in the use of computer tools, especially simulation environments, the pedagogical values of hardware-based laboratory instruction are still essential, but sometimes forgotten. One of the purposes of this work has been to cover the gap between technological learning improvements and pedagogic theory. Using some statistical tools, a qualitative and quantitative analysis about the impact of a learning innovation will be performed. The final goal is to offer a methodology for evaluating the impact of an innovation to obtain conclusions based on a strong theoretical support and not only based on instructors’ perceptions. Microcontroller course and curriculum overview Our microcontroller course is a sophomore-level embedded systems subject based on the popular eight-bit Motorola 68HC11 microcontroller. The 68HC11 microcontroller is used at numerous universities across the world, and it is supported with many textbooks and educational packages. Target system: degree of interactivity Users' attitude User learning performance Learning styles Figure 2. Conceptual research model Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 Improving learning performance in laboratory instruction 413 Since 1998, the applied teaching methodology composes of theoretical concepts and practical experiences at the same ratio, to guarantee a ‘hands-on learning’ philosophy. The course is designed to provide the students with 70 hours of instruction in one semester, divided into two hours of lecture and two-hour lab per week. Instruction links theoretical content and laboratory experiments using a 68HC11 development platform. The learning objectives for the course included: ● To understand the hardware features, specifications and technologies of the microprocessor system components: CPUs and peripherals. ● To understand the software features of Motorola 68HC11 microcontroller. ● To be aware of the current use of the development platform. The students develop hardware and software required to apply microprocessors to real-world problems. Originally, the practical content of the course was based on simulations and the realization of software applications, which run in the MC68HC11 development board and control basic external input/output (I/O) devices and peripherals, like external LCD displays or keyboards. Through the years, it has been detected that this practical content is far from reality, and consequently students do not feel motivated enough to perform better. Recently, these peripherals have been replaced with a more interactive and motivating external device, a Philips Savvy® GSM (Global System for Mobile) phone that is electrically connected and controlled by the 68HC11. The novel interface uses the microcontroller general purpose I/O pins to communicate with the cellular phone using its keyboard, emulating phone key pulsation. This new approach represents a sensible change with respect to the traditional practical content. Notice that the mobile technology, including the well-known SMS, has rapidly emerged, including the educational environments. SMS can be defined as the ability to send and receive text messages (up to 160 alphanumeric characters combination) to and from mobile telephones. Since SMS was created as part of the GSM Phase 1 standard—the first short message is believed to have been sent in December 1992 from a personal computer to a mobile phone on the Vodafone GSM network in the UK—there is no doubt about its success: the European market can be resumed in more than a billion messages per month. Notice that younger people, who represent the majority of the educational tools end-users, are the principal customers of this technology. As a consequence, SMS has become an interesting educational technology with the fundamental ability of improving a student’s attention and motivation when using it. A lab prototype, based on a MC68HC11A8 Motorola® microcontroller and a Philips Savvy® GSM phone, has been designed to train students in microcontroller handling. Students must manage and program the microcontroller to communicate with the cellular phone. They have to emulate phone key pulsation to send a required SMS message to the professor or their own mobile phone using general purpose I/O pins. Instead of programming poorly interactive peripherals or external devices, students handle a modern and nearby electronic system, which can substantially improve their motivation and interest in the job. Figure 3 shows a picture of the SMS-based laboratory environment. The cellular terminal keyboard is a matrix of rows and columns where short-circuiting a row and a column implies that a key is pressed. Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 414 M. R. Martínez-Torres et al. Figure 3. SMS-based laboratory environment During the course, students must control the cell phone and be able to transmit and receive SMS messages. Figure 4 illustrates how the practical content of the course, based in the described lab prototype, is organized. Figure 4. Organization of lab sessions As is shown in Figure 4, lab bench is composed of three different interfaces. The first one, the human interface, represents the interaction teacher–student and student–student. Two groups of students work over the bench during the same lab session. One group organizes the interaction with the sensors and actuators of the system (hardware configuration, measurement components, value of the maximum input range, etc.). This group implements a measurement stage based on several sensors, such as a position/angular sensor, a movement sensor, and a proximity sensor. The other group of students program the microcontroller to manage a Figure 3. SMS-based laboratory environment Figure 4. Organization of lab sessions Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 Improving learning performance in laboratory instruction 415 required flowchart. One example of lab work is the implementation of a car alarm. In this case, two external sensors are used: a proximity sensor and an angular sensor. When the proximity sensor inside a vehicle is activated, the alarm will advise the user. The angular sensor will detect events like a wheel being taken off. A close collaboration between both groups must be reinforced because of the heavy dependence of programming and interfacing activities—thereby, promoting close collaborative learning. The system monitoring is implemented using traditional outputs like sirens and horns, and the interface with the cell phone, offering the possibility of using SMS messages to advise users about different events. The teacher, responsible for monitoring all the activities and human groups, is defined like the final user of each lab task. When the traditional interface with sirens and horns works properly, students must configure the cell interface to send a SMS message to the teacher. These SMS messages are processed in order to identify the event and allow the evaluation of the lab work with minimal resources. The proposed lab organization strengthens the collaborative work and enables simultaneous working groups (groups of students working on different benches, programming different applications and configuring different hardware interfaces, such as car alarms, domestic systems, aeroplane navigation systems, automotive simulators, and so on), with a low cost of time and low teacher dependence. Results In order to validate the effectiveness of the novel laboratory environment and to evaluate the observed learning performance, a comparison between the traditional and enhanced laboratory instruction has been done attending to the performance learning models. First, a qualitative comparison is done in order to identify the dimensions that potentially could be improved using the proposed approach. Next, a quantitative analysis is performed to evaluate the magnitude of improvements. In accordance with the model of Figure 2, the causal relationship between interactivity and attitude is mediated by the learning styles. All the learning theories previously described can be observed in the laboratory instruction. Several interviews were held with students in order to analyze and identify the key features associated with these learning styles. Mapping these key features with the notion of interactivity, the obtained results for an interactive lab instruction system can be defined in terms of the following broad guidelines. ● Reinforcement through (thrust of Behaviorist theory ): ● Feedback: to strengthen users’ learning via verification. Errors (made by users) to be identified (by the system), users to be informed of reasons for their occurrence, and corrective actions may be suggested. ● Learner control: to offer users the capability to control the instructional sequence, pace (time spent on learning), review/repeat instruction and select the content covered. ● Learning progress assessment: to help learners with self-monitoring by assessing their overall performance. ● Efficient and effective transfer of knowledge through (thrust of Cognitivist theory ): ● Questions and clarifications: to encourage and enable users to ask questions and seek clarifications on specific subjects, topics or words (e.g. hot spots). Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 416 M. R. Martínez-Torres et al. ● Real examples and practice: to enable users to practice as often as necessary. Also, build in realistic examples to help learners to develop necessary concepts and to be connected with the real world. ● Flexibility and discovery-based learning through (thrust of Constructivist theory ): ● Promotion of one’s own initiative: to permit developing learners’ interests. ● The user attitude is related with the lab instruction itself, content, format, user control, feedback, and ease of use, but also with the degree of motivation generated by the interaction with the practical exercise. The previous guidelines suggest the dimensions to be taken into account in order to perform the qualitative analysis. Table 1 details this qualitative analysis about the improvement of the learning performance of the traditional lab exercises against the enhanced SMS-based lab instruction. The main improvements achieved using the SMS facility are focused on interactivity (row numbers 2 and 3), learning assessment (row number 4), real example and practice (row number 5), feedback (row number 10), and motivation (row number 12). Once the main improvements achieved using the SMS service in the lab instruction have been identified, it would be useful to quantify these benefits. The quantification is important to prove that a significant improvement in the learning performance has been obtained. A questionnaire (detailed in Table 2) was distributed to the 142 students of the course in order to identify the relative importance of the different dimensions. The questionnaire compiles the dimensions of Table 1. Students were asked to sort the 12 dimensions according to their improvement in learning performance in laboratory instruction, taking into account that the first one has the maximum importance and the last one (12th), the minimum importance. Table 3 shows the obtained mean value for each dimension. An analysis of variance (ANOVA) was conducted to test the hypothesis of equal means. The high value of the F statistic (41.13) suggests that the differences between the dimension means are highly significant. The results from Table 3 (the dimensions have been sorted according to their mean values) suggests that motivation, real examples and practices, content, and feedback are the most important dimensions. All of these have a significant influence in the learning performance in the laboratory instruction. Notice that the SMS-based lab instruction offers better learning performance than the traditional lab instruction in three of the four most significant dimensions (motivation, real examples and practices, and feedback). A cluster analysis was performed in order to group the detected dimensions into a small set of variables (Selim et al ., 1984) in order to facilitate the generalization of the results. The dendrogram that was obtained is shown in Figure 5. In this figure, the numbers along the horizontal axis represent the identified dimensions from Table 2, while the links between them are represented as upside down U-shaped lines. The height of an inverted U indicates the distance between the objects. Figure 5. Dendrogram of cluster analysis One way to determine the natural cluster divisions in a data set is to compare the height of each link in a cluster tree with the heights of neighboring links below it in the tree. If a link is approximately the same height as neighboring links, it indicates that there are similarities between the objects joined at this level of the hierarchy. These links are said to exhibit a high level of consistency. If the height of a link differs from neighboring links, it indicates that there Downloaded By: [Universidad de Sevilla] At: 10:32 2 November 2007 Improving learning performance in laboratory instruction 417 are dissimilarities between the objects at this level in the cluster tree. This link is said to be inconsistent with the links around it. In cluster analysis, inconsistent links can indicate the border of a natural division in a data set. The relative consistency of each link in a hierarchical cluster tree can be quantified and expressed as the inconsistency coefficient. This value compares the height of a link in a cluster hierarchy with the average height of neighboring links. Table 1. Comparative analysis of the proposed experimental tasks according to the conceptual research model Dimensions Traditional lab instruction SMS-based lab instruction Interactive frequency Determined by laboratory access Interactive range Supported by the results that can be shown in usual peripherals or devices (displays, matrix keyboards) Supported by SMS service, offering a broader range of possibilities and a full duplex communication Interactive significance The impact of the users’ choices is restricted to lab environment The impact of the users’ choices is extended outside the lab environment. A message can be sent to any cellular phone Behaviorist learning theory • Feedback • Learning control • Learning assessment Debugging tools and display provide a correctional feedback and learning control. Learning assessment can only be done using traditional methods, with physical presence of the lecturer Debugging tools and SMS also provide a correctional feedback and learning control. Improved motivational feedback is achieved using cellular phones. Learning assessment can be done using mobile possibilities: the experimental work may be done and assessed without the physical presence of the lecturer Cognitivist learning theory • Questions and clarifications • Real example and practice Tutorial action is supported by the lecturer. Examples and practice are limited to the traditional peripherals and devices Tutorial action is supported by the lecturer. SMS service provides a real example of modern technology Constructivist learning theory The learning process is determined by the proposed exercises. Notice that additional work may be addressed using the SMS service Content The student task is based on microcontroller programming Format The format in which knowledge is delivered is based on the microcontroller debugging tools User control The extent of control that the learners could exercise during the learning process is subject to the goals of the experimental work Feedback Supported by the lecturer Supported by the lecturer. A motivational feedback is also provided by the reception of a message Easy of use Guaranteed by the previous knowledge of students and the debugging tools Motivation The reward of the experimental task is low. It only consists of observing the peripheral state The reward has been notably increased. It consists of sending an SMS message to their cellular phone, to the lecture cellular phone, and detecting incoming messages