Self-assessment for autonomous language learners

Self-assessment for autonomous language learners David Gardner University of Hong Kong September 1999 Abstract This paper discusses why self-assessment is an important tool in the toolkit of autonomous language learners. It can be used both as a testing device leading to accreditation and as a device for personal self-monitoring. It provides the learner with immediate feedback to determine language proficiency and to reflect on learning strategies. There are great benefits to be derived from self-assessment but it is a technique that needs to be introduced carefully and accompanied by considerable awareness raising and support. This paper discusses how to maximise the benefits while minimising the pitfalls. It also describes the teacher's role in the process and the different levels of learner involvement from self-administering a pre-packaged test through to using self-produced tests. Key Words: Autonomy, Generic Assessments, Self-assessment, Self-monitoring, Selftesting, Teachers' Roles. 1. Introduction This paper discusses the use of self-assessment as a tool in the toolkit of autonomous language learners. There is some diversity in the way the terms ‘self-assessment’ and ‘autonomous learners’ are used in the literature, particularly the latter. It may be wise, therefore, to start by defining how these terms will be used in this paper. Assessments for autonomous learners may cover a Links & Letters 7, 2000 49-60 1. Introduction 2. Benefits and pitfalls 3. The role of teachers/counsellors in the process 4. Levels of learner involvement 5. Conclusion References Appendix: A generic self-assessment Table of Contents wide area of knowledge (for example, reading an authentic language text) or a small, tightly focused area (for example, questions at the end of a worksheet on a specific grammar point). Assessments may serve one or more of a number of purposes, such as confidence building, demonstrating learning gain, or motivation, and they may be constructed in a number of ways, for example, by the teacher, by the learner, collaboratively or as a portfolio (for a fuller discussion of assessments for autonomous learners, see Gardner and Miller, 1999). Assessments with any combination of the above criteria can be self-assessments because this term refers simply to the mode of administration, i.e., assessments which are self-administered. Self-assessments may be constructed by anyone, including, but not necessarily, the learner. There are good reasons for learners to self-assess and those reasons will be a focus of this paper. The term autonomous learners is more difficult to define. It has been widely used in the literature and, as Pemberton (1996: 2) points out in reviewing titles of books in the field of autonomy, “different terms are often used to refer to the same thing... [and]... the same term is often used to mean different things”. Generally, writers, including the writer of this paper, adhere more or less strictly to Holec's view of autonomy as “the ability to take charge of one's learning” (Holec, 1981: 3). It is important to note that learners' levels of autonomy are not fixed and are likely to fluctuate with time, contexts and learners' moods. In addition, it has been pointed out by a number of writers (for example, Little, 1996; Nunan, 1997; Gardner and Miller, 1999) that total autonomy is an ideal rarely reached. In this paper the term autonomous language learning is used to refer to the learning undertaken by autonomous learners, i.e., the process undertaken as a result of learners taking responsibility for their own learning. It is reasonable to assume that autonomous learners would benefit from feedback on achievements in their learning through engaging in some kind of assessment procedure. The individualised nature of autonomous learning makes large-scale, institutionalised assessments problematic although an autonomous learner may make the decision to include these as part of a personalised assessment regime. Self-assessment seems to accommodate itself much more easily to the diverse and flexible requirements of an autonomous learner. Indeed, it has been argued that self-assessment is an integral part of autonomous learning (Holec, 1981; Tudor, 1996; Thomson, 1996; Gardner and Miller, 1999) and that all learners engage in it (Holec, 1985) although not necessarily knowingly (Thomson, 1996). While the above assumption seems reasonable, it would not be reasonable to assume that autonomous learners understand the benefits of engaging in self-assessment nor that they know how to do it effectively. The purpose of this paper is to discuss the benefits and pitfalls of self-assessment; reflect on the importance of teachers/counsellors in the process; and consider the role of the learners. An example of a self-assessment is given to illustrate my points. 50 Links & Letters 7, 2000 David Gardner 2. Benefits and pitfalls The benefits of self-assessment can extend beyond learners to teachers and also to the institution (figure 1). Learners benefit the most and in quite diverse ways. Benefits to teachers and institutions are less diverse and the extent to which the benefits can be realised depends to a large extent on whether they are able to gain access to the results of the self-assessments. The ways in which self-assessment is beneficial are described in detail below. This will be followed by a discussion of the potential pitfalls. 2.1. The benefits of self-assessment If, as Holec (1981: 3) suggests, “autonomy is the ability to take charge of one's learning”, then self-assessment is a tool which supports those with that ability. Autonomous learners decide what to learn, when to learn and how to learn. Self-assessors decide what to assess, when to assess it and how to assess it. Autonomous learners take responsibility for their learning and this includes taking responsibility for monitoring their progress. Self-assessment provides an opportunity to self-tailor an assessment regime which can parallel a selftailored study regime. Dickinson (1987) argues that self-evaluation of performance is an important skill for all language learners but of particular importance to autonomous language learners. Thomson (1996) implemented a self-assessment project as a way of getting learners involved in self-directed learning. Autonomous learning is about individualisation of learning and selfassessment helps learners monitor their individualised progress. An important aspect of the monitoring process for learners is simply knowing how they are doing in their learning. They want to know if they are becoming more proficient as users of the target language. Oxford (1990: 162) includes “gauging either general language progress or progress in any of the four skills” as one Self-assessment for autonomous language learners Links & Letters 7, 2000 51 BENEFITS Individualisation Reflection Motivation Evaluation Monitoring Support Accreditation Justification Figure 1. The benefits of self-assessment. learner { teacher } institution } of her language learning strategies. Self-assessments help learners monitor their level of success in specific learning tasks. A series of self-assessments will contribute to monitoring progress towards specific learning objectives. They can also have a motivational effect. Success breeds confidence. Self-assessment does not always demonstrate success but where it does, even on a small scale, learners' motivation will be enhanced. Teachers also need to know how well learners are doing. They have a professional responsibility to help learners learn. As well as monitoring language proficiency, self-assessments provide learners with personalised feedback on the effectiveness of their learning strategies, specific learning methods and learning materials. Learners can use this feedback to evaluate their approach to language learning. This is part of learners' reflection about learning. In selecting, administering and considering the results of self-assessments learners must necessarily reflect on their goals, strategies and achievements. Self-assessments provide milestones in the ongoing process of reflection that all autonomous learners are engaged in. Evaluative feedback can also help teachers to enhance their support of learners. Through self-assessment, learners can identify specific areas where they need more support and can seek help from teachers or language counsellors. Information about support requirements can enable teachers/counsellors to focus on areas where guidance is needed at a time when it is needed most. Managed correctly, self-assessments can contribute to formal assessment requirements although there are concerns about reliability (discussed later). This is of benefit to those learners who want or need certification to demonstrate their learning gain. Accreditation can also be of benefit to institutions whose interests lie in demonstrating the learning achieved by their students. A further benefit to institutions is the evidence which self-assessments provide of the appropriate use of resources. This may contribute to justifying the use of funds, equipment and teachers' time as well as providing satisfaction that resources are being used optimally. 2.2. The potential pitfalls of self-assessment There are a number of issues which are causes of concern to those encouraging learners to experiment with self-assessment. The most obvious and perhaps most likely to discourage teachers and learners is the issue of reliability. The other issues revolve around the change in roles of both learners and teachers brought about by introducing self-assessment. Unreliability The jury is still out on whether self-assessment is reliable but there has been considerable discussion. Dickinson (1987: 136) suggests “it is likely that teachers and other specialists will be more reliable in their assessment and make accurate assessments more often than the learners”. It should be noted, 52 Links & Letters 7, 2000 David Gardner however, that he finds other convincing arguments for recommending selfassessment. In a study of multilingual, multicultural postgraduate learners in Britain, Blue (1988) compared learners' ratings of their performance with ratings giving by their tutors. He found considerable divergence. The work of Pierce et al. (1993) is based on school aged learners in a French immersion program in Canada. Learners assessed themselves against two criteria: by comparing themselves with a native speaker and by reflecting on the difficulty they experienced with everyday tasks in French. Results were compared against learners' results on proficiency tests of the four skills. The researchers concluded that self-assessment is not a reliable indicator of proficiency. However, as they point out, many of the subjects have little or no access to the target language or native speakers outside the classroom. In effect, it would be difficult for them to imagine how they would perform. In a comparison of a test of Dutch as a second language for adult learners and a self-assessed version of the same test, Janssen-van Dieten (1989) found the self-assessed version less reliable although earlier studies and her own pilot studies had been more encouraging. For her, the value of self-assessment is “its positive influence on the learning process” (Janssen-van Dieten, 1989: 44). Thomson (1996), in studying learners of Japanese as a foreign language (situated in Australia but from diverse cultural backgrounds), also felt very positive about using self-assessment despite finding considerable diversity in the accuracy of self-marking. Other studies have concluded that self-assessment is reliable. Bachman and Palmer (1989), for example, found that members of a multilingual, multiracial group of adult learners of English as a foreign language in the US were able to reliably self-rate themselves for their communicative language abilities. Another example of success with self-assessment comes from Blanche (1990) who studied the ability of a group of adult learners of French as a foreign language (in the US) to estimate their own speaking ability. He concludes that “the overall accuracy of the self-evaluations... is impressive” (Blanche 1990: 226). The variability in findings is difficult to account for with any certainty because the variables among the cited studies are many. Just among these five studies there is variation in sample size, age of subjects, cultural backgrounds, target language, whether learners were in a target language environment, format of tests, language skills being tested and what the self-assessment scores were being compared with. Any one of these variables, or any combination of them, may affect reliability. It is, perhaps, comforting that even in the studies where results were disappointing, researchers maintained a belief in the value of self-assessment. Undoubtedly, reliability is an issue that needs to be kept in mind but it is not one which should prevent self-assessments from being tried. Perhaps teachers would feel more confident about self-assessment if they conducted their own research on its reliability within their own contexts. The findings could be communicated to learners to let them make their own judgements. Self-assessment for autonomous language learners Links & Letters 7, 2000 53 The degree to which a certain amount of unreliability can be tolerated depends on the uses to which the assessments will be put. Where used for individual monitoring of progress, absolute reliability may be of less importance. Assessments which are not totally reliable still offer many of the benefits listed in figure 1. Where assessments are to be used for accreditation, reliability is obviously important. It is essential to ensure accurate measurement of standards and hence fairness. One way to validate individual self-assessments is to have a teacher randomly check some of the results. This would encourage learners to be honest and realistic in their self-marking and would contribute to accreditation. In addition, regular random checking would provide a clearer understanding of the reliability issue. Changing roles Self-assessment indicates a change in the roles of learners and teachers. The degree to which there is a change will depend on the degree to which learners are already autonomous. Less autonomous learners (and teachers who are not used to dealing with autonomous learners) will need relatively more support. A lot of the issues in this category are related to perceptions which are potentially negative. Firstly, there is the perception of worth. Assessments are only useful if they are accepted as valid. Many learners, and perhaps also teachers, administrators and the public, expect tests to be prepared and administered by teachers. They may find unacceptable tests which are self-administered suspecting that it will be easy to cheat and thus consider them worthless. Awareness of the usefulness of self-assessment needs to be raised if their face validity is not to be compromised. Secondly, is the potential of self-assessment to upset the perceived balance of power. Both teachers and learners may feel that assessment is a teacher's job. Learners may resist the extra work and may also lose respect for their teachers if they perceive self-assessment as a ruse to off-load part of the teachers' burden on them. Teachers may see self-assessment as a threat to their jobs or at least a loss of power. Alternatively, it may simply be perceived as a change in the status quo as was the case in the study conducted by Thomson (1996). Thirdly, learners may feel unequipped or unwilling to produce, conduct and interpret their own assessments. These are skills language teachers possess and language learners may well feel they lack. However, there is an important difference between self-administration and self-creation of tests. The former is the starting point of self-assessment and the latter is a more distant goal. Teachers can help learners acquire the requisite skills. Interpreting test results will become a familiar activity as learners engage in reflection. Finally, there is the problem of self-consciousness. Just as some learners feel self-conscious about speaking a foreign language so too will some react badly to the idea of conducting their own assessments. In younger learners it is possible that this will be translated into immature behaviour. In older learners it may foster resentment of the process. 54 Links & Letters 7, 2000 David Gardner 2.3 Balancing benefits and pitfalls It is clear that a lot is to be gained from learners self-assessing themselves. It could be argued that many of the benefits mentioned above are true of all kinds of assessment. For example, even the results of a formal examination will allow a learner to reflect on goals, strategies and achievements. They will also allow learners and teachers to evaluate learning materials. If the results meet the learner's aspirations they will, no doubt, motivate. The additional bonus of self-assessment is that it allows a high level of individualisation as well as providing immediate feedback. It is an important tool for autonomous learners (indeed, for all learners). We must acknowledge, nevertheless, that self-assessment is not without its potential pitfalls. This is where the teacher comes in. The potential pitfalls listed above can be neutralised by teachers who have skills and experience that exceed those of students. The ways in which they can do this will be discussed below. 3. The role of teachers/counsellors in the process The role of teachers in facilitating self-assessment falls into three parts: first to raise awareness among learners of the benefits of self-assessment; second to provide guidance on, and materials for, conducting self-assessments; and third to help learners understand the significance of the results. Teachers have a unique combination of knowledge, expertise and skills which not only allows them to undertake this role but actually makes them the best people for the job (figure 2). It should be clear that self-assessment is not about leaving students to fend for themselves. It is about teachers creating opportunities for students to make responsible choices which individualise assessments to their own needs. In some cases this may mean considerable support from teachers and, in others, less. As learners become more autonomous, and certainly as they become more skilled at self-assessment, they are likely to be more inventive in their assessments and require less support. 4. Levels of learner involvement A distinction must be drawn between producing an assessment and conducting an assessment. Self-assessment does not mean ‘think of a way to assess yourself’ although it does also include that option. Various combinations of teacher-learner involvement characterise different varieties of selfassessments. Figure 3 illustrates the combinations. At one end of the spectrum the teacher prepares everything. For a teacherprepared assessment the teacher thinks of a way of assessing something, prepares the content, prepares the answer key and writes a set of instructions for use. Learners use the assessment. At the other extreme is a learner-prepared Self-assessment for autonomous language learners Links & Letters 7, 2000 55 assessment in which all the work is done by the learner. A generic assessment is a halfway house. The teacher develops an idea for how to do an assessment, explains the idea clearly in a set of instructions and leaves the learners to create their own assessments. The advantage of generic assessments is that the same idea can be used many times with the learner varying the content but still with some of the face validity of teacher-prepared assessments. Generic assessments are a balance between teacher-prepared and learner-prepared assessments in a number of ways (figure 4). In a sense they also help to balance out the benefits against the pitfalls of self-assessments. 56 Links & Letters 7, 2000 David Gardner Assessment Content Criteria Administered by TeacherTeacher Teacher Learner prepared Generic Learner Teacher Learner LearnerLearner Learner Learner prepared Figure 3. Varieties of self-assessments. Role Teachers’ Knowledge, Expertise and Skills Awareness raising Knowledge: Teachers know their students (educational background, cultural context, etc), so they know the problems which need dealing with. Expertise: Teachers can identify the benefits of self-assessment for their students. Skills: Teachers can get their students’ attention. Provision of guidance Knowledge: Teachers know about assessing and materials learning. Expertise: Teachers are skilled assessors of learning. Skills: Teachers can produce assessment materials. Teachers can help learners focus an assessment. Interpretation Knowledge: Teachers know about the target results language. Expertise: Teachers can judge the significance of results. Skills: Teachers can translate assessment outcomes into learning goals. Figure 2. The roles of teachers in self-assessment. Producing a good assessment is fairly work-intensive but is usually justified on the grounds that it will be used with a large number of students who all need to be tested using the same criteria. The high level of individualisation among autonomous learners, however, negates this justification unless there happens to be a particular area in which reasonably large numbers of learners want to self-assess themselves. Generic assessments, however, give learners a procedure rather than a fixed test. They can apply the procedure frequently by varying the content and they can adapt the procedure to suit their changing needs. An example of a generic assessment appears in the appendix. To optimise the benefits of generic assessments, Gardner and Miller (1999: 210) suggest they should contain the following pieces of information: —the purpose of the assessment — the benefit to the learner — the procedure for conducting the assessment — the procedure for marking the assessment — a suggested marking scale — a choice of follow up actions based on the score achieved. Self-assessment for autonomous language learners Links & Letters 7, 2000 57 Teacherprepared Generic Learner-prepared Face validity Cost effectiveness Expert authoring Reliability Ease of preparation ✓ ✖ ✓ ✓ ✓ ✖ ✓ ✖ ✖ ✖ ✓ ✓ ✓ ✖✓ ✖✓ ✖ Figure 4. The characteristics of different kinds of assessments.