Assessment for the course was based on three extremely tough sit-down examinations as well as a term paper. At the very beginning of the course, O’Grady explicitly justified his use of examinations by saying that we were future teachers of linguistics and so we needed to know the material in detail. I happen to disagree with this argument, and under other circumstances I might have polemicized with O’Grady; but I was very favorably impressed with the fact that the professor had questioned his own assessment procedures and had made decisions about how to evaluate the students not out of an unthinking adherence to custom but out of a conviction (I would further suggest, a moral conviction) that it was the good and right thing to do, and furthermore, that he respected his students as thinking adults enough to explain his position to them. This brings me to another key component of our role as teachers in determining assessment procedures. All educational work is fundamentally rooted in context. In the case of assessment, then, I suggest that we have an ongoing moral duty to interrogate the context in which assessment is being used, in order to determine what is the good and right way to proceed with these particular learners at this particular point in their learning. In the case of Professor O’Grady, a key element in the equation were the needs of the students in their coming professional lives: His learners were all doctoral students in linguistics, and so the examination format was, in his view, appropriate. In any given case, this equation will be a complex one, including the future needs of students, their expectations, the nature of what is being assessed (vocabulary? writing skills? communicative ability?), systemic requirements, cultural preferences, and so on. Moreover, calling the decision-making process an equation is also inaccurate, because in reality the weighing of many factors is nearly always done in a holistic and flexible way. It is equally important to interrogate the nature and ostensible and real purpose of existing tests: that is, to take what Shohamy (1998), after Pennycook (1994) and others, called a “critical” approach to language testing. According to Shohamy, among other things such a stance:
• “views language tests as…deeply embedded in cultural, educational, and political arenas where different ideological and social forms struggle for dominance” (p. 332);
• “asks questions about what sort of agendas are delivered through tests and whose agendas they are” (p. 332);
• “challenges psychometric traditions and considers interpretive ones” (p. 332);
• “asks questions about whose knowledge the tests are based on,” including whether what is included in tests can be “negotiated, challenged and appropriated” (p. 333);
• suggests that “the notion of ‘just a test’ is an impossibility because it is impossible to separate language testing from the many contexts in which it operates” (p. 333). 77 Values in English Language Teaching The one key word that Shohamy (1998) did not use, although she implies it in all the foregoing, is value. What values are enshrined in, or presupposed by, the kinds of tests we use and the ways in which we use them?

Continue Reading

The preceding discussion suggests another central moral paradox of language teaching. On the one hand, without some form of evaluation our students cannot be sure of their progress or how this matches up with the requirements of the systems within which they are studying; furthermore, although administrative convenience may seem a paltry motivation for particular forms of testing, it is also true that it represents one domain in which learners are treated in some way equally. (One could also argue that, in principle, an efficiently functioning administration should also be in the best interests of students.) On the other hand, any form of testing or assessment that we use is unavoidably only a partial, indirect, and subjectively judged reflection of the student’s actual abilities; this is true both because of the inherent qualities of assessment procedures and the impossibility of ever conclusively determining what it means to know a language. Thus, we have a moral imperative to offer some form of assessment, yet any form of assessment is morally suspect and fallible. This clearly leaves us as teachers in an uncomfortable position. Is there any way out of it? Well, at one level there is not; like the other moral paradoxes and dilemmas we have The Morality of Testing and Assessment 76 examined in this book, the paradox of testing simply represents a constant factor in our work. It is better seen as a dynamic rather than a problem; that is, it is simply a permanent characteristic of what we do, rather than some obstacle that will eventually be overcome. On another level, however, I believe there are ways forward. First of all, I suggest it is incumbent on each of us as teachers to continually reflect on our own values and continue to question whether these values are accurately reflected in our assessment procedures. A few years ago, when I was in graduate school, I took a course in transformational syntax from William O’Grady, a well-known theoretical linguist.

Continue Reading

To what extent is subjectivity an advantage, and how can I know when it becomes a hindrance? In some teaching situations, teachers address this problem by being highly specific about the components of the portfolio. My daughter’s sixth-grade teacher used this approach. For example, draft writing components must include a 1-page outline (100–120 words), two drafts, and a final draft. Furthermore, many teachers simply note the presence or absence of certain items, not paying attention to their quality. This simplifies the procedure and renders grading more uniform and less subjective, but it also has the effect of shifting the portfolio back toward traditional assessment in its inflexibility and reliance on quantity and not quality. Each of these decisions carries complex and usually contradictory moral consequences. At this stage in the book I hope it is not necessary to point out that the preceding discussion is not intended to argue against portfolios. As I mentioned, I continue to use portfolios myself, and I see them as being considerably superior to traditional forms of testing. My point in this section is that even if we choose to use portfolios or other forms of alternative assessment, we still very much need to be aware of the moral consequences of our decision and the moral complexities with which it is fraught. It is only through reflection on these issues that we can move toward what for us is a morally grounded approach to evaluation.

Continue Reading

On the one hand, the field as a whole supports student autonomy, responsibility, and empowerment; we teachers sometimes question whether there is even such a thing as “teaching,” and we portray ourselves rather as “people who help others to learn.” Yet on the other hand, we know both intellectually and personally that there is such a thing as teaching, and that there are considerably better and considerably worse ways of doing it, and we take personal responsibility for the successes and failures of our learners. It is terribly difficult to figure out where the responsibility of the teacher ends and that of the learner begins. The notion of the teacherstudent relation to some extent addresses this conundrum, suggesting that it is neither one nor the other, but the rela-tion between them that is the key factor. Yet at the end of the (school) day each of us teachers is an individual, and each of us wonders about our own agency, its moral obligations and moral limits. The practice of portfolio assessment is one attempt to shift the balance of this dynamic in one particular direction, but the underlying dynamic remains. The last moral dilemma I examine in relation to student portfolios is this: How exactly are we to evaluate them? It is here that the paradox of subjectivity in testing comes back to haunt us, along with Noddings’ (1984) problem of the moral dissonance that occurs when teachers become evaluators. The fact is that although the other desirable characteristics of portfolios—student choice, lack of stress, a capacity for capturing both deeper understanding and development over time—remain in place, many of the moral dilemmas attendant on traditional testing are still present. How is student work to be graded? As before, do we reward progress over time, or final ability, or hard work, or all three? In what combinations? Also as before, though the evidence in portfolios is considerably richer than that in multiple-choice tests, it is still only indirect evidence, and still must be filtered through the interpretive understanding of the teacher. 75 Values in English Language Teaching Moreover, the apparent flexibility of the portfolio also conceals another uncomfortable moral dynamic: The greater the freedom and flexibility of the design of the portfolio, the harder it is for the teacher to evaluate it. If I allow one student to write an essay for one component, and another student to write a poem for the same assignment how can the
two be compared? Furthermore, when it comes to grading, I face the same problems mentioned earlier. If I give a student extra leeway for a poorly done assignment handed in at a difficult personal time for that student, am I using or abusing the special relation I have with him and with other students? Conversely, how can I grade evenly and yet also reward outstanding work that I believe deserves special recognition?

Continue Reading

Third, in contrast with traditional tests, which provide at best only a snapshot in time of the learner’s competence, portfolios aim to show the growth of that competence; to this end, they often include drafts of papers along with the final versions, or a series of written pieces that show improvement over time. Last, because a portfolio is assembled over the period of a semester or a term (or longer), the stressful practice of cramming all one knows into a single 2-hr exam at the end of the course is avoided. I truly believe that portfolios represent a huge improvement over traditional forms of assessment; I use them myself, and there is growing evidence that they constitute an effective assessment tool (Torrance, 1995). Nevertheless, as I mentioned earlier, portfolios are by no means exempt from the complex moral dilemmas that inform other kinds of evaluation. In the remainder of this section I outline the principal moral issues that the use of portfolios entails. First, we must acknowledge that, for many students, it is considerably easier to memorize a few vocabulary items and grammar structures than to compile a portfolio; that is, traditional form of assessment are often easier on certain students. In deciding whether to institute portfolio assessment, then, we face the moral decision of whether this innovation truly serves the best interests of the students. Can we be sure that the gains from the portfolio are worth the effort required of our students (and ourselves)? Related to this are what are sometimes termed ecological considerations: How does the portfolio relate to the broader curriculum and the educational and social system in which students are situated? In Japan, for example, many students need English primarily to pass The Morality of Testing and Assessment 74 grammar-oriented tests that in turn will allow them to enter a good college; in such cases, assessment by portfolio, while in principle justifiable, may not in practice be defensible if the students’ needs and goals are factored into the equation. Such decisions involve moral considerations of what is right and good for particular students in particular sociopolitical, cultural, and educational contexts. Even in contexts where portfolios might be more appropriate—for example, U. S. academic English programs—there is often considerable resistance from the learners themselves. Although I certainly do not believe that such resistance represents an absolute impediment to change, I also believe that we have a moral responsibility to take our learners’ viewpoints seriously. This is part of the dialogue that forms the foundation of the teacher-student relation. Furthermore, for education to take place, we cannot simply bemoan the fact that our students are not where we would like them to be. All true education takes the students where they are and leads them from there. This notion has been echoed by authors as diverse as Nel Noddings (1984) and Paulo Freire (1972). Another aspect of the teacher-student relation that arises here is that of responsibility. Though this varies from case to case, use of portfolios always involves a considerable shift of responsibility (for selecting material, ordering it, and presenting it) from the teacher to the student. Of course, practical problems often arise in such circumstances. Some students fail to take on this responsibility, whether out of rebellion, inertia, or some other reason. Yet underlying these practical matters is one of the fundamental moral paradoxes of our profession.

Continue Reading

I practice this both in my teacher education classes and my ESL teaching. Nevertheless, my point here is that even if one agrees that these methods are superior, they are still value laden, and they still involve complex moral issues and moral dilemmas. In this section I explore the moral underpinnings of alternative approaches to assessment. I center the discussion around a series of moral dilemmas that inhere in the processes of instituting and maintaining portfolio assessment in the language classroom (Genesee & Upshur, 1996). Many other approaches are possible in alternative assessment, but the portfolio is probably the best known of its techniques. In addition, such a focus allows me to be more concrete in my discussion. A portfolio is an organized collection of different pieces of work by a student that is presented in lieu of a traditional examination for the purposes of assessment. Portfolios are often thematic but loosely structured; their function is to demonstrate both the range and the quality of a student’s work (Cole, Ryan, & Kick, 1995; De Fina, 1992; Genesee & Upshur, 1996). Several important features of the portfolio contrast with aspects of traditional assessment mentioned earlier. First, the portfolio is designed to show what the student can do and does know as opposed to what he or she cannot do and does not know. For this reason, an important element in the process of compiling a portfolio is that the student be able to choose which pieces of work are included and which are not. Second, portfolios are intended to give a richer picture of the student’s abilities and understanding than can be gleaned from one-word multiple-choice answers; thus, portfolios often focus on more extended, contextualized pieces of work such as written essays.

Continue Reading

1. Look again at the situation described by Wen-Hsing. How would you have handled this situation? What other options were open to Wen-Hsing? What values would underlie these options?
2. What did you think about the story of Alison’s student in French? Would it be reasonable to apportion blame in this story? How else might it have been resolved?
3. What forms of evaluation do you use in your own teaching? What values underlie these kinds of evaluation?
4. What do you think we should be assessing in the testing of language learning? How can tests or other forms of assessments measure this? What problems are there?
5. To what extent should we as teachers be responsible not just for students’ learning but also their study habits, their behavior, and their values?
6. What, in your view, is a good student?
7. Liz Hamp-Lyons (1998) suggested that test preparation programs are ethically (morally) wrong. What is your view of this? Have you prepared students of your own for standardized tests such as the TOEFL? In light of your experiences, what do think of Hamp-Lyons’ arguments?

Continue Reading

Whose values are these? What kinds of value, in turn, are assigned to students on the basis of these tests? I suggest that in addressing the moral complexities of teaching each individual teacher needs to consider the values inherent in the tests used in her own educational setting. Finally, though I am rendering myself particularly vulnerable to attack here, I would strongly advocate the need for flexibility in assessment procedures. Given the multiple uncertainties that attend the design and the taking of a test, we simply cannot rely on raw, unmediated scores to give us accurate and fair information about a student’s level, ability, or amount of learning. The learning process is a highly individual one, and the teacherstudent relation is similarly unrepeatable. If assessment is to be an integral part of teaching—which, I have argued, it needs to be—then it must be included in that relation, inside what Noddings called “the uniqueness of human encounters” (1984, p. 3). This does not exclude the use of externally written and scored standardized tests, but I believe that, in essence, assessment in the classroom must be brought within the bounds of the unique relation between teacher and student and that in order for this to happen, we need the flexibility that comes from a deep knowledge of our students and their circumstances. Of course, it is also clear that another word for this flexibility is subjectivity, and to misquote a famous saying as it might apply to teaching, the price of subjectivity is eternal vigilance.

Continue Reading

Over the last 20 years or so, a significantly different approach to assessment has been developed (Genesee & Upshur, 1996; Herman et al., 1992; A. Katz, 2000; O’Malley & Valdez Pierce, 1996; Torrance, 1995). Alternative assessment, as it is generally known (the term authentic assessment is also used), takes as its starting point a vigorous critique of conventionally used forms of assessment such as multiple-choice tests. The case against traditional assessment includes many of the arguments I have already mentioned in this chapter: that these forms of assessment test the wrong kinds of knowledge, appealing to memorization and simplistic knowing of facts rather than deeper understanding; that they are designed with administrative convenience in mind rather than being grounded in the best interests of the students; that they are unnecessarily stressful; and that they aim to catch students out with what they do not know rather than allowing them to show what they do. In place of such tests, alternative assessment offers various options, including portfolio assessment, “kidwatching” (Goodman, 1985) and other forms of continual assessment, teacher-student learning contracts, and a range of other ideas. 73 Values in English Language Teaching Before I continue this discussion I would like to state for the record that I find the case against traditional assessment rather convincing. I have not given an exam in more than 7 years; I do all of my assessment by alternative methods, including portfolios, journals, written assignments, and so on.

Continue Reading