Assessment for the course was based on three extremely tough sit-down examinations as well as a term paper. At the very beginning of the course, O’Grady explicitly justified his use of examinations by saying that we were future teachers of linguistics and so we needed to know the material in detail. I happen to disagree with this argument, and under other circumstances I might have polemicized with O’Grady; but I was very favorably impressed with the fact that the professor had questioned his own assessment procedures and had made decisions about how to evaluate the students not out of an unthinking adherence to custom but out of a conviction (I would further suggest, a moral conviction) that it was the good and right thing to do, and furthermore, that he respected his students as thinking adults enough to explain his position to them. This brings me to another key component of our role as teachers in determining assessment procedures. All educational work is fundamentally rooted in context. In the case of assessment, then, I suggest that we have an ongoing moral duty to interrogate the context in which assessment is being used, in order to determine what is the good and right way to proceed with these particular learners at this particular point in their learning. In the case of Professor O’Grady, a key element in the equation were the needs of the students in their coming professional lives: His learners were all doctoral students in linguistics, and so the examination format was, in his view, appropriate. In any given case, this equation will be a complex one, including the future needs of students, their expectations, the nature of what is being assessed (vocabulary? writing skills? communicative ability?), systemic requirements, cultural preferences, and so on. Moreover, calling the decision-making process an equation is also inaccurate, because in reality the weighing of many factors is nearly always done in a holistic and flexible way. It is equally important to interrogate the nature and ostensible and real purpose of existing tests: that is, to take what Shohamy (1998), after Pennycook (1994) and others, called a “critical” approach to language testing. According to Shohamy, among other things such a stance:
• “views language tests as…deeply embedded in cultural, educational, and political arenas where different ideological and social forms struggle for dominance” (p. 332);
• “asks questions about what sort of agendas are delivered through tests and whose agendas they are” (p. 332);
• “challenges psychometric traditions and considers interpretive ones” (p. 332);
• “asks questions about whose knowledge the tests are based on,” including whether what is included in tests can be “negotiated, challenged and appropriated” (p. 333);
• suggests that “the notion of ‘just a test’ is an impossibility because it is impossible to separate language testing from the many contexts in which it operates” (p. 333). 77 Values in English Language Teaching The one key word that Shohamy (1998) did not use, although she implies it in all the foregoing, is value. What values are enshrined in, or presupposed by, the kinds of tests we use and the ways in which we use them?

Continue Reading

The preceding discussion suggests another central moral paradox of language teaching. On the one hand, without some form of evaluation our students cannot be sure of their progress or how this matches up with the requirements of the systems within which they are studying; furthermore, although administrative convenience may seem a paltry motivation for particular forms of testing, it is also true that it represents one domain in which learners are treated in some way equally. (One could also argue that, in principle, an efficiently functioning administration should also be in the best interests of students.) On the other hand, any form of testing or assessment that we use is unavoidably only a partial, indirect, and subjectively judged reflection of the student’s actual abilities; this is true both because of the inherent qualities of assessment procedures and the impossibility of ever conclusively determining what it means to know a language. Thus, we have a moral imperative to offer some form of assessment, yet any form of assessment is morally suspect and fallible. This clearly leaves us as teachers in an uncomfortable position. Is there any way out of it? Well, at one level there is not; like the other moral paradoxes and dilemmas we have The Morality of Testing and Assessment 76 examined in this book, the paradox of testing simply represents a constant factor in our work. It is better seen as a dynamic rather than a problem; that is, it is simply a permanent characteristic of what we do, rather than some obstacle that will eventually be overcome. On another level, however, I believe there are ways forward. First of all, I suggest it is incumbent on each of us as teachers to continually reflect on our own values and continue to question whether these values are accurately reflected in our assessment procedures. A few years ago, when I was in graduate school, I took a course in transformational syntax from William O’Grady, a well-known theoretical linguist.

Continue Reading

To what extent is subjectivity an advantage, and how can I know when it becomes a hindrance? In some teaching situations, teachers address this problem by being highly specific about the components of the portfolio. My daughter’s sixth-grade teacher used this approach. For example, draft writing components must include a 1-page outline (100–120 words), two drafts, and a final draft. Furthermore, many teachers simply note the presence or absence of certain items, not paying attention to their quality. This simplifies the procedure and renders grading more uniform and less subjective, but it also has the effect of shifting the portfolio back toward traditional assessment in its inflexibility and reliance on quantity and not quality. Each of these decisions carries complex and usually contradictory moral consequences. At this stage in the book I hope it is not necessary to point out that the preceding discussion is not intended to argue against portfolios. As I mentioned, I continue to use portfolios myself, and I see them as being considerably superior to traditional forms of testing. My point in this section is that even if we choose to use portfolios or other forms of alternative assessment, we still very much need to be aware of the moral consequences of our decision and the moral complexities with which it is fraught. It is only through reflection on these issues that we can move toward what for us is a morally grounded approach to evaluation.

Continue Reading

On the one hand, the field as a whole supports student autonomy, responsibility, and empowerment; we teachers sometimes question whether there is even such a thing as “teaching,” and we portray ourselves rather as “people who help others to learn.” Yet on the other hand, we know both intellectually and personally that there is such a thing as teaching, and that there are considerably better and considerably worse ways of doing it, and we take personal responsibility for the successes and failures of our learners. It is terribly difficult to figure out where the responsibility of the teacher ends and that of the learner begins. The notion of the teacherstudent relation to some extent addresses this conundrum, suggesting that it is neither one nor the other, but the rela-tion between them that is the key factor. Yet at the end of the (school) day each of us teachers is an individual, and each of us wonders about our own agency, its moral obligations and moral limits. The practice of portfolio assessment is one attempt to shift the balance of this dynamic in one particular direction, but the underlying dynamic remains. The last moral dilemma I examine in relation to student portfolios is this: How exactly are we to evaluate them? It is here that the paradox of subjectivity in testing comes back to haunt us, along with Noddings’ (1984) problem of the moral dissonance that occurs when teachers become evaluators. The fact is that although the other desirable characteristics of portfolios—student choice, lack of stress, a capacity for capturing both deeper understanding and development over time—remain in place, many of the moral dilemmas attendant on traditional testing are still present. How is student work to be graded? As before, do we reward progress over time, or final ability, or hard work, or all three? In what combinations? Also as before, though the evidence in portfolios is considerably richer than that in multiple-choice tests, it is still only indirect evidence, and still must be filtered through the interpretive understanding of the teacher. 75 Values in English Language Teaching Moreover, the apparent flexibility of the portfolio also conceals another uncomfortable moral dynamic: The greater the freedom and flexibility of the design of the portfolio, the harder it is for the teacher to evaluate it. If I allow one student to write an essay for one component, and another student to write a poem for the same assignment how can the
two be compared? Furthermore, when it comes to grading, I face the same problems mentioned earlier. If I give a student extra leeway for a poorly done assignment handed in at a difficult personal time for that student, am I using or abusing the special relation I have with him and with other students? Conversely, how can I grade evenly and yet also reward outstanding work that I believe deserves special recognition?

Continue Reading

Third, in contrast with traditional tests, which provide at best only a snapshot in time of the learner’s competence, portfolios aim to show the growth of that competence; to this end, they often include drafts of papers along with the final versions, or a series of written pieces that show improvement over time. Last, because a portfolio is assembled over the period of a semester or a term (or longer), the stressful practice of cramming all one knows into a single 2-hr exam at the end of the course is avoided. I truly believe that portfolios represent a huge improvement over traditional forms of assessment; I use them myself, and there is growing evidence that they constitute an effective assessment tool (Torrance, 1995). Nevertheless, as I mentioned earlier, portfolios are by no means exempt from the complex moral dilemmas that inform other kinds of evaluation. In the remainder of this section I outline the principal moral issues that the use of portfolios entails. First, we must acknowledge that, for many students, it is considerably easier to memorize a few vocabulary items and grammar structures than to compile a portfolio; that is, traditional form of assessment are often easier on certain students. In deciding whether to institute portfolio assessment, then, we face the moral decision of whether this innovation truly serves the best interests of the students. Can we be sure that the gains from the portfolio are worth the effort required of our students (and ourselves)? Related to this are what are sometimes termed ecological considerations: How does the portfolio relate to the broader curriculum and the educational and social system in which students are situated? In Japan, for example, many students need English primarily to pass The Morality of Testing and Assessment 74 grammar-oriented tests that in turn will allow them to enter a good college; in such cases, assessment by portfolio, while in principle justifiable, may not in practice be defensible if the students’ needs and goals are factored into the equation. Such decisions involve moral considerations of what is right and good for particular students in particular sociopolitical, cultural, and educational contexts. Even in contexts where portfolios might be more appropriate—for example, U. S. academic English programs—there is often considerable resistance from the learners themselves. Although I certainly do not believe that such resistance represents an absolute impediment to change, I also believe that we have a moral responsibility to take our learners’ viewpoints seriously. This is part of the dialogue that forms the foundation of the teacher-student relation. Furthermore, for education to take place, we cannot simply bemoan the fact that our students are not where we would like them to be. All true education takes the students where they are and leads them from there. This notion has been echoed by authors as diverse as Nel Noddings (1984) and Paulo Freire (1972). Another aspect of the teacher-student relation that arises here is that of responsibility. Though this varies from case to case, use of portfolios always involves a considerable shift of responsibility (for selecting material, ordering it, and presenting it) from the teacher to the student. Of course, practical problems often arise in such circumstances. Some students fail to take on this responsibility, whether out of rebellion, inertia, or some other reason. Yet underlying these practical matters is one of the fundamental moral paradoxes of our profession.

Continue Reading

I practice this both in my teacher education classes and my ESL teaching. Nevertheless, my point here is that even if one agrees that these methods are superior, they are still value laden, and they still involve complex moral issues and moral dilemmas. In this section I explore the moral underpinnings of alternative approaches to assessment. I center the discussion around a series of moral dilemmas that inhere in the processes of instituting and maintaining portfolio assessment in the language classroom (Genesee & Upshur, 1996). Many other approaches are possible in alternative assessment, but the portfolio is probably the best known of its techniques. In addition, such a focus allows me to be more concrete in my discussion. A portfolio is an organized collection of different pieces of work by a student that is presented in lieu of a traditional examination for the purposes of assessment. Portfolios are often thematic but loosely structured; their function is to demonstrate both the range and the quality of a student’s work (Cole, Ryan, & Kick, 1995; De Fina, 1992; Genesee & Upshur, 1996). Several important features of the portfolio contrast with aspects of traditional assessment mentioned earlier. First, the portfolio is designed to show what the student can do and does know as opposed to what he or she cannot do and does not know. For this reason, an important element in the process of compiling a portfolio is that the student be able to choose which pieces of work are included and which are not. Second, portfolios are intended to give a richer picture of the student’s abilities and understanding than can be gleaned from one-word multiple-choice answers; thus, portfolios often focus on more extended, contextualized pieces of work such as written essays.

Continue Reading

1. Look again at the situation described by Wen-Hsing. How would you have handled this situation? What other options were open to Wen-Hsing? What values would underlie these options?
2. What did you think about the story of Alison’s student in French? Would it be reasonable to apportion blame in this story? How else might it have been resolved?
3. What forms of evaluation do you use in your own teaching? What values underlie these kinds of evaluation?
4. What do you think we should be assessing in the testing of language learning? How can tests or other forms of assessments measure this? What problems are there?
5. To what extent should we as teachers be responsible not just for students’ learning but also their study habits, their behavior, and their values?
6. What, in your view, is a good student?
7. Liz Hamp-Lyons (1998) suggested that test preparation programs are ethically (morally) wrong. What is your view of this? Have you prepared students of your own for standardized tests such as the TOEFL? In light of your experiences, what do think of Hamp-Lyons’ arguments?

Continue Reading

Whose values are these? What kinds of value, in turn, are assigned to students on the basis of these tests? I suggest that in addressing the moral complexities of teaching each individual teacher needs to consider the values inherent in the tests used in her own educational setting. Finally, though I am rendering myself particularly vulnerable to attack here, I would strongly advocate the need for flexibility in assessment procedures. Given the multiple uncertainties that attend the design and the taking of a test, we simply cannot rely on raw, unmediated scores to give us accurate and fair information about a student’s level, ability, or amount of learning. The learning process is a highly individual one, and the teacherstudent relation is similarly unrepeatable. If assessment is to be an integral part of teaching—which, I have argued, it needs to be—then it must be included in that relation, inside what Noddings called “the uniqueness of human encounters” (1984, p. 3). This does not exclude the use of externally written and scored standardized tests, but I believe that, in essence, assessment in the classroom must be brought within the bounds of the unique relation between teacher and student and that in order for this to happen, we need the flexibility that comes from a deep knowledge of our students and their circumstances. Of course, it is also clear that another word for this flexibility is subjectivity, and to misquote a famous saying as it might apply to teaching, the price of subjectivity is eternal vigilance.

Continue Reading

Over the last 20 years or so, a significantly different approach to assessment has been developed (Genesee & Upshur, 1996; Herman et al., 1992; A. Katz, 2000; O’Malley & Valdez Pierce, 1996; Torrance, 1995). Alternative assessment, as it is generally known (the term authentic assessment is also used), takes as its starting point a vigorous critique of conventionally used forms of assessment such as multiple-choice tests. The case against traditional assessment includes many of the arguments I have already mentioned in this chapter: that these forms of assessment test the wrong kinds of knowledge, appealing to memorization and simplistic knowing of facts rather than deeper understanding; that they are designed with administrative convenience in mind rather than being grounded in the best interests of the students; that they are unnecessarily stressful; and that they aim to catch students out with what they do not know rather than allowing them to show what they do. In place of such tests, alternative assessment offers various options, including portfolio assessment, “kidwatching” (Goodman, 1985) and other forms of continual assessment, teacher-student learning contracts, and a range of other ideas. 73 Values in English Language Teaching Before I continue this discussion I would like to state for the record that I find the case against traditional assessment rather convincing. I have not given an exam in more than 7 years; I do all of my assessment by alternative methods, including portfolios, journals, written assignments, and so on.

Continue Reading

Or is it the fault of the students who demand a certain kind of teaching? […] Is it perhaps the fault of the teaching institutions, which do not provide any kind of teacher training in TOEFL preparation? […] Or is training in teaching test preparation the responsibility of the textbook writers and publishers? (p. 335) Hamp-Lyons’ (1998) questions remind me of the social nature of morality. The point of her questions, as I understand them, is not to apportion ultimate blame (“yes, it’s the students’ fault”) but to point up the fact that adjudicating on moral issues is a highly complex process in which many individuals and institutions have a stake. Standardized tests are social phenomena par excellence; any consideration of their moral significance must begin from this starting point. 2Of course, I am sidestepping the fact that with a so-called communicative test it is still necessary to define what knowledge of language is and to ignore the fact that an examination virtually by definition cannot involve genuine communication and therefore will always be only an indirect and artificial indication of the candidate’s “true” ability, however ability is defined. The Morality of Testing and Assessment 72 For example, much as I loathe the whole business of tests, when it comes to my own students I can understand why they would want to have extra preparation. Although at one level such an approach demolishes both the illusion of the snapshot-of-ability principle and the principle of equality and fairness, on the other hand, the moral importance of relation creeps in. It is never the case, and I would argue that it never should be, that a teacher’s own students are not more important than some other students in another state or country. We want the best for our own students, even if in general moral terms it could give them an “unfair” advantage. This is a classic instance of the way in which individual circumstances and specific relations color our approach to moral dilemmas. This brings me to a final point, which Noddings (1984) raised in her discussion of assessment. Contrary to received wisdom regarding the preferability of local, teacherdeveloped forms of assessment over mass standardized testing, Noddings wrote that she
is “convinced…that grading—summative evaluation of any kind—should not be done by teachers. If it must be done, it should be done by external examiners, persons hired to look at students as objects. Then teacher and students would be recognized as together in the battle against ignorance” (p. 195). Despite my instinctive and growing distaste for standardized tests of all kinds, I find Noddings’ argument curiously persuasive because, like her, I believe that our prime duty as teachers is to focus on the learning of our own students. Turning the problem of evaluation over to outsiders moves it from the immediate, local teacher-student relation, nd spares that relation the “grinding” experience mentioned earlier in which the teacher switches caps from advocate to judge. Noddings’ (1984) suggestion does not justify indiscriminate use of testing and it does not offer any justification of current testing practices or excuse test makers from an obligation to continually rethink the format and nature of their tests. It does, however, remind us that we are dealing with issues of immense moral complexity, in which unequivocal good and bad, right and wrong, are terribly hard to pin down.

Continue Reading

One candidate from a western Eu-ropean country had an English mother, and her entire application confirmed her own categorization of herself as “virtually bilingual,” yet her TOEFL score was a mere 597, which, the computer-generated form from the university’s Office of International Admissions told us, indicated that she “may need supplementary English training.” A doctoral candidate from an African country in which English is widely spoken had attended an Englishlanguage university for his undergraduate and master’s degrees and had sterling references and published academic work, yet his scores on the TOEFL and Graduate Record Examination were both abysmally low. What should we do in such situations? Both candidates were admitted, but in each case the test scores complicated the decision rather than making it easier; in the case of scores on the Graduate Record Examination, for example, we are required by the university’s graduate school to obtain an official exception for any candidate who does not score the required minimum in this test. 71 Values in English Language Teaching Another moral paradox is the disjuncture between testing and current pedagogical practice in language teaching (Hamp-Lyons, 1998). The communicative model that, in various forms, is widely used across the world encourages students to engage in meaningful interaction using whatever linguistic means they have at their disposal; it specifically downplays the importance of grammatical accuracy over communicative effectiveness. Although some tests have made more or less successful attempts to integrate communicative competence in their evaluations of students (the Cambridge suite of examinations and the ACTFL’s [American Council for the Teaching of Foreign Languages] proficiency guidelines for foreign languages [Byrnes & Canale, 1987] come to mind; see also Powers, Schedl, & Wilson, 1999), the TOEFL has notably lagged behind in this regard, and many other tests still focus narrowly on grammar and vocabulary.2 One consequence of this disjuncture is the so-called washback effect: the ways in which test format affects teaching. The theory behind the TOEFL, like any test of its kind, is that it is a snapshot of a candidate’s language ability at a given moment in time; thus, it should not be possible to improve one’s performance other than by more study of the language. In reality, of course, TOEFL preparation courses and programs abound. Liz Hamp-Lyons (1998) offered a very thoughtful analysis of some of the ethical (what I would call moral) issues that arise from the “powerful” (p. 331) washback effect of such an influential test as the TOEFL. Drawing on the work of Mehrens and Kaminsky (1989) and Popham (1991), Hamp-Lyons posed the question of what constitutes “ethical test preparation” (p. 334) and argued that existing materials (and hence a great deal of existing test preparation programs across the world) are “educationally indefensible (boosting scores without mastery) and of dubious ethicality (coaching merely for score gain)” (p. 334). She went on to ask a series of provocative and important questions, all of which have a strong moral dimension: Can a test be blamed for the ways in which some teachers teach towards it? […]

Continue Reading

This, combined with the strict and ritualistic way in which standardized tests are conducted, gives the knowledge they enshrine a solemn, almost sacred significance. The political, “interested” nature of knowledge featured in chapter 3 is a powerful component here too. I argue that there is deep moral meaning in such an approach to knowledge: By reducing learners to recipients of knowledge rather than creators of it, one is also reducing their capacity for moral agency. There is also a question of honesty here. Because of the veil of objectivity behind which they hide, standardized tests ride roughshod over the unavoidable difficulties of matching score with actual ability. The final score is presented (and in the overwhelming majority of cases is also treated) as an objective measure: The uncertainties and ambiguities that attend test development, and the myriad psychological factors that affect a candidate’s performance on a given day, are invisible. Furthermore, because of the physical and administrative distance between the testers and those tested, appeals are difficult, if not impossible. A teacher might possibly be inclined to be lenient on a student whose grandfather died a few days before the exam, or to give a student who has difficulty writing an extra minute or two at the end of a test. A standardized test can offer neither of these possibilities or anything like them. What is missing here is relation: The human relation between tester and testee, which exists when teachers prepare tests, and which informed the whole of the previous section, is entirely absent in the standardized test. By this account, the moral contours of the test are quite different. The educational process is suddenly deprived of its deepest and most meaningful component. This feature is underlined even more in the current shift to computerized testing in the TOEFL and many other common tests. The impersonal nature of such tests, and the impossibility of our understanding the human dimension of the test-taking experience of any specific individual, makes it very difficult for the consumers of test score information to know how to interpret them. As language professionals, we know the complexities I have been discussing here; as a result, reading the scores is very much a matter of interpretation rather than a simple acknowledgment of a score. Just 2 days ago I was reviewing some late admissions for our own master’s and doctoral programs.

Continue Reading

Everything I wrote earlier about the value-laden nature of assessment practices—that they are oriented to product rather than process, that they favor certain candidates over others, that they are used for administrative convenience rather than serving the needs of the learners—applies in spades in the case of standardized tests such as the TOEFL. Yet the TOEFL and its ilk also raise an additional set of moral concerns and dilemmas. Elana Shohamy (1998), one of the first people to raise questions about the “ethical” dimensions of language testing, described the widespread use of standardized tests to promote bureaucratic and political agendas. She identified three ethical consequences of such uses of tests:

1. The “institutionalized knowledge” (p. 339) that tests canonize is “narrow, simplistic and often different from experts’ knowledge” (p. 339). The kind of knowledge tested, which often involves single-word answers in multiple-choice formats, “overlooks the complexities of subject matter and is unmeaningful for repair” (p. 339).

2. A “parallel system” (p. 340) is created whereby stated policy is at odds with the “organizational aspirations” reflected in the tests. Shohamy gave the example of Israel, where “both Hebrew and Arabic are official languages, yet, on the high school entrance exam Arabs are tested in Hebrew, while Hebrew speakers are not tested in Arabic” (p. 340).

3. Ethical problems arise when “the test becomes a means through which the policy makers communicate priorities to the system” (p. 340). Shohamy sees this as “undemocratic and unethical” (p. 340) because those affected by the test—the students who take it and the teachers who teach them—have no say in the design and implementation of the test. This last point deserves further consideration. I would argue that the most serious moral concerns with such tests arise from their imper-sonal nature. As Shohamy (1998) pointed The Morality of Testing and Assessment 70 out, the people affected by the test have no say in its creation; through such procedures it is much easier to maintain the myth of the objective test, because the people who create the questions and assess performance are nowhere around—unlike with a teacher or school department, to which students usually have some kind of access.

Continue Reading

Who Is a Good Student?

Throughout this discussion I have deliberately been using the words good and bad. This whole discussion ultimately, revolves around a fundamental ambiguity inherent in the phrase good student (Amirault, 1995). On the one hand, a good student is one who does well: learns, passes tests and exams, and so on. These qualities and achievements are moral in nature the way that education in general is moral in nature. It is good to learn, to know more, to have more skills and abilities. Yet even here there is ambiguity. What exactly does it mean to do well, to succeed? Such questions once again go to the heart of our purpose in teaching. In an adult literacy class, for example, is a student successful if he reads a newspaper article? Or passes his GED (the high school equivalency examination)? Or if he gets a job? We might also ask: What of the student who learns well but does not pass the exam? Or what of the EFL student who gets only a C in English yet is promptly hired to teach English in an elementary school? (I have known such teachers myself.) Furthermore, there is a social notion of the “good” student that is also moral in nature, yet in a different way. This notion of the “good student” takes good to mean obedient, pleasant, willing, hard working, conscientious, persistent—all of which, of course, are 69 Values in English Language Teaching also morally desirable characteristics, and which, other things being equal, equip students better to benefit from their education. Yet this meaning of “good student” cannot always be reconciled with that mentioned in the preceding paragraph: Some students work hard and are pleasant but do not properly grasp the subject matter; others are sullen and lazy yet smart. What do we—what do you—mean when you use the expression “She’s a good student”? Which of these meanings is more important to you, and to the student concerned? Which meanings are reflected in the system of values underlying the forms of assessment you use?
It is important to emphasize the symbiotic relationship between the moral messages
sent by our assessment practices and our notions of what it is to be a good student. It is
through whatever assessment practices we use that the identity of good or bad student is
encoded in schools; conversely, our idea of the good student affects the kinds of
assessment we select. In either case, multiple powerful and complex moral meanings are
to be found in the kinds of tests and other forms of evaluation that we use in our
classrooms.

Continue Reading

Assessment Beyond Language

The value-laden nature of assessment, moreover, goes far beyond the simple matter of how to measure language ability. There are also crucial educational considerations to take into account. A central moral dilemma for many teachers, for example, at least in this country, is the extent to which they should reward effort, or ability, or achievement. Up until this point, I have been assuming that evaluation is intended to measure the student’s ability in English. But in much ESL teaching in the United States and certain other countries, great emphasis is placed on a student’s engagement in, or commitment to, her work. It is thought important to reward effort—the time and energy devoted to an assignment, rather than merely the quality of the finished product, or the willingness to participate in classroom discussion rather than the grammatical correctness of the contributions or the value of their substance. In this there is very clearly an issue of moral judgment: In rewarding “good” behavior, we are standing in judgment over the learner; we are adjudicating “good” and “bad” ways to be as well as knowledge of the subject matter. There is a strong component of moral education in the old-fashioned sense, of instilling 67 Values in English Language Teaching and reinforcing desirable behaviors, habits, and attitudes in our students (Jackson, Boostrom & Hansen, 1993). At the same time, another aspect of the moral dimension of power emerges, as we punish those who do not behave in approved ways, for example, giving lower grades to students who do not willingly take part in classroom activities, fail to turn in journals or other written work on time, and so on. What function does this punishment serve? As a warning for the future? As a sign to others? In any case, surely its consequences are not restricted to the moment in which a bad grade is given and received. Let me share an example of the complex issues at play here. For her doctoral dissertation, Ewald (2001) interviewed university-level students and teachers of Spanish about their attitudes toward group work. One teacher she spoke to, Gonzalo, explained that he graded students on their contributions to small-group work. The students to whom Ewald spoke, however, felt that this was an unfair practice, pointing out that although they accept the usefulness of small-group work, for some students participation in such groups is rendered difficult for nonlinguistic reasons such as shyness. Ewald (2001, p. 166) reported that Gonzalo’s practice is grounded in a belief that evaluation of this aspect of their work in class will motivate students to participate more and help them to see the value of small-group work (and we know that in language classes, the more you speak, the more you learn). He might also have wished to be able to reward the students who contribute more willingly. Yet, as the students’ reaction shows, this practice brings with it several moral dilemmas. First, there is the question of the extent to which personality traits such as shyness should affect one’s grade. Second, Ewald pointed out that the students were already aware of the expectation of participation and did not need to be reminded of it. This becomes a matter of trust (p. 167): That is, the practice of evaluating contributions to group work carries with it the implication that without the pressure of the evaluation students cannot be trusted to participate of their own accord. I would also point out a third issue of measurement: the problem of how to assign scores fairly to something as complex as participation in a small group. The practice of rewarding hard work as well as “objectively” measured ability gives rise to its own moral dilemmas. What do we do with those students who work terribly hard and yet simply do not have the wherewithal to do A-grade work? Conversely, what do we do with the bright but disaffected students who are able to speak fluently and write expressively yet will not take part in classroom dialogue and do the minimum to scrape by in their written work? Once, many years ago (when I still gave exams), in an undergraduate class on second language acquisition I had a student who barely came to class at all yet turned up for the midterm exam and did tolerably well. What is one to do in such a situation? What was I to do? At one level, the student had done what she was supposed to: She had learned the material the course covered. At another level, she had flouted the (in this case unwritten) rules of engagement of the academy, which state that good students are expected to do the things that good students do: come to class, take part in discussions, show interest, and so on. A related issue is whether one aims to measure ability or achievement. Often I have had students who come to the class knowing very little about the matter at hand and who learn a lot during the class. Do these students deserve a better grade than those who knew a great deal more at the beginning yet at the end may still know more than their colleagues? The Morality of Testing and Assessment 68 Such questions raise the specter of our purpose in teaching in the first place. If indeed we aim merely to transmit information or knowledge, then we should reward the student who has, or has acquired, more information or knowledge. However, throughout this book I have been arguing that teaching cannot and should not be reduced to the transfer of information. It is primarily about the moral relation between teacher and student. This said, however, the teacher, as one-caring (Noddings, 1984), is in a different position than the student, the cared-for. What is the moral responsibility of the latter toward the former? Noddings (1984) suggested that while the teacher’s responsibility is greater, there is still a need for reciprocity (pp. 69–74). Yet to what extent is it our responsibility to judge the student on matters of character or innate ability? I argued in chapter 3 that there is an element of moral education in adult ESL settings. Yet what of other contexts? How far do our duties go beyond teaching the language and into the territory of character formation? An additional point is the moral dilemma that arises from the fact that students have different levels of ability. The questions I have just raised—whether students should be rewarded for effort or for achievement and whether progress is as important as final achievement—cannot really be answered without referring to differing levels of aptitude. Some students, for whatever reason, are simply good at languages; others, to use a Polish expression, are anti-talents when it comes to language learning. In a sense, this is a matter of “moral luck” (Statman, 1993): Some people are “born better” in one regard or another. At college, my friend Brett would regularly infuri-ate me by finishing his French essays in a scrawl as we were walking to class together; he invariably got an A. I think most of us have known other Bretts, whether as friends or students of ours. Should he and his kind be rewarded for simply being better and faster? It seems to me that try as we might to evaluate students on language alone, we cannot help but take other, morally charged circumstances into consideration; the question is, are we aware of this? If so, have we thought through the moral consequences of our decision?

Continue Reading

Assessing Knowledge of Language

A central question in the assessment of language learning—possibly the most important of all—is: What does it mean to know a language? Anyone designing any kind of evaluation has to answer this question; yet to do so is already to begin to make morally significant judgments. Is language knowing vocabulary? Being able to recite grammar rules? To buy an airplane ticket? To translate sentences? To write a persuasive essay? In choosing between these and a thousand other options, we are making choices that will have significant effects on our students and their performance. Consider the relatively simple case of Wen-Hsing mentioned earlier, where choices of what is and is not acceptable, reached by group consensus, left students who had given grammatically acceptable answers with a worse score. Furthermore, the fact of the matter is that our choices themselves are largely based on what I have called faith, that is, our beliefs about the nature of language, learning language and knowing language that are grounded only partly in logic and can never be fully confirmed or disproved (see chap. 1). Knowing a language is a phenomenally complicated thing; in determining how to test that knowledge we are forced to make choices that oversimplify the picture (McNamara, 1996). Our choices, furthermore, have demonstrable consequences for students. Ania, my elder daughter, who is bilingual in English and Polish, returned to Poland for some of her high school education. In one of her English classes she failed a major exam because she did not “know” the grammar of English and so was unable to understand instructions such as: “Convert the following sentences into the present perfect tense,” even though she was able to use such structures with nativelike ability in her speech. Her teachers had chosen to define knowledge of English as knowledge of grammatical terminology rather than the actual ability to speak the language (which for Ania would not have been a problem). Of The Morality of Testing and Assessment 66 course, we have some general guidelines—it is good pedagogical practice, for example, to test what has been covered in class and not what has not (Genesee & Upshur, 1996; Herman, Aschbacher, & Winters, 1992)—but this merely begs the question of what should be taught in class. The business of testing is even more complicated because there is only ever an indirect relation between our notion of what it is to know a language and the form of evaluation we devise. Even if we believe that language learning is a matter of vocabulary only, we have to select certain lexical items to be included in the test and exclude others. The situation is, of course, infinitely more complex if we have a more sophisticated understanding of knowledge of language, including areas such as pragmatics and discourse. In parallel fashion, there is only ever an indirect relationship between a student’s performance in a test and her actual knowledge of the language, whether for reasons of nerves or having a good day or bad day, or from the universally acknowledged slippage between competence and performance. All of these factors mean that to devise a test and to assign scores or grades to those who take it is to sail out onto very dark and deep moral waters indeed. Last, another fundamental conundrum is that neither language nor competence in language is naturally measurable. If we are judging how high a person can jump, we can pretty much agree on who jumps higher than others: Height is simple to measure. It is not at all clear, however, how we can objectively measure how well someone speaks another language. We find ourselves resorting to subjective terms such as fluent, hesitant, and difficulty (Richard-Amato, 1996, pp. 99–100), which require constant interpretation, and once more, the more sophisticated our attempts at measurement become, the harder they are to pull together into a cohesive overall assessment. The fundamental immeasurability of language competence lends a further moral dimension to our work in language assessment; the decisions we are forced to make about how competence will be assessed are always subjective and thus can only be rooted in our beliefs about what is right and good, beliefs which, we must always acknowledge, could be mistaken.

Continue Reading

In this section I focus on forms of assessment that are designed, administered, and evaluated by teachers themselves. Although, as I mentioned earlier, I believe that all forms of assessment have moral meaning, assessments by teachers of their own students carry particular kinds of moral significance.

Assessment and the Teacher-Student Relation
However we evaluate our students, when we come to do so we are always and inevitably faced with an insurmountable moral problem. In all that I have read on testing and teaching, I have nowhere seen it better expressed than by Nel Noddings (1984). In the following passage she begins by reaffirming the paramount importance of the teacherstudent relation: Teaching involves two persons in a special relationship. Usually, there is a fairly well-defined “something” in which the two engage, but this is not always true. Sometimes teacher and student just explore. They explore something, of course, but this something is not always prespecified; nor need it remain constant or, for that matter, even lead somewhere definite. The essence is in the relationship. In the relationship, the teacher has become a duality; she shares the view of the objects under study with the student. Then suddenly, grindingly, she must wrench herself from the relationship and make her student into an object of scrutiny, (p. 195) This “grinding” quality of assessment practices is an unavoidable consequence of the teacher-student relation. If we were merely technicians conveying information, there would be no moral dimension to assessment. However, we are not, and this dimension 65 Values in English Language Teaching not only exists but is of central importance in our approach to assessment. As teachers, we wish to be supportive—to push our students, yes, but to do so in ways that make them feel challenged yet also free to fail without consequences. At the same time, the need to evaluate—which, as I argued in the preceding section, is also a moral imperative—not only does not promote that kind of relation but actively works against it. It is important to point out that this moral dilemma does not go away when teachers do not have control over the testing practices used in their classrooms. They still participate in the processes of evaluation; from the point of view of the teacher-student relation, the net result is the same: the “grinding” sensation described by Noddings. The only difference is that the teacher has not had a voice in determining what material counts as knowledge for the purposes of the examination. In the previous section, I mentioned several ways in which assessment procedures cannot help but influence the teacher-student relation. One other crucial aspect of this influence must be mentioned here: the question of trust. Implicit in a great many aspects of testing is a lack of trust toward students: Everything from seating patterns to the meticulously controlled matter of test security are established in ways that assume a default tendency to cheat on their part. Trust, in turn, is an implicit belief in the fundamental goodness of the other. Absence of trust, by the same token, indicates a lack of a belief in the other’s basic goodness. In our mechanisms of control we are passing moral judgment on our learners.

Continue Reading

These two paradoxes, profound and central as they are, do not exhaust the moral contours of assessment procedures. Assessment—any form of assessment—is moral in a number of ways. First and foremost, assessment is moral because, as I mentioned earlier, it quite literally places a value on each individual student. This value is usually given either as a percentage or some other fraction, a score, or a letter grade. In the colossal majority of cases, the student falls short of mathematical “perfection.” Assessment is also moral in that most forms of assessment in some way measure one student against another—that is, they assign not just an absolute value but also a relative value. Here issues of justice creep in: There is a delicate balance between treating everyone equally and rewarding those who do better, whether through hard work, innate ability, or a combination of these (an important issue to which I return a bit later). Furthermore, assessment is moral because it often has serious real-world consequences for learners. The grades we give our students, and the scores they obtain on standardized tests, often have huge significance in their lives: Because of these scores and grades, they get or do not get accepted into programs, they are or are not given scholarships and funds, they are or are not promoted, are or are not given a raise, and so on. (Remember the consequences faced by Peter’s Palestinian student in the story told at the beginning of chap. 1.) Our decisions form the direct or indirect sources of these assessments and thus carry great moral weight, because we have to be sure (as in fact we rarely can be) that the aforementioned kinds of decisions are just and fair, that the “best” candidates (once again a moral expression) have in fact been successful. Last, assessment is also moral because, like everything else in teaching, it is conducted in complex and morally ambiguous real-world contexts, and to be understood properly it cannot be divorced from those contexts or seen to be merely about the learning of languages. The dilemma faced by Peter in chapter 1 is an example of the way in which the world beyond the language classroom can impinge hugely on our decision making in 63 Values in English Language Teaching student evaluation: Here, the political realities of life outside class made an apparently straightforward evaluation of the student’s abilities horribly complex in moral terms. The decision that Peter made had, of course, no impact on that student’s knowledge of English. My point is precisely that sometimes, whether we like it or not, a student’s abilities are not the only thing that needs to be taken into consideration. Let me share an example parallel with that of Peter. A friend of mine, Alison, recently took a junior faculty position in the French department of a well-known private university. She had a student who wished to take a minor in French; to do so, he needed a B minimum in his language classes. Yet the student was lazy and rather arrogant and signally failed to do work at a level that would allow him to receive a B.Alison gave him a lower grade. She was called to the dean, who quietly explained to Alison that this student’s parents had donated millions of dollars to the university and that the university was counting on further donations. Like Peter, Alison reluctantly changed the grade she had awarded. I tell this story not to condemn Alison, but quite the opposite—to show how complicated the real world of evaluating students can be. As with Peter, the final grade Alison gave did not in the least represent some reassessment of the student but was a result of external factors; nevertheless, the final grade is what counts in the real world.1 Neither do the moral contours of evaluation end here. An additional moral dilemma is the constant and unresolved (indeed, unresolvable) dynamic between formative and summative functions of assessment. Assessment specialists commonly draw a distinction between formative assessment—that is, assessment designed to indicate to a student how he or she is doing—with summative assessment, which measures final achievement in a course or program (Rea-Dickins & Gardner, 2000; Torrance & Pryor, 1998). Yet in reality the distinction is not clear. Certain summative forms of evaluation take on a formative role: For example, when I received what is known as an Upper Second bachelor’s degree from my university in England—a summative qualification—I also took this as a formative indication that, without a first-class degree, I had no business returning to postgraduate education. It took me some time to revise my interpretation and enter a doctoral program. Conversely—and, I believe, more commonly—grades or marks that are meant to have a formative role take on certain summative qualities. For example, many kinds of evaluation, such as quizzes and midterms, are intended to let students know how they are doing. Yet often scores from these sources also factor into final grades; thus, the evaluation is also summative in that it forms part of the summative grade. Moreover, even when this is not the case, formative evaluations look like summative ones; they often come in the form of scores on tests in which there is little in the way of feedback. I suggest that this resemblance leads learners to see the teacher as a judge rather than as a teacher, once again affecting the teacher-student relation. 1Julian Edge has pointed out to me that there are other moral aspects to this story, too. The parents’ desire to look after their son’s interests is also morally justifiable, as are the potential benefits from the expected donations to many other students at the university, including some whom scholarship money would allow to participate in otherwise prohibitively expensive programs. The Morality of Testing and Assessment 64 Last, the truly dilemmatic nature of approaches to assessment is underscored by the fact that, in light of the paradox of the necessary evil, a decision not to use any form of assessment at all is also a moral act. By choosing not to give any exams or other methods of assessment, a teacher is of course relieving students of the stress and all the attendant vagaries of determining what counts as knowledge. However, such a teacher is also sending other moral messages to her students. Many students might believe that this decision reflects an underlying indifference to what the students learn and hence to them as people; that is, once again it will affect the teacher-student relation. Students who would normally strive to excel will have a reduced motivation to do so—in fact, only those students truly engaged in the subject matter are likely to remain unaffected and, as most teachers would agree, such students are rarely in the majority (Milton, Pollio, & Eison, 1986). Furthermore, regardless of the theoretical arguments, exams can function to help students distinguish important from less important aspects of what they cover; the absence of exams makes such distinctions much harder. In many ways, then, the moral landscape of assessment is complex and difficult terrain. Both the inner workings of assessment procedures and their broader sociopolitical context are such that questions of assessment are always also questions of values. These values, in turn, are never straightforward but always fraught with conflict.

Continue Reading

The first paradox is that of test subjectivity. On the one hand, testers place great emphasis on the goal of objectivity in testing. This seems, by and large, a worthy goal. Yet testing, more than any other aspect of teaching, is value laden—and, as we have seen, values are inherently subjective in nature (Gipps & Murphy, 1994). The selection of what to test, how it will be tested, and how scores are to be interpreted are all acts that require human judgment; that is, they are subjective acts. The preceding example of Wen-Hsing and her colleagues is a miniature example of this process. It follows quite naturally that the process of assigning and grading a test or other work by a student—that is, the process of evaluation—is precisely that: a process of placing a value on the head of each individual student. If this is not a moral act, nothing is. Furthermore, while subjectivity is on the whole not a desirable quality, flexibility is; and flexibility in testing—accepting the word playing in Wen-Hsing’s test, for example, when the official key allows only talking—can be achieved only through the subjective decision making of a particular teacher with particular students who happen to produce these answers (once again, we find ourselves back at the critical place of the teacher-student relation). The second paradox, which intermeshes with and compounds the first, is what I shall refer to as the paradox of the necessary evil. On the one hand, tests and other forms of assessment are undesirable, and a great many teachers dislike them. Not only are they unreliable and inherently disposed to unfairness, as described earlier, but they are also stressful on students (and, in different ways, on teachers, too), and they take precious time and attention from what most of us see as the real purpose of education: learning. Tests are often designed more from the point of view of administrative convenience than that of the students’ needs. Furthermore, there is an ever-present political aspect to testing The Morality of Testing and Assessment 62 that often takes over: The already unhealthy societal preoccupation with testing magnifies these problems to the point where the processes and goals of education are seriously undermined. On the other hand, however, most teachers would also recognize that some form of evaluation, though it may not be pleasant, is in fact essential not just for the convenience of the teacher or the school but for the learners themselves. Learners, and their teachers, need to have a sense of how well they are doing: of their progress, of how their work measures up to expectations, maybe even of how they stand in relation to their peers. Without this information, they can feel lost and adrift. Furthermore, many teachers (myself included) like to be able to have some way of rewarding outstanding work and giving due recognition to those who perform particularly well. Thus, while evaluation is undesirable for moral reasons, it is at the same time necessary, also for moral reasons; it is a necessary evil. These, then, are the two fundamental paradoxes with which we enter the discussion of values and assessment in language teaching: the first is the dynamic of objectivity and subjectivity in testing; the second is the simultaneous desirability and undesirability of assessment.

Continue Reading

I start this chapter with a very simple story about tests told to me by Wen-Hsing, a former student of mine who now teaches at a secondary school back home in Taiwan. Wen- Hsing recently sent me an e-mail about what happened when she and her colleagues were grading the English language component of an entrance exam: One section of the English test was: “According to the picture, answer the following five questions.” It was a picture of a classroom, where there is a teacher standing and six students seated. It looked like two of the students were talking to each other and the teacher was not happy about it. One of the five questions was “Why is the teacher angry? The teacher is angry because students are__.” This blank only allowed one word and the “standard” answer, according to the test-giver, was “talking.” When we were grading the answer sheet, we found there were a variety of answers and some of them that seemed possible were “playing,” “noisy,” “bad.” Thus, we voted to decide if we would accept these answers. Interestingly, “noisy” and “bad” were accepted but “playing” was rejected. The reason of the majority was that we could not tell from the picture whether these two students were playing or not. Well then, I asked them, “Can you tell from the picture that these two are bad?” The answer I got was “We all agreed not to include ‘playing’ in the answers. If we reached an agreement, it is fine.” Although I would like to give students whose answer was “playing” credit, I couldn’t do it and I graded those answer sheets the way I was told to do. This case was not unique. It happened every time I graded in the entrance exam. I don’t know why some possible answers were accepted but some were not. I think it is good to have students answer questions according to the picture they see, but is it necessary to restrict the number of answers? You know what, I always felt “not so good” after grading because there was always one or two answers that would arise dispute. The problems faced by Wen-Hsing and her colleagues reveal the profoundly moral nature of assessment in language teaching. These Taiwanese teachers are striving to adjudicate which knowledge is sanctioned and which is not; their deliberations involve drawing lines in the sand where there are few if any objective criteria unambiguously separating right from wrong. Yet the consequences of their decisions will be visited on the children of their classes and, over time, will become part of each child’s permanent record. Of course, one could argue that the item in question is simply badly designed and that what is needed is just a better test composed of less ambiguous questions. Yet I believe that anyone who has tried to write a test, whether a professional test designer or a classroom teacher, will recognize the difficulties the Taiwanese teachers face. With such a phenomenally complex thing as a language, there are limitless problems that arise in determining ways of testing students’ knowledge; the more complicated and interesting that knowledge becomes, the harder it is to test (Bachman, 2000). Furthermore, those who are most adept at writing test items—professional testers—are also those farthest removed from the classroom, and thus they lack information about what has been covered in class by particular groups of students. All of us are obliged to make do with faulty tools in the work of evaluating students. In this chapter I explore the moral dynamics underlying various aspects of testing and evaluation. During the discussion, I raise many complex moral questions both about traditional forms of evaluation such as standardized tests and examinations, and about alternative approaches to assessment such as portfolios. I argue, however, that two profound moral paradoxes underlie the entire realm of language testing and assessment.

Continue Reading