Assessing Knowledge of Language
A central question in the assessment of language learning—possibly the most important of all—is: What does it mean to know a language? Anyone designing any kind of evaluation has to answer this question; yet to do so is already to begin to make morally significant judgments. Is language knowing vocabulary? Being able to recite grammar rules? To buy an airplane ticket? To translate sentences? To write a persuasive essay? In choosing between these and a thousand other options, we are making choices that will have significant effects on our students and their performance. Consider the relatively simple case of Wen-Hsing mentioned earlier, where choices of what is and is not acceptable, reached by group consensus, left students who had given grammatically acceptable answers with a worse score. Furthermore, the fact of the matter is that our choices themselves are largely based on what I have called faith, that is, our beliefs about the nature of language, learning language and knowing language that are grounded only partly in logic and can never be fully confirmed or disproved (see chap. 1). Knowing a language is a phenomenally complicated thing; in determining how to test that knowledge we are forced to make choices that oversimplify the picture (McNamara, 1996). Our choices, furthermore, have demonstrable consequences for students. Ania, my elder daughter, who is bilingual in English and Polish, returned to Poland for some of her high school education. In one of her English classes she failed a major exam because she did not “know” the grammar of English and so was unable to understand instructions such as: “Convert the following sentences into the present perfect tense,” even though she was able to use such structures with nativelike ability in her speech. Her teachers had chosen to define knowledge of English as knowledge of grammatical terminology rather than the actual ability to speak the language (which for Ania would not have been a problem). Of The Morality of Testing and Assessment 66 course, we have some general guidelines—it is good pedagogical practice, for example, to test what has been covered in class and not what has not (Genesee & Upshur, 1996; Herman, Aschbacher, & Winters, 1992)—but this merely begs the question of what should be taught in class. The business of testing is even more complicated because there is only ever an indirect relation between our notion of what it is to know a language and the form of evaluation we devise. Even if we believe that language learning is a matter of vocabulary only, we have to select certain lexical items to be included in the test and exclude others. The situation is, of course, infinitely more complex if we have a more sophisticated understanding of knowledge of language, including areas such as pragmatics and discourse. In parallel fashion, there is only ever an indirect relationship between a student’s performance in a test and her actual knowledge of the language, whether for reasons of nerves or having a good day or bad day, or from the universally acknowledged slippage between competence and performance. All of these factors mean that to devise a test and to assign scores or grades to those who take it is to sail out onto very dark and deep moral waters indeed. Last, another fundamental conundrum is that neither language nor competence in language is naturally measurable. If we are judging how high a person can jump, we can pretty much agree on who jumps higher than others: Height is simple to measure. It is not at all clear, however, how we can objectively measure how well someone speaks another language. We find ourselves resorting to subjective terms such as fluent, hesitant, and difficulty (Richard-Amato, 1996, pp. 99–100), which require constant interpretation, and once more, the more sophisticated our attempts at measurement become, the harder they are to pull together into a cohesive overall assessment. The fundamental immeasurability of language competence lends a further moral dimension to our work in language assessment; the decisions we are forced to make about how competence will be assessed are always subjective and thus can only be rooted in our beliefs about what is right and good, beliefs which, we must always acknowledge, could be mistaken.
A central question in the assessment of language learning—possibly the most important of all—is: What does it mean to know a language? Anyone designing any kind of evaluation has to answer this question; yet to do so is already to begin to make morally significant judgments. Is language knowing vocabulary? Being able to recite grammar rules? To buy an airplane ticket? To translate sentences? To write a persuasive essay? In choosing between these and a thousand other options, we are making choices that will have significant effects on our students and their performance. Consider the relatively simple case of Wen-Hsing mentioned earlier, where choices of what is and is not acceptable, reached by group consensus, left students who had given grammatically acceptable answers with a worse score. Furthermore, the fact of the matter is that our choices themselves are largely based on what I have called faith, that is, our beliefs about the nature of language, learning language and knowing language that are grounded only partly in logic and can never be fully confirmed or disproved (see chap. 1). Knowing a language is a phenomenally complicated thing; in determining how to test that knowledge we are forced to make choices that oversimplify the picture (McNamara, 1996). Our choices, furthermore, have demonstrable consequences for students. Ania, my elder daughter, who is bilingual in English and Polish, returned to Poland for some of her high school education. In one of her English classes she failed a major exam because she did not “know” the grammar of English and so was unable to understand instructions such as: “Convert the following sentences into the present perfect tense,” even though she was able to use such structures with nativelike ability in her speech. Her teachers had chosen to define knowledge of English as knowledge of grammatical terminology rather than the actual ability to speak the language (which for Ania would not have been a problem). Of The Morality of Testing and Assessment 66 course, we have some general guidelines—it is good pedagogical practice, for example, to test what has been covered in class and not what has not (Genesee & Upshur, 1996; Herman, Aschbacher, & Winters, 1992)—but this merely begs the question of what should be taught in class. The business of testing is even more complicated because there is only ever an indirect relation between our notion of what it is to know a language and the form of evaluation we devise. Even if we believe that language learning is a matter of vocabulary only, we have to select certain lexical items to be included in the test and exclude others. The situation is, of course, infinitely more complex if we have a more sophisticated understanding of knowledge of language, including areas such as pragmatics and discourse. In parallel fashion, there is only ever an indirect relationship between a student’s performance in a test and her actual knowledge of the language, whether for reasons of nerves or having a good day or bad day, or from the universally acknowledged slippage between competence and performance. All of these factors mean that to devise a test and to assign scores or grades to those who take it is to sail out onto very dark and deep moral waters indeed. Last, another fundamental conundrum is that neither language nor competence in language is naturally measurable. If we are judging how high a person can jump, we can pretty much agree on who jumps higher than others: Height is simple to measure. It is not at all clear, however, how we can objectively measure how well someone speaks another language. We find ourselves resorting to subjective terms such as fluent, hesitant, and difficulty (Richard-Amato, 1996, pp. 99–100), which require constant interpretation, and once more, the more sophisticated our attempts at measurement become, the harder they are to pull together into a cohesive overall assessment. The fundamental immeasurability of language competence lends a further moral dimension to our work in language assessment; the decisions we are forced to make about how competence will be assessed are always subjective and thus can only be rooted in our beliefs about what is right and good, beliefs which, we must always acknowledge, could be mistaken.
0 comments
Post a Comment