Japan Language Testing Association Newsletter. No. 10. 31 Aug. 2001 (p. 3, 4)
Special Lecture Report:
Language for Specific Purpose Testing: The State of the Artby Dan Douglas (Iowa State University)
After outlining the key points in his recent work Assessing Languages for Specific Purposes, Douglas mentioned how the notion of language testing for specific purposes merits further consideration. He indicated that it is becoming increasingly difficult to distinguish language for specific purpose (LSP) tests from general language (GL) tests. LSP tests, Douglas contends, should be regarded as one type of communicative test. Nearly all major LSP tests today, he pointed out, are stringently developed according to Bachman and Palmer's (1996) criteria for communicative language testing which, among other things, states that all tests should reflect target language use. For this reason, Douglas suggested that language tests are best described in terms of a continuum of specificity, with tests such as the TOEFL® at one end of the spectrum, and the Air Traffic Contoller's Proficiency Exam at the other. The key difference between general and LSP tests is not in the construction process, but the nature of the construct being measured. "Looking at the way tests are constructed is no longer a viable way to identify what is a LSP test or a more general test," Douglas remarked.
Discussing the relationship between general language knowledge and specific purpose background knowledge, Douglas asserted that "background knowledge, far from being a factor leading to construct irrelevant variance, may be an essential aspect of the construct of specific language purpose ability." He echoed Fulcher (1999) in suggesting that some aspects of contextual background knowledge should be regarded as part of an underlying test construct, rather than a source of test error.
Effective LSP tests . . . integrate elements of linguistic knowledge with specific background knowledge relevant to a particular field. Pointing out how difficult it is to separate background knowledge from general language knowledge, Douglas advocated specifying specific background knowledge in test constructs. At the same time, he cautioned against attempting to assess some types of non-linguistic abilities beyond the capacity of the test developers. It would be inappropriate, for example, for most LSP test developers to attempt to measure a person's engineering expertise. Effective LSP tests, Douglas maintains, integrate elements of linguistic knowledge with specific background knowledge relevant to a particular field. To achieve this, it is often necessary to discover what people in a given field consider to be valid performance criteria. Mentioning how tests often fail to measure critical skills which score users are seeking to discern, Douglas commented, "The ways we judge performance in specific language tests need to be considered more closely."
Clapham, Davies, Fulcher, and Widdowson have recommended that linguistic knowledge should be a main focus of communicative tests. Widdowson (2001), in particular, has argued that we should not even try to measure communicative language ability, since it is a dynamic concept defying measurement. Douglas points out that language researchers in the coming decades will need to explore how Widdowson's assertion that tests should focus on linguistic knowledge. Their answer to that issue will, in part, influence LSP test development. Douglas believes that the concept of general language knowledge itself should be questioned. He conceives of language as a variable capacity rather than a monolithic construct.
In addition, Douglas mentioned how the distinction between dialect and register proposed by Davies (1984) may not always be valid. Davies contends that dialects are acquired as native languages, while registers may be acquired for specific purposes at any time, then disregarded when not called for. Douglas, however, believes that both dialects and registers are learned and discarded as part of social behavior. In Douglas' view, one of the main challenges in the years ahead is to define what is meant by 'specific purpose language testing'. Currently LSP has been criticized by Davies and others as being too atheoretical and undefined. Douglas counters by asserting that, "LSP testing is in fact justified on theoretical grounds . . . but . . . often not carried out for pragmatic reasons – it is too expensive or impractical to produce the variety of field-specific tests that theory would justify." Because language performance varies with context and since academic or vocational contexts often call for very specific types of language use, Douglas regards LSP testing as theoretically justified.
After briefly reviewing current testing research, Douglas remarked how language tests in general are moving away from discrete point multiple-choice tasks towards more integrative, communicative tasks. He voiced concern over the tendency in the field to prefer GL tests over LSP test. He suggested that LSP tests are often superior to general language tests in that they tend to contain more situationally relevant assessment criteria.
In his presentation, Douglas also expressed concern about the lack of pragmatic reality in many test simulations. Acknowledging that realism is hard to simulate, he felt test developers must strive to make their tests as realistic as possible.
Douglas concluded by noting the vibrant activities being conducted in the field of LSP testing. He mentioned about 30 ongoing research projects, then underscored the need for further studies on generalizability, the influence of information technology on test performance, assessment criteria, backwash, and what actually makes texts field-specific.
- reported by Tim NewfieldsBachman, L. & A. Palmer. (1996). Language Testing in Practice. Oxford: Oxford University Press.
Davies, A. (1984). ESL expectations in examining: the problem of English as a Foreign Language and English as a Mother Tongue. Language Testing, 1, 82 - 98.
Fulcher, G. (1999). Assessment in English for Academic Purposes: Putting Content Validity in Its Place. Applied Linguistics, 20 (2) 221 - 236.
Widdowson, H. (2001). Communicative Language Testing: The Art of the Possible. In C. Elder, A. Brown, et al. (Eds.) Experimenting with Uncertainty: Essays in Honour of Alan Davies. (pp. 12-21). Cambridge: Cambridge University Press.