In 1998 and 1999 three of the largest providers of educational tests introduced computer-based versions of proficiency tests for English as a foreign language. At the same time, many institutions and individuals began to offer Web-based tests for particular language programs and classes. These two phenomena have added immeasurably to the momentum of work in computer-assisted testing that began with a few research projects on computer-assisted language testing (CALT). Despite the fact that computer-assisted testing in large-scale programs is not without its problems, at the start of the 21st century, it seems clear that CALT is becoming a fact for all language learners in educational settings, and therefore for all teachers and applied linguists. As a growing number of people are affected by CALT, many of them express the fear that computer delivery may influence test performance.

The concern, which is echoed by SLA researchers about the results of computer-assisted methods in research, is that the computer-delivered methods are less valid than those requiring conventional means such as paper and pencil or face-to-face interview. However, to address this and other issues, a means for evaluating CALT is needed. This chapter begins by explaining the concern that test method may influence test performance in undesirable ways and then presents a practical method for evaluating the validity of computer-assisted tests which is based on the framework for test usefulness developed by Bachman and Palmer (1996).