1.1 Target Readership

This Element is written for teachers working in different language teaching contexts. We acknowledge that, in contemporary education, the view that only language teachers and assessment specialists need to be concerned with language assessment is changing. For example, in content–language integrated learning (CLIL) and English medium instruction (EMI) contexts, the teachers involved are not necessarily language teachers. Content teachers will likely implement some form of language testing and assessment (LTA) with their students as part of their curriculum work. For example, a CLIL, Year 3 maths teacher needs to help students comprehend a maths question such as ‘Tyler’s mobile service provider charges him at 5 pence per 30 seconds. How much will it cost him if his phone call lasts 6.5 minutes?’. In this mathematics question, some students’ success in answering it can be influenced by the target mathematical knowledge and the ability to construct meaning from a long noun phrase. In this CLIL context, academic language is likely to be taught and explained together with mathematical knowledge. Therefore, all teachers involved would benefit from an understanding that students’ content and language knowledge can impact their performance.

In this Element, we mainly use ‘teacher/s’ to refer to our readers and ‘students’ as individuals studying in a language programme. Because other terms are also used to describe those being tested or assessed (e.g., test-takers, candidates, and applicants) in the educational literature, we may also use them where they are terminologically and/or rhetorically apposite.

1.2 Overview of the Element

There are eight sections in this Element.

• Section 1 (Introduction) presents the concept of language assessment literacy, target readership and overview of this Element.

• Section 2 (Assessment) provides a background description of some key assessment contexts in English language teaching (ELT). The functions and purposes of assessment are discussed.

• Section 3 (Essential Concepts in Assessment) discusses the terminological distinctions and conceptual connections between testing, assessment, and evaluation. In addition, it provides the essential background of LTA as an academic and professional discipline that teachers can draw on for their understanding of assessment practice.

• Section 4 (Types of Language Assessment) explains two main types of assessment (i.e., formative assessment (FA) and summative assessment (SA)). While not intending to polarise them, this distinction is helpful for teachers to grasp some differences and interconnections.

• Section 5 (Key Theoretical and Technical Concepts) presents a set of key concepts in the design and development of language tests and assessments. Topics addressed in this section include constructs, tasks, and test item types, classical test theory (CTT), and errors in assessment.

• Section 6 (Summative Assessment Design: Types and Processes) considers major iterative stages in developing SA tasks: planning, developing, administrating, and using assessment results. This section includes a discussion of test and assessment techniques.

• Section 7 (Quality Aspects in Assessment) discusses essential quality criteria that ensure effective assessment. Criteria discussed include validity, reliability, practicality, ethics, fairness, and impact.

• Section 8 (Further Developments) points to areas for classroom assessment research that are relevant to teachers and recommends additional resources for professional development.

1.3 Special Features

This Element incorporates a call-out box, Scan Me, at various places. This call-out box contains a quick-response (QR) code linked to a selected YouTube video (with a URL also provided) relevant to a topic or issue under discussion. The videos provide additional information or perspectives about the issues being discussed. This shift to an audiovisual mode can make learning about assessment more accessible and meaningful. A guided reflection box is also provided at the end of each main section. Underneath these, a Padlet link is provided for readers to post and share their thoughts with others working in different parts of the world. The authors will engage with the readers’ comments and opinions as appropriate. In the final part of this Element, a Glossary of key terms in language assessment is provided for ease of reference and an Appendix contains example test specifications that teachers can consider if/when they wish to develop an SA.

2.1 Power and Influences of Assessment

All assessment activities have power and consequences (see, e.g., Reference Fulcher and HardingFulcher & Harding, 2022; Reference KunnanKunnan, 2018; Reference ShohamyShohamy, 2001). Sometimes their influence can be subtle and non-directly observable; at other times they can be profound and immediate. For example, when teachers use informal or embedded classroom assessments (e.g., checking student understanding by asking confirmation, clarification, or comprehension questions), they can be said to engage in subtle and non-directly observable but powerful assessment for pedagogic purposes. Teachers can help students improve their learning after observing their answers by providing immediate feedback to their students and modifying their subsequent teaching activities. While such teacher formative assessment practice is implicitly embedded in the ongoing classroom activities, it can have a powerful impact on the quality of learning. With FA, students can learn to independently assess their understanding, gain confidence in their ability to use the target language, and sustain their motivation to learn.

The impact and power of summative assessment (SA) can be more immediate and noticeable, such as when students are asked to take the midterm and final examinations and complete assigned coursework (e.g., individual, paired, or group tasks) that are used to represent their overall achievement in the course (e.g., final scores and grades to be recorded in academic transcripts). The power of such assessment is immediately apparent to both teachers and students. The assessment outcomes can determine students’ future opportunities. For example, if students fail a mandatory assessment task, they cannot move on to the next level or stage of their study. They have to repeat the same course. It also means that they will be behind their peers in terms of study completion, and they can lose their confidence, self-esteem, motivation to study harder, and so on.

Scan Me 1

Formative Assessment | Walford Anglican School for Girls | Australian Education Research Organisation (posted to YouTube by Australian Education Research Organisation (AERO))

https://bit.ly/46M8uAg

The power of external assessment (e.g., college or university admission tests, state or government tests) can impact classroom teaching and learning situations. Students may be required to take a college entrance examination at the end of secondary education. Teachers are often tasked to help students prepare for such an examination. This situation can shift teachers’ and students’ attention away from the curriculum content (i.e., they gear their teaching and learning specifically to the test). Teachers may get promoted or rewarded if the students’ success rate for admission is high. Another example of the power of external assessment is a standards-based assessment of students’ literacies and progression by governments and education agencies. Such assessment regimes enable governments to use assessment data to enforce or introduce some academic curriculums nationally and impose assessment outcomes–driven funding models that can directly impact schools, educational sectors, and society.

Globally speaking, governments often devise national educational policies and promulgate standards or benchmarks and curriculums that need to be followed by educational institutions. They rely on standardised testing and assessment to adjudge students’ educational standards and attainment levels nationally. Large-scale standardised testing is often deployed to ascertain student learning attainment at a state, provincial, or national level. One underlying, often hidden reason for using standardised assessment by governments is a perceived lack of rigour and trust in teacher- or school-based assessment by external government agencies. For this reason, there is a need to help teachers understand the roles and practices of assessment to change such negative perceptions of their professional knowledge and skills.

Scan Me 2

Testing, Testing | Linda Darling-Hammond | TEDxStanford (posted to YouTube by TEDx Talks)

https://bit.ly/46WFUw2 >

Education systems can enhance their assessment frameworks by providing supportive resources and professional development for teachers so that the educational value of teacher assessment can be positively promoted. With a more positive public perception of teachers’ ability to conduct a sound assessment, governments will be more willing to adopt a combination of standardised and teacher-led assessments for evaluating students. This approach is growing in popularity in some educational jurisdictions. For example, in New South Wales, Australia, 50 per cent of school students’ achievement is determined by school-based assessment (e.g., teacher and curriculum assessment) and the other 50 per cent by formal state-based examinations. Furthermore, there has been some strong resistance by various state governments to the Australian federal government’s attempt to mandate the same national tests for all Australian states. Another example is the Hong Kong Diploma of Secondary Education (HKDSE) English language examination which has a teacher assessment component.

2.2 Assessment in Education

Approaches and practices of assessment vary across language classrooms, educational contexts, and countries. For instance, SA is mandated or implemented, for instance, by official policies (e.g., all students must take part in the national examination program), entrance requirements (e.g., all students must pass the final examination to be eligible for the next course), or authorities (e.g., applicants need to obtain an overall IELTS Band 7 to be eligible for student visas). Standardised tests are often adopted to evaluate students according to established criteria, such as classroom learning outcomes stipulated by state or provincial educational curriculums. Standardised tests are carried out in accordance with strict administrative conditions and are marked or evaluated in the same way for all students; as such, they are seen to be fair because all students are treated in the same way, allowing comparisons among students’ performances. On the strength of this reasoning, decisions on students’ achievement levels, represented by marks or grades, can often be based solely on their test performances. While standardised tests can be valuable and transparent for decision-making on students, teachers know that the content of many tests does not require their students to perform tasks related to their actual classroom activities or real-life situations.

We note that in anglophone applied linguistics and assessment of English as a foreign/second language, there is a tendency to see the notion of ‘stakes’ (high versus low) in terms of impact on the individual or institutional functions. The concept of high-stakes within this purview can vary in degree and consequences. For instance, the impact of passing or failing a high-stakes exit or national test in English is first and foremost felt by the student. At the same time, such a test serves an important institutional function for educational institutions (e.g., student selection), employers (e.g., personnel selection of qualified professionals), and society (e.g., fairness for all). Seen in this light, high or low stakes relate to how assessment use affects individuals and society.

Scan Me 3

What Is High Stakes Testing – and Why Hasn’t ItWorked? (posted to YouTube by Gregg Clemons)

https://bit.ly/3RiXMLI>

Classroom assessment focusses on activities in the classroom, which can be informal and embedded in teaching and learning activities. Teachers can employ various assessment activities to ascertain student learning and support effective teaching. Teacher assessment can be a dynamic or impromptu reaction to a given teaching–learning situation (e.g., teachers ask students to explain the target grammar rule after they have completed an exercise together). Follow-on activities such as this can reinforce learning. Teachers can also provide feedback comments on students’ responses in informal assessment to support further learning. Informal assessment, such as a teacher-led question-and-answer activity, may not immediately impact the student (e.g., passing or failing a mandatory public examination or test, as discussed already). For this reason, informal assessment is sometimes seen as low-stakes. If the purpose of assessment is to find out what has been learnt and how to improve learning further, then such informal assessment should not be seen as low-stakes because it is pivotal to the central purpose of education.

Assessments in education vary in the degree of stakes, and they can have different roles in teaching, learning, and decision-making, depending on whether they are immediately impactful or have delayed effects on students and society. A conventionally accepted view that teacher-led classroom assessment is generally low-stakes has been questioned. There is a case to suggest that the use of teacher assessment to enhance learning is educationally high-stakes (see Reference Black and WiliamBlack and Wiliam, 1998, Reference Black and Wiliam2009, Reference Black and Wiliam2018; Reference James and GardnerJames, 2006; Reference Leung and KunnanLeung, 2014; Reference Lewkowicz and LeungLewkowicz and Leung, 2021; Reference Stobart and GardnerStobart, 2006).

This Element presents and discusses other formal non-test tasks for evaluating students’ learning (e.g., assigned coursework, project-based assessment, portfolios). These forms of classroom assessment have gained acceptability in recent years because they can fully capture students’ overall learning and development across time points. The use of high or low diverse assessment frameworks and practices in different contexts suggests a more fluid educational view of assessment impact than the narrower view of high or low stakes traditionally associated with language assessments in English as a foreign/second language contexts. The movement towards balanced types of assessment suggests that teachers need to develop and use a range of learning-oriented and standardised assessments in their educational context.

2.3 Purposes and Functions of Assessment

We have thus far provided a broad account of assessment in language education. We will now look at some major functions that language assessment can serve.

1. 1. Formative function – to enable students to achieve or attain the target learning objectives. The verb ‘enable’ suggests that the focus of FA or assessment for learning (AfL) is on supporting the achievement of learning targets. When conducting FA, language teachers devise or use various assessment and learning activities to help students develop their knowledge and skills relevant to the learning outcomes. Thus, FA informs teaching and learning. It is designed to help students overcome difficulties (e.g., when they cannot recognise their mistakes, errors, or problems on their own). Teachers’ feedback and guidance can help them develop further learning and problem-solving skills or promote positive attitudes towards learning and self-regulation.

2. 2. Summative function – to assess learning achievement or attainment levels at the end of a learning period, whether it is a lesson, a term, or a course. This kind of assessment includes achievement tests, end-of-year examinations, and a class quiz led by the teacher (to check what has been learnt in the lesson). Periodic quizzes may also check student achievement or attainment level as students progress through various lessons. Results of SA can be used to decide whether, how much, or how well students can use the language according to the learning objectives and/or formatively to determine whether students have learnt the specified syllabus content and how best to support their further learning to ensure that they have fulfilled the learning outcomes. Grades or final scores derived from SA can be considered quantitative information indicating students’ attainment or mastery relative to the target learning outcomes (discussed further in Section 4).

3. 3. Diagnostic function – to diagnose students’ readiness or preparedness. Diagnostic language assessment identifies students’ or individuals’ strengths and weaknesses regarding specified language abilities or skills. There are various ways in which diagnostic assessments are used. For instance, students can take a diagnostic language test that is relevant to what they will be studying. If the course is related to writing, error detection tasks (e.g., identifying errors in a sentence) may be used to determine whether students can notice grammatical errors. If the diagnostic assessment outcomes suggest they cannot, teachers can use the information to plan further learning support. The DELNA (Diagnostic English Language Needs Assessment) is an example of using assessment for diagnostic purposes for English as a second language (ESL) international students.

4. 4. Placement function – to place students in an appropriate programme or course. The outcomes of an assessment of language proficiency and/or ability can be used to assign students to an approrpiate class or programme. Such assessment often takes the form of a placement test. For example, it will be challenging for students with a beginner’s proficiency level to be placed in a more advanced language class that requires advanced reading. Therefore, knowing whether a student’s current language ability and skills are suitable for a given class is essential.

5. 5. Gatekeeping function – to admit student applicants into an academic or professional training programme. Admission or selection tests serve gatekeeping purposes. Usually, there are student quotas for an academic or training programme. Therefore, applicants’ scores may be rank-ordered before selection or admission. University and college entrance examinations are examples of admission tests (e.g., the National College Entrance Examination (NCEE) or China’s Gaokao).

Scan Me 4

Understanding Assessments | Different Types of Assessments | Kathleen Jasper (posted to YouTube by Kathleen Jasper)

https://bit.ly/487xx1I

3.1 Testing, Assessment, and Evaluation

The discussion of language assessment involves various related terms such as testing, assessment, and evaluation. These terms are associated with both FA and SA. Figure 1 presents the overlapping natures of testing, assessment, measurement, evaluation, and utilisation. This graphic representation suggests a set of complex and embedded concepts in language assessment in educational settings.

Figure 1 The complex and embedded concepts testing, assessment, and evaluation

3.1.3 Evaluation

Figure 1 shows that testing and assessment implicate evaluation – a process of judging the value or quality of information collected by tests, measures, or other assessment tasks. Evaluation is a broader concept than assessment (depicted in another larger circle). Evaluation requires language teachers to take a broader view of assessment outcomes and to interpret them with reference to other relevant issues. That is, it may require some logical inferencing and reasoning processes. For instance, teachers ask what scores or observed performance mean to students’ learning status.

Scan Me 5

The Principles of Testing and Assessment (posted to YouTube by Advance Consulting for Education)

https://bit.ly/41UG9Xr >

3.2 Language Ability and Assessment

In this section, we discuss the development of a theoretical framework informing an understanding of general language proficiency. Assessment has been used as a tool for social, educational, and political power and control (see Reference SpolskySpolsky, 1995, Reference Spolsky, Shohamy, Or and May2017; Reference KunnanKunnan, 2018, who provide a review of the history of language assessment). For instance, the Imperial Civil Service Examination in China is considered the first examination in the world to produce an administrative elite for governance, dating back to the Sui dynasty, 581–618 CE (Reference KunnanKunnan, 2018). The establishment of this prestigious public examination is a very good example of how assessment can be built into the fabric of the sociopolitical affairs of a society.

In the anglophone academy, the book Language Testing by Reference LadoLado (1961) took a primarily structural view of language in which underlying linguistic systems and rules such as vocabulary and grammar are considered significant in effective language communication; thus, it formed a crucial part of language testing. Reference LadoLado’s (1961) book was a milestone in testing and language assessment because it helped formalise language testing as a discipline within applied linguistics (see Reference Read, Paltridge and PhakitiRead, 2015; Reference SpolskySpolsky, 1995). In the past forty years or so, the communicative model of language has been prominent (see Section 2.2.1). Further, LTA research in the English language has made invaluable contributions to many aspects of language education (e.g., Reference Fulcher and HardingFulcher & Harding, 2022; Reference Winke and BrunfautWinke & Brunfaut, 2021). The following subsections discuss the related theoretical concepts influencing our understanding of language proficiency.

3.2.1 General Language Proficiency

Researchers and scholars of LTA have long been developing a theoretical concept of language proficiency that can explain what and how people know and use language across various contexts and situations (see Reference Bachman and PalmerBachman & Palmer, 2010; Reference Jeon and In’namiJeon & In’nami, 2022; Reference McNamaraMcNamara, 1996). Language proficiency is a general language ability independent of specific classroom learning or teaching. It may reflect accumulative knowledge and skills acquired through classroom and informal learning. Language proficiency tests assess what individuals can do with a particular language without referencing a specific language syllabus. Language proficiency can be divided into levels, for example beginner, pre-intermediate, intermediate, and advanced. Students at different levels can be described in terms of what they can do with language and how well they can or cannot do it. It is important to know that language proficiency tests are traditionally psychometrically oriented and standardised.

Until the 1960s, LTA was based on a theory that language use behaviours were underlined by human mental structures manifested through the linguistic system, which at the time was understood to be related to lexico-grammatical features (e.g., vocabulary and grammatical knowledge). Hence, language assessment research at the time developed and used test techniques such as multiple-choice, cloze, and short-answer formats for assessing lexico-grammatical features of the linguistic system (see Reference LeungLeung, 2022b; Reference McNamaraMcNamara, 1996; Reference PurpuraPurpura, 1999).

That said, research in the 1970s onwards started to suggest that language proficiency was multi-componential in that it was made up of more than lexico-grammatical knowledge (see Reference BachmanBachman, 1990; Reference Bachman and PalmerBachman & Palmer, 1996, Reference Bachman and Palmer2010; Reference Canale and OllerCanale, 1983; Reference Canale and SwainCanale & Swain, 1980; Reference Jeon and In’namiJeon & In’nami, 2022; Reference SavignonSavignon, 1972, Reference Savignon1983). Figure 2 provides a sociocognitive model of language proficiency (adapted from Reference Bachman and PalmerBachman & Palmer, 2010), encompassing cognitive and social dimensions.

Figure 2 Model of language proficiency

(adapted from Reference Bachman and PalmerBachman and Palmer, 2010)

The model suggests that various internal linguistic processes interact in language use or learning. In Figure 2, areas within the oval represent internal mental processing and correspond with language tasks and settings. Language proficiency comprises language knowledge and strategic competence (Reference Bachman and PalmerBachman & Palmer, 2010), both of which have a social dimension.

• Language knowledge is concerned with knowing and understanding the formal rule-governed structure of language regarding grammatical and vocabulary (also referred to as lexis formally), pronunciation, and conventions of language use in context, including a social and cultural dimension (see also Reference BachmanBachman, 1990). Various aspects of language knowledge are highly connected (e.g., ‘went’ is a lexical item (word) and the grammatical past tense for ‘go’). While various linguistic features are conceptually distinctive (e.g., vocabulary versus grammar), they are not easily separated in language use. In the contemporary language assessment literature, a term such as lexico-grammatical knowledge is preferred for describing the knowledge of rule-governed structures.

• Strategic competence is the ability to employ cognitive and metacognitive knowledge and regulation (including practical know how of language use in context), such as working out what things to say and how to say them to best express one’s meaning in context (Reference Bachman and PalmerBachman & Palmer, 2010). Students use various strategies, such as cognitive and metacognitive strategies, to help them process information and complete given tasks (see also Reference OxfordOxford, 2017). According to Reference Bachman and PalmerBachman and Palmer (2010), strategic competence mediates the language knowledge and non-linguistic factors of a given person, such as personal characteristics, as well as knowledge of the world (or topical knowledge) and affective schemata (e.g., emotions and feelings). In Figure 2, a double-headed arrow links language proficiency to non-linguistic factors. Our current understanding is that strategic competence is not only a cognitive aspect of language use control; it also has a social dimension in that particular ways of thought regulation are shaped by a specific social and cultural context.

In Figure 2, language use, observed performance, and scores at the top of the model are seen as the results of interactions between a given person’s language proficiency and the context (i.e., language tasks and settings). This multi-componential view of general language proficiency has had a significant methodological influence on how people’s language proficiency is tested or assessed. This view underpins the design of test items requiring students to produce their responses communicatively. This orientation is also reflected by the scoring rubrics in language proficiency tests such as TOEFL and IELTS that focus on assessing test-takers’ vocabulary range, pragmatic appropriacy, task fulfilment, and grammatical accuracy.

Scan Me 6

Assessing General Language Proficiency (posted to YouTube by British Council Aptis Official)

https://bit.ly/3IPcfvd

In language assessment practice, it is often necessary to gather evidence of students’ specific-purpose language proficiency in a given language within a specific target language use (TLU) domain or situation. Specialist assessment designers can combine features of general proficiency and specific-purpose proficiency to meet the requirements of different disciplines and professions (Reference DouglasDouglas, 2000; Reference Moder, Halleck, Fulcher and HardingModer & Halleck, 2022). An example of a specific-purpose language domain in language teaching is academic writing for engineering. The idea of specific-purpose language proficiency can be useful outside the language classroom. For example, the Occupational English Test (OET) for medical professionals, the Objective Structured Clinical Exam (OSCE), and the Aviation English Language Proficiency Test are specific-purpose language proficiency tests for professional certification and accreditation (see Reference Elder and McNamaraElder & McNamara, 2016).

3.2.2 Language-Using Skills

Language-using skills typically refer to four distinct modes: listening, speaking, reading, and writing. These are also grouped into receptive and productive modes. Receptive modes comprise reading and listening skills (decoding and making sense of information), whereas speaking and writing skills are productive modes (encoding and creating information). Language-using skills can be related to general or specific-purpose language proficiency.

Although language use is naturally multimodal (meaning that it requires multiple skills or modes to operate or make meaning), in FA and SA situations, test items and assessment tasks are usually designed to elicit performance that can provide information on using a particular language skill. There are also practical reasons to assess one skill at a time: it is easier to design, administer, and interpret results. Nonetheless, a recent trend has been to use integrated language tasks that reflect real-life situations, such as in an academic or professional setting (discussed further in Section 4.2.2). We should not automatically assume that all language proficiency tests would cover all four skills equally; much would depend on the purpose of the test concerned.

The following provides an overview of each language-using skill from the perspective of an individual language user.

• Reading skills refer to the ability to process and understand the meaning of written texts (see Reference AldersonAlderson, 2000; Reference Brunfaut, Fulcher and HardingBrunfaut, 2022; Reference GrabeGrabe, 2009). Reading comprehension is derived from the reader’s construction and interpretation of meaning from a text through language processing and activation of their prior world knowledge.

• Listening skills refer to processing and understanding audio and spoken texts and obtaining information (see Reference FieldField, 2008; Reference RostRost, 2016). Unlike reading a written text where the reader can move back and forth, spoken language is temporally streamed, making retrieving what has been missed more difficult.

• Speaking skills refer to the ability to process and produce coherent speech that is meaningful and appropriate for a given purpose in a given context (see Reference FulcherFulcher, 2014; Reference LuomaLuoma, 2004). Speaking can be one-way (e.g., reading aloud, talks, announcements) or two-way (e.g., conversations, interviews).

• Writing skills refer to the ability to process and produce ideas or information using a given writing system (e.g., alphabets and letters, word orders, punctuations, syntax) to convey meaning (see Reference HylandHyland, 2016; Reference WeigleWeigle, 2002).

Tables 1 and 2 are examples of language-using skills that underpin constructs in language assessment (see also Reference Brown and AbeywickramaBrown & Abeywickrama, 2019; Reference GreenGreen, 2020; Reference HughesHughes, 2003). We will focus on linguistic and discourse components for the purpose at hand.

Table 1 Reading and listening skills (in alphabetical order) of the verbal processes involved

Reading skills	Listening skills
Adjust reading rates to suit a reading purpose and context.	Distinguish main ideas from supporting details or examples.
Identify main ideas and the author’s purposes.	Identify and retain specific details.
Infer what writers imply.	Identify main ideas and the speaker’s purpose, tone, attitude, or intention.
Predict the content from an introduction paragraph.	Infer the speaker’s implied speech or meaning in context.
Recognise spellings and word meanings in context.	Predict what speakers may say next based on the background information.
Recognise types of text and genre.	Process incoming speech at an efficient rate.
Search for and recall specific details.	Recognise discourse markers and cohesive devices in spoken texts.
Skim and scan for general and specific information.	Recognise grammatical functions of words (e.g., nouns, pronouns, verbs, prepositions) and tenses in speech.
Understand discourse markers and cohesive devices in written texts.	Recognise speech sounds (e.g., stressed patterns and reduced forms) and meanings in utterances, and retain them in working memory temporarily.
Understand grammatical functions of words (e.g., nouns, pronouns, verbs, prepositions) and recognise tenses in written texts.	Search for general or specific information.
Use headings, paragraphs, and enhanced texts (e.g., bold, italicised) to assist reading.	Understand spoken discourse meaning (e.g., monologue, dialogue) using real-world knowledge.
Use prior knowledge and/or experiences to relate to the topic and aid reading comprehension.	Use non-verbal clues and body language to make meaningful interpretations.

Table 2 Speaking and writing skills (in alphabetical order) of the verbal processes involved

Speaking skills	Writing skills
Apply pragmatic knowledge such as speech acts and implicatures, as well as conversation rules and social skills (e.g., politeness, turn-taking, and cross-cultural knowledge).	Apply appropriate writing rhetorical forms and conventions (e.g., form completing and personal, professional, and academic writing).
Connect topics, ideas, or arguments to a given situation or event (e.g., use of signposts, conjunctions, and cohesive devices).
Differentiate and pronounce individual words or sounds intelligibly.	Construct phrases, clauses, and sentences appropriately based on the grammatical system.
Express meaning and speaking intention clearly for a given audience.	Express meaning and writing aim clearly for a given audience.
Produce correct and intelligible stress patterns and intonation contours.	Spell individual words correctly (i.e., spelling).
Produce fluent (e.g., smooth) and natural-sounding speech.	Use appropriate cohesive devices to enhance textual clarity.
Use cohesive devices (e.g., adverbs and pronouns) appropriately.	Use correct subject–verb agreements, function words, word orders, pronouns, prepositions, articles, and so on.
Use correct subject–verb agreements, function words, word orders, pronouns, prepositions, articles, and so on.	Write at an appropriate pace in context.

Scan Me 7

Productive and Receptive Skills (posted to YouTube by ITTT International TEFL & TESOL Training)

https://bit.ly/4afk4qA

3.2.5 Multilingualism and Language Proficiency

Until recently, the concept of language proficiency used in LTA was modelled on what native speakers of a given language could do when they communicated (see Reference McNamaraMcNamara, 1996). Such modelling is based on a monoglossic ideology underpinning a language proficiency view that idealises the notion of native-speaker competence as the gold standard for learners of English. It follows that language tests and assessments based on such conceptual assumptions have tended to norm the TLU on that of native speakers.

However, an idealised language proficiency model requires some revisions on some theoretical and methodical grounds. For instance, there are many native-speaker varieties of English (e.g., American, British, Australian, Singaporean, and Indian Englishes). Furthermore, a one-standard approach in multicultural, multilingual, and globalised societies can be problematic in LTA. Especially in a localised context, there can be different models and acceptable norms of language proficiency. Many individuals in ethnolinguistically diverse communities routinely communicate with one another through their multilingual repertoire without regard for idealised native-speaker models. Language communication tends to focus more on comprehensibility than on observance of putative native-speaker norms.

Teachers can benefit from understanding the contemporary notion of fluid/flexible multilingualismFootnote ¹ broadly discussed in language education and applied linguistics. Multilingualism acknowledges the coexistence of different languages within individuals in and across societies. In the contemporary language assessment literature, assessing multilingual constructs has been increasingly considered, but it is challenging to accomplish well (e.g., Reference Chalhoub-DevilleChalhoub-Deville, 2019; Reference Schissel, Leung and Chalhoub-DevilleSchissel et al., 2019). Reference Chalhoub-DevilleChalhoub-Deville (2019) acknowledged that multilingual constructs could be more problematic for test designers to assess. For example, how should the different languages used by a speaker in a multilingual social interaction be recognised and scored?

Scan Me 8

Embracing Multilingualism and Eradicating Linguistic Bias | Karen Leung | TEDxWWU (posted to YouTube by TEDx Talks)

https://bit.ly/3GBSI01

An implication for recognising multilingualism in language teaching is that teachers can appreciate the roles of multilingualism in classroom teaching, learning, and assessment. A traditional focus of language teaching and assessing has been on helping students learn and acquire an idealised native-speaker-like language proficiency (as seen by an emphasis on fluency, accuracy, and appropriacy in using a given language in language teaching curriculums and tests). Although relevant research has shown that teachers’ and students’ own languages (other than English) are used in teaching and learning activities, assessment of students’ achievement still focusses only on their ability to use the target language accurately, fluently, and appropriately. An ideal native-speaker proficiency is inadequate and inappropriate to capture multilinguals’ language ability and performance in social interaction (for further discussion, see Reference Leung, Little and FiguerasLeung, 2022a, Reference Leung2022b).

Reflection Box 2

What are challenges you often face when assessing students’ language learning?

Share your thoughts here: https://bit.ly/3ReJOKX..

4.1 Formative Assessment (FA)

Carrying out FA serves various educational functions and purposes, including:

1. 1. gauging the extent to which students are developing or have developed a body of knowledge and skills or abilities as defined by the target learning outcomes; and

2. 2. observing student learning behaviours to inform teaching and learning and using teacher feedback to help students recognise the strengths and gaps in their learning (e.g., by pointing out good and weak aspects of performance) and feedforward to help students improve their future performance (e.g., how to avoid similar mistakes or errors in future similar tasks or situations, how to move towards the desired goal).

In the professional literature, the term AfL is also used to refer to assessment with a formative purpose. Generally, AfL is embedded within teaching and learning. It can range from being informal or ‘on the run’ (e.g., spontaneous or impromptu assessment of students’ current knowledge or understanding during classroom activities, such as teachers’ use of questions and clarification requests) to being formal (e.g., planned assessment activities, such as portfolios and project-based assessment). Teachers engage in AfL to ensure that students develop the required knowledge and skills.

As indicated earlier, Figure 3 shows that FA and SA are interconnected. We will elaborate on this point further here. For example, teachers need to understand the standards or learning outcomes assessed at the end of a learning period. Such understanding helps teachers observe students’ current learning status, thereby enabling teachers to provide helpful feedback and feedforward that will shape students’ overall achievement or attainment. Formative insights from in-class SA can yield information for practical teaching plans and material development. We, hence, argue that teachers’ SA is directly related to FA in classroom contexts. In language teaching, assessment of learning (AoL) can be teacher-made assessments or tasks and standardised assessments by others (e.g., by the department, school, university, and international testing companies).

Scan Me 9

Formative assessment in the classroom (Posted to YouTube by AITSL)

https://bit.ly/3RVO1nj

4.2 Summative Assessment (SA)

Using SA has long been integral to formal education. The term summative implies that assessment focusses on collecting cumulative evidence of students’ knowledge, skills, or abilities derived from the teaching and learning activities and engagement at the end of the semester or the teaching and learning period. In a broad sense, SA and AoL are synonymous; they provide accountability for student learning (e.g., grades in academic transcripts and institutional certificates). Official requirements for AoL are also known as mandates, which are the frameworks that regulate classroom assessment regimes (Reference Davidson and LynchDavidson & Lynch, 2002).

Use of SA is associated with a given syllabus’s standards or learning outcomes. Standards are the pre-specified criteria of language ability, skills, or learning outcomes that determine what counts as success (or not). Standards-based assessment, such as that provided by the state or by government agencies, is also considered summative. See Reference Brown and AbeywickramaBrown and Abeywickrama (2019), Reference Harsch, Malone, Winke and BrunfaltHarsch and Malone (2021), and Reference PophamPopham (2017) for a detailed discussion of summative and standards-based assessments. Although the boundaries between AoL and standards-based assessment can sometimes be blurry since both involve standards, in practical terms AoL tends to be closely associated with a specific learning point. Standards-based assessments tend to be associated with external mandatory targets or requirements.

4.2.1 Assessment of Learning (AoL)

Traditionally, AoL seeks to determine whether students have met the desired learning objectives of a course. It does not primarily focus on providing feedback for learning or improving learning processes. Typically, AoL is a component of a language syllabus design (e.g., when and how to assess, what kinds of assessment tasks, and weights of assessment tasks). Generally, the academic committee that approves a proposed syllabus also checks the information about the assessment tasks.

In principle, AoL focusses on evaluating the outcomes of students’ learning that are compared against the set of learning outcomes or defined standards that underly a language syllabus or curriculum, teaching and learning activities, and AoL (e.g., midterm and final examinations). An achievement test that collects summative information on students’ learning attainment at the end of a course is also considered to be AoL. It is important to note that AoL does not always rely on midterm and final examinations in contemporary education. Classroom-based SA can combine traditional (e.g., tests and quizzes) and process-oriented assessments (e.g., take-home tests, assignments, group work, and portfolio submissions). The standards for reading skills for Level 1 on the ESL scales used in Australia for English as an additional language/dialect (EAL/D) learners can be used to exemplify AoL activities:

Standard 1

1. 1. Students can obtain meaning from reading a short, simple text.

2. 2. Students can use basic familiar vocabulary, sounds, and sentence structures to aid their reading comprehension.

3. 3. Students can independently use their classroom reading experiences to complete similar reading tasks and activities.

After AoL data have been evaluated, students are awarded scores or grades that best describe their level of the prescribed standards. In AoL, students’ performance should be assessed against the standards (i.e., criteria or benchmarks) rather than being compared with their peers. This approach is called criterion-referenced approach (CRA).

Alignment is an important aspect of AoL. Since all forms of assessment can impact student learning quality in various ways, alignments among the target learning outcomes, teaching and learning activities, and assessment are essential. Alignments can warrant assessment relevance and promote fairness in assessment. We illustrate an example of such alignments in Figure 4.

Figure 4 Alignments among learning outcomes, teaching and learning, and AoL

The basics of alignments are that, first, teaching and learning in the classroom should be relevant to the learning outcomes to ensure that students develop their language repertoires in association with the learning outcomes. Second, assessment should be aligned with the learning outcomes and the teaching and learning activities. Figure 5 shows the alignments among learning outcomes, teaching and learning activities, and assessment tasks.

Figure 5 Detailed example of alignments among learning objectives, teaching and learning, and assessment tasks in a pre-intermediate reading unit of study

(based on NSW Department of Education and Training, 2004)

4.2.2 Standards-Based Assessment

A hallmark of large-scale standards-based assessment is standardisation of content and process. In standards-based assessment, the design, administrative, and scoring procedures strictly follow predetermined steps and requirements (e.g., time allowance, proctoring procedures, and scoring methods). It is a procedure that requires all students or test-takers to complete assessment tasks within a strictly observed administrative procedure.

We denote standards-based assessments as those implemented by external agencies or authorities on a system-wide basis, for example a suite of nationwide school leaving examinations. Government standards or benchmarks largely influence school curriculums, syllabuses, and SAs. Influential standards-based tests include nation- or state-wide standards-based examinations at statutorily specified times (e.g., by ministries of education). Officially mandated standards-based assessments can have government funding implications for schools according to students’ performance, and some teachers may be rewarded or penalised depending on students’ performance. Governments use such external standards-based assessments to identify, for instance, the level of national literacy and skills required to help determine their country’s international economic competitiveness and inform education reform policies.

Other examples of external standards-based frameworks are the American Council on the Teaching of Foreign Languages (ACTFL) Proficiency Guidelines, the Canadian Language Benchmarks (CLB) for ESL, the CEFR, the EAL Assessment Framework for Schools in the UK (published by the Bell Foundation), and the OECD Programme for International Student Assessment (PISA). In Australia, for example, NAPLAN (National Assessment Program – Literacy and Numeracy) is an example of an annual standards-based assessment for Years 3, 5, 7, and 9.

Scan Me 10

NAPLAN Nightmare: National Test Results Show The School System Is Going Backwards (posted to YouTube by The Project)

https://bit.ly/41iCIti

Table 3 is an example of an ESL scale in Australia for describing progressions of EAL/D students in different standard phases (e.g., emerging and developing). Table 3 shows some overlapping and transitional content at each competency level.

Table 3 Example of the ESL standards (emerging and developing phases) for EAL/D students in Years 3–6

Emerging level

Developing level

At the beginning of this phase, students demonstrate the ability to understand simple short texts with varying success. Students are able to: 1. develop linguistic knowledge of predictable English sound–symbol relationships and some common letter patterns 2. recognise common subject-specific words 3. comprehension 4. understand familiar prints around the classroom (e.g., posters and signs) 5. read their own writing correctly 6. read short texts with predictable structures and everyday language 7. read texts out loud, following the print from left to right and top to bottom, using appropriate pauses and intonation 8. understand some gist or main ideas in simple short texts 9. use their current spelling and pronunciation knowledge to attempt pronouncing new words. As they reach the end of this phase, they become more independent in their reading and realise the purpose of reading.

At the beginning of this phase, students demonstrate the ability to understand more complex and longer texts with a good success rate. Students are able to: 1. become more independent in their reading than in the emerging phase 2. when instructed, recognise common suffixes and prefixes, and use these to construct meaning (e.g., -ly for the adverbs of adjectives) 3. identify the gist or the main idea in most class texts correctly, and in more complex and lengthier texts that have predictable structures and contain familiar vocabulary, although they may still depend on some illustrations 4. identify characters in texts 5. search for specific factual information. As they reach the end of this phase, they are able to: 6. recognise various purposes of texts and reading comprehension 7. recall and retell sequences of events or stories accurately 8. make some inferences 9. adjust their reading rate to cope with available time allowance and tasks 10. read aloud most class texts correctly, with confidence and appropriate intonation (e.g., statements, questions, and dialogue).

(based on Australian Curriculum, Assessment and Reporting Authority (ACARA), 2014, pp. 9, 16)

4.2.3 Types of Decision-Making in SA

We have seen already that SA is used to decide on students’ achievement or success levels. Two types of decision-making approach are commonly used in assessment practices: the norm-referenced approach (NRA) and the CRA.

Norm-Referenced Approach (NRA)

The term ‘norm’ is derived from the normal distribution concept of test scores (bell-shaped distribution; see Figure 6). The normal distribution is a probabilistic frequency distribution of quantitative data or scores in which the average or mean score is used as the midpoint in the distribution (the 50 per cent point). The standard deviation (SD), a statistical measure that estimates the spread of scores in a given sample or population, is used to help establish the distribution curve.

Figure 6 The normal distribution

As illustrated in Figure 6, SDs are placed around the mean (average score). A minus sign appears before them when they are below the mean, but no plus sign is used when they are above the mean. Theoretically, this kind of score distribution suggests a ranking system. For example, students whose scores are below the mean score are considered below the average, whereas those above it are considered above the norm; how far below or above is defined by the values of SDs.

The perfect normal distribution of scores is symmetrical (mirror image) on the left and the right sides of the mean. When the normal distribution is considered, scores are placed along a continuum of the lowest and highest possible score. When there is a normal distribution, many students will likely be placed around the average score (approximately 68 per cent within ±1SD). To illustrate, if the mean score is 60 and the SD is 10, the scores between the mean and 1SD can range from 60 to 70 (making up approximately 34 per cent of all students), and the scores between the mean and –1SD can range from 50 to 60 (also making up around 34 per cent of all students).

However, a perfect normal distribution (i.e., symmetrical) is unlikely in a dataset because the normal distribution is affected by sample size and the heterogeneity of students’ abilities (high, medium, and low), among other things. Through the normal distribution principle, NRA adopts a ranking system of students’ scores (e.g., the highest to the lowest) for decision-making on students. There are reasons for NRA to be used in many standardised standards-based assessment contexts, for example where students are compared with others in a given school across different states or provinces. The type of decision-making in the NRA is relative (see Reference BrownBrown, 2005). Scores are first ordered and calculated in percentiles. In statistics, a percentile of a given student indicates the proportion of students who score above or below them. For example, students with a percentile score of 90 perform better than 90 out of 100 but worse than 10 out of 100. In Australia, for example, students who have completed a two-year higher school certificate in most states are given an Australian Tertiary Admissions Rank (ATAR) score. Their grades and subjects are calculated to derive each student’s ATAR score, which is out of 100 (i.e., a percentile). Therefore, if a student’s ATAR is 95, it means that this student performs better than 95 per cent of the students (i.e., this student is in the top 5 per cent). Generally speaking, a high ATAR is specified for a competitive field of study (e.g., medicine) in a highly ranked university.

In summary, NRA is used in SA when limited places, quotas, or positions are available. It is also used in general or specific-purpose language proficiency tests in which test-takers must be identified as beginners, intermediate, or advanced, and so forth. In practice, even though scores may not be normally distributed, scores are rank-ordered, and those with the highest scores are selected.

Criterion-Referenced Approach (CRA)

An example of a CRA is a driving test. It is of no interest to the licensing authorities whether candidates are the best or the worst drivers among all candidates before they are given a driving license. The driving test officer has the official criteria and must determine whether a given candidate has met each of them (e.g., speed management and vehicle control). In the end, the decision is a yes/no decision. Candidates of a given day are not compared against one another for decision-making.

The driving test scenario can be applied to classroom-based SA. Students’ achievement or attainment performances are decided using the given learning outcomes (i.e., criteria). For example, in a reading lesson, if one objective is ‘students can identify the main idea in an expository text’, and if students can correctly answer the questions related to the main ideas in a reading test, then it can be said that those students have met the objective (i.e., passed). In practice, the cut or minimum expected scores decide whether students have met the criteria or standards (e.g., at least 70 per cent correct). In this example, students’ scores are not ranked as in the NRA. Accordingly, CRA focusses on matching or aligning students’ performance against the given standards or criteria (see also Reference FulcherFulcher, 2010). In other words, the CRA decision is absolute as it focusses on whether a student has achieved the stipulated benchmark, be it in the form of a single answer or a minimum number of acceptable answers/responses.

Of course, in many language courses, students are not necessarily given a pass or fail grade. There may be different letter grades that are associated with a range of percentage scores (e.g., 0−49 = F, 50−59 = D, 60−69 = C, 70−79 = B, 80−89 = A, and 90−100 = A+). The decision to award particular grades to students would depend on where their scores fall in the ranges. In this grading system, teachers should use percentages (rather than percentiles) to decide students’ achievement levels or grades. If all students have scores between 90 and 100, they should all receive A+.

Reflection Box 3

Should assessment for and of learning be norm-referenced? Why or why not?

Share your thoughts here: https://bit.ly/4aaPyhx.

5.1 Constructs in Assessment

Teachers may encounter the word construct when they read a test manual or technical report of a given test or assessment. Constructs represent abstract concepts of knowledge and skills. Language proficiency is an example of a construct because it is abstract. We cannot observe it directly, but we can make it ‘knowable’ through tests or assessments, allowing inferences and evaluations to be drawn from responses to test questions or tasks. Constructs such as language proficiency or skills are theorised to manifest observed language use or responses to test tasks for inferring the constructs of interest. For example, speaking fluency can be inferred by observing how students talk when they respond to a given task. Reading comprehension can be inferred by observing whether students answer comprehension questions correctly. Assessing what we cannot define is difficult because we do not know what we seek in students’ abilities or knowledge. Teachers may not use the term construct, but terms such as language skills and learning outcomes or objectives in their assessment would be part of the established professional terminologies.

Knowing what to assess is important for at least two reasons. The first is related to FA. When teachers employ FA during their teaching, they can be more aware of what they are looking for in their students (e.g., abstract ability, language skills) as they interact with classroom activities. For example, teachers can recognise students’ errors or inappropriate language use. Knowing what to assess allows teachers to effectively use their time and effort to gauge and support students’ learning. The second implication is related to SA. In AoL, test-writers and teachers can explicitly define what language knowledge or skills they aim to assess in their students. Explicit definitions can provide a checklist for evaluating questions or tasks that students will complete (e.g., quizzes, midterm and final test questions, and tasks for portfolios). By being explicit about what to assess, teachers can check whether their assessment constructs are aligned with classroom tasks and activities. Likewise, test-writers can check the alignment between test design and curriculum and/or syllabus specifications.

Scan Me 11

CEFR and Language Assessment (posted to YouTube by British Council Aptis Official)

https://bit.ly/41cNktX

5.2 Ways to Elicit Performance (as Evidence of Learning/Knowing)

This section elaborates on fundamental considerations when teachers elicit or collect evidence of student learning or performance (as specified in the learning outcomes). Suppose teachers aim to know whether students can orally tell others about themselves (e.g., their names, hobbies, or interests) after a lesson. In that case, they can design a paired task where students exchange information about themselves. This method seems appropriate and relevant. If, however, students are asked to complete missing gaps in a written conversation dialogue, their speaking is not appropriately assessed. In this situation, their performance also depends on reading comprehension and writing skills.

Language assessment professionals have developed and fine-tuned the use of test or assessment techniques for assessing language skills or components (see Reference Brown and AbeywickramaBrown & Abeywickrama, 2019; Reference GreenGreen, 2020; Reference Coombe and LiontasCoombe, 2018; Reference WeirWeir, 1990, Reference Weir2005; Reference Winke and BrunfautWinke & Brunfaut, 2021). This section considers three (interrelated) methods for eliciting students’ performance.

5.3 Classical Test Theory (CTT)

Developed in the early to mid-twentieth century, CTT is related to a body of psychometric theory in ensuring accuracy and reliability in test scores (see Reference Brown, Fulcher and HardingBrown, 2022; Reference Sawaki and KunnanSawaki, 2014, for detailed discussion). Reference Brown, Fulcher and HardingBrown (2022) argues out that CTT helps teachers understand abstract testing and assessment concepts. Constructs, for instance, are abstract, so it is plausible that errors can occur in an assessment process. Test reliability and error in testing and assessment are well-conceptualised within CTT. In a nutshell, CTT is a measurement theory that focusses on the accuracy and consistency of collecting relevant and sufficient information about an aspect or construct of interest. It aims to connect an observation (e.g., as represented by performance criteria and/or test scores) to an underlying construct of interest.

Reference Brown, Fulcher and HardingBrown (2022) argues that it is a misconception that CTT is a thing of the past or an old-fashioned theory that is no longer relevant to language assessment. He asserts that CTT is still relevant to many practices in language assessment and is widely used, especially in educational and professional settings. He suggests that some may perceive it to be old-fashioned because it has been overshadowed by more sophisticated alternative measurement theories, such as the Item Response Theory (IRT) and the Generalisability Theory (G-theory) used by assessment researchers and test companies (e.g., Educational Testing Services, Pearson Education).

One of CTT’s core assumptions is that a test score comprises what we are looking for plus some irrelevant information (labelled as errors) that can interfere with or give a false understanding of students’ ability, knowledge, or attainment, for example. We summarise this concept as follows:

According to this principle, the first component of a test score is a true score that is explained by what we aim to assess (e.g., the underlying construct(s)) by asking students to complete test tasks or questions. For example, we seek to know whether students can give directions using a map. In that case, their scores should represent their ability to use appropriate words, comprehensible expressions, and specific information relevant to the map. As can be seen from this example, the ability we are interested in includes grammatical and lexical knowledge, pronunciations, social skills (e.g., turn-taking, politeness), and knowledge of how to read a map. This is the part of scores we often think of as a true score – what underlies a given score.

Scan Me 12

Classical Test Theory and Operationalization (posted to YouTube by Sprightly Pedagogue)

https://bit.ly/3PykZcG

Hypothetically, if a score comprises a true score plus zero error (true score + 0 error), a test or assessment has captured 100 per cent of the underlying construct. However, it is unlikely that a test score is made up only of a true score since various interfering factors can impact students’ performance. Therefore, in CTT, the products of interfering factors or conditions are known as ‘errors’ or an ‘error score’. Now let us return to the example of the map-reading and giving direction task and think about possible errors that may affect a score. We will use two scenarios to explore plausible errors:

Scenario 1: Jay is generally good at speaking and is familiar with giving directions to places using a map from the classroom activities. She is short-sighted but forgot to bring her glasses to the test room that day. She also had her wisdom tooth removed two days before the test. Consequently, in this test, she could not see the map clearly (as it was also relatively small and in black and white), and she felt uncomfortable when speaking owing to some pain from the dental surgery. She had to guess some street names and places on the map. Jay felt upset and worried during this test because she could not pronounce words intelligibly. Jay subsequently got a low score owing to her poor performance. Is Jay’s score an accurate picture of her ability? What could be the errors in this test? What might be a problem with using the current score to decide her speaking ability?

Scenario 2: Mike is an average speaker and happened to know what map was to be used in the test beforehand. Therefore, he memorised street names and places and practised giving various directions based on the map several times. Mike produced an impressive speaking performance with fluency and accuracy in this test. Subsequently, Mike received a top score. Is Mike’s score an accurate picture of his ability? What could be the errors in this test? What might be a problem with using the current score to decide his speaking ability?

As can be seen in Scenario 1, some factors other than the target ability influence Jay’s performance (e.g., no eyeglasses, a small map, an inability to speak comfortably, and the associated effects they have on Jay, such as causing her to become upset and worried). Because Jay’s ability is high, her test score is not representative of her ability and, hence, the observed score contains many errors. In Scenario 2, we do not know how much Mike’s score represents his ability because he was familiar with the map used and had memorised what to say. We might know more by asking him to do the task with a map he has never seen and finding out whether he performs similarly well. Nonetheless, we know that his current score cannot represent his ability. Should we construe Mike’s background knowledge and familiarity with the test task as an interfering factor contributing to the measurement error?

In CTT, an error score is theorised to be contributed by factors other than the construct(s) or aspect(s) of language knowledge and ability of interest. Such known factors include testing or assessment methods, administrative and scoring conditions, and students’ factors such as current illness and fatigue (see also Reference Bachman and PalmerBachman & Palmer, 1996; Reference BrownBrown, 2005 for further discussion). Our hypothetical examples of Jay and Mike suggest that the more the true score is included in an observed score, the more valuable the score can be when it comes to decision-making (Reference Brown and KunnanBrown, 2014).

The proponents of CTT have developed and used rigorous and robust systems in test design, standardised administration, and scoring to minimise errors. For example, a test should not rely only on one task or question because it cannot adequately sample students’ abilities. Therefore, CTT has a tradition of dissecting errors from true scores and improving a test or assessment that can eliminate systematic errors. The development of CTT, no doubt, has led to statistical analyses of test reliability such as internal consistency (with reference to parallel tests and items), item difficulty, discrimination of different ability levels (note the discussion of test discrimination as a test function is not related to racism or sexism), test–retest reliability, split–half reliability, and so on (see Reference BachmanBachman, 2004; Reference BrownBrown, 2005, Reference Brown, Fulcher and Harding2022; Reference LynchLynch, 2003; Reference Roever and PhakitiRoever & Phakiti, 2018; Reference Sawaki and KunnanSawaki, 2014, Reference Sawaki, Tsagari and Banerjee2017 for further details).

To understand CTT, it is also important not to confuse the notion of CTT errors with ‘mistakes’ that students make in assessment. In FA, teachers benefit from knowing students’ mistakes since they can provide corrective feedback and other support to help students correct their errors. Students’ mistakes are not the same as measurement errors in assessment (manifested through testing and test scores). For example, when a teacher asks her students to explain the grammar rule related to a communicative task they are learning, one student representative explains it correctly. Therefore, the teacher feels delighted that her students have got the concept right. However, in the exam, most students fail to correctly use the grammar rule in a similar task they had accomplished in the classroom. To the teacher, this is a shocking discovery. In this scenario, the teacher might have made an error in her generalisation about student learning: while the student representative might have correctly understood the grammatical concept, the rest of the class might not have. Therefore, more students with different ability levels should have been asked to avoid such errors. In this example, the teacher’s inaccurate understanding creates the illusion of student collective knowing. Such an observation error can be likened to an error score in CTT, leading to ineffective assessment.

Finally, CTT has limitations, for example its reliance on the use of raw scores to estimate errors, its assumption of item equivalence across the whole test (i.e., sum scores), and error of measurement being treated the same for all ability levels.

Understanding CTT is beneficial for teachers when they design a test or assessment task or use an already developed test; it makes them aware that a single observation is likely to result in a misleading conclusion about performance. Awareness of errors in assessment is relevant to tests and other non-test tasks such as portfolios and assigned coursework. For example, in portfolio assessment of young children, parents may assist their children in completing their portfolios. Therefore, the portfolio may include contributions from the parents involved. Furthermore, in another non-test condition, many unknown factors can contribute to errors in assessment. For example, students may use Chat Generative Pre-trained Transformer (ChatGPT) to help them write an essay that can be completed at home, but they cannot access any such assistance when they are asked to hand-write an essay in a controlled testing condition. Their writing scores can differ significantly under these different conditions.

6.1 Test Design

There can be situations in which teachers need to develop a test (e.g., midterm and final examinations, screening tests, admission tests). In such situations, teachers are assumed to be knowledgeable in creating test tasks or quizzes for their students. Their school or academic institute may ask them to design a test. The following design stages, as illustrated in Figure 7, can be seen as a road map for test development.

Figure 7 Five stages in test design

6.1.5 Stage 5 (Use)

Finally, assessment use should correspond to the intended purpose(s) identified earlier in the planning stage. We note that the reasoning underpinning this stage is applicable to both tests and non-test assessments. Once students’ responses are scored, teachers decide their success (e.g., pass/fail, admit/not admit, employ/not employ, and so on). In language education, decisions based on assessment can go beyond simply a pass-fail decision (e.g., awarding different grades). If there is a limited number of positions available, scores are typically ranked, and candidates with the highest test scores are generally considered for selection (as discussed in the NRA).

In summary, developing summative tests or assessments is a complex process. Typically, the development of standardised testing for large-scale assessment is not a task for a teacher working on their own; it requires teamwork and expert knowledge, with each team member responsible for leading a specific stage. To learn more about the stages discussed in this section, see Reference Bachman and DamböckBachman and Damböck (2017), Reference Davidson and LynchDavidson and Lynch (2002), and Reference GreenGreen (2020).

Scan Me 13

Test Development (posted to YouTube by British Council Aptis Official)

https://bit.ly/47wD2X9

6.1.6 Types of Test Techniques

This section presents examples of selected- and constructed-response techniques teachers may consider in their question (also known as ‘test item’) design (see Reference Brown and AbeywickramaBrown & Abeywickrama, 2019; Reference PurpuraPurpura, 2016; Reference WeirWeir, 1990, Reference Weir2005 for a comprehensive discussion of test and assessment techniques).

Selected-Response Techniques

In selected-response techniques, students are limited in how they can respond to questions or tasks. Such techniques include discrete-point techniques such as checklists, analogies, multiple-choice, true/false, and matching. These techniques are often used for assessing receptive language skills or linguistic knowledge (e.g., grammar and vocabulary) because the focus is often on comprehension, understanding, and/or identification. The following are selected examples of this technique.

1. a. A dichotomous technique (e.g., yes/no, true/false, A/B, present/absent) can be used to assess receptive skills but is unsuitable for assessing productive skills. A test using this technique is relatively easy to construct (e.g., ideas or information from the text can be presented as statements with prompts) and quick to score. In FA, teachers can use this technique to check students’ understanding in the classroom. In binary or dichotomous items, e.g., yes/no questions, students have a 50 per cent chance of being correct using blind guessing – a source of error in scores. They may answer incorrectly owing to flaws, a lack of clarity, or inaccuracy in the statement, and they may obtain the correct answer for the wrong reason or by merely picking randomly. Teachers can use FA techniques to overcome this by asking students to explain their answers. Providing grounds for answers is an extra task that requires reflexive thinking skills and additional writing requirements.

2. b. A multiple-choice technique is often adopted in language assessment. These items allow students to select an answer from three to five choices, although four may be the most common. The popularity of this technique is driven by the practical nature of its administration and scoring, especially in large-scale standardised assessments. This technique can test a broad range of reading and listening constructs or knowledge of productive skills such as grammatical, vocabulary, pronunciation, and pragmatic ability. Figure 8 illustrates an example of a multiple-choice reading comprehension test.

   As with the dichotomous techniques, students are asked to choose only one of the multiple options as the correct answer. It is essential to pre-test, edit, and check the difficulty levels and distractor functions for this type of test. The sole use of and reliance on this technique in a test has been criticised and discouraged (Reference AldersonAlderson, 2000). For instance, there is the possibility of guessing. Even when five-choice questions are used, the chance of being correct by guessing is 20 per cent, so the effect of guessing is not negligible. Furthermore, the choosing activities are unnatural in real-world language use (e.g., people are rarely given options in real-life conversations). It is, therefore, difficult to generalise high test scores to real-life language use success.

Figure 8 Example of a multiple-choice test

1. c. A checklist technique is used to ask students to select and produce a list of items of information based on a reading or listening text (e.g., check all that are relevant to the question). Checklists can assess scanning and skimming skills and the accuracy of information retrieval and retention. Figure 9 provides an example of a checklist technique.

Figure 9 Example of a test using a checklist technique

According to information given in the conversation between Thomas and Jane, put a tick (✓) next to what Thomas has planned to do on Saturday. There may more than one answer.

1. □ Go to the beach

2. □ Go to the dentist

3. □ Do some gardening

4. □ Get a haircut

5. □ Go shopping

Partial correctness can be used in scoring this technique, though this may result in a complex procedure. In FA, checklists are useful for checking students’ knowledge and understanding. For example, in FA of writing, a list can be used to remind students whether they have completed or engaged in some activities (e.g., ☑ I have checked subject–verb agreements; ☑ I have checked my spelling before submitting). Test instructions need to be precise because students may understand the written or spoken text but may be confused by the instructions provided. For example, some students may believe that they must choose just one item in the given list (as in multiple-choice tests) when they need to select more than one, and this leads to failure.

Constructed-Response Techniques

Constructed-response techniques ask students to produce answers or responses on their own. Compared to the selected-response techniques, these minimise the effect of guessing. These techniques can be used to elicit short or extended responses. Short constructed-response techniques are, for instance, short-answer questions (e.g., one word only, or no more than three words), cloze or gap-filling (e.g., one word per one missing space in a text), and diagram completion (e.g., no more than three words). These techniques are somewhat restricting and controlling. The following are examples of this technique in more detail:

1. a. A cloze technique is mainly used for assessing reading skills. Every nth word (e.g., every fifth, sixth, or seventh word) in a paragraph is deleted, and students must devise a word to replace it. This technique taps into students’ broad comprehension of the stimulus text. They have to work out what might be missing that could complete the meaning. When they complete the gap, they must pay attention to content meaning, grammatical features (e.g., tenses, verb forms), and word forms (e.g., plurals, nouns, adjectives, or adverbs). Figure 10 presents an example of a cloze technique in which every sixth word is removed.

   This technique can infer students’ ability to understand text coherence since a word that correctly fills a given space needs to be meaningful and fit logically with the remaining parts of the sentence or paragraph. This technique, hence, requires the integrative skills discussed in Section 4.2.2. The cloze technique can be combined with other techniques, such as multiple-choice or providing a list of words to select the correct answer from. This technique can give students corrective feedback in FA because they must notice their vocabulary, grammar, spelling, and reading comprehension. The cloze technique, however, is known to lack authenticity in the sense that in ordinary real-life reading, not every nth word is missing; therefore, assessing reading this way requires more than reading ability since this is also a problem-solving activity.

The cloze technique can be (1) __________ with other techniques, such as (2) __________ mixing it with a multiple-choice (3) __________ or by providing a list (4) __________ words from which students or (5) __________ can select the correct answer. (6) __________ combination makes test tasks easier (7) __________ the cloze format on its own.

Answers (based on the original text): (1) combined (2) by (3) technique (4) of (5) candidates (6) This (7) than

Figure 10 Example of a test using a cloze technique

1. b. A gap-filling technique can have different variants ranging from one word only to a few words. A one-word gap-filling technique is similar to a cloze technique, but a particular word is selected to be removed. That is, word deletion is motivated by assessment objectives, whereas the cloze technique is more format-driven (i.e., deleting every nth word) as the cloze technique. A gap-filling technique can assess reading and other language-using skills, such as listening, writing, and speaking, as well as grammatical knowledge. In listening, students can fill in a gap in a written text, diagram, or graph as they listen to the audio passage. This technique can be successfully employed in dictations in which students listen to an audio text and fill in the missing words according to what is heard. This is also known as partial dictation. In writing, they may complete a form or application. In speaking, they may complete a conversation (e.g., what the person will likely say after a question in a discourse completion task). Discourse completion tasks are indirect assessments of speaking. In a grammar test, they may complete a space that makes the sentence grammatically correct.

2. c. A short-answer technique is considered to be a semi-objective, constructed-response technique. Assessments using this technique involve students giving responses that are somewhat restricted. This technique can assess reading or listening comprehension skills (e.g., what, where, when, why, and how questions). The number of words for short answers can vary but should range between three and six in reading or listening tests. Long answers will shift students’ attention from listening to writing answers in listening assessment. The writing-up of ‘long’ answers is likely to be construct-irrelevant (i.e., the ability has nothing to do with listening comprehension skills). In some reading tests, students can write a sentence or two as their answers. Again, the instructions for students regarding the length of response required should be explicit and easy to understand to avoid misunderstanding, leading to failures.

   An information-transfer technique is a variant of the short constructed-response technique. This technique is usually linked to visual representations or mind maps of texts. In a reading or listening test, students read or listen to a text and complete a graph, diagram, chart, or table with missing information, for example.

   A short-answer technique can also be applied in speaking assessment. Students may complete a series of questions by responding to them verbally (e.g., What’s your name? Where do you come from? What is your favourite subject this semester?). Such a test inevitably involves listening skills since students need to be able to understand questions before they can answer them. In a pronunciation test, tasks that require reading aloud a series of words, phrases, or sentences may be considered a short-answer technique. In a writing test, a short-answer technique is exemplified by completing a form or application. It can also ask students to construct a sentence based on the provided words.

   The target skill or ability assessed through constructed-response techniques depends on other language and/or cognitive skills not directly assessed. For example, in a listening test, skills other than listening skills include reading comprehension and writing, as students need to read questions and options or write answers to questions.

Extended constructed-response techniques require students to produce lengthier answers or responses (e.g., essays, interviews, role plays, oral presentations). These techniques can assess productive skills such as writing and speaking.

1. a. A pictures or events description technique is an extended constructed-response technique. However, responses remain somewhat restricted because students can only speak or write what a series of sequentially ordered pictures or events suggests. At the same time, students need to construct their own spoken or written discourse as they describe the pictures or events. To achieve this task, students need to have sufficient vocabulary knowledge related to the photographs or events and the grammatical ability to construct their speech or sentences. This technique is suitable for specific-purpose tests. For example, a doctor may need to explain a series of medical procedures to colleagues or patients; a chef may explain the steps in which a particular dish is to be prepared; and a tourist guide can explain the history of a tourist destination or cultural artefact. Since all students describe the same pictures or events, scoring is less complicated than scoring for interview or role-play techniques. Figure 11 provides an example of a picture task for a speaking test.

   There are some cautions to keep in mind when using a picture or events task. For instance, students’ performance may depend on the quality of the pictures used and their content-specific knowledge. They may not interpret the pictures or events in the same way as the test designers, teachers, or essay markers. Furthermore, pictures may be culturally sensitive regardless of test designers’ best attempts to find culturally neutral scenes. A picture that puts test-takers in a negative frame of mind is unlikely to elicit their optimal performance. Finally, some students may be more creative than others when describing pictures or telling events. However, creativity may not be a construct of interest in the assessment.

Figure 11 A speaking task based on a picture

1. b. A dictation technique is an integrative constructed-response technique that requires students to listen to a read-aloud text and then write what they have heard verbatim. Students use listening, reading, and writing skills as well as memorisation (which may not be a relevant construct of interest that differentiates performance). Once the appropriateness of the content of the dictation passage has been checked, the difficulty levels are determined simply by deciding whether the text is read once or twice and whether the reading speed is slow, moderate, or fast. Partial dictation, which may involve students writing a phrase or sentences in specific areas in a given text, may be considered to reduce writing volume. Figure 12 is an example of a partial dictation technique.

   The feedback on performance involving dictation or partial dictation can be formative. A dictation test can help students improve their working memory, self-monitoring and self-assessment, and fluent recall of information. In scoring, issues such as spelling mistakes and other writing skills (e.g., capitalisation and tenses) should be considered and addressed.

Figure 12 Example of a partial dictation test

Instructions: You are to listen to a talk about the use of dictation tests to control immigration in Australia. You are to write down exactly what you have heard in the missing spaces. First, the text will be read once at a natural speed; second, the text will be read with a pause between sentences; third, the text will be read again at a natural speed. You should check your spelling and use of capital letters.

Test-takers hear:

The ‘White Australia Policy’ and the dictation test under which it was infamously enforced provided central policy tools in the quest to control Australia’s immigrant population from Federation in 1901 until well into the twentieth century. The dictation test, which was a key element of the Immigration Restriction Act 1901, has always been associated with the question of race.

It was administered to ‘coloureds’ and ‘Asians’ in order to have an apparently neutral reason to deport them. The last person to pass the test did so in 1909. It became foolproof, as it was designed to be. The applicant would be given the test in a language that their background firmly indicated they would not know and, upon failing, they would be told that the authorities could go on giving them tests in languages that they did not know, infinitely.

Test-takers see:

The ‘White Australia Policy’ and the dictation test under which it was infamously enforced provided central policy tools in the quest to (1) _________________________________________________________________________________________________________ until well into the twentieth century. The dictation test, which was a key element of the Immigration Restriction Act 1901, (2) ____________________________________________________________________________

____________________________________________________.

It was administered to ‘coloureds’ and ‘Asians’ in order to have an apparently neutral reason to deport them. (3) ______________________________________________________________. It became foolproof, as it was designed to be. The applicant would be given the test in a language that (4) _____________________________________________

__________________________________________________________ and, upon failing, they would be told that the authorities (5) ____________________________________________________________________.

Source: Adapted from Reference Robertson, Hohmann and StewartRobertson et al. (2005, pp. 241–2).

1. c. A summary technique can assess receptive and productive language skills. It assesses integrative skills. The focus, however, is on accuracy in expressing, summarising, or paraphrasing the main ideas, sequences, and essential information from a text. A summary technique can be combined with the dictation technique. Students read and summarise a text through writing or speaking. Instructions must ensure that students understand the requirement (e.g., Write no more than 50 words; Speak within 30 seconds after the beep sound.).

2. d. A free-verbal or free-written recall technique is a variant of the dictation and summary technique. This is an extended integrative technique for assessing reading, listening, speaking, or writing. In a reading or listening test, students read or listen to a text and then speak or write about what they have understood from their reading or listening. This type of assessment focusses on accurate comprehension of a given text. In a speaking assessment, in addition to correct recollections of information, the assessment can focus on the intelligibility of speech, pronunciation, and fluency. In writing tests, the assessment criteria can include content accuracy, spelling, grammatical accuracy, vocabulary use, and mechanics (e.g., punctuation). This technique can become an integrated task when students are required to add or relate their position to the topic and their examples or support in an essay or speech. This technique seems natural in the sense that there are no test questions that intervene in students’ receptive or productive language processes.

   In SA, performance is, however, dependent on task familiarity, practice, memory capacity, and the topics and the complexity of texts. Some students can comprehend a text well while reading or listening but may struggle to remember what they have just read or heard shortly after finishing reading or listening. In a reading or listening assessment, this technique may not reveal students’ natural ways of reading or listening because they do not usually need to tell people what and how much they can recall. The relevance of this technique for FA is that it helps students learn to use memory strategies, which are essential for enhancing language learning and use. Teachers can help students with feedback and memory strategy instructions. Students can practise recalling what they have read or heard, and they can start with simple texts, then move to more complex and lengthier texts.

3. e. An essay technique is a flexible constructed-response method for assessing writing. Generally, students are asked to independently write an essay responding to a statement or question (see integrated tasks in Section 4.2.2). Essay topics can range from personal topics, such as my last holiday, my hobby, or my family, to academic or social topics, such as issues in climate change, technology, education, or social issues. In EMI and CLIL settings, essays are most likely to be subject topic–related. Students are expected to state their position, provide support, reasons, and examples, and organise an essay based on what they have been taught (e.g., introduction–body–conclusion). As far as possible, it would be advisable to avoid offensive topics that may impact negatively on student performance. Instructions need to be explicit and transparent (e.g., the number of words students need to write, the time allowance, and the criteria used to judge their essays).

4. f. A role-play technique is used frequently for assessing speaking as it is adaptable for various student or test-taker levels. This technique is commonly used in standardised language proficiency tests and can be suitable for scenario-based assessment. It is versatile and communication-oriented to capture various speaking repertoires, from basic pronunciations to pragmatic and turn-taking skills. It is often used in specific-purpose assessments (e.g., business English, English for aviation and medical professionals). Students imagine themselves being in a specific role in a given scenario and use the target language to exchange information or solve a problem, for instance. A role-play technique can be carried out between an interviewer (examiner) and a student or test-taker, or it can be assigned in a pair-speaking task in which two students are responsible for a particular role. Similar to the essay technique, instructions must be clear regarding the criteria for evaluating performance and the preparation time and time for task completion.

5. g. Oral presentation: Oral presentation is a versatile and authentic technique for assessing speaking and communication skills and content knowledge. In professional and educational contexts (e.g., meetings, conferences, advertisements, product or service promotions), oral presentations are common. In classroom contexts, oral presentations can be part of a larger project or assessed coursework students have been working on. Students can present a topic with or without a PowerPoint presentation or other audiovisual or multimedia aids. Students need to understand task requirements and practise before their oral presentations.

   In some educational contexts, for example in some Italian universities, final degree exams can involve oral presentations to the professor, which is, in effect, a verbal essay on a topic. For example, there is an element of oral presentations in North American, New Zealand, and UK university practice for PhD viva voce. Such oral presentations typically include a question-and-answer session after the presentation. Oral presentations can also be included as part of paired or group work, as they can involve collaboration and leadership. That said, they can also be challenging for some students owing to a lack of language proficiency and collegiality with group members.

6.2 Non-test Assessment Design

Standardised summative tests follow specific stages in design, development, and administration. Therefore, while some considerations in test design can be applicable for non-test assessments (e.g., planning, designing, and evaluating), other considerations differ significantly because, typically, non-test assessment tasks can serve both formative and summative purposes. Unlike standardised tests focussing on individual students, non-test assessments can be conducted as pair or group tasks.

6.2.1 Contemporary Approaches to FA

This section explores four approaches that can inform FA practices: dynamic assessment, learning-oriented assessment, usage-based, and scenario-based assessments. In the language assessment literature, these assessment approaches have been discussed in relation to assessment for and of learning.

Dynamic Assessment

Dynamic assessment focusses on the development and processes of learning during teacher-guided activities. Generally speaking, it tends to focus on an individual student who is regarded unique in their language learning needs and their ability to tackle a learning task successfully. When students encounter difficulties, teachers provide scaffolding and feedback to support task engagement, considering the student’s current knowledge and ability. Guidance is fine-tuned to each student’s particular learning needs or difficulties with the learning task. The aim is not to spoon-feed students in terms of how to correct their errors but to provide advice on how to address problems, taking account of their learning dispositions. Teacher feedback in dynamic assessment should be premised on a clear view of the learning processes or steps involved. To achieve this, teachers have to analyse and understand the nature of learning involved in any pedagogic task. This feature makes dynamic assessment different from other kinds of formative support. The principles of dynamic assessment have been extended to cover whole class teaching contexts. See Reference Poehner, Infante, Tsagari and BanerjeePoehner and Infante (2017) for further discussion.

Scan Me 14

Dynamic Assessment for Speech (posted to YouTube by Bilinguistics – Speech Therapy)

https://bit.ly/47oUBIw

Learning-Oriented Assessment

Learning-oriented assessment (LoA) takes a learning approach in which the real-life educational environment (e.g., learning outcomes, mandatory internal and external assessments), cognition (e.g., thinking processes required to develop and learn the target language features), affect of students (e.g., feeling and emotion), and social contexts (classroom settings, teachers and peers) are considered (Reference CarlessCarless, 2007, Reference Carless2015; Reference Turner, Purpura, Tsagari and BanerjeeTurner & Purpura, 2017). Thus, LoA can involve a cycle of teaching, learning, and assessment both inside and outside the classroom, and it can use all forms of assessment, including both FA and SA. For example, it can begin with teachers using the learning outcomes or objectives to develop teaching, learning, and assessment activities (e.g., how to engage in small talk). Students are then introduced to the concept of small talk with real-life examples of small talk. They work individually, in pairs, or in groups to learn from the examples. After that, they are encouraged to personalise their own small talk. In-class activities include teachers monitoring student learning, tracking student progress, and providing feedback on their learning and performance (e.g., suggesting correct pronunciation or word choice). Self- and peer-assessment can also be promoted as students complete learning activities. Teachers can also use SA techniques, such as quizzes and achievement tests, and non-test assessment tasks, such as project-based and portfolio assessments, to help students realise real-life applications of small talk. Both FA and SA techniques can be aligned with the learning outcomes or objectives. In out-of-class activities, students are invited to engage in independent study projects (on topics and materials guided by teachers). They may be encouraged to use digital technology to conduct their studies multimodally. Teachers can also offer a follow-up discussion with students about their out-of-class learning during class time. The fundamental premise of LoA is that students should learn new skills or useful knowledge while participating in assessment activities. In LoA, targeted language-use scenarios are adopted to ensure that the learning activities meet students’ personal, educational, and social goals as appropriate. See Reference Chong and ReindersChong and Reinders (2023), Reference Jones and SavilleJones and Saville (2016), and Reference Turner, Purpura, Tsagari and BanerjeeTurner and Purpura (2017) for further discussion.

Scan Me 15

Learning Oriented Assessment: The different stages of the LOA cycle (posted to YouTube by English with Cambridge)

https://bit.ly/3SdbnG8

Usage-Based Assessment

Reference Ellis, O’Donnell, Römer, MacWhinney and O’GradyEllis et al. (2015) and Douglas Fir Group (2016) discuss a theoretical framework that describes and explains the nature of language use as language learning. This framework postulates that people recognise the function of language through how it is used. For example, frequently or repeatedly used language features in a given context are likely to be more noticed in terms of how and what they are used for than those infrequently used. This idea recognises that language use in a real-life activity is often patterned (see Reference Cadierno and EskildsenCadierno & Eskildsen, 2015; Reference Dolgova, Tyler and GaoDolgova & Tyler, 2019; Reference HallHall, 2019). For that reason, concepts in usage-based language learning have implications for language teaching and assessment in terms of what to teach and assess. For example, Thai students learning to use English in Thailand require different foci on content and activities from those studying English in England because the needs for and exposures to English usage are different. Therefore, usage-based assessment should be sensitive to local language use and learning.

Scenario-Based Assessment

Reference Purpura, Masperi, Cervini and BardièrePurpura (2021) provides a useful discussion of scenario-based assessment. A scenario is an imaginary language-use scene that is used to generate a real-life experience in which a student interacts with a task. The task may include a collaborative activity among students and teachers. Language use in a given scenario can be dynamic and flexible. Reference Purpura, Masperi, Cervini and BardièrePurpura (2021) operationalises scenario-based assessment through computer technology, although it can be carried out in an in-person classroom. Scenario-based assessment focusses on using real-life, interconnected tasks. For example, in academic language use, students are likely to engage in various tasks both individually and in groups. They later create a nexus of interconnected language use to accomplish a larger communicative goal. Students may read several texts about a topic of interest and watch a documentary or news about that topic, and so forth, to discuss main issues with other students and professors. Before that, they may summarise key messages from various resources through notes or texts. They may read their summaries aloud to rehearse their speech or check their intelligibility. This example shows that different language uses are integrated or combined to serve a meaningful overall purpose.

These four assessment approaches are relevant to effective classroom language teaching and assessment. They share three common concerns. First, it is vital to help students imagine a real-life language-use situation they will face and to relate their language learning and use to such a situation. Second, various multimodal learning and assessment tasks can be created as a pedagogic space (comprising tasks of increasing complexity) in which students can connect smaller pieces or segments to form a larger language-use goal for a specific purpose or scenario. Third, students can receive feedback on language use as part of their engagement with the assessment activities.

6.2.3 Examples of Non-test Assessment Tasks

The following provides an overview of non-test assessment tasks that have gained popularity in classroom assessments because they promote task authenticity, integrated language skills, application skills, and creativity. Students are not constrained by the extent of time pressure as they are in a standardised test condition.

Figure 13 Timeline for completing a piece of coursework

Portfolio Assessment

Portfolio assessment is a broad approach that focusses on a collection of students’ work over time (e.g., an assigned task that cannot be completed in standardised conditions). It can accommodate collections of separate or integrated language skills. It takes a learner-centred approach to assessment because it focusses on students’ ongoing learning processes and the outcomes of such processes. Therefore, students collect various drafts and outlines of their work that have been commented on for improvement, the materials they have used and produced, and their reflections as they completed the assessment task. Portfolios can be constructed in the form of physical folders or artefacts. E-portfolios are portfolios that embrace computer and information technology as well as cloud-based storage and sharing. E-portfolios have gained popularity as they are flexible regarding simultaneous and synchronous student–student or student–teacher communication (see Reference LamLam, 2023 for further discussion).

Figure 14 Assessment rubric

(based on an assessment rubric of an English for academic purposes unit of study)

Task-Based Assessment

Task-based assessment is directly related to a task-based language teaching approach, which focusses on helping students learn and use the target language communicatively in authentic language-use situations. Task-based teaching and learning is a communicative approach to language teaching and learning that focusses on fulfilling communicative tasks. It does not ignore the importance of raising awareness of relevant linguistic form that makes language use accurate and/or appropriate (i.e., focus on form). Task accomplishment tends to be integrative and integrated as students must combine various language skills for successful communication. Task-based language assessment can be both formative and summative as students can receive feedback on their performance to improve their communicative skills, contributing to overall learning attainment. Task-based assessment may be designed similarly to portfolio assessment, but students may not be required to submit all their drafts and other artefacts for evaluation. Instead, the focus may be on the outcome of fulfilling the task rather than on the detailed processes and development that led to the outcome, as in portfolio assessment.

Reflection Box 5

Is it necessary to plan an SA for a language classroom? Why or why not?

Share your thoughts here: https://bit.ly/46WJmXw.

7.1 Reliability (Revisited)

We have introduced the concept of reliability under CTT in regard to measurement errors. Reliability relates to how assessment practices and results are consistent in standardised tests. In SA, we can think of reliability as the level of consistency of students’ performance. When students complete the same test, they should perform similarly today, yesterday, and tomorrow. If students pass the test today but fail the same test tomorrow, then there is a problem with reliability. We cannot rely on this test for decision-making. In statistics, a reliability estimate of a test ranges from 0 (completely unreliable or random) to 1 (completely reliable). A reliability estimate of 0.90 is expected for tests used to make a high-stakes decision.

It is important to note that the concepts of reliability in CTT cannot be applied to some aspects of classroom assessment, such as learning-oriented, formative, and dynamic assessment. It is unreasonable to expect that students’ performances can be consistently measured because they are developing their knowledge and skills. Their performances can fluctuate over time and across contexts and tasks. Similarly, in portfolio assessment, much of students’ work can be in the form of drafts and revised drafts, so their performances cannot be expected to be consistent. Therefore, in learning-oriented or dynamic assessment, trustworthiness is more appropriately used. The notion of ‘trustworthiness’ in language assessment refers to the quality of assessment activities that can be dependable, useful, and relevant to teaching and learning activities.

7.2 Validity

In the mainstream language assessment literature, validity refers to the extent to which scores infer the target ability or construct and are used appropriately and ethically in decision-making (Reference ChapelleChapelle, 2021; Reference Chapelle, Lee, Fulcher and HardingChapelle & Lee, 2022; Reference Phakiti and IsaacsPhakiti & Isaacs, 2021). In this section, we explain three aspects of validity that are relevant for consideration in FA and SA: construct, content, and face validity.

1. 1. Construct validity: Constructs are theoretical and abstract concepts that cannot be observed directly. When we use scores to indicate students’ abilities or skills, we can ask whether the test or assessment accurately and appropriately captures the abstract abilities or skills that we claim to be testing or assessing. If so, what evidence do we have to support our claim? The same question can be applied to classroom assessment. The focus of assessment should always be aligned with assessment tasks, be they in the form of test questions or portfolios compiled by students.

2. 2. Content validity: We all know that examinations and tests are unusually concerned with a particular subject or area of knowledge and skills. So, the selection and the design of test/exam items (questions) have to tap into the focal construct. Content validity refers to the extent to which the tasks and activities involved in assessment are linked to the focal construct. Therefore, only sample tasks or activities relevant to and representative of the target constructs should be used. Therefore, content validity can be thought of as the extent to which the content of the test is relevant and sufficient to assess the target constructs of interest. By relevance, we mean that assessment tasks or questions must prompt students to produce the language related to the target constructs or abilities. For example, suppose teachers give students a multiple-choice grammar test and use the scores to infer the extent of their speaking skills. In that case, the test is both construct and content invalid simply because knowledge of grammar does not equate to speaking skills, and the tasks cannot elicit samples of the speaking skills. By sufficiency, we mean obtaining an adequate amount and quality of task responses from students to enable evaluation and making an evidence-based decision. This consideration applies to both standardised testing and classroom assessment. In classroom assessment, content validity can be checked by asking whether test tasks or questions are similar to what students have done in the classroom. If they are not, then there are issues with the content validity.

3. 3. Face validity: Generally, this refers to the appearance or perceived relevance of the assessment or test to stakeholders including teachers, students, university administrators, employers, and possibly even the general public. If a test or assessment task does not look as though it measures what it claims to measure, it does not have face validity. We used an example of a grammar test to claim about students’ speaking ability in Point (2), which lacks both construct and content validity. This is also an example of a test that lacks face validity. In addition, face validity can be considered in terms of assessment formats, sequences of tasks, and how fairness and test security are addressed. Although the primary focus of language assessment should be construct and content validity, it is undeniable that face validity is essential.

Scan Me 16

What every teacher needs to know about assessment (posted to YouTube by Dylan Wiliam)

https://bit.ly/3Nkw8gk

7.3 Practicality

In language assessment, ideally, we would like to collect as much language and language-related information as possible to have confidence in our decisions. Nonetheless, there is a need to consider whether such an expectation is realistic because assessment needs to be practical. The term practicality is related to the extent to which an assessment is feasible in a given context. In both FA and SA, teachers need to consider the cost, resources, and time required and constrained to develop, administer, and score or evaluate students’ responses.

The practicality considerations of FA differ from those of SA in various ways. For example, classroom AfL is ongoing and embedded in teaching and learning activities. The teaching and learning activities, whether in-person or online, are designed to lead to qualitative feedback to the students. Thus, AfL can be time-demanding for teachers and students. It may not be practical to give individualised feedback to each student. In SA, since the main assessment concern is mainly obtaining outcomes accurately and efficiently, test designers routinely consider how much time will be needed to develop a test or assessment task, how much time students will need to complete the test or assessment, and how much money will be spent on development and administration. Practicality considerations can also cover issues such as throughput efficiency (the number of students/test-takers per administration) and marking efficiency (use of machine marking or human raters). A lengthy and complex assessment can collect extensive evidence of learning, but it is more expensive and time-consuming to develop and administer. A shorter assessment may be time-efficient and affordable, but may not sufficiently collect students’ knowledge or skills of interest.

7.4 Ethics

The issues of ethics and fairness impact the quality aspects of testing and assessment. These concepts are intertwined (see the International Language Testing Association (ILTA) Guidelines for Practice, which comprehensively cover ethical assessment). Generally speaking, the term ‘ethics’ concerns the broad educational principles and social responsibilities that assessment developers, curriculum designers, teachers, and policymakers have to address. Ethics are at the heart of testing and assessment practice. Ethical considerations are part of the decision-making when teachers or test developers select test questions or assessment tasks for a given purpose and group of students, decide on the marking and reporting framework, and design protocols for safeguarding students’ personal and assessment data.

Teachers are expected to ensure that students learn as effectively as possible. It is considered unethical if teachers never monitor their students’ learning. Using a range of assessment methods (e.g., questions, exercises, quizzes, homework) to help students engage in language learning activities allows teachers and students to find out what has been learnt and what has been a challenge. These are ways to establish trust, shared purpose, and collaboration in the classroom. Teachers can guide students to take responsibility for their own learning. For example, students can be taught to engage in self- and peer-assessment. As they become more and more independent, they begin to develop learner autonomy. These are ethical principles that have been embedded in FA.

But SA is imbued with ethical considerations too. For instance, final degree examinations are a means of providing accountability – examination results can be used as an indicator of the quality of the education programs involved. Ethical practice in SA includes:

1. 1. designing and using tests and assessments relevant to the unit of study or the subject being taught. Using tests or assessments not aligned with the learning outcomes or the teaching and learning activities is unethical;

2. 2. informing students about essential tests and assessments they must complete in the course and the consequences of not succeeding. Students should be aware of their assessment responsibilities;

3. 3. reminding students of assessment dates and allowing them to ask questions to help them prepare for assessment tasks;

4. 4. ensuring that students have equal access to essential resources for learning and attainment. Assessment is unfair if some students can access resources while others cannot;

5. 5. conducting the SA in accordance with the appropriate rubrics and protocols within the given assessment framework. Making ungrounded, unfair, and prejudicial judgements on test performance is unethical;

6. 6. providing feedback on students’ performance individually or in groups so that they can confirm their understanding and realise critical areas they need to correct or improve on. In tests, quizzes, or assessments that produce scores, this includes providing numeric feedback with explanations of the scores that also cover areas of strength and weakness and what students should consider improving;

7. 7. treating all students fairly, regardless of their ability level, gender, race/ethnicity, disabilities, religion, and socio-economic status;

8. 8. calling out when dishonesty, such as cheating and plagiarism, is found. If such issues are not raised and addressed in accordance with the established code of conduct, students who engage in such behaviours may think that it is acceptable to do so. Not doing anything about this is unethical because such an experience can engender further misdeeds. Furthermore, cheating, if unchecked, can result in unfairness for other students and test-takers.

7.5 Fairness

In the discussion on ethics in Section 6.4, issues of fairness are implicated. Fairness is related to equality of treatment and opportunity for all students (see Reference KunnanKunnan, 2018). In FA, for example, if teachers give detailed feedback and support only to some students, but not to others, they are unfair in their assessment practice. If a high-stakes SA offers advantages only to some students, but not to others (e.g., owing to gender, race, religion, or socio-economic status), it is not a fair assessment. An assessment can be considered unfair if the test items and tasks involve knowledge and skills that are accessible only to some of the students and/or test-takers, for example topics that some students may know more about owing to their backgrounds in terms of language, ethnicity/race, gender, religion, and socio-economic status (for further discussion, see Reference Mirhosseini and De CostaMirhosseini & De Costa, 2019). The following scenario illustrates how fairness and ethics can be intertwined, presenting plausible dilemmas that teachers may face.

In this scenario, the principal has not advanced a ‘fairness’ argument for the use of computer auto-marking. The professional dilemma for the teachers is whether to challenge the principal’s authority. To do that might cause friction between the teachers and the principal. At the same time, the teachers cannot simply ignore the concerns; that course of (in)action might lead to students failing the test simply because it does not align with what they have been taught.

In many contemporary public education systems, schools and colleges serve minoritised students from diverse language backgrounds (e.g., some Hispanic backgrounds, Spanish-speaking communities in the USA, or speakers of indigenous languages in Australia). Suppose the language of academic communication is standard English (however defined). In that case, many minoritised students may receive unequal treatment and opportunity, particularly those at an early stage of learning English. For these students, the educational provision is unfair.

On a positive note, fairness can be facilitated by special considerations and accommodations. Considerations of accommodations for students with disabilities (both physical and cognitive diabilities) or those in difficult circumstances (e.g., unexpected illness or family tragedy) are essential (see Reference Abedi and KunnanAbedi, 2014; Reference Abedi, Zhang, Rowe and LeeAbedi et al., 2020). Accommodations in language assessment may require some changes or modifications to the test questions, assessment tasks, administration conditions, and procedures. For example, accommodations may include extending the amount of time for completion of the test, allowing the use of text-to-speech software to read a text in a reading test or to produce speech in response to a speaking task, or providing an amanuensis (a person writing down answers for the student or test-taker).

7.6 Impact

Earlier in this Element, we discussed the meanings of ‘stakes’ in testing and assessment. Mandatory standardised tests can have a high impact on teachers as well as students. Teachers’ careers may be impacted when their students perform poorly. Therefore, introducing a test or assessment will always have some degree of influence on the participants in various ways, as discussed earlier.

Test and assessment impact can be felt broadly by individual students, other stakeholders (e.g., teachers, schools, parents, employers), and society. Generally, intended consequences are directly related to using tests and assessments for a given purpose (e.g., to provide feedback to improve learning, to decide students’ achievement or mastery of the learning outcomes, to certify, to admit, and to employ). Unintended consequences refer to the negative impacts of assessment use beyond intended results, for example the high-stakes nature of a given test or assessment causing cheating behaviours on the part of weak or low-ability students. Another example of unintended consequences of a high-stakes test is the massive use of private tutoring for test preparation, popularly known as ‘cramming’.

We now focus on a well-recognised impact – washback (e.g., Reference Tsagari and BanerjeeTsagari & Cheng, 2017; Reference Wall, Fulcher and DavidsonWall, 2012). Washback refers to the influences of tests or assessments on language teaching, learning, and/or curriculum design. For example, if a national language test is to be given to all students in their final year at school, teachers and students will pay attention to ways of maximising success. There may be extra teaching and learning activities to prepare students to take this test at the expense of other learning activities. This is an example of negative washback. In the language assessment literature, washback can be positive or negative.

• Positive washback refers to the impact of assessment that helps students progress, retaining or transferring what they have learnt. The test or assessment can prepare students to use the knowledge and skills in their future careers and lives. For example, if a test asks students to write or speak in the target language, they are likely to prepare themselves to do so. Consequently, this knowledge is also available for non-test language use after the test.

• Negative washback refers to a situation in which student learning is overly restricted to the knowledge and skills covered by a test or assessment task. In other words, the nature of the learning is curtailed to fit the parameters of the test. For example, a language assessment that measures only students’ grammatical knowledge using a multiple-choice technique can have negative washback impact because isolated grammatical knowledge is insufficient for reading, listening, speaking, and writing in real-life contexts. Knowing grammar rules and how to answer multiple-choice questions is not sufficient for real-life language use or applications.

Scan Me 17

Positive washback (posted to YouTube by Judith Granados)

https://bit.ly/4aNCZsL

The concept of washback is relevant to both FA and SA. Teachers’ AfL can have positive washback impacts on students’ linguistic, cognitive, and social engagement. For example, when students receive some corrective feedback on their learning, they can realise their current difficulty or weakness and pay attention to improve it. This is an example of positive washback. When students are guided to become autonomous and self-regulated in their learning through the use of assessment criteria (often in the form of rubrics and sample questions/tasks), self-practices or rehearsals, and self-tests and assessments, they are likely to develop a capacity for autonomous learning as they have realised the benefits of self-monitoring and exploratory enquiry. The impact of assessment is omnipresent in formal education. The impact of assessment in English language education is most keenly felt in SA. The washback of large-scale commercially marketed tests such as IELTS and TOEFL often impacts the content of teaching programmes.

8.1 Pedagogical Practice

8.1.2 Formative Feedback

Readers will notice that the discussion in this section on pedagogic practice contains some repetition of points made in other sections. We intend to make explicit the connections between theory and practice. Feedback matters in additional/second language acquisition (Reference Gass, Behney and PlonskyGass et al., 2020). For example, it can raise awareness of students’ language use, behaviours, or errors. According to Reference Duckor and HolmbergDuckor and Holmberg (2023), it matters ‘in a world where we grow, where we learn, and where we expect everyone to change for the better’ (p. 2). These authors also point out that students need assistance to help them understand where they are going and what steps will lead them there; FA can help students achieve this by offering appropriate guidance (Reference Duckor and HolmbergDuckor & Holmberg, 2023). Teachers should ask the following questions before giving feedback: (1) Should feedback be provided? (2) What feedback should be provided? (3) When should it be delivered? (4) Who should give it? and (5) How should it be given?

Five Considerations for Effective Formative Feedback

This section presents five points language teachers should consider when providing formative feedback to students.

1. 1. Types of formative feedback: Different types of feedback can be considered when providing formative feedback.

   • Performance feedback focusses on helping students realise the correctness or appropriateness of their language use or responses to test tasks or activities.

   • Numerical feedback (e.g., scores) is also performance feedback. It tells students the level of performance relative to the maximum score they can achieve. It is helpful to provide an average score so that students know if they are above or below the average. However, numerical feedback alone is insufficient because students may not know how to use it.

   • Corrective feedback is complementary to numerical feedback and part of performance feedback because teachers inform students whether their language use or responses are accurate or appropriate and provide an explanation.

   • Metacognitive feedback focusses on helping students understand cognitive processes that can help them succeed in language use or task completion. It concerns how well students process information for monitoring and planning purposes (e.g., planning and goal-setting, monitoring, remembering, recalling, analysing, and synthesising). In a writing task, students’ poor knowhow may result in a lack of outlining or weak brainstorming of ideas before writing. After realising this, teachers can discuss how to improve their outlining with students.

2. 2. Configuration of feedback: Formative feedback needs to be delivered in ways that would benefit the student. Formative feedback configuration is related to efficacy, effectiveness, and fairness. According to Reference Duckor and HolmbergDuckor and Holmberg (2023), formative feedback can be designed for whole class, group, and individual configurations. Whole class formative feedback provides a general guidance in areas in which students perform and do not perform well. Group feedback focusses on the work of a specific group of (more or less homogenous) students, whereas individual feedback is tailored to a particular student.

3. 3. Explicitness of feedback: Teachers can vary the degree of explicitness when giving feedback to students.

   • Explicit feedback provides direct information to students on whether they are right or wrong in their responses and what the correct responses are (e.g., ‘You misspelt “weather” here.’).

   • Implicit feedback offers indirect correction of students’ responses but does not promptly provide the correct answer or response (e.g., ‘Is “weather” spelt correctly?’).

   • Recasts are examples of implicit feedback when teachers use a proper form without explicitly correcting the student(s). For instance, in an interview task, a student says, ‘I did not had breakfast, so I am quite hungry’; in response, the teacher says, ‘Oh, you didn’t have breakfast. Why?’.

   • Other types of feedback include clarification requests and elicitation and repetition techniques (see Reference Lyster and RantaLyster & Ranta, 1997).

4. 4. Time of feedback: This consideration is related to promptness of feedback provision.

   • Immediate feedback occurs when teachers provide feedback to students promptly when they realise students’ errors or incorrect or inappropriate language use or responses. Immediate feedback may disrupt students’ cognitive processes of language use or learning or embarrass them, so teachers must decide whether it is necessary and when it is appropriate. With automated scoring, numerical and generic feedback can be provided soon after students have submitted the work or test. A benefit of immediate feedback is that students can associate the correction with the error.

   • Delayed feedback can be provided after students have completed a given activity or complex language task. In some educational settings where teachers design and administer their own tests to groups of students, teachers will need time first to score or evaluate students’ responses, so feedback can be delayed. Delayed feedback can be both ‘holistic’ and detailed – giving specific information to help students learn about their work.

5. 5. Feedback agencies: Agency is concerned with the directionality of the formative feedback framework (Reference Duckor and HolmbergDuckor & Holmberg, 2023). Formative feedback can be teacher-driven, but it is not always the case. Students can be encouraged to assess their own work (e.g., self-assessment) or provide feedback to one another (e.g., peer-assessment). Teachers need to support and guide students when they engage in self- and peer-assessment (e.g., how to use assessment criteria and what to look for in their work).

Scan Me 18

Assessment for learning 4 Giving formative feedback (posted to YouTube by COGNITA LATAM)

https://bit.ly/3GAVvGR

8.2 Research and Enquiries in Language Assessment

A number of issues in language assessment require further research at this time. We present six issues and areas for discussion for illustrative purposes. Many of these issues cut across both FA and SA (illustrated in Figure 16).

Figure 16 Topics and areas for assessment research

1. 1. Research that investigates applications of language proficiency or language competency frameworks or benchmarks that are used to inform language curriculums, policies, and language teaching and assessment across various regions around the world (e.g., ACTFL, CEFR, the ESL scales in Australia, to name but a few). Several frameworks have been influential and revised over time. Therefore, knowing whether and to what extent such frameworks have impacted or been used in local, national, and international assessment contexts is essential. The benefits and drawbacks of adopting a particular language framework for teaching, learning, and assessment in localised contexts are under-researched.

2. 2. Research that considers and explores the nature of multilingual proficiency in language assessment. Insights from this area of study can inform how language proficiency can be assessed differently across localised multilingual contexts. Multilingual assessment requires re-conceptualising the prevailing English language proficiency models, which are largely based on anglophone monolingual language proficiency theories. Multilinguals are currently assessed based on this view in many language proficiency tests. In both FA and SA, English language learning outcomes are strongly influenced by anglophone theories of language proficiency. This is evident when schools, universities, or language institutes adopt or subscribe to commercial language books from mainstream publishers that provide them with language tests or assessment tools for classroom use. There is a need to embrace and accept the presence of multilingualism in English language use, learning, and assessment. There has been a call to consider flexible multilingualism and translanguaging, which occurs when English language learner-users (who, by definition, are multilingual) use all their language resources from their multilingual repertoire to make meaning when communicating with others (Reference Chalhoub-DevilleChalhoub-Deville, 2019; Reference Jenkins, Leung and ShohamyJenkins & Leung, 2016; Reference LeungLeung, 2022b; Reference Schissel, Leung and Chalhoub-DevilleSchissel et al., 2019).

3. 3. Research that investigates various roles and influences of social dimensions in language learning, use, and performance. For example, SA research can explore how students co-construct their language use with others, such as interviewers or partners, as they complete communicative language tasks. What are the features of social interaction that manifest in a given type of task completion? (For further discussion, see Reference KramschKramsch, 1986).

4. 4. Research that promotes assessment of integrated language skills (multimodal language use) that reflect real-world language-use situations. Language skills are often assessed separately. There are pedagogical reasons for separating language skills (to suit learner levels and to break down the complexity of language learning). While this practice is likely to continue, it is essential to note that in real-life language use, people mix various skills simultaneously and/or asynchronously.

5. 5. Research that reviews and evaluates applications and influences of computer technology in language assessment. The past decades have seen the popularity of Internet- or technology-based, large-scale assessments. Computer and mobile technology use for testing and assessment has become common in FA and SA. However, teachers need to be vigilant and critical about the use of technology since its introduction changes the nature of assessment methods, language use, and ways of observing performance. Teachers should aim to examine and address the theoretical, methodological, and fairness issues and other challenges they face, for example, in test design, administration, and scoring, as well as those faced by students and teachers (e.g., test preparation, access to resources, and technology).

   In addition to the use of technology for test delivery, there has been a rise in the use of in automated scoring technology (AST) in which AI and natural language processing (NLP), for example, have been adopted to replace human scoring. Developers who adopt AST as part of their test or assessment design have been challenged by critical questions about the reliability, the accuracy, and the suitability of automated scoring use (in terms of the real-world construct of writing or speaking ability). This, incidentally, has triggered public suspicion in relation to claims about the reliability and the validity of automated scoring made by researchers who are affiliated with the institutions that are developing and marketing the products.

   Teachers should enquire about the use and the impact of AST on their classroom practice, particularly as there could be unforeseen or unintended consequences for students, test-takers, and stakeholders. For example, much attention has been drawn to AI technology that can threaten the integrity of assessments (e.g., it writes on behalf of students). Already, AI technology is sufficiently sophisticated that it can be challenging for AST to detect AI plagiarism. Teaching students to use AI ethically is also essential. If teachers do not have the knowledge and/or the technology to see such use, it would be challenging to determine students’ actual learning attainment. Therefore, when technology has become an integral part of language assessment, it is essential to research its impacts on the validity and the trustworthiness of assessment practice and students’ lifelong learning.

6. 6. Research that focusses on fairness and ethics in language assessment. Fairness and ethical considerations in classroom language assessment are integral to the abovementioned topics and areas. Fairness and ethics are complex matters as all assessment contexts are different; each case has to be considered in its own context.

   Although there is no one-size-fits-all, bullet-proof measure to guarantee fairness in language assessment, we argue that research into practices of fair and ethical assessment would allow teachers to realise the problems that they and their students face when assessment is unfair. For example, teachers can focus on transparency of purpose in language assessment (e.g., reasons for promoting self- and peer-assessment in the classroom; clarity in the task, the assessment instructions, and the evaluation criteria used; and how classroom assessment includes students with special needs). Teachers can also investigate how they and their students shape the classroom environment in which all students have access to equitable support and resources (e.g., through appropriate feedback provision and opportunity for students to access educational material or technology) that give them an equal chance to be successful in their learning and achievement.

We hope that teachers’ enquiries into research that is relevant to language assessment practice in their contexts will help enhance their expertise and professional repertoire. Teachers can also conduct research into aspect(s) of their language assessment practice. For instance, teachers, working individually or together with colleagues, can look into the way(s) in which they design their end-of-term tests or non-test tasks, analyse student performance, and provide feedback.

8.3 Further Professional Development and Further Reading

We recognise that a high level of teacher knowledge in assessment matters is vital to professionalism. We therefore encourage teachers to join international or local professional associations that work with their members to promote good practice in language assessment (e.g., the International Language Testing Association, the European Association for Language Testing and Assessment, the Asian Association for Language Assessment, the Association of Language Testing and Assessment of Australia and New Zealand), and other local/national professional associations. Several prominent academic journals, e.g., Language Testing, Language Assessment Quarterly, Assessing Writing, Language Testing in Asia, Language Education and Assessment, and Assessment in Education: Principles, Policy & Practice, and websites maintained by professional associations provide up-to-date information on language assessment developments and research findings. Open access (free-to-use) publication of journals has made it easier in recent years for everyone to read up on research and development.

Although we have provided many citations throughout this Element, we recommend the following books for teachers to further their language assessment knowledge:

1. - Reference Bachman and DamböckBachman and Damböck (2017) presents theoretical and practical concepts and issues in classroom-based assessments, such as the relative importance of decisions based on assessment (e.g., formative and summative) and assessment design;

2. - Reference Fulcher and HardingFulcher and Harding (2022) is the second edition of Reference FulcherFulcher and Davidson (2012). It provides updated chapters from the first edition and new chapters on issues and key considerations in language assessment research and use;

3. - Reference GriffeeGriffee (2012) encourages teachers to do research and providing an accessible introduction to research methods (e.g., research design and data collection) that can be useful for language assessment research;

4. - Reference Jeon and In’namiJeon and In’nami (2022) is an edited volume based on a research synthesis approach. It presents four areas in which language skills have been considered, assessed, and researched, and provides an essential theoretical explanation of each language skill that can help teachers extend their theoretical repertoires for informing their teaching and assessment;

5. - Reference McMillanMcMillan (2013) provides comprehensive coverage of classroom assessment topics and areas, and offers articulated research questions for classroom assessment (e.g., FA, SA, quality assessment criteria) so that teachers can relate specific research questions to their localised context;

6. - Reference Shohamy, Or and MayShohamy et al. (2017) is an edited volume that provides innovative perspectives and methodologies on critical language assessment use and practice issues. Key themes include assessing language domains (e.g., language proficiency, multilingualism, and lingual franca), assessment methods (e.g., use of technology, methods for test validation), assessment in education (e.g., dynamic assessment, washback, and test impacts), and assessment in society (e.g., test stakes, critical language testing);

7. - Reference Tsagari and BanerjeeTsagari and Banerjee (2017) presents the essential foundations of language assessment (e.g., assessment purposes, quality factors), key assessment areas (e.g., assessing language skills), contexts of assessment (e.g., assessment in educational or workplace settings), and contemporary issues in assessment (e.g., assessing young learners and students with learning and other disabilities);

8. - Reference Winke and BrunfautWinke and Brunfaut (2021) comprehensively treats language assessment methodology for second language acquisition research, focussing on the interface between language assessment and second language acquisition research. Topics include fundamental concepts in assessment and measurement, research instruments, measuring individual differences, and assessment of language development.

Also, the UK Association for Language Testing and Assessment (UKALTA) has published a number of briefing sheets on various topics, including AI and language assessment and flexible multilingualism. See https://ukalta.org/ukalta-reports-and-briefing-papers/.

8.4 Concluding Remarks

We must recognise the significance of language assessment in the educational enterprise. Reference Spolsky, Fulcher and DavidsonSpolsky’s (2012, p. 502) metaphorical remark that ‘tests, like guns, are potentially so powerful as to be commonly misused’ is a moral reminder that we should all be aware of the power of assessment.