2. Reference Level Descriptions

The Council of Europe (2005) recognises in the development of Reference Level Descriptions (RLDs) the need for a concerted effort to reconcile the competing aims of comparability and flexibility and to interpret and elaborate the necessarily broad descriptions of levels in the CEFR so that they may more readily inform practices relating to specific languages and to specific applications (such as syllabus design or test development). The essential requirements are set out in a set of draft guidelines as follows:

In common with other elements of what has come to be known as the CEFR toolkit, RLDs are intended to assist users in applying the CEFR to meet their local needs as language learners and educators.

The guidelines specify that RLDs should provide ‘inventories of the linguistic realisations of general notions, acts of discourse and specific notions/lexical elements and morpho-syntactic elements’ (CoE 2005: 5) that ground the CEFR descriptors. There is no requirement to limit the descriptors employed to those used in the CEFR; indeed, developers are encouraged to incorporate descriptors from European Language Portfolio (ELP) models. Each RLD should explain the process by which these inventories are arrived at, the knowledge of a linguistic form expected of learners at a given level (receptive or productive) and the relationships between the lists presented.

RLDs are thus intended to mediate between the CEFR and specific contexts for its use. They offer meaningful illustrative learning objectives that more fully operationalise the CEFR descriptions for users by providing linguistic exponents. In considering how best to proceed in developing RLDs, our first step should be to consider how learning objectives are presented in the CEFR and the extent to which these objectives already meet the needs of CEFR stakeholders as revealed through the available literature.

3. Objectives and ‘can do’ statements in the Common European Framework

The CEFR embodies an instrumentalist ‘action-oriented approach’; the key concern is with what learners are able to accomplish when using a language rather than with their knowledge about language. This approach leads to an emphasis on potential language activities (and tasks) – ‘any purposeful actions considered by an individual as necessary in order to achieve a given result in the context of a problem to be solved, an obligation to fulfil or an objective to be achieved’ (CoE 2001: 10) – as meaningful outcomes of learning.

To assist in the process of setting objectives for language learning, teaching and assessment, the CEFR presents the user with conceptual questions for consideration and a bank of ‘illustrative’ descriptors arranged into 54 scales, each relating either to an aspect of a ‘competence’ (e.g. scales for ‘vocabulary range’ and ‘vocabulary control’ [CoE 2001: 112] are aspects of ‘lexical competence’ [CoE 2001: 30]) or to a language ‘activity’ (e.g. for oral production, ‘Sustained monologue: describing experience’, ‘Sustained monologue: putting a case (e.g. in debate)’, ‘Public announcements’, ‘Addressing audiences’). These illustrative descriptors are arranged into six proficiency levels. According to the CEFR, ‘The intention of providing a concrete illustrative set of descriptors, together with criteria and methodologies for the further development of descriptors, is to help decision-makers design applications to suit their contexts’ (CoE 2001: 36).

It is acknowledged in the opening chapter of the CEFR that ‘In order to fulfil its functions. . . a Common European Framework must be comprehensive, transparent and coherent’ (CoE 2001: 7). The CEFR offers the following definitions:

The descriptors provided in the framework take the form of positively worded ‘can do’ statements describing activities that the learner might carry out in the target language. They are said to draw on three main sources: ‘(a) the theoretical work of the [CEFR] authoring group, (b) the analysis of existing scales of proficiency and (c) the practical workshops with teachers’ (CoE 2001: 30). It is claimed that the scaling of these descriptors based on the rating of performance samples using the Rasch model (Bond and Fox Reference Bond and Fox2007) creates an empirically derived scale of language ability.

The following section locates the CEFR in two descriptive traditions that influenced its development: behavioural objectives and the proficiency movement. The CEFR is considered in relation to its own stated purposes and in relation to the goals of these two approaches to level description.

4. Behavioural objectives and the proficiency movement

5. Use of behavioural objectives and proficiency scale descriptors in the CEFR

Both proficiency scales and learning objectives were incorporated into the development of the CEFR illustrative descriptor pool (North Reference North2000). The ACTFL Proficiency Guidelines and related proficiency scales such as the Australian Second Language Proficiency Ratings (Ingram Reference Ingram1990), the Foreign Service Institute Absolute Proficiency Ratings (Wilds Reference Wilds, Jones and Spolsky1975) and the Interagency Language Roundtable Language Skill Level Descriptions (ILR 1985) were included, as were such objectives-based schemes as the English National Curriculum: Modern Languages (1991) and the Eurocentres Scale of Language Proficiency (1993). Borrowing from both behavioural objectives and proficiency scales in this way sits well with the aim of building shared understandings between learner and teacher assessment of ongoing learning and consolidated summative information derived from programme external tests that may be used for certification and accountability.

The illustrative scales in the CEFR are made up of individual statements taken from the sources listed above, among others (30 schemes altogether), giving an initial pool of 1,679 descriptors in total (North Reference North2000). The descriptors were screened for repetition and edited to produce ‘positively worded, “stand-alone” statements that could be independently calibrated’ (North Reference North2000: 184).

Editing and trialling were intended to produce descriptors that would be suitable both as learning objectives and for distinguishing levels (North Reference North2000). Five requirements for adequate proficiency descriptors emerging from North's study are listed in Appendix A of the CEFR (CoE 2001: 205–207):

Note the similarities between positiveness and independence and Mager's (Reference Mager1991) performance, and between clarity and definiteness and Mager's condition and criterion in formulating behavioural objectives.

Employing the Rasch model, the statements were then calibrated on the basis of teachers’ ratings of student performance – either ratings of their own students or of video samples of learner speech. It is claimed that

Hudson (Reference Hudson2005: 218) questions the validity of the claim of empiricism, observing that ‘whereas the descriptors were empirically scaled based on performance ratings, the particular descriptors were not subsequently cast as actual test prompts and then calibrated again to determine if they still scale hierarchically’. North (Reference North2000) acknowledges the further objection that the scales are empirical only to the extent that they calibrate teacher perceptions: they are not empirically derived from L2 learner data (Hulstijn Reference Hulstijn2007). Reference level descriptions will need to address the links between teacher perceptions as operationalised in the scales and observable learner performance.

As the CEFR draws on both behavioural objectives and proficiency scales, it has attracted much of the criticism directed at both approaches as well as doubts concerning the extent to which it is possible to reconcile diverse purposes within a single scheme.

The criticisms made of the ACTFL Proficiency Guidelines (Lantolf & Frawley Reference Lantolf and Frawley1985; Savignon Reference Savignon1985; Kramsch Reference Kramsch1986) that they were not based on research into the nature of second language acquisition have been repeated with respect to the CEFR by Bausch, Christ and Königs (2002, cited in Morrow Reference Morrow2004) and by Hulstijn (Reference Hulstijn2007). Although, unlike the ACTFL Proficiency Guidelines, the CEFR is based on a componential model of communicative language ability, Hulstijn (Reference Hulstijn2007: 666) notes the continuing lack of ‘empirical evidence that, in following the overall oral proficiency scale, all learners first attain the functional level of A1, then the level of A2, and so on, until they reach their individual plateau’. Conversely, of course, there is equally little evidence to suggest that learners do not all proceed in this way.

Echoing criticisms of the ACTFL Proficiency Guidelines made by Bachman and Savignon (Reference Bachman and Savignon1986), Hulstijn (Reference Hulstijn2007) notes that the CEFR levels integrate, through the ‘activities’ and ‘competences’ scales, issues of quantity – the number of ‘domains, functions, notions, situations, locations, topics, and roles’ (de Jong Reference de Jong2004) that a learner is able to cope with – and issues of quality – the extent to which communication is effective and efficient (although, as noted above, the degree of integration is inconsistent). Hulstijn questions the evidential basis for supposing that a learner placed at a given level on the basis of ability to perform the requisite activities will necessarily possess equivalent quality in terms of the linguistic (grammar and vocabulary) scales at that same level (2007: 666). Rather, he suggests, there may be learners who are limited in terms of quantity, but who are able to perform their limited range of roles with high linguistic quality, while others may be able to operate in a wide range of contexts, but with only limited linguistic resources: the distinction between horizontality and verticality made by Richterich & Schneider (Reference Richterich and Schneider1992) is not sufficiently clear in the CEFR.

The tasks that learners are asked to perform have a substantial impact on the nature and quality of the language that they will be able to produce. Further work is needed to tease out the relationship between elements of linguistic quantity and quality. The complex relationship between the two may partly explain the feeling that the higher levels of the CEFR are inappropriate or unattainable for certain groups of learners – especially young learners (Little Reference Little2007: 651) as the activities envisaged at these levels ‘lie beyond the cognitive and experiential range of children and the great majority of adolescents’.

An objection raised by Weir (Reference Weir2005a) is that the framework is insufficiently explicit to inform test or task specifications because it fails to include the contextual parameters that affect performance – Mager's (Reference Mager1991) ‘condition’ or constraints on performance and Weir's (Reference Weir2005b) context validity. It is argued in the CEFR that ‘very detailed concrete decisions on the content of texts, exercises, activities, tests, etc.’ (CoE 2001: 44) will be required, taking into account the impact of physical conditions (e.g. levels of ambient noise), social conditions (e.g. relative status of interlocutors) and time pressures (e.g. conversational interaction allowing participants little time to prepare their utterances) on student/test-taker performance. Although a list of external conditions is provided (CoE 2001: 46–47), these are not incorporated into or explicitly related to the ‘can do’ descriptors that define the levels. As Weir (Reference Weir2005a) has pointed out, this must seriously undermine the interpretability of the levels as it is unclear which conditions should apply when we judge what learners ‘can do’.

Furthermore, Weir (Reference Weir2005a) and Alderson (Reference Alderson2007) argue, the scales fail to take sufficient account of the cognitive and metacognitive processes engaged by learners as they perform tasks at each CEFR level. These are dealt with in the framework (CoE 2001: 90–93), but as Alderson (Reference Alderson2004) observes, the CEFR provides only ‘a taxonomy of behaviours rather than a theory of development in listening and reading activities’. In the scale for ‘Overall Reading Comprehension’, by way of illustration, the descriptor ‘Can read straightforward factual texts on subjects related to his/her field and interest with a satisfactory level of comprehension’ not only fails to define what degree of comprehension is meant by ‘satisfactory’, but provides no details on the operations required to arrive at this level of comprehension. It is unclear, for example, whether, following the options set out on page 68 of the framework, learners should be able to read such a text for gist, for specific information, for detailed understanding or for implications, or whether such differences in reading purpose might have no implications for the level of the reading task.

The approach taken by the CEFR is contrasted by McNamara & Roever (Reference McNamara and Roever2006) with schemes such as the Australian Certificates in Spoken and Written English (CSWE) or the related Canadian Language Benchmarks (CLB 2000) which provide elaboration and sample tasks to exemplify levels. Such elaboration is intended to help test developers, teachers responsible for assessment and learners themselves to arrive at comparable understandings of learner abilities. In the Canadian Language Benchmarks 2000 document, for example, the following ‘can do’ statement is given for the skill of reading at ‘Benchmark 5: Initial intermediate proficiency’.

Note that Table 1 includes criteria for success as well as task definition and exemplification. A list of performance conditions is also provided – for Reading Benchmark 5 these include the following:

• • Text is two or three paragraphs long and related to personal experience or familiar context.

• • Text is legible, easy to read; is in print or neat handwriting.

• • Tasks are in a standard format: with items to circle, match, fill in a blank, complete a chart, answer questions, etc.

• • Learner is adequately briefed for focused reading (has at least minimal knowledge to activate knowledge schemata for top-down processing).

• • Instructions are clear and explicit, for everyday situations, used with some visual clues, presented step by step. Pictures occasionally accompany text.

• • Text has clear organization.

• • Text is two or three paragraphs long, printed or electronic.

• • Language is mostly concrete and literal, with some abstract words.

• • Context and topic are often familiar and partly predictable for learner.

• • Content is relevant and can be related to personal experience.

• • Text types: newspaper articles, educational/content materials, stories, encyclopedia entries, short reports. (Pawlikowska-Smith Reference Pawlikowska-Smith2002: 88)

In a companion volume (CLB Sample Tasks) examples of representative tasks are offered for each of the three CLB domains of ‘community access’, ‘study/academic’ and ‘workplace’. The following reading tasks are suggested for the academic domain at Benchmark 5:

Table 1 Reading Benchmark 5: Initial intermediate proficiency (Pawlikowska-Smith Reference Pawlikowska-Smith2000: 89).

Study/Academic tasks

• • Read a brochure about a training program.

• • Follow instructions regarding school assignments.

• • Follow simple-language, user-friendly computer screen commands.

• • Number a set of pictures in an appropriate sequence based on the information in the text. Identify pictures that do not belong in a story sequence.

• • Skim ten texts for 30 seconds each. Fill out a sheet with information on what each article is about, its purpose, if it is interesting or useful for you to read or not.

Guidance is also given on the proportion of items based on such tasks that a learner would need to answer correctly in order to achieve the benchmark.

The extent to which the CLB tasks might, in fact, yield comparable tasks and comparable results across teaching and learning contexts is, of course, open to question – how long after all are ‘two to three paragraphs’ and how might one be sure that a learner is ‘adequately briefed’? There is insufficient information on required item types to ensure the comparability of tasks produced by different teachers based on these lists and so any assumption that 80% correct responses on one task would be even very roughly comparable to 80% on another seems overly optimistic. Nonetheless, the amplification of the ‘can do’ is far closer to the level of detail that Weir (Reference Weir2005a) has called for. Providing a similar level of detail for purposes of illustration might benefit the interpretability of RLDs for English.

The CSWE scheme goes even further than the CLB in supporting comparability in teacher assessment, providing sample tasks whose difficulty levels have been calibrated (using the Rasch model) for teachers to draw on when they assess learners (Brindley Reference Brindley2001). Of course, such calibration may be more meaningful when a scheme is designed to work within an educational programme (the case for the CSWE) than when it is intended to apply across programmes (the case for the CEFR and the proposed RLD).

Martyniuk & Noijons (Reference Martyniuk and Noijons2007) in their survey of the use of the CEFR across Europe found that users ‘stress the need for general clarification (such as comments on theoretical concepts, examples and good illustrations, sets of tasks for use in specific contexts’. Further development of the sample materials provided by the Council of Europe or of open-access schemes such as DIALANG might help to ground the CEFR levels in a way that meets these needs. On the other hand, the more detail is provided, the greater the risk that illustrative tasks become required tasks and this distinction must be made clear.

The lack of guidance on criteria and conditions in the CEFR leads directly to questions about comparability. In posing the question, ‘how does one know for certain that a test of Greek calibrated at level B1 in Finland is equivalent to a test of Polish considered to be at level B1 in Portugal?’, Bonnet (Reference Bonnet2007: 670) points to two threats to comparability: local norms and inter-linguistic variation. In the absence of elaboration, exemplification and, most crucially, moderation of standards, it is likely that users in one setting may interpret the illustrative descriptors differently to users in another. It is possible that different interpretations of levels might develop so that, for example, a learner judged to be at level B1 in one school in Finland might be rated as level C1 at another in Portugal. The risks associated with such inconsistency are well illustrated by Crossey's (Reference Crossey and Alderson2009) account of the NATO STANAG 6001 scheme intended to provide agreed international language standards for military personnel. Faith in the scheme was undermined when it became clear that learners certified as being at a given level in one context did not satisfy the criteria as interpreted in another. Equally (although beyond the scope of a reference level description for English), the effect of language differences on the calibration of ‘can do’ statements is far from clear. It is important that RLDs incorporate a mechanism for ongoing interaction between users so that common understandings can be fostered and maintained.

Alderson is critical of inconsistencies in the wording of the descriptors, which perhaps reflect their varied provenance (Alderson et al. Reference Alderson, Figueras, Kuijper, Nold, Takala and Tardieu2006; Alderson Reference Alderson2007). For example, at level C1 on the ‘Coherence and Cohesion’ scale (CoE 2001: 125) there is reference to ‘clear, smoothly flowing, well-structured speech’, but at level C2, ‘coherent and cohesive text’. At level C1, learners use ‘organisational patterns, connectors and cohesive devices’; level C2 only includes ‘organisational patterns and cohesive devices’. However, in the global scale on page 28, ‘connectors’ do appear at level C2. On the scale for ‘Vocabulary Range’ (CoE 2001: 112), learners at level C have a ‘lexical repertoire’; those at level B2 have a ‘range of vocabulary’. Alderson et al. (Reference Alderson, Figueras, Kuijper, Nold, Takala and Tardieu2006) note that eight different verbs are used to indicate understanding at level B2, but that no gloss is provided to explain the implications, if any, of these differences. Without elaboration and exemplification it is unclear whether such differences are significant. RLDs for English should serve to clarify or minimise such variation.

The CEFR includes criticism of other scales for the use of vague language and qualifiers to define level differences, but such problems are not consistently avoided in the illustrative descriptors. In the ‘Grammatical Accuracy’ scale (CoE 2001: 114), a B2 learner ‘Shows a relatively high degree of grammatical control’, while a B2+ learner has ‘Good grammatical control’ and a C1 learner ‘Consistently maintains a high degree of grammatical accuracy’. In attempting to apply these descriptions, the user may wonder, ‘relative to what?’ or ‘how consistently?’, but no guidance is offered in the CEFR. Descriptors for reading and listening make use of verbs such as ‘understand’, ‘follow’ or ‘read’. These descriptors fall short on the criterion of clarity as they fail to specify observable behaviours that would demonstrate the learner's ability. Where comparisons are to be made across contexts it is important that users share similar expectations of learner performance.

There is considerable variation among descriptors even within scales in the extent to which they specify activities or refer to performance quality. In the self-assessment grid presented as Table 2 on pages 26–27 of the framework a B1 descriptor for spoken interaction reads, ‘I can enter unprepared into conversation on topics that are familiar, of personal interest or pertinent to everyday life (e.g. family, hobbies, work, travel and current events)’, while a C1 descriptor is ‘I can express myself fluently and spontaneously without much obvious searching for expressions’. The B1 descriptor, in common with many of those offered for the lower levels, describes a language activity (‘conversation’ – although no definition of a conversation is offered) and operative conditions (the speaker is ‘unprepared’ and the topics are ‘familiar, of personal interest or pertinent to everyday life’). However, no criteria are suggested for the qualities of the learner's contribution to the conversation (nor are any provided for the other descriptor placed at this level). The C1 descriptor, in contrast, is exclusively concerned with quality or ‘how well’ (‘fluently and spontaneously without much obvious searching for expressions’), but offers no guidance on activities or tasks beyond the vague ‘express myself’. There is no indication of the conditions or situational constraints under which the speaker might be asked to interact.

Table 2 Performance monitoring, evaluation and the Benchmark achievement report in CLB 2000 (Pawlikowska-Smith Reference Pawlikowska-Smith2000: 146).

Many case studies, in using the framework, call for further elaboration and exemplification of the levels (Alderson Reference Alderson2002; Morrow Reference Morrow2004; Martyniuk & Noijons Reference Martyniuk and Noijons2007; Figueras & Noijons Reference Figueras and Noijons2009; Martyniuk forthcoming). This need has been addressed by the Council of Europe through the provision of materials exemplifying the framework levels and the manual for relating language examinations to the CEFR. However, these materials remain sparse and seem to raise as many questions as they answer. North (Reference North2000) has suggested that ‘Nobody has the same level across the 54 scales. Everybody has a profile’: that the scales apply to aspects of competence rather than to learners. In performance a learner may, for example, display certain competences at level C1, others at B2. It is therefore important to understand which scales do apply to the sample performances or tasks and which do not. To what extent can a given level (rather than a profile) characterise a learner, a task or a performance? Sample performances provided to accompany RLDs for English need to be accompanied by commentary explaining in some detail how these relate to the descriptors.

6. Tension between purposes

Brindley (Reference Brindley1998) lists challenges encountered in the development of outcomes-based assessment. Some of these are equally applicable to proficiency scales and include issues such as comparability and consistency of judgements outlined above. Additionally, there is a danger, also seen in the mastery learning experience of the 1960s and 1970s, that objectives can come to dominate the classroom. In finding an appropriate level of specificity in setting objectives, steering a course between the Scylla of inexplicit generalisation and the Charybdis of atomisation has proved to be a persistent challenge for educators (Popham Reference Popham1973; Spady Reference Spady1994; Brindley Reference Brindley1998). Almost every learning activity could be specified and recorded in terms of an objective, but this is administratively overwhelming and it is far from clear how information might be aggregated to provide information that might be of use to stakeholders outside the immediate learning context.

Within language programmes, teachers often employ relatively inconsequential ‘intermediate objectives’ (Trim Reference Trim1981) to motivate learners and to acknowledge success on specific learning tasks; they may be less concerned with how performance on a task generalises to performance beyond the classroom. In contrast, external audiences such as sponsors and employers are interested in what are sometimes called ‘terminal objectives’ (Trim Reference Trim1981) or summative representations of learner proficiency – the ‘end-results of the learning process’ (Trim Reference Trim1981). In the CEFR context, the question arises of how many of the illustrative descriptors from the finer-grained category scales would need to apply before a learner could be judged to be at a certain level on the global scale (Council of Europe 2001: 24).

Teasdale and Leung (Reference Teasdale and Leung2000: 167) raise ‘the question of whether assessments can at the same time adequately serve formative/diagnostic purposes whilst adequately performing as valid (in the psychometric sense) measurement instruments within particular contexts’. Arkoudis and O'Loughlin (Reference Arkoudis and O'Loughlin2004) and Burrows (Reference Burrows1998) point to the variation that teachers may exhibit in their interpretation of frameworks and to the ways in which they may adapt these to local circumstances in divergent ways. Such customisation is welcomed in the CEFR in its role as a heuristic for the elaboration of language programmes, but is likely to work against the comparability of outcomes from programmes purportedly situated at the same level.

A further objection to outcomes-based schemes is that they ‘obscure, deform and trivialize education’ (Egan 1983, quoted in Lantolf & Frawley Reference Lantolf and Frawley1985). Objectives are said to be bureaucratic and coercive, serving to circumscribe learner roles and forcing teachers to work towards predetermined ends. In the field of ESL, Auerbach (Reference Auerbach1986) criticises outcomes-based approaches for casting ESL learners in menial work roles and for narrowing the educational choices open to teachers. McNamara & Roever (Reference McNamara and Roever2006: 212–213) are concerned that the CEFR is achieving such dominance that it imposes its interpretation of language learning on previously diverse teaching and testing programmes: ‘testing organizations wishing to operate in Europe. . .have had to align their assessments with levels of the CEFR, despite radically different constructs, as a pure political necessity. Funding for reform of school language syllabi. . .is tied to conformity to the CEFR in more than one European country’. Morrow (Reference Morrow2004) stresses that the framework is explicitly not intended as ‘a set of suggestions, recommendations or guidelines’ and cites the case studies that he introduces to the diversity of ways in which it may be applied. Although critical of some of these uses, Fulcher & Davidson (Reference Fulcher and Davidson2007) take a less pessimistic view than McNamara & Roever (Reference McNamara and Roever2006), seeing the framework's role in raising awareness as a key strength. In line with the ethos of the CEFR, RLDs for English should be presented as a resource for teachers, course developers and other educators and should be developed in consultation and collaboration with users.

It is apparent that the CEFR levels have provided a convenient set of standards that have been enthusiastically adopted by government agencies in mandating targets for achievement or setting requirements for immigration programmes, often without any clear justification (Alderson Reference Alderson2007; Krumm Reference Krumm2007). Alderson is concerned that

7. Operational Reference Level Descriptions

The Threshold series (van Ek and Trim) provides the classical approach to CEFR specification covering A1 (Breakthrough), A2 (Waystage), B1 (Threshold) and B2 and beyond (Vantage). These documents specify linguistic competence for English in a way that complements the CEFR. Rather than starting from language forms, as in the CEFR scales for range and control, the Threshold series starts from a classification of communicative functions and of notions, presenting lexical and grammatical forms as their exponents. Profile deutsch (Glaboniat et al. Reference Glaboniat, Müller, Schmitz, Rusch and Wertenschlag2002), a reference level description for German, makes use of information technology to allow users to approach the material from either direction. In the Profile deutsch interface, clicking on a function brings up grammatical and lexical exponents; clicking on a grammatical item brings up a list of functions that the item could be used to realise. RLDs for English should make use of similar technologies to allow users to access content in directions that are of most relevance to their requirements.

Alderson (Reference Alderson2004: 1), reporting on the experience of applying the CEFR scales to the DIALANG diagnostic assessment project, identifies a key challenge for users: ‘it is not easy to determine what sort of written and spoken texts might be appropriate for each level, what topics might be more or less suitable at any given level, and what sort of operation – be that strategy, subskill or pragmatic inference – or sociolinguistic context might be applicable at which level’. In addition to supplying language-specific specifications of grammar and vocabulary, the integration of cognitive and contextual parameters into the CEFR levels should be a key contribution of RLDs for English. Work contributing to the Council of Europe (2009) manual for relating examinations to the Common European Framework, such as the task content analysis checklists (Alderson et al Reference Alderson, Figueras, Kuijper, Nold, Takala and Tardieu2006; Association of Language Testers in Europe 2005), has provided a framework for approaching such work. Shaw & Weir (Reference Shaw and Weir2007) and Khalifa & Weir (Reference Khalifa and Weir2009) have taken up the challenge and have already established how contextual and cognitive parameters relate to the levels of Cambridge ESOL examinations. This work needs to be applied more broadly in RLDs to indicate how level differences are operationalised in a range of settings.