2.1 Its origins in industry

In order to understand the place of QM in language education one needs to know where it began. Historically, the quality movement arose in the field of industrial production, and was extended to service industries and subsequently to public institutions, including schools and universities, and to specific areas such as language teaching. In industry the goal was to improve productivity and efficiency; this was initially influenced by Taylorian principles (Taylor Reference Taylor1911), according to which production processes were split up into the smallest possible operational units, allowing production lines (like those in Chaplin's Modern Times) to contribute to the development of mass production, replacing custom-building. ‘Quality’ was achieved by systems of control carried out by ‘inspectors’ who checked for errors; putting defects right was typically a significant part of production cost.

Before and during the Second World War in the United States, and after 1945 in Japan, especially, QM principles were developed by a number of academics and consultants. Most of them were working in manufacturing industry, linked closely to the war effort in the USA, and to reconstruction of the economy in Japan. The best known of them is W. Ernest Deming (1900–1993), who developed a series of principles for the management of quality in industry, including an approach to management known as ‘Deming's Wheel’ or (from its initials) ‘PDCA’, a continuous sequence consisting of four steps:

• • Plan (establish targets, objectives and processes)

• • Do (implement the plan and collect data)

• • Check (study the results and compare with the plan, noting differences)

• • Act (analyse differences, carry out corrective actions; if no corrective action needed apply the PDCA sequence to a wider range of actions).

Deming (Reference Deming1986) also developed a set of principles which have profoundly influenced the QM movement: an emphasis on leadership, on giving responsibility to workers, on the idea of production being driven by planned processes, on a systems approach, on the importance of data collection and of basing decisions on factual information. They are in direct opposition to previous Taylorian approaches. Deming claimed that by giving responsibility to small groups of workers, defects could be eliminated at the outset: ‘get it right the first time, every time!’. Juran (Reference Juran1951) developed further the quality control applications of the approach, adding emphasis on human aspects of quality control and on rigorous analysis of processes.

2.2 Applications of QM in service industries

The application of QM principles to services saw the addition of a focus on the customer and of strategic approaches to quality. Feigenbaum (Reference Feigenbaum1961) developed the concept of TQM (Total Quality Management), emphasising the need for customer satisfaction underpinned by clear service standards. Since services, unlike goods, are not tangible, quality has been equated with customer satisfaction, with the adoption of slogans such as ‘quality means meeting customer expectations’ or ‘quality means exceeding customer expectations’ (Juran Reference Juran1951). In the provision of services the contract between the provider and the consumer of services is a relational one and the quality of the service is often defined through the keeping of promises expressed in the form of customer charters or service guarantees. An essential element of quality in services is a clear description of what is offered: ‘say what you do/do what you say you do’.

The International Organization for Standardization (ISO 2008) is the body which has the most influence on the implementation of quality approaches in services. In ISO 9001 it has developed sets of standards for products and services, based on principles derived from Deming and other QM gurus. In the applications of ISO 9001 the emphasis is placed mainly on the process and systems approaches. Rather than assessing the actual quality of a service or product, ISO in its initial versions was concerned to verify that (a) there is a system for delivering quality and (b) that there are procedures for establishing and checking that the processes generated by the system are being applied.

2.3 Clients and client satisfaction

An important feature of the development of quality approaches in the service sector was the integration of customer relationships at the centre of QM and the definition of the concept of the customer or client (Winder & Judd Reference Winder and Judd1996). If the quality of a service is judged by the satisfaction of client needs or expectations, this raises the question of ‘who is the client?’ Is it the person who pays for the service, or the one who experiences it? For example, is the client of a school the learner in the classroom, the parents, the local authority, or society in general? To answer this question, various categorisations of ‘clients’ have been formulated, such as the primary client (usually the one who pays for the service) and secondary client, if the consumer of the service is different (Juran Reference Juran1951). Another category is the ‘unwitting’ client, who may be affected by the delivery of a service, even though they have not initiated it. Some approaches to organisational development, including that of Juran (Reference Juran1951), have proposed the idea of the organisation as an interlocking system of mutual client relationships, in which everyone is both client and provider of services. This view distinguishes between external clients, who purchase a service, and internal clients, who have needs and expectations which they are entitled to have satisfied, but who are also accountable to others in turn for the satisfaction of their legitimate expectations.

2.4 QM in public institutions

The notion of accountability is especially important when QM approaches are applied to public institutions; it reinforces the idea of client relationships and has been especially influential in healthcare and education. Patients, parents, children and society in general are seen as having rights, and public institutions are accountable to them in the delivery of these rights. Such rights are often described in charters or guarantees: public statements of the specific promises for which institutions can be held accountable.

Another notion which has been applied principally, but not exclusively, to public institutions is that of fitness of, and for, purpose, which includes the need to have valid objectives and to achieve them effectively. As Harvey & Green (Reference Harvey and Green1993: 9) put it: ‘Fitness for purpose sees quality as fulfilling a customer's requirements, needs or desires. Theoretically, it is the customer who specifies requirements. In education, fitness for purpose is usually based on the ability of an institution to fulfil its mission or a programme of study to fulfil its aims’. This is an alternative way of expressing the slogan that quality is achieved by ‘doing the right things’ and ‘doing things right’. It goes beyond the notion of customer satisfaction or meeting clients’ needs, as fitness of purpose can examine notions of social usefulness – educational worth as well as individual satisfaction – and this makes it a suitable model for public educational institutions. In a study published in the collection of papers on the legitimacy of QM in higher education, Kohler (Reference Kohler, Weber and Dolgova-Dreyer2007: 66) illustrates the concept as a ‘quality spiral’ which states that ‘valid’ objectives lead to a ‘fitting concept’, with ‘true’ implementation.

2.5 Summary of the historical background

From this historical background we can see that QM principles and practices cover a broad range of different concepts: concepts that are not necessarily mutually consistent. In industrial contexts the emphasis is on efficiency and productivity, through careful analysis of processes and the basing of decisions on observed facts. In services, these are complemented by the development of guaranteed service standards and the primacy of the customer and customer satisfaction. Public institutions have seen quality in terms of legitimacy and accountability and have developed the concepts of fitness of purpose and for purpose. When we examine how these are applied in language education, we will see that the different aspects are often bundled together, without distinction or clarity, as ‘quality’.

4.1 Definitions and elements of quality in language education

The concept of quality is difficult to pin down and can be very subjective. ‘In trying to grasp the concept of quality in education, one starting point is to look for a definition. Theoreticians have struggled and come up with a variety of definitions, including quality being defined by the degree to which set objectives are achieved, added value, fitness for purpose, and client satisfaction’ (Thomas Reference Thomas2003: 232). The origin of the word is Latin qualis – ‘what kind?’ – and until relatively recently its use was neutral unless modified by an attribute such as low or high, so the concept was descriptive rather than prescriptive. The descriptive element of quality is crucial, as applications of systems for QM depend on being able to define what it is one is talking about.

When we talk about quality, we are concerned with defining the nature of the thing described, the resources used to provide it, and steps taken to make it better, with the following phases in any quality initiative (Brown & Heyworth Reference Brown and Heyworth1999):

• • Purpose – Why are we undertaking this activity? Is the purpose a ‘fit’ purpose?

• • Description – What is the nature of the thing we are talking about? What kind of thing is it?

• • Comparison – What are ‘good’ instances of the subject? What is ‘good practice’ in a particular activity? What criteria do we use for this?

• • Evaluation – How good an example of its kind is it? How does it compare with set standards?

• • Management – What can we do to ensure that quality is maintained? How can we make it better? What standards do we apply to this?

• • Guarantees – How do we know we can rely on the quality of a particular thing or activity? How can it be reliably accredited? By whom?

4.2 The purposes of language education

In addition to contributing to the obvious aim of acquiring communicative competence, QM in language teaching must have relevance to broader educational aims. These include the importance of language learning in cognitive development (Cummins Reference Cummins2000) and its role as a necessary element of successful learning in schools. It can also be a means of ensuring equality of opportunity for children of migrants and socially disadvantaged groups, and contribute to the acceptance of differences and respect for otherness. It can even be seen as a key factor in the exercise of democratic citizenship (Beacco & Byram Reference Beacco and Byram2007).

The aims of foreign and second (or third) language learning in academic settings differ from those of language education for migrants, as they do if languages are being learnt as part of school or university system rather than as a single goal in a language school or adult education.

4.3 Description, comparison and evaluation: models of QM

The concept of quality is concerned first with description – what kind of thing are we talking about? – as well as the practical means by which it might be improved. Comprehensive and systematic descriptions, such as the Common European Framework of Reference (CEFR) (Council of Europe 2001), and the associated frameworks of reference which have been developed from this model – for teacher education, for Content and Language Integrated Learning (CLIL), for pluralistic approaches to language teaching, and for evaluating resources – are necessary points of departure for setting objectives and for comparing and accrediting quality.

Descriptions lead naturally to a need for comparison: how can we define good instances of a particular activity? What do we mean by ‘good practice’ or ‘excellence’? Various audit systems have developed scales and grading systems for assessing the quality of institutions, and there are research projects which identify and describe good practice in different contexts, for example in the work of the European Centre for Modern Languages in Graz (ECML 2011).

The definition of what is ‘good’ raises the question of ‘how good is it?’ and the need for the evaluation of quality, which in turn requires the definition of what can be measured reliably and with validity. Classic quality theory in industry or services tends to be based on a mix of a number of different aspects (Heyworth, in Muresan Reference Muresan, Heyworth, Mateva and Rose2007):

• • Detailed management of processes in the three phases of design, implementation and outcome, with the aim of achieving ‘zero defect’ (i.e. fault-free operations) through inspection and control.

• • Viewing quality as a feature of the relationship with the client, whose satisfaction is the measure of the quality achieved, and ‘exceeding clients’ expectations’ is the goal.

• • Measuring the results or outcomes of the processes – financial success for a commercial enterprise, growth for an economy, and examination results for a school.

Cheng & Tam (Reference Cheng and Tam1997) cite seven models of quality in education: the goals and specifications model, the resources input model, the process model, the satisfaction model, the legitimacy model, the absence of problems model and the organisational learning model. They maintain that a combination of these models can provide a comprehensive approach to quality in education.

All of these models have been used as bases for the evaluation of quality in language education; school inspections have examined the processes by which curriculum is translated into syllabus, and how this in turn generates schemes of work and lesson planning. Assessment procedures involve a process from initial placement through progress testing to final evaluation of proficiency. In both state and private education, quality ‘charters’ or guarantees have been developed which define what the ‘providers’ of educational services promise to those who use them. In several countries league tables of examination results are used, with differing degrees of validity, to judge school performance. In England the inspecting authority for mainstream schools, Ofsted, makes these reports available to the public on its website (Ofsted 2012).

4.4 QM and accreditation

The proactive aspect in relation to quality seeks to give answers to the question ‘what can we do to make things better?’ and leads to QM. This is typically divided into quality assurance, carried out before and during an activity to make sure it is done well, and quality control, which checks the finished product. In language teaching activities, this covers a wide spectrum: the appropriateness and relevance of curriculum and syllabus, teacher performance, the quality of the resources used, the reliability and validity of assessment procedures, the amount and quality of learning which is achieved, and more general aspects such as the security and well-being of participants and the educational values of the school (EAQUALS 2010; British Council 2012). As a consequence of this, QM is concerned with features that apply to the whole of an institution, such as the way in which its leadership can create a culture of quality, often involving methodologies like action research, quality circles – in which small groups of staff meet to propose ways of improving quality – client focus groups, encouragement of innovation, and intensive use of class observations. It functions at an individual level, too, through the development of reflective practice and continuous professional development.

The final aspect of QM is directed towards the clients and other stakeholders involved in the process. How can they make an informed choice (when they have one) from among providers of language teaching? What independent guarantees of quality can be obtained to build confidence and trust in the institution? Accreditation procedures are described in more detail in section 8 of this review.

5.1 Methodologies for Quality Assurance

Quality Assurance (QA) is a concept that covers all policies and actions implemented within a quality system. It is a broader term than quality control (which often refers only to post hoc controls) and usually refers to the internal workings of an organisation.

Muresan et al. provide, in their volume Qualitraining (Reference Muresan, Heyworth, Mateva and Rose2007), an overview of the applications of QA in language teaching; in educational environments, approaches to QA have adapted general QA principles to an educational environment. The general methodologies are:

• • the drafting and presence of mission statements setting quality objectives

• • the explicit development of a culture of quality in an organisation

• • the identification of different forms of leadership and ensuring that these function in a way conducive to the creation of ‘quality’

• • regular quality reviews of how far quality objectives have been attained.

More specific mechanisms include:

• • benchmarking: choosing standards, often in relation to the general performance of other institutions in the same field, or to competitors against whom performance will be judged

• • identifying indicators which will demonstrate how far the standards are being met

• • drafting client charters which state what the institution promises or guarantees to its stakeholders, and displaying these prominently

• • procedures for observation, data gathering, client surveys and questionnaires as ways of verifying quality and assessing what indicators are showing about quality.

5.2 Accountability management

The approaches just described – reflective practice, different forms of observation, action research and commitment to innovation – are all aspects of the processes through which QM can be practised in language education. They are more concerned with the internal workings of schools and teachers’ experience than with outcomes, and their use in QM has often been an offshoot of concerns with professional development or teacher training.

Accountability management adapts the client satisfaction models developed in the application of QM in the service sector. Farmer (Reference Farmer2006) argues that language teaching has been slow to develop practices that respond coherently to clients’ needs and that research has been inward-looking rather than centred on issues related to client accountability. Language education could be considered as a ‘service industry like any other requiring attention to the parameters set by a quality assurance programme’ (Farmer Reference Farmer2006: 161). He cites a TESOL paper (TESOL 2000) which establishes quality indicators and performance standards for adult ESOL teaching programmes under headings which include programme structure, administration and planning, curriculum, instruction, retention and transition, assessment and learner gains, staffing, professional development and staff evaluation.

Farmer suggests that, although it is very difficult to establish objectively measurable indicators in an educational setting and elements of public trust and educational independence of judgment are important, too, QM procedures nevertheless need to take greater account of client needs. He suggests, for example (p. 166), that indicators for curricula could include ‘setting goals, objectives, outcomes, approaches, methods, materials, technological resources and evaluation measures that are appropriate for clients’ needs and goals’. Possible indicators for good programme structure include the presence of a clear mission statement and an organisational philosophy which encourages input from internal and external stakeholders. The notion of ‘stakeholder’ in language education is a complex one – stakeholders might be the learners in the classroom, parents, other educational institutions (i.e. school accountability to universities), employers or society at large. How can the needs of all these stakeholders be reconciled? How should long-term development needs be balanced against short-term wishes which affect motivation and engagement in learning? In service industries the notion of client and stakeholder is especially complex, with organisations seen as working within a system of mutual service provision, each department and category of staff having responsibility for meeting the needs of others and the right to claim efficient services from them.

Gottlieb & Nguyen (Reference Gottlieb and Nguyen2007) examine the systems for accountability management in foreign language provision in schools in the United States and emphasise the need for data and assessment results to be made accessible to learners, teachers, school and state administrators over the full period of foreign language instruction. They propose a ‘pivotal portfolio’ as a means for doing this: a portfolio which gathers all the relevant data and is available to all stakeholders, its form differing according to need.

5.3 Programme evaluation

This is an approach which attempts an overall view of all the elements of QM as a way of judging the effectiveness and efficiency of an institution or a particular programme. It seeks to provide regular and reliable information to managers of teaching institutions on how far performance targets are being met. It is meant to enable institutions to ensure quality service to students, and, importantly, to demonstrate that they are doing this to the authorities or other bodies to whom they are accountable. To set up such a system requires the design of meaningful and useful indicators and the involvement of both the beneficiaries of the service and the teachers and other staff concerned.

Mackay et al. (Reference Mackay, Wellesley, Tasman, Bazerga, Rea-Dickins and Germaine1998) describe a system of programme evaluation set up in a language centre in a public sector institute for oil and gas technology in Indonesia. The first step in the process was to identify the key characteristics of the language centre as a prelude to determining key indicators. These characteristics would apply to most institutions: their environment, governance, the programmes they offer, their teaching and support staff and the teaching resources available. Mackay et al. argued that good performance indicators are:

• • relevant – addressing policies and practices related to established goals and specific responsibilities

• • informative – providing information about factors which influence outcomes

• • acceptable – relating to goals which have been agreed upon and can be defined operationally, and used in training if necessary

• • cost effective – can be implemented without excessive use of resources and time.

For innovation and quality measures to be successful, involvement in decision-making is important. There are many examples of unsuccessful top-down change, in which innovation is rejected because it is imposed without the involvement of those who have to carry it out. Fullan (Reference Fullan1993: 163) points out that changes in schools will not succeed unless ‘teachers become simultaneously and seamlessly inquiry-oriented, skilled, reflective and collaborative professionals’ and that this is only possible if the school environment encourages this kind of involvement and collaboration. Karavas-Doukas (Reference Karavas-Doukas, Rea-Dickins and Germaine1998) describes how a major top-down introduction of communicative methodology in Greek schools was ineffective because it did not involve teachers, whose educational culture and mindset did not buy into the changes.

5.4 A culture of quality and leadership

A coherent approach to QM can only be achieved if the institution can introduce what Rose (Reference Rose and Muresan2007) calls a ‘culture of quality’, requiring explicit statements of quality objectives and leadership and including regular reflective practice, with staff taking personal responsibility for the quality of their work and organisational procedures to recognise and reward it. The commitment to quality needs to be documented in mission statements, charters and strategic plans that are communicated to all stakeholders. Aims and objectives for the institution need to be set, making the values of the institution explicit, describing quality, identifying clients, both internal and external, and making commitments and promises which involve all those working in the institution. Muresan et al. (Reference Muresan, Heyworth, Mateva and Rose2007) includes examples of mission statements and charters mainly from private institutions and associations around Europe directly connected to language education.

Leadership is an essential feature of a culture of quality, but not just top-down leadership, as it can take numerous forms: moral leadership, grounded in personal and professional values; strategic, focused on policy and organisation; managerial, focused on function and tasks; and collaborative, based on democratic principles and encouraging the participation of all stakeholders. Interpersonal leadership centres on collaboration, interpersonal relationships and distributed leadership that engages a range of people in leadership activity; it extends the boundaries of leadership beyond delegation. ‘Effective leaders will draw from a personal repertoire based on “fitness for purpose”. Whatever the approach to leadership style, it is the leadership processes which underpin this that promote quality’ (Rose Reference Rose and Muresan2007: 30).

Another qualitative aspect of educational leadership is related to the capacity-building of the institution. This involves the development not just of professional, technical competence but also of ‘emotional capacity’ and intercultural capacity. Timperley & Philips’ (Reference Timperley and Phillips2003) study in New Zealand noted that teachers’ feelings of self-efficacy had an effect on learner achievement and that learner achievement promoted positive attitudes in teachers. The authors (p. 639) discovered that ‘a complex interplay of new knowledge, how to teach it and unanticipated changes in children's achievement helped to achieve changes in teachers’ feelings of self-efficacy and in their expectations of students’. They conclude that teachers’ ‘beliefs and improvement in their practice need to be addressed simultaneously in order for professional development to have a lasting effect on the expectations and achievement of low income children’. Bartolomé (Reference Bartolomé and Beykont2002) describes how ideological and affective cultural changes in teachers’ attitudes enable them successfully to bridge the cultural gap between them and learners from disadvantaged environments.

5.5 Identifying best practice and benchmarking

In benchmarking approaches to QM there is an attempt to identify best feasible practice and establish it as the target standard. The concept of best practice is widely used as a tool in QM, but it has been questioned. Aalto et al. (Reference Aalto, Abel, Atanasoska, Boeckmann and Lamb2011) set out to determine best practice in teaching the language of schooling as a subject (i.e. French in schools in France) in multilingual classrooms. They discovered problems with the concept of best practice: teaching and learning are so context-determined that what is best in one situation is often not applicable elsewhere, so they reduced their ambitions to a description of ‘practice’. They found that successful practices are often successful because of the capacity for innovation, creativity and ability to collaborate of those who initiate them, and practice in complex situations cannot be simply transferred from one setting to another.

Rea-Dickins & Germaine (Reference Rea-Dickins and Germaine1992) list a series of evaluation activities to enable teachers to assess, for example, their use of materials or how to motivate learners. They also use reflective checklists to identify useful practice which could be adopted.

One of the uses of the CEFR has been the application of the levels as benchmarks either for individuals, classes, schools or national curricula. This has had a beneficial effect in setting clear aims, especially when linked to the use of portfolio and self-assessment procedures, but can also have a standardising effect, especially if learners are assigned to a broad level, rather than to a differentiated profile. Cambridge ESOL (2012) offers a language benchmarking service that uses a series of tests and assessment activities that are compared against international standards. Techniques such as diagnostic testing and linguistic gap analysis have been used in a number of countries as a way of setting objectives for their language teaching programmes. The European Language Survey – SurveyLang (2012) – has compared average levels achieved by learners in different countries and this data is likely to be used by governments to set benchmarks for learner achievement.

To summarise this section on methodology for quality assessment: there are established approaches to achieving good practice in QM in education. The process of establishing quality depends on a coherent implementation of regular long-term use of such tools as reflective practice, different forms of observation, action research as an instrument for change and innovation, to establish the involvement of all staff in an evaluative and innovative process. To implement this coherent approach requires leadership in the creation of a quality culture and a combination of top-down and bottom-up approaches. In the context of language education, especially in schools, issues related to cultural and linguistic identity are crucial. It is probably fair to say that relatively few institutions manage to combine all the features described in this section; there is little emphasis on quantitative measurement of their effectiveness and almost none on measuring the link between the various approaches and learner outcomes.

7.1 Criteria and standards for quality of curriculum, learning and teaching

The CEFR has several features that are relevant to quality definitions and to QM systems. First of all, the wide acceptance of the CEFR levels as the basis for curriculum design, materials production and examinations means that there are objective standards for comparing achievement in different learning environments and for different languages. It enables benchmarks to be set for the target levels of achievement and the amount of time devoted to study; it allows international comparisons of attainment to be made and helps schools to map out their curricula more coherently. This broad acceptance can also have drawbacks, however, with the CEFR levels used for political aims such as the imposition of rigid national curricula. The French education system, for example, sets CEFR level B1 as the aim for the first foreign language in the Baccalauréat, without any reference to the profiling of individual skills proposed in the CEFR.

Although the CEFR claims that it does not wish to ‘tell practitioners what to do, or how to do it’ (p. xi), the ‘model’ is in fact widely quoted, and an emphasis frequently placed on the notions of action orientation and learner-centredness both to inform didactic approaches, and to act as implicit quality criteria. Language schools in France use the term ‘l'approche actionnelle’ as the counterpart to the ‘communicative method’ or ‘approach’ in English-teaching environments. The description of language learning as the activation of competences – defined (p. 9) as ‘the sum of knowledge, skills and characteristics that allow a person to perform actions’ – to accomplish social tasks is widely accepted.

The CEFR also implies a privileged position for language teaching in curricula, as a vehicle for the development of personal and cultural identity, the exercise of democratic citizenship and the acquisition of autonomy and learning skills which can be applied to other domains. In general, the CEFR and its comprehensive descriptions have a major part to play in any applications of QM in language education, especially those related to assessment. There are, however, pitfalls in thinking that CEFR ‘compliance’ – a term often used – is the answer to all quality issues in language education. The CEFR does not, for example, say very much about cultural aspects of language learning, nor do the descriptions of an action-centred approach address the need for learning and teaching to be informed by a coherent learning theory, or sets of learning theories.

Another framework produced by the Council of Europe – CARAP-FREPA, a Framework of Reference for Pluralistic Approaches to Language Learning and Teaching (Candelier et al. Reference Candelier, de Pietro, Facciol, Lorincz, Pascual and Schröder-Sura2011) – addresses issues of plurilingual and intercultural approaches to language learning. At its core is a set of descriptors which start from the view that language learning necessarily involves at least two languages – the target language and the learner's language – and can usefully involve more than two. The authors maintain that these aspects are an essential element of efficiency and quality in language education.

In the United States, the National Standards for Foreign Language Education (2005) set quality standards for language teaching that put the emphasis on what students can do as the outcome of their learning, identifying cognitive development and intercultural competences as well as communication skills.

These three examples of scales and standards for teaching indicate that QM should relate not only to the development of communicative competence, but to a wide range of educational and social purposes.

7.2 Descriptions of the quality of teachers

What do we mean by ‘good’ teachers and ‘good’ teaching? How much of this aspect of quality is common to teachers of all subjects and how much is specific to language teaching? In order to establish quality criteria and standards we need descriptive frameworks. There are a number of descriptions of quality requirements both for teachers in general, and for teachers of languages specifically. Typically they are based on a competence model, with competence defined generally as a combination of values (or attitudes), knowledge and understanding and skills.

7.3 Quality of examinations and assessment

The most obvious quality criteria for examinations are those of reliability, validity and practicality (see, for example, Genesee & Upshur Reference Genesee and Upshur1996). ALTE (the Association of Language Testers in Europe) has, in addition, developed a set of quality standards for examinations to which all members of ALTE adhere (Saville Reference Saville2005). In its code of practice all members are committed to meeting the standards in examination development, as well as in interpreting the results, striving for fairness, and providing information for examination takers. It sets minimum standards for the development of examinations as part of an auditing system which aims to be developmental, impartial and supportive (Saville Reference Saville2005). Criteria of quality in test construction include:

• • examinations based on a theoretical construct, such as a model of communicative competence

• • description of the purpose and context of use of the examination

• • criteria for selection and training of test constructors and expert judgment

• • comparability of parallel examinations in terms of content, stability, consistency and grade boundaries

• • evidence of claims that the examination is linked to an external reference system such as the CEFR.

The Council of Europe has produced a manual (ALTE 2009) for the alignment of examinations and tests to the CEFR. It provides a methodology for constructing examinations and combines descriptions of the language learning and teaching model of the CEFR and considerations of reliability, validity, fairness and ethics in testing. It points out (p. 11) that there can be no single standard for doing this and that it is important to gather multiple kinds of evidence to support quality standards in examinations. A project called RELEX (Noijons 2010) from the European Centre for Modern Languages produced guidelines for applying the manual in local situations.

The European Association for Language Testing and Assessment – EALTA –has produced a similar document: ‘Guidelines for good practice in language testing and assessment’ (Erickson & Figueras Reference Erickson and Figueras2011), translated into 35 languages and complementing the ALTE standards: ‘members of the association are committed to adhering to principles of transparency, accountability and quality’. The guidelines are all in question form and include standards for training of teachers in testing and assessment and for approaches to assessment in the classroom as well as more formal aspects of test and examination construction. The 14 questions on quality control ask assessors to specify aspects such as piloting of testing, double-marking as routine practice and statistical data-gathering.

The ALTE and EALTA guidelines have broad international application and adapted forms of these are used by the Goethe Institut for German, the Instituto Cervantes for Spanish and the Centre International d'Etudes Pédagogiques and the Alliance Française for French.

7.4 Quality aspects of resources

In 1998 the European Commission produced a ‘Quality guide for the evaluation and design of language learning and teaching programmes and materials’ (Lasnier et al. 1998). Its emphasis was more on materials than programmes, and it formulated a set of nine quality principles – relevance, transparency, reliability, attractiveness, flexibility, generativeness, participation, efficiency and socialisation – integrated into a guide and CD-ROM which provided a self-assessment tool to help in the design and implementation of materials and in judging the outcomes.

In ‘Approaches to materials design in European textbooks: Implementing principles of authenticity, learner autonomy and cultural awareness’ (Fenner & Newby Reference Fenner and Newby1999), materials design and the way materials are used are examined, first, from the point of view of authenticity, which includes the use of ‘real’ texts, authentic activities in relation to the input in the materials and what Newby calls ‘personal authenticity’. This refers to the fact that materials should encourage the learner to become personally involved in the way s/he uses language to express authentic feelings or communicate authentic opinions. The second principle is that of learner autonomy, and states that materials should provide opportunities for learner choice (of the level of text, for example), learner self-assessment and reflection on objectives, progress and use of the language. Many recent coursebooks for children, in particular, include portfolio-inspired activities with self-assessment checklists and opportunities to reflect on preferred learning styles. The third principle is that of the need for learning materials to promote cultural awareness.

These principles are reflected in good practice in materials production and publishing, though there is perhaps less emphasis on authenticity of text in ELT materials than in those for French, Spanish and German teaching, in which the defence and propagation of national or language area and cultural aspects are given greater importance. Many major publishers declare a commitment to conformity with CEFR levels as one of their quality standards; others use additional aspects of resource management such as modern technology with resources for interactive whiteboards, web links to books, and iPhones and iPads as supports for resources.

8.1 Higher education

Most QM applications in higher education are part of the quality policy of the whole institution; there are tensions, however, between academic freedom and increasing demands that universities should be accountable. A Council of Europe report on Higher Education Governance (cited in Weber & Dolgova-Dreyer Reference Weber and Dolgova-Dreyer2007) recommends that universities should avoid micromanagement and that quality assessment should be built on trust and be internal rather than external. The Lisbon declaration (European University Association 2007) proposed over 100 quality indicators for higher education. Kok (Reference Kok2004: 43) pointed out that the very proliferation of these tended to make them self-defeating.

As a result, the literature on QM in university contexts tends to be methodological and procedural, rather than content-based. Costello (Reference Costello, Weber and Dolgova-Dreyer2007) describes the process as one of finding the answers to four questions: What is the institution trying to do? How is the institution trying to do it? How does the institution know it works? What can the institution change in order to improve? This assumes a certain legitimacy for the institution and implied trust in its mission and operations.

There have been initiatives to provide standards for language centres in universities. CercleS is an association of 34 university language centres in 12 countries which have produced the Wulkow Memorandum (2010). It expresses their commitment, though without any auditing system to check it, to QM procedures – internal and external reviews and cross-European benchmarking frameworks – to further their aims of ‘fostering and developing not only linguistic but also transferable professional skills such as

• • The ability to communicate across cultures and to work in inter-cultural teams

• • The strategies for life-long learning

• • The ability to transfer academic skills to new working environments

• • The ability for self-reflection on and self-evaluation of communicative behaviour

• • The ability to negotiate communication effectively in different working environments’.

8.2 Education of migrants

The purposes of language education for migrants are social and political as well as linguistic. For the children of migrants the dominant theme is effective integration into the school system; for the migrants themselves, social insertion and the practicalities of life in society are paramount. This area is one in which it is easier to identify bad practice than good. In a number of countries there is a tendency to see the children only as ‘problems’ for the school system and their language difficulties often lead them to being placed in classes for those with special needs. Until quite recently there was a practice to discourage the use of the L1, both at school and at home, with negative consequences for the development of social identity.

Debaene (Reference Debaene, Komorowska and Aleksandrowicz-Pedich2010) carried out a comparative study of the education of Polish migrant children in France and Ireland and identified a number of areas of good practice. Acknowledgement of the children's L1 and encouraging them to use and develop it seems to contribute to successful integration into the school. Providing language support in close complement to mainstream teaching seems to work better than providing special and segregated classes; encouraging children to produce creative work in the L1 and then to translate it enables children to use their cognitive capacities instead of being restricted because of their limited language capacity. The work of Cummins (Reference Cummins2000) has had enormous influence in developing these approaches: he states that they have been successful for migrants in Canada. A whole-school approach which encourages multicultural and plurilingual education for all pupils, not just children speaking immigrant languages, also contributes to successful cognitive development for all. Systematic vocabulary development in classes with bilingual children in the Cambridge (UK) area was also found to have beneficial effects for mother tongue children from socially disadvantaged families (Alexopoulo & Katsos Reference Alexopoulo and Katsos2011). Aalto (Reference Aalto, Abel, Atanasoska, Boeckmann and Lamb2011) describes effective practices in integrating other cultures into the teaching of the language of schooling as a subject, again with an emphasis on the importance of establishing links between the home culture and that of the school and its social environment.

The French version of the Threshold level, the Niveau Seuil (Coste et al. Reference Coste, Courtilion, Ferenczi, Martins-Baltar and Papo1983), added language activities specifically related to the needs of adult migrants to the more travel-related descriptors of the English Threshold version. These included practical aspects of dealing with bureaucracy, understanding regulations and form-filling. Rossner, in a ‘Thematic Study’ on a Council of Europe project on language education for adult migrants (2008: 5), states that ‘adult migrants have some of the most urgent language learning needs of any language learner. The range of pressing practical concerns about day-to-day life may, however, make it hard for migrants to give language learning the priority it requires’. To meet these needs he proposes a system of QM inspired by those established in other sectors (specifically the model developed by EAQUALS). Standards would ensure that appropriate curricula were developed taking account of CEFR levels as well as the intercultural needs of migrants. There is a need to introduce standards for the quality of teaching methods, staff, the quality of providers and course delivery, which would, ideally, include specific accreditation for migrant education and a code of practice referring to international standards, self-assessment and training guides for managers. These standards could be embodied in charters which would include commitments related to integration, citizenship and social cohesion, human rights (including the right to education) and equality of opportunity.

8.3 Young learners

There is not a great deal of literature dealing specifically with QM aspects of language teaching for young learners. Nikolov & Curtain (Reference Nikolov and Curtain2000: 12) regard the following criteria as common aspects of successful language teaching in primary schools across a range of countries: ‘the importance of focusing on meaning, the integration of language instruction with the mainstream curriculum, a task-based and content-based approach, the need of fun and success in the classrooms, and the learning to learn element leading to the autonomy of learners in early language programs’. McKay (2000: 225), writing about primary foreign languages in Australia, considers the purposes of primary language teaching to be the fostering of intercultural relationships and a contribution to the future learning of all subjects: specifically languages. One problem is how to achieve coherent continuity with secondary education. ‘Pleasant early classroom experiences and initial success are later influenced by other important variables learners encounter, for example the lack of continuity, different methodology, and how useful they perceive the target language at a later stage of their studies’ (ibid.).

In a number of countries the teaching of languages has been complemented by the development of ‘Eveil aux Langues’ (called ‘language awareness’ by Hawkins Reference Hawkins1987, but now generally translated as ‘awakening to’ or ‘discovery of languages’). Perregaux (Reference Perregaux1994, translated by the author), in an account of the application of the concept in Geneva, identifies its aims as acceptance and legitimacy for the languages of all the children in school, raising awareness of plurilingualism as an aspect of the immediate and general environment, and a structuring of knowledge about the world and others.

The approach has been widely used in primary education in France, Switzerland and Spain and is a key aspect of the Council of Europe's approaches to plurilingual education, as described in ‘A platform of resources and references for plurilingual and intercultural education’ www.coe.int/t/dg4/linguistic/langeduc/le_platformintro_EN.asp? (Council of Europe 2011).

The descriptions of the different purposes, objectives and standards for language teaching and learning in the various sectors makes it clear that there cannot be a single approach to QM for languages, and that the requirements of an ‘education of quality for all’ must be defined in relation to each specific educational environment – at national, regional, institutional and, perhaps, even classroom level.

8.4 Institutional quality accreditation

The range of different possible approaches to accrediting quality are well summarised by Kohler (Reference Kohler, Weber and Dolgova-Dreyer2007: 65) who identifies six possible criteria for applying QM to higher education institutions. He lists excellence, fitness of and for purpose, compliance with standards, compliance with directives (e.g. curricula), client/customer satisfaction, value for money/time invested (efficiency), individual enhancement and (institutional) capacity for change. In fact, QM programmes have often suffered from trying to fit quality into a single concept – all of the above approaches are relevant in different ways.

In the next section we will describe various ways in which quality control, recognition and accreditation schemes are designed and implemented. In most cases these criteria are not made explicit and tend to combine client satisfaction criteria with those related to compliance (with CEFR levels, for example), together with notions of excellence (which are not always clearly defined).

9.1 ISO certification

By 2009 over a million organisations held one of the ISO's 18,000 different certifications. ISO 9001 is a certification extensively used in educational organisations; its full title is ‘Quality management systems – Requirements’, which is why you can find the ‘label’ displayed almost anywhere, including public toilets, motorway service stations, and on the back of vans. ISO 9001 is designed generically and emphasises procedures rather than content, guaranteeing that such procedures have been properly documented, and that systems are in place for controlling quality, communicating with stakeholders and managing resources. There is a full description on the ISO website, www.iso.org.

The benefits of ISO 9001 for efficient management are significant, and institutions which have obtained certification have usually found it beneficial. However, many complement it with more specific educational or language education accreditation, as it cannot help much with the content of decisions on curriculum content or teaching methods and does not provide specific standards for these. Its emphasis on customer and stakeholder relationships is useful but leaves aside all the other issues related to educational purpose and values. It has sometimes been criticised (see Seddon Reference Seddon2000, for example) for being overly bureaucratic and not contributing to real quality.

The Conseil Européen de Normalisation (CEN) has recently drafted standards for language travel, after wide consultation with the language travel profession. It drafts minimum standards for customer information and supervision of minors in schools and during travel to and from the school, as well as basic standards for course information and the conduct of courses.

9.2 The methodology of accreditation schemes

There are many similarities between the approaches of the different accrediting and recognition bodies. Their standards are embodied either in a checklist (more usual in the case of accrediting bodies) or in a public statement, such as the charter of an association or commitment to a certain number of guarantees for customers. The purpose of the audit/inspection process is to verify that the institution is respecting the relevant standards or charters. Matheidesz (Reference Matheidesz2010: 37) describes standards for auditing/inspection systems as ‘inclusive, comprehensive and complete, evidence-based, tangible, non-prescriptive, reliable, developmental and flexible’. The process is usually organised in a series of phases:

1. 1. An institutional self-evaluation is carried out, often using checklists designed by the accrediting organisation. In the audits carried out by the Accrediting Council for Continuing Education and Training (ACCET: 2012), self-assessment is at the core of the process, and the visit to the institution is to verify the accuracy of the self-assessment. The 15 Goethe Institut schools in Germany work on a three-year QM cycle, with an institutional self-assessment in the first year, an internal mutual audit in the second year carried out with one of the other schools in the group, and an external audit in the third.

2. 2. The submission of documentary evidence is a key feature of accreditation schemes. Schools are usually asked to submit a substantial amount of documentation of academic and administrative evidence. The French quality label has a list of 85 indicators, all of which must be supported by documentary evidence; checking these against what actually happens is a major aspect of the audit. A good indicator of the fit between the accreditation process and the actual working of the schools is how many of these documents have to be specially prepared for the audit and how many are useful in the day-to-day workings of the institution. An audit visit to the school is usually carried out by at least two auditors and typically lasts two days or more, according to the size of the school. The visit includes physical inspection of the site, observation of classes, interviews with academic management staff, inspection of security and safety arrangements, meetings with teachers and with course participants, and relevant inspection of host families. Some audit schemes try as far as possible to observe all teachers and all course types present during the inspection visit, usually with brief 15–20-minute visits to each class. Others rely more on documentary evidence, with a relatively small sampling of the actual teaching. Most inspection systems finish the visit with a feedback session with the institution's management, in which they summarise their main findings.

3. 3. The accrediting visit is followed up by a written report which can vary in length; in some cases it can be a short ticking off (or not) of checklists of indicators; in other schemes the report includes substantial description and assessment of the auditors’ findings. Almost all accrediting bodies have some kind of moderating committee which accepts or modifies the proposed verdict of the auditing team. Good practice recommends that this should include representatives of both user groups and independent experts. Accreditation UK, for example, has representatives from the British Council, English UK and outside advisers. The French accreditation results are confirmed by a commission composed of the accrediting ministries and the various professional associations involved in the scheme. In most systems there is a right of appeal on the result.

A number of methodological issues are raised by the practice of language teaching accreditation, some relating to the legitimacy of accrediting bodies. This is not an issue when they are national ministries or bodies mandated by them, but is a concern when they are private associations or commercial accrediting bodies, which is the case with ISO, for example. There is currently no formal evaluation of accrediting bodies, although a new association, IFCES (the International Forum for the Certification of Educational Services), with nine members from Europe, North America and Asia, has been formed with the mission ‘to contribute to the improvement of quality in the provision of learning services by developing international expertise to support certification against ISO 29990 and related standards’ (www.ifces.org). In general, legitimacy depends on a combination of voluntary acceptance and the perceived values and transparency of the relevant institutions.

Further methodological issues relate to the impartiality of inspectors. It is not easy to combine the requirements that auditors should be completely independent, and at the same time informed and professional. Most accrediting bodies have developed training schemes and codes of conduct for auditors, and there are increasing numbers of freelance auditors with relevant previous experience. Another issue is differentiation of quality assessment. A number of accreditation schemes have tried to do this. Accreditation UK has experimented with publishable statements, the French quality label has a series of one to three stars and EAQUALS identifies ‘areas of excellence’. None of these has provided a complete solution: the star system tends to devalue the one-star schools and the positive aspects of being accredited. There are also problems in drafting meaningful and accurate publishable statements.

Reliability of assessment is another issue. School environments are very context-specific and diverse, an especially relevant consideration in international accreditation schemes, and objectivity in the assessment of, for example, lesson observation is difficult to achieve. The various schemes declare that they do not wish to dictate didactic approaches or methodology, but there is nevertheless a perception on the part of schools that inspectors are looking for specific features such as group and pair work; sometimes the teaching is planned to provide what schools think the inspectors are looking for. Various schemes have developed ways to improve the reliability of grading: using audit reports (edited for anonymity) as case studies, for example, or using videoed lessons for joint grading sessions in auditor training activities. There are, however, genuine difficulties in the assessment of quality: how to come to a reliable overall assessment on the basis of a number of lesson chunks from different teachers at different levels, and combating observer fatigue after long intensive periods of observation. There is a danger of applying minimal general standards of acceptability in this area.

There is a choice to be made between holistic and checklist assessment; in order to achieve a high degree of reliability it is important to have measurable and observable indicators, so there is a temptation to create long checklists that contribute to the reliability of the schemes but can encourage teachers to concentrate on inessentials. A more descriptive and holistic approach demands the creation of an appropriate mode of discourse, and considerable professional confidence, but it offers more opportunities for formative assessment and real integration of accreditation as part of a coherent approach to QM.