Book contents
- Frontmatter
- Contents
- List of tables, boxes and figures
- List of contributors
- Preface
- 1 Introduction: changes in training
- 2 Workplace-based assessment methods: literature overview
- 3 Case-based discussion
- 4 The mini-Assessed Clinical Encounter (mini-ACE)
- 5 The Assessment of Clinical Expertise (ACE)
- 6 Multi-source feedback
- 7 Direct Observation of Non-Clinical Skills: a new tool to assess higher psychiatric trainees
- 8 Workplace-based assessments in psychotherapy
- 9 Educational supervisor's report
- 10 Portfolios
- 11 Annual Review of Competence Progression (ARCP)
- 12 Examinations in the era of competency training
- 13 Piloting workplace-based assessments in psychiatry
- 14 Developing and delivering an online assessment system: Assessments Online
- 15 A trainee perspective of workplace-based assessments
- 16 Conclusions
- Appendix 1 Assessment forms
- Appendix 2 Guide for ARCP panels in core psychiatry training
- Appendix 3 The MRCPsych examination
- Index
2 - Workplace-based assessment methods: literature overview
Published online by Cambridge University Press: 01 January 2018
- Frontmatter
- Contents
- List of tables, boxes and figures
- List of contributors
- Preface
- 1 Introduction: changes in training
- 2 Workplace-based assessment methods: literature overview
- 3 Case-based discussion
- 4 The mini-Assessed Clinical Encounter (mini-ACE)
- 5 The Assessment of Clinical Expertise (ACE)
- 6 Multi-source feedback
- 7 Direct Observation of Non-Clinical Skills: a new tool to assess higher psychiatric trainees
- 8 Workplace-based assessments in psychotherapy
- 9 Educational supervisor's report
- 10 Portfolios
- 11 Annual Review of Competence Progression (ARCP)
- 12 Examinations in the era of competency training
- 13 Piloting workplace-based assessments in psychiatry
- 14 Developing and delivering an online assessment system: Assessments Online
- 15 A trainee perspective of workplace-based assessments
- 16 Conclusions
- Appendix 1 Assessment forms
- Appendix 2 Guide for ARCP panels in core psychiatry training
- Appendix 3 The MRCPsych examination
- Index
Summary
This chapter provides a short introduction and background to some of the WPBA methods: the long case, Assessment of Clinical Expertise (ACE), multi-source feedback (MSF), mini-Clinical Evaluation Exercise (mini-CEX), Direct Observation of Procedural Skills (DOPS), case-based discussion, and Journal Club Presentation.
For each assessment method, what the approach practically involves is first defined, before considering the key messages and research evidence from the literature.
The long case
Across most medical specialties, the traditional long case has historically occupied a central and critical role in the evaluation of clinical skills (Weiss, 2002). In the long case, trainees are given 30–60 min of unobserved time to interview and examine an unstandardised patient, before presenting and discussing the case with one or more examiners. This assessment can take up to an hour.
For examination purposes, the underlying belief is that within a single long case, active and usually unstructured questioning by an experienced examiner can determine a trainee's competency. The key assessment strength of this approach is that trainees are required to formulate differential diagnosis and management plans for real patients in an actual clinical setting. However, the method has been criticised for the poor reliability of its assessments, and the lack of direct examiner observation of the trainee/patient encounter (reducing the validity of assessments). Consequently, a new instrument for undertaking long-case assessments with psychiatric trainees has been developed – the ACE.
Reliability of the long case
Concerns have repeatedly been voiced about the reliability of information generated through the traditional long case. This is because it is usually based upon a single patient encounter and unstructured examiner questioning. This causes three problems (Norcini, 2001, 2002).
Inter-case reliability. The long case is typically based on one in-depth patient encounter. However, trainees’ performances will vary across cases, reflecting their strengths and weaknesses across different patient problems, and the different responses of patients to them. Good inter-case reliability requires that a larger number, and broader sample, of different cases are included.
Interrater reliability. The long case is typically based upon the scores of no more than two examiners. Research shows that examiners differ in their ratings when assessing the same event. Good interrater reliability requires that multiple examiners are used.
- Type
- Chapter
- Information
- Workplace-Based Assessments in Psychiatry , pp. 14 - 27Publisher: Royal College of PsychiatristsPrint publication year: 2011