Hostname: page-component-7f64f4797f-n5rmq Total loading time: 0 Render date: 2025-11-04T08:18:00.291Z Has data issue: false hasContentIssue false
  • English
  • Français

The challenge of training supervisors to use direct assessments of clinical competence in CBT consistently: a systematic review and exploratory training study

Published online by Cambridge University Press:  14 October 2016

Maria E. Loades  and
Peter Armstrong
Maria E. Loades*
Affiliation:
Department of Psychology, University of Bath, Claverton Down, Bath, UK
Peter Armstrong
Affiliation:
Newcastle, UK
*
*Author for correspondence: Dr M. E. Loades, Department of Psychology, University of Bath, Claverton Down, Bath BA2 7AY, UK (email: M.E.Loades@bath.ac.uk).
Get access Rights & Permissions [Opens in a new window]

Abstract

Evaluating and enhancing supervisee competence is a key function of supervision and can be aided by the use of direct assessments of clinical competence, e.g. the Cognitive Therapy Scale – Revised (CTS-R). We aimed to review the literature regarding inter-rater reliability and training on the CTS and CTS-R to present exploratory data on training raters to use this measure. We employed a systematic review. An exploratory study evaluated the outcomes of a CTS-R supervisor training workshop (n = 34), including self-reported familiarity with and confidence in using the tool, and inter-rater consistency on three CTS-R subscales, pre- and post-training. CTS and CTS-R inter-rater reliability was variable, with evidence of rater training enhancing reliability, although the form, duration and frequency of such training is unclear. The exploratory study found that supervisors rated themselves as more familiar with and confident in using the CTS-R at the end of training compared to at the beginning. However, inter-rater reliability was poor at the beginning and end of the training. Rating competence requires supervisors to make qualitative judgements, which is inherently variable. Training raters has been shown to improve rater reliability, although this was not demonstrated in the exploratory study. Practice implications and future research priorities are identified.

Keywords

Cognitive therapyCognitive Therapy ScalecompetenceCTS-Rinter-rater trainingsupervisor training

Information

Type
Special Issue: International Developments in Supporting and Developing CBT Supervisors
Information
the Cognitive Behaviour Therapist , Volume 9 , 2016 , e27
Copyright
Copyright © British Association for Behavioural and Cognitive Psychotherapies 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Recommended follow-up reading

Milne, DL, Reiser, R (2016). Supporting our supervisors: sending out an SOS. The Cognitive Behaviour Therapist. doi:10.1017/S1754470X15000616.CrossRefGoogle Scholar
Muse, K, McManus, F (2013). A systematic review of methods for assessing competence in cognitive-behavioral therapy. Clinical Psychology Review 33, 484–499.Google Scholar
Watkins, CE, Milne, DL (2014). The Wiley International Handbook of Clinical Supervision. Oxford, UK: John Wiley & Sons Ltd.CrossRefGoogle Scholar

References

Adler, AD, Strunk, DR, Fazio, RH (2015). What changes in cognitive therapy for depression? An examination of cognitive therapy skills and maladaptive beliefs. Behavior Therapy 46, 96–109.Google Scholar
Bennett-Levy, J, Padesky, CA (2014). Use it or lose it: post-workshop reflection enhances learning and utilization of CBT skills. Cognitive and Behavioral Practice 21, 12–19.CrossRefGoogle Scholar
Bernard, JM, Goodyear, RK (2013). Fundamentals of Clinical Supervision. Boston: Pearson Education.Google Scholar
Blackburn, I-M, James, IA, Milne, DL, Baker, C, Standart, S, Garland, A, Reichelt, FK (2001). The Revised Cognitive Therapy Scale (CTS-R): psychometric properties. Behavioural and Cognitive Psychotherapy 29, 431–446.Google Scholar
Branson, A, Shafran, R, Myles, P (2015). Investigating the relationship between competence and patient outcome with CBT. Behaviour Research and Therapy 68, 19–26.Google Scholar
Brosan, L, Reynolds, S, Moore, RG (2006). Factors associated with competence in cognitive therapists. Behavioural and Cognitive Psychotherapy 35, 179.Google Scholar
Brown, LA, Craske, MG, Glenn, DE, Stein, MB, Sullivan, G, Sherbourne, C, Bystritsky, A, Welch, SS, Campbell-Sills, L, Lang, A, Roy-Byrne, P, Rose, RD (2013). CBT competence in novice therapists improves anxiety outcomes. Depression & Anxiety 30, 97–115.CrossRefGoogle ScholarPubMed
Bryant, MJ, Simons, AD, Thase, ME (1999). Therapist skill and patient variables in homework compliance: controlling an uncontrolled variable in cognitive therapy outcome research. Cognitive Therapy and Research 23, 381–399.CrossRefGoogle Scholar
Culloty, T, Milne, D, Sheikh, AI (2010). Evaluating the training of clinical supervisors: a pilot study using the fidelity framework. The Cognitive Behaviour Therapist 3, 132–144.Google Scholar
Devane, SM, Haddock, G, Lancashire, S, Baguley, I, Butterworth, T, Tarrier, N, James, A, Molyneux, P (1998). The clinical skills of community psychiatric nurses working with patients who have severe and enduring mental health problems: an empirical analysis. Journal of Advanced Nursing 27, 253–260.CrossRefGoogle ScholarPubMed
Dimeff, LA, Koerner, K, Woodcock, EA, Beadnell, B, Brown, MZ, Skutch, JM, Paves, AP, Bazinet, A, Harned, MS (2009). Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills. Behaviour Research and Therapy 47, 921–930.CrossRefGoogle Scholar
Dimidjian, S, Hollon, SD, Dobson, KS, Schmaling, KB, Kohlenberg, RJ, Addis, ME, Gallop, R, McGlinchey, JB, Markley, DK, Gollan, JK, Atkins, DC, Dunner, DL, Jacobson, NS (2006). Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology 74, 658–670.Google Scholar
Dobson, KS, Shaw, BF, Vallis, TM (1985). Reliability of a measure of the quality of cognitive therapy. British Journal of Clinical Psychology 24, 295–300.CrossRefGoogle ScholarPubMed
Dreyfus, HL, Dreyfus, SE (1986). Mind over Machine: the Power of Human Intuition and Expertise in the Age of the Computer. Oxford: Blackwell.Google Scholar
Ginzburg, DM, Bohn, C, Hofling, V, Weck, F, Clark, DM, Stangier, U (2012). Treatment specific competence predicts outcome in cognitive therapy for social anxiety disorder. Behaviour Research and Therapy 50, 747–752.Google Scholar
Gordon, PK (2006). A comparison of two versions of the Cognitive Therapy Scale. Behavioural and Cognitive Psychotherapy 35, 343.Google Scholar
Haddock, G, Devane, S, Bradshaw, T, McGovern, J, Tarrier, N, Kinderman, P, Baguley, I, Lancashire, S, Harris, N (2001). An investigation into the psychometric properties of the Cognitive Therapy Scale for Psychosis (CTS-Psy). Behavioural and Cognitive Psychotherapy 29, 221–233.CrossRefGoogle Scholar
Hayes, AF, Krippendorff, K (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures 1, 77–89.Google Scholar
Inskipp, F, Proctor, B (1993). The Art, Craft and Tasks of Counselling Supervision, Part 1 – Making the Most of Supervision. Twickenham: Cascade Publications.Google Scholar
Inskipp, F, Proctor, B (1995). The Art, Craft and Tasks of Counselling Supervision, Part 2 – Becoming a Supervisor: Twickenham: Cascade Publications.Google Scholar
Jahoda, A, Willner, P, Rose, J, Kroese, BS, Lammie, C, Shead, J, Woodgate, C, Gillespie, D, Townson, J, Felce, D, Stimpson, A, Rose, N, MacMahon, P, Nuttall, J, Hood, K (2013). Development of a scale to measure fidelity to manualized group-based cognitive behavioural interventions for people with intellectual disabilities. Research in Developmental Disabilities 34, 4210–4221.CrossRefGoogle ScholarPubMed
James, IA, Blackburn, I-M, Milne, D, Reichfelt, FK (2001). Moderators of trainee therapists’ competence in cognitive therapy. British Journal of Clinical Psychology 40, 131–141.Google Scholar
Keen, AJ, Freeston, MH (2008). Assessing competence in cognitive-behavioural therapy. British Journal of Psychiatry 193, 60–64.CrossRefGoogle ScholarPubMed
Kolb, DA (1984). Experiential Learning: Experience as the Source of Learning and Development. New York: Prentice-Hall Englewood Cliffs.Google Scholar
Kuyken, W, Tsivrikos, D (2009). Therapist competence, comorbidity and cognitive-behavioral therapy for depression. Psychotherapy and Psychosomatics 78, 42–48.Google Scholar
Landis, JR, Koch, GG (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159–174.CrossRefGoogle ScholarPubMed
Lysaker, PH, Davis, LW, Bryson, GJ, Bell, MD (2009). Effects of cognitive behavioral therapy on work outcomes in vocational rehabilitation for participants with schizophrenia spectrum disorders. Schizophrenia Research 107, 186–191.CrossRefGoogle ScholarPubMed
Maissi, E, Ridge, K, Treasure, J, Chalder, T, Roche, S, Bartlett, J, Schmidt, U, Thomas, S, Ismail, K (2011). Nurse-led psychological interventions to improve diabetes control: assessing competencies. Patient Education and Counseling 84, e37–e43.Google Scholar
McManus, F, Rakovshik, S, Kennerley, H, Fennell, M, Westbrook, D (2012). An investigation of the accuracy of therapists' self-assessment of cognitive-behaviour therapy skills. British Journal of Clinical Psychology 51, 292–306.Google Scholar
Milne, D (2007). An empirical definition of clinical supervision. British Journal of Clinical Psychology 46, 437–447.Google Scholar
Milne, D (2010). Can we enhance the training of clinical supervisors? A national pilot study of an evidence-based approach. Clinical Psychology & Psychotherapy 17, 321–328.Google Scholar
Milne, D, Baker, C, Blackburn, I-M, James, I, Reichelt, K (1999). Effectiveness of cognitive therapy training. Journal of Behavior Therapy and Experimental Psychiatry 30, 81–92.Google Scholar
Milne, D, Claydon, T, Blackburn, I-M, James, IA (2001). Rationale for a new measure of competence in therapy. Behavioural and Cognitive Psychotherapy 29, 21–33.Google Scholar
Milne, DL, Reiser, R (2016). Supporting our supervisors: sending out an SOS. The Cognitive Behaviour Therapist. doi:10.1017/S1754470X15000616.Google Scholar
Muse, K, McManus, F (2013). A systematic review of methods for assessing competence in cognitive-behavioural therapy. Clinical Psychology Review 33, 484–499.CrossRefGoogle ScholarPubMed
Nichol, D, MacFarlane-Dick, D (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31, 199–218.CrossRefGoogle Scholar
O'Donovan, A, Kavanagh, DJ (2014). Measuring competence in supervisees and supervisors In: The Wiley International Handbook of Clinical Supervision, pp. 458–467. Chichester: John Wiley & Sons, Ltd.Google Scholar
Reichelt, FK, James, IA, Blackburn, I-M (2003). Impact of training on rating competence in cognitive therapy. Journal of Behavior Therapy and Experimental Psychiatry 34, 87–99.Google Scholar
Rosenbaum, M, Ronen, T (1998). Clinical supervision from the standpoint of cognitive-behavior therapy. Psychotherapy: Theory, Research, Practice, Training 35, 220–230.Google Scholar
Roth, AD, Pilling, S (2008). A competence framework for the supervision of psychological therapies. Recuperado em 10, 2010.Google Scholar
Ryum, T, Stiles, TC, Svartberg, M, McCullough, L (2010). The effects of therapist competence in assigning homework in cognitive therapy with cluster C personality disorders: results from a randomized controlled trial. Cognitive and Behavioral Practice 17, 283–289.Google Scholar
Shafran, R, Clark, DM, Fairburn, CG, Arntz, A, Barlow, DH, Ehlers, A, Freeston, M, Garety, PA, Hollon, SD, Ost, LG, Salkovskis, PM, Williams, JM, Wilson, GT (2009). Mind the gap: improving the dissemination of CBT. Behaviour Research and Therapy 47, 902–909.CrossRefGoogle Scholar
Sharpless, BA, Barber, JP (2009). A conceptual and empirical review of the meaning, measurement, development, and teaching of intervention competence in clinical psychology. Clinical Psychology Review 29, 47–56.Google Scholar
Shaw, BF, Elkin, I, Yamaguchi, J, Olmsted, M, Vallis, TM, Dobson, KS, Lowry, A, Sotsky, SM, Watkins, JT, Imber, SD (1999). Therapist competence ratings in relation to clinical outcome in cognitive therapy of depression. Journal of Consulting and Clinical Psychology 67, 837–846.Google Scholar
Simons, AD, Rozek, DC, Serrano, JL (2013). Wanted: reliable and valid measures for the science of cognitive behavioral therapy dissemination and implementation. Clinical Psychology: Science and Practice 20, 181–194.Google Scholar
Stallard, P, Myles, P, Branson, A (2014). The Cognitive Behaviour Therapy Scale for Children and Young People (CBTS-CYP): development and psychometric properties. Behavioural and Cognitive Psychotherapy 42, 269–282.Google Scholar
Strunk, DR, Brotman, MA, DeRubeis, RJ (2010). The process of change in cognitive therapy for depression: predictors of early inter-session symptom gains. Behaviour Research and Therapy 48, 599–606.CrossRefGoogle ScholarPubMed
Trepka, C, Rees, A, Shapiro, D, Hardy, G, Barkham, M (2004). Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy and Research 28, 143–157.Google Scholar
Tversky, A, Kahneman, D (1974). Judgment under uncertainty: heuristics and biases. Science 185, 1124–1131.Google Scholar
Vallis, TM, Shaw, BF, Dobson, KS (1986). The Cognitive Therapy Scale: psychometric properties. Journal of Consulting and Clinical Psychology 54, 381–385.Google Scholar
von Consbruch, K, Clark, DM, Stangier, U (2012). Assessing therapeutic competence in cognitive therapy for social phobia: psychometric properties of the cognitive therapy competence scale for social phobia (CTCS-SP). Behavioural and Cognitive Psychotherapy 40, 149–161.Google Scholar
Watkins, CE, Wang, CDC (2014). On the education of clinical supervisors The Wiley International Handbook of Clinical Supervision (pp. 177–203): John Wiley & Sons, Ltd.Google Scholar
Weck, F, Grikscheit, F, Hofling, V, Stangier, U (2014). Assessing treatment integrity in cognitive-behavioral therapy: comparing session segments with entire sessions. Behavior Therapy 45, 541–552.Google Scholar
Williams, RM, Moorey, S, Cobb, J (1991). Training in cognitive-behaviour therapy: pilot evaluation of a training course using the Cognitive Therapy Scale. Behavioural Psychotherapy 19, 373–376.Google Scholar
Wittorf, A, Jakobi-Malterre, UE, Beulen, S, Bechdolf, A, Müller, BW, Sartory, G, Wagner, M, Wiedemann, G, Wölwer, W, Herrlich, J, Klingberg, S (2013). Associations between therapy skills and patient experiences of change processes in cognitive behavioral therapy for psychosis. Psychiatry Research 210, 702–709.Google Scholar
Young, J, Beck, AT (1980). Cognitive Therapy Scale: Rating manual (unpublished manuscript).Google Scholar
Submit a response

Comments

No Comments have been published for this article.