Hostname: page-component-848d4c4894-x24gv Total loading time: 0 Render date: 2024-05-31T02:18:59.445Z Has data issue: false hasContentIssue false

Brief scales for the measurement of target variables and processes of change in cognitive behaviour therapy for major depression, panic disorder and social anxiety disorder

Published online by Cambridge University Press:  21 November 2023

Erland Axelsson*
Affiliation:
Division of Family Medicine and Primary Care, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden Liljeholmen University Primary Health Care Center, Region Stockholm, Stockholm, Sweden Academic Primary Health Care Center, Region Stockholm, Stockholm, Sweden
Fredrik Santoft
Affiliation:
Center for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden
Josefin Särnholm
Affiliation:
Division of Psychology, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden
Brjánn Ljótsson
Affiliation:
Division of Psychology, Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden
*
Corresponding author: Erland Axelsson; Email: erland.axelsson@ki.se
Rights & Permissions [Opens in a new window]

Abstract

Background:

The measurement of process variables derived from cognitive behavioural theory can aid treatment development and support the clinician in following treatment progress. Self-report process measures are ideally brief, which reduces the burden on patients and facilitates the implementation of repeated measurements.

Aims:

To develop 13 brief versions (3–6 items) of existing cognitive behavioural process scales for three common mental disorders: major depression, panic disorder, and social anxiety disorder.

Method:

Using data from a real-world teaching clinic offering internet-delivered cognitive behavior therapy (n=370), we drafted brief process scales and then validated these scales in later cohorts (n=293).

Results:

In the validation data, change in the brief process scales significantly mediated change in the corresponding domain outcomes, with standardized coefficient point estimates in the range of –0.53 to –0.21. Correlations with the original process scales were substantial (r=.83–.96), internal consistency was mostly adequate (α=0.65–0.86), and change scores were moderate to large (|d|=0.51–1.18). For depression, the brief Behavioral Activation for Depression Scale-Activation subscale was especially promising. For panic disorder, the brief Agoraphobic Cognitions Questionnaire-Physical Consequences subscale was especially promising. For social anxiety disorder, the Social Cognitions Questionnaire, the Social Probability and Cost Questionnaire, and the Social Behavior Questionnaire-Avoidance and Impression Management subscales were all promising.

Conclusions:

Several brief process scales showed promise as measures of treatment processes in cognitive behaviour therapy. There is a need for replication and further evaluation using experimental designs, in other clinical settings, and preferably in larger samples.

Type
Main
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of British Association for Behavioural and Cognitive Psychotherapies

Introduction

Depression and the common anxiety disorders such as panic disorder and social anxiety disorder are among the worldwide leading causes of disease burden (Bandelow and Michaelis, Reference Bandelow and Michaelis2022; Liu et al., Reference Liu, He, Yang, Feng, Zhao and Lyu2020). Cognitive behaviour therapy (CBT) is a widely researched and recommended treatment for these conditions, with typical remission rates around 43–59% (Loerinc et al., Reference Loerinc, Meuret, Twohig, Rosenfield, Bluett and Craske2015; Santoft et al., Reference Santoft, Axelsson, Öst, Hedman-Lagerlöf, Fust and Hedman-Lagerlöf2019a). Certain processes are widely believed to maintain depression and anxiety over time, and are therefore targeted in mainstream CBT. Cognitive theory tends to emphasize how the formation of dysfunctional schemas and assumptions about the self and the world can influence information processing and result in problematic cognitive, emotional and behavioural responses (Beck and Haigh, Reference Beck and Haigh2014; Bennett-Levy et al., Reference Bennett-Levy, Butler, Fennell, Hackmann, Mueller and Westbrook2004). Behavioural theory instead focuses on the functional analysis of behavioural responses to the environment, and tends to emphasize the role of conditioning and the beneficial effects of increasing positively reinforced behaviours while reducing negatively reinforced safety and avoidance behaviours (Mowrer, Reference Mowrer1960; Sturmey, Reference Sturmey2007). Mainstream CBT aims to systematically (a) modify dysfunctional assumptions, (b) achieve beneficial changes in behaviour and environmental factors in accordance with functional analysis, or (c) a combination of the two. The measurement of process variables derived from cognitive behavioural theory can aid treatment development, open new avenues for research into processes, and support the clinician in following treatment progress.

In both research and clinical practice, several self-report questionnaires have been developed to measure key processes in therapy in accordance with cognitive behavioural theory. Here follows a list of relatively widespread self-report process measures of relevance for the present study. In depression, one existing process scale is the Automatic Thoughts Questionnaire (ATQ) which measures ‘the covert self-statements reported by depressives as being representative of the kinds of cognitions they experience’ (Hollon and Kendall, Reference Hollon and Kendall1980), i.e. cognitions indicative of dysfunctional schema and core assumptions targeted with cognitive interventions (Clark and Beck, Reference Clark and Beck2010). The ATQ was originally developed as a 30-item instrument, but has been shortened to 15- and 8-item versions with adequate psychometric properties (Netemeyer et al., Reference Netemeyer, Williamson, Burton, Biswas, Jindal, Landreth, Mills and Primeaux2016). Variations on the ATQ have been found to change with depression symptoms during CBT, and to mediate the effect of CBT on depressive symptoms (i.e. the ATQ appears to be a variable that changes contingent on the treatment, and as part of the same process as the outcome; Garratt et al., Reference Garratt, Ingram, Rand and Sawalani2007; Stice et al., Reference Stice, Rohde, Seeley and Gau2010). Also relevant for depression, the Behavioral Activation for Depression Scale (BADS; Kanter et al., Reference Kanter, Mulick, Busch, Berlin and Martell2006; Kanter et al., Reference Kanter, Rusch, Busch and Sedivy2008) is an existing measure of behavioural activation, behavioural avoidance, and rumination, which are key constructs of the mainstream learning theory account of depression (Kanter et al., Reference Kanter, Manos, Bowe, Baruch, Busch and Rusch2010; Martell et al., Reference Martell, Dimidjian, Herman-Dunn and Lewinsohn2013; Ramnerö et al., Reference Ramnerö, Folke and Kanter2016; Spasojevic and Alloy, Reference Spasojevic and Alloy2001). The conventional BADS has 25 items, although an abbreviated 9-item version has also been developed with psychometric evaluation rendering mixed results (Fuhr et al., Reference Fuhr, Hautzinger, Krisch, Berking and Ebert2016; Manos et al., Reference Manos, Kanter and Luo2011). There is some evidence indicating that an improvement in the BADS is typically associated with beneficial effects of CBT on depression symptoms (Manos et al., Reference Manos, Kanter and Busch2010), and that behavioural activation as measured using the BADS can be regarded a mediator of the treatment effect at least under some operationalizations (Seeley et al., Reference Seeley, Sheeber, Feil, Leve, Davis, Sorensen and Allan2019; van Luenen et al., Reference van Luenen, Kraaij, Spinhoven, Wilderjans and Garnefski2019). Also based on learning theory, the Reward Probability Index (RPI) measures the degree to which individuals are exposed to rewarding environmental events (first subfactor) and environmental suppressors (second subfactor); the former usually regarded a proxy measure of response-contingent positive reinforcement, and the latter referring to properties of the environment that discourage or prevent an individual from engaging in activities that result in response-contingent positive reinforcement (Carvalho et al., Reference Carvalho, Gawrysiak, Hellmuth, McNulty, Magidson, Lejuez and Hopko2011; Lewinsohn and Graf, Reference Lewinsohn and Graf1973; Martell et al., Reference Martell, Dimidjian, Herman-Dunn and Lewinsohn2013). The RPI is a 20-item instrument, and to our knowledge no abbreviated version has yet been developed. The RPI and its subfactors has been found to mediate the effect of behavioural activation and avoidance on mood in healthy adults (Carvalho and Hopko, Reference Carvalho and Hopko2011; Gill et al., Reference Gill, Kane and Mazzucchelli2019; Hill et al., Reference Hill, Buitron and Pettit2017). However, we are only aware of one adequately powered clinical trial where the RPI was analysed as a potential mediator of the beneficial effects of treatment for depression. This study focused on pregnant women and reported promising effects, but warrants replication (Dimidjian et al., Reference Dimidjian, Goodman, Sherwood, Simon, Ludman, Gallop, Welch, Boggs, Metcalf, Hubley, Powers and Beck2017).

In panic disorder, the Agoraphobic Cognitions Questionnaire (ACQ) incorporates 14 items pertaining to catastrophic thoughts relating to the ‘fear of fear’ (Goldstein and Chambless, Reference Goldstein and Chambless1978) in terms of physical consequences (e.g. having a heart attack or a stroke) and in terms of the loss of control (e.g. acting strange or going crazy) (Chambless et al., Reference Chambless, Caputo, Bright and Gallagher1984) that are indicative of dysfunctional assumptions of the kind typically targeted in cognitive interventions (Bennett-Levy et al., Reference Bennett-Levy, Butler, Fennell, Hackmann, Mueller and Westbrook2004). Supporting the idea of agoraphobic cognitions as a key target in treatment, a 2021 meta-analysis found that CBT for the anxiety disorders produces large average improvements in threat reappraisal that are especially pronounced in panic disorder (Draheim and Anderson, Reference Draheim and Anderson2021). The ACQ has been evaluated as a process measure in CBT, and has been found to reduce over the course of treatment and mediate treatment effects (Gloster et al., Reference Gloster, Klotsche, Gerlach, Hamm, Strohle, Gauggel, Kircher, Alpers, Deckert and Wittchen2014; Vogele et al., Reference Vogele, Ehlers, Meyer, Frank, Hahlweg and Margraf2010). Also relevant for panic disorder, the Behaviors Questionnaire (BQ) measures the existence of panic disorder-related avoidance and safety behaviours. These are behaviours targeted in mainstream CBT (Craske and Barlow, Reference Craske, Barlow and Barlow2014) that occur in response to panic-related anxiety or fear, and which are commonly believed to persist due to the resulting short-term reduction in anxiety, the lack of opportunities for learning new behaviours, and the lack of opportunities for disproving dysfunctional assumptions (Centre for Anxiety Disorders & Trauma (CADAT) at King’s College London, 2020; Marks and Mathews, Reference Marks and Mathews1979). Although we have been unable to identify a primary peer-reviewed publication, the most widespread version of the BQ appears to have 31 items grounded in cognitive behavioural theory, and the avoidance subscale appears to have been developed on the basis of the Fear Questionnaire by Marks and Mathews (Reference Marks and Mathews1979), which according to its primary publication has been found to change with treatment. Highlighting the potential benefits of the BQ as a behavioural process measure, a 2015 systematic review identified a higher baseline level of behavioural avoidance as the most consistent predictor of a smaller symptom improvement in CBT for panic disorder (Porter and Chambless, Reference Porter and Chambless2015). To our knowledge, no abbreviated version of the ACQ or BQ has ever been developed.

In social anxiety disorder, the Social Cognitions Questionnaire (SCQ) surveys the frequency and believability of catastrophic thoughts about social situations (Clark, Reference Clark, Crozier and Alden2001; Oxford Centre for Anxiety Disorders and Trauma (OxCADAT), 2022). Although we are not aware of a primary peer-reviewed publication detailing the development of the SCQ, the instrument consists of 22 items grounded in cognitive behavioural theory, appears to be relatively widespread which could be argued to speak for its face validity, and its internal consistency has been found to be excellent (Thew et al., Reference Thew, Ehlers, Grey, Wild, Warnock-Parkes, Dawson and Clark2020) including with regard to an adapted version for adolescents (Chiu et al., Reference Chiu, Clark and Leigh2021). The SCQ has been found to change with CBT, and to mediate the treatment’s effect on overall social phobia (Thew et al., Reference Thew, Ehlers, Grey, Wild, Warnock-Parkes, Dawson and Clark2020; Thew et al., Reference Thew, Kwok, Lissillour Chan, Powell, Wild, Leung and Clark2022). At least one longitudinal cohort study has also found a variation on the SCQ to be predictive of increased social anxiety as studied outside of treatment (Chiu et al., Reference Chiu, Clark and Leigh2021). Also relevant for social anxiety, the Social Probability and Cost Questionnaire (SPCQ) probes the perceived probability and cost of adverse social outcomes, indicative of dysfunctional assumptions that can be targeted in therapy (Foa et al., Reference Foa, Franklin, Perry and Herbert1996; Hoffart et al., Reference Hoffart, Borge, Sexton and Clark2009; Hoffart et al., Reference Hoffart, Borge and Clark2016; McManus et al., Reference McManus, Clark and Hackmann2000). Although the most widespread version of the SPCQ has 33 items (McManus et al., Reference McManus, Clark and Hackmann2000), an 8-item version has also been used (Hoffart et al., Reference Hoffart, Borge, Sexton and Clark2009; Santoft et al., Reference Santoft, Salomonsson, Hesser, Lindsäter, Ljótsson, Lekander, Kecklund, Öst and Hedman-Lagerlöf2019b). Probability and cost biases have been found to respond to CBT for social anxiety disorder (Benbow and Anderson, Reference Benbow and Anderson2018), and at least one study has been indicative of a reciprocal relationship between the SPCQ and social anxiety over the course of CBT (Santoft et al., Reference Santoft, Salomonsson, Hesser, Lindsäter, Ljótsson, Lekander, Kecklund, Öst and Hedman-Lagerlöf2019b). Another process scale is the 28-item Social Behavior Questionnaire (SBQ) which measures the presence of avoidance and safety behaviours common in social anxiety disorder, including strategies for impression management (Gray et al., Reference Gray, Beierl and Clark2019). These behaviours are at the core of most cognitive behavioural conceptualizations of the maintenance of social anxiety disorder, and are believed to maintain or perhaps even exacerbate dysfunctional patterns in cognition and emotion (Lervik et al., Reference Lervik, Hoffart, Knapstad and Smith2022; Wong and Rapee, Reference Wong and Rapee2016). In line with this view, avoidance behaviours including as measured using the SBQ have been found to mediate the effect of CBT on social anxiety disorder (Santoft et al., Reference Santoft, Salomonsson, Hesser, Lindsäter, Ljótsson, Lekander, Kecklund, Öst and Hedman-Lagerlöf2019b; Thew et al., Reference Thew, Kwok, Lissillour Chan, Powell, Wild, Leung and Clark2022). In summary, there are several existing self-report measures of targets of CBT for clinical depression, panic disorder, and social anxiety disorder.

For these scales to be more useful in clinical work and research, it would be preferable to develop shorter and yet psychometrically sound versions. Brief versions place less burden on the respondent, can be more easily combined with other scales, and administered on a repeated basis. This makes it easier to monitor progress and provide personalized feedback in the clinic, and more accurate conclusions can be drawn in research (Ziegler et al., Reference Ziegler, Kemper and Kruyen2014). The use of repeated measurement is also important for mediational analyses and investigations into processes of change (Kazdin, Reference Kazdin2007). In this study, we aimed to develop brief versions of the aforementioned CBT process scales, as based on data from a real-world teaching clinic offering internet-delivered CBT for depression, panic disorder, and social anxiety disorder. In order to establish a frame of reference and ensure that the beneficial properties of the original full scales were maintained, we also systematically investigated the psychometric properties of the original process scales including as potential mediators of the treatment effects in CBT.

Method

Design

We developed 13 brief CBT process scales at an ICBT teaching clinic run by master-level psychologist students and their clinical supervisors at Karolinska Institutet, Stockholm, Sweden, as described previously (Niles et al., Reference Niles, Axelsson, Andersson, Hedman-Lagerlöf, Carlbring, Andersson, Johansson, Widén, Driessen, Santoft and Ljótsson2021). This study was based on longitudinal data from all 674 patients attending the clinic between 2014 and 2021, with the exception of 11 patients who did not give informed consent for participating in research. Thus, we analysed data from 663 patients.

Participants

Advertisements for the ICBT teaching clinic were posted on social media and in newspapers, under the heading ‘Depression? Social phobia? Panic attacks?’. Patients that showed interest were instructed to complete an online self-report screening battery after which those who reported subclinical symptoms, suicidal ideation, severe depression, or an alcohol use disorder were referred to routine care. Those remaining completed an eligibility telephone interview with a psychologist student working under the supervision of a licensed psychologist. In order to be offered treatment, patients were required to meet full criteria for a primary diagnosis of depression, panic disorder, or social anxiety disorder as assessed using the Mini International Neuropsychiatric Interview (MINI; Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998). Patients were also required not to report manic or psychotic symptoms, meet full criteria for post-traumatic stress disorder, or to have an alcohol, substance use, or eating disorder. Although no reliability checks were conducted, previous studies have indicated that the MINI is a reliable instrument (Sheehan et al., Reference Sheehan, Lecrubier, Sheehan, Amorim, Janavs, Weiller, Hergueta, Baker and Dunbar1998) that is appreciated by clinicians (Pettersson et al., Reference Pettersson, Modin, Wahlstrom, af Winklerfelt Hammarberg and Krakau2018), and the diagnoses were validated by a licensed psychologist. The recruitment flow is illustrated in Fig. 1, and patient characteristics are listed in Table 1. In total, 663 patients were included, of whom 218 had received ICBT for depression, 183 ICBT for panic disorder, and 262 ICBT for social anxiety disorder. We divided the sample into a selection/training sample (n=370) of patients treated in 2014–2017 and a validation sample (n=293) of patients treated in 2018–2021.

Figure 1. Flowchart of the recruitment procedure at the Internet-delivered cognitive behavior therapy teaching clinic. AUDIT = Alcohol Use Disorders Identification Test; CBT = cognitive behavior therapy; MADRS-S = Montgomery-Åsberg Depression Rating Scale – Self report version; MDD = major depressive disorder; PD = panic disorder; SAD = social anxiety disorder; ITT = intention to treat.

Table 1. Sociodemographic and clinical characteristics of the two samples

ICBT, therapist-guided internet-delivered cognitive behavior therapy; MADRS-S, Montgomery-Åsberg Depression Rating Scale-Self-report version; LSAS-SR, Liebowitz Social Anxiety Scale-Self-report version; PDSS-SR, Panic Disorder Severity Scale-Self-report version; USS, upper secondary school, equivalent to International Standard Classification of Education (ISCED) 1997 level 3.

a Data on age missing for 19 patients (18%) in the validation depression group, 20 (26%) in the validation panic disorder group, and 13 (12%) in the validation social anxiety disorder group.

b Valid percentage, i.e. proportion of non-missing. Data on education missing for 5 patients (4%) in the selection/testing depression group, 1 (1%) in the selection/testing panic disorder group, 1 (1%) in the validation depression group, 2 (3%) in the validation panic disorder group, and 4 (4%) in the validation social anxiety disorder group.

Treatments

Patients were enrolled in 10 weeks of internet-delivered cognitive behaviour therapy (ICBT) for their primary diagnosis (depression, panic disorder, or social anxiety disorder) based on mainstream cognitive behavioural theory and guided by a masters-level psychologist student under the supervision of a clinical psychologist. An advantage of ICBT for the study of treatment processes is the highly structured treatment format which ensures that all patients are exposed to the same content, and reduces the risk that treatment components are omitted. It is also often argued that the supportive role of the therapist, who may devote as little as 10–20 minutes per patient and week, may reduce the influence of unintended therapist behaviours (so called ‘therapist drift’) (Andersson, Reference Andersson2016). The treatment was text-based and delivered via a secure web platform where each patient had a personal account, and logged in regularly to complete eight modules that resembled book chapters with bundled homework exercises. Access to new modules was given contingent on progress, meaning that the completion of module 1 was necessary to reach module 2, the completion of module 2 was necessary to reach module 3, and so on. Patients and therapists could also communicate via an asynchronous email-like messaging system where a typical message consisted of 100–400 words. The aim was to convey the same educational content, and to have the patient engage in the same strategies and behaviour changes, as in conventional face-to-face CBT even though the therapist was expected to devote less time per patient and week.

ICBT for depression was primarily based on behavioural activation. This protocol had not previously been empirically validated, but followed best practice learning theory principles found to be effective in randomized controlled trials (Cuijpers et al., Reference Cuijpers, van Straten and Warmerdam2007; Dimidjian et al., Reference Dimidjian, Barrera, Martell, Munoz and Lewinsohn2011; Santoft et al., Reference Santoft, Axelsson, Öst, Hedman-Lagerlöf, Fust and Hedman-Lagerlöf2019a). Educational components emphasized the role of functional analysis and the importance of reducing behavioural avoidance. Patients were also encouraged to practice functional analysis of negative thoughts, to assess whether ruminating thoughts were helpful, and to engage in structured problem solving. ICBT for panic disorder adhered to a published manual (Carlbring et al., Reference Carlbring, Westling, Ljungstrand, Ekselius and Andersson2001) and emphasized exposure and response prevention including interoceptive exposure (Pompoli et al., Reference Pompoli, Furukawa, Efthimiou, Imai, Tajika and Salanti2018), while also incorporating the identification and systematic exploration of negative automatic thoughts, breathing retraining, mindfulness, and acceptance strategies. ICBT for social anxiety disorder was based on another published manual (Andersson et al., Reference Andersson, Carlbring, Holmström, Sparthan, Furmark, Nilsson-Ihrfelt, Buhrman and Ekselius2006) and emphasized the use of behavioural experiments and exposure as a means of challenging dysfunctional patterns in cognition. This treatment also highlighted the importance of reducing self-focus as a type of safety behaviour (e.g. McManus et al., Reference McManus, Sacadura and Clark2008), and promoted strategies for enhancing social skills, primarily in terms of active listening and the use of open-ended questions to maintain a conversation. For a more extensive description of the ICBTs, see the supplementary material of Niles et al. (Reference Niles, Axelsson, Andersson, Hedman-Lagerlöf, Carlbring, Andersson, Johansson, Widén, Driessen, Santoft and Ljótsson2021).

Measures

The patients completed self-report questionnaires via the same web platform used for delivering ICBT. Patients completed the domain symptom scale and all process scales that corresponded to their primary diagnosis, at the beginning of each week over the 10-week treatment, thus resulting in 10 assessment points. The first and last assessment were more exhaustive, as all patients completed all domain symptom scales. Each intermediate weekly assessment consisted of approximately 45–52 items. All scales were administered in Swedish, and simple (non-weighted) sum scoring was used for all questionnaires throughout the study.

Domain-specific symptom outcomes

We measured depression symptoms using the Montgomery-Åsberg Depression Rating Scale-Self-report version (MADRS-S; Svanborg and Åsberg, Reference Svanborg and Åsberg1994) which is well documented to have adequate psychometric properties (Fantino and Moore, Reference Fantino and Moore2009; Thorndike et al., Reference Thorndike, Carlbring, Smyth, Magee, Gonder-Frederick, Öst and Ritterband2009). The MADRS-S consists of nine items, each scored 0–6, resulting in a 0–54 sum score. In the selection/training data, at baseline, the MADRS-S was unifactorial and exhibited adequate internal consistency (α=0.85). We measured panic disorder symptoms using the Panic Disorder Severity Scale-Self-report (PDSS-SR; Houck et al., Reference Houck, Spiegel, Shear and Rucci2002) which is typically found to have adequate psychometric properties (Houck et al., Reference Houck, Spiegel, Shear and Rucci2002; Shear et al., Reference Shear, Rucci, Williams, Frank, Grochocinski, Vander Bilt, Houck and Wang2001). The PDSS–SR consists of seven items, each scored 0–4, resulting in a 0–28 sum score. In the selection/training data, at baseline, the PDSS-SR was unifactorial and exhibited excellent internal consistency (α=0.91). We measured social anxiety disorder symptoms using the Liebowitz Social Anxiety Scale-Self-report (LSAS-SR; Fresco et al., Reference Fresco, Coles, Heimberg, Liebowitz, Hami, Stein and Goetz2001) which is typically found to have adequate psychometric properties (Baker et al., Reference Baker, Heinrichs, Kim and Hofmann2002; Fresco et al., Reference Fresco, Coles, Heimberg, Liebowitz, Hami, Stein and Goetz2001; Hedman et al., Reference Hedman, Ljótsson, Rück, Furmark, Carlbring, Lindefors and Andersson2010). The LSAS-SR consists of 24 items, each with one fear and one avoidance subitem each scored 0–3, resulting in a 0–144 sum score. In the selection/training data, at baseline, the LSAS-SR was unifactorial and exhibited excellent internal consistency (α=0.97).

Depression process scales

This study involved three process scales focusing on the treatment of depression. We administered an 8-item version of the Automatic Thoughts Questionnaire (ATQ) which is usually found to be unifactorial (Hollon and Kendall, Reference Hollon and Kendall1980; Netemeyer et al., Reference Netemeyer, Williamson, Burton, Biswas, Jindal, Landreth, Mills and Primeaux2016). From the Behavioral Activation for Depression Scale (BADS), we administered two established subfactors: the 7-item Activation subscale (BADS-AC) and the 8-item Avoidance/Rumination subscale (BADS-AV/R) (Kanter et al., Reference Kanter, Mulick, Busch, Berlin and Martell2006; Kanter et al., Reference Kanter, Rusch, Busch and Sedivy2008). The Reward Probability Index (RPI) is commonly found to have two subfactors: Reward Probability (RPI-RP) and Environmental Suppressors (RPI-ES) (Carvalho et al., Reference Carvalho, Gawrysiak, Hellmuth, McNulty, Magidson, Lejuez and Hopko2011). We administered a subset of 13 items from the original RPI (seven items from the RPI-RP and six from the RPI-ES), as originating from an earlier project (ClinicalTrials.gov: NCT01636791, NCT01667822) where these items were chosen by clinical psychologists with expert knowledge in CBT. This was done on the basis of face validity and coherence relative to theory including coverage of the construct and its facets, phrasings suitable for repeated measurements, and high factor loadings.

Panic disorder process scales

This study involved two process scales focusing on the treatment of panic disorder. The Agoraphobic Cognitions Questionnaire (ACQ) has 14 items, and is commonly found to have two subfactors: the 7-item Physical Consequences subscale (ACQ-PC) and the 7-item Loss of Control subscale (ACQ-LOC) (Chambless et al., Reference Chambless, Caputo, Bright and Gallagher1984). The Behaviors Questionnaire (BQ) is usually divided into the Avoidance (BQ-A) and Safety Behaviors (BQ-SB) subscales (Centre for Anxiety Disorders & Trauma (CADAT) at King’s College London, 2020; Marks and Mathews, Reference Marks and Mathews1979). We administered a subset of 10 items from the conventional BQ-A and 15 items from the conventional BQ-SB, as originating from an earlier project (ClinicalTrials.gov: NCT01636791, NCT01667822; see above).

Social anxiety disorder process scales

This study involved three process scales focusing on the treatment of social anxiety disorder. We administered 10 items from the Social Cognitions Questionnaire (SCQ); five relating to the believability, and five to the frequency, of cognitions common in social anxiety (Oxford Centre for Anxiety Disorders and Trauma (OxCADAT), 2022), as originating from an earlier project (ClinicalTrials.gov: NCT01636791, NCT01667822; see above). In accordance with Hoffart et al. (Reference Hoffart, Borge, Sexton and Clark2009), we also administered eight items from the Social Probability and Cost Questionnaire (SPCQ) which concerns the perceived probability and valence of adverse social outcomes (Foa et al., Reference Foa, Franklin, Perry and Herbert1996; Hoffart et al., Reference Hoffart, Borge and Clark2016; McManus et al., Reference McManus, Clark and Hackmann2000). The Social Behavior Questionnaire (SBQ) is usually found to have two subfactors: Avoidance (SBQ-A) and Impression Management (SBQ-IM) (Gray et al., Reference Gray, Beierl and Clark2019). Originating from the same study as the items of the SCQ, we also administered a subset of five items from the SBQ-A and five from the SBQ-IM.

Statistical analyses

Effectiveness of ICBT

We conducted the statistical analyses using Stata 15 and R 4.1.0 (R Core Team, 2016) with lavaan 0.6-8 (Rosseel, Reference Rosseel2012). In the assessment of adherence to the protocol, patients were classified as probable treatment drop-outs if they either (i) missed at least three assessment points in a row and did not complete subsequent assessment points or (ii) completed less than four assessment points in total. Prior to the main analyses pertaining to the development of brief process scales, we assessed change in depression symptoms, panic disorder symptoms, and social anxiety disorder symptoms over the course of ICBT to ensure that there were beneficial effects on each domain outcome. For this purpose, all patients with at least one measurement point were included in linear mixed effects regression models with a random intercept and slope, unstructured covariance over these, and an autoregressive residual covariance structure. We calculated Cohen’s d standardized effects as the model-implied change score (intention to treat, i.e. using data from all patients regardless of adherence), divided by the standard deviation of change as based on observed data. This type of effect size which weighs in both the mean change and the variance in change (the pre- to post-treatment correlation) can be thought of as a ‘signal to noise ratio’ that is similar to a traditional test statistic (Morris and DeShon, Reference Morris and DeShon2002).

Main phase 1: Item selection and drafting of the brief process scales

In accordance with recommendations, in reducing the number of process scale items, we strived to marry theoretical considerations with empirical considerations (Ziegler et al., Reference Ziegler, Kemper and Kruyen2014) and revised our selection of items in an iterative manner based on the psychometric evidence (Furr, Reference Furr2011). We first formulated each target construct, and then decided on which psychometric properties that were desirable for a brief CBT process scale, before assessing these in the selection/training data (n=370; Kleka and Soroko, Reference Kleka and Soroko2018). Whenever a full process scale had 10 or fewer items, we attempted to draft a brief version of this scale as a whole (the ATQ, SCQ, SPCQ, and initially also the SBQ). If the full process scale had more than 10 items, we instead drafted a brief version of each subscale, and, treated these as full scales to be shortened (the BADS-AC separate from the BADS-AV/R, the RPI-RP from the RP-ES, the ACQ-PC from the ACQ-LOC, and the BQ-A from the BQ-SB). Two assessors (E.A. and B.L.) independently reviewed the psychometric evidence derived from the selection/training data. Each assessor selected three to four items from each scale (or subscale) based on the following:

  • We conducted a factor analysis with all scale items and the domain symptom scale. This was based on principal axis factoring with promax rotation, and the idea was to retain items that allowed us to capture all dimensions, ideally with standardized factor loadings ≥0.40. The Kaiser-Meyer-Olkin (KMO) statistic was at least 0.70 in conjunction with a significant Barlett’s test for all full scales (see Supplementary material, Table B3). All item distributions are tabulated in the Supplementary material, Table B1.

  • We modelled item change over time using linear mixed effects models and standardized effect sizes. A quadratic time term (time×time) reflective of a curvilinear effect was added if this improved the model fit. The idea was to give priority to items that changed more over time in terms of a standardized within-group effect (d). The numerator for d was derived from the linear mixed model, i.e. either the coefficient for the simple effect of time whenever a curvilinear relationship was lacking, or the sum of the coefficients for time and time×time whenever inclusion of the latter had improved model fit.

  • We explored the relationship between the fitted item slopes and the corresponding domain symptom slopes, as based on Pearson correlations and also the factor analysis of fitted slopes. That is, having derived the fitted item slopes from the linear mixed models, we did two things: first, for the slopes of each item, we calculated the correlation with the corresponding domain outcome slopes; second, we conducted a factor analysis of the item slopes of each scale. Ideally, we wanted to proceed with items that captured change in the full instrument and showed a clear relationship with the domain outcome in terms of change.

  • We explored if change (fluctuations) in each item (e.g. of the ATQ) was uniquely predictive of subsequent change (fluctuations) in the domain outcome (e.g. the MADRS-S) on a week-by-week basis, using random-intercepts cross-lagged panel models (Hamaker et al., Reference Hamaker, Kuiper and Grasman2015) specified analogous to Axelsson et al. (Reference Axelsson, Hesser, Andersson, Ljótsson and Hedman-Lagerlöf2020). This effect (γ) was constrained to be equal from week to week (i.e. the unstandardized effect), and we preferred to see that change (fluctuations) in item scores were predictive of subsequent change (fluctuations) in the domain outcome.

  • We assessed the face validity of each item in terms of its phrasing (Was it easy to understand? Could it easily be misunderstood?) and the relationship to cognitive behavioural theory (Does it measure what the scale is intended to measure? Does the selection of items cover the theoretical construct?). This was to ensure that each brief scale would measure what it was intended to measure, and that the brief scale sums would accurately capture the same latent traits, and pick up on the same processes of change, as the full scale sums.

A third assessor (F.S.) reviewed the suggestions for item selection by E.A. and B.L., and formulated suggestions for 3–4 item brief scales. This formed the basis for a subsequent discussion where preliminary brief scales were drafted by means of voting. Still using the selection/training data, these items sets were analysed as preliminary brief scales, primarily to ensure that the internal consistency appeared to be acceptable, and that change in the summed scale correlated with change in the domain outcome in the same manner as the individual items. This was an important step, considering that it is theoretically possible, although unlikely, for the psychometric properties of the sum scale not to mirror properties of the individual items. We employed unweighted item summation for all scales except the SCQ, where we first multiplied the a items (scored 1–5) by 2 to approximate the range of the b items (scored 0–10). This review of the brief scale sums resulted in a few changes to the preliminary versions (see Results). We then had a final list of brief scales to be validated.

Main phase 2: Validation in a new sample

We evaluated the psychometric properties of the brief scales in a new, separate, validation dataset. Five aspects were explored. First, the correlation with the corresponding full scale. Second, the internal consistency of the brief scale, compared with the full scale. Third, the standardized effect size pertaining to change in the brief scale, compared with the full scale. This was based on linear mixed models, as detailed above. Fourth, we tested if there was a week-by-week unique predictive effect of change (fluctuations) in the brief scale on subsequent change (fluctuations) in the domain outcome, and contrasted the standardized effect related to that of the full scale. This was based on random-intercepts cross-lagged panel models (Hamaker et al., Reference Hamaker, Kuiper and Grasman2015), as detailed above. Fifth, we tested if the brief scale mediated change in the domain outcome, and contrasted the standardized effect to that of the full scale. This test for mediation was based on separate linear mixed effects models for the a path from time to the potential mediator, and the b path from the potential mediator to the domain outcome. The models were fitted on all measurement points, and we constructed 95% confidence intervals for the a×b products using bias-corrected cluster bootstrapping (MacKinnon et al., Reference MacKinnon, Lockwood and Williams2004).

Results

Adherence to the protocol

Missing data rates are presented in Fig. 1. The average patient completed 8.4 out of 10 (SD=2.6) assessment points. The rate of probable treatment drop-outs was 13% (28/218) in ICBT for depression, 17% (31/183) in ICBT for panic disorder, and 16% (43/262) in ICBT for social anxiety disorder.

Effectiveness of ICBT

As was first reported by Niles et al. (Reference Niles, Axelsson, Andersson, Hedman-Lagerlöf, Carlbring, Andersson, Johansson, Widén, Driessen, Santoft and Ljótsson2021), patients in the 2014–2017 selection/training sample (n=370) improved over ICBT for depression (MADRS-S: b=–7.9; 95% CI=–9.3 to –6.6; d=1.11), ICBT for panic disorder (PDSS-SR: b=–5.3; 95% CI=–6.2 to –4.3; d=1.07), and ICBT for social anxiety disorder (LSAS-SR: b=–25.5; 95% CI=–29.2 to –21.7; d=1.12). Similarly, the patients of the 2018–2021 validation sample (n=293) improved over ICBT for depression (MADRS-S: b=–8.7; 95% CI=–10.1 to –7.3; d=1.31), ICBT for panic disorder (PDSS-SR: b=–5.5; 95% CI=–6.6 to –4.4; d=1.23), and ICBT for social anxiety disorder (LSAS-SR: b=–24.5; 95% CI=–28.9 to –20.1; d=1.12). Results were similar, with slightly higher point estimates, in patients who completed all 10 weekly assessments (Supplementary material, supplement B).

Main phase 1: Item selection and drafting of brief process scales

Each of the two independent assessors selected 43 out of 103 items for the brief scales, and agreed on 29 of these (67%; κ=0.44). Absolute agreement was achieved for the RPI-ES and SBQ. After the discussion with assessor 3, the resulting preliminary brief scales were scored in the selection/training data. We found two preliminary scales to be problematic. The BADS–AV/R had been reduced to a 4-item scale where α was 0.62, which we did not find satisfactory. Because we could not draft an alternative, more satisfactory, 3- or 4-item version, we instead proceeded with a longer 6-item version. Also, the SBQ had been reduced to a 4–item scale where α was 0.59. We therefore split the SBQ into two brief subscales: the SBQ–A and SBQ-IM. For the final selection of items for the brief process scales, see the ‘Brief’ column of Table B1 in the Supplementary material.

Main phase 2: Validation in a new sample

The main outcomes of the validation phase are presented in Table 2. Non-adjusted correlations were substantial (≥.83). Internal consistency was adequate for all brief scales except the RPI-ES (α=0.65). Most brief scales changed to a moderate to large degree, similar to their full-scale equivalents. Week-by-week temporal precedence was found for brief versions of the BADS-AC, ACQ-PC, SCQ, SPCQ, SBQ-A and SBQ-IM. All brief scales mediated the effect of time on the domain outcome as indicated by significant a×b products. The final brief scales can be found in Swedish and English in the Supplementary material.

Table 2. Brief process scales: psychometric properties and relationship to the corresponding full scales in the validation data (n=293)

Gamma (γ) is derived from random-intercepts cross-lagged panel models, and stands for the median completely standardized unique effect of deviations in the brief process scale score of one week, on deviations in the domain outcome score of the following week. ACQ-LOC, Agoraphobic Cognitions Questionnaire-Loss of Control subscale; ACQ-PC, Agoraphobic Cognitions Questionnaire-Physical Consequences subscale; ATQ, Automatic Thoughts Questionnaire; BADS-AC, Behavioral Activation for Depression Scale-Activation subscale; BADS-AV/R, Behavioral Activation for Depression Scale-Avoidance/Rumination subscale; BQ-A, Behaviors Questionnaire-Avoidance; BQ-SB, Behaviors Questionnaire-Safety Behaviors; MDD, major depressive disorder; PD, panic disorder; RPI-ES, Reward Probability Index-Environmental Suppressors subscale; RPI-RP, Reward Probability Index-Reward Probability subscale; SAD, social anxiety disorder; SBQ-A, Social Behavior Questionnaire-Avoidance subscale; SBQ-IM, Social Behavior Questionnaire-Impression Management subscale; SCQ, Social Cognitions Questionnaire; SPCQ, Social Probability and Cost Questionnaire.

a Non-adjusted Pearson correlation. Each full scale was rescored as the brief scale, meaning that all scores on shared items were identical.

b Including the curvilinear effect of time on the brief process scale, which improved model fit when modelling the a path.

Discussion

In this study, we developed and validated brief versions of 13 questionnaires intended to capture target variables and core processes of change in CBT for depression, panic disorder, and social anxiety disorder. We found that several of the original scales showed promise in terms of their psychometric properties and also as mediators of the treatment effect. As is illustrated in Table 2, most of these beneficial properties appeared to be maintained also when the number of items per scale was roughly cut in half. We regard this to be an important finding, considering that around two times as many constructs can be measured using equal resources. The use of brief process scales can also facilitate repeated measurement regimens which could prove helpful for clinicians and researchers alike (Kazdin, Reference Kazdin2007; Lutz et al., Reference Lutz, Rubel, Deisenhofer and Moggia2022; Ziegler et al., Reference Ziegler, Kemper and Kruyen2014).

Specific recommendations

For depression, we recommend use of the brief BADS-AC which showed promise as a process measure of behavioural activation. The brief ATQ, RPI-RP and BADS-AV/R are also worth studying further, although a week-by-week time-lagged effect on depression was not seen here. The brief RPI-ES which purports to measure environmental suppressors of rewards exhibited questionable internal consistency and face validity, and is probably not worth studying further. Notably, very few previous studies have been concerned with brief CBT process scales for depression. In one study, the BADS-AC was reduced to 5–6 items, i.e. twice as long as the 3-item version evaluated here (Manos et al., Reference Manos, Kanter and Luo2011). In two single-subject cases, that version appeared to be promising for tracking treatment progress but no more extensive longitudinal evaluation was ever conducted. Interestingly, the brief version developed here was not part of the 5- to 6-item version but yet exhibited adequate psychometric properties.

For panic disorder, we recommend use of the brief ACQ-PC which showed promise as a measure of catastrophic thoughts about the physical consequences for panic attacks. The brief ACQ-LOC, BQ-A and BQ-SB are also worth studying further. To our knowledge, no previous study has evaluated the psychometric properties of brief CBT process scales for panic disorder. There are, however, recent examples of longer alternatives; one being the Oxford Cognitions and Defences Questionnaire with subscales for threat cognitions, anxious avoidance, and safety behaviours each consisting of 8–14 items, i.e. each at least twice as many items as the brief scales evaluated here (Rosebrock et al., Reference Rosebrock, Lambe, Mulhall, Petit, Loe, Saidel, Pervez, Mitchell, Chauhan, Prouten, Chan, Aynsworth, Murphy, Jones, Powling, Chapman, Dudley, Morrison, O’Regan and Freeman2022).

For social anxiety disorder, all four brief process scales – i.e. the SCQ, SPCQ, SBQ-A, and SBQ-IM – exhibited promising psychometric properties and, tentatively, we recommend further use and evaluation of all of these. The authors of a recent systematic review of cognitive components in social anxiety concluded that ‘future studies would benefit from the inclusion of […] multiple well-validated measures within the same domain of social cognition […] outside of the laboratory’ (Alvi et al., Reference Alvi, Kumar and Tabak2022). Considering that we are not aware of previous attempts at developing brief CBT process scales for social anxiety disorder, the scales evaluated here could potentially be of use in this endeavour.

Caveats regarding temporal precedence

We wish to comment briefly on the random-intercepts cross-lagged panel models that we used to assess whether change in the brief scales was systematically predictive of subsequent change in the corresponding symptom domain on a week-by-week basis. Although establishing a timeline from change in the process scale to subsequent change in the presumed outcome is an important aspect of investigating causation (Kazdin, Reference Kazdin2007), for many of the target variables probed by these brief scales it is not clear over what time frame (seconds, minutes, hours, days, weeks, months?) effects on the domain outcome are to be expected. Thus, even though we could not demonstrate a week-by-week temporal precedence for any of the depression target variables except behavioural activation (the brief BADS-AC), nor any of the panic disorder targets except catastrophic thoughts about the physical consequences of panic attacks (the brief ACQ-PC), it cannot be ruled out that the expected temporal ordering effects could be observed with another temporal resolution (e.g. minute to minute, or hour to hour) than in this study. There is also an ongoing debate over which statistical modelling techniques that are to be preferred for the study of time-lagged effects, and how such effects should be interpreted (Usami et al., Reference Usami, Murayama and Hamaker2019).

Strengths and limitations

An important strength of this study is that it was conducted in a structured setting where all patients were diagnosed with a primary psychiatric disorder corresponding to their treatment, and the ICBT format ensured that similar treatment content was conveyed to all participants, which speaks for reproducibility. Another strength is that all process measures were administered over 10 assessment points, which made analyses of change relatively robust. An important limitation is that there was no control group, which means that estimates of change may not mirror causal effects. We administered several scales in parallel which may have led to fatigue that affected estimates. Another interesting threat to the validity of this study is that, although we used widely recognized symptom scales, there is some degree of conceptual overlap between these and the process measures. For example, the ATQ measured ‘automatic negative thoughts’ while 1 out of 9 items of the MADRS-S established the level of depression by asking the patient about pessimism. This said, symptom scales of the kind used in this study are widely recognized as valid indicators of psychopathology and of primary interest in much of psychological treatment research. Another limitation is that while we drafted brief scales to tap into the processes of change that occurred naturally as part of CBT, our results say less about the usefulness of the brief process scales as clinical tools for the prospective adaptation of CBT during the course of a treatment. Importantly, the present study did not focus on retaining items in the sense that we wanted for example every type of avoidance behaviour to be surveyed for each psychiatric condition. Rather, we focused on whether each brief scale sum would be appropriate to use instead of the corresponding full-scale sum. Overall, results were promising in this regard, although we wish to remind the reader that the BQ and SCQ appear to lack primary peer-reviewed journal publications which means that even though their brief scales versions appeared to be promising in this study, there is little reference data pertaining to the psychometric properties of the full-scale equivalents beyond what was presented here. We also wish to bring to the reader’s attention that CBT protocols can differ in their components, and that we evaluated the role of these brief process scales in relation to three specific CBT protocols. It is conceivable that some of the scales evaluated in this study had seemed more promising, and some less promising, had we focused on CBT protocols with a slightly different emphasis. For example, we found the BADS-AC to be especially promising as a process variable in CBT for depression, but this may, at least to some degree, be because our CBT protocol for depression focused heavily on behavioural activation. The implication of this is that it is important to evaluate the brief scales also with other CBT protocols in order to determine their usefulness as process scales in that particular context. Another limitation is that the samples sizes were generally smaller than ideal, and power was limited for example in the study of time-lagged effects for panic disorder in the validation data (n=77), which means that these estimates need be interpreted with caution and warrant replication. Last, the validation phase of this study was based on further rescoring of the existing full process scales rather than actual administration of the brief scales. Considering the potential impact of ordering effects (e.g. Knowles, Reference Knowles1988), this further highlights the need for replication.

Conclusion

In this study, we have presented several brief scales that can be used to measure target variables in CBT for depression, panic disorder, and social anxiety disorder. These brief scales can potentially facilitate treatment development, research into processes, and the evaluation of treatment progress. Results were generally promising, but based on limited sample sizes, with data derived from one clinical setting only, and without a control group. We wish to emphasize that there is a need for replication and further psychometric evaluation of these brief scales, ideally using variations on CBT protocols, based on experimental designs, and in larger samples.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S1352465823000541

Data availability statement

Data from the current study, and results derived from these data, are made available to the extent that this is deemed to be consistent with Swedish and European Union (EU) data protection and privacy legislation. Decisions pertaining to the sharing of data are taken upon reasonable request, on a case-by-case basis, and in accordance with judicial expertise.

Acknowledgements

This work used the BASS platform from the eHealth Core Facility at Karolinska Institutet.

Author contributions

Erland Axelsson: Conceptualization (equal), Data curation (lead), Formal analysis (lead), Investigation (equal), Methodology (lead), Project administration (equal), Visualization (lead), Writing – original draft (lead), Writing – review & editing (lead); Fredrik Santoft: Investigation (equal), Methodology (supporting), Writing – review & editing (equal); Josefin Särnholm: Investigation (equal), Writing – review & editing (equal); Brjánn Ljótsson: Conceptualization (equal), Data curation (supporting), Formal analysis (supporting), Investigation (equal), Methodology (supporting), Resources (lead), Software (lead), Writing – review & editing (equal).

Financial support

Open access publication was funded by Karolinska Institutet. The authors received no additional funding from an external source. No organization or entity over and above the authors influenced the study design, data collection, analysis, interpretation, or publication process.

Competing interests

The authors declare none.

Ethical standard

This study was approved by the Regional Ethics Committee of Stockholm and the Swedish Ethical Review Authority (2018/932-31/2 with amendments). All procedures were in accordance with Swedish and European Union (EU) data protection and privacy legislation and the Declaration of Helsinki. All patients in this study gave informed consent for participating in research.

References

Alvi, T., Kumar, D., & Tabak, B. A. (2022). Social anxiety and behavioral assessments of social cognition: a systematic review. Journal of Affective Disorders, 311, 1730. https://doi.org/10.1016/j.jad.2022.04.130 CrossRefGoogle ScholarPubMed
Andersson, G. (2016). Internet-delivered psychological treatments. Annual Review of Clinical Psychology, 12, 157179. https://doi.org/10.1146/annurev-clinpsy-021815-093006 CrossRefGoogle ScholarPubMed
Andersson, G., Carlbring, P., Holmström, A., Sparthan, E., Furmark, T., Nilsson-Ihrfelt, E., Buhrman, M., & Ekselius, L. (2006). Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. Journal of Consulting and Clinical Psychology, 74, 677686. https://doi.org/10.1037/0022-006X.74.4.677 CrossRefGoogle ScholarPubMed
Axelsson, E., Hesser, H., Andersson, E., Ljótsson, B., & Hedman-Lagerlöf, E. (2020). Mediators of treatment effect in minimal-contact cognitive behaviour therapy for severe health anxiety: a theory-driven analysis based on a randomised controlled trial. Journal of Anxiety Disorders, 69, 102172. https://doi.org/10.1016/j.janxdis.2019.102172 CrossRefGoogle ScholarPubMed
Baker, S. L., Heinrichs, N., Kim, H. J., & Hofmann, S. G. (2002). The Liebowitz social anxiety scale as a self-report instrument: a preliminary psychometric analysis. Behaviour Research and Therapy, 40, 701715. https://doi.org/10.1016/s0005-7967(01)00060-2 CrossRefGoogle ScholarPubMed
Bandelow, B., & Michaelis, S. (2022). Epidemiology of anxiety disorders in the 21st century. Dialogues in Clinical Neuroscience, 17, 327335. https://doi.org/10.31887/DCNS.2015.17.3/bbandelow CrossRefGoogle Scholar
Beck, A. T., & Haigh, E. A. P. (2014). Advances in cognitive theory and therapy: the generic cognitive model. In T. D. Cannon & T. Widiger (eds), Annual Review of Clinical Psychology, 10, 1–24). https://doi.org/10.1146/annurev-clinpsy-032813-153734 CrossRefGoogle Scholar
Benbow, A. A., & Anderson, P. L. (2018). Long-term improvements in probability and cost biases following brief cognitive behavioral therapy for social anxiety disorder. Cognitive Therapy and Research, 43, 412418. https://doi.org/10.1007/s10608-018-9947-0 CrossRefGoogle Scholar
Bennett-Levy, J., Butler, G., Fennell, M., Hackmann, A., Mueller, M., & Westbrook, D. (2004). Oxford Guide to Behavioural Experiments in Cognitive Therapy. Oxford University Press.CrossRefGoogle Scholar
Carlbring, P., Westling, B. E., Ljungstrand, P., Ekselius, L., & Andersson, G. (2001). Treatment of panic disorder via the internet: a randomized trial of a self-help program. Behavior Therapy, 32, 751764. https://doi.org/10.1016/s0005-7894(01)80019-8 CrossRefGoogle Scholar
Carvalho, J. P., Gawrysiak, M. J., Hellmuth, J. C., McNulty, J. K., Magidson, J. F., Lejuez, C. W., & Hopko, D. R. (2011). The reward probability index: design and validation of a scale measuring access to environmental reward. Behavior Therapy, 42, 249262. https://doi.org/10.1016/j.beth.2010.05.004 CrossRefGoogle ScholarPubMed
Carvalho, J. P., & Hopko, D. R. (2011). Behavioral theory of depression: reinforcement as a mediating variable between avoidance and depression. Journal of Behavior Therapy and Experimental Psychiatry, 42, 154162. https://doi.org/10.1016/j.jbtep.2010.10.001 CrossRefGoogle ScholarPubMed
Centre for Anxiety Disorders & Trauma (CADAT) at King’s College London (2020). Panic and Agoraphobia Questionnaires. Retrieved from: https://www.kcl.ac.uk/ioppn/depts/psychology/research/ResearchGroupings/CADAT/Research/Panic-and-Agoraphobia-Questionnaires (accessed October 2020).Google Scholar
Chambless, D. L., Caputo, G. C., Bright, P., & Gallagher, R. (1984). Assessment of fear of fear in agoraphobics: the body sensations questionnaire and the agoraphobic cognitions questionnaire. Journal of Consulting and Clinical Psychology, 52, 10901097. https://doi.org/10.1037//0022-006x.52.6.1090 CrossRefGoogle ScholarPubMed
Chiu, K., Clark, D. M., & Leigh, E. (2021). Cognitive predictors of adolescent social anxiety. Behaviour Research and Therapy, 137, 103801. https://doi.org/10.1016/j.brat.2020.103801 CrossRefGoogle ScholarPubMed
Clark, D. A., & Beck, A. T. (2010). Cognitive theory and therapy of anxiety and depression: convergence with neurobiological findings. Trends in Cognitive Sciences, 14, 418424. https://doi.org/10.1016/j.tics.2010.06.007 CrossRefGoogle ScholarPubMed
Clark, D. M. (2001). A cognitive perspective on social phobia. In Crozier, W. R. & Alden, L. E. (eds), International Handbook of Social Anxiety: Concepts, Research and Interventions Relating to the Self and Shyness. John Wiley & Sons, Ltd.Google Scholar
Craske, M. G., & Barlow, D. H. (2014). Panic disorder and agoraphobia. In Barlow, D. H. (ed), Clinical Handbook of Psychological Disorders: A Step-by-Step Treatment Manual (5th edn, pp. 161). New York: Guilford Press.Google Scholar
Cuijpers, P., van Straten, A., & Warmerdam, L. (2007). Behavioral activation treatments of depression: a meta-analysis. Clinical Psychology Review, 27, 318326. https://doi.org/10.1016/j.cpr.2006.11.001 CrossRefGoogle ScholarPubMed
Dimidjian, S., Barrera, M. Jr, Martell, C., Munoz, R. F., & Lewinsohn, P. M. (2011). The origins and current status of behavioral activation treatments for depression. Annual Review of Clinical Psychology, 7, 138. https://doi.org/10.1146/annurev-clinpsy-032210-104535 CrossRefGoogle ScholarPubMed
Dimidjian, S., Goodman, S. H., Sherwood, N. E., Simon, G. E., Ludman, E., Gallop, R., Welch, S. S., Boggs, J. M., Metcalf, C. A., Hubley, S., Powers, J. D., & Beck, A. (2017). A pragmatic randomized clinical trial of behavioral activation for depressed pregnant women. Journal of Consulting and Clinical Psychology, 85, 2636. https://doi.org/10.1037/ccp0000151 CrossRefGoogle ScholarPubMed
Draheim, A. A., & Anderson, P. L. (2021). Does cognitive behavioral therapy for anxiety disorders improve threat reappraisal?: a meta-analysis. Journal of Behavioral and Cognitive Therapy, 31, 125135. https://doi.org/10.1016/j.jbct.2020.12.004 CrossRefGoogle Scholar
Fantino, B., & Moore, N. (2009). The self-reported Montgomery-Åsberg depression rating scale is a useful evaluative tool in major depressive disorder. BMC Psychiatry, 9, 26. https://doi.org/10.1186/1471-244X-9-26 CrossRefGoogle ScholarPubMed
Foa, E. B., Franklin, M. E., Perry, K. J., & Herbert, J. D. (1996). Cognitive biases in generalized social phobia. Journal of Abnormal Psychology, 105, 433439. https://doi.org/10.1037/0021-843x.105.3.433 CrossRefGoogle ScholarPubMed
Fresco, D. M., Coles, M. E., Heimberg, R. G., Liebowitz, M. R., Hami, S., Stein, M. B., & Goetz, D. (2001). The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychological Medicine, 31, 10251035. https://doi.org/10.1017/s0033291701004056 CrossRefGoogle Scholar
Fuhr, K., Hautzinger, M., Krisch, K., Berking, M., & Ebert, D. D. (2016). Validation of the Behavioral Activation for Depression Scale (BADS) – psychometric properties of the long and short form. Comprehensive Psychiatry, 66, 209218. https://doi.org/10.1016/j.comppsych.2016.02.004 CrossRefGoogle Scholar
Furr, R. M. (2011). Scale Construction and Psychometrics for Social and Personality Psychology. SAGE Publications Ltd.CrossRefGoogle Scholar
Garratt, G., Ingram, R. E., Rand, K. L., & Sawalani, G. (2007). Cognitive processes in cognitive therapy: evaluation of the mechanisms of change in the treatment of depression. Clinical Psychology: Science and Practice, 14, 224239. https://doi.org/10.1111/j.1468-2850.2007.00081.x Google Scholar
Gill, C. C., Kane, R. T., & Mazzucchelli, T. G. (2019). Activation, avoidance, and response-contingent positive reinforcement predict subjective wellbeing. Journal of Happiness Studies, 20, 331349. https://doi.org/10.1007/s10902-017-9949-8 CrossRefGoogle Scholar
Gloster, A. T., Klotsche, J., Gerlach, A. L., Hamm, A., Strohle, A., Gauggel, S., Kircher, T., Alpers, G. W., Deckert, J., & Wittchen, H. U. (2014). Timing matters: change depends on the stage of treatment in cognitive behavioral therapy for panic disorder with agoraphobia. Journal of Consulting and Clinical Psychology, 82, 141153. https://doi.org/10.1037/a0034555 CrossRefGoogle ScholarPubMed
Goldstein, A. J., & Chambless, D. L. (1978). A reanalysis of agoraphobia. Behavior Therapy, 9, 4759. https://doi.org/10.1016/s0005-7894(78)80053-7 CrossRefGoogle Scholar
Gray, E., Beierl, E. T., & Clark, D. M. (2019). Sub-types of safety behaviours and their effects on social anxiety disorder. PloS One, 14, e0223165. https://doi.org/10.1371/journal.pone.0223165 CrossRefGoogle ScholarPubMed
Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. (2015). A critique of the cross-lagged panel model. Psychological Methods, 20, 102116. https://doi.org/10.1037/a0038889 CrossRefGoogle ScholarPubMed
Hedman, E., Ljótsson, B., Rück, C., Furmark, T., Carlbring, P., Lindefors, N., & Andersson, G. (2010). Internet administration of self-report measures commonly used in research on social anxiety disorder: a psychometric evaluation. Computers in Human Behavior, 26, 736740. https://doi.org/10.1016/j.chb.2010.01.010 CrossRefGoogle Scholar
Hill, R. M., Buitron, V., & Pettit, J. W. (2017). Unpacking response contingent positive reinforcement: reward probability, but not environmental suppressors, prospectively predicts depressive symptoms via behavioral activation. Journal of Psychopathology and Behavioral Assessment, 39, 498505. https://doi.org/10.1007/s10862-017-9600-3 CrossRefGoogle Scholar
Hoffart, A., Borge, F.-M., & Clark, D. M. (2016). Within-person process-outcome relationships in residential cognitive and interpersonal psychotherapy for social anxiety disorder: a reanalysis using disaggregated data. Journal of Experimental Psychopathology, 7, 671683. https://doi.org/10.5127/jep.056116 CrossRefGoogle Scholar
Hoffart, A., Borge, F.-M., Sexton, H., & Clark, D. M. (2009). Change processes in residential cognitive and interpersonal psychotherapy for social phobia: a process-outcome study. Behavior Therapy, 40, 1022. https://doi.org/10.1016/j.beth.2007.12.003 CrossRefGoogle ScholarPubMed
Hollon, S. D., & Kendall, P. C. (1980). Cognitive self-statements in depression: development of an automatic thoughts questionnaire. Cognitive Therapy and Research, 4, 383395. https://doi.org/10.1007/bf01178214 CrossRefGoogle Scholar
Houck, P. R., Spiegel, D. A., Shear, M. K., & Rucci, P. (2002). Reliability of the self-report version of the panic disorder severity scale. Depression and Anxiety, 15, 183185. https://doi.org/10.1002/da.10049 CrossRefGoogle ScholarPubMed
Kanter, J. W., Manos, R. C., Bowe, W. M., Baruch, D. E., Busch, A. M., & Rusch, L. C. (2010). What is behavioral activation? A review of the empirical literature. Clinical Psychology Review, 30, 608620. https://doi.org/10.1016/j.cpr.2010.04.001 CrossRefGoogle ScholarPubMed
Kanter, J. W., Mulick, P. S., Busch, A. M., Berlin, K. S., & Martell, C. R. (2006). The Behavioral Activation for Depression Scale (BADS): psychometric properties and factor structure. Journal of Psychopathology and Behavioral Assessment, 29, 191202. https://doi.org/10.1007/s10862-006-9038-5 CrossRefGoogle Scholar
Kanter, J. W., Rusch, L. C., Busch, A. M., & Sedivy, S. K. (2008). Validation of the Behavioral Activation for Depression Scale (BADS) in a community sample with elevated depressive symptoms. Journal of Psychopathology and Behavioral Assessment, 31, 3642. https://doi.org/10.1007/s10862-008-9088-y CrossRefGoogle Scholar
Kazdin, A. E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 127. https://doi.org/10.1146/annurev.clinpsy.3.022806.091432 CrossRefGoogle ScholarPubMed
Kleka, P., & Soroko, E. (2018). How to abbreviate questionnaires and avoid the sins? Survey Research Methods, 12, 147160. https://doi.org/10.18148/srm/2018.v12i2.7224 Google Scholar
Knowles, E. S. (1988). Item context effects on personality scales: measuring changes the measure. Journal of Personality and Social Psychology, 55, 312320. https://doi.org/10.1037/0022-3514.55.2.312 CrossRefGoogle Scholar
Lervik, L. V., Hoffart, A., Knapstad, M., & Smith, O. R. F. (2022). Exploring the temporal associations between avoidance behavior and cognitions during the course of cognitive behavioral therapy for clients with symptoms of social anxiety disorder. Psychother Res, 32, 195208. https://doi.org/10.1080/10503307.2021.1930243 CrossRefGoogle ScholarPubMed
Lewinsohn, P. M., & Graf, M. (1973). Pleasant activities and depression. Journal of Consulting and Clinical Psychology, 41, 261268. https://doi.org/10.1037/h0035142 CrossRefGoogle ScholarPubMed
Liu, Q., He, H., Yang, J., Feng, X., Zhao, F., & Lyu, J. (2020). Changes in the global burden of depression from 1990 to 2017: findings from the Global Burden of Disease study. Journal of Psychiatric Research, 126, 134140. https://doi.org/10.1016/j.jpsychires.2019.08.002 CrossRefGoogle ScholarPubMed
Loerinc, A. G., Meuret, A. E., Twohig, M. P., Rosenfield, D., Bluett, E. J., & Craske, M. G. (2015). Response rates for CBT for anxiety disorders: need for standardized criteria. Clinical Psychology Review, 42, 7282. https://doi.org/10.1016/j.cpr.2015.08.004 CrossRefGoogle ScholarPubMed
Lutz, W., Rubel, J., Deisenhofer, A. K., & Moggia, D. (2022). Continuous outcome measurement in modern data-informed psychotherapies. World Psychiatry, 21, 215216. https://doi.org/10.1002/wps.20988 CrossRefGoogle ScholarPubMed
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99. https://doi.org/10.1207/s15327906mbr3901_4 CrossRefGoogle ScholarPubMed
Manos, R. C., Kanter, J. W., & Busch, A. M. (2010). A critical review of assessment strategies to measure the behavioral activation model of depression. Clinical Psychology Review, 30, 547561. https://doi.org/10.1016/j.cpr.2010.03.008 CrossRefGoogle ScholarPubMed
Manos, R. C., Kanter, J. W., & Luo, W. (2011). The behavioral activation for depression scale-short form: development and validation. Behavior Therapy, 42, 726739. https://doi.org/10.1016/j.beth.2011.04.004 CrossRefGoogle ScholarPubMed
Marks, I. M., & Mathews, A. M. (1979). Brief standard self-rating for phobic patients. Behaviour Research and Therapy, 17, 263267. https://doi.org/10.1016/0005-7967(79)90041-x CrossRefGoogle ScholarPubMed
Martell, C. R., Dimidjian, S., Herman-Dunn, R., & Lewinsohn, P. M. (2013). Behavioral Activation for Depression: A Clinician’s Guide. New York: Guilford Press.Google Scholar
McManus, F., Clark, D. M., & Hackmann, A. (2000). Specificity of cognitive biases in social phobia and their role in recovery. Behavioural and Cognitive Psychotherapy, 28, 201209. https://doi.org/10.1017/s1352465800003015 CrossRefGoogle Scholar
McManus, F., Sacadura, C., & Clark, D. M. (2008). Why social anxiety persists: an experimental investigation of the role of safety behaviours as a maintaining factor. Journal of Behavior Therapy and Experimental Psychiatry, 39, 147161. https://doi.org/10.1016/j.jbtep.2006.12.002 CrossRefGoogle ScholarPubMed
Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105125. https://doi.org/10.1037/1082-989x.7.1.105 CrossRefGoogle ScholarPubMed
Mowrer, O. H. (1960). Learning Theory and Behavior. John Wiley & Sons, Inc.CrossRefGoogle Scholar
Netemeyer, R. G., Williamson, D. A., Burton, S., Biswas, D., Jindal, S., Landreth, S., Mills, G., & Primeaux, S. (2016). Psychometric properties of shortened versions of the Automatic Thoughts Questionnaire. Educational and Psychological Measurement, 62, 111129. https://doi.org/10.1177/0013164402062001008 CrossRefGoogle Scholar
Niles, A. N., Axelsson, E., Andersson, E., Hedman-Lagerlöf, E., Carlbring, P., Andersson, G., Johansson, R., Widén, S., Driessen, J., Santoft, F., & Ljótsson, B. (2021). Internet-based cognitive behavior therapy for depression, social anxiety disorder, and panic disorder: effectiveness and predictors of response in a teaching clinic. Behaviour Research and Therapy, 136, 103767. https://doi.org/10.1016/j.brat.2020.103767 CrossRefGoogle Scholar
Oxford Centre for Anxiety Disorders and Trauma (OxCADAT) (2022). OxCADAT Resources: Resources for Cognitive Therapy for PTSD, Social Anxiety Disorder and Panic Disorder. https://oxcadatresources.com/questionnaires/ Google Scholar
Pettersson, A., Modin, S., Wahlstrom, R., af Winklerfelt Hammarberg, S., & Krakau, I. (2018). The Mini-International Neuropsychiatric Interview is useful and well accepted as part of the clinical assessment for depression and anxiety in primary care: a mixed-methods study. BMC Family Practice, 19, 19. https://doi.org/10.1186/s12875-017-0674-5 CrossRefGoogle ScholarPubMed
Pompoli, A., Furukawa, T. A., Efthimiou, O., Imai, H., Tajika, A., & Salanti, G. (2018). Dismantling cognitive-behaviour therapy for panic disorder: a systematic review and component network meta-analysis. Psychological Medicine, 48, 19451953. https://doi.org/10.1017/S0033291717003919 CrossRefGoogle ScholarPubMed
Porter, E., & Chambless, D. L. (2015). A systematic review of predictors and moderators of improvement in cognitive-behavioral therapy for panic disorder and agoraphobia. Clinical Psychology Review, 42, 179192. https://doi.org/10.1016/j.cpr.2015.09.004 CrossRefGoogle ScholarPubMed
Ramnerö, J., Folke, F., & Kanter, J. W. (2016). A learning theory account of depression. Scandinavian Journal of Psychology, 57, 7382. https://doi.org/10.1111/sjop.12233 CrossRefGoogle ScholarPubMed
R Core Team (2016). R: A Language and Environment for Statistical Computing. In R Foundation for Statistical Computing. https://www.R-project.org/ Google Scholar
Rosebrock, L., Lambe, S., Mulhall, S., Petit, A., Loe, B. S., Saidel, S., Pervez, M., Mitchell, J., Chauhan, N., Prouten, E., Chan, C., Aynsworth, C., Murphy, E., Jones, J., Powling, R., Chapman, K., Dudley, R., Morrison, A., O’Regan, E., … & Freeman, D. (2022). Understanding agoraphobic avoidance: the development of the Oxford Cognitions and Defences Questionnaire (O-CDQ). Behavioural and Cognitive Psychotherapy, 112. https://doi.org/10.1017/S1352465822000030 Google ScholarPubMed
Rosseel, Y. (2012). lavaan: an R Package for Structural Equation Modeling. Journal of Statistical Software, 48, 136. https://doi.org/10.18637/jss.v048.i02 CrossRefGoogle Scholar
Santoft, F., Axelsson, E., Öst, L.-G., Hedman-Lagerlöf, M., Fust, J., & Hedman-Lagerlöf, E. (2019a). Cognitive behaviour therapy for depression in primary care: systematic review and meta-analysis. Psychological Medicine, 49, 12661274. https://doi.org/10.1017/S0033291718004208 CrossRefGoogle ScholarPubMed
Santoft, F., Salomonsson, S., Hesser, H., Lindsäter, E., Ljótsson, B., Lekander, M., Kecklund, G., Öst, L.-G., & Hedman-Lagerlöf, E. (2019b). Processes in cognitive behavior therapy for social anxiety disorder: predicting subsequent symptom change. Journal of Anxiety Disorders. https://doi.org/10.1016/j.janxdis.2019.102118 CrossRefGoogle ScholarPubMed
Seeley, J. R., Sheeber, L. B., Feil, E. G., Leve, C., Davis, B., Sorensen, E., & Allan, S. (2019). Mediation analyses of internet-facilitated cognitive behavioral intervention for maternal depression. Cognitive Behaviour Therapy, 48, 337352. https://doi.org/10.1080/16506073.2018.1513554 CrossRefGoogle ScholarPubMed
Shear, M. K., Rucci, P., Williams, J., Frank, E., Grochocinski, V., Vander Bilt, J., Houck, P., & Wang, T. (2001). Reliability and validity of the Panic Disorder Severity Scale: replication and extension. Journal of Psychiatric Research, 35, 293296. https://doi.org/10.1016/s0022-3956(01)00028-0 CrossRefGoogle ScholarPubMed
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., Hergueta, T., Baker, R., & Dunbar, G. C. (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59, 2233. https://www.ncbi.nlm.nih.gov/pubmed/9881538 Google ScholarPubMed
Spasojevic, J., & Alloy, L. B. (2001). Rumination as a common mechanism relating depressive risk factors to depression. Emotion, 1, 2537. https://doi.org/10.1037/1528-3542.1.1.25 CrossRefGoogle ScholarPubMed
Stice, E., Rohde, P., Seeley, J. R., & Gau, J. M. (2010). Testing mediators of intervention effects in randomized controlled trials: an evaluation of three depression prevention programs. Journal of Consulting and Clinical Psychology, 78, 273280. https://doi.org/10.1037/a0018396 CrossRefGoogle ScholarPubMed
Sturmey, P. (2007). Functional Analysis in Clinical Treatment. Elsevier.Google Scholar
Svanborg, P., & Åsberg, M. (1994). A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatrica Scandinavica, 89, 2128. https://doi.org/10.1111/j.1600-0447.1994.tb01480.x CrossRefGoogle ScholarPubMed
Thew, G. R., Ehlers, A., Grey, N., Wild, J., Warnock-Parkes, E., Dawson, R. L., & Clark, D. M. (2020). Change processes in cognitive therapy for social anxiety disorder delivered in routine clinical practice. Clinical Psychology in Europe, 2. https://doi.org/10.32872/cpe.v2i2.2947 CrossRefGoogle Scholar
Thew, G. R., Kwok, A. P. L., Lissillour Chan, M. H., Powell, C., Wild, J., Leung, P. W. L., & Clark, D. M. (2022). Internet-delivered cognitive therapy for social anxiety disorder in Hong Kong: a randomized controlled trial. Internet Interventions, 28, 100539. https://doi.org/10.1016/j.invent.2022.100539 CrossRefGoogle Scholar
Thorndike, F. P., Carlbring, P., Smyth, F. L., Magee, J. C., Gonder-Frederick, L., Öst, L.-G., & Ritterband, L. M. (2009). Web-based measurement: effect of completing single or multiple items per webpage. Computers in Human Behavior, 25, 393401. https://doi.org/10.1016/j.chb.2008.05.006 CrossRefGoogle Scholar
Usami, S., Murayama, K., & Hamaker, E. L. (2019). A unified framework of longitudinal models to examine reciprocal relations. Psychological Methods, 24(5), 637657. https://doi.org/10.1037/met0000210 CrossRefGoogle ScholarPubMed
van Luenen, S., Kraaij, V., Spinhoven, P., Wilderjans, T. F., & Garnefski, N. (2019). Exploring mediators of a guided web-based self-help intervention for people with HIV and depressive symptoms: randomized controlled trial. JMIR Mental Health, 6, e12711. https://doi.org/10.2196/12711 CrossRefGoogle ScholarPubMed
Vogele, C., Ehlers, A., Meyer, A. H., Frank, M., Hahlweg, K., & Margraf, J. (2010). Cognitive mediation of clinical improvement after intensive exposure therapy of agoraphobia and social phobia. Depression and Anxiety, 27, 294301. https://doi.org/10.1002/da.20651 CrossRefGoogle ScholarPubMed
Wong, Q. J. J., & Rapee, R. M. (2016). The aetiology and maintenance of social anxiety disorder: a synthesis of complimentary theoretical models and formulation of a new integrated model. Journal of Affective Disorders, 203, 84100. https://doi.org/10.1016/j.jad.2016.05.069 CrossRefGoogle ScholarPubMed
Ziegler, M., Kemper, C. J., & Kruyen, P. (2014). Short scales – five misunderstandings and ways to overcome them. Journal of Individual Differences, 35, 185189. https://doi.org/10.1027/1614-0001/a000148 CrossRefGoogle Scholar
Figure 0

Figure 1. Flowchart of the recruitment procedure at the Internet-delivered cognitive behavior therapy teaching clinic. AUDIT = Alcohol Use Disorders Identification Test; CBT = cognitive behavior therapy; MADRS-S = Montgomery-Åsberg Depression Rating Scale – Self report version; MDD = major depressive disorder; PD = panic disorder; SAD = social anxiety disorder; ITT = intention to treat.

Figure 1

Table 1. Sociodemographic and clinical characteristics of the two samples

Figure 2

Table 2. Brief process scales: psychometric properties and relationship to the corresponding full scales in the validation data (n=293)

Supplementary material: PDF

Axelsson et al. supplementary material

Axelsson et al. supplementary material 1

Download Axelsson et al. supplementary material(PDF)
PDF 1.1 MB
Supplementary material: PDF

Axelsson et al. supplementary material

Axelsson et al. supplementary material 2

Download Axelsson et al. supplementary material(PDF)
PDF 993.5 KB
Submit a response

Comments

No Comments have been published for this article.