Setting

The study was conducted in South Verona (about 75 000 inhabitants), an area that includes part of the city of Verona and two small neighbouring towns. The main agency providing psychiatric care for the adult population is the South Verona Community Psychiatric Service (CPS), which is run by the Section of Psychiatry, Department of Medicine and Public Health, University of Verona.

The CPS supplies a wide range of well-integrated hospital and community services. With the exception of hospital nurses, all staff work both within and outside the hospital. This ensures continuity of care through the different phases of treatment and across the various components of service provision (Reference Sytema, Micciolo and TansellaSytema et al, 1997). Two private in-patient units (with a total of 220 beds), an out-patient service for children and adolescents, an out-patient service for those with addictions and a small number of general hospital neurological wards also provide psychiatric care to the residents in the Province of Verona, a wider area that includes South Verona (Reference Tansella, Amaddeo, Burti, Goldberg and ThornicroftTansella et al, 1998).

The South Verona Psychiatric Case Register

The South Verona Psychiatric Case Register (PCR), which began in 1979, records socio-demographic characteristics, past psychiatric and medical history, clinical data, and contacts with psychiatrists, psychologists, social workers and psychiatric nurses. The PCR collects information not only from the South Verona CPS but also from all public and private psychiatric services of the Province of Verona. Contacts with general practitioners (GPs), psychiatrists and psychologists in private practice are not reported to the PCR. Each attendance at an out-patient clinic and each home visit is counted as a contact. The PCR also routinely records details of patients who leave the catchment area and those who die. Estimates of time spent for each out-patient contact and each domiciliary visit are routinely recorded by the professionals providing care (Reference Tansella, Amaddeo, Burti, Goldberg and ThornicroftTansella et al, 1998), as are the types and numbers of professionals involved. This information forms the basis for calculating costs of specialist mental health care (Reference Amaddeo, Beecham and BonizzatoAmaddeo et al, 1997).

Patients

This study is part of the South Verona Outcome Project, in which cross-sectional standardised assessments of patients in contact with the South Verona CPS have been made each year since 1994. Both first-ever patients and patients already in contact with the service are assessed, using several outcome measures, but in this study we report only the use of the Global Assessment of Functioning Scale (GAF) and the Verona Service Satisfaction Scale (VSSS) for all those seen by a psychiatrist or a psychologist. The Outcome Project study excludes contacts which take place in the casualty department or in the liaison psychiatry department because of logistical difficulties in assessing patients in these settings. Full details of the design of the Outcome Project are given in Ruggeri et al (Reference Ruggeri, Biggeri and Rucci1998).

The official Italian versions of GAF and VSSS were used. The GAF is a measure of individual well-being in the previous month on a continuous scale, where 0 denotes extremely severe dysfunction and 90 extremely good function (Reference Endicott, Spitzer and FleissEndicott et al, 1976). The VSSS consists of 54 items covering 7 dimensions of the patient's experience of mental health services in the previous year: overall satisfaction, the skills and behaviour of professionals, information, access, efficacy, type of intervention and involvement of relatives items are rated on a 5-point Likert scale (1=terrible; 5=excellent) (Reference Ruggeri and Dall'AgnolaRuggeri & Dall'Agnola, 1993; Reference Ruggeri, Dall'Agnola and AgostiniRuggeri et al, 1994).

All key professionals were trained in the correct use of these standardised instruments. Interrater reliability for GAF scores was assessed during the project and was always higher than 0.70 (intraclass correlation coefficient). If necessary, the research team helped the patients to complete the VSSS and assessed their understanding of items and coherence of assessments; confidentiality was fully preserved. The test-retest reliability and the validity of the VSSS have been assessed previously and proved to be good (Reference Ruggeri and Dall'AgnolaRuggeri & Dall'Agnola, 1993; Reference Ruggeri, Leese and ThornicroftRuggeri et al, 2000).

This study includes all first-ever and all previously treated patients who were seen in the cross-sectional assessment period between October and December 1994, and for whom both GAF and VSSS were completed. Using the PCR, each patient was followed-up for 2 years after his/her first contact during the 3-month assessment period. Patients who died or moved away from the catchment area during the first year after the index contact were excluded from the study.

Drop-out patients were defined as those who (a) had a period without psychiatric contacts lasting at least 365 consecutive days, either immediately after the index contact or after further occasional contacts occurring in the following year and (b) those whose termination of treatment was not rated as appropriate.

To rate appropriateness of termination of contact, we considered the reason. This was independently assessed from the case notes of the last recorded contact and rated by a psychiatrist. From these case records, a rating was made for each patient in the following categories who terminated contact: (a) clinical resolution of the episode, (b) termination agreed between patient and clinician for other reasons, (c) termination not agreed, or (d) referral to the GP. When the reason for termination of contact could not be assessed from the case notes, the psychiatrist used the case notes to make a GAF rating of the overall functional level of each patient during the month preceding the date of the last recorded contact. This rating was blind to all previous GAF ratings and to the status of the patient in terms of contact termination. This retrospective method of rating the GAF from case records has been shown to be highly reliable (Reference Mirandola, Baldassari and BeneduceMirandola et al, 2000).

Using the information gathered in the steps outlined above, an appropriate termination of contact with services was defined as applying to: (i) those patients for whom the recorded reason of termination referred to categories (a), (b) or (d) above or (ii) those patients in which the clinical condition at termination showed only a minor degree of disability/symptom severity, as shown by a GAF score of 70, for the month preceding the date of termination of contact, indicating only a mild degree of disability.

Measures used

For each patient, the following data were collected in relation to the index contact.

1. (a) Socio-demographic and diagnostic information (ICD-10 clinical descriptions, case register diagnosis; World Health Organization, 1992), which is reported in the following groups: schizophrenia and related disorders, affective disorders, anxiety-related and somatoform disorders, personality disorders and other disorders.

2. (b) Presence of severe mental illness, defined using the criteria in Ruggeri et al (Reference Ruggeri, Leese and Thornicroft2000): (a) GAF score ≤ 50 and (b) over 2 years of contact with mental health services.

3. (c) Length of contact with services, before entry into the study.

4. (d) GAF score at entry into the study.

5. (e) VSSS score at entry into the study.

6. (f) Psychiatric care received in the 365 days before entry into the study (date on which the GAF and VSSS were rated): admitted or not admitted to hospital, number of day care contacts, number of out-patient contacts, number of domiciliary visits (data from the PCR).

7. (g) Direct costs in the year preceding entry into the study. Costs were attached to each service contact recorded on the PCR so as to give the best local estimates of long-run marginal opportunity costs. Direct costs included are those concerning contacts with public and private specialist mental health services; costs of care provided by GPs, private psychiatrists and psychologists, and medication payments by patients were excluded. As we decided to use the most recent and more comprehensive list of unit costs, costs are expressed in Italian lire at 1999 price levels. All indirect costs were excluded from this study. The cost of out-patient contacts was calculated by taking the cost of a working minute for the different professionals (psychiatrists, psychologists, social workers, nurses, etc.) and multiplying this value by the estimated time spent in each contact. For the in-patient services, the cost per day was calculated, and for the rehabilitation groups and day care, an estimate was made of the cost per contact (taking into account the contact duration). The costs of private in-patient clinics were based on the prices paid by the Italian National Health Service. The support and treatment services listed here are mainly delivered, coordinated and funded by the public sector health service. Full details on the preparation of the unit cost list and the cost calculation have been reported elsewhere (Amaddeo et al, Reference Amaddeo, Beecham and Bonizzato1997, Reference Amaddeo, Beecham and Bonizzato1998).

Statistical analysis

The probability of being a drop-out was assessed by logistical regression. Since the study only included subjects (‘respondents’) for whom complete GAF and VSSS data were available, weights were applied to make the sample representative of all the eligible patients (patients who had been seen in the cross-sectional assessment in October—December 1994). The dependent variable was patient status (drop-out or not drop-out). The independent variables were: socio-demographic characteristics (gender, age, marital status, living situation, education and employment status); clinical characteristics (diagnosis and whether the patient had a severe mental illness or not); length of contact with services before entry into the study; service utilisation characteristics of the patients in the year preceding entry into the study (days of admission to hospital, days of contact with day care, number of contacts with out-patients or domiciliary care and total service costs in the previous year); patient overall functioning at entry into the study as rated by GAF score; and patient satisfaction characteristics at entry into the study, as rated by the VSSS total score and the scores in the seven VSSS dimensions.

Weights for non-response were proportional to the inverse probability of responding, estimated from a logistical regression on the whole group of eligible patients. The weights are greater for respondents with a lower response probability, who are therefore underrepresented in the analysed sample (Reference Iannacchione, Milne and FolsomIannacchione et al, 1991). The independent variables were socio-demographic and clinical information available both for respondents and non-respondents, and the dependent variable was the response status. A ‘missing at random’ mechanism for non-response is assumed, given the characteristics included in the logistical regression model (Reference Brick and KaltonBrick & Kalton, 1996). All statistical analyses were performed using STATA Release 7.0 (STATA Corporation, 2000).

Predicting drop-out

All patients with schizophrenia remained in contact with services during the study period, so the diagnosis of schizophrenia was a perfect predictor for not dropping out. Table 6 shows the final logistic regression model calculated for the remaining 177 patients (excluding those with a diagnosis of schizophrenia, n=46). A backward selection was performed, and in the initial model, socio-demographic characteristics (living condition, working status, educational level, marital status), clinical variables (diagnosis, GAF, length of contact, number of contacts in the previous year), total costs and satisfaction with services were introduced. The prediction formula used in Table 6 and an example of a prediction for a typical patient are shown in the Appendix.

Table 6 Predictors of the probability of dropping out (n=177, excluding those with a diagnosis of schizophrenia (for example of prediction analysis, see Appendix)

	Odds ratio1	s.e.	P	95% Cl
Satisfaction with professionals' skills and behaviour	0.403	0.187	0.05	0.2-1.0
Satisfaction with type of intervention	5.973	4.538	0.02	1.3-26.7
Age	0.952	0.0158	0.01	0.92-0.98
Day care contacts in the previous year	0.917	0.0379	0.04	0.8-0.9
Out-patient contacts in the previous year	0.931	0.0323	0.04	0.9-1

It was found that lower age, less use of day care, and less use of out-patient care in the previous year all increased the risk of dropping out of treatment. Premature termination of treatment was not associated with the other socio-demographic characteristics, psychiatric history or diagnosis (except schizophrenia), in-patient days or community care contact in the previous year.

No significant effect on dropping out was found for GAF score, VSSS total score and for total costs. Instead, associations were found between some aspects of patient satisfaction with services and dropping out: patients with greater satisfaction with the skill and behaviour of professionals had a greater probability of staying in contact, whereas those who were more satisfied with the type of intervention received were 5.9 times more likely to drop out.

Rate of inappropriate termination of contact (drop-out)

Among the 261 patients included in the cross-sectional South Verona Outcome Project we found that 17% (44) had an inappropriate termination of contact during the year after the index contact. Our findings are not consistent with the results of other studies (Reference PekarikPekarik, 1983; Reference Tehrani, Krussel and BorgTehrani et al, 1996; Reference Young, Grusky and JordanYoung et al, 2000), which estimated that between 26% and 40% of patients may inappropriately leave out-patient care in a 1-year period. The lower drop-out found in our study may be explained by several factors. First, the South Verona service is designed to promote continuity of care, especially for people with severe mental illness (none of the patients with schizophrenia dropped out), which explains why only 27% (70 out of 261) of the total group discontinued contact during the year after the index contact. Second, previous studies have defined the concept of inappropriate termination of treatment less stringently (Reference Baekeland and LundwallBaekeland & Lundwall, 1975; Reference Louks, Mason and BackusLouks et al, 1989; Reference Koch and GillisKoch & Gillis, 1991; Reference Mohl, Martinez and TicknorMohl et al, 1991), relating dropping out to the number of out-patient visits made or to the length of time in treatment (Reference Atwood and BeckAtwood & Beck, 1985; Reference Dworkin, Adams and TelshowDworkin et al, 1986; Reference Axelrod and WetzlerAxelrod & Wetzler, 1989; Reference Mohl, Martinez and TicknorMohl et al, 1991). In these investigations, a patient was considered to be a drop-out if he or she terminated treatment before an arbitrary cut-off point, whether the clinician agreed with the termination or not. By contrast, our definition distinguished between appropriate and inappropriate termination. In addition, the possibility that some of our drop-out patients did not, in fact, drop out of contact with services, but rather transferred care to other providers not reporting to the case register, would further reduce the proportion of cases dropping out of care, and would increase the difference between our findings and those of previous studies. In these respects, the South Verona CPS acts more as an assertive outreach team than as a general adult mental health service within the UK context (Department of Health, 2001). This is because it has relatively fewer referrals of patients with lesser disability than catchment area teams in Britain (which often combine both consultation assessments at the request of primary care practitioners and the treatment of a longer-term case-load of patients with greater disability), as shown by previous UK—Italian comparative studies (Reference Amaddeo, Gater and GoldbergAmaddeo et al, 1995; Reference Gater, Amaddeo and TansellaGater et al, 1995).

Variables associated with dropping out of care

Our findings relating drop-out to younger age are consistent with the results of Kline & King (Reference Kline and King1973), Molnar & Pinchoff (Reference Molnar and Pinchoff1993), Tehrani et al (Reference Tehrani, Krussel and Borg1996) and Young et al (Reference Young, Grusky and Jordan2000), although the finding from the univariate analysis that drop-outs are more likely both to be younger and unmarried must be interpreted with caution, as these two variables are often associated in psychiatric datasets. In terms of clinical status, our results show that there was a significant difference in the level of disability between drop-out patients and patients who remained in contact. These results are consistent with those of Robin (Reference Robin1976). However, a recent controlled prospective study at a psychiatric out-patient service in London (without an outreach service) showed that those who missed appointments were more unwell and had higher levels of disability than those who did attend (Reference Killaspy, Banerjee and KingKillaspy et al, 2000). This study, however, referred only to loss of contact with the out-patient component of the service, rather than with any part of the service, and therefore addressed a more restricted issue. In addition, our results show that direct costs of patients who dropped out of treatment in the previous year were significantly lower than those of patients both with and without schizophrenia, who remained in contact.

Predictors of dropping out of care

For the multivariate analysis, we excluded the 46 patients with schizophrenia because none of them dropped out of care in the year following the index contact; the diagnosis of schizophrenia thus might be considered a perfect predictor of non-drop-out. This is because the South Verona CPS is deliberately targeted at those with severe mental illness and if such a patient fails to attend for an appointment, the staff actively arrange to visit them at home to ensure continuity of clinical contact.

Termination of treatment might be assumed to represent a behavioural sign of dissatisfaction, so a strong inverse relationship between inappropriate termination of treatment and satisfaction with care might be expected. Our use of a detailed service satisfaction scale allowed us to differentiate between different aspects of satisfaction. In terms of the bivariate analysis, we found a trend, but no significant differences, for lower satisfaction in drop-out patients (Table 3). However, when multivariate analyses were performed to identify predictors of dropping out, after excluding patients with a diagnosis of schizophrenia, satisfaction with type of intervention received became the most significant predictor. This sub-scale summarises ratings made on 17 items of the VSSS referring to patients' perceptions of a wide range of treatment and care received, from medication to sheltered work and advice on welfare benefits. This suggests that, for a group of patients predominantly without psychoses, dropping out of contact with services is strongly associated with being more satisfied with the interventions received in the period prior to the index contact and this implies that, from the perspective of these patients, the termination of contact was appropriate.

These results indicate that different criteria may be used by staff and by patients not suffering from schizophrenia when judging at which point to discontinue clinical contact, and that these different priorities may well warrant a more detailed investigation. This leads us to the tentative conclusion that a basis for planning a mutually agreed termination of treatment by clinicians should be open discussion with patients as to whether they are satisfied with the type and amount of treatment received, and when they feel that they have had sufficient care.

At the same time, as expected, patients who are less satisfied with the professional skills and behaviour (rating professionalism, competence and thoroughness of staff) are also more likely to drop out of care. Therefore, these aspects of satisfaction may also have important consequences for whether patients without a diagnosis of schizophrenia allow continuing clinical contact, and therefore potentially effective treatment, to take place at all. This also indicates a further avenue for research, namely the interrelationships between the processes and the outcomes of care, in this case where the processes are rated from a patient perspective.