Clozapine aside, any efficacy differences between available antipsychotic drugs are at best minimal. Reference Jones, Barnes, Davies, Dunn, Lloyd and Hayhurst1–Reference Leucht, Komossa, Rummel-Kluge, Corves, Hunger and Schmid3 Perceptions of the adverse effect profiles of different antipsychotics are consequently a major influence on drug choice in clinical practice. To provide a robust evidence base for prescribing decisions, it is important that clinical studies of antipsychotics collect and report data about adverse effects in a systematic way to enable a true and fair comparison to be made of their relative propensity for both common and potentially serious adverse effects. Such information is needed to support patients in making informed choices about their drug treatment, in accordance with national clinical guidelines. 4 However, investigators face several methodological challenges when attempting to quantify drug safety and tolerability in research settings. A number of outcome measures have been developed to collect safety and tolerability data, but these vary in the number, type and definition of adverse effects that they cover. Reference Jordan, Knight and Pointon5 The choice of outcome measure used to evaluate adverse treatment effects has the potential to bias safety data in studies comparing antipsychotics head-to-head, particularly when the pharmacological profiles of the agents involved indicate that they are liable to induce different effects. Studies that rely solely on clinical observation and/or spontaneous reporting to collect safety and tolerability data are likely to underestimate the prevalence of subjective, intimate or hidden adverse effects because patients appear to underreport treatment-related symptoms when asked general questions about antipsychotic tolerability. Reference Byerly, Nakonezny, Fisher, Magouirk and Rush6,Reference Yusufi, Mukherjee, Flanagan, Paton, Dunn and Page7 In addition to the difficulties associated with collecting comprehensive adverse effect data, several reviews have shown that the reports of safety information from clinical drug studies in different fields of medicine are often difficult to interpret. Reference Edwards, McQuay, Moore and Collins8–Reference Papanikolaou, Churchill, Wahlbeck and Ioannidis11 Deficiencies noted include failures to substantiate general statements regarding adverse drug effects with numerical data and to describe the severity of adverse effects in adequate detail. Reference Ioannidis and Lau9–Reference Papanikolaou, Churchill, Wahlbeck and Ioannidis11 The primary focus on efficacy in reports of clinical drug studies arguably biases the evidence base towards symptom control data, making it harder for clinicians to use research findings to evaluate relative risks associated with different treatment options. To investigate how safety and tolerability data are collected and presented in clinical studies of antipsychotic agents, a survey was undertaken of a representative sample of published reports.
The two primary objectives of the survey were to investigate how information about the adverse effects of antipsychotics is elicited in research settings, and to examine how the adverse effect and tolerability data captured using assessment tools are reported.
Method
Survey sample
Data were extracted from a sample of journal articles recorded on the register of the Cochrane Schizophrenia Group (CSG). Reference Adams, Coutinho, Davis, Duggan, Leucht and Li12
Studies are included in the register if they are relevant to the treatment, rehabilitation and management of people with non-affective functional psychoses, schizophrenia or chronic/severe mental illness. The register was established in 1991 and is compiled by searches of 77 databases and manual searching of 30 journals and 23 conference proceedings. At the time of our data extraction, the register comprised 12 866 entries, which related to 10 979 separate studies published from 1949 to July 2007. The decision was taken to seek permission to access the CSG register because it was considered that this would provide a more wide-ranging, representative sample of reports than an electronic database search.
Inclusion criteria
Studies were selected from the CSG register for inclusion in the survey if they were published in English between January 2002 and July 2007 and if their key outcome measures were the efficacy and/or tolerability of antipsychotic drugs used in the treatment of acute psychotic illness or schizophrenia. The sample was not restricted to clearly randomised controlled trials but could also include controlled clinical trials with less explicit descriptions of allocation.
Exclusion criteria
Conference proceedings, posters, letters to journal editors, entries in national research registers and review articles that were included in the CSG register were excluded from the survey on the grounds that trial methods and outcomes could not be expected to be fully reported in these media. Also excluded were studies investigating the efficacy of non-drug treatments and service interventions; studies investigating antipsychotics not listed in the British National Formulary (BNF); 13 studies examining adjunctive treatment with drugs not listed as antipsychotics in the BNF; and studies examining the efficacy of drugs prescribed to treat antipsychotic side-effects (e.g. anti-obesity drugs) or to manage comorbid substance misuse.
Data collection
The following information was extracted from each of the articles surveyed and entered into Microsoft Access 2003 on Windows XP, Professional Service Pack 2 using an electronic data collection form: publication details (journal, date, authors and reference), study design information, names of study drugs and participant demographics (age, gender, diagnosis, treatment setting). A score indicating the degree of bias in a study's design was also entered into the database as a measure of study quality, in accordance with published rating criteria. Reference Jadad, Moore, Carroll, Jenkinson, Reynolds and Gavaghan14
The primary outcomes of interest were the nature of the adverse effects recorded in the trial reports, the methods used to assess them and the way in which data from formal assessment tools were reported. Owing to the diverse range of adverse effects associated with antipsychotic medication, the data collection form was designed to capture information about effects that are most frequently associated with antipsychotic treatment (extrapyramidal symptoms, prolactin elevation, weight gain), those linked to physical heath problems of a long-term or serious nature (lipid disturbances, glucose dysregulation, cardiac abnormalities) and those reported to be particularly distressing to patients (sexual dysfunction). Reference Fakhoury, Wright and Wallace15,Reference Finn, Bailey, Schultz and Faber16 The data collection form also enabled reports to be identified that referred to assessing or recording data about other general treatment-emergent problems, such as dizziness, sedation, constipation and dry mouth, or which referred to the evaluation of patients' subjective experience of treatment.
The electronic data collection form was tested on a sample of reports. Changes made as a result of this trial included the addition of a checkbox for reports that referred to general, unspecified laboratory tests and the inclusion of an expanded range of assessment methods for generic adverse effects (e.g. self-report, observation and systematic evaluation). A notes section was also added to the form to enable information to be recorded about adverse effects not covered by the existing options, (e.g. injection site pain for depot antipsychotic treatment).
All entries on the CSG register that were published between January 2002 and July 2007 were exported into a Microsoft Access database, which was set up to assist in screening entries to identify whether they met the inclusion criteria. If it was unclear from the CSG register title whether the entry met the inclusion criteria, the full paper was accessed for further scrutiny. Use of a checkbox in the data collection form enabled all entries meeting the review criteria to be easily identified. An additional checkbox enabled studies to be labelled if there was any doubt regarding their eligibility, which was then jointly reviewed by three of the investigators (A.P., C.P. and T.R.E.B.).
During the review process, reports presenting data from the same trial as another report were marked in the database as repeats. If several reports were found presenting sub-analyses of data from a single study, data from all of the reports were combined into a single database entry. Where it was clear from a report that details of the study design and methodology were described elsewhere, the original report was located and data on study quality, adverse effect assessment and reporting were extracted from both reports and combined in a single database entry for the study. If separate reports described different phases of a trial (e.g. a double-blind, randomised acute treatment phase and open-label follow-up phase), these were treated as separate entries in the database on the grounds that different approaches to assessing treatment safety may be adopted at different stages in a trial. For example, diagnostic research criteria for tardive dyskinesia require the condition to be present for a minimum of 3 months, Reference Schooler and Kane17 so it might be more appropriate to assess tardive dyskinesia in long-term follow-up studies rather than acute treatment trials.
Once data had been entered into the database for all of the reports that met the survey inclusion criteria, queries were run to identify missing data or anomalous values, which were then rectified. A series of reports was also run to identify where multiple reports presented data from a single trial.
Results
A total of 2087 entries published in English between January 2002 and July 2007 were retrieved from the CSG register. Of these, a total of 196 reports met the inclusion criteria for the survey, of which 59 presented subsets of data from 25 separate studies. Each study was allocated a single entry in the database, unless it consisted of discrete phases reported in separate papers. For analytic purposes distinct phases of larger trials (such as long-term follow-up) were treated as separate studies. Relevant data were extracted from 27 reports published before 2002 that were related to studies in the review sample (e.g. study protocols). The final sample consisted of 167 database entries, 15 of which presented data from discrete phases of seven separate trials. For ease of reference, these final database entries are referred to as ‘studies’ in the remainder of this report.
Characteristics of the reports surveyed
The final sample consisted of 103 controlled trials of antipsychotic drugs, 90 of which were double-blind. There were 59 open-label studies and 5 that used a cross-sectional or pragmatic design in order to capture ‘real world’ treatment conditions. Approximately half of the studies surveyed investigated treatment with second-generation antipsychotic drugs, 42% compared treatment between first-generation and second-generation antipsychotics and 6% exclusively investigated treatment with first-generation antipsychotics. The mean quality score was 2.6 for the whole sample, 3.3 for the randomised controlled studies and 1.5 for the open-label studies. Reference Jadad, Moore, Carroll, Jenkinson, Reynolds and Gavaghan14 Overall, 79 studies in the sample (47%) had a quality score of 2 or less out of 5, reflecting the finding that relatively few papers provided explicit information about methods of randomisation and masking that would minimise bias. The primary objective of the majority of the studies surveyed was to examine antipsychotic efficacy; just 37 studies (22%) were designed to investigate specific types of adverse effect, predominantly extrapyramidal side-effects (EPS), metabolic side-effects and adverse subjective experiences.
Percentage of the sample (167 studies) that described the assessment method and/or reported on specific adverse effects of antipsychotic medication. ECG, electrocardiography; RCT, randomised controlled trial.
Side-effects reported
Figure 1 shows the proportion of the 167 studies assessing specific adverse effects. Movement disorders were most frequently reported and assessed using published rating scales in the articles surveyed. Few articles (12%) evaluated patients' subjective experience of treatment or assessed sexual dysfunction associated with antipsychotic use.
Adverse effect assessment methods
Figure 2 shows the proportion of the 167 studies reporting specific adverse effects that referred to using a published rating scale or questionnaire to assess them. Such scales were frequently used to assess EPS, but other types of adverse effect did not tend to be assessed systematically.
Proportion of the sample (167 studies) reporting specific types of adverse effect, indicating those that used a published rating scale or questionnaire for systematic data collection. TEAE, treatment-emergent adverse effect.
General treatment-emergent adverse effects
For the purposes of the survey, general treatment-emergent adverse effects were classified as problems other than drug-related movement disorders, sexual dysfunction, weight gain, metabolic abnormalities and hyperprolactinaemia. Examples include anticholinergic effects (e.g. dry mouth, constipation), gastrointestinal problems and sedation. Just under 30% (48) of the study reports surveyed did not provide any information about general treatment-emergent adverse effects. Of the 119 reports that did refer to general treatment-emergent adverse effects, collection methods were either unclear or not stated in more than 40% (51). Only 35 reports (21%) explicitly stated that data about general treatment-emergent adverse effects were collected systematically; of these, 19 stated that they had used a published rating scale, representing just 11% of the total sample.
Extrapyramidal side-effects
Of the 167 studies surveyed, 86% made some reference to assessment of EPS (Fig. 2) and more than 90% of these employed a published movement disorder rating scale. The most frequently cited scale was the Simpson–Angus Scale (SAS), Reference Simpson and Angus18 used in 71% of the studies that formally evaluated these symptoms. The next most frequently cited scales were the Barnes Akathisia Rating Scale (BARS), Reference Barnes19 employed in 58% of studies, and the Abnormal Involuntary Movement Scale (AIMS), Reference Guy20 used in 52%. A total of 46 studies (28%) used all three scales to assess the full range of EPS associated with antipsychotics.
Metabolic parameters
Weight gain, referred to in 57% of the reports surveyed, was the adverse effect most frequently assessed and reported after EPS. Most of the studies (82%) that reported weight data compared treatment groups’ mean weight change. Thirty-five per cent of the studies that measured drug-related weight change reported the proportion of patients who had gained above a certain percentage of their baseline body weight (usually 7%), and less than a third (28%) reported changes in body mass index (BMI). Only 9 of the 96 studies (9%) that referred to weight change reported all three parameters (mean weight change, BMI and the percentage of patients gaining more than a specified percentage of their baseline body weight). Few studies made explicit reference to assessing metabolic disturbances: 30 studies (18%) reported testing lipids, 31 (19%) measured glucose levels and 38 (23%) assessed patients' blood pressure. Only 7 studies, all of which were double-blind randomised controlled trials, explicitly referred to assessing lipids, glucose, blood pressure and weight gain, which are the key markers of the metabolic syndrome. Of the reports surveyed, 40% referred to conducting routine laboratory tests in the method section, but without defining which tests were requested. A large number of reports also referred to monitoring ‘vital signs’ in the method section, so it is possible that some of these studies also collected metabolic assessment data that were not fully reported.
Cardiac monitoring
Just over a third of the studies surveyed (57 in total) referred to electrocardiographic assessments.
Prolactin levels
Approximately a third (34%) of the reports referred to measuring patients' plasma prolactin levels. Of these, 16 (10% of the total sample) explicitly considered the relationship between prolactin abnormalities, adverse effects on sexual function and menstrual disturbances.
Sexual side-effects
Twenty (12%) articles reported sexual dysfunction associated with antipsychotic use, of which six used a formal assessment measure to evaluate drug-related changes in sexual desire, arousal and satisfaction.
Subjective experience
Only 20 studies (12%) assessed patients' subjective experience of antipsychotic treatment. Of the 20 studies that did so, 15 used a formal rating scale. The most frequently cited scale was the Drug Attitude Inventory. Reference Hogan, Awad and Eastwood21
Reporting of adverse effect data
The majority of studies that provided information about general treatment-emergent adverse effects included tables reporting the number and/or percentage of patients experiencing specific problems. Tables of adverse effects could be difficult to interpret because they sometimes included symptoms of schizophrenia (e.g. hallucinations) as well as physiological problems that could be related to drug-induced or psychiatric symptoms (e.g. restlessness, which may be due to akathisia or anxiety). Of the 119 study reports that referred to adverse effects or events, 48 (40%) limited inclusion of such data to problems observed in at least 5% or 10% of patients. Few reports provided specific information about the severity, frequency or duration of adverse effects. Statements regarding severity tended to be vague (e.g. ‘most adverse effects were mild to moderate’) and it was frequently unclear how different degrees of severity had been defined. Authors typically failed to state whether investigators attributed adverse effects to the study drug, dosage or other factors (e.g. withdrawal of previous treatment).
Data from EPS rating scales tended to be used statistically to describe treatment groups (e.g. mean group score) or to draw inferences about the risk of developing EPS associated with different antipsychotics. Only about 25% reported the number or percentage of patients developing EPS or fulfilling recognised diagnostic criteria (such as reaching a diagnostic threshold score on a rating scale). The general trend towards reporting mean numerical values and significance testing was not limited to EPS scale data. Most studies that measured patients' prolactin levels reported statistical data, but did not include any systematic assessment of adverse effects potentially associated with plasma prolactin elevation.
Discussion
The aim of this survey was to investigate the methods used to assess and report adverse effects in clinical studies of antipsychotics used for the treatment of schizophrenia. Assessment measures are of particular interest in this field of research given the diverse range of adverse effects associated with antipsychotics, the common use of these drugs long-term in people with schizophrenia, and the clinical need to understand the distinct safety and tolerability profiles of the individual antipsychotic drugs available. We found that standardised measures were not routinely employed to capture information about the incidence and severity of non-neurological adverse effects. In contrast, more than 90% of the reports that referred to movement disorders used published rating scales to assess them. The majority referred to using one or more of three rating scales (SAS, BARS and AIMS) to assess movement disorders, although most failed to evaluate the full range of EPS associated with antipsychotics.
The common use of rating scales to assess EPS in these recent studies contrasts with a survey by Thornley & Adams of 2000 reports published between 1950 and 1998, which found that 25% did not use rating scales to assess tolerability or efficacy outcomes: only 9% of the whole sample referred to using the SAS and 6% to the AIMS. Reference Thornley and Adams22 Although Thornley & Adams surveyed study reports of a broad range of interventions used in the treatment of schizophrenia, including psychotherapy and case management, studies of antipsychotics made up nearly 60% of the total sample. The marked difference between these findings and those of our survey suggests that the use of published rating scales to assess EPS has become accepted as standard practice in clinical studies of antipsychotics published in English.
Despite broad uniformity in the choice of measures used to evaluate EPS, findings from different studies remain difficult to interpret owing to variations in reporting. Scores derived from movement disorder rating scales tend to be presented in statistical terms (e.g. mean total score), which do not provide clinicians with meaningful information about the prevalence of individual movement disorders at defined threshold levels of severity. Data on certain other side-effects were often presented in statistical terms, making it difficult to gauge the proportion of patients with a clinically relevant problem. For example, most studies that measured prolactin levels presented mean values for the treatment groups, and did not report any systematic assessment of symptoms that might be associated with hyperprolactinaemia, such as sexual dysfunction or menstrual irregularities.
Change in weight was the second most frequently assessed safety parameter in the reports surveyed, partly reflecting increasing concerns about physical health risks. Reference Allison and Casey23,Reference Marder, Schooler, Miller, Buchanan, Casey and Davis24 Given the burgeoning evidence for metabolic disturbance associated with antipsychotic medication, it was notable that relatively few of these recent studies explicitly referred to markers of the metabolic syndrome (i.e. abnormal glucose concentrations, lipid levels and blood pressure). This finding may partly reflect the time lag between the planning and execution of such studies and publication of the findings. It is also possible that some reports may have failed to mention assessments of metabolic functioning, as 40% of the sample referred to laboratory screening without specifying which tests were undertaken.
Investigators appeared to prioritise the collection of information about EPS and drug-related weight gain; other adverse effects associated with antipsychotics were assessed much less frequently and less consistently. Sexual dysfunction and aversive subjective responses to antipsychotic treatment were the adverse effects least likely to be assessed or reported in the survey sample. Although most studies reported information about general treatment-emergent adverse effects, the methods used to collect these data were unclear or not stated in more than 40% of the reports that mentioned them. This is comparable to findings in other clinical settings. For example, an examination of 52 reports of double-blind, randomised trials of single-dose analgesics used in postoperative pain found that 37% failed to specify how information about adverse effects had been collected. Reference Edwards, McQuay, Moore and Collins8 When these authors compared findings among studies that described the methods of collecting information about adverse effects, they found that the reported incidence of symptoms was higher with patient diaries than when direct questioning was used. Reference Edwards, McQuay, Moore and Collins8 The impact of the assessment method used has also been noted in studies of antipsychotics: patients typically report more side-effects when they are questioned systematically than when they are asked general questions about treatment tolerability. Reference Byerly, Nakonezny, Fisher, Magouirk and Rush6,Reference Yusufi, Mukherjee, Flanagan, Paton, Dunn and Page7 Patients may underreport adverse effects for a number of reasons, including misattribution of symptoms, forgetfulness and embarrassment about raising intimate adverse effects. Reference Rosenberg, Bleiberg, Koscis and Gross25
If standardised checklists or rating scales designed to assess a comprehensive range of adverse effects are not employed to collect safety and tolerability data in a systematic way, the prevalence of certain types of adverse effect is likely to be underestimated. Inconsistent collection and reporting of information about adverse effects frustrates attempts to learn about the relative risk profiles of different antipsychotics as it limits the ability to pool data from different studies. For example, a recent study that examined the prevalence of anticholinergic effects by analysing data from 177 randomised trials of antipsychotics included in Cochrane reviews found that an estimated 40 000 trial participants did not have data on these outcomes either recorded or reported. Reference Ozbilen and Adams26 These investigators had predicted that the UKU side-effect rating scale and the Liverpool University Neuroleptic Side-Effect Rating Scale (LUNSERS) would typically be used in trials to assess anticholinergic effects. Reference Lingjærde, Ahlfors, Bech, Dencker and Elgen27,Reference Day, Wood, Dewey and Bentall28 In line with our findings, only seven of the trials reviewed explicitly referred to using the UKU and none reported using the LUNSERS.
Study limitations and strengths
The survey data were extracted by a single researcher. However, to minimise the risk of relevant data being missed, an electronic data collection form was used to provide a checklist for the researcher when extracting information from reports. A series of Microsoft Access queries was also run following completion of the survey in order to identify and correct missing data and anomalous values such as large participant numbers or invalid quality scores. One limitation of our survey is that the findings are limited to trial reports published in English. Previous research has shown that there is international variation in reporting of safety data. Reference Hayashi and Walker29
The primary strengths of the study are the comparatively large number of trials surveyed, focused on a specific type of drug and condition. The fact that the findings were so consistent, with clear differences found between the assessment of EPS and non-neurological side-effects suggests that the results are robust. Further, the survey findings replicate those of previous studies in other fields of medicine and are also supported by more recent evidence from a study that used data from antipsychotic trials to investigate the prevalence of anticholinergic effects. Reference Edwards, McQuay, Moore and Collins8,Reference Loke and Derry10,Reference Ozbilen and Adams26
Clinical implications
Findings from a meta-analysis of randomised controlled trials comparing second-generation antipsychotics head-to-head in the treatment of schizophrenia led the investigators to conclude that efficacy differences between these drugs are likely to be modest, and that when selecting drugs for individual patients, any potential small advantage in efficacy must be balanced against relatively large differences in adverse effect profiles and cost. Reference Leucht, Komossa, Rummel-Kluge, Corves, Hunger and Schmid3 To achieve this in practice, and to inform patient choice, clinicians need to have a clear understanding of both the short-term and longer-term risks associated with the antipsychotic agents currently available. The findings of our survey demonstrate why it can be difficult to use outcome data from clinical studies to obtain reliable estimates of the relative propensity of individual antipsychotics to cause specific adverse effects. In clinical practice, the available sources of information about the relative risk profiles of different antipsychotics, such as conferences, lectures and training courses, are likely to summarise data from clinical studies. Similarly, national treatment guidelines referring to the assessment and management of antipsychotic side-effects will be based in part on evidence derived from systematic reviews of published clinical trials. Another key source of information for clinicians is their clinical experience of the side-effects identified in their patients, and the shared experience of their peers in this regard. However, such information may be distorted by selective patient report and lack of systematic assessment. For example, the prevailing knowledge about the tolerability profiles of individual antipsychotics derived from clinical trials may in turn cause clinicians to preferentially ask patients about particular types of adverse effects. Although knowledge about certain types of adverse effects, notably EPS and weight gain, will be acquired, other troubling problems that may be harder to evaluate systematically such as anticholinergic effects, sexual dysfunction and aversive subjective experiences may be neglected. There is consequently a need for a more consistent approach to assessing and reporting adverse effects in clinical studies of antipsychotics. The Consolidated Standards of Reporting Trials (CONSORT) statement has been developed to promote transparent reporting of randomised controlled trials. Reference Moher, Schulz and Altman30,Reference Altman, Schulz, Moher, Egger, Davidoff and Elbourne31 It advocates the appropriate use of ‘validated scales or consensus guidelines’ for measuring primary and secondary outcomes in clinical trials, and states that all important adverse events and side-effects should be reported in each intervention group, with measures of the severity of adverse events operationally defined. Reference Altman, Schulz, Moher, Egger, Davidoff and Elbourne31 To support consistent implementation of CONSORT, a set of international standards could be developed for use in clinical studies of antipsychotics outlining the minimum acceptable set of data that should be collected on adverse effects, the outcome measures that should be used to evaluate treatment safety and tolerability, operational definitions of side-effect severity, and guidelines for reporting adverse effect data in a clinically meaningful way. In the field of rheumatology, for example, international collaborative work has enabled agreement to be reached on core sets of outcome measures for use in clinical trials. Reference Tugwell, Boers, Brooks, Simon, Strand and Idzerda32 Developing agreement within the research community on standards for assessing and reporting adverse effects in studies of antipsychotics could minimise the potential for the selection of assessment measures to bias tolerability data in favour of specific drugs with particular side-effect liabilities, and would assist regulatory bodies in ensuring that the summary of product characteristics for each drug contained a more accurate and complete list of side-effects. A consistent approach to collecting and presenting side-effect data from studies of antipsychotics would also contribute to the development of a more robust evidence base for comparing the safety and tolerability of individual drugs, thus enabling prescribers and patients to make more informed decisions about treatment options.
eLetters
No eLetters have been published for this article.