Background
Stroke is a leading cause of mortality and morbidity. Reference Feigin, Forouzanfar and Krishnamurthi1 Ongoing evaluation of the organized stroke systems of care established across Canadian provinces is necessary to ensure excellence in care and outcomes. Reference Kapral, Laupacis and Phillips2 Such studies typically use linked administrative health data to include large populations over long time periods. Reference Yu, Holodinsky and Zerna3 However, accurate and fair comparisons of patient outcomes require risk adjustment for baseline stroke severity because it is one of the most important predictors of outcomes. Reference Fonarow, Pan and Saver4–Reference Katzan, Spertus and Bettger6 An important limitation of stroke research using administrative data is the lack of a measure of baseline stroke severity. Even in clinical databases created using primary data collection, stroke severity is often missing. Reference Rost, Bottle and Lee5,Reference Thompson, Luo, Gardiner, Burke, Nickles and Reeves7,Reference Smith, Shobha and Dai8
The Passive Surveillance Stroke SeVerity (PaSSV) score was derived using Ontario administrative data, and was found to be associated with 30-day all-cause mortality after stroke with a similar magnitude of effect as for observed stroke severity ascertained from clinical data in the Ontario Stroke Registry. Reference Yu, Austin and Rashid9 The PaSSV score has the potential to be used as a risk-adjustment tool for multi-jurisdiction stroke outcome comparisons across Canadian provinces. However, health data structure is different in each province, necessitating dedicated external validation analyses for each province. 10
We calculated the PaSSV score using administrative data in British Columbia (BC) and Nova Scotia (NS) between 2014 and 2019, and we assessed its association with stroke mortality and admission to long-term care in each province. These analyses were also carried out in Ontario to assess validity PaSSV over time. We hypothesized that PaSSV will show similar validity across provinces and over time.
Methods
Study cohort
We used diagnosis codes from the Canadian Institute for Health Information’s Discharge Abstract Database 11 (DAD) to identify hospitalizations with a most responsible diagnosis of ischemic stroke (H34.1, I63 except I63.6, I64) or intracerebral hemorrhage (I61) between April 1, 2014 and March 31, 2019. These codes have been shown to have high accuracy with positive predictive value of 92% for intracerebral hemorrhage and 97% for ischemic stroke. Reference Porter, Mondor, Kapral, Fang and Hall12 The beginning of the cohort accrual period falls after the period during which the PaSSV score was derived in Ontario (2002–2013), thereby allowing us to assess the temporal validity of the score in Ontario, as well as its geographic external validity to other provinces.
To exclude elective admissions, we only included records where the admission was through an emergency department and the corresponding National Ambulatory Care Reporting System (NACRS) record was available. 13 While the NACRS coverage of emergency departments is complete in Ontario, it is incomplete in BC and NS. 13 According to the 2019–20 CIHI estimates, only 27.8% (30 of 108) of emergency departments in BC and 21.1% (8 of 38) in NS were mandated to report to NACRS, accounting for an estimated coverage of 71% of all emergency visits in BC and 49% in NS. 14 We also excluded patients aged < 18 or > 105 years, those with an invalid health card number, and those who experienced a stroke while hospitalized for a different reason. Among individuals with multiple eligible events, we only included the first one (Table 1). Given inter-hospital transfers are common in stroke, we created an episode of care for each hospitalization to avoid double counting events. The index date was the first day of the episode of care. In Ontario, we used ICES’ (previously Institute for Clinical Evaluative Sciences) standard definition: any admissions within 6 hours of the previous discharge, any admissions within 12 hours of the previous discharge where discharge codes indicate transfer between two acute care hospitals, and any admissions within 48 hours of the previous discharge where the “institution from” and “institution to” numbers match. In BC and NS, we included all emergency department visits within 48 hours of the hospitalization as long as these were also within 24 hours of each other.
Table 1: Cohort creation flow in each province
NACRS = National Ambulatory Care Reporting System; CTAS = Canadian Triage and Acuity Scale.
Passive surveillance stroke SeVerity (PaSSV) score
The PaSSV score was developed in Ontario, where observed stroke severity was available in the Ontario Stroke Registry between 2003 and 2013. Reference Yu, Austin and Rashid9 The Ontario Stroke Registry is a population-based clinical stroke registry with information on the Canadian Neurological Scale (CNS) score, a stroke severity scale ranging from 1.5 to 11.5 where a lower score indicates higher severity, and this clinical score is always an integer or ends in 0.5. Reference Cote, Battista, Wolfson, Boucher, Adam and Hachinski15 The PaSSV score was derived by fitting a multivariable linear regression model in which we regressed the CNS as a continuous variable on predictor variables obtained from administrative databases. Reference Yu, Kapral and Park16 The components of PaSSV include information from the emergency department (Canadian Triage and Acuity Scale score, arrival by ambulance), hyperacute stroke care (transfer to a higher-level stroke centre, mechanical ventilation within two days of the index date) and the International Classification of Diseases 10th revisions Canadian codes for stroke symptoms (Supplemental Table 1).
Outcomes
The main outcome was the time to all-cause mortality within 30 days of the index date obtained from the BC vital statistics 17 , NS vital statistics and Insured Patient Registry and the Ontario Registered Persons Database. We also evaluated the association between PaSSV and admission to long-term care within 365 days among patients who were not in a long-term care facility at baseline. Admission to long-term care was identified using the Home and Community Care database and prescriptions dispensed under the PharmaNet long-term care plan in BC, the Continuing Care Reporting System in Ontario, and the Eligibility Group database in NS.
Research ethics, privacy and data access
Datasets were linked deterministically using unique encoded identifiers in each province. In Ontario, data were linked and analyzed at ICES and the use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act without requirement for review by a Research Ethics Board. In British Columbia, data were linked at Population Data BC and analyzed at the UBC Centre for Health Services and Policy Research. Access to data provided by the Data Stewards is subject to approval but can be requested for research projects through the Data Stewards or their designated service providers. The following data sets were used in this study: consolidation (census geocodes, demographics, registry), Home and Community Care, DAD, NACRS, PharmaNet, vital statistics and medical services plan payment information file. You can find further information regarding these data sets by visiting the PopData project webpage at: https://my.popdata.bc.ca/project_listings/22-001/collection_approval_dates. All inferences, opinions and conclusions drawn in this publication are those of the author(s), and do not reflect the opinions or policies of the Data Steward(s). In Nova Scotia, data were linked and analyzed at Health Data Nova Scotia with approval from the Nova Scotia Health Research Ethics Board (REB file #1027160).
Statistical methods
We described the patient characteristics and stroke severity using PaSSV in each province. We calculated the PaSSV score using the previously published beta coefficients. Reference Yu, Kapral and Park16 The theoretical range of PaSSV is from −2.3 to 13.1 and unlike the clinical CNS score, PaSSV is not constrained to be either an integer or to end in 0.5, and it was modeled as a continuous variable in our analyses. We estimated the hazard ratios (HR) and 95% confidence intervals (CI) for 30-day all-cause mortality after stroke using two Cox proportional hazards models, one with PaSSV and one without, in each province. All models adjusted for age (continuous), sex, Charlson comorbidity index (dichotomized < 2 versus ≥ 2) with a 5-year look-back period, Reference Quan, Sundararajan and Halfon18 and stroke type (ischemic versus intracerebral hemorrhage). We compared the models’ discriminative value using Uno’s c-statistics and 95% CI. Reference Uno, Cai, Pencina, D’Agostino and Wei19 We repeated the analyses with the outcome of admission to long-term care within 365 days using adjusted cause-specific hazard models in order to account for the competing risk of death. Analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, North Carolina).
Results
We identified 25,933 hospitalizations for ischemic stroke or intracerebral hemorrhage in BC, 78,412 in Ontario, and 6,385 in NS. After applying the exclusion criteria, 18,387 (70.9%) hospitalization records from unique patients were included for analysis in BC, 65,082 (83.0%) records in Ontario, and 2,673 (41.9%) records in NS. As shown in Table 1, the proportion of hospitalizations excluded for each exclusion criterion was similar across provinces except for linkage to an emergency department record in NACRS. A higher proportion of hospitalizations in BC and NS, than in Ontario were excluded due to lack of linkage to NACRS: 17.9% (n = 4,654) of patients in BC, 3.6% (n = 2,810) in Ontario, and 48.3% (n = 3,082) in NS.
Table 2 shows the patient baseline characteristics by province, the composite PaSSV score, as well as the frequency of the individual components of PaSSV. There were more patients with a Charlson comorbidity index of ≥ 2 in Ontario than in BC or NS. The mean and median PaSSV scores were similar in the three provinces, but there were differences in the components of the composite score. Compared to the other two provinces, a lower proportion of patients in BC were triaged to the highest acuity CTAS score or underwent transfer to a hospital with a higher level of stroke care, but a higher proportion arrived by ambulance. Use of mechanical ventilation and stroke symptoms recorded in administrative data were similar across provinces.
Table 2: Baseline characteristics by province
In BC, 2,678 (14.6%) patients died within 30 days and among the community-dwelling patients at baseline, 2,355/17,536 (13.4%) were admitted to long-term care within 365 days. In Ontario, 9,199 (14.1%) died and 9,045/61,420 (14.7%) were admitted to long-term care. In NS, 349 (13.1%) patients died and 200/2,587 (7.7%) were admitted to long-term care. In multivariable models, every 1-unit increase in the PaSSV score, where a higher score indicates lower stroke severity, was associated with a lower hazard of death (HR and 95% CI 0.70 [0.68, 0.71] in BC, 0.69 [0.68, 0.69] in Ontario, 0.72 [0.68, 0.75] in NS) and lower hazard of admission to long-term care (0.77 [0.76, 0.79] in BC, 0.84 [0.83, 0.85] in Ontario, 0.86 [0.79, 0.93] in NS), shown in Table 3. Adding PaSSV as a covariate in the multivariable models for 30-day mortality improved the models’ discriminative ability as demonstrated by higher c-statistics (Table 3). We made a similar observation for the 365-day long-term care outcome, but the increase in c-statistics was to a lesser extent.
Table 3: Uno’s c-statistics [95% confidence intervals] and hazard ratios [95% confidence intervals] for multivariable models for all-cause mortality within 30 days and admission to long-term care within 365 days with and without the PaSSV variable
A higher proportion of patients were excluded in BC (29.1%) and NS (58.1%) than in Ontario (17.0%) primarily based on the lack of linkage to an emergency department visit in the NACRS database preceding the admission. Most patients without NACRS records had a flag in the DAD database that the hospitalization was preceded by an emergency department visit (96.5% in BC [4,307/4,465] and 80.6% in NS [2,484/3,082]), indicating that most of these hospitalizations were not elective in nature. We observed several differences in patient characteristics comparing those with versus without NACRS record, the most striking one being that there was a higher proportion of rural residents among those without a NACRS record: 40.2% of patients without a NACRS record in BC and 42.7% in NS were living in rural areas, compared to only 6.2% of those with a NACRS record in BC and 31.0% in NS were living in rural areas (Supplemental Table 2).
Discussion
We successfully calculated the PaSSV score using provincial linked administrative data in 86,142 patients who were admitted to hospital with an ischemic stroke or intracerebral hemorrhage over a 5-year period in three Canadian provinces and made several interesting observations. First, the composite PaSSV score was similar among patients in all three provinces. This is consistent with our clinical expectation that stroke severity would be similar across provinces. Our study confirms that the global PaSSV score is not affected by potential variations in clinical practice patterns or coding practices in different Canadian provinces. Second, we showed that a higher PaSSV score (lower stroke severity) was associated with a lower hazard of death within 30 days and admission to long-term care within 365 days, again consistent with the clinical expectation of that lower stroke severity is associated with better outcomes. Third, we showed that adding PaSSV to multivariable models to predict death or long-term care admission improved the models’ discriminative ability compared to those without PaSSV. The c-statistic and 95% CI for the 30-day all-cause mortality models in all three provinces in the current study were similar to the c-statistic of that published in the original PaSSV derivation cohort (0.76 [0.75, 0.76]). Reference Yu, Austin and Rashid9
These observations suggest that the PaSSV score, initially developed in Ontario, has geographical and temporal external validity. The ability to account for differences in stroke severity is important for stroke outcomes research and quality improvement initiatives within each province and when comparing care and outcomes across jurisdictions. For example, prior work showed that patients being evaluated in comprehensive stroke centers with advanced stroke care and treatments were more likely to be experiencing more severe strokes compared to those treated in primary stroke centers or non-designated centers, and accounting for PaSSV reclassified 18.5% of 157 acute care hospitals across Ontario with regards to their risk-standardized stroke mortality performance compared to a model without PaSSV.(16) PaSSV has also been used for risk-adjustment in a recent American study on population-based access to thrombectomy. Reference Kamel, Parikh and Chatterjee20
A critical limitation is that PaSSV requires linkage to the NACRS database, which contains important information related to the emergency department visit that reflect stroke severity, including the emergency triage acuity, arrival by ambulance and inter-hospital transfers. Stroke is a medical emergency and as expected, most hospitalizations were through an emergency department in BC and NS, even when there was no NACRS record. Prior work using Alberta administrative data, where reporting to NACRS is complete, showed that PaSSV is associated with clinical outcomes after stroke. Reference Joundi, King and Stang21 However, in BC and NS, reporting is incomplete, and hospitals that do not report to NACRS tend to be smaller centers located in rural areas, and as a result, patients with stroke living in rural area were more likely to be excluded from our analyses. This is particularly regrettable because rural residents have been shown to have reduced access to standard stroke investigations and services. Reference Kapral, Hall and Gozdyra22 Incomplete reporting to NACRS in BC and NS creates additional challenges to province-wide quality evaluations. Analyses in NS were particularly affected because only 41.9% of patients with stroke could be included in this study. Thus, our results may not be generalizable to the entire province. We hope our findings will encourage more provinces to fully mandate reporting to NACRS for all emergency departments.
In addition to incomplete NACRS coverage, our study had other limitations. We did not have access to observed clinical stroke severity and therefore could not directly compare PaSSV to observed stroke severity, but these comparisons have been previously reported. Reference Yu, Austin and Rashid9,Reference Joundi, King and Stang21 We also acknowledge that certain components of the composite PaSSV score, such as inter-hospital transfer to a hospital with higher level of care or use of mechanical ventilation, reflect hyperacute stroke care. Nevertheless, hyperacute care decisions in clinical practice are most often guided by the severity of the stroke and we showed that these can be used as proxy measurements for stroke severity.
Conclusion
Our findings suggest that PaSSV, a measure of stroke severity derived from administrative data, has geographical and temporal external validity across multiple Canadian provinces, making it a valuable tool for risk-adjustment in stroke outcomes research. We recommend the use of a clinical measure of observed stroke severity where possible, but in the absence of this information, the use of PaSSV through linkage with administrative health data is an alternative validated option.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/cjn.2024.36.
Author contributions
AYXY: conception, design, analysis, interpretation, drafting and critical revision, funding management, PCA: design, analysis, interpretation, critical revision, ALP: design, analysis, interpretation, critical revision, JF: design, analysis, interpretation, critical revision, MDH: design, interpretation, critical revision, NK: design, interpretation, critical revision, TSF: design, interpretation, critical revision, RAJ: design, interpretation, critical revision, SP: design, analysis, interpretation, critical revision, YZ: design, analysis, interpretation, critical revision, MKK: conception, design, analysis, interpretation, critical revision. All authors give final approval of the version to be published.
Funding statement
This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and the Ministry of Long‐Term Care. This document used data adapted from the Statistics Canada Postal Code Conversion File, which is based on data licensed from Canada Post Corporation and/or data adapted from the Ontario Ministry of Health Postal Code Conversion File, which contains data copied under license from Canada Post Corporation and Statistics Canada. Parts of this material are based on data and information compiled and provided by MOH and the Canadian Institute for Health Information. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. This work is made possible through the support of Health Data Research Network Canada and the SPOR-Canadian Data Platform. Access to Population Data BC data provided by the Data Steward(s) is subject to approval, but can be requested for research projects through the Data Steward(s) or their designated service providers. All inferences, opinions and conclusions drawn in this publication are those of the author(s), and do not reflect the opinions or policies of the Data Steward(s). Portions of the data used in this report were made available by Health Data Nova Scotia of Dalhousie University. Although this research analysis is based on data obtained from the Nova Scotia Department of Health and Wellness, the observations and opinions expressed are those of the authors and do not represent those of either Health Data Nova Scotia or the Department of Health and Wellness. AY holds a National New Investigator Award from the Heart & Stroke Foundation of Canada, MKK holds the Lillian Love Chair in Women’s Health at the University Health Network, Toronto, Canada. TSF holds the Sauder Family/Heart and Stroke Professorship of Stroke Research from the University of British Columbia.
Competing interests
MDH reports personal fees from Sun Pharma, grants from Boehringer-Ingelheim, Stryker Inc., NoNO Inc., Medtronic LLC, a patent Systems and Methods for Assisting in Decision-Making and Triaging for Acute Stroke Patients issued to US Patent office Number: 62/086,077 and owns stock in Pure Web Incorporated, is a director of the Canadian Federation of Neurological Sciences and the Canadian Stroke Consortium (not-for-profit groups), is a director of Circle NeuroVascular Inc., and has received grant support from Alberta Innovates Health Solutions, CIHR, Heart & Stroke Foundation of Canada, National Institutes of Neurological Disorders and Stroke (outside of current study), NK holds grant funding from CIHR, NSERC Discovery, NSERC Alliance and Mitacs; these include matching funds from industrial partners: Medtronic (NSERC Alliance), Lumiio (NSERC Alliance) and Synaptive (Mitacs); she has received consultation money from Roche, is part owner of DESTINE Health, TSF declares disclosures: Bayer Canada; advisory board, HLS Therapeutics, Roche Canada, AstraZeneca; Board: DESTINE Health. The other authors declare no disclosures.
