According to the latest national prevalence survey in France in 2017, surgical-site infections (SSIs) are the second most frequent hospital-acquired infection (HAI) in France [0.82% (95% confidence interval [CI], 0.72–0.95) of hospitalized patients] after urinary tract infections. 1 They are associated with prolonged hospitalization, unscheduled repeat surgery, additional management costs, and higher mortality. 2
Surveillance of SSI has been widely extended in many countries, including EU countries, first based on the US National Nosocomial Infections Surveillance System (NNIS) model developed in 1991 by the Centers for Disease Control and Prevention (CDC). Reference Culver, Horan and Gaynes3 In the European Union, many countries, such as France (ie, RAISIN), Germany (ie, KISS), UK (ie, UKSHA), have developed comparable surveillance systems that include SSI as a main target. 4 Initially based on manual data collection, these systems have evolved in recent years to automated or semiautomated processes using data extracted from hospital databases. The main reason for this change is the time saved in collecting data, and therefore the potential for reducing costs and improving effectiveness. Reference Chalfine, Cauet and Lin5–Reference Knepper, Young, Jenkins and Price7
For SSI, data collection is needed on patient case mix to allow benchmarking and comparison among surgical units. Indeed, comparing SSI rates among surgery wards requires statistical adjustment for risk factors. The different components of the NNIS risk index could be extracted if they are collected routinely, but their accessibility in the hospital database is not always warranted.
In France, a new national SSI surveillance system was launched in 2020 (Surveillance and Prevention of Infectious Risk in Surgery and Interventional Medicine or SPICMI). SPICMI is a locally implemented and semiautomated system for detection of SSI based on data extraction from the electronic hospital database. 8 The surgeon is required to validate SSI identified by the algorithmic system. Case mix is also estimated based on individual data that include NNIS risk index components and other operative risk factors that must be collected from the operating-room software. In addition, patient comorbidities could be extracted directly from the hospital discharge database (HDD). 8,9 In France, the HDD 8 is first designed for activity-based pricing. However, this system also represents a large source of clinical data that could be used for both epidemiological studies and surveillance in every hospital. The data are then coming from different data sources, which are not always related in the participating hospitals and, therefore, not easily extractable for routine surveillance.
We evaluated the performance of a comorbidity-based risk adjustment model using a panel of variables extracted from the HDD compared to other models that use variables from different data sources. Second, we studied the benefit of using these individual comorbidities as adjustment variables of SSI risk.
Methods and study population
Participation
During 2020–2021, each public or private hospital that participated in the program selected at least 1 target procedure to be monitored for the first 6 months of the year in patients aged ≥18 years. The procedures were defined by their Common Classification of Medical Procedures (CCAM) coding 8 : colectomy and appendectomy in digestive surgery; breast surgery and caesarean section in obstetrics and gynecology; laminectomy and lumbar disc herniation surgery in neurosurgery; coronary artery bypass grafting and valve replacements in cardiac surgery; hip and knee replacement and revision surgery in orthopedics; and ureteroscopy, prostatectomy, and transurethral resection of the prostate in urology.
Definition of SSI cases and detection procedure
The detection of suspected SSIs was based on a semiautomated protocol, using data extractions from the information system of each hospital (Fig. 1). 8 For all procedures except in urology, SSI was classified in 3 categories according to a combination of 2 criteria: (1) reoperation during the index stay or rehospitalization and (2) positive microbiological sample from 1 of the 3 levels of the surgical site (superficial, deep or organ space). These 2 events should have occurred within the 30 days following the initial surgery (90 in case of remaining implants) according to standard usual definition. 9 The SSI was defined as highly probable based on the presence of the 2 criteria. In this case, the detected SSI was submitted to the surgeon for review and confirmation. The SSI was defined as mildly probable if only 1 criterion was present. In this case, the infection control team confirmed whether the case met the CDC criteria 9 based on data in the medical record. Surgeon review was also recommended in such cases. An SSI diagnosis was rejected if either of the 2 criteria was present.
Figure 1. Surgical-site infection (SSI) detection protocol used for all surgical departments except urology.
The same process was used for urological surgery based on either positive urine culture and/or antibiotic prescription for >48 hours within the 30 days after surgery.
Data extraction was performed at least 1 month after SSI surveillance period, in accordance with the hospital IT department.
Risk factors
The NNIS risk index components (ASA anesthetic risk score, Altemeier contamination class, duration of the operation, Reference Culver, Horan and Gaynes3 and emergency versus elective) were recovered from the operative room database which is separated from the HDD. 10
These 6 comorbidities were selected according to literature review: arterial hypertension, Reference Martin, Kaye and Knott14,Reference Thelwall, Harrington, Sheridan and Lamagni19 diabetes, Reference Molla, Temesgen, Seyoum and Melkamu11–Reference Xu, Qu, Kanani, Guo, Ren and Chen15 cancer, Reference Miwa, Shirai and Yamamoto16,Reference Anatone, Danford, Jang, Smartt, Konigsberg and Tyler17 immunosuppression, Reference Tsantes, Papadopoulos and Lytras18 malnutrition, Reference Thelwall, Harrington, Sheridan and Lamagni19 and obesity. Reference Cai, Wang, Wang and Zhou12,Reference Thelwall, Harrington, Sheridan and Lamagni19 They were directly extracted from the HDD using defined International Classification of Diseases Tenth Revision (ICD-10) codes, as well as age, sex, length of preoperative stay, and outpatient surgery. Comorbidities were extracted at the same moment of data extraction for detection. Coded comorbidities during the initial stay for intervention, rehospitalization, or collected in the year before initial intervention were considered.
The hospital type (public, private, and nonprofit private) was transmitted at the time of surveillance registration.
Missing data management
For our analysis, we excluded all participating wards with >90% missing data for NHSN risk index components and comorbidities. After assuming “missing at random” (MAR) hypothesis, missing data were imputed in a multiple chain equation algorithm using the “mice” package in R Studio software. Reference van Buuren and Groothuis-Oudshoorn20 The imputation model included all variables collected by the program for which the proportion of missing data was <50% and the NHSN risk index was calculated individually after imputation of its components. Ten data sets were imputed given the fraction of missing data Reference Cottrell, Cot and Mary21 specified in the results. The estimators were combined according to Rubin rules. Reference Rubin22
Statistical analysis
After univariate analysis by calculating the odds ratio of SSI for each predictor, a multivariate logistic regression was conducted. The regression coefficients and then the ORs were estimated by the maximum likelihood method. Confidence intervals are provided at 95% (CI95). Four adjustment models were calculated using imputed data, and each model was adjusted for surgical specialty. Model 1 included only NNIS risk index. Model 2 included NNIS components and other perioperative variables (ie, emergency vs elective surgery, sex, outpatient surgery vs inpatient surgery, preoperative length of stay ≥2 days, and type of hospital) but not comorbidities. Model 3 included NNIS components, the other perioperative variables and comorbidities. Model 4 included all study-defined variables exclusively from the HDD. For models 2, 3, and 4, all predictors with a P < 0.2 in univariate analysis were included in the full baseline model. The variable with the largest median P was removed from the full model using a backward stepwise method until a threshold median P = .05 for all variables in the final model. Reference Zhao and Long23
The qualities of the models were calculated using the pooled receiver operating characteristic (ROC) area under the curve (AUC) for discriminatory accuracy and a cross-validation on 10 subsets per imputed data set was performed. Reference Rubin22 The calibration was studied by the Hosmer-Lemeshow test and the AUC of the models were compared with a Delong test. Reference DeLong, DeLong and Clarke-Pearson24 We used R Studio version 1.3.1093 software (R Foundation for Statistical Computing, Vienna, Austria) for these analyses.
Results
Population characteristics
Overall, 11,975 surgical procedures from 27 participating hospitals (11 public, 11 private, and 5 nonprofit private) were finally analyzed over the study period (Fig. 2). Compared to the imputed data set, only 4,782 surgical procedures (40%) would have been studied in complete-case analyses. Missing data were more frequent for the emergency status (36.0%) and NNIS risk index components: 21.5% for ASA score, 8.7% for Altemeier contamination class and 18% for intervention duration. Consequently, missing data were more frequent for global NNIS risk index (38.9%) than for comorbidities, ranging from 11.0% for obesity to 16.3% for arterial hypertension.
Figure 2. Flow diagram of analyzed surgical procedures.
SSI
Among these procedures, 294 SSIs were detected, for an overall incidence rate of 2.46% (95% CI, 2.18%–2.73%). Among these SSIs, 141 were diagnosed with reoperation combined with positive microbiological sample. Also, 17 SSIs were diagnosed with reoperation with clinical signs of infection but no positive microbiological sample. Furthermore, 103 SSIs were diagnosed with a positive microbiological sample and clinical signs of infections but without reoperation, and 8 SSIs were diagnosed with an antibiotic prescription and clinical infection signs concerning urology only. Data were not provided for 25 cases.
Univariate analysis
The following variables were positively associated with the risk of SSI: male sex; ASA score >2; procedure duration greater than the 75th percentile; NNIS risk index 1, 2, and 3 compared with score 0; preoperative length of stay ≥2 days; outpatient surgery; hospital type ‘private nonprofit’ or ‘public’ compared with ‘private’ hospitals; and cardiac, digestive, obstetrics and gynecology, and neurosurgical specialties compared with orthopedics (Table 1). All studied comorbidities (ie, cancer, diabetes, obesity, immunosuppression, hypertension, and malnutrition) were significantly associated with the risk of SSI. Age, Altemeier contamination class, and emergency surgery were not significantly associated. Compared with nonimputed data, similar results were observed using imputed data.
Table 1. Demographic Characteristics and Comorbid Conditions in the Cohort (N=11,975) of Surgical-Site Infections (Univariate Analysis Using Imputed Data)
Note. OR, odds ratio; CI, credibility interval; NNIS, National Nosocomial Infections Surveillance System; Ref, reference; SSI, surgical-site infection; ASA, American Society of Anesthesiologists. Percentages might not total 100 because of rounding.
* Indicates statistical significance.
Multivariate analysis and models comparison
In the multivariate analysis, the final models with predictors independently associated with SSI risk are presented in Table 2. The corrected AUC for reference model 1, with only NNIS risk index, was 0.598 (95% CI, 0.564–0.630). Model 2 finally included ASA score, duration of surgery greater than the 75th percentile, preoperative length of stay, outpatient surgery, hospital status, and surgical specialty with a corrected AUC of 0.641 (95% CI, 0.609–0.672). Model 3 finally included duration of surgery greater than the 75th percentile, preoperative length of stay, surgical specialty, as well as cancer, diabetes, obesity, and malnutrition for a corrected AUC of 0.675 (95% CI, 0.642–0.707). Model 4 finally included outpatient surgery, surgical specialty, as well as cancer, diabetes, obesity, and malnutrition, for a corrected AUC of 0.659 (95% CI, 0.625–0.692).
Table 2. Results of the 4 Selected Predictive Models Using Imputed Data
Note. NSHN, National Healthcare Safety Network; NNIS, National Nosocomial Infections Surveillance System; HDD, hospital discharge database; OR, odds ratio; CI, confidence interval; X, excluded during stepwise selection; Ref, reference; ASA, American Society of Anesthesiologists; AUC, area under the curve.
The areas under the ROC curves of models 2, 3, and 4 were significantly greater than those for reference model 1 (P = 0.01; P < .001; P = 0.002, respectively). Model 2 had a significantly lower pooled AUC than model 3 (P = .016). There was no significant difference in the pooled AUCs between model 4 and models 2 and 3 (P = .23 and P = .052, respectively). Model calibrations were acceptable (nonsignificant Hosmer-Lemeshow tests).
Discussion
Our study yielded 3 main findings. First, we demonstrated the potential to statistically predict SSI risk for defined patient comorbidities, which were directly extracted from the HDD. This potential is at least similar to the components of the historical NNIS risk index, and with models including NNIS risk index components and other perioperative individual and hospital data such as NHSN now use. Our findings are consistent with some other studies in different countries in which a routine hospital database was used for SSI surveillance. Grammatico-Guillon et al Reference Grammatico-Guillon, Miliani and Banaei-Bouchareb25 reported on the construction of a quality indicator used by the French National Authority for Health to evaluate the rates of SSI after total hip and knee replacement surgeries. Outlier hospitals were displayed using a standardized incidence ratio Reference Rioux, Grandbastien and Astagneau26 calculation based on individual comorbidities. Using a retrospective cohort study, Jackson et al Reference Jackson, Leekha and Magder27 showed that a model including individual comorbidity variables allowed a better adjustment of the risk of SSI in 28 US hospitals for colectomies, hysterectomies, and hip and knee replacements, compared with models without comorbidities. In Spain, Angel Garcia et al Reference Angel García, Martínez Nicolás, García Marín and Soria Aledo28 also developed 2 adjustment models for clean surgery and colectomy, based on a panel of variables suspected of being risk factors after review of the literature including comorbidities.
The second finding is that these relevant comorbidities could be used for adjustment in routine automated or semiautomated surveillance using electronically extracted hospital data, whenever available. The HDD is mainly used for activity-based pricing but is also used for epidemiological studies, Reference Grammatico-Guillon, Baron, Gaborit, Rusch and Astagneau29 including infections after hip or knee prosthetic surgery. Reference Grammatico-Guillon, Miliani and Banaei-Bouchareb25 Although this system is compelling, some limitations of SSI surveillance are related to the lack of the NNIS risk index components and some other surgical risk factors. These variables may exist in other databases (ie, patient medical record or operating room register), but they are not always computerized or easily extractable with a routinely accessible information technology (IT) system. Among hospitals participating in both NNIS components and comorbidity data collection, the proportion of missing data was lower for comorbidities (from the HDD) than the NNIS components and the other perioperative risk factors. To correct for that possible flaw, we used multiple imputation analysis. Although the missing at random (MAR) assumption for multiple imputation is not testable, all possible predictors collected in the protocol were included in the imputation model. An adjustment of easily extractable variables, such as comorbidities, could be an interesting alternative to explore when SSI risk factors are not available.
The third finding was the ability of a simple algorithm to detect SSI in a semiautomated surveillance. This algorithm combined 2 criteria based on electronically accessible data from the HDD (reoperation) and laboratory (microbiological samples) for all surgeries except for urology, for which it combined laboratory and antibiotic prescription data. For definitively confirmed SSI (with the exception of urology), a culture-positive microbiological sample was a much more frequent criterion than reintervention. The extraction procedure is probably a highly time saving for surgical staff because only a few patient files required validation by the surgeon or the infection control practitioner; however, more detailed studies are needed. Other pilot studies have been published recently using algorithm for automated or semiautomated HAI detection, combining mainly clinical and microbiological data in addition to antibiotic prescriptions. Reference Grammatico-Guillon, Baron, Gaborit, Rusch and Astagneau29,Reference Verberk, Aghdassi and Abbas30 However, performance indicators such as specificity, sensitivity, and predictive values of the algorithms have not been clearly evaluated at this time, and more surveillance data are needed. For instance, in our study, deep-incisional or organ-space SSIs are probably more easily detected than superficial SSI, because these more frequently require reoperation, although the algorithm also detects possible SSI based on positive microbiological samples with no reoperation. This factor could lead to underestimates of the global SSI rate, although deep-incisional or organ-space SSIs are the most important target for surveillance given their potential severity and human or financial extra cost. On the other hand, patients with no reoperation and negative microbiological samples could be considered wrongly without infection, thus being false-negative cases. To better estimate the performance of the algorithm, we should calculate the negative predictive value in patients who were considered without SSI based on no reoperation and negative microbiological results. This development is expected in the near future.
Our study had some potential limitations. Only a targeted selection of procedures has been followed in the surveillance. These procedures have been chosen according to the European protocol 4 and validated by a panel of experts belonging to the steering committee of the program. This will allow comparisons between countries and over time, but many common surgical procedures are not monitored. Due to the number of observations and low SSI rate, adjustment was made for surgical specialty rather than at the procedure level. We decided to include all targeted procedures for the specialty and, thus, to adjust at the specialty level due to statistical constraints. The length of preoperative stay was included in the model as a classical SSI risk factor, which could correspond to more severe underlying patient conditions as well as mismanagement in the hospital. The hospitals’ participation in the network was voluntary. Thus, optimal representativeness and exhaustivity of the surveillance could not be achieved. Compared to our former network, 10 the participation of surgery wards was relatively lower. It increased significantly between 2020 and 2021: 24,217 interventions with individual risk factors collection in 2017; 2,150 in 2020; and 9,825 in 2021. Moreover, to get a homogenous sample of hospitals participating in both NNIS components and comorbidity data collection, we excluded a consistent number of hospitals and interventions. This could be partly explained by the fact that the automated surveillance program was launched only recently, and many hospitals have not yet set up an effective information technology (IT) system for automated data extraction. Indeed, the program was launched during the COVID-19 pandemic, which tremendously disturbed surgery activity and the work of the infection control team.
Despite the improvements to be made, our HDD-based model including electronically extractable comorbidities should be considered as an interesting alternative to the NNIS risk index as well as other SSI risk-adjustment models including various perioperative variables, which simplify data collection. The success depends primarily on the capacity of each hospital to enhance the quality level of their own IT system.
Acknowledgments
We thank Bafodé Minte for his contribution to the surveillance information system, Karin Lebascle for her help to scientific references, and the other members of the steering committee of the SPICMI program: S. Aho, J. Auraix, P. Baillet, L. Banaei-Bouchareb, T. Bauer, A. Berger-Carbonne, G. Birgand, F. Bruyere, S. Chassy, N. Christou, G. Cisse, V. Cluzaud, K. Cohen, C. Daniau, C. Decoene, A. Florentin, L. Grammatico-Guillon, M. Huang, L. May-Michelangeli, S. Malavaud, B. Marcheix, V. Merle, N. Osinski, E. Piednoir, J. Tremoulet, C. Vaislic, V. Vallee, D. Verjat-Trannoy, V. Villefranque, C. Vons, and E. Vuillet.
Financial support
The national program for surveillance and prevention of infection in surgery and interventional medicine (SPICMI) is granted by Public Health France. The study also received a grant from the Regional Health Agency of Bretagne.
Competing interests
All authors report no conflicts of interest relevant to this article.
