Hostname: page-component-5c6d5d7d68-sv6ng Total loading time: 0 Render date: 2024-08-15T00:24:43.850Z Has data issue: false hasContentIssue false
  • English
  • Français

Severity of Disease Estimation and Risk-Adjustment for Comparison of Outcomes in Mechanically Ventilated Patients Using Electronic Routine Care Data

Published online by Cambridge University Press:  17 April 2015

Maaike S. M. van Mourik ,
Karel G. M. Moons ,
Michael V. Murphy ,
Marc J. M. Bonten ,
Michael Klompas  and
MICU Registry
Maaike S. M. van Mourik*
Affiliation:
Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts Department of Medical Microbiology, University Medical Center Utrecht, Utrecht, The Netherlands
Karel G. M. Moons
Affiliation:
Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
Michael V. Murphy
Affiliation:
Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts
Marc J. M. Bonten
Affiliation:
Department of Medical Microbiology, University Medical Center Utrecht, Utrecht, The Netherlands Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
Michael Klompas
Affiliation:
Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts Department of Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
MICU Registry
Affiliation:
Department of Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
*
Address correspondence to Maaike S. M. van Mourik, Department of Medical Microbiology, HP G04.614, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands (M.S.M.vanMourik-2@umcutrecht.nl).
Get access Rights & Permissions [Opens in a new window]

Abstract

BACKGROUND

Valid comparison between hospitals for benchmarking or pay-for-performance incentives requires accurate correction for underlying disease severity (case-mix). However, existing models are either very simplistic or require extensive manual data collection.

OBJECTIVE

To develop a disease severity prediction model based solely on data routinely available in electronic health records for risk-adjustment in mechanically ventilated patients.

DESIGN

Retrospective cohort study.

PARTICIPANTS

Mechanically ventilated patients from a single tertiary medical center (2006–2012).

METHODS

Predictors were extracted from electronic data repositories (demographic characteristics, laboratory tests, medications, microbiology results, procedure codes, and comorbidities) and assessed for feasibility and generalizability of data collection. Models for in-hospital mortality of increasing complexity were built using logistic regression. Estimated disease severity from these models was linked to rates of ventilator-associated events.

RESULTS

A total of 20,028 patients were initiated on mechanical ventilation, of whom 3,027 deceased in hospital. For models of incremental complexity, area under the receiver operating characteristic curve ranged from 0.83 to 0.88. A simple model including demographic characteristics, type of intensive care unit, time to intubation, blood culture sampling, 8 common laboratory tests, and surgical status achieved an area under the receiver operating characteristic curve of 0.87 (95% CI, 0.86–0.88) with adequate calibration. The estimated disease severity was associated with occurrence of ventilator-associated events.

CONCLUSIONS

Accurate estimation of disease severity in ventilated patients using electronic, routine care data was feasible using simple models. These estimates may be useful for risk-adjustment in ventilated patients. Additional research is necessary to validate and refine these models.

Infect. Control Hosp. Epidemiol. 2015;36(7):807–815

Type
Original Articles
Information
Infection Control & Hospital Epidemiology , Volume 36 , Supplement 7 , July 2015 , pp. 807 - 815
Copyright
© 2015 by The Society for Healthcare Epidemiology of America. All rights reserved 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Shared last author.

Presented in part: IDWeek 2013; San Francisco, California; October 4, 2013 (Abstract p1076).

References

1. Panzer, RJ, Gitomer, RS, Greene, WH, Webster, PR, Landry, KR, Riccobono, CA. Increasing demands for quality measurement. JAMA 2013;310:1971–1980.Google Scholar
2. Rosenthal, MB. Nonpayment for performance? Medicare’s new reimbursement rule. N Engl J Med 2007;357:1573–1575.Google Scholar
3. Centers for Medicare and Medicaid Services(CMS). Hospital-acquired conditions. CMS website. Available at: http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/HospitalAcqCond/Hospital-Acquired_Conditions.html. Updated September 20, 2012. Accessed November 25, 2013.Google Scholar
4. Association of Professionals in Infection Control and Epidemiology (APIC). HAI reporting laws and regulations. APIC website. Available at: http://www.apic.org/Resource_/TinyMceFileManager/Advocacy-PDFs/Advocacy_Updates/Static_map_-_HAI_revised_7-6-11.gif. Updated July 6, 2011. Accessed February 13, 2012.Google Scholar
5. Haustein, T, Gastmeier, P, Holmes, A, et al. Use of benchmarking and public reporting for infection control in four high-income countries. Lancet Infect Dis 2011;11:471–481.Google Scholar
6. Mu, Y, Edwards, JR, Horan, TC, Berrios-Torres, SI, Fridkin, SK. Improving risk-adjusted measures of surgical site infection for the National Healthcare Safety Network. Infect Control Hosp Epidemiol 2011;32:970–986.CrossRefGoogle ScholarPubMed
7. Harris, AD, McGregor, JC. The importance of case-mix adjustment for infection rates and the need for more research. Infect Control Hosp Epidemiol 2008;29:693–694.Google Scholar
8. Breslow, MJ, Badawi, O. Severity scoring in the critically ill, part 1: interpretation and accuracy of outcome prediction scoring systems. Chest 2012;141:245–252.Google Scholar
9. van Dishoeck, AM, Koek, MB, Steyerberg, EW, van Benthem, BH, Vos, MC, Lingsma, HF. Use of surgical-site infection rates to rank hospital performance across several types of surgery. Br J Surg 2013;100:628–636.Google Scholar
10. Kritsotakis, EI, Dimitriadis, I, Roumbelaki, M, et al. Case-mix adjustment approach to benchmarking prevalence rates of nosocomial infection in hospitals in Cyprus and Greece. Infect Control Hosp Epidemiol 2008;29:685–692.Google Scholar
11. Magill, SS, Klompas, M, Balk, R, et al. Developing a new, national approach to surveillance for ventilator-associated events. Crit Care Med 2013;41:2467–2475.Google Scholar
12. Centers for Disease Control and Prevention (CDC). Surveillance for ventilator-associated events. CDC website. Available at: http://www.cdc.gov/nhsn/acute-care-hospital/vae/. Updated November 25, 2013. Accessed November 26, 2013.Google Scholar
13. Dudeck, MA, Weiner, LM, len-Bridson, K, et al. National Healthcare Safety Network (NHSN) report, data summary for 2012, device-associated module. Am J Infect Control 2013;41:1148–1166.Google Scholar
14. Moreno, RP, Metnitz, PG, Almeida, E, et al. SAPS 3: from evaluation of the patient to evaluation of the intensive care unit, part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 2005;31:1345–1355.CrossRefGoogle ScholarPubMed
15. Zimmerman, JE, Kramer, AA, McNair, DS, Malila, FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 2006;34:1297–1310.Google Scholar
16. Higgins, TL, Teres, D, Copes, WS, Nathanson, BH, Stark, M, Kramer, AA. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Crit Care Med 2007;35:827–835.Google Scholar
17. Jha, AK, DesRoches, CM, Campbell, EG, et al. Use of electronic health records in US hospitals. N Engl J Med 2009;360:1628–1638.Google Scholar
18. Kuzniewicz, MW, Vasilevskis, EE, Lane, R, et al. Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest 2008;133:1319–1327.Google Scholar
19. Johnson, AE, Kramer, AA, Clifford, GD. A new severity of illness scale using a subset of Acute Physiology, Age, and Chronic Health Evaluation data elements shows comparable predictive accuracy. Crit Care Med 2013;41:1711–1718.Google Scholar
20. Render, ML, Deddens, J, Freyberg, R, et al. Veterans Affairs intensive care unit risk adjustment model: validation, updating, recalibration. Crit Care Med 2008;36:1031–1042.CrossRefGoogle ScholarPubMed
21. Liu, V, Turk, BJ, Ragins, AI, Kipnis, P, Escobar, GJ. An electronic Simplified Acute Physiology Score-based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med 2013;41:41–48.Google Scholar
22. Graham, PL, Cook, DA. Prediction of risk of death using 30-day outcome: a practical end point for quality auditing in intensive care. Chest 2004;125:1458–1466.Google Scholar
23. Drye, EE, Normand, SL, Wang, Y, et al. Comparison of hospital risk-standardized mortality rates calculated by using in-hospital and 30-day models: an observational study with implications for hospital profiling. Ann Intern Med 2012;156:19–26.CrossRefGoogle ScholarPubMed
24. Pouw, ME, Peelen, LM, Moons, KG, Kalkman, CJ, Lingsma, HF. Including post-discharge mortality in calculation of hospital standardised mortality ratios: retrospective analysis of hospital episode statistics. BMJ 2013;347:f5913.Google Scholar
25. Partners Healthcare. Research Patient Data Registry (RPDR). Partners Healthcare website. Available at: http://rc.partners.org/rpdr. Updated November 1, 2013. Accessed November 11, 2013.Google Scholar
26. Higgins, TL, Kramer, AA, Nathanson, BH, Copes, W, Stark, M, Teres, D. Prospective validation of the intensive care unit admission Mortality Probability Admission Model (MPM0-III). Crit Care Med 2009;37:1619–1623.Google Scholar
27. Healthcare Cost and Utilization Project. Comorbidity Software, version 3.7. Available at: http://www.hcup-us.ahrq.gov/toolssoftware/comorbidity/comorbidity.jsp. Updated 2012. Accessed November 1, 2013.Google Scholar
28. Elixhauser, A, Steiner, C, Harris, DR, Coffey, RM. Comorbidity measures for use with administrative data. Med Care 1998;36:8–27.Google Scholar
29. van Buuren, S, Groothuis-Oudshoorn, K. MICE: Multivariable Imputation by Chained Equations in R. J Stat Softw 2011;45:1–67.Google Scholar
30. Harrell, FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York: Springer-Verlag, 2001.Google Scholar
31. Steyerberg, EW, Harrell, FE, Borsboom, GJ, Eijkemans, MJ, Vergouwe, Y, Habbema, JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–781.Google Scholar
32. Rubin, DB. Multiple Imputation for Nonresponse in Surveys. Hoboken: J. Wiley & Sons, 1987.Google Scholar
33. Centers for Disease Control and Prevention (CDC) Division of Healthcare Quality Promotion. The National Healthcare Safety Network Device-Associated Module: Ventilator-Associated Event Protocol. Atlanta: CDC, 2013.Google Scholar
34. Dolinay, T, Kim, YS, Howrylak, J, et al. Inflammasome-regulated cytokines are critical mediators of acute lung injury. Am J Respir Crit Care Med 2012;185:1225–1234.Google Scholar
35. Polderman, KH, Thijs, LG, Girbes, AR. Interobserver variability in the use of APACHE II scores. Lancet 1999;353:380.Google Scholar
36. Bosman, RJ, Oudemane van Straaten, HM, Zandstra, DF. The use of intensive care information systems alters outcome prediction. Intensive Care Med 1998;24:953–958.Google Scholar
37. Afessa, B, Keegan, MT, Gajic, O, Hubmayr, RD, Peters, SG. The influence of missing components of the Acute Physiology Score of APACHE III on the measurement of ICU performance. Intensive Care Med 2005;31:1537–1543.Google Scholar
38. Kramer, AA, Zimmerman, JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med 2007;35:2052–2056.Google Scholar
39. Keegan, MT, Gajic, O, Afessa, B. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0-III and influence of resuscitation status on model performance. Chest 2012;142:851–858.Google Scholar
Supplementary material: PDF

van Mourik supplementary material

van Mourik supplementary material 1

Download van Mourik supplementary material(PDF)
PDF 123.4 KB