Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-18T08:18:00.120Z Has data issue: false hasContentIssue false
  • English
  • Français

Inter-Agency Communication and Operations Capabilities during a Hospital Functional Exercise: Reliability and Validity of a Measurement Tool

Published online by Cambridge University Press:  28 June 2012

Elena Savoia ,
Paul D. Biddinger ,
Jon Burstein  and
Michael A. Stoto
Elena Savoia*
Affiliation:
Center for Public Health Preparedness, Harvard School of Public Health, Boston, Massachusetts, USA
Paul D. Biddinger
Affiliation:
Center for Public Health Preparedness, Harvard School of Public Health, Boston, Massachusetts, USA Department of Emergency Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA
Jon Burstein
Affiliation:
Center for Public Health Preparedness, Harvard School of Public Health, Boston, Massachusetts, USA Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
Michael A. Stoto
Affiliation:
Center for Public Health Preparedness, Harvard School of Public Health, Boston, Massachusetts, USA School of Nursing and Health Studies, Georgetown University, Washington, DC, USA
*
Center for Public Health PreparednessDivision of Public Health PracticeHarvard School of Public Health677 Huntington AvenueBoston, MassachusettsUSA E-mail: esavoia@hsph.harvard.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Introduction:

As proxies for actual emergencies, drills and exercises can raise awareness, stimulate improvements in planning and training, and provide an opportunity to examine how different components of the public health system would combine to respond to a challenge. Despite these benefits, there remains a substantial need for widely accepted and prospectively validated tools to evaluate agencies' and hospitals' performance during such events. Unfortunately, to date, few studies have focused on addressing this need. The purpose of this study was to assess the validity and reliability of a qualitative performance assessment tool designed to measure hospitals' communication and operational capabilities during a functional exercise.

Methods:

The study population included 154 hospital personnel representing nine hospitals that participated in a functional exercise in Massachusetts in June 2008. A 25-item questionnaire was developed to assess the following three hospital functional capabilities: (1) inter-agency communication; (2) communication with the public; and (3) disaster operations. Analyses were conducted to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement.

Results:

Twenty-two questions were retained in the final instrument, which demonstrated reliability with alpha coefficients of 0.83 or higher for all scales. A three-factor solution from the principal components analysis accounted for 57% of the total variance, and the factor structure was consistent with the original hypothesized domains. Inter-rater agreement between participants' self-reported scores and external evaluators' scores ranged from moderate to good.

Conclusions:

The resulting 22-item performance measurement tool reliably measured hospital capabilities in a functional exercise setting, with preliminary evidence of concurrent and criterion-related validity.

Keywords

communicationfunctional exerciseshospital preparednessmeasurementpublic healthvalidity
Type
Original Research
Information
Prehospital and Disaster Medicine , Volume 25 , Issue 1 , February 2010 , pp. 52 - 58
Copyright
Copyright © World Association for Disaster and Emergency Medicine 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Dausey, DJ, Buehler, JW, Lurie, N: Designing and conducting tabletop exercises to assess public health preparedness for manmade and naturally occurring biological threats. BMC Public Health 2007;7:92.CrossRefGoogle ScholarPubMed
2.Asch, SM, Stoto, MA, Mendes, M, Valdez, M, Gallagher, ME, Halverson, P, Lurie, N: A review of instruments assessing public health preparedness. Public Health Rep 2005;120(5):532–542.CrossRefGoogle ScholarPubMed
3.Kaji, AH, Lewis, RJ: Assessment of the reliability of the Johns Hopkins/Agency for Healthcare Research and Quality hospital disaster drill evaluation tool. Ann Emerg Med 2008;52(3):204–210.CrossRefGoogle Scholar
4.Savoia, E, Testa, MA, Biddinger, PD, et al. : Assessing public health capabilities during emergency preparedness tabletop exercises: Reliability and validity of a measurement tool. Public Health Rep 2009;124(1):138–148CrossRefGoogle ScholarPubMed
5.Biddinger, PD, Cadigan, R, Auerbacj, B, et al. : Using tabletop exercises to identify systems-level changes for improving preparedness. Public Health Rep 2008;123(1):96–101.CrossRefGoogle Scholar
6.Cronbach, LJ: Coefficient alpha and the internal structure of tests. Psykometrika 1951;16:297–333.CrossRefGoogle Scholar
7.Hair, JF, Anderson, RE, Tatham, RL, et al. : Multivariate Analysis: With Readings. 4th ed.Englewood Cliffs, New Jersey: Prentice-Hall, 1995.Google Scholar
8.Duteman, GH: Principal Components Analysis. Newbury Park, California: Sage Publications, 1989.CrossRefGoogle Scholar
9.SPSS for Windows, Release 16.0.1 2007. Chicago: SPSS Inc.Google Scholar
10.Agency for Healthcare Research and Quality: Optimizing Surge Capacity: Hospital Assessment and Planning. Bioterrorism and Health System Preparedness, Issue Brief No.3. AHRQ Publication No. 04-P008. Available at http://www.ahrq.gov/news/ulp/btbriefs/btbrief3.htm.Google Scholar