Abstract
Rationale: Checklists may reduce errors of omission for critically ill patients.
Objectives: To determine whether prompting to use a checklist improves process of care and clinical outcomes.
Methods: We conducted a cohort study in the medical intensive care unit (MICU) of a tertiary care university hospital. Patients admitted to either of two independent MICU teams were included. Intervention team physicians were prompted to address six parameters from a daily rounding checklist if overlooked during morning work rounds. The second team (control) used the identical checklist without prompting.
Measurements and Main Results: One hundred and forty prompted group patients were compared with 125 control and 1,283 preintervention patients. Compared with control, prompting increased median ventilator-free duration, decreased empirical antibiotic and central venous catheter duration, and increased rates of deep vein thrombosis and stress ulcer prophylaxis. Prompted group patients had lower risk-adjusted ICU mortality compared with the control group (odds ratio, 0.36; 95% confidence interval, 0.13–0.96; P = 0.041) and lower hospital mortality compared with the control group (10.0 vs. 20.8%; P = 0.014), which remained significant after risk adjustment (odds ratio, 0.34; 95% confidence interval, 0.15–0.76; P = 0.008). Observed-to-predicted ICU length of stay was lower in the prompted group compared with control (0.59 vs. 0.87; P = 0.02). Checklist availability alone did not improve mortality or length of stay compared with preintervention patients.
Conclusions: In this single-site, preliminary study, checklist-based prompting improved multiple processes of care, and may have improved mortality and length of stay, compared with a stand-alone checklist. The manner in which checklists are implemented is of great consequence in the care of critically ill patients.
Checklists prevent medical errors and improve outcomes for critically ill patients.
In this study, prompting to use a checklist improved process of care measures and may have reduced severity-adjusted mortality and length of stay. Prompting may provide a novel and more effective approach to checklist implementation.
The complexity of the ICU environment places critically ill patients at increased risk of medical error, particularly errors of omission (1–3). Checklists have become synonymous with attempts to simplify care and reduce errors of omission (4). ICU checklists increase compliance with clinical practice guidelines (5) and lead to earlier extubation (6) and transfer from ICU to telemetry units (7), and decreased rates of catheter-related bloodstream infection (8, 9).
Despite these encouraging results, the benefit of widespread checklist use remains unclear. Prior checklist research has generally addressed a single topic (e.g., catheter-related bloodstream infections). No rigorously controlled studies of the benefits of checklists exist: published studies have used mostly the before/after model (7, 9, 10), which can be compromised by coincident interventions (8).
Implementation issues raise the greatest concern regarding effective use of checklists. Successful quality improvement interventions often provide a mechanism to encourage or enforce the intervention's adoption (11). However, administrative mandates for checklists often do not consider the importance of implementation strategy in achieving the culture change essential for their success (12).
Despite general appreciation of the benefits of checklists by clinicians, results of checklist introduction to many critical care services have been disappointing. We therefore sought to determine whether regular verbal prompting to address multiple quality parameters on a checklist leads to superior process of care and improved clinical outcomes than the unprompted use of an identical checklist.
Some of the results of this study have been previously reported in the form of an abstract (13).
The study was conducted in the medical intensive care unit (MICU) at Northwestern Memorial Hospital (NMH, Chicago, IL), a tertiary care urban university-affiliated hospital. The MICU is a closed-unit staffed by two separate teams, each with an independent patient census. The two teams admit patients on alternating days. Each team consists of one pulmonary/critical care attending physician, one fellow, one pharmacist, and several residents and interns. Attendings and fellows have weekday rotations of 1–4 weeks, often with different weekend coverage (see the online supplement).
Before the study we designed and instituted an MICU daily rounding checklist as a quality improvement tool (Figure 1). Faculty, fellows, pharmacists, and nurses were trained to complete the checklist daily (see the online supplement).
Figure 1. Daily rounding checklist. The medical intensive care unit (MICU) checklist was introduced in March 2009. Faculty, fellows, pharmacists, and nurses were trained in its use. Multiple parameters are included along the left side. Nurses are responsible for filling out the yellow section, pharmacists the orange section, and physicians the green section. The checklist was designed to be able to follow parameters over time, with each column of boxes representing each ICU day. The attending or fellow is required to initial the checklist on each day. ARDS = acute respiratory distress syndrome; DVT = deep vein thrombosis; GI = gastrointestinal; HOB = head of bed; SCD = sequential compression device; VAP = ventilator-associated pneumonia.
[More] [Minimize]We conducted a prospective concurrently controlled cohort study. The study was approved by the Institutional Review Board with a waiver of consent. All patients admitted to the MICU service on or after June 25, 2009 and discharged on or before September 15, 2009 were eligible for inclusion. This time period was chosen to minimize the crossover of attendings and fellows between MICU teams. Exclusion criteria included the following: (1) patients physically located in a different ICU for more than the first 72 hours of their ICU stay, (2) patients transferred from a different ICU service, and (3) patients transferred to another ICU service within 12 hours of MICU admission. Only the first MICU admission was included for patients admitted more than once without intervening hospital discharge (14, 15). All included patients were monitored through hospital discharge. We also performed a retrospective analysis of the 1,283 MICU patients admitted from July 1, 2008 to June 25, 2009 (preintervention group) who otherwise met the above inclusion criteria.
A non–care-providing resident physician (the prompter) initiated discussion with one of the MICU teams (prompted team), using scripted questions if any of the following six parameters under investigation were overlooked on daily work rounds: mechanical ventilation weaning, empirical antibiotics, central venous catheters (CVCs), Foley urinary catheters, and deep vein thrombosis (DVT) and stress ulcer prophylaxis. A verbal prompting script addressing the selected checklist topics was developed before study initiation (see Figure E1 in the online supplement). For example, if the rounding team failed to discuss the presence or management of a CVC, the prompter would ask, “The CVC has been in place for [X] days. Do you want to continue it”? The prompters had no patient care responsibilities; there was no contact between the prompters and any patient. Verbal prompting was directed at the attending and fellow.
Any patient admitted to the prompted team was included regardless of whether the prompter was present during their ICU stay (e.g., patients admitted and discharged over the weekend). Prompting began during the first rounds after a patient's MICU admission, occurred after a care-providing resident's presentation but before the MICU team entered the patient's room, and continued daily (whenever the prompter was present) until MICU discharge.
The unprompted MICU team, with availability of the identical checklist, served as a control (see the online supplement). Both teams were unaware of the goals of the study.
For each patient, the prompter recorded the number of days he/she was present, and for each checklist parameter the number of days on which prompting occurred and whether prompting led to a change in medical management. We retrieved checklists from both groups at ICU discharge. Primary outcomes included differences between the prompted and control groups related to the process of care parameters under investigation: ventilator-free days (16); duration of empirical antibiotics (defined as antibiotics administered without culture-documented infection), total antibiotics, and CVCs (excluding hemodialysis catheters and peripherally inserted central catheters); Foley urinary catheter duration in eligible patients; pharmacological deep vein thrombosis (DVT) prophylaxis per eligible days; and stress ulcer prophylaxis per eligible days. We collected primary outcomes from the electronic medical record, except for Foley catheter duration, which was obtained from patients’ checklists.
Secondary outcomes included ICU and hospital mortality, ICU length of stay (LOS), and ventilator-associated pneumonia (see the online supplement). Data necessary to calculate Acute Physiology and Chronic Health Evaluation (APACHE) IV–predicted mortality and ICU LOS were collected retrospectively (14, 15). We also collected hospital mortality and ICU LOS for preintervention group patients. We obtained the results of bronchoalveolar lavage fluid cultures for mechanically ventilated patients.
Descriptive data are summarized as mean (standard deviation, SD), median (interquartile range, IQR), or number (%). We used a χ2 test to compare categorical variables, and Student's t test and Wilcoxon or Kruskal-Wallis rank-sum tests to compare continuous variables, as appropriate. Patients were excluded from a specific outcome analysis if they were not exposed to the parameter in question (e.g., patients who did not receive antibiotics were excluded from the antibiotic analyses).
We constructed logistic regression models to adjust ICU and hospital mortality for APACHE IV–predicted hospital mortality, age, sex, and night or weekend ICU admission status. Differences are expressed as the odds ratio (OR) for death with 95% confidence intervals (CIs). We calculated standardized mortality ratios (SMRs, observed/APACHE IV–predicted mortality), reported with 95% CIs. We analyzed mean ICU LOS by Monte Carlo hypothesis testing (100,000 synthetic samples), to remove the effects of random variation in nonnormally distributed variables. Observed/APACHE IV–predicted LOS ratios were calculated. We constructed Kaplan-Meier curves and Cox proportional hazards models of the proportion remaining in the ICU. All tests are two-tailed, and a P value less than 0.05 was considered significant. Analyses were performed with SAS (version 9.2; SAS Institute, Cary, NC), except for the Monte Carlo simulations (17).
One hundred and forty prompted, 125 control, and 1,283 preintervention group patients were enrolled. We found no significant differences in baseline characteristics between the prompted and control groups (Table 1).
| Prompted | Control | ||
| Characteristic | (n = 140) | (n = 125) | P Value |
| Age (yr), mean (SD) | 58.5 (17.8) | 57.3 (17.8) | 0.60 |
| Sex (male), no. (%) | 69 (49.3) | 51 (40.8) | 0.17 |
| Race, no. (%) | |||
| White | 71 (50.7) | 69 (55.2) | |
| African American | 47 (33.6) | 42 (33.6) | 0.54 |
| Hispanic/other | 22 (15.7) | 14 (11.2) | |
| Location before MICU, no. (%) | |||
| Emergency department | 79 (56.4) | 76 (60.8) | |
| General medical ward | 54 (38.6) | 44 (35.2) | 0.75 |
| Outside hospital transfer | 7 (5.0) | 5 (4.0) | |
| Diagnosis, no. (%)* | |||
| Sepsis | 32 (22.9) | 32 (25.6) | |
| Pneumonia | 20 (14.3) | 11 (8.8) | |
| Obstructive airway disease | 8 (5.7) | 15 (12.0) | |
| Other respiratory† | 13 (9.3) | 14 (11.2) | |
| GI hemorrhage | 17 (12.1) | 12 (9.6) | 0.39 |
| Metabolic | 7 (5.0) | 11 (8.8) | |
| Neurological | 10 (7.1) | 6 (4.8) | |
| Drug intoxication/withdrawal | 6 (4.3) | 6 (4.8) | |
| Other | 27 (19.3) | 18 (14.4) | |
| Sepsis subdiagnosis, no. (%)* | |||
| Pulmonary | 6 (18.8) | 7 (21.9) | |
| GI | 7 (21.9) | 8 (25.0) | |
| Urinary | 9 (28.1) | 5 (15.6) | 0.82 |
| Soft tissue | 2 (6.3) | 3 (9.4) | |
| Unknown | 8 (25.0) | 9 (28.1) | |
| Mechanical ventilation, no. (%) | 36 (28.8) | 41 (29.3) | 0.93 |
| Hospital discharge disposition, no. (%) | 0.45 | ||
| Home | 96 (76.2) | 71 (71.7) | |
| Long-term acute care or skilled nursing facility | 30 (23.8) | 28 (28.3) |
A prompter was present on 67.9% of prompted group daily rounds during the 82-day intervention period. Prompting was required on 64.7% of patient-days. The amount of prompting for each care practice is provided in Table 2.
| Variable | Prompting Patient-days/Rounding Patient-days (%)* |
| Overall† | 334/516 (64.7) |
| Foley catheter‡ | 98/238 (41.2) |
| Empirical antibiotics | 103/284 (36.3) |
| Central venous catheter | 53/206 (25.7) |
| Mechanical ventilation | 35/249 (14.1) |
| DVT prophylaxis§ | 5/336 (1.5) |
| Stress ulcer prophylaxis§ | 5/497 (1.0) |
Compared with the control group, the prompted group had increased median (IQR) ventilator-free days (22 [14–26] vs. 16 [0–21.5] d; P = 0.028), shorter empirical antibiotic (2 [1–3] vs. 3 [2–7] d; P = 0.012) and central venous catheter (3 [2–5] vs. 5 [2–8] d; P = 0.007) duration, and increased mean (SD) proportion of pharmacological DVT (0.96 [0.18] vs. 0.76 [0.35]; P < 0.001) and stress ulcer prophylaxis (0.93 [0.22] vs. 0.83 [0.31]; P < 0.001). The mean (SD) proportion of days of antibiotic use that were empirical also decreased with prompting (0.77 [0.32] vs. 0.91 [0.29]; P < 0.001). Foley catheter duration was not significantly reduced with prompting (3 [2–7] vs. 4 [2–11] d; P = 0.29) (Table 3).
| Variable | Prompted (n = 140) | Control (n = 125) | P Value |
| Ventilator-free days | 22 (14–26) | 16 (0–21.5) | 0.028 |
| Antibiotics | |||
| Empirical antibiotics, d | 2 (1–3) | 3 (2–7) | 0.012 |
| Total antibiotics, d | 3 (2–5) | 3 (2–7) | 0.41 |
| Proportion empirical antibiotics* | 0.77 (0.32) | 0.91 (0.29) | <0.001 |
| Central venous catheter, d | 3 (2–5) | 5 (2–8) | 0.007 |
| Foley catheter, d | 3 (2–7) | 4 (2–11) | 0.29 |
| DVT prophylaxis/eligible | 1 (1, 1) | 1 (0.6–1) | <0.001 |
| Proportion DVT prophylaxis* | 0.96 (0.18) | 0.76 (0.35) | |
| Stress ulcer prophylaxis/eligible | 1 (1, 1) | 1 (0.75–1) | <0.001 |
| Proportion stress ulcer prophylaxis* | 0.93 (0.22) | 0.83 (0.31) |
We conducted post-hoc analyses of two intermediate outcomes. In the prompted group, 14 of 51 (27.5%) bronchoalveolar lavage cultures were positive (bacterial colony-forming units ≥ 104/ml) (18–21), compared with 17 of 40 (42.5%) in the control group (P = 0.13). No catheter-related bloodstream infections were documented during the study period in any patient.
There was no difference in hospital mortality between the control and preintervention groups. Both ICU and hospital mortality were lower in the prompted group compared with the control group, whereas APACHE IV–predicted mortality was similar (Table 4 and Figure 2A).
| Variable | Prompted (n = 140) | Control (n = 125) | P Value |
| Mortality | |||
| APACHE IV–predicted hospital mortality, no. (%) | 31.1 (22.2) | 27.2 (21.7) | 0.86 |
| ICU mortality, no. (%) | 9 (6.4) | 17 (13.6) | 0.050 |
| Adjusted odds ratio (95% CI) | 0.36 (0.13–0.96) | Reference | 0.041 |
| Hospital mortality, no. (%) | 14 (10.0) | 26 (20.8) | 0.014 |
| Adjusted odds ratio (95% CI) | 0.34 (0.15–0.76) | Reference | 0.008 |
| SMR (95% CI)* | 0.45 (0.25–0.76) | 0.96 (0.63–1.4) | |
| Length of Stay | |||
| APACHE IV–predicted LOS, d† | 5.8 (2.0) | 5.5 (2.0) | 0.17 |
| ICU LOS, d | |||
| Mean (SD) | 3.5 (4.3) | 4.9 (7.0) | 0.07‡ |
| Median (IQR) | 1.9 (1.0–3.7) | 1.9 (1.2–4.9) | 0.45 |
| Observed/predicted LOS ratio (95% CI)* | 0.59 (0.48–0.69) | 0.87 (0.68–1.06) | 0.02‡ |
Figure 2. Hospital mortality. (A) Observed (light blue and red columns) and Acute Physiology and Chronic Health Evaluation (APACHE) IV–predicted (dark blue and red columns) hospital mortality in the prompted and control groups, and observed preintervention hospital mortality (gray column). Mortality rates (%) and standardized mortality ratios (SMRs) are shown. (B) Observed hospital mortality in the control group (blue columns) and prompted group (red columns) according to patient quartile of predicted risk. Prompted and control patients were pooled and divided into equal quartiles of predicted risk. The range of predicted mortality for patients in each quartile is shown underneath each quartile number.
[More] [Minimize]In multivariate analyses, severity-adjusted ICU mortality as well as severity-adjusted hospital mortality were reduced in the prompted group (ICU mortality: OR, 0.36; 95% CI, 0.13–0.96; P = 0.041; hospital mortality: OR, 0.34; 95% CI, 0.15–0.76; P = 0.008). The prompted group SMR was 0.45 (95% CI, 0.25–0.76); the SMR for the control group was 0.96 (95% CI, 0.63–1.40). The hospital mortality difference was observed in the second through fourth quartiles of predicted mortality (Figure 2B). Age, sex, night or weekend ICU admission status, and attending experience (Table E1) were not predictors of death.
We found significant differences in hospital mortality between the prompted group and both a seasonal preintervention subgroup (1 yr before the intervention and before checklist institution, 24.0%), and the remainder of the preintervention group (20.2%) (P = 0.008); there was no difference between the checklist-only control group and either preintervention group.
Post-hoc review revealed lower ICU and hospital mortality rates in the prompted group for patients with sepsis (especially gastrointestinal sepsis), pneumonia, and the category Other Respiratory, with a similar distribution of diagnoses (Table 5). Patients who died in the hospital in the prompted group had higher APACHE IV–predicted mortality (53.7%; SD, 26.5%) than those who died in the control group (40.5%; SD, 24.8%); this was not statistically significant (P = 0.16).
| Prompting Group | Control Group | |||
| Deaths/Total | Mortality Rate | Deaths/Total | Mortality Rate | |
| Hospital Mortality | ||||
| Sepsis | 6/32 | 0.19 | 11/32 | 0.34 |
| GI | 1/7 | 0.14 | 6/8 | 0.75 |
| Pulmonary | 2/6 | 0.33 | 2/7 | 0.29 |
| Urinary | 0/9 | 0 | 1/5 | 0.20 |
| Soft tissue | 0/2 | 0 | 0/3 | 0 |
| Unknown | 3/8 | 0.38 | 2/9 | 0.22 |
| Pneumonia | 4/20 | 0.20 | 5/11 | 0.46 |
| Obstructive lung disease | 1/8 | 0.13 | 0/15 | 0 |
| Other respiratory* | 0/13 | 0 | 6/14 | 0.43 |
| All other diagnoses | 3/67 | 0.045 | 4/53 | 0.075 |
| ICU Mortality | ||||
| Sepsis | 3/32 | 0.094 | 7/32 | 0.22 |
| GI | 1/7 | 0.14 | 4/8 | 0.50 |
| Pulmonary | 1/6 | 0.17 | 1/7 | 0.14 |
| Urinary | 0/9 | 0 | 0/5 | 0 |
| Soft tissue | 0/2 | 0 | 0/3 | 0 |
| Unknown | 1/8 | 0.13 | 2/9 | 0.22 |
| Pneumonia | 4/20 | 0.20 | 4/11 | 0.36 |
| Obstructive lung disease | 0/8 | 0 | 0/15 | 0 |
| Other respiratory* | 0/13 | 0 | 5/14 | 0.36 |
| All other diagnoses | 2/67 | 0.030 | 1/53 | 0.019 |
APACHE IV–predicted ICU length of stay was similar in the prompted and control groups. Mean ICU LOS in the prompted group was shorter compared with control (Table 4). There was no difference in ICU LOS between the checklist-only control and preintervention (4.3 d) groups. Observed/predicted LOS ratios (95% CI) for the prompted and control groups were 0.59 (0.48–0.69) and 0.87 (0.68–1.06), respectively (P = 0.02). The difference between prompted and control group ICU LOS was pronounced after 4 ICU days (Figure 3).
Figure 3. Kaplan-Meier analysis of intensive care unit (ICU) length of stay. Among ICU survivors, the proportion of patients remaining in the ICU is shown according to their ICU length of stay; tick marks represent ICU deaths. The Acute Physiology and Chronic Health Evaluation (APACHE) IV–adjusted hazard ratio (95% confidence interval) determined by Cox proportional hazards model was 0.67 (0.52–0.88; P = 0.003).
[More] [Minimize]In this single-site, preliminary study, we demonstrate that prompting physicians on one MICU team to discuss care practice parameters on a checklist improves multiple processes of care compared with a similar team that received checklists but no prompting. Prompting was associated with shortened duration of mechanical ventilation, empirical antibiotic use and central venous catheter use, and increased use of DVT and stress ulcer prophylaxis. Using the checklist without ongoing prompting did not result in improvement compared with baseline measurement of outcomes. The prompted group, however, had lower severity-adjusted mortality and length of stay.
Our study contrasts active checklist use versus passive implementation, the latter representing a quality improvement paradigm in which a tool or process is initiated through regulatory or administrative mandate (22). Indeed, a comparison of checklist-only control group outcomes with the preintervention cohort suggests that the checklist itself had little effect. Instead, our study suggests that effective quality improvement requires a robust implementation and adherence strategy. Constant attentiveness to the care practices under investigation, driven by prompting, achieved improvements in processes of care. We do not believe the checklist is superfluous: prompting by memory would be prone to similar errors of omission that it attempts to prevent.
Overlooked in the enthusiasm for checklists is the fact that the most prominent examples enforced changes in behavior. Bedside nurses were empowered to stop CVC insertion with overt support from hospital administrators in the Keystone Project (9). Surgeons were not allowed to begin an operation or transfer a patient to the recovery room without checklist completion in the World Health Organization Surgical Safety Checklist Study (10). Even in the oft-cited airplane pilot illustration (23), participation by a copilot is compulsory. We describe our intervention as prompting rather than enforcing, to reflect that the checklist addressed more nuanced decisions (e.g., when to discontinue empirical antibiotics or wean mechanical ventilation) for which a prompted discussion is more appropriate than a mandatory decision. Our study suggests that checklists require an accountability strategy to change behavior and ultimately progress to culture change. This finding is critical to quality improvement strategies based on checklists.
Our study has several strengths that build on other successful quality improvement interventions (5, 7–11). First, prompting was face-to-face and repetitive, which strongly encouraged physicians to change their management behavior, similar to prior studies that employed local quality champions and close monitoring (9, 10). In contrast, checklist implementation without repetitive prompting in the control group had no mechanism of accountability, and did not yield improvement in outcomes.
Second, our intervention targeted multiple care practices. This may have led to more clinical benefit than any single practice alone (10, 11). Previous strategies that individually targeted our prompting topics—including empirical antibiotics (18, 24, 25), mechanical ventilation weaning (26, 27), and CVCs (8, 9)—are each associated with improved outcomes. Also, duration of mechanical ventilation, prior antibiotic use, and prior broad-spectrum antibiotic use are all independent predictors of ventilator-associated pneumonia due to potentially drug-resistant bacteria (28), which itself increases mortality (29). The improvement of several factors in combination may have plausibly led to the improvement of outcomes observed in our study.
Third, an individual prompt may impact patient care or alter provider behavior beyond that single instance of prompting. In the example of mechanical ventilation, prompting for a weaning trial occurred on only 15% of patient-days, yet a substantial 6-day increase in ventilator-free days was observed. A single prompt for mechanical ventilation weaning could have prevented multiple future days of ventilation if weaning was successful on that prompted day. Prompting at one time also may have led the prompted physician to order spontaneous breathing trials on other patients or the same patient on subsequent days, even when no prompter was present. In addition, the prompt may have elicited a discussion of the rationale for the prompt, resulting in an enhanced educational effect that would carry over to other patients on other days.
Although we hypothesized that prompting would improve quality of care, mortality and LOS benefits were unanticipated. The association of prompting with reduced mortality was observed for both severity-adjusted ICU and hospital mortality, suggesting that the effects of prompting in the ICU had a direct impact on ICU outcome and that this effect increased by hospital discharge. However, this was a small study, with a 95% confidence interval for the adjusted odds ratio of mortality that was wide (0.15–0.76), which precludes a definitive analysis of the factors that may have contributed to the mortality reduction.
Some of the mortality difference may have been due to chance, as small studies may be prone to variation. Nevertheless, even a conservative estimate based on the upper 95% confidence interval limit for adjusted mortality suggests that the intervention may have led to a reduction in deaths. Several findings may support this. There was no difference in severity of illness, baseline characteristics, or discharge disposition. The results suggest lower mortality in the prompted group for patients with sepsis, pneumonia, and other respiratory conditions, diagnoses directly related to the processes of care targeted for prompting. We found no difference in mortality in the lowest patient quartile or highest range of predicted mortality (Figure E2), as expected because ICU interventions would not be expected to affect mortality when risk of death is very high or very low. We observed no early difference in LOS; an early difference would suggest that repetitive prompting was unlikely to be responsible. Last, neither seasonal nor overall variation in hospital mortality or ICU LOS between the preintervention and control groups was demonstrated, and control group SMR matched that of a prior 1-year prospective quality project in the same ICU (1,619 patients admitted in 2006–2007; SMR; 0.96; our unpublished data). Despite these findings, our intervention deserves further exploratory analyses and study in a large, multicenter study.
Our study has several potential limitations. First, It was a single-site preliminary study, potentially limiting the generalizability of the results, particularly the mortality effect, of prompting based on our specific checklist to other settings. However, the benefit of a prompter or other similar accountability strategy is likely generalizable to improvements in different processes of care addressed by a different unit-specific checklist. Reproduction of the clinical outcome benefits in larger studies will depend on the linkage between the process of care issues addressed by the checklist and disease-specific outcomes, as well as baseline compliance.
Second, the study cohorts were relatively small. We purposefully limited the length of the study to minimize crossover of attending physicians and fellows from one team to the other, which could have reduced the apparent influence of prompting. Frequent physician rotation within teams could have impacted patients with longer ICU lengths of stay. Third, no research personnel were attached to the control team. As a result, the degree of checklist use by the control team was not directly observed, which also limited our ability to ascertain other differences in team characteristics. Last, although baseline characteristics, severity of illness, discharge disposition, and night or weekend ICU admission were not confounders, other residual confounders could exist.
Using a resident physician as a prompter is clearly an artificial construct of this study. To test effectiveness and widespread implementation, future studies should focus on determining the optimal approach to prompting, such as an electronic decision-support tool (30, 31) or virtual prompting (11). Alternative forms of prompting may increase the feasibility of future multicenter studies; however, the benefit of face-to-face prompting in our study may not be reproduced electronically.
In summary, this single-site study suggests that simply having a checklist available for reference without consideration of a robust implementation and adherence strategy is unlikely to maximize desired patient outcomes. The complexity of critical care medicine may benefit from, and provide an opportunity to investigate, novel approaches to reduce errors of omission.
The authors thank the faculty and fellows of the Division of Pulmonary and Critical Care Medicine, the residents of the Departments of Internal Medicine and Emergency Medicine, and the pharmacy and nursing staffs of the Medical Intensive Care Unit for using the checklist and including us on daily MICU rounds; Julie Garrett, M.B.A.-H.C.M., R.N. (MICU Nurse Manager, Northwestern Memorial Hospital), for implementing the checklist with the nursing staff; Ognjen Gajic, M.D. (Mayo Clinic, Rochester, MN), for inspiration to conduct quality improvement research; Joe Feinglass, Ph.D. (Northwestern University), for support with quality improvement measurements and data review; Cynthia Barnard, M.B.A., M.S.J.S., C.P.H.Q. (Northwestern Memorial Hospital), for assistance with quality improvement measurements; Helen Donnelly, R.N., B.S.N., C.C.R.C. (Northwestern Memorial Hospital), for assistance with data collection; and Eric Just, M.S. (Northwestern University), for assistance with data collection from the Enterprise Data Warehouse. None of these people received compensation beyond their normal salaries.
| 1. | Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. Med J Aust 1995;163:458–471. |
| 2. | Cook DJ, Montori VM, McMullin JP, Finfer SR, Rocker GM. Improving patients’ safety locally: changing clinician behaviour. Lancet 2004;363:1224–1230. |
| 3. | Weiss CH, Amaral LA. Moving the science of quality improvement in critical care medicine forward. Am J Respir Crit Care Med 2010;182:1461–1462. |
| 4. | Hales BM, Pronovost PJ. The checklist—a tool for error management and performance improvement. J Crit Care 2006;21:231–235. |
| 5. | DuBose JJ, Inaba K, Shiflett A, Trankiem C, Teixeira PG, Salim A, Rhee P, Demetriades D, Belzberg H. Measurable outcomes of quality improvement in the trauma intensive care unit: the impact of a daily quality rounding checklist. J Trauma 2008;64:22–27, discussion 27–29. |
| 6. | Walsh TS, Dodds S, McArdle F. Evaluation of simple criteria to predict successful weaning from mechanical ventilation in intensive care patients. Br J Anaesth 2004;92:793–799. |
| 7. | Byrnes MC, Schuerer DJ, Schallom ME, Sona CS, Mazuski JE, Taylor BE, McKenzie W, Thomas JM, Emerson JS, Nemeth JL, et al.. Implementation of a mandatory checklist of protocols and objectives improves compliance with a wide range of evidence-based intensive care unit practices. Crit Care Med 2009;37:2775–2781. |
| 8. | Berenholtz SM, Pronovost PJ, Lipsett PA, Hobson D, Earsing K, Farley JE, Milanovich S, Garrett-Mayer E, Winters BD, Rubin HR, et al.. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med 2004;32:2014–2020. |
| 9. | Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, Sexton B, Hyzy R, Welsh R, Roth G, et al.. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006;355:2725–2732. |
| 10. | Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, et al.. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–499. |
| 11. | Scales DC, Dainty K, Hales B, Pinto R, Fowler RA, Adhikari NK, Zwarenstein M. A multifaceted intervention for quality improvement in a network of intensive care units: a cluster randomized trial. JAMA 2011;305:363–372. |
| 12. | Pronovost PJ, Murphy DJ, Needham DM. The science of translating research into practice in intensive care. Am J Respir Crit Care Med 2010;182:1463–1464. |
| 13. | Weiss CH, Moazed F, McEvoy C, Singer B, Szleifer I, Amaral LAN, Watts C, Sznajder JI, Wunderink RG. Checklist-based prompting improves ICU outcomes [abstract]. Am J Respir Crit Care Med 2010;181:A2397. |
| 14. | Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med 2006;34:1297–1310. |
| 15. | Zimmerman JE, Kramer AA, McNair DS, Malila FM, Shaffer VL. Intensive care unit length of stay: benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV. Crit Care Med 2006;34:2517–2529. |
| 16. | Rubenfeld GD, Angus DC, Pinsky MR, Curtis JR, Connors AF, Bernard GR. Outcomes research in critical care: results of the American Thoracic Society critical care assembly workshop on outcomes research. Am J Respir Crit Care Med 1999;160:358–367. |
| 17. | Malmgren RD, Stouffer DB, Motter AE, Amaral LA. A Poissonian explanation for heavy tails in e-mail communication. Proc Natl Acad Sci USA 2008;105:18153–18158. |
| 18. | Fagon JY, Chastre J, Wolff M, Gervais C, Parer-Aubas S, Stephan F, Similowski T, Mercat A, Diehl JL, Sollet JP, et al.. Invasive and noninvasive strategies for management of suspected ventilator-associated pneumonia: a randomized trial. Ann Intern Med 2000;132:621–630. |
| 19. | Chastre J, Fagon JY, Bornet-Lecso M, Calvat S, Dombret MC, al Khani R, Basset F, Gibert C. Evaluation of bronchoscopic techniques for the diagnosis of nosocomial pneumonia. Am J Respir Crit Care Med 1995;152:231–240. |
| 20. | Canadian Critical Care Trials Group. A randomized trial of diagnostic techniques for ventilator-associated pneumonia. N Engl J Med 2006;355:2619–2630. |
| 21. | American Thoracic Society, Infectious Diseases Society of America. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia. Am J Respir Crit Care Med 2005;171:388–416. |
| 22. | Joint Commission. Hospital: 2011 national patient safety goals. (Accessed June, 2011.) Available at http://www.jointcommission.org/hap_2011_npsgs/ |
| 23. | Gawande A. The checklist: if something so simple can transform intensive care, what else can it do? New Yorker 2007;86–101. |
| 24. | Aarts MA, Brun-Buisson C, Cook DJ, Kumar A, Opal S, Rocker G, Smith T, Vincent JL, Marshall JC. Antibiotic management of suspected nosocomial ICU-acquired infection: does prolonged empiric therapy improve outcome? Intensive Care Med 2007;33:1369–1378. |
| 25. | Singh N, Rogers P, Atwood CW, Wagener MM, Yu VL. Short-course empiric antibiotic therapy for patients with pulmonary infiltrates in the intensive care unit: a proposed solution for indiscriminate antibiotic prescription. Am J Respir Crit Care Med 2000;162:505–511. |
| 26. | Feng Y, Amoateng-Adjepong Y, Kaufman D, Gheorghe C, Manthous CA. Age, duration of mechanical ventilation, and outcomes of patients who are critically ill. Chest 2009;136:759–764. |
| 27. | Kress JP, Pohlman AS, O'Connor MF, Hall JB. Daily interruption of sedative infusions in critically ill patients undergoing mechanical ventilation. N Engl J Med 2000;342:1471–1477. |
| 28. | Trouillet JL, Chastre J, Vuagnat A, Joly-Guillou ML, Combaux D, Dombret MC, Gibert C. Ventilator-associated pneumonia caused by potentially drug-resistant bacteria. Am J Respir Crit Care Med 1998;157:531–539. |
| 29. | Fagon JY, Chastre J, Vuagnat A, Trouillet JL, Novara A, Gibert C. Nosocomial pneumonia and mortality among patients in intensive care units. JAMA 1996;275:866–869. |
| 30. | Riggio JM, Sorokin R, Moxey ED, Mather P, Gould S, Kane GC. Effectiveness of a clinical-decision-support system in improving compliance with cardiac-care quality measures and supporting resident training. Acad Med 2009;84:1719–1726. |
| 31. | Hart EM, Owen H. Errors and omissions in anesthesia: a pilot study using a pilot's checklist. Anesth Analg 2005;101:246–250. |
Supported by the National Heart Lung and Blood Institute to C.H.W. (T32HL076139-07), M.K. (P01HL071643-07), and J.I.S. (P01HL071643-07 and R37HL048129-16), and the Parker B. Francis Fellowship to C.H.W.
Author contributions: C.H.W., F.M., C.A.M., C.M.W., J.I.S., and R.G.W. contributed to the study design; C.H.W., F.M., C.A.M., B.D.S., S.D.P., D.W.B., and R.G.W. contributed to the implementation and supervision of the study, and acquisition of data; C.H.W., I.S., L.A.N.A., M.K., S.D.P., D.W.B., J.I.S., and R.G.W. performed the statistical analysis and verified its accuracy. All authors had full access to the data. All authors participated in drafting and critically revising the manuscript. All authors approved the final version of the manuscript.
This article has an online supplement, which is available from this issue's table of contents at www.atsjournals.org
Originally Published in Press as DOI: 10.1164/rccm.201101-0037OC on May 26, 2011
Author disclosures
