Abstract
Rationale: Objective strategies are needed to improve the diagnosis of severe community-acquired pneumonia in the emergency department setting.
Objectives: To develop and validate a clinical prediction rule for identifying patients with severe community-acquired pneumonia, comparing it with other prognostic rules.
Methods: Data collected from clinical information and physical examination of 1,057 patients visiting the emergency department of a hospital were used to derive a clinical prediction rule, which was then validated in two different populations: 719 patients from the same center and 1,121 patients from four other hospitals.
Measurements and Main Results: In the multivariate analyses, eight independent predictive factors were correlated with severe community-acquired pneumonia: arterial pH < 7.30, systolic blood pressure < 90 mm Hg, respiratory rate > 30 breaths/min, altered mental status, blood urea nitrogen > 30 mg/dl, oxygen arterial pressure < 54 mm Hg or ratio of arterial oxygen tension to fraction of inspired oxygen < 250 mm Hg, age ⩾ 80 yr, and multilobar/bilateral lung affectation. From the β parameter obtained in the multivariate model, a score was assigned to each predictive variable. The model shows an area under the curve of 0.92. This rule proved better at identifying patients evolving toward severe community-acquired pneumonia than either the modified American Thoracic Society rule, the British Thoracic Society's CURB-65, or the Pneumonia Severity Index.
Conclusions: A simple score using clinical data available at the time of the emergency department visit provides a practical diagnostic decision aid, and predicts the development of severe community-acquired pneumonia.
Current rules for severe community-acquired pneumonia identify only patients with mortality at 30 days or patients requiring ICU admission.
Our proposed rule aims to identify, at first evaluation, patients at increased risk of complicated community-acquired pneumonia evolution.
Community-acquired pneumonia (CAP) is a common and potentially serious illness. Management decisions regarding site of care, extent of assessment, and level of treatment are based primarily on disease severity. Severe CAP (SCAP) is a life-threatening condition that requires intensive care. Estimates of the frequency of SCAP range from 5 to 35% (1), with mortality ranging from 20 to 50% (2). These relatively wide ranges indicate disparities between definitions of SCAP.
There is no universally accepted definition of SCAP. During the last decade, the term has been used for cases that ultimately result in death (3, 4), and/or patients requiring admission to an intensive care unit (ICU). Such practical definitions seem to be insufficient because the risk of death from CAP is not the same as the need for inpatient care. On the other hand, the decision to admit a patient to the ICU depends on the clinical judgment of individual clinicians and the local practices of their hospitals, differences that could account for much of the variability regarding ICU admission (3–8). Studies focused on the evaluation of patients admitted to the ICU (9–11) mix some variables evident at the time of admission with other potentially evolutionary criteria, which are not applicable to early hospital admission. Other criteria, such as mechanical ventilation and septic shock (11), are less subject to interpretive variability and better reflect illness severity.
Identification of patients likely to have a major adverse outcome is a key step in reducing the mortality rate of SCAP. Unfortunately, many patients may not appear severely ill at initial presentation. In these patients and others, suboptimal management in the period between hospital admission and ICU referral has been associated with increased mortality (12). Early identification of the sickest patients or those with rapidly progressing CAP may allow for earlier intervention and thus potentially improve outcomes.
Approximately 75% of CAP cases are initially evaluated and treated in hospital emergency departments (EDs) (13), making these centers the ideal location for the establishment of a medical practice guideline to evaluate SCAP. The identification of patients at risk for adverse outcomes can greatly aid clinical decision making in the ED.
The main goal of the present study was the development and validation of a prediction rule for SCAP used in the ED. We used hospital mortality, mechanical ventilation, and/or septic shock as dependent variables to define SCAP, then compared the results of applying this prediction rule with those derived from the application of three other validated prediction rules: the modified American Thoracic Society rule (m-ATS) (9), the British Thoracic Society's CURB-65 (confusion, urea, respiratory rate, and blood pressure at age 65 years or older) (3), and the Pneumonia Severity Index (PSI) (4, 14).
Patients included in the study were nonimmunosuppressed adults aged 18 yr or older with a pulmonary infiltrate on chest X-ray not known to be old and with symptoms consistent with pneumonia, including cough, dyspnea, fever, and/or pleuritic chest pain. (Additional details are provided in the online supplement.) Patients with an expected terminal event, defined as any patient with metastatic cancer, advanced dementia, or a disease or condition with a high likelihood of predicted fatality during the next 30 d, were explicitly included in the study.
The prediction rule was internally derived and validated at Galdakao Hospital, a 400-bed teaching hospital that serves a population of 300,000 inhabitants. The study included all consecutive patients with a diagnosis of CAP who visited the hospital's emergency room between March 2000 and March 2004. Patients were randomized into two groups to create a derivation cohort with 60% of the patients and an internal validation cohort with the other 40% (Figure 1). Data were collected prospectively.
Figure 1. Cohort distribution in the prediction rule. Patients assigned to the derivation and validation cohorts are shown together with the flow used for establishing the rule. *Excluded patients were known to be positive for the human immunodeficiency virus, chronically immunosuppressed, or hospitalized during the previous 14 d.
[More] [Minimize]An external validation cohort was formed with patients admitted to four other hospitals in the same health network. Patient records were randomly selected and retrospectively analyzed for two periods, March 1998 to March 1999 and March 2000 to September 2001. This cohort had been used in a previous study (6). Sample design and hospital characteristics, as well as additional details regarding identification, evaluation, and exclusion criteria, have been described previously elsewhere (15). (Additional details are available in the online supplement.)
Local ethics committees approved the project.
Predictive variables were clearly defined and selected after an exhaustive literature review. Patients in the derivation and internal validation cohorts were managed according to a clinical guideline that guaranteed the prospective and systematic collection of relevant clinical information. The data were managed using a computer-based score with previously defined dedicated software (16). The same variables were gathered for the clinical history reviews of the external validation cohort.
A protocol was defined for the collection of data and definition of variables such as septic shock, severe sepsis, and mechanical ventilation. (Additional details are in the online supplement.)
Descriptive statistics included frequency tables and mean and standard deviations. The χ2 and Fisher's exact tests were used to test for statistical significance among proportions. For continuous variables, the Student t test or analysis of variance was performed when appropriate in the univariate analysis of the three cohorts.
To identify the variables correlated with SCAP, we performed univariate analyses using the χ2 test. Patients with statistically significant results at p < 0.15 were entered in a multivariate model.
We performed logistic regression models to select the variables for our prediction rule. We provide the area under the curve (AUC) of the final model and the odds ratios (OR) and 95% confidence intervals (CI) of all selected variables. No assumptions were made in relation to missing values.
We also estimated the same parameters for other prediction rules (m-ATS, CURB-65, and PSI) as applied to the three cohorts. All effects were considered significant at p < 0.05, unless otherwise noted. (Additional details are available in the online supplement.)
A total of 1,940 episodes of CAP diagnosis were recorded in the ED; 1,776 satisfied all criteria for inclusion in the outcome analysis (Figure 1). Of these, 46 episodes were classified as an expected terminal event at the time of diagnosis. A total of 1,057 patients were randomly assigned to the derivation cohort and 719 to the internal validation cohort. The external validation cohort was formed by 1,121 patients admitted to four other hospitals. The sociodemographic features of the derivation, internal validation, and external validation cohorts are described in Table 1. The average age was 64.7 yr (SD, 19.7 yr). Some variables were missing in the external validation cohort, although only respiratory rate, arterial pH, and PaO2/FiO2 < 250 mm Hg showed missing percentages higher than 5%. Patients in the derivation and internal validation cohorts were managed following a clinical guideline, so they did not show any missing values. Pulse oximetry was performed on 4.1% of patients, whereas the rest underwent blood gas analysis. The rate of SCAP among admitted patients was 11.5% in the derivation cohort, 9.8% in the internal validation cohort, and 12% in the external cohort, whereas in-hospital mortality was 9.1, 8.2, and 9.7%, respectively. Of the total number of patients evaluated at the ED, 3% were admitted to the ICU; among inpatients, 4% were admitted to the ED. The sociodemographic characteristics of admitted patients in the derivation and validation cohorts are described in Table E1 in the online supplement.
Characteristics | Derivation (n = 1,057) (%) | Internal Validation (n = 719) (%) | External Validation‡ (n = 1,121) (%) | Missing§ (%) |
|---|---|---|---|---|
| Demographic factors | ||||
| Age ⩾ 80 yr | 25.0 | 22.3 | 31.3 | |
| Women | 35.6 | 38.4 | 37.4 | |
| Nursing home | 6.8 | 4.2 | 6.9 | |
| Prior antibiotic treatment | 19.5 | 20.2 | 22.8 | |
| Comorbidity | ||||
| Neoplastic disease | 4.4 | 3.6 | 7.1 | |
| Liver disease | 4.0 | 2.8 | 3.9 | |
| Congestive heart failure | 5.1 | 6.5 | 12.9 | |
| Cerebrovascular disease | 7.6 | 8.9 | 14.0 | |
| Renal disease | 6.4 | 6.5 | 4.4 | |
| Chronic obstructive lung disease | 21.3 | 18.8 | 29.1 | |
| Physical exam | ||||
| Altered mental status | 11.1 | 8.8 | 9.4 | |
| Pulse > 125/min | 6.7 | 7.4 | 8.4 | 3.5 |
| Respiratory rate > 30 breaths/min | 11.7 | 10.7 | 28.0 | 39.7 |
| Systolic blood pressure < 90 mm Hg | 3.2 | 3.2 | 3.4 | 2.0 |
| Temperature < 35°C or > 40°C | 3.0 | 2.8 | 4.1 | 1.6 |
| Laboratory and X-ray findings | ||||
| Blood urea nitrogen > 30 mg/dl | 19.2 | 19.2 | 24.9 | 3.4 |
| Glucose > 250 mg/dl | 5.8 | 6.3 | 7.9 | 2.8 |
| Hematocrit < 30% | 0.4 | 0.3 | 0.9 | 2.8 |
| Sodium < 130 mmol/L | 4.0 | 4.9 | 4.4 | 3.9 |
| PaO2/FiO2 < 250 mg/dl | 16.7 | 15.2 | 25.6 | 17.1 |
| Arterial pH < 7.30 | 2.3 | 2.1 | 3.2 | 16.9 |
| Pleural effusion | 10.0 | 8.1 | 7.6 | |
| Multilobar/bilateral X-ray | 17.2 | 18.6 | 24.1 | |
| SCAP criteria* | 11.5 | 9.8 | 12.0 | |
| In-hospital death | 9.1 | 8.2 | 9.7 | |
| Mechanical ventilation | 1.5 | 1.8 | 2.0 | |
| Shock | 2.3 | 1.6 | 2.3 | |
| ICU† | 3.9 | 4.4 | 3.4 |
Table 2 shows the 14 variables analyzed in the derivation cohort associated with SCAP in the univariate analysis. Of those, two were sociodemographic, one was comorbidity, five involved the physical exam, four were analytical, and two were related to X-ray results.
Variables | Presence SCAP (n = 76) (%) | Nonpresence SCAP (n = 981) (%) | p Value | OR (95% CI) |
|---|---|---|---|---|
| Age ⩾ 80 yr | 63.2 | 22.0 | < 0.001 | 6.1 (3.7–9.9) |
| Nursing home resident | 26.3 | 5.3 | < 0.001 | 6.4 (3.6–11.4) |
| Cerebrovascular disease | 21.0 | 6.5 | < 0.001 | 3.8 (2.1–7.0) |
| Altered mental status | 43.4 | 8.6 | < 0.001 | 8.2 (4.9–13.6) |
| Pulse > 125/min | 15.8 | 6.0 | < 0.01 | 2.9 (1.5–5.7) |
| Respiratory rate > 30 breaths/min | 54.0 | 8.5 | < 0.001 | 12.7 (7.7–21.0) |
| Systolic pressure < 90 mm Hg | 19.7 | 1.9 | < 0.001 | 12.5 (6.0–25.7) |
| Temperature < 35C° or > 40C° | 7.9 | 2.7 | < 0.05 | 3.1 (1.3–7.9) |
| Blood urea nitrogen > 30 mg/dl | 59.2 | 16.1 | < 0.001 | 7.6 (4.6–12.3) |
| Glucose > 250 mg/dl | 11.8 | 5.3 | < 0.05 | 2.4 (1.1–5.1) |
| PaO2/FiO2 < 250 mm Hg | 51.3 | 14.0 | < 0.001 | 6.5 (4.0–10.5) |
| Arterial pH < 7.30 | 17.1 | 1.1 | < 0.001 | 18.2 (7.8–42.3) |
| Pleural effusion | 23.7 | 9.0 | < 0.001 | 3.1 (1.8–5.6) |
| Multilobar/bilateral X-ray | 47.4 | 14.9 | < 0.001 | 5.1 (3.2–8.3) |
In the multivariate analyses, eight independent predictive factors were correlated with SCAP: systolic blood pressure < 90 mm Hg, arterial pH < 7.30, respiratory rate > 30 breaths/min, blood urea nitrogen (BUN) > 30 mg/dl, oxygen arterial pressure < 54 mm Hg or PaO2/FiO2 < 250 mm Hg, altered mental status, age ⩾ 80 yr, and multilobar/bilateral lung affectation in X-rays. Although living in a nursing home was also correlated with SCAP (OR, 2.4; 95% CI, 1.1–5.5), it was not included in the predictive model. Results from the multivariate analyses were used to develop a clinical prediction rule. From the β parameter obtained in the multivariate model, a score was assigned to each predictive variable (Table 3). By adding up the points assigned to each predictive variable, a score was given to each patient, with a higher score corresponding to a higher likelihood of SCAP. This enabled us to group patients in different risk categories (Figure 2). The prediction rule showed good discriminating power, as measured by the AUC of 0.92. Applying this prediction rule to the cohorts yielded an AUC of 0.92 for the internal cohort and 0.80 for the external validation cohort. The prediction rule also showed good discriminating power when applied to in-hospital mortality (see Figure E2).
Figure 2. Severe community-acquired pneumonia (SCAP) prediction rule behavior in the derivation and validation cohorts. Percentage of SCAP for each cohort according to the cutoff intervals of the rule.
[More] [Minimize]Variables | β Parameter | OR (95% CI) | Points | Criteria |
|---|---|---|---|---|
| Intercept | −4.79 | |||
| pH < 7.30 | 2.38 | 10.8 (3.5–34.0) | 13 | Major |
| Systolic pressure < 90 mm Hg | 2.19 | 8.9 (3.2–25.2) | 11 | Major |
| Respiratory rate > 30 breaths/min | 1.83 | 6.3 (3.4–11.7) | 9 | Minor |
| Blood urea nitrogen > 30 mg/dl | 0.92 | 2.5 (1.4–4.7) | 5 | Minor |
| Altered mental status | 0.87 | 2.4 (1.2–4.6) | 5 | Minor |
| PaO2/FiO2 < 250 mm Hg | 1.12 | 3.1 (1.7–5.7) | 6 | Minor |
| Age ⩾ 80 yr | 0.86 | 2.4 (1.3–4.4) | 5 | Minor |
| Multilobar/bilateral X-ray | 0.68 | 2.0 (1.1–3.7) | 5 | Minor |
When applying a cutoff point of 10 or greater, our prediction rule showed an AUC of 0.83 for the derivation cohort, 0.86 for the internal validation cohort, and 0.72 for the external validation cohort. Values of sensitivity, specificity, and positive and negative predictive values for our rule and three other predictive rules are shown in Table 4. Both m-ATS and CURB-65 had low sensitivity (51.3 and 68.4%, respectively), whereas PSI risk class IV–V and adjusted PSI demonstrated poor specificity (68.1 and 57.5%) for the derivation cohort, a trend lessened in the validation cohorts. The performance of the different rules was consistent across cohorts.
Rule | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | AUC (%) | p Value |
|---|---|---|---|---|---|---|
| SCAP prediction rule | ||||||
| Derivation | 92.1 | 73.8 | 21.4 | 99.2 | 0.83 | |
| Internal validation | 95.5 | 76.7 | 21.1 | 99.6 | 0.86 | |
| External validation | 84.3 | 60.3 | 22.0 | 96.7 | 0.72 | |
| m-ATS* | ||||||
| Derivation | 51.3 | 95.9 | 49.4 | 96.2 | 0.74 | <0.01 |
| Internal validation | 61.4 | 96.7 | 55.1 | 97.5 | 0.79 | 0.07 |
| External validation | 50.4 | 91.9 | 47.2 | 92.8 | 0.71 | 0.37 |
| CURB-65 (3–5)† | ||||||
| Derivation | 68.4 | 86.8 | 28.6 | 97.3 | 0.78 | <0.05 |
| Internal validation | 63.6 | 87.3 | 24.6 | 97.4 | 0.75 | <0.01 |
| External validation | 60.3 | 78.4 | 26.7 | 93.8 | 0.69 | 0.17 |
| PSI (IV-V)‡ | ||||||
| Derivation | 94.7 | 68.1 | 18.7 | 99.4 | 0.81 | 0.24 |
| Internal validation | 88.6 | 69.3 | 15.9 | 98.9 | 0.79 | <0.01 |
| External validation | 91.4 | 50.7 | 19.8 | 97.8 | 0.71 | 0.32 |
| Adjusted PSI§ | ||||||
| Derivation | 97.4 | 57.5 | 15.1 | 99.7 | 0.77 | <0.01 |
| Internal validation | 95.5 | 61.2 | 13.8 | 99.5 | 0.78 | <0.01 |
| External validation | 95.7 | 36.4 | 16.7 | 98.5 | 0.66 | <0.05 |
When applied to the entire prospective cohort (derivation and internal validation cohorts), our prediction rule did not detect eight false negatives out of 1,250 patients. All eight died, although only one patient with a history of congestive heart failure was admitted to the ICU on the third day after admission, requiring mechanical ventilation and vasopressors. Another patient died of complications from lung thromboembolism, diagnosed via computerized axial tomography, without requiring any modification of the antibiotic treatment. Another patient suffered from neurologic degenerative disease and was treated with home-administered noninvasive mechanical ventilation; this patient presented with a pH of 7.35 and a Pco2 of 69 mm Hg at the time of admission. Three patients were between 76 and 80 yrs of age, and fulfilled, in addition, one of the other criteria in the rule (either Po2 or BUN). The other two were 80 yr or older. One of these patients had advanced-stage neoplastic disease and the other presented with three criteria (BUN of 29 mg/dl, Po2 of 59 mm Hg, and respiratory rate of 30 breaths/min, just under the limit of the cutoff for this variable) at the time of diagnosis (see Table E2). Of the 526 patients positively identified as having SCAP by the prediction rule, there were 414 false positives. Fourteen required admission to the ICU for monitoring. The contributors to the false positives were systolic blood pressure < 90 mm Hg (32 cases), arterial pH < 7.30 (21 cases), BUN > 30 mg/dl (204 cases), respiratory rate > 30 breaths/min (121 cases), PaO2/FiO2 < 250 mm Hg (162 cases), altered mental status (102 cases), multilobar/bilateral lung affectation in X-rays (157 cases), and age > 80 yr (262 cases). Of the 414 patients with false-positive results, 82% presented with severe sepsis early in the hospital course, defined as infection plus new-onset acute organ dysfunction using consensus criteria. Among the 80 patients without severe sepsis, the relevant variables were age, BUN, multilobar lung affectation, and respiratory rate.
After excluding patients with expected terminal events, who were more likely to receive aggressive treatment, we evaluated the rule for predicting SCAP in the entire prospective cohort. The AUC was 0.92; when a cutoff of 10 or greater was used, the AUC was 0.84.
Our findings suggest that a simple model that uses only eight variables, which are easily accessible and interpretable at the time of ED presentation, can identify seriously ill patients with CAP who are at risk for an adverse outcome and can provide a practical diagnostic decision aid. Using data initially collected in the ED, our rule can identify at admission patients developing SCAP within and beyond the first 24 h. To our knowledge, this is the first such prediction rule that achieves this goal.
This rule achieved high sensitivity and high AUC, which were maintained in validation samples. It was derived from a significant number of consecutive patients from the ED of a single hospital and validated in two cohorts with a wide sample of patients, thus suggesting that such a rule could be applied with some confidence in medical practice while undergoing future prospective evaluations. The model has a higher discriminatory power for predicting SCAP than do the m-ATS, CURB-65, and PSI.
Instead of using ICU admission, which is highly variable according to individual ICU admission practices (3–8), as a criterion for determining SCAP, we considered in-hospital death, mechanical ventilation, and/or septic shock as endpoints, given their more objective nature as variables (11).
The main value of this prediction rule is its ability to identify patients who need additional monitoring and more aggressive treatment after the first evaluation in the ED either at the ICU or other alternative settings, such as intermediate care units or a specialized regular ward, depending on the severity of the patient's condition.
Multiple logistic regression results identified eight independent predictive factors that correlated with SCAP. As stated earlier, the “nursing home” variable, although significantly associated with SCAP, was excluded from the prediction rule because cultural and social differences regarding the use of nursing homes could affect the rule's generalizability.
The strongest clinical predictors of SCAP were pH < 7.30 and systolic pressure < 90 mm Hg. A depressed pH, which is likely a side effect of metabolic acidosis derived from sepsis (17), is not included in other prediction rules such as CURB-65 or m-ATS (3, 9). On the other hand, it is given the highest score in the PSI (4) and also appears as a clinical characteristic associated with mortality in patients with SCAP admitted to the ICU (18).
All of the variables used to create our score are included in the PSI, except for multilobar/bilateral lung affectation in X-rays, which has recently been related to treatment failure and poor prognosis (7). Our rule presents a simplified and reweighted PSI with a different cutoff for pH, Po2, and age. Dealing with only eight variables instead of 20 offers greater simplicity and applicability. The covariates in our rule are readily obtainable in the normal course of a patient's ED work-up. An overall score was computed from the scores assigned to each of the eight predictive variables. It must be noted that having eight variables with different scores makes memorization difficult. For this reason, the score may be used as a continuous variable or with a cutoff of 10 or greater, with easy-to-use major and minor criteria that do not require any tool. Thus, our score includes variables present in other well-known prediction rules, such as the CURB and PSI, and recalculates the relative weight of these variables and comes up with a simple, easy-to-memorize algorithm that can easily identify patients at risk of an adverse evolution. Our rule is less complicated than the PSI; it adds variables to the CURB-65 that have been demonstrated to improve the ability to predict severity; and it is more useful than the m-ATS for patients in the ED. It was not our goal to prove or disprove the validity of these rules when they are applied in a way consistent with their original purposes.
The variables we used are also included in the CURB-65, although this rule uses an age cutoff of 65 yr, which is lower than our cutoff of 80 yr. In addition, we included three more variables—pH, PaO2/FiO2, and multilobar/bilateral lung affectation in X-rays—that the authors of the CURB-65 had pointed out would likely be necessary to consider (19). Hypoxemia has been associated with impending respiratory failure, subsequent ICU admission (4), and mortality (20) in patients with CAP, reflecting the severity of primary organ impairment in this illness. Furthermore, the identification of arterial hypoxemia has direct treatment implications, which include the delivery of supplemental oxygen, ventilatory support (21), as well as hospitalization for more intensive clinical observation. Arterial blood gas provides useful information about arterial pH and PaO2, although in patients with hemodynamic stability, it could be avoided with a pulse oximetry finding of greater than 90%. With respect to the m-ATS criteria, our score shares some variables with this system. In our score, however, the two major m-ATS criteria, septic shock and mechanical ventilation, were defined as endpoints because they are potentially evolutionary criteria. Consequently, they are not applicable to early hospital admission.
The variables can be grouped into major (pH < 7.30 and systolic pressure < 90 mm Hg) and minor (respiratory rate ⩾ 30 breaths/min, confusion, BUN > 30 mg/dl, PaO2 < 54 mm Hg or PaO2/FiO2 < 250 mm Hg, age ⩾ 80 yr, and multilobar/bilateral lung affectation in X-rays). If we apply the score with a cutoff of 10 or greater, the evaluation of SCAP is based on the presence of one major criterion, or two or more minor criteria (see Table 3 and Figure 3).
Figure 3. The variables of score grouped in major and minor criteria. The evaluation of SCAP is based on the presence of one major criterion or two or more minor criteria. P = arterial pH; S = systolic pressure; C = confusion; U = blood urea nitrogen; R = respiratory rate; X = X-ray; O = PaO2; 80 = Age ⩾ 80 years.
[More] [Minimize]At a cutoff of 10 or greater, the prediction rule showed 92.1% sensitivity and 73.8% specificity, which proved superior when compared with the application of m-ATS or CURB-65 with three or more factors, although it showed less sensitivity than PSI classes IV and V and the adjusted PSI. The specificity findings suggest that our rule has the potential to identify a larger proportion of patients as low risk than the PSI. We chose a cutoff of 10 or more because it presented good sensitivity while showing the best balance between sensitivity and specificity. The low sensitivity of m-ATS is probably due to the use of data available at initial evaluation. On the other hand, the low sensitivity of CURB-65 may be due to the fact that the class with two factors had a 7% mortality rate and that the cutoff for this rule is established with three or more factors.
Due to the substantial proportion of outpatients included in our study, the prevalence of SCAP in the prospective cohort was lower than in the external derivation cohort and in other studies primarily focused on inpatients (3, 5, 8–11). This explains why the negative predictive values for all prediction rules were uniformly high, whereas the positive predictive values were generally low in our study cohort. The low positive predictive value of the rules often failed to select the patients at risk for subsequent SCAP. Therefore, its best use as a stratification tool may be as an adjunct to clinical judgment. The negative predictive value was excellent for all rules, which means that all patients who do not fulfill the criteria could be safely managed without intensive care.
Approximately 80% of the patients with false-positive results presented with severe sepsis. Severe sepsis is common in CAP, occurring in approximately one-half of all hospitalized patients with CAP and early in the course of the hospital stay (22). It has been shown that early intervention in subjects with incipient severe sepsis may improve outcome (23). Our rule did not detect a small population of eight patients who developed SCAP, only one of whom required admission to the ICU. This patient was admitted with associated comorbidity and evolved to shock and respiratory failure on the third day after admission, thus underscoring the importance of evolution during the first hours of acute-onset diseases such as CAP. Given that pneumonia is a dynamic process, a small yet important number of patients who might not fulfill severity criteria on hospital admission may nevertheless be at risk of developing SCAP in the following days. In connection with this, Ewig and colleagues (5) observed that the m-ATS rule predicted 100% of ICU admissions later in the course of hospitalization (within 48–72 h).
Although the aforementioned prediction rules currently established by scientific societies (24–28), such as CURB-65 and PSI, adequately predict the risk of death and therefore the decision for hospital admission, they do not specify the level of care most likely required. In an effort to fill that gap, our study focused on the identification of patients whose disease can suddenly take a turn for the worse, and thus who may require additional monitoring and care (hemodynamic monitoring, ventilatory support, and nursing observation). For that reason, despite the high number of false positives, our rule is adequate for identifying seriously ill patients with CAP at the time of ED evaluation who are at risk for an adverse outcome, which is a more efficient tool from a clinical point of view. It is important to increase the degree of awareness about signs of severity in CAP, because it has been demonstrated that treatment for patients with obvious signs of severity, such as confusion and hypotension, often does not measure up to treatment guidelines (29).
Exclusion from the derivation and internal validation cohorts of the 46 patients considered to be having terminal events maintained the model's predictive capability. It is difficult to predict the terminal phase of many diseases (30), which is why these patients were not excluded initially. In either case, this rule is valid for identifying all at-risk patients in the ED as well as those suitable for more aggressive treatment.
This study has four main limitations:
The rule was derived from patients from a single hospital and validated with an external cohort in which some data were missing due to the retrospective manner of their collection. To evaluate whether our findings could be generalized to other populations, we compared our patients with other populations of hospitalized patients with CAP described in the literature (3, 31). Using a well-established prediction rule for risk of death and other adverse outcomes, the data indicate that the population of patients with pneumonia in our hospital is similar to the populations of patients admitted to other hospitals. In addition, our score was validated in an external cohort of patients from four other hospitals. Still, a prospective validation is required to assess the generalizability of these findings.
The external validation cohort does not include outpatients, because it involved a retrospective analysis of the clinical history of admitted patients.
Our data suggest underuse of the ICU in our center. It is possible that older age was used as a restrictive criterion for ICU admission even though there is evidence to justify care given to critically ill, elderly patients in units with closer monitoring (32, 33).
4.Our prediction rule did not take into account the potential diagnostic utility of new investigative techniques, such as markers of inflammatory response (34–38). However, these markers are not necessarily available outside tertiary centers and may not yield immediate results at the time of ED consultation.
The clinical heterogeneity of CAP is a global limitation when attempting to create a scoring system capable of separating patients into appropriate management groups. This limitation compels the notion that clinical judgment, which is difficult to define in objective terms, must remain an indispensable part of decision making. Our proposed score aims to identify as early as possible patients at increased risk of complicated CAP evolution. The use of this score in EDs may help in triage decisions. It may also be used to design future studies and health care strategies aimed at reducing CAP mortality. We suggest that patients scored as high risk (score ⩾ 10) should receive closer monitoring than low-risk patients.
In summary, a prediction rule based on eight variables that are easily accessible at the time of ED admission is able to identify patients whose CAP may substantially worsen. The similarity of these prognostic variables to those of previously published models suggests that the results could be generalized to other institutions. The value of this rule is supported by the small number of false negatives and the fact that it can be used in any type of population visiting an ED.
The authors appreciate the support of the Direction of Osakidetza, staff members of different services, and the medical record sections of the participating hospitals. They also thank Mr. Patrick J. Skerrett for his assistance in the editing work of the manuscript and Dr. A. Torres for the review of the manuscript.
| 1. | Alvarez-Lerma F, Torres A. Severe community-acquired pneumonia. Curr Opin Crit Care 2004;10:369–374. |
| 2. | Rodriguez de Castro F, Torres A. Optimizing treatment outcomes in severe community-acquired pneumonia. Am J Respir Med 2003;2:39–54. |
| 3. | Lim WS, Van der Eerden MM, Laing R, Boersma WG, Karalus N, Town GI, Lewis SA, Macfarlane JT. Defining community-acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax 2003;58:377–382. |
| 4. | Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, Coley CM, Marrie TJ, Kapoor WN. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 1997;336:243−250. |
| 5. | Ewig S, de Roux A, Bauer T, Garcia E, Niederman M, Torres A. Validation of predictive rules and indices of severity for community acquired pneumonia. Thorax 2004;59:421–427. |
| 6. | Capelastegui A, España PP, Quintana JM, Gorordo I, Ortega M, Idoiaga I, Bilbao A. Improvement of process-of-care and outcomes after implementing a guideline for management of community-acquired pneumonia: a controlled before-and-after study. Clin Infect Dis 2004;39:955–963. |
| 7. | Menéndez R, Torres A, Zalacaín R, Aspa J, Martín Villasclaras JJ, Borderías L, Benítez Moya JM, Ruiz-Manzano J, Rodríguez de Castro F, Blanquer J, et al. Risk factors to treatment failure in community acquired pneumonia: implications for disease outcome. Thorax 2004;59:960–965. |
| 8. | Riley PD, Aronsky D, Dean NC. Validation of the 2001 American Thoracic Society criteria for severe community-acquired pneumonia. Crit Care Med 2004;32:2398–2402. |
| 9. | Ewig S, Ruiz M, Mensa J, Marcos MA, Martinez JA, Arancibia F, Niederman MS, Torres A. Severe community-acquired pneumonia: assessment of severity criteria. Am J Respir Crit Care Med 1998;158:1102–1108. |
| 10. | Leroy O, Georges H, Beuscart C, Guery B, Coffinier C, Vandenbussche C, Thevenin D, Beaucaire G. Severe community-acquired pneumonia in ICUs: prospective validation of a prognostic score. Intensive Care Med 1996;22:1307–1314. |
| 11. | Angus DC, Marrie TJ, Obrosky DS, Clermont G, Dremsizov TT, Coley C, Fine MJ, Singer DE, Kapoor WN. Severe community-acquired pneumonia: use of intensive care services and evaluation of American and British Thoracic Society diagnosis criteria. Am J Respir Crit Care Med 2002;166:717–723. |
| 12. | McQuillan P, Pilkington S, Allan A, Taylor B, Short A, Morgan G, Nielsen M, Barrett D, Smith G. Confidential inquiry into quality of care before admission to intensive care. BMJ 1998;316:1853–1858. |
| 13. | Yealy DM, Auble TE, Stone RA, Lave JR, Meehan TP, Graff LG, Fine JM, Obrosky DS, Edick SM, Hough LJ, et al. The emergency department community-acquired pneumonia trial: methodology of a quality improvement intervention. Ann Emerg Med 2004;43:770–782. |
| 14. | Yealy DM, Auble TE, Stone RA, Lave JR, Meehan TP, Graff LG, Fine JM, Obrosky DS, Mor MK, Whittle J, et al. Effect of increasing the intensity of implementing pneumonia guidelines: a randomized, controlled trial. Ann Intern Med 2005;143:881–894. |
| 15. | Capelastegui A, España PP, Quintana JM, Gorordo I, Gallardo MS, Idoiaga I, Bilbao A. Management of community-acquired pneumonia and secular trends at different hospitals. Respir Med 2005;99:268–278. |
| 16. | España PP, Capelastegui A, Quintana JM, Soto A, Gorordo I, García-Urbaneja M, Bilbao A. A prediction rule to identify allocation of inpatient care in community-acquired pneumonia. Eur Respir J 2003;21:695–701. |
| 17. | Fall PJ, Szerlip HM. Lactic acidosis: from sour milk to septic shock. J Intensive Care Med 2005;20:255–271. |
| 18. | Yosimoto A, Nakamura H, Fujimura M, Nakao S. Severe community-acquired pneumonia in an intensive care unit: risk factors for mortality. Intern Med 2005;44:710–716. |
| 19. | British Thoracic Society Standards of Care Committee. BTS guidelines for management of community acquired pneumonia in adults. Thorax 2001;56(Suppl 4):IV 1–64. |
| 20. | Bowton DL, Scuderi PE, Haponik EF. The incidence and effect of outcome of hypoxemia in hospitalized medical patients. Am J Med 1994;97:38–46. |
| 21. | Ferrer M, Esquinas A, Leon M, Gonzalez G, Alarcon A, Torres A. Noninvasive ventilation in severe hypoxemic respiratory failure: a randomized clinical trial. Am J Respir Crit Care Med 2003;168:1438–1444. |
| 22. | Dremsizov T, Clermont G, Kellum JA, Kalassian KG, Fine MJ, Angus DC. Severe sepsis in community-acquired pneumonia: when does it happen, and do systemic inflammatory response syndrome criteria help predict course? Chest 2006;129:968–978. |
| 23. | Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, Knoblich B, Peterson E, Tomlanovich M; Early Goal-Directed Therapy Collaborative Group. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 2001;345:1368–1377. |
| 24. | Bartlett JG, Dowell SF, Mandell LA, File Jr TM, Musher DM, Fine MJ. Practice guidelines for the management of community-acquired pneumonia in adults: Infectious Diseases Society of America. Clin Infect Dis 2000;31:347–382. |
| 25. | American College of Emergency Physicians Clinical Policies Committee and the Clinical Policies Subcommittee on Community-acquired Pneumonia. Clinical policy for the management and risk stratification of community acquired pneumonia in adults in the emergency department. Ann Emerg Med 2001;38:107–113. |
| 26. | Mandell LA, Bartlett JG, Dowell SF, File TM, Musher DM, Whitney C. Update of practice guidelines, for the management of community-acquired pneumonia in immunocompetent adults. Clin Infect Dis 2003;37:1405–1433. |
| 27. | MacFarlane JT, Boldy D. 2004 update of BTS pneumonia guidelines: what's new? Thorax 2004;59:364–366. |
| 28. | Alfageme I, Aspa J, Bello S, Blanquer J, Blanquer R, Borderías L, Bravo C, de Celis R, de Gracia X, Dorca J, et al. Guidelines for the diagnosis and management of community-acquired pneumonia. Spanish Society of Pulmonology and Thoracic Surgery. Arch Bronconeumol 2005;41:272–289. |
| 29. | Menéndez R, Torres A, Zalacaín R, Aspa J, Martin-Villasclaras JJ, Borderías L, Benítez-Moya JM, Ruiz-Manzano J, de Castro FR, Blanquer J, et al. Guidelines for the treatment of community-acquired pneumonia: predictors of adherence and outcome. Am J Respir Crit Care Med 2005;172:757–762. |
| 30. | Brandt HE, Deliens L, Ooms ME, van der Steen JT, van der Wal G, Ribbe MW. Symptoms, signs, problems, and diseases of terminally ill nursing home patients. Arch Intern Med 2005;165:314–320. |
| 31. | Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG. A controlled trial of critical pathway for treatment of community-acquired pneumonia. JAMA 2000;283:749–755. |
| 32. | Kaplan V, Angus DC, Griffin MF, Clermont G, Scott Watson R, Linde-Zwirble WT. Hospitalized community-acquired pneumonia in the elderly: age- and sex-related patterns of care and outcome in the United States. Am J Respir Crit Care Med 2002;165:766–772. |
| 33. | Díaz A, Alvarez M, Callejas C, Rosso R, Schnettler K, Saldías F. Clinical picture and prognostic factors for severe community-acquired pneumonia in adults admitted to the intensive care unit. Arch Bronconeumol 2005;41:20–26. |
| 34. | Hedlund J, Hansson LO. Procalcitonin and C-reactive protein levels in community-acquired pneumonia: correlation with etiology and prognosis. Infection 2000;28:68–73. |
| 35. | Gallagher PM, Lowe G, Fitzgerald T, Bella A, Greene CM, McElvaney NG, O'Neill SJ. Association of IL-10 polymorphism with severity of illness in community acquired pneumonia. Thorax 2003;58:154–156. |
| 36. | Yende S, Tuomanen E, Wunderink R, Kanaya A, Newman A, Harris T, de Rekeneire N, Kritchevky B; Health ABC Study. Preinfection systemic inflammatory markers and risk of hospitalization due to pneumonia. Am J Respir Crit Care Med 2005;172:1440–1446. |
| 37. | Schaaf B, Boehmke F, Esnaashari H, Seitzer U, Kothe H, Maass M, Zabel P, Dalhoff K. Pneumococcal septic shock is associated with the interleukin-10–1082 gene promoter polymorphism. Am J Respir Crit Care Med 2002;168:476–480. |
| 38. | Christ-Crain M, Stolz D, Bingisser R, Muller C, Miedinger D, Huber PR, Zimmerli W, Harbarth S, Tamm M, Muller B. Procalcitonin guidance of antibiotic therapy in community-acquired pneumonia: a randomized trial. Am J Respir Crit Care Med 2006;174:84–93. |
