Abstract
Rationale: Observational studies suggest a 50% mortality reduction for older patients receiving influenza vaccination; some deem this magnitude of benefit implausible and invoke confounding by the “healthy user effect” as an alternate explanation.
Objectives: To evaluate unrecognized confounding by hypothesizing the presence of a 50% mortality reduction with vaccination for patients with pneumonia outside of influenza season.
Methods: Clinical, laboratory, and functional data were prospectively collected on 1,813 adults with community-acquired pneumonia admitted to six hospitals outside of influenza season in the Capital Health region (AB, Canada). Vaccination status was ascertained by interview and chart review. Outcome was in-hospital mortality. Influenza-vaccinated patients were matched to a nonvaccinated control using propensity scores, and then multivariable regression was used to determine the independent association between vaccination and mortality.
Measurements and Main Results: The cohort consisted of 352 vaccine recipients and 352 matched control subjects. Most (85%) patients were 65 years or older, 29% had severe pneumonia, and 12% died. Influenza vaccination was associated with a 51% mortality reduction (28 of 352 [8%] died vs. 53 of 352 [15%] control subjects; unadjusted odds ratio [OR], 0.49; 95% confidence interval [CI], 0.30–0.79; P = 0.004) outside influenza season. Adjustment for age, sex, and comorbidities did not alter these findings (adjusted OR, 0.45; 95% CI, 0.27–0.76). More complete adjustment for confounding (e.g., functional and socioeconomic status) markedly attenuated these benefits and their statistical significance (adjusted OR, 0.81; 95% CI, 0.35–1.85; P = 0.61).
Conclusions: The 51% reduction in mortality with vaccination initially observed in patients with pneumonia who did not have influenza was most likely a result of confounding. Previous observational studies may have overestimated mortality benefits of influenza vaccination.
Observational studies report a 50% reduction in mortality associated with influenza vaccination; however, many scientists contend that this magnitude of benefit is implausible.
The results of this study suggest that many previous observational studies have overestimated the mortality benefits of influenza vaccination due to difficult-to-correct confounding attributable to the “healthy-user” effect.
First, although few randomized trials have been completed, no trial data support a mortality benefit with influenza vaccination (3–5). Second, over the last two decades in the United States, even while vaccination rates among the elderly have increased from 15 to 65%, there has been no commensurate decrease in hospital admissions or all-cause mortality (1–3). In fact, both admission rates and mortality in those 65 years and older have increased with increasing vaccine coverage over time (1–3). Of course, it is plausible that secular changes in multiple unmeasured patient factors during this time period of increased vaccination might have still led to increased rates of hospitalization and mortality. Third, some studies have observed mortality reduction with influenza vaccination in the “off-season” (i.e., the time of year when there is little if any circulating virus) (5, 9, 10). Last, studies that are able to better adjust for health and functional status or other measures of frailty demonstrate attenuation or abolition of mortality benefit, suggesting substantial residual confounding in most reported observational studies of vaccine effectiveness (4, 5, 11). This phenomenon has been variously referred to as frailty bias or the healthy-vaccinee effect or, more generally, as the “healthy-user” effect (4, 5, 11–13). The healthy-user is postulated to be a patient who, despite the presence of various coexisting conditions, is relatively healthier and has a predilection for better lifestyle behaviors (e.g., diet and exercise), more health seeking and preventative activities (e.g., cancer screening or treatment with medications like statins), better adherence to medical advice and therapies, and greater likelihood to be vaccinated against pneumococcus and influenza (11–13). Most observational studies of influenza vaccination have not been able to adequately control for this healthy-user effect.
We hypothesized that if the healthy-user effect was responsible for the mortality benefit associated with influenza vaccination seen in observational studies, there should also be a significant mortality benefit present during the off-season. Because serious influenza episodes in the elderly usually lead to admission to hospital for lower respiratory tract infection, we took advantage of a previously assembled large and clinically detailed population-based cohort of consecutive adults admitted to hospital with pneumonia during the off-season to test our hypothesis. Our database was particularly well suited to examination of these questions because we prospectively collected vaccination status on all patients and we had a number of rarely collected potential indicators of the healthy-user effect (13).
The study and data collection methods have been previously described in detail (13–15). Briefly, between 2000 and 2002, all six hospitals within the Capital Health region implemented a previously validated and efficacious clinical pathway for the management of community-acquired pneumonia (CAP) (13–15). Capital Health is the largest integrated health system in Canada, serving over 1 million people within Edmonton, Alberta, Canada. During this period, data were prospectively collected on all 3,415 patients older than 17 years who were admitted to any hospital within the region for CAP (13–15). The only patients excluded from the population-based cohort were those who had tuberculosis or cystic fibrosis; those who were immunocompromised; or those who were pregnant. Then, for purposes of the present study, we excluded patients admitted to hospital during influenza season. National and regional laboratory surveillance data were used to define each influenza season by identifying the first and last occurrence of influenza isolated within the region. To evaluate the effect of influenza vaccination in the off-season, all 1,602 patients admitted to hospital for CAP during the so-defined 10 months of flu season were excluded from analyses of outcomes (i.e., between the months of November 2000 and March 2001 and between January 2002 and May 2002). Thus, all patients admitted to hospital between April to December 2001 and June to November 2002 were included in the present study. The study was approved by the Health Research Ethics Board of the University of Alberta.
Standardized abstraction forms were used by six trained nurses to prospectively collect all data from time of presentation to the emergency room or hospital until either death or patient discharge. Similar to large administrative databases, data collected included age, sex, comorbidities, and prescription medications. Importantly, however, the data ascertained also included clinical characteristics (e.g., documentation of advanced directives, smoking status), immunization history, and premorbid functional status. Functional status was determined for the week before hospital admission, based on patient or proxy interview, and classified as independent in ambulation versus consistently needing some type of walking aid or a wheelchair versus bedridden (13–15). As a derived measure of socioeconomic status, we used postal (zip) codes and neighborhood-level census data to assign a median household income to each patient (15). Last, the Pneumonia Severity Index (PSI), a validated measure of the severity of pneumonia-specific illness at presentation on the basis of three demographic variables, five comorbidities, five physical findings, and seven laboratory tests, was also calculated on all patients (13–15). The PSI is perhaps best considered a reflection of an individual's overall health and risk of death from an episode of pneumonia as opposed to a true marker of “pneumonia severity” because many of the variables and their relative weights within this index predate development of pneumonia. In fact, by their relative scoring, age, sex, and comorbidities are the major drivers of the PSI score and therefore pneumonia severity may be overestimated in older individuals.
Our independent exposure variable of interest was a dichotomous variable representing up-to-date (current) vaccination with influenza vaccine for each specific vaccination cycle. For example, if a patient was hospitalized in September 2000, he or she would be considered up-to-date based on receipt of the 1999–2000 vaccine; however, if the patient was hospitalized in October 2000, he or she would be considered up-to-date based on receipt of the 2000–2001 vaccine because that was the month the new vaccine became available. At admission to hospital, trained research nurses determined vaccination status using multiple complementary methods, which included patient and proxy interview, chart review, and contact with physicians and public health offices (14). These data elements were determined on admission and entered into case-report forms. Therefore, they were collected independently of postadmission outcomes and were inaccessible to the physicians providing patient care (14). Similar methods were used to collect pneumococcal vaccination status, although we could only determine lifetime (“ever”) immunization with 23-valent polysaccharide pneumococcal vaccine (14).
Our primary outcome measure was in-hospital mortality. Secondary outcomes included need for intensive care unit (ICU) admission and a composite endpoint of death or ICU admission; the latter represents a measure of total burden of in-hospital illness for patients with CAP. For patients admitted to the ICU who subsequently died, only their death was counted.
As previously described (13–16), multivariable logistic regression was used to construct a propensity score that reflected the patient's likelihood of receiving influenza vaccination. Because we hypothesized that influenza vaccination was a proxy measure for the healthy-user effect, baseline characteristics, including sociodemographic, clinical (e.g., comorbidities), and other information that might reflect a healthy-user (e.g., statin use, functional status) before hospital admission, were included in the propensity score (13). Of note, the actual PSI score (calculated and derived during hospital admission after vaccination occurred) was not included in the propensity score because it could not conceptually contribute to the vaccination decision. Nonetheless, some individual components of the PSI (age, sex, comorbidities) were among the 36 variables used in the propensity model. The variables and propensity model are available from the first author (D.T.E.) on request. Because coreceipt of influenza vaccine and pneumococcal vaccine was so common, the latter could not be included in propensity score models for influenza vaccination. Then, to ensure that patients vaccinated for influenza were comparable to those who were not vaccinated, to maximize statistical efficiency, and to ensure better control of residual confounding, we used a 5-digit greedy algorithm that matched the propensity score of each patient exposed to influenza vaccination to a patient who had not been vaccinated (14, 17).
Using this matched sample, we then completed a series of multivariable logistic regression analyses to evaluate the association between influenza vaccination and mortality. First, we calculated unadjusted estimates of mortality benefit. Second, we conducted simple adjustment using only age and sex. Third, we undertook what would be considered a “typical” administrative database analysis, adjusting for age, sex, admission from a nursing home, comorbidities, and number of medications. Last, we completed an analysis that incorporated clinical and laboratory data (e.g., PSI according to five risk classes) as well as data that are not routinely available in most administrative databases or retrospective chart reviews and might reflect healthy-user status (e.g., functional status, smoking status, pneumococcal immunizations). Some might question our a priori modeling strategy with respect to adjustment for PSI, a strategy that has been routinely advocated by us (14) as well as others (18, 19) when examining the potential benefits of vaccination. Given that the PSI is such an excellent risk-adjustment tool for pneumonia-related mortality, it seems to us inappropriate not to include it in models examining pneumonia-related mortality regardless of whether or not it is associated with (a potential confounder of) vaccine receipt (20). By analogy, male sex is well known to substantially increase the risk of pneumonia-related mortality (21), although it is not consistently associated with vaccine receipt—it would be difficult if not impossible to justify not including male sex in a multivariable model examining pneumonia mortality. Furthermore, a substantial proportion of the PSI score can be considered, in aggregate, a measure of chronic ill health (e.g., demographics, comorbidities) that was present “before” the episode of pneumonia occurred. Thus, it may also capture by proxy potential attributes related to qualities present in healthy-users, patients who are relatively healthier and more likely to seek out and undertake various screening activities as well as initiate preventive treatments such as use of statins or immunizations (12).
We report unadjusted and adjusted odds ratios (ORs) from unconditional logistic regression models with their respective 95% confidence intervals (95% CIs) and associated P values. All analyses were conducted using SPSS version 14 (SPSS, Inc., Chicago IL) and Stata Intercool version 9 (StataCorp LP, College Station, TX). Similar analytic approaches were undertaken for the outcomes of ICU admission and the composite endpoint of death or ICU admission.
To evaluate the robustness of the results, we also conducted several sensitivity analyses. First, we repeated the analyses with conditional multivariable logistic regression using generalized estimating equation methods to account for the matched nature of the data (22). The results of this analysis differed negligibly from our main analysis and are not otherwise presented. Second, to ensure patients included in the analyses were indeed off-season patients, we further extended the influenza season to include one additional month before and one additional month after the first and last influenza isolates were identified and also excluded the patients admitted during this additional 4 months. Third, we restricted all analyses to patients aged 65 years or older. Fourth, because the magnitude of bias on the estimates of mortality benefits of influenza vaccination may be greatest in the immediate “pre–influenza season” period and wane in the postseason (9), we included an interaction term with time in our models to evaluate potential differences in benefit in the months before the influenza season compared with benefits seen in the months immediately after the influenza season. Fifth, we evaluated the robustness of our results by rerunning analyses using the entire (nonmatched) cohort of patients admitted to hospital during the off season. Finally, as described by others (23), we undertook a thought-experiment to illustrate how prevalent (and how strongly associated with mortality) an unmeasured hypothetical confounder would need to be in the unvaccinated control patients to entirely offset the apparent mortality benefits of influenza vaccination.
During 2000 to 2002, 1,813 patients 17 years or older were admitted to hospital for CAP during the off-season. Overall, admission to hospital occurred evenly throughout the off seasons: 353 (50%) of patients were admitted within 2 months of the start or end of influenza season and 351 (50%) were admitted during the remaining months; similarly, 303 (43%) patients were admitted after influenza season and 401 (57%) were admitted before the next flu season. Influenza vaccination status was documented for all patients: 370 (20%) had been vaccinated. As expected, influenza vaccination rates increased across quintiles of increasing propensity score, representing an increased predicted probability of receiving influenza vaccination: 4% versus 13% versus 21% versus 27% versus 35% of patients (P value for trend < 0.001).
Using the propensity score, 352 of 370 influenza-vaccinated patients were matched to 352 unvaccinated patients, representing a 95% success rate (i.e., no match was available for 18 vaccinated subjects). The median age of the matched sample was 78 years, 54% were male, 28% were from nursing homes, 38% had chronic obstructive pulmonary disease, and 29% had severe (PSI class V) pneumonia. Patients were well matched and there were no statistically significant or clinically important differences in patient characteristics according to influenza vaccination status—with the exception of pneumococcal vaccination which was not part of the propensity score model and which was noted significantly more often among those vaccinated for influenza (Table 1).
Variables | Not Vaccinated (n = 352) | Vaccinated (n = 352) | P Value | |||
|---|---|---|---|---|---|---|
| Age, yr (mean ± SD) | 76.3 ± 12.7 | 75.9 ± 12.6 | 0.70 | |||
| Age ≥65 yr | 295 (84) | 301 (86) | 0.53 | |||
| Male | 200 (57) | 181 (51) | 0.15 | |||
| Aboriginal origin | 7 (2) | 5 (1) | 0.56 | |||
| Median household income (thousands of dollars) | ||||||
| <30 | 28 (8) | 21 (6) | 0.65 | |||
| 30–39 | 73 (21) | 83 (24) | ||||
| 40–49 | 128 (36) | 127 (36) | ||||
| >50 | 123 (35) | 121 (34) | ||||
| Previous comorbidities | ||||||
| Stroke | 37 (11) | 42 (12) | 0.55 | |||
| Ischemic heart disease | 130 (37) | 133 (38) | 0.82 | |||
| Heart failure | 75 (21) | 79 (22) | 0.72 | |||
| Cancer | 70 (20) | 58 (16) | 0.24 | |||
| Liver disease | 3 (1) | 6 (2) | 0.31 | |||
| Renal disease | 60 (17) | 66 (19) | 0.56 | |||
| COPD | 132 (38) | 135 (38) | 0.82 | |||
| Neuropsychiatric illness | 77 (22) | 72 (20) | 0.65 | |||
| Pneumococcal vaccination | 18 (5) | 241 (68) | <0.001 | |||
| Statin therapy | 45 (13) | 49 (14) | 0.66 | |||
| Five or more medications | 85 (24) | 77 (22) | 0.47 | |||
| Smoking status | ||||||
| Nonsmoker | 154 (44) | 151 (43) | 0.89 | |||
| Former smoker | 141 (40) | 147 (42) | ||||
| Current smoker | 57 (16) | 54 (15) | ||||
| Nursing home resident | 98 (28) | 99 (28) | 0.93 | |||
| Advance directive | 67 (19) | 59 (17) | 0.43 | |||
| Premorbid functional status | ||||||
| Independent mobility | 300 (85) | 306 (87) | 0.75 | |||
| Wheelchair/prosthesis | 39 (11) | 36 (10) | ||||
| Bedridden | 13 (4) | 10 (3) | ||||
| Pneumonia Severity Index | ||||||
| Class I or II | 26 (7) | 31 (9) | 0.10 | |||
| Class III | 67 (19) | 66 (19) | ||||
| Class IV | 144 (41) | 168 (48) | ||||
| Class V | 115 (33) | 87 (25) | ||||
Overall, 81 (12%) patients died in hospital. In unadjusted analyses, patients receiving influenza vaccination were significantly less likely to die compared with nonvaccinated patients (28 [8%] vs. 53 [15%]; OR, 0.49; 95% CI, 0.30–0.79; P = 0.004) (Figure 1). Using sequential multivariable models, the adjusted OR for the use of influenza vaccination and in-hospital mortality changed to 0.48 (95% CI, 0.30–0.79; P = 0.003) after adjustment for age and sex to 0.45 (95% CI, 0.27–0.76; P = 0.003) after the typical adjustments possible in large administrative databases (Figure 2). Subsequent models adjusting for clinical data not routinely available in administrative datasets yielded substantially different results. Inclusion of detailed clinical and laboratory data decreased the adjusted OR for death to 0.52 (95% CI, 0.30–0.90; P = 0.02) and, with full adjustment for all available data including functional status, need for an advance directive, pneumococcal immunizations, and socioeconomic status, yielded a much smaller and nonsignificant adjusted OR for death of 0.81 (95% CI, 0.35–1.85; P = 0.61; Figure 2 and Table 2). Similar results were observed for a more parsimonious model that included only age, sex, severity of pneumonia, functional status, smoking status, need for an advanced directive, and pneumococcal immunizations (OR, 0.79; 95% CI, 0.35–1.75; P = 0.56).
Figure 1. Outcomes of 704 propensity score–matched patients hospitalized for pneumonia during the off-season, according to influenza vaccination status. ICU = intensive care unit.
[More] [Minimize]
Figure 2. Sequentially adjusted models of the association between influenza vaccination and in-hospital mortality in 704 propensity score–matched patients admitted to hospital with pneumonia during the off-season. PSI = Pneumonia Severity Index.
[More] [Minimize]Variables | Adjusted OR* | 95% CI | P Value | |||
|---|---|---|---|---|---|---|
| Previous comorbidities | ||||||
| Cancer | 2.16 | 1.13–4.15 | 0.02 | |||
| Premorbid functional status | ||||||
| Independent mobility | 1.0 (reference) | |||||
| Wheelchair or prosthesis | 2.39 | 1.08–5.32 | 0.032 | |||
| Bedridden | 2.53 | 0.75–8.59 | 0.136 | |||
| Advanced directive | 2.93 | 1.42–6.02 | 0.004 | |||
| Influenza vaccination | 0.81 | 0.35–1.85 | 0.61 | |||
Compared with those who were not vaccinated, patients who received influenza vaccine were less likely to be admitted to the ICU (Figure 1), regardless of model adjustment (unadjusted OR, 0.08; 95% CI, 0.02–0.26; P < 0.001; fully adjusted model OR, 0.17; 95% CI 0.04–0.71; P = 0.014). For the composite endpoint of ICU admission or death (Figure 1), patients receiving influenza vaccination were significantly less likely to be admitted to the ICU or die compared with nonvaccinated patients (unadjusted OR, 0.33; 95% CI, 0.21–0.52; P < 0.001). Fully adjusted models still suggest a trend toward a reduction in this composite endpoint for patients receiving influenza vaccination compared with those not vaccinated (OR, 0.50; 95% CI, 0.25–1.00; P = 0.05).
Further exclusion of another 208 patients admitted to hospital within 1 month before and 1 month after the influenza seasons that we initially defined yielded a smaller study cohort of 496 patients. But even this did not materially influence our findings with respect to in-hospital mortality (unadjusted, 38 [15%] vs. 21 [9%]; OR, 0.51; 95% CI, 0.29–0.90; P = 0.02; and fully adjusted model OR, 0.78; 95% CI, 0.29–2.16; P = 0.64). For analyses restricted to only patients 65 years or older (n = 602), the ORs for the use of influenza vaccination and in-hospital mortality were unchanged (unadjusted, 52 [17%] vs. 26 [9%]; OR, 0.45; 95% CI, 0.27–0.75; P = 0.002; and fully adjusted model OR, 0.81; 95% CI, 0.35–1.92; P = 0.64). Similarly, no difference in the effect of influenza vaccination on mortality over time was observed in patients admitted to hospital before or after influenza season (P = 0.1 for interaction). Analyses based on the entire nonmatched cohort (n = 1,813) did not materially affect our conclusions with respect to vaccination: the fully adjusted OR for mortality was 0.81 (95% CI, 0.42–1.55; P = 0.52).
Our final sensitivity analyses related to an unmeasured and hypothetical single confounder in the unvaccinated control group. A confounding variable present in only 10% of control subjects that increased the relative risk of death by 3 or more would be entirely sufficient to explain the adjusted (albeit nonsignificant) 19% relative mortality reduction we observed. Similarly, an unmeasured confounder present in 20% of control subjects would only need to increase the risk of death by 2 to abolish the residual 19% mortality benefit of influenza vaccine seen in our analyses. In the overall parent dataset, differences in variables such as impaired functional status and need for an advance directive approach these hypothetical boundaries with respect to differential prevalence and risk of mortality.
We observed a large mortality benefit in patients who had received influenza vaccination before their hospitalization for pneumonia, even though it is extremely unlikely they had an influenza-related illness. Specifically, in unadjusted analyses, we found a statistically significant 51% relative reduction in all-cause mortality during hospitalization, from 15 to 8%, for those who were previously vaccinated. However, with progressively more careful adjustment for disease severity and measures of the healthy-user effect, the estimated benefit of influenza vaccination was markedly attenuated (19% reduction in mortality) and no longer statistically significant (P = 0.61). Our results are most consistent with residual and difficult-to-correct confounding.
The initial (unadjusted) 51% reduction in all-cause mortality that we observed is consistent with what would be expected on the basis of previous observational studies that suggested mortality reduction on the order of 48 to 52% with influenza vaccination (3–8). To our knowledge, there have only been two studies that have previously specifically examined outcomes related to influenza vaccination in patients hospitalized with pneumonia (18, 19). In the first study, Herzog and colleagues studied 12,566 Medicare beneficiaries hospitalized with pneumonia during one influenza season; 61% of patients had unknown vaccination status, but most likely were not vaccinated (19). Thirty-day mortality was 24% among those who had been vaccinated before admission versus 32% among those not vaccinated (adjusted 35% reduction in mortality with vaccination, P < 0.001) (19). In the second study, Spaude and coworkers examined 17,393 adults hospitalized during four consecutive influenza seasons; 47% of patients had unknown vaccination status (18). In-hospital mortality was 7%, and influenza vaccination was associated with an adjusted 43% mortality reduction (P < 0.001) (18). Although both of these studies were rigorously conducted, they share three common limitations (18, 19): (1) ascertainment of vaccination status was incomplete, (2) data from the influenza off season were excluded from all analyses, and (3) neither study had information available with respect to functional status or other indicators of the healthy user. These last two limitations are common to almost all published observational studies of the effectiveness of influenza vaccination (5–10).
Given that we were able, to some degree, to overcome a number of limitations endemic to previous studies, we believe that our results empirically demonstrate that the mortality benefits of influenza vaccination may have been largely overestimated. That is, we observed that a striking and highly significant association became attenuated and nonsignificant after accounting for disease severity and measures of functional status. The same confounding by the healthy-user effect has been implicated in observational studies that found reductions in the risk of myocardial infarction or stroke with hormone therapy (12), as well as the noncardiovascular “pleiotropic effects” of preventive medications such as statin therapy (13). Moreover, even previous studies of the mortality benefits of influenza vaccination in elderly patients have suggested potential confounding by the healthy-user effect that cannot be adjusted for using routine administrative data, possibility due to the fact that various groupings of administrative data are relatively poor markers for disease severity or overall frailty (9, 11). Alternately, because our estimate of mortality reduction with influenza vaccination is still consistent with a benefit (adjusted OR, 0.81; 95% CI, 0.35–1.85; P = 0.61), our data could also be interpreted as demonstrating an unanticipated pleiotropic benefit of influenza vaccination. It has been posited that even in patients with bacterial pneumonia, influenza vaccination might reduce the severity of illness by one of two mechanisms. Vaccination could improve outcomes either through augmentation of innate immunity (24) or by virtue of the fact that vaccination prevents influenza infection (which can itself predispose individuals to serious bacterial infections with Streptococcus pneumoniae or Staphylococcus aureus) and leads to a shift in the microbial spectrum toward far less virulent pathogens (18). Despite such speculations, we believe that our results are most consistent with difficult-to-control confounding.
This observational study has at least four major limitations. First, because we studied only patients hospitalized with pneumonia, our findings may not necessarily be generalized to broader population-based studies, which tend to include much healthier and lower risk patients who never become ill enough to require admission. Although this might be expected to lead to differences in baseline risk and absolute mortality rates, there is no reason to believe that the relative benefits of vaccination should be different in our study population (25, 26). Second, we did not have any measures of immunity and very limited data regarding etiology, and so it is possible that some patients we studied did actually have influenza infection. However, we restricted analyses to time periods when influenza was not circulating in the community and we conducted a sensitivity analysis that extended the beginning and end of influenza season by 1 month in each direction and this did not materially affect our results. Third, we only have information related to outcomes during hospitalization, although most of the morbidity and eventual mortality associated with influenza infection in the elderly occurs during hospital admission (5, 7, 8, 18, 19). Last, although we had more clinical data than most studies, our measures of the healthy-user effect were fairly rudimentary and limited. Ideally, we would also have had more detailed information about socioeconomic status; health care visits before illness; habits related to diet, alcohol intake, regular exercise, and health screening and promotion activities; and, especially, adherence to prescribed medications (12, 13).
In conclusion, the fact that we were able to find such a large and fairly robust mortality benefit with influenza vaccination in patients hospitalized with pneumonia during the off-season implies that studies that have restricted their analyses to the influenza season have overestimated the potential mortality benefit of vaccination. If this is the case, it has important implications for researchers (e.g., need for better vaccines and need for better observational study designs) and for policy makers (e.g., different guideline recommendations and more rational resource allocation strategies). Many initiatives are underway to increase rates of annual influenza vaccination in the elderly because of the putative mortality benefits, despite increasing concerns about the evidence underpinning these well-intended recommendations (27). It would seem prudent to us that before the implementation of such overarching recommendations, that higher quality evidence of benefit be generated first. Furthermore, because we were also unable to control completely for the healthy-user effect, we hope our findings might help tilt the balance toward clinical equipoise and permit much needed and adequately powered randomized trials of influenza vaccine in the elderly to take place.
For their dedication and diligence, the authors thank the Community-Acquired Pneumonia Pathway research nurses and the staff of EPICORE (University of Alberta) for data management services.
| 1. | Thompson WW, Shay DK, Weintraub E, Brammer L, Bridges CB, Cox NJ, Fukuda K. Influenza-associated hospitalizations in the United States. JAMA 2004;292:1333–1340. |
| 2. | Thompson WW, Shay DK, Weintraub E, Brammer L, Cox N, Anderson LJ, Fukuda K. Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA 2003;289:179–186. |
| 3. | Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet 2005;366:1165–1174. |
| 4. | Jefferson T, Di Pietrantonj C. Inactivated influenza vaccines in the elderly: are you sure? Lancet 2007;370:1199–1200. |
| 5. | Simonsen L, Taylor RJ, Viboud C, Miller MA, Jackson LA. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infect Dis 2007;7:658–666. |
| 6. | Nichol KL, Nordin JD, Nelson DB, Mullooly JP, Hak E. Effectiveness of influenza vaccine in the community-dwelling elderly. N Engl J Med 2007;357:1373–1381. |
| 7. | Christenson B, Lundbergh P, Hedlund J, Ortqvist A. Effects of a large-scale intervention with influenza and 23-valent pneumococcal vaccines in adults aged 65 years or older: a prospective study. Lancet 2001;357:1008–1011. |
| 8. | Fedson DS, Wajda A, Nicol JP, Hammond GW, Kaiser DL, Roos LL. Clinical effectiveness of influenza vaccination in Manitoba. JAMA 1993;270:1956–1961. |
| 9. | Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. Int J Epidemiol 2006;35:337–344. |
| 10. | Mangtani P, Cumberland P, Hodgson CR, Roberts JA, Cutts FT, Hall AJ. A cohort study of the effectiveness of influenza vaccine in older people, performed using the UK general practice research database. J Infect Dis 2004;190:1–10. |
| 11. | Jackson LA, Nelson JC, Benson P, Neuzil KM, Reid RJ, Psaty BM, Heckbert SR, Larson EB, Weiss NS. Functional status is a confounder of the association of influenza vaccine and risk of all cause mortality in seniors. Int J Epidemiol 2006;35:345–352. |
| 12. | Brookhart MA, Patrick AR, Dormuth C, Avorn J, Shrank W, Cadarette SM, Solomon DH. Adherence to lipid-lowering therapy and the use of preventive health services: an investigation of the healthy user effect. Am J Epidemiol 2007;166:348–354. |
| 13. | Majumdar SR, McAlister FA, Eurich DT, Padwal RS, Marrie TJ. Statins and outcomes in patients admitted to hospital with community acquired pneumonia: population based prospective cohort study. BMJ 2006;333:999. |
| 14. | Johnstone J, Marrie TJ, Eurich DT, Majumdar SR. Effect of pneumococcal vaccination in hospitalized adults with community-acquired pneumonia. Arch Intern Med 2007;167:1938–1943. |
| 15. | Marrie TJ, Wu L. Factors influencing in-hospital mortality in community-acquired pneumonia: a prospective study of patients not initially admitted to the ICU. Chest 2005;127:1260–1270. |
| 16. | Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70:41–55. |
| 17. | Parsons LS. Reducing bias in a propensity score matched-pair sample using greedy matching techniques (Internet) (accessed 2007 October 30). Available from: http://www2.sas.com/proceedings/sugi26/p214-26.pdf |
| 18. | Spaude KA, Abrutyn E, Kirchner C, Kim A, Daley J, Fisman DN. Influenza vaccination and risk of mortality among adults hospitalized with community-acquired pneumonia. Arch Intern Med 2007;167:53–59. |
| 19. | Herzog NS, Bratzler DW, Houck PM, Jiang H, Nsa W, Shook C, Weingarten SR. Effects of previous influenza vaccination on subsequent readmission and mortality in elderly patients hospitalized with pneumonia. Am J Med 2003;115:454–461. |
| 20. | Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol 1996;25:1107–1116. |
| 21. | Kaplan V, Clermont G, Griffin MF, Kasal J, Watson RS, Linde-Zwirble WT, Angus DC. Pneumonia: still the old man's friend? Arch Intern Med 2003;163:317–323. |
| 22. | Austin PC. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med 2007;23:2037–2049. |
| 23. | Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf 2006;15:291–303. |
| 24. | Weinberger B, Herndler-Brandstetter D, Schwanninger A, Weiskopf D, Grubeck-Loebenstein B. Vaccines: biology of immune responses to vaccines in elderly persons. Clin Infect Dis 2008;46:1078–1084. |
| 25. | Furukawa TA, Guyatt GH, Griffith LE. Can we individualize the “number needed to treat”? An empirical study of summary effect measures in meta-analyses. Int J Epidemiol 2002;31:72–76. |
| 26. | McAlister FA. Commentary: relative treatment effects are consistent across the spectrum of underlying risks … usually. Int J Epidemiol 2002;31:76–77. |
| 27. | Fiore AE, Shay DK, Haber P, Iskander JK, Uyeki TM, Mootrey G, Bresee JS, Cox NJ; Advisory Committee on Immunization Practices (ACIP), Centers for Disease Control and Prevention (CDC). Prevention and control of influenza. Recommendations of the Advisory Committee on Immunization Practices (ACIP), 2007. MMWR Recomm Rep 2007;56:1–54. |
