Numerous nonrandomized studies have reported that older adults who receive influenza vaccine are much less likely to die during influenza season than their nonvaccinated peers (1–4). Although these findings have been widely interpreted as evidence that influenza vaccination reduces mortality in older adults, there is increasing recognition of an alternate explanation—that the apparent vaccine effect is due, at least in part, to preferential selection of vaccination by healthier persons who are at relatively low risk of death during the winter months (5). In general, questions of causality versus bias in nonrandomized studies are very difficult to resolve. In the case of influenza vaccine evaluations, however, we have a unique and powerful tool. The seasonality of influenza infection creates natural control time periods, during which no true vaccine effect is expected. Because influenza vaccine cannot prevent influenza-related complications when influenza is not circulating, in the absence of bias there should be no difference in the risk of death between vaccinated and unvaccinated persons outside of influenza seasons. If vaccinated persons are at lower risk of death than unvaccinated persons outside of influenza season, this demonstrates that relatively healthier persons were more likely to select vaccination.

Eurich and colleagues (see pages 527–533 in this issue) have used this control period approach to estimate the contribution of bias to estimates of the association between influenza vaccination and risk of death among adults hospitalized for pneumonia (6). Two previous studies have reported that influenza vaccination was associated with a 35 to 47% reduction in risk of death among adults hospitalized with pneumonia during influenza season and concluded that the findings were consistent with a protective effect of influenza vaccination against pneumonia mortality (3, 4). In their evaluation of adults in Edmonton, Canada, who were hospitalized for pneumonia outside of influenza season, Eurich and colleagues found that persons who had received influenza vaccine were approximately 50% less likely to die than those who were not vaccinated.

The reduction in risk of death found outside of influenza season in the Edmonton study directly demonstrates the presence of bias. We have performed similar analyses in the Group Health population and found that, among seniors hospitalized for pneumonia, influenza vaccination was associated with a 35% lower risk of 30-day mortality, in both analyses of persons hospitalized during and outside of influenza season (7). We have also used the control period approach to show that the lower risks of all-cause mortality and pneumonia hospitalization consistently observed in studies comparing vaccinated and unvaccinated community-dwelling seniors during influenza season are largely, or perhaps entirely, due to bias, because differences of similar magnitude are found during noninfluenza periods (8). In those analyses, we found that the influence of bias diminishes with time and so the greatest differences are found in analyses of control periods before the onset of influenza season. The results of studies using the control period approach clearly indicate that evaluations of noninfluenza periods, and particularly periods prior to the onset of influenza season, are an essential component of influenza vaccine effectiveness evaluations. Without the control period results, differences in risk between vaccinated and unvaccinated persons observed during influenza season cannot be reliably interpreted.

In the Edmonton study, Eurich and colleagues also found that adjustment for general indicators of comorbidity, such as the presence or absence of chronic conditions and number of prescription medications, often assessed in studies relying on administrative data sources did not substantively alter the estimate of the association. In contrast, adjustment for prospectively collected frailty indicators, such as ability to ambulate and the need for an advanced directive, did reduce the influence of bias, resulting in an adjusted odds ratio of 0.81 (95% confidence interval, 0.35–1.85). We have also found that unvaccinated seniors are more likely to have functional status limitations, such as requiring assistance to ambulate or bathe, and that those indicators of frailty are associated with an increased risk of death and so are important confounders of the association of influenza vaccination and mortality in seniors (9).

These analyses therefore provide another message: reducing the influence of bias in influenza vaccine studies requires information beyond that available from administration data sources. Furthermore, even with more rigorous methods, it may not be possible to characterize individuals accurately enough to allow complete adjustment for the important differences between the vaccinated and unvaccinated groups. Eurich and coworkers found that adjustment for prospectively collected indicators did not fully control for bias, as evidenced by the adjusted odds ratio of 0.81. Studies of persons assigned to placebo in randomized controlled trials have provided similar cautionary notes regarding the limitations of methods of adjustment (10, 11). In those studies, large differences in mortality have been reported between participants who adhered to their placebo treatment and those who did not, and there is evidence that this adherence bias cannot be reduced even by adjustment for the numerous detailed health indicators prospectively collected at the onset of the clinical trial (10).

The signs point to a hard road ahead in the quest for less biased estimates of influenza vaccine effectiveness in older adults. Eurich and colleagues call for randomized trials of influenza vaccine in the elderly. If such trials include the groups at highest risk of influenza-related morbidity and mortality, namely the oldest old and those with serious underlying health conditions, then the finding of a reduction in risk of influenza infection would provide convincing evidence that vaccination could impact the burden of those serious outcomes. Placebo-controlled trials are likely not feasible in the United States or other countries with policies for routine vaccination of seniors, however, and sources of funding for evaluations in other regions are not obvious. In the absence of information from clinical trials among the high-risk elderly, valuable knowledge may potentially still be gained from nonrandomized studies that incorporate the best available methods to assess and adjust for important confounders, provided that they utilize noninfluenza periods to gauge the success of those methods in reducing the influence of bias.

References

Section:

ChooseTop of pageReferences <<CITING ARTICLES