Abstract
Idiopathic pulmonary fibrosis causes progressive morbidity and has a worldwide incidence that is increasing. There are a number of promising therapies, one of which has been approved in Europe, parts of Asia, and India, and others that are at various stages of development. Despite this, there continues to be debate about the most appropriate clinical endpoint that should be used in future randomized controlled clinical trials of novel therapies in idiopathic pulmonary fibrosis. In a recent Pulmonary Perspective in this journal, the case for the use of a variety of clinical endpoints was analyzed, and the article concluded that FVC, the endpoint most commonly used recently and in ongoing studies, was not an appropriate option. In this Pulmonary Perspective we present a counterpoint in which we explore the basis on which this conclusion is drawn and present data that strongly and logically support the use of FVC as a valid and robust measure that fulfils the criteria for an ideal clinical endpoint and that is meaningful to patient and clinician alike.
Idiopathic pulmonary fibrosis (IPF) is an orphan disease and is the most common and lethal of all the idiopathic interstitial pneumonias. It has a worldwide prevalence estimate of at least 5 million people (1), with evidence that the incidence is increasing (2). Most individuals experience relentless disease progression with worsening functional limitations, increasing morbidity, and reliance on the assistance of others to live out their limited lifespan (3). Referral to tertiary lung centers where the goal is often to be listed for lung transplantation is an option; unfortunately, only a very small subset of patients are candidates. Frequently, however, there is a delay in diagnosis that results in death before transplantation. The earlier the diagnosis is attained, the better the outcomes. Indeed, recent studies have shown that outcome is better for those individuals whose FVC is more preserved at baseline (4–6).
To find ways of halting this relentless progression, the last decade has witnessed a flurry of randomized clinical trials (RCT) involving novel therapies for IPF (7). This is in marked contrast to the previous decade, in which placebo-controlled trials were not attempted, largely because of the belief based on underpowered studies that corticosteroids with or without immunosuppressants were effective (8). To date, most RCT have had negative outcomes, with the notable exceptions being the pirfenidone trials conducted in Japan and the rest of the world that together consisted of four phase II/III studies (9–11). In a metaanalysis of these studies, Spagnolo and colleagues, using highly robust and rigorous independent statistical analysis under the aegis of the Cochrane collaboration, concluded that “…pirfenidone appears to improve progression-free survival and, to a lesser extent, pulmonary function in patients with idiopathic pulmonary fibrosis” (12). Importantly, and in support of the concept that the outcomes were not spurious, the magnitude of change in the primary outcome measures, vital capacity or FVC, were remarkably similar in the two positive phase III studies—a reduction in the risk of disease progression by 36%.
The primary endpoint from these positive studies used a measure of lung function, a robust, reproducible, and highly relevant measurement when studying a disease whose archetypal pathophysiology is a scarring process that reduces the size of the lungs (13). Importantly, more formal analysis of this measurement, using data from two large negative RCT, found FVC to be reliable, based on the stability of percent predicted FVC values between the study screening visit (Day −28 to Day −1) and the baseline visit (Day 1); valid, based on relationships between percent predicted FVC and the following parameters at the same visit: percent predicted diffusing capacity of carbon monoxide, alveolar–arterial gradient, and indices of patient symptoms and quality of life; and responsive, based on examining the relationship between change in percent predicted FVC over 24 weeks and 1-year risk of death (4). In this regard, it was shown that a 10% or greater absolute reduction in FVC over a 6-month period was associated with an almost fivefold increase in the risk of death over the subsequent 1-year period. Furthermore, a decline of only 5 to 10% was also predictive, with a greater than twofold increased risk of death over the subsequent year (4). These data have been reproduced by others (14), and more recently it has been reported that even a relative fall in FVC by 10% (e.g., from 60 to 54%) as opposed to an absolute 10% decline (e.g., from 60 to 50%) was found to be just as predictive of a worse outcome (15). Collectively, these studies demonstrate that FVC represents a robust clinical measurement that is a standard for the determination of progression of disease in day to day clinical management and that categorical decrements of FVC are powerful predictors of mortality. We therefore believe that maintaining lung function for an extended period of time should be a laudable goal, a view likely shared by most treating physicians and patients alike.
The medical and patient communities should feel optimistic that a good start has been made that potentially heralds an era of new drug development that could result in even more efficacious therapies for this dreaded disease that causes relentless progression and morbidity. A significant cloud has been cast over the ray of hope by the publication of a Pulmonary Perspective by Raghu and colleagues in a recent issue of the Journal (16).
In this Pulmonary Perspective, the authors apply a process of analysis of “clinically meaningful” endpoints in phase III RCT and conclude: “we believe the endpoints that best meet these criteria [how a patient feels, functions, or survives] are all-cause mortality and all-cause non-elective hospitalization” (16). The authors are very critical and dismissive of other endpoints that have been widely used in recent, successful RCT. They continue: “there are currently no validated surrogate endpoints” (16). They make no suggestions of how to design RCT using their preferred endpoints, do not discuss the very significant obstacles to conducting clinical trials using these endpoints, and make no recommendations on how surrogate endpoints can be validated—not a trivial task in IPF. Furthermore, they cast doubt on the value of FVC as a sensible, appropriate endpoint for trials of novel therapy.
This document is more destructive than constructive and risks leaving the community of patients with IPF disenfranchised, given that the pharmaceutical industry would be challenged to support future studies if these sorts of constraints on endpoints were to be widely accepted by regulatory authorities. Raghu and colleagues (16) put great emphasis on the concept of “clinically meaningful endpoints” and the absence of validation for any surrogate endpoints in IPF while attempting to lay the foundation for their recommendation to use only all-cause mortality and all-cause nonelective hospitalization as clinical trial endpoints. We believe that this Pulmonary Perspective is an example of process at the expense of progress and wish in our counterpoint to highlight how the basis of the authors’ interpretation of the definition of a clinical endpoint is open to argument and that their consequent judgements of endpoint acceptability are questionable. Specifically, the stance on the acceptability of FVC as a valid clinical endpoint prompts us to urge the readership to view the conclusions drawn by the authors with some caution.
What is really meant by “clinically meaningful”? Raghu and colleagues (16) define this term as being an endpoint that directly measures how a patient feels (symptoms), functions (the ability to perform functions in daily life), or survives. This definition is “validated” by reference to an article that was the product of deliberations by a working group of the National Institutes of Health (NIH) Director’s Initiative on Biomarkers and Surrogate Endpoints (17). In this well-argued document that focuses mainly on biomarkers and surrogate markers, the authors wished to avoid misunderstanding by defining a number of terms that have historically been confusing and overlapping. Of these definitions, they suggest “clinical endpoints [note, not “clinically meaningful endpoints”] are distinct measurements or analyses of disease characteristics observed in a study or a clinical trial that reflect the effect of a therapeutic intervention. Clinical endpoints are the most credible characteristics used in the assessment of the benefits and risks of a therapeutic intervention in randomized clinical trials” (17). In this regard, FVC is clearly acceptable as a clinical endpoint using these defining recommendations because it is a measure of function that we would argue is a “distinct measurement” of the characteristics of IPF. To argue differently is specious. We would also argue that it is “meaningful,” being directly relevant to the goals of patients.
Raghu and colleagues continue: “The ideal clinically meaningful endpoint should also be well defined, reliable, measurable, interpretable, and sensitive to the effects of the intervention” (16). To recommend that all-cause hospitalization is an ideal endpoint using their own description of the ideal endpoint does not bear scrutiny. The reasons that patients are hospitalized are not universally defined and are broad, with wide variability in clinical practice, not only internationally but also regionally and even locally. The reason for hospitalizations can be difficult to discern, especially if patients are admitted to outside hospitals. All-cause hospitalization is an imperfect endpoint for IPF because patients with extensive fibrosis and advanced physiologic impairment may well require hospitalization due to processes that are unrelated to the biology of progressive fibrosis. Many times hospitalization will reflect differences in end-of-life clinical management and not disease progression. It makes little sense to think that an antifibrotic drug will impact hospitalizations in patients with advanced disease. Indeed, hospitalization as a viable endpoint has never been tested or validated prospectively in an IPF clinical trial, the same critique that the authors bring to bear on FVC.
To argue that all-cause mortality is also an ideal or recommended endpoint is also questionable given the impracticality of developing a mortality study in mild to moderate IPF disease. Specifically, it is widely acknowledged that such studies would be prolonged, with patient retention problematic, and would require large numbers of patients, rendering it prohibitively expensive and impractical. In addition, when dealing with an elderly patient population followed over the course of years, many may die with their IPF rather than from their IPF, diminishing the specificity of all-cause mortality as an endpoint and, importantly, possibly underestimating the efficacy of a therapy that is not designed to impact on causes of death that are due to events other than disease progression. One of the many hurdles for IPF studies is patient recruitment and patient retention: the longer the study, the greater the likelihood of dropouts, which could jeopardize the integrity of any well-designed study. The problem of powering such a study is underscored by a recent phase III RCT that reported a placebo mortality of only 2% in a prematurely discontinued study (18). One needs to question how big a trial would be needed to demonstrate a reduction in that magnitude of event rate.
It could be argued that smaller studies would be practical if patients who had more severe disease were enrolled. We would suggest that such a population of patients does not represent the disease as a whole and that patients with advanced disease behave quite differently from those with mild to moderate disease, often dying of pulmonary vascular complications rather than progressive fibrosis. This may explain the results from the recently published RCT of sildenafil, which targeted advanced IPF and was enriched for patients with complicating pulmonary hypertension by virtue of the inclusion criterion of a diffusing capacity of carbon monoxide less than 35% predicted (19). The short 12-week duration of this RCT and the properties of the agent studied suggest that any active drug benefits reported were due to factors other than an antifibrotic effect. Disease behavior at end stage cannot be used as a template for the design of clinical trials of patients with less advanced disease, in whom disease stabilization should be regarded as a successful outcome.
It could also be argued that the value of FVC as an endpoint is lessened because in neither of two recently reported trials of therapy in IPF was the rate of decline in FVC between the active and placebo limbs different even though mortality rates did differ (18, 20). We suggest that because the mortality rates in these studies were higher in the active limbs, it is possible that the worse outcome in these studies might not be related simply to disease progression and would not therefore necessarily be preceded by a decline in FVC.
Taken together, therefore, we believe that change in FVC meets all the criteria of a clinical endpoint for IPF studies as defined by the NIH working group document, is meaningful in the context of assessing change in a measure of how scarred the lungs are, and meets all the defining features of an ideal endpoint; in short “the ideal clinically meaningful endpoint.” Is this not what we all want?
The NIH working group defines a surrogate endpoint as: “a biomarker that is intended to substitute for a clinical endpoint. A surrogate endpoint is expected to predict clinical benefit (or harm or lack of benefit or harm) based on epidemiologic, therapeutic, pathophysiologic, or other scientific evidence” (17). In some contexts, change in a surrogate biomarker status alone has been acceptable as an index of drug efficacy even in the absence of a direct demonstration or prediction of an effect on clinical outcomes. A good example is therapy for HIV, in which plasma viral load and CD4+ T-cell counts were used as substitutes for clinical outcome (21–23). In diseases, including IPF, that can have lethal outcomes, it seems that a measurement that predicts outcome is a very acceptable substitute measurement on which to base judgements of drug efficacy. Perhaps the closest analogy in this regard comes from the field of pulmonary arterial hypertension, in which at least five of the seven currently available agents in the United States have been approved based on change in the 6-minute walk test distance (24). It is widely recognized that these agents have helped to modify the natural history of this equally devastating disease.
However, these theoretical arguments aside, we actually suggest that FVC is not a substitute for a clinical endpoint, as it is itself a clinical endpoint as defined by the NIH working group and is not therefore a surrogate at all. In the interest of the academic exercise, however, let us explore whether or not FVC could be regarded as a reasonable choice of measurement that has an impact on likely mortality (i.e., an endpoint, the change in which would “…predict clinical benefit” [17]). Its measurement characteristics are impeccable. Categorical changes in percent predicted FVC values are consistently associated with changes in other clinically important outcomes and are strong predictors of mortality (4, 5). It is reasonable to presume, therefore, that preventing such categorical FVC change is good and has an impact on rate of survival. In other words it “predicts clinical benefit.” To have to prove or “validate” this for an orphan disease like IPF is a somewhat pedantic suggestion given this evidence, quite difficult in practice, and yet to be shown in any study. In this regard, for a drug effect on FVC to be conclusively proven to accurately predict mortality would require it to be shown that all individuals whose FVC reduction was due to drug (and would not have happened during a quiescent phase of disease without drug) had a better outcome. To simply show that a group prevention of a categorical reduction in FVC by active drug was associated with a reduced mortality in the same group is no proof of them being causally related and is false logic. Furthermore, it is unrealistic to mandate all-cause mortality as the endpoint required for regulatory approval in an orphan disease with enormous heterogeneity and variable rates of progression. It is disingenuous of the authors to cite the two studies on which they base their assertion of the feasibility of mortality studies in IPF (18, 25). Specifically, one published RCT (IFN-γ; INSPIRE) did not show any difference in mortality between the two groups in a study that was powered to demonstrate a 50% reduction in risk of mortality, an extremely large and rarely observed magnitude of treatment effect (25). Paradoxically, the only reason there was a difference in mortality in the second (PANTHER-IPF) study was because of a profound increase in mortality rate in the active treatment limb (18). It is somewhat ironic that prednisone/azathioprine, previously touted and widely accepted as standard therapy for IPF, based on a small study with mortality as the endpoint (by Raghu and colleagues) (26), was subsequently suggested to be more harmful in the context of a larger study in which change in the FVC was the primary endpoint!
Given the overwhelming evidence of the importance of FVC in reflecting the clinical condition of the patient with IPF and predicting outcome, why would one choose to stack the deck against identifying novel therapies? We argue that it is fundamentally flawed and unfair to patients with IPF to hold their disease(s) to a threshold that is not feasible.
In the Pulmonary Perspective from Raghu and colleagues, which sets out to explore all endpoints that have been used or mooted in trials of novel therapy for IPF, a formal process has been followed that takes each endpoint, analyzes pros and cons of the use of that endpoint, and comes to a decision on each (16). The facade looks balanced, but we believe the process was flawed by the authors’ questionable interpretation of a document on which many of their analyses are based. More importantly, their Perspective is at risk of obstructing the emergence of any other therapies for IPF by making suggestions that, if followed, would present totally insuperable barriers for further drug development. The authors appear to particularly endorse mortality trials, which, if broadly accepted as the status quo, will make it almost impossible to design phase II trials that can reasonably inform phase III trials. From a practical standpoint, it is highly unlikely that any sponsor will proceed directly to phase III trials using mortality as an endpoint that will require very large numbers of patients followed over a period of years.
For the first time in the history of IPF drug development there has been major investment by industry in biologically plausible agents identified by high-impact science. This therefore represents an extraordinary time for patients with IPF, as confidence has grown that high-quality RCT can be performed with a feasible pathway to regulatory approval. Mandating mortality as the primary endpoint for phase III trials in IPF has the potential to seriously cripple this enormous opportunity for patients with IPF.
We suggest that what is needed to move forward constructively beyond just an academic debate requires the involvement of a broader group of individuals, including expert clinicians from not only the United States but also Europe and the rest of the world, patient representatives, representation from the pharmaceutical industry, as well as regulatory agencies. We need effective patient options that take into account the unique challenges of an orphan disease with tremendous heterogeneity and unpredictable rates of progression. This requires a more constructive approach rather than the recommendations presented by Raghu and colleagues (16). We hope that the community will be able to balance the deliberations of Raghu and colleagues with the counterpoints provided in this Pulmonary Perspective, and we would encourage the readership of the American Journal of Respiratory and Critical Care Medicine to engage in this important debate through the correspondence section of our journal.
| 1. | Meltzer EB, Noble PW. Idiopathic pulmonary fibrosis. Orphanet J Rare Dis 2008;3:1–15. |
| 2. | Navaratnam V, Fleming KM, West J, Smith CJ, Jenkins RG, Fogarty A, Hubbard RB. The rising incidence of idiopathic pulmonary fibrosis in the UK. Thorax 2011;66:462–467. |
| 3. | Raghu G, Collard HR, Egan JJ, Martinez FJ, Behr J, Brown KK, Colby TV, Cordier J-F, Flaherty KR, Lasky JA, et al.. An official ATS/ERS/JRS/ALAT statement: idiopathic pulmonary fibrosis: evidence-based guidelines for diagnosis and management. Am J Respir Crit Care Med 2011;183:788–824. |
| 4. | du Bois RM, Weycker D, Albera C, Bradford WZ, Costabel U, Kartashov A, King TE, Lancaster L, Noble PW, Sahn SA, et al.. Forced vital capacity in patients with idiopathic pulmonary fibrosis: test properties and minimal clinically important difference. Am J Respir Crit Care Med 2011;184:1382–1389. |
| 5. | du Bois RM, Weycker D, Albera C, Bradford WZ, Costabel U, Kartashov A, Lancaster L, Noble PW, Raghu G, Sahn SA, et al.. Ascertainment of individual risk of mortality for patients with idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 2011;184:459–466. |
| 6. | Nathan SD, Shlobin OA, Weir N, Ahmad S, Kaldjob JM, Battle E, Sheridan MJ, du Bois RM. Long-term course and prognosis of idiopathic pulmonary fibrosis in the new millennium. Chest 2011;140:221–229. |
| 7. | du Bois RM. Strategies for treating idiopathic pulmonary fibrosis. Nat Rev Drug Discov 2010;9:129–140. |
| 8. | Idiopathic Pulmonary Fibrosis. Diagnosis and Treatment. International consensus statement. Am J Respir Crit Care Med 2000;161:646–664. |
| 9. | Azuma A, Nukiwa T, Tsuboi E, Suga M, Abe S, Nakata K, Taguchi Y, Nagai S, Itoh H, Ohi M, et al.. Double-blind, placebo-controlled trial of pirfenidone in patients with idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 2005;171:1040–1047. |
| 10. | Taniguchi H, Ebina M, Kondoh Y, Ogura T, Azuma A, Suga M, Taguchi Y, Takahashi H, Nakata K, Sato A, et al.. Pirfenidone in idiopathic pulmonary fibrosis. Eur Respir J 2010;35:821–829. |
| 11. | Noble PW, Albera C, Bradford WZ, Costabel U, Glassberg MK, Kardatzke D, King TE, Lancaster L, Sahn SA, Szwarcberg J, et al.. Pirfenidone in patients with idiopathic pulmonary fibrosis (CAPACITY): two randomised trials. Lancet 2011;377:1760–1769. |
| 12. | Spagnolo P, Del Giovane C, Luppi F, Cerri S, Balduzzi S, Walters EH, D'Amico R, Richeldi L. Non-steroid agents for idiopathic pulmonary fibrosis. Cochrane Database Syst Rev 2010;9:CD003134. |
| 13. | Vancheri C, Failla M, Crimi NC, Raghu G. Idiopathic pulmonary fibrosis: a disease with similarities and links to cancer biology. Eur Respir J 2010;35:496–504. |
| 14. | Zappala CJ, Latsi PI, Nicholson AG, Colby TV, Cramer D, Renzoni EA, Hansell DM, du Bois RM, Wells AU. Marginal decline in forced vital capacity is associated with a poor outcome in idiopathic pulmonary fibrosis. Eur Respir J 2010;35:830–836. |
| 15. | Richeldi L, Ryerson CJ, Lee JS, Wolters PJ, Koth LL, Ley B, Elicker BM, Jones KD, King TE, Ryu JH, et al.. Relative versus absolute change in forced vital capacity in idiopathic pulmonary fibrosis. Thorax 2012;67:407–411. |
| 16. | Raghu G, Collard HR, Anstrom KJ, Flaherty KR, Fleming TR, King TE, Martinez FJ, Brown KK. Idiopathic pulmonary fibrosis: clinically meaningful primary endpoints in phase 3 clinical trials. Am J Respir Crit Care Med 2012;185:1044–1048. |
| 17. | Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther 2001;69:89–95. |
| 18. | Raghu G, Anstrom KJ, King TE, Lasky JA, Martinez FJ. Prednisone, azathioprine, and N-acetylcysteine for pulmonary fibrosis. N Engl J Med 2012;366:1968–1977. |
| 19. | Zisman DA, Schwarz M, Anstrom KJ, Collard HR, Flaherty KR, Hunninghake GW. A controlled trial of sildenafil in advanced idiopathic pulmonary fibrosis. N Engl J Med 2010;363:620–628. |
| 20. | Noth I, Anstrom KJ, Calvert SB, de Andrade J, Flaherty KR, Glazer C, Kaner RJ, Olman MA, The Idiopathic Pulmonary Fibrosis Clinical Research Network (IPFnet). A placebo-controlled randomized trial of warfarin in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 2012;186:88–95. |
| 21. | Deyton L. Importance of surrogate markers in evaluation of antiviral therapy for HIV infection. JAMA 1996;276:159–160. |
| 22. | Pozniak A. Surrogacy in HIV-1 clinical trials. Lancet 1998;351:536–537. |
| 23. | Lagakos SW, Hoth DF. Surrogate markers in AIDS: where are we? Where are we going? Ann Intern Med 1992;116:599–601. |
| 24. | O’Callaghan DS, Savale L, Montani D, Jais X, Sitbon O, Simonneau G, Humbert M. Treatment of pulmonary arterial hypertension with targeted therapies. Nat Rev Cardiol 2011;8:526–538. |
| 25. | King TE, Albera C, Bradford WZ, Costabel U, Hormel P, Lancaster L, Noble PW, Sahn SA, Szwarcberg J, Thomeer M, et al.. Effect of interferon gamma-1b on survival in patients with idiopathic pulmonary fibrosis (INSPIRE): a multicentre, randomised, placebo-controlled trial. Lancet 2009;374:222–228. |
| 26. | Raghu G, DePaso WJ, Cain K, Hammar SP, Wetzel CE, Dreis DF, Hutchinson J, Pardee NE, Winterbauer RH. Azathioprine combined with prednisolone in the treatment of idiopathic pulmonary fibrosis: a prospective, double-blind, randomized, placebo-controlled clinical trial. Am Rev Respir Dis 1991;144:291–296. |
Originally Published in Press as DOI: 10.1164/rccm.201206-1010PP on June 12, 2012
Author disclosures
