Abstract
It is not known if a surgical lung biopsy is necessary in all patients for the diagnosis of idiopathic pulmonary fibrosis (IPF). We conducted a blinded, prospective study at eight referring centers. Initially, cases were evaluated by clinical history and examination, transbronchial biopsy, and high-resolution lung computed tomography scans. Pulmonologists at the referring centers then assessed their certainty of the diagnosis of IPF and provided an overall diagnosis, before surgical lung biopsy. The lung biopsies were reviewed by a pathology core and 54 of 91 patients received a pathologic diagnosis of IPF. The positive predictive value of a confident (certain) clinical diagnosis of IPF by the referring centers was 80%. The positive predictive value of a confident clinical diagnosis was higher, when the cases were reviewed by a core of pulmonologists (87%) or radiologists (96%). Lung biopsy was most important for diagnosis in those patients with an uncertain diagnosis and those thought unlikely to have IPF. These studies suggest that clinical and radiologic data that result in a confident diagnosis of IPF by an experienced pulmonologist or radiologist are sufficient to obviate the need for a lung biopsy. Lung biopsy is most helpful when clinical and radiologic data result in an uncertain diagnosis or when patients are thought not to have IPF.
Idiopathic pulmonary fibrosis (IPF) is a lung disorder with a poor prognosis (1-7). Because patients with other disorders that mimic IPF (2, 8-16) have a better prognosis or require different therapy, it is recommended that patients suspected of having IPF undergo a surgical lung biopsy. The amount of tissue obtained with transbronchial biopsy is not sufficient to make a diagnosis (17). Recent retrospective studies have suggested that radiologic findings, using high-resolution computed tomography (HRCT) scans, are highly specific for IPF and can be used to make a diagnosis without a lung biopsy (2, 13, 18-22). The present study determined the value of clinical and radiologic findings for the diagnosis of IPF.
We performed a prospective, blinded study at eight referring centers. All new patients suspected of having IPF were entered into the study if their medical condition did not preclude performing the biopsy. The study was approved by the institutional review boards for protection of human subjects at each of the centers and the subjects gave written informed consent.
The study was designed to mimic the usual clinical evaluation of patients suspected of having IPF. Patients with an underlying connective tissue disorder, exposure to environmental agents or drugs known to cause pulmonary fibrosis, or other underlying disorders known to cause pulmonary fibrosis were excluded. The remaining patients had a HRCT scan and a bronchoscopy with a transbronchial lung biopsy. The transbronchial biopsy was performed to detect lung diseases other than IPF. A diagnosis was obtained by transbronchial biopsy in only two patients. If the transbronchial biopsy did not provide a specific diagnosis, patients underwent a surgical (open or thorascopic) lung biopsy. The lung HRCT scan was not used to determine if a patient should undergo a surgical biopsy.
Before the surgical biopsy but after the results of the lung HRCT scan and transbronchial biopsy, one pulmonologist at each of the referring centers rated the certainty of the diagnosis of IPF (as certain, uncertain, or unlikely) and provided an overall clinical diagnosis, even if the diagnosis was uncertain. The center investigators could use any clinical information that was available for the patient to provide this assessment. No predetermined clinical or radiologic criteria were used to make a clinical diagnosis or to determine the level of certainty of the diagnosis of IPF.
The following information was provided by the referring centers for review by a clinical core of three pulmonologists: presence and duration of cough; presence and duration of dyspnea; history of smoking; history of fever, weight loss, myalgias, arthralgias, rash, and arthritis; presence of finger clubbing; and pulmonary function tests. The clinical core directly evaluated chest radiographs and HRCT scans. Each independently rated their certainty of the diagnosis of IPF (as certain, uncertain, or unlikely) and provided an overall clinical diagnosis, even if the diagnosis was uncertain.
A core of four chest radiologists independently evaluated the HRCT scans. No clinical information was provided. Each rated their certainty of the diagnosis of IPF (as certain, uncertain, or unlikely), and provided an overall clinical diagnosis, even if the diagnosis was uncertain.
A core of three lung pathologists independently evaluated the same sets of pathology slides. No clinical information was provided. They provided an overall pathologic diagnosis, and if they were unsure of the diagnosis, they provided a secondary diagnosis.
The kappa statistic (23) was used to assess agreement in IPF diagnosis for all observers within each of the cores. The kappa coefficient used to measure agreement within the cores was based on the form proposed by Kraemer (24), which allowed for unequal numbers of observations per subject. The probability of agreement between any two members of a core, estimated as the average proportion of concordant pairs for all possible pairings of raters per subject, was also estimated (25). Using the pathology diagnosis of IPF as the gold standard, sensitivity, specificity, accuracy, and positive predictive value of the diagnosis of IPF of the cores and centers were calculated. For the combinations of overall clinical diagnosis and certainty of diagnosis of each of the cores and referring centers, Bayesian posterior probabilities corresponding to the predictive value of the diagnosis/certainty of diagnosis in identifying IPF were calculated (26). A prior probability of IPF of 0.60, corresponding to the prevalence of IPF in new suspected patients who present for diagnosis, was used in these calculations. This estimate was based on the proportion of patients in whom IPF was diagnosed by the pathology diagnosis. A high level of sensitivity, specificity, accuracy, positive predictive value, and Bayesian posterior conditional predictive probability for a confident IPF diagnosis would suggest that biopsy is not necessary for cases with a confident diagnosis of IPF.
The eight referring centers entered 91 patients; 82 of these patients had no complications from the biopsy. In four patients, an air leak persisted for up to 6 d. Three patients were readmitted because of a pneumothorax which resolved with placement of a chest tube. One patient developed a postoperative fever without a change on the chest radiograph. Another patient, a 70-yr-old white male, was hospitalized with dyspnea due to a pneumothorax. The patient was evaluated using the protocol because of a chest radiograph consistent with interstitial lung disease. During his hospital stay, he had recurrent pneumothoraces. Because of this, he had a thoracoscopy for removal of blebs and a pleurodesis. A lung biopsy was performed during the procedure. Postoperatively, he developed increased respiratory failure and died.
Pathologic findings consistent with usual interstitial pneumonia were required for the clinical diagnosis of IPF (27). When only the primary diagnosis was used, the probability that any two pathologists in the pathology core would agree regarding the presence or absence of IPF 0.85 (Table 1). If the secondary diagnosis was considered, the probability of agreement was 0.87. The probability of agreement regarding a specific primary diagnosis was 0.72. When both the primary and secondary diagnoses were considered, the probability of agreement for a specific diagnosis was 0.85. The agreement regarding a specific diagnosis was not as high when compared with the agreement regarding the presence or absence of IPF because there was less agreement in the group of patients who did not have IPF. Of the 37 patients in whom IPF was not diagnosed by at least two of the pathologists, the probability of agreement regarding a specific primary diagnosis was 0.48. For this study, we considered agreement between two pathologists regarding a specific diagnosis or the presence or absence of IPF, as the standard for the study. This allowed us to have agreement on the presence or absence of IPF on all patients. Overall, IPF was present in 54 cases and not present in 37 cases. Of the 37 non-IPF cases, one had silicosis, 12 had respiratory bronchiolitis, seven had hypersensitivity pneumonitis, three had sarcoidosis, one had histiocytosis-X, two had emphysema, one had bronchiolitis obliterans with organizing pneumonia, six had nonspecific interstitial pneumonitis, two had bronchoalveolar carcinoma, one had pulmonary hypertension, and one had eosinophilic pneumonia.
| Core | IPF versus Non-IPF* | Specific Diagnosis† | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Agreement | Kappa Score | 95% CI | Agreement | Kappa Score | 95% CI | |||||||
| Clinical | 0.79 | 0.59 (± 0.06 SE) | 0.46–0.72 | 0.49 | 0.32 (± 0.05 SE) | 0.22–0.42 | ||||||
| Radiology | 0.77 | 0.54 (± 0.06 SE) | 0.42–0.65 | 0.54 | 0.31 (± 0.05 SE) | 0.20–0.41 | ||||||
| Pathology | 0.85 | 0.68 (± 0.06 SE) | 0.56–0.80 | 0.72 | 0.55 (± 0.05 SE) | 0.45–0.64 | ||||||
Overall, the sensitivity, specificity, accuracy, and positive predictive value of the diagnosis of IPF were 85%, 43%, 68%, and 69%, respectively (Table 2). When the uncertain cases were excluded, the sensitivity, specificity, accuracy, and positive predictive value of a confident diagnosis were 93%, 36%, 78%, and 80%, respectively (Table 3). The probability that a patient has IPF given a confident (certain) diagnosis by the referring center was 0.81 (Table 4).
| Overall IPF Diagnosis* | ||||||||
|---|---|---|---|---|---|---|---|---|
| Sensitivity | Specificity | Accuracy | Positive Predictive Value | |||||
| Clinical core | 39/54 (72%) | 31/37 (84%) | 70/91 (77%) | 39/45 (87%) | ||||
| Radiology core† | 41/53 (77%) | 26/36 (72%) | 67/89 (75%) | 41/48 (85%) | ||||
| Referring center | 46/54 (85%) | 16/37 (43%) | 62/91 (68%) | 46/67 (69%) | ||||
| Sensitivity | Specificity | Accuracy | Positive Predictive Value | |||||
|---|---|---|---|---|---|---|---|---|
| Clinical core | 26/33 (79%) | 26/30 (87%) | 52/63 (86%) | 26/30 (87%) | ||||
| Radiology core | 26/30 (87%) | 21/22 (95%) | 47/52 (90%) | 26/27 (96%) | ||||
| Referring center | 37/40 (93%) | 5/14 (36%) | 42/54 (78%) | 37/46 (80%) |
| Diagnosis/ Certainty | Clinical Core | Radiology Core | Referring Center | |||
|---|---|---|---|---|---|---|
| IPF/Certain | 0.870 | 0.964 | 0.809 | |||
| IPF/Uncertain | 0.870 | 0.720 | 0.435 | |||
| No agreement | — | 0.578 | — | |||
| Not IPF/Uncertain | 0.622 | 0.451 | 0.318 | |||
| Not IPF/Certain | 0.217 | 0.164 | 0.381 |
A diagnosis of IPF/usual interstitial pneumonia uniformly resulted in aggressive therapy with corticosteroids, usually with a cytotoxic agent. Of the 54 patients with IPF, 11 were not thought to have IPF before biopsy. The information was also used for counseling regarding prognosis for these patients with IPF. For the remaining 37 patients, less aggressive therapy appropriate for the diagnosis was used. For nine patients, a certain diagnosis of IPF underwent a major change to: lung cancer (two patients), sarcoidosis (two), hypersensitivity pneumonitis (three), silicosis (one), and emphysema (one).
The probability that any two members of the core would agree with respect to the presence or absence of IPF was 0.77 (Table 1). Agreement within the core regarding a specific diagnosis was lower at 0.54 because there was less agreement for the patients who were believed not to have IPF. Overall, the sensitivity, specificity, accuracy, and positive predictive value of the diagnosis of IPF were 77%, 72%, 75%, and 85%, respectively (Table 2). When the uncertain cases were excluded, sensitivity, specificity, accuracy, and positive predictive value were 87%, 95%, 90%, and 96%, respectively (Table 3). The probability that a patient has IPF given a confident (certain) diagnosis of IPF by the radiology core was 0.96 (Table 4). One additional observation of the radiology core was that there were no instances where the diagnosis of IPF was associated with a normal HRCT scan (data not shown).
The overall agreement within the core related to the presence or absence of IPF was 0.79 (Table 1). The agreement within the core related to a specific diagnosis was much lower at 0.49. Overall, sensitivity, specificity, accuracy, and positive predictive value of the diagnosis of IPF were 72%, 84%, 77%, and 87%, respectively (Table 2). When the uncertain cases were excluded, sensitivity, specificity, accuracy, and positive predictive value of a confident diagnosis were 79%, 87%, 86%, and 87% (Table 3). The probability that a patient has IPF given a confident (certain) diagnosis of IPF by the clinical core was 0.87 (Table 4).
Overall, this study showed that clinical and radiologic data that result in a confident diagnosis by a pulmonologist or radiologist with extensive experience in the care of patients with interstitial lung diseases are sufficient to obviate the need for a lung biopsy. For these clinicians, the sensitivity, specificity, accuracy, and positive predictive value of a confident diagnosis are very high. It is important to note, however, that a confident clinical diagnosis of IPF identified only one-half of the patients who actually have this disorder. This study also suggests that lung biopsy may be required for diagnosis when patients are cared for by less experienced clinicians, when the diagnosis is uncertain, and when the clinical diagnosis is not IPF. The study also showed that it is easier to determine the presence or absence of IPF without lung biopsy than it is to make a specific diagnosis. This is due, in large part, to difficulty in determining a diagnosis in patients who do not have IPF.
For this study, we defined agreement on the diagnosis of IPF in the pathology core as two of the three members agreeing, rather than developing a consensus within the core. Although this may have resulted in chance agreement in a few instances, it is unlikely that this study design resulted in a change in overall diagnosis. In addition, because of the design of the study, we were able to evaluate, for the first time, the frequency of agreement between experienced pathologists or pulmonologists regarding the diagnosis of an interstitial lung disease. The frequency of agreement in the pathology core regarding the presence or absence of IPF was approximately 85%. The accuracy of this observation is supported by the observation that the agreement between pathologists at the clinical centers and the pathology core was also approximately 80% (data not shown). These observations suggest that in populations of patients with interstitial lung disease, a specific diagnosis might be difficult in approximately 15 to 20% of the cases, even with a surgical biopsy.
Several retrospective studies have evaluated agreement by radiologists for evaluation of lung HRCT scans of patients with interstitial lung disease. Collins and coworkers evaluated interobserver and intraobserver variability in evaluating lung HRCT scans in patients with IPF (19). The overall kappa score for the pattern type on the lung HRCT scans in the study was 0.48, a value that is not significantly different from that observed in this study. In a study by Daniloff and associates (28), the kappa score for interobserver agreement for the presence of nodules, septal lines, and ground-glass attenuation on lung HRCT scans from patients with chronic beryllium disease was 0.53, 0.44, and 0.53, respectively. These levels of agreement are also similar to that observed in this study. Kazerooni and coworkers correlated evidence of “fibrosis” on lung HRCT scans with evidence of fibrosis on pathologic specimens (29). In the study, interobserver agreement on the lung HRCT scans was also similar to the present study. These observations, as an aggregate, suggest that the levels of interobserver agreement in the present study are consistent with prior studies.
It is important to emphasize that the results of this study were generated by pathologists, pulmonologists, and radiologists with a significant amount of experience with IPF and other interstitial lung diseases. Whether these results could be duplicated in other clinical settings is not clear. It should also be noted that members of the pathology and radiology cores evaluated the cases without knowledge of the clinical history, which is often available in clinical practice. This was not the case, however, in the referring centers where there was an exchange of information among clinicians, radiologists, and pathologists. This clinical setting most closely replicated the care of patients by pulmonary physicians in the community. In addition, a number of patients with interstitial disease, i.e., those with an underlying connective tissue disease or occupational lung disease, were excluded from the study because they often do not require open lung biopsy for diagnosis.
The overall findings of this study are similar to that of Raghu and colleagues (30). It is reassuring, for both studies, that the sensitivity, specificity, and positive predictive value of a confident diagnosis of IPF were almost the same. The study by Raghu and colleagues was performed within a single institution, whereas this was a multicenter study with review of the data by clinical, radiologic, and pathology cores. A major strength of the current study is the broad and extensive experience of the participating investigators. In addition, a thorough analysis of agreement between these experienced clinicians, radiologists, and pathologists for the diagnosis of IPF is presented. This information should be critical for the design of future clinical studies of IPF.
In summary, this study defined in a prospective and blinded manner the capacity of experienced pulmonologists and radiologists to determine the presence or absence of IPF, using only clinical and radiologic information. In clinical practice, only a small percentage of patients undergo lung biopsy, in spite of recommendations that patients have a pathologic confirmation of the diagnosis of the disease. This may be due, in part, to complications associated with the procedure, as noted for this study. The information from this study should help clinicians determine when a surgical biopsy will be most helpful in various patient care settings. In addition, the data from this study suggest that, for some patients, biopsy proof of IPF may not, in the future, be regarded as a necessary criterion for inclusion in clinical studies. For these clinical studies, inclusion of patients without a lung biopsy may require the expertise of experienced cores of clinicians and radiologists, as was the case in this study.
The authors thank DeAnna O'Quinn for her expert assistance in the preparation of this manuscript and Julia Stutzman for her assistance in conducting this study.
Supported in part by the NHLBI SCOR program on interstitial lung disease.
| 1. | Bjoraker JA, Ryu JH, Edwin MK, Myers JL, Tazelaar HD, Schroeder DR, Offord KPPrognostic significance of histopathologic subsets in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med1571998199203 |
| 2. | Wells AU, Cullinan P, Hansell DM, Rubens MB, Black CM, Newman-Taylor AJ, Du Bois RMFibrosing alveolitis associated with systemic sclerosis has a better prognosis than lone cryptogenic fibrosing alveolitis. Am J Respir Crit Care Med149199415831590 |
| 3. | Tukiainen P, Taskinen E, Holsti P, Korhola O, Valle MPrognosis of cryptogenic fibrosing alveolitis. Thorax381983349355 |
| 4. | Crystal RG, Gadek JE, Ferrans VJ, Fulmer JD, Line BR, Hunninghake GWInterstitial lung disease: current concepts of pathogenesis, staging and therapy. Am J Med701981542568 |
| 5. | Schwartz DA, Van Fossen DS, Davis CS, Helmers RA, Dayton CS, Burmeister LF, Hunninghake GWDeterminants of progression in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med1491994444449 |
| 6. | Cherniack RM, Colby TV, Flint A, Thurlbeck WM, Waldron J, Ackerson L, King TEQuantitative assessment of lung pathology in idiopathic pulmonary fibrosis. The BAL Cooperative Group Steering Committee. Am Rev Respir Dis1441991892900 |
| 7. | Carrington CB, Gaensler EA, Coutu RE, FitzGerald MX, Gupta RGNatural history and treated course of usual and desquamative interstitial pneumonia. N Engl J Med2981978801809 |
| 8. | Padley SP, Padhani AR, Nicholson A, Hansell DMPulmonary sarcoidosis mimicking cryptogenic fibrosing alveolitis on CT. Clin Radiol511996807810 |
| 9. | Lynch DA, Newell JD, Logan PM, King TE, Muller NLCan CT distinguish hypersensitivity pneumonitis from idiopathic pulmonary fibrosis? AJR Am J Roentgenol1651995807811 |
| 10. | Yoshida K, Shijubo N, Koba H, Mori Y, Satoh M, Morikawa T, Abe SChronic eosinophilic pneumonia progressing to lung fibrosis. Eur Respir J7199415411544 |
| 11. | Katzenstein AL, Fiorelli RFNonspecific interstitial pneumonia/fibrosis: histologic features and clinical significance. Am J Surg Pathol181994136147 |
| 12. | Perez-Padilla R, Salas J, Chapela R, Sanchez M, Carrillo G, Perez R, Sansores R, Gaxiola M, Selman MMortality in Mexican patients with chronic pigeon breeder's lung compared with those with usual interstitial pneumonia. Am Rev Respir Dis14819934953 |
| 13. | Hartman TE, Primack SL, Swensen SJ, Hansell D, McGuinness G, Muller NLDesquamative interstitial pneumonia: thin-section CT findings in 22 patients. Radiology1871993787790 |
| 14. | Kitaichi MPathologic features and the classification of interstitial pneumonia of unknown etiology. Bull Chest Dis Res Inst Kyoto Univ231990118 |
| 15. | Aisner SC, Albin RJDiffuse interstitial pneumonitis and fibrosis in sarcoidosis. Chest941988193195 |
| 16. | Muller NL, Guerry-Force ML, Staples CA, Wright JL, Wiggs B, Coppin C, Pare P, Hogg JCDifferential diagnosis of bronchiolitis obliterans with organizing pneumonia and usual interstitial pneumonia: clinical, functional, and radiologic findings. Radiology1621987151156 |
| 17. | Wall CP, Gaensler EA, Carrington CB, Hayes JAComparison of transbronchial and open biopsies in chronic infiltrative lung diseases. Am Rev Respir Dis1231981280285 |
| 18. | Coxson HO, Hogg JC, Mayo JR, Behzad H, Whittall KP, Schwartz DA, Hartley PG, Galvin JR, Wilson JS, Hunninghake GWQuantification of idiopathic pulmonary fibrosis using computed tomography and histology. Am J Respir Crit Care Med155199716491656 |
| 19. | Collins CD, Wells AU, Hansell DM, Morgan RA, MacSweeney JE, du Bois RM, Rubens MBObserver variation in pattern type and extent of disease in fibrosing alveolitis on thin section computed tomography and chest radiography. Clin Radiol491994236240 |
| 20. | Tung KT, Wells AU, Rubens MB, Kirk JM, du Bois RM, Hansell DMAccuracy of the typical computed tomographic appearances of fibrosing alveolitis. Thorax481993334338 |
| 21. | Bergin CJ, Muller NLCT of interstitial lung disease: a diagnostic approach. AJR Am J Roentgenol1481987915 |
| 22. | Muller NL, Miller RR, Webb WR, Evans KG, Ostrow DNFibrosing alveolitis: CT–pathologic correlation. Radiology1601986585588 |
| 23. | Fleiss J. Statistical methods for rates and proportions. New York: John Wiley and Sons; 1981. |
| 24. | Kraemer HCExtension of the kappa coefficient. Biometrics361980207216 |
| 25. | Woolson R. Statistical methods for the analysis of biomedical data. New York: John Wiley and Sons; 1987. |
| 26. | Jones RH, McClatchey MWBeyond sensitivity, specificity and statistical independence. Stat Med7198812891295 |
| 27. | Katzenstein AL, Myers JLIdiopathic pulmonary fibrosis: clinical relevance of pathologic classification. Am J Respir Crit Care Med157199813011315 |
| 28. | Daniloff EM, Lynch DA, Bartelson BB, Newell JD, Bernstein SM, Newman LSObserver variation and relationship of computed tomography to severity of beryllium disease. Am J Respir Crit Care Med155199720472056 |
| 29. | Kazerooni EA, Martinez FJ, Flint A, Jamadar DA, Gross BH, Spizarny DL, Cascade PN, Whyte RI, Lynch JP, Toews GThin-section CT obtained at 10-mm increments versus limited three-level thin-section CT for idiopathic pulmonary fibrosis: correlation with pathologic scoring. AJR Am J Roentgenol1691997977983 |
| 30. | Raghu G, Mageto YN, Lockhart D, Schmidt RA, Wood DE, Godwin JDThe accuracy of the clinical diagnosis of new-onset idiopathic pulmonary fibrosis and other interstitial lung disease: a prospective study. Chest116199911681174 |
1 Current address: Dept. of Internal Medicine, Pulmonary Disease Division, San Francisco General Hospital, San Francisco, California.
2 Current address: Dept of Radio Pathology, Armed Forces Institute of Pathology, Washington, DC.
