Current guidelines recommend that the clinician, radiologist, and pathologist work together to establish a diagnosis of idiopathic interstitial pneumonia. Three clinicians, two radiologists, and two pathologists reviewed 58 consecutive cases of suspected idiopathic interstitial pneumonia. Each participant was provided information in a sequential manner and was asked to record their diagnostic impression and level of confidence at each step. Interobserver agreement improved from the beginning to the end of the review. After the presentation of histopathologic information, radiologists changed their diagnostic impression more often than did clinicians. In general, as more information was provided the confidence level for a given diagnosis improved, and the diagnoses rendered with a high level of confidence were more likely congruent with the final pathologic consensus diagnosis. The final consensus pathologist diagnosis was idiopathic pulmonary fibrosis in 30 cases. Clinicians identified 75% and radiologists identified 48% of these cases before presentation of the histopathologic information. Histopathologic information has the greatest impact on the final diagnosis, especially when the initial clinical/radiographic diagnosis is not idiopathic pulmonary fibrosis. We conclude that dynamic interactions between clinicians, radiologists, and pathologists improve interobserver agreement and diagnostic confidence.
Histopathologic subsets of idiopathic interstitial pneumonia exhibit different prognoses (1–8). Therefore, an accurate diagnosis is critical to the management of patients with idiopathic interstitial pneumonia. Clinical features, high-resolution computed tomography (HRCT) (9–12), and surgical lung biopsy (13) all play a role in establishing a diagnosis. An American Thoracic Society/European Respiratory Society (ATS/ERS) committee emphasized the need for a dynamic diagnostic integrated process in which clinicians, radiologists, and pathologists exchange information in the determination of a diagnosis in individual patients with suspected idiopathic interstitial pneumonia (14). Using histology alone as the “gold standard” for diagnosis can be complicated by difficulties with interrater agreement (15) and the potential for sampling error (2). The combination of HRCT and histologic features also better predicts prognosis compared with either modality alone (16). In the current study, we hypothesized that an interactive process would improve the interobserver agreement between expert clinicians, radiologists, and pathologists in consecutive patients with suspected idiopathic interstitial pneumonia compared with each group working in diagnostic isolation. This study illustrates that a consensus diagnosis, reached after a careful exchange of clinical, radiographic, and histopathologic information, often differs from the initial diagnosis reached by the individual clinician, radiologist, or pathologist working in isolation. Some of these results have been previously reported in the form of an abstract (17, 18).
Patients in this study represented consecutive patients referred to the University of Michigan Specialized Center of Research (SCOR) in the Pathobiology of Fibrotic Lung Disease for potential participation in research protocols between January 2002 and August 2002. Patients with suspected idiopathic interstitial pneumonia were referred to the study center by participants in the University of Michigan Fibrotic Lung Disease Network (see acknowledgment). Through the course of evaluation all patients underwent a history, physical examination, complete pulmonary function testing, HRCT, and surgical lung biopsy. Patients without an HRCT scan or a surgical lung biopsy were excluded. Patients with known collagen vascular disease at the time of presentation were excluded. Patients without collagen vascular disease at initial presentation, but that developed a discrete collagen vascular disease during the course of follow-up, were included.
A standard form was used to collect clinical information including symptoms, environmental exposures, comorbid illnesses, medication use, smoking history, family history, physical examination findings, and serologic data. Pulmonary function data (spirometry, lung volumes, and diffusion capacity for carbon monoxide) and HRCT within 6 months of surgical lung biopsy were reviewed.
Two expert pulmonary pathologists (T.C. and W.T.) individually reviewed each patient's surgical lung biopsy without clinical or HRCT information. The slides from each patient's surgical lung biopsy were independently reviewed by the two pathologists before the study meeting. At the study meeting, during Steps 1–3 (see below) the pathologists were physically separated from the clinicians and radiologists and resolved individual differences in histopathologic opinion through joint review and a consensus histopathologic diagnosis was reached.
The overall study format was designed to evaluate whether the addition of specific clinical, radiographic, and pathologic information impacted the diagnostic impression of each participant. Participants met at the University of Michigan and were given information in a stepwise fashion as outlined below and in Figure 1. Participants could change diagnostic impression and/or level of confidence at each step of the review process.
Step 1: Expert clinicians (J.L., T.K., and G.R.) and radiologists (E.K. and B.G.) independently reviewed HRCTs without clinical or pathologic information. Each participant was asked independently to provide their opinion as to the most likely diagnosis and a confidence level for that diagnosis (1 = definite, 2 = probable, 3 = possible, 4 = definitely not). Acknowledging the prognostic importance of usual interstitial pneumonia (UIP), participants were asked to record their confidence that the case could represent UIP.
Step 2: Clinicians and radiologists reviewed HRCTs with a standardized presentation of clinical information and recorded their diagnostic impression and confidence as in Step 1; no discussion occurred between the participants.
Step 3: Clinicians and radiologists discussed diagnostic impressions and again recorded their individual diagnosis and confidence level.
Step 4: Pathologists (T.C. and W.T.) entered the study arena. The standardized clinical information and HRCT were again reviewed. The pathologists then discussed their interpretation of the surgical lung biopsy. The clinicians, radiologists, and pathologists discussed each case and again recorded their individual final clinical, radiologic, and pathologic diagnosis and confidence level. For this part of the study the two pathologists functioned as a single observer, that is, they did not each record their individual interpretation.
Step 5: Participants discussed their own interpretation of each case. When disagreement was present an attempt was made to reach a consensus diagnosis. A consensus clinical, radiologic, and pathologic diagnosis could not be reached in all cases and therefore all participants again recorded their individual diagnoses and confidence levels.
Each observer's diagnosis was coded into one of six categories—idiopathic pulmonary fibrosis (IPF), nonspecific interstitial pneumonia (NSIP), respiratory bronchiolitis interstitial lung disease/desquamative interstitial pneumonia (RBILD/DIP), hypersensitivity pneumonitis, bronchiolar disease, and other. McNemar tests were subsequently used to test whether two probabilities of agreement conducted during different steps or by different raters were equal. A κ statistic allowing for multiple raters was also used to assess agreement in diagnosis. κ Scores are rated as almost perfect agreement (above 0.8), substantial agreement (scores between 0.6 and 0.8), moderate agreement (scores between 0.4 and 0.6), fair agreement (scores between 0.2 and 0.4), slight agreement (scores between 0.0 and 0.2), and poor agreement (scores below 0.0) (19). An estimating equation approach to the analysis of correlated κ statistics was used in comparisons of κ statistics estimated throughout the study and in producing confidence intervals for the κ statistics (20). A χ2 or Fisher exact test was used to examine the relationship between the confidence of each participant for their diagnosis and the final pathologist consensus diagnosis. A recorded confidence level of 1 was considered high in confidence and a confidence level of 2, 3, or 4 was considered low in confidence.
Fifty-eight patients were evaluated. The κ for the two pathologists without clinical or radiologic information (pathology review before Step 1) was 0.72 (95% confidence interval, 0.57 to 0.86) with a probability of agreement of 0.81 (95% confidence interval, 0.71 to 0.91). IPF and NSIP were the most common diagnoses (Tables 1 and 2)
| Other|| 4|| 3|| 5|| 5|| 4|| 5|
|Step||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]|
|1||0.41 (0.29, 0.52)||0.72 (0.57, 0.86)||0.39 (0.29, 0.49)||NA|
|2||0.51 (0.37, 0.64)||0.80 (0.67, 0.93)||0.44 (0.34, 0.54)||NA|
|3||0.67 (0.54, 0.79)||0.78 (0.65, 0.91)||0.55 (0.44, 0.66)||NA|
|4||0.75 (0.64, 0.86)||0.84 (0.72, 0.96)||0.78 (0.70, 0.86)||0.79 (0.71, 0.86)|
|5||0.86 (0.76, 0.95)||0.90 (0.80, 0.99)||0.88 (0.81, 0.96)||0.88 (0.81, 0.94)|
|Steps||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]|
|1 and 2||0.75 (0.61, 0.90)||0.73 (0.59, 0.88)||0.80 (0.68, 0.92)||0.92 (0.84, 1.00)||0.90 (0.80, 0.99)|
|2 and 3||0.86 (0.75, 0.98)||0.75 (0.61, 0.89)||0.92 (0.84, 1.00)||0.95 (0.88, 1.00)||0.98 (0.92, 1.00)|
|3 and 4||0.60 (0.45, 0.76)||0.55 (0.39, 0.71)||0.50 (0.35, 0.66)||0.20 (0.06, 0.34)||0.32 (0.16, 0.48)|
|4 and 5||0.92 (0.83, 1.00)||0.89 (0.79, 0.99)||0.79 (0.66, 0.92)||0.84 (0.72, 0.96)||0.94 (0.88, 1.00)|
The agreement level of clinicians and radiologists with the pathologist consensus diagnosis tended to increase as additional information was provided (Table 5)
|Step||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]||[κ (95% CI)]|
|1||0.22 (0.07, 0.36)||0.22 (0.09, 0.36)||0.38 (0.23, 0.54)||0.09 (0.0, 0.21)||0.14 (0.02, 0.27)|
|2||0.34 (0.20, 0.48)||0.20 (0.05, 0.34)||0.39 (0.24, 0.55)||0.12 (0.0, 0.26)||0.17 (0.04, 0.30)|
|3||0.34 (0.20, 0.48)||0.32 (0.17, 0.47)||0.39 (0.23, 0.54)||0.13 (0.0, 0.26)||0.19 (0.05, 0.32)|
|4||0.76 (0.63, 0.88)||0.79 (0.67, 0.91)||0.74 (0.61, 0.88)||0.81 (0.69, 0.94)||0.92 (0.84, 1.00)|
|5||0.89 (0.80, 0.99)||0.78 (0.65, 0.91)||0.89 (0.79, 0.99)||0.87 (0.76, 0.97)||0.92 (0.84, 1.00)|
|UIP||ILD associated with CVID|
|Bronchiolar disease||Pulmonary veno-occlusive disease|
Clinicians tended to be more confident than radiologists in the early steps of the evaluation process, although the number of confident diagnoses increased for all observers as more information was provided (Table 7)
Self-reported Confidence Level†
|Clinician A||Clinician B||Clinician C||Radiologist A||Radiologist B|
|Diagnosis Matched Final
|p Value*||< 0.001||0.665||< 0.001||0.031||1|
| p Value*||0.003||0.10||0.026||0.17|| 1|
Total agreement (3 clinicians, 2 radiologists, and 2 pathologists) on a final diagnosis was reached in 47 (81%) of the cases (IPF, n = 28; NSIP, n = 13; RBILD, n = 1; hypersensitivity pneumonitis, n = 1; other, n = 4). In 6 (10%) cases all but one rater was in agreement (IPF, n = 2; NSIP, n = 2; hypersensitivity pneumonitis, n = 2). In four cases all but two raters were in agreement (RBILD, n = 1; bronchiolar, n = 3). In one case the raters were equally divided between RBILD versus a bronchiolar disorder.
Assigning a diagnosis to a patient with interstitial lung disease is difficult and at times imprecise. In clinical practice, pulmonary physicians, radiologists, and pathologists are separated by time, geographic location, and different schedules. This challenges the pulmonary physician confronted with the actual patient in the clinic to individually integrate and analyze the gathered clinical, radiographic, and pathologic data and make decisions about a given patient rather than discussing the data with the radiologist and/or pathologist in a collective manner. In an attempt to improve diagnostic accuracy the ATS/ERS consensus statement recommended a dynamic integrated approach to the diagnosis of interstitial lung diseases that involves an interaction between clinicians, radiologists, and pathologists to achieve a final clinical, radiologic, and pathologic diagnosis (14). In this study we examined whether such a dynamic interactive process impacted the perceived diagnosis, and confidence level for that diagnosis, for a group of consecutive patients with suspected idiopathic interstitial pneumonia. We demonstrate that:
The level of agreement between observers and diagnostic confidence improves as more data (clinical, radiographic, and pathologic) are provided.
Patients may not require surgical lung biopsy when the clinical and radiographic (HRCT) impression is consistent with a confident diagnosis of IPF.
For non-IPF idiopathic interstitial pneumonias, a surgical lung biopsy may be required as a final consensus diagnosis is most influenced by the histopathologic pattern in this setting.
This study quantifies how a dynamic exchange of clinical, radiographic, and pathologic information changes the diagnostic impression of expert physicians when assigning a diagnosis of idiopathic interstitial pneumonia. Exchanges of information lead to both changes in diagnosis and improved confidence with the assigned diagnosis. The effect of interaction between physicians seems most critical for non-IPF idiopathic interstitial pneumonia cases. After review of the clinical, radiographic, and pathologic information the pathologists thought that IPF was the diagnosis in 30 cases. Of these 30 cases, the two radiologists and three clinicians each identified 14, 15, 22, 23, and 23 (respectively) after the review of clinical/radiographic information before obtaining histopathologic information. Importantly, in almost half of the overall cases the histopathologic findings were critical in determining the final diagnosis from each participant. The discussion and exchange of clinical, radiographic, and histopathologic information also tended to improve each participant's confidence in their diagnosis. This is important as diagnoses at any stage rendered with a high level of confidence tended to correspond with the final consensus diagnosis. This suggests that when a physician expresses uncertainty about an idiopathic interstitial pneumonia diagnosis, extra effort should be made to seek additional information and/or opinions with the hope of achieving greater confidence in their diagnostic impression. Furthermore, these data strongly support the role of surgical lung biopsy in patients with a non-IPF HRCT and clinical scenario (1, 14).
A unique feature of this study is the lack of the arbitrary assignment of a gold standard for the final diagnosis. By allowing all participants the opportunity to change their initial diagnostic interpretations we monitored how diagnoses changed with more information and the level of agreement at each stage in the evaluation process. Our data demonstrate that a change in the final clinical–radiographic–pathologic diagnosis is particularly likely to occur in patients with a clinical and radiographic scenario suggestive of non-IPF idiopathic interstitial pneumonia. Importantly, providing the clinical and radiographic information to pathologists led to an alternative or clarified pathologic opinion in 19% of the cases. These findings extend previous work that evaluated agreement between clinicians, radiologists, and pathologists (5, 10, 16, 21–24).
The level of agreement between participants was good at onset and improved with discussion of additional clinical and histopathologic information. The highest initial agreement was between radiologists. Radiologists were also less likely than clinicians to believe that the clinical scenario was characteristic of IPF and were more likely, compared with clinicians, to change their diagnosis on the basis of histopathologic findings. Previous work demonstrated the limited sensitivity (16, 24, 25) yet high positive predictive value (10, 12, 16, 26) of HRCT for detecting IPF. Our data highlight the value of clinicians reviewing the actual HRCT with a radiologist versus relying on reported findings. The greatest improvement for the clinician's interobserver agreement occurred after the discussion of clinical and radiographic features with radiologists.
The ATS/ERS consensus panel on idiopathic pulmonary fibrosis states that in the absence of a surgical lung biopsy, the diagnosis of IPF/UIP remains uncertain. However, the panel proposed diagnostic criteria that could be used by clinicians to increase the likelihood of a correct clinical diagnosis of IPF (Table 8)
Major criteria (all must be present)
|1. Exclusion of other known causes of ILD, such as certain drug toxicities, environmental exposures, and connective tissue diseases|
|2. Abnormal pulmonary function studies that include evidence of restriction (reduced VC, often with an increased FEV1/FVC ratio) and impaired gas exchange [increased (A–a)PO2 with rest or exercise or decreased DLCO]|
|3. Bibasilar reticular abnormalities with minimal ground glass opacities on HRCT scans|
|4. Transbronchial lung biopsy or bronchoalveolar lavage showing no features to support an alternative diagnosis|
|Minor criteria (three of four must be present)|
|1. Age, 50 yr|
|2. Insidious onset of otherwise unexplained dyspnea on exertion|
|3. Duration of illness greater than 3 mo|
|4. Bibasilar, inspiratory crackles (dry or “Velcro” type in quality)|
A limitation of our study is the use of participants with significant expertise in the field of interstitial lung disease. Some investigators have suggested that expert radiologists exhibit improved interobserver agreement compared with radiologists with lesser expertise (27–30). No similar data have examined this topic in the evaluation of suspected idiopathic interstitial pneumonia by less experienced clinicians. Additional data are required to assess the role of a dynamic, interactive diagnostic process in the hands of less experienced clinicians, radiologists, and pathologists. These data also do not address how individual personalities may have impacted the degree of agreement (or disagreement) at various steps throughout the experiment. The participants in this study are recognized, senior experts and are accustomed to rendering an “expert opinion.” As such, other physicians may render diagnostic opinions that are more (or less) malleable compared with this study group.
In summary, our data quantify the essence of a dynamic clinical, radiographic, and pathologic approach to the diagnosis of patients with suspected idiopathic interstitial pneumonia. We believe that at a minimum the evaluation of patients with idiopathic interstitial pneumonia should include an interaction between the clinician and thoracic radiologist. If this initial step results in a confident diagnosis of IPF a surgical lung biopsy is not required. In all other cases a surgical lung biopsy is likely to impact the final diagnosis and should be performed if possible. If a surgical lung biopsy is available, the pathologist should be involved in the diagnostic analytic discussion. It appears the current gold standard for the diagnosis of idiopathic interstitial pneumonia is a dynamic integrated process that requires direct interaction between clinicians and radiologists as well as pathologists when a surgical lung biopsy is available. Further research to clarify this process, examine its effect in the hands of less experienced physicians, and evaluate the implications in terms of response to therapy and prognosis is required.
The University of Michigan Fibrotic Lung Disease Network includes: University of Michigan, Division of Pulmonary and Critical Care, Ann Arbor, MI—D. Arenberg, W. Bria, D. Dahlgren, C. Grum, V. Lama, T. Ojo, M. Peters-Golden, R. Simon, T. Sisson, T. Standiford, V. Thannickal, E. White; Internal Medicine Clinic, Alpena, MI—P. Bachwich, C. Easton, J. Mazur; The Lung Center, Battle Creek, MI—S. Chaparala, G. Harrington, N. Potempa; Bay City, MI—S. Manawar, J. Summer; Clawson, MI—P. Hukku, J. Sung; Clinton Township, MI—R. Babcock; Pulmonary and Critical Care Medicine Consultants, Commerce, MI—J. Belen, M. Dunn, D. Maxwell, R. Reagle, R. Sherman, S. Simecek; Oakwood Hospital, Dearborn, MI—L. Victor; Henry Ford Hospital, Detroit, MI—B. DiGiovine, M. Eichenhorn, R. Hyzy, J. Popovich, Jr., D. Spizarny; Botsford General Hospital, Farmington Hills, MI—B. Rabinowitz; Pulmonary and Critical Care Specialists, Farmington Hills, MI—G. Ferguson, P. Kaplan, S. Sklar, W. VanderRoest; Pulmonary Associates, PC, Flint, MI—O. Filos, V. Rao, M. V. Thomas, J. Varghese, J. Vyskocil, F. Wadenstorer; Grand Valley Internal Medicine, Grand Rapids, MI—J. Cantor, W. Katz, R. Johnson, Jr., D. Listello, J. Wilt; Michigan Medical Professional Company, Grand Rapids, MI—C. Acharya, W. Couwenhoven, T. Daum, M. Harrison, M. Koets, G. Sandman, G. VanOtteren; Michigan Medical, PC, Holland, MI—S. Kraker; Huntington Woods, MI—M. Greenberger, A. O'Neill, D. Wu; Pulmonary Clinics of Southern Michigan, Jackson, MI—R. C. Albertson, III, J. Chauncey, T. Murray, G. Patten; Associated Pulmonary and Critical Care Specialists, PC, Kalamazoo, MI—T. Abraham, J. Dirks, B. Dykstra, G. Grambau, J. Schoell; Pulmonary and Critical Care Associates, PC, Kalamazoo, MI—R. Brush, S. Jefferson, J. Miller, S. Schuldheisz, M. Warlick; Pulmonary and Critical Care Consultants, Lansing, MI—J. Armstrong, A. Atkinson, T. Kantra, L. Rawsthorne, D. Young; Pulmonary Services, Lansing, MI—A. Abbasi, C. M. Gera, G. Kashyap, J. Morlock; Respiratory Medicine, Marquette, MI—S. Danek, A. Saari; Midland, MI—S. Yadam; Central Michigan Healthcare System, Mt. Pleasant, MI—E. Obeid; Muskegon Pulmonary Associates, Muskegon, MI—D. Hoch, A. Kleaveland; Owosso Medical Group, Owosso, MI—A. Allam, M. A. Gad, Jr.; Lung Associates, Pontiac, MI—A. Desai, U. Dhanjal, A. Sethi; St. Joseph's Hospital, Pontiac, MI—F. Ahmad, L. Kaiser, L. Rosenthal, D. Sak; Physician HealthCare Network, Port Huron, MI—R. Ailani, M. Basha, A. Hadar, S. Holstine; Pulmonary, Critical Care, and Sleep, PC, Rochester Hills, MI—M. W. Al-Ameri, R. Go, M. Kashlan; Rochester, MI—K. Aggarwal; Roseville, MI—W. Hanna, R. Marchese; William Beaumont Hospital, Royal Oak, MI—R. Begle, D. Erb, K. P. Ravikrishnan, J. Seidman, S. Sherman; Saginaw, MI—R. Agarwal, F. Ansari, T. Damuth, C. Indira; Spring Lake, MI—M. Ivey; Lakeside Healthcare Specialists, St. Joseph, MI—S. Deskins, A. Palmer, S. Shastri; Pulmonary and Critical Care Associates, St. Clair Shores, MI and Troy, MI—R. DiLisio, S. Galens, K. Grady, D. Harrington, R. Herbert, C. Hughes, J. Lee, A. Starrico, K. Stevens, M. Trunsky, W. Ventimiglia; Taylor, MI—D. Mahajan. Pulmonary Medicine Associates, Warren, MI—H. Kaplan, L. Tankanow; Henry Ford Wyandotte Hospital, Wyandotte, MI—M. Pensler; Toledo Pulmonary and Sleep Specialists, Toledo, OH—F. O. Horton, III, A. Nathanson, R. Wainz.
|1.||Flaherty KR, Toews GB, Travis WD, Colby TV, Kazerooni EA, Gross BH, Jain A, Strawderman RL III, Paine R III, Flint A, et al. Clinical significance of histological classification of idiopathic interstitial pneumonia. Eur Respir J 2002;19:275–283.|
|2.||Flaherty KR, Travis WD, Colby TV, Toews GB, Kazerooni EA, Gross BH, Jain A, Strawderman RL III, Flint A, Lynch JP III. et al. Histologic variability in usual and nonspecific interstitial pneumonias. Am J Respir Crit Care Med 2001;164:1722–1727.|
|3.||Katzenstein ALA, Fiorelli RF. Nonspecific interstitial pneumonia/fibrosis: histologic features and clinical significance. Am J Surg Pathol 1994;18:136–147.|
|4.||Nagai S, Kitaichi M, Itoh H, Nishimura K, Izumi T, Colby TV. Idiopathic nonspecific interstitial pneumonia/fibrosis: comparison with idiopathic pulmonary fibrosis and BOOP. Eur Respir J 1998;12:1010–1019.|
|5.||Nicholson AG, Colby TV, DuBois RM, Hansell DM, Wells AU. The prognostic significance of the histologic pattern of interstitial pneumonia in patients presenting with the clinical entity of cryptogenic fibrosing alveolitis. Am J Respir Crit Care Med 2000;162:2213–2217.|
|6.||Travis WD, Matsui K, Moss J, Ferrans VJ. Idiopathic nonspecific intersititial pneumonia: prognostic signficance of cellular and fibrosing patterns. Am J Surg Pathol 2000;24:19–33.|
|7.||Bjoraker JA, Ryu JH, Edwin MK, Myers JL, Tazelaar HD, Schoreder DR, Offord KP. Prognostic significance of histopathologic subsets in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 1998;157:199–203.|
|8.||Lama VN, Flaherty KR, Toews GB, Colby TV, Travis WD, Long Q, Murray S, Kazerooni EA, Gross BH, Lynch JP III, et al. Prognostic value of desaturation during a 6-minute walk test in idiopathic interstitial pneumonia. Am J Respir Crit Care Med 2003;168:1084–1090.|
|9.||Hunninghake GW, Lynch DA, Galvin JR, Muller N, Schwartz D, King TE Jr, Lynch JP III, Hegele R, Waldron JA Jr, Colby TV, et al. Radiologic findings are strongly associated with a pathologic diagnosis of usual interstitial pneumonia. Chest 2003;124:1215–1223.|
|10.||Hunninghake GW, Zimmerman MB, Schwartz DA, King TE Jr, Lynch J, Hegele R, Waldron J, Colby T, Muller N, Lynch D, et al. Utility of a lung biopsy for the diagnosis of idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 2001;164:193–196.|
|11.||Raghu G. Interstitial lung disease—a diagnostic approach: are CT scan and lung biopsy indicated for every patient? Am J Respir Crit Care Med 1995;151:909–914.|
|12.||Raghu G, Mageto YN, Lockhart D, Schmidt RA, Wood DE, Godwin JD. The accuracy of the clinical diagnosis of new-onset idiopathic pulmonary fibrosis and other interstitial lung disease: a prospective study. Chest 1999;116:1168–1174.|
|13.||Katzenstein ALA, Myers JL. Idiopathic pulmonary fibrosis: clinical relevance of pathologic classification. Am J Respir Crit Care Med 1998;157:1301–1315.|
|14.||American Thoracic Society, European Respiratory Society. American Thoracic Society/European Respiratory Society international multidisciplinary consensus classification of the idiopathic interstitial pneumonias. Am J Respir Crit Care Med 2002;165:277–304.|
|15.||Nicholson AG, Fulford LG, Colby TV, Du Bois RM, Hansell DM, Wells AU. The relationship between individual histologic features and disease progression in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med 2002;166:173–177.|
|16.||Flaherty KR, Thwaite EL, Kazerooni EA, Gross BH, Toews GB, Colby TV, Travis WD, Mumford JA, Murray S, Flint A, et al. Radiological versus histological diagnosis in UIP and NSIP: survival implications. Thorax 2003;58:143–148.|
|17.||Flaherty KR, King TE Jr, Raghu G, Kazerooni EA, Gross BH, Travis WD, Colby TV, Lynch JP III, Long Q, Murray S, et al. A confident clinical and radiographic diagnosis of usual interstitial pneumonia or nonspecific interstitial pneumonia is associated with a higher likelihood of finding histologic UIP or NSIP on biopsy [abstract]. Am J Respir Crit Care Med 2004;169:A706.|
|18.||Flaherty KR, Lama VN, King TE Jr, Raghu G, Lynch JP III, Travis WD, Colby TV, Kazerooni EA, Gross BH, Long Q, et al. Clinical–radiographic–pathologic diagnosis in idiopathic interstitial pneumonia (IIP) [abstract]. Am J Respir Crit Care Med 2003;167:A299.|
|19.||Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–174.|
|20.||Thompson JR. Estimating equations for κ statistics. Stat Med 2001;20:2895–2906.|
|21.||Johkoh T, Muller NL, Colby TV, Ichikado K, Taniguchi H, Kondoh Y, Fujimoto K, Kinoshita M, Arakawa H, Yamada H, et al. Nonspecific interstitial pneumonia: correlation between thin-section CT findings and pathologic subgroups in 55 patients. Radiology 2002;225:199–204.|
|22.||Kazerooni EA, Martinez FJ, Flint A, Jamadar DA, Gross BH, Spizarny DL, Cascade PN, Whyte RI, Lynch JP III, Toews G. Thin-section CT obtained at 10-mm increments versus limited three-level thin-section CT for idiopathic pulmonary fibrosis: correlation with pathologic scoring. AJR Am J Roentgenol 1997;169:977–983.|
|23.||Wells AU, Rubens MB, du Bois RM, Hansell DM. Serial CT in fibrosing alveolitis: prognostic significance of the initial pattern. AJR Am J Roentgenol 1993;161:1159–1165.|
|24.||MacDonald SLS, Rubens MB, Hansell DM, Copley SJ, Desai SR, DuBois RM, Nicholson AG, Colby TV, Wells AU. Nonspecific interstitial pneumonia and usual interstitial pneumonia: comparative appearances at and diagnostic accuracy of thin-section CT. Radiology 2001;221:600–605.|
|25.||Johkoh T, Muller NL, Cartier Y, Kavanagh PV, Hartman TE, Akira M, Ichikado K, Ando M, Nakamura H. Idiopathic interstitial pneumonias: diagnostic accuracy of thin-section CT in 129 patients. Radiology 1999;211:555–560.|
|26.||Swensen SJ, Aughenbaugh GL, Myers JL. Diffuse lung disease: diagnostic accuracy of CT in patients undergoing surgical biopsy of the lung. Radiology 1997;205:229–234.|
|27.||Collins CD, Wells AU, Hansell DM, Morgan RA, MacSweeney JE, du Bois RM, Rubens MB. Observer variation in pattern type and extent of disease in fibrosing alveolitis on thin section computed tomography and chest radiography. Clin Radiol 1994;49:236–240.|
|28.||Tsuda K, Yu KK, Coakley FV, Srivastav SK, Scheidler JE, Hricak H. Detection of extracapsular extension of prostate cancer: role of fat suppression endorectal MRI. J Comput Assist Tomogr 1999;23:74–78.|
|29.||Yu KK, Hricak H, Alagappan R, Chernoff DM, Bacchetti P, Zaloudek CJ. Detection of extracapsular extension of prostate carcinoma with endorectal and phased-array coil MR imaging: multivariate feature analysis. Radiology 1997;202:697–702.|
|30.||Lee MG, Baker ME, Sostman HD, Spritzer CE, Paine S, Paulson EK, Keogan MT. The diagnostic accuracy/efficacy of MRI in differentiating hepatic hemangiomas from metastatic colorectal/breast carcinoma: a multiple reader ROC analysis using a jackknife technique. J Comput Assist Tomogr 1996;20:905–913.|
|31.||American Thoracic Society. Idiopathic pulmonary fibrosis: diagnosis and treatment. International consensus statement. Am J Respir Crit Care Med 2000;161:646–664.|