Comments
Abstract
Rationale: One important concern during high-flow nasal cannula (HFNC) therapy in patients with acute hypoxemic respiratory failure is to not delay intubation.
Objectives: To validate the diagnostic accuracy of an index (termed ROX and defined as the ratio of oxygen saturation as measured by pulse oximetry/FiO2 to respiratory rate) for determining HFNC outcome (need or not for intubation).
Methods: This was a 2-year multicenter prospective observational cohort study including patients with pneumonia treated with HFNC. Identification was through Cox proportional hazards modeling of ROX association with HFNC outcome. The most specific cutoff of the ROX index to predict HFNC failure and success was assessed.
Measurements and Main Results: Among the 191 patients treated with HFNC in the validation cohort, 68 (35.6%) required intubation. The prediction accuracy of the ROX index increased over time (area under the receiver operating characteristic curve: 2 h, 0.679; 6 h, 0.703; 12 h, 0.759). ROX greater than or equal to 4.88 measured at 2 (hazard ratio, 0.434; 95% confidence interval, 0.264–0.715; P = 0.001), 6 (hazard ratio, 0.304; 95% confidence interval, 0.182–0.509; P < 0.001), or 12 hours (hazard ratio, 0.291; 95% confidence interval, 0.161–0.524; P < 0.001) after HFNC initiation was consistently associated with a lower risk for intubation. A ROX less than 2.85, less than 3.47, and less than 3.85 at 2, 6, and 12 hours of HFNC initiation, respectively, were predictors of HFNC failure. Patients who failed presented a lower increase in the values of the ROX index over the 12 hours. Among components of the index, oxygen saturation as measured by pulse oximetry/FiO2 had a greater weight than respiratory rate.
Conclusions: In patients with pneumonia with acute respiratory failure treated with HFNC, ROX is an index that can help identify those patients with low and those with high risk for intubation.
Clinical trial registered with www.clinicaltrials.gov (NCT 02845128).
Delayed intubation of spontaneously breathing patients with hypoxemic acute respiratory failure is associated with an excess mortality. Although several studies have described factors associated with higher risk for intubation in patients treated with high-flow oxygen, none was designed and powered to validate them. We recently described the utility of the ROX index, defined as the ratio of oxygen saturation as measured by pulse oximetry/FiO2 to respiratory rate, for determining which patients treated with nasal high flow will not require intubation.
We confirm the ROX index’s accuracy for predicting nasal high-flow oxygen outcome of pneumonia-related respiratory failure: ROX index greater than or equal to 4.88 measured at 2, 6, or 12 hours is a determinant of high-flow success. Additionally, we identified and validated values at different time-points of the ROX index, which predict high-flow failure. Because the ROX index is easily measured and repeated at the bedside, we show that changes of the index over time are also predictive of high-flow outcome. This index can thus be incorporated in the day-to-day clinical decision-making process of critically ill patients treated with nasal high flow.
A growing interest in noninvasive management of acute hypoxemic respiratory failure (AHRF) has been fueled by the advent of high-flow nasal cannula oxygen therapy (HFNC) (1) and by recent data showing that use of HFNC was associated with lower mortality, more ventilator-free days, and lower risk for intubation in subsets of patients with PaO2/FiO2 less than or equal to 200 mm Hg or in those who were immunocompromised in comparison with noninvasive ventilation (NIV) or standard oxygen (2, 3). These positive results followed physiologic studies showing improvements in oxygenation, lung mechanics, and comfort associated with HFNC (4–6). Pneumonia, which is a frequent cause of acute respiratory distress syndrome (7), was the most frequent cause of AHRF in these studies (8). This has led clinicians to try this technique in patients with the most severe respiratory failure, those precisely with acute respiratory distress syndrome (9, 10).
A consequence of the increasing use of HFNC is the risk of delaying a needed intubation. This is an important concern because a large body of evidence has shown that patients that fail NIV management of de novo AHRF have a worse outcome. This has been convincingly shown with NIV (11), especially in patients treated for pneumonia (12) and also with HFNC (13). The new European Respiratory Society/American Thoracic Society guidelines for acute respiratory failure made no formal recommendation for NIV in this context (14). In addition, there are no prospectively validated and accepted intubation criteria for AHRF. This may lead to considerable differences among clinicians in terms of timing of intubation that could impact outcome (15). A core set of parameters that should prompt intubation are generally agreed on, but precise cutoffs may vary considerably. Therefore, to describe clinical variables that could be easily used at the bedside to help decide on intubation in a timely fashion is a point of special interest to avoid delaying a needed intubation. To address this unmet need, we recently described the ROX index, defined as the ratio of oxygen saturation as measured by pulse oximetry (SpO2)/FiO2 to respiratory rate (RR). This index outperformed the diagnostic accuracy of the two variables separately (16). Patients who had a ROX index greater than or equal to 4.88 after 12 hours of HFNC therapy were less likely to be intubated, even after adjusting for potential covariates. Like any other scoring system, an independent validation of the score is necessary. We therefore undertook a multicenter, prospective study to validate the ROX index’s diagnostic accuracy for determining which patients will succeed and which will fail on HFNC.
This is a multicenter prospective observational cohort study performed over a 2-year period (2016–2017) including patients with pneumonia treated with HFNC who were admitted in five different ICUs in Spain and France (see the online supplement for detail). Local ethics committee approved the studies in Spain and written patient’s informed consent was obtained before inclusion. For the French centers, the Ethics Committee of the French Intensive Care Society also approved the study. Because of its purely observational design, written consent was not required. Patients were informed of the nature of the study, its purpose and objectives, and of their right to decline participation.
All consecutive patients admitted to the ICU with pneumonia and treated with HFNC were included. No patients declined to participate. Pneumonia was diagnosed according to Infectious Diseases Society of America/American Thoracic Society 2007 guidelines (17). Patients younger than 18 years old, patients with indication for immediate intubation (18), and those with do-not-intubate order were excluded. Patients electively intubated for diagnostic or therapeutic procedures (fibrobronchoscopy, surgery) were also not included. Patients were followed until death or hospital discharge.
Management of HFNC therapy and criteria for mechanical ventilation (MV) did not differ between training (16) and validation studies. High flow was provided either with the Optiflow device (MR850 heated humidified RT202 delivery tubing and RT050/051 nasal cannula; Fisher and Paykel Healthcare) or with Airvo 2 (Fisher and Paykel Healthcare). HFNC was initiated with a minimum flow of 30 L/min with a FiO2 of 1 in those patients that were unable to maintain an SpO2 higher than 92% and an RR of 25 breaths/min or greater while receiving standard oxygen administered through a face mask at 10 L/min or more. Then, FiO2 was titrated targeting an SpO2 above 92% and flow rate was adjusted according to the maximum tolerated. In all patients, the maximum tolerated flow was achieved within the first 10 minutes of HFNC treatment.
HFNC failure was defined as the subsequent need for invasive MV. The participating ICUs agreed on a common set of intubation criteria to help the attending physicians decide when to intubate. These criteria included a decreased level of consciousness (Glasgow coma score <12), cardiac arrest/arrhythmias and severe hemodynamic instability (norepinephrine >0.1 μg/kg/min), or persisting or worsening respiratory condition defined as at least two of the following criteria: failure to achieve correct oxygenation (PaO2 <60 mm Hg or SpO2 <90% despite HFNC flow ≥30 L/min and FiO2 of 1), respiratory acidosis (PaCO2 >50 mm Hg or PvCO2 >55 mm Hg with pH <7.25), RR >30 breaths/min, or inability to clear secretions.
In the previous prospective exploratory study, the ROX index was calculated from the respiratory variables that were significantly different among groups (16), aiming to obtain an additive effect on the accuracy for discriminating between patients who succeeded and those who failed with HFNC. The ROX index was defined as the ratio of SpO2/FiO2 (%) to RR (breaths/min). In the numerator were placed the variables with a positive association with HFNC success, whereas in the denominator were placed those variables that had an inverse relation with HFNC success. In the present study, the ROX index was not used to decide on intubation, which was guided by criteria agreed on and defined previously.
Quantitative variables were expressed as median (interquartile range), categorical variables were expressed as frequency (percentage). Continuous variables were compared using the Student’s t test or Mann-Whitney U test, as appropriate. Differences in categorical variables were assessed with chi-square or Fisher exact test, as appropriate. To assess the accuracy of different variables for correctly classifying patients who would succeed or fail on HFNC, receiver operating characteristic curves (ROCs) were performed and the areas under the curves (AUROCs) were calculated. Differences between ROC curves were estimated using a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix (19). Because the effect of RR and FiO2, and ROX index in predicting HFNC success are the opposite, we have used the AUROC of the inverse of RR and FiO2 for its comparison with ROX index AUROC. Because reported intubation rates in AHRF treated with HFNC range from 28% to 48% (3, 9, 10, 16), the sample size was estimated assuming an intubation rate of 35%. According to the previous reported value of the ROX index after 12 hours, we predicted AUROC value of 0.8 in the validation cohort with a noninferiority margin of 0.08. The noninferiority design included a power of 0.8 and a type I error of 0.05. These conditions required 189 patients.
We have used the previously defined cutoff point described for the ROX index of 4.88 (16). According to this value, Kaplan-Meier curves were used to determine the probability of MV for patients with higher and lower ROX index at different time points. These curves were compared using the log-rank test. To identify if the ROX index was associated with the need for MV, Cox proportional hazards modeling was chosen, while simultaneously adjusting for other covariates. Variables with P value less than 0.1 in the univariate analysis and other variables that could influence the value of the ROX index were considered as potential covariates. We have also adjusted by severity scores (Acute Physiology and Chronical Health Evaluation II and Sequential Organ Failure Assessment score). To prevent model overfitting, we introduced all potential confounding one at a time. Moreover, a general model for predicting the need for MV in the overall cohort (validation and training together) was constructed using those variables with a P value less than 0.1 in the univariate analysis and the variable ROX index greater than or equal to 4.88 at different time points. Further validation was also performed in the FLORALI cohort (3).
Finally, we investigated a cutoff of the ROX index with higher specificity for predicting the risk of HFNC failure. Differences in the values of the ROX index at different time points between patients who succeeded and those patients who fail on HFNC were also assessed and their role was confirmed using the Cox proportional hazards modeling and adjusting for the previous value of the ROX index. A two-sided P value of 0.05 or less was considered statistically significant. Statistical analyses were performed using the STATA 14 software (Stata Corp. Stata Statistical Software: Release 14. Statistical software; StataCorp LP).
A total of 191 and 157 patients were treated with HFNC in the validation and in the training cohort, respectively. Their baseline characteristics are reported in Table E1 in the online supplement. Results regarding the training cohort were reported elsewhere (16). Patients included in the validation cohort were older and presented a higher prevalence of chronic heart failure compared with those patients included in the training cohort. Moreover, in the validation cohort, the type of pneumonia was more frequently a healthcare-associated pneumonia and patients also presented a higher prevalence of shock (39 [20.4%] patients vs. 13 [8.4%] patients; P = 0.002) and a trend toward high Acute Physiology and Chronical Health Evaluation II score. Higher SpO2/FiO2 values were observed and higher flow rates were used in the validation cohort throughout the study period (see Table E2). Finally, whereas no differences in ICU mortality or length of stay were observed between the two cohorts, patients included in the validation cohort presented a higher hospital mortality (50 [27.3%] patients vs. 22 [14.2%] patients; P = 0.003).
In the validation cohort, 68 (35.6%) patients required subsequent intubation and MV. The median duration of HFNC therapy in success and failure groups was 96 (48–144) hours and 24 (12–60) hours, respectively (P < 0.001). After 2, 6, and 12 hours, 190 (99.5%), 182 (95.2%), and 169 (88.4%) patients were still on HFNC, respectively. Within the first 2 hours of HFNC therapy only one (0.5%) patient needed to be intubated. Between 2 and 6 hours, six (3.1%) patients were intubated and between 6 and 12 hours, 11 (5.7%) needed to be intubated. The cumulative risk of being free of MV in HFNC failure is represented in Figure E1. HFNC failure was associated with higher ICU and hospital mortality and length of stay (see Table E3). HFNC failure patients had a higher prevalence of immunosuppression (Table 1). HFNC success patients had higher SpO2/FiO2 and lower RR after HFNC and throughout the study period (Table 2). Likewise, higher ROX index values were observed in those patients who succeeded with HFNC. AUROC values of different variables are reported in Table 3. There was no difference in the diagnostic accuracy of the ROX index between the validation and training cohorts whatever the time point (see Table E4). In the validation cohort, AUROC values of ROX index for discriminating those patients who will succeed with HFNC were higher than those found with SpO2/FiO2 at 18 hours, RR at 6, 12, and 24 hours, and FiO2 at 12 and 18 hours (see Table E5). We considered the same cutoff point for the ROX index than previously reported (16).
| Success (n = 123) | Failure (n = 68) | P Value | |
|---|---|---|---|
| Sex, male, n (%) | 76 (61.8) | 42 (61.8) | 0.997 |
| Age, yr | 64 (52–73) | 60 (52–71) | 0.412 |
| Comorbidities, n (%) | |||
| Immunosuppression | 32 (26.0) | 28 (41.2) | 0.031 |
| Chronic heart failure | 26 (21.1) | 14 (20.6) | 0.929 |
| Chronic liver disease | 13 (10.6) | 3 (4.4) | 0.141 |
| Chronic respiratory disease | 45 (36.6) | 24 (35.6) | 0.859 |
| Chronic renal failure | 9 (7.3) | 6 (8.8) | 0.711 |
| Type of pneumonia, n (%) | 0.177 | ||
| Bacterial | |||
| Community acquired | 78 (63.4) | 34 (50.0) | |
| Health care related | 31 (25.2) | 25 (36.8) | |
| Viral pneumonitis | 14 (11.4) | 9 (13.2) | |
| Pneumonia severity index | 112 (74–153) | 121 (94–146) | 0.445 |
| APACHE II of 24-h ICU admission | 16 (11–21) | 18 (14–21) | 0.140 |
| SOFA score at ICU admission | 5 (2–8) | 4 (3–7) | 0.198 |
| NIV requirement, n (%) | 6 (4.9) | 4 (5.9) | 0.746 |
| Number of quadrants affected on chest X-ray | 2.5 (2–4) | 3 (2–4) | 0.095 |
| Variable | Time (h) | Success (n = 123) | Failure (n = 68) | P Value |
|---|---|---|---|---|
| SpO2/FiO2 | Prior to HFNC | 180 (113–223) | 106 (94–190) | 0.005 |
| 2 | 155 (106–165) | 109 (96–159) | 0.003 | |
| 6 | 160 (127–192) | 115 (98–167) | 0.001 | |
| 12 | 165 (127–200) | 113 (97–190) | 0.001 | |
| 18 | 176 (140–203) | 118 (98–193) | 0.002 | |
| 24 | 194 (152–239) | 120 (96–192) | <0.001 | |
| RR, breaths/min | Prior to HFNC | 28 (26–32) | 32 (25–34) | 0.778 |
| 2 | 25 (22–28) | 28 (22–32) | 0.023 | |
| 6 | 24 (20–27) | 26 (22–30) | 0.003 | |
| 12 | 23 (19–26) | 26 (21–29) | <0.001 | |
| 18 | 22 (18–26) | 25 (22–28) | 0.001 | |
| 24 | 21 (18–25) | 24 (20–30) | 0.004 | |
| PaCO2, mm Hg | Prior to HFNC | 36 (31–42) | 38 (30–45) | 0.468 |
| 2 | 38 (33–44) | 38 (33–47) | 0.317 | |
| 6 | 38 (33–46) | 37 (32–45) | 0.650 | |
| 12 | 38 (32–44) | 36 (31–43) | 0.940 | |
| 18 | 39 (33–45) | 35 (28–43) | 0.230 | |
| 24 | 38 (32–43) | 34 (28–44) | 0.415 | |
| Flow, L/min | Prior to HFNC | 7 (4–12) | 9 (5–15) | 0.140 |
| 2 | 50 (40–60) | 50 (40–60) | 0.256 | |
| 6 | 50 (40–60) | 50 (40–60) | 0.729 | |
| 12 | 50 (40–60) | 40 (40–55) | 0.185 | |
| 18 | 50 (40–60) | 40 (40–53) | 0.140 | |
| 24 | 50 (40–59) | 45 (40–60) | 0.495 | |
| ROX index | Prior to HFNC | 5.81 (4.21–8.00) | 4.06 (2.98–6.54) | 0.169 |
| 2 | 5.71 (4.62–7.28) | 4.43 (3.57–6.16) | 0.001 | |
| 6 | 6.55 (5.44–8.17) | 4.86 (3.43–6.64) | <0.001 | |
| 12 | 7.53 (5.83–9.93) | 4.78 (3.67–6.99) | <0.001 | |
| 18 | 8.60 (6.30–10.03) | 5.10 (3.84–7.31) | <0.001 | |
| 24 | 8.68 (6.93–11.77) | 5.05 (4.00–6.74) | <0.001 | |
| SpO2, % | Prior to HFNC | 92 (90–96) | 93 (90–96) | 0.381 |
| 2 | 97 (95–99) | 96 (94–98) | 0.032 | |
| 6 | 97 (96–98) | 96 (95–99) | 0.015 | |
| 12 | 97 (95–99) | 96 (94–97) | <0.001 | |
| 18 | 97 (96–99) | 96 (94–98) | 0.001 | |
| 24 | 97 (96–99) | 96 (95–98) | 0.013 | |
| FiO2 | Prior to HFNC | 0.50 (0.40–0.81) | 0.89 (0.50–0.98) | 0.000 |
| 2 | 0.60 (0.60–0.90) | 0.80 (0.60–1.00) | 0.002 | |
| 6 | 0.60 (0.50–0.75) | 0.84 (0.60–1.00) | <0.001 | |
| 12 | 0.60 (0.50–0.75) | 0.85 (0.50–1.00) | 0.291 | |
| 18 | 0.53 (0.45–0.70) | 0.80 (0.50–1.00) | 0.001 | |
| 24 | 0.50 (0.40–0.63) | 0.80 (0.50–1.00) | <0.001 | |
| Lactate, mmol/L | Prior to HFNC | — | — | — |
| 2 | 1.60 (1.00–2.30) | 1.50 (1.00–2.35) | 0.363 | |
| 6 | 1.30 (0.88–1.90) | 1.40 (1.09–2.08) | 0.495 | |
| 12 | 1.40 (1.00–2.00) | 1.45 (1.10–2.08) | 0.933 | |
| 18 | 1.40 (0.95–1.89) | 1.44 (1.00–2.50) | 0.644 | |
| 24 | 1.22 (0.90–1.90) | 1.27 (1.03–2.48) | 0.868 |
| Variable | Time | AUROC | 95% CI | P Value |
|---|---|---|---|---|
| SpO2/FiO2 | Prior to HFNC | 0.641 | 0.550–0.731 | 0.002 |
| 2 h | 0.648 | 0.561–0.734 | 0.001 | |
| 6 h | 0.672 | 0.580–0.764 | <0.001 | |
| 12 h | 0.695 | 0.598–0.791 | <0.001 | |
| 18 h | 0.685 | 0.575–0.796 | 0.001 | |
| 24 h | 0.749 | 0.648–0.850 | <0.001 | |
| RR, breaths/min | Prior to HFNC | 0.460 | 0.367–0.553 | 0.383 |
| 2 h | 0.393 | 0.303–0.482 | 0.017 | |
| 6 h | 0.381 | 0.293–0.470 | 0.010 | |
| 12 h | 0.341 | 0.246–0.436 | 0.002 | |
| 18 h | 0.323 | 0.227–0.419 | 0.001 | |
| 24 h | 0.349 | 0.243–0.456 | 0.007 | |
| PaCO2, mm Hg | Prior to HFNC | 0.462 | 0.358–0.567 | 0.469 |
| 2 h | 0.481 | 0.388–0.575 | 0.682 | |
| 6 h | 0.539 | 0.442–0.637 | 0.420 | |
| 12 h | 0.535 | 0.434–0.635 | 0.499 | |
| 18 h | 0.607 | 0.505–0.710 | 0.047 | |
| 24 h | 0.559 | 0.445–0.673 | 0.295 | |
| Flow, L/min | Prior to HFNC | — | — | — |
| 2 h | 0.543 | 0.456–0.631 | 0.326 | |
| 6 h | 0.516 | 0.425–0.607 | 0.720 | |
| 12 h | 0.569 | 0.474–0.664 | 0.162 | |
| 18 h | 0.568 | 0.466–0.670 | 0.191 | |
| 24 h | 0.534 | 0.424–0.643 | 0.530 | |
| ROX index | Prior to HFNC | 0.659 | 0.566–0.751 | 0.001 |
| 2 h | 0.679 | 0.594–0.763 | <0.001 | |
| 6 h | 0.703 | 0.616–0.790 | <0.001 | |
| 12 h | 0.752 | 0.664–0.840 | <0.001 | |
| 18 h | 0.755 | 0.662–0.847 | <0.001 | |
| 24 h | 0.801 | 0.709–0.893 | <0.001 | |
| SpO2, % | Prior to HFNC | 0.451 | 0.364–0.537 | 0.273 |
| 2 h | 0.582 | 0.496–0.668 | 0.063 | |
| 6 h | 0.596 | 0.503–0.689 | 0.034 | |
| 12 h | 0.693 | 0.608–0.778 | <0.001 | |
| 18 h | 0.675 | 0.575–0.775 | 0.001 | |
| 24 h | 0.625 | 0.517–0.734 | 0.025 | |
| FiO2 | Prior to HFNC | 0.644 | 0.558–0.729 | 0.002 |
| 2 h | 0.629 | 0.544–0.715 | 0.003 | |
| 6 h | 0.669 | 0.579–0.760 | <0.001 | |
| 12 h | 0.672 | 0.574–0.770 | 0.001 | |
| 18 h | 0.676 | 0.565–0.787 | 0.001 | |
| 24 h | 0.747 | 0.645–0.849 | <0.001 | |
| Lactate, mmol/L | Prior to HFNC | — | — | — |
| 2 h | 0.506 | 0.413–0.600 | 0.894 | |
| 6 h | 0.432 | 0.335–0.530 | 0.195 | |
| 12 h | 0.505 | 0.401–0.608 | 0.931 | |
| 18 h | 0.501 | 0.387–0.616 | 0.984 | |
| 24 h | 0.483 | 0.373–0.593 | 0.768 |
The values of sensitivity, specificity, positive and negative predictive value, and the positive and negative likelihood ratio for a ROX index greater than or equal to 4.88 are presented in Table 4. Kaplan-Meier plots showing the probability of MV according to the ROX value at different time points are shown in Figure 1. Patients with ROX index score greater than or equal to 4.88 after 2 hours of HFNC were less likely to need MV. These differences increased throughout the study period. To validate the association between the ROX index during HFNC and the risk of MV, a Cox proportional hazards model was performed. A ROX index greater than or equal to 4.88 was consistently associated with a lower risk of MV, even after adjusting for potential confounding variables (Table 5). Finally, Table E6 shows the cutoff values of the ROX index that have a sensitivity or specificity greater than or equal to 90% and those with the maximum value of positive and negative likelihood ratio for HFNC success.
| Se (%) | Sp (%) | PPV (%) | NPV (%) | LR+ | LR− | ||
|---|---|---|---|---|---|---|---|
| 2 h | Validation | 69.6 | 60.0 | 75.5 | 52.7 | 1.74 | 0.51 |
| Training | 37.1 | 73.8 | 76.6 | 33.7 | 1.41 | 0.85 | |
| 6 h | Validation | 83.8 | 50.0 | 76.6 | 61.2 | 1.68 | 0.32 |
| Training | 50.5 | 60.0 | 76.6 | 31.8 | 1.26 | 0.82 | |
| 12 h | Validation | 86.8 | 52.2 | 81.8 | 61.5 | 1.82 | 0.25 |
| Training | 70.1 | 72.4 | 89.4 | 42.0 | 2.54 | 0.41 | |
| 18 h | Validation | 87.7 | 47.4 | 83.3 | 56.2 | 2.00 | 0.22 |
| Training | 81.0 | 66.7 | 88.9 | 51.6 | 2.43 | 0.28 | |
| 24 h | Validation | 89.1 | 42.9 | 83.1 | 55.6 | 1.56 | 0.25 |
| Training | 80.7 | 72.2 | 93.1 | 44.8 | 2.90 | 0.27 |
Figure 1. Kaplan-Meier plots showing the probability of mechanical ventilation according to the ROX group at (A) 2 hours, (B) 6 hours, and (C) 12 hours after high-flow nasal cannula onset. HFNC = high-flow nasal cannula; MV = mechanical ventilation.
[More] [Minimize]| Hazard Ratio | 95% CI | P Value | |
|---|---|---|---|
| Unadjusted ROX ≥ 4.88 | |||
| At 2 h after HFNC onset | 0.434 | 0.264–0.715 | 0.001 |
| At 6 h after HFNC onset | 0.304 | 0.182–0.509 | <0.001 |
| At 12 h after HFNC onset | 0.291 | 0.161–0.524 | <0.001 |
| Adjusted by immunosuppression | |||
| At 2 h after HFNC onset | 0.455 | 0.274–0.756 | 0.002 |
| At 6 h after HFNC onset | 0.322 | 0.190–0.546 | <0.001 |
| At 12 h after HFNC onset | 0.311 | 0.170–0.569 | <0.001 |
| Adjusted by number of quadrants affected in chest X-ray | |||
| At 2 h after HFNC onset | 0.449 | 0.271–0.744 | 0.002 |
| At 6 h after HFNC onset | 0.308 | 0.184–0.516 | <0.001 |
| At 12 h after HFNC onset | 0.326 | 0.178–0.597 | <0.001 |
| Adjusted by shock at HFNC onset | |||
| At 2 h after HFNC onset | 0.435 | 0.264–0.717 | 0.001 |
| At 6 h after HFNC onset | 0.300 | 0.179–0.501 | <0.001 |
| At 12 h after HFNC onset | 0.303 | 0.168–0.548 | <0.001 |
| Adjusted by SOFA | |||
| At 2 h after HFNC onset | 0.444 | 0.269–0.733 | 0.001 |
| At 6 h after HFNC onset | 0.306 | 0.183–0.512 | <0.001 |
| At 12 h after HFNC onset | 0.296 | 0.164–0.534 | <0.001 |
| Adjusted by APACHE II | |||
| At 2 h after HFNC onset | 0.442 | 0.268–0.729 | 0.001 |
| At 6 h after HFNC onset | 0.310 | 0.184–0.522 | <0.001 |
| At 12 h after HFNC onset | 0.290 | 0.158–0.533 | <0.001 |
| Adjusted by flow rate | |||
| At 2 h after HFNC onset | 0.417 | 0.252–0.690 | 0.001 |
| At 6 h after HFNC onset | 0.282 | 0.169–0.472 | <0.001 |
| At 12 h after HFNC onset | 0.289 | 0.158–0.528 | <0.001 |
| Adjusted by center | |||
| At 2 h after HFNC onset | 0.400 | 0.240–0.668 | <0.001 |
| At 6 h after HFNC onset | 0.283 | 0.169–0.474 | <0.001 |
| At 12 h after HFNC onset | 0.292 | 0.152–0.698 | <0.001 |
First, to give a clearer statement regarding when the ROX index should be calculated to decide on intubation we determined in the whole sample (the training and validation cohort together) the excess of mortality in different time frames taking as reference the first 6 hours of HFNC therapy. Those patients intubated after 12 hours or more of HFNC had an increased risk of hospital death (Figure 2). Second, variables associated with risk of intubation and MV were also analyzed in the whole sample. HFNC success was associated with a higher ROX index, regardless of the time point considered. In contrast, HFNC failure was associated with a greater number of quadrants affected in chest X-ray (3 [2–4] vs. 2 [2–4]; P < 0.001) and a higher prevalence of immunosuppressed patients (39.3% vs. 29.7%; P = 0.049). All variables with P less than 0.1 were included in a multivariate model using Cox proportional hazards modeling (number of quadrants involved in chest X-ray, immunosuppression, viral pneumonia and ROX index). Another model was constructed using ROX index measured at 2, 6, and 12 hours after HFNC onset. ROX index was the unique variable constantly associated with the risk of intubation, regardless of the time-point used (see Table E7).
Figure 2. Relative risk of death according to the time of intubation in patients who failed on high-flow nasal cannula.
[More] [Minimize]Table E8 shows the cutoff value of the ROX index with a higher specificity and the maximum likelihood positive ratio for predicting HFNC failure at 2, 6, and 12 hours. A ROX smaller than 2.85, 3.47, and 3.85 at 2, 6, and 12 hours of HFNC initiation had specificities of 99.2%, 99.2%, and 98.4%, respectively. Kaplan-Meier plot showing the difference in probability of MV between patients with ROX index less than 3.85 and greater than or equal to 4.88 are shown in Figure E2. Patients who failed presented a smaller increase in ROX index values from 2 to 12 and 6 to 12 hours compared with those patients who succeeded (see Table E9). The differences in the ROX index were associated with the risk of HFNC failure after adjusting for the value of the ROX index at the beginning of the analyzed period (see Table E10). Similar and consistent results were observed in the training cohort (see Tables E8–E10).
A second external validation was performed using the FLORALI cohort. No differences were observed in the diagnostic accuracy of the ROX index in the FLORALI cohort compared with both the validation and training cohorts at 2, 6, and 12 hours (see Table E11). The values of sensitivity, specificity, the positive and negative predictive values, and the positive and negative likelihood ratios for a ROX index greater than or equal to 4.88 to predict HFNC success and different cutoff values to predict HFNC failure are presented in Tables E12 and E13. Patients who failed presented a lower increase in the values of the ROX index from 1 to 12 hours (see Tables E14 and E15).
Predicting outcome of noninvasive management of patients with AHRF to avoid delaying a needed intubation is a major and daily challenge for clinicians in the ICU. In the present study, we confirm that a ROX index greater than or equal to 4.88 measured at 2, 6, or 12 hours is a determinant of HFNC success, even after adjusting for potential confounding variables. Similar results are found when applying the ROX index to the FLORALI database. Additionally, we provide specific cutoff points of the ROX index with very high specificity allowing identification of patients who need to be intubated within the first 12 hours of treatment with HFNC. These results have the potential to change and improve practices in the monitoring of patients treated with HFNC.
Consistent data indicate that “late” intubation is associated with worse outcome in patients with acute respiratory failure (20, 21). The same has been found true in patients treated with HFNC (13). However, prediction of HFNC outcome is still challenging. Although many respiratory (oxygenation [5, 22, 23], RR, thoracoabdominal asynchrony [5]) and nonrespiratory (need for vasopressors [22–24], baseline Sequential Organ Failure Assessment score [25, 26], severity of disease [9]) criteria have been found to be associated with HFNC failure, none of them has been tested prospectively to predict HFNC outcome. More and more patients are being treated with HFNC including patients with acute respiratory distress syndrome (9, 10) and a noticeable proportion (30–40%) of them will require subsequent intubation. It is therefore crucial that they may be identified as early as possible by clinicians so as to anticipate intubation. We previously showed that the ROX index measured 12 hours after HFNC initiation was a better predictor of treatment success than SpO2/FiO2 or RR alone (16). Furthermore, patients with a ROX index greater than or equal to 4.88 after 12 hours of HFNC therapy were less likely to be intubated, even after adjusting for potential covariates.
To ensure robustness of the ROX index, three different analyses were performed. First, by examining the AUROC of the different variables, we found that ROX index’s AUROC measured in the validation cohort was comparable with the one reported in the training cohort (16) and also in the FLORALI cohort (3). In addition, ROX index’s AUROC were superior to the ones of other respiratory variables. Second, using a Cox proportional hazards modeling, we found that the ROX index less than 4.88 was independently associated with a higher risk of intubation, even after adjusting for the potential confounders. Because predicting success is not the same as predicting failure, we were able to determine ROX index cutoff points with higher specificity and positive likelihood ratio for predicting HFNC failure. Finally, to take into account the dynamic dimension of decision-making, we analyzed the variations in the ROX index over time and observed a lesser increase (and in some instances a decrease) in the ROX index values between different time-points in those patients who failed compared with those who succeeded with HFNC. Interestingly, similar to what was observed with NIV (27), patients who failed on HFNC and were intubated after more than 12 hours of HFNC presented an increased risk of death.
How can these results be applied by the clinician? Data from studies on HFNC that reported the time of intubation shows that most intubations occur between the 12th and the 24th hour. We therefore suggest monitoring the ROX index over time with a special focus from the 12th hour onward: if the ROX is greater than or equal to 4.88, then the patient has a high chance of success, if it is less than 3.85, then the risk of failure is high, and intubating the patient should be discussed. No predictive index is perfect, and a gray zone obviously exists between 3.85 and 4.88 in which it is difficult to conclude. At 12 hours, 21 patients only (11% [21/191] of the entire population) were in this zone. Among them, seven were ultimately intubated. One could imagine that if a patient is in the gray zone at 12 hours, the ROX could be repeated 1 or 2 hours later: 1) if the score has increased, the patient should be considered with a greater likelihood of success; 2) if it has decreased, then intubation has a greater likelihood to occur; and 3) if the score is unchanged, then reassessment should be performed after 1 or 2 more hours. Such a strategy obviously requires a prospective evaluation.
The limitations of the study listed below deserve consideration. We deliberately included only patients with pneumonia-related AHRF because pneumonia is by far the leading cause of AHRF and the major indication for HFNC (3) (82% of the FLORALI patients had pneumonia). Our results may thus not be generalizable to other less frequent causes of AHRF. In some instances, SpO2/FiO2 was almost as good as ROX. However, adding the RR generally improved the diagnostic accuracy. It is universally accepted as a highly determinant vital sign that is easily measured at the bedside. Patients’ dyspnea and discomfort under HFNC were not assessed, although they might be a potential indicator of HFNC failure. Whether they are superior or not to the ROX index requires further assessment. Because of the design of the study and construction of the models, some analyses were retrospective. Application of the ROX index to the FLORALI cohort yielded consistent results with those obtained with the training and validation cohorts, although in some instances weaker than expected. A potential explanation is that the FLORALI criteria for intubation required a 10-point higher RR than in the present study keeping in mind that RR has a strong impact on the ROX index. Finally, we cannot ascertain that the ROX index was not used to make any decision of intubation. However, even though all variables were collected prospectively, the ROX index calculation was performed during the statistical analysis.
In conclusion, our results indicate that the ROX index helps predict outcome of HFNC therapy of patients with AHRF caused by pneumonia. They also suggest that the dynamic of changes of its value may help discriminate those patients who will succeed with HFNC from those patients who will fail. Among the components of the index, SpO2/FiO2 has a greater weight than RR. The index can be easily and repeatedly measured at the bedside thereby contributing to the day-to-day clinical decision-making process of critically ill patients treated with HFNC. Further studies are needed to determine whether the use of the ROX index can avoid delaying a needed intubation and improve outcomes.
| 1. | Goligher EC, Slutsky AS. Not just oxygen? Mechanisms of benefit from high-flow nasal cannula in hypoxemic respiratory failure. Am J Respir Crit Care Med 2017;195:1128–1131. |
| 2. | Frat JP, Ragot S, Girault C, Perbet S, Prat G, Boulain T, et al.; REVA Network. Effect of non-invasive oxygenation strategies in immunocompromised patients with severe acute respiratory failure: a post-hoc analysis of a randomised trial. Lancet Respir Med 2016;4:646–652. |
| 3. | Frat JP, Thille AW, Mercat A, Girault C, Ragot S, Perbet S, et al.; FLORALI Study Group; REVA Network. High-flow oxygen through nasal cannula in acute hypoxemic respiratory failure. N Engl J Med 2015;372:2185–2196. |
| 4. | Roca O, Riera J, Torres F, Masclans JR. High-flow oxygen therapy in acute respiratory failure. Respir Care 2010;55:408–413. |
| 5. | Sztrymf B, Messika J, Bertrand F, Hurel D, Leon R, Dreyfuss D, et al. Beneficial effects of humidified high flow nasal oxygen in critical care patients: a prospective pilot study. Intensive Care Med 2011;37:1780–1786. |
| 6. | Mauri T, Turrini C, Eronia N, Grasselli G, Volta CA, Bellani G, et al. Physiologic effects of high-flow nasal cannula in acute hypoxemic respiratory failure. Am J Respir Crit Care Med 2017;195:1207–1215. |
| 7. | Bellani G, Laffey JG, Pham T, Fan E, Brochard L, Esteban A, et al.; LUNG SAFE Investigators; ESICM Trials Group. Epidemiology, patterns of care, and mortality for patients with acute respiratory distress syndrome in intensive care units in 50 countries. JAMA 2016;315:788–800. |
| 8. | Frat JP, Ragot S, Thille AW. High-flow nasal cannula oxygen in respiratory failure. N Engl J Med 2015;373:1374–1375. |
| 9. | Messika J, Ben Ahmed K, Gaudry S, Miguel-Montanes R, Rafat C, Sztrymf B, et al. Use of high-flow nasal cannula oxygen therapy in subjects with ARDS: a 1-year observational study. Respir Care 2015;60:162–169. |
| 10. | García-de-Acilu M, Marin-Corral J, Vázquez A, Ruano L, Magret M, Ferrer R, et al. Hypoxemic patients with bilateral infiltrates treated with high-flow nasal cannula present a similar pattern of biomarkers of inflammation and injury to acute respiratory distress syndrome patients. Crit Care Med 2017;45:1845–1853. |
| 11. | Demoule A, Girou E, Richard JC, Taille S, Brochard L. Benefits and risks of success or failure of noninvasive ventilation. Intensive Care Med 2006;32:1756–1765. |
| 12. | Carrillo A, Gonzalez-Diaz G, Ferrer M, Martinez-Quintana ME, Lopez-Martinez A, Llamas N, et al. Non-invasive ventilation in community-acquired pneumonia and severe acute respiratory failure. Intensive Care Med 2012;38:458–466. |
| 13. | Kang BJ, Koh Y, Lim CM, Huh JW, Baek S, Han M, et al. Failure of high-flow nasal cannula therapy may delay intubation and increase mortality. Intensive Care Med 2015;41:623–632. |
| 14. | Rochwerg B, Brochard L, Elliott MW, Hess D, Hill NS, Nava S, et al. Official ERS/ATS clinical practice guidelines: noninvasive ventilation for acute respiratory failure. Eur Respir J 2017;50:1602426. |
| 15. | Delbove A, Darreau C, Hamel JF, Asfar P, Lerolle N. Impact of endotracheal intubation on septic shock outcome: a post hoc analysis of the SEPSISPAM trial. J Crit Care 2015;30:1174–1178. |
| 16. | Roca O, Messika J, Caralt B, García-de-Acilu M, Sztrymf B, Ricard JD, et al. Predicting success of high-flow nasal cannula in pneumonia patients with hypoxemic respiratory failure: the utility of the ROX index. J Crit Care 2016;35:200–205. |
| 17. | Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, et al.; Infectious Diseases Society of America; American Thoracic Society. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis 2007;44:S27–S72. |
| 18. | Brochard L, Mancebo J, Wysocki M, Lofaso F, Conti G, Rauss A, et al. Noninvasive ventilation for acute exacerbations of chronic obstructive pulmonary disease. N Engl J Med 1995;333:817–822. |
| 19. | DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–845. |
| 20. | Kangelaris KN, Ware LB, Wang CY, Janz DR, Zhuo H, Matthay MA, et al. Timing of intubation and clinical outcomes in adults with acute respiratory distress syndrome. Crit Care Med 2016;44:120–129. |
| 21. | Bauer PR, Gajic O, Nanchal R, Kashyap R, Martin-Loeches I, Sakr Y, et al.; ICON Investigators (Supplemental Appendix 1). Association between timing of intubation and outcome in critically ill patients: a secondary analysis of the ICON audit. J Crit Care 2017;42:1–5. |
| 22. | Rello J, Perez M, Roca O, Poulakou G, Souto J, Laborda C, et al. High-flow nasal therapy in adults with severe acute respiratory infection: a cohort study in patients with 2009 influenza A/H1N1v. J Crit Care 2012;27:434–439. |
| 23. | Hyun Cho W, Ju Yeo H, Hoon Yoon S, Lee S, SooJeon D, Seong Kim Y, et al. High-flow nasal cannula therapy for acute hypoxemic respiratory failure in adults: a retrospective analysis. Int Med (Tokyo, Japan) 2015;54:2307–2313. |
| 24. | Roca O, de Acilu MG, Caralt B, Sacanell J, Masclans JR; ICU Collaborators. Humidified high flow nasal cannula supportive therapy improves outcomes in lung transplant recipients readmitted to the intensive care unit because of acute respiratory failure. Transplantation 2015;99:1092–1098. |
| 25. | Koga Y, Kaneda K, Mizuguchi I, Nakahara T, Miyauchi T, Fujita M, et al. Extent of pleural effusion on chest radiograph is associated with failure of high-flow nasal cannula oxygen therapy. J Crit Care 2016;32:165–169. |
| 26. | Kim WY, Sung H, Hong SB, Lim CM, Koh Y, Huh JW. Predictors of high flow nasal cannula failure in immunocompromised patients with acute respiratory failure due to non-HIV pneumocystis pneumonia. J Thorac Dis 2017;9:3013–3022. |
| 27. | Duan J, Han X, Bai L, Zhou L, Huang S. Assessment of heart rate, acidosis, consciousness, oxygenation, and respiratory rate to predict noninvasive ventilation failure in hypoxemic patients. Intensive Care Med 2017;43:192–199. |
Author Contributions: O.R. and J.-D.R. designed the study; contributed to the acquisition, analysis, and interpretation of data; wrote the manuscript; and revised the manuscript. B.C. contributed to the acquisition, analysis, and interpretation of data; wrote the manuscript; and revised the manuscript. J.M., M.S., B.S., G.H., M.G.-d.-A., J.-P.F., and J.R.M. designed the study; contributed to the acquisition, analysis, and interpretation of data; and revised the manuscript.
This article has an online supplement, which is accessible from this issue’s table of contents at www.atsjournals.org.
Originally Published in Press as DOI: 10.1164/rccm.201803-0589OC on December 21, 2018
Author disclosures are available with the text of this article at www.atsjournals.org.
