From the Authors:

We appreciate the letter from Dr. Walkey and Dr. Bor about our article (1). They cleverly suggest ways in which regression discontinuity designs (RDD) may be quite helpful. RDD may be superior to other observational approaches to monitor ongoing implementation of the results of randomized clinical trials (RCTs) that have been analyzed to detect heterogeneity of treatment effect (HTE). This is a potentially useful strategy for organizations willing to commit organizational resources to both a nuanced discussion of RCT results and big data infrastructure to support it. We note, of course, that this can only occur if the original RCT has been reported and analyzed for HTE by baseline risk.

One of the benefits of examination for HTE by baseline risk of death is that it provides data about the net benefits of a treatment. Rather than the RCT providing a single population average estimate, with HTE analyses, we can move toward personalizing estimates of the benefits of a therapy. These data can support the implicit judgments clinicians are constantly called on to make. Net benefit thinking recognizes that some patients may benefit very much from a therapy, in which case most such patients should get it. Some patients are much more likely to be harmed than helped, and those patients should not get that therapy. And some patients are in a middle range of modest net benefit, in which case individual preferences and clinician-patient-family conversation are key.

To implement an RDD monitoring strategy, an organization would need to come to a consensus as to where these thresholds should be set after each RCT. In particular, the organization would need to agree on: where are the everyone/some and some/no-one thresholds for our hospital or clinician group? Such a collective conversation in the face of new evidence might be of great value but also requires willingness to standardize practice regarding that therapy.

We note that Dr. Walkey and Dr. Bor suggest new approaches to refining knowledge about HTE after the primary observations have been made in RCTs. True randomization remains a gold standard, and we do not believe their letter argues that even sophisticated observational approaches such as RDD could replace the primary role of intentional randomization. We are excited that they have suggested another reason to report and analyze RCTs for HTE by baseline risk. Such HTE reporting and analysis must become an accepted standard to provide the foundation for the work they are developing.