Nancy Cartwright on the Limits of RCTs

Guest Post by Bill Gardner @Bill_Gardner

Many researchers and physicians assert that randomized clinical trials (RCTs) are the “gold standard” for evidence about what works in medicine. But many others have pointed to both strengths and limitations in RCTs (see, for example, Austin Frakt’s comments on Angus Deaton here). Nancy Cartwright is a major philosopher of science. In this Lancet paper she provides insights into why RCTs are so highly valued and also why they are by themselves insufficient to answer the most important questions in medicine.

RCTs have been taken to be a gold standard because they are, according to Cartwright, “self-validating.” What this means is that an RCT can establish a causal connection between a treatment and an outcome more or less by virtue of the design.

all features causally relevant to the outcome other than the treatment (and its downstream effects) are distributed identically between treatment and control groups. If the outcome is more probable in the treatment than the control group,… the only explanation possible is that the treatment caused the outcome in some members of that group.

An RCT done right means that correlation between treatment and outcome does imply causation. In most sciences, you need strong theories to understand what the data mean and to justify your causal interpretation of an experiment’s results (see here). There are no control groups in astrophysics. In an RCT, the evidence for the causal effect of the treatment comes from the experimental design and doesn’t depend on your theory about what makes the treatment work.

Because RCTs stand on their own without a superstructure of theory, Cartwright says that RCTs provide “clinchers” to arguments.

The views, opinions and positions expressed by these authors and blogs are theirs and do not necessarily represent that of the Bioethics Research Library and Kennedy Institute of Ethics or Georgetown University.