Understanding Randomized Controlled Trials: A Historical Medical Example

Here's the explanation, using the CAST trial (Cardiac Arrhythmia Suppression Trial, 1989) as the single example. THE SETUP After a heart attack, roughly 10% of patients develop ventricular arrhythmias (abnormal extra heartbeats), and observational data strongly suggested these predicted sudden death. The logic seemed airtight: suppress the arrhythmias and you save lives. Three drugs — encainide, flecainide, moricizine — were known to suppress arrhythmias. CAST randomized ~1,500 post-heart-attack patients with arrhythmias to drug or placebo, double-blind. WHAT RANDOMIZATION DOES It's the only design that removes confounding. By assigning treatment by coin flip, the drug and placebo groups become comparable on everything — age, heart function, diabetes, and crucially the unknown confounders you can't even measure. Any later difference in outcomes can only plausibly be attributed to the treatment, not to "sicker patients happened to get the drug." This is what turns association into causation. CAST's randomization is why its result outranked all the observational data that came before it. WHAT THE CONTROL GROUP DOES The placebo arm provides the counterfactual — what happens without the drug. Without it you'd watch arrhythmias drop and conclude the drugs worked. The control group absorbs the placebo effect, natural recovery, and regression to the mean (patients enrolled when they're at their worst tend to improve regardless). Blinding — neither patient nor doctor knows the arm — stops expectation from contaminating the comparison. WHAT THE TRIAL FOUND After about 10 months, mortality was 7.7% on encainide/flecainide versus 3.0% on placebo — roughly a 2.5x higher death rate. The trial was stopped early and the drugs were pulled from the market. The drugs did exactly what they were designed to do — suppress arrhythmias — and yet killed patients. WHAT AN RCT CAN TELL YOU With randomization, a control group, and blinding, you get a trustworthy causal estimate of a treatment's effect on the outcomes you measured, in the population you enrolled, over the follow-up you ran. CAST definitively answered "does suppressing arrhythmias save lives?" — No, it kills. WHAT AN RCT CANNOT TELL YOU 1. Mechanism. CAST showed the drugs were lethal but didn't explain why (later attributed to proarrhythmia). A trial answers "whether," not "why." 2. Generalizability. It only speaks to the enrolled population — post-MI patients with asymptomatic arrhythmias. Nothing about other patients, doses, or durations. 3. Long-term effects beyond the follow-up window, or rare harms the trial is underpowered to detect. 4. Individual effects. The result is an average; it can't tell you which specific patient benefits or is harmed. 5. Certainty. A single trial can be wrong by chance; results require replication. P-HACKING Randomization shrinks but doesn't eliminate the ability to manufacture significance. The classic moves — fishing through subgroups ("maybe it's only harmful in patients with poor heart function"), switching the primary endpoint after seeing data, stopping early because results look favorable, running 20 analyses and reporting the one with p<0.05 — all inflate false positives. CAST itself illustrates the early-stopping version: a Data Safety Monitoring Board looks at accumulating data repeatedly, and each look is a statistical test. Without pre-specified stopping boundaries (adjusted for repeated looks), the repeated testing alone can produce a spurious "significant" result. The telltale sign of a p-hacked finding is that it fails to replicate. WHY SIGNIFICANCE CAN BE PRACTICALLY MEANINGLESS p<0.05 only means "if the true effect were zero, this result would be unlikely." It says nothing about the size or importance of the effect. A 50,000-patient trial can return p<0.001 for a risk reduction from 20% to

19.9% — real, reproducible, and clinically useless (you'd treat 1,000 patients to help one). CAST is the sharper version: the arrhythmia suppression was statistically significant — the drugs genuinely did what they were designed to do — yet that significance was not just meaningless but actively misleading, because the endpoint everyone cared about (arrhythmia counts) was a surrogate with no bearing on the outcome that mattered (death). Significance on the wrong endpoint, or significance with a trivial effect size, is fully compatible with a drug that is worthless or lethal. BOTTOM LINE An RCT is the gold standard for one narrow question: does this treatment change this outcome in this population? It earns that status only through randomization, a proper control group, and blinding. It cannot hand you mechanism, generalizability, or certainty, and it is only as honest as the people analyzing it — p-hacking and over-interpretation of p-values can corrupt even a well-run trial.