Using Baseline Data to Guide Experimental Design

Abstract: Randomized controlled trials (RCTs) are costly. Designs with optimal treatment assignment probabilities — the Neyman allocation — may provide more powerful inference for a fixed sample size. Work implementing such designs usually relies on pilot data which are rarely available in practice. However, availability of baseline outcome and treatment data is common. I pose the experimental design as a decision problem and show how baseline data may be used to inform it. This yields the minimax and minimax regret optimal assignment probabilities that result in asymptotic variances that are minimax and minimax regret optimal under a large class of assignment mechanisms, including stratified block randomization. I illustrate the utility of the findings using empirically calibrated simulations.