Statistical Independence and Compromised Randomization

Chapter 14 — Statistical Independence and Compromised Randomization Randomization is supposed to ensure that treatment assignment is independent of potential outcomes—but what happens when something goes wrong? This chapter addresses two scenarios: when the experimenter controls the assignment mechanism but randomization is compromised, and when the experimenter receives data from a third-party randomization that has gone awry. Solutions include rerandomization, permutation-test inference, and decision-theoretic randomization-test inference. The overarching lesson: careful design prevention is worth far more than any post-hoc regression cure.

1. Failures of statistical independence imply potentially compromised internal validity.
2. There are two major categories of compromised randomization: Case 1, when the experimenter controls the assignment mechanism; Case 2, when the experimenter receives data wherein someone else has done the randomization and something has gone amiss.
3. While a rerandomization procedure is appropriate for Case 1, for Case 2 the tools of observational data science can be applied, but a researcher must leverage knowledge of the initial assignment mechanism and an economic model.
4. Since an ounce of design prevention is worth a ton of regression cure, the experimentalist should pay careful attention to key design elements.