Generalizability and Scaling

Chapter 16 — Generalizability and Scaling Internal validity is a prerequisite, not a destination. This chapter tackles Experimental Problem 2: whether causal effects found in one environment transfer to others. It develops the assumptions required for external validity—external unconfoundedness, overlap, parallelism, and investigator neutrality—and distinguishes local, global, and zero generalizability. It then turns to scaling, showing that portability of benefit-cost profiles requires a behavioral model of the data-generating process. The chapter introduces “Option C thinking”—backward-inducting from the target environment at the design stage to ensure EP2 is achievable.

1. Internal validity is a prerequisite for external validity (generalizability) because results that deviate from the true effect because of systematic error lack the basis for generalizability.
2. In the social sciences, all experimental results generalize to some setting, and no experimental result generalizes to all settings.
3. Whereas external validity explores whether experimental results are portable, scaling pertains to the portability of benefit-cost profiles.
4. A behavioral model is necessary to describe the data-generating process and relate it to other settings: researchers should backward induct by using Option C thinking in the design stage to achieve EP2.