In this  workshop, we will discuss the logic of experimentation, its strengths  and weaknesses compared to other methodologies, and the ways in which  experimentation has been -- and could be -- used to investigate  political, social, and economic phenomena.  Emphasis will be placed on  field experiments, randomized trials conducted in real-world settings.   Examples will be drawn from a broad array of disciplines.

After  describing the attractive statistical properties of experiments, we  consider a variety of potential threats to core assumptions.  In  particular, we consider the complications that arise when (1) treatment  and control conditions different in systematic ways other than the  intended treatment, (2) treatments are not administered according to the  randomly assigned plan, (3) subjects are affected by the treatments  assigned or administered to others, and (4) outcome measures are not  obtained for all subjects.  In each case, we discuss possible  statistical and design solutions.  We conclude by discussing the  practical and ethical issues that arise when conducting experiments in  field settings.

​

​

Audience & Prerequisites

This  workshop serves as a broad introduction to social science experiments.  Participants are expected to come to the session with some experience  designing a research study and with a question or hypothesis that they  wish to investigate using experimental design.

​

​

Readings & Resources

The primary text for the course is

Gerber, Alan S., and Donald P. Green.  2012.  Field Experiments: Design, Analysis, and Interpretation.  New York: W.W. Norton.

This  textbook (FEDAI for short) is too extensive to be covered in two  sessions, but we will make our way through much of the first half, which  covers core topics.

Supplementary  readings are designed to illustrate a wide range of experimental  applications.  They will be provided to participants via a shared  Dropbox folder.  Further readings beyond the dropbox PDFs are available  on request.

Data  analysis examples will be provided in both R and Stata format, but no  special programming expertise is required.  If you have some experience  with statistical software, you can join in when we analyze data  together.

​

​

Schedule

Session 1.

What are  experiments?  Why conduct experimental research?  Experiments and  Models of Potential Outcomes.  Varieties of experimental designs.  What  are covariates and why are they useful?

• FEDAI: Chapters 1 and 2, but skim 3 and 4.

Varieties of survey experiments:

• Hainmueller, Jens, and Daniel J. Hopkins. The hidden American  immigration consensus: A conjoint analysis of attitudes toward  immigrants. American Journal of Political Science 59.3 (2015): 529-548.

• Healy, Andrew, Katrina Kosec, and Cecilia Hyunjung Mo. Economic  development, mobility, and political discontent: An experimental test of  Tocqueville’s thesis in Pakistan. American Political Science Review 111.3 (2017): 605-621.

• Kuklinski, James H., Michael D. Cobb, and Martin Gilens. Racial attitudes and the New South. Journal of Politics 59.2 (1997): 323-349.

​

Varieties of Field experiments:

• Kalla, Joshua L., and David E. Broockman.  2015. Campaign Contributions  Facilitate Access to Congressional Officials: A Randomized Field  Experiment.  American Journal of Political Science 60(3): 545-558.

• Bertrand, Marianne and Sendhil Mullanathan. 2004. Are Emily and Greg  More Employable than Lakisha and Jamil? A Field Experiment on Labor  Market Discrimination. American Economic Review 94(4):  991-1013.

​

Varieties of Lab experiments:

• Mutz, Diana C. and Byron Reeves. 2005. The New Video Malaise: Effects of Televised Incivility on Political Trust. American Political Science Review 99 (February):1-15.

• Casella, Alessandra, and Thomas R. Palfrey.  2021. “Trading votes for votes: A laboratory study.” Games and Economic Behavior 125: 1-26.

​

Session 2.

Experiments with One-sided Noncompliance (Failure-to-Treat):

• FEDAI: Chapter 5.

In  addition, read the following article, which we will use in class to  illustrate the analysis of experiments with one-sided noncompliance:

• Gerber, Alan S., and Donald P. Green. 2000. The Effects of Canvassing,  Direct Mail, and Telephone Contact on Voter Turnout: A Field Experiment.  American Political Science Review 94:653-63.

​

Experiments with Attrition:

• FEDAI: Chapter 7.

​

Implementing an Experiment and Reporting the Results:

• FEDAI: Chapter 13, Appendix A, and Appendix B.

​

Special topics: heterogeneous effects, causal mechanisms, and meta-analysis:

• FEDAI: Chapters 9, 10, and 11.