没有合适的资源?快使用搜索试试~ 我知道了~
资源详情
资源评论
资源推荐
NBER TECHNICAL WORKING PAPER SERIES
USING RANDOMIZATION IN DEVELOPMENT ECONOMICS RESEARCH:
A TOOLKIT
Esther Duflo
Rachel Glennerster
Michael Kremer
Technical Working Paper 333
http://www.nber.org/papers/t0333
NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
December 2006
We thank the editor T.Paul Schultz, as well Abhijit Banerjee, Guido Imbens and Jeffrey Kling for
extensive discussions, David Clingingsmith, Greg Fischer, Trang Nguyen and Heidi Williams for outstanding
research assistance, and Paul Glewwe and Emmanuel Saez, whose previous collaboration with us inspired
parts of this chapter. The views expressed herein are those of the author(s) and do not necessarily
reflect the views of the National Bureau of Economic Research.
© 2006 by Esther Duflo, Rachel Glennerster, and Michael Kremer. All rights reserved. Short sections
of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full
credit, including © notice, is given to the source.
Using Randomization in Development Economics Research: A Toolkit
Esther Duflo, Rachel Glennerster, and Michael Kremer
NBER Technical Working Paper No. 333
December 2006
JEL No. C93,I0,J0,O0
ABSTRACT
This paper is a practical guide (a toolkit) for researchers, students and practitioners wishing to introduce
randomization as part of a research design in the field. It first covers the rationale for the use of randomization,
as a solution to selection bias and a partial solution to publication biases. Second, it discusses various
ways in which randomization can be practically introduced in a field settings. Third, it discusses designs
issues such as sample size requirements, stratification, level of randomization and data collection methods.
Fourth, it discusses how to analyze data from randomized evaluations when there are departures from
the basic framework. It reviews in particular how to handle imperfect compliance and externalities.
Finally, it discusses some of the issues involved in drawing general conclusions from randomized
evaluations, including the necessary use of theory as a guide when designing evaluations and interpreting
results.
Esther Duflo
Department of Economics
MIT, E52-252G
50 Memorial Drive
Cambridge, MA 02142
and NBER
eduflo@mit.edu
Rachel Glennerster
Poverty Action Lab
MIT Department of Economics
E60-246
Cambridge MA 02139
rglenner@mit.edu
Michael Kremer
Harvard University
Department of Economics
Littauer Center 207
Cambridge, MA 02138
and NBER
mkremer@fas.harvard.edu
Con
tents
1 Introduction 3
2 Why Randomize? 4
2.1 The Problem of Causal Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Randomization Solves the Selection Bias . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Other Methods to Control for Selection Bias . . . . . . . . . . . . . . . . . . . . . 10
2.3.1 Controlling for Selection Bias by Controlling for Observables . . . . . . . 10
2.3.2 Regression Discontinuity Design Estimates . . . . . . . . . . . . . . . . . 11
2.3.3 Difference-in-Differences and Fixed Effects . . . . . . . . . . . . . . . . . . 12
2.4 Comparing Experimental and Non-Experimental Estimates . . . . . . . . . . . . 13
2.5 Publication Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.5.1 Publication bias in non-experimental studies . . . . . . . . . . . . . . . . 15
2.5.2 Randomization and publication bias . . . . . . . . . . . . . . . . . . . . . 17
3 Incorporating Randomized Evaluation in a Research Design 19
3.1 Partners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 Pilot projects: From program evaluations to field experiments . . . . . . . . . . . 22
3.3 Alternative Methods of Randomization . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3.1 Oversubscription Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3.2 Randomized Order of Phase-In . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3.3 Within-Group Randomization . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.3.4 Encouragement Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4 Sample size, design, and the power of experiments 28
4.1 Basic Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.2 Grouped Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.3 Imperfect Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.4 Control Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.5 Stratification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.6 Power calculations in practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1
5
Practical Design and Implementation Issues 40
5.1 Level of Randomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.2 Cross-Cutting Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.3 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.3.1 Conducting Baseline Surveys . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.3.2 Using Administrative Data . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6 Analysis with Departures from Perfect Randomization 47
6.1 The Probability of Selection Depends on the Strata . . . . . . . . . . . . . . . . . 47
6.2 Partial Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.2.1 From Intention To Treat to Average Treatment Effects . . . . . . 51
6.2.2 When is IV Not Appropriate . . . . . . . . . . . . . . . . . . . . . . . 55
6.3 Externalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.4 Attrition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
7 Inference Issues 61
7.1 Grouped Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
7.2 Multiple Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
7.3 Subgroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7.4 Covariates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
8 External Validity and Generalizing Randomized Evaluations 66
8.1 Partial and General Equilibrium Effects . . . . . . . . . . . . . . . . . . . . . . . 67
8.2 Hawthorne and John Henry Effects . . . . . . . . . . . . . . . . . . . . . . . . . . 68
8.3 Generalizing Beyond Specific Programs and Samples . . . . . . . . . . . . . . . . 70
8.4 Evidence on the Generalizability of Randomized Evaluation Results . . . . . . . 71
8.5 Field Experiments and Theoretical Models . . . . . . . . . . . . . . . . . . . . . . 73
2
1
Introduction
Randomization is now an integral part of a development economist’s toolbox. Over the last ten
years, a growing number of randomized evaluations have been conducted by economists or with
their input. These evaluations, on topics as diverse as the effect of school inputs on learning
(Glewwe and Kremer 2005), the adoption of new technologies in agriculture (Duflo, Kremer, and
Robinson 2006), corruption in driving licenses administration (Bertrand, Djankov, Hanna, and
Mullainathan 2006), or moral hazard and adverse selection in consumer credit markets (Karlan
and Zinman 2005b), have attempted to answer important policy questions and have also been
used by economists as a testing ground for their theories.
Unlike the early “social experiments” conducted in the United States—with their large bud-
gets, large teams, and complex implementations—many of the randomized evaluations that have
been conducted in recent years in developing countries have had fairly small budgets, making
them affordable for development economists. Working with local partners on a smaller scale has
also given more flexibility to researchers, who can often influence program design. As a result,
randomized evaluation has become a powerful research tool.
While research involving randomization still represents a small proportion of work in de-
velopment economics, there is now a considerable body of theoretical knowledge and practical
experience on how to run these projects. In this chapter, we attempt to draw together in one
place the main lessons of this experience and provide a reference for researchers planning to con-
duct such projects. The chapter thus provides practical guidance on how to conduct, analyze,
and interpret randomized evaluations in developing countries and on how to use such evaluations
to answer questions about economic behavior.
This chapter is not a review of research using randomization in development economics.
1
Nor is its main purpose to justify the use of randomization as a complement or substitute to
other research methods, although we touch upon these issues along the way.
2
Rather, it is
a practical guide, a “toolkit,” which we hope will be useful to those interested in including
1
Kremer
(2003) and Glewwe and Kremer (2005) provide a review of randomized evaluations in education;
Banerjee and Duflo (2005) review the results from randomized evaluations on ways to improve teacher’s and
nurse’s attendance in developing countries; Duflo (2006) reviews the lessons on incentives, social learning, and
hyperb olic discounting.
2
We have provided such arguments elsewhere, see Duflo (2004) and Duflo and Kremer (2005).
3
剩余88页未读,继续阅读
caesor9806
- 粉丝: 0
- 资源: 1
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论0