IMPACT-L Archives

Moderated conference on impact assessment of agricultural research: May 2014

Impact-L@LISTSERV.FAO.ORG

Options: Use Classic View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Topic: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Mon, 26 May 2014 13:21:10 +0200
text/plain (29 lines)
This is Daniel Suryadarma again. In my previous message (nr. 64), I discussed the main drawback of propensity score matching (PSM). Earlier today, I started out writing about a number of evaluation methods that should be considered ahead of PSM. Halfway through, however, I began to think that it is much more efficient to just recommend a book that I had used when I was teaching quantitative impact evaluation in Australia - Gertler et al on Impact Evaluation in Practice, which can be freely downloaded from the World Bank website (reference below). This book has not been mentioned before in this conference, and I think it goes much deeper into each method than Khandker et al's book. [Khandker et al (2010) ' Handbook on impact evaluation: Quantitative methods and practices', also from the World Bank, was recommended in Messages 14 and 15...Moderator].

Having taken the methods discussion out of the way, I think it may be useful to reiterate the purpose and, from there, the most basic foundation of impact evaluation. From my point of view, the purpose of any impact evaluation is to do two things: measure effect size - that is: how big the impact is; and determine causality. This also applies to ex post impact assessment of agricultural research.

Based on the purpose, an impact evaluation rises and falls on one thing: the counterfactual. That is: what would the outcome have been, had the program/research not been implemented? The central issue of impact evaluation is the validity of the counterfactual. Without an explicit construction of a counterfactual, there is no impact evaluation.

Finally, I am going to make a statement that may be controversial. Since the main purpose of an impact evaluation is to measure effect size, quantitative methods must be used. Without quantitative estimates, there is no impact evaluation.

Dr. Daniel Suryadarma
Senior Scientist - Impact Assessment
Center for International Forestry Research
P.O. Box 0113 BOCBD
Bogor 16000
Indonesia
www.cifor.org
e-mail: d.suryadarma (at) cgiar.org
www.danielsuryadarma.com
twitter.com/dsuryadarma

Reference:
- Gertler P.J., Martinez S., Premand P., Rawlings, L.B and C.M.J.  Vermeersch. 2011. Impact Evaluation in Practice. World Bank, Washington. http://siteresources.worldbank.org/EXTHDOFFICE/Resources/5485726-1295455628620/Impact_Evaluation_in_Practice.pdf  (3 MB)

[To contribute to this conference, send your message to [log in to unmask] For further information, see http://www.fao.org/nr/research-extension-systems/res-home/news/detail/en/c/217706/ ].

########################################################################

To unsubscribe from the Impact-L list, click the following link:
https://listserv.fao.org/cgi-bin/wa?SUBED1=Impact-L&A=1

ATOM RSS1 RSS2