IMPACT-L Archives

Moderated conference on impact assessment of agricultural research: May 2014

Impact-L@LISTSERV.FAO.ORG

Options: Use Classic View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Topic: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Fri, 30 May 2014 09:59:03 +0200
text/plain (34 lines)
This is Daniel Suryadarma, replying to Matthieu Stigler (Message 95) in order to provide an example that was requested. Although the example is not in the field of agricultural research, I hope it suffices as it is in the education sector - which I would argue is at least as complex as the agricultural research for development (AR4D) field. 

I was involved in a project that implemented four interventions to strengthen school committees in Indonesia, with the ultimate aim of improving learning outcomes. The project extensively used a theory of change (ToC) developed at the start of the project. I have put the ToC up on my website: http://www.danielsuryadarma.com/pdf/ToC.png. The resulting research paper by Pradhan et al. (2011), in which we estimate the merits of each box in the intermediary outcomes, can be downloaded (also published in Pradhan et al., 2014). We also implemented a qualitative study (Bjork, 2009 - which unfortunately is not publicly available) to illuminate the quantitative findings. I'm not taking any credit (all due to my brilliant colleagues), but this project is an example of how a rigorous quantitative impact evaluation is then enriched by a similarly rigorous qualitative evaluation. 

And I think, this is how proper impact evaluations should be implemented. The issue is not about qualitative vs quantitative methods. The best impact assessment studies that I know of combine many methods, building the body of evidence, explaining the intermediate outcomes, consider external validity, and internalizing (as much as possible) the complexities and contexts - including taking into account other causal factors. I take the point (and the skepticism) that this is not (yet) the norm in the world of impact evaluation, but the world is moving in this direction - and we all need to play a part. 

I have another point, to respond to Matthieu's last paragraph. In my opinion, the goal of impact evaluation is not accountability. It is R&D. Impact evaluations allow us to learn which interventions work the best, have the highest efficiency, etc. This is why Impact evaluations require quantitative estimates. We need to be able to compare the merits of one intervention with other potential interventions with the same goal. The choice is not between one intervention and no intervention, but between one intervention and a plethora of other interventions. 

Dr. Daniel Suryadarma
Senior Scientist - Impact Assessment
Center for International Forestry Research
P.O. Box 0113 BOCBD
Bogor 16000
Indonesia
www.cifor.org
e-mail: d.suryadarma (at) cgiar.org
www.danielsuryadarma.com
twitter.com/dsuryadarma

References:

- Bjork, C. 2009, Improving educational quality through community participation - qualitative study. Report for World Bank

- Pradhan, M., D. Suryadarma, A. Beatty, M. Wong, A. Alisjahbana, A. Gaduh, and R.P. Artha. 2011. Improving educational quality through enhancing community participation: Results from a randomized field experiment in Indonesia. World bank Policy Research Working Paper 5795 http://elibrary.worldbank.org/doi/book/10.1596/1813-9450-5795 

- Pradhan, M., D. Suryadarma, A. Beatty, M. Wong, A. Gaduh, A. Alisjahbana and R.P. Artha. 2014. Improving educational quality through enhancing community participation: Results from a randomized field experiment in Indonesia. American Economic Journal: Applied Economics, 6(2): 105-26. April 2014 issue. http://www.aeaweb.org/articles.php?doi=10.1257/app.6.2.105 

[To contribute to this conference, send your message to [log in to unmask] The last day for sending messages to the conference is 1 June. The searchable message archive is at https://listserv.fao.org/cgi-bin/wa?A0=Impact-L ].

########################################################################

To unsubscribe from the Impact-L list, click the following link:
https://listserv.fao.org/cgi-bin/wa?SUBED1=Impact-L&A=1

ATOM RSS1 RSS2