IMPACT-L Archives

Moderated conference on impact assessment of agricultural research: May 2014

Impact-L@LISTSERV.FAO.ORG

Options: Use Classic View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Topic: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Tue, 13 May 2014 11:32:41 +0200
text/plain (37 lines)
I am Matthieu Stigler, I have been working on quantitative impact evaluation in various places, including FAO, and am currently working on qualitative methods for the case-studies component of IMPRESA (http://www.impresa-project.eu/).

In response to Amadou (Message 13):

Matching methods have become quite popular recently, so you should find plenty of tutorials and introductory papers, that will give you a good overview of the method. Find below a suggested introductory reading list, that should make you feel comfortable to start using the methods.

However, as this e-conference features many people with a great background in qualitative methods, I guess you might benefit particularly in discussing how qualitative methods can enrich your initial quantitative design. As an example, qualitative methods with semi-structured interviews might be particularly suited to investigate how the self-selection process comes into place. This might give you a better overall understanding for the narrative of your study, help you in the quantitative step to select variables for matching, or have an idea of possible effect heterogeneity. By 'effect heterogeneity', the idea is that the effect is not the same for everyone: people might be affected differently by the programme, some even negatively. See for an example, the Integrated Nutrition Project in India reported by Leeuw and Vaessen (2009, http://siteresources.worldbank.org/EXTOED/Resources/nonie_guidance.pdf), where it is mentioned (p. 70) that girls benefited more than boys (although this reversed later on). If one thinks that there is self-selection, it could be the case that girls self-selected more into the programme (as they knew they would benefit more). Knowing of the latter (qualitative analysis of self-selection) would be very informative for the former (quantitative analysis of an heterogeneous impact).

But I have to admit that I have no reference to recommend on this: given my limited knowledge, I haven't seen (m)any papers in qualitative evaluation methods addressing self-selection (which comes at odds with quantitative practice where self-selection is the main and biggest worry of every quantitative analysis), but hopefully more knowledgeable qualitative practitioners will have better suggestions on this!

Suggested reading on matching could come in that order:

- Khandker, S.R, G.B. Koolwal and H.A. Samad. (2010). Handbook on impact evaluation: Quantitative methods and practices.  World Bank. https://openknowledge.worldbank.org/bitstream/handle/10986/2693/520990PUB0EPI1101Official0Use0Only1.pdf (3 MB).

- Stuart, E.A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science 25(1): 1-21. http://biostat.jhsph.edu/~estuart/Stuart10.StatSci.pdf  

-Caliendo, M. and S. Kopeinig (2008) Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys,
Vol. 22, No. 1, pp. 31-72. http://ideas.repec.org/a/bla/jecsur/v22y2008i1p31-72.html  

Regarding software, there is an excellent free package for the open source software R, called MatchIt: 
http://cran.r-project.org/web/packages/MatchIt/index.html  . It is well documented (in a published review) so that you will quickly be able to run your own analysis: http://www.jstatsoft.org/v42/i08 

Matthieu Stigler 
Institut de recherche de lšagriculture biologique (FiBL) 
Ackerstrasse 113, Case postale 219
5070 Frick, 
Switzerland
www.fibl.org 
e-mail: matthieu.stigler (at) gmail.com 

[To contribute to this conference, send your message to [log in to unmask] For further information, see http://www.fao.org/nr/research-extension-systems/res-home/news/detail/en/c/217706/ ].

########################################################################

To unsubscribe from the Impact-L list, click the following link:
https://listserv.fao.org/cgi-bin/wa?SUBED1=Impact-L&A=1

ATOM RSS1 RSS2