IMPACT-L Archives

Moderated conference on impact assessment of agricultural research: May 2014

Impact-L@LISTSERV.FAO.ORG

FAO is one of nine partners participating in a project on the Impact of Research on EU Agriculture (IMPRESA), funded by the European Union Seventh Framework Programme (FP7). In the recently-launched project, FAO is leading work package 1 (on Concept Development and Learning), which has as one of its main aims the establishment of a common framework to update concepts and methodologies for impact assessment of agricultural research. As part of the work package's activities, FAO is hosting in May 2014 this e-mail conference whose provisional title is "Ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives".
Options: Use Classic View
Hide Table of Contents

Tailoring rural advisory services to family farms: An FAO e-mail conference, 1-18 December 2014
Summary document: FAO e-conference on ex post impact assessment of agricultural research
All messages in a single PDF on web // Stay informed about IMPRESA
End of FAO e-conference on ex post impact assessment
109: Ex-post impact assessment and smallholder resource-poor farmers
108: Operational considerations - refrained
107: Questions about the participations
106: Different points from the e-conference
105: Re: A holistic framework for assessing impact of AR4D projects
104: Re: Can social network analysis be helpful to impact evaluation?
103: EpIA methods for forestry
102: The benefits of impact assessment - Burundi
101: Re: Attribution vs contribution in impact evaluation
100: Re: Attribution vs contribution in impact evaluation
Last days of the FAO e-mail conference on impact assessment
99: Institutional impact analysis
98: Re: Measuring the impacts of 2012 flooding on livelihoods
97: Cirad's work on IE // A few comments about the e-conf discussions
96: Re: Can social network analysis be helpful to impact evaluation?
95: Re: Attribution vs contribution in impact evaluation
94: More incentives for research: EpIA results and Research Information Systems
93: Importance of capacity for conducting impact assessment
92: Re: Attribution vs contribution in impact evaluation
91: Measuring the impacts of 2012 flooding on livelihoods
90: Re: Can social network analysis be helpful to impact evaluation?
89: Re: Quantitative data on social and environmental dimensions
88: Re: Can social network analysis be helpful to impact evaluation?
87: Re: Difficulties in assessing and evaluating agricultural research
86: Re: Attribution vs contribution in impact evaluation
85: Evaluating the social impacts of INIA´s technologies
84: Attribution vs contribution in impact evaluation
83: Quantitative data on social and environmental dimensions
82: Re: Macro-level epIA - Four basic issues
81: Quantitative methods - Instrumental variable methods
80: Participation of actors in EpIA
79: Re: Project design and impact evaluation
78: Re: Can social network analysis be helpful to impact evaluation?
77: Re: Can social network analysis be helpful to impact evaluation?
76: Re: Alternatives to propensity score matching // Purpose of impact evaluation
75: Re: Difficulties in assessing and evaluating agricultural research
74: Re: Seeing impacts in a broader and longer term context
73: Re: Issues in assessment of non-economic impact
72: Can social network analysis be helpful to impact evaluation?
71: Difficulties in assessing and evaluating agricultural research
70: Alternatives to propensity score matching // Purpose of impact evaluation
Moderator's message: Coming into the final week of this FAO e-mail conference
67: Evaluating impacts of capacity development in research
68: Evaluation of INIA, Uruguay
66: Re: Issues in assessment of non-economic impact
65: Issues in assessment of non-economic impact
64: Re: Ex-post impact evaluation of new farming techniques in Senegal - propensity score matching method?
63: Issues in agricultural research impact evaluation
62: Outcome mapping
61: Re: Participatory procedures
60: Re: Two caveats to the definition of impact
59: Communicating the epIA findings
58: Two caveats to the definition of impact
57: Re: Project design and impact evaluation
56: Institutional innovation in Latin America and the Caribbean: Evaluation of FONTAGRO
55: Re: Some lessons learned from assessment of economic impacts of agricultural research technologies
Moderator's message: Just passed the half-way stage in this FAO e-mail conference
54: Communicating epIA findings to relevant stakeholders
53: Participatory monitoring and evaluation in the Philippines
52: Re: Case studies and selection bias
51: Some lessons learned from assessment of economic impacts of agricultural research technologies
50: Re: Partial innovation adoption; project impact evaluation; and network analysis
49: Partial innovation adoption; project impact evaluation; and network analysis
48: Main lessons learned from evaluation of INIA, Uruguay
47: Re: Towards effective communication strategies for ex post impact assessment
46: Re: Case studies and selection bias // CGIAR SPIA resources
45: Re: Project design and impact evaluation
44: Re: Case studies and selection bias
43: Project design and impact evaluation
42: Seed and planting material- Impact assessment
41: EpIA when there is partial adaption of innovations
40: Different epIA methods and a control group
39: Contribution from Egypt
38: Technology adoption and impact evaluation
37: Re: Participatory procedures
36: Re: Participatory procedures
35: Partial acceptance of research results // infrastructure
34: Participatory procedures
33: Re: Towards effective communication strategies for ex post impact assessment
32: Impact assessment and small scale farmers
31: Macro-level epIA - Four basic issues
30: Re: Towards effective communication strategies for ex post impact assessment
29: Re: Case studies and selection bias
28: Re: A holistic framework for assessing impact of AR4D projects
27: Transferability, cost effectiveness and communication of research information
26: Responding to issues raised in Section 4 of the Background Document
25: Communication and research impact
24: Re: Case studies and selection bias
23: Re: Ex-post impact evaluation of new farming techniques in Senegal - propensity score matching method?
22: Re: Message from Kolkata
21: Re: A holistic framework for assessing impact of AR4D projects
20: Case studies and selection bias
19: Macro-level assessments of agricultural research - TFP vs. partial productivity measures
18: Towards effective communication strategies for ex post impact assessment
17: Message from Kolkata
16: The evaluation of each investigation for acceptance by farmers and political agents
15: Re: Ex-post impact evaluation of new farming techniques in Senegal - propensity score matching method?
14: Re: Ex-post impact evaluation of new farming techniques in Senegal - propensity score matching method?
13: Ex-post impact evaluation of new farming techniques in Senegal - propensity score matching method?
12: Assessing the impact of research: long- vs. short-term and quantitative vs. qualitative measures
Update on the FAO e-conference on approaches and methodologies in epIA of agricultural research
11: Re: Integrating value of new knowledge in epIA
10: Operational limitations of smallholder producers
9: Diverse issues related to epIA of agricultural research in the EU
8: Impact evaluation for AR4D: Causality in complex settings
7: Impact assessment: Looking at mobiles hubs of networks and their arrangements over time...
6: Two issues influencing agricultural research and objectivity of epIA
5: A holistic framework for assessing impact of AR4D projects
4: Integrating value of new knowledge in epIA
3: Re: How to consider the aspects of systemic change in impact assessment strategies?
2: How to consider the aspects of systemic change in impact assessment strategies?
1: Five selected issues fundamental to epIA of agricultural research
Welcome to the FAO e-mail conference on Ex post impact assessment of agricultural research
Background Document to the FAO e-mail conference
Message from moderator of FAO e-mail conference on ex post impact assessment

AIS
Fri, 7 Nov 2014 10:37:41 +0100
Reply
Dear Colleagues,

In case any of you might be interested in joining, I thought I would let you know that I will soon be moderating an FAO e-mail conference entitled "Tailoring rural advisory services to family farms". So, please feel free to subscribe and I also ask you to spread the word about the conference through your networks, social media and to any of your colleagues who might be interested in joining.

[More ...]
Reply
No Replies
AIS
Mon, 15 Sep 2014 17:30:51 +0200
Reply
Dear Colleagues,

Here below is the Summary Document for the FAO e-mail conference on "Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives" that took place from 5 May to 1 June 2014.

The conference was organized by FAO as part of its contributions to an EU-FP7 project called IMPRESA that is looking at the impacts of agricultural research in Europe (http://www.impresa-project.eu/).

[More ...]
Reply
No Replies
AIS
Tue, 3 Jun 2014 17:31:11 +0200
Reply
Dear Participants,

We have now put all of the messages from this FAO e-mail conference on "Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives" into a single PDF and made it available on the web.

The file contains in chronological order:
- the Welcome message I sent before the conference began
- the 109 messages from participants posted during the 4-week conference
- the final message I posted on 2 June closing the conference, providing also a brief summary about participation.

[More ...]
Reply
No Replies
AIS
Mon, 2 Jun 2014 15:38:57 +0200
Reply
Dear Colleagues,

The last messages have just been posted (numbers 106 to 109), so this FAO conference on "Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives" is now officially closed.

I would like to sincerely thank all of you who participated actively in this conference. My impression is that there is a very large demand for clear and easily-accessible knowledge about how to best assess the impacts that agricultural research is having on people's lives, their communities and their countries, and that a large number of people joined this conference in order

[More ...]
Reply
No Replies
AIS
Mon, 2 Jun 2014 15:38:07 +0200
Reply
I am Datta Rangnekar from Ahmedabad, India, involved since long in agriculture (crop and livestock) based rural livelihood development work in western and central India.

Firstly, let me thank the FAO for organizing these e-conferences as these provide good opportunities for learning and exchange of views and experiences. I was hesitant to convey my viewpoints on post-facto evaluation of agricultural research since I am more of a development worker than a specialist.

[More ...]
Reply
No Replies
AIS
Mon, 2 Jun 2014 15:37:15 +0200
Reply
This is Dick Tinsley, Colorado State University, again.

While I have refrained from submitting comments for the last couple of weeks, I have read all 105 postings, much of it dealing with social issues and economics, which I am not all that well versed in. However, agriculture research is still a biological activity and needs to be substantially evaluated in biological terms. Thus, I would like to go back to my original posting (nr. 10) concerning the operational limits farmers have in accepting research results, particularly smallholders in developing countries with limited labor, even more limited energy to fuel that

[More ...]
Reply
No Replies
AIS
Mon, 2 Jun 2014 15:36:34 +0200
Reply
My name is Marisa Gonnella. I'm a professor of Rural Sociology and Rural Extension of the Universidad Nacional de Rosario, Argentina.

I send questions by participants. Questions:

1- For the participants, is it more important to analyze the impact or the process of intervention through which it is expected to have an impact? For example, for the Conference participants, is it more important to meet the goals of the Conference or to analyse the impact the conference may have on them from the interaction gained from the knowledge of different realities. The answer to this question would be that when

[More ...]
Reply
No Replies
AIS
Mon, 2 Jun 2014 15:35:56 +0200
Reply
I am Shams Fawki again (my previous message was nr. 39).

At the end of this great email conference I would like to thank John Ruane and FAO for the coordination. I would like also to thank all the professors, consultants and participants in these very interesting discussions and knowledge exchange.

[More ...]
Reply
No Replies
AIS
Sat, 31 May 2014 17:50:04 +0200
Reply
My name is Uche N. Chikezie from the Federal University of Technology (FUTO), Nigeria.

I agree with the views of many participants in this FAO email conference.

Huu-Nhuan Nguyen in message 5 said that impact assessment of agricultural research projects in in the Northwest Highlands of Vietnam have been conducted in the national or majority languages rather than the minority ethnic communities. This is similar to the impact assessment situation in Nigeria.

[More ...]
Reply
No Replies
AIS
Sat, 31 May 2014 17:30:00 +0200
Reply
This is Petya Slavova, a sociologist from the Department of Sociology, Sofia University, Bulgaria and a member of the IMPRESA project team.

I would like to comment on Message 72 by Matthieu Stigler,Dominique Barjolle and Sylvain Quiédeville.

In my view the usefulness of social network analysis (SNA) in ex post impact assessment (epIA) of case studies is problematic for several reasons:

[More ...]
Reply
No Replies
AIS
Sat, 31 May 2014 17:16:14 +0200
Reply
This is E.M. Muralidharan from India. I work with the Kerala Forest Research Institute, Peechi and am involved in research in  forest biotechnology.

Although a bit late in the conference, I thought it would be pertinent to ask a question here in the hope that someone could provide  information.

Not surprisingly, I found little reference to ex post impact assessment (epIA) in forestry in the messages so far. I would have expected that the methods would be different and more challenging when compared to the other sectors, given the  longer time frame involved in a typical forestry intervention and several non-tangible and

[More ...]
Reply
No Replies
AIS
Fri, 30 May 2014 12:36:59 +0200
Reply
I am Leonard Ntakirutimana, Msc, Territory Intelligence specialist, Burundi. I work in the unit of ''Institutional Audit and monitoring and evaluation of institutions performances" (ASEP), Bureau d'Etudes Stratégiques et de Développement (BESD). We do evaluation of Annual Action Plans (AAP) of the Government of Burundi each semestrial level.

The Impact of impact assessment is very interesting. Assessments are made in different localities  (territory, community). Each community has its own behaviour, history, culture, and so on. The result of assessment are different because of the characteristics of the community. In Burundi, the impact of impact assessment is transparency, good governance, comprehension

[More ...]
Reply
No Replies
AIS
Fri, 30 May 2014 11:19:52 +0200
Reply
This is Ekanath Khatiwada again (previous messages 2 and 60).

I liked the discussion about the attribution vs contribution.

It is evident that most of the projects/programmes prefer to generate information about what is changing during the life of the particular project/programme. Some time is not reflected much about the extent to which those changes were caused by the programmes or the projects. In some cases they would have happened anyway without the project/programme interventions? This is the question of attribution; to what extent were observed improvements actually caused by the particular project?

[More ...]
Reply
No Replies
AIS
Fri, 30 May 2014 09:59:03 +0200
Reply
This is Daniel Suryadarma, replying to Matthieu Stigler (Message 95) in order to provide an example that was requested. Although the example is not in the field of agricultural research, I hope it suffices as it is in the education sector - which I would argue is at least as complex as the agricultural research for development (AR4D) field.  

[More ...]
Reply
No Replies
AIS
Fri, 30 May 2014 09:22:21 +0200
Reply
Dear Colleagues,

There are just 3 days left, so I especially encourage those who haven't yet done so to share your views, experiences and knowledge regarding the approaches and methodologies that can be used for ex post impact assessment (epIA) of agricultural research, based on the specific questions in Section 4 of the Background Document or responding to issues raised in the very many high-quality and stimulating messages that have been posted so far.

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 18:59:04 +0200
Reply
This is Mario Pareja, again. In this message I would like to describe the approach we followed, and some of the methods used, to evaluate something that has not been referred to, in this excellent exchange organized by FAO, nor clearly fully defined: the institutional impacts.
 
The reasons we undertook this perspective for impact analysis were basically two:
(1) the assumption, later proved correct, that the potential economic, social and environmental impacts of 20 years of investment in agricultural research & development & innovation (R&D&I) could be explained by the type and level of development (legal and juridical framework,

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 18:45:02 +0200
Reply
This is Mario Pareja, again, replying to two messages, one from A. Bonou (No. 81) and the following one from Dr. Yapi (No. 91).

I am in full agreement with Dr. Yapi in all the points that he makes regarding the "results chain" and time lags. Moreover, to evaluate the impact of a flooding falls outside the "results chain of impacts from ag-R&D&I". Since I have quite a bit of experience in disaster management (I worked for CARE for several years), mainly on the issues of environment in disasters, I recommend that you, A. Bonou, visit the group M&E for

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 18:41:03 +0200
Reply
From Danielle Barret, Senior advisor to DG Research & Strategy of the “French Agricultural Research Centre for International Development” (Cirad) in charge of “Innovation-Impact Task-force” co- coordination.

I have been highly interested in the discussions launched by John Ruane’s synthetic document on impact evaluation of agricultural research. Some of us thought that the IMPRESA project was addressing the European zone. But surprisingly, most of the participants talked about impact evaluation in Southern contexts. A subject of high interest for CIRAD. [The macro-level and micro-level impact assessments that will be carried out through the Impact of Research on EU Agriculture (IMPRESA,

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 14:06:19 +0200
Reply
This is Silvia Andrea Perez again, responding to a request from a conference subscriber for a brief easily understandable description of social network analysis (SNA). I try and do this here.

When you want to do SNA for ex post impact assessment (epIA), let's say of farmers who are adopting the positive selection technology on potatoes, what I did was to go and interview the group of farmers. I used a semi-structured questionnaire to ask farmers various things:

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 13:57:44 +0200
Reply
This is Matthieu Stigler again (Message 15).

I would like to react to two issues: attribution vs. contribution (Message 84, by Hailemichael Taye), and the claim that impact evaluation requires to quantify an impact, and hence can only be done through quantitative methods (Message 70, by Daniel Suryadarma).

I am a little bit skeptical about Daniel Suryadarma's point (Message 86) about the fact that quantitative impact evaluation (IE) methods do investigate intermediary steps in a pathway, and I would welcome references to studies that do this. [Daniel wrote "attribution analyses can, and have been used to, provide an in-depth understanding

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 11:45:51 +0200
Reply
I am Birge Wolf, from the Faculty of Organic Agricultural Science of the University of Kassel, situated in Witzenhausen, Germany.

Thanks a lot for organizing this very interesting e-mail conference. I am very delighted about all these competent and reflective contributions – I will need some more time to read them all.

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 10:50:22 +0200
Reply
I am Suresh Babu, currently a Senior Research Fellow at the international Food Policy Research Institute (IFPRI) in Washington. I am conducting research on institutional development including national agricultural research systems (NARS) and their management. Previously I was a research economist at Cornell University's Food And Nutrition Policy Program.

This has been a rich exchange of ideas and views on this forum. At IFPRI we have been looking at these issues for the past 30 years. Much of what was done as part of this effort is available on www.IFPRI.org. I am appreciative of the issues and challenges raised in

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 09:57:18 +0200
Reply
I am Hailemichael Taye again. This is a response to the comment given by Daniel Suryadarma (Message 86) after my email on attribution vs contribution in impact evaluation (Message 84).

Let me clarify some issues first. I have never attempted to compare the two approaches in general terms and concluded one is always best over the other. What I was repeatedly saying is the impact evaluation (IE) approach (contribution vs attribution) to be selected should consider the characteristics of the intervention. Second, I only was questioning the appropriateness of ''attribution'' for complex and dynamic interventions and never criticized any of

[More ...]
Reply
No Replies
AIS
Thu, 29 May 2014 09:18:05 +0200
Reply
This is Atse M. Yapi again, trying to respond to the first of the concerns of Alice Bonou in her message (nr. 81).

Alice wrote: "For my PhD research, I am trying to estimate the impact of 2012 flooding on the livelihood of farmers in the Niger Basin (micro-level Impact Evaluation). I have some concerns and would appreciate your help regarding them: My first concern is which outcome of interest will be appropriate (yield, income, total expenditure, school expenditure, health expenditure, calorie intake, subjective wellbeing....)?"

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 16:16:08 +0200
Reply
This is Silvia Andrea Perez again.

I have used social network analysis and I think that it is very useful for post-impact evaluation and even for doing the baseline.

To give you some examples: it is useful to track the groups of actors participating in a process or project, to map their linkages (or missing linkages), how they exchange resources (for instance, information) and the direction of exchanges; the redundancy of linkages (power issues involved, types of leadership?), the flow (or not flow at all) of resources, etc. It can be useful to identify the diversity and types of interactions

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 16:03:16 +0200
Reply
This is Mario Pareja again, responding to Message 83, from Dr. Daniel Suryadarma.

I should state my full agreement with Dr. Suryadarma about the need of quantitative data for the social evaluation of the impact of agricultural research & development & innovation (R&D&I). But I still think that some social impacts could be identified, qualitatively, by perception of the target population per se, the farmers and their families, as well as by technicians and decision makers.

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 15:07:37 +0200
Reply
This is Matthieu, Sylvain and Dominique, again (Message 72).

We would like to thank Ed Garrett (Message 77) and Hippolyte Affognon (Message 78) for their responses to our previous message (72) !

We must say however that we are still not totally convinced about the potential of social network analysis (SNA) for ex-post impact evaluation. In one word, how can a method based on observing links between people say anything about changes and evolution of individual people?

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 12:40:09 +0200
Reply
I am Francis Mukoyi from Zimbabwe. I am a plant breeder currently working on cotton with the Cotton Research Institute, Kadoma. Greetings to all participants who have shared their knowledge with us. The whole conference has been an eye opener in many respects of project evaluation.

My career is mainly in crop development, having worked with various crops including rice and Irish potato in the last years. Participatory crop improvement techniques have been the mainstay activity in our programs and I work with farmers from the selection of segregating breeding materials through to the selection of stable advanced materials. I have often realized that there are problems

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 11:46:31 +0200
Reply
This is Daniel Suryadarma, yet again. I found the comments by Hailemichael Taye (Message 84) interesting, but I disagree with the main thrust of his email. 

Hailemichael criticized attribution analysis for lacking internal validity [he wrote: "using attribution analysis in research and extension interventions would lead us false claims"]. He proposed that contribution analysis can be a solution [he wrote: "The essential value of contribution analysis is that it offers an approach designed to reduce uncertainty about the contribution the intervention is making to the observed results through an increased understanding of why the observed results have occurred (or not!)

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 11:11:56 +0200
Reply
This is, again, Mario R. Pareja, from Uruguay, continuing to share our experience on the impact evaluation of the National Institute for Agricultural Research in Uruguay (INIA-UY). This message attempts to summarize our experience in evaluating the social impacts of INIA's technologies in 20 years. To do it, I should state that I am quoting from the social impact chapter of our evaluation and so give the proper credit to the sociologist that worked in our team, Dr. Mariela Bianco, Full Professor of the University of Uruguay. Since I am writing on their  behalf, I repeat the credits to the 

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 10:39:00 +0200
Reply
This is Hailemichael Taye, again. In my earlier email (Message 63), I discussed the challenges of impact evaluation (IE) in agricultural research and extension from defining IE itself to methodological/conceptual limitations in terms of attributing impact to research and extension (R&E) interventions. One of the challenges emanates from the complex and dynamic nature of agricultural research and extension interventions and processes. Now, it is widely understood that agricultural R&E is not a linear process where research develops the innovations, extension disseminates and farmers will adopt and use it. Rather, the innovation development to dissemination to adoption to impact continuum is characterized 

[More ...]
Reply
No Replies
AIS
Wed, 28 May 2014 09:20:50 +0200
Reply
This is Daniel Suryadarma, again, replying to Dr Mario Pareja (message 76). 

While I take the point about data scarcity in developing countries, many social and environmental outcomes are still quantifiable. A quick look through World Development Indicators (http://data.worldbank.org/indicator/all) proves this. And in impact evaluations, we should use these quantitative data as much as possible - including collecting primary data when necessary. As I said previously, solely relying on qualitative data does not allow an estimation of impact.

[More ...]
Reply
No Replies
AIS
Tue, 27 May 2014 17:38:49 +0200
Reply
This is Peter Midmore (coordinator of IMPRESA) here again.

I'm responding to query 1 of Alessandra Coli and Barbara Pacini's Message 31. They seek empirical studies addressing the relationship between public and private R&D funding, particularly in terms of impacts deriving from any complementarity or substitutability between the two.

I have just come across an article by Fuglie and Toole (2014) in early view of the American Journal of Agricultural Economics entitled "The Evolving Institutional Structure of Public and Private Agricultural Research", which raises and explores a number of issues regarding the role of private biotech activities. Also, the reference

[More ...]
Reply
No Replies
AIS
Tue, 27 May 2014 17:23:04 +0200
Reply
I am Alice Bonou from Benin Republic (West Africa). I am an Agricultural Economist and am currently a PhD student in economics of climate change. Hello to everyone, and thanks for all of your contributions which I have read with interest.

First I would like to thank the organizers of this email conference. It is a great opportunity for us to share and learn from each other. I am only sorry that I joined the conference a bit late so I have just completed the reading of all posts.

[More ...]
Reply
No Replies
Ruane, John (DDNR)
Tue, 27 May 2014 11:45:16 +0200
Reply
This is Andrew Fieldsend of the Research Institute of Agricultural Economics in Budapest and IMPRESA national expert for Hungary. IMPRESA (http://www.impresa-project.eu/) aims to evaluate the impact of EU research on agriculture, collecting data on recent trends in investment in agricultural research, and developing a framework combining case studies, econometric analysis and modelling for assessing its impact. Hungary is one of 20 European countries in which IMPRESA has begun a country-level analysis of the agricultural research expenditures and an assessment of the availabilities of data regarding public and private investments in agricultural research. 

[More ...]
Reply
No Replies
AIS
Tue, 27 May 2014 09:53:54 +0200
Reply
This is Litha Magingxa from the Agricultural Research Council of South Africa where I am responsible for overseeing the package of work that analyses and packages outputs from bio-physical agricultural research for the different stakeholders including impact assessment.

I am joining this fascinating discussion rather late. I agree fully with the points raised by Silvia Andrea Perez (Message 57).

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 18:08:21 +0200
Reply
This is Hippolyte Affognon from the International Centre of Insect Physiology and Ecology (Head of Social Science and Impact Assessment Unit).

A contribution to message 72 (from Matthieu Stigler, Dominique Barjolle and Sylvain Quiédeville):

I think that social network analysis (SNA) can be very helpful to impact evaluation as social network is very important for knowledge dissemination. As we know, knowledge and information are elements that propel an increase in agricultural productivity and rural incomes. Impact is consecutive to adoption following knowledge and information dissemination. SNA can then be used as a method for identifying effective and relevant pathways for

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 18:05:07 +0200
Reply
This is Ed Garrett, previous Fulbright Fellow at Hungarian Research Institute of Organic Agriculture (Messages 4, 29 and 37), in response to Message 72 by Matthieu Stigler and colleagues on the use of Social Network Analysis (SNA).

There are several people at the University of California, Davis who are working on the use of SNA as a tool for epIA.  When I combine the questions, comments, and concerns of Matthieu and colleagues in Message 72 with education models, there seem to be some methods to manage complexity and cost. One that seems most promising is the development of electronic forums

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 18:04:15 +0200
Reply
This is Mario Pareja, from Uruguay, again. This is in reply, or better in support of, Dr Suryadarma (Message 70).

I stress the importance of the statement made in that message that "The central issue of impact evaluation is the validity of the counterfactual. Without an explicit construction of a counterfactual, there is no impact evaluation.". Absolutely!

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 18:03:31 +0200
Reply
I am Abdulmojeed Yakubu, a Senior Lecturer of Animal Breeding and Genetics at Nasarawa State University, Keffi, Shabu-Lafia Campus, Lafia, Nigeria.
 
I quite agree with the view of AbdulJabbar Alkhazraji (Message 71) as regards the difficulties faced by most of the developing countries in assessing and evaluating agricultural research. In Nigeria for instance, having access to internal research funds is a very serious problem as there are very few of such research granting bodies. Evaluating research output is not given a priority attention by the government as this is done mainly by research institutes and organizations of foreign extractions. Take,

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 18:02:46 +0200
Reply
From Mark Reader, Research Assistant in Rural Business, at the University of Cambridge, UK. 27 years in agricultural research, development and economics.

As someone whose job is now to measure physical production and economic viability of farms in England, I would like to support the view of Dr Lele [Message 69] in drawing attention to a role for ongoing monitoring in the context of ex-post Impact Assessment.

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 18:01:55 +0200
Reply
My name is Fred Zaal, I'm a senior advisor at the Royal Tropical Institute (KIT) in Amsterdam. Please look at www.kit.nl/sed for more information on my affiliation.

The issue discussed must occur on many occasions and in many ways. The non-economic impact is hard to assess in comparison with the quantitative methods developed for the economic impact of agricultural research for development (AR4D). Indeed, in our work we've come across this often, and have used the livelihoods approach to try to collect qualitative data in a systematic way. The issues mentioned (baseline, adoption, indicators and attribution) by Mario Pareja (Message 66),

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 13:41:53 +0200
Reply
This is a joint contribution from Matthieu Stigler (I previously posted Messages 15 and 44), Dominique Barjolle (Message 61) and Sylvain Quiédeville from the Research Institute of Organic Farming (FiBL) in Switzerland. Part of FibL's work in the IMPRESA project (http://www.impresa-project.eu/) is to prepare a case study manual which will document the methodological framework to be used later in the project for six case studies of scientific research-based innovation. 

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 13:27:17 +0200
Reply
I'm AbdulJabbar Alkhazraji, an Iraqi  scientist and researcher holding a PhD, with 36 years of experience working with Governmental Departments, Educational Faculties and Private laboratory equipment's companies with different positions in Iraq.

I see that the difficulties we're faced with in Iraq and most of developing countries in assessing and evaluating agricultural research include the following:
1st: The  lack of strategic policies in the agricultural sector.
2nd: The absence of coordinating system between research scientific centers (universities, research center...etc) and private sector.
3rd: The instability of agricultural planning and programs for long time, due to instability of the political situation.

[More ...]
Reply
No Replies
AIS
Mon, 26 May 2014 13:21:10 +0200
Reply
This is Daniel Suryadarma again. In my previous message (nr. 64), I discussed the main drawback of propensity score matching (PSM). Earlier today, I started out writing about a number of evaluation methods that should be considered ahead of PSM. Halfway through, however, I began to think that it is much more efficient to just recommend a book that I had used when I was teaching quantitative impact evaluation in Australia - Gertler et al on Impact Evaluation in Practice, which can be freely downloaded from the World Bank website (reference below). This book has not been mentioned before in 

[More ...]
Reply
No Replies
AIS
Sun, 25 May 2014 18:00:31 +0200
Reply
Dear Participants,

This e-mail conference began on 5 May and the last day for sending messages to me for this conference is 1 June. The final messages will be posted on 2 June and the conference is then finished.

About 620 people have subscribed themselves to the conference and it represents a unique opportunity for people to share their knowledge and experiences and to learn from each other regarding the approaches and methodologies that can be used for ex post impact assessment (epIA) of agricultural research.

[More ...]
Reply
No Replies
AIS
Sun, 25 May 2014 17:56:27 +0200
Reply
This is Julien de Meyer, until recently working as Agricultural Research Officer in the FAO Research and Extension Unit in Rome and now a freelance consultant in Australia.

FAO in collaboration with the SOLINSA project (www.solinsa.net, funded by the 7th Framework Programme of the European Commission) recently conducted a case study in South Tyrol in Italy to assess their apple production innovation system. I led that study and we found that in the last decades, the productivity of the system had increased dramatically: In 40 years, from 1960 to 2000, the labor needed to farm a 1 ha apple orchard

[More ...]
Reply
No Replies
AIS
Sun, 25 May 2014 17:55:56 +0200
Reply
This is Mario Pareja, from Uruguay, attempting to share with all of you our experiences in evaluating 20 years of investment in agricultural research & development & innovation (R&D&I) by the National Institute of Agricultural Research (INIA) in Uruguay (INIA-UY). In this message, I would like to provide you with a general framework for the evaluation, the team we formed and other issues related to the juridical-legal and institutional contexts, as well as some of the conclusions. Not all these may be applicable to the CGIAR centres but we believe thay could be of use to the national agricultural research 

[More ...]
Reply
No Replies
AIS
Fri, 23 May 2014 09:11:46 +0200
Reply
This is Mario Pareja, again, responding to Message 65 by Huu-Nhuan Nguyen.

I am in agreement that the sustainable livelihood framework could be used for measuring impacts from agricultural research for development (AR4D). However, I expand, and place stress on the limitations  - additional to the ones mentioned by Nhuan and to the ones I stated in my previous messages - that need to be taken into consideration. They are the following:

[More ...]
Reply
No Replies
AIS
Thu, 22 May 2014 16:01:39 +0200
Reply
This is Huu-Nhuan Nguyen again.

In my previous discussion (Message 5), I already mentioned that current impact assessment practices tend to focus more on economic impact and ignore non-economic impacts (e.g., human, social, physical and natural impacts) which are also vital for measuring the contribution of agricultural research for development (AR4D) to local livelihood. Mario R. Pareja (Message 48 & 58), also discussed that "the change must be reflected in the quality of people's livelihoods" (Message 58).

[More ...]
Reply
No Replies
AIS
Thu, 22 May 2014 09:23:01 +0200
Reply
My name is Daniel Suryadarma, I am the senior scientist in charge of impact assessment at the Center for International Forestry Research (CIFOR). 

I am replying to the question by Amadou Binta Ba about impact evaluation methods (Message 13). Basically, identification of propensity score matching (PSM) requires two assumptions:

First, the unconfoundedness assumption. That is: after controlling for all observable characteristics, no other characteristics - observed or unobserved - influence both participation in the program and the outcomes on which the impact is being evaluated.

[More ...]
Reply
No Replies
AIS
Wed, 21 May 2014 13:53:24 +0200
Reply
My name is Hailemichael Taye from Ethiopia. I am Results Based Monitoring and Evaluation Expert in a R4D project, at International Livestock Research Institute (ILRI). This is very interesting topic and discussion where I expect to learn a lot. I have got information about this program lately and this is my first email. 

[More ...]
Reply
No Replies
AIS
Wed, 21 May 2014 13:27:47 +0200
Reply
I am Dr. Deogratias Lwezaura, again, Principal Economist in the Ministry of Agriculture, Tanzania.
 
I am in agreement with Mario Pareja (Message 58) in the definition of outcome, impact and sustainability. However,  I want to allude to the definition and approaches derived from the outcome mapping tool developed by the International Development Research Centre (IDRC). According to outcome mapping: success of a program or project is assessed or looked at the outcome level, of which significantly I am in agreement. Outcome is any observed changes in people's attitudes, behaviour, practices, activities, values, etc. This is the most important as far

[More ...]
Reply
No Replies
AIS
Wed, 21 May 2014 13:02:38 +0200
Reply
I am Dominique Barjolle, deputy director of FiBL, the Research Institute of Organic Farming in Switzerland. I have done much research (and worked directly) in the past on Innovation in Agriculture, and am currently involved in the IMPRESA project (http://www.impresa-project.eu/), with the role of designing the case studies together, among others, with Matthieu Stigler. 

[More ...]
Reply
No Replies
AIS
Wed, 21 May 2014 11:54:46 +0200
Reply
This is Ekanath Khatiwada, again (previous message 2).

I fully agree with Mario R. Pareja (Message 58) on the logic of results based management (RBM) system, which is becoming one of the popular tools within the UN, World Bank groups and other development programmes on the ground. RBM system captures the results attributions beyond the technical aspects.

[More ...]
Reply
No Replies
AIS
Wed, 21 May 2014 11:46:44 +0200
Reply
I am Shahid Sheikh from Pakistan, freelance consultant on standardization in agriculture sector to add more value in international trade. I started my carrier from a PL 480 project of US on "Sterile Male Technique for the control of fruit flies in Pakistan" later shifted my services to R&D activities on chemical pesticides up to retirement. 

[More ...]
Reply
No Replies
AIS
Wed, 21 May 2014 10:03:31 +0200
Reply
This is Mario R Pareja, again.

I congratulate John Ruane for having done an excellent summary on the conference background paper: “Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives”. I am only sorry that I joined the conference a bit late so I have just completed its reading. I would like to point  to one of the phrases in such a paper included in the following paragraph because I think it is relevant and I often see terms being confused. The paragraph is: 

[More ...]
Reply
No Replies
AIS
Wed, 21 May 2014 09:57:45 +0200
Reply
This is Silvia Andrea Perez, Wageningen University, again (previous message 7).

I agree with Justus Kavoi (Message 45) that it is important that participation from different stakeholders takes place in the process of impact assessment. I would like to add that the participatory monitoring and evaluation should be done even over time, during the process, going beyond a static pre and post assessment with very predefined plans and goals. There might be opportunities to change and even innovate in the process of a project, so that these also need to be monitored, tracking the events when stakeholders adjust their actions

[More ...]
Reply
No Replies
AIS
Tue, 20 May 2014 22:01:38 +0200
Reply
This is Priscila Henríquez, specialist on innovation at the Inter-American Institute for Cooperation on Agriculture (IICA), based at Washington DC. First of all, a disclaimer: I am not an evaluation specialist, so my contributions stem from a pragmatic experience of 25 years working in agricultural research and innovation.

One of my current responsibilities is to provide technical support for the Regional Fund for Agricultural Technology (FONTAGRO), sponsored by IICA and the Inter-American Development Bank (IDB). This is a unique cooperation mechanism to promote agricultural innovation in Latin America and the Caribbean (LAC), formed by 15 countries. FONTAGRO's capital - contributed

[More ...]
Reply
No Replies
AIS
Tue, 20 May 2014 15:03:03 +0200
Reply
This is Mario R Pareja, from Uruguay, again (previous messages 48 and 50).

I would like to respond to Dr Yapi, Message 51, in relation to economic impacts.

I am in full agreement with the need for participatory and multidisciplinary approaches. In our case (see my previous messages), the evaluation was done by a team composed of economists, sociologists, environment and institutional specialists and we conducted personal interviews (farmers, researchers, authorities, professors, farmer's associations, etc.), field surveys (directed to farmers) and focal groups with technicians that advise farmers. 

[More ...]
Reply
No Replies
AIS
Tue, 20 May 2014 13:36:32 +0200
Reply
Dear Colleagues,

This conference began on Monday 5 May while the last day for receiving messages will be Sunday 1 June 2014. We have therefore just passed the half-way stage in this 4-week conference on "Approaches and methodologies in ex post impact assessment (epIA) of agricultural research: Experiences, lessons learned and perspectives".

[More ...]
Reply
No Replies
AIS
Tue, 20 May 2014 10:35:00 +0200
Reply
This is Huu-Nhuan Nguyen again.

Thank for the active discussion from participants of the conference. I would like to discuss more about how the findings of impact assessment should be communicated among actors. From the question topic 4.7 on ' Communicating the epIA findings' in the conference background document, I realized that existing impact assessments seem to aim at reporting epIA findings to donors and policy makers rather than farmers and agricultural extension networks. Therefore, in this message, I want to focus more on how to communicate epIA findings to relevant stakeholders.

[More ...]
Reply
No Replies
AIS
Tue, 20 May 2014 10:21:49 +0200
Reply
I am Sheilah S. Vergara, Program Specialist for Training and Capacity Building, Regional Center for Asia, International Institute of Rural Reconstruction (IIRR), the Philippines. I was also the Program Manager of the Community Agricultural Technology Program (CATP) in the Philippines funded by the Australian Center for International Agricultural Research (ACIAR). CATP was a project of ACIAR but was a program that had several agricultural projects focusing on adaption of select ACIAR technologies by farmer beneficiaries through the assistance of local NGOs. The program aimed to give farmers the opportunity to choose the technology that will suit them best. They were 

[More ...]
Reply
No Replies
AIS
Tue, 20 May 2014 09:42:39 +0200
Reply
This is Markus Olapade, Evaluation Specialist at the International Initiative for Impact Evaluation (3ie), for which I am primarily working on the Agricultural Innovation Thematic Window.

I would like to add to a point mentioned by Matthieu Stigler (Message 44) regarding results that vary across different studies and draw your attention to two important tools. If some studies show significant effects while other studies, on the same subject, don't, it becomes difficult to judge which study to believe and which not. Here Systematic Reviews and Meta-Analysis become important.

[More ...]
Reply
No Replies
AIS
Tue, 20 May 2014 09:27:59 +0200
Reply
This is Atse M. Yapi again.

This time, I would like to share three lessons learned from my experience in assessing the economic impacts of agricultural research technologies:

1) Impact assessment is a multi-disciplinary undertaking, even though it is in many cases led by the socio-economic divisions of the research institutions. It is participatory as it needs the contributions of all the stakeholders who participated in (i) the development of the technology (research scientists from plant breeders to agronomists, socio-economists and research managers/administrators; as well as farmers); (ii) the dissemination of the technology (extension technicians, seed centers and distribution agents,

[More ...]
Reply
No Replies
AIS
Mon, 19 May 2014 17:35:00 +0200
Reply
This is Mario R Pareja, again. I have just posted a message with the experience we, as a team (with specialists in economics, sociology, environment, and institutions), developed during the evaluation of 20 years investment in agricultural research, development and innovation (R&D&I) by the Instituto Nacional de Investigación Agropecuaria (National Institute for Agricultural Research, INIA) in Uruguay. 

[More ...]
Reply
No Replies
AIS
Mon, 19 May 2014 16:37:19 +0200
Reply
I am Doris Marquardt, active in GCARD (Global Conference on Agricultural Research for Development) matters since 2009, currently working at the European Academy (EURAC) in Bolzano, Italy, before that at the Germany-based Leibniz Institute for Agricultural Development in Central and Eastern Europe (IAMO), primarily focusing in my on RD and community development issues in practice supplemented by social networking and policy design at a theoretical and practical level. Thanks to the organizers and contributors to the interesting conference.
 
Picking up Message 41 (by Atse Yapi) on partial adoption of innovation, two other approaches appear also feasible:
a) not to ask

[More ...]
Reply
No Replies
AIS
Mon, 19 May 2014 15:53:17 +0200
Reply
This is from Mario Pareja. I am an independent consultant from Uruguay who has worked in more than 50 countries (Latin America and the Caribbean, Africa, Asia) around the world in themes including the development, monitoring and evaluation of projects and programs, natural resource management, agriculture, livelihood security and others. 

[More ...]
Reply
No Replies
AIS
Mon, 19 May 2014 13:32:18 +0200
Reply
This is Anna Augustyn again (I sent Message 9).
 
I would like to add some points to discussion on the use of participatory methods in ex post impact assessment (epIA) of agricultural research, mentioned by a couple of colleagues. I’ve been working extensively with both conventional and participatory methods, in scientific research and in public policy consultancy. My experience is that a good way is to combine both.
 
I'm screening now various types of IA (such as social impact assessment (SIA), heritage impact assessment (HIA), environmental impact assessment (EIA) etc.) and I am realizing that concerns are pretty

[More ...]
Reply
No Replies
AIS
Mon, 19 May 2014 12:57:12 +0200
Reply
This is James Stevenson again (previously message 14, on selection bias), from the Standing Panel on Impact Assessment (SPIA) of the Consultative Group on International Agricultural Research (CGIAR).

Last week we re-launched the CGIAR impact assessment website following a major overhaul. You can find a lot of useful materials for impact assessment of agricultural research at: http://impact.cgiar.org

[More ...]
Reply
No Replies
AIS
Mon, 19 May 2014 11:44:00 +0200
Reply
This is from Justus M Kavoi again (previous message was nr. 36).

I agree with the views of many participants in the on-going electronic conference on ex-post impact assessment - quite often, whenever a new development project is being designed, in most cases, the issue of participatory monitoring and evaluation (PM&E) is overlooked, especially more so with respect to the budget to carry out PM&E. PM&E is at times viewed as a stand-alone project activity. PM&E needs to be embraced by all development partners and stakeholders that PM&E is part and parcel of the entire project and should be planned

[More ...]
Reply
No Replies
AIS
Mon, 19 May 2014 11:38:33 +0200
Reply
This is Matthieu Stigler again (my first message was nr. 15).

Coming back to Message 20 by Peter Midmore, I think there is some confusion that has slipped in between two similar, yet distinct concepts: self-selection (Message 13, by Amadou Binta Ba) and sample selection (Message 20). While similar, these concepts have implications at different levels, threatening either "internal" or "external" validity. Quickly defined, internal validity is whether conclusions of a study validly apply to the sample considered, while external validity is whether conclusions that are valid for a sample also apply to the larger population, or other populations not

[More ...]
Reply
No Replies
AIS
Sun, 18 May 2014 07:46:52 +0200
Reply
This is Andrea Sonnino, currently working at FAO as chief of the Research and Extension Unit, but soon back to the Italian research institute the Agenzia nazionale per le nuove tecnologie, l’energia e lo sviluppo economico sostenibile (ENEA).

I appreciate very much the discussion of this electronic conference. I would like to underline a point that has been made by several contributors, especially by Dr. Deogratias Lwezaura (message 38): evaluation of impact should properly be planned or designed right away from project design. It seems quite obvious, but this is not always the case.

[More ...]
Reply
No Replies
AIS
Sat, 17 May 2014 06:52:31 +0200
Reply
This is Dr V Arunachalam from the Indian Council of Agricultural Research (ICAR) Complex at Goa, India. I''ve been involved in production of seeds and planting materials of horticultural crops.

I found the messages interesting and useful especially the specific protocols in Message 8 by Dr John Mayne for natural resource management (NRM) research. I found them very specific. But I did not find such specific protocols for seed and planting materials. I would be grateful to know whether such specific protocols exist for seeds and planting materials of improved/local varieties. Because, seed or planting material plays a major role

[More ...]
Reply
No Replies
AIS
Fri, 16 May 2014 18:20:52 +0200
Reply
This is Atse Yapi again.

Dr. Dick Tinsley's contribution (message 35) raised many interesting points, one of which I wish to comment on here.

Dick wrote: "... and wonder how you would factor into your ex-post facto evaluation farmers who have understood the innovations, like it and want to adopt it, but don’t have the means to do so, or do so only on a limited area, while make some substantial modifications to accommodate their limited operational resource base".

[More ...]
Reply
No Replies
AIS
Fri, 16 May 2014 17:12:18 +0200
Reply
I am Maria Carolina De la Fuente. I am M.Sc Agricultural Economics and agronomist. Three years ago, I was introduced to impact assessment through my Master's thesis in Agricultural Economics. I currently serve as the responsible person for impact assessment at the Institute of Agricultural Research (INIA), Chile.

My interest in being part of this conference arises because of the different types of evaluation methodologies that I have met through my work, since I have come to understand that the application of different methods of impact assessment depends on the area you want to cover. For example, when doing my

[More ...]
Reply
No Replies
AIS
Fri, 16 May 2014 17:06:17 +0200
Reply
Hello, I am Shams F. El-Shamy, lecturer in Entomology Department, Faculty of Science, Ain-Shams University, Cairo, Egypt. I worked in nutritional ecology of predatory insects (beetles) for agricultural pest in my master in Denmark. In my PhD I worked with different natural control methods to control stored-grain insects in Egypt. Generally, I am interested in natural control and integrated pest management (IPM). 

[More ...]
Reply
No Replies
AIS
Fri, 16 May 2014 11:37:49 +0200
Reply
My name is Dr. Deogratias Lwezaura, Principal Economist, Ministry of Agriculture, Tanzania. I also serve as M&E specialist of the Eastern Africa Agricultural Productivity Project, Tanzania.
 
I have been following the good discussion with interest and I have some comments to make.

When I read through the discussions, I see there are still critical methodological issues whose consensus is still not conclusive. For example, evaluating adoption: how do we sample the categories of respondents such as project participants, non-participants, participating district/region or non-participating district/region/province so that we came up with statistical conclusion of adoption rates or proportion of adopters

[More ...]
Reply
No Replies
AIS
Thu, 15 May 2014 16:45:21 +0200
Reply
Hello again, Ed Garrett finishing my Fulbright Fellowship at the Hungarian Research Institute of Organic Agriculture.

Specifically to Dr. Tinsley's concern about getting good data from participatory research and post impact assessment dialogues (Message 34), I want to stress that each field of research has its own needs for specialized training.

We do treat validation of data very seriously in ethnographic research by working with interview scripts, having our own questions evaluated by those outside of our research to understand in which direction they are leading. We do know that the act of interviewing changes the ideas of the interview subject,

[More ...]
Reply
No Replies
AIS
Thu, 15 May 2014 16:44:58 +0200
Reply
I am Justus M. Kavoi, a senior researcher with the Kenya Agricultural Research Institute (KARI), currently in the final year of my PhD studies (Agricultural extension) at Egerton University, Njoro, Kenya.

My observations on the on-going discussions about participatory procedures in impact assessment are that, there is need to have a blend of the two sides of the coin: 1) Embrace participation/involvement of all concerned stakeholders, especially the farming community on one side because they are better placed to state whether or not the project under review has had any impact on their livelihoods or not and if they could

[More ...]
Reply
No Replies
AIS
Thu, 15 May 2014 16:44:27 +0200
Reply
This is Dick Tinsley again.

As I have been reviewing the various contributions to this E-consultation, I get the impression that acceptance of agriculture research results is 100% discretionary on the part of the farmers including smallholder farmers, and the transfer of knowledge on research innovations is all that is needed for willing farmers to apply it. I would question this, and wonder how you would factor into your ex-post facto evaluation farmers who have understood the innovations, like it and want to adopt it, but don’t have the means to do so, or do so only on a limited

[More ...]
Reply
No Replies
AIS
Thu, 15 May 2014 11:14:08 +0200
Reply
This is Dick Tinsley again.

I have noted considerable discussion on participatory methods in this conference. I would like to enter a level of caution on this as participatory procedures can easily be leveraged toward the bias of the research promoting an innovation. Too easy to ask leading questions, or receive appeasement replies. Thus these need to be complimented with some more solid observations or other measurements. As much as I have always enjoyed my many informal discussions with smallholder producers in all the countries I have had the pleasure to work, I tend to be skeptical of reliance on

[More ...]
Reply
No Replies
AIS
Thu, 15 May 2014 11:05:40 +0200
Reply
This is Huu-Nhuan Nguyen again.

I would like to have further discussion about communication issues in ex-post impact assessment (epIA). Thanks to Matthieu Stigler (Message 30) for your good comments. I would like to discuss two concerns that Mathieu and myself have mentioned:

1) If the active participation of key stakeholders and beneficiaries is needed for ex-post impact assessment

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 17:39:22 +0200
Reply
This is Dr C. Virginie Mfegue, plant pathologist in Cameroon.

I’ve been involved in research projects aiming to empower small scale farmers in terms of diseases and pests control. After setting and conducting the project in a more or less participatory approach, impact assessment is often left besides. When impact assessment is integrated to the project package, the time allocated to this step is somehow short cut. One reason is that research projects are overlapping, within the same regions, and the same groups of farmers, and we scientists are sometimes in a rush to implement new projects. I totally agree

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 15:57:06 +0200
Reply
This is from Alessandra Coli and Barbara Pacini, statisticians at the University of Pisa (UniPi) - members of the UniPi team in the IMPRESA project (http://www.impresa-project.eu/).
 
While we have some expertise in experimental and observational methods to assess causal effects of specific interventions at the micro-level,
we would like to have some insights on assessing the impact of agricultural research at macro-level, starting from some basic issues.
 
Regarding methodologies and data requirement, we would like to have some feedback on the following issues:

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 15:10:01 +0200
Reply
This is Matthieu Stigler again (Message 15).

I would be interested in having more details for the message of Huu-Nhuan Nguyen, who advocates the use of participatory methods.

I understand from your message that participatory methods can be used from two perspectives, one trying to involve more stakeholders in the project simply by involving them in the evaluation, the second perspective of full empowerment. Maybe one should add to these a third category, an "instrumental" one, stating that participation will increase the quality of the evaluation itself, just for the sake of the evaluation. I am not sure however whether

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 14:46:17 +0200
Reply
This is Ed Garrett again, from the Hungarian Research Institute of Organic Agriculture (ÖMKi) in Budapest, Hungary, working with innovation at the farm level and farmer education needs.

I response to the issue of bias in selecting case studies. I would recommend that all cases be recorded and then a randomization of the cases be used for selection. This could help to control for selection of "interesting" studies and also encourage deeper investigation of the cases to fully report on them.

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 12:54:16 +0200
Reply
This is Dr. Amadou Issaka again (Message 3).

I agree with Dr. Atse M. Yapi (Message 21) in the sense that impact cannot be attributed to the sole project in a target area. The problem we face is, very often, donors, governments and other policy-makers ask impact assessment specialists to demonstrate (quality/or quantity) what is attributed to the intervention. In addition to that, one of the challenge is how to please or to satisfy everyone regarding your assessment findings, whatever the methodology used. In the past, evaluation is done on demand, while nowadays it is done "if you want to

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 12:45:11 +0200
Reply
I am Annie Murimi from Kenya. I am a certified Monitoring and Evaluation professional and currently pursuing a masters degree in Project Management. My undergraduate was in social sciences. I work as a Development Manager at Utooni Development Organization; a Kenyan NGO working with farmers in the arid and semi-arid areas. 

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 11:50:18 +0200
Reply
I am Dr. Stanley Weeraratna, a Sri Lankan. I was Professor of Agronomy at Ruhuna University, and later Professor of Soils and Water Resources at Rajarata University, both in Sri Lanka. I worked as a FAO Soil Fertility and Microbiology Expert and later as the Chairman, Sri Lanka Sugarcane Research Institute. 

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 11:31:59 +0200
Reply
My name is Peter Stradiot, a private crop consultant in vegetable growing in greenhouses. The regions I advise are mainly in the Mediterranean area and hot climate regions -worldwide. I am also expert in the EU for research evaluation in horticulture. In the need to know more and faster we created also a network of more than 20 independent consultants worldwide. 

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 11:23:12 +0200
Reply
This is Amadou Binta Ba, from Senegal, again (Message 13).

Thanks to Peter Midmore for your contribution (Message 20), you are raising very important issues. As far as I am concerned I don't have experience about evaluation because I am a beginner. But I have some concern about this issue regarding my study. During an interview with local community leaders of my study area I have been told that the project had an effect beyond the target villages. It impacted some neighboring villages because of their position. I think that failure to take into account this type of selection bias

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 11:11:37 +0200
Reply
This is Dr. Atse M. Yapi again. I read with interest Binta's contribution (Message 13) and would like to make few comments and suggestions:

1) Binta you want to assessment the impacts (in terms of income, crop yield and food security) of new farming techniques introduced to family farmers in Senegal; and you are not sure about the best methodology to adopt to get the job done. From your objectives, it may be best to adopt a combination of quantitative and qualitative methods of assessment. The rationale here as I see it is that the new farming techniques have succeeded

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 11:07:16 +0200
Reply
This is Dr. Atse M. Yapi again.

The second point I wish to raise is a comment with regard to S.K.T Nasar's observation in Message 17 that most ex post impact assessments (epIAs) are net present value (NPV) based, and as such they ignore completely the opportunity cost (OC).

I wish to express my conviction that the use of NPV in combination with economic surplus method provides a robust micro-economic foundation to epIAs of agricultural technologies; and as such should not be overlooked as a valid methodological instrument for epIA. In my view, the opportunity cost issue raised by S.K.T.

[More ...]
Reply
No Replies
AIS
Wed, 14 May 2014 11:00:13 +0200
Reply
I am Dr. Atse M. Yapi, an Agriculture and Natural Resource Policy Consultant at the FAO Regional office for Africa, Accra, Ghana. I have been involved in impact assessment of agricultural research technologies from joint research efforts by national agricultural research systems (NARS) and the International Crops Research Institute for the Semi-Arid Tropics (ICRISAT) in Mali, Northern Cameroon and Chad for a number of years. Based on my experience, I would like to make some comments on issues raised in previous messages. 

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 17:19:52 +0200
Reply
Hello everyone, and thanks for all of your contributions which I have read with interest. I am Peter Midmore, coordinator of the IMPRESA project and professor of economics at Aberystwyth University.

My first contribution to this debate is to follow up Messages 13 (by Amadou Binta Ba) and 15 (by Matthieu Stigler) and discuss case study selection issues. While I am a fan of the approach in general for its ability to deal with complex, multi-effect processes at a system level, and its relevance in participatory evaluation, in recent years I have been concerned about the process of selection and

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 15:29:03 +0200
Reply
Hi all, a great discussion. My name is Karlheinz Knickel, I am an independent researcher and consultant based in Frankfurt/Main. I have a particular interest in interfaces of practical, research and policy levels as well as analyses that support the effectiveness of rural, agricultural and environmental policies. I work with the Institute for Rural Development Research (IfLS), which is one of the nine partners in the IMPRESA project (http://www.impresa-project.eu/). 

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 12:16:49 +0200
Reply
This is Huu-Nhuan Nguyen again (Message 5).

As Godswill in Message 6 mentioned, the communication issues exist in ex post impact assessment of agricultural research. From my review of various impact assessment work for agricultural research, especially those in developing countries, I realize that top-down or one-way communication approach has been practiced by many agricultural research projects. In addition, the single quantitative based approach with structured questionnaires and closed questions is likely to help evaluators to get expected information and results rather than to understand how outcomes and impacts are generated and contributed to development.

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 12:15:55 +0200
Reply
This is S.K.T. Nasar, from Kolkata, India, again (Message 1), with thanks to contributors for underscoring the inbuilt complications in the thematic issue. I wish here to mention the ‘idea of agriculture’ and to categorise the types of investment in agricultural research to straighten some complexity of ex ante impact projection (eaIP) vis-ã-vis ex post impact assessment (epIA).
 
1. The enormity of agriculture encompasses aspects from extraterrestrial photon to sub-molecular nano particle, from intracellular structures to supra-organismic agro-ecosystem, and from farming-related ground operations to all spheres related to local, national and global marketing. To this may be added country-based

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 12:14:57 +0200
Reply
My name is Gabriel Esquivel Lopez. My background is in agricultural production from the Universidad Veracruzana in Mexico. I am currently doing postgraduate studies on an evaluation of the aquaculture sector, looking at social aspects, marginalization, production groups, gender, water use, the environment, food, political factors, laws, among others. My interest in this conference on “Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives” is in all aspects, from evaluating at the macro- and micro-level , to the reporting of results to those responsible for public policy. 

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 11:32:41 +0200
Reply
I am Matthieu Stigler, I have been working on quantitative impact evaluation in various places, including FAO, and am currently working on qualitative methods for the case-studies component of IMPRESA (http://www.impresa-project.eu/).

In response to Amadou (Message 13):

Matching methods have become quite popular recently, so you should find plenty of tutorials and introductory papers, that will give you a good overview of the method. Find below a suggested introductory reading list, that should make you feel comfortable to start using the methods.

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 10:49:30 +0200
Reply
My name is James Stevenson, I'm based at FAO in the Secretariat for the CGIAR Independent Science and Partnership Council. I spend most of my time supporting the Standing Panel on Impact Assessment which has a responsibility for oversight of impact assessment activities in the Consultative Group on International Agricultural Research (CGIAR). 

[More ...]
Reply
No Replies
AIS
Tue, 13 May 2014 06:29:29 +0200
Reply
I am Amadou Binta Ba from Senegal, a PhD student in Climate Change Economics. For my thesis, I am doing an ex-post impact evaluation of a climate change adaptation intervention. To achieve this, I need to learn about impact evaluation methods and so for this reason I am happy to take part in this e-mail conference. 

[More ...]
Reply
No Replies
AIS
Mon, 12 May 2014 14:24:49 +0200
Reply
Olawale F. Olaniyan is a researcher and also a consultant with specialization in the management and sustainable utilization of agricultural resources. I particularly have interests in providing evidence-based information that can guide policy and other regulatory frameworks.

Assessing the impact of research in the context of agricultural sciences is multidimensional just like the challenges of the farmers. Also, it is really amazing to realize that the results or outputs of many agricultural research for development cannot be replicated at the farmers' level and thereby making an objective assessment somehow challenging! This matter even gets worse when the impact of such

[More ...]
Reply
No Replies
AIS
Thu, 8 May 2014 17:47:21 +0200
Reply
Dear Participants,

This is just a quick update about the conference on "Approaches and methodologies in ex post impact assessment (epIA) of agricultural research: Experiences, lessons learned and perspectives" which began on Monday.

Over 500 people have subscribed themselves from around the world.

The conference background document is available at http://www.fao.org/docrep/019/as549e/as549e.pdf. It describes the main approaches and methodologies used for carrying out epIA of agricultural research. These were classified into two broad groups: macro-level assessments looking, for example, at the impacts of agricultural research at the regional or national level and micro-level assessments, looking at the impacts of specific research-derived

[More ...]
Reply
No Replies
AIS
Thu, 8 May 2014 10:11:11 +0200
Reply
From David Steane, for many years Head of Animal Breeding and Genetics in the British development agency the Meat & Livestock Commission and later working in FAO - for the last six years before retiring, as chief technical advisor of a project on Conservation and Use of Animal Genetic Resources in Asia. 

[More ...]
Reply
No Replies
AIS
Thu, 8 May 2014 10:03:04 +0200
Reply
Initial introduction: Dick Tinsley, Professor Emeritus, Soils & Crop Sciences, Colorado State University, approaching 40 years working with smallholder farmers and their communities. Author book: Developing Smallholder Agriculture: A Global Perspective and Manager of the website: www.smallholderagriculture.com

As I reviewed the material I think there is another dimension to evaluating the impact of agriculture research. That is the drag imposed on the results by the limited operational resources available to farmers, particularly smallholders. This is basically a decade long oversight resulting from the limitations of agronomy or other biological research. Agronomy does an excellent job of determining what is physically

[More ...]
Reply
No Replies
AIS
Wed, 7 May 2014 17:07:53 +0200
Reply
My name is Anna Augustyn. I come from Poland, have a background in social sciences (rural sociology) and in the last 10 years have been working as a researcher and consultant in rural / agricultural policies. Among others, I contributed to the EU FP7-funded project RuDI – Impact Assessment of Rural Development Policies incl. LEADER (http://www.rudi-europe.net/) - and in the recent 4 years with various expertise for the European Network for Rural Development (http://enrd.ec.europa.eu/) e.g. developing methodologies of network self-assessment, evidence-based and strongly applied policy research and analysis for the European Commission. I am an outspoken fan of participatory action 

[More ...]
Reply
No Replies
AIS
Wed, 7 May 2014 15:14:18 +0200
Reply
I am John Mayne an independent evaluation consultant working primarily on development interventions, and in particular working within the Consultative Group on International Agricultural Research (CGIAR) and several of its research programs over the past several years.

The issue of non-experimental approaches to assessing impact is of great interest to me. Several messages to date have referenced the use of impact pathways and theories of change in this light. I wanted to draw your attention to publications that address this issue, one done for the UK Department for International Development (DFID) and one more specifically on agricultural research for development

[More ...]
Reply
No Replies
AIS
Wed, 7 May 2014 10:50:21 +0200
Reply
I am Silvia Andrea Perez Perdomo. I am from Colombia, but I lived in Uganda for 8 years, which I can say is my second home. Now my third home is here in the Netherlands, where I have been 3 years.

I am particularly interested in this conference on impact assessment of agricultural research because I have been working for more than 15 years in projects related to agricultural research and innovation, working with research and local organisations in Latin American and African countries. Currently I am a PhD candidate at Wageningen University. I am looking at dynamics of innovation-social

[More ...]
Reply
No Replies
AIS
Tue, 6 May 2014 17:07:36 +0200
Reply
This is Godswill Ntsomboh Ntsefong from the Institute of Agricultural Research for Development (IRAD), Cameroon. I am presently the Lipids Analysis Laboratory Manager and Chief of improved oil palm seed production unit at IRAD of La Dibamba. I am equally a researcher and reviewer in peer review journals. My research interventions so far are focused on lipids analysis and integrated crop protection with particular concern on the oil palm. I also focus on the improvement of oil palm smallholder sector as well as Good Laboratory Practices and Management Principles. 

[More ...]
Reply
No Replies
AIS
Tue, 6 May 2014 16:52:21 +0200
Reply
I am Huu-Nhuan Nguyen, PhD candidate at University of Queensland, Australia. I am interested in participatory communication approaches for social change and developing innovative impact assessment frameworks for agricultural research for development (AR4D) projects.

I am currently working on developing a holistic framework for assessing impact of AR4D projects underpinned by participatory communication strategies in culturally diverse environments in the Northwest Highlands of Vietnam. By reviewing existing impact assessment practices in the regions I also found out that the assessment of impact of AR4D projects remains problematic in terms of both objectives and methods.

[More ...]
Reply
No Replies
AIS
Tue, 6 May 2014 16:29:06 +0200
Reply
My name is Ed Garrett, previously a farmer from the United States having a diverse background with livestock, crops, farm systems, and multiple scale and locations. I am completing a Fulbright Fellowship at the Hungarian Research Institute of Organic Agriculture in Budapest where I have been doing ethnographic research looking at On-farm Innovation and the needs of beginning farmers in Hungary.
 
Responding in part to the fifth issue raised by S.K.T. Nasar (Message 1), I want to recommend we take into account the changing body of knowledge on Agriculture (causes and effects) we have when doing our post evaluation

[More ...]
Reply
No Replies
AIS
Tue, 6 May 2014 11:12:56 +0200
Reply
I’m Amadou Issaka, holding a PhD in Agriculture, with over 20 years experiences in Agriculture education, Natural Resource Management & Poverty Reduction. I have specific experience over 12 years in Monitoring & Evaluation within various institutions (including but not limited to - African Development Bank, World Bank, UNDP, African Union, European Union, etc…). In the past I monitored projects in water resources, animal production as well as Agriculture Education. I’m currently working with the International Institute of Tropical Agriculture (IITA Ibadan-Nigeria) as M&E specialist to implement a five-years R&D Project aiming at strengthening African countries to improve four strategic crops 

[More ...]
Reply
No Replies
AIS
Tue, 6 May 2014 10:01:11 +0200
Reply
I am Ekanath Khatiwada, a private sector development specialist with more than 15 years experience both in Africa (ESA Region) and Asia. My sectoral experience includes; promoting various rural agribusiness enterprises, renewable energy enterprises and programme monitoring and evaluations (PME). I conducted a number of market studies; some of them are cardamom sub sector market study in Nepal, gum acacia value chain study in South Sudan, shea butter market assessment in South Sudan and coordinated a biogas feasibility study for Zambia. I am currently an independent consultant working on market development, present assignments include; a Making Markets Work for the 

[More ...]
Reply
No Replies
AIS
Mon, 5 May 2014 19:00:40 +0200
Reply
[Thanks to S.K.T. Nasar from India for sending in the first message of this FAO e-mail conference on the "Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives". All messages will be numbered chronologically. If during the conference you notice that you are missing any messages, just contact me at [log in to unmask] ...Moderator]. 

[More ...]
Reply
No Replies
AIS
Fri, 2 May 2014 11:26:25 +0200
Reply
Dear Colleagues,

Welcome to this FAO e-mail conference on "Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives" !! Thank you for joining.

You can send messages now (send them to [log in to unmask]). Messages will be posted from Monday 5 May onwards while the last day for receiving messages will be Sunday 1 June 2014.

[More ...]
Reply
No Replies
AIS
Wed, 30 Apr 2014 16:14:10 +0200
Reply
Dear Colleagues,

As indicated in my message to you yesterday, I am now sending you the Background Document to the FAO e-mail conference on "Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives"

It aims to provide an easily-understandable introduction and brief overview to the conference topic that you, as participants, will find useful for the e-mail conference. The Introductory Section to the 18-page background document explains first that FAO is hosting the conference as part of its activities in the Impact of Research on EU Agriculture (IMPRESA) project. It then provides an

[More ...]
Reply
No Replies
AIS
Tue, 29 Apr 2014 15:29:46 +0200
Reply
Dear Colleagues,

Welcome to all of you who have subscribed so far to this FAO e-mail conference on "Approaches and methodologies in ex post impact assessment of agricultural research: Experiences, lessons learned and perspectives" !!

The 4-week long conference begins on Monday 5 May.

Later this week, I will send you the Background Document to the conference. Its aim is to provide an easily-understandable introduction and brief overview to the approaches and methodologies used in ex post impact assessment of agricultural research. It also gives specific guidance to participants regarding the questions they should address in the conference.

[More ...]
Reply
No Replies

ATOM RSS1 RSS2