This is from Mario Pareja. I am an independent consultant from Uruguay who has worked in more than 50 countries (Latin America and the Caribbean, Africa, Asia) around the world in themes including the development, monitoring and evaluation of projects and programs, natural resource management, agriculture, livelihood security and others. In 2010 we responded to a call for proposals by INIA (the National Institute of Agricultural Research of Uruguay), forming a four-member team, under my coordination and led and organized by the Instituto Interamericano de Cooperación para la Agricultura (IICA) in Uruguay. We presented a very complete proposal for ex-post impact evaluation of 20 years of investment in agricultural research, development and innovation (R&D&I) at INIA. After winning the bid, we conducted, during the years 2010-2011, the multidimensional (technological, economical, social, environmental and institutional) impact evaluation, combining desk as well as field studies. This was a comprehensive evaluation, including the various dimensions mentioned above, and focusing on institutional, programmatic and project impacts. The process included desk (review of data recorded by INIA and others) as well as field research (various farmers surveys, interviews, focal groups, etc.). We had a local, Uruguayan, evaluation team of 4 experts, which conducted most of the field work supported and guided by an advisory board that included high level scientists and experts on impact evaluation studies, such as Derek Byerlee and Julian Alston, among others, as well as representatives from IICA and the farmers. The report was published in Spanish. There is a summary and a very extensive and complete report (2 volumes). The Executive Summary is available on the web (reference given at the end of the message). We have a long list of lessons learned but, in this message, I just want to emphasize a few of them; the more general. 1) In order to evaluate the impact of the results and products of an agricultural R&D&I institution there is a need to look at the micro- as well as the macro-level. When you get started, it is impossible to separate and isolate the institutional, programmatic and project impacts in the field. Impacts of innovations are measured after adoption, and probably after a few years farmers have implemented, but there are technologies recently released showing results (not yet impacts) that need to be considered and others in the pipeline that need to be looked at as "potential future impacts". 2) The main limitation for impact evaluation is the absence of reliable information. From a solid base line (very difficult to establish in a complex production system such as agriculture) all the way to follow up adoption rates and evidence-based economic, social and environmental impacts. Agricultural R&D&I institutions must develop their own, on-going systems and mechanisms to follow up the technologies they produce and release to the farmers and be able to monitor and measure results and impacts. And, foremost, make sure that a base line is taken before releasing the technology. 3) It is easier to evaluate economic impacts (return to investments and other indicators could be traced and measured at the macro scale) than social and environmental impacts. This is due to a) the limited work done in developing reliable and measurable indicators for social and environmental impacts, and b) the absence, or difficulty to obtain reliable data on these dimensions [examples are i) social changes which are difficult to attribute to agricultural technologies because they are often strongly affected by the implementation of public policies (employment generation, social safety nets, etc.) and ii) environmental impacts of agricultural technologies (soil and water conservation, biodiversity protection, etc.) often need to be looked at a more macro, regional, scale which poses methodological, and financial, difficulties to an impact evaluation study like the one we conducted]. 4) The attribution issue. Usually, an agricultural technology is a result of an initial institutional effort that leads the work, e.g. INIA. But others (universities, farmers' organizations, private enterprises, etc.) may also contribute to its development. Due to this condition, impact is usually difficult to be attributed to a particular agent. I leave it there, for the time being, but I will be glad to expand in any of these issues as well as others related to our experience. Mario R. Pareja Ingeniero Agrónomo, M.S., Ph.D. independent consultant Paraje El Colorado, Canelones, Uruguay Telephone: 598-2-3654394 or 598-98372634 e-mail: parejamr (at) gmail.com Reference: M. Pareja; J. Bervejillo; M. Bianco; A. Ruíz y A. Torres. 2011. Evaluación de los impactos económicos, sociales, ambientales e institucionales de 20 años de inversión en investigación e innovación agropecuaria por parte del Instituto Nacional de Investigación Agropecuaria (INIA) - Uruguay (Evaluation of the Economic, Social, Environmental and Institutional Impacts of 20 years of Investment in Agricultural Research and Innovation by the National Institute for Agricultural Research (INIA), Uruguay). Executive summary (in Spanish), 41 pages. http://www.inia.uy/Publicaciones/Documentos%20compartidos/18429151211092410.pdf (400 KB). [To contribute to this conference, send your message to [log in to unmask] For further information, see http://www.fao.org/nr/research-extension-systems/res-home/news/detail/en/c/217706/ ]. ######################################################################## To unsubscribe from the Impact-L list, click the following link: https://listserv.fao.org/cgi-bin/wa?SUBED1=Impact-L&A=1