This is from Alessandra Coli and Barbara Pacini, statisticians at the University of Pisa (UniPi) - members of the UniPi team in the IMPRESA project (http://www.impresa-project.eu/).
While we have some expertise in experimental and observational methods to assess causal effects of specific interventions at the micro-level,
we would like to have some insights on assessing the impact of agricultural research at macro-level, starting from some basic issues.
Regarding methodologies and data requirement, we would like to have some feedback on the following issues:
1) Public versus private R&D funding:
To the best of our knowledge, research expenditure (R&D funding) in agriculture plays the role of treatment variable in this literature. Public R&D funding may have an impact on private R&D funding. In our opinion, the relationship between the two (complementarity or substitutability) should be taken into account when trying to correctly attribute the effects. We wonder whether any participants could suggest to us some empirical studies addressing this issue.
2) Time lag between research investments and selected outcomes:
R&D expenditures cover basic research, applied research, and experimental development. The different kind of research implies a different time interval separating the investment and the research productivity assessment. Therefore it would be necessary to analyse the impact of basic research, applied research, and experimental development expenditures separately and accurately define the related time lag. To the best of your knowledge, are there European data (at country or, even better, at NUTS 1 level) and studies fitting these requirements? [The NUTS (Nomenclature of territorial units for statistics) classification is a hierarchical system for dividing up the economic territory of the EU. The NUTS 1 level divides it up into major socio-economic regions (http://epp.eurostat.ec.europa.eu/portal/page/portal/nuts_nomenclature/introduction) ...Moderator].
3) Focus on the causal mechanism:
A well-defined causal effect should consider the specific target and objectives of R&D investments. The original objectives of the funding should guide in assessing the effectiveness of the investment. In a causal perspective, it does not make sense to compare investments primarily designed to reach different targets and different objectives (economic, social, environmental, etc.). To this end, it would be useful to analyse funding policies (i.e. R&D expenditure) by purpose. We are not aware of studies in this direction.
4) On the use of a counterfactual approach at macro-level:
We agree that the issue of attribution, (i.e. isolating the effect of the intervention from other factors and potential confounders) is central in the epIA literature. As there is no counterfactual situation to observe at the macro level, most studies focus on statistical relationships among research expenditures variables and one or more outcomes of interest. It is possible to interpret such relationships as causal effects under assumptions that may be quite strong (for example, the estimate of the economic return to R&D as a constant elasticity parameter in a parametric production function). We would be interested in methods to conduct proper impact evaluation analysis using a causal inference approach, trying to reproduce a counterfactual at the aggregate level. Our experience in this applied context is quite limited, but we found some interesting suggestions in the recent econometric literature, see for example the synthetic control methods proposed by Abadie et al (2010) for comparative case studies with macro data.
Alessandra Coli
Assistent Professor of Economic statistics
Department of Economics and Management (University of Pisa)
Via Cosimo Ridolfi 10, 56124
a.coli(at)ec.unipi.it
Barbara Pacini
Associate Professor of Statistics
Department of Political Science (University of Pisa)
Via Cosimo Ridolfi 10, 56124
barbara.pacini(at)sc.unipi.it
Reference:
Abadie, Alberto, Alexis Diamond, and Jens Hainmueller (2010), "Synthetic control methods for comparative case studies: Estimating the effect of California's tobacco control program," Journal of the American Statistical Association 105:490, 493-505.
http://dspace.mit.edu/openaccess-disseminate/1721.1/59447
[To contribute to this conference, send your message to [log in to unmask] For further information, see http://www.fao.org/nr/research-extension-systems/res-home/news/detail/en/c/217706/ ].
########################################################################
To unsubscribe from the Impact-L list, click the following link:
https://listserv.fao.org/cgi-bin/wa?SUBED1=Impact-L&A=1
|