Research and Development: Show me the Impact…
Scientific research has historically been assessed by the level of citations a publication or researcher has – the more the better being the mantra. The reasoning being that the “credible” researcher (or significant work) would automatically lead to citations or popularity: the more “credible” leading to more citations. Problems with this model are quite obvious, as it leads to a “publish or perish” mentality and encourages “popular” or trendy research. In addition, frequently cited publications are, at times, cited for their controversial nature, and not necessarily for their significance or impact in terms of research. But what does that mean for agricultural research?
The final product of agricultural research should, at the end of the day, have a measurable positive impact on the lives of the poor. If that is taken as a given, then we must reconsider our current evaluation models for agricultural scientific research. Various other strategies have been considered to address some of the shortcomings of the above model. However, most of these strategies aim to include the end-users either in the developing of the project or in training at the tail end of the project.
Is this enough? Is there not a better way to measure impact? How can we better link outputs to results? What about accountability?
During a workshop held last year at the GFAR meetings in India, Frank Rijsberman, Sanjini De Silva, and I presented to the participants a model based on Outcome Contracting (you can access the article here). The basis of this model is accountability, both in terms of project design and funding. If the primary goal of our work and research is poverty reduction, should we not be held accountable for it? In the new model, researchers, along with the end users, partners etc, identify the impact pathway of any particular project, and decide up to which point the project can be held responsible. Accountability is established and funding, or partial funding, is awarded upon achieving the intended goal.
Can such an inclusive model work in our current environment? How would that affect our current approach to research? What would be the drawbacks for using such a model?
No Responses to “Research and Development: Show me the Impact…”
Leave a Reply
Enrica Porcari
The CGIAR's first Chief Information Officer is hosted by Bioversity International in Rome, Italy.
All content created by ICT-KM is published under a Creative Commons 3.0 license. You are free to share, republish or remix so long as you attribute ICT-KM clearly as the original source.
I agree with the opinion expressed in the blog. It is not only at the Individual level but also at the Institution and Research System level that we need to relook at assessment.
In GFAR’s case National Agricultural Research Systems (NARS) receive very little direct financial support from GFAR. Our main partners are the Regional Forums. The regional forums’ partners are the (NARS). We also support Regional Networks, Inter-regional networks, Global Partnership Programs and Projects where NARS linkages are established directly and through the regional forums. We also have parallel direct linkages with Civil society, Non Government, Farmer organizations and the private sector. Our linkages at the Global level are with the United Nationals Food and Agricultural Organization (FAO), the Consultative Group on International Agricultural Research (CGIAR) and its International Agricultural Research Centres (IARCs), Non CGIAR IARCS, Advanced research institutions such as Universities.
As you can see there is high complexity on assessing how GFAR achieves its outcomes of fostering dialogues on critical issues related to agricultural research and innovation development, facilitating action and in assessing outcomes of global action in agricultural research for development (ARD). The output based approach would be too shallow to measure the role and impact of GFAR.
A large part of the assessment framework in all ARD activities uses the Log Frame approach because those who invest financially in ARD (Donors) are Log frame based in their assessment. There are many limitations when this approach is applied to organizations such as GFAR whose influence on ARD is significantly larger than the financially support it has or gives to its partners.
I have drawn attention to the need to review current approaches to assessment of IARCs in the meta-EPMR conducted by CGIAR recently. I am of the opinion that the CGIAR as a system and its Institutes face similar problems as GFAR in its assessments.
As a community interested in agricultural research for development, we should all look at whether current approaches to monitoring, evaluation and impact assessment are yielding us neccessary information to gauge how well we are doing for development through agricultural research and innovation.
Ajit Maru
It is good to see that someone is finally beginning to dismantle the false assumptions behind current CGIAR evaluation indicators. In the best case scenario, even the most respectable and oft-cited articles have limited or no impact on those the system claims to assist. At the same time, capacity building activities such as training, direct work with development agencies or with farmer associations receives little weight in our evaluation system. If the CGIAR were expected to contribute to science in a way similar to a northern university, then this evaluation system is fine. For a system that strives to reduce rural hunger and poverty it is clearly not.
While I agree with the overall tone of this post, there are a couple of oversights that are troubling. First, the post seems to accept that scientific knowledge is developed by a researchers in a vacuum. No less troubling is the continued vision of the end user as a passive recipient of knowledge from an advanced research center.
Work over the last several decades (!!!) in areas like participatory and action research shows that both assumptions are wrong. On the one hand, few scientists in the CG fit the mold of lone researchers shut up in laboratories. Nearly all my colleagues work with other people and many of their best ideas come from interactions with people who are not “scientists”. Secondly, knowledge creation rarely comes as a bolt out of the blue. Rather it is a cumulative effort that builds on the work of others.
The issue of users as mere recipients of scientific largess is also mistaken. Time and again studies have shown that users evaluate, test, improve and further develop ideas that come from science to meet their own needs. Failure to recognize that the CG is itself but a small part of a much larger innovation system are disingenuous at best. We are not, however much we might want to be, the center of knowledge creation in agriculture. That title belongs to millions of farmers and other supply chain actors.
Mark Lundy