Insider-Outsiders: Do Experts Know How Far their Evidence Travels in the Policymaking World?

H.E. Thabo Mbeki, Former President of South Africa at a high-level policymaker discussion conducted by the Brookings Institution Africa Growth Initiative on February 17, 2016. Photo courtesy of the Brookings Institution via Flickr Commons.  

I recently commenced my research fellowship at the Woodrow Wilson International Center for Scholars. I have begun navigating the research and policy landscape of the United States and discussed elements of my research with a couple of experts in Washington.During one of my recent engagements, I asked an expert in academia who uses his research to engage with the U.S. government on peacebuilding in Africa what the outcome of his engagements have been. His response caught my attention, and I quote: "It's very difficult to know what effect, if any, the styles of engagement [I have used] have on any concrete decisions."

This piqued my curiosity because my research hinges on the value of research evidence on the quality of policymaking in Africa.

Why Evidence Matters

The use of research findings in policymaking is not a new phenomenon in the developed world, as seen in the case of the United States. In fact, some countries in the global south are borrowing best practices from countries like the United States. Scholars define evidence-informed policymaking (EIPM)[1] as the consideration of the best available research evidence for policymaking purposes. The evidence-informed policymaking conversation is specific to testable research evidence, as its inherent value is in the fact that it is generated in a systematic and rigorous manner. Not all research evidence is relevant for policymaking.[2] This conversation, however, is on policy-relevant research.

Much of the literature, however, on the evidence-to-policy discourse relates to the health sector. Perhaps this is due to past efforts by the World Health Organization (WHO) — who aimed to have evidence-based policy decisions in the health sector as countries worked towards the attainment of the Millennium Development Goals (MDGs).[3] With time, conversations on evidence and its role in policymaking have floated across other sectors beyond health.

Policymaking can be an opaque process; while well-meaning policymakers want to base their decisions (of which policies are part) on research evidence, there are other variables at play, including personal interests, ideologies, socio-political interests, and the influence of interest groups, among others.[4] For instance, policymakers may push policies that will help them gain political mileage as opposed to addressing real problems in the most effective way.

Further, the impact of research on policymaking is neither linear nor simple.[5] With so many variables, research evidence faces strong competition and at times is at the bottom of the factors driving policy decisions. For example, an expert at a recent research report[6] launch hosted by the Brookings Institution on the use of evidence by decision-makers in the education sector noted that data is rarely the factor that determines the ultimate decision by policymakers. While data is rarely the decisive factor, it can play a supporting role, as can research evidence.

The complex nature of policymaking and the impact policies have on the development trajectory of countries warrants keen attention on these issues. Moreover, policies anchored in evidence respond more effectively to challenges affecting day-to-day citizens.

Tracking Progress 

Once policy-relevant research evidence has reached the policymaking table, does the influence of researchers and knowledge intermediaries end? Experts have pointed out various mechanisms through which one can track the impact of communication and policy engagement efforts.[7] An explicit citation of research in policy documents is one way to attribute the use of specific research. However, what about in cases where the impact of research is not explicitly documented in policy documents, but the research has influenced the framing of policy issues or set the agenda for a new policy issue?

As the expert I spoke to noted, tracking the outcomes of engagements with policymakers is not easy. In addition, more often than not, there are other players working with the same policymaker(s) on the same policy issue, which complicates attribution. Furthermore, policymakers may make progress on policy issues using research evidence supplied by various players. However, the details of the policymaking process are not in the public realm and therefore researchers and other experts in the research-to-policy ecosystem are often left in the dark.

The Conversation Has Not Ended

The use of research evidence in the research-to-policy ecosystem is still a growing field. Therefore, seeking answers to how researchers, knowledge intermediaries, and other experts can track the outcomes of communicating and engaging policymakers using research evidence is an evolving landscape that presents an opportunity for continued research. As I continue with my research, I will seek out the perspectives of different experts on how they track the impact of their engagement with policymakers. For those who have built strong relationships with policymakers, are they really insiders or are they out in the cold in the evidence-to-policy journey?

Experts being privy to the current status of policy issues is critical for various reasons. One, if progress is not made, experts can evaluate whether this is due to the nature of the research evidence provided to policymakers. For instance, was the evidence not strong enough to inform policy decisions? Did other factors come into play such as a shift in government priorities? The point I am making is that getting research evidence to the policymaking table is not enough. Experts working to inform and/or influence policy using research evidence should stay informed on the policy's progress. This reiterates the value of maintaining strong relationships with policymakers beyond the ''launch event" of research findings. If one truly desires to make significant contributions to evidence-informed policymaking, it is a long-term commitment. Fight to stay on the inside.

Diana Warira is a Southern Voices Network Scholar for Peacebuilding for the Spring 2018 term. She is a Communications Officer for the African Institute for Development Policy in Nairobi, Kenya, which is a member organization of the Southern Voices Network for Peacebuilding.  

[1] Andrew D. Oxman, John N. Lavis, Simon Lewin, and Atle Fretheim, "SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking? Health Research Policy and Systems 7, Suppl 1, (December 16, 2009), https://doi.org/10.1186/1478-4505-7-S1-S1.

[2] Diane Stone, Simon Maxwell, and Michael Keating, "Bridging Research and Policy: " (international workshop, Radcliffe House, University of Warwick, July 16-17, 2001).

[3] Paul M. Wilson, Mark Petticrew, Michael W Calnan, and Irwin Nazareth, "Does Dissemination Extend Beyond Publication: A Survey of a Cross Section of Public Funded Research in the UK," Implementation Science 5, no. 61 (2010): 1, https://doi.org/10.1186/1748-5908-5-61.

[4] Jon D. Brehaut and Don Juzwishin, "Bridging the Gap: The Use of Research Evidence in Policy Development," HTA Initiative, no. 18, (September 2005): 23. https://mafiadoc.com/bridging-the-gap-the-use-of-research-evidence-in-policy-development_59cdbce01723ddf665c91434.html.

[5] Ajoy Datta and Arnaldo Pellini. "Communicating Research: A beginner's guide for researchers in Vietnam"Overseas Development Institute, (August 2011): 1, https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/7183.pdf.

[6] Samantha Custer, Elizabeth M. King, Tamar Manuelyan Atinc, Lindsay Read, and Tanya Sethi, "Toward data-driven education systems: Insights into using information to measure results and manage change," Brookings Institution, (February 20,  2018), https://www.brookings.edu/research/toward-data-driven-education-systems-insights-into-using-information-to-measure-results-and-manage-change/.

[7] Josephine Tsui, Simon Hearn, and John Young, "Monitoring and evaluation of policy influence and advocacy," Overseas Development Institute 395, (2014): 9, https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/8928.pdf.