Experience in Policy Impact
Vanesa Weyrauch
Independent consultant and Associate Researcher at CIPPEC
Within current debates on what is policy influence and how to assess whether a policy research organisation has been successful in its efforts or not, there has been recently an increasing acknowledgment of the complexity of this task, and questions about how much can be really measured or is worth measuring abound.
Indeed, one of the main discoveries among those who take our online course focused in MEL of policy influence is the need to redefine policy influence itself: should only affecting a specific program or policy be considered policy influence? How about achieving changes in attitudes, beliefs, frameworks, ideas, resources, capacities and relationships among stakeholders who influence or can be influenced by policy? Is really contributing to a new policy, or helping modify an existing one, the Holy Grail?
Through rich and continuous exchange with colleagues in developing countries, we have found significant agreement and awareness on the need to expand what is enclosed under the concept of policy influence, to include, for example, short term outcomes at the level of actors (changes in attitudes, behaviors, beliefs, discourses, etc.). Often these changes are needed to then be able to affect their decisions and consequently modify or create new policies informed by the produced research. Therefore, recognition of the long way and diverse nature of influence is crucial in terms of assessing what has worked or has not, including who needs to be touched by our work. In fact, what might seem a win situation: a policymaker adopting a direct recommendation that emerged from our research might turn to be a failure when she decides to provide a new service without carefully managing its implied costs. In this post at P&I, Ulviyya Mikayilova, Policy Unit manager at the Center for Innovations in Education (CIE), from Azerbaijan, describes this paradoxical situation and argues about the need to re-consider what policy influence really is.
Moreover, having contributed to a very specific policy change (i.e. having developed a formula to calculate a more equitable distribution of new funds targeted at expanding school hours for poorest students) is just a new chapter of complex and non-linear story. It is not enough to acknowledge the level of efforts, activities, strategies and relationships that a group of researchers or a research institution has deployed to inform, convince, help others develop such a change. Once the new content is there – or new procedures take place – there is again a long path to walk to contribute to turn that change into a real impact on beneficiaries of that policy. The bottom-line question for many donors and grantees is: did research finally help to improve people´s lives? Is it worth investing in this?
Responding to this question in an intelligent and useful way implies new discussions and reflections from us. For example, are think tanks/policy research institutions accountable for final impact? Should/can they be? Is it enough to claim contribution to a policy change based on high quality and relevant research and not produce later on evidence on how that policy worked or not?
Not a minor dilemma at all. What are the boundaries for our work? Where should the policy influence efforts stop? Is it at the design stage, i.e. having been successful bringing into the table good ideas emerging from research that lead to w new policy or the modification of an existing one? What happens, then, with implementation?
Some organisations may immediately shy away from what happens after. It is the role of the State and it is within the government capacity to strive to ensure that policies are deployed in a way that they reach the intended results as much as possible. External stakeholders can never be accountable for that and should not try to influence or control that process. Furthermore, doing so would diminish the State´s capacity and accountability.
Others, on the contrary, decide to get further involved: some by monitoring and evaluating results of the policy so as to inform future efforts and become social watchdogs in the name of the intended beneficiaries. Some others become engaged in policy implementation: they get their feet in the mud, and work providing technical assistance or developing capacity of public servants so that the policy is well implemented or at least stays in the right direction. This also provides learning for future efforts and recommendations. However, what happens if they can only do this in the first pilots, or only for some level of policymakers? Do these organisations have enough resources (human and financial) to really play a role in policies of mid or large scale? Would they lose independence, autonomy and capacity to continue innovating?
Of course, an approach that acknowledges complexity reveals there is no unique or right answer to these challenges. Quite the other way: it demands those playing this game to further reflect and more wisely determine what their best role would be. Answers may vary according to diverse political contexts, organizational priorities and values, existing capacity of other external stakeholders to play similar roles, etc.
Policy influence is a changing kaleidoscope and one cannot ignore the need to remain flexible and dynamic. However, one should also avoid the risk of just following the flow, responding to where demand and opportunities from others arise without re-visiting at some critical points in development its main goals, mission and vision. Some structure makes a policy research organization healthy, by conserving its identity. Some flexibility allows it to stay relevant and be valued and needed. To strike the balance is not easy at all but probably will help the institution thrive in an ever changing and increasingly complex policy world.
Vanesa Weyrauch
Independent consultant and Associate Researcher at CIPPEC
Within current debates on what is policy influence and how to assess whether a policy research organisation has been successful in its efforts or not, there has been recently an increasing acknowledgment of the complexity of this task, and questions about how much can be really measured or is worth measuring abound.
Indeed, one of the main discoveries among those who take our online course focused in MEL of policy influence is the need to redefine policy influence itself: should only affecting a specific program or policy be considered policy influence? How about achieving changes in attitudes, beliefs, frameworks, ideas, resources, capacities and relationships among stakeholders who influence or can be influenced by policy? Is really contributing to a new policy, or helping modify an existing one, the Holy Grail?
Through rich and continuous exchange with colleagues in developing countries, we have found significant agreement and awareness on the need to expand what is enclosed under the concept of policy influence, to include, for example, short term outcomes at the level of actors (changes in attitudes, behaviors, beliefs, discourses, etc.). Often these changes are needed to then be able to affect their decisions and consequently modify or create new policies informed by the produced research. Therefore, recognition of the long way and diverse nature of influence is crucial in terms of assessing what has worked or has not, including who needs to be touched by our work. In fact, what might seem a win situation: a policymaker adopting a direct recommendation that emerged from our research might turn to be a failure when she decides to provide a new service without carefully managing its implied costs. In this post at P&I, Ulviyya Mikayilova, Policy Unit manager at the Center for Innovations in Education (CIE), from Azerbaijan, describes this paradoxical situation and argues about the need to re-consider what policy influence really is.
Moreover, having contributed to a very specific policy change (i.e. having developed a formula to calculate a more equitable distribution of new funds targeted at expanding school hours for poorest students) is just a new chapter of complex and non-linear story. It is not enough to acknowledge the level of efforts, activities, strategies and relationships that a group of researchers or a research institution has deployed to inform, convince, help others develop such a change. Once the new content is there – or new procedures take place – there is again a long path to walk to contribute to turn that change into a real impact on beneficiaries of that policy. The bottom-line question for many donors and grantees is: did research finally help to improve people´s lives? Is it worth investing in this?
Responding to this question in an intelligent and useful way implies new discussions and reflections from us. For example, are think tanks/policy research institutions accountable for final impact? Should/can they be? Is it enough to claim contribution to a policy change based on high quality and relevant research and not produce later on evidence on how that policy worked or not?
Not a minor dilemma at all. What are the boundaries for our work? Where should the policy influence efforts stop? Is it at the design stage, i.e. having been successful bringing into the table good ideas emerging from research that lead to w new policy or the modification of an existing one? What happens, then, with implementation?
Some organisations may immediately shy away from what happens after. It is the role of the State and it is within the government capacity to strive to ensure that policies are deployed in a way that they reach the intended results as much as possible. External stakeholders can never be accountable for that and should not try to influence or control that process. Furthermore, doing so would diminish the State´s capacity and accountability.
Others, on the contrary, decide to get further involved: some by monitoring and evaluating results of the policy so as to inform future efforts and become social watchdogs in the name of the intended beneficiaries. Some others become engaged in policy implementation: they get their feet in the mud, and work providing technical assistance or developing capacity of public servants so that the policy is well implemented or at least stays in the right direction. This also provides learning for future efforts and recommendations. However, what happens if they can only do this in the first pilots, or only for some level of policymakers? Do these organisations have enough resources (human and financial) to really play a role in policies of mid or large scale? Would they lose independence, autonomy and capacity to continue innovating?
Of course, an approach that acknowledges complexity reveals there is no unique or right answer to these challenges. Quite the other way: it demands those playing this game to further reflect and more wisely determine what their best role would be. Answers may vary according to diverse political contexts, organizational priorities and values, existing capacity of other external stakeholders to play similar roles, etc.
Policy influence is a changing kaleidoscope and one cannot ignore the need to remain flexible and dynamic. However, one should also avoid the risk of just following the flow, responding to where demand and opportunities from others arise without re-visiting at some critical points in development its main goals, mission and vision. Some structure makes a policy research organization healthy, by conserving its identity. Some flexibility allows it to stay relevant and be valued and needed. To strike the balance is not easy at all but probably will help the institution thrive in an ever changing and increasingly complex policy world.
No comments:
Post a Comment