Methodological choices to inform policy
Andrea Ordonez
Associate of Politics & Ideas
There is a persistent perception that there is a tension between rigorous research and research being relevant, thus increasing the chance of having an impact. Of course, I understand the origins of this tension. For example, some rigorous methods require more time than those given by policy windows, a reconciliation that is not always easy.
Nonetheless, I believe that there has to be ways by which a research can make better methodological choices that can be rigorous and also coherent with the policy context. This is why in Politics and Ideas, we are developing a set of tools, and capacity building strategies that aim at helping researchers to navigate the methodological choices with a compass set on policy priorities. Here, I will share some ideas of navigating methodological choices that researchers can keep in mind for their current and future projects.
1. The distinction between qualitative and quantitative research is not very informative
When researchers describe the methodological choices, they usually state whether they will use quantitative, qualitative or mixed-methods. These categories have become popular, both are not very helpful to understand exactly what is being done, and how this will answer as a research question. Furthermore, this can scale up into a competition, between those in the “qualitative camp” and those in the “quantitative camp”, a discussion that is not very fruitful, as each method has its strengths for certain types of questions, and its limitations for others. When considering what the best method is, try to think beyond these categories.
2. Research methods as your own Swiss knife
One way of thinking about methods, is to imagine that they are a Swiss knife: a collection of tools, each one with a particular strength. For example, you wouldn’t use a knife to open a bottle of wine when you have a Corkscrew among your tools. The same should be true for methods: researchers focused on informing policy, should develop a variety of methodological skills to choose from, depending on the specific need of each occasion. Given that you have these tools at hand, the key next step is to choose among them wisely.
3. Policy problem: the compass to choose methods
But how do we know when to use each method? I believe that our chances of informing policy depend highly on our choices at this stage. To decide what methods to use, it is important to look at the bigger picture.
1. Define a policy problem, and be able to describe it in both technical and political terms. This takes some initial effort for us to have a good sense of the context.
2. Identify what is the purpose that research can play in each specific case. Will research be used to find a solution? To set a topic in the agenda? To facilitate a political negotiation? As you can already imagine, not all methods can serve all these purposes.
3. Formulate a better research question, that is more clearly align with the specific problem at hand, and what research can do in that particular case with the clarity of the first two steps.
4. Select the methodology or a combination of methods to use, that make more sense in your context, and with your purpose in mind.
4. A practical example
Let’s imagine a country that has been implementing a Conditional Cash Transfer Program for the last 10 years, and now, there is an interest to reform it because of an economic crisis that has decreased the government’s budget.
In the first step you identify what is the problem, although there is political support for a reform to the CCT program, different actors have very different positions on the issue. You notice many times they are referencing research, but the evidence is much dispersed and in some cases even contradictory!
In the second stage, you consider that you can get involved by making sense of the existing evidence. You have identified more than 10 studies on the CCT’s in your country alone – no wonder everyone was referring to such divergent results! Your objective becomes to give more clarity on the existing evidence.
In the third stage, you plan to answer the following questions: what does all these evidence mean at the end of the day? Is it quality evidence? How can we make sense of contradicting results?
Finally, and only here, do you identify your method, and choose to do a meta-analysis of the existing studies on the CCT program, and to place this analysis within the broader evidence of CCTs in other countries.
As you can see, making a choice about a method, in the context of informing policy, is much more complex than saying quantitative or qualitative methods.
5. Learning More
If you are interested in learning more about frameworks and tools for policy relevant research, you are invited to the Short Course on doing policy relevant research that will be carried out in September, 2016. You can also read a variety of blogs on this issues at Politics and Ideas.
Andrea Ordonez
Associate of Politics & Ideas
There is a persistent perception that there is a tension between rigorous research and research being relevant, thus increasing the chance of having an impact. Of course, I understand the origins of this tension. For example, some rigorous methods require more time than those given by policy windows, a reconciliation that is not always easy.
Nonetheless, I believe that there has to be ways by which a research can make better methodological choices that can be rigorous and also coherent with the policy context. This is why in Politics and Ideas, we are developing a set of tools, and capacity building strategies that aim at helping researchers to navigate the methodological choices with a compass set on policy priorities. Here, I will share some ideas of navigating methodological choices that researchers can keep in mind for their current and future projects.
1. The distinction between qualitative and quantitative research is not very informative
When researchers describe the methodological choices, they usually state whether they will use quantitative, qualitative or mixed-methods. These categories have become popular, both are not very helpful to understand exactly what is being done, and how this will answer as a research question. Furthermore, this can scale up into a competition, between those in the “qualitative camp” and those in the “quantitative camp”, a discussion that is not very fruitful, as each method has its strengths for certain types of questions, and its limitations for others. When considering what the best method is, try to think beyond these categories.
2. Research methods as your own Swiss knife
One way of thinking about methods, is to imagine that they are a Swiss knife: a collection of tools, each one with a particular strength. For example, you wouldn’t use a knife to open a bottle of wine when you have a Corkscrew among your tools. The same should be true for methods: researchers focused on informing policy, should develop a variety of methodological skills to choose from, depending on the specific need of each occasion. Given that you have these tools at hand, the key next step is to choose among them wisely.
3. Policy problem: the compass to choose methods
But how do we know when to use each method? I believe that our chances of informing policy depend highly on our choices at this stage. To decide what methods to use, it is important to look at the bigger picture.
1. Define a policy problem, and be able to describe it in both technical and political terms. This takes some initial effort for us to have a good sense of the context.
2. Identify what is the purpose that research can play in each specific case. Will research be used to find a solution? To set a topic in the agenda? To facilitate a political negotiation? As you can already imagine, not all methods can serve all these purposes.
3. Formulate a better research question, that is more clearly align with the specific problem at hand, and what research can do in that particular case with the clarity of the first two steps.
4. Select the methodology or a combination of methods to use, that make more sense in your context, and with your purpose in mind.
Let’s imagine a country that has been implementing a Conditional Cash Transfer Program for the last 10 years, and now, there is an interest to reform it because of an economic crisis that has decreased the government’s budget.
In the first step you identify what is the problem, although there is political support for a reform to the CCT program, different actors have very different positions on the issue. You notice many times they are referencing research, but the evidence is much dispersed and in some cases even contradictory!
In the second stage, you consider that you can get involved by making sense of the existing evidence. You have identified more than 10 studies on the CCT’s in your country alone – no wonder everyone was referring to such divergent results! Your objective becomes to give more clarity on the existing evidence.
In the third stage, you plan to answer the following questions: what does all these evidence mean at the end of the day? Is it quality evidence? How can we make sense of contradicting results?
Finally, and only here, do you identify your method, and choose to do a meta-analysis of the existing studies on the CCT program, and to place this analysis within the broader evidence of CCTs in other countries.
As you can see, making a choice about a method, in the context of informing policy, is much more complex than saying quantitative or qualitative methods.
5. Learning More
If you are interested in learning more about frameworks and tools for policy relevant research, you are invited to the Short Course on doing policy relevant research that will be carried out in September, 2016. You can also read a variety of blogs on this issues at Politics and Ideas.
No comments:
Post a Comment