Interview - Samar Verma

Dr. Samar Verma

Senior Program Officer

IDRC, New Delhi

Q: What is your understanding of policy impact?

Informed by numerous ways in which policy research institutions and multiple other stakeholders define policy ‘impact’, and learning from our own need to better understand this term, at the Think Tank Initiative (TTI) programme, we know that there are multiple dimensions to this term. Some institutions refer to changes in public policy documents as ‘impact’, while some others prefer to use it to changes in public discourse, or terms of debate among the society at large. Some others believe that limiting impact to changes only at public policy level is restrictive and should be extended to understanding how the changes in public policies are eventually implemented on the ground. Relatively more academic-minded Think Tanks also often include international credible publications as impact. While all of these dimensions are useful and important to capture, we often like to distinguish among policy access, influence and use, and understand the policy impact more as in the ‘use’ of the knowledge generated. The ‘use’ could be by multiple stakeholders, principally the government, but not limited to them. Access and influence, including publications, are seen more as ways to achieve ‘use’ and hence impact.

Q: What are the innovative examples you have observed on institutions making policy impact?

It is truly amazing to see the number of interesting ways in which institutions continue to innovate for policy success, often in very challenging political context, and sometimes even in situations of conflict. Innovations span from diversity in business models (driven by sustainability strategies) and governance structures to institutional systems of recruitment and retention of key personnel (which is by far the most important challenge that think tanks continue to face) and communication & policy engagement strategies. We also have seen institutions collaborating to innovate (where for instance, partner’s strengths are complimentary), which is one of the special features that the TTI encourages among its member institutions. However, the one key factor that drives this all, and remains central to their success, is the dynamism of institutional leadership actively supported by the governing board.

Q: What are the latent opportunities you see on how institutions could make policy impact?

The institutions that we at the TTI work with are a very diverse cohort. Many have been in existence, playing a key role in national government policies for a long time, while several others have emerged and grown rapidly in recent decades. There is a great diversity in size, research themes that they work on, and of course their institutional drivers and leadership. However, in my personal view, I have always felt how institutions could perhaps leverage their strengths significantly more with greater focus on at least two areas, viz., more engaged governing boards and improved institutional communication and outreach strategy.

Q: Do you have any thoughts on the nature of environment that makes it conducive for institutions/organisations to effect policy impact?


One clear dimension of the ecosystem is effective demand for good evidence from policy making community and the society at large. That not only spurs generation of more robust evidence, but also creates a more informed and mature society characterised by improved governance and accountability that are bedrocks of democratic politics. In TTI, we have made a modest effort to begin understanding the demand side of public policy making through the Policy Community Surveys (PCS). While these surveys are not designed as statistically valid analysis, they do throw up interesting insights into perceptions of senior national policy making community on quality of public policy making in their countries, their preferences for research-based inputs, and their opinion on the role of multiple supplier types of evidence, including Think Tanks and universities. Conducted at regular intervals, we also expect them to show some temporal trends that might become visible over a period of time. The completed surveys are available on www.thinktankinitiative.org. .

Interview - Anil Jain

Anil Kumar Jain
Adviser (Energy, Climate Change, and Overseas Engagements), NITI Aayog 


What is your understanding of policy impact?

Policy impact, in my view, is an observable change in certain pre-identified parameters in a particular sector of a country. Whether or not this change is observable depends on a variety of metrics: the nature of parameters, their definitions, the sample space, and the time-frame of a particular intervention.

Gauging the impact of a policy/intervention is an important step in the cycle that starts from policy formulation, goes on to policy appraisal, and finally culminates in the implementation of the same. Measuring the impact that an intervention is making is particularly useful in mid-course correction of any intervention, for which a large amount of public expenditure is mobilised. It is also important for formulating follow up strategies for tracking specific components of a problem that are found to not respond fully to the interventions being carried out.

 According to you what do you think could be the parameters?

The first and foremost metric to gauge policy impact is the construction of a baseline to monitor against progress.. The next is to define objectives in terms of set deliverables, to facilitate ease in monitoring. Focusing on an apt evaluation by constructing a sample size correctly, including techniques that take into account the multitude of factors - geographic, socio-economic, political, that influence the implementation and adopting interventions, are also factors that help understand the impact that a particular policy is making.

What are the innovative examples you have observed on institutions making policy impact?

I would like to cite an example from the energy efficiency space in India. Energy efficiency in India is a concept that is restricted by a lot of market barriers, the major one being information asymmetry.

An innovative example in the space of energy efficiency is a public, real time, national dashboard for monitoring the adoption and impact of the ‘Domestic Efficient Lighting Programme’ of the Government of India (http://www.delp.in). This new dashboard, accessible to anyone with an internet connection, aims to tackle the barrier of information asymmetry by making users aware of the impact that the adoption of efficient lighting systems are having on parameters like the energy saved per day, the cost saved per day, the peak demand that was avoided and the carbon dioxide reduction per day. Such an application, which measures policy impact in a transparent fashion, helps make the users aware of the benefits of the intervention, which increases adoption, and in turn maximises the impact of the intervention.

What are the latent opportunities you see on how institutions could make policy impact?

In my opinion, institutions could contribute towards making policy impact by utilising their strengths of being repositories of evidence based sector expertise.

I would like to illustrate this with an example of a project that we are carrying out at NITI Aayog. With the new developmental goals of the Government in the space of renewable energy, NITI Aayog, in collaboration with a leading Think Tank in India and an international research agency, is facilitating the development of a tool for geospatial analysis for Wind and Solar Energy. This analysis, which includes data across states, upto taluka levels, would help investors identify the different areas in India available for installation, and their inherent potential for generation.

This is an excellent example of an institution utilising its sector specific expertise to aid the Government by providing a tool that would help in the achievement of a country’s developmental goals and also maximising policy impact.


Do you have any thoughts on the nature of environment that makes it conducive for institutions/organisations to effect policy impact?

A strong environment of collaboration between the policy makers, and the institutions/organisations is important for positively affecting policy impact. Different institutions have expertise in different dimensions like sector specific research, advisory, and advocacy. Bringing in these institutions and their expertise into the policy formulation process, inculcates a strong evidence based backing to any new policy/ intervention.

For instance, we at NITI Aayog, developed one of the Government of India’s first, open source, interactive, dynamic, scenario building tools to encourage dialogue, and consensus building on energy policy. In this exercise, we roped in a multitude of Think Tanks, non-governmental organisations, international research organisations, industry bodies, and the academia. Each institution brought in its own expertise to the table, which enabled us to develop a comprehensive product for the nation. With an environment of collaboration and intellectual exchange, different institutions were able to contribute to the energy story of India and in turn play a part in the formulation and strengthen the energy policy in India.

Interview - Andrew Hurst

Andrew Hurst
Program Leader, TTI, IDRC Canada


Q: What is your understanding of policy impact?

I understand policy impact quite broadly. In our work, we tend to assume policy means public policy, which delimits a certain domain of responsibility belonging to governments on behalf of their citizens. In this sense, impact could run the gamut from influencing how a policy issue is framed, to outlining policy choices open to government, to implementing a policy or set of policies, to helping assess the effectiveness of that policy in achieving its intended outcomes. However, I also think of impact on other social actors whose decisions and behaviours have bearing - deliberately or not - on social, political, economic and environmental issues. Here I am thinking of private firms, civil society organisations or even individuals. While there may be no recognisable “private policy”, a researcher could nevertheless consider whether their research has had any impact on the ends that a public policy would have. So for instance, working to change how companies accommodate workers’ rights in global value chains, or seeking to influence the attitudes and behaviours of men in relation to violence against women. This last example may not at first seem like “policy impact”. But I would argue that it should, because even in private spheres (like households), there are issues of public concern (like women’s rights). In the end, I try to think about how research can help constructively address socio-economic, political or environmental issues.

Q: According to the donor’s perspective what do you think could be the parameters?

It very much depends on the donor. Some donors, particularly the bilateral donors, are more concerned with downstream impacts. This is both because they are interested in change and how to bring it about (although aren’t we all?) but also because there is immense pressure on them to demonstrate value for the taxpayers monies they are spending. This has led, in my opinion, to both a narrowing of what is considered impact, while at the same time increasing expectations of research and its ability to contribute to that impact. Other donors whose accountabilities are different (such as large private foundations) can afford to take a broader view of impact. But again, this varies between them.

Q: What are the innovative examples you have observed on institutions making policy impact?

There are so many I am always reluctant to highlight one or two. Our website is filled with Stories of Influence that show how the organisations TTI supports – including CSTEP -  are making a difference and contributing to positive social change. Instead of particular examples, I will mention one innovative approach, and that is research which takes into account questions of research integrity and legitimacy. This normally means seeking the active involvement of people with a stake in the research process. IDRC considers approaches that incorporate these considerations as essential to the production of quality research. And while these approaches to research may not be innovative in the sense of being new approaches conceptually, they are too often not undertaken, and yet they can have significant bearing on the likelihood that the research findings will have impact.

Q: What are the latent opportunities you see on how institutions could make policy impact? 

New digital technologies have really blown open the traditional, formal policy making process. They facilitate access to ideas, accelerate expectations of citizens and consumers and facilitate responsiveness to those expectations, change the nature of production and consumption, and change social relations in many ways. I think within these changing circumstances there are interesting opportunities for influence to be explored. For instance, using these technologies for communications and outreach (this is already happening but way more could be done) using them in the research process (e.g. crowdsourcing data collection), or even taking some of these changes up as policy-relevant research topics (e.g. how to bring internet access to the more than 60% of households that, according to the International Telecommunications Union, do not currently have access, and more importantly, what could such access facilitate?).

Q: Do you have any thoughts on the nature of environment that makes it conducive for institutions/organisations to effect policy impact? 

Clearly, context matters  (: http://www.thinktankinitiative.org/news/context-study-linking-think-tank-performance-decisions-and-context)  but as has been noted elsewhere, this is a rather uncontroversial and unhelpful statement (http://onthinktanks.org/2015/04/20/context-matters-so-what/).  And how you answer this question depends very much on your organisation, where you sit within it, and what you as an individual understand context to mean. But let me offer two thoughts on your question from my previous experience as a policymaker.

The first thought is how crucial it is to acknowledge the social nature of context and the importance of personal relationships within this. Organisations operate in a social environment, by which I mean ones that are constituted by social relations of all kinds that are governed by a whole range of overlapping social norms and values. Developing cordial and professional relationships on a personal level with those individuals that organisations seek to influence is an essential element in building trust, which plays a central role in the effectiveness of communication. Quality and timely research is important, but I believe having a foundation of trust with the intended audience helps accelerate the process of influence. There are risks obviously – cooptation being the most dangerous – but are manageable. Moreover, because the social environments organisations operate within are not fixed, what organisations do, and how they do what they do, can actually help change existing social relations for the better – or even help constitute new ones. These outcomes will be complementary to whatever impact they seek through their research outputs but I would argue the power that comes from the way an organisation is socially embedded should very much be borne in mind when conceiving organisational purpose and approach. Establishing and nurturing diverse, meaningful personal networks is part and parcel of this process.

My second thought on context is the opportunity that arises from “disruptive moments.” More often than not, this would be an unanticipated event that creates an opening for good, evidence-based ideas to gain traction. The collapse of the Rana Plaza Complex in Dhaka in 2013 and the aftermath is one example. The Centre for Policy Dialogue, through its “Post-Rana Plaza Collapse Civil Society Initiative”, not only galvanized action to sustain civil society monitoring of the promises for support for the victims, it was also able to use the moment to have a broader conversation with government and private sector actors on finding better ways of ensuring workplace safety and compliance with workers’ rights in the apparels sector of Bangladesh.

These thoughts are likely very familiar to those who manage Think Tanks, so I don’t feel like this is anything new but to the extent that they could be generalised, it might be as follows: be proactive in shaping the elements of your environment that you can control, and be ready to take advantage when the influences you cannot control create openings for you.

Interview - Carlos Gustavo Machiado

Carlos Gustavo Machicado
Executive Director,  INESAD



Q: What is your understanding of policy impact?

Policy impact means that the research that a Think Tank performs guides some of the policy decisions that the national or local government take. If a Think Tank is specialised in economic issues, of course the impact has to be in the decisions that the government takes in the economic area. An illustrative way of thinking about policy impact is that the outputs of a Think Tank are used as inputs for the government's decisions or actions.

Q: As a Head of a Think Tank, what do you think could be the parameters?

The most difficult thing is to measure impact because the parameters can certainly be very different according to the policy. I can think of the following parameters:

1. The researcher or researchers who have conducted a research are invited by a public authority to present the results and policy implications of the paper or working paper.
2. The conclusions or recommendations of the paper or working paper have been used as arguments for policy decision.
3. It is possible to identify the same words used in the paper or working paper in the official publications (from the government).
4. After concluding a research project, the researcher(s) are invited by the government as a consultant or are invited to improve the research according to the government's needs. It can also be that the government funds the new or the extension of a research.

Q: What are the innovative examples you have observed in institutions making policy impact?

At INESAD, we are using games to guide some policy decisions. We cannot affirm yet that these games are impacting the government's decisions, but at least it seems that they are explaining in an easy way, how people interact in different situations. The games that we are implementing try to explain how people interact in labour markets, but games can be used in all other type of markets and areas.

Q: What are the latent opportunities you see in how institutions could make policy impact?

For Think Tanks that are specialised in economic issues, as it is in the case of INESAD, it is clear that governments would be open for suggestions in times where the economic situation of the country is turning complicated and in times where policy decisions will be associated with important costs for the society. It is clear that this is the situation or will be the situation for most Latin American countries in the next years. Governments of Latin American countries will face several trade-offs from which they will have to choose. So, I think there will be many opportunities for Think Tanks in the next few years.

Q: Do you have any thoughts on the nature of environment that makes it conducive for institutions/organisations to effect policy impact?

Certainly, it is important that governments are open to the academy and to institutions like Think Tanks that provide evidence-based researches on data and models. Unfortunately, this is not the case in some countries (for example Bolivia), so it is very difficult to effect policy impact. In addition, the public servants that take the policy decisions are not well-educated, so they just don´t believe in research as a guide for taking decisions, and therefore take decisions based on their intuition, and that is really a problem for Think Tanks that have as their main objective to provide suggestions that impact policy decisions.

Interview - Valerie Traore

Valerie Traore
Executive Director, NIYEL



Q: What is your understanding of policy impact?

It depends if we are talking about the impact of the policy itself, or the process of impacting policy making. Assuming we are referring to the latter, there are two levels of policy impact. The first is being able to influence the process of policy making at different stages, from agenda setting, to policy formulation, adoption or review. The second, often more challenging, is ensuring that existing policies are actually implemented.

Q: According to you, what do you think could be the parameters to measure policy impact?

Measuring policy impact begins with understanding the baseline. Do policies exist? If yes, are they adequate, inclusive, and realistic and do they uphold human rights? If policies do not exist, what are the barriers and at what levels should they be?

The second element of measuring policy impact, is to understand where should we seek impact of the policies we are pushing for. Is it at the family level that the policy should have an impact (such as with domestic violence or access to education), at the community level, the district or national level, regional (such as in trade and migration) or global level (as in climate change or illicit financial flow).

The third element to consider is the dimension of change that the policy is going to affect. An education policy for example has implications on the economy, social relations, food security, governance, just to name a few dimensions that are relevant at each level of impact.

Finally, assessing policy impact is also looking at the actors with the mandate to effect the change we are looking for. Being able to assess the small shifts in the balance of power, such as the language a minister uses in favour of a certain policy, the championing by an MP, or the increased participation of community members in discussions on the issue allows for monitoring and evaluation of policy impact.

Once we start looking at these three parameters of policy impact and map out indicators based on these, measuring policy impact becomes a lot more concrete.

Q: Based on your interactions with various Think Tanks, what are the innovative examples you have observed in institutions making policy impact?

There are a range of approaches that Think Tanks use and how innovative they are depends on the context and what has been used before within that context. Reaching out to religious leaders with research findings and bringing them on board as advocates of an issue is one approach. Another widely used tool is scorecards, which tend to play on the competitive nature of districts or even states and motivates action.

Q: In your opinion, does the nature of policy impact change from organisation to organisation/region to region?

I don’t think the nature of impact changes from organisation to organisation or region to region. I think the process by which one impacts policy varies greatly. Every policy process aims at having a specific impact and that is where we should focus. The process of influencing policy in order to have impact does vary depending on the specific nature and mandate of the Think Tank, the issue, the target, the socio-political context, and the timing just to cite a few factors.

Q: In the world of Think Tanks, why is effective communication important to policy impact?

In the world of Think Tanks, effective communication is not just important, it is crucial to their effectiveness. Most Think Tanks exist to provide research and solutions to development challenges. The recommendations they offer are meant to serve policy making or practice changes, but without those recommendations reaching the desired targets through communication, they are obsolete.

Interview - S.V. Ranganath

Public Policy: An Indian Perspective

S.V. Ranganath, IAS (Retd.)
Former Chief Secretary, Government of Karnataka and Board Member, CSTEP


What is Public Policy?

Policy making may be defined as “the process by which governments translate their vision into programmes and actions that deliver outcomes (desired changes) in the real world.”  It is presumed that governments act as a perfectly rational individual who always chooses the best options available in framing any policy. However, this is not so in the real world.  In practice not all the options available are analysed or explored.  An incremental approach is adopted in framing policy changes. According to Herbert Simon, everyone is governed by “bounded rationality” which implies that none can gather all information and process it perfectly to arrive at a rational decision.

How is Public Policy formulated?

In India there is a national or federal government and also governments at the state level and the distribution of powers between the federal government and the states is clearly enunciated in the constitution.  Both at the federal and the state levels, public policy is laid down by the political government i.e. the parliament or the legislature as the case may be. The political arm of the government consists of Prime Minister/Chief Minister and Members of the Council of Ministers. The laws passed by the Parliament or the State Legislature provide the frame work for detailed policy making by the executive. In practice, it is the job of the Secretary to Government to initiate policy making and to explore the various options available and the final decision is taken by the Minister or the Cabinet.  Each department of government is responsible for formulating the sector policy but it cannot take an independent decision.  The existing Rules of Business require a consultative process to be followed before a final decision is evolved.  Matters having financial implication need the finance department’s concurrence; legal issues needs the opinion and approval of law department. The formulation of any major policy involves eliciting the views of various departments connected with the implementation of that particular policy. Thereafter, the matter is sent to the Cabinet where different positions are reconciled and the policy is approved with such modifications as deemed necessary.

Although subjects within the domain of the central and state governments are clearly specified in the constitution, the central government formulates policies on subjects in the concurrent list.  The National Health Policy and the National Housing policy are two examples of this phenomenon. Several important central sector schemes where large amount of money are allocated under schemes like MNREGA, JnNurm and National Rural Health Mission, are implemented by State Governments. Therefore even though the policies of these sectors are framed by the Central government, the implementation depends upon the involvement and dedication of the state machinery. This has a powerful impact on the success of such policies.

There are several other stake holders who are involved both directly and indirectly before a certain policy emerges.  Interested groups, lobbies or individuals exert influence on the policy makers both at the political and bureaucratic level and their views play a part in shaping policy.  Such pressure group could include farmers’ organisations, industries and trade associations, teacher associations and certain sections of the civil society.

The judiciary has been playing an important role in policy making. A large number of legislations need interpretation by the courts.  The courts have also intervened in actual implementation of the policies and issued directions to the executive government.  The recent examples of judicial intervention in policy implementation pertain to regulating mining activities in Karnataka and distribution of 2G spectrum.  In the 2G cases, the Supreme Court observed “under the Constitution, judicial review is one of its basic features and in exercise of such judicial review, the court can certainly scrutinize and even strike down policy decisions of the executive when such decisions are unconstitutional.”  In another case, the Supreme Court noted: “Judges must know their limits and must not try to run the government”.  Judicial intervention in policy making is not peculiar to India and is common to all democratic countries where the courts have sought to intervene where public interest is involved.

What are the challenges faced by policy makers?

Most Government policies and programmes which appear well conceived, logical and appropriate fall far short of expectation of policy makers at the implementation level.  The reasons for the sub-par implementation of government policies and programmes can be categorised into four broad categories.

Poor inter-departmental coordination:  In Government, the Rules of Business Allocation specifies the allocation of various subjects to the relevant government departments.  These allocations are based on the government’s perception of the management needs, and generally work very well.  However, most real life issues involve problems that cut across the jurisdiction of many departments and herein lies the rub.  For example, a programme on nutrition will involve four departments of government viz. Department of Health and Family Welfare, Department of Women and Child Development, Department of Education (Mid-Day Meals) and Department of Food.  Most of the subjects that touch the life of a common man would fall within the purview of three or four departments of government.  Sadly, the levels of coordination among the departments are poor; most departments assiduously guard their turf, without giving adequate thought to the needs of quality programme implementation.  This is the fundamental reason why many government programmes fail at the implementation level.  Therefore, the lacuna of inadequate attention to issues of departmental coordination and dedicated “turf guarding” by the relevant department needs to be addressed without further ado.

Insufficient attention given to capacity building: The responsibility for implementation of the policies and programmes pertaining to welfare of the common man rests almost entirely with the State Governments.  Efficient time-lines, initiatives, innovation, dedication, integrity and result orientation are the essential features of the management structure of any competent implementation unit.  In addition, the ability to take calculated risks, to instill team spirit and ensure that the members of the team give their best are the characteristics of any successful project manager.  All public servants involved in implementing public policy need to have these qualities in ample measure.

However, an additional dimension of openness, transparency, addiction to the principles of natural justice and above all compassion are essential ingredients of a successful public administrator. It addition, it is necessary to create an environment where public servants are encouraged to take risks.  The present environment, not only does not reward risk takers, but also seeks to penalise them.  Not all the member of teams involved in implementing policies and programmes have got all these qualities.  This is one of the important reasons for poor implementation of well conceived policies. It is necessary that the capacity building of those involved in implementation of public policy is taken up in right earnest.

Archaic systems and procedures: Most of the systems and procedures prevalent in public administration in colonial India have been continued from the time of independence. The intention of the colonial government was to ensure that public administration was an instrument of control and command.  Today, the priorities have undergone a sea of changes and the intent of the government is to connect with the people.  Most of the procedures which are in vogue today are not citizen friendly; every citizen has to repeatedly visit the government offices to get what is their legitimate right.  These archaic procedures lead to delay, frustration and is often accompanied by whispers of corruption.  There is an immediate need to review and radically modify the existing procedures to make it more transparent and citizen friendly.  It is also necessary to make use of technology in a big way to make the existing procedures more open and citizen friendly.

Judicial intervention: Most policies involve capturing the intention of the Government into an appropriate legislation.  A poorly drafted legislation leads to giving discretion to the implementing authority apart from endless litigation in courts of law.  Since courts have been quite active in intervention of cases where there is a violation of provisions of the constitution and other relevant laws, the framing of Acts and Rules by the legislature as well as by the bureaucracy needs much closer attention than what it is receiving today.

How can Think Tanks help policy making?

Most of the policy makers in the government are rushed for time. A policy maker needs inputs, which are multi-disciplinary in character.  Apart  from in depth domain knowledge, a policy maker needs to have more than a fleeting acquaintance with public administration, modern management, economic theory, behavioural economics, law, political science, psychology and sociology apart from the technological options available.  A Think tank is best suited to give these multi-disciplinary inputs, which the policy makers are craving for. At present this expertise does not exist in any department of government and this is one of the main areas where government interaction with Think Tanks will contribute decisively to improved policy making.

As mentioned earlier, policy makers, normally, do not consider all the options available while framing policy, thanks to the limitation of “bounded rationality”. A Think Tank can give more options to the policy maker to explore and this should assist the policy maker considerably.  Most of the welfare programmes could be implemented much better if an evaluation is done by a competent Think Tank who could identify the bottle necks and road blocks and suggest appropriate modifications. As of now, evaluation of programmes depends on data collected through questionnaires.  It would help if a Think Tank is asked to evaluate major policies and programmes of government through “action research”.

Tool Review

Review of Monitoring and Evaluation of Policy Influence and Advocacy
Authors: Josephine Tsui, Simon Hearn, and John Young

ODI Working Paper 395  http://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/8928.pdf 
This working paper commissioned by the Bill and Melinda Gates Foundation is a useful resource for organisations trying to influence public policy and for agencies which fund them.  Policy influence is a complex and ambiguous area, making M&E difficult. Yet monitoring and evaluating the impact of such advocacy interventions is essential from the perspective of learning as well as to establish accountability.
This thoughtful, exhaustive, and well-researched and referenced paper could guide target organisations towards better monitoring and assessment of their advocacy interventions.   It provides a comprehensive view of important issues in M&E of advocacy interventions, and of existing methods and approaches (referred to as frameworks) as well as conceptual frameworks to understand and improve the M & E process. It is well organised, with five chapters apart from the Introduction and Appendices.

The introduction is written in an engaging, almost conversational style, as the authors touch upon what M & E is, why it is required, different ways to describe the outcomes of an advocacy intervention, the importance of analysing causality and of synthesising the results of different kinds of analysis to assess overall success.  Do not skip this section.  It provides a clear perspective on a rather heavy subject, which is useful in understanding some of the more complicated and summarized content later.

Chapter 1 reviews current trends in the field, discussing how effective policy influence interventions can take different forms and have diverse outcomes, making it harder to measure their impact. Hence the need for multiple approaches and methods, presented in Chapters 2 & 3.
Chapter 2 presents frameworks to understand how policy change happens, so that interventions are planned better and enabling evaluation of an intervention at the planning stage itself.   The chapter presents three sets of frameworks to understand how policy change happens, frameworks to guide intervention planning and to gauge the level of influence that interventions have on policy change.
Chapter 3 presents four sets of frameworks for monitoring and evaluating advocacy interventions, representing four dimensions of an advocacy programme.

• Strategy and direction
• Management and outputs
• Outcomes and impact (the authors explain the difference between ‘outputs’ and ‘outcomes’)
• Understanding causes of the outcomes observed.

In Chapters 2 & 3 discussion on each set of frameworks opens with a handy box briefly explaining what these frameworks are and why and then they are useful.  Each set covers a fairly wide spectrum of methods and approaches.  For instance, the set relating to management and outputs (pages 11 and 12) has tools ranging from behavioral descriptions of  multiple levels of success (Scoring Rubrics), to one  assessing  specific capabilities of the organization (Coalition Capacity Checklist),  to one which tabulates and weights multiple criteria to arrive at effectiveness scores (Multi-criteria Decision Analysis).  While the chapters themselves contain summarized information presented in a tabular format, the appendices contain details of each framework with references, opening with a brief explanation of why and when the framework is useful.

Six case studies in Chapter 4 illustrate the application of one of the frameworks.  Some of the cases are well written and clearly illustrate the method; others (e.g. DFID page 17) are confusing, though clarity is easily obtained on consulting the attributed source.

This paper is a great resource for organisations trying to influence public policy, but it requires the reader to be comfortable with abstraction and concepts.  In organisations where the advocacy team has difficulty dealing with these, leaders could step in or organise workshops around the theme of M&E using the contents of this paper.


Reviewed by: Neeta Krishna
Associate Professor - HR 
Father C Rodrigues Institute of Management Mumbai