Aditi Bulletin Issue 5

Note from Managing Editor
The current issue focuses on Research Methodology. Several Think Tanks as a policy research organisation conduct different types of researches, some follow particular methodologies, some develop methodologies based on the research topic, some use random approaches.  While there is no defined research methodology/ies to use for a topic or theme, researchers adopt methods to suit their research. In order to arrive at the desired result/s researchers opt for research methodologies to ensure that the research is gainful and meaningful. As always, we received several contributions from a varied list of people and organisation. I would like to thank all the members of the Editorial Board to provide their continued support.

Regards
Managing Editor, Aditi

Editorial 

Will Paxton
Director of Kivu International 



Guy Lodge
Director of Kivu International




A Think Tank’s reputation will rise and fall depending on the quality of its research. Producing a shoddy piece of work that unravels in the face of external scrutiny can take a Think Tank a long time to regain its credibility.

If research is to be strong enough to withstand the full force of interrogation from policy-makers, the media and others, then self-evidently it is important to use appropriate and robust research methodologies.

But one needs to be clear what we mean by ‘appropriate’ and ‘robust’ and relate these  to the environment in which Think Tanks operate. This means being clear about the two most significant ways in which Think Tank research differs from that undertaken by academic institutions.

Firstly, to be influential Think Tanks have to work within timelines set by policy-makers and politicians. These can be incredibly tight and constrain the type of research methods a Think Tank can adapt. While an academic research project might last 5 years, the typical length of a Think Tank project is often between 6-12 months. Randomised control trials, to pick a voguish academic methodology, are just not an option when you have so little time to play with. (RCTs are also very expensive – and Think Tanks, in our experience, seldom have the resources to pay for them).

Time pressures force Think Tanks to adopt mixed methodologies – very often combining secondary analysis of existing research, with selective primary research (such as a poll or some original quantitative work). A good Think Tank will add considerable value by synthesising and clarifying academic research, and using it to help inform policy advocacy. Balancing a mix of secondary and primary research methods means a Think Tank has to be versatile.

Second, Think Tanks are primarily in the business of influencing policy. This has important implications for research methodologies. Practically it means you have to give more weight to methods that help you solve problems, not analyse them. This means being able to cost the policy options you are proposing – and saying how they will be paid for when resources are scarce. It also means being able to conduct distributional analysis, showing who the “winners” and “losers” will be from the policy change you are pushing. Comparative research is also important, as it can help identify and adapt policies that have been shown to address the problem your own country faces.

Of course Think Tanks vary significantly in terms of their size, resources and staff skills – they’re not all locked into trying to influence short-term policy debates. A good Think Tank will often be juggling a mix of projects using a broad range of methodologies: some responding to the here and now, and others engaged in more long-term thinking that is trying to anticipate the future.

Finally, Think Tank research methods must be judged against standards that reflect the reality of the environment in which they operate. The test should not be methodological purity for the sake of it. Instead, Think Tanks should ask whether their research is policy-relevant? Have the research methods been designed to help you understand – and get over - the barriers to change on an issue? Is it politically savvy - that is, do the recommendations stand a chance of being implemented?

Articles

Research Methods to study on Think Tanks
by Enrique Mendizabal, Founder of On Think Tanks

Research Methods Vs Methodologies
by Shrimoyee Bhattacharya, Senior Research Scientist, CSTEP, Sandhya Sundararagavan, Research Scientist, CSTEP and Debapriya Das, Senior Research Economist, CSTEP

Methodological choices to inform policy 
by Andrea Ordonez, Associate of Politics & Ideas

Lessons from Teaching Research Methodology at IPS
by Nisha Arunatilake, Fellow at Institute Of Policy Studies Of Sri Lanka

Sharing the Whole Research Map
by Bruno Paschoal, Director of OndaPolitica

Interviews

Sabyasachi Kar - Associate Professor Institute of Economic Growth

Sebastian Maslow, Assistant Professor (Political Science) at Tohoku University

Thoughts from  ZHU Xufeng, Professor Ph.D., Tsinghua University, China

Case Study

Using research methodologies to study impact evaluation of a Think Tank in Latin America

by Lorena Alcazar Valdivia, Director of Research at GRADE and Miguel Jaramillo, Executive Director and Senior Researcher at GRADE

Advantage of Pragmatic Research Methods: Case of Smart Cities Study by CSTEP

by Shrimoyee Bhattacharya, Senior Research Scientist, CSTEP

Peri-urban Case Study

Sujatha Byravan Principal Research Scientist, CSTEP

                         

Humour

Concept - Dr. Jai Asundi, Principal Research Scientist, CSTEP

Illustration - Sandeep Khasnavis, Graphic Designer, CSTEP

Cross Posting

Creating innovative research designs: the 10-year Methodological Think Tank case study

by Katerndahl David A, Professor at University of Texas Health Science Center at San Antonio, Family & Community Medicine, San Antonio, TX

                            

Interesting Readings

Involving Children and Young People in Research

Summary: The papers demonstrate that there is now a considerable wealth of experience with participatory research in Australia. Together the papers identify the strengths, the challenges, the complexities —and the enjoyment — of participatory research. The Think Tank provided a unique opportunity for experts from many sectors and from all around Australia to discuss their collective experience and knowledge of participatory research. We hope the compendium is a first step toward developing a collective understanding of how best to involve children and young people in research for their benefit, the benefit of their communities, and for the benefit of research.

URL: http://telethonkids.org.au/media/54379/involvingchildrenandyoungpeopleinresearch_1_.pdf

Webinar on Research Methodologies especially designed for Think Tanks and organised by Atlas Network

This webinar is part of a series by Atlas Leadership Academy, which provides a robust series of credit-based training opportunities that allow participants to sharpen their skills in management, communications, and fundraising while building their free-market organization. This episode deals with conducting policy research within think tanks by Jeff Miron, Director of Economic Studies at the Cato Institute. Jeff provided an introduction to key research methodologies, when to use them, and how to employ research to effect policy change.

URL: http://www.liberties.gr/features/15-liberal-monitor-news/news-international/722-atlas-holds-a-webinar-on-research-methodologies-for-think-tanks.html

Conference on Methodologies for Researching Think Tanks: Case Studies

MARCOS GONZALEZ HERNANDO

This piece is cross posted from On Think Tanks

URL: https://onthinktanks.org/articles/conference-on-methodologies-for-researching-think-tanks-case-studies/

Compiled by: Dr. Annapoorna Ravichander, Head, Communication and Policy Engagement, CSTEP

                              

Article - Enrique Mendizabal

Research Methods to study on Think Tanks

Enrique Mendizabal
Founder of On Think Tanks





Editor’s note: This article has been written by Marcos Gonzalez Hernando and Jordan Tchilingirian with contributions from Enrique Mendizabal and Andrea Ordoñez. It incorporates the views and suggestions of those present at a pre-launch of the First Online Conference on Research Methods to Study Think Tanks as well as online participants.

This article served as a discussion paper for a series of webinars on studying Think Tanks. It is structured in the same way as the pre-launch event and involves a few statements from the organisers and a compilation of the views of the participants. Richard Darlington, Rosie Clayton, Brendan Martin, and Alyaa Ebbiary joined us at the event and their views have been incorporated in the discussion.

The event addressed several pertinent questions and one of the key one was-What methods could be used to study them?

Different methods were explored to employ when studying think tanks. This is the focus of the series of webinars running through September and November. Different methods are useful for different situations and questions. The list is not exhaustive but we hope it will provide a useful source of ideas to Think Tank researchers:

• By type of data
• Qualitative: interviews, ethnography, document analysis (discourse analysis/grounded theory)
• Quantitative: surveys, SNA, content analysis, bibliometrics analysis, social media and big data.
• Mixed methods: including case studies and episode studies.
• Temporal frame: Historical, Snapshot
• Context: National, Multinational, Institutional, Network, Political-wing, Policy area
• Unit of analysis: Institution, Researcher, Policy report, Policy Idea, Tweet, Media presence
• Research Question: Impact, research capacity, networks, institutional structure and change

Interesting References
#TTmethods – Session 1: Case studies, qualitative methods and diachronic perspectives: This session explores an approach to study think-tanks that relies on an array of qualitative methods – including interviews, ethnography, document analysis – to understand these organisations from a closer perspective.
#TTmethods – Session 2: Researching think-tanks with social network analysis: This session will look at how social network analysis (SNA) can be applied to the political world and to public policy research.
#TTmethods – Session 3: Quantitative Survey Analysis: This session explores an approach to study think-tanks that relies on quantitative methods –namely survey analysis– to understand these organisations in a manner that may allow for generalisations.
For more resources and literature visit: On Think Tanks Methods for Researching Think Tanks

Article - Shrimoyee, Sandhya, Debapriya

Research Methods vs Methodologies

Shrimoyee Bhattacharya
Senior Research Scientist, CSTEP



Sandhya Sundararagavan
Research Scientist, CSTEP

Debapriya Das
Senior Research Economist, CSTEP

The science of solving a problem systematically is research methodology. People often confuse methods with methodologies. Methodology is how the research is conducted while methods are means by which a research is conducted. Thus methodologies are generally very unique to a particular research, whereas methods are more established ways with theoretical backing to achieve a methodology. Selection of suitable methodologies and methods are critical while conducting policy research in Think Tanks. Methodologies include milestones or intermediate goals that help answer the larger research question at the end. Methods are selected to perform activities that can help reach intended goals.

For example, if the purpose of a research is to measure the extent of change in topography of a certain geographic area over a given period of time, then the methodology for the same could include producing maps of existing topography. The methods deployed for this goal could include primary data collection and interpretation through surveys or analysis, based on available secondary data from different sources such as available satellite data from open sources. It is here that there could be a blurred understanding of the difference between research methods and techniques. Techniques or tools are finer aspects of a research method. For example, primary data collection through topographic surveys could be done with a variety of techniques such as Areal surveying, Transit-stadia surveying, Plane-table surveying, or sophisticated Geographic Positioning Systems (GPS) methods. In this case, a researcher can select a specific technique primarily based on the quality of output produced and identify if this is adequate in answering the larger research question. Criteria such as time sensitivity, cost, and availability of skilled human resources etc. are also important in this regard.

Research methods are categorised in the following broad categories: 

Quantitative methods help in quantifying or testing a policy problem by using different kinds of data sets. In quantitative methods mathematical or/and statistical analysis is used and is generalised to a larger population within known error margins. Surveys and review of documents and records with numeric information are common quantitative methods prescribed in research.

Qualitative methods are explored to understand the policy problem in terms of opinions, reasons and an underlying motivation. These methods describe the policy problem from the point of view of those who are experiencing it. Comparatively, qualitative methods are more subjective than quantitative methods. Focus group discussions, in depth interviews, review of documents pertaining to a policy problem are few of the common qualitative methods used in research.

Pragmatic/combined methods – as the name suggests is a combination of qualitative and quantitative methods. Once a certain statistical or numerical analysis is completed for a policy problem, the same can be tested via qualitative methods in terms of interviews and focus group discussions. Pragmatic or combined methods help in balancing the limitation of one type of method with the strength of the other.

On the other hand, research methodologies are selected based on type of research such as longitudinal research, empirical research, quantitative or qualitative research, historical research, analytical research, experimental and exploratory research etc.

The choice of research methods from the categories mentioned above are based on the research methodology adopted for a particular type of research that the policy problem concentrates on.
Often there are debates among researchers on the choice between qualitative and quantitative methods. It is important to remember that public policy research has two important dimensions- knowledge and value. The degree of certainty (or lack of it) in each of these determines whether a policy problem is structured, unstructured or moderately structured (Politics and Ideas, “Doing policy relevant research – responding to policy problems”, Module 4, 2015-16). Brainstorming on research methodologies and methods at the primary stage of research can be valuable for a Think Tank in understanding the kind of challenges for a particular research eco system and thus be cognizant of the type of research output to be expected. To summarise, both are important factors in generating favourable conditions for making the desired impact from a research.

Article - Andrea Ordonez

Methodological choices to inform policy 

Andrea Ordonez
Associate of Politics & Ideas





There is a persistent perception that there is a tension between rigorous research and research being relevant, thus increasing the chance of having an impact. Of course, I understand the origins of this tension. For example, some rigorous methods require more time than those given by policy windows, a reconciliation that is not always easy. 

Nonetheless, I believe that there has to be ways by which a research can make better methodological choices that can be rigorous and also coherent with the policy context. This is why in Politics and Ideas, we are developing a set of tools, and capacity building strategies that aim at helping researchers to navigate the methodological choices with a compass set on policy priorities. Here, I will share some ideas of navigating methodological choices that researchers can keep in mind for their current and future projects. 

1. The distinction between qualitative and quantitative research is not very informative
When researchers describe the methodological choices, they usually state whether they will use quantitative, qualitative or mixed-methods. These categories have become popular, both are not very helpful to understand exactly what is being done, and how this will answer as a research question. Furthermore, this can scale up into a competition, between those in the “qualitative camp” and those in the “quantitative camp”, a discussion that is not very fruitful, as each method has its strengths for certain types of questions, and its limitations for others. When considering what the best method is, try to think beyond these categories.

2. Research methods as your own Swiss knife
One way of thinking about methods, is to imagine that they are a Swiss knife: a collection of tools, each one with a particular strength. For example, you wouldn’t use a knife to open a bottle of wine when you have a Corkscrew among your tools. The same should be true for methods: researchers focused on informing policy, should develop a variety of methodological skills to choose from, depending on the specific need of each occasion. Given that you have these tools at hand, the key next step is to choose among them wisely. 

3. Policy problem: the compass to choose methods
But how do we know when to use each method? I believe that our chances of informing policy depend highly on our choices at this stage. To decide what methods to use, it is important to look at the bigger picture. 
1. Define a policy problem, and be able to describe it in both technical and political terms. This takes some initial effort for us to have a good sense of the context.
2. Identify what is the purpose that research can play in each specific case. Will research be used to find a solution? To set a topic in the agenda? To facilitate a political negotiation? As you can already imagine, not all methods can serve all these purposes.  
3. Formulate a better research question, that is more clearly align with the specific problem at hand, and what research can do in that particular case with the clarity of the first two steps. 
4. Select the methodology or a combination of methods to use, that make more sense in your context, and with your purpose in mind. 

4. A practical example
Let’s imagine a country that has been implementing a Conditional Cash Transfer Program for the last 10 years, and now, there is an interest to reform it because of an economic crisis that has decreased the government’s budget. 
In the first step you identify what is the problem, although there is political support for a reform to the CCT program, different actors have very different positions on the issue. You notice many times they are referencing research, but the evidence is much dispersed and in some cases even contradictory!
In the second stage, you consider that you can get involved by making sense of the existing evidence. You have identified more than 10 studies on the CCT’s in your country alone – no wonder everyone was referring to such divergent results! Your objective becomes to give more clarity on the existing evidence. 
In the third stage, you plan to answer the following questions: what does all these evidence mean at the end of the day? Is it quality evidence? How can we make sense of contradicting results?
Finally, and only here, do you identify your method, and choose to do a meta-analysis of the existing studies on the CCT program, and to place this analysis within the broader evidence of CCTs in other countries. 
As you can see, making a choice about a method, in the context of informing policy, is much more complex than saying quantitative or qualitative methods. 

5. Learning More
If you are interested in learning more about frameworks and tools for policy relevant research, you are invited to the Short Course on doing policy relevant research that will be carried out in September, 2016. You can also read a variety of blogs on this issues at Politics and Ideas. 

Article - Nisha Arunatilake

Lessons from Teaching Research Methodology at IPS

Nisha Arunatilake
Fellow at Institute Of Policy Studies Of Sri Lanka




A good cook takes the trouble to source the best ingredients and knows a variety of ways of putting these ingredients together to create signature dishes. Break through research is a bit like cooking. It requires knowledge of available data, knowledge of research techniques and a bit of creativity. At the Institute of Policy Studies of Sri Lanka (IPS) I have been involved in teaching research methods to junior economists over the last five years. In this blog I share with you what I learnt in my attempts to improve methodological rigour amongst our researchers.

Basics are important, both in cooking and in conducting research. One of the first methodological training courses at IPS was to teach distributional analysis. The course started with explaining income concepts and aggregating income information to arrive at disposable income. But, many junior researchers did not have the basic competencies in handling big datasets. The researchers got bogged down by the details of computer programming, although many completed the course, they were unable to fully appreciate distributional analysis and use it effectively to answer research questions.

Learning from this experience, IPS launched a series of basic courses to teach researchers how to handle big data sets, rearrange them and create variables so that when more advance methodological courses are conducted, they are free to focus on the methodological training being taught without being hindered by the details of analysis. Although the second method was more successful in teaching advance methodological concepts, many junior researchers failed to use the techniques taught in their research work. A discussion with the researchers revealed that this is because although they knew the theory of economic techniques they lacked confidence in applying the same.

Practice builds confidence. To overcome this problem, the next phase of the methodological training at IPS concentrated on giving junior researchers practical experience in conducting research. Each researcher was matched with a senior researcher according to their research interest. The junior researchers were asked to then handle projects which used one of the research methods they have learned with the guidance of senior researchers. Although this method was successful in generating a series of good research papers, it did not achieve the greater objective of improving the output of methodologically rigorous research papers.  

Creativity helps one to stand apart. A good cook can make a basic dish, like an omelet, look special. My colleague recently wrote a research paper on social determinants of health outcomes. She used a sophisticated research methodology developed by the World Bank applied on to the Demographic and Health Survey data of Sri Lanka. The paper was methodologically sound, and produced results that were largely relevant for Sri Lanka. However, she had trouble getting it published internationally. The problem was, most international journals have already published many similar papers.

Unlike ten years ago with increased access to the internet and standard data sources researchers are able to conduct good quality policy relevant research very easily. Given this easier access to data and computer, researchers tend to try to fit methodologies to research questions, which results in large numbers of very similar research papers.  But, in order for the research to stand apart and be relevant at a particular juncture, the research needs should start at asking the correct research questions. Once research questions are asked, one must see the availability of data to answer the research question being asked and choose the appropriate research methodology that would best answer the research question being asked given the available data. This requires being creative.

The main task of a policy researcher is to provide recommendations to help improve the economic outcomes of a country. This is done by identifying topical research questions. Research methodologies help researchers to find answers to these questions, given the availability of data. A wide knowledge of research methodologies and available data sources will expand the scope of research, but to conduct ground breaking research that stand apart, one needs to be creative – like a cook who can make the most basic dish taste special!
 

Article - Bruno Paschoal

Sharing the Whole Research Map

Bruno Paschoal 
Director of OndaPolitica




In any social science or policy research project, a lot of data is collected and produced by researchers and their team to include publications, databases, workflows, slides, metadata, logs, interview transcriptions, legislation, among others. These are research objects that will be analysed and findings and reports will be substantiated. However, most of thesis never published or shared with anyone outside the research team.

To use a metaphor, if all these objects formed a research map, what researchers make public is just a path through the map - a paper, a presentation, a talk - but never the entire map. Readers cannot navigate the map on their own and are forced to follow linear paths set by the researcher.
 Prezi presentation link: https://prezi.com/embed/2mlurujmuon2/

In Onda, we believe that sharing the complete research map (or a great part of it) can bring different benefits not only for researchers but for the whole society.  Why? Because by doing this it has the potential to increase the impact of research projects by:

1. Giving more transparency and allowing better replicability: when readers have easy access to objects in which a research analysis was based on (the “raw” data), they can better evaluate the reliability of the information, assess if the analysis and findings are accurate, replicate results and even contact authors for suggestions, correction and improvements for future versions.

2. Broadening a potential audience: many times, one is looking for information about a given topic (the history of a policy, for instance) and finds a paper about it, but with a different approach (a study of its effectiveness). However, in that paper, one sees a superficial (secondary/background) chapter about its history, even citing an interview made or a document collected but not available on the Internet. In that case, if one had access to the research map, one could obtain the relevant (secondary) objects, re-focus on it and still give credit to the author/s by citing the research project from where the data was obtained.

3. Saving time and resources from others: in many projects, collecting and preparing data is the most time and resource consuming step: compiling or transcribing quantitative data from different sources can take much longer than running a regression and analysing its results; conducting an interview with a key-actor usually implies in travelling and opportunity costs that some can’t afford. By sharing the map with others, one can reduce costs and save other’s time by allowing them to re-use and recycle one’s previous efforts.

In Onda, we have been experimenting different ways to open the research map with others: from sharing the research main folder with others using Dropbox or Google Drive to creating multimedia and interactive reports using software like Scalar. All attempts are analysed and obstacles faced, skills required and desirable incentives are identified and shared.

So far we have identified three missing elements that would have potential to change the situation:

1. Missing incentives: stakeholders who could more easily inspire this change – journals and funders – hardly require researchers to share their research objects (especially in the qualitative sciences).

2. Missing tools: although there are a growing number of online data repositories (theoretically the place to share the data), they are nothing more than a storage space and do not allow easy and linked access to data by users.

3. Missing community: currently, there are very few people working on this issue and there is no community using and adapting existing tools, sharing their lessons and knowledge on how to share data (like in an open-source software movement).

To sum up, if we are to see more research maps available online, we need to start conducting experiments and prove to others the benefits of sharing. Nothing better than the power of example to inspire change, start out a movement and a community with enough power to influence the right stakeholders and change the status quo. Let’s be the change we want to see in the world; in Onda, we already gave our initial step. Check out our latest project and send us your feedback.

Interview - Sabyasachi Kar

Sabyasachi Kar
Associate Professor, Institute of Economic Growth

Kunal Sen
Professor of Development Economics & Policy, SEED, University of Manchester, UK





Note: These questions are based on the research for a forthcoming book from Palgrave MacMillan:
The Political Economy of India's Growth Episodes by Sabyasachi Kar (Institute of Economic Growth, India) and Kunal Sen (IDPM, University of Manchester, UK)


Abstract of the Book
For decades following its independence, the Indian economy suffered from poor growth outcomes which famously came to be described as 'the Hindu rate'. Sometime during the late seventies or early eighties, things started to change for the better and by the second half of the 1990s and 2000s, there was a complete turnaround with India joining the small group of countries that were growth success stories. Recently, this narrative of India's emerging growth miracle came to be questioned when growth rates slowed down considerably after 2010. This book attempts to explain these distinct growth episodes in India by going beyond immediate determinants of growth like investment, export, infrastructure etc., and providing a political economy framework relating these episodes to deeper changes in the economy and polity. We argue that the transitions from one growth episode to another can be explained by the bi-directional relationship between growth outcomes and institutional arrangements, and the manner in which institutional arrangements and their transitions are determined by the political bargains struck between the elite groups in Indian society.

Q: How did this research begin and evolve in the minds of the researchers such as yourself Sabyasachi and Kunal Sen? Was Lant Pritchett’s framework/theory a thinking space you had already inhabited before this research began?
Lant's framework helps explain some of the puzzles associated with institutions and growth, particularly in developing countries. Methodologically, this framework can be tested using both cross-country evidence and country-specific case studies. Our previous work and publications were based on cross-country data and we had always planned to follow this up with country case studies, as these provide more detailed understanding of these issues. This is what led us to take up the India study.

Q: Could you tell us about the research methodology that you used for this book? How would you describe the nature of evidence and findings from this study? For the reader this is of great interest as the findings appear to offer a learning framework of practical use and experimentation (especially for policy makers) rather than evidence as the final truth
There are four distinct types of methodology that we have used in this book. The first is a statistical approach that has been used to identify the distinct growth regimes of the Indian economy in the post-independence period.  The second is a set of exploratory key-informant surveys that helped us understand the nature of informal institutions in India. The third is an analytical approach that has used existing data and literature to support our hypotheses. Finally, the fourth methodology used is a set of sector-specific and firm-specific case studies that have provided a deeper understanding of the issues.

Q:  What is the point of departure in this methodology used for this research study from previous methodologies you use? How is this different? 
The issues addressed in this book are multi-dimensional, complex and not easily amenable to quantification. We knew right from the beginning that one standard methodological approach would not throw sufficient light on them. Thus, the main departure - methodologically speaking - was the adoption of several alternative methodologies and tools to make sense of the issues.

Q:  What were the challenges you experienced (in methodology or in other areas such as capturing the attention of different audience etc.) in doing a research such as this one?
One of the most critical challenges has been to get information on activities that are either legally or morally seen to be wrong (for example rent-sharing among the elite groups and crony-capitalism) or are non-transparent by nature ("deals" between political and economic elite).

Q: What do you think would be the impact of this research? 
The cutting-edge literature on economic growth in developing countries has shifted its focus from immediate determinants like investment, exports, policy etc. to deeper factors like economic and political institutions. An analysis of the role of these deeper determinants in driving economic growth in India, has however, remained sparse. This book attempts to correct this shortcoming and will definitely encourage more contributions to this important area of study.     

Interviewed by Susan Koshy
Designer of Learning Experiences, Knowledge Translator and Curator
Innohealth Systems/Blank Canvas

Interview - Sebastian Maslow

Sebastian Maslow
Assistant Professor (Political Science) at Tohoku University




Q: How do you design/assess a methodology?
My research focuses on the role of knowledge networks, in particular Think Tanks in Japan's foreign policy process. At various levels of a policy process, problem definition and agenda setting evolve informally; this is also in the case for Japan, where the close ties between the conservative government coalitions, the bureaucracy and the corporate sector has been a defining feature for most of the country's post-war political history. Accounting for the causal impact of ideas is a complex if not impossible task for political science analysis. Hence, assessing the impact of Think Tanks in this political environment requires a command of qualitative research methods, combining in-depth interviews of Think Tank affiliates. The research design should cover longer time horizons in order to assess the changing institutional conditions under which Think Tanks yield influence. Qualitative research of this sort seeks to trace the evolution of policy ideas (my own research focuses on the recent debate over Japan's national security institutions and collective self-defense as it has evolved under the administration of Prime Minister Shinzo Abe since 2012). A comprehensive command of Japanese language skills is required to take into account the Think Tank publications and media debates to locate Think Tanks in the interaction with government agencies in Japan's complex knowledge networks.

Q: Is your research primarily demand or supply driven?
My research is perhaps both, demand and supply driven. Supply driven means scholarship which derives from a genuine academic problem perception. As for research on Japan's Think Tank community, little comprehensive analysis exists assessing their role in the policy process. Meanwhile, the need for policy expertise and external advise has risen as Japan faces challenges to its economic and social institutions in the form of an on-going economic crisis of slow-growth and rapid demographic change. In this regard, scholarship on knowledge networks in Japan is also demand driven. Finally, the growing literature on Think Tanks and knowledge regimes in political science has yet to turn its attention on Asia.

Q: How important is academic rigour in terms of technical, economic, social, political and environmental dimensions?
Aiming at a high-degree of academic rigour across all dimensions is crucial for scholarly analysis, as only rigorous social science analysis will be granted the necessary trust for it to be of relevance in public discourse. Scholarship, as other sites of public discourse, needs to be accountable and transparent in its conduct; only by upholding the highest standards in social inquiry can academia play an important role in shaping society and holding political power accountable.

Q:  In your opinion, is consultative approach and engagement with diverse stakeholders a feasible model?
Social inquiry needs to distinguish itself from consultative engagement in social debate. Academia is influential because of its critical distance to its research objects and not because of its direct entanglement with it. The latter jeopardises public trust in social inquiry as consultative engagement comes along with the risk of losing objectivity. A clear line should therefore be drawn between social inquiry and policy advisory; scholars on both sides of policy analysis must be critically aware of their role in shaping social debate.

Thoughts - ZHU Xufeng

Thoughts from Xufeng
Professor Ph.D., Tsinghua University, China




A research methodology is chosen based on a project. We first understand the problem statement and then choose an appropriate methodology. In my opinion, there is no particular way or system to learn research methodology. Coming from an academic background we primarily depend on reading papers. I also believe that a good research is question driven not method driven.

Case Study - Lorena & Miguel

Using research methodologies to study impact evaluation of a Think Tank in Latin America

Lorena Alcazar Valdivia
Director of Research at GRADE



Miguel Jaramillo
Executive Director and Senior Researcher at GRADE

The study used qualitative and quantitative information research used in a balanced way. The methodology included:

• Generated a database of information
• Gathered survey information
• Administered a survey
• Conducted a workshop
• Conducted in-depth interviews

For more information on this study and results you can read http://www.3ieimpact.org/media/filer_public/2015/10/01/wp24-impact_evaluation_and_policy.pdf

Case Study - Shrimoyee Bhattacharya

Advantage of Pragmatic Research Methods: Case of Smart Cities Study by CSTEP

Shrimoyee Bhattacharya
Senior Research Scientist, CSTEP

Context

The announcement of Smart Cities Mission by the Government of India in 2014 had led to intense deliberations in expert and public domain. CSTEP’s research focus was on bringing better clarity regarding what is a smart city and what is the level of knowledge available globally to support the development of smart cities, whether and how it can be aligned to India’s larger national development agenda and how such an urban development model can be effectively designed on principles of sustainability.

Policy Problem 

There is a substantial knowledge gap regarding the definition and concept of smart cities which is still emerging along with immature standards and regulatory regime, and lack of evidence for success of smart city models globally. The deliberations in the Indian context also triggered opinionated positions among experts on whether such a technology driven city development model should be adopted by India which could create exclusionary development trends. Thus, there is both uncertainty of knowledge and lack of agreement of values on smart cities which has rendered it as an “unstructured” problem ((Politics and Ideas, “Doing policy relevant research – responding to policy problems”, Module 4, 2015-16).

Methodology used

The larger goal of the study was to assess the extent of knowledge uncertainty regarding the concept of smart cities and thus assess the existing urban development ecosystem in India for its preparedness with respect to the concept of smart cities. The adopted methodology for the study thus included major milestones to address these goals.  The study included extensive review of literature drawn on both theoretical and practical discourses, as well as interaction with experts and relevant stakeholders. Both quantitative and qualitative methods were deployed as that is a pragmatic way to analyse urban issues. Interacting with stakeholders was an important source of understanding subtle realities that significantly enhanced the perception gathered from secondary literature.  

Output and Outcome

The immediate research output were a set of criteria for selection of cities under the government mission, and a reference framework for smart urban development in India based on the principles of sustainable development. The process oriented interventions at the forefront of the research recommendations were developed to enable a robust and accountable decision making system at a city level. Additionally avast amount of literature referred for the purpose of this research also enabled the creation of a "Compendium of Resources" for smart cities worldwide. The larger outcome included enhancement of knowledge to create room for better informed deliberations and identification of key issues for a long-term phase that could significantly impact success of the mission.

Learning

The problem was generally viewed as more of a technological issue by many stakeholders in the initial stage of the research. This perception changed as researchers started gaining insights into the types of problems associated with smart cities. While this led to changes in the scope of the research at a later stage, adopting a pragmatic research method helped in incorporating such changes in research design. As a result, the research was able to identify ‘city governance’ as the key area seeking attention and technology as the enabler in the Smart Cities Mission. 

Read more about CSTEP’s smart cities work at: 

http://www.cstep.in/uploads/default/files/publications/stuff/6b9cb2a72ca68b2c2ab3e98ba4c5e649.pdf

http://www.cstep.in/uploads/default/files/publications/stuff/320f73a9fd318538a3d258e8dfe180e4.pdf

Case Study - Sujatha Byravan

Peri-urban Case Study

Sujatha Byravan
Principal Research Scientist, CSTEP




CSTEP is carrying out a research project that examines dynamic changes taking place in peri-urban areas in India. The two geographical areas include, Doddaballapur and Sriperambudur, which lie outside of the cities of Bangalore and Chennai respectively. Such peri-urban areas are transition zones between urbanised pockets and agricultural hinterland.  Changes in these zones are triggered by various conditions depending on the social, economic and political context. In addition, global environmental changes including climate change are also transforming these areas.

The research project is adopting a number of approaches/methods that can help gain an understanding of Doddaballapur. For example, the Geographical Information System (GIS) is used to learn about land use changes over time, analysing census data to learn about socioeconomic changes, carrying out household surveys to learn about water, sanitation, multidimensional poverty, women and their empowerment. These data and tools are being complemented by participatory appraisal methods such as focus group discussions in a couple of gram panchayats. Since climate variability and change will mainly affect temperature and rainfall, governance in this peri-urban area with regard to water and land are also being studied by CSTEP.


 After completing these analyses, we will carry out an exercise called participatory scenario development. In such a forum, some members of the community, decision makers, local leaders and experts will consider the research findings, place them in the context of the kind of future the community seeks, their challenges, and then back cast to decide what they should be doing today and in the near term to reach their goals. For instance, what kinds of policies would be needed to ensure that everyone has access to drinking water? What kinds of actions are needed? At the end of this research, we hope to gain a layered understanding, through the several different kinds of approaches we have taken, of different aspects of this taluk located outside of Bangalore. Furthermore, we hope that these findings could be used by local policy makers to better understand the problems and challenges of people living there.   

Cross Posting

Creating innovative research designs: the 10-year Methodological Think Tank case study

Katerndahl David A
Professor at University of Texas Health Science Center at San Antonio, Family & Community Medicine, San Antonio, TX




Abstract

PURPOSE:
Addressing important but complex research questions often necessitates the creation of innovative mixed methods designs. This report describes an approach to developing research designs for studying important but methodologically challenging research questions.

METHODS:
The Methodological Think Tank has been held annually in conjunction with the Primary Care Research Methods and Statistics Conference in San Antonio since 1994. A group of 3 to 4 methodologists with expertise balanced between quantitative and qualitative backgrounds is invited by the think tank coordinators to serve on a 2-day think tank to discuss a research question selected from those submitted in response to a call for proposals. During the first half-day, these experts explore the content area with the investigator, often challenging beliefs and assumptions. During the second half-day, the think tank participants systematically prune potential approaches until a desirable research method is identified.

CONCLUSION:
Researchers and academic departments could use this process locally to develop innovative research designs.
https://www.researchgate.net/publication/6791712_Creating_Innovative_Research_Designs_The_10-Year_Methodological_Think_Tank_Case_Study