Sacrifices of Comparison Made in Political Sciences

Political science focuses on the analysis of how political decisions are made and implemented. ‘Comparative politics, as a field of study, provides us with a ready array of conceptual and analytical tools that we can use to address and answer a wide range of questions about the social world.’ (Lim, 2010). Political scientists aim to have a better understanding of how political institutions and systems function, how problems have occurred and how other problems may come about in the future. However, there is much debate on what method should be used to achieve this. The quantitative method (large-N analysis) uses mass data from statistics and inquiries, to establish trends and patterns. The qualitative method uses singular case studies to control variables and provide in-depth analysis. Small-N analysis binds these two methods together, by comparing a small-handpicked selection of case studies. Many researchers champion the large-N analysis, whilst others argue that this sacrifices depth and therefore produces unreliable data. Due to the nature of comparative politics, although depth is sacrificed for breadth in many cases, it is necessary to draw comparisons and make assumptions about various democracies. This essay will explore large-N and small-N comparisons, along with individual case studies, comparing individual research reports, all using different methods, to then assess their strengths and weaknesses.

Comparative politics relies on breadth to generate variables and create typologies. The aim of comparative research is to formulate rules and to apply them to similar cases. Therefore, exploring multiple cases in a breadth study is imperative to generalise and fulfill the requirements of the hypothesis. Hypotheses are key in comparative politics, especially when considering the interaction of different variables. Consequently, it is imperative that various results from different studies are considered. Scientists must study, from alternative perspectives, political systems, which should formulate and test similar hypotheses. Wieviroka (1992:163) argues that to avoid starting again from scratch, earlier findings have to be borne in mind. Therefore, when conducting a study, there must be a basic knowledge of political systems as most political scientists usually adopt the typical comparative politics methods; the method of difference and the method of agreement. These make sure that variables representing the differences and similarities can be identified to give the study good foundations. For example, there are broad consensuses over approaches used in comparative politics such as institutionalism, pluralism, corporatism, behaviourism, cultural perspectives and policy analyses. This emphasises that there is a need to integrate findings of various studies to gain a better understanding of how institutions influence the individual’s choice.

Various approaches have aided comparative politics in its ability to create a complex picture of political systems and the factors that contribute to the structuring of the state. These breadth analyses then provide further foundations for typologies and classifications. For example, Amorim Neto & Cox’s “Electoral Institutions, Cleavage Structures, and the Number of Parties” conducts a large-N study to analyse if there is a correlation (and if so, how far it goes) between the different measures of electoral system permissiveness, the number of effective parties and ethnic fragmentation. The data collected was from 54 elections around the world, including both presidential and parliamentary elections. From this, Neto and Cox were able to deduce that the effective number of parties are dependent on the diversity of the state and the types of electoral systems. The benefit therefore of conducting a large-N analysis is that statistical controls can be used. Statistical control (SC) refers to the ‘technique of separating out the effect of one particular independent variable from the effects of the remaining variables on the dependant variable in a multivariate analysis’ (Gujarati, 2003). Using SC can aid political scientists in ruling out rival explanations for why outcomes are produced. Within this, it is easier to identify ‘outliers’ and then make generalisations as their theory is tested over a larger sample and in turn becomes much more representative. One of the main problems with this type of study is that it is often expensive and time consuming, as Collier (1993) notes that there is a problem with “collecting adequate information in a sufficient amount of time”. However, this is a necessity to gather evidence and draw comparisons, which is what comparative politics is founded upon.

However, case studies can prove to be valuable for scientists as they provide in depth analysis, which takes into consideration multiple variables. This broaches the issue that often arises from breadth studies that many people have different opinions. For example, the question of ‘What is Democracy?’ is extremely controversial, as many have different opinions of what it actually entails. This is often contentious as different states carry different political cultures. This is a problem often exacerbated by the fact many countries have different political cultures. However, quantitative study is often guided on the basis of generally acknowledged and accepted mass data. Dogan (1994) argues that a false sense of security often arises with mass data and therefore this prevents researchers from assessing the ‘validity of quantitative data’. Especially as some worldwide studies that come from specific sources (e.g. the World Values Survey) use statistics from specific institutions such as the United Nations, World Bank and the European Union. These sources are not always precise and therefore support Dogan’s claim. For example, political participation in Islamic countries is very unalike the patterns we see in the Western world. Norris and Inglehart (2004) link this with beliefs about how gender roles should be carried out. Researchers often do not have the resources to conduct all of their own studies, which allow them to control variables, and even if they did so, by the time they had conducted the studies, the economic and social realities may have changed. Here, case studies provide valuable, as they are able to construct hypotheses, contribute to theory building, and produce in-depth analysis of outliers that are found through large-N studies (Landman, 2000).

The benefits of a singular case study is portrayed in Robert Putnam’s Making Democracy Work whereby he analysed 20 different regions throughout Italy across 20 years of study. He studied the impact that institutional reform had on institutional performance. When he came across discrepancies in these findings, he assessed the reasons for cross-temporal and cross-sectional variation in institutional performance, and he studied six of these regions in more depth as a result. This would not have been possible with a large-N study. Hence, a key benefit of case studies is that you can explain outcomes with process tracing (George and Bennett, 2005). Bryman (1974) adds that qualitative research gives political scientists more freedom to shape their own design and therefore adapt to ‘social complexities’ to a much larger extent than quantitative methods of study. Nevertheless comparative politics as a whole does not sacrifice breadth for depth as case studies mean that only one entity is analysed and are therefore of a limited value to political scientists. Case studies merely lay foundations as an explorative method to further understanding quantitative analysis (Lijphart,1975:160) as they are only useful to “disconfirm a regularity to a limited degree” (Sartori, 1994:23). Therefore, in the field of comparative politics they have limited value, as generalisations cannot be drawn from them.

The idea that brings both the large-N design and case studies together is the practice of small-N studies. Small-N analysis examines a small number of cases in depth, which are all selectively handpicked. One of the main strengths of these types of studies are that they are “specified, complex models that are sensitive to variations by time and place.” (Coppedge, 1999). “Perils of Presidentialism” (Linz, 1990) is an example of small-N analysis. Linz considers the consequence that presidential and parliamentary government types have on states’ democratic ability. Linz’s research was carried out through selected cases (countries) from Western Europe (e.g. Italy, Spain and France), Latin America (such as Chile, Argentina and Brazil) and North America. His hypothesis was based on proving if the nature of parliamentary rule was superior nature of presidential rule. Small-N analysis enabled him to intentionally select case studies that had alike characteristics to aide specific hypothesis testing. The Comparative Method (Collier, 1993) argues that small-N designs such as Linz’s enable the intensive analysis of a few cases with less energy expenditure, financial resources and time. Therefore, intensive analysis can be more productive than superficial statistical analysis, which can be time consuming and difficult to successfully execute as the collection of large date can be extremely difficult. A benefit of utilising small-N instead of large-N is that the studies can be operationalised at a lower level and consequently the results are likely to be valid as the concepts chosen are being accurately measured. Small-N scientists are critical of the case study method as they believe that patterns must come from theory or observation which is “validated by intimate knowledge of the detail, nuance, and history of the small number of cases” (Paul et al. 2013).

However, once the number of cases expands, analysts can no longer “hold all the cases in their head” and the information is too large to be compared holistically and qualitatively without expecting a margin of error. Lijphart argues that this is because small-N analyses can focus on “comparable cases” that are matched on many variables that are not central to the study. This means that they can effectively ‘control’ these variables. They can then choose countries, which differ in terms of key variables that are the focus of the study which allows a more reliable assessment of their influence. Yet, small-N analysis has various weaknesses, which make it inferior to its large-N counterpart. Goggin (1986) comments on the nature of small-N analysis, as there are many variables yet a small number of cases. Therefore, it is more efficient to study more countries and consequently conduct a large-N study instead. As a result, Linz’s study has come under great criticism for its underdevelopment. Kerlinger (1973) argues that the ideal research design must answer the research question, introduce the element of control for extraneous independent variables and permit the investigator to generalize from their findings. Small-N studies are incapable of fulfilling these criteria. However, Prezworski et. al in Democracy and Development (2000) studies 150 countries over 40 years to achieve a similar objective to Linz. Conversely, unlike Linz’s analysis, this study complies with Kerlinger’s ideal research design as it allows generalisation due to the increased scale of the project and randomisation of case studies.

After analysing the evidence, it portrays that none of the aforementioned methods achieve a perfect outcome, but that large-N analysis serves the purpose of comparative politics best. This is due to the fact that it is the only method, which can successfully draw large comparisons and create generalisations to create typologies. Without large-N studies, theories could not be widely developed and created. Although small-N studies are useful for those who do not possess the time or resources to conduct a large-N study, they carry a selection bias and therefore results are not always accurate. Similarly, both case studies and small-N studies still have the issue whereby even the smaller samples they handpick/select are not always guaranteed to be completely representative. This means that large-N comparisons are more valid and representative as they study a much larger percentage of the population and additional countries. Even though case studies can take variables into consideration and therefore provide a more accurate picture, they are limited as to what conclusions political scientists can draw from them. Subsequently, although depth is inevitably lost when comparing a large-N study to a small-N study, breadth is essential to carry out the main function of comparative politics, which is to draw substantial comparisons.