Virtuous people and evil elites? The role of moralizing frames and normative distinctions in identifying populist discourse

Hunger, Sophia Article — Published Version Virtuous people and evil elites? The role of moralizing frames and normative distinctions in identifying populist discourse Political Research Exchange Provided in Cooperation with: WZB Berlin Social Science Center Suggested Citation: Hunger, Sophia (2024) : Virtuous people and evil elites? The role of moralizing frames and normative distinctions in identifying populist discourse, Political Research Exchange, ISSN 2474-736X, Taylor & Francis, London, Vol. 6, Iss. 1, pp. 1-26, https://doi.org/10.1080/2474736X.2024.2370308 This Version is available at: https://hdl.handle.net/10419/308502 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. http://creativecommons.org/licenses/by/4.0/ RESEARCH ARTICLE Virtuous people and evil elites? The role of moralizing frames and normative distinctions in identifying populist discourse Sophia Hunger a,b a University of Bremen/SOCIUM, Bremen, Germany; b WZB Berlin Social Science Center, Berlin, Germany ABSTRACT Populism has been a much-studied concept in Social Science research. A great share of research on the concept has been dedicated to measuring the occurrence of populism in political text. In this article, I propose a novel measure of populism that considers established theoretical assumptions, i.e. presenting the people as morally superior and the elite as evil. This moral framing of the antagonistic groups is necessary to identify populist discourse while keeping it separate from empirically related concepts, e.g. radical right-wing ideology. The novel two-step dictionary detects morallyframed references to both groups. I apply this approach to a text corpus of all speeches given in the European Parliament from 1999 to 2014 and carry out extensive validity checks. Taking the moralizing notion of populism more seriously not only contributes to our theoretical understanding of populist discourse and its impact on the political sphere, but the new measure also improves previous approaches to measuring populism. ARTICLE HISTORY Received 13 November 2023 Accepted 13 June 2024 KEYWORDS Populism, parties, European parliament, text-as-data, moralization Introduction Populism has become a buzzword (Hunger and Paxton 2022): It is used attributed to politicians of all kinds; is said to be contagious for mainstream parties; and supposedly affects citizens in their political behaviour. Particularly if we are interested in how populist discourse affects other actors, e.g. parties and voters, we need a theoretically informed way to measure populism on a large scale. Many influential studies (e.g. Jagers and Walgrave 2007; Rooduijn and Pauwels 2011) have laid important groundwork for the endeavour of measuring populist discourse across large quantities of text. Drawing on these studies, this article uses text-as-data methods for a new measurement of populist discourse, which emphasizes the moral notion of populism. Theoretical accounts of populism stress the centrality of the morally-charged Manichean distinction between the people and the elites. That is, populists refer to the people by portraying them as homogeneous, superior and good, while the elites are presented as corrupt, evil and degenerated. While theoretically well-established, this moral © 2024 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/ licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The terms on which this article has been published allow the posting of the Accepted Manuscript in a repository by the author (s) or with their consent. CONTACT Sophia Hunger [email protected] POLITICAL RESEARCH EXCHANGE 2024, VOL. 6, 2370308 https://doi.org/10.1080/2474736X.2024.2370308 distinction has not travelled to quantitative measurements of populism. This poses several problems, as not only populism, but also thick ideologies are based on a differentiation between the people and an out-group, e.g. in the case of nationalism the people understood as ‘natives’ vs. immigrants as ‘non-natives’. This example is particularly crucial, as populist radical right parties experienced a leap of support in recent years in Europe. I argue that this normative distinction between the two groups needs to be taken more seriously in measuring populism and in the study of its consequences, e.g. its effect on other parties, voters, or the media In order to ground my populism measurement thoroughly in the theory, I propose a two-step dictionary approach. The first step employs two baseline dictionaries, one for ‘the people’ and one for ‘the elite’, which consist of morally neutral words for both groups, such as ‘elite’, ‘politicians’, ‘citizens’, or ‘taxpayers’. These baseline dictionaries are used to detect neutral mentions of each of the groups and for extracting the section or snippets of the speeches in which these mentions occur. In the second step, these snippets are then analysed using frame dictionaries, again one for each group. The frame dictionaries contain words and phrases which can be used to present the groups as morally superior, i.e. the people, or as morally corrupted, i.e. the elite. My approach comes with several advantages. First, by using a different measurement for anti-elitism and people-centrism, it allows for assessing how these different features of populism are used by different actors. Second, by taking the moral distinction between the people and the elites into account, it provides a way of measuring populism that is less prone to be conflated with references to the people in nationalist, socialist, or other ways. Third, after having established and validated my measure, I compare my discourse-based measures to categorizations of populist parties based on experts in order to assess if ‘populist is what populists do’. Following Forrest Gump’s famous dictum of ‘Stupid is as stupid does’, I assess whether parties, that are considered as populist use more populist rhetoric based on my measure and test whether there are differences based on radicality and ideology. Lastly, the combination of base and frame dictionaries provides a straight-forward approach to quantitative frame analysis more generally. This might be of interest for a broader range of scholarship beyond populism research, for instance for assessing how specific groups of people, e.g. female politicians, experts, etc., are presented in different sets of texts. Populism as a moralized discourse The ideational definition of populism The concept of populism has often been criticized for lacking conceptual clarity and is subject to vivid academic debates about a worthwhile definition. However, several scholars (Aslanidis 2016, 89; Bonikowski et al. 2019, 62; Hawkins and Rovira Kaltwasser 2017, 527; Mudde and Rovira Kaltwasser 2018, 2–3; Rooduijn 2019, 363) have noted a trend towards agreeing on an ideational definition. These approaches share some conceptions of populism: (1) the antagonism between the people and the elites as populism’s core concept, (2) the idea that populism lacks strong programmatic appeals and is most often combined with ‘full’ ideologies, as well as (3) a ‘degreeist’ understanding of populism. While the ‘ideational camp’ includes scholars that define populism as ‘framing device’ (Bonikowski et al. 2019, 62), ‘discursive frame’ (Aslanidis 2016, 98), ‘communication style’ 2 S. HUNGER (Brubaker 2017a, 2017b), ‘communication phenomenon’ (Vreese et al. 2018), or ‘worldview’ (Hawkins 2009), most famously phrased by Mudde (2004, 543) describing populism ‘thin ideology’ that: considers society to be ultimately separated into two homogeneous and antagonistic groups, the ‘pure people’ and ‘the corrupt elite’, and which argues that politics should be an expression of the volontè gènèrale (general will) of the people. A further common denominator of different ideational definitions is the understanding of populism as discursive feature rather than a fixed policy orientation: ‘unlike a [thick] ideology, populism is a latent set of ideas or a world view that lacks significant exposition and (…) and is usually low on policy specifics’ (Hawkins 2009, 6). While full ideologies result in programmatic orientations of parties, the populism becomes visible through the use of a specific discourse (Hawkins 2010, 10). Closely connected to the ideational understanding of populism is the debate around the ‘degree-ism’ of populism. Aslanidis (2016, 96) proposes to consider populism as gradual rather than dichotomous since this is ‘more or less, how the concept has been operationalised in the growing quantitative literature’ (Bernhard, Kriesi, and Weber 2015; see e.g. Jagers and Walgrave 2007; Reungoat 2010; Rooduijn and Pauwels 2011). This allows for varying degrees of populist discourse across different political actors and contexts. Similarly, Cammack (2000, 155) makes a case for rather using the adjective ‘populist’ instead of the noun as a ‘qualifier of some substantive political project’. DeeganKrause and Haughton (2009, 822) also suggest to shift ‘our understanding of populism (…) to a description of party appeals rather than parties themselves’. Additionally, treating populism solely as graded allows for a more nuanced study of the phenomenon and decreases normative biases and the compulsion for academics to take sides in an essentialist struggle (Aslanidis 2016). The degreeist understanding of populism is central to this study. It allows for developing a measurement that is independent from ‘external’ categorizations of actors into a binary measure of populist and non-populist. On the other hand, it makes it crucial to develop a measurement which is strongly based on theoretical assumptions instead of relying on a ‘we know it when we see it’-logic. Both the opposition to the ‘evil’ elites and the praise for the ‘good’ people are wellestablished core features of populist discourse. While nearly all ideologies appeal to ‘the people’ in a specific way, e.g. socialism or nationalism, the moral distinction between the people and the elites is essential to populism. This moral elevation of the people, however, requires a references category, an ‘other’, an anti-pole. This reference category is embodied by the ‘morally degenerated’ elites and hence completes this dyadic relationship. In the following section, I discuss that references to both groups and their moralized framing are inextricable markers of populism, that are both necessary but not sufficient (see also Table 1). Table 1. Morality and antagonistic groups as markers of populist discourse. Moralized language Antagonistic groups: elite vs. people Yes No Yes populist not sufficient No not sufficient not populist POLITICAL RESEARCH EXCHANGE 3 The centrality of moral framing for populist discourse Actors, who employ populist discourse, need to discriminate between the people and the elites and they do so by presenting one side as pure, good and virtuous and the other side as evil, corrupt and rotten. Various authors have emphasized this normative distinction between both groups as central to populism, as it is defined as ‘a Manichean and moralistic discourse that divides society into el pueblo and oligarchy […]’ (De la Torre 2000, 4). In a similar vein, Müller (2017, 19–20) defines populism as ‘a particular moralistic imagination of politics, a way of perceiving the political world that sets a morally pure and fully unified (…) people against elites who are deemed corrupt or in some other way morally inferior’. Wiles (1969, 167) connects the ideational nature of populism and its moral notion stating that populism ‘is moralistic rather than programmatic’. Similarly, Panizza (2005, 22) argues that populists substitute a moral discourse for a political discourse in order to be able ‘to talk politics while denouncing it as a dirty game’. That is, by employing a moralistic discourse that differentiates them from other political actors, they can be part of politics while condemning the whole political sphere and its actors. By using universal abstractions, they ‘contrast the high moral grounding of [their] message with the corruption and betrayal of the political establishment’ (Panizza 2005, 22-23), In this moralistic logic, the elites, e.g. other political actors, are presented as enemies who are not ‘sharing a common symbolic space within which the conflict takes place’ (Mouffe 2000, 20). This makes compromise or even dialogue with the opponent impossible which is unacceptable for populist parties (De la Torre 2000). Panizza (2005, 22–23) also points out that moral divides are used by populists in order to disqualify adversaries thereby making legitimate dissent impossible. Mudde dedicates a significant share of his chapter on the ideational approach to populism to morality, which he calls ‘the essence of the populist division’ (2017, 29). Thus, purity and authenticity are for him at the core of the distinction between the two groups. This moralistic distinction is, however, tied to the presence of antielitism and people-centrism, in order to be considered as populism, as Table 1 shows. Despite this multitude of scholars emphasizing the moralized notion of populism, Stavrakakis and Jäger (2017, 12) argue that morality is ‘the shaky basis’ of ‘the “new” mainstream in contemporary populism study’ which is both under-defined and too broad to be a distinct feature of populist rhetoric. Moreover, they (Stavrakakis and Jäger 2017, 13–14) criticize that the morally charged distinction between ‘evil’ and ‘good’ has been employed by politicians of manifold ideologies, especially as it is ‘more or less unavoidable in any political conjuncture, and especially pronounced in crucial turning points’. They provide several exemplary quotes by politicians, e.g. by Thatcher: ‘I am in politics because of the conflict between good and evil, I believe that in the end good will triumph’. While this statement certainly includes moral elements, it falls short of qualifying as populist as the antagonistic divide. Thatcher refers to ‘good’ and ‘bad’ in an unspecific way, which is from references to people or elites. Moralized language is certainly employed by manifold actors (Jasper 1992; see e.g. Jung 2019) and purity and idealization may be used for various political arguments (e.g. during the recent political crisis in Europe, see Narotzky (2016)), but Stavrakakis and Jäger (2017) neglect the necessary combination of moralized framing with the references to the people and the elites. In this regard, their critique of morality is under-specified and does not apply to a shared theoretical understanding of populism. 4 S. HUNGER The antagonism between the people and the elites However, morality and moralized discourse alone are not sufficient markers for populist discourse, the same applies for sole references to the people and the elites. It is not just bare references to ‘the people’ and ‘the elites’ that characterize populism, these groups are presenting and framing them in a morally charged way. First, ‘appeals to the people’ as marker are not explicit enough about how ‘the people’ are perceived by populists. Referring to the people is a trait that is shared by politicians from nearly all political camps and colour. In political discourse, various understandings of ‘the people’ may be at play: for instance, in nationalistic, economic, cultural, or political terms (Meny and Surel 2002). Radical right-wing actors present the ‘people as a nation’, while radical left actors rather emphasize class (Kriesi 2014, 362). What sets populism apart is how these groups are charged with meaning which is core to this ideology (Zehnter 2023). While various constructions of the people can be found in modern democracies, I argue that the construction of the people as morally superior, honest and virtuous lies at the core of populism, which is then to be contrasted by the rejection of the ‘evil elites’. Many authors (Hawkins 2009; e.g. Jagers and Walgrave 2007; Reungoat 2010; Rooduijn and Pauwels 2011) have used ‘appeals to the people’ as identifier for people-centrism When measuring appeals to the people, Jagers and Walgrave (2007, 339) stress that references to the population (group) as an inseparable unity mostly preceded by a definite article (e.g. the voter, the people, the consumer) ‘can be regarded as the most solid indicator of populism’. Reungoat (2010) operationalizes people-centrism as the frequent use of ‘terms that referred to the population (population as a whole or population categories)’ but does not specify any moral evaluation or construction as superior and therefore might overestimate people-centrism. Aslanidis (2017, 9) points out that with this approach ‘people-centrism tends to get over-coded, since all references to ‘the people’ are coded indiscriminately into this category’. When not considering this moral framing of the antagonistic groups, we might rather be measuring concepts other than populism, e.g. nationalism. The necessary combination of references to the antagonistic groups and a moralized framing of these requires careful operationalization. In the next section, I shall present my approach which takes both markers into account. The moral distinction between the people and the elites – a tool for identifying populist discourse? Measuring populism using dictionaries Populism has often been measured in political texts of all kinds, since common approaches to measuring ideology, such as using roll-call votes, are not transferable to the concept of populism. Next to text-based measures, expert surveys are increasingly used to measure populism in political parties (see e.g. Meijers and Zaslove 2021). However, the surveys are not suitable to retrospectively assess how populism developed (Levick and Olavarria-Gambi 2020). Measuring populism using automated text analysis has shown to be a tedious task, that comes – due to the conceptual complexity of populism – with many pitfalls and challenges (Jankowski and Huber 2023). Recent advances in the social sciences have also POLITICAL RESEARCH EXCHANGE 5 sparked approaches to measures populism using supervised machine learning. Di Cocco and Monechi (2022) and Celico, Rode, and Rodriguez-Carreño (2024) use externallydefined (binary and continuous) measures of parties’ populism and train classifiers on this information. This party-based approach, however, does not allow to distinguish between populism and other party characteristics, such as communication style or thick ideology, in the prediction and is thus largely detached from a theoretically-based conception of populism (see also Jankowski and Huber 2023). Erhard et al. (2023) and Bonikowski, Luo, and Stuhler (2022) use transformer models in order to measure parties’ level of populism, in the German Bundestag and US electoral campaign respectively. Weighting between dictionaries and supervised machine-learning is strongly tied to the level of transparency the classification should entail. Weighting between dictionaries and supervised machine-learning is strongly tied to the level of transparency the classification should entail. Supervised machine-learning approaches are often black-boxes for researchers and readers alike and ‘due to [their] complexity [they] lack transparency for validating the results sufficiently. Hence, inherently interpretable models should be preferred for highly complex tasks, similar to the task at hand’ (Rudin 2019). Since I aim to implement the theoretically well-established centrality of a moral framing into a valid measurement of populism, for me transparency outweighs the benefits of transformer models. While often feature scores are not discussed in machine-learning approaches, using a dictionary model gives us full transparency and control of which features are tied to our concept of interest. I believe this is particularly valuable in my case, where my contribution also aims at showing the importance of morality in order to measure populism. Some of the aforementioned studies use dictionaries to identify and measure populist discourse (Bonikowski and Gidron 2016; Rooduijn and Pauwels 2011; see also Gründl 2022, for an application Breyer 2022). Dictionaries draw on word frequencies that are used in a respective text. They are designed to capture categories of a theoretical concept ‘by allocating words to these categories using a combination of a priori and empirical criteria’ (Laver and Garry 2000, 626). Applying a dictionary, i.e. a list of theoretically motivated keywords, then results in counts of these keywords’ occurrences in different texts. By assessing the frequency in which the keywords are used, it measures to which extent a certain concept is prevalent in a document (Grimmer and Stewart 2013, 8). Hence, designing such a dictionary requires a deep understanding of the concept at hand, as it is only valid with a precise choice of keywords and it might otherwise result in numerous false positives (Ruedin and Morales 2017). Furthermore, qualitative, in-depth assessment and validation are crucial in order to assess the number of false positives. Deleting and adding keywords based on this, however, may result in rather ad hoc dictionaries of either extreme brevity or excessive length (Aslanidis 2017). Hence, the terms included in populism dictionaries often seem too broad, e.g. class, politic*; propaganda; referend*; regime, people (Rooduijn and Pauwels 2011), as there is no strong theoretical reason why – for instance using the word politics – qualifies for a populist statement. Other keywords seem very arbitrary and tailored to a very specific context, such as loophole and long nose (Bonikowski and Gidron 2016), or donors (Oliver and Rahn 2016). Many of these words are not theoretically connected to the concept of populism and hence capture other phenomena. 6 S. HUNGER I base my approach on strong theoretical assumptions, i.e. the centrality of moral framing for populist discourse. I argue that this theoretical foundation allows us to choose keywords for the two reference groups and their respective framing that permits us to identify populist discourse. This approach draws on a communicationcentred rather than an actor-centred understanding of populism (Stanyer, Salgado, and Strömbäck 2016, 354). While the first focuses on the theoretical characteristics of populist communication, the latter uses the characteristics of populist political actors as a starting point. The actor-centred approach assumes that we already know who populist actors are and draws conclusions about populist communication based on these actors’ political communication. Focusing on a pre-defined set of populist actors and the way they talk, comes at risk, I argue, of conflating populist discourse with the programmatic profile of the populists’ host ideologies. Furthermore, an actor-based approach is at odds with an understanding of populism as a matter of degree, as discussed in the previous section. If we use a set of actors as a heuristic to measure populist discourse, we discount the possibility that populist discourse can vary across settings, contexts, issues and actors. Identifying the moral distinction in populist discourse through a two-step dictionary In order to use the normative distinction between the elite and the people as a tool for identifying populist discourse, I suggest using a novel two-step dictionary approach, that allows for detecting whether one of the groups is referred to in a morally charged way. Unlike most previous studies, I use separate dictionaries to capture the reference to the two groups. This allows for measuring people-centrism and anti-elitism, i.e. two core features of populism, separately and thus for assessing differences in the number of references across actors. My two-step dictionary approach is set up as follows: First, I design two baseline dictionaries for references to the people and the elites. This allows me to identify the references to the two groups as distinct features and to show how they are used by different parties as well as when they occur together. Studying microlevel populist attitudes, Schulz et al. (2017, 2) make a similar argument stating ‘that a unidimensional model fails to adequately describe populist attitudes, as it does not account for the different political ideas that have been identified as distinct yet correlated facets of a populist ideology’. As argued above, mentioning the elites and the people is not finegrained enough to measure populist discourse. As referring to the people can be done in neutral, nationalist and many other ways, I suggest that the construction of the people as morally superior captures populism more precisely. Similarly, not all references to the elite are necessarily populist, i.e. negative, as politicians could refer to the elite positively as well. In order to capture this moral framing, I develop two additional frame dictionaries which aim to capture whether the people are presented in a morally superior and the elites in a morally degenerated manner. In the first step, the baseline dictionaries are tailored to identify references to the people or the elites (see Table 2). I created them drawing on previous automated approaches and codebooks for hand-coding (Bonikowski and Gidron 2018; Hameleers and Vliegenthart 2019; Jagers and Walgrave 2007; Reungoat 2010; Rooduijn and Pauwels 2011). Next, I assessed the quality of the two dictionaries by looking at a sample of appearances of every word in the people – and the elite-dictionary and judged whether they capture POLITICAL RESEARCH EXCHANGE 7 the right concepts. I excluded words as false positives if they were mostly used to refer to specific, narrow groups of people or other concepts that are not related to ‘the people’ or ‘the elites’. For instance, some studies claim that caste is often used in populist discourse in order to refer to the elites (Rooduijn 2015). However, most of the occurrences of the word caste were in fact referring to the Indian caste system. Hence, the keyword ‘caste’ was removed from the dictionary. Some keywords were included using globing, i.e. wildcard characters (depicted by an asterisk *), thus allowing for any string following the specified keywords. This means that the keyword elit* will pick up elite, elitism, elitist, elite-focused, and so on. Other words, e.g. resident, are only included in specific grammatical forms, as the keyword with an asterisk otherwise would pick up words such as ‘residential’ disproportionally. The final set of words for the base-dictionaries is depicted in Table 2. Some of the keywords are quite broad and neutral, e.g. citizen, while others – especially for the elite – carry a value judgement, for instance, cronies. I consider these morally charged keywords as sufficient markers for a moral framing. These words are highlighted in bold in Table 2. Using the baseline dictionaries, I find a total of 9817 references to the elite and 133,625 references to the people. The overrepresentation of references to the people serves as an indication that referring to the people is done by most politicians in a multitude of ways, that are not populist. Table A2 in the appendix shows the frequency of the individual keywords for people and elites in the corpus. As I have argued above, references to the people and the elites ought to be presented in a moralized way in order to be considered populist rhetoric. To perform the next step of my analysis, i.e. determining if the two groups are framed in a morally-charged way, I select ‘snippets’ around the occurrences of the peopleand the elite-dictionary. This means, that I create a new text corpus, that does not consist of the full speeches anymore, but only of the parts of the speeches in which we find a reference to the people or the elites. Considering the length of the speeches and the sentences within them, I decided to select a window of eight words before and after each keyword. Additionally, the length of the frame window is based on the German election manifestos since 1998. The Manifesto Project (Lehmann et al. 2023) uses the source in order to code parties’ issue position. The main coding unit in the project are so-called quasi-sentences, where one quasi-sentence includes exactly one positional statement on one issue. I assessed the summary statistics for the quasi-sentences coded in all German manifestos in that timeframe. The median length is 13 words, the mean is 14.71 words. I used this Table 2. Keywords for references to the people and the elite. People Elite citizen population’s apparatchik* europhil* citizen’s resident aristocrat* financial sharks citizens resident’s bad apples* financier* citizens’ residents banks mafia europeans residents’ bosses mafia’s europeans’ taxpayer bosses’ old elites families taxpayer’s Brussels mafia oligarch* family taxpayers bureaucrat political class* man on the street* taxpayers’ circles* politician* ordinary person* the public cronies power monger the public’s people ruling circles elit* people’s population establishment ruling class establishment’s sycophancy eurocra* 8 S. HUNGER already discussed above – a discussion on populism research that could be summarized as ‘Populist is as populist does?’ Or put differently: How do we define whether an actor or a party is populist? Gump’s answer implies that a person should rather be judged by actions and not her appearance. Transferred to the study of parties and party actors, this calls for perceiving populism as a ‘description of parties’ appeals rather than parties themselves’ (Deegan-Krause and Haughton 2009, 822) as already discussed in section 2. Table 5 shows the total number of speeches given by MEPs of parties that are classified as populist or non-populist following Rooduijn et al. (2019). The three last columns show the classification of populist speech based on my measure, and the measures of Bonikowski and Gidron (2018) and Rooduijn and Pauwels (2011), separately for populist and non-populist parties. Of all speeches in the sample, about 20 per cent were given by actors belonging to parties that are traditionally considered populist. The first difference that sticks out is the mismatch regarding the total share of speeches classified as populist between the measures. While the dictionary by Rooduijn and Pauwels (2011) hits in nearly 40 per cent of the speeches, my measures classify around 2.5 per cent as populist and the approach of Bonikowski and Gidron (2018) classifies only around 1.4 per cent of all speeches as populist. 3 This serves as an additional indication that Rooduijn and Pauwels (2011) approach drastically overestimates populism. This is in line with the expectation of the EP as a hard test for populism. While the share of populist discourse is similar to Bonikowski and Gidron’s (2018) approach, their small accuracy serves as an indication that they identify different speeches than my approach. Looking at the set of parties that are classified as populist a priori based on Rooduijn et al. (2019), all three measures identify a higher share of populist rhetoric among them compared to the parties categorized as non-populist. Hence, it seems that the parties generally seen as populist in academic literature also show higher levels of populist rhetoric in the EP. However, the difference between both groups is fairly small, or in the case of Rooduijn and Pauwels (2011) even negligible. Core features – anti-elitism and people-centrism Next, I delve more into the core features anti-elitsm and people-centrism and break up my measurement. Other than many previous approaches, having separate dictionaries provides the opportunity to study how they relate to each other. In order to assess how their use coincides, I present two different figures. First, I show how often both concepts are present in the very same speech. Second, I show whether actors who are using people-centrism also use anti-elitism, i.e. whether the use of references to both groups is correlated within actors. Table 6 shows the presence of anti-elitism and people-centrism in single speeches. The two-by-two table shows that people-centrism is around six times more common than anti-elitism. Additionally, the table allows to assess the co-occurrence of anti-elitism and people-centrism within a single speech. This is, however, an extremely rare event. It appears in 61 speeches in the whole corpus i.e. in only 0.03 per cent of the speeches. Table 5. Classification of speeches for different dictionaries. Populist party Own approach Bonikowski & Girdon Rooduijn & Pauwels No Count 3623 2003 53,282 No Proportion 0.025 0.014 0.371 Yes Count 1148 880 13,715 Yes Proportion 0.032 0.025 0.389 POLITICAL RESEARCH EXCHANGE 15 Next, I turn to the MEP-level, assessing whether there is any relationship between the use of the two core features anti-elitism and people-centrism within individuals. Figure 1 presents as scatter plot showing the percentage of speeches in which MEPs used the core features of populism. The percentage of speeches including people-centrism is shown on the x-axis, while the y-axis shows the percentage of anti-elitism. The black dots represent MEPs who are classified as populist a priori following Rooduijn et al. (2019), while each grey dot represents a non-populist MEP. Additionally, the graph presents the fitted lines for both groups separately, again populist MEPs in black and non-populist MEPs in grey. 4 The graph confirms the findings already presented above: people-centrism is far more used than anti-elitism. However, there seems to be a positive correlation for the usage of both core concepts for populist parties, while this correlation is nearly absent for non-populist parties. These findings speak to a conceptual issue which has sparked a debate among populism scholars: Do markers of populism have to co-occur in the same speech or text or is it ‘enough’ if an actor uses both core features separately? Rooduijn, de Lange, and van der Brug (2014, 567) argue that ‘it is the combination of people-centrism and anti-elitism that defines populism. Only if a critique on the (bad) elite coincides with an emphasis on the (good) people, can we speak of populism’. However, if they re-run their analyses removing this condition, their results are robust. Engesser et al. (2017) argue for a ‘fragmented concept’, i.e. the core features of populism do not necessarily have to be present in one speech by a politician in order to consider the speech populist. Along similar lines, Ernst et al. (2017) show that the core features occur in a fragmented manner, but still emphasize the importance of combining them. The results presented above support this view, showing that a co-occurrence of both features is rather rare in the EP, but the use of both features by a single MEP is correlated for MEPs belonging to parties that are classified as populist. Populism left and right? Lastly, I present evidence that populism is more commonly used by parties on the fringes of the political spectrum. Several previous studies (Bernhard and Kriesi 2019; Ernst et al. 2017; Rooduijn and Akkerman 2017) have argued and shown that parties at the fringes of the political system tend to use populism more than those positioned in the centre. Figure 2 shows the use of anti-elitism, people-centrism, my combined measures and the measures by Rooduijn and Pauwels (2011) and Bonikowski and Gidron (2018). The parties are clustered along their left-right dimension using the ParlGov (Döring and Manow 2016) left-right measures, where 0 denotes the left end and 10 the right end of the spectrum. While ParlGov provides the most exhaustive measure of parties’ left-right position, I also include the category ‘NA’ in order to show that the missings do not introduce additional bias. Table 6. Cross tabulation of anti-elitism and people-centrism being present in the same speech. Anti-elitism People-centrism Yes No Yes 61 594 No 4055 174,195 16 S. HUNGER Panel A clearly shows that far-right parties use anti-elitism to the largest extent, followed by far-left parties. Unsurprisingly, mainstream parties use very little anti-elitism, given that they are considered part of ‘the establishment’ themselves. For people-centrism (panel B), the differences shrink. While the far-left and the far-right use appeals to the people nearly equally often, mainstream parties also do employ the people-centrist rhetoric, however to a smaller extent. This partly confirms March’s (2017, 290) findings of ‘demoticism (i.e. closeness to ‘ordinary’ people without this antagonistic identity)’ as Figure 1. Scatter plot for percentage of people-centrism and anti-elitism in MEPs’ speeches. Figure 2. Box plots for parties by left-right dimension. POLITICAL RESEARCH EXCHANGE 17 a common trait of mainstream parties’ rhetoric. My combined measures (panel C) still show that radical left and radical right parties use more populism than centrist parties. Turning to the outcomes of the other two dictionaries, we also see the inverse u-shaped curve in Bonikowski and Gidron’s (2018) (panel D) measure. It is worth pointing out that here radical left parties turn out to be more populist than radical right parties. For the measure by Rooduijn and Pauwels (2011) (panel E) the differences between parties of different ideological orientations nearly vanish. Conclusion In this article, I have argued that the moral distinction between the people and the elites is a central component of populist discourse. While this is theoretically well-established, quantitative approaches to measuring populism have often neglected this moral divide. Building on this centrality of the two antagonistic groups for populism discourse, I developed a novel two-step dictionary approach to measure populist discourse. My approach comes with several advantages. First, as my dictionaries are well-grounded in the literature, they are less at risk of being context-dependent and hence less ad hoc fashioned. Second, as I designed the approach based on theoretical assumptions and not actorbased, it is less prone to conflate populism with empirically co-occurring concepts, such as nationalism and radical right ideology (Hunger and Paxton 2022). Additionally, combining dictionaries for references to groups and then identifying their framing with a second set of dictionaries is a research design which can be transferred to other fields of study. I validated my measures carefully, against a hand-coded gold standard and compared their performance to other populism dictionaries. While the accuracy is not skyrocketing, my dictionaries outperform previous approaches and are hence a valuable tool for identifying populist discourse. Bonikowski and Gidron (2018) obtain very similar results in terms of the share of speeches containing populist discourse, however, their accuracy measures differ significantly from my approach. This might be due to the fact that their dictionary mostly includes words related to anti-elitism rather than people-centrism. Hence, it is very likely that their approach is not suitable for measuring people-centrism. A valid automated approach to measure populist discourse is particularly valuable for large sets of texts, as manual approaches often require an unmanageable effort, Zulianello, Albertini, and Ceccobelli (2018) for instance hand-coded 24,240 Facebook posts. I use my measures to analyse the occurrence of populist discourse in the European Parliament from 1999 to 2014 drawing on a corpus of 178,905 speeches held by 1704 individual MEPs. A comparison to a predefined classification of populist parties shows that indeed ‘Populist is as populist does’, meaning that populist parties use more populist speech than mainstream parties, however only to a small margin. The two core features of populism interact with each other on MEP level, however, their co-occurrence in one single speech is empirically scarce. Nevertheless, actors traditionally defined as populist are more prone to use both features across their speeches than mainstream party parliamentarians. Lastly, I show that radical left and radical right actors use populist discourse to a greater extent than mainstream parties. This difference is, however, more pronounced for anti-elitism than for people-centrism. Apart from serving as a tool to identify populist discourse, showing how moralized framing is central to populism provides further avenues for future research. The first 18 S. HUNGER possible expansion is to study the effect of moral framing on individuals. This might well speak to scholarship on individuals’ motivation to vote for populist parties (Arzheimer 2009; Bakker, Rooduijn, and Schumacher 2016; Dunn 2015). We might argue that aside from classical voting theories, populists manage to activate people’s moral convictions (see e.g. Ryan (2014); Ryan (2017)) by using their moralized framing and hence gain their votes (see e.g. Hameleers, Bos, and de Vreese 2017). Second – and connected to scholarship on individual-level populist attitudes (see e.g. Akkerman, Mudde, and Zaslove 2014; Elchardus and Spruyt 2016; Schulz et al. 2017) we could ask which role the moralized framing and moral convictions play in these populist attitudes. Notes 1. Previous hand-coded analyses (see e.g. Ernst et al. (2017, 1359) show that ‘The dimension of restoring sovereignty is almost absent’. 2. In order to exclude interposed questions which are often technical or procedural, I introduce a minimum length of 26 words. The maximum speech length is 2528 words, with a mean of 210 words. Additionally, I remove speeches on foreign policy, as they often contain references to people and elites of non-European countries debated (Watanabe 2018), which amounts to 43,309 speeches. The removal of speeches on foreign policy was done using the newsmap dictionary created by Watanabe (2018). Specifically, I removed all speeches with references to non-European countries with exception of the US. 3. The hand-coded random sample contains 6 percent of populist speeches. Other approaches find similar numbers for different text corpora: Bernhard and Kriesi (2019) in 5 percent of parties’ press releases in 11 countries, Ernst et al. (2017) in about 10 percent of their analysed twitter and facebook posts, Zulianello, Albertini, and Ceccobelli (2018) in about 1.9 percent of the Facebook posts in their sample. 4. For the sake of readability, I removed outliers from the graph, they were however included in the estimation of the fitted lines. Acknowledgements For helpful comments on previous versions of this article, the author wishes to thank Paris Aslanidis, Endre Borbath, Björn Bremer, Theresa Gessler, Edgar Grande, Swen Hutter, Ellen Immergut, Heiko Klüver, Hanspeter Kriesi, Sarah de Lange, Daniel Saldivia Gonzatti, Mariken van der Velden, Teresa Völker, Mara Weber, Lisa Zehnter, the editors and three anonymous reviewers of PRX, as well as the participants of the following workshops and conferences, where the article was presented and received helpful feedback: EPSA Annual General Conference 2017, LSE Graduate Conference “Populism: Left Right Centre” 2017, and the Colloquium of the Zentrum für Zivilgesellschaftsforschung. Disclosure statement No potential conflict of interest was reported by the author(s). Data availability statement The data that support the findings of this study are openly available in [Harvard Dataverse] at https://doi.org/10.7910/DVN/9UGJVM, reference number [9UGJVM]. <will be created (including replication material) upon publication> The raw data from the EU Parliamentary Speeches can be fund via https://github.com/Connected-Politics-Lab/EUDD/tree/main/Datasets/EP_Corpus POLITICAL RESEARCH EXCHANGE 19 References Adcock, R., and D. Collier. 2001. “Measurement Validity: A Shared Standard for Qualitative and Quantitative Research.” The American Political Science Review 95 (3): 529–546. doi:10.1017/ S0003055401003100 Akkerman, A., C. Mudde, and A. Zaslove. 2014. “How Populist are the People? Measuring Populist Attitudes in Voters.” Comparative Political Studies 47 (9): 1324–1353. doi:10.1177/ 0010414013512600 Armony, A. C., and V. Armony. 2005. “Indictments, Myths, and Citizen Mobilization in Argentina: A Discourse Analysis.” Latin American Politics & Society 47 (4): 27–54. doi:10.1111/j.1548-2456.2005. tb00327.x Arzheimer, K. 2009. “Contextual Factors and the Extreme Right Vote in Western Europe, 1980-2002.” American Journal of Political Science 53 (2): 259–275. doi:10.1111/j.1540-5907.2009.00369.x Aslanidis, P. 2016. “Is Populism an Ideology? A Refutation and a new Perspective.” Political Studies 64 (1_suppl): 88–104. doi:10.1111/1467-9248.12224 Aslanidis, P. 2017. “Measuring Populist Discourse with Semantic Text Analysis: An Application on Grassroots Populist Mobilization.” Quality & Quantity 52: 1241–1263. Bakker, B. N., M. Rooduijn, and G. Schumacher. 2016. “The Psychological Roots of Populist Voting: Evidence from the United States, the Netherlands and Germany.” European Journal of Political Research 55 (2): 302–320. doi:10.1111/1475-6765.12121 Benoit, K., K. Watanabe, H. Wang, P. Nulty, A. Obeng, S. Müller, A. Matsuo, et al. 2018. “Quanteda: An r Package for the Quantitative Analysis of Textual Data.” Journal of Open Source Software 3 (30): 774. doi:10.21105/joss.00774 Bernhard, L., and H. Kriesi. 2019. “Populism in Election Times: A Comparative Analysis of 11 Countries in Western Europe.” West European Politics 42 (6): 1188–1208. doi:10.1080/01402382.2019. 1596694 Bernhard, L., H. Kriesi, and E. Weber. 2015. “The Populist Discourse of the Swiss People’s Party.” In European Populism in the Shadow of the Great Recession, edited by H. Kriesi, and T. S. Pappas, 125–139. Colchester: ECPR Press. Bonikowski, B., and N. Gidron. 2016. “The Populist Style in American Politics: Presidential Campaign Rhetoric, 1952-1996.” Social Forces 94 (4): 1593–1621. doi:10.1093/sf/sov120 Bonikowski, B., and N. Gidron. 2018. “Populism in Legislative Discourse: Evidence from the European Parliament, 1999-2004.” Working Paper. Bonikowski, B., D. Halikiopoulou, E. Kaufmann, and M. Rooduijn. 2019. “Populism and Nationalism in a Comparative Perspective: A Scholarly Exchange.” Nations and Nationalism 25 (1): 58–81. doi:10. 1111/nana.12480 Bonikowski, B., Y. Luo, and O. Stuhler. 2022. “Politics as Usual? Measuring Populism, Nationalism, and Authoritarianism in U.S. Presidential Campaigns (1952–2020) with Neural Language Models.” Sociological Methods & Research 51 (4): 1721–1787. doi:10.1177/00491241221122317 Brack, N. 2015. “The Roles of Eurosceptic Members of the European Parliament and Their Implications for the EU.” International Political Science Review 36 (3): 337–350. doi:10.1177/ 0192512115571590 Breyer, M. 2022. “Populist Positions in Party Competition: Do Parties Strategically Vary Their Degree of Populism in Reaction to Vote and Office Loss?” Party Politics 29 (4): 672–684. Brubaker, R. 2017a. “Why Populism?” Theory and Society 46 (5): 357–385. doi:10.1007/s11186-0179301-7 Brubaker, R. 2017b. “Between Nationalism and Civilizationism: The European Populist Moment in Comparative Perspective.” Ethnic and Racial Studies 40 (8): 1191–1226. doi:10.1080/01419870. 2017.1294700 Caiani, M., and D. della Porta. 2011. “The Elitist Populism of the Extreme Right: A Frame Analysis of Extreme Right-Wing Discourses in Italy and Germany.” Acta Politica 46 (2): 180–202. doi:10.1057/ ap.2010.28 Cammack, P. 2000. “The Resurgence of Populism in Latin America.” Bulletin of Latin American Research 19 (2): 159–161. doi:10.1111/j.1470-9856.2000.tb00096.x 20 S. HUNGER Celico, A., M. Rode, and I. Rodriguez-Carreño. 2024. “Will the Real Populists Please Stand up? A Machine Learning Index of Party Populism.” European Journal of Political Economy 82: 102529. doi:10.1016/j.ejpoleco.2024.102529 Cranmer, M. 2011. “Populist Communication and Publicity: An Empirical Study of Contextual Differences in Switzerland.” Swiss Political Science Review 17 (3): 286–307. doi:10.1111/j.16626370.2011.02019.x Cross, J., and D. Greene. 2016. “Replication Data for: Exploring the Political Agenda of the European Parliament Using a Dynamic Topic Modeling Approach.” Havard Dataverse 25 (1): 77–94. doi:10. 7910/DVN/LVHLZK Deegan-Krause, K., and T. Haughton. 2009. “Toward a More Useful Conceptualization of Populism: Types and Degrees of Populist Appeals in the Case of Slovakia.” Politics & Policy 37 (4): 821–841. doi:10.1111/j.1747-1346.2009.00200.x De la Torre, C. 2000. Populist Seduction in Latin America: The Ecuadorian Experience. Athens: Ohio University Press. Di Cocco, J., and B. Monechi. 2022. “How Populist are Parties? Measuring Degrees of Populism in Party Manifestos Using Supervised Machine Learning.” Political Analysis 30 (3): 311–327. doi:10. 1017/pan.2021.29 Döring, H., and P. Manow. 2016. Parliaments and Governments Database (ParlGov): Information on Parties, Elections and Cabinets in Modern Democracies. Development Version. Dunn, K. 2015. “Preference for Radical Right-Wing Populist Parties among Exclusive-Nationalists and Authoritarians.” Party Politics 21 (3): 367–380. doi:10.1177/1354068812472587 Elchardus, M., and B. Spruyt. 2016. “Populism, Persistent Republicanism and Declinism: An Empirical Analysis of Populism as a Thin Ideology.” Government and Opposition 51 (1): 111–133. doi:10. 1017/gov.2014.27 Engesser, S., N. Ernst, F. Esser, and F. Büchel. 2017. “Populism and Social Media: How Politicians Spread a Fragmented Ideology.” Information, Communication & Society 20 (8): 1109–1126. doi:10.1080/1369118X.2016.1207697 Erhard, L., S. Hanke, U. Remer, A. Falenska, and R. Heiberger. 2023. PopBERT. Detecting populism and its host ideologies in the German Bundestag. arXiv preprint arXiv:2309.14355. Ernst, N., S. Blassnig, S. Engesser, F. Büchel, and F. Esser. 2019. “Populists Prefer Social Media Over Talk Shows: An Analysis of Populist Messages and Stylistic Elements Across six Countries.” Social Media + Society 5 (1): 1–14. doi:10.1177/2056305118823358 Ernst, N., S. Engesser, F. Büchel, S. Blassnig, and F. Esser. 2017. “Extreme Parties and Populism: An Analysis of Facebook and Twitter Across six Countries.” Information, Communication & Society 20 (9): 1347–1364. doi:10.1080/1369118X.2017.1329333 European Parliament. 2012. Amendment of Rule 181 on Verbatim Reports and Rule 182 on Audiovisual Record of Proceedings. Grabbe, H., and N. Groot. 2014. “Populism in the European Parliament: What Implications for the Open Society?” The International Spectator 49 (4): 33–46. doi:10.1080/03932729.2014.961768 Graham, J., J. Haidt, and B. A. Nosek. 2009. “Liberals and Conservatives Rely on Different Sets of Moral Foundations.” Journal of Personality and Social Psychology 96 (5): 1029–1046. doi:10.1037/ a0015141 Greene, D., and J. Cross. 2017. “Exploring the Political Agenda of the European Parliament Using a Dynamic Topic Modeling Approach.” Political Analysis 25(1): 77–94. Grimmer, J., and B. M. Stewart. 2013. “Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts.” Political Analysis 21 (3): 267–297. doi:10.1093/ pan/mps028 Gründl, J. 2022. “Populist Ideas on Social Media: A Dictionary-Based Measurement of Populist Communication.” New Media & Society 24 (6): 1481–1499. doi:10.1177/ 1461444820976970 Haidt, J., and J. Graham. 2007. “When Morality Opposes Justice: Conservatives Have Moral Intuitions That Liberals may not Recognize.” Social Justice Research 20 (1): 98–116. doi:10. 1007/s11211-007-0034-z POLITICAL RESEARCH EXCHANGE 21 Haidt, J., and C. Joseph. 2005. “The Moral Mind: How Five Sets of Innate Intuitions Guide the Development of Many Culture-Specific Virtues, and Perhaps Even Modules.” In The Innate Mind, edited by P. Carruthers, S. Laurence, and S. P. Stich, 367–391. New York: Oxford University Press. Hameleers, M., L. Bos, and C. H. de Vreese. 2017. “They did it”: The Effects of Emotionalized Blame Attribution in Populist Communication.” Communication Research 44 (6): 870–900. doi:10.1177/ 0093650216644026 Hameleers, M., and R. Vliegenthart. 2019. “The Rise of a Populist Zeitgeist? A Content Analysis of Populist Media Coverage in Newspapers Published Between 1990 and 2017.” Journalism Studies 1–18. Hawkins, K. A. 2009. “Is Chávez Populist? Measuring Populist Discourse in Comparative Perspective.” Comparative Political Studies 42 (8): 1040–1067. Hawkins, K. A. 2010. Venezuela’s Chavismo and Populism in Comparative Perspective. Cambridge University Press. Hawkins, K. A., and C. Rovira Kaltwasser. 2017. “What the (Ideational) Study of Populism Can Teach us, and What it Can’t.” Swiss Political Science Review 23 (4): 526–542. doi:10.1111/spsr.12281 Hunger, S., and F. Paxton. 2022. “What’s in a Buzzword? A Systematic Review of the State of Populism Research in Political Science.” Political Science Research and Methods 10 (3): 617–633. doi:10.1017/psrm.2021.44 Jagers, J., and S. Walgrave. 2007. “Populism as Political Communication Style: An Empirical Study of Political Parties’ Discourse in Belgium.” European Journal of Political Research 46 (3): 319–345. doi:10.1111/j.1475-6765.2006.00690.x Jankowski, M., and R. Huber. 2023. “When Correlation is not Enough: Validating Populism Scores from Supervised Machine-Learning Models.” Political Analysis 31 (4): 591–605. doi:10.1017/pan. 2022.32 Jasper, J. M. 1992. “The Politics of Abstractions: Instrumental and Moralist Rhetorics in Public Debate.” Social Research 59 (2): 315–344. Jung, J.-H. 2019. “The Mobilizing Effect of Parties’ Moral Rhetoric.” American Journal of Political Science 64 (2): 341–355. Kantola, J., and C. Miller. 2021. “Party Politics and Radical Right Populism in the European Parliament: Analysing Political Groups as Democratic Actors.” JCMS: Journal of Common Market Studies 59 (4): 782–801. doi:10.1111/jcms.13181 Kriesi, H. 2014. “The Populist Challenge.” West European Politics 37 (2): 361–378. doi:10.1080/ 01402382.2014.887879 Laver, M., and J. Garry. 2000. “Estimating Policy Positions from Political Texts.” American Journal of Political Science 44 (3): 619–634. doi:10.2307/2669268 Lehmann, P., S. Franzmann, T. Burst, S. Regel, F. Riethmüller, A. Volkens, B. Bernhard, and L. Zehnter. 2023. The Manifesto Data Collection. Manifesto Project (MRG/CMP/MARPOR) . Version 2023a. Berlin: Wissenschaftszentrum Berlin für Sozialforschung (WZB) / Göttingen: Institut für Demokratieforschung (IfDem). Levick, L., and M. Olavarria-Gambi. 2020. “Hindsight Bias in Expert Surveys: How Democratic Crises Influence Retrospective Evaluations.” Politics 40 (4): 494–509. doi:10.1177/ 0263395720914571 Manucci, L., and E. Weber. 2017. “Why the big Picture Matters: Political and Media Populism in Western Europe Since the 1970s.” Swiss Political Science Review 23 (4): 313–334. doi:10.1111/ spsr.12267 March, L. 2017. “Left and Right Populism Compared: The British Case.” The British Journal of Politics and International Relations 19 (2): 282–303. doi:10.1177/1369148117701753 Meijers, M. J., and A. Zaslove. 2021. “Measuring Populism in Political Parties: Appraisal of a new Approach.” Comparative Political Studies 54 (2): 372–407. doi:10.1177/0010414020938081 Mény, Y., and Y. Surel. 2000. Par le Peuple, Pour le Peuple: Le Populisme et les Démocraties. Paris: Fayard. Meny, Y., and Y. Surel. 2002. “The Constitutive Ambiguity of Populism.” In Democracies and the Populist Challenge, edited by Y. Meny, and Y. Surel, 1–21. London: Palgrave. 22 S. HUNGER Mouffe, C. 2000. The Democratic Paradox. London: Verso. Mudde, C. 2004. “The Populist Zeitgeist.” Government and Opposition 39 (4): 541–563. doi:10.1111/j. 1477-7053.2004.00135.x Mudde, C. 2017. “Populism: An Ideational Approach.” In The Oxford Handbook of Populism, edited by C. Rovira Kaltwasser, P. Taggart, P. Ochoa Espejo, and P. Ostiguy, 28–47. New York: Oxford University Press. Mudde, C., and C. Rovira Kaltwasser. 2018. “Studying Populism in Comparative Perspective: Reflections on the Contemporary and Future Research Agenda.” Comparative Political Studies 51 (13): 1667–1693. doi:10.1177/0010414018789490 Müller, J.-W. 2017. What is Populism?. London: Penguin UK. Narotzky, S. 2016. “Between Inequality and Injustice: Dignity as a Motive for Mobilization During the Crisis.” History and Anthropology 27 (1): 74–92. doi:10.1080/02757206.2015.1111209 Oliver, J. E., and W. M. Rahn. 2016. “Rise of the Trumpenvolk: Populism in the 2016 Election.” The ANNALS of the American Academy of Political and Social Science 667 (1): 189–206. doi:10.1177/ 0002716216662639 Panizza, F. 2005. Populism and the Mirror of Democracy. London: Verso. Pauwels, T. 2011. “Measuring Populism: A Quantitative Text Analysis of Party Literature in Belgium.” Journal of Elections, Public Opinion and Parties 21 (1): 97–119. doi:10.1080/17457289.2011.539483 Reungoat, E. 2010. “Anti-EU Parties and the People: An Analysis of Populism in French Euromanifestos.” Perspectives on European Politics and Society 11 (3): 292–312. doi:10.1080/ 15705854.2010.503034 Rooduijn, M. 2015. “The Rise of the Populist Radical Right in Western Europe.” European View 14 (1): 3–11. doi:10.1007/s12290-015-0347-5 Rooduijn, M. 2019. “State of the Field: How to Study Populism and Adjacent Topics? A Plea for Both More and Less Focus.” European Journal of Political Research 58 (1): 362–372. doi:10.1111/14756765.12314 Rooduijn, M., and T. Akkerman. 2017. “Flank Attacks: Populism and Left-Right Radicalism in Western Europe.” Party Politics 23 (3): 193–204. doi:10.1177/1354068815596514 Rooduijn, M., S. L. de Lange, and W. van der Brug. 2014. “A Populist Zeitgeist? Programmatic Contagion by Populist Parties in Western Europe.” Party Politics 20 (4): 563–575. doi:10.1177/ 1354068811436065 Rooduijn, M., and T. Pauwels. 2011. “Measuring Populism: Comparing two Methods of Content Analysis.” West European Politics 34 (6): 1272–1283. doi:10.1080/01402382.2011.616665 Rooduijn, M., S. Van Kessel, C. Froio, A. Pirro, and S. L. de Lange. 2019. The PopuList: An Overview of Populist, Far Right, Far Left and Eurosceptic Parties in Europe. http://www.populist.org. Rudin, C. 2019. “Stop Explaining Black box Machine Learning Models for High Stakes Decisions and use Interpretable Models Instead.” Nature Machine Intelligence 1 (5): 206–215. doi:10.1038/ s42256-019-0048-x Ruedin, D., and L. Morales. 2017. “Estimating Party Positions on Immigration: Assessing the Reliability and Validity of Different Methods.” Party Politics 25 (3): 1–12. Ryan, T. J. 2014. “Reconsidering Moral Issues in Politics.” The Journal of Politics 76 (2): 380–397. doi:10.1017/S0022381613001357 Ryan, T. J. 2017. “No Compromise: Political Consequences of Moralized Attitudes.” American Journal of Political Science 61 (2): 409–423. doi:10.1111/ajps.12248 Schulte-Cloos, J. 2018. “Do European Parliament Elections Foster Challenger Parties’ Success on the National Level?” European Union Politics 19 (3): 408–426. doi:10.1177/ 1465116518773486 Schulz, A., P. Müller, C. Schemer, D. S. Wirz, M. Wettstein, and W. Wirth. 2017. “Measuring Populist Attitudes on Three Dimensions.” International Journal of Public Opinion Research 30 (2): 316–326. Slapin, J. B., and S.-O. Proksch. 2010. “Look Who’s Talking: Parliamentary Debate in the European Union.” European Union Politics 11 (3): 333–357. doi:10.1177/1465116510369266 Stanyer, J., S. Salgado, and J. Strömbäck. 2016. “Populist Actors as Communicators or Political Actors as Populist Communicators. Cross-National Findings and Perspectives.” In Populist Political POLITICAL RESEARCH EXCHANGE 23 Communication in Europe, edited by T. Aalberg, F. Esser, C. Reinemann, J. Strömbäck, and C. H. de Vreese, 353–364. New York: Routledge. Stavrakakis, Y., and A. Jäger. 2017. “Accomplishments and Limitations of the “new” Mainstream in Contemporary Populism Studies.” European Journal of Social Theory 21 (4): 547–565. Vreese, C. H., F. de Esser, T. Aalberg, C. Reinemann, and J. Stanyer. 2018. “Populism as an Expression of Political Communication Content and Style: A new Perspective.” The International Journal of Press/Politics 23 (4): 423–438. doi:10.1177/1940161218790035 Watanabe, K. 2018. “Newsmap - a Semi-Supervised Approach to Geographical News Classification.” Digital Journalism 6 (3): 294–309. doi:10.1080/21670811.2017.1293487 Wiles, P. 1969. “A Syndrome, not a Doctrine.” In Populism: Its Meaning and National Characteristics, edited by G. Ionescu, and E. Gellner, 166–179. Macmillan. Zehnter, L. 2023. “Identity Through Distinction. Contextualizing inand Outgroups in Populist Political Communication.” Paper Presented at the ECPR General Conference 2023. Zulianello, M., A. Albertini, and D. Ceccobelli. 2018. “A Populist Zeitgeist? The Communication Strategies of Western and Latin American Political Leaders on Facebook.” The International Journal of Press/Politics 23 (4): 439–457. doi:10.1177/1940161218783836 Appendix Frame dictionaries Table A1. Keywords for positive and negative framing of the people and the elite. negative frame positive frame abuse* deprav* hungry for power political class sins betray* noble alienate desecrat* ignore* power grabbing spurn blameless ordinary man annihilate deserted immoral* power hungry stain* brave ordinary men apostasy deserter* impair power monger stomp bravely ordinary person apostate deserting imperialist* powergrabbing sycophany cleanliness ordinary apparatchik* destroy impiety powerhungry taint* decency praiseworthy arrogan* detriment* impious privilege tarnish* decent principled bad discrimnat* imposter privileges traitor* defen* reasonable bad apples disgust* indecen* profan* transgress* digni* refined betray* dishones inequitable profligate trashy ethic* the good bias* disloyal* inhuman* ravage treacher* expense of upright bigot* disobe* instable refuse treason* fair upstanding blemish disproportion insubordinat* refuse* unacceptable* forget valor Brussels mafia disrespect* insurgent renegade uncaring forgetting valour* contagio* dissociate intemperate repuls* unchaste hard earned virtuous corrupt* dysfunctional* jilt* ruin* unclean* hard working coward* endanger* lawless* ruling circles undemocratic* hard-earned cronies enem* lobby* ruling circles unelected hard-working crooked* eurocrat* miscreant sediti* unequal* hardworking cynical* europhil* mutinous selfinterested unfaithful honest* damag* evil nonconformist self-satisfied unhealthy honor* debase* favouritism not in touch self-serving unresponsive honour* debauche* filth* obscen* sequester unscrupulos ignor* deceiv* fraud obstruct sick useless interest defector greed offend* sin venal loyal defian* gross old elites sinful* wantona loyalt* defile* harmful* oppose sinned wealthy (Continued) 24 S. HUNGER