Hybrid intelligence strategies for identifying, classifying and analyzing political bots

$ € £¥ social sciences Article Hybrid Intelligence Strategies for Identifying, Classifying and Analyzing Political Bots Berta García-Orosa 1,*, Pablo Gamallo 2, Patricia Martín-Rodilla 3and Rodrigo Martínez-Castaño 2   Citation: García-Orosa, Berta, Pablo Gamallo, Patricia Martín-Rodilla, and Rodrigo Martínez-Castaño. 2021. Hybrid Intelligence Strategies for Identifying, Classifying and Analyzing Political Bots. Social Sciences 10: 357. https://doi.org/ 10.3390/socsci10100357 Academic Editor: Andreu Casero-Ripollés Received: 7 June 2021 Accepted: 10 September 2021 Published: 27 September 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). 1Department of Communication Sciences, Universidad de Santiago, 15782 Santiago de Compostela, Spain 2Centro de Investigación en Tecnoloxías da Información, Universidade de Santiago de Compostela, 15705 Santiago de Compostela, Spain; [email protected] (P.G.); r[email protected] (R.M.-C.) 3Information Retrieval Lab, Centro de Investigación en Tecnoloxías da Información e as Comunicacións (CITIC), Universidade da Coruña, Campus de Elviña s/n CP, 15071 A Coruña, Spain; [email protected] *Correspondence: berta.gar[email protected] Abstract: Political bots, through astroturfing and other strategies, have become important players in recent elections in several countries. This study aims to provide researchers and the citizenry with the necessary knowledge to design strategies to identify bots and counteract what international organizations have deemed bots’ harmful effects on democracy and, simultaneously, improve automatic detection of them. This study is based on two innovative methodological approaches: (1) dealing with bots using hybrid intelligence (HI), a multidisciplinary perspective that combines artificial intelligence (AI), natural language processing, political science, and communication science, and (2) applying framing theory to political bots. This paper contributes to the literature in the field by (a) applying framing to the analysis of political bots, (b) defining characteristics to identify signs of automation in Spanish, (c) building a Spanish-language bot database, (d) developing a specific classifier for Spanish-language accounts, (e) using HI to detect bots, and (f) developing tools that enable the everyday citizen to identify political bots through framing. Keywords: bots; framing; hybrid intelligence; empowerment; social media 1. Introduction No longer just a software mediator, bots have become important players in various political systems (Salge and Karahanna 2018;Lewis et al. 2019). Since 2010, political parties and governments have spent more than USD 500 million on research and development in this field (Bradshaw and Howard 2018). Their effect on election and referendum results increases each year, with formally organized social media manipulation campaigns in 48 countries, compared to 28 in the previous year (Bradshaw and Howard 2018), though they develop and exert influence differently depending on the context. Bots’ political influence continues to grow (Montal and Reich 2017) due to their ability to create artificial public opinion and turn non-existent or minority opinions into majority or dominant ones (Ross et al. 2019). The receivers of these messages are defenseless given that they are yet unable to distinguish real from fake speeches, which employ deception so as to conceal their nature (van der Kaa and Krahmer 2014;Waddell 2018;Wölker and Powell 2018;Kušen and Strembeck 2020). This research aims primarily to provide citizens with the knowledge necessary to create strategies for identifying political bots as a safeguard against their potentially negative impact on democracy, as a tool for achieving equality of opportunities in public debate for all political options, and to improve automated bot detection. We seek to do this by applying hybrid intelligence (HI), which will allow us to combine machine and human intelligence to overcome deficiencies in current artificial intelligence systems Soc. Sci. 2021,10, 357. https://doi.org/10.3390/socsci10100357 https://www.mdpi.com/journal/socsci Soc. Sci. 2021,10, 357 2 of 18 (Dellermann et al. 2019; Kamar 2016). The idea is to leverage a multidisciplinary approach through AI, natural language processing, political science, and communication sciences. We used HI to analyze the bots in Spain’s April 2019 elections, leveraging previous studies on automatic bot detection (Perdana et al. 2015;Morstatter et al. 2016;Ramalingam and Chinnaiah 2018;Gamallo and Almatarneh 2019), a novel idea that has developed a great deal in the past year (e.g., the HI4NLP workshop proposed by two of this paper’s authors and accepted by ECAI 2020, a referential European conference on artificial intelligence). Moreover, it has been applied in other contexts and fields, mostly to tackle problems with a clear social component or goal, like the one at hand in this paper. For example, it has been used in market analyses (Dellermann et al. 2019) or analyses of big data collected by sensors (e.g., smart cities), thanks to its focus on the Internet of Things (IoT) (Dellermann et al. 2017). Moreover, we leveraged framing theory, taking frames as a speech-constructing and processing strategy (Pan and Kosicki 1993) that emphasize certain aspects of reality to encourage the desired interpretation thereof (Gitlin 1980;Gamson and Modigliani 1989; Scheufele 1999;Entman and Usher 2018). We established four specific goals: 1. Develop Spain’s first account classifier and define characteristics to identify signs of automation. We collected and indexed a massive amount of text from Twitter, analyzing the political bots of the resulting corpus so as to determine their defining characteristics. We measured their effectiveness at the individual and aggregate levels, leveraging various sets of characteristics so as to find the most effective. Among the heuristics explored were those featured in previous studies on Twitter. 2. Compile Spain’s first bot database for Twitter. The database will allow for a subsequent analysis of bots’ presence in and influence on Spanish public opinion. 3. Determine the typical characteristics of political bots during political campaigns based on political bots’ profiles and tweets. 4. Analyze and develop tools for the public at large to identify bots without relying on automated machine detection. This article contributes to the literature by (a) applying framing to the analysis of political bots, (b) defining a set of characteristics so as to identify signs of automation in Spanish, (c) compiling a Spanish-language bot database, (d) developing a classifier for Spanish-language accounts, (e) applying HI to bot detection, (f) identifying key information for the public at large to identify political bots, and (g) improving automated detection of bots. Having set forth an introduction, we will now briefly review the relevant literature and describe the perspective we adopted in our research. Then, we will detail the methods and samples used, examine the set of features used by the bot classifier, discuss the results of the bot analysis, and propose a detection tool. Finally, we will discuss trends in the field, as well as the conclusions and limitations of our research. 2. Framework 2.1. Political Bots: Identification and Social Impact Previous research focused on creating techniques to automatically identify bots by analyzing messaging behavior in the Twitter ecosystem (Schuchard et al. 2019;Badawy et al. 2019;Lai et al. 2019;Wang et al. 2020;Zheng et al. 2019). Bots’ movement throughout Twitter was primarily analyzed so as to improve automatic detection systems. In keeping with Woolley and Howard (2016), political bots were defined as, “[ . . . ] the algorithms that operate over social media, written to learn from and mimic real people so as to manipulate public opinion across a diverse range of social media and device networks.” The most noteworthy results from previous research revolve around four central points: (a) influence on electoral processes, (b) functions, (c) taxonomies, and (d) regulation of political bots. Soc. Sci. 2021,10, 357 3 of 18 The literature indicates that between 5 and 25% of Twitter accounts are bots and that they are used more during election campaigns (Keller and Klinger 2019). Additionally, previous studies confirm political bots’ influence on electoral processes in distinct political systems and contexts (e.g., Mexico (Glowacki et al. 2018), Venezuela (Forelle et al. 2015), Chile (Santana and Cánepa 2019), Colombia (López Urrea et al. 2016), the United Kingdom (Murthy et al. 2016), the United States (Howard and Kollanyi 2017;Frey et al. 2018;Luceri et al. 2019), Ecuador (Puyosa 2017), France (Ferrara 2017), Argentina (Filer and Fredheim 2017), Spain (Campos-Domínguez and García-Orosa 2018), Russia (Sanovich n.d.), 2017). Other performed comparative analyses of different countries (Anelli et al. 2019). Still, developments in bots and the influence they exert have varied country to country, and some have even sought to promote the use of good bots (McKelvey and Dubois 2017). Political parties and other political players can track personal data to send campaign ads, determine the ideological makeup of their potential voters, and send personalized messages adapted to voters’ needs in real time with no need for human intervention. Algorithms can reveal the public’s state of mind, opinion, location, ideology, and needs. Moreover, they can be used to build and send in real-time messages designed to support the sender’s positions and influence each and every type of voter. Bots can even hold conversations with people or amongst themselves. Here, the literature describes astroturfing as a system for creating fake public opinion, highlighting several bot actions (Treré2016; Bastos and Mercea 2017): (a) pro-government ads, (b) creating fake opinion leaders, (c) delegitimizing systems of government, (d) supporting opposition groups, (e) empowering the public, (f) establishing political agendas and debates, and (g) weakening political dissent. Taxonomies revolve around the bots’ dynamics (Dagon et al. 2008;Chu et al. 2010; McKelvey and Dubois 2017) or devise specific categories for corpus analysis, as in Stukal et al. (2019), where bots were categorized as pro-regime, anti-regime, or neutral. One of the most thorough taxonomies includes various characteristics: professional news content, professional political content, polarizing and conspiracy content, and other policy news and information (Machado et al. 2018). The detection of a large number of bots spreading false information and polarizing the political conversation (Bessi and Ferrara 2016) gave rise to various initiatives, an evaluation of changes to legislation, and proposals for intervention. Though some criticize regulation for potentially limiting freedom of speech (Lamo and Calo 2019), many countries and international organizations have recently developed such rules. The European Union warned of the threat to democracy if political parties generate automated messages adapted to the needs of each person based on big data analysis, which may even be manipulated by fake news (European Commission 2018;European Parliament 2017). Additionally, some countries have enacted laws to regulate artificial intelligence and computational propaganda (Italy in 2014, France in 2016, the UK in 2017). In November 2018, the European Commission adopted the Action Plan against Disinformation to minimize disinformation in European elections. 2.2. Our Approaches: Hybrid Intelligence Our paper uses this context as a starting point, understanding that the sophistication of bots necessitates the development of mixed analysis methods that combine statistical methods with social sciences, which we seek to achieve through HI. Natural language processing and new AI techniques in the field of machine/deep learning (ML/DL) have been used in recent years to detect bots at the account level, processing a large number of social network posts and leveraging information on the network’s structure, temporal dynamics, and sentiment analysis, and even using neural networks in large compilations of text data (Kudugunta and Ferrara 2018;Stukal et al. 2019). Nonetheless, we believe that bots’ increasing complexity necessitates mixed methodologies that combine expert human knowledge with ML/DL and NLP systems. Soc. Sci. 2021,10, 357 4 of 18 This approach will allow us to feed the classifier with new information that will improve and allow for immediate detection of political bots, and will also allow individual users to do so, despite them not typically having access to such technological tools nor the ability to visualize a large number of bots. We tackle bot detection through framing not only because it can connect with the messages’ propagation dynamics and their cascade influence, but also because it is a model that successfully maps out the complexity of thought by paying attention to the elements of greatest significance in a given communicative context. We assume that frames play an important role in public opinion and that bots can rapidly spread frames (tell people what to think about) so as to, as Entman (2010) points out, monitor public attitudes to influence people’s behavior. This situation, combined with the spiral of silence, could become a powerful weapon for hiding opinions not just because they are a minority position but because they differ from those proposed by bots. Moreover, according to Chong and Druckman (2007), bots have two of the three frame-strengthening elements: frequency, accessibility, and relevance. The influence bots may exert through frame-spreading is portrayed in the following Figure 1. Soc. Sci. 2021, 10, x FOR PEER REVIEW 4 of 19 text data (Kudugunta and Ferrara 2018; Stukal et al. 2019). Nonetheless, we believe that bots’ increasing complexity necessitates mixed methodologies that combine expert human knowledge with ML/DL and NLP systems. This approach will allow us to feed the classifier with new information that will improve and allow for immediate detection of political bots, and will also allow individual users to do so, despite them not typically having access to such technological tools nor the ability to visualize a large number of bots. We tackle bot detection through framing not only because it can connect with the messages’ propagation dynamics and their cascade influence, but also because it is a model that successfully maps out the complexity of thought by paying attention to the elements of greatest significance in a given communicative context. We assume that frames play an important role in public opinion and that bots can rapidly spread frames (tell people what to think about) so as to, as Entman (2010) points out, monitor public attitudes to influence people’s behavior. This situation, combined with the spiral of silence, could become a powerful weapon for hiding opinions not just because they are a minority position but because they differ from those proposed by bots. Moreover, according to Chong and Druckman (2007), bots have two of the three framestrengthening elements: frequency, accessibility, and relevance. The influence bots may exert through frame-spreading is portrayed in the following Figure 1. Figure 1. Source: created by authors. Although bots may have initially been geared towards increasing a politician’s nonexistent popularity, they are not typically used to spread frames that tend to receivers’ needs. A frame’s success partially revolves around awareness and adaptation of pre-existing frames. As seen in the diagram, this is one of bots’ strengths: Namely, the user imbues the bots with data analysis, allowing them to incorporate preexisting frames. If the spiral of silence were applied to political bots, a bot creator could quickly spread new frames consistent with receivers’ preexisting frames so as to transform those initially rare or nonexistent frames into dominant ones. Here we turn to the conceptualization and operationalization of the frame analysis, as well as to the trends in social network frames that underpin the tools used in our paper’s empirical analysis. Frames are a key concept in political communication, though they have been used in other fields as well (Bateson 2002; Goffman 1974). Frames can carry out four functions: defining problems, interpreting causes, moral judgments, and recommendations for treatment (Entman 2003). Their success is linked to interaction with individuals’ pre-existing schema, pre-existing frames, and current information (Entman and Usher 2018). The framing approach presents distinct typologies (Matthes 2009). For our purposes, the relevant typology is that which distinguishes between specific and generic frames (De Vreese 2005). Whereas specific frames revolve around specific topics or isolated events, generic frames transcend thematic limitations because the same issue can be identified in different contexts (De Vreese 2002). A great deal of research has applied such classifications: specific frames in Matthes (2009), Lengauer and Höller (2013), Hänggli and Kriesi (2010), Sheafer and Gabay (2009), Zhou and Moy (2007), and Matthes (2009), and generic Figure 1. Source: created by authors. Although bots may have initially been geared towards increasing a politician’s nonexistent popularity, they are not typically used to spread frames that tend to receivers’ needs. A frame’s success partially revolves around awareness and adaptation of preexisting frames. As seen in the diagram, this is one of bots’ strengths: Namely, the user imbues the bots with data analysis, allowing them to incorporate preexisting frames. If the spiral of silence were applied to political bots, a bot creator could quickly spread new frames consistent with receivers’ preexisting frames so as to transform those initially rare or nonexistent frames into dominant ones. Here we turn to the conceptualization and operationalization of the frame analysis, as well as to the trends in social network frames that underpin the tools used in our paper’s empirical analysis. Frames are a key concept in political communication, though they have been used in other fields as well (Bateson 2002;Goffman 1974). Frames can carry out four functions: defining problems, interpreting causes, moral judgments, and recommendations for treatment (Entman 2003). Their success is linked to interaction with individuals’ pre-existing schema, pre-existing frames, and current information (Entman and Usher 2018). The framing approach presents distinct typologies (Matthes 2009). For our purposes, the relevant typology is that which distinguishes between specific and generic frames (De Vreese 2005) . Whereas specific frames revolve around specific topics or isolated events, generic frames transcend thematic limitations because the same issue can be identified in different contexts (De Vreese 2002). A great deal of research has applied such classifications: specific frames in Matthes (2009), Lengauer and Höller (2013), Hänggli and Kriesi (2010), Sheafer and Gabay (2009), Zhou and Moy (2007), and Matthes (2009), and generic frames in Aalberg et al. (2012), Iyengar (1991), Semetko and Valkenburg (2000), Neuman et al. (1992), and Lengauer and Höller (2013). In keeping with Aalberg et al.’s (2012) work on generic frames, we dealt with game frames and strategic frames. Game frames take politics as a game that consists of winning Soc. Sci. 2021,10, 357 5 of 18 or losing, employs bellicose language, and emphasizes polling data. They typically focus on politics in general, legislative debates, the winners and losers of an election, and the battle for public opinion (polls), and speculate on election or political results and potential coalitions. The strategic frame refers to campaign strategies and tactics, motives and instrumental actions, personality, style, and metacoverage. It also includes different media strategies, including news that covers the press’s behavior. The first generation of research demonstrated that frames can affect public opinion on a wide range of issues (Aarøe 2011), though not to the same degree (Entman 2003;Aarøe 2011;Chong and Druckman 2007). Public actors compete to strengthen certain frames and several factors determine their success or failure in spreading new frames (Chong and Druckman 2007;Hänggli and Kriesi 2012). Given social networks’ consolidation as a form of political communication, framing processes, as well as the processes of information production and consumption, among others, need to be reevaluated (Entman and Usher 2018). Digitalization of online framing processes has significantly influenced interpersonal, family, and organizational communication and increased opportunities for extremism and balkanization (Entman and Usher 2018). Initial studies applied framing theory to blogs and websites (Bichard 2006;Goldman and Kuypers 2010). To date, many framing-based studies of social networks have considered frame propagation dynamics in distinct but inter-related communication ecosystems (Wasike 2013;Aruguete and Calvo 2018). Twitter has become a major political communication tool, especially during campaign seasons when parties and candidates use it to provide information about their campaign and its events or to link to their website (Jungherr 2016). Previous studies affirmed Twitter’s ability to get out the vote and increase civic engagement (Gainous and Wagner 2014) and to change political commitment (Lee and Park 2013;Grˇci´c et al. 2017). Academic research highlights bi-directionality, interactivity, the capacity for dialogue, and even promoting a participatory, deliberative democracy. More importantly, Twitter is the appropriate tool for politization given its ability to disseminate and cause messages to circulate, in addition to its fostering of public engagement (Utz et al. 2013;Abitbol and Lee 2017;Ji et al. 2018; Painter 2015). Additionally, our paper considers Twitter a shared communicative space in which democratic deliberation could arise and each political actor could defend their positions. Moreover, we understand that for this to happen, an equal playing field is an important premise. Bots violate that condition by giving too much power to too few. Given that platforms refuse to take responsibility for this situation, the frames spread therein, and the beneficiaries (Entman and Usher 2018), we propose empowering the public to detect bots, promoting horizontal surveillance put into practice by people themselves in a somewhat planned way through cell phones and other devices that allow for reporting and sharing political actions with which they disagree. Our research aims to foster public detection of political bots, whose messages are often indistinguishable from those created by humans. Indeed, much of their success lies in this concealment. 3. Materials and Methods We proposed a multi-disciplinary approach leveraging artificial intelligence, natural language processing, political science, and communication science. We applied hybrid intelligence to our analysis of political bots from Spain’s April 2019 election, referencing previous studies on automatic bot detection. Below we summarize and explain our extraction methods and our analysis of tweets (Appendix A): Step 1. Crawler Design. The Polypus Twitter crawler (Martínez-Castaño et al. 2018b) retrieves tweets in real time from Twitter’s public and anonymous Search API, used by its official web client. The tweets are formatted in HTML so they can be directly inserted in the site’s feed. The API limits the number of returned tweets by returning a sample of the total that matches a Soc. Sci. 2021,10, 357 6 of 18 given query. Any user of the web interface is expected to receive the same set of tweets when executing the same query within the same time windows. The high-level architecture of the system is shown in Figure 2. The crawler can be configured both for targeted and untargeted searches. The number of threads is configurable so that the queries are equally distributed and processed in parallel. In addition, multiple instances of the crawler can be executed on different machines so that the crawler scales horizontally. To avoid repeated tweets, a distributed memory-based key-value store is used to store the known identifiers for several days. Soc. Sci. 2021, 10, x FOR PEER REVIEW 6 of 19 The Polypus Twitter crawler (Martínez-Castaño et al. 2018b) retrieves tweets in real time from Twitter’s public and anonymous Search API, used by its official web client. The tweets are formatted in HTML so they can be directly inserted in the site’s feed. The API limits the number of returned tweets by returning a sample of the total that matches a given query. Any user of the web interface is expected to receive the same set of tweets when executing the same query within the same time windows. The high-level architecture of the system is shown in Figure 2. The crawler can be configured both for targeted and untargeted searches. The number of threads is configurable so that the queries are equally distributed and processed in parallel. In addition, multiple instances of the crawler can be executed on different machines so that the crawler scales horizontally. To avoid repeated tweets, a distributed memory-based key-value store is used to store the known identifiers for several days. Figure 2. High-level Twitter crawler architecture. The list of queries can be set as the list of the most frequent terms or expressions in a set of languages. With this strategy, the crawler can retrieve huge amounts of tweets without any specific target. There is not a linear relation with the available resources (due to the aforementioned limitations of Twitter’s API). However, for specific targets such as in this study, the extraction can practically match the actual production of tweets through the use of specific terms, hashtags, or Twitter accounts. Dynamic data (e.g., retweets, number of replies) are collected afterwards since the tweets are extracted in real time and these attributes do not offer useful information about the users’ interaction initially. Polypus’s Twitter crawler is now part of a set of social media crawlers integrated into Catenae (Martínez-Castaño et al. 2018a, 2018c), a Python framework for easy design, development, and deployment of stream processing applications with Docker containers1. Step 2. Classifier features. Extraction parameters (explained below). Step 3. Discourse analysis through framing. Tweet content analysis parameters (explained below). Crawler Threads IDs Search Queries HTML Cod e Twitter Anonymous Search API KV Store IDs Tweets Tweets Target Queries Figure 2. High-level Twitter crawler architecture. The list of queries can be set as the list of the most frequent terms or expressions in a set of languages. With this strategy, the crawler can retrieve huge amounts of tweets without any specific target. There is not a linear relation with the available resources (due to the aforementioned limitations of Twitter’s API). However, for specific targets such as in this study, the extraction can practically match the actual production of tweets through the use of specific terms, hashtags, or Twitter accounts. Dynamic data (e.g., retweets, number of replies) are collected afterwards since the tweets are extracted in real time and these attributes do not offer useful information about the users’ interaction initially. Polypus’s Twitter crawler is now part of a set of social media crawlers integrated into Catenae (Martínez-Castaño et al. 2018a,2018c), a Python framework for easy design, development, and deployment of stream processing applications with Docker containers 1 . Step 2. Classifier features. Extraction parameters (explained below). Step 3. Discourse analysis through framing. Tweet content analysis parameters (explained below). The sample was in keeping with those of other studies (Hedman et al. 2018;Schäfer et al. 2017). In addition to the campaign season, it also included the period between April 15 and May 5, with 575 candidate or political party accounts, as well as the hashtags promoted by those parties. The sample yielded the following Table 1data: Soc. Sci. 2021,10, 357 7 of 18 Table 1. Capture time interval: [2019/04/16 10:15:00 UTC, 2019/05/09 10:59:19 UTC]. Unique users: 1,036,920 Tweets: 4,547,482 Tweets plus retweets: 22,296,826 Additionally, it combined analysis of parties’ and candidates’ accounts, as well as hashtags as we can see in the Table 2. Table 2. Accounts, hashtags, and terms. Accounts, Hashtags and Terms Accounts @PSOE, @PPopular, @ahorapodemos, @CiudadanosCs, @eajpnv, @JuntsXCat, @compromis, @vox_es, @navarra_suma, @ForoAsturias, @coalicion@Esquerra_ERC, @ehbildu, @Nueva_Canarias, @sanchezcastejon, @pablocasado, @Pablo_Iglesias, @Albert_Rivera, @Aitor_Esteban, @jordialapreso, @LauraBorras, @joanbaldovi, @junqueras, @gabrielrufian, @sergiosayas, @PedroQuevedoIt, @anioramas, @Santi_ABASCAL, @meritxell_batet, @InesArrimadas, @cayetanaAT @Jaumeasens Hashtags #elecciones2019, #debates, #eleccionesgenerales2019, #28deabril, #eleccionesgenerales28A, #LaEspañaQueQuieres, #HazQuePase, #ValorSeguro, #VamosCiudadanos, #PorEspaña, #Perotampocoteconformes, #ahorapodemos Terms PSOE, PP, Podemos, Ciudadanos, PNV, Junts per Catalunya, Junts, Compromís, Navarra Suma, Partido Popular, Coalición Canaria, Esquerra Republicana, EH-Bildu, Nueva Canarias, Vox, Pedro Sánchez, Pablo Casado, Pablo Iglesias, Albert Rivera, Aitor Esteban, Jordi Sánchez, Laura Borrás, Joan Baldoví, Paloma Gázquez, Oriol Junqueras, Gabriel Rufián, Sergio Sayas, Garazi Dorronsoro, Pedro Quevedo, Ana Oramas, Santiago Abascal, Meritxell Batet, Inés Arrimadas, Cayetana Álvarez, Jaume Asens We manually classified frames based on a sample of 50 accounts identified as bots and selected randomly, using the following categories: message dynamics, interaction on the network (links, retweets, replies), topics, goals, level of message repetition, end game, frequency, and frames, divided along the following lines: (a) Structural level (syntactic and communicative). At the communicative level, we analyzed feedback in the network, the development of threads and references to previous messages, the use of denotative language, irony and double entendre, and connection to offline messages. (b) Content level (framing). At the content level, we pinpointed three components: number of frames, issue-specific frames, and generic frames. Lastly, we evaluated to what extent each of these categories can be automated. For this paper’s context, we chose Spain’s 28 April 2019 elections, which are particularly relevant for several reasons. First is the 74.65% voter participation, much higher than that of the 2016 elections won by the Spanish Socialist Workers’ Party (PSOE in Spanish), at a time in which Spain’s two-party system was splintering due to the emergence of two new parties (Podemos and Ciudadanos). Second, VOX, following its performance in the province of Andalusia’s regional elections, could be anticipated to capture seats in the national parliament, as well. Our study was warranted, moreover, because in Spain we had yet to recognize the extent of computational propaganda given there was only one exploratory study (Campos-Domínguez and García-Orosa 2018) , which indicated that in 2019 the political algorithm would finally take hold in the country. With this phenomenon having advanced in recent years, it became necessary to undertake a multi-disciplinary study in Spain that further analyzed the situation and consequences for digital society and the electoral processes of 2019 and 2020. Soc. Sci. 2021,10, 357 8 of 18 4. Features of the Classifier Feature selection is a critical process for classification tasks that rely on traditional machine learning. In this section, we describe the different types of features we used to design and implement a hybrid classifier, leveraging a model trained from annotated datasets and some generic heuristics determined based on prior knowledge formalized by experts in the domain. The system is therefore based on the HI paradigm, since it hybridizes automatic learning with information from experts. 4.1. Features In order to train a classifier, we defined three types of features: social network, contentbased, and lexical features. 4.1.1. Social Network Features These are specific characteristics of the language used in social networks, consisting of textual elements that can only be found on Twitter: • Ratio of the number of hashtags, i.e., number of hashtags used by a user account divided by total number of tweets sent from that account; •Ratio of the number of retweets; •Ratio of the number of URL links; •Ratio of the number of user references; •Ratio of the number of emojis; •Ratio of the number of textual emoticons; •Ratio of the number of onomatopoeias, e.g., “haha” in English or jeje in Spanish. • Ratio of the number of language abbreviations, e.g., “b4” (before) or “btw” (by the way) in English, and “q” (que) or “xq” (porque), in Spanish; •Ratio of the number of alliterations, e.g., repetition of vowel sounds. 4.1.2. Content-Based Features These are features that can be extracted from any text message: •Ratio of the size of tweets; •Ratio of the number of identical pairs of tweets; •Lexical richness, defined as lemma/token ratio; • Similarity between sequential pairs of tweets. To obtain the final similarity ratio associated with a user account, all similarity scores between pairs of sequential tweets are added, and the result is divided by the total number of tweets. These content-based features were created with just lexical words (i.e., nouns, adjectives, verbs, and adverbs) by making use of PoS tagging so as to identify them. 4.1.3. Lexical Features Lexical features were derived from several domain-specific lexicons; in particular, two different weighted lexicons were automatically built for each language: • A human/bot lexicon consisting of specific words belonging to two classes: the language of bots and the language of humans in Twitter; • A sentiment lexicon consisting of polarity words (positive or negative) used by bots or humans. Each lexicon was built by making use of the annotated corpora provided by the PAN Shared Task organizers and a ranking algorithm defined in Almatarneh and Gamallo (2018). As in the case of content-based features, only words tagged as nouns, verbs, adjectives, and adverbs were considered. In order to find the best feature configuration in a classification task, we used a Bayesian algorithm. In addition to its simplicity and efficiency, Naive Bayes performs well in this type of task, as described in Alarifi et al. (2016), where the Bayesian classifier obtained the best results in the bot/human classification. Our classifier was implemented Soc. Sci. 2021,10, 357 9 of 18 with the Naïve Bayes Perl module (https://metacpan.org/pod/Algorithm::NaiveBayes, accessed on 30 June 2021) In order to lemmatize and identify lexical PoS tags, tweets were processed using the multilingual toolkit LinguaKit (Gamallo et al. 2018). The classifier was trained with the dataset provided by the PAN Shared Task. 4.2. Heuristics Our hybrid approach features a system with two modules: a rule-based module consisting of generic heuristics defined with expert knowledge and the Bayes classifier developed with the features described above. The generic heuristics are applied before the Bayes classifier. The generic heuristics use some of the features defined above; for instance, a user is considered a bot if the similarity of its tweets is above a given threshold, or if the number of hashtags and user references is very high yet the lexical richness is very low. Thresholds were set empirically. Preliminary results obtained with the PAN dataset collection (Rangel and Rosso 2019) showed that the hybrid approach, with rules and a statistical classifier, works slightly better than using just the rules or the classifier. 5. Analysis Based on our data and the referenced literature, we created a set of characteristics for political bots during election campaigns, designed a strategy for identifying them, and indicated which ones are likely to be automated and included in our classifier, in addition to evaluating their effects on the discourse. In the analyzed election campaign, the bot:non-bot ratio was 0.063%, and bots sent 1.903% of messages. Nonetheless, despite these low percentages, bots were highly active and tweeted an average of 132.30 times, compared to the 4.3 tweets of the average human account. Likewise, the average bot account tweeted 6.30 times per day, compared to 4.31 daily tweets by the non-bot accounts. Political bots were flexible and fast but rarely interacted with previous messages and elicited little interaction from other users (10.96% of posts received likes, 9.95% were retweeted, and none received replies, n= 9466 messages). Those that elicited likes typically received one per post, except during particularly active periods of message repetition, like the “EquiparacionYa” (a protest against the gender pay gap in Spain’s security forces) and “Talidomida” campaigns (referencing those harmed by thalidomide, a pharmaceutical drug developed by the German company Grunenthal GmbH, sold in Spain between 1957 and 1963 as a sedative and nausea suppressor and that caused thousands of birth defects). The campaign featured, among others, the following messages: #ElDebateDecisivo #ILPJusapol @jusapol @PSOE @populares @ahorapodemos @CiudadanosCs @vox_es @europapress @EFEnoticias//Los talidomidicos hacen público su voto. Comparte 2 . #Avite #talidomida #28A #28Abril #CampañaElectoral #EleccionesGenerales #YoVotoGrunenthal #28AbrilElecciones #EleccionesGenerales2019 #Elecciones2019 #LaEspañaQueQuieres #110compromisosPSOE https://youtu.be/klCrtCJBkwQ(accessed on 30 June 2021). Such automated political messages tended to be part of synchronized, planned, goaloriented mediated campaigns that featured high concentrations of messages during a short period (for example, intense criticism of another party’s leader based on a specific act during a short period). Thus, we detected high-frequency tweets concentrated in a specific time interval and normally with a specific common goal. Such was the case in a relatively short time interval with the identified-as-bot account <user name=“jucilcantabria”>. The aforementioned account sent the following messages: El nombre es lo de menos, JUSAPOL SOMOS TODOS Estamos en cada rincón de este país y ¡No vamos a parar! #ILPJusapol #EquiparacionYa and similar retweets:// #EquiparacionYa #ILPJusapol @jusapol, eliciting a great number of retweets and likes. 3 Soc. Sci. 2021,10, 357 16 of 18 European Commission. 2018. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. Tackling Online Disinformation: A European Approach. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52018DC0236 (accessed on 30 June 2021). European Parliament. 2017. European Parliament Resolution of 14 March 2017 on Fundamental Rights Implications of Big Data: Privacy, Data Protection, Non-Discrimination, Security and Law-Enforcement. Available online: https://www.europarl.europa. eu/doceo/document/TA-8-2017-0076_EN.html (accessed on 30 June 2021). Ferrara, Emilio. 2017. Disinformation and Social Bots Operations in the Run Up to the 2017 French Presidential Election. arXiv arXiv:1707.00086. Filer, Tanya, and Rolf Fredheim. 2017. Popular with the Robots: Accusation and Automation in the Argentine Presidential Elections, 2015. International Journal of Politics, Culture and Society 30: 259–74. [CrossRef] Forelle, Michelle, Phillip Howard, Andres Monroy-Hernandez, and Saiph Savage. 2015. Political Bots and the Manipulation of Public Opinion in Venezuela. Available online: https://doi.org/10.2139/ssrn.2635800 (accessed on 30 June 2021). Frey, Carl Benedikt, Thor Berger, and Chinchih Chen. 2018. Political machinery: Did robots swing the 2016 US presidential election? Oxford Review of Economic Policy 34: 418–42. [CrossRef] Gainous, Jason, and Kevin Wagner. 2014. Tweeting to Power: The Social Media Revolution in American Politics. Public Opinion Quarterly 78: 1026–28. [CrossRef] Gamallo, Pablo, Marcos Garcia, César Piñeiro, Rodrigo Martínez-Castaño, and Juan Pichel. 2018. LinguaKit: A Big Data-Based Multilingual Tool for Linguistic Analysis and Information Extraction. Paper presented at the 2018 Fifth International Conference on Social Networks Analysis, Management and Security (SNAMS), Valencia, Spain, October 15–18. Gamallo, Pablo, and Sattam Almatarneh. 2019. Naive-Bayesian Classification for Bot Detection in Twitter. Paper presented at the Conference and Labs of the Evaluation Forum 2019, Lugano, Switzerland, September 9–12. Gamson, William, and Andre Modigliani. 1989. Media discourse and public opinion on nuclear power: A constructionist approach. American Journal of Sociology 95: 1–35. [CrossRef] García-Orosa, Berta, Pablo Vázquez-Sande, and XoséLópez-García. 2017. Digital narratives of the major political parties of Spain, France, Portugal and the United States. El Profesional de La Información26: 4. [CrossRef] Gitlin, Todd. 1980. The Whole World Is Watching: Mass Media in the Makin and unmaking of the New Left. Berkeley: University of California Press. Glowacki, Monika, Vidya Narayanan, Sam Maynard, Gustavo Hirsch, Bence Kollanyi, Lisa-Maria Neudert, Phil Howard, Thomas Lederer, and Vlad Barash. 2018. News and Political Information Consumption in Mexico: Mapping the 2018 Mexican Presidential Election on Twitter and Facebook. pp. 1–6. Available online: https://demtech.oii.ox.ac.uk/wp-content/uploads/sites/93/2018/ 06/Mexico2018.pdf (accessed on 30 June 2021). Goffman, Erving. 1974. Frame Analysis: An Essay on the Organization of Experience. Cambridge: Harvard University Press. Goldman, Adria, and Jim Kuypers. 2010. Contrasts in News Coverage. Relevant Rethoric: A New Journal of Rhetorical Studies 1: 1–18. Grˇci´c, Klara, Marina Bagic Babac, and Vedran Podobnik. 2017. Generating politician profiles based on content analysis of social network datasets. Journal of Universal Computer Science 23: 236–55. Hagen, Loni, Stephen Neely, Thomas E. Keller, Ryan Scharf, and Fatima Espinoza Vasquez. 2020. Rise of the Machines? Examining the Influence of Social Bots on a Political Discussion Network. Social Science Computer Review. [CrossRef] Hänggli, Regula, and Hanspeter Kriesi. 2010. Political framing strategies and their impact on media framing in a swiss direct-democratic campaign. Political Communication 27: 141–57. [CrossRef] Hänggli, Regula, and Hanspeter Kriesi. 2012. Frame Construction and Frame Promotion (Strategic Framing Choices) Title. American Behavioral Scientist 56: 9. [CrossRef] Hedman, Freja, Fabian Sivnert, Bence Kollanyi, Vidya Narayanan, Lisa-Maria Neudert, and Philip Howard. 2018. News and Political Information Consumption in Sweden: Mapping the 2018 Swedish General Election on Twitter. Oxford: Oxford Internet Institute. Howard, Philip, and Bence Kollanyi. 2017. Bots, #Strongerin, and #Brexit: Computational Propaganda During the UK-EU Referendum. Available online: https://doi.org/10.2139/ssrn.2798311 (accessed on 30 June 2021). Iyengar, Shanto. 1991. Is Anyone Responsible? How Television Frames Political Issues. Chicago: University of Chicago Press. Ji, Yi Grace, Zifei Fay Chen, Weiting Tao, and Zongchao Cathy Li. 2018. Functional and emotional traits of corporate social media message strategies: Behavioral insights from SandP 500 Facebook data. Public Relations Review 45: 88–103. [CrossRef] Jungherr, Andreas. 2016. Twitter use in election campaigns: A systematic literature reviewNo Title. Journal of Information Technology and Politics 13: 72–91. [CrossRef] Kamar, Ece. 2016. Direction in Hybrid Intelligence: Complementing AI System with Human Intelligence. Paper presented at the 25th International Joint Conference on Artificial Intelligence, New York, NY, USA, July 9–15; pp. 4070–73. Keller, Tobias, and Ulrike Klinger. 2019. Social Bots in Election Campaigns: Theoretical, Empirical, and Methodological Implications. Political Communication 36: 171–89. [CrossRef] Kudugunta, Sneha, and Emilio Ferrara. 2018. Deep Neural Networks for Bot Detection. Information Sciences 467: 312–22. [CrossRef] Kušen, Ema, and Mark Strembeck. 2020. You talkin’ to me? Exploring Human/Bot Communication Patterns during Riot Events. Information Processing and Management 57: 102126. [CrossRef] Lai, Mirko, Marcella Tambuscio, Viviana Patti, Giancarlo Ruffo, and Paolo Rosso. 2019. Stance polarity in political debates: A diachronic perspective of network homophily and conversations on Twitter. Data and Knowledge Engineering 124: 101738. [CrossRef] Soc. Sci. 2021,10, 357 17 of 18 Lamo, Madeline, and Ryan Calo. 2019. Regulating Bot Speech (July 16, 2018). UCLA Law Review. Available online: https: //ssrn.com/abstract=3214572 (accessed on 30 June 2021). Lee, Hyunmin, and Hyojung Park. 2013. Testing the impact of message interactivity on relationship management and organizational reputation. Journal of Public Relations Research 25: 188–206. [CrossRef] Lengauer, Günter, and Iris Höller. 2013. Generic frame building in the 2008 Austrian elections. Public Relations Review 39: 303–14. [CrossRef] Lewis, Seth C., Andrea Guzman, and Thomas Schmidt. 2019. Automation, Journalism, and Human–Machine Communication: Rethinking Roles and Relationships of Humans and Machines in News. Digital Journalism 7: 409–27. [CrossRef] López Urrea, Laura María, Julián Enrique Páez Valdez, and Arlex Darwin Cuellar Rodríguez. 2016. El Discurso Político Mediado Por Ordenadores. Revista Nexus Comunicación19: 110–29. [CrossRef] Luceri, Luca, Ashok Badawy, Adam Deb, and Emilio Ferrara. 2019. Red bots do it better: Comparative analysis of social bot partisan behavior. Paper presented at the Web Conference 2019—Companion of the World Wide Web Conference (WWW 2019), San Francisco, CA, USA, May 13–17; pp. 1007–12. [CrossRef] Machado, Caio, Beatriz Kira, Gustavo Hirsch, Nahema Marchal, Bence Kollanyi, Philip Howard, Thomas Lederer, and Vlad Barash. 2018. News and Political Information Consumption in Brazil: Mapping the First Round of the 2018 Brazilian Presidential Election on Twitter. Comprop Data Memo 2018.4/October 5. pp. 1–7. Available online: https://demtech.oii.ox.ac.uk/wp-content/ uploads/sites/93/2018/10/machado_et_al.pdf (accessed on 30 June 2021). Martínez-Castaño, Rodrigo, Juan Pichel, and David Losada. 2018a. Building python-based topologies for massive processing of social media data in real time. Paper presented at the 5th Spanish Conference on Information Retrieva, Zaragoza, Spain, June 25–27; pp. 1–8. Martínez-Castaño, Rodrigo, Juan Pichel, and Pablo Gamallo. 2018b. Polypus: A Big Data Self-Deployable Architecture for Microblogging Text Extraction and Real-Time Sentiment Analysis. arXiv arXiv:1801.03710. Martínez-Castaño, Rodrigo, Juan Pichel, David Losada, and Fabio Crestani. 2018c. A Micromodule Approach for Building Real-Time Systems with Python-Based Models: Application to Early Risk Detection of Depression on Social MediaTitle. Paper presented at the European Conference on Information Retrieval, Grenoble, France, March 26–29; pp. 801–5. Matthes, Jörg. 2009. What’s in a frame? A content analysis of media framing studies in the world’s leading communication journals, 1990-2005. Journalism and Mass Communication Quarterly 86: 349–67. [CrossRef] McKelvey, F., and E. Dubois. 2017. Computational Propaganda in Canada: The Use of Political Bots. Working Paper No. 2017.6. Oxford: University of Oxford. Montal, Tal, and Zvi Reich. 2017. I, Robot. You, Journalist. Who is the Author? Authorship, bylines and full disclosure in automated journalism. Digital Journalism 5: 829–49. [CrossRef] Morstatter, Fred, Liang Wu, Tahora Nazer, Kathlenn Carley, and Huan Liu. 2016. A New Approach to Bot Detection. Paper presented at the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), San Francisco, CA, USA, August 18–21; pp. 533–40. Murthy, Dhiraj, Alison Powell, Ramine Tinati, Nick Anstead, Leslie Carr, Susan Halford, and Mark Weal. 2016. Bots and political influence: A sociotechnical investigation of social network capital. International Journal of Communication 10: 4952–71. Neuman, Russell, Marion Just, and Ann Crigler. 1992. News and the Construction of Political Meaning. Chicago: The University of Chicago. Painter, David Lynn. 2015. Online Political Public Relations and Trust: Source and Interactivity Effects in the 2012 US Presidential Campaign. Public Relations Review 41: 801–8. [CrossRef] Pan, Zhongdang, and Gerald Kosicki. 1993. Framing analysis: An approach to news discourse. Political Communication 10: 55–75. [CrossRef] Pedersen, Ramus Tue. 2012. The game frame and political efficacy: Beyond the spiral of cynicism. European Journal of Communication 27: 225–40. [CrossRef] Perdana, Rizal Setya, Tri Hadiah Muliawati, and Reddy Alexandro. 2015. Bot Spammer Detection in Twitter Using Tweet Similarity and Time Interval Entropy. Jurnal Ilmu Komputer Dan Informasi 8: 19. [CrossRef] Puyosa, Iria. 2017. Political Bots on Twitter in #Ecuador2017 Presidential Campaigns. Contratexto 27: 39–60. [CrossRef] Ramalingam, Devakunchari, and Valliyammai Chinnaiah. 2018. Fake profile detection techniques in large-scale online social networks: A comprehensive review. Computers and Electrical Engineering 65: 165–77. [CrossRef] Rangel, Francisco, and Paolo Rosso. 2019. Overview of the 7th author profiling task at PAN 2019: Bots and gender profiling in twitter. Paper presented at the CEUR Workshop Proceedings, Lugano, Switzerland, September 9–12. Ross, Björn, Laura Pilz, Benjamin Cabrera, Florian Brachten, German Neubaum, and S. Stieglitz. 2019. Are social bots a real threat? An agent-based model of the spiral of silence to analyse the impact of manipulative actors in social networks. European Journal of Information Systems 28: 394–412. [CrossRef] Salge, Carolina Alves de Lima, and Elena Karahanna. 2018. Protesting Corruption on Twitter: Is It a Bot or Is It a Person? Academy of Management Discoveries 4: 32–49. [CrossRef] Sanovich, Sergey. n.d. Computational Propaganda in Russia: The Origins of Digital Misinformation. Oxford: The University of Oxford. Santana, Luis, and Gonzalo Huerta Cánepa. 2019. ¿Son bots? Automatización en redes sociales durante las elecciones presidenciales de Chile 2017. Cuadernos.info 44: 61–77. [CrossRef] Soc. Sci. 2021,10, 357 18 of 18 Schäfer, Fabian, Evert Stefan, and Philip Heinrich. 2017. Japan’s 2014 general election: Political bots, right-wing internet activism, and prime minister Shinz¯ o Abe’s hidden nationalist agenda. Big Data 5: 294–309. [CrossRef] Scheufele, Dietram. 1999. Framing as a theory of media effects. Journal of Communication 49: 103–22. [CrossRef] Schuchard, Ross, Andrew Crooks, Anthony Stefanidis, and Arie Croitoru. 2019. Bot stamina: Examining the influence and staying power of bots in online social networks. Applied Network Science 4: 55. [CrossRef] Semetko, Holli, and Patti Valkenburg. 2000. Framing European Politics: A content analysis of press and television news. JOurnal of Communication 50: 93–109. [CrossRef] Sheafer, Tamir, and Itay Gabay. 2009. Mediated public diplomacy: A strategic contest over international agenda building and frame building. Political Communication 26: 447–67. [CrossRef] Stukal, Denis, Sergey Sanovich, Joshua Tucker, and Richard Bonneau. 2019. For Whom the Bot Tolls: A Neural Networks Approach to Measuring Political Orientation of Twitter Bots in Russia. SAGE Open 9: 2158244019827715. [CrossRef] Treré, Emiliano. 2016. The dark side of digital politics: Understanding the algorithmic manufacturing of consent and the hindering of online dissidence. IDS Bulletin 47: 127–38. [CrossRef] Utz, Sonja, Schultz Friederike, and Sandra Glocka. 2013. Crisis communication online: How medium, crisis type and emotions affected public reactions in the Fukushima Daiichi nuclear disaster. Public Relations Review 39: 40–46. [CrossRef] van der Kaa, Hille, and Emiel Krahmer. 2014. Journalist versus news consumer: The perceived credibility of machine written news. Paper presented at the Computation+ Journalism Conference, New York, NY, USA, October 24–25; pp. 1–4. Waddell, Franklin. 2018. A Robot Wrote This? Digital Journalism 6: 236–55. [CrossRef] Wang, Wei, Yaoyao Shang, Yongzhong He, Yidong Li, and Jiqiang Liu. 2020. BotMark: Automated botnet detection with hybrid analysis of flow-based and graph-based traffic behaviors. Information Sciences 511: 284–96. [CrossRef] Wasike, Ben. 2013. Framing news in 140 characters: How social media Editors frame the news and interact with audiences via Twitter. Global Media Journal, Canadian Edition 6: 5–23. Wölker, Anja, and Thomas Powell. 2018. Algorithms in the newsroom? News readers’ perceived credibility and selection of automated journalism. Journalism 22: 86–103. [CrossRef] Woolley, Samuel, and Philip Howard. 2016. Political communication, Computational Propaganda, and autonomous agents: Introduction. International Journal of Communication 10: 4882–90. Zheng, Lei, Chistopher Albano, Neev Vora, Feng Mai, and Jeffrey Nickerson. 2019. The roles bots play in Wikipedia. Paper presented at ACM on Human-Computer Interaction, 3(CSCW), Boston, MA, USA, June 30–July 3. Zhou, Yuqiong, and Patricia Moy. 2007. Parsing Framing Processes: The Interplay Between Online Public Opinion and Media Coverage. Journal of Communication 57: 79–98. [CrossRef]