Dark Design Patterns : An End-User Perspective

ISSN: 1795-6889 www.humantechnology.jyu.fi Volume 16(2), August 2020, 170–199 170 DARK DESIGN PATTERNS: AN END-USER PERSPECTIVE Abstract: The number of websites and mobile applications available is growing continually, as are the persuasive approaches to influence human behavior and decision making. Although designing for persuasion offers several potential benefits, recent developments expose various deceptive designs, that is, dark patterns, that utilize psychological factors to nudge people toward, from someone else’s perspective, desired directions. This paper contributes to an increased awareness of the phenomenon of dark patterns through our exploring how users perceive and experience these patterns. Hence, we chose a qualitative research approach, with focus groups and interviews, for our exploration. Our analysis shows that participants were moderately aware of these deceptive techniques, several of which were perceived as sneaky and dishonest. Respondents further expressed a resigned attitude toward such techniques and primarily blamed businesses for their occurrence. Users considered their dependency on services employing these practices, thus making it difficult to avoid fully dark patterns. Keywords: dark patterns, persuasive design, deceptive design, evil design, qualitative study. ©2020 Maximilian Maier & Rikard Harr, and the Open Science Centre, University of Jyväskylä DOI: https://doi.org/10.17011/ht/urn.202008245641 This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Maximilian Maier Department of Informatics Umeå University Sweden Rikard Harr Department of Informatics Umeå University Sweden Dark Design Patterns: An End-User Perspective 171 INTRODUCTION1 Technologies are ubiquitous in society today and the number of available websites and mobile applications is growing at a fast pace. Designers create these interfaces predominately with the intention to aid users in reaching their goals by being self-explanatory, presenting information in an accessible way, and allowing easy navigation (Sommerer, Jain, & Mignonneau, 2008). In recent years, methods to influence human decision-making processes through the utilization of technology have skyrocketed and have become an important dimension of human–computer interaction (HCI) research. Based upon an understanding of psychological principles through which the human mind operates, designers can cater to user experiences that effectively and purposefully influence people’s behaviors in ways desirable for these users. This resembles, in many ways, what a therapist, based upon an understanding of the human mind, can do for a patient within a therapeutic session. Persuasive technology approaches (e.g., Fogg, 2003) can be applied to promote, for instance, sustainable lifestyles (Midden, McCalley, Ham, & Zaalberg, 2008), health and wellness (Orji & Moffatt, 2018), or anxiety disorders management (Farvolden, Denisoff, Selby, Bagby, & Rudy, 2005). Although persuasive design methods in themselves are not to be feared, the very same understanding of the human mind could just as well be used for malicious purposes, paving the way for manipulating people’s decision making in a manner unaligned with their own goals and desires. In several ways, this phenomenon is nothing new. History is replete with examples of some people exerting influence on others to make decisions in certain ways, sometimes even to the decision makers’ own detriment. Most people have experienced visiting a commercial establishment and leaving with an item they did not know they wanted or perhaps did not want at all. Moreover, most people have developed, over time, a readiness for handling such situations when they occur by devising strategies for managing them. However, when it comes to persuasive technologies, users’ experiences are likely to be much more limited and, as a consequence, so is their ability to detect and handle them. These seductive design approaches that people might encounter when accessing websites and mobile applications, known as dark patterns, exploit the user by implementing deceptive functionality in the user interface. The term, introduced by user experience designer Harry Brignull (2013), describes design mechanisms that mislead users into making decisions that primarily are not in their immediate interest (see also Brownlee, 2016). Such deceptions include accepting undesirable privacy settings due to the difficulty of changing them or going through with a purchase even if the cost is higher than what was anticipated (due to costs being added at checkout and the trouble of reversing the entire procedure). Using such techniques might be considered beneficial from a business perspective—for example, to retrieve as much user data as possible or increasing sales—but they are, however, on the verge of illegality and immorality. In online interactions, user interfaces can affect people’s emotions and behavioral patterns on a deep level and it is uncertain how aware the common user is of these underlying design patterns, given their recent development and uptake. Up to this point, practitioners and the media have exposed several dark persuasive design strategies (e.g., Brignull, 2013; Jaiswal, 2018), but the phenomenon of dark patterns is rather understudied in current HCI literature. Previous work includes a critical view on the dark side of proxemic interactions (Greenberg, Boring, Vermeulen, & Dostal, 2014), focused on how systems relying on proxemic relations between people and technology, either intentionally or Maier & Harr 172 unintentionally, might end up abusing people. Other research focused on making classifications of regular, anti-, and dark patterns (Mirnig & Tscheligi, 2017), outlining the differences between these patterns and the risks associated with confusing them. A third focus consisted of a thorough analysis and categorization of dark patterns through a content analysis of a wide range of examples to determine their ethical implications (Gray, Kou, Battles, Hoggatt, & Toombs, 2018). To understand this phenomenon in more detail, further research is required. Our intention in the current paper is to contribute to an increased awareness of the phenomenon of dark patterns by exploring how users perceive, experience, and respond to these patterns. We selected a qualitative research approach because such approaches enable researchers to “answer questions about experience, meaning and perspective” (Hammarberg, Kirkman, & de Lacey, 2016, p. 499), and, in comparison to quantitative approaches, “provide a deeper understanding of social phenomena” (Gill, Stewart, Treasure, & Chadwick, 2008, p. 292). Such an understanding might unravel insights regarding the awareness of such practices, that is, which patterns are ethically appropriate to deploy and which are considered unacceptable in the eyes of the user. Thus, the research question that framed our inquiry into these issues was How does the end user perceive, experience, and respond to dark patterns? RELATED RESEARCH In the following, we present a brief introduction to human decision-making processes and brain functions, followed by a concise overview of design for behavioral change and the field of persuasive technology. Lastly, we present the subject of dark patterns, an analysis of the way these patterns are used and possible to resist, and ethical reflections of dark persuasive designs. Decision-making Processes The behavior of a person, in most cases, can be seen as a result of decisions made. An employee speaking up at a meeting commonly is caused by a felt need and a decision to do so. By understanding these decision-making processes, people’s behaviors, for the most part, can be predicted, and user experiences can be designed to support or alter such behavior. Cognitive psychology research starts from the premise that two different systems are active in human decision making, System 1 and System 2 (Kahneman, 2012). System 1 works unconsciously and effortlessly, and it relies on emotions and simplifying heuristics to make decisions, causing predictable biases. The slower and conscious System 2, on the other hand, “construct[s] thoughts in an orderly series of steps” (Kahneman, 2012, p. 21) and is linked to the experience of concentration and choice. The two systems have different individual functions, and abilities as well as limitations. Moreover, as the human mind is not able to process all the data encountered in one’s changing environment and interactions, about 95% of one’s cognitive activities take place in a nonconscious manner (van Rymenant, 2008). System 1 suggests intuitive responses while System 2 oversees the quality of these responses, which “it may endorse, correct, or override” (Kahneman & Frederick, 2002, p. 51). Two factors that are influential in human decision making are heuristics and biases. Heuristics are mental shortcuts, useful for making judgments and solving problems. However, these heuristics can result in a logical fallacy based on cognitive factors, that is, a Dark Design Patterns: An End-User Perspective 173 cognitive bias. On the one hand, heuristics can be used in design by countering those biases to help people make better decisions or by exploiting those in whatever way desired (Fischhoff, 2002). Well-studied cognitive biases and heuristics are the confirmation bias, framing, anchoring, the status quo bias, and social proof.2 People’s decision-making process also is influenced by their mood, which can be affected by numerous factors and determines whether a decision is based on logical reasons (Shafir, Simonson, & Tversky, 1993) or emotions (Bechara, 2004; Norman, 2005). One example of this phenomenon is how different colors can affect a person’s mood and influence his/her behavior in various ways (Mehta & Zhu, 2009). Two different types of affect have an influence on decision-making processes, integral and incidental emotions (Adel, 2017; Västfjäll et al., 2016). Integral affects are emotions that are part of the internal representation of a decision and influence the decision directly. These emotions can be controlled, whereas incidental affects cannot. Incidental emotions can work as heuristics for the sake of carrying out evaluative decisions. One way through which designers may consider emotions is through nudging. Nudges, “changes in choice architecture that predictably influence decisions without restricting freedom of choice” (Peer et al., 2019, p. 2), are effective tools for influencing people’s behavior and have been applied successfully in areas such as finance, education, and health. Known examples include setting defaults for organ donation (Johnson & Goldstein, 2003) or displaying social norms to combat water wasting (Bernedo, Ferraro, & Price, 2014). A nudge triggers the quick and automatic decision-making processes of System 1 to promote desired outcomes (Campbell-Arvai, Arvai, & Kalof, 2014). Peer et al. (2019) showed that a thorough understanding of the decision-making style of people could significantly improve the design, evaluation, and implementation of nudges online. Design for Behavioral Change By considering decision-making processes, design can be used to influence and shape human behavior. Such an approach draws on theories of social, environmental, and behavioral characteristics as drivers for changes in behavior. Originally, work by Norman (1988) introduced concepts such as affordances or constraint feedback, providing principles with the user’s experience in mind. Norman (1999) advocated the importance of perceived affordance in design: The user’s proper perception and understanding of potential actions supports appropriate decision making when facing a new challenge. Basic principles commonly used in human-centered design also are relevant when designing for behavioral change, for instance, tailoring a design for a specific group of users, making use of people’s current understanding of a situation, the use of challenges and storytelling, or handling users’ behavioral errors in an appropriate manner (Lockton, 2013). We note here the significance of the concept of gamification: Using elements from games as motivational drivers when designing interfaces potentially leads to increased user engagement and desirable behavior (Alsawaier, 2018). Zachrisson, Storrø, and Boks (2012) visualized the distribution of control between the user and the product in a spectrum (see Figure 1). Between the extremes of informing (where the user makes decisions) and determining (when a product makes most, or all, decisions), a design might enable, encourage, guide, seduce, and steer users toward the desired behavioral outcome. Designers can create a degree of control a user should have in a particular situation and match that with relevant behavioral factors to accomplish an expected behavioral change. These factors may Maier & Harr 174 Figure 1. Distribution of control between the user and the product when designing for behavior change (Zachrisson et al., 2012, p. 363). include the degree of behavior normality, the user’s desire to act in the way the designer wants, and the level of attention the user needs for the task (Lockton, 2013). Alongside cognitive factors, we consider the contextual factors in understanding human behavior (Lockton, 2013). Many tactics are applied to modify the context in which people act or aim to change people’s thinking to make them behave in a specific manner. Design is able to address the context of the behavior and the way in which people translate their perceptions into decisions. The human mind is constantly making conscious and unconscious decisions; this reality affects the way people interact with designs. According to Norman (2005), three interconnected levels of the emotional system exist, all influencing how a design is processed. These are visceral, behavioral, and reflective, and each influences the others, shaping the overall experience. Understanding these levels can support the creation of successful and enjoyable user interface designs by making them visually appealing, effective, pleasurable, and memorable (Komninos, 2019). In order to facilitate the process of designing for behavior change, designers/researchers have developed numerous constructs such as mindful design (Niedderer, 2007) and guiding toolkits, such as design with intent (Lockton, 2013), or embedded design (Kaufman & Flanagan, 2015). Another such construct is persuasive technology (Fogg, 2003), a well-established and commonly used construct within HCI. Persuasive Technology With digital technology becoming ubiquitous in people’s lives, new possibilities have surfaced to influence behavior considerably. Much research has focused on how the affordances and constraints on the Web, in computer systems, and associated with applications could be utilized to influence behavior, such as for commercial or political purposes. Digital architecture and interface design connect also with societal effects, enabled by “mass communication, distribution of information, and social networking” (Lockton, 2012, p. 1). Nowadays, anyone is able to build services online that can enable and influence a variety of behaviors. The psychological principles for fostering behavioral change remain the same regardless of the medium, and by employing those methods in user interface design, they become subject to the field of persuasive technology. Persuasive technology is defined broadly as “any interactive computing system designed to change people’s attitudes or behaviors” (Fogg, 2003, p. 1). Fogg coined the term captology to describe “computers as persuasive technologies,” a perspective that includes “design, research, analysis, and ethics of interactive computing products created for the purpose of changing Dark Design Patterns: An End-User Perspective 175 people’s attitudes or behaviors” (Fogg, 2003, p. 5). The field is growing and various computing products, such as mobile phones, websites, mobile applications, and video games are designed to influence what people think and do. These designs offer several advantages when compared to more traditional persuaders, such as human interaction or print marketing, and can be much more persistent, collect and process massive amounts of data, scale easily, offer anonymity, provide tailored responses, and adapt to different contexts (Lockton, 2012). Research on persuasive technology has focused on, to a large extent, supporting users to adapt their behavior in a desirable way and examples of such efforts include changing health behavior (Orji & Moffatt, 2018), promoting sustainable lifestyles (Midden et al., 2008), or managing anxiety disorders (Farvolden et al., 2005). In the Fogg behavior model (FBM) for persuasive design, Fogg (2009) asserted that for a behavior to occur, a sufficient amount of motivation and ability and a trigger or prompt are needed. The behavior takes place only if all these are in place; the higher the motivation, or the easier a task appears, the more responsive people are likely to be. The FBM (see Figure 2) illustrates that “motivation and ability have a compensatory relationship to each other” (Jain, 2018, What Causes Behavior Change? section, para. 3). If the motivation is substantial, the behavior can occur even if the ability might be low and vice versa. Figure 2. The Fogg behavior model (Musso, 2017, Actions section, adapted from Fogg, 2009) clarifies how a trigger for behavior (or action) is impacted by the actor’s motivation for and ability to manage a given behavior. Maier & Harr 176 The FBM connects personal and contextual factors and presents an easy method for assessing situations and the various aspects needed to influence behavior (Lockton, 2012). Designers can engage with these elements, but have most influence over triggers and ability. Addressing the basic elements of ability—time, resources, effort, cognitive load, social acceptance, and routine—are important; however, designers must recognize that in circumstances when the user’s ability is low, the need high motivation becomes essential for success. Motivation means to be moved to act (Ryan & Deci, 2000), and people are motivated to act by several factors. However, in general, a person energized or mobilized toward a goal is characterized as motivated, while someone who is not is considered unmotivated (Lockton, 2013). Two main types of motivation have been identified: intrinsic and extrinsic (Pink, 2011). The first concerns internal factors, such as autonomy, curiosity, or meaning, while extrinsic motivation refers to external factors, such as money or rewards. Extrinsic factors are more effective for straightforward routine tasks, while intrinsic motivational techniques are more suited for tasks requiring higher cognitive functions. Motivational drivers used for creating compelling user experiences typically are based on real-world observations of what spurs people, for example, a desire for completion, collecting and order, achievements and rewards, or feedback (Jain, 2018; Kumar & Herger, 2014). It is crucial to consider both types of motivation in designing persuasive technology. In his book Beyond Boredom and Anxiety, Mihály Csíkszentmihayi (1975, p. 36) described the concept of flow as “the holistic sensation that people feel when they act with total involvement,” where a “person experiences it as a unified flowing from one moment to the next, in which he is in control of his actions.” The state of flow is one of fully focused motivation. A goal of a business might, or perhaps should, be to get its customers into this flow; achieving this requires understanding human motivational drivers fully and developing a strategy to influence the behavior of people toward the business’ goal. With an understanding of human psychology, designers can extract various implications for a design (Lockton, 2013). Various contexts, such as social environments or affordances integrated into digital systems, affect and shape people’s behavior before, during, and after actions. Designers can draw on a range of methods for influencing the way people perceive available actions, such as manipulation of information, affordances and choices, application of social proof, or conditioning behaviors. Further, people need to feel a freedom of choice, provided by, for example, multiple ways to complete a task. Thus, designers must be aware of the emotions they provoke, understand how these emotions are triggered, and be able to control them in order to address what the user is intuitively picking up. Moreover, within a certain context, cognitive thinking and decision making influence the behavior. Designers may influence cognition by providing adequate feedback in correspondence with the users’ needs and expectations, displaying a reasonable and manageable amount of information, and taking measures to motivate users. Fogg (2003) listed seven persuasive technology tools or strategies to influence attitudes or behavior (Table 1). Oftentimes, these tools or strategies are used together as part of a system to create a persuasive experience. To summarize, the commercial sector has employed persuasive design techniques frequently in the past to generate revenue, business growth, and competitive advantage. More recently, persuasion methods have focused on causing positive outcomes for the individual, and Fogg (2003) displayed how computing technologies, such as websites, mobile devices, and software applications, have the ability to change people’s attitudes and behaviors. When used responsibly, Dark Design Patterns: An End-User Perspective 177 Table 1. Seven Persuasive Technology Strategies, as Stated by Fogg (2003). Tool Description Reduction Simplifying a task toward the users’ desired outcome; for example, reducing the required steps needed for task completion. This might encourage users to perform the task (correctly) and trust their abilities in tackling the task with a positive approach. Tunneling Guiding users toward the desired outcome through a series of steps, frequently initiated by people wishing to change their own behavior. Tailoring Providing personalized experiences in terms of the presented interface, information, options, and/or feedback, based on the users’ needs and actions. This increases the likelihood that an experience will be perceived as relevant. Tailoring often is employed in conjunction with tunneling. Suggestion Giving users suggestions at the right moment. This requires recognition of an individual user’s current situation with variables that help to find out when, where, and how suggestions should/could be presented. Self-monitoring Enabling real-time tracking of people’s behavior to allow them the chance to adjust in reaching desired outcomes. When people know how they are doing, they are more likely to continue with that behavior. Surveillance Monitoring the behavior of others to adjust the targets’ behaviors in a particular manner. Observing others makes the achievement of a desired outcome more likely as people will try to act to meet the expectations of the observer. Conditioning Providing rewards or punishments to shape behaviors and help users toward desired outcomes. persuasive design can add value to the user’s experience and increase user engagement. However, persuasive design strategies directly raise questions of another kind: Who benefits from persuasive techniques? Are people sometimes intentionally misled? Where is the ethical line? Clever and malicious utilization of these strategies could easily pave the way for a manipulation of the user. Dark Patterns When interacting with the online world, people typically are unaware of existing underlying design patterns crafted with the goal of directing their attention to whatever a company wants the users to see and, alternatively, keeping other things hidden. Even if people have not noticed these patterns, they have most likely faced them while engaging digital platforms, such as shopping websites (Brignull, 2018), social media (Frobrukerrådet, 2018), video games (Zagal, Björk, & Lewis, 2013), and mobile applications (Bösch, Erb, Kargl, Kopp, & Pfattheicher, 2016). Such experiences are becoming increasingly common. By designing interfaces strategically and manipulatively, companies can steer users toward—from the companies’ perspective—a desired outcome. This phenomenon of exploiting the user was termed dark pattern by Harry Brignull, who defined it as “a user interface carefully crafted to trick users into doing things they might not otherwise do” (Brignull, 2013, What is a dark pattern? section, para. 1). In his book Evil by Design, author Chris Nodder (2013) described various patterns that allow exploring the user based on human characteristics, additionally providing advice to interface designers on how to utilize them Maier & Harr 178 on a spectrum from persuasive to manipulative in their own work. From a user’s perspective, the effect of these patterns may be annoying and frustrating on one hand but, on the other, could lead to financial loss (Asbury, 2014), addictive behavior (Dow Schüll, 2014), or unintentionally sharing personal data (Frobrukerrådet, 2018). Many articles about dark patterns have been published by practitioners (e.g., Brownlee, 2016; Jaiswal, 2018; Singer, 2016), but research on the topic is scarce, suggesting that the subject is somewhat understudied. One exception is research conducted by Mathur and colleagues (Mathur et al., 2019), who used an automated technique—a web crawler developed for experts—to identify instances of dark patterns on approximately 11,000 shopping websites. Even if the crawler analyzed only textual information—thus excluding patterns enabled by color, style, and other nontextual features—it uncovered 1,818 instances of dark patterns within these sites. Recently, researchers at Purdue University began studying the issue (e.g., Chivukula, Brier, & Gray, 2018; Gray et al., 2018), primarily by focusing on ethical considerations. In this paper, dark patterns are defined as functionality, drawing on psychological insights, implemented within user interfaces that is deceptive to users and in not in their best interests. Brignull (2018) formulated 12 different types of dark patterns to describe various deceptive strategies, while Gray et al. (2018) contributed by sorting these into five distinct categories: nagging, obstruction, sneaking, interface interference, and forced action (see Figure 3). Nagging: Redirecting expected functionality that perseveres one or more interactions (Gray et al., 2018). Nagging often occurs as the user is pursuing a goal and the task is interrupted by an action not related to the original goal. Examples are pop-ups blocking the interface from the user and sound or video that is played automatically when not expected or desired. Obstruction: Making a process more complicated than it has to be in order to dissuade an action (Gray et al., 2018). An example of this is how some companies make it difficult for users to access and/or change privacy settings. Sneaking: Designing an interface with the intention to “hide, disguise, or delay” (Gray et al., 2018, p. 6) information of interest to the user. It baits users to act in a way they likely would not if they were aware of it. An example would be an X button in an interface that, instead of closing the window, performs another unexpected action. Figure 3. Summary of dark pattern strategies, taken from Gray et al. (2018, p. 5). The items within quotation marks are from Brignull’s (2018) typology. Dark Design Patterns: An End-User Perspective 185  While reviewing the clusters, data that was either redundant or seemed neither relevant nor related to the study aim were removed, reducing the volume of information.  We then identified significant connections and patterns among the groups and added captions describing the content, which led to three final themes with six subcategories from which meaning was drawn.  Finally, this article represents the reporting process of the thematic analysis. RESULTS AND ANALYSIS The data analysis resulted in a diagram consisting of three themes (Perception, Conduct, Countermeasures) and six categories (see Figure 4). Although the themes Perception and Conduct encapsulated directly the participants’ answers, the theme Countermeasures was derived simply from the participants’ feedback without any associated categories. Perception The theme Perception encompasses the categories Impression, Assessment, Balance, and Acceptability. The category Impression covers users’ comprehension of dark patterns as well as their thoughts and general understanding of the topic. How participants evaluate dark patterns is described in the Assessment category. The Balance category focuses on the tension between values and manipulation, while Acceptability outlines the circumstances under which participants accept or reject dark patterns. Impression The participants recognized the subject as a “difficult topic” that “many don’t know about or don’t want to know about” (P1), with the specific term dark pattern being completely unfamiliar Figure 4. Themes, categories, and connections resulting from the thematic analysis of user-based focus groups and one-on-one interviews. Maier & Harr 186 to the participants. Some of the presented examples of dark patterns were recognized. Interviewees had “encountered them before” (P1), and a few were considered “pretty obvious” (P1) and predictable, especially techniques that toy with emotion and try to pressure people into doing something. The interviewees pointed to nagging as one such example often encountered that is always considered annoying. The preselection of checkboxes is another omnipresent phenomenon, for instance, that was experienced when installing computer programs, which also entailed the risk of unintentionally installing something that was not desired (e.g., a toolbar). Other techniques were new to the participants and left them astonished. In general, the respondents were “somewhat aware” (FG7) of the existence of manipulative techniques on the Web. Awareness of these practices, according to the respondents, has increased in recent years, and they noted this originated from the continuous media coverage of scandals, with the Facebook–Cambridge Analytica data incident leading the way. The EU legislation on cookies,6 which requires informed consent from people through cookie prompts, also sensitized participants regarding “what happens with your data” (FG2), as did GDPR enactment. Because of these events, participants noted awareness of some of the patterns’ prohibition by these laws, for example, “Some of them were outlawed” (FG2). As a group, the participants’ personal experiences with dark patterns have influenced their perceptions, and especially their emotions, toward the subject. Assessment Overall, the informants considered dark patterns as sneaky, hidden, intentionally implemented, and a “dishonest way to conduct business and interface design” (P3). According to FG3, the patterns were “usually noticed afterwards, after they tricked you.” Several participants acknowledged that these tactics exist to trick people, that applying dark patterns is “in the company’s best interest, not the user’s” (P4) and that “they trigger you with certain things and then profit from that” (FG5). One participant boiled it down to the following: “They are forcing me to do something like buy stuff, click or see something, get some data from me, get advantage of me, and influence me in my behavior” (P2). Respondents see no way to fully avoid dark patterns and believe that people simply must live with it. One reason for this resignation is that users seem to “depend on websites” (P3). At that point, participants projected their personal apprehension onto the general population. Because certain interfaces must be used (“Some companies are just too big and you need them”; P4), running into dark patterns on those sites oftentimes cannot be prevented. Interestingly, two attendees of the second focus group shared the opinion that such techniques will appear more often in the future, that the Internet is becoming a “darker” place that will grant users less and less freedom. Therefore, people need to “use the Internet in a functional way, with awareness” (FG1), always pay attention, think about whether a service is interested in manipulating the user, and find the “best strategy to minimize the damage” (FG4). In terms of what emotions were felt when facing dark design patterns—noting they are not always spotted—it became evident that everyone participating in the study was feeling annoyed. The reoccurring character of some of the patterns fueled this annoyance. Other emotional reactions included anger, irritation, feeling stupid or pressured, frustration, worry (especially when it comes to data security), and stress. The common point of view of the participants was that the reaction depends on the perceived damage. If the damage is minor, Dark Design Patterns: An End-User Perspective 187 people “just get annoyed and live on with it” (FG3); another participant remarked that she had “gotten used to it, because so many do it” (P1). There seemed to be a feeling of resignation, that one just “has to accept it” (FG6). In another immediate reaction regarding encountering dark patterns, Participant P3 stated it “makes me not trust services,” while P1 “tries to reverse the steps” mentally to check what triggered the experience. The willingness to counter the dark patterns is low, according to P2, because it is an “effort I have to make and a waste of time, so I just ignore it.” Balance Although the experience of facing behavior-influencing elements makes some participants want to leave a website completely, it seems as if the benefits that come with using a website or application often outweigh the encountered negative aspects. In general, answers from the participants suggest that integrating deceiving techniques into user interfaces is “always shitty behavior” (P3), regardless of the company applying it. However, it appears that such behavior is excused more when the benefits are high or come from certain companies. If users profit from a company in different, valuable ways, and maybe even are dependent on a service, they are inclined to accept possible minor negative aspects, especially if companies “provide a free service to the user” (FG1). The participants agreed that they will presumably keep using a service if it “still has good information, benefits, or if friends use it” (P2). P1 formulated additional reasons by saying, “I know that it is influencing me in some way and I still use some of the services because sometimes there is no alternative; sometimes it is just convenient; sometimes I just don’t care and accept it.” The participants noted an understanding that the purpose of many services is to keep the user engaged “and I accept it when I enjoy it” (FG4). The opinion is that more established, larger companies can afford to implement dark patterns more often because they can manage the potential loss of users from these practices. Furthermore, several participants expressed a fear of missing out on something if they do not use a particular (often popular) service. This means that if users find enough reason to keep using a service, they would do that, even if it means encountering dark patterns. A focus group attendee expressed his thoughts by saying, With companies I trust, at least I know the worst case is that they want to steal my time or data, but their behavior is not malicious nor are they trying to cheat me out of 50 bucks with some hidden fee. (FG2) However, if dark patterns on user interfaces are “interfering with the original task too much” (FG4) or are frustrating enough, participants would consider “stop using it” (FG1), “might just abort” (P2), or “try to avoid it as much as possible” (FG4). Naturally, it comes down to the user, who has to decide whether to keep using a service despite of the knowledge of deceptive design patterns. Nevertheless, it might depend on the balance between values and manipulation whether a person keeps on using that service. Acceptability Specific dark patterns considered more acceptable to most of the respondents include the categories nagging and forced action. This is particularly the case because these categories are Maier & Harr 188 thought to be clearly visible but more annoying than dangerous. Many variations of these techniques leave the user with a choice because of their visible character: “Here you can just choose to wait or close the page” (P4). P3 remarked that some practices on a given website are “enough to make me leave, but are not really hurting me.” Further, P1 noted, “Giving me trouble to unsubscribe is not as bad in comparison to if it’s something really important that I’m going to miss.” Moreover, methods comparable to the classic salesmanship-like toying with emotions are considered commonplace and ordinary and are therefore widely accepted among all study participants. Our informants viewed these patterns as annoying, but “not too dangerous” (P4). P3 added, “Those are not really malicious, so I think they are ok.” In contrast, deceiving behavior is not excused if coming from institutions that handle personal, sensitive data: “I wouldn’t excuse it from my bank” (P1). People find the utilization unacceptable if it could have more severe consequences or if “it is something really important that is missed” (P1). Therefore, the participants’ responses indicated that hiding information is generally considered more dangerous to the user because the patterns are harder to detect and invisible. As one participant expressed, “I can’t excuse dark patterns that can cause more damage in the future” (FG7). However, two interviewees remarked that because they probably would not notice that information was hidden from them, they would inherently accept the pattern, even though it does “not make it ok” (P2). P4 added, “If it’s in the background and you can’t see it and it is not obvious, then I accept it more.” However, the participants’ responses suggest that users’ trust of a company is weakened and its credibility compromised if it utilizes too many manipulative techniques. Participants especially did not excuse the sneaking or hiding of additional costs or informing the user about sharing their personal information. Regarding specific dark patterns, it became clear how difficult it is to generalize which techniques are accepted more and which are rejected the most. Some might care about particular techniques that others might not mind at all. However, the participants could not imagine a scenario in which they actually desire being influenced in their behavior by dark patterns. Only interviewee P2 tried to see a positive side of techniques pressuring people through emotions: “They may help me to decide faster, which is sometimes good because it reduces my time wasting.” To conclude, the participants felt that they, and perhaps people in general, have become more critical when interacting with user interfaces in recent years. They look for deceiving techniques and try not to be tricked. However, they oftentimes fail. Such behavior can quickly result in them abandoning these services. Nevertheless, they sometimes end up accepting them, by choice. Conduct The Conduct theme consists of the categories Responsibility and Motivation, described below. When considering the question of who is responsible for utilizing dark patterns, all participants agreed that it is mostly the business owner’s fault for making the conscious decision to apply manipulative techniques and then benefiting from that. Two participants noted that lower-level employees, such as the designers or programmers who actually implement the dark pattern, hardly can be blamed as they probably just execute managerial orders. If a higher ranked employee agrees to utilize a technique, “then the whole company is to blame” (FG2). However, participants remarked that users are at least partly responsible for their fate and should view websites and mobile applications in a critical way at all times: Many dark patterns potentially could be avoided if the average users were more aware of the application of these techniques Dark Design Patterns: An End-User Perspective 189 and payed more attention to their usage. Yet, because the most effective dark patterns are those that remain undetected by the user, participants acknowledged that the user could not be at fault for running into those. Regarding the intention and motivation behind implementing dark patterns, participants listed numerous possible explanations. They include organizational growth in general, gaining (psychological) influence and/or control over users, enhancing popularity, promoting a certain ideology or goal, getting people’s attention, accessing people’s data, and marketing and sales. P1 described an unpleasant sneaking incident that occurred when interacting with an airline’s website: The price that they put in the advertisement didn’t apply for me in the end because it was just for a special payment option. ... I had to go all the way in the process, through putting in my personal information, to just realize, “Oh this price doesn’t apply for me.” (P1) Another very common example of companies pressing for sales, and one often encountered by our participants, is a type of psychological pressure being applied on the user through toying with emotions (e.g., “only three rooms left” on booking.com). The reasons that companies apply malicious techniques seem to be many; however, all participants identified profit as the main motivation. Countermeasures A great variety of possibilities to fight dark patterns in user interfaces came to the respondents’ minds in focus groups as well as during individual interviews, even though the common thread was that the endeavor is difficult and regulating certain patterns might just lead to companies coming up with different schemes. Additionally, it was noted that while stopping them completely is next to impossible, it is “all about how to best live with them” (FG4), which can be achieved best by being careful and thinking critically when interacting in an online environment, as well as keeping in mind that someone might be interested in manipulating you. Nevertheless, participants identified several actual countermeasures for combatting these strategies. First, the topic of people’s awareness of these tactics was a significant topic of discussion during the focus groups. According to the participants, this could be achieved by educating people, by “getting the message out there” (P4), and even warning users about existing dark patterns through media articles, posts on social media (e.g., Facebook, Twitter, YouTube), or documentaries. Raising awareness about dark patterns is a process that could start even in educational institutions, as one participant mentioned. In addition, “official objective channels” (P1), possibly even governmental agencies, could provide more information on the subject. Participants mentioned some countering tactics in their discussions: The development of some kind of software, along with ad blockers and scripts, which can act as a shield to protect users from dark patterns. Moreover, all participants referred to laws or some kind of consumer protection regulation as perhaps the most suitable way to fight manipulative design because companies otherwise would “just keep on using them because they profit so much from it” (P4). As a participant expressed, “If you want to get rid of them, laws might be the best solution” (P3). One focus group attendee was aware that regulations have been implemented already: “Some of them are now outlawed with the new GDPR” (FG2). Another possible countermeasure considered was to apply pressure on companies by denouncing them publicly for utilizing dark patterns (e.g., through social media use). This could be Maier & Harr 190 achieved through boycotting websites and mobile applications that make use of dark patterns in their interfaces, perhaps even building “a community to fight the patterns” (P4). Another participant agreed, “If I see some articles or news that some companies do bad stuff, it increases chances that I stop using them” (P2). Additionally, “using alternatives that don’t utilize dark patterns” (FG5) also potentially could help in influencing companies to change their behavior. Finally, rather drastic solutions for dodging the behavior-influencing techniques on the Web were discussed. One idea, beyond simply “leaving the Internet” (FG1) completely, was that of “a monthly flat rate to the Internet” (FG2). Such an option would incorporate several benefits beyond just avoiding dark patterns or masking advertisements. DISCUSSION Although the term dark pattern was unknown to the participants, they nevertheless were moderately aware of the existence of such techniques and recognized some presented examples. This is in line with previous reports on an increased awareness of these patterns among users (Chivukula, Watkins, McKay, & Gray, 2019). The respondents’ personal experiences significantly influenced their perceptions on the topic and about techniques considered to be sneaky, scheming, and dishonest. The impact and effects of dark patterns remain difficult to generalize as the level of experienced deceptiveness, as well as the effectiveness of the techniques, depend very much on the person facing them. The discussion with our study participants oftentimes shifted to specific dark patterns that dealt with privacy, and the moderator pointed out that those are not the only area where dark patterns are found. This indicates that privacy concerns are omnipresent and participants quickly connected these to dark patterns. A reason for that is certainly the repeated media coverage of data scandals in recent years, such as the case of Cambridge Analytica and Facebook. The respondents believed there is no way to avoid dark patterns fully. They named the dependency on certain services as a reason for that. Therefore, more influential companies can afford to experiment with deceiving techniques without users leaving. Lanier (2018) suggested that addiction is an important reason why people accept manipulation by information technology: People are afraid of missing out. Companies and their designers are able to manipulate the basic elements of time, resources, effort, cognitive load, social acceptance, and routine (see Fogg’s FBM, 2009)—and influence the user in their behavior. Digital platforms deliver value and benefits to its users. Alternatives disappear because many benefits appear only when everyone uses the same service (e.g., social media platforms), which in return leads to people losing options and/or flexibility. The word annoyance was mentioned frequently throughout the study. Thus, we argue that many people, at first, are angry toward certain patterns, but then get used to them (e.g., the “salesmanship” pattern). At that point, these practices just become annoying, and eventually normalized. The indication is that people might feel better when getting used to these patterns or if they do not know about their existence at all. Most of these websites or mobile applications operate within a business model where the main goal to make money, often achieved through finding customers and influencing them in a desirable way toward the organization’s product or service. It seems from their actions, however, that some companies consider behavior modification to be almost the only possibility Dark Design Patterns: An End-User Perspective 191 to survive. The resulting business model is that people’s values might have to be supplanted in favor of business needs in order for a service or product to be successful. We argue that this might not be true. Previous research indicates that, in the long term, customer happiness, trust, and credibility is likely reduced by these deceiving strategies (Brownlee, 2016), which damages the brand and eventually leads people to stop using a service or product. These longitudinal effects, however, need further attention because, as Mathur et al. (2019) found, the more popular shopping websites were more likely to feature dark patterns. Moreover, such approaches could be seen as risky strategies nowadays, especially with the ubiquity of social media: When dark pattern practices are used extensively as a marketing tool, accusations of wrongdoing by a company become easily visible to consumers. Thus, we can foresee a future where deceiving techniques might be exposed and businesses employing such tactics will have to deal with the negative consequences. The question of when influencing behavior is accepted by people and when not remains difficult to answer due to the subjective nature of persuasion. The results of this study indicate that the acceptability of certain dark patterns increases when they are visible, when the user has a choice, and/or the user enjoys the interaction and does not fail to notice anything of importance. Users might ignore manipulative techniques if they benefit from the service in other valuable ways. We could draw a parallel to research on the privacy paradox, when people claim to care about their information privacy, but share excessive amounts of personal data to reach certain gains (Draper & Turow, 2019). One is that people are unaware of how their private data are used (Barnes, 2006; Kokolakis, 2017), and another explanation is that people are willing to share such data when the expected rewards outweigh the perceived risk (Draper, 2017; Hoofnagle & Urban, 2014). A third explanation is that people feel resigned to the situation and consider pervasive monitoring as inescapable (Draper & Turow, 2019). All these explanations were brought up by respondents in the present study but in relation to manipulative techniques. However, this does not exclude the general disapproval of utilizing such deceptive techniques. If people start to accept dark patterns that let them have the freedom in making a choice, then what about the invisible behavior modification? Weinschenk (2013) stated that people are susceptible to influences from persuasive technology even after learning that they exist. This clearly also applies to the phenomenon of dark patterns, where design decisions are made that might harm the user in one way or another. The impression from this study is that participants were not shy in making a case for the reasons that make them continue to use a website or mobile application even if they are being tricked when using them. As previously mentioned by Kahneman (2012), people’s knowledge of their biases does not make it easier to act against them. This means that manipulation literacy—the ability to recognize deceptive techniques, as suggested by Lewis (2013)—is just a starting point in terms of countermeasures against dark patterns. Media coverage about dark design practices (e.g., in the wake of scandals), as well as people talking about it on social media to promote a more ethical usage or to condemn businesses, clearly raises awareness about the subject. This is in line with the contribution of Mathur and colleagues (2019), who made their data set and automated techniques publicly available journalists, researchers, and regulators to support that specific purpose. The existence of data corpuses such as from Brignull or Gray and colleagues (see Endnotes 2 and 3) further contributes to the generation of a knowledge base and sensitizes people to deceptive strategies. However, the knowledge of the existence of a dark pattern alone does not protect people from being deceived. As history has shown with the consequences of smoking, for instance, literacy Maier & Harr 192 alone does not always lead to the desired results. Because raising public awareness of the phenomenon cannot and will not be sufficient, we argue that one has to go further to properly fight such manipulative techniques. That however can be a difficult endeavor. Therefore, we advocate for making a case against organizations utilizing unethical dark design patterns. This can start at an educational level, ensuring that designers, technology developers, and their places of employment understand how such deceptive practices ultimately will be bad for business and then educate regarding ethical principles. A study by Chivukula et al. (2018) showed that design students applied manipulative techniques to exploit known user values for the purpose of stakeholder-directed outcomes if the goal was boosting conversion rates; they thereby ignored the ethical implications. Teaching about the effects of cognitive load, emotions, design for behavior change, and persuasive technology will help all involved in online and mobile design to consider the ethical implications connected with utilizing certain persuasive techniques. Various concepts and methods from scholars are available for integrating ethical principles into the work processes, such as value-sensitive design (Friedman et al., 2002) or values at play (Flanagan & Nissenbaum, 2014). The adaptation of this advice into the world of practitioners happens haltingly, however. Furthermore, business models can be adjusted to make money in alternative ways. For instance, one participant suggested monetizing services on the Internet as a way to avoid dark patterns. Brignull (as cited in Andersen, 2016, para 8) believed that “when most people encounter these kinds of patterns, they tend to blame themselves.” Our study shows, however, that the participants primarily see the business owners at fault, with growth (e.g., influence, popularity, and eventually money) providing the motivation for deceptive activities due to the reality that individual users typically are not able to evade these dark patterns. Yet they do recognize that one should be critical when interacting online and pay attention at all times. Hence, our participants acknowledge that users hold some responsibility for their own protection. CONCLUSIONS In this article, we have discussed the psychological aspects of human decision-making processes. Psychology plays an important role in understanding persuasive design. We have outlined further the concept of dark patterns and identified a gap in the research regarding how users perceive, experience, and respond to such patterns. Dark design patterns have a conscious component of deceit intending to nudge people toward making a certain decision. By intentionally implementing such manipulative techniques, user values are de-emphasized in favor of processes that profit the business. The central value of our research lies in the generation of a catalogue of various perspectives on dark patterns from users’ points of view. Our participants were somewhat aware of the existence of such strategies, but unaware of the totality and frequency of them. In the eyes of these respondents, the business owners are mostly at fault for applying manipulative designs. They believe, however, that people are partly responsible for their own fate and should view online content critically. In addition, the acceptability of such techniques shifts depending on the respective dark pattern and on the balance users accept between value received and exploitation endured. Persuasive design techniques have been used in the commercial sector for quite some time with the motivation to generate revenue and growth. Before the digital age became pervasive, Dark Design Patterns: An End-User Perspective 193 people were persuaded to buy products by salespersons in stores, on the streets, or on their own doorstep. Even then, it did not take long for the population to realize that sellers oftentimes tricked them while trying to make sales by any means, with little thought regarding the actual benefit to customer. This often led to people ignoring them on the streets or simply not opening the door. Consumers consciously stopped buying products that were not useful or beneficial for them. Nowadays, in the digital era, things have changed. People use social media and websites they know are influencing their behavior. Despite being aware of persuasive and manipulative techniques, they keep using these services, making excuses why it still has benefits for them. People accept such behavior by sellers in an online environment, triggering a different behavior compared to that in the real life and generations past. We postulate that users—as individuals and societies— are moving toward a point where deceptive behavior is becoming accepted and normalized. Nevertheless, a number of solutions are available to fight such dark design techniques. Considering the threats posed to users by deceiving strategies, such countermeasures undoubtedly need to be pursued. Moreover, by coming up with proactive approaches for undermining the harm, individuals and regulatory agencies are working against and mitigating the effect of these deceptive and potentially harmful patterns. Because this topic is rather understudied in the field of HCI, further research is needed. The participants involved in our study were all university students, on either undergraduate or graduate level. Consequently, most if not all of our participants are experienced Internet users and have substantial experience of interacting with a wide range of applications and platforms. This is clearly a limitation; a suitable next step would be to conduct a similar study with a more diverse set of participants. An additional aspect to consider is that we conducted the focus group and interviews in English, which was not the mother tongue of any of the participants. It is possible—perhaps likely—that working in a second language restricted participants when they were sharing their thoughts and opinions. Even if all participants partake in education that, to some or large extent, is provided in English, it might be worthwhile to initiate a similar study in which participants were enabled to provide their insights in their first, and not second, language. Given the qualitative approach we chose for this study, a quantitative study of how people perceive and respond to dark patterns would be an appropriate future step. This would provide objective empirical data that, for instance, could explore the average end user’s perspective on dark patterns, statistics about what user group is the more susceptible to certain behavior manipulation in user interfaces, or information about varying reactions from different groups of people. Moreover, further qualitative and quantitative testing experiments of dark persuasive design strategies can produce insights on how these techniques function in comparison to more ethically applied designs. Given that the longitudinal effects for businesses applying these techniques is unclear and to some extent contradictory, more research on actual business practices is essential. Such insights would be valuable for customers, businesses, and policy makers. Additionally, it would be interesting to conduct research regarding where exactly users can or typically come across dark patterns: What kind of businesses, websites, and mobile applications utilize them and why? IMPLICATIONS FOR THEORY, PRACTICE, AND POLICY By their very nature, dark patterns and deceptive practices manipulate users for the organizations’ gain. Our research provides information on how users are perceiving these practices—for Maier & Harr 194 themselves and for society. Thus, this research provides findings that could impact future research, user practices, and regulatory policies. For the research perspective, this research provides insights into how users perceive dark patterns, how they feel about the practices, how they accept or avoid such tactics, and what countermeasures they feel are useful. Although we provide just a small piece of the puzzle, it would be useful for future research to build on our findings and expand both the data gathering methods and participant profile. Our research also suggests ideas on how users themselves can become informed of the deceptive practices and protect themselves. Education and encouragement could be useful for users’ protection and further research can facilitate better practices as well. The same is true for helping organizations to choose honest and transparent options to attain their business goals. Finally, this research suggests avenues for government and association regulators who may need new means to address the deceptive trends and discourage such practices. Although much more research is needed, both quantitative and qualitative to advance the field, our research provides value evidence of a focused group’s perspectives on the challenges posed by dark patterns. ENDNOTES 1. The paper is based on Maximilian Maier’s master’s thesis written in 2019 at Department of Informatics, Umeå University, Sweden. 2. Social proof, a tendency to copy the behavior of others, is a term coined by Cialdini back in 1984. 3. Occurrences of deceptive patterns discovered by users are retweeted by the following Twitter account: twitter.com/darkpatterns 4. Colin Gray and his research group at Purdue University developed a website for raising awareness of deceptive patterns and for enabling users to share their own discovered examples: darkpatterns.uxp2.com 5. This scandal in 2018 consisted of a data breach where personal data from millions of Facebook users was harvested by the British political consulting firm Cambridge Analytica without users’ consent and employed primarily for political advertising. 6. Cookies are small files stored on users’ computers/smartphones/tablets to monitor their Web activities. Laws regulating their use have been in place for some time, but were revitalized by the GDPR regulation because cookies are based on, and handle, personal data. REFERENCES Adel, B. (2017, May 15). Persuasive UX: The role of emotions in decision making [Web log post]. Retrieved April 27, 2019, from https://medium.com/@beckiiadel/persuasive-ux-the-role-of-emotions-in-decision- making-4f83076dde0a Alsawaier, R. (2018). The effect of gamification on motivation and engagement. International Journal of Information and Learning Technology, 35(1), 56–79. https://doi.org/10.1108/IJILT-02-2017-0009 Andersen, M. (2016, November 2). Deceptive design is illegal now, so why are you still getting swindled? Retrieved June 1, 2019, from http://eyeondesign.aiga.org/deceptive-design-is-illegal-now-so-why-are-you- still-getting-swindled/