Systematic Literature Review on Customer Emotions in Social Media

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY-NC-ND 4.0 https://creativecommons.org/licenses/by-nc-nd/4.0/ Systematic Literature Review on Customer Emotions in Social Media © the Authors, 2018. Published version Madhala, Prashanth; Jussila, Jari; Aramo-Immonen, Heli; Suominen, Anu Madhala, P., Jussila, J., Aramo-Immonen, H., & Suominen, A. (2018). Systematic Literature Review on Customer Emotions in Social Media. In V. Cunnane, & N. Corcoran (Eds.), ECSM 2018 : Proceedings of the 5th European Conference on Social Media (pp. 154-162). Academic Conferences and Publishing International Limited. 2018 Systematic Literature Review on Customer Emotions in Social Media Prashanth Madhala1, Jari Jussila1, 2, Heli Aramo-Immonen3 and Anu Suominen1 1Tampere University of Technology, Finland 2Örebro University, Sweden 3University of Jyväskylä, Finland prashanth.ma[email protected] [email protected] heli.aramo-immo[email protected] [email protected] Abstract: Customers are human beings who express their emotions openly on social media platforms. There is a wealth of social media data that companies can make use of to improve their business decision making and tailor their marketing strategies. In order to benefit from this, organizations need to apply computational methods, which can save time and effort rather than applying traditional consumer research approaches, such as surveys or interviews. The purpose of this study is to investigate existing computational studies on detecting consumer emotions from social media data. We conducted a systematic literature review on articles published in ScienceDirect, IEEE Explore, ACM Digital Library, and Emerald Insight from the period 2009-2017. The aim was to discover how social media data was extracted, how large datasets were used in detecting emotions, the type of computational methods used, and the accuracy of the results obtained from the existing studies. Most of the studies were focused on sentiment analysis and different machine learning algorithms. The computational methods were applied in business decision making and marketing functions. Practical scenarios included emotion detection in customer reviews and sentiment analysis of retail brands. Based on these studies, we have uncovered situations where the results of the analysis are either sufficiently accurate or supportive for decision making. We provide recommendations for organizations and managers on developing their resources to make use of different computational methods for emotion and sentiment detection. Finally, we present the limitations of these methods and provide recommendations for aligning future research studies toward big social data analytics on customer emotions. Keywords: Social media, big data, emotions, consumer behavior, sentiment analysis 1. Introduction The widespread diffusion of social media platforms, such as Facebook and Twitter, are driving the world-wide spread of the phenomenon called ‘Big Data’ (Fosso Wamba et al., 2015). Moreover, social media has transformed the ways in which companies and customers interact. A great deal of social media interaction is emotionally loaded. Furthermore, competition is increasingly based on a brand’s ability to inspire emotional experiences. (Jalonen and Jussila, 2016) According to a recent study, fully emotionally connected customers are 52% more valuable on average than just highly satisfied ones (Magids, Zorfas, and Leemon, 2015). In fact, for more than a decade, emotions have been recognized as being essential in marketing and consumer behavior studies (Laros and Steenkamp, 2005). These studies have highlighted the fact that, instead of rational decision making based on utilitarian product attributes and benefits, consumer decisions are “biased” by emotions (Jalonen and Jussila, 2016). There is a wealth of social media data on consumer emotions available for companies to collect and analyze, yet the sheer volume of the data makes its manual analysis difficult, time-consuming, and costly. However, social media analytics tools and methods can provide a cost-effective way to gather relevant data and the means of processing the data into knowledge, enabling more accurate and valuable marketing decisions (Jussila, Boedeker et al., 2017) In their systematic review and longitudinal case study, Fosso Wamba et al. (2015) studied the impacts that Big Data may bring about. They discovered that big data should be used for understanding customers better. Therefore, Fosso Wamba et al. (2015) called for future research in the big data domain, particularly focusing on developing explanatory and predictive theories that encompass the cross functional facets of the domain. In the literature, Big Social Data has been further distinguished from the broader category of big data. Big data has been, for instance, conceptualized as any data that is produced as a result of the quantification of the world that may include data from sensors, multiple industrial and domestic networks as well as financial markets, whereas big social data is produced as a result of the mediated communication practices of our everyday life, “whenever we go online, use our smartphone, use an app or make a purchase” (Coté, 2014). In recent years, several approaches to big social data analytics (Bravo-Marquez, Mendoza and Poblete, 2014; 154 Prashanth Madhala et al. Batrinca and Treleaven, 2015; Bello-Orgaz, Jung and Camacho, 2016; Jussila, Menon, et al., 2017) have been introduced that make use of computational methods for detecting and analyzing consumer emotions. Criticism has been also leveled against current big data research. For instance, Boyd and Crawford (2012) question the objectivity and accuracy of the data, and challenge whether big data is always more suited to the research task than small data and whether it can preserve the contexts of what it is intended to describe. Bruns (2013) continues this by stating that the challenge is to improve the quality rather than (or at least the same time as) the quantity of the data gathered. Researchers must carefully consider the data they collect, and what the limitations of these sources are – rather than use the datasets that were the easiest to capture while still appearing to contain meaningful data at face value (Bruns, 2013). Bruns (2013) also points out that discussion of research choices in big data research should not be limited to data gathering approaches, for it is just as important that the further steps in processing the data are documented in detail in order to ensure the replicability of results. Taking these recommendations into consideration, we conducted a systematic literature review on customer emotions in social media guided by the following research questions: RQ1. How is social media data on consumer emotions currently collected? RQ2. How are large datasets used in detecting emotions? RQ3. What types of computational methods are being used to detect consumer emotions? RQ4. What are the limitations and the accuracy of current computational methods in detecting consumer emotions? 2. Methodology According to Tranfield et al. (2003), the systematic literature review method aims at locating all the relevant studies without the biased view of the researcher. This is achieved by making explicit the values and assumptions that constitute a review. Unlike narrative reviews, systematic reviews adopt a replicable, scientific, and transparent process in order to minimize bias, furthermore offering an audit trail for reviewers. We followed a seven-stage process model for conducting systematic literature reviews (Fink, 2005): 1) selecting research questions, 2) selecting the bibliographic or article database, 3) choosing search terms, 4) applying practical screening criteria, 5) applying methodological screening criteria, 6) performing the review, and 7) synthesizing the results. First, we defined the research questions, which are described in the introduction section. Second, we selected the appropriate databases and the type of literature in which we were interested. In order to guarantee a comprehensive sample covering the most important data related to our research objectives, we used the following databases: ScienceDirect, Emerald Insight, IEEE Explore, and ACM Digital Library. The combination of the chosen databases was diverse, covering both businessand technology-related academic journal and conference articles. Third, we used as search terms “social media,” “emotions,” and “consumer,” as the aim was to understand especially emotions related to consumer behavior in social media. Therefore, we created two search strings: “social media emotions” and “social media emotions consumer.” All of their results are combined and documented in the findings. Fourth, we applied the practical screening criteria outlined in Table 1. Table 1: Practical screening criteria Inclusion criteria Type Include only studies written in English Publication language Include studies focused on social media Content Include studies focused on emotions Content Include studies focused on consumer behavior Content Exclusion criteria Type Exclude duplicates Content Exclude articles with missing description of data extraction methods Content Fifth, as methodological screening criteria we included only empirical studies, excluding conceptual articles and studies that either introduce tools or perform experiments without collecting any social media data. Sixth, 155 Prashanth Madhala et al. in reading the full papers we further excluded articles that talk about emotions but did not use any computational method to classify social media text into sentiment or emotion categories. Table 2 outlines the resulting articles. 3. Results Table 2: Overview of the reviewed empirical studies. No. Author Title Publication 1 (Sun et al., 2017) Detecting users' anomalous emotion using social media for business intelligence Journal of Computational Science 2 (Bernabé-Moreno et al., 2015) Emotional profiling of locations based on social media Information Technology And Quantitative Management 3 (He et al., 2015) A novel social media competitive analytics framework with sentiment benchmarks Information and Management 4 (Mostafa, 2013) More than words: Social networks' text mining for consumer brand sentiments Expert Systems with Applications 5 (Ghiassi, Skinner and Zimbra, 2013) Twitter brand sentiment analysis: A hybrid system using n - gram analysis and dynamic artificial neural network Expert Systems with Applications 6 (Kontopoulos et al., 2013) Ontology-based sentiment analysis of Twitter posts Expert Systems with Applications 7 (Gao et al., 2018) Identifying competitors through comparative relation mining of online reviews in the restaurant industry International Journal of Hospitality Management 8 (Li and Xu, 2014) Text-based emotion classification using emotion cause extraction Expert Systems with Applications 9 (Deng, Sinha and Zhao, 2017) Adapting sentiment lexicons to domain-specific social media texts Decision Support Systems 10 (Howells and Ertugan, 2017) Applying fuzzy logic for sentiment analysis of social media network data in marketing International Conference on Theory and Application of Soft Computing, Computing with Words and Perception 11 (Lee, 2017) Social media analytics for enterprises: Typology, methods, and processes Business Horizons 12 (Benthaus, Risius and Beck, 2016) Social media management strategies for organizational impression management and their effect on public perception Journal of Strategic Information Systems 13 (Wang et al., 2016) Fine-Grained Sentiment Analysis of Social Media with Emotion Sensing Future Technologies Conference 14 (Zhao et al., 2014) PEARL: An Interactive Visual Analytic Tool for Understanding Personal Emotion Style Derived from Social Media IEEE Conference on Visual Analytics Science and Te chnology 15 (Hong Keel Sul, Dennis and Yuan, 2014) Trading on Twitter: The Financial Information Content of Emotion in Social Media Hawaii International Conference on System Sci ences 16 (Larsen et al., 2016) TV ratings vs. Social media Engagement Big Social Data Analytics of the Scandinavian TV Talk Show Skavlan IEEE International Conference on Big Data 17 (Shukri et al., 2015) Twitter Sentiment Analysis: A Case Study in the Automotive Industry IEEE Jordan Conference on Applied Electrical Engineeri ng and Computing Technologies 18 (Sarakit et al., 2015) Classifying Emotion in Thai YouTube Comments International Conference of Information and Communication Technology for Embedded Systems 19 (Bing, Chan and Ou, 2014) Public Sentiment Analysis in Twitter Data for Prediction of A Company's Stock Price Movements IEEE International Conference on eBusiness Engineering 20 (Castellanos et al., 2011) LivePulse: Tapping Social Media for Sentiments in Real-Time International conference companion on World Wide Web 21 (Xu et al., 2017) A New Chatbot for Customer Service on Social Media Conference on Human Factors in Computing Systems 22 (Uren et al., 2016) Social media and sentiment in bioenergy consultation International Journal of Energy Sector Management 23 (He et al., 2017) Application of social media analytics: a case of analyzing online hotel reviews Online Information Review 156 Prashanth Madhala et al. The reviewed empirical articles consist of 11 journal articles and 12 conference articles, with the earliest article from the year 2011 and the latest from 2017. As our aim was to understand data collection, dataset attributes and the types of emotions investigated in the studies, we compiled Table 3 that describes the social media platforms where the data was extracted, the data extraction method used to collect the data, and the emotion classification used in the study. Table 3: Social media platforms, data extraction, dataset size and emotion classifications used in the studies. No. Social Media Platform Data Extraction Method Dataset Size Emotion Classification 1 Sina weibo Crawler 10275 microblogs on 100 users Distinct sentiment – 3 categories, Discreet emotions – 5 categories 2 Twitter Harvester 852319 interactions Valence, Arousal, Dominance of emotions – 51 emotional states. 3 Twitter API Not described Distinct sentiment – 2 categories 4 Twitter Miner 3516 tweets Distinct sentiment – 2 categories 5 Twitter API 10345184 tweets 5 categories of sentiment 6 Twitter API Not described Distinct sentiment – 2 categories 7 Dianping Crawler 35872 reviews Distinct sentiment – 2 categories 8 Sina weibo Crawler 16845 microblogs Discreet emotions – 6 categories 9 Twitter Querying 743069 tweets Distinct sentiment – 2 categories 10 Twitter Open source intelligence Not described 5 categories of sentiment 11 Yelp Nvivo 50 customer postings(d) Distinct sentiment – 4 categories 12 Twitter API 17 million user generated tweets, 200000 sent by companies. Distinct sentiment – 5 categories 13 Twitter Crawler Not described Distinct sentiment – 3 categories, Emotions – 4 categories 14 Twitter PEARL Study 1: 10 participants, 308 labeled tweets. Study 2: 6 participants, 600 tweets per participant Discreet emotions – 8 categories 15 Twitter CRSP 2503385 tweets Distinct sentiment – 2 categories 16 Facebook SODATA 24000 comments Distinct sentiment –3 categories, Emotions – 5 categories. 17 Twitter API 3000 tweets Distinct sentiment– 2 categories, Emotions – 7 categories 18 YouTube API Movies: 2771 comments Advertisement: 3077 comments Discreet emotions – 6 categories 19 Twitter Crawler 15 million tweets Distinct sentiment – 5 categories 20 Twitter API Not described Distinct sentiment – 3 categories 21 Twitter API 703 study participants Distinct emotions – 3 categories 22 Twitter API 300 tweets Distinct sentiment – 3 categories 23 TripAdvisor API 11403 reviews Distinct sentiment – 3 categories Regarding the size of data, from as little as 50 postings to 17 million tweets have been analyzed in terms of customer emotion detection. The largest size of dataset was extracted from the Twitter platform. The datasets were extracted using APIs, crawlers, miners, harvesters, open source intelligence, and custom-built software (e.g., SODATA, PEARL, and VOZIQ). A variety of different software tools were employed in the studies described in the articles. For instance, visual analytics was carried out by custom-built PEARL software in Zhao et al. (2014) and MATLAB was used for graphical representation of data in Sun et al. (2017). R was used for data pre-processing in several articles (He et al., 2015, 2017; Shukri et al., 2015). In addition, several R libraries were used in sentiment score calculation and visualization, such as twitteR, plyr, stringr, and ggplot2. Restaurant review pre-processing, element extraction, and web crawling were implement in Python using MySQL for data storage (Gao et al., 2018). Similarly, Larsen et al. (2016) carried out an experiment using Python. However, some of the studies did not explicitly report the software and analysis tools used in the study. Most of the studies categorized emotions into 2-5 different sentiment categories. Seven studies categorized sentiment into positive and negative categories. Six studies categorized sentiment into positive, neutral, and negative categories. Lee (2017) categorized emotions into four sentiment categories: very positive, moderately 157 Prashanth Madhala et al. positive, very negative, and moderately negative. In three studies, emotions were categorized into five sentiment categories, e.g., strongly positive, mildly positive, neutral, mildly negative, and strongly negative (Ghiassi, Skinner, and Zimbra, 2013). The exceptions were Benthaus et al. (2016), who differentiated sentiment into average, positive, negative, ambivalent, and neutral and Bing et al. (2014), who classified sentiment into positive+, positive, neutral, negative, negative-. In seven studies, emotions were categorized into distinct emotion categories, ranging from 4 to 51 distinct emotions. Bernabé-Moreno (2015) classified emotions into 51 categories based on Russel’s (1980) circumplex model: ambitious, amorous, amused, apathetic, ashamed, attentive, bitter, conscientious, contemplative, convinced, courageous, dejected, desperate, despondent, determined, disappointed, discontented, disgusted, dissatisfied, distrustful, embarrassed, enthusiastic, envious, expectant, feeling_guilt, feeling_superior, friendly, hesitant, hopeful, impatient, impressed, indignant, insulted, interested, jealous, joyous, languid, light-hearted, loathing, longing, melancholic, passionate, peaceful, pensive, polite, serious, solemn, suspicious, taken_aback, uncomfortable, worried. In some studies several emotion theories were used, but the most common theories included Ekman’s (1992) basic emotions and Plutchik’s (2001) emotion theory. Wang et al. (2016) categorized emotions into anxiety, sadness, anger, and others based on combined emotion studies (Ekman, 1992; Plutchik and Robert, 2001; Socher et al., 2013; Chafale and Pimpalkar, 2014). Zhao et al. (2014) used Plutchik and Robert’s (2001) categorical model for primary emotion pairs such as anger-fear, anticipation-surprise, joy-sadness, and trustdisgust; (Mehrabian, 1980; Russell, 1980) for dimensionality (valence, arousal, dominance); and (Davidson and Sharon Begley, 2012) for emotional styles on the above-mentioned emotions and dimensions. Shukri et al. (2015) employed (Strapparava and Valitutti, 2004; Wilson, Wiebe and Hoffmann, 2005) to categorize emotions into classes such as anger, disgust, fear, joy, sadness, surprise, and unknown. In the Li and Xu (2014) study, emotion classification was carried out based on basic emotions (Ekman and Friesen, 1971): happiness, anger, disgust, fear, sadness, and surprise, Larsen et al. (2016) and Sarakit et al. (2015) also employed Ekman’s (1992) model of basic emotions. Sun et al. (2017) classified emotion into neutral, happy, surprised, sad, and angry, without referring to any emotion theories. In addition, Xu et al. (2017) classified emotions without referring to any emotion theories, first, into two categories: emotional and informational, and second, into three additional categories: appropriateness, empathy, and helpfulness. The studies introduced a variety of approaches for detecting the above described emotions categories. We compiled the used computational methods, the evaluation and assessment of the methods, as well as, primary limitations outlined by the authors into Table 4. Table 4: Limitations, computational methods, and their evaluation in reviewed empirical studies. Article Computational Methods Evaluation of Computational Methods Primary Limitations 1 Anomaly detection: single and multivariate Gaussian distribution Accuracy of abnormal user emotion detection = 83.49% Sparseness of micro-blog data of individual user, abnormal user reaction can only be detected in a week or in a month. 2 Multivariate kernel density function Visual evaluation Analyzed only geo-located social media users and not the entire population at the airport. 3 Modified Chi-square feature selection, modified N -Gram model N-Gram correctly recognized 79% of the comments. The values of a ccuracy showed that the N -Gram correctly marked positive message and negative mess ages at a success rate of 82.42%. Not described 4 Hu and Liu Lexicon T-mobile brand has highest negative sentiment of 72% , DHL has highest positive sentiment of 60% Analysis does not reveal the reason b ehind consumers’ expression of sentiment, meaning that it fails to identify the sentiment topic. 5 Supervised machine learning algorithm - DAN2, Support vector machine (SVM). Accuracy for strong positive emotions using DAN2 – 96%. Accuracy for mildly negative and strongly negative emotions – 89.9% and 95.1% Analysis was performed only on a single corpus (single brand). 6 Ontology based semantically enabled system (SEM), Ontology support (ONT) ; Custom built system without ontology learning techniques (CUS). Calculates sentiment score for tweets: (a) SEM outperforms ONT with regards to recall ratios, (b) SEM, ONT perform better than CUS. Presence of high ratio of advertising tweets could contribute to the distortion of results. 158 Prashanth Madhala et al. Article Computational Methods Evaluation of Computational Methods Primary Limitations 7 Network construction: Singleline graph, Dichotomic -line graph, and Multi-line graph. Sentiment strength dictionary. Proportion of 5 star rating – R 20 (56.8%) Service - R2 (9.1/10) Environment - R2 (8.9/10),R20 (8.9/10) Price - R2 (89/100) Taste - R20 (9.0/10) Restaurant no 20 is better in 4/7 categories. Dataset used is limited to 50 restaurants. 8 Support vector machine (SVM), Support vector regression (SVR), Emotion cause extraction. Higher precision for happiness, anger, fear, and surprise emotions. Increase in statistical fscore percentage in happiness, anger, disgust, surprise (1.6%, 2%, 9.2% , and 0.8%) respectively. Emotion causes derived from sad posts are not effective when classifying them from other posts. 9 Lexicon that learns sentiment words generated by corpus. General Inquirer (GI), MPQA subjectivity lexicon (MPQA), Opinion lexico n (OL), SentiWordNet (SWN). Domain specific: Loughran and McDonald (LM). F-measure - Experimental results – 80.31%, Class level results - 81.05%, political tweets results – 85.46% Proposed algorithm - combined lexicon with part -of-speech tagging excluding SWN (Combined4_ARVN) outperforms in Fmeasure by almost 10% . Higher false-positive rate is obtained when many words are added to the lexicon which have no sentiment and higher fa lse-negative rate when fewer words are added during which sentiment words are missed. Therefore, choosing the right threshold is important. 10 Fuzzy logic analysis Social bot account follower rate = 27% of the user population, Information gathered from 12% of the user population for customer relationship management. Not described 11 Sentiment analysis, Descriptive statistics Stage 1: evaluation of reviews on sentiment polarity Stage 2: Statistics performed on six factors , where c ustomer satisfaction evident on high correlation between restaurant’s food, price , and atmosphere. Small dataset size. 12 SentiStrength 2, Naïve bayes, Data analysis User generated positive tweets increased by 4.3% and negative tweets decreases by 3.5%, successful deployment of social media management tools. Analysis was based on companies that have success with social media management tools and unconscious interviewer bias. 13 Fuzzy rule inference technique, multi -source lexicon combination. Real-time analysis of tweets/data and emotions of tweets are visualized with geographic information. Negative tweets are broken down into finer emotions for real - time crisis management. Not described 14 NRC, ANEW, WordNet, Clustering -based mood detection In the proposed tool, anger mood accuracy is estimated to be 0.81 (Cohen’s Kappa). The tool can help a user visualize his/her own emotional resilience, emotional style. A small reliability issue in the PEARL analytic engine is that it should be evaluated with a large number of users or using a broader population. 15 Word analysis strategy (sentiment analysis), HarvardIV dictionary, regression using adjusted R squared value, correlation coefficient. Positive correlation between emotional val ence and same day stock returns. More impact from users with more followers (median: 177) Not described 16 No mention of any method exclusively. Only general mention of sentiment analysis. No correlation between activity on Facebook and viewing figures. Controversial views expressed are received with negative reactions which may be a n opportunity for brand exposure and attentio n. However, it may affect the brand in the long run. The time interval used may have been too short t o study the significance of the relationship between the viewing figures and Facebook activity . 17 Naïve Bayes (NB), Wiebe’s polarity lexicon, Strapparava emotions lexicon. Positive polarity of AUDI brand (83%), which was higher when compared to other brands. Not described 18 LexTo: Thai Lexeme Tokenizer, Lexitron dictionary, Support vector machine (SVM), Multinomial naïve bayes (MNB) , J48 decision tree, Weka. 10 -fold cross validation TF - In movie category, MNB has the best accuracy of 84.48%. In advertisement category, SVM has the best accuracy of 76.14% . For the TF - IDF, SVM performs better for both movie and advertisement categories with 82.28% and 72.41%, respectively. Accuracy is affected by the presence of ambiguous comments. 19 SentiWordNet 3.0, Naïve Bayes (NB), Support vector machine (SVM), C4.5 Proposed algorithm accuracy for sentiment detection in IT industry 76.12% , Media – 73.78% performs better than other classifiers. Sometimes, tweets contain no meaning which can hamper the accuracy of the model and the best prediction accuracy is obtained only after a 3-day period. 159 Prashanth Madhala et al. Article Computational Methods Evaluation of Computational Methods Primary Limitations 20 General sentiment lexicon, WordNet, Language heuristics Successful classification of positive and negative sentiments from tweets. Not described 21 Long short-term memory networks (LTSM), Sequence-tosequence learning, Word embeddings Deep learning performs better than information retrieval method in emotional context. Performance decrease (from emotional requests to informational requests) of information retrieval and deep learning systems. 22 Entity-level contextual sentiment extraction Negative emotions are mainly targeted by opposition groups. Limitations due to the analysis being performed on a small dataset. 23 Natural language pre-processing, text mining (cluster analysis, categorization, concept maps) and sentiment analysis. Customers, both satisfied and unsatisfied, share a common interest such as location, food, rooms, service quality , and staff. Not described In majority of the studies, marketing and business intelligence were the context for detecting emotions. In Sun et al. (2017), the main context was business intelligence, where user emotions were detected to help companies make better decisions with respect to product image and also to help the government monitor public opinion. Geographically tailored marketing advertisements and messages based on the emotional profile of the location were the main focus in Bernabé-Moreno (2015). He et al. (2015) focused on customer opinions on competitors’ product and service offerings for improved decision making and marketing intelligence. Xu et al. (2017) applied emotion detection in customer service chatbots. In Mostafa (2013), social media data was used to redesign marketing and advertising campaigns. Ghiassi et al. (2013) similarly focused on marketing and public relations. Gao et al. (2018) focused on competitor environment analysis, market positioning, and development of service strategy. Howells and Ertugan (2017) study was directed at customer relationship management, marketing, and retention of customers. Lee (2017) presented a 2x2 typology of enterprise social media analytics. Benthaus et al. (2016) evaluated the use of social media management tools to help firms influence public perception besides marketing and customer engagement activities. In Wang et al. (2016), the focus was on customer behavior and consumer preference context. Zhao et al. (2014) focused on providing individualized customer care. Larsen et al. (2016) and Shukri et al. (2015) both studied customer emotions for assisting marketing activities. In Bing et al. (2014) and Hong et al. (2014), the focus of emotion detection was on financial markets, and the impact on stock prices and stock returns, for example. 4. Discussion and Conclusions In our systematic literature review we first sought answers to three research questions, i.e., how social media data on consumer emotions is currently collected, how large datasets are used in detecting emotions, and what type of computational methods are being used to detect consumer emotions. We discovered that social media data on consumer emotions was collected by eight types of methods, most commonly using APIs and web crawling. The datasets used in detecting emotions ranged from small data (e.g., 50 customer postings) to what can be considered as big data in terms of volume (e.g., 17 million tweets). We found that a large variety of computational methods were being used in the process of detecting consumer emotions. Some of the studies combined several computational methods to achieve more accurate results (Sarakit et al., 2015; Deng, Sinha and Zhao, 2017). The answers to our last research question “What are the limitations and the accuracy of current computational methods in detecting consumer emotions?” revealed a fundamental weakness of the current research on detection of sentiment or emotions from social media, in particular regarding the limitations and evaluation of computational methods. First of all, a major deficiency in the research is that less than half of the studies report the accuracy of the computational method used. Furthermore, in those studies that report the accuracy, it ranged between 73% and 95%. Additionally, context seems to affect the accuracy of the computational methods. For example, in the study of Sarakit et al. (2015), Support Vector Machine yielded the best result with an accuracy of 76% in the advertisement genre, whereas the multinomial naïve Bayes yielded the best result with an accuracy of 84% in the music video genre. Therefore, a custom-built tool trained with a particular dataset, e.g., data collected from restaurant reviews, may not be transferable directly to a different context; at minimum the accuracy should be estimated in the new context. This finding affirms the view of Boyd and Crawford (2012) that objectivity and the accuracy of the data is the challenge of big data research. Therefore, our results indicate that, in addition to data gathering approaches, researchers should also focus on the processing of the data, as well as on its documentation in detail to ensure the replicability of results, as suggested by Bruns (2013). 160