User-generated reviews and official star ratings: Can the two work together in hotel evaluation systems?

Odebiyi, Femi; Gontar, Vitaliya Article User-generated reviews and official star ratings: Can the two work together in hotel evaluation systems? Economic Review: Journal of Economics and Business Provided in Cooperation with: Faculty of Economics, University of Tuzla Suggested Citation: Odebiyi, Femi; Gontar, Vitaliya (2020) : User-generated reviews and official star ratings: Can the two work together in hotel evaluation systems?, Economic Review: Journal of Economics and Business, ISSN 2303-680X, University of Tuzla, Faculty of Economics, Tuzla, Vol. 18, Iss. 1, pp. 37-48, https://er.ef.untz.ba/index.php/er/article/view/58 This Version is available at: https://hdl.handle.net/10419/307821 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc-nd/4.0/ . Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020/// * Department of Business Administration, LCC International University, Lithuania, [email protected] ** St. Petersburg University, Graduate School of Management (GSOM), Russia, [email protected] 37 /// USER-GENERATED REVIEWS AND OFFICIAL STAR RATINGS: CAN THE TWO WORK TOGETHER IN HOTEL EVALUATION SYSTEMS? Femi Odebiyi*, Vitaliya Gontar** Abstract The purpose of this study is to understand the emerging influence of e-WOM (electronic word- of-mouth) as a contemporary feedback system, its role for hotels, and importance for guests. Furthermore, the study identifies the nature of user-generated feedback in terms of determining hotel ratings, and establishing whether similarities between official hotel ratings and guests’ feedback evaluations exist. The qualitative study is based on the analysis of 240 feedbacks, taken from the TripAdvisor.com, for 20 hotels of 3 and 4 star ratings in Klaipeda (Lithuania) and Kaliningrad (Russia), and interviews of eight hoteliers. The most important review categories for guests were found to be that of amenities, service, and location. The findings reveal that the online review system is still not structured enough to become the main evaluation system of a hotel. Keywords: e-WOM, guest feedback, online review, Star rating JEL Classification: M31, M39 1. Introduction The conventional star rating system in the hotel industry has been one of the guiding mechanisms for customers in their decisionmaking process for a long time, with its first prototype invented in 1900 in France as a means of travel guide (Whitelaw & Jago, 2009). Even though each country has its own requirements with differing specific criteria (Minazzi, 2010), the main roles of rating systems remain similar: to guide people in the process of hotel selection; to set specific standards, requiring hotels to maintain them; to adjust guests’ expectations regarding services provided; and to set limits on prices (Felix & Clever, 2014). However, the emerging system of online guests’ reviews has put into question the effectiveness of the conventional method with the assumption that in many cases guests can provide a more reliable and realistic overview of the service quality and amenities of a hotel. On the other hand, some critics emphasize difficulties in recognizing authenticity and objectivity of this emerging system (World Tourism Organization, 2014). A major issue preventing the emerging feedback system from taking over the conventional one is that of mistrust and skepticism regarding validity and honesty of information presented on the Internet (Hensens, 2015). On a similar note, there are concerns about the conventional system because people have experienced situations where their expectations-- set by conventional rating systems-- were not met due to experiencing a lower quality of service (Whitelaw & Jago, 2009). This study explores user-generated feedbacks as a way of evaluating hotels, focusing on what guests pay attention to when they write reviews. The findings of this study provide an overview on complementariness of the two evaluation systems (the conventional and user-generated ones) and the ways hoteliers use feedbacks in their managerial decisions. Online reviews from TripAdvisor website serve as the main data source for this study. Tripadvisor.com is considered as the largest platform for travelers’ feedbacks and one of the most popular websites for consumergenerated comments in the world (Hensens, Struwig & Dayan, 2010). In accordance, the research objectives of this study are: RO1. To determine which review categories are of main importance for guests. ///. Odebiyi, F., Gontar, V. /// 38 Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020 RO2. To establish to what extent there exists complementariness between hotel star ratings and guest feedback evaluations. RO3. To discover the manner by which hoteliers use online reviews in their managerial decisions. 2. Literature review 2.1. Conventional star rating system The hotel rating system is usually referred to as the grading system of hotels based on various attributes which evaluates physical and service characteristics, and is established and managed on governmental, sectoral or private levels (Tefera & Govender, 2015). The elements evaluated by a conventional rating system do vary according to countries, and this is the result of unsuccessful attempts to unify this system internationally (Minazzi, 2010). Many of the studies on service quality in the hotel industry have paid little attention to the intricacies of ratings, describing only the overall picture of the system without actually exploring possible substitutions (Cronin et al., 2000; Keshavarz & Jamshidi, 2018; Parasuraman et al., 1988). Wouter Hensens (2010) was among the early researchers in the field to successfully study the conventional star rating systems from a more critical perspective. First, he acknowledged that the star rating system is influential on customers’ decision because of the governmental regulatory component to the system. Similarly, Whitelaw and Jago (2009) alluded to the usefulness of the star rating system because it provides a valuable tool for government in setting rules, regulating, and ensuring safety in hotels. Hotels find their star rating useful for creating marketing strategies aimed at specific target market, and for branding and promotion as well. In contrast to Whitelaw and Jago (2009), Hensens (2015) concluded that the star rating system has proved to be not quite effective and accurate in providing clear and realistic expectations for potential customers. 2.2. Rating system through guests’ feedbacks The rating system through guests’ feedback is neither official nor approved by any governing structures, but over the years it has gained prominence in playing a key role in consumer decision-making process; therefore, hotels see the need to embrace this relatively new system in order to attract guests. Tefera and Govender (2015) concluded that people find electronic Word of Mouth (e-WOM) and guests’ feedbacks as trustworthy (and sometimes even showing a more realistic and precise picture) as the rating provided by the conventional system. However, the shortcomings of guests’ feedbacks system include both the lack of consistency and absence of well-defined criteria. The question on reliability and trustworthiness of the reviews also points to the weakness of the system. The criteria that online travel websites use for assessing hotels differ from one website to another, resulting in the inconsistency in the system (Guillet & Law, 2010). Nevertheless, the same problem is present in the conventional star rating system standards, which usually differ from one country to another, more so that not all hotels use either the standardized US AAA Tourism Rating System (Tefera & Govender, 2015) or the European star rating system. Subjectivity in guests’ feedbacks is another mitigating issue (Hensens et al., 2010). 2.3. Service quality expectations through rating systems Tefera and Govender (2015) emphasized that the initial purpose of hotel ratings is to protect customers from a disappointing experience and to ensure service quality. Both the star rating system and guests’ feedbacks address the issue of providing potential customers with useful and reliable information regarding services. The star rating system was first introduced as a means of informing guests about the basic facilities of hotel without little or no reference to the subjective attributes (Hensens, 2010). Basically, a hotel will put forward a list of its facilities and a “star” is assigned to it. Nevertheless, since service quality is more than just facilities, a conventional star rating system is therefore incomplete in its assessment of a hotel. Lopez Fernandez and Serrano Bedia (2004) in their study of Spanish hotels pointed out that rankings provided by . User-generated reviews and official star ratings: can the two work together in hotel … /// Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020 39 /// the star-rating system did not correspond with the actual level of quality experienced by guests. What is more, they found a significant difference in the categories of hotels (from one-star to five-stars), emphasizing that “quality is not synonymous with luxury but rather with providing that which a client seeks” (Lopez Fernandez & Serrano Bedia, 2004). Nonetheless, the authors did not measure the overall customer satisfaction, which could have provided a better picture of service quality evaluation because in doing so, other influential subjective factors would have been included. History of the hospitality industry’s development has shown that soon after the introduction of the star rating system, hoteliers found out that presence of the star rating certificate does not provide an absolute advantage in the market nor does it signify good reputation; for most clients it has been only a moderately important factor. For hotel clients, what really matters is the actual level of the rendered services. The practice has shown that more than a few hotels have started to use various additional measuring techniques to improve service quality, and therefore increase customer satisfaction. The most obvious way was, and still is, to ask guests to document their experience and provide suggestions for improvements. That is how the Guest Comment Cards (GCCs) were invented. GCCs are questionnaires (sometimes anonymous) that are unobtrusively distributed in various visible places around the hotel (in rooms, at the reception desk, at the lounge area, etc.) and have gained in popularity due to their simplicity and subsequent fruitful results (Felix & Clever, 2014). Guests’ feedbacks have indeed changed the way customers assess hotel ratings, as it has given them an ample opportunity to influence ratings according to their experiences. In recognition of the importance of guests’ feedback, hoteliers have found ways to integrate their systems with various platforms in order to receive guests’ direct evaluation (Duan et al., 2016). Hensens (2010) proposed that despite the fact that conventional rating systems provide a more independent overview, feedbacks can become the “most accurate way to provide potential travelers with a realistic expectation” (p. 14) or at least “be effective in improving existing systems” (p. 46). Nevertheless, the question regarding objectivity and reliability of guests’ feedbacks gets in the way of putting this system on the top pedestal when evaluating hotel service quality. One of the major problems is that there is hardly a way to influence how feedbacks are written to conform to a standardized format. Many are informal narrations written with a free style language and little structure; personal opinion or inappropriate comments are quite often present. Therefore, significant proportions of feedbacks do not contain useful information or reflect actual quality of service being evaluated. To avoid this problem and make the system more stable while working towards a common benefit, some websites have started requiring guests to additionally evaluate several essential amenities when leaving a feedback. This step has considerably impacted the reliability and structuralizing of this system, thus giving it a better chance to supplant the traditional star rating system. 2.4. Rating systems in meeting price expectations Price has always been an influential factor not only when customers decide on making their purchase decision but also for the postpurchase evaluation. In a situation where they have to react on the rendered services, price is one of the highly impacting factors in moving these evaluations towards or away from a met expectation. Even if the quality of service remains the same, differences in price affect overall impression and customer satisfaction. At the same time, the following tendency has been noticed: the lower the price, the lower the expectations towards the service; thus, it is easier to exceed those expectations and increase customer satisfaction by trying to improve service quality and leave price unchanged. The inverse logic also works, when high price sets expectations too high; in this situation the failure in meeting those expectations leads to a strong negative fall in customer satisfaction and overall experience (Toncar, Alon & Misati, 2010). Toncar et al. (2010) argues that the service sector in contrast to the tangible goods sector, has more flexible price expectations, with the ///. Odebiyi, F., Gontar, V. /// 40 Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020 relationship between expected and real price weaker due to the fact that services are less tangible and visible, which then makes customers to have a greater price range set in mind for particular services. When taking the conventional rating system into account, one may find it quite easy for customers to adjust their expectation based on the fact that hotels from one sector (for example, three-star hotel) have a comparable price-range, because types of services and facilities provided within this sector are similar (Whitelaw & Jago, 2009). Tefera and Govender (2015) have also stated that online guests’ feedbacks system plays a little bit different role in pricing system. With the ability to view the feedbacks left by actual guests, one can compare the actual level of service quality and the provided price, thus evaluating their compatibility. Guests’ feedbacks can serve as a measure of the degree to which hotel’s services correspond to the ones stated, influencing significantly the popularity and demand for a particular hotel. 2.5. An integrated approach for rating systems There is no doubt that both the traditional rating system and guests’ feedbacks have their own unique advantages, as well as drawbacks, in the sense that ideally none of them should be used independently. Conversely, these systems can work together, taking the strong sides from both of them. Since identifying possible integrated approaches is one the objectives of this study, postulated integration models were reviewed, one of which is the model suggested by the World Tourism Organization (2014). The main push factor is the idea that these systems should have complementary rather than competing roles in the industry. Potential customers can sort their preferences and decide upon several hotels of interest with the help of the conventional star rating system, while guests’ feedbacks can help in making a final decision by getting an overview of a hotel through the past experience of real customers. The proponents of such integration point to its advantage in decreasing the level of uncertainty in customers’ decision-making process. The World Tourism Organization (2014) proposes two broad integration models: full integration and comparative performance. The first one suggests a full inclusion of weighted average of the guest ratings into the conventional star rating system, thus allowing hotels to move to a different star level, adjusting to the real guests’ experience. It can also move a hotel down if the guest ratings will evaluate it lower than the conventional system. Therefore, although star ratings take the guiding role in the model, the final influential decision is up to the guests’ ratings. The second model of comparative performance simply allows guest ratings to be reflected and taken into account on the supplement level, “providing additional guidance for the consumer” (p. 18). However, two ratings will be displayed independently without integration, and the guests’ reviews system will not have enough influence to change the official rating of a hotel (World Tourism Organization, 2014). Somewhat similar, Hensens (2010) suggested a different model of paying attention to the efficient way of the star rating system in controlling the quality of some tangible services through regular inspections and audits. These services should be monitored more often than annually in order to avoid situations where appropriate services are only maintained during the inspection period. Certainly a few adjustments will have to be done in both systems in the way of integration. Guest feedbacks indeed have strong influence on the consumer buying behavior, therefore, for making an integration system work there should be devised ways of ensuring their reliability and authenticity. 2.6. Hoteliers’ usage of online reviews The positive role of e-WOM usage as a tool for improving service quality has been explored by many researchers. A survey conducted by Nielsen (2013) found that 70 percent of the respondents trust reviews posted on online platforms. This gives hoteliers an intention to pay attention to the customers’ evaluation of services and consequently gain advantage from guests’ feedbacks in a way of knowing first-hand opinion regarding strong and weak sides of a hotel (Buhalis et al., 2016). At the same time, e-WOM and online feedbacks were found to be efficient in improving loyalty level, relationships with customers and company’s reputation as well, if the managers of service organizations regu- . User-generated reviews and official star ratings: can the two work together in hotel … /// Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020 41 /// larly and properly interact with customers on online platforms. Executive and operational managers are highly encouraged to actively participate in conversations, giving “immediate and authentic responses” (Kwok & Xie, 2016, p. 2157). Increasing consumers’ satisfaction is also viewed as an outcome of the regular and proper communication with customers (Xie et al., 2016). When guests receive a response from a hotel manager to their positive review, they feel valuable and important (Xie et al., 2016). However, it was found that positive reviews are rarely responded to, possibly due to the fact that they are viewed by other customers as less credible, thereby attracting lower helpfulness ratings (Kwok & Xie, 2016). Nonetheless, when feedbacks contain negative evaluation of hotel services, managers are encouraged to provide “consumers with a solution or method to address the problem” (Kwok & Xie, 2016, p. 2163). Such behavior serves several purposes, such as demonstration of hotel’s acceptance of the poor service provided, willingness to listen to customer’s voice and concerns, and eagerness to work on a problem that occurred in an effort to improve services. Adequate mana-gerial reaction to negative e- WOM tends to have higher helpfulness feedbacks ratings; it can eliminate conflicts at the very beginning, or at least prevents issues from getting worse and this “is one of the most salient predictors of hotel performance” (Xie et al., 2016, pp. 2017-2018). Taking e- WOM and guests’ feedbacks into account is a way towards be-tter service quality but its usage through ma-nagerial responses can benefit hotels in many more ways. However, in achieving fruitful results it is crucial to manage reviews properly: immediately and with enough authenticity. This strategy does not only restore previous consumers’ satisfaction, but at the same time improves hotel reputation and influences purchasing decisions of future customers. 3. Methodology and research results 3.1. Research design This study followed a qualitative approach which included collecting and analyzing guests’ reviews of sampled hotels, as well as conducting interviews with managers of the selected hotels. The analysis of guests’ feedbacks was based on the text analysis research conducted by Han, Mankad, Gavirneni and Verma (2016), where the authors examined text of the reviews of hotels in Moscow, Russia in order to develop reliable insights on comments’ importance and the ways of how hoteliers can apply this information. Guest feedbacks were collected from TripAdvisor.com. and the convenience sampling technique was used in choosing reviews from 10 hotels each in Klaipeda (Lithuania) and Kaliningrad (Russia), all belonging to four and three stars in the conventional rating system. For each city only the hotels having the official star rating certificate according to TripAdvisor.com were included, and then the scope was narrowed to the first 10 in terms of popularity. Only reviews in English were selected, and accumulatively 240 reviews were extracted for the analysis (the latest 12 per each hotel). The reviews were coded for further analysis by manually extracting main distinctive words and afterwards they were placed into these categories as developed by Han et al. (2016): amenities, location, transactions, value, and experience. In addition to the aforementioned categories, additional set of categories was developed by the authors-- rooms, location, value, cleanliness, sleep quality, and service-- adopted from TripAdvisor.com. MAXQDA 12 software was used for qualitative data analysis. 3.2. Interviews In order to obtain additional perspective from hoteliers on how rating systems can work in cohesion and how they use online reviews in their managerial decisions, eight interviews with different hoteliers (operational managers) were conducted. Interview questions were adapted from the questionnaire for hoteliers developed by the World Tourism Organization (2014) in their article on the integrated approach of both rating systems. ///. Odebiyi, F., Gontar, V. /// 42 Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020 The review categorization allowed the tracking of the importance of particular categories for travelers by the number of times when they were mentioned. Based on this premise, the number of coded segments 3.3. Results The dataset was taken from reviews from 40 countries. Distribution of countries represented among reviewers is presented in Figure 1. in each category was analyzed using MAXQDA software and is represented in the graph shown in Figure 2. The percentage on this graph represents the weight of each category among all coded segments. Figure 1. Distribution of countries among reviewers Source: Author's own empirical research Figure 2. Coded segments in categories distribution Source: Author's own empirical research . User-generated reviews and official star ratings: can the two work together in hotel … /// Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020 43 /// Furthermore, it was also valuable to understand the relationship among the variables themselves. This factor proved to be important in finding out about expected and As shown in Figures 3 and 4, Amenities and Location were co-mentioned more often than any other categories. Thus, it may be inferred that guests treat these categories as interrelated. unexpected patterns of code usage and relationships in the specific context of the study. This indicates a variety of associations and naturally occurring connections. Figures 3 and 4 present a co-occurrence of categories. Similarity between ratings and feedback In comparing how guests rate hotels of different star ratings on online platforms, we observed the tendency for three-star hotels to be ranked lower by guests. These observed pa- Figure 3. Coded code relations co-occurrence (nodal representation) Source: Author's own empirical research Figure 4. Coded Code Relations Co-Occurrence Number Source: Author's own empirical research Figure 5. Overall rating for 3* hotels Source: Author's own empirical research ///. Odebiyi, F., Gontar, V. /// 44 Economic Review – Journal of Economics and Business, Vol. XVIII, Issue 1, May 2020 tterns are presented in Figures 5 and 6. Figure 5 plots each of the values of 1-5 rating system (x-axis) and the corresponding percentage obtained for each of the values for (1-5) rating system from total reviews (yaxis). A value of 4 translates to guests’ perceived value is better than what was expected, Figure 6 plots each of the values of 1 -5 rating system (x-axis) and the corresponding percentage obtained for each of the values for (1- 5) rating system from total reviews (yaxis). A value of 4 translates to guests’ perceived value is better than what was expected, while a rating of value 5 is interpreted as hotel services were exceptional and of superb value. For four-star hotels, 84% of the reviewers (compared to 71.9% for three-star hotels) perceived the hotel services to be of a better than expected value or of exceptional and superb value. From this observed pattern, we conclude that the higher star rating classification a hotel has, the better the chances of it meeting customers’ expectations. However, it is necessary to notice that guests’ expectations are influenced by a number of factors, such as individual preferences, personal needs, personal philosophy, past experience, word of mouth, background, and situational factors (e.g. health issues, mood, etc.) (Hensens, 2010). Interviews analysis Semi-structured interviews were conducted with four hoteliers each in both Kaliningrad and Klaipeda. The results are somewhat unexpected, because encouraging guests to leave a feedback is among the most important while a rating of value 5 is interpreted as hotel services were exceptional and of superb value. For three-star hotels, 71.9% (28.1% + 43.8%) of the reviewers perceived the hotel services to be of a better than expected value or of exceptional and superb value. topics for hoteliers when talking about guests’ reviews. The interviewees overall have similar ways of encouraging guests to write reviews which is usually done by asking front desk staff to collect e-mail addresses of guests, either during check-in or check-out process for the purpose of sending e-questionnaires for internal feedback. One hotelier pointed out a practice in their chain which is to ask for guests’ e-mail addresses during the check-in, not during the check-out process. This practice is to prevent front desk staff from cherry-picking only guests whom they know has had a rather pleasant experience in the hotel. However, according to one of the interviewees only 30% to 40% of guests fill out the questionnaires sent to guests’ email addresses. With that level of response rate, hoteliers cannot neglect the benefits and influence of external systems, such as reviews left by guests on sites like TripAdvisor.com, Booking.com, and Expedia.com. The interviewees declared that they have put in place different services that collect all the reviews on a regular basis from various websites in order to see a broader picture and be able to track their progress more efficiently. Even though each hotelier alluded to importance of guests’ feedback and Figure 6. Overall rating for 3* hotels Source: Author's own empirical research