Website Design and Trust Elements: A/B Testing on a Start-up's Website

Schmitt, Lars; Haupenthal, Isabel; Bin Ahmed, Faisal Article Website Design and Trust Elements: A/B Testing on a Start-up's Website ENTRENOVA - ENTerprise REsearch InNOVAtion Provided in Cooperation with: IRENET - Society for Advancing Innovation and Research in Economy, Zagreb Suggested Citation: Schmitt, Lars; Haupenthal, Isabel; Bin Ahmed, Faisal (2021) : Website Design and Trust Elements: A/B Testing on a Start-up's Website, ENTRENOVA - ENTerprise REsearch InNOVAtion, ISSN 2706-4735, IRENET - Society for Advancing Innovation and Research in Economy, Zagreb, Vol. 7, Iss. 1, pp. 166-177, https://doi.org/10.54820/ZOWH5239 This Version is available at: https://hdl.handle.net/10419/262244 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc/4.0/ 166 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 Website Design and Trust Elements: A/B Testing on a Start-up's Website Lars Schmitt Technical University of Munich, Germany Isabel Haupenthal Technical University of Munich, Germany Faisal Bin Ahmed Technical University of Munich, Germany Abstract Start-ups are young companies that are hardly known, especially during their early stages, by the relevant stakeholders. A start-up's website is, therefore, often the first point of contact for potential customers, investors, or partners. Such a website usually explains the new product or service and presents the founding team with its competencies. The user's perception of the website and its design can be crucial in determining whether the user is interested in getting in touch with the start-up or even considering the purchase of the respective product or service. User’s trust in the website and its operator is essential for this. The so-called trust elements, such as logos, testimonials, or seals, are intended to create trust on websites. So far, the influence of these elements on user behaviour has hardly been empirically proven in a real-life context. Therefore, we have applied the method of A/B testing to the website of a fictive start-up. Trust elements were placed on one variant of the website (A), whereas on the other variant, there were none (B). The experiment shows that the duration of the user sessions does not differ between the two variants. However, more requests were made on the website variant with trust elements. Keywords: website design, e-trust, online trust, e-commerce, A/B testing JEL classification: L26, C99 Paper type: Research article Received: Feb 23, 2021 Accepted: May 15, 2021 DOI: 10.54820/ZOWH5239 167 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 Introduction Due to the growing importance of digitalization and the associated increase in using the internet, traditional commerce has become much more digital, what is called electronic commerce or e-commerce (Muñoz‐Leiva et al., 2010). E-commerce has changed a lot, particularly in the business-to-consumer sector (B2C), as new possibilities occurred to distribute goods and services directly to the customer (Walia et al., 2013). Companies can use their websites as communication channels (Rahimnia et al., 2013) to offer products and services beyond their offices and shops (Beldad et al., 2010). With the help of these channels, companies can get in touch with existing customers as well as potential new ones (Rahimnia et al., 2013). Furthermore, e-commerce enables significant benefits for businesses and consumers, such as the reduction of costs (Rahimnia et al., 2013). From the perspective of potential customers, websites can be used to satisfy their needs and demands, such as obtaining a new product or service (Kim et al., 2010). Website users can interact and conduct transactions with the supplying party without any temporal or spatial constraints (Beldad et al., 2010; Lowry et al., 2014). Consequently, the purchase of a product or service might be possible under better conditions (Muñoz‐Leiva et al., 2010). Hence, an increasing number of customers favor e-commerce over traditional commerce for these reasons (Li et al., 2010). With a focus on start-ups and young companies, e-commerce facilitates its entry into the global market and enables them to target a high-volume customer base (Rahimnia et al., 2013). In addition, e-commerce reduces marketing costs, promotes closer relationships with business partners as well as with customers, and can thus improve the popularity of the company (Rahimnia et al., 2013). Therefore, successful e-commerce businesses require websites that are visually appealing, easily navigable, informative, and secure (Cyr, 2013). As e-commerce lacks any kind of typical social presence, many concerns emerge, which cause people to be reluctant when operating online (Beldad et al., 2010). The larger the amount of money, the more concerned customers are about completing an online transaction (Muñoz‐Leiva et al., 2010). As fraud also takes place online and e-commerce grows rapidly, one can assume that fraud cases continue to increase immensely (Walia et al., 2013). Therefore, the essential question is raised on how to establish trust in interactions or transactions conducted on the Internet, which is also called “e-trust” (Taddeo, 2009). Several researchers have already found an answer to this question by identifying various website features and elements that influence users’ trust and, consequently, user behavior. These are often called “trust-inducing features” (Wang et al., 2005) or “trust elements” (Sivaji et al., 2011). However, their effect has hardly been empirically proven and tested in a real-life context. In contrast, laboratory experiments with control groups are much more common in the field of e-trust. We attempt to contribute to the e-trust literature through an experiment using A/B testing in a reallife context, meaning there are at least two variants of the same website. Differences in user behavior become, therefore, apparent in A/B testing. In our case, the website variants differ in the presence of trust elements so that on one variant, trust elements were visible (A), while on the other, these were removed (B). As far as we know, this application of website-based A/B testing in a real-life context is unique in the field of e-trust. 168 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 Theoretical background According to Rotter (1967), trust is “the belief that one party will reliably keep its word or promise and fulfill its obligations in an exchange relationship.” Gefen et al. (2003) define trust as “the expectations that other individuals or companies with which one interacts will not take improper advantage resulting from the dependence one has on them.” Coming from these offline dimensions of trust, Urban et al. (2000) transferred trust concepts to online dimensions by explaining how website trust is built. The authors also point out different contexts in which website trust might be important, e.g., online sales advisors, product presentation, advertising, or pricing (Urban et al., 2000). Due to the increasing importance of e-commerce, the term “e-trust” was soon introduced (Merrilees et al., 2003) as well as “online trust” (Wang et al., 2005; Kracher et al., 2005). Taddeo (2009) defines e-trust as “trust in digital contexts” and states that it “occurs in environments where direct and physical contacts do not take place, where moral and social pressures can be differently perceived, and where interactions are mediated by digital devices.” E-trust is placed in a website and its content when the customer assumes that the other party is reliable and will fulfill its obligations (Muñoz‐Leiva et al., 2012). We use the terms – website trust, online trust, and e-trust – as synonyms in the following. Another terminological distinction should nevertheless be made regarding the term “WebTrust”. WebTrust represents guidelines for e-commerce assurance services which were developed by the American Institute of Certified Public Accountants, jointly with the Canadian Institute of Chartered Accountants (Chang et al., 2011). Salam et al. (2003) differentiate between the trustee and the trustor. The trustee represents the party that is being trusted, so the website’s operator who offers products or services. Hence, the trustor is the user who places trust in the trustee. Trust between these parties is based on the user's perception of the operator's ability, benevolence, and integrity (Mayer et al., 1995; McKnight et al., 2001; Palvou, 2003). Ability is here defined as the perceived competencies and skills of the website operator (McKnight et al., 2002). Benevolence is described as the degree of empathy that the operator has towards the user, whereas integrity refers to the aspect that the website’s operator follows ethical and moral standards. Similar aspects are mentioned by Grabner-Kräuter et al. (2006), who distinguish between a soft and a hard dimension of trust. Soft characteristics of the trusted party are benevolence, honesty, integrity, and credibility, whereas hard characteristics are competence, predictability, reliability, correctness, and availability. Both dimensions affect the trustworthiness and the (perceived) functionality of the trusted party. Further aspects for assessing the trustworthiness are the operator's reputation, the appearance and design of the website, and its performance (Beldad et al., 2010; Pengnate et al., 2013). It is important to mention as well that potential customers focus heavily on reviews and other persons’ feedback, even if they do not know them personally (Beldad et al., 2010). This is because people are "truth-biased", which means that they tend to believe criticism and reviews from other people (Liu et al., 2012). Tamimi et al. (2015) emphasize that more online experience reduces perceived risks. Hence, experienced users are mainly influenced by ratings of reviews and the price of products when making their purchase decision. In contrast, less experienced users perceive the product type as the most important aspect. Altogether many aspects play a role in the context of e-trust. The main objective from the perspective of the website’s operator is to influence the user’s purchase and repurchase intention. Lim (2015) states that this intention is influenced by the user’s attitude as well as the perceived ease of use of the website. According to 169 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 Zhang et al. (2011), the repurchase intention is closely related to online customer loyalty, which might bring a competitive advantage to the website’s operator. In our study, we focus on website design and, therefore, on trust elements that (might) influence user behavior and e-trust. For this, we use the framework of trustinducing interface design features from Wang et al. (2005). The authors developed this framework based on the existing literature. Furthermore, they categorized the identified trust elements in four dimensions: graphic design, structure design, content design, and social-cue design (see Table 1). Their dimensions were later confirmed by Seckler et al. (2015) through a web-based survey. When filling out the survey, the study’s participants should think of an occasion where they felt “exceptionally trustful/distrustful” using a website (Seckler et al., 2015). Consequently, their study relied on participants’ memory and prior experiences with websites. Table 1 Design elements influencing e-trust Dimension Explanation Examples Graphic design Refers to the graphical and visual design factors on the website that normally give consumers a first impression • Use of three-dimensional dynamic • Use of moderate pastel colors • Use of well-chosen photographs Structure design Defines the overall organization and accessibility of displayed information on the website • Implementation of easy-to-use navigation, i.e., simplicity and consistency • Use of accessible information, e.g., no broken links • Application of page design techniques, e.g., white spaces, grouping, visual density Content design Refers to the informational components that can be included on the website, either textual or graphical • Display of brand-promoting information, e.g., company logo, slogan • Up-front disclosure of all aspects of the customer relationship, e.g., financial, legal concerns • Display of seals of approval or thirdparty certificate • Use of comprehensive and correct product information Social-cue design Relates to embedding social cues, such as face-to-face interaction and social presence, into web interface via different communication media • Inclusion of a representative photograph or video • Use of synchronous communication media, e.g., messaging and chat tools, video telephony Source: Wang et al. (2005) Methodology Two research fields are in particular relevant in the context of this study which has not yet been brought together, although they could enrich each other in our opinion. On the one hand, there are practitioners and researchers in the field of A/B testing who mostly present and discuss the methodology (Hynninen et al., 2014; Langmann, 2018) or provide practical business examples (Kohavi et al., 2007; Crook et al., 2009; Kohavi et al., 2011; Kohavi, 2012; Kohavi et al., 2012; Kohavi et al., 2014; Kohavi, 2015). Practitioners usually keep the results of experiments within their company 170 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 scope. On the other hand, we have numerous theoretical models and concepts in the field of e-trust (see pp. 2-3), which were mostly derived from “offline” trust research and have not yet been tested in a real online environment. We are convinced that the method of A/B testing can be used to verify and expand existing theories. A/B testing has so far mainly been used in practice to improve the website design and to encourage a certain user behavior, e.g., at companies like Airbnb, Amazon, Facebook, Linked In, or Netflix (Kohavi et al., 2020). According to Kohavi et al. (2017), the experimenter normally creates two experiences in A/B testing: “A” usually represents the current website, often considered as the “champion”, whereas “B” includes a modification of “A” and can, therefore, be considered as the “challenger”. Modifications can be changes regarding the user interface or website layout as well as the implementation of a new website feature. The users are randomly assigned to the two variants, usually with a 50/50 ratio. The key metrics are collected, computed, and analyzed (see Table 2). Table 2 Selection of typical metrics in an A/B testing Metrics Description Pageviews The total number of pages viewed. Repeated views of a single page are counted Session duration Length of a session in seconds. A session lasts as long as there is continued activity Bounces The total number of single-page visits Transactions The total number of completed purchases on the website Revenue The total revenue from web transactions Click-Through Rate (CTR) The rate is calculated by dividing the total number of clicks on an element, e.g., buttons, by the number of people who have seen the element Conversion Rate The rate that is calculated by dividing the total number of conversions by the total number of visitors, e.g., an e-commerce website receives 200 visitors/month and has 50 sales; the conversion rate would be 50 divided by 200, or 25% Sources: Google Optimize (2020); Optimizely (2020) Experiment We have chosen the start-up context as the overall setting of our experiment. The reason for this is that we believe that the website of a young company is of particular importance, as a variety of stakeholders is usually addressed, such as customers, investors, or partners. At the same time, start-ups are hardly known, so that the website becomes, metaphorically speaking, a storefront. Consequently, we created a fictive start-up and the corresponding website with two variants, A and B. The startup was called SECUPROTECT. The idea of the start-up was a "platform for security services", which was accessible via the following URL: www.secuprotect.de. Private and commercial customers could therefore find the right security service provider more quickly with the help of SECUPROTECT. The market for security services is very fragmented. This means that there are many small providers, most of whom are active locally or regionally. It is, therefore, a market where a platform business model potentially makes sense and might bring value for the customer. The fictitious service portfolio of SECUPROTECT included the following security services: object protection (private), object protection (commercial), event protection, and personal protection (see Figure 1). 171 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 Figure 1 Screenshot of the SECUPROTECT website Source: Authors’ work In our experiment, we assume that user’s trust has an impact on the mentioned key metrics, especially regarding session duration. More precisely, we assume that users who trust a website stay longer on it and also show a different click behavior, i.e. users click more. Our website variants differ in the presence of trust elements, so on one variant trust elements were visible (A), while on the other these were removed (B). We have decided on three trust elements (see Figure 2): (i) Logos of the (pretended) network of security service providers; (ii) (Fake) Testimonial of a 172 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 customer, and (iii) LinkedIn buttons (possibility to check the authenticity of the startup founders). Figure 2 Screenshot of the trust elements on variant A (removed on variant B) Source: Authors’ work A/B testing should be scheduled for a specific time frame. We have chosen a data collection period of 90 days (May 11, 2020, till August 9, 2020). Meanwhile, we also started the Google Ads campaign to make users aware of our website. Under the keyword "find security services" (German: Sicherheitsdienstleistungen finden) the website appeared mostly on page one or two in the German Google search during the campaign (see Figure 3). Figure 3 Illustration of the Google advertisement in the German language Source: Authors’ work Technical and methodical limitations We used the free version of Google Optimize for our experiment which comes with some limitations. In our case, the primary goal was to track the session duration as 173 ENTRENOVA - ENTerprise REsearch InNOVAtion Vol. 7 No. 1 well as the number of clicks on a particular button. While Google Optimize does not support click counts on links or buttons, we decided to use a link shortener service called Cuttly as a workaround. With Cuttly, it is possible to track links or button clicks for free. We created two unique custom links for the same button, so one for each variant. We did this for the button named “Make a request.” Much more problematic from a scientific point of view, was that Google Optimize or Google Analytics does not allow to export of the raw data of the experiment. This is only possible with Google Analytics 360, which we were not aware of before and during the experiment. The use of Google Analytics 360 comes with an annual fee of EUR 135,000 which was of course outside the budget for this project. This also has consequences for the methodology of this study since without raw data no own statistical analyses can be conducted, e.g., t-test. For this reason, the following results are based entirely on Google tools. Results According to the statistics from Google Optimize, there were a total of 456 sessions during the 90-day testing phase, which corresponds to the number of visits. The number of visits has to be distinguished from the number of visitors. The number of visitors was 398, meaning that some users have visited the website again. These were most likely the users who either made a request or registered as a security company. Of 456 sessions, 238 sessions are allocated to variant A and 218 to variant B. On average, users spent 36 seconds on variant A and 31 seconds on variant B (see Table 3). The trust elements, therefore, had no substantial influence on the length of stay. Table 3 Comparison between variant A and B regarding the session duration Sessions Total Session Duration Calculated Duration per Session Variant A (with trust elements) 238 02:20:53 (hh:mm:ss) 00:00:36 (hh:mm:ss) Variant B (no trust elements) 218 01:52:20 (hh:mm:ss) 00:00:31 (hh:mm:ss) Source: Authors’ work Regarding the number of clicks on the request button, there was a considerable difference between the two variants. In variant A, the request button was clicked 41 times, and we received five real requests. In variant B, in comparison, 26 clicks were made on the button, and no requests were submitted (see Table 4). Consequently, the trust elements on variant A led to more requests. Table 4 Comparison between variant A and B regarding the requests Number of Clicks on Request Button Number of Requests Made Variant A (with trust elements) 41 5 Variant B (no trust elements) 26 0 Source: Authors’ work