GraphQL or REST for mobile applications?

GraphQLorRESTforMobile Applications? Antonio Qui˜na-Mera1,2(B),Jos´eMar´ıa Garc´ıa3, Pablo Fern´andez3, Pa´ul Vega-Molina4, and Antonio Ruiz-Cort´es3 1Universidad T´ecnica del Norte. FICA Faculty. eCIER Research Group, Av. 17 de Julio 5-21, Ibarra, Ecuador [email protected] 2SCORE Lab. Universidad de Sevilla, Av. Reina Mercedes s/n, 41012 Sevilla, Spain 3SCORE Lab. I3US Institute. Universidad de Sevilla, Av. Reina Mercedes s/n, 41012 Sevilla, Spain {josemgarcia,pablofm,aruiz}@us.es 4Universidad de las Fuerzas Armadas ESPE, Sangolqu´ı, Ecuador [email protected] Abstract. Currently, GraphQL has emerged as a query language for developing web APIs that propose to improve several data access problems of RESTful APIs. The present paper aims to study the effects on software quality of APIs developed with REST and GraphQL architectures consumed from mobile applications. For this, we design a computational experiment that compares the quality characteristic “performance efficiency” of mobile application consumption to three APIs; one GraphQL API and two REST APIs (one exposes complex queries on several endpoints, the other exposes complex queries on a single endpoint). The results show that the software quality of the API developed with GraphQL architecture is higher than that developed with REST architecture. Keywords: GraphQL API ·REST API ·Quality evaluation ·Mobile application 1 Introduction Today, Microservice Architecture (MSA) is a trend to develop reliable and efficient applications that overcome technical and functional deficiencies such as redundancy, recovery, and scalability. The MSA consists of small applications (microservices) with a single responsibility (functional, non-functional, or crosscutting requirements) that can be deployed, scaled, and tested independently [2,25]. Microservices communicate over the HTTP(s) protocol via lightweight mechanisms that commonly use the REpresentational State Transfer (REST) architectural style to build Application Programming Interfaces (APIs). REST has been rapidly accepted in the industry since its emergence in 2000 [29]. However, it has shown some problems, such as query complexity, requiring multiple https://doi.org/10.1007/978-3-031-20319-0_2 GraphQL or REST for Mobile Applications? 17 HTTP requests to obtain multiple resources. In addition, over-fetching occurs when a request returns more data than necessary; and under-fetching occurs when an API resource does not return enough information, thus requiring additional requests [18,28]. In this sense, in 2015, GraphQL emerged as an alternative to solve several reported problems of REST [24]; the application of this paradigm in software development has had a growing interest in academia, and industry [21,28]. Therefore, we have observed that several studies compare various quality characteristics between REST and GraphQL; however, we found that no study compares these paradigms in the context of consumption from mobile applications. For this reason, we defined the following research question for this study: –RQ: What effect do the development architectures of APIs consumed from mobile applications have on software quality? To answer the research question presented, we propose to conduct a computational experiment that compares the software quality of the REST and GraphQL architectures in the development of APIs consumed from a mobile application. The rest of the document is: Sect. 2shows the theoretical foundation of REST, GraphQL, and software quality characteristics. Section 3shows the configuration of the computational experiment. Section 4shows the execution of the experimental design proposed in the Sect. 3. Section 5displays the experiment results. Section 6shows the threats to the validity of the experiment. Section 7 shows a discussion of the results with related work. Finally, Sect. 8shows the conclusions of the study and future work. 2 Theoretical Foundation REST (REpresentational State Transfer) is an architectural style for distributed hypermedia systems [6]; based on the HTTP standard for the development of web services, specifically for the development of REST APIs or also known as RESTful APIs [17,19]. The main design criteria of the REST architecture are i) all things on the Internet are abstracted as resources. ii) Each resource corresponds to a URI (unique resource identifier). iii) Standard actions such as GET, POST, PUT, DELETE, and HEAD of the HTTP protocol are used for the operation of resources. iv) The resource identifier will not change when applying multiple operations; what changes is the operator. v) Operations do not have status [8]. GraphQL is a query language, and execution engine for describing capabilities and data model requirements for client-server applications [24]. To implement GraphQL application servers (API GraphQL), it is not necessary to use a specific programming language or persistence mechanism. Instead, GraphQL encodes the capabilities of the type-system-based data model using a uniform language that responds to the following design principles: product-centric, hierarchical, strong typing, client-specified queries, and introspective. The primary operations of GraphQL are queries, mutations, and data subscriptions [5,24]. 18 A. Qui˜na-Mera et al. The Software Quality Characteristics for this study are based on the ISO/IEC 25010 standards that establish a model with characteristics and subcharacteristics to measure the software’s internal, external, and use quality [10]. In addition, the ISO/IEC 25023 standard establishes measures, including measurement functions associated with the quality characteristics in the software product quality model [11]. The Experiment (or controlled experiment) in software engineering is an empirical investigation that manipulates a factor or variable of the studied environment. Therefore, the researcher needs laboratories to study practitioners’ problems and develop and evolve solutions from experimentation [30]. Because experimentation addresses the context of the problem presented in Sect. 1and is the most commonly used strategy in software engineering [7], we will apply it to this study. 3 Experimental Setting 3.1 Goal Definition We define the objective of the experiment using the Goal Question Metric (GQM) Approach [1], considering the following: –Analyze the software development process –for the purpose to compare the performance efficiency of software architectures on mobile devices –with respect to software quality –from the point of view of the researcher –in the context of a computational laboratory. 3.2 Factors and Treatments The factor under study is the software architecture, specifically in developing APIs that mobile applications will consume. The treatments applied to this factor are: – GraphQL architecture for API development. – REST architecture for API development (considered as the control level). We will compare the performance efficiency of these architectures consumed by a mobile application. 3.3 Variables The Independent variable is the “Software architecture for API development”, applied with two treatments: GraphQL architecture and REST architecture. We operationalize this variable by developing three APIs: a GraphQL API, a REST API that exposes composite queries on separate endpoints, and another GraphQL or REST for Mobile Applications? 19 REST API that exposes composite queries on a single endpoint (similar to the GraphQL API). The dependent variable is “Software Quality”, which we operationalize using the “Performance efficiency” characteristic of the external software quality according to ISO/IEC 25010 [10]. Performance efficiency is the evaluation of performance concerning the number of resources used under stated conditions [10]. Below, we detail the metrics chosen to quantify the dependent variable: –Average response time, is the time it takes to complete a job or an asynchronous process [11]. The measurement function is: X= n  I=1 (BI−AI)/n (1) where AIis the time to start job I;BIis the operation time to perform the tasks at observation I;andnis the number of observations [11]. –Transaction processing capacity, ask the question: How many transactions can be processed per unit of time? [11]. The calculation function is: X=A/B (2) where Ais the number of transactions completed during the observation time; Bis the observation duration [11]. 3.4 Hypothesis In this section, we posed the following hypotheses based on the research question RQ: –H0(Null Hypothesis): There is no difference in the software quality effects of APIs developed with the GraphQL architecture and consumed by a mobile application. –H1(Alternative Hypothesis 1): There is a difference in the software quality effects of APIs developed with GraphQL architecture and consumed by a mobile application. The quality of software produced by GraphQL architectures is higher than that of REST architecture. –H2(Alternative Hypothesis 2): There is a difference in the software quality effects of APIs developed with GraphQL architecture and consumed by a mobile application. The quality of software produced by GraphQL architectures is lower than that of REST architecture. 3.5 Design In this experiment, we design a computational laboratory to establish the necessary conditions to compare the performance efficiency between the GraphQL architecture and the REST architecture. Where we set up three experimental tasks that will implement common data queries and mutation scenarios (four 20 A. Qui˜na-Mera et al. use cases), the first task consists of exposing a GraphQL API. The second task involves exposing a REST API with several endpoints for complex queries. Finally, the third task involves exposing a REST API with one endpoint for complex queries. We define three data query use cases with different levels of complexity, and one data insertion use case, which will be executed three times with varying amounts of records. The three tasks will be consumed by a mobile application, which will measure with the different metrics established in Sect. 3.3 the efficiency of the software quality performance of the GraphQL and REST architectures. Table 1shows the design of the experiment. Table 1. Experiment Design Use cases Repetitions GraphQL architecture REST architecture REST single endpoint architecture Use case 1 3 1, 100, 1.000, 10.000, 30.000, 50.000 (records) Use case 2 3 1, 100, 1.000, 10.000, 30.000, 50.000 (records) Use case 3 3 1, 100, 1.000, 10.000, 30.000, 50.000 (records) Use case 4 3 5 - 10 - 20 - 30 (seconds) 3.6 Use Cases The first three use cases describe data query requirements, and the fourth describes data insertion according to the data structure chosen for the study, which is described below: Use Case 1: Activities execution: consult the execution of activities performed by employees. – Query complexity level: 1 – Fields: •ConsumptionTimes: detail, EndDate, StartDate, activity, customer id, pre-invoice id, project id, type id UseCase2:Activities in detail: query the detail of the employees’ activities. – Query complexity level: 2 – Fields: •Activity: activity id, detail, status, priority •Project: project id, detail, request id, company id •Pre-invoice: prefinvoice id, detail, amount, customer id UseCase3:Solution Tasks: query the solution tasks performed in the employees’ activities. – Query complexity level: 3 – Fields: •Activity: activity id, detail, status, priority •Project: project id, detail, request id, company id •Pre-invoice: prefinvoice id, detail, amount, customer id GraphQL or REST for Mobile Applications? 21 •Distributor: consumer id, detail •Rate: type id, detail, concept Use Case 4: Activity execution: is the time of the execution of the activities planned for the employees. – Mutation: data insertion. – Fields •ConsumptionTimes: detail, EndDate, StartDate, activity, customer id, pre-invoice id, project id, type id. 3.7 Experimental Tasks In this section, we describe and design the experimental tasks of the computational laboratory according to the treatments (see Sect. 3.2), variables (see Sect. 3.3), design (see Sect. 3.5), and use cases (see Sect. 3.6) defined for this empirical study. Experimental Task 1: GraphQL API. This task aims to implement the use cases defined in the Sect. 3.6 in a GraphQL API, which a mobile application will then consume. Experimental Task 2: REST API. This task aims to implement the use cases defined in Sect. 3.6 in a REST API, exposing each data access resource at different endpoints (normal behavior in REST APIs). Experimental Task 3: API REST Queries Single Endpoint. This task aims to implement the use cases defined in Sect. 3.6 in a REST API that exposes in an endpoint the compound queries described in the use cases (behavior similar to GraphQL APIs). In this sense, we complement the computational lab architecture by developing a mobile client application that consumes the APIs described in the experimental tasks. Figure 1shows the architecture of the experimental lab. Experimental laboratory Local Network Local PC GraphQL API Mobile Application GraphQL API use cases consumption REST API use cases consumption REST API Single endpoint use cases consumption REST API REST API Single endpoint Fig. 1. Experimental laboratory architecture 22 A. Qui˜na-Mera et al. 3.8 Instrumentation The following is a description of the instrumentation of the computational laboratory used in the experiment: Local-PC specifications, is a computer that has the following characteristics: – Operating System: Windows 10 Pro 64-bit – Processor: Intel (R) Core (TM) i7-5500U CPU @ 2.40GHz – Memory: RAM 8 GB – Internet connection of 75 MB bandwidth. Mobile device specifications, is a cell phone with the following features: – Brand: Huawei P30 lite – Model: MAR-LX3A – Operating System: Android version 10 – Processor: Hisilicon Kirin 710 –RAM:4GB – Storage: 128GB – Resolution: 2312 ×1080 Development environment, consists of the following applications and technologies: – Integrated development environment: Visual Studio Code v1.62 – Programming language: JavaScript – JavaScript runtime environment: NodeJS v14.17.5 – Client application npm libraries: React Native 16.8.6, react-apollo 2.5.5 – GraphQL API npm libraries: graphql v15.5.0, graphql-server-express v1.4.1, graphql-import v1.0.2, y graphql-tools v7.0.5, cors 2.8.5 – REST API npm libraries: graphql v15.5.0, graphql-server-express v1.4.1, graphql-import v1.0.2, y graphql-tools v7.0.5, cors 2.8.5 Data collection and analysis, consists of the following applications: – Microsoft Excel 365 – R Studio Desktop 0.99.903. 4 Experiment Execution The experiment was run in Octuber 2020, using the experimental setting proposed in Sect. 3. 4.1 Sample The sample data repository belongs to a technology company in the city of Latacunga - Ecuador, which has the adequate amount of data to run the experiment according to the design exposed in Sect. 3.5. GraphQL or REST for Mobile Applications? 23 4.2 Preparation We started the preparation of the computational lab by verifying the connection to the enterprise database with the GraphQL, REST, and REST single endpoint APIs deployed on a Local PC. Next, we confirmed the connection and operation of the experimental tasks described in Sect. 3.7 from the mobile application to each exposed APIs. Finally, after verifying the operation of the experimental environment, we ran the experiment according to the design established in Sect. 3.5. 4.3 Data Collection We collected the results, on the one hand, the response times after the execution of use cases 1, 2, and 3 performed from the mobile application; on the other hand, and the number of transactions after the execution of use case 4. In each run of the use cases, the mobile application printed the results on screen, which we copied and recorded in a Microsoft Excel 365 file. 5 Results In this section, we present the results obtained from the execution of the experiment (Sect. 4.3); the statistical analysis of the metrics of average response time and transaction processing capacity of the quality sub-characteristic performance efficiency; and a summary of the effects on the software quality of the APIs developed with the GraphQL and REST architectures consumed by a mobile application. 5.1 Average Response Time After analyzing the results of the average response times obtained by executing cases 1, 2, and 3 of the experiment designed in Table 1, we observe that the response times of the GraphQL API are approximately fourteen times lower than the times of the REST API, and API REST Sigle Endpoint, as shown in Fig. 2. Statistical Analysis of Average Response Time. First, we analyze for each level of query complexity the degree of linear relationship that the APIs have, for which we use Pearson’s correlation matrices, as shown in Fig. 3. Where we highlight the following correlation values that are located between -1 and +1, level 1 of query complexity shows that API REST Single Endpoint and API REST have a positive linear correlation of 0.718. At level 2, API REST Single Endpoint and API GraphQL positively correlate to 0.754. Finally, at level 3, API GraphQL and API REST Single Endpoint have a positive linear correlation of 0.580, while API REST has a similar positive correlation of 0.583. 24 A. Qui˜na-Mera et al. Fig. 2. Average response time by query complexity level Fig. 3. Average response time correlation After performing the correlation matrix, we apply the Wilks’ Lambda statistic as a discriminant analysis that allows us to visualize if there are significant differences or not between the groups of results of the average response time of the application of the different architectures of the APIs in the experiment. Table 2shows the test of group means. In table 2, we observe that the significance level (SIG) of the GraphQL API is 0.004, being the lowest, which concludes that there are differences between the groups and that the GraphQL API has a significant advantage over the others. 5.2 Transaction Processing Capacity In this section, we analyze the results of the transaction processing capacity obtained when executing use case 4 of the experiment designed in Table 1. GraphQL has the highest processing capacity, reaching up to three times more