A case study of transactional workload running in virtual machines: the performance evaluation of a flight seats availability service

A Case Study of Transactional Workload Running in Virtual Machines: The Performance Evaluation of a Flight Seats Availability Service CARLOS JUIZ 1, (Senior Member, IEEE), BARTOMEU CAPO1,2, BELEN BERMEJO 1, (Member, IEEE), ALEJANDRO FERNÁNDEZ-MONTES 3, AND DAMIÁN FERNÁNDEZ-CERERO 3 1Computer Science Department, University of the Balearic Islands, 07122 Palma, Spain 2Multinucleo Soluciones de Alto Rendimiento S.L., 07006 Palma, Spain 3Department of Computer Languages and Systems, University of Seville, 41012 Seville, Spain Corresponding author: Carlos Juiz ([email protected]) This publication is part of the TED2021-132695B-I00 project, financed by MCIN/AEI/10.13039/501100011033 and by the European Union ‘‘NextGenerationEU’’/PRTR. ABSTRACT Much of the research has focused on performance evaluation and, particularly, the response time of clusters in cloud computing. However, one important topic has hardly been addressed: the impact of virtual machine consolidation on real business cases, on companies driven by requirements for high performance in transaction response time, specifically on intermediation trip companies. The ability to provide quality service, guaranteed within several milliseconds, is crucial to the business success of these cluster platforms. We present a case study for evaluating the performance of the seat availability service used by a flight carrier. The case study is the application of the performance evaluation methodology that ranges from monitoring to tuning options of a real-world service running on virtual machines, to understand capacity planning or possible substitution by other configurations of virtualization or containerization of the architecture of the cloud platform. This case study also proposes a workload characterisation using data clusters, allowing the architecture to be modeled as a simple network of multiclass queues of any virtual machine on the platform. Additionally, we estimate the new transaction response time by the possibility of either reducing or incrementing the number of virtual machines and their replacement by containers INDEX TERMS Virtual machines, performance evaluation, monitoring, workload characterization, discreteevent simulation, overhead, containers. I. INTRODUCTION Cloud computing aims to shift the location of the computing infrastructure to the Internet to reduce the costs of management and maintenance of hardware and software resources [1]. Cloud service providers offer high performance, scalability, security, and availability [2]. However, performance issues lead to the question of how to guarantee that the system can offer the required quality of service. This article presents the performance evaluation case study for a The associate editor coordinating the review of this manuscript and approving it for publication was Tomás F. Pena . cloud platform based on the transaction response time of a service executed in virtual machines (VM). Response time in this article is considered as the time for a transaction to be serviced (with a requirement of a few milliseconds), in other words, the sum of the delays, waiting and servicing times in the VM. Therefore, this research considers the performance journey, from the transaction arriving on the cloud platform until its service is finalised. The real-world case study consists of a virtualization platform for calculating the availability of flight inventory in a flag carrier. Only the availability of the inventory is calculated, that is, the calculation of the available seats based 81600 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ VOLUME 11, 2023 C. Juiz et al.: Case Study of Transactional Workload Running in VM on different B2B (Business-to-Business) demanding systems, for example, Global Distribution Systems (GDSs) to the flight carrier. This is a crucial business activity for companies in the tourism industry, which offer airplane seats, both the companies that mediate in the business (buying and reselling seats for their tourism packages), as well as the airlines that directly market their services. Thus, the case presented in this article can be directly applied to almost all commercial and public platforms dedicated to serving transactions as quickly as possible on VM platforms. In summary, this article includes a case study of the performance evaluation of a virtual machine platform in the cloud that is used to execute a critical service for a flight carrier, the engine for determining seat availability. The flight carrier expects this service to be executed, at most, in a response time of a few milliseconds, to be able to attend to the transactions that arrive from the main stakeholders of the airline business, e.g., the GDSs, even the airline’s e-commerce. Consequently, this article collects how the performance study problem has been faced to satisfy the following question, the IT supplier and SaaS provider that runs and monitors the service: How much residual capacity is left in the current virtual machine configuration to maintain the quality of service? This question brings us to respond to two previous issues; first, what characterises the transaction workload that the virtual machines receive? And second, is it possible to build a performance model and the corresponding evaluation of the virtual machines running that service? As soon as these questions are answered, newer questions may arise as to how to improve the service through tuning and not upgrading the current hardware. Therefore, applying the method for solving a performance problem and performance evaluation phases described in [3] and [4], in this case study we proceed as follows: 1. Understand the system to be evaluated (scenarios monitoring). 2. Characterize the current workload of the system. 3. Build a workload model. 4. Collect data and parameters of a performance model. 5. Build a performance model. 6. Evaluate the performance model. 7. Analyze the results of the performance evaluation. 8. Determine the current capacity and future capacity planning. 9. Propose options for tuning or upgrading the system. 10. Evaluate the modified performance model. Thus, the detailed contributions of this performance case study are mainly: •The current workload characterization of the transactions arriving at the VMs and their clustering in different transaction classes that are not currently differentiated at the SaaS supplier. •The analytical and simulation models and their corresponding evaluation with the current workload characterization to determine the residual capacity of VMs. The models are extended to consider the new transaction clustering when stressing the VM capacity. •The example of forecasting study of workload scenario to see the possible seasonality of the transactions and their relationship with their characterization and the proposed clustering. •The study of the possible tuning of the current architecture estimates the overhead saved or added by either adding or subtracting one VM per physical machine (PM) or even replacing VMs with containers and its estimated effects on the performance of the platform. Therefore, the remainder of the paper is organized as follows. Section II details the closer related work on queuing system cases to address the problem of providing performance evaluation with VMs. In Section III, the case-study scenarios are presented. This section corresponds with the first phase of the performance evaluation method. Section IV overviews the experimental setup. Section Vis devoted to workload monitoring and characterization, covering phases second and third of the methodology. Section VI presents our modelling proposal to design the transactional system with a very simple queueing network. We compare the real workload and the queueing model mean response times, and we show the capacity planning by stressing the queueing network model, i.e., the VMs utilization. This corresponds to phases four to seven in the methodology for evaluating the performance of the transactional system. In Section VII, we show a scenario example of the possible forecasting of the transactional system (phase number eight). Section VIII is devoted to estimating the tuning effects on the original set of VMs with other similar configurations and estimating the overheads, ending the methodology for phases nine to ten. The discussion is in section IX. Finally, section Xoutlines the main conclusions and future work. II. RELATED WORK The problem of cloud computing performance modelling considering the quality-of-service metrics as response time has been extensively studied in [5]. In [6], the authors obtained the response time distribution of a cloud system modelled on a classical open M/M/m network, assuming an exponential density function of arrival interval and service time. Response time distributions were used to determine the optimal service tier and the ratio of the maximum number of tasks to the minimum number of resources (VMs). Response time considers both wait time in queue and service time. For a particular service resource, the author has determined the level of her QoS service that can be guaranteed in terms of response time. In [7] the authors extend the e-health model developed in [8]. The nodes that form the cloud architecture can be analysed individually if they form an open Jackson network. They only consider M/M/1 and M/M/m nodes because they assume that the analysis constraints arrival and service rates have only exponential distributions. Work [7] extends the model presented in [9] by modelling a cloud VOLUME 11, 2023 81601 C. Juiz et al.: Case Study of Transactional Workload Running in VM architecture instead of web servers. The model should also serve as a guide for creating/deleting VMs as in [10] or as a guide for investigating the root cause of bottlenecks and providing solutions as in [11]. In [10], the author presented an integer programming mechanism that automatically scales compute instances to cloud-based workload information and performance requirements. It was based on an activity plan for starting and stopping VM instances. This mechanism allowed cloud applications to complete submitted jobs on time by controlling the number of underlying instances and reducing user costs by choosing the right instance type. In [12] the performance study about a monitoring system for IoP (Internet of People), the integration of chips inside people that link to other chips and the Internet, is evaluated using an M/M/c/K queuing network. The authors then propose a queueing-based model to evaluate the performance of fog-supported IoP systems. They first develop a mathematical model of a fog node that considers the processing time, service time, and arrival rate of data requests. They then extend this model to a multi-fog-node scenario, considering the transmission time and delay between fog nodes. Finally, they evaluate the performance of the proposed model through simulation experiments. The paper ‘‘Performance evaluation of message routing strategies in the Internet of robotic things using the D/M/c/K queuing network’’ by Feitosa et al. [13], focuses on evaluating the performance of message routing strategies in the Internet of Robotic Things (IoRT). The authors begin by discussing the challenges of communication in IoRT and the need for efficient message-routing strategies. They then introduce the D/M/c/K queuing network as a tool for evaluating the performance of these strategies. The article [13] focuses on the use of Internet of Things (IoT) sensor networks in smart buildings and uses queuing models to evaluate their performance. The authors then introduce queuing models to evaluate the performance of IoT sensor networks in smart buildings. They explain how queuing models work and how they can be used to simulate different scenarios and evaluate the performance of the sensor network under different conditions. In [14], the authors present a study on the performance of an Internet of Healthcare Things (IoHT) system for medical monitoring, using M/M/c/K queuing models. The IoHT system consists of a network of interconnected devices and sensors that collect and transmit healthcare data to a central server for analysis and decision-making. Our research work is a performance engineering case study covering not only the queueing modelling and evaluation but earlier phases of workload monitoring and characterization and posterior phases such as the forecasting and tuning phases. Particularly, our case study is built upon monitoring the VMs at the cloud platform and its workload characterization during several periods. From the monitoring and workload characterization and clustering, we build simple queuing network models for each VM. The network is evaluated through approximate analytical models (relaxed to exponential queues) and discrete-event simulation. The queueing network model also permits the study of the possible capacity planning future workload and anticipates several VM changes tuning the number of VMs or replacing them with containers. The queueing network model allows the cloud system to be scaled optimally to guarantee the quality of service for the transaction mean response time and plan the proper deployment and removal of VMs or virtual processors according to the workload. To comply with the mean response time expected, the cloud platform should be able to add/remove VMs according to the results obtained by approximate analysis or simulation. We summarize and compare some of the related works with ours in table 1. III. TRANSACTIONAL SYSTEM FOR FLIGHT SEATS AVAILABILITY SERVICE The flight seat availability algorithm for an airline carrier runs in an IT supplier datacenter with their PMs hosted in the cloud. In particular, the two PMs are Xeon Gold 6148 processors at 2.40 GHz as IaaS. These PMs are hosting four VMs each, to handle the transactions (availability requests) that come from different organizations and flight business intermediaries, such as Global Distribution Systems (GDS), to the flight carrier. Thus, the seat availability requests arrive in a traditional B2B relationship between the business supplier (the airline), the IT supplier (the datacenter), and the SaaS provider (the flight seats availability service) to connect to other intermediaries as input of other services, e.g., selling aircraft seats to the final customers. Mainly, for each flight seat request, a thread is launched to a vcore (virtual core) in the VM that attends to that request, the service could be summarized as: •getting data from persistence in RAM, mainly moving data from RAM to vcore; •loading local caches in the thread (move data from RAM to TLS (Thread Local Storage) to some object caches for later use; •executing linear record search operations or using Hash Maps: Operations O(n) or O(n2) and even O(1), date/time operations, and simple arithmetic operations including a few trigonometric to calculate miles. Regarding the cloud platform at the IT supplier, one PM hosts four VM, servicing transactions. To guarantee VM availability, there is another twin set of four VM in another PM, that is, the service consists of a total of eight VM, in two sets of four VM each. Each VM consists of 16 vcores, one of which hosts the operating system (in a type II virtualization architecture), and the other 15 handle business transactions in parallel. Figure 1shows a graphical schematic representation of PM hosting four VM and their 15 service vcores. IV. EXPERIMENTAL SETUP To monitor the workload of the four VMs consolidated in both PMs, the Telegraf agent1is used, which allows the collection of all the metrics of each VM of the flight availability service 1https://www.influxdata.com/time-series-platform/telegraf/ 81602 VOLUME 11, 2023 C. Juiz et al.: Case Study of Transactional Workload Running in VM TABLE 1. Comparison among some related research works. FIGURE 1. Organization of PMs, VMs, and vcores. engine. These data are obtained via SOAP and are sent to the time series database where they can be consulted with Grafana2(see figure 2). Regarding the k-means clustering of the monitored transactions we use Weka [16], and the Minitab [17] tool has been used for the analysis of the statistical distributions. Finally, we use the QNAP2 [18] discreteevent simulator and the corresponding analytical solver for the queue modelling and evaluation. V. WORKLOAD MONITORING AND CHARACTERIZATION The IaaS for processing flight seat availability can receive several million transactions per minute (TPM). The number 2https://grafana.com/ FIGURE 2. Agent monitoring the Flight Availability service. of requests/s sometimes fluctuates seasonally and during the same day but other times not. In table 2, we collect three statistics of different monitoring periods that correspond to three different scenarios of requests/s received at one PM with four consolidated VMs. Their duration has been adjusted to the availability of PM monitoring. We monitored the four VMs together, even though there are consolidated in the same PM and sharing all the hardware and software resources, i.e., without any distributed allocation or network or intercommunication among them. On the one hand, the three scenarios present very different mean response times and in the case of the first scenario the standard deviation is very high. On the other hand, it is confirmed that VOLUME 11, 2023 81603 C. Juiz et al.: Case Study of Transactional Workload Running in VM TABLE 2. Requests arrival statistics of a PM (sampling). FIGURE 3. Requests/s monitored during scenario 1. the four VMs consolidated in the same PM behave similarly due to the uniform sharing of transactions since the standard deviation between the four VMs is almost negligible for both the transactions’ mean response time and the VMs’ mean utilization. A. VM WORKLOAD MONITORING Taking any VM of the four consolidated at one PM, the observed behaviour explains the differences between scenarios (see table 3). The three scenarios could be considered different because of the variance in requests/s for a VM. So, the standard deviation of the interarrival times of requests should be considered for the workload characterization. Figures 3to 5show the profile of these requests/s for the three scenarios. In figure 3, the cycles of the graph correspond to the five days of the duration of the observation period of scenario 1. In the same way, in figure 4, each peak and valley correspond to a calendar day of scenario 2 from 28 days. However, for scenario 3, there is no such calendar pattern or seasonality (see figure 5) for 57 days. This can be demonstrated by normalizing each monitoring data (subtracting the mean requests/s and dividing by their standard deviation). We took all the combinations of consecutive five-day periods in scenarios 2 and 3, taking scenario 1 as the cycle to compare. In figure 6, normalized scenario 1 is shown, whereas the first five-day period of scenarios 2 and 3 are shown in figures 7and 8, respectively. With these paradigmatic examples, it is observed that scenarios 1 and 2 have a similar pattern of arrival requests/s while scenario 3 is different. B. VM RESPONSE TIME MONITORING Once the transaction request arrives at a VM, its service consists of two consecutive phases: processing and sending. FIGURE 4. Requests/s monitored during scenario 2. FIGURE 5. Requests/s monitored during scenario 3. FIGURE 6. Normalized requests/s monitored during scenario 1. Additionally, a transaction can remain stopped by a request scheduler (delayed) until its service can be carried out, that is, the transaction processing and sending. The resulting response time of a transaction executed in a thread by a vcore is the service time plus the delay time. The delay time of a transaction is not a queue time of the transaction waiting for its service as we understand the traditional continuous service of a FIFO queue. This can be verified in the following workload scenarios. The delay time corresponds to a different 81604 VOLUME 11, 2023 C. Juiz et al.: Case Study of Transactional Workload Running in VM TABLE 3. Requests arrival statistics of a VM (sampling). FIGURE 7. Normalized requests/s monitored during scenario 2 (example of five days). FIGURE 8. Normalized requests/s monitored during scenario 3 (example of five days). cumulated time due to scheduling or any other VM activities before transactions are queued to receive processing and sending times in a thread executed in a vcore. Between the processing and the sending times, there is no waiting time. So that as soon as a request of flight seat availability arrives at one of the four VMs the transaction request is delayed sometime due to some scheduling actions (and maybe other software cumulated delays) to determine the assigned vcore, that executes the transaction service (the flight seat availability) after some queueing at vcore processor if there would be high vcore utilization. In table 4we show the monitoring of one of the threads at vcores in a VM for a transaction in the three scenarios. Firstly, there does not seem to be any noticeable difference in the sending service, since the mean and standard deviation of the three scenarios are very similar and their values are smaller in comparison with process times. Secondly, the processing time is very similar in the three scenarios with a higher standard deviation for scenario 1. However, the biggest difference among the three samples is the delay time. Even though scenarios 2 and 3 have similarly low values compared again to scenario 1, the latter has a lot of variances. C. WORKLOAD STATISTICAL DISTRIBUTION FITTING To build even the simplest queueing model and perform the statistical analysis, it is necessary to know the distribution of the arrivals, i.e., the requests/s (or the corresponding interarrival times), the delay and the service (process and sending) times. To avoid inaccurate guessing of any distribution, we may try to confirm whether a certain distribution fits the four monitored timed variables in all the monitored scenarios. Unfortunately, the distribution fitting of the probability distribution to the series of arrivals, i.e., requests/s and the delay, process and send times were not valid for the data of any scenario. In appendix tables A1 to A4, we provide the distribution fitting test [17] of the requests/s, process, delay, and sending times, respectively, in scenario 1 (the rest of the scenarios and delay and sending times are similar). Since the p-value is lower than 0,05 in all four tables, we cannot conclude the requests/s, delay time, process time, and sending time fit with well-known distributions. D. WORKLOAD AND RESPONSE TIME CLUSTERIZATION Even if the distributions do not fit, the monitorization of the three scenarios shows some features that may conduct a more detailed classification of the transactions: •there is an enormous variance in scenario 1 in mean process time and mean delay time; •scenarios 1 and 2 are similar in arrival frequencies to each VM; •scenarios 2 and 3 are very similar in mean service times (process plus sending times) and delay time; •scenario 3 is very different in the frequency of arrivals from the other two; •scenario 1 is very different in the delay time (and somewhat in the service time). By applying clustering to the three scenarios, we may represent different classes of transactions by selecting the centroids of the clusters as representatives of a respective class. These classes were not distinguished in the original monitoring. If simple k-means are applied to the three scenarios, the following clusters can be determined based on the Euclidean distance from their centroids [16]. We transform the request/s original data to interarrival times, i.e., transform average frequencies to mean interarrival times to facilitate operationalization. VOLUME 11, 2023 81605 C. Juiz et al.: Case Study of Transactional Workload Running in VM TABLE 4. Mean delay, process, and sending times of a transaction in a VM. Therefore, the monitorization of scenario 1 could include 3 different clusters where the major differences come from the interarrival time, process time, and delay time, respectively, whereas the sending time is almost common to every cluster (see table 5). Selecting more than 4 clusters would produce two or more clusters closer than the rest, and selecting two clusters are too simplistic to characterize scenario 1. Particularly centroid 1.2 represents transactions with more mean process time and mean delay time compared to clusters 1.1 and 1.3. Whereas the delay time of centroid 1.3 doubles the one from cluster centroid 1.1. However, centroids 1.1 and 1.3 have a huge delay time variance compared to centroid 1.3. In table 5, delay time appears as the variable with more standard deviation in any cluster. To be coherent, scenario 2 is also classified into 3 clusters, even though the scenario is less variable (see centroids in table 6). Centroid 2.2 has more mean process, mean delay, and mean sending time than the two other clusters even though all three centroids are more similar in comparison with the previous scenario. Then, scenario 3 is also classified into 3 clusters (see centroids in table 7) than the previous two. Centroid 3.2 is an outlier but important for clusterization since it has a higher mean interarrival time and mean delay time than the two other centroids. With this data clusterization, we may conclude the following: •The mean processing time for transactions is higher than 2.1 ms and exceptionally may arrive to double this value (even though the maximum almost arrives at 70 ms). •The delay time is the variable that defines the transaction’s performance. •The sending time last usually slightly more than 0.1 ms but does not influence the performance of the transaction as the rest of the variables and it is very stable compared with process time and particularly with delay time. VI. QUEUE MODELLING AND CAPACITY PLANNING The cloud platform at the datacenter consists of two sets of four VMs. The queuing model is shown in figure 9. Each PM is modeled as an open network [19] where requests are generated by a source queue of customers (transactions), arriving at a H/H/∞delay queue that sends them in a uniform distribution scheduling. Each VM is a multiserver of 15 vcores. The vcores are modelled as a H/H/15 queue, solved with a discrete-event simulator. All H distributions were assumed hyper exponential with the mean and the standard deviation FIGURE 9. One PM modelled as a queue network with transactions as customers. shown in tables 2,3and 4. The hyper-exponential distribution permits to parametrize of both central statistics. Table 8shows the comparison between the mean response time measured in the PM with four consolidated VMs and the mean response times of the four simulated VMs. The results show very similar behaviour to the four VMs (results expected by the distribution of the workload), with low relative errors concerning the measured mean response time and negligible differences between VMs. Since the performance measurements and simulation seem to confirm the similarity among the four VMs consolidated at the same PM sharing uniformly the workload, we concentrate the rest of the case study in one VM but considering the clustering built to distinguish different customer classes in the queueing model. Thus, each VM is modelled with an open network where requests are generated by a source queue of multiclass customers (transactions) weighted as the data cluster frequencies, arriving at a H/H/∞delay queue that sends them to vcores. The vcores are modelled as a H/H/15 queue, solved with a discrete-event simulator (relaxed, in general, to exponential queues for approximate analytical solutions). To compare results against the discrete-event simulation approximations, we also model a VM with exponential queues for a unique class customer source and the exponential service of vcores, even for the delay time the hyperexponential distribution remains used. Thus, the delay queue was modelled as an M/H/∞and the vcores as H/M/15 with two consecutive exponential services (process and sending times). Assuming that each cluster centroid is representative of a customer class, the multiclass queue model is shown in figure 10. The queueing network model was simulated with a confidence interval of 95% for all queues and more than 10 million transaction instances (customers) were simulated per scenario, even though results converged with just thousands of customers, like the number of samples in monitoring data. 81606 VOLUME 11, 2023 C. Juiz et al.: Case Study of Transactional Workload Running in VM TABLE 5. Cluster output of scenario 1 through k-means. TABLE 6. Cluster output of scenario 2 through k-means. TABLE 7. Cluster output of scenario 3 through k-means. TABLE 8. Response times of a PM (monitoring and simulation). FIGURE 10. One VM modelled as a queue network with transactions as customers. No exact analytical or numerical solution was applicable but the convolution method was applied to have an analytical approximation. In table 9, we show the mean response times of a VM in the three scenarios. The queuing model approximations work well in discrete-event simulation and in the analytical approximation since errors compared with real data are less than 0.018% in any case, except in scenario 3 but less than 4.24%. For the data cluster centroids taken as representatives of customer classes, no analytical or approximation for a multiclass model was possible even relaxing disciplines or distributions, so we show the discrete-event simulation results in table 10. Reducing the variance intra-cluster and increasing the variance inter-cluster produces less % relative error per class, compared with the mean response times of a unique transaction class. Moreover, in scenario 3, where the outliers are isolated in class 3.2, the model reduces the aggregated class relative error in the simulation results, ranging from - 4.232 % to -0.004 %. However, in scenario 2, the aggregated class relative error increases from 0.018 % to 1.374 % and, in scenario 1 aggregated class simulation results are equal. The utilization of one VM in the three scenarios during monitoring was 34.51%, 16.57%, and 11.41%, respectively. Since the 4 VM share the workload in each PM, there is still room to receive more requests/s, until PM saturation. If the arrival frequencies monitored in the three scenarios were increasing, capacity planning would be like the ones shown in figures 11 to 13. In these figures, the arrival frequency of transactions has been increasing from 10% to 90%, close to model saturation. The mean response times are almost equal to the service times due to the buffer effect of the scheduler and its corresponding delay time until the utilization of the VM is higher than 70%. VII. FORECASTING Scenarios 1 and 2 are seasonal, each day they have a peak and valley with the same shape, although have different intensities. When normalized, both scenarios look very similar. The trend lines are slightly increasing or decreasing but do not provide much information since they are averages (see figures 3to 8). To predict the requests/s of these two scenarios, observing the daily transaction response times, we use moving averages to predict the approximate number of requests/s in the VOLUME 11, 2023 81607 C. Juiz et al.: Case Study of Transactional Workload Running in VM TABLE 9. Response times of a VM (monitoring, simulation, and analytical approximation). TABLE 10. Response times of a multiclass VM (monitoring, clustering, and simulation). FIGURE 11. Capacity planning for scenario 1. FIGURE 12. Capacity planning for scenario 2. following instants. This is quite predictable for scenario 1 (see figure 6). In the case of scenario 2, its high coefficient of variation makes the absolute value of immediate future FIGURE 13. Capacity planning for scenario 3. requests/s less predictable. In the case of scenario 3, there is no daily seasonality but the coefficient of variation is close to 30%, and the average starts to lose meaning. As an example, in figure 14, the individualized response time samples of each scenario 1 have been represented and their corresponding trend through moving averages, with periods of 90 samples. As in requests/s, there is the same seasonal behaviour for sample transaction response times. Thus, the mean and the standard deviation of response time to depict the performance behaviour using just one class of transactions could lose descriptive meaning, not only for the performance evaluation of the VM but also forecasting. This is the reason why we clustered the transactions depending on the four variables monitored. The moving averages with 90 samples period seem to confirm that consider three mean response time centroids with values computed in table 11, for scenario 1, representing the averages of valleys (1.1), peaks (1.2), and ascending and descending values (1.3), 81608 VOLUME 11, 2023