Recommendation and Classification Systems: A Systematic Mapping Study

Review Article Recommendation and Classification Systems: A Systematic Mapping Study J. G. Enr´ ıquez , 1 L. Morales-Trujillo , 1 Fernando Calle-Alonso , 2 F. J. Dom´ ınguez-Mayo , 1 and J. M. Lucas-Rodr´ ıguez 3 1 Computer Languages and Systems Department, University of Seville, Avd. Reina Mercedes s/n, 41012 Seville, Spain 2 Statistics and Operational Research Area, University of Malaga, Bulevar Louis Pasteur 31, 29010 Malaga, Spain 3 Servinform S.A., Calle Manufactura, 5, 41927 Mairena del Aljarafe, Spain Correspondence should be addressed to J. G. Enr´ ıquez; [email protected] Received 15 February 2019; Revised 14 May 2019; Accepted 12 June 2019; Published 27 June 2019 Guest Editor: Luis Iribarne Copyright ©2019 J. G. Enr´ ıquez et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Today, recommendation algorithms are widely used by companies in multiple sectors with the aim of increasing their profits or offering a more specialized service to their customers. Moreover, there are countless applications in which classification algorithms are used, seeking to find patterns that are difficult for people to detect or whose detection cost is very high. Sometimes, it is necessary to use a mixture of both algorithms to give an optimal solution to a problem. This is the case of the ADAGIO, a R&D project that combines machine learning (ML) strategies from heterogeneous data sources to generate valuable knowledge based on the available open data. In order to support the ADAGIO project requirements, the main objective of this paper is to provide a clear vision of the existing classification and recommendation ML systems to help researchers and practitioners to choose the best option. To achieve this goal, this work presents a systematic review applied in two contexts: scientific and industrial. More than a thousand papers have been analyzed resulting in 80 primary studies. Conclusions show that the combination of these two algorithms (classification and recommendation) is not very used in practice. In fact, the validation presented for both cases is very scarce in the industrial environment. From the point of view of software development life cycle, this review also shows that the work being done in the ML (for classification and recommendation) research and industrial environment is far from earlier stages such as business requirements and analysis. This makes it very difficult to find efficient and effective solutions that support real business needs from an early stage. It is therefore that the article suggests the development of new ML research lines to facilitate its application in the different domains. 1. Introduction The great growth in the amount of data and information that can be accessed (the known Big Data), coupled with government collaboration to provide open information (Open Data), makes companies very interested in this issue. One of the biggest problems in this area is that this information is not found in one single place, not even in a common interpretation format. Therefore, it is necessary to create solutions that collect these dispersed data and apply a specific treatment so that they can be offered to their customers. The collection of dispersed information and its unification in order to be able to work with it would open a new market niche, a new business unit, considering the possibility of generating valuable data automatically. In addition, it would increase independence when making decisions or solving problems without having to resort to an expert in business management. The ADAGIO project was born in this context. It is a R&D project that combines Big Data and machine learning (ML) strategies for the treatment of geolocated data extracted from heterogeneous data sources. It enables the aggregation, consolidation, and normalization of data from Hindawi Scientific Programming Volume 2019, Article ID 8043905, 18 pages https://doi.org/10.1155/2019/8043905 different semantic fields obtained from the sources mentioned before. Its purpose is to allow reconciled information to be consulted using specific variables, thus facilitating the generation of knowledge. The application of classification and recommendation systems in this project is of great interest for the interrelation and periodic consolidation of the data process so that the system develops capabilities for transformation, interrelation, and integration of data through supervised learning. In addition, these systems provide a great value for the management of queries, to enhance the performance of queries by users in a language as natural and high level as possible. The fact that the user obtains good results during the searches in the ADAGIO platform is one of the main objectives of the project. In order to improve the user’s experience, suggestions are proposed during the phase of filling the search parameters. For this phase, the collaboration of the system users will also be required, evaluating the results of the searches according to their quality and precision. This study has been performed to facilitate researchers and practitioners the task of choosing the most appropriate system, technology, or algorithm to include in the ADAGIO project for satisfying their requirements. In this sense, this paper presents a systematic mapping study (SMS) that analyzes the current state of the art of the recommendation and classification systems and how they work together. Then, from the point of view of the software development life cycle, this review also shows that the work being done in the ML (for classification and recommendation) research and industrial environment is far from earlier stages such as business requirements and analysis. This makes it very difficult to find efficient and effective solutions that support real business needs from an early stage. Then, this paper suggests the development of new ML research lines to facilitate its application in the different domains. This paper is organized as follows. Section 2 describes the closest related work to our proposal; Section 3 details the selected method to carry out the SMS; Sections 4 to 8 illustrate the execution of the different phases of the SMS; and finally, Section 9 summarizes the conclusions obtained from the study and presents a set of future work. 2. Related Work Recommendation and classification systems are acquiring much interest within the scientific community. In this section, the closest related works to the research proposed in this article are presented. Jaysri et al. [1] presented a complete review of the recommendation systems, focusing on the collaborative filtering. It shows different algorithms based on this filtering for both the user profile and the product characteristics. In addition, it demonstrates several classification methods that may be part of the input for recommendation systems. Ekstrand et al. [2] presented a general overview and focused on the field of recommendation systems. Their purpose was to learn more about the current development of recommendation methods, specially systems making use of the collaborative filtering. Obtaining a research perspective on how to make decisions when choosing algorithms to propose recommendations can be found in the paper presented by Gunawardana and Shani [3]. It criticizes the use of online methods, which can offer measures to choose recommendation algorithms, and determines as a crucial element the use of offline tools to obtain these measures. In addition, it discards the use of traditional metrics to make the algorithm choice and reviews the proper elaboration of experiments to carry it out. To do this, the authors perform an analysis of important tasks of the recommendation systems and classify a set of appropriate and well-known assessment measures for each task. Poussevin et al. [4] exposed the challenge of considering the preferences of users when recommending. The authors analyzed a combination of recommendation systems and classifiers that highlight words that indicate a gap between users’ expectations and their actual experience. They conclude that traditional recommendation systems analyze the past classifications; that is, they consider the users’ preferences history, while the recommendation systems that analyze the opinion classifications consider the existing evaluations at that moment. Within the scope of ML, there has been an increase in the interest of the research community, being the subject of many papers. Some of the proposals use lexical classifiers to detect possible feelings using content-based recommendations [5]. Other authors have focused on more traditional branches of ML, using well-known and proven statistical methods such as logistic regression, the Pearson correlation coefficient, or the application of the naive Bayes theorem based on probability, among others [6]. The authors of this paper focused on making extensions of these methods to solve problems inherent in recommendation systems such as cold start or scalability. The cold start [7] is a typical problem since the beginning of the recommendation systems because when a system does not have enough data, precision cannot be assured when recommending. This is a problem that gets worse at the beginning of the implementation of a system when data are not available. Scalability becomes a quite difficult task due to the increase of information in recent years and the amount of data that systems must manage. Recommendation systems, both product and user-based, affect performance and accuracy when these amounts of data are very large. The work presented by Ghazanfar and Pr¨ ugel- Bennett [8] has been also focused on this problem, generally for the user-based recommendation, which is the most used. Alternative interesting related work focused in the use of ML is the survey in sentiment classification presented by Hailong et al. [9]. In this work, the authors also provide a comparative study of the techniques found, concluding that supervised ML present a higher accuracy, while lexiconbased methods are likewise competitive because they require less effort and they are not sensitive to the quantity and quality of the training dataset. The survey presented by Mu [10] delivers a review of deep learning-based recommender systems. The authors conclude this work summarizing a set 2Scientific Programming of future research lines such as cross domain, scalability, explainability, or deep composite model-based recommender systems, among others. The paper presented by Portugal et al. [11] presents a systematic review of the use of ML in recommender systems. The authors analyzed 121 primary studies classified in different categories: content-based and neighbor-based of content-based filtering, neighborhood-based and modelbased of collaborative filtering, and hybrid filtering. This work helps developers to recognize the algorithms, their types, and trends in the use of specific algorithms. It also offers current-type evaluation metrics and categorizes the algorithms based on these metrics. Ouhbi et al. [12] proposed a deep learning-based recommender system to overcome some limitations of existing approaches. In the related work section of this paper, the authors describe a small state of the art of deep learning-based recommender systems, detailing the method, approach, metric, dataset, advantages, and disadvantages of seven proposals. Zhang et al. [13] delivered a wide review of deep learning-based recommender systems, proposing a classification and highlighting a group of the most influential. The authors debate the pros and cons of using deep learning techniques for recommendation tasks. Additionally, some of the most pressing open problems and promising future extensions are detailed. In summary, the literature review presented different topics, which may come close to the objective pursued. But there are several differences between these papers and the one presented in this work: (i) the review process: unlike the rest of the papers, this research presents a systematic and rigorous process, ensuring the quality of the results obtained; (ii) the context of application: usually reviews are carried out on the scientific literature; in this case, this research also presents a review on the industrial scope, analyzing the main existing solutions to the problem; and (iii) the scope of application: in this systematic review, the state of the art of the classification and recommendation systems is presented working together, something that in the related works already mentioned is not carried out or it is done independently for classification or recommendation. 3. Methodology A systematic literature review is an effective way of knowing the state of the art of a subject. This procedure ensures a certain level of quality of information and has the support of the research community. The monitoring of a systematic and guided process guarantees reliable and interesting results and facilitates the work of gathering information. The review presented in this paper is placed within the context of the recommendation and classification systems from two perspectives: scientific and industrial. When carrying out a systematic literature review (SLR), the main methodology to be considered is the one presented by Kitchenham and Charters [14]. This is one of the most widely accepted methods in the area of software engineering. It offers a way of performing a SLR consisting in three phases: planning and conducting the review and reporting of results. However, instead of performing a deep review of the papers comparing them, which is the main goal of a SLR, this study seeks to provide an overview of an interesting topic and to identify the number and type of published-related researches, as well as the related results available. Therefore, the best methodology to be applied is the systematic mapping study (SMS) presented by Petersen et al. [15], a type of the systematic review but with a broader objective. This method will allow identifying the subjects that lack empirical evidence and which are necessary to carry out more empirical studies. SMSs show many similarities with respect to the SLRs. As possible to see in activity diagram of Figure 1, this method stablishes a set of five steps, where each of them produces an output. These steps are as follows: (i) Definition of the Research Questions. Formulation of the research questions (RQs) that will guide the work. (ii) Conduct Search. The search is normally executed in different digital libraries and based on some keywords extracted from the RQs. (iii) Screening of Papers. Applying the inclusion and exclusion criteria with the aim of selecting the most relevant and close papers to the topic of the research. (iv) Keywording Using Abstract. Building of the classification scheme, where all the primary papers selected in the previous phase will be categorized. (v) Data Extraction and Mapping Process. Data extraction and mapping process based on the results obtained in the keywording activity. This activity will let the researchers to classify which is the state of the art of the topic and to identify gaps and possibilities for future research. 4. Definition of Research Questions A Research Question (RQ) is the fundamental core of a research project, study, or literature review. Therefore, to know and better understand the existing literature related to the recommendation and classifications systems, it is necessary to formulate a set of research questions. These questions will focus the study, will determine the methodology that will be established, and will guide all the stages of this research. In this sense, the RQs that have been proposed for this SMS are as follows: (i) RQ1. Which recommendation and classification systems have been researched? (ii) RQ2. Which recommendation and classification systems have been used? (iii) RQ3. Which is the nature of the systems found? (iv) RQ4. Which are the objectives pursued in the proposals found? 5. Conduct Search Before performing the search in the different digital libraries, it is necessary to complete two operations: define the digital Scientific Programming 3 libraries where the searches will be executed and establish the keywords that will compose the search strings. Selected digital libraries to carry out the search have been the following: SCOPUS, IEEE Xplore, ACM, and ScienceDirect. In addition, for the industrial scope, the search engines that have been selected are Google, Yahoo, and Bing. To specify the search, keywords were defined, and it is a fundamental part when creating the queries for each digital library. These keywords were obtained after carrying out an analysis of the field of study to which this research applies, recommendation and classification systems. Table 1 shows the complete set of keywords used, and equation (1) shows the formula applied to these keywords to create the final queries. Boolean expression of keywords is as follows: E1�V3 i�1Ai 􏼐 􏼑∧V4 j�1Bj 􏼐 􏼑∧V4 k�1Ck 􏼐 􏼑.(1) Once all the keywords were defined, the queries were constructed. These queries were different for each digital library, and they had different boundary characteristics, depending on the possibilities of the digital library. Digital libraries have certain limitations when conducting searches. For example, some of them do not allow the use of complete search strings; in others, it is necessary to complement these strings with simple textual searches. For this reason, there is the need to create individual queries for each library and, subsequently, to treat the search results to obtain the same results that could have been obtained using the originally proposed query. Table 2 shows a set of examples for each of digital library. The search was executed on the title, abstract, and keywords of the papers, except in those digital libraries that did not allow it. In such cases, the search was performed on the complete text. Search strings, metadata of found elements (title, author, and year of publication), and summaries of the documents were stored for each search source. Once the first search was executed, it obtained an initial set of 1,195 potential primary studies. 6. Screening of Papers There are different metrics to define the quality criteria that make a paper relevant. In this work, in addition to those related to the structure of the papers, the quality assurance criteria defined by those scientific papers found that were classified in the following accepted indexes: (i) “Journal Citation Report (JCR)” [16] part of the company Thomson Scientific (ii) The Australian classification created by the “Computing Research and Education Association of Australasia (CORE)” [17] Definition of research quesiton Start End Conducting search Screening of papers Keywording using abstracts Data extraction and mapping process Review scope All papers Relevant papers Classification scheme Systematic map Figure 1: SMS workflow. Table 1: Keyword definition. A B C A1. Machine learning B1. Recommender system C1. Classifier A2. Deep learning B3. Recommended C2. Classification A3. Neuronal networks B4. Content-based filtering C3. Classified B5. Collaborative filtering C4. Classification system Table 2: Example of queries. Digital library Query Scopus TITLE-ABS-KEY (“machine learning” OR “deep learning” OR “neuronal networks”) AND TITLEABS-KEY (“classifier” OR “classification” OR “classified” OR “classification system”) AND TITLE-ABS-KEY (“recommended” OR “recommender system” OR “collaborative filtering” OR “content-based filtering”) IEEE Xplore (“Machine learning” OR “deep learning” OR “neuronal networks”) AND (“classifier” OR “classification” OR “classified” OR “classification system”) AND (“recommended” OR “recommender system” OR “collaborative filtering” OR “content-based filtering”) ACM acmdlTitle: (“machine learning” “deep learning” “neuronal networks”) OR recordAbstract: (“machine learning” “deep learning” “neuronal networks”) AND (acmdlTitle: (“classifier” “classification” “classified” “classification system”) OR recordAbstract: (“classifier” “classification” “classified” “classification system”)) AND (acmdlTitle: (“recommended” “recommender system” “collaborative filtering” “content-based filtering”) OR recordAbstract: (“recommended” “recommender system” “collaborative filtering” “content-based filtering”)) Science direct (“Machine learning” OR “deep learning” OR “neuronal networks”) AND (“classifier” OR “classification” OR “classified” OR “classification system”) AND (“recommended” OR “recommender system” OR “collaborative filtering” OR “content-based filtering”) 4Scientific Programming (iii) The ranking of relevant congresses for the Scientific Information Society of Spain (SCIE) [18], advising the use of the ranking developed by the Italian associations GII and GRIN [19] In addition, the following inclusion and exclusion criteria were defined for including or being not a publication into the selected primary studies: (i) C1, Criterion 1. The classification of the publication in question must be “Computer Science” (ii) C2, Criterion 2. Written in English (iii) C3, Criterion 3. The research must be related to the classification and recommendation of data using machine learning systems (iv) C4, Criterion 4. Searches cannot be repeated. Multiple appearances must be eliminated (v) C5, Criterion 5. As mentioned above, papers must be classified into the JCR or SCIE rakings (vi) C6, Criterion 6. The reading of the abstract must fit with the dealt topic Finally, some recommendations from experts in the subject dealt with in this SMS have also been considered. If these studies were not found after the execution of the different searches, they were included in the final selection of primary studies. Once defined the quality and inclusion and exclusion criteria, the screening of the papers was performed. According to the C1 of inclusion/exclusion of papers which scope is related to “Computer science,” a total of 923 results were obtained, having discarded 272 papers that did not meet this criterion. C2 was applied to the 923 papers obtained from C1 resulting on 909 papers. To the results obtained from C2, C3 criterion was applied leaving a total of 432 results. Once C4 was applied, a total of 96 papers were removed remaining 336. A total of 259 papers was the result of applying C5. The last filter, C6, was applied resulting on 99 papers considering that 160 of the removed ones did not fit the topic of this research. Finally, repeated papers were removed. This process ended up removing duplicated entries between the different digital libraries. The result of applying all the quality and inclusion and exclusion criteria was a total of 80 primary studies which will be categorized into the classification schema. The number of papers found corresponds (roughly) to 6% of the results found in the first search. Table 3 shows the primary studies selected. Figure 2 shows the list of keywords discovered in the different primary studies. In this figure, the keywords are classified based on the total number of matches found between all these primary studies. Figure 3 depicts the complete process of selecting primary studies. It shows the search procedure for each digital library and the results after the application of each quality and inclusion and exclusion criteria. By the same token, the process carried out previously was executed for the industrial scope for detecting and selecting Table 3: Selected primary studies. Title Reference Building accurate and practical recommender system algorithms using machine learning classifier and collaborative filtering [20] DGA botnet detection using collaborative filtering and density-based clustering [21] A multistage collaborative filtering method for fall detection [22] Analysis and performance of collaborative filtering and classification algorithms [1] Extracting a vocabulary of surprise by collaborative filtering mixture and analysis of feelings [4] Content based filtering in online social network using inference algorithm [23] Building switching hybrid recommender system using machine learning classifiers and collaborative filtering [8] Imputation-boosted collaborative filtering using machine learning classifiers [24] CRISP-an interruption management algorithm based on collaborative filtering [25] A credit scoring model based on collaborative filtering [26] Collaborative filtering recommender systems [2] An improved switching hybrid recommender system using naive Bayes classifier and collaborative filtering [6] Tweet modeling with LSTM recurrent neural networks for hashtag recommendation [27] A two-stage cross-domain recommendation for cold start problem in cyber-physical systems [28] ELM based imputation-boosted proactive recommender systems [29] Twitter-user recommender system using tweets: a content-based approach [30] A personalized time-bound activity recommendation system [31] Automated content based short text classification for filtering undesired posts on Facebook [32] Shilling attack detection in collaborative recommender systems using a meta learning strategy [33] Building a distributed generic recommender using scalable data mining library [34] Context-aware movie recommendation based on signal processing and machine learning [35] Recommender systems using linear classifiers [36] A survey of accuracy evaluation metrics of recommendation tasks [3] Incorporating user control into recommender systems based on naive Bayesian classification [37] Classification features for attack detection in collaborative recommender systems [38] Automatic tag recommendation algorithms for social recommender systems [39] Optimizing similar item recommendations in a semistructured marketplace to maximize conversion [40] Capturing knowledge of user preferences: ontologies in recommender systems [41] Emotion-based music recommendation using supervised learning [42] AWESOME—a data warehouse-based system for adaptive website recommendations [43] Scientific Programming 5 the primary technologies or tools that companies offer. The search engines returned multiple results (Table 4), with a total of 21 proposals remaining were potential candidates. Table 3: Continued. Title Reference Lexical and syntactic features selection for an adaptive reading recommendation system based on text complexity [5] A smart-device news recommendation technology based on the user click behavior [44] Recommendation as link prediction in bipartite graphs: A graph kernel-based machine learning approach [45] A novel approach towards context based recommendations using support vector machine methodology [46] A smartphone-based activity-aware system for music streaming recommendation [47] An app usage recommender system: improving prediction accuracy for both warm and cold start users [48] Proposing design recommendations for an intelligent recommender system logging stress [49] A recommender system based on implicit feedback for selective dissemination of eBooks [50] A novel recommender system based on FFT with machine learning for predicting and identifying heart diseases [51] Shilling attack detection in collaborative recommender systems using a meta learning strategy [33] An approach to content based recommender systems using decision list based classification with k-DNF rule set [52] Probabilistic approach for QoS-aware recommender system for trustworthy web service selection [53] Approach to cold-start problem in recommender systems in the context of web-based education [54] Context and intention-awareness in POIs recommender systems [55] A collaborative filtering-based re-ranking strategy for search in digital libraries [56] Learning users’ interests by quality classification in market-based recommender systems [57] Mobile content recommendation system for revisiting user using content-based filtering and clientside user profile [58] A hybrid collaborative filtering algorithm based on KNN and gradient boosting [59] A scalable collaborative filtering algorithm based on localized preference [60] Recommended or not recommended? Review classification through opinion extraction [61] Meta-feature based data mining service selection and recommendation using machine learning models [62] Personalized channel recommendation deep learning from a switch sequence [63] Affective labeling in a content-based recommender system for images [64] A novel approach towards context sensitive recommendations based on machine learning methodology [65] A distance-based approach for action recommendation [66] Ranking and classifying attractiveness of photos in folksonomies [67] Table 3: Continued. Title Reference Consequences of variability in classifier performance estimates [68] Machine learning and lexicon based methods for sentiment classification: a survey [9] Machine learning algorithm selection for forecasting behavior of global institutional investors [69] Towards rapid interactive machine learning: evaluating tradeoffs of classification without representation [70] Towards a method for automatically evolving Bayesian network classifiers [71] A machine learning based trust evaluation framework for online social networks [72] Automated problem identification: regression vs. classification via evolutionary deep networks [73] Empirical evaluation of ranking prediction methods for gene expression data classification [74] Inferring contextual preferences using deep autoencoding [75] Automatic recognition of text difficulty from consumers health information [76] A hybrid approach for automatic model recommendation [77] Learning instance greedily cloning naive Bayes for ranking [78] Pairwise-ranking based collaborative recurrent neural networks for clinical event prediction [79] Accurate multi-criteria decision making methodology for recommending machine learning algorithm [80] A general extensible learning approach for multidisease recommendations in a telehealth environment [81] An efficient recommendation generation using relevant jaccard similarity [82] An image-based segmentation recommender using crowdsourcing and transfer learning for skin lesion extraction [83] Automatic classification of high resolution land cover using a new data weighting procedure: the combination of k-means clustering algorithm and central tendency measures (KMC–CTM) [84] Building a hospital referral expert system with a prediction and optimization-based decision support system algorithm [85] Classification techniques on computerized systems to predict and/or to detect apnea: a systematic review [86] Identification of category associations using a multilabel classifier [87] Making use of associative classifiers in order to alleviate typical drawbacks in recommender systems [88] S3Mining: a model-driven engineering approach for supporting novice data miners in selecting suitable classifiers [89] The use of machine learning algorithms in recommender systems: a systematic review [11] 6Scientific Programming 7. Keywording using Abstracts To create the classification scheme for categorizing the selected primary studies, an attempt was made to answer each of the research questions formulated in the planning phase and, in addition, to identify each of them with a set of features. Moreover, two complete iterations were carried out to classify all the studies and to verify that all the features that had been found included the content of each study. Table 5 shows and describes the classification scheme defined. Thereupon, process for the definition of the classification scheme is repeated for the industrial area. Through the answer to the research questions and the extraction of the technologies’ features, a classification scheme was defined (Table 6). 8. Data Extraction and Mapping Process 8.1. Scientific Report. This section describes the most important aspects obtained from the information collected. To achieve this purpose, each of the research questions will be answered and validated, showing the data obtained for each of them. It is important to note that some of the features may appear in several studies; therefore, the totals may not always correspond to 100%. (i) Research Question RQ1 finds the methods, techniques, and/or tools that have been investigated for the classification and recommendation systems. Figure 4 shows that the predominant type of studies is methods, which represent 35.00% of the total of the studies, followed by the complete system studies, with a 23.75%. The rest of studies correspond to algorithms with 20.00%, analysis with a presence of 18.75%, and finally, frameworks with a 6.25% of the total primary studies. From a software development life-cycle perspective (and avoiding methodological discussions), requirements and analysis phases differ from the design phase because it is an earlier stage and closer to the business (or the application model) and is completely technology independent. Then, the found works are contextualized in the technological design phase. No contextualized work was found in early stages (business requirements or analysis). (ii) Research Question RQ2 seeks to know the validation of the studies found, which may be practical or theoretical, identifying if they are within the scientific or industrial scope. The results obtained (Figure 5) show that all the primary studies were academic focused. Most of them were validated by some way (97.50%), while 10.00% were not validated. It is important to note that three different groups have been distinguished within the validation category. The experimentation subgroup includes all those studies whose proposal was tested and validated by experimentation with synthetic and real data sources. This group contains most of the results found that were validated, 72.50% of the total. Another important category is the one that validates the proposals by a case study, which represent 13.75%. Only the 5.00% of the primary studies were carried out through surveys, and just one primary study was focused on the industrial context, representing the 1.25% of the total. 13 12 8 77 66 5 444 3333333333 222222222222222222 0 2 4 6 8 10 12 14 Recommender systems Collaborative filtering Machine learning Classification Data mining Classification algorithms Recommender systems Deep learning Recommender systems Machine learning Pattern classification Context Support vector machines Collaborative filtering Machine learning Learning systems Support vector machine Learning (artificial intelligence) Filtering Machine learning algorithms Accuracy Prediction algorithms Artificial intelligence Decision trees Content-based filtering Social networking (online) Feature extraction Internet Neural network Recommender systems Training Motion pictures Recommender systems Classification Text categorization Information technology User profiling Computational modeling Collaboration Figure 2: Keyword mapping. Scientific Programming 7 (iii) Research Question 3 aims to identify the nature of the methods, techniques, and/or tools about the classification and recommendation systems found in the literature. Figure 6 groups two main categories that contain the whole set of features of the primary studies found: recommendation and classification. Within the recommendation group, content-based and collaborative filtering proposals are very balanced, representing the 36.25% and 38.75%, respectively. Hybrid systems are the worst classified with 17.50% of the papers. Furthermore, the classification group is described, where both supervised and unsupervised learning features are presented. Two features stand out for their use: naive Bayes to classify according to probabilities with a 28.75% and support vectors, representing the 20.00% of total. Target based and Random Forest are the less used, with a presence of just 1 primary study. (iv) Research Question RQ4 indicates which are the main points of interest of the research and which areas have been less investigated. This interest is classified into four categories: novelty, analysis, research, and improvement (Figure 7). The novelty contains those primary studies whose goal is to present something that lacked in the literature, and this category represents 22.50%, with 18 primary studies. Analysis category contains those results that are comparison or study of different existing techniques, and it represents the 7.50% of total. The improvement category represents that 30.00% of the results whose main objective is to improve an existing approach. Finally, the largest category is the research one, were a search on existing or new approaches in the literature is dealt with. It represents the 36.25% of total with 29 primary studies. At last, it is interesting to analyze other results that are not related to the research questions but with the objective of this document. These results can help to know the evolution of the research of the classification and recommendation systems. 583 studies Digital libraries Scopus IEEE Xplore ACM ScienceDirect C1 C2 C3 C4 C5 C6 C1C1C1 C2 C3 C4 C5 C6 C2 C3 C4 C5 C6 C2 C3 C4 C5 C6 99 primary studies 80 primary studies 442 studies 428 studies 64 studies 64 studies 40 studies 27 studies 148 studies 148 studies 148 studies 35 studies 35 studies 24 studies 16 studies 362 studies 277 studies 277 studies 277 studies 181 studies 142 studies 39 studies 102 studies 56 studies 56 studies 56 studies 56 studies 53 studies 17 studies Removing duplicates C1: computer science C2: written in english C3: do not fit the selected topic C4: removing duplicates C5: are not JCR or SCIE classified C6: filtering reading abstracts Figure 3: Search flow diagram: scientific field. 8Scientific Programming Table 5: Scientific classification scheme. Research Question Feature Description RQ1 Algorithm This feature defines if the primary study proposes an algorithm or series of algorithms System This feature defines if the primary study is a software system based on different components Framework This feature defines whether the primary study is based on a framework Method This feature defines if the primary study is a set of procedures to obtain a result Analysis This feature defines if the primary study is a theoretical study based on surveys or systematic reviews among others RQ2 Validated This feature defines whether the primary study has been validated with experiments, use cases, or surveys Not validated This feature defines if the primary study has not been validated with experiments, use cases, or surveys Academic This feature defines if the primary study has been validated with some academic case study Industrial This feature defines if the primary study has been validated with some case study in the industry Experiment This feature defines if the primary study has been validated with the elaboration of different experiments Use case This feature defines whether the primary study has been validated with the study of a use case Survey This feature defines if the primary study has been validated with the elaboration of some type of survey Table 4: Selected primary technologies/tools. Technology Reference Scikit-learn [90] Surprise [91] LightFM [92] Rexy [93] PredictionIO [94] HapiGER [95] LensKit [96] SuggestGrid [97] SLI Systems Recommender [98] AmazonWebService Machine Learning [99] Azure ML Studio [100] Yusp [101] IBM Watson [102] Recombee [103] Mr. DLib [104] Caret [105] Shiny [106] RandomForest [107] KlaR [108] CORElearn [109] RecommenderLab [110] Scientific Programming 9 [17] J. L. C. Izquierdo, V. Cosentino, and J. Cabot, “Analysis of co-authorship graphs of CORE-ranked software conferences,” Scientometrics, vol. 109, no. 3, pp. 1665–1693, 2016. [18] SCIE, “La Sociedad Cient´ ıfica Inform´ atica de España,” 2017. [19] SCIE, “GII-GRIN-SCIE (GGS) Conference Rating,”2019. [20] A. Sattar, M. A. Ghazanfar, and M. Iqbal, “Building accurate and practical recommender system algorithms using machine learning classifier and collaborative filtering,” Arabian Journal for Science and Engineering, vol. 42, no. 8, pp. 3229–3247, 2017. [21] T.-D. Nguyen, T.-D. Cao, and L.-G. Nguyen, “DGA botnet detection using collaborative filtering and density-based clustering,” in Proceedings of the Sixth International Symposium on Information and Communication Technology, pp. 203–209, Hue City, Vietnam, December 2015. [22] T. Xie, Y. Chen, L. Hu, C. Gao, C. Hu, and J. Shen, “A multistage collaborative filtering method for fall detection,” in Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Rio, Brazil, August 2017. [23] N. Thilagavathi and R. Taarika, “Content based filtering in online social network using inference algorithm,” in Proceedings of the 2014 International Conference on Circuits, Power and Computing Technologies (ICCPCT), Nagercoil, India, March 2014. [24] X. Su, T. M. Khoshgoftaar, X. Zhu, and R. Greiner, “Imputation-boosted collaborative filtering using machine learning classifiers,” in Proceedings of the 2008 ACM Symposium on Applied Computing—SAC ’08, Fortaleza, Cear´ a, Brazil, March 2008. [25] T. Shrot, A. Rosenfeld, J. Golbeck, and S. Kraus, “CRISP -an interruption management algorithm based on collaborative filtering,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, Canada, 2014. [26] X. Zheng, “A credit scoring model based on collaborative filtering,” in Proceedings of the 9th International Conference on Computational Intelligence and Security, Emei Mountain, Sichuan, China, December 2013. [27] J. Li, H. Xu, X. He, J. Deng, and X. Sun, “Tweet modeling with LSTM recurrent neural networks for hashtag recommendation,” in Proceedings of the International Joint Conference on Neural Networks, Vancouver, British Columbia, Canada, 2016. [28] P. Liu, J. Cao, X. Liang, and W. Li, “A two-stage crossdomain recommendation for cold start problem in cyberphysical systems,” in Proceedings of the International Conference on Machine Learning and Cybernetics, Guangzhou, China, 2015. [29] P. Bedi, Richa, S. K. Agarwal, and V. Bhasin, “ELM based imputation-boosted proactive recommender systems,” in Proceedings of the 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Jaipur, India, September 2016. [30] R. H. Nidhi and B. Annappa, “Twitter-user recommender system using tweets: a content-based approach,” in Proceedings of the ICCIDS 2017 International Conference on Computational Intelligence in Data Science, pp. 1–6, Chennai, India, June 2017. [31] R. Mittal and V. Sinha, “A personalized time-bound activity recommendation system,” in Proceedings of the 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, USA, January 2017. [32] A. S. Vairagade and R. A. Fadnavis, “Automated content based short text classification for filtering undesired posts on Facebook,” in Proceedings of the IEEE World Conference on Futuristic Trends in Research and Innovation for Social Welfare (WCTFTR), Coimbatore, India, 2016. [33] W. Bhebe and O. P. Kogeda, “Shilling attack detection in collaborative recommender systems using a meta learning strategy,” in Proceedings of the 2015 International Conference on Emerging Trends in Networks and Computer Communications (ETNCC), pp. 56–61, IEEE, Windhoek, Namibia, May 2015. [34] L. Bhatia and S. S. Prasad, “Building a distributed generic recommender using scalable data mining library,” in Proceedings of the 2015 IEEE International Conference on Computational Intelligence and Communication Technology (CICT), Ghaziabad, India, 2015. [35] C. Biancalana, F. Gasparetti, A. Micarelli, A. Miola, and G. Sansonetti, “Context-aware movie recommendation based on signal processing and machine learning,” in Proceedings of the 2nd Challenge on Context-Aware Movie Recommendation, Chicago, IL, USA, 2011. [36] T. Zhang and V. S. Iyengar, “Recommender systems using linear classifiers,” Journal of Machine Learning Research, pp. 313–334, 2002. [37] V. Pronk, W. Verhaegh, A. Proidl, and M. Tiemann, “Incorporating user control into recommender systems based on naive Bayesian classification,” in Proceedings of the ACM International Conference on Recommender Systems, Minneapolis, MN, USA, 2007. [38] R. Burke, B. Mobasher, C. Williams, and R. Bhaumik, “Classification features for attack detection in collaborative recommender systems,” in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining—KDD ’06, Philadelphia, PA, USA, August 2006. [39] Y. Song, L. Zhang, and C. L. Giles, “Automatic tag recommendation algorithms for social recommender systems,” ACM Transactions on the Web, vol. 5, no. 1, p. 31, 2011. [40] Y. M. Brovman, “Optimizing similar item recommendations in a semi-structured marketplace to maximize conversion,” in Proceedings of the 10th ACM Conference on Recommender Systems—RecSys ’16, Boston, MA, USA, September 2016. [41] S. E. Middleton, D. C. De Roure, and N. R. Shadbolt, “Capturing knowledge of user preferences,” in Proceedings of the International Conference on Knowledge capture—K-CAP, Victoria, BC, Canada, 2001. [42] P. P. Jean-Jacques, J. Noack, and K. Bodarw´e, “Emotionbased music recommendation using supervised learning,” in Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia, Linz, Austria, December 2015. [43] A. Thor and E. Rahm, “AWESOME—A Data Warehouse- Based System for Adaptive Website Recommendations,” in Proceedings of the Thirtieth International Conference on Very Large Data Bases, vol. 30, pp. 384–395, VLDB Endowment, Toronto, Ontario, Canada, September 2004. [44] Y. H. Gu, S. J. Yoo, Z. Piao, J. No, Z. Jiang, and H. Yin, “A smart-device news recommendation technology based on the user click behavior,” in Proceedings of the Sixth International Conference on Emerging Databases: Technologies, Applications, and Theory, pp. 9–16, Jeju Island, Republic of Korea, October 2016. [45] X. Li and H. Chen, “Recommendation as link prediction in bipartite graphs: a graph kernel-based machine learning approach,” Decision Support Systems, vol. 54, no. 2, pp. 880–890, 2013. [46] A. A. Kothari and W. D. Patel, “A novel approach towards context based recommendations using support vector 16 Scientific Programming machine methodology,” Procedia Computer Science, vol. 57, pp. 1171–1178, 2015. [47] W. P. Lee, C. T. Chen, J. Y. Huang, and J. Y. Liang, “A smartphone-based activity-aware system for music streaming recommendation,” Knowledge-Based Systems, vol. 131, pp. 70–82, 2017. [48] D. Han, J. Li, W. Li, R. Liu, and H. Chen, An App Usage Recommender System: Improving Prediction Accuracy for Both Warm and Cold Start Users, Multimedia Systems, 2019. [49] A. Visuri, R. Poguntke, and E. Kuosmanen, Proposing Design Recommendations for an Intelligent Recommender System Logging Stress, Association for Computing Machinery, New York, NY, USA, 2018. [50] E. R. N´uñez-Valdez, D. Quintana, R. G. Crespo, P. Isasi, and E. Herrera-Viedma, “A recommender system based on implicit feedback for selective dissemination of ebooks,” Information Sciences, vol. 467, pp. 87–98, 2018. [51] S. Narayan and E. Sathiyamoorthy, “A novel recommender system based on FFT with machine learning for predicting and identifying heart diseases,” Neural Computing and Applications, vol. 31, no. S1, pp. 93–102, 2019. [52] A. Pujahari and V. Padmanabhan, “An approach to content based recommender systems using decision list based classification with k-DNF rule set,” in Proceedings of the 2014 13th International Conference on Information Technology (ICIT), Bhubaneswar, India, December 2014. [53] M. Mehdi, N. Bouguila, and J. Bentahar, “Probabilistic approach for QoS-aware recommender system for trustworthy web service selection,” Applied Intelligence, vol. 41, no. 2, pp. 503–524, 2014. [54] R. A. Gotardo, E. R. Hruschka, S. D. Zorzo, and P. R. M. Cereda, “Approach to cold-start problem in recommender systems in the context of web-based education,” in Proceedings of the 2013 12th International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA, December 2013. [55] H. Costa, B. Furtado, D. Pires, L. Macedo, and A. Cardoso, “Context and intention-awareness in POIs recommender systems,” in Proceedings of the 6th ACM Recommender Systems Conference, 4th Workshop on Context-Aware Recommender Systems (RecSys), vol. 12, p. 5, Dubai, UAE, September 2012. [56] U. Rohini and V. Ambati, “A collaborative filtering based reranking strategy for search in digital libraries,” in Lecture Notes in Computer Science, Springer, Berlin, Germany, 2005. [57] Y. Z. Wei, L. Moreau, and N. R. Jennings, “Learning users’ interests by quality classification in market-based recommender systems,” IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 12, pp. 1678–1688, 2005. [58] W. Paireekreng, “Mobile content recommendation system for re-visiting user using content-based filtering and clientside user profile,” in Proceedings—International Conference on Machine Learning and Cybernetics, Lanzhou, China, 2013. [59] S. Lu, B. Wang, H. Wang, and Q. Hong, “A hybrid collaborative filtering algorithm based on KNN and gradient boosting,” in Proceedings of the 13th International Conference on Computer Science and Education (ICCSE), Colombo, Sri Lanka, August 2018. [60] L. Zhang, B. Xiao, J. Guo, and C. Zhu, “A scalable collaborative filtering algorithm based on localized preference,” in Proceedings of the 7th International Conference on Machine Learning and Cybernetics (ICMLC), Melbourne, Australia, December 2008. [61] S. Feng, M. Zhang, Y. Zhang, and Z. Deng, “Recommended or not recommended? Review classification through opinion extraction,” in Proceedings of the 12th Asia-Pacific Web Conference, Advances in Web Technologies and Applications (APWeb), Busan, Korea, April 2010. [62] B. Alghofaily and C. Ding, “Meta-feature based data mining service selection and recommendation using machine learning models,” in Proceedings of the 2018 IEEE 15th International Conference on e-Business Engineering (ICEBE), Xi’an, China, October 2018. [63] C. Yang, S. Ren, Y. Liu, H. Cao, Q. Yuan, and G. Han, “Personalized channel recommendation deep learning from a switch sequence,” IEEE Access, vol. 6, pp. 50824–50838, 2018. [64] M. Tkalˇ ciˇ c, A. Odi´ c, A. KoTkalˇ siˇ cir, and J. Tasiˇ c, “Affective labeling in a content-based recommender system for images,” IEEE Transactions on Multimedia, vol. 15, no. 2, pp. 391–400, 2013. [65] A. A. Kothari and W. D. Patel, “A novel approach towards context sensitive recommendations based on machine learning methodology,” in Proceedings of the 2015 5th International Conference on Communication Systems and Network Technologies (CSNT), Gwalior, MP, India, April 2015. [66] R. Trepos, A. Salleb, M. O. Cordier, V. Masson, and C. Gascuel, “A distance-based approach for action recommendation,” in Lecture Notes in Computer Science, Springer, Berlin, Germany, 2005. [67] J. S. Pedro and S. Siersdorfer, “Ranking and Classifying Attractiveness of Photos in Folksonomies,” in Proceedings of the 18th International Conference on World Wide Web, pp. 771–780, ACM, Madrid, Spain, April 2009. [68] T. Raeder, T. R. Hoens, and N. V. Chawla, “Consequences of variability in classifier performance estimates,” in Proceedings of the IEEE International Conference on Data Mining (ICDM), Sydney, Australia, 2010. [69] J. J. Ahn, S. J. Lee, K. J. Oh, T. Y. Kim, H. Y. Lee, and M. S. Kim, “Machine learning algorithm selection for forecasting behavior of global institutional investors,” in Proceedings of the 42nd Annual Hawaii International Conference on System Sciences (HICSS), Waikoloa, Hawaii, January 2009. [70] D. Arendt, E. Saldanha, R. Wesslen, S. Volkova, and W. Dou, “Towards rapid interactive machine learning: evaluating tradeoffs of classification without representation,” in Proceedings of the 24th International Conference on Intelligent User Interfaces, pp. 591–602, Marina del Ray, CA, USA, March 2019. [71] A. G. C. de S´a and G. L. Pappa, “Towards a method for automatically evolving bayesian network classifiers,” in Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 1505–1512, ACM, Amsterdam, Netherlands, July 2013. [72] K. Zhao and L. Pan, “A machine learning based trust evaluation framework for online social networks,” in Proceedings of the 2014 IEEE 13th International Conference on Trust, Security and Privacy in Computing and Communications, Beijing, China, September 2014. [73] E. Dufourq and B. A. Bassett, “Automated problem identification: regression vs. classification via evolutionary deep networks,” in Proceedings of the South African Institute of Computer Scientists and Information Technologists, p. 12, Thaba Nchu, South Africa, September 2017. Scientific Programming 17 [74] B. F. De Souza, A. C. P. L. F. De Carvalho, and C. Soares, “Empirical evaluation of ranking prediction methods for gene expression data classification,” in Lecture Notes in Computer Science, Springer, Berlin, Germany, 2010. [75] M. Unger, B. Shapira, L. Rokach, and A. Bar, “Inferring contextual preferences using deep auto-encoding,” in Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, pp. 221–229, ACM, Bratislava, Slovakia, July 2017. [76] W. Yunli, “Automatic recognition of text difficulty from consumers health information,” in Proceedings of the IEEE Symposium on Computer-Based Medical Systems, Salt Lake City, Utah, 2006. [77] R. Vainshtein, A. Greenstein-Messica, G. Katz, B. Shapira, and L. Rokach, “A hybrid approach for automatic model recommendation,” in Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 1623–1626, ACM, Turin, Italy, October 2018. [78] L. Jiang and H. Zhang, “Learning instance greedily cloning naive Bayes for ranking,” in Proceedings of the IEEE International Conference on Data Mining (ICDM), p. 8, IEEE, Houston, TX, USA, 2005. [79] Z. Qiao, S. Zhao, C. Xiao, X. Li, Y. Qin, and F. Wang, “Pairwise-ranking based collaborative recurrent neural networks for clinical event prediction,” in Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Stockholm, Sweden, July 2018. [80] R. Ali, S. Lee, and T. C. Chung, “Accurate multi-criteria decision making methodology for recommending machine learning algorithm,” Expert Systems with Applications, vol. 71, pp. 257–278, 2017. [81] R. Lafta, J. Zhang, X. Tao et al., “A general extensible learning approach for multi-disease recommendations in a telehealth environment,” Pattern Recognition Letters, 2018. [82] S. Bag, S. K. Kumar, and M. K. Tiwari, “An efficient recommendation generation using relevant jaccard similarity,” Information Sciences, vol. 483, pp. 53–64, 2019. [83] A. Soudani and W. Barhoumi, “An image-based segmentation recommender using crowdsourcing and transfer learning for skin lesion extraction,” Expert Systems with Applications, vol. 118, pp. 400–410, 2019. [84] S. S. Durduran, “Automatic classification of high resolution land cover using a new data weighting procedure: the combination of k-means clustering algorithm and central tendency measures (KMC-CTM),” Applied Soft Computing, vol. 35, pp. 136–150, 2015. [85] C. L. Chi, W. N. Street, and M. M. Ward, “Building a hospital referral expert system with a prediction and optimizationbased decision support system algorithm,” Journal of Biomedical Informatics, vol. 41, no. 2, pp. 371–386, 2008. [86] N. Pombo, N. Garcia, and K. Bousson, “Classification techniques on computerized systems to predict and/or to detect apnea: a systematic review,” Computer Methods and Programs in Biomedicine, vol. 140, pp. 265–274, 2017. [87] J. Szyma´ nski and J. Rzeniewicz, “Identification of category associations using a multilabel classifier,” Expert Systems with Applications, vol. 61, pp. 327–342, 2016. [88] J. Pinho Lucas, S. Segrera, and M. N. Moreno, “Making use of associative classifiers in order to alleviate typical drawbacks in recommender systems,” Expert Systems with Applications, vol. 39, no. 1, pp. 1273–1283, 2012. [89] R. Espinosa, D. Garc´ ıa-Saiz, M. Zorrilla, J. J. Zubcoff, and J. N. Maz´ on, “S3mining: a model-driven engineering approach for supporting novice data miners in selecting suitable classifiers,” Computer Standards & Interfaces, vol. 65, pp. 143–158, 2019. [90] D. Cournapeau, “Scikit-learn,” 2019. [91] N. Hug, “Surprise,” 2019. [92] M. Kula, “LightFM,” in Proceedings of the 2nd Workshop on New Trends on Content-Based Recommender Systems Co- Located with 9th ACM, Vienna, Austria, September 2015. [93] K. Vand, “Rexy,” 2019. [94] A. S. Foundation, PredictionIO, A. S. Foundation, Pune, Maharashtra, 2019. [95] G. Jenson, “HapiGER,” 2019. [96] L. C. A. and Credits, “LensKit,” 2019. [97] I. SuggestGrid, “SuggestGrid,” 2019. [98] S. Systems, “SLI Systems Recommender,” 2019. [99] A. W. Services, “AmazonWebService Machine Learning,” 2019. [100] Microsoft, “Azure ML Studio,” 2019. [101] Gravity Research & Development, “Yusp,” 2019. [102] IBM Watson Studio, “IBM Watson,” 2019. [103] Recombee, “Recombee,” 2019. [104] Mr. Dlib, “Mr. DLib,” 2019. [105] Caret, “Caret,” 2019. [106] Shiny, “Shiny,” 2019. [107] RandomForest, “RandomForest,” 2019. [108] KlaR, “KlaR,” 2019. [109] CORElearn, “CORElearn,” 2019. [110] RecommenderLab, “RecommenderLab,” 2019. [111] F. J. Dom´ ınguez-Mayo, M. J. Escalona, and M. Mej´ ıas, “QuEF (quality evaluation framework) for model-driven web methodologies,” in Lecture Notes in Computer Science, Springer, Berlin, Germany, 2010. 18 Scientific Programming Computer Games Technology International Journal of Hindawi www.hindawi.com Volume 2018 Hindawi www.hindawi.com Journal of Engineering Volume 2018 Advances in Fuzzy Systems Hindawi www.hindawi.com Volume 2018 International Journal of Reconfigurable Computing Hindawi www.hindawi.com Volume 2018 Hindawi www.hindawi.com Volume 2018 Applied Computational Intelligence and Soft Computing Advances in Artificial Intelligence Hindawi www.hindawi.com Volume 2018 Hindawi www.hindawi.com Volume 2018 Civil Engineering Advances in Hindawi www.hindawi.com Volume 2018 Electrical and Computer Engineering Journal of Journal of Computer Networks and Communications Hindawi www.hindawi.com Volume 2018 Hindawi www.hindawi.com Volume 2018 Advances in Multimedia International Journal of Biomedical Imaging Hindawi www.hindawi.com Volume 2018 Hindawi www.hindawi.com Volume 2018 Engineering Mathematics International Journal of Robotics Journal of Hindawi www.hindawi.com Volume 2018 Hindawi www.hindawi.com Volume 2018 Computational Intelligence and Neuroscience Hindawi www.hindawi.com Volume 2018 Mathematical Problems in Engineering Modelling & Simulation in Engineering Hindawi www.hindawi.com Volume 2018 Hindawi Publishing Corporation http://www.hindawi.com Volume 2013 Hindawi www.hindawi.com The Scientific World Journal Volume 2018 Hindawi www.hindawi.com Volume 2018 Human-Computer Interaction Advances in Hindawi www.hindawi.com Volume 2018 Scientific Programming Submit your manuscripts at www.hindawi.com