Analyzing inconsistencies in software product lines using an ontological rule-based approach

Analyzing inconsistencies in software product lines using an ontological rule-based approach Megha Bhushan a, ∗, Shivani Goel b, Karamjit Kaur a a CSED, Thapar University, Patiala, Punjab, India, University of Seville, Spain b Department of Computer Science Engineering, Bennett University, Greater Noida, U.P., India a r t i c l e i n f o Article history: Received 23 March 2016 Revised 14 January 2017 Accepted 1 June 2017 Available online 3 June 2017 Keywords: Feature model Software product line Rule-based approach Ontology Inconsistency Defects a b s t r a c t Software product line engineering (SPLE) is an evolving technical paradigm for generating software products. Feature model (FM) represents commonality and variability of a group of software products that appears within a specific domain. The quality of FMs is one of the factors that impacts the correctness of software product line (SPL). Developing FMs might also incorporate inaccurate relationships among features which cause numerous defects in models. Inconsistency is one of such defect that decreases the benefits of SPL. Existing approaches have focused in identifying inconsistencies in FMs however, only a few of these approaches are able to provide their causes. In this paper FM is formalized from an ontological view by converting model into a predicate-based ontology and defining a set of first-order logic based rules for identifying FM inconsistencies along with their causes in natural language in order to assist developers with solutions to fix defects. A FM available in software product lines online tools repository has been used to explain the presented approach and validated using 24 FMs of varied sizes up to 22,035 features. Evaluation results demonstrate that our approach is effective and accurate for the FMs scalable up to thousands of features and thus, improves SPL. 1. Introduction A software product line (SPL) is a family of related software intensive systems, sharing a common and managed set of features that fulfill the exact requirements of an appropriate market segment ( Clements and Northrop, 2001 ). The main focus of SPL is software reuse in an attempt to improve the quality and productivity while reducing cost as well as time to market. Although other variability models for modeling variability and commonality exist, such as decision models ( Schmid and John, 2004 ), dopler variability models ( Dhungana et al., 2011 ), orthogonal variability models ( Pohl et al., 2005 ) and textual variability language ( Classen et al., 2011 ). However, feature model (FM) is the most popular variability model for SPL that illustrates the features and their relationships ( Kang et al., 1990 ). There are various factors that impact the success of SPL such as (i) testing which includes to validate and verify the generated software products, (ii) poor interface among customers and organization hinders communication of information and customer satisfaction, (iii) reusability decreases with developing product line (PL) which includes bug ∗Corresponding author. E-mail addresses: [email protected] (M. Bhushan), [email protected] fixing, performance etc., (iv) overloaded software development system becomes a challenge in the development and maintenance of the system, and (iv) low quality software products can lead to the development of defective new products. Therefore, ensuring quality in the software development paradigm is of utmost importance. The complexity of FMs increases with growing number of features and relationships which cause an increase in the contradictory relations. These contradictory relationships arise due to the combination of features derived from multiple software products in order to develop a new product. Presence of a single contradictory relation in a FM can cause an inconsistency defect which makes the model inconsistent. Inconsistency in FMs is a critical issue as software products are derived by reusing models. A FM with inconsistency defects has contradictory information i.e., the information which conflicts with other information in the similar model. It will not allow deriving products, as these products may include inconsistent configurations. Therefore, defect due to inconsistency is one of the major factors that deteriorates the reusability and quality of models. For example, various developers that belong to the same business unit working on a SPL of automotive assembly cause an inconsistency defect by adding automatic and manual gears where both gears cannot exist at the same time. Other examples include Nokia ( Thao, 2012 ), LG Industrial Systems ( Pohl et al., 2005 ) etc. Several techniques exist for the identification of FM inconsistencies such as constraint satisfaction techniques, logic truth 606 maintenance systems, formal semantics, logic based methods, genetic algorithms and many more. Few approaches based on firstorder logic (FOL) provide an effective way to formalize FMs using ontology ( Gruninger et al., 2008; Àlvez, 2012 and Bhushan and Goel, 2016 ), which is a formal, explicit specification of a shared conceptualization ( Gruber, 1993 ). It improves the level of expressivity when compared to FMs and facilitates to infer interesting information related to the models; for instance, to verify consistency among the FM and its meta-model, and to retrieve child features. Other researchers have also used methods based on ontologies ( Wang et al., 2007; Noorian et al., 2011 and Guo et al., 2012 ) and FOL ( Mannion, 2002 and Elfaki, 2016 ) to deal with defects due to inconsistency. Although several methods exist that have identified inconsistency ( Mannion, 2002; Gheyi et al., 2011; Zhang and Moller- Perdersen, 2013; Yang and Dong, 2013; Asadi et al., 2014 and Thüm et al., 2014b ) but only a few methods have provided explanation for their causes in a manner which is difficult to understand by developers ( Segura et al., 2010; Noorian et al., 2011; Felfernig et al., 2013 and Lesta et al., 2015 ). However, Thüm et al. (2014a) have only detected the constraints involved in dead features along with false-optional features and Elfaki (2016) has only prevented indirect inconsistency. The lack of methods to explain the cause of inconsistency defects in a user friendly language and to recommend solutions for resolving inconsistencies motivated us to propose an effective approach. Moreover, manually inspecting larger FMs to detect inconsistencies is a laborious task. Therefore, handling defects due to inconsistency is a critical task in order to derive defect free valid products from SPL, to improve reusability and quality of SPL models. We have developed an ontological approach based on FOL rules that handles FM inconsistencies to improve the quality of FMs in SPL and the results of evaluation using 24 FMs verified its effectiveness, accuracy and scalability with thousands of features in FMs. Following are the contributions of the proposed approach: I. We classify FM defects due to inconsistencies in the form of cases. II. We formalize FM using FOL predicate-based feature model ontology (FMO) which is one of the important contributions of the proposed approach that provides a correspondence among FM representations and FMO. III. We define and implement a set of FOL rules in Prolog ( Wielemaker, 2015 ) to deal with inconsistencies in FMs. IV. We identify FM inconsistencies and their causes using a user friendly natural language in the presented classification. V. The causes explained in natural language are easily understandable by developers and this information assists them to fix inconsistency defects by recommending solutions (i.e., by eliminating relationships involved in the defect). VI. Evaluation results of the proposed approach using real-world FMs from online software product lines online tools (SPLOT 1 ) repository as well as randomly generated FMs with thousands of features verifies our approach to be effective, accurate and scalable up to 22,035 features in FMs. Thus, it allows deriving defect free software products by subsequently enhancing the reusability and quality of FMs in SPL. Following is the structure of remaining paper: Section 2 presents preliminaries required to understand the approach presented in Section 3 . Section 4 analyzes performance and evaluates the scalability, execution time and accuracy of proposed approach. Section 5 compares our approach with related works. Section 6 describes concluding remarks and future directions. 1 http://www.splot-research.org/ . 2. Preliminaries 2.1. Inconsistencies Various types of inconsistency defects discussed in this paper are explained with the help of Fig. 1 ( Salinesi et al., 2010 and Elfaki, 2016 ), where r represents root feature which is mandatory to be included in each valid product of the PL, p1 and p2 represent parent child features, and f1 and f2 represent child features. Rule 1: Mandatory child features f1 and f2 have same parent root feature r, and f1 implies f2 whereas f2 excludes f1 . Thus, it is an inconsistency as both features f1 and f2 can never be selected simultaneously for the configuration of a valid product. Rule 2: Mandatory child features f1 and f2 have same parent root feature r where f1 excludes f2. Thus, it is an inconsistency as both features f1 and f2 are mutually excluded and these features can never be selected together for the configuration of a valid product. Rule 3: Mandatory parent features p1 and p2 have same root feature r where p1 excludes p2 and p1 has a mandatory child feature f1 which implies p2 . Rule 4: Mandatory parent features p1 and p2 have same root feature r where p1 excludes p2 and p1 has a mandatory child feature f1 which implies an optional child feature f2 whose parent is p2 . Rule 5: A mandatory parent feature p2 implies an optional parent feature p1 where both features have same root feature r and p1 has a mandatory child feature f1 which excludes p2. Rule 6: A mandatory parent feature p1 implies an optional parent feature p2 where both features have same root feature r and p1 has a mandatory child feature f1 which excludes the other mandatory child feature f2 having parent p2 . Rule 7: A mandatory parent feature p1 implies an optional parent feature p2 where both features have same root feature r and p1 has a mandatory child feature f1 which excludes p2. Rule 8: Mandatory child features f1 and f2 belong to the group cardinality < 1..1 > with a mandatory parent feature p where f1 implies f2 . In this particular case, p can also be connected to the root feature. The implication relationship among the child features exceeds the upper limit of the group cardinality and does not even allow to incorporate one sub feature. 2.2. Running example of FM The variabilities and commonalities in SPLE are represented using feature modeling notation by means of relationships among the features. A feature is defined as a unique element that is of relevance to the user. It is a hierarchical tree structure comprising of features and relationships among them. The entire SPL is represented with the help of root of the FM tree. An adapted version of the address FM available in the SPLOT repository ( Mendonca et al., 2009 ) is used as a running example in this paper. Fig. 2 describes the address FM using feature modeling notation where “addressbook ”is the root feature. There can be more than one child feature associated with a parent feature. A unique name has been assigned to each feature in FM. The cross tree constraint relationships i.e., exclusion and implication among features have also been shown for better interpretation of the proposed approach. To illustrate the proposed approach, 14 cross tree constraints and 18 additional features were introduced in the primary model to inject inconsistency defects. Following illustrates various feature modeling relationships using Fig. 2: 607 Fig. 1. Inconsistency defects. Fig. 2. Address feature model representing inconsistency defects. Optional : An optional feature may or may not be included in valid products associated with its parent feature. For instance, it is optional to include contactmanagement whenever addressbook is selected. Other optional features are x3, y2, interoperability, telephonenumber, e_mailaddress, addcontact, edictcontact and deletecontact. Mandatory : A mandatory feature appears in each product whenever its parent feature is selected. For instance, it is mandatory to incorporate x4 whenever addressbook is chosen. Other mandatory features are x1, x2, authentification, x5, x6, contactaccess, contactcharacteristics, x7, x8, x9, x10, y1, y3, name, y4, y5, y6, y7 and y8. Group cardinality: It describes an interval that limits the count of child features that can be included in a valid product whenever their parent feature is selected. The mandatory child features y5 and y6 belong to a group cardinality < 1..1 > where single feature can be included in a valid configuration. Additionally, if y5 is incorporated, y6 must also be included due to the fact that implication relationship exists between them. The cross tree constraints are as follows: Exclusion: Two features related with exclusion relationship cannot be included simultaneously in the same valid configuration. For instance, if there exists an exclusion between authentification and contactaccess, it will not allow to incorporate both features simultaneously. The address FM includes seven exclusion relationships. Implication : The source feature X of implication relationship implies target feature Y, which means that Y should be incorporated in valid products with X. For instance, the implication between contactcharacteristics and interoperability , represents that contactcharacteristics implies interoperability . The address FM includes seven implication relationships. 3. Proposed approach The proposed approach is explained in this section which includes construction of FMO, defining and dealing with inconsistency defects in the presented classification of defects in FMs (see Section 3.2.2 ). A set of rules is defined and implemented using Prolog describing certain cases of wrong usage of relationships among features in a FM that leads to inconsistency defects. The implementation details of the proposed approach to deal with inconsis- 608 Fig. 3. Feature model ontology of address feature model. tencies are explained using a running example of address FM from SPLOT repository: 3.1. Conversion from FM to ontology FeatureIDE tool ( Thüm et al., 2014a ) automatically reads an address FM by importing it in SXFM (Simple XML feature model) format from SPLOT repository. The graphic representation of the address FM can be generated using FeatureIDE’s visualization functionalities. Additional features (e.g., x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, y1, y2,y3, y4, y5, y6, y7 and y8 as given in Fig. 2 ) and cross tree constraints are introduced in the input address FM by enabling editing in the model editor of FeatureIDE in order to cause inconsistencies. Then, a converter 2 is developed that converts modified input address FM 3 from FeatureIDE to predicate-based FMO as shown in Fig. 3 . The concepts of FM and the semanticrelationships between them are represented using an ontological representation. The constructed FMO corresponds to the address FMs features and their relationships. The FMO mainly comprises of four types of predicates: feature, relation, parent and card. Following are the form of predicates defined in Prolog which are used to represent classes and properties in FMO: feature(Name,Category): Name represents the name of feature and Category describes that the feature Name can be either optional (“o ”) e.g., feature(contactmanagement,o) or mandatory (“m ”) e.g., feature(contactaccess,m). parent(Name,ParentName) : The name of child feature is represented by Name and ParentName signifies the name of parent feature. It shows that Name has parent ParentName e.g., parent(contactmanagement,addressbook). relation(Name1,Name2,Relationship): Name1 and Name2 respectively represents the names of source and target features intervening in the relationship. Relationship can be either implication e.g., relation(contactcharacteristics, interoperability,implication) or exclusion e.g., relation (authentification,contactaccess,exclusion). card(Name1, [Name2,Name3], [Min,Max]) : It represents that child features Name2 and Name3 are related in a < Min..Max > group cardinality with a mandatory parent feature Name1 . It determines the minimum ( Min ) and the maximum ( Max ) number of child features which are allowed to be specified in a cardinality relationship. For example, 2 http://bit.ly/2gSbiGs . 3 http://bit.ly/2hNWsm3 . card(x7,[ y 5, y 6],[1,1]) describes that only one feature can be selected from a set of features (i.e., y5 and y6) with group cardinality < 1..1 > . 3.2. Identification process The identification process for defects due to inconsistency in FMs is represented using an ontological rule-based approach. The predicate-based FMO in Prolog presents the existing facts. A set of FOL rules is developed and it represents eight specific cases of misuse among the relationships in a FM that caused inconsistency defects. These rules are applied as FOL queries to the generated FMO using SWI-Prolog. The identified inconsistencies with explanations for their causes in the given classification suggest solutions to developers for fixing defects are illustrated in the results (see Section 3.2.2 and Fig. 4 ). Each rule leads to the generation of identified inconsistency defects. For all rules we, (i) specify the existing facts (ii) state the FOL rules (iii) present the corresponding results (see Fig. 4 ) after implementing each rule which includes the identified inconsistency defect along with its cause (iv) present the combined results (see Fig. 5 ) produced after implementing all rules to the running example specified in Fig. 2 . The identification process for defects due to inconsistency in SPL is described as follows: 3.2.1. General criteria The general criteria followed to describe inconsistency defects are: 1. Mandatory features f2 and f3 are connected to the root feature f1, and f2 implies f3 and f3 excludes f2. 2. Mandatory features f2 and f3 are connected to the root feature f1 2.1 f2 excludes f3. 2.2 f2 excludes f3 and mandatory child feature f4 implies f3. 2.3 f2 excludes f3 and mandatory child feature f4 implies an optional child feature f5. 3. Mandatory feature f2 implies an optional feature f3 where both features are connected to the root feature f1 3.1 mandatory child feature f4 having parent f3 excludes f2. 3.2 mandatory child feature f4 excludes another mandatory child feature f5. 3.3 f3 excludes mandatory child feature f4 having parent f2. 609 Fig. 4. Results of rules for inconsistency defects. Fig. 5. Snapshot of the results produced after applying all rules to address feature model. 3.2.2. Identification rules The defects due to inconsistency are classified into four cases: (i) exclusion and implication simultaneously, (ii) mandatory and exclusion, (iii) optional and implication, and (iv) implication between alternative child features. The cases of inconsistency defects are described using facts, rules and results as follow: Case 1: Exclusion and implication simultaneously. This case identifies inconsistency defect using rule 1 indicated by general criteria number 1. Rule 1 (refer Fig. 1 (Rule 1)) Facts: feature(r,m). feature(f1,m). feature(f2,m). parent(f1,r). parent(f2,r). relation(f1,f2, implication). relation(f2,f1,exclusion). rule1 :-feature(R,m), feature(F1,m), feature(F2,m), parent(F1,R), parent(F2,R), relation(F1,F2, implication), relation(F2,F1,exclusion), write(‘ \ nDefect: Inconsistency1 \ nExplanation: implicati-on and exclusion between’), write(F1), write(‘and’), write(F2). Result: Fig. 4 (Rule 1) gives the result produced after applying this rule. Case 2: Mandatory and exclusion. In this case, rules 2, 3 and 4 identify inconsistency defects indicated by general criteria numbers 2.1, 2.2 and 2.3 respectively. Rule 2 (refer Fig. 1 (Rule 2)) Facts: feature(r,m). feature(f1,m). feature(f2,m). parent(f1,r). parent(f2,r). relation(f1,f2, exclusion). rule2 :- feature(R,m), feature(F1,m), feature(F2,m), parent(F1,R), parent(F2,R), relation(F1,F2, exclusion), 610 write(‘ \ nDefect: Inconsistency2 \ nExplanation: exclusion between’), write(F1), write(‘and’), write(F2). Result: The result obtained after applying this rule is shown in Fig. 4 (Rule 2). Rule 3 (refer Fig. 1 (Rule 3)) Facts: feature(r,m). feature(p1,m). feature(p2,m). feature(f1,m). parent(p1,r). parent(p2,r). parent(f1,p1). relation(p1,p2,exclusion). relation(f1,p2,implication). rule3 :-feature(R,m), feature(P1,m), feature(P2,m), feature(F1,m), parent(P1,R), parent(P2,R), parent(F1,P1), relation(P1,P2,exclusion), relation(F1,P2,implication), write(‘ \ nDefect: Inconsistency3 \ nExplanation: exclusion between’), write(P1), write(‘and’), write(P2), write(‘ \ nand implication between’), write(F1), write(‘and’), write(P2). Result: Fig. 4 (Rule 3) shows the result obtained after implementing this rule. Rule 4 (refer Fig. 1 (Rule 4)) Facts: feature(r,m). feature(p1,m). feature(p2,m). feature(f1,m). feature(f2,o). parent(p1,r). parent(p2,r). parent(f1,p1). parent(f2,p2). relation(f1,f2,implication). relation(p1,p2,exclusion). rule4 : - feature(R,m), feature(P1,m), feature(P2,m), feature(F1,m), feature(F2,o), parent(P1,R), parent(P2,R), parent(F1,P1), parent(F2,P2), relation(P1,P2,exclusion), relation(F1,F2, implication),write(‘ \ nDefect: Inconsistency4 \ nExplanation: exclusion between’), write(P1), write(‘and’), write(P2), write(‘ \ nand implication between’), write(F1), write(‘and’), write(F2). Result: The result produced after applying this rule is given in Figure 4 (Rule 4). Case 3: Optional and implication. In this case, rules 5, 6 and 7 identify inconsistency defects indicated by general criteria numbers 3.1, 3.2 and 3.3 respectively. Rule 5 (refer Fig. 1 (Rule 5)) Facts: feature(r,m). feature(p1,o). feature(p2,m). feature(f1,m). parent(p1,r). parent(p2,r). parent(f1,p1). relation(p2,p1,implication). relation(f1,p2,exclusion). rule5 : -feature(R,m), feature(P1,o), feature(P2,m), feature(F1,m), parent(P1,R), parent(P2,R), parent(F1,P1), relation(P2,P1,implication), relation(F1,P2,exclusion), write(‘ \ nDefect: Inconsistency5 \ nExplanation: implication between’), write(P2), write(‘and’), write(P1), write(‘ \ n and exclusion between’), write(F1), write(‘and’), write(P2). Result: Fig. 4 (Rule 5) shows the result generated after implementing this rule. Rule 6 (refer Fig. 1 (Rule 6)) Facts: feature(r,m). feature(p1,m). feature(p2,o). feature(f1,m). feature(f2,m). parent(p1,r). parent(p2,r). parent(f1,p1). parent(f2,p2). relation(p1,p2,implication). relation(f1,f2,exclusion). rule6 :- feature(R,m), feature(P1,m), feature(P2,o), feature(F1,m), feature(F2,m), parent(P1,R), parent(P2,R), parent(F1,P1), parent(F2,P2), relation(P1,P2,implication), relation(F1,F2, exclusion),write(‘ \ nDefect: Inconsistency6 \ nExplanation: implication’), write(P1), write(‘and’), write(P2), write(‘ \ nand exclusion’), write(F1), write(‘and’), write(F2). Result: Fig. 4 (Rule 6) presents the result obtained after applying this rule. Rule 7 (refer Fig. 1 (Rule 7)) Facts: feature(r,m). feature(p1,m). feature(p2,o). feature(f1,m). parent(p1,r). parent(p2,r). parent(f1,p1). relation(p1,p2,implication). relation(f1,p2,exclusion). rule7 :-feature(R,m), feature(P1,m), feature(P2,o), feature(F1,m), parent(P1,R), parent(P2,R), parent(F1,P1), relation(P1,P2,implication), relation(F1,P2,exclusion), write(‘ \ nDefect: Inconsistency7 \ nExplanation: implication between’), write(P1), write(‘and’), write(P2), write(‘ \ nand exclusion between’), write(F1), write(‘and’), write(P2). Result : Fig. 4 (Rule 7) presents the result generated after applying this rule. Case 4: Implication between alternative child features. Rule 8 identifies inconsistency defect in this case. Rule 8 (refer Fig. 1 (Rule 8)) Facts: feature(p,m). feature(f1,m). feature(f2,m). parent(f1,p). parent(f2,p). card(p,[ f 1, f 2],[1,1]). relation(f1,f2,implication). Rule 8 :-feature(P,m), feature(F1,m), feature(F2,m), parent(F1,P), parent(F2,P), relation(F1,F2, implication), card(P,[ F 1, F 2],[1,1]), write(‘ \ nDefect: Inconsistency8 \ nExplanation: implication between’), write(F1), write(‘and’), write(F2). Result : The result generated after applying this rule is shown in Fig. 4 (Rule 8). 3.2.3. Implementation to the running example The FMO based on predicates for running example of address FM (see Fig. 2 ) is represented in Fig. 3 . Fig. 5 depicts a snapshot of the results generated after implementing all rules to address FMO. The generated results can be further used by the developers or modelers to fix inconsistency defects i.e., by modifying or deleting relationships involved in the cause of defect. 4. Performance analysis and evaluation The performance of the proposed approach has been evaluated by focusing on accuracy, scalability and execution time. The experiments were performed on 24 models, out of which 16 were from the SPLOT repository and other 8 models were randomly generated. The environment for the evaluation was a single laptop with Windows 8.1 of 64 bits, CPU 2.40 GHz, processor Intel(R) Core(TM) i7-3630QM, RAM of 8.00 GB, FeatureIDE 2.7.2, SPLOT repository and SWI-Prolog (Multi-threaded, 64 bits, Version 7.2.3). 4.1. Accuracy The proposed approach identified 100% of the defects due to inconsistency in our set of rules with 0% false positive. Our approach was evaluated with 24 models of varied sizes from small size to large size FMs (i.e., 30 to 22,035 features) and the results comprise of identifying defects due to inconsistency with explanation for their causes and their related execution times as shown in Table 1 . Moreover, it is found that all 8 rules (individually and simultaneously) identified all expected defects due to inconsistency and their causes signifying 100% accuracy of the proposed approach. 4.2. Scalability The performance of proposed approach was tested by executing all 8 rules to identify defects due to inconsistency and their causes on 24 FMs with range from 30 to 22,035 features, and calculating their corresponding execution time in seconds as shown in Fig. 6 . Number of features in all models and time are represented on X -axis and Y -axis respectively. The results conclude that the queries over FM up to 800 features take less than 2.7 seconds and large sized FM with 22,035 features takes a reasonable time of 24.24 seconds. All rules were executed 50 times for all the 24 models, which mean 9600 (8 ∗24 ∗50) queries in whole were executed in order to attain the accurate and reliable measures of execution times. The execution time computed is the average of 50 611 Table 1 Identified inconsistency defects in feature models and their corresponding execution times. Feature model OF AF TF NE NI NOI Rules Time (in sec) AddressBook 12 18 30 7 7 8 All rules 0.04 Celular 10 31 41 7 7 8 All rules 0.05 GuideAgent 17 31 48 7 7 8 All rules 0.06 Bcmscontext 35 31 66 7 7 8 All rules 0.12 Experiment environment 35 45 80 7 7 8 All rules 0.15 Reuso - UFRJ - Eclipse1 72 31 103 7 7 8 All rules 0.37 Car Selection 72 38 110 7 7 8 All rules 0.38 Billing 88 35 123 7 7 8 All rules 0.53 SmartTV 96 31 127 7 7 8 All rules 0.56 xtext 137 31 168 7 7 8 All rules 0.60 BattleofTanks 144 31 175 7 7 8 All rules 0.68 FM_Test 168 31 199 7 7 8 All rules 0.73 Printers 172 31 203 7 7 8 All rules 0.93 BankingSoftware 176 31 207 7 7 8 All rules 0.99 Electronic Shopping 290 31 321 7 7 8 All rules 1.00 A Model for Decision-making for Investments on Enterprise Information Systems 366 34 400 7 7 8 All rules 1.14 FM with 535 features 500 35 535 7 7 8 All rules 1.27 FM with 635 features 600 35 635 7 7 8 All rules 2.14 FM with 835 features 800 35 835 7 7 8 All rules 2.71 FM with 1035 features 10 0 035 1035 7 7 8 All rules 3.57 FM with 5035 features 50 0 035 5035 7 7 8 All rules 10.38 FM with 10,035 features 10,0 0 0 35 10,035 7 7 8 All rules 13.20 FM with 15,035 features 15,0 0 0 35 15,035 7 7 8 All rules 19.13 FM with 22,035 features 22,0 0 0 35 22,035 7 7 8 All rules 24.24 OF = Original features, AF = Additional features, TF = Total features, NE = No. of exclusions, NI = No. of implications, NOI = No. of inconsistencies Fig. 6. Execution time, for all rules, per number of features. executions for all rules on each one of the model (i.e., 192 combined results). All the rules (see Section 3.2.2 ) that deal with defects due to inconsistency are merged into a single program and implemented over models in order to determine the scalability of inconsistency defects identification process. For instance, the results of this program for running example include the identified inconsistencies with their explanations and corresponding execution time as shown in Fig. 5 . 4.3. Execution time The proposed approach was implemented on FMs with features 20 0, 40 0, 60 0, 80 0 and 10 0 0, and measured the execution time Fig. 7. Execution time, for all rules, per number of features for the performance. Fig. 8. Execution time of all rules, per number of features for the height of tree. consumed by all rules in dealing with inconsistencies. This information is used to compare the execution time considered for the factors that affect the performance of our approach: number of mandatory and optional features, number of implications, number of exclusions, number of implications and exclusions (see Fig. 7 ), and height of tree (see Fig. 8 ). These factors were chosen due to their impact on execution time. As shown in Tables 2 and 3 , R 2 (R-squared) values for each of the plot in Fig. 7 and 8 have been calculated respectively, in order to measure the data’s goodness of fit. The nearer is the value of R 2 to 1, better the fit. It means the line passes closely through all points. Following discusses the factors impacting performance of our approach: Number of mandatory and optional features : Fig. 7 (1) shows the execution time of FMs with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 pro- 612 Table 2 R 2 value calculated for performance. R 2 value Features Fig.7 (1) Implications Fig. 7 (2) Exclusions Fig. 7 (3) Implications and exclusions Fig. 7 (4) (a) Mandatory features 0.98 0.97 0.97 0.87 (b) Optional features 1.00 1.00 1.00 1.00 (c) Mandatory and optional features 0.91 0.89 0.90 0.97 Table 3 R 2 value calculated for height of tree. Height of tree (R 2 value) Features Fig. 8 (1) Implications Fig. 8 (2) Exclusions Fig. 8 (3) Implications and exclusions Fig. 8 (4) (a) Mandatory features 0.95 0.93 0.93 0.94 (b) Optional features 1.00 1.00 1.00 1.00 (c) Mandatory and optional features 0.89 0.93 0.93 0.88 portion. The analysis showed that performance is improved with (b) while it deteriorated with (c). It means the approach required less time to deal with FM inconsistencies in (b) than from FMs with (a) and (c). For this case, the line nearly goes through all points in (b) with R 2 value 1.00 as shown in Table 2 . Number of implications : Fig. 7 (2) depicts the execution time of FMs including implication relationships with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 proportion. As an overall trend, (b) improved performance whereas (a) and (c) rapidly declined it. This indicates the approach required more time to deal with FM inconsistencies in (a) and (c) than from FMs with (b). As shown in Table 2 , the R 2 value for (b) is 1.00 which indicates a perfect fit of data. Number of exclusions : Fig. 7 (3) illustrates the execution time of FMs including exclusion relationships with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 proportion. It can be seen that performance increased with (b) while it decreased with (c). It implies that the approach required less time to deal with FM inconsistencies in (b) as compared to FMs with (a) and (c). The R 2 value for (b) is 1.00 as given in Table 2 , which means the line closely passes through all points in this case. Number of implications and exclusions : Fig. 7 (4) represents the execution time of FMs including implication and exclusion relationships with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 proportion. The results showed that performance is inclined with (b) while it is declined with (c). This demonstrates that the approach devoted more time to deal with FM inconsistencies in (a) and (c) as compared to FMs with (b). As given in Table 2 , the value of R 2 for (b) is 1.00 which represents that the line nearly goes through all points. Fig. 8 demonstrates the execution time of all rules as per the height of tree with increasing levels 7, 11, 15, 19 and 22 in the models with features 200, 400, 600, 800 and 1000 respectively. Height of tree includes number of mandatory and optional features : Fig. 8 (1) corresponds to the execution time of FMs with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 proportion. The analysis showed that performance is improved with (b) while it rapidly reduced with (c) i.e. the approach required less time to deal with FM inconsistencies in (b) than FMs with (a) and (c). In this case, the line nearly goes through all points in (b) with R 2 value 1.00 as shown in Table 3 . Height of tree includes number of implications : Fig. 8 (2) depicts execution time of FMs including implication relationships with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 proportion. As an overall trend, (b) improved performance whereas (a) deteriorated it. This indicates that the approach consumed more time to deal with FM inconsistencies in (a) and (c) than FMs with (b). As shown in Table 3 , the R 2 value for (b) is 1.00 which leads to a perfect fit of data. Height of tree includes number of exclusions : Fig. 8 (3) illustrates execution time of FMs including exclusion relationships with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 proportion. It can be clearly seen that performance increased with (b) while it decreased with (a). It implies the approach required less time to deal with FM inconsistencies in (b) than FMs with (a) and (c). The R 2 value for (b) is 1.00 as given in Table 3 , indicating that the line closely passes through all points in this case. Height of tree includes number of implications and exclusions : Fig. 8 (4) represents execution time of FMs including implication and exclusion relationships with (a) mandatory features, (b) optional features, and (c) mandatory and optional features in 1:1 proportion. The outcomes revealed that performance is persuaded with (b) while it is radically dropped with (c). This demonstrates that the approach consumed more time to deal with FM inconsistencies in (a) and (c) as compared to FMs with (b). As given in Table 3 , the value of R 2 for (b) is 1.00 which represents that the line nearly goes through all points. All rules can be checked based on integral square error (ISE) and Spearman rank correlation coefficient, between the execution time for each rule and the number of features in the models as shown in Table 4 . The curve fitting technique is used to calculate ISE for determining the most appropriate rule. Rules 4 and 6 have least errors as given in Table 4 which stimulate the best predictions for our approach. All rules can be examined in a polynomial time (i.e., with 5th degree polynomial), which is due to the fact that our approach just evaluates the features related to the rule rather than whole model. The good recommendation of rules dependency or independency is given by Spearman rank correlation coefficient which is nearer to 0 for rule 1. Despite the fact that the complexity of this kind of issue associated with predicatebased FMO is NP, the results proved that the proposed rules seem to be scalable to large sized models with the help of our technique. 4.4. Discussion and threats to validity The results of experiment evaluation show that our approach efficiently works with large size FMs (see Fig. 6 ) as it handles FM inconsistencies in 24.24 seconds even for large size model up to 20,035 features. Additionally, the execution time also scales with increasing size of FMs (see Table 1 ). Thüm et al. (2014a) have provided a precision of 68.75% when their technique is validated and contrasted with our set of rules, while the accuracy of the proposed approach is considered to be 100% with 0% false positive since the identified inconsistencies and their corresponding explanations are accurate according to the rule definitions. We have calculated the execution time between FM with varying size from 200 to 10 0 0 features and various number of factors (see Figs. 7 and 8 ) influencing the performance. The plots shown M. Bhushan et al. / The Journal of Systems and Software 137 (2018) 605–617 613 Table 4 Integral square errors and spearman rank correlation coefficients among number of features and execution time per each rule over feature models. Rules 1 2 3 4 5 6 7 8 Integral square error 141,245.5 97,223.0 11,911 3322 31,851.8 2291.3 59,990.8 6737.4 Correlation coefficient 0.05 0.71 0.91 0.97 0.83 0.97 0.84 0.97 Table 5 The comparison of execution time to detect inconsistencies between proposed approach and existing approaches. Articles Number of features Inconsistency detection time (in sec) Existing approaches Proposed approach Wang et al. (2010) 4400 <65 6.367 White et al. (2010) 50 0 0≈50 10.38 Gheyi et al. (2011) 20 0 094 4.71 Noorian et al. (2011) 31 147 0.04 Felfernig et al. (2013) 172 FASTDIAG algorithm (10.54) HSDAG algorithm (100) 0.63 White et al. (2014) 10 0 0 4 3.55 Elfaki (2016) 20,0 0 0 Identification(97.309) Prevention(67.515) 22.37 Proposed approach 22,035 –NA 24.242 in Figs. 7, 8 (1) and (4) indicate that the FMs with optional features required less runtime to handle inconsistency defects as compared to FMs with mandatory and optional features in 1:1 proportion. However, according to plots shown in Fig. 8 (2) and (3), it can be ascertained that FMs with mandatory features required more time to handle inconsistency defects when compared to FMs with optional features. Table 5 compares the proposed approach with existing approaches based on the execution time required to deal with inconsistencies. Our approach handled inconsistencies in FMs up to 22,035 features in 24.24 seconds when compared with other existing methods leading to better scalability in an extremely reasonable time. Elfaki (2016) identified three types of inconsistencies in 97.309 seconds and prevented only direct inconsistency in 67.515 seconds for 20,0 0 0 features while our approach identified and explained causes of direct inconsistency along with other defects due to inconsistency (see Fig. 1 ) in 22.37 seconds for 20,0 0 0 features. Threats to validity : The experimental results show that the proposed approach is valid, however the validity of these experiments may get affected by some conditions. Following are various validity threats that could affect the experiments. External validity . Even though the majority of the input models used in the experiments are real world models available in SPLOT repository, there are also some randomly generated models (with 500 to 22,035 features) that we have used for the experimentation, which may not reflect real world models and may cause threat to external validity. The complexity of the model and the problem size vary with real world models. Therefore, this effect is minimized by generating models using a well known FeatureIDE tool. Moreover, the analysis of FMs is run independently in order to reduce the impact of other threads on the calculated execution time such as threads of operating system. All rules were executed 50 times on each one of the model and its average is used as a result to minimize the impact. Internal validity . The calculated execution time to analyze a FM affects the threat to internal validity. Various factors that impact the performance are number of mandatory and optional features, number of implications, number of exclusions, number of implications and exclusions (see Fig. 7 ) and height of tree (see Fig. 8 ). The internal validity of the experiments is improved by experimenting the FMs, with varying size of the model ranging from 200 to 10 0 0 features and by increasing the levels in order of 7, 11, 15, 19 and 22 in the model’s height with features 20 0, 40 0, 60 0, 80 0 and 10 0 0 respectively. 5. Related work Many existing notations for representing FMs have been proposed such as UML ( Alférez et al., 2010 ), constraint programming, domain specific language (DSL) ( Groher and Voelter, 2007 ), higher order logic ( Janota and Kiniry, 2007 ), extensible markup language, propositional and FOL ( Mannion et al., 2002 ) and description logic (DL). The main drawback of UML is the use of object constraint language only as a validation tool because it is carried out over a particular object or an object structure ( Czarnecki, 1998 ). DSL being a domain specific programming language does not give solutions to general problems ( Mernik et al., 2005 ) and therefore, does not provide standardization. All real-life cases cannot be represented using propositional logic due to absence of variables ( Copi and Cohen, 2009 ). These restrictions have encouraged the use of FOL for validating SPL ( Sun et al., 2005; Bhushan and Goel, 2016 and Elfaki, 2016 ) as it is more representative and manages most of the real-life cases ( Gallier, 2003 ). Researchers have represented FMs based on ontology: Wang et al. (2007) have identified invalid configurations in FMs and its causes by means of ontological terms, Lee et al. (2007) have analyzed the semantic similarity of the FMs (i.e. FMs variability and commonality), Abo et al. (2008) have also validated FM consistency with Semantic Web Rule Language based rules by integrating different FMs, and Noorian et al. (2011) have used OWL-DL and Pellet reasoner ( Sirin et al., 2007 ) for the identification along with corrections of FM inconsistencies. However, the corrections build in OWL-DL are difficult to understand and are inefficient with expressive ontologies e.g, DOLCE ( Gangemi et al., 2002 ) and SUMO ( Niles and Pease, 2001 ). These drawbacks have resulted in the development of FOL predicates, a more expressive language based on logics ( Berardi et al., 2004 ). First-order language is a predicate-based ontology language for modeling a FM which includes classes of objects and properties represented using binary predicates and ternary predicates respectively ( Goldstein and Storey, 1991 and De Bruijn and Heymans, 2006 ). FOL based FLOWS ontology has been developed for web services by extending process specification language ( Gruninger et al., 2008 ). It increases the level of expressiveness by including rich quantified sentences along with n -ary predicates, and also enables modeling and reasoning on web services semantics. Therefore, the above limitations motivated us to transform FM into FOL predicatebased ontology as it increases the expressiveness and provides formalization.