Process Mining to Unleash Variability Management: Discovering Configuration Workflows Using Logs

Process Mining to Unleash Variability Management: Discovering Configuration Workflows Using Logs Ángel Jesús Varela-Vaca, José A. Galindo, Belén Ramos-Gutiérrez María Teresa Gómez-López and David Benavides Universidad de Sevilla Seville, Spain {ajvarela,jagalindo,brgutierrez,maytegomez,benavides}@us.es ABSTRACT Variability models are used to build configurators. Configurators are programs that guide users through the configuration process to reach a desired configuration that fulfils user requirements. The same variability model can be used to design different configurators employing different techniques. One of the elements that can change in a configurator is the configuration workflow, i.e., the order and sequence in which the different configuration elements are presented to the configuration stakeholders. When developing a configurator, a challenge is to decide the configuration workflow that better suites stakeholders according to previous configurations. For example, when configuring a Linux distribution, the configuration process start by choosing the network or the graphic card, and then other packages with respect to a given sequence. In this paper, we present COnfiguration workfLOw proceSS mIning (COLOSSI), an automated technique that given a set of logs of previous configurations and a variability model can automatically assist to determine the configuration workflow that better fits the configuration logs generated by user activities. The technique is based on process discovery, commonly used in the process mining area, with an adaptation to configuration contexts. Our proposal is validated using existing data from an ERP configuration environment showing its feasibility. Furthermore, we open the door to new applications of process mining techniques in different areas of software product line engineering. KEYWORDS variability, configuration workflow, process mining, process discov ery, clustering 1 INTRODUCTION Variability models such as Feature Models (FMs) [ 22 ] describe commonalities and variabilities in Software Product Lines (SPLs) and are used along all the SPL development process. After an FM is defined, products can be configured and derived. In the configuration and derivation process, users select and deselect features using a configurator. A configurator [ 21 ][ 19 ] is a software tool that presents configuration options to the users in different stages. An example of a configurator tool is KConfig [ 58 ] where developers can configure the Linux kernel with more than 12.000 configuration options. An important aspect of a configurator is to determine the configuration workflow [ 28 ], i.e., the order in which features and options are presented to configuration stakeholders. For instance, when configuring the Linux kernel using KConfig [ 58 ], there can be different user configuration profiles depending on interests or skills. The configuration workflow used by a configurator can impact the user experience in the configuration process. Therefore, selecting a well suited configuration workflow is a challenge. Up to now – to the best of our knowledge– the selection of a configuration workflow is made either intuitively or following the structure and properties of a variability model [21, 66]. In this paper, we present COLOSSI, an approach that takes a feature model and a set of existing configuration logs and automatically retrieves configuration workflows. A configuration log is a set of configurations performed in the past in a given domain taking into account a configuration order. Our solution relies on process mining [ 3 ] techniques. Process mining is a well established area of business process management that uses different techniques to extract business processes from traces of execution. In our approach, we conceptually map a business process model to a configuration workflow and traces to configuration logs making possible to reuse process mining techniques to infer configuration workflows. Although using process mining can automatically retrieve configuration workflows, the results can be difficult to interpret to domain engineers in order to build a configurator. This is mainly due to the fact that, very often, mined processes are “spaghetti-like” models in which the same activity needs to be duplicated [ 62 ]. To illustrate the difficulty, Figure 1 shows the result of directly applying process mining techniques to the ERP system presented in [ 46 ] and detailed in Section 3. The simplification of spaghetti processes is an open problem in the process mining domain [ 3 ]. Variability models have special characteristics that can help to guide the discovery process. To overcome this difficulty, our solution adapts a clustering algorithm 1 2 3 1 3 1 1 1 1 1 1 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 3 1 1 2 1 1 1 1 1 1 1 1 1 1 12 1 2 1 2 1 1 1 1 1 1 3 3 1 3 2 1 2 1 1 1 1 2 1 1 1 2 1 1 1 1 1 1 2 1 4 1 3 1 2 1 1 2 2 1 2 2 2 1 2 3 1 2 1 1 2 2 2 1 2 1 1 1 1 1 1 2 1 1 1 1 2 2 1 1 2 4 2 2 1 2 1 2 1 1 1 2 1 1 1 1 2 2 1 1 1 2 1 1 3 2 4 1 1 1 2 2 2 1 2 3 1 2 1 1 2 1 1 1 2 2 3 1 2 1 2 2 1 2 2 2 2 2 1 1 1 1 2 1 1 1 12 1 1 2 2 4 1 1 1 1 2 4 3 1 2 1 1 1 1 1 1 1 1 1 1 4 2 2 1 2 1 1 1 1 2 1 1 2 3 1 1 1 1 1 1 3 1 2 1 1 1 2 3 2 1 2 1 1 4 1 2 1 2 1 2 1 3 2 1 2 1 2 1 1 2 2 2 2 2 1 1 1 2 1 2 2 1 2 1 1 1 1 2 1 1 3 3 2 1 1 2 2 1 1 2 1 2 1 1 1 2 1 1 1 1 1 1 2 1 2 1 2 1 1 1 1 1 1 3 1 1 1 1 1 1 2 2 1 1 1 3 1 1 1 1 1 2 3 2 4 2 1 1 1 1 3 1 1 1 1 1 1 2 1 1 1 2 3 1 1 2 1 1 2 1 3 1 11 1 1 1 1 1 2 3 1 1 3 1 4 2 1 1 2 1 2 1 2 1 1 1 1 1 1 1 1 2 1 1 2 3 1 1 1 1 2 2 1 1 2 2 1 2 1 3 1 1 1 2 1 1 1 1 1 2 1 1 3 3 1 1 3 1 2 2 1 2 1 1 1 1 1 3 3 1 1 2 2 1 3 1 3 3 1 4 1 2 3 3 1 1 4 2 1 1 2 1 1 1 1 2 1 1 2 2 4 1 3 1 2 1 1 1 3 1 1 1 3 1 1 1 1 2 1 3 3 1 1 1 1 1 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 2 1 3 1 1 1 1 4 2 1 1 1 1 1 2 2 1 2 1 2 2 1 1 2 1 1 2 1 2 1 1 1 1 3 2 1 1 1 1 1 2 1 1 1 1 3 1 1 1 1 2 1 1 1 2 1 1 1 2 1 1 1 2 1 1 2 3 1 3 1 2 2 1 1 1 3 2 1 1 1 2 1 1 1 1 1 10 3 2 3 4 3 2 1 1 1 2 2 1 1 1 1 1 1 2 1 2 1 1 1 3 2 1 1 2 3 1 1 1 2 2 4 2 2 1 3 1 3 2 1 1 1 2 2 1 2 2 1 2 1 1 2 2 1 1 2 1 1 1 1 2 1 2 1 3 1 1 3 2 1 1 2 1 1 1 2 1 2 3 1 1 1 2 1 1 1 1 3 1 1 1 2 1 1 2 1 2 1 1 2 1 1 1 1 1 3 1 1 1 2 3 2 2 1 3 2 1 1 1 1 2 3 1 1 1 1 1 2 1 1 2 3 1 1 1 2 2 1 2 1 1 3 1 1 1 1 1 1 1 1 2 1 1 1 1 2 2 1 2 2 3 1 1 3 1 1 2 2 1 1 2 1 1 1 2 2 1 2 1 2 1 1 1 1 3 3 1 3 1 1 3 2 1 1 1 2 2 1 1 2 2 1 3 1 1 3 1 1 3 1 2 1 1 4 1 1 2 1 3 1 1 1 1 4 2 3 2 1 1 1 1 1 1 2 5 3 2 1 1 2 2 3 2 5 1 4 2 2 1 1 1 1 1 1 2 1 1 1 2 3 1 1 2 2 1 1 2 1 4 1 2 4 1 1 2 2 3 2 1 1 2 1 1 1 1 2 1 2 2 1 1 1 1 3 4 1 1 1 1 1 1 1 1 2 1 2 2 2 1 1 1 1 2 2 1 1 2 1 2 1 1 1 1 1 42 1 2 1 2 1 2 1 1 1 1 1 2 2 1 1 2 3 3 1 2 1 1 2 2 2 1 1 3 1 3 1 2 1 1 1 1 3 1 1 2 2 3 2 1 1 1 1 2 1 2 3 1 1 3 1 2 4 1 3 1 1 1 1 1 2 1 2 1 2 1 21 2 1 1 3 3 2 1 1 2 3 3 1 1 1 4 1 1 2 1 1 4 2 3 3 1 3 1 1 2 3 1 1 1 1 1 1 1 1 2 2 1 1 1 2 2 3 2 1 1 1 1 2 1 1 1 3 1 2 1 2 2 1 1 2 1 1 1 3 1 2 1 1 4 1 1 2 2 1 1 2 1 1 1 1 1 3 2 1 1 1 1 2 3 1 2 1 1 1 2 2 1 1 1 2 1 3 1 1 3 3 3 1 2 5 2 1 2 1 1 1 1 1 3 1 1 1 1 1 1 1 1 2 2 1 3 1 1 3 2 1 2 3 1 1 1 1 2 2 1 3 2 3 1 1 1 1 2 3 1 1 2 1 1 1 2 1 1 1 1 1 2 1 1 1 3 3 1 1 1 4 2 1 1 1 1 3 1 1 3 1 1 2 1 2 1 2 1 1 1 1 1 2 2 3 1 1 1 1 2 2 3 1 3 1 1 3 2 2 1 2 1 1 2 1 1 4 1 1 1 1 1 1 3 2 3 1 2 2 1 1 1 1 1 3 2 1 1 1 1 1 2 1 2 2 3 1 1 1 1 1 1 2 2 2 1 2 1 1 2 1 1 1 1 1 1 1 2 1 1 1 1 1 4 1 4 1 2 1 1 1 1 1 1 2 1 1 3 2 1 4 4 1 1 1 1 1 1 1 1 2 1 1 3 1 1 1 2 1 1 2 1 2 3 2 1 1 2 2 1 1 1 2 2 1 1 1 3 1 1 2 1 1 1 1 2 2 1 1 2 2 1 1 3 1 1 1 1 1 2 1 3 1 2 1 4 1 1 1 1 1 1 1 1 1 1 1 1 1 3 1 1 1 1 1 1 2 2 3 1 1 2 1 1 1 3 1 1 1 1 1 1 2 1 1 1 11 1 3 1 2 3 1 1 2 1 1 1 1 1 3 1 1 2 1 2 2 1 2 1 2 1 1 1 1 1 2 1 1 2 1 1 2 1 1 1 1 1 1 2 1 1 2 1 2 1 1 1 1 2 3 2 1 2 1 4 1 4 3 2 1 1 1 2 2 1 1 1 1 1 2 3 2 1 1 3 1 1 3 1 1 1 1 1 4 1 2 4 1 1 1 1 1 3 2 3 1 1 1 2 1 2 1 1 1 2 2 1 1 1 1 1 1 1 1 1 2 1 1 2 1 3 1 1 1 1 1 3 1 2 1 1 2 2 1 2 2 1 1 1 2 2 1 1 1 2 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 2 1 1 1 1 1 3 1 1 1 1 1 3 1 1 2 1 1 1 3 1 1 1 3 1 1 3 1 1 1 2 3 1 1 1 1 1 1 2 2 1 1 1 2 1 1 3 1 1 4 2 2 1 1 1 1 3 1 1 2 3 2 1 1 4 1 2 1 2 2 2 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 2 2 3 1 1 1 1 1 1 2 2 2 3 2 2 1 1 2 2 3 1 1 1 1 4 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 2 1 1 2 1 2 1 1 1 2 1 1 1 4 1 1 2 1 2 1 1 2 1 2 1 1 1 2 1 2 1 1 1 1 1 1 3 1 2 1 1 1 1 1 1 3 2 1 1 2 1 1 2 1 2 1 3 1 1 1 1 1 1 3 1 1 1 1 1 3 1 1 2 1 5 2 1 1 3 2 2 1 2 1 1 1 3 1 1 3 3 2 1 2 1 1 1 1 1 1 1 1 1 3 1 1 1 2 1 1 2 1 1 1 3 1 2 1 1 1 1 2 1 1 2 5 4 1 1 1 1 2 1 3 1 1 1 1 1 1 1 2 3 1 2 3 1 2 1 8 4 1755 41 8271 14 1702 27 3702 20 8179 47 1740 36 1762 31 1118 152 1707 92 1701 51 1754 33 1782 14 1709 29 1759 25 1706 36 8166 50 1757 39 8209 11 8205 11 3701 25 8045 99 1752 20 1751 12 8049 64 8154 58 307 50 955 43 1432 44 3714 19 235 38 1768 15 8172 54 8170 30 3705 28 335 146 1764 22 1721 26 1106 35 8026 55 8204 23 8167 50 8206 24 967 96 3706 21 1708 28 8014 64 936 44 1457 138 932 111 336 103 1704 28 8047 64 949 64 8160 30 8165 50 8072 127 946 64 1322 95 1452 135 1728 33 8207 25 3709 21 3715 25 8247 9 8202 18 1756 21 3810 54 317 121 8153 48 1727 15 1723 28 111 62 1725 16 279 49 1775 17 282 68 8201 11 1722 31 1726 32 1781 23 1401 63 203 48 3722 15 8073 62 8042 64 8048 64 3721 22 9801 23 1402 38 1504 108 1406 41 3735 20 8003 99 904 34 8257 17 322 103 1303 37 8137 90 8050 84 372 92 350 100 1463 44 8131 88 1319 33 8016 51 1788 24 8169 66 8004 76 969 77 8171 75 8067 86 283 82 348 94 8077 84 8069 31 8099 87 8173 24 1455 20 8260 20 521 35 8002 99 1312 20 8129 79 8052 44 1278 51 8157 45 980 30 361 94 325 92 8178 57 342 92 1732 59 313 94 8113 90 8130 88 8059 31 365 38 349 61 311 63 332 92 334 94 8128 90 1451 36 399 92 905 33 302 115 1409 38 520 32 1008 44 1710 30 8136 90 367 92 8041 94 8058 93 3315 85 8081 91 308 37 908 34 8023 34 8085 91 1912 92 1926 104 364 92 388 92 8138 90 8019 34 396 92 8028 33 8133 90 8044 34 8143 90 8145 45 8057 93 8061 94 329 89 8080 31 8229 49 1306 43 8075 31 356 64 146 98 8181 69 8095 87 8013 96 1913 93 340 92 8142 90 8134 90 503 94 286 94 8123 90 8110 87 8177 36 338 92 1214 52 371 92 1347 40 8031 39 1067 47 1477 35 1441 33 1474 49 305 92 1203 120 421 31 8124 90 316 82 1763 26 8146 67 285 46 8098 87 326 100 8140 90 257 77 1346 35 8127 90 929 101 8216 35 572 45 8008 100 8158 44 1905 107 310 94 915 24 315 96 8289 11 1450 36 1445 46 8161 56 518 35 8215 40 8071 31 1711 27 8097 87 8086 91 1418 43 8139 90 321 101 8141 90 312 94 931 73 331 92 323 101 8020 34 296 60 8109 87 320 58 8038 46 8092 87 360 94 1902 105 333 92 1326 36 8005 91 8147 41 8135 90 1301 59 1316 35 309 92 1453 47 901 38 8001 99 750 93 1906 95 114 95 1442 10 8017 26 369 95 8217 55 8195 38 8155 30 8126 87 347 95 344 92 337 93 314 94 1311 106 749 54 318 107 909 34 8084 91 351 100 3313 85 3312 85 8115 87 8163 59 303 94 8114 90 1181 57 8091 31 1195 103 324 89 800 10 966 10 8183 17 8101 9 8235 20 8065 20 8277 19 8333 17 963 10 8060 15 8340 17 8236 26 1177 10 8256 24 1045 24 1466 10 8111 18 8039 12 125 11 8255 22 8261 16 1714 11 224 14 244 14 8021 12 8105 28 8116 15 8078 15 975 10 8009 12 148 12 8264 16 8120 15 728 28 8088 11 8117 16 8070 11 605 33 150 51 921 10 8151 14 8083 28 8186 14 1028 24 8214 16 8227 13 8213 16 1245 35 106 11 221 14 8240 20 1243 34 1060 24 8238 20 280 16 1424 10 1820 16 8079 15 1814 15 117 16 1082 24 1055 24 281 14 8094 14 157 19 8121 16 8055 30 104 25 8106 27 8051 15 118 35 8224 19 8188 29 8037 17 8022 11 1462 10 139 15 8335 17 8152 17 8144 15 8339 17 8102 37 8030 31 8027 41 717 45 8066 18 8334 17 8191 28 124 16 8259 11 8148 10 8192 9 1223 39 1061 24 918 10 8046 14 1828 16 645 17 1240 39 217 15 8150 16 8176 14 8184 16 1033 24 8011 15 771 25 8344 17 1092 24 107 11 8108 11 101 26 8015 11 292 11 1797 11 1047 24 1808 16 8040 17 615 12 123 13 156 25 1773 17 8190 28 8346 18 8164 12 229 17 8118 18 8104 11 1410 10 641 12 8063 17 8100 9 151 32 147 25 8210 14 143 16 8103 35 739 25 277 11 920 12 911 12 8043 16 725 26 1826 16 8062 20 8006 11 8087 11 8263 20 8230 9 1221 39 1767 16 8318 10 8286 13 8007 28 961 12 1772 11 105 46 8234 20 8208 14 8243 22 953 10 1273 31 1071 24 152 18 1261 35 1016 24 1819 15 8282 23 968 10 1275 12 8174 9 616 12 8180 19 8343 17 997 10 719 45 8168 13 1029 24 8332 17 8189 37 913 12 274 14 606 18 319 33 8233 10 1783 11 8262 15 1057 24 154 11 288 11 8132 9 1464 10 768 25 1284 32 8222 9 8032 20 8228 15 145 26 1078 24 1037 24 8221 9 1253 35 8012 10 8074 16 8185 25 236 14 8162 17 8203 10 8010 15 8268 24 8241 13 8024 21 8107 11 8054 34 8218 11 8056 13 1813 14 8331 17 1220 39 1075 24 8090 30 1232 39 8239 13 239 14 8033 11 8182 24 8245 15 103 25 956 12 149 31 276 11 8089 23 1018 24 228 20 8093 18 119 17 1244 14 8254 9 8237 22 8187 10 8053 36 273 28 923 10 964 10 1817 15 8064 19 1428 11 102 26 1233 42 1417 10 1014 24 1040 24 8122 9 1811 14 120 26 1031 24 763 24 8025 12 8199 11 1431 11 8285 13 8125 9 8018 31 627 12 1222 39 219 33 1825 16 8246 46 8119 15 8076 20 8200 10 1087 24 1219 44 8035 14 1465 10 914 12 509 57 8280 10 110 26 158 15 637 12 8096 21 218 38 948 12 718 45 742 25 8295 18 8251 9 8212 12 8336 17 8149 15 1810 14 298 14 934 11 290 17 246 24 8068 12 1809 38 8196 20 8338 17 8225 32 766 25 1079 24 8345 17 8269 22 720 26 8034 14 1050 24 912 10 8029 19 634 12 910 12 942 10 8082 20 8220 11 8337 17 8112 18 1454 12 202 42 1007 36 737 31 748 11 744 14 747 11 517 22 220 17 1737 13 1000 15 387 13 533 9 775 12 3913 9 562 9 2510 9 1345 9 1264 9 1786 11 393 9 1184 9 253 9 234 31 1511 10 1201 10 712 11 2810 9 903 9 1801 14 1012 9 3011 9 1109 10 1903 14 2129 23 1730 11 1117 10 703 17 944 9 511 9 807 11 1174 9 3321 9 1327 22 2901 9 743 11 1512 13 1914 17 2513 9 1259 9 1186 9 3703 10 976 9 2905 9 2803 9 536 14 1247 9 1456 12 2804 9 1351 10 1515 9 1291 29 1238 9 2119 23 385 9 1936 17 1217 14 1205 31 1743 13 412 9 1024 24 1062 23 2127 23 1411 10 935 9 965 9 516 10 1305 11 1134 9 765 9 3912 9 1314 31 1034 9 3713 11 2114 23 1806 12 2100 23 3911 9 2811 9 2103 23 1053 18 1281 9 1127 9 1447 9 204 9 1069 9 1277 9 237 12 528 15 530 15 131 10 275 35 266 23 3712 13 522 12 366 9 1437 9 1194 9 1795 11 1140 9 1133 9 3012 9 1076 23 414 9 1121 9 1469 9 1066 9 1267 9 1832 12 2807 9 1152 9 1433 9 850 9 3907 9 2123 23 724 9 1821 9 1901 14 734 11 1193 19 3502 9 1030 9 1329 9 2112 23 1909 11 1143 9 210 37 1188 10 1269 30 1750 11 3407 9 957 9 1250 10 2122 23 916 9 635 10 2518 9 330 9 727 9 2707 9 1187 9 566 9 2046 9 2505 9 556 9 1802 15 1120 9 623 9 906 9 1506 10 3014 9 376 9 1340 9 419 9 1739 10 586 9 1093 9 1286 16 2130 23 2534 9 353 56 1027 23 343 9 510 9 2101 23 1173 9 527 10 2117 23 1508 10 1771 11 2560 9 1509 13 559 12 1470 9 1439 9 1130 21 2503 9 287 9 1784 10 3409 9 3419 9 1753 10 2806 9 1317 10 1265 9 1824 12 3723 10 1048 9 640 10 642 9 1017 23 523 10 3302 9 618 20 505 12 271 16 3734 9 880 9 632 20 1307 40 502 9 529 9 1207 31 1236 9 542 9 3904 9 1304 9 998 9 1422 9 553 9 2511 9 1235 9 2554 9 769 9 2530 9 1956 17 1135 12 2106 23 922 9 2701 9 1325 12 1146 9 1052 23 1013 23 406 21 1206 43 588 9 2110 23 2805 9 1423 9 2132 23 1179 9 1141 15 1287 9 1026 23 981 9 2602 9 999 9 1829 9 2705 9 525 10 1309 14 1145 10 2504 9 1320 10 583 9 1084 23 1425 9 1237 9 1443 9 938 9 9901 10 1285 9 251 15 1911 11 289 9 3919 9 383 9 1790 10 227 32 506 9 1330 9 1041 23 2557 9 3410 9 952 9 3415 9 555 10 2709 9 565 23 1712 10 3404 9 3311 9 403 10 1818 11 1011 9 3909 9 2808 9 974 9 3411 9 1085 9 3314 14 561 12 1400 9 132 10 507 9 514 20 3306 9 300 9 535 10 543 17 223 9 391 11 603 10 726 9 596 22 3406 9 245 31 1823 18 1162 9 1360 9 339 9 249 32 1049 9 1446 9 1032 9 1332 9 1310 23 1946 17 1803 23 2704 9 3413 9 416 9 2501 9 363 9 2115 23 2104 23 1407 11 1302 22 927 9 3501 9 937 9 214 9 3401 9 589 15 1478 9 1288 9 1197 9 1807 13 2128 23 598 10 1283 9 1724 11 1165 9 1917 17 2800 9 3906 9 1056 9 1461 9 3905 9 3707 10 3412 9 1089 23 548 9 563 9 1459 9 397 9 732 9 1904 11 568 9 994 9 772 9 564 31 3725 11 2133 23 745 20 767 10 2118 23 1252 9 1248 28 3717 13 1404 29 1338 9 3920 9 1185 9 2097 9 643 9 1523 10 639 10 231 9 706 9 1171 9 401 9 1705 14 238 18 1460 9 215 31 1501 16 8325 14 415 10 1230 30 958 9 972 9 Figure 1: Spaghetti process of the ERP presented in [46]. solution that instead of retrieving a single–complicated configuration workflow, is able to cluster the configurations according to different metrics. Our solution takes information from the variability model as input and retrieves less complex configuration workflows that can assist the development of better configurators. COLOSSI is validated using an ERP case study taken from [ 46 ]. Results show that the metrics of the retrieved configuration workflows are improved by a 99% in the best case and 81% in the worst case with respect to the spaghetti like first solution. The contributions of this paper are as follows: • An automated technique based on process mining to select a configuration workflow that better fits the configuration logs according to a set of metrics. •A validation of the proposal using a realistic ERP scenario. • An available implementation that can be applied to other datasets. The remainder of this paper is organised as follows: Section 2 details the solution and concepts that grounds our proposal; Section 3 presents empirical results from analysing COLOSSI; Section 4 presents the related work and Section 5 presents concluding remarks and lessons learned. 2 COLOSSI: CONFIGURATION WORKFLOW PROCESS MINING SOLUTION In order to create a configuration workflow, a feature model and a configuration log must be combined. Figure 2 shows an overview of the COLOSSI approach. Using the configuration log, it is possible to apply process mining techniques to derive a valid configuration workflow representing all the possible paths defined in the configuration logs. It is likely that the resulting workflow follows the so-called spaghetti-style [ 62 ] and it is therefore difficult to understand and manipulate. Nevertheless, it is important to remark that it can be already exploited by process mining automated tools to extract metrics, perform simplification over the workflow as well as many additional analysis. Also, any generated configuration workflow can be already used to build automatically a configurator. In addition, to the usage of process mining techniques, we propose handling and clustering methods to reduce and group similar configuration traces according to some properties. Those clusters can then be used again as input of process mining techniques to obtaining a set of configuration workflows depending on the observed behaviour of the configuration logs. Those workflows will obtain better metrics with respect to the original complex workflows of step 1 ○ . Our conjecture is that the resulting configuration workflows of step 2 ○ will better guide the domain engineers in the construction of a configurator as well as the analysis mentioned previously. Following, we describe the details of the different elements of COLOSSI. 2.1 Inputs A feature model is an arranged set of features that describes variability and commonality using features and relationships among them. [ 18 , 57 ]. FMs describe all the potential combinations of features. Figure 3 shows an excerpt of a feature model of the ERP domain where features are arranged in a tree–like structure and different relationships are established among them. FMs can be used to build configurators that are pieces of software that guide the configuration process while selecting and deselecting features. An example of a configurator is KConfig, a tool that helps configuring the Linux kernel. As an FM can define a configuration space defined by all the possible feature combinations, it can also define different possible configuration workflows that can be derived using the same FM. COLOSSI takes as input a FM and a configuration log. To define a configuration log, we use some definitions that are used in process mining area to define events and traces and we map those definitions to define a configuration log. An event log is a multiset of traces: Definition 2.1. (Event Log). Let L be an event log L=[τ1,· · · ,τm] as a multiset of traces τi. A trace is a tuple with an identifier and a sequence of events that occurred at some point in time: Definition 2.2. (Trace). Let τ be a trace τ = ⟨case_id , E⟩ which consists of a case_id which identifies the case, and a sequence of events E={ε1,· · · ,εn} , εi occurring at a time index i relative to the other events in E. An event occurrence is a 3–tuple with an identifier of an activity that occurred at some timestamp and that can have additional information: Definition 2.3. (Event occurrence). Let ε be an event occurrence ε = ⟨activity_id , timestamps , others⟩ which is specified by the identity of an activity which produces it and the timestamps. It can store more information (i.e., states, labels, resources, etc.) In COLOSSI, we conceptually map elements from the feature modelling domain to the process mining domain as shown in Table 1. Concretely, an event log is conceptually a configuration log. A trace is an ordered configuration, i.e., a configuration trace, thus, it is a set of selected features that follow a given order. Finally, an event occurrence is a feature. Additionally, a feature can have more information like attributes associated with this feature such as preferences, metrics or the like. 2.2 Configuration logs extractor A configuration log is composed of a set of configuration traces where each configuration trace encodes not only the features of a configuration but the timestamps indicating when each feature Raw Configuration Log conf 1 conf 2 conf 3 ... conf n Configuration log handler Configuration log cluster generator Process discovery Feature model Clustered Configuration Logs Configuration Log Extractor Process discovery Process Mining Configuration workflow Clustered configuration workflows  XES Configuration Log 1 2  XES conf 1 conf 3  XES conf 1 conf 3  XES conf 1 conf 3  XES Configuration Log' Figure 2: COLOSSI solution overview. Feature model ERP requires CRM Project Managment Accountancy TaskListCalendar CostSpend Check Credit Card 1stconfiguration:{crm,tasklist,cost,accountancy} 2ndconfiguration:{projectmanagement,accountancy,creditcard,calendar} 3rdconfiguration:{projectmanagement,check,crm,tasklist} 4thconfiguration:{crm,tasklist,calendar,creditcard} Configuration log Figure 3: ERP domain based example. Process Mining Product Line Event log Configuration log Trace Configuration trace Event occurrence Feature Table 1: Mapping concepts. was selected. In a raw configuration log, we can find a diversity of meta-information among the selected or deselected features. Moreover, this meta-information can be presented in a unstructured or structured fashion. In this first step, we take as input a raw configuration log and output a set of configuration traces. Therefore, we need to i) search for the meta-information encoding the timestamps for each feature. Note that this might not be explicit and can be provided using other mechanisms (e.g., line numbers in a plain text format); ii) use this meta-information to represent the feature selection order, and; iii) store the set of configuration traces in a format that can be used throughout the configuration workflow retrieval process (e.g., XES serialization [ 1 ]). After this, we end up with a set of configuration traces that represent the selection order used by the configurator users. However, there might be non-valid configurations and other erroneous configurations w.r.t domain information. 2.3 Configuration logs handler At this step, the configuration log might contain non-valid configurations, erroneous partial selection of features among other domain-related errors such as those depicted in [ 6 ]. To remove clutter and noise out of the workflows, users might prefer to remove such information from the configuration log. This cleaning step consist on removing wrong selection of features (a.k.a non-valid partial configurations) as well as generate metrics that can be latter exploited to optimise the workflow retrieval process. For example, the use of atomic-sets to complete partial configurations. Depending on the expected workflow usage, domain engineers have to define the meaning of a valid configuration and the metrics to rely on. For example, a SPL engineer might consider only configurations with complete assignments of features to develop a configuration while other might find interesting to consider full assignments (i.e., to configure only the variability part of the product line, keeping aside the common parts). 2.4 Process mining - discovery Process mining is a family of techniques based on event logs that can be categorised as process discovery, conformance checking and enhancement [ 63 ]. In this paper, we are focused on the use of process mining to analyse the configuration logs for the discovering of configuration workflows based on the user experiences. Process discovery in process mining brings together a set of algorithms to generate a workflow process model that covers the traces of activities observed in an organisation [40]. The evolution of algorithms during last decades has allowed the discovery of complex models that are able to involve not only the activities executed in the daily work of companies, but also the persons who execute them and the used resources. Process mining is an important topic that has been well received by the enterprises, bringing about the evolution of the research solution tools (e.g., ProM [ 64 ]) to commercial solutions (e.g., Disco ™ and Celonis ™ ). This facilitates its applicability to several contexts and areas, although variability has been out of the scope of these techniques before this paper. Process discovery in process mining uses a set of traces similar to the configuration log shown in Figure 3, to obtain a model that covers the possible traces. Figure 4 shows the process discovered by Disco tool-suite, which covers every possibility configuration trace. The relational patterns among the definition of the features become part of the model. For example, two features can be the first in the traces (CRM or Project management) or after CRM always Task List is selected. Figure 4 also shows the number of traces that are represented by each transition, giving information about the importance o each part of the traces in the obtained model. 1 1 1 Project managment (2) 1 Check (1) 3 CRM (3) 1 1 1 Task List (3) 1 Cost (1) 1 1 Accountancy (2) 1Credit card (2) 1 Calendar (2) 2 2 Figure 4: Process discovered for configuration log of ERP domain based example. In the framework proposed in this paper (Figure 2), Process mining - process discovery module enables to read an event log and generates a process model that fits these traces. In the case of the variability context, a configuration log is read and a configuration workflow is obtained using the same techniques used for classical process mining. 2.5 Configuration logs cluster generator Configuration processes have a high degree of variability, specially when the configuration order is defined by human decisions. The application of process discovery in this type of scenarios tends to produce spaghetti-like processes, being necessary to apply a pre-processing. Configurability contexts are specially variable in relation to the executed activities derived from the high human intervention, thereby, we propose to divide the traces into subsets, to model different profiles of users and avoiding the discovery of non-user understandable processes. In these contexts, where process discovery is used to infer spaghetti-like processes, frequently clustering techniques such as a pre-processing step are applied [ 26 ]. To adapt the solution to configuration tasks, we propose the division of the configuration traces into multiple clusters before the application of a process discovery. This division lets to discover configuration workflows with more quality. This section describes what a cluster is and the metric (e.g., entropy) used to divide the traces among them. In following sections, we describe how quality is measured and how the clusters can be created. Being L a configuration log composed of a set of configuration traces (i.e., [τ1,· · · ,τm] ), a cluster is a subset of configuration traces from Lthat complies certain properties. Definition 2.4. (Cluster of Configuration Traces). A cluster of configuration logs, c=[τi,· · · ,τj] ⊆ L , where ∀τk⊆L , ∃c|τk∈ c and c’ ,c where τk∈c’. The distribution of configuration traces between various clusters depends on the purpose of the practitioners. In our case, the goal is to group the more similar configuration traces. In this paper, the meaning of ’similar’ is related to both features and transitions involved in the logs. For this reason, we adapted the classical information entropy metric [ 37 ] by introducing two different custom entropy metrics for clustering in the configuration context: • Entropy-features ( Sf eatures )of a cluster: a metric which measures the similarity between a set of traces according to the features that belong to the same cluster. Thus, it is the ratio between the number of features that do not appear in all configuration traces ( f eaturesnat ) and the number of different features in all the configuration traces (f eaturesdif f ): Sf eatures =|f eaturesnat | |f eaturesdif f |(1) • Entropy-transitions ( Stransitions )of a cluster: a metric which measures the similarity between a set of traces according to the transitions that belong to the same cluster. Thus, it is the ratio between the transitions that do not appear in all configuration traces ( transitionsnat ) and the number of different transitions in all the configuration traces (transitionsdif f ): Stransitions =|transitionsnat | |transitionsdif f |(2) In order to illustrate the calculation of entropies, the Sf eatur es and Stransitions for the Cluster 1 and Cluster 2 of Figure 5 are determined in Table 2. Entropy-features Entropytransitions Cluster 1 0 4=00 3=0 Cluster 2 6 8=0,75 6 15 =0,4 Table 2: Entropies for the clusters of the Figure 5. Note that the range of the entropy is [ 0 .. 1 ] . The values of entropy that are close to 0represent more similar traces, whilst when they are close to 1represent that there are different features involved in the traces of the cluster. The best configuration of clusters obtained from a set of configurations traces is the one that minimize the summation of the entropy of all clusters obtained. The challenge is how to obtain the best configuration of clusters as a pre-processing of a process discovery. In order to find out the best configuration traces divided into clusters, minimising the entropy of the resulting clusters, different algorithms for clustering can be used. In accordance with [ 30 ] clustering provides an unsupervised classification of patterns (observations, data items, or feature vectors) into groups (clusters). Clustering [ 31 ] brings together a large set of algorithms that can be classified in different ways according to the point of view necessary in the case of study. We propose the use of the well-known hierarchical agglomerative clustering algorithm base on the work in [69] as the combination of hierarchical and agglomerative clustering. On the first hand, hierarchical clustering is defined as a procedure to form hierarchical groups of mutually exclusive subsets, each of which has members that are maximally similar with respect to the specified characteristics [ 69 ]. In the same study, authors define the process as: assuming we start from n sets, it permits their reduction to n − 1 mutually exclusive sets by considering the union of all possible n(n2 −1) pairs and selecting a union having a maximal value for the objective function. On the other hand, agglomerative clustering is an algorithm that starts from the assumption that each element constitutes a cluster by itself (singleton) and it successively merges these singletons together forming clusters until a stopping criterion is satisfied which is also determined by the objective function. The characteristics used in our solution is based on both entropies presented (features and transitions), combined with the objective function the Ward’s minimum variance method [ 69 ]. This function aims to minimise the sum of the squared differences within all clusters, which means, a variance-minimising approach. Figure 5 shows the obtained dendrogram1 for the example in Figure 3 by using Entropy-features. Project managment (1) Check (1) 1 CRM (3) 1 Task List (3) 3 Cost (1) 1 Accountancy (2) 1 Credit card (2) 1 Calendar (2) 1 1 1 1 1 2 Cluster 2 Traces 1 3 42 Entropy 3.2 3.0 0.2 0.0 Project managment (1) Accountancy (1) 1 Credit card (1) 1 Calendar (1) 1 1 Cluster 1 0.4 0.8 Figure 5: Clustering for the ERP excerpt using the Entropyfeatures. It is well-known that every clustering algorithm builds a distance matrix during its execution based on a given methodology (euclidean, manhattan, etc.). However, hierarchical agglomerative clustering is an exception, since it can be carried out from the distance matrix itself. We consider the entropy matrix as the distance matrix, this leads us to decide for this method of clustering, which can be performed also using other methods, such as single-linkage, complete-linkage,average-linkage, and Ward. Whenever a clustering process is executed, one of the first problems to deal with is to decide which is the optimal number of clusters. Many studies carried out about the discovery of the optimum number of clusters, therefore, we decided to study a significant 1 Dendrogram is a branching diagram which represents the arrangement of the clusters produced by the corresponding analyses number of them to choose by voting the number of clusters that most indicators had selected as optimal. In this regard, 17 different indicators [ 4 , 5 , 9 , 14 , 16 , 17 , 20 , 24 , 25 , 29 , 32 , 33 , 42 , 44 , 45 , 50 , 51 ] are used as reference to choose the best number of clusters adapted to our scenario. By using these indicators with the different clustering methods mentioned above, all of them selected a too large number of clusters as the optimal solution. In addition, the values of the indicators themselves are in some of the cases too dissimilar. For all the methods the range [0-10] is proposed to determine the number of clusters. In the single-linkage method always retrieved the maximum (i.e., 10). For the average-linkage retrieved always an average of 2 clusters. Both methods help to bound the limits of number of clusters between 2 and 10. Only with the complete-linkage and Ward methods, more homogeneous, similar and an assumable number of clusters as results were obtained for the dendrograms showed in Figures 6 and 7. Finally, by making use of dendrograms, it is observed that applying the Ward method, the samples becomes better distributed among the clusters, thereby, generating more differentiated clusters and better structured dendrograms. This fact determines the Ward’s method as the best option for our approach. 2.6 Leveraging the results of COLOSSI The COLOSSI approach can be used in different scenarios to leverage process mining in variability management. One of the scenarios presented in this paper is the building of configurators. However, we envision other areas where process mining can be used to automate different tasks. Next, we describe those scenarios, also related to software product lines, from our experience and perspective: •Configurator building . Up to now, configurators building is performed using manual mechanisms or, at most, using the information present in the variability model (e.g., tree traversal in feature models). With COLOSSI, we open the door to use existing configuration logs to build configurators. This novel approach can open the door to new ways of assisting configurators builders by using the generated configuration workflow to optimise configurators. •Data analysis . From the generated configuration workflow it is possible to perform many analysis in terms of graph metrics. Deadlocks identifications, misalignment analysis, metrics extraction –to just mention a few– are areas where process mining techniques can be useful. •Testing . From the data extracted in the former item, it could be possible to define new sampling techniques [ 61 ] that can improve the identification of bugs or feature interactions in existing product lines. •Variability reduction . One of the challenges for companies that develop software product lines is variability reduction [ 8 ]. While variability is a must in a software product line approach, it is always difficult to find a trade off between a high degree of variability and a systematic management of such a variability. In this context, experts claim for techniques and tools to reduce variability while preserving configurability. Process mining techniques presented in this paper can be a first step towards defining tools to assist in the decision of variability reduction. •Reverse engineering . One of the inputs used when reverse engineering feature models are configurations (a.k.a. product matrix). We envision that the techniques described in this paper can be used in reverse engineering of variability models. For instance, the generated configuration workflow can be analysed to better guide reverse engineering algorithms 3 EVALUATION In this section we present the evaluation of COLOSSI. The evaluation consists of the application of the framework detailed in Section 2 to a configuration log obtained from a real scenario. The possible clusters derived from the application of the defined entropies are analysed. 3.1 Experimentation data In order to analyse the applicability of our example in a configuration real scenario, we used the raw information from [ 46 ]. The used data include a configuration model representing a real ERP, as well as a raw configuration log. The ERP feature model has 1920 features and 59044 cross-tree constraints. Also, the configuration log is formed of 35193 event occurrences that represent a total of 170 different configuration traces with an average of 207 features per configuration trace. 3.2 Framework application In this section we detail each task of the framework presented in Figure 2. 3.2.1 Configuration Log Extractor. The input data of the configuration of the ERP is represented in a CSV file with two elements, the configuration id and the feature that is configured. Note that a feature can appear in one or more traces, but no more than once in the same trace. Then, the timestamp required to extract the traces was taken by the line number in which the features where appearing throughout the file in a sequential order. This is, we assume that the timestamps were implicit based on the order of appearance (i.e., line numbers) then, transformed them into a more standard format for traces. Concretely we use in our solution the IEEE Standard for eXtensible Event Stream (XES) [ 1 ]. This is a standard to serialise, store, exchange events data and it is commonly used in process mining techniques. 3.2.2 Configuration Log Handler. To clean up the set of configurations retrieved by the extractor we decided to consider only valid partial and full configurations. This filtering operation is performed by using the FaMa framework [ 7 ]. After filtering, the valid partial configurations using automated analysis [ 6 ], we ended up considering 61 configuration traces from the initial set of 170. 3.2.3 Configuration Log Cluster Generator and Process Discovery. Figure 6 and 7 represent the obtained clusters according to the dendrogram built by means of the entropy of features and transitions analysis respectively. In the case of feature entropy (Figure 6), five clusters are obtained. On the other hand, when the transition entropy is used, three clusters are derived to split the configuration logs into simpler configuration workflows. In the following subsections, the details about the obtained configuration workflows and clusters are analysed. 3.3 Analysis of Results In order to evaluate how the application of clustering can improve the configuration workflows obtained by COLOSSI, in this section, we compare the models discovered: (1) the original configuration logs obtained from the initial raw configuration log (cf. Section 3.1); (2) the filtered version of the same log including only valid configurations (i.e., after applying configuration log handler), and; (3) two set of clusters based on the proposed entropies (features and transitions explained in Section 2.5). The analysis is carried out following two different perspectives: (1) the analysis of the discovered configuration workflows and (2) the analysis of the set of configuration traces involved in each cluster used in the process discovery. 3.3.1 Analysis of discovered configuration workflows. First, highlight that inductive process discovery techniques used by COLOSSI, ensure the soundness and correctness of the process models obtained [ 35 ]. Thus, an analysis of the soundness and correctness of the configuration workflows are unnecessary since processes discovered is always complete, have a proper completion, and have no dead transitions. However, the complexity of the configuration models is affected by the number of features, the number of configuration traces and the number of transitions. Obviously, the filtering of the configuration traces or the division of the logs will bring about simpler configuration workflows. Figure 8 depicts in a comparative way the number of features, configuration traces and transitions of the set of configuration logs using in each scenario: original configuration log, filtered configuration log only with valid traces, the 5clusters obtained by using entropy-features, and the 3clusters obtained by using entropy-transitions. Regarding general parameters, there is an enormous difference between the original, filtered version, and both that use clustering. The original configuration workflow contains 1652 features and 3330 transitions, whilst the filtered version has one less magnitude order of features and transitions. The clusters seem similar to the filtered version, however, each cluster contains, at least, less than half regarding features. In case of the transitions, the filtered version reached four times fewer transitions than the original. However, clusters reached in the worst case 500 transitions less than the filtered and more than one third fewer transitions in the best case. Due to the clusters group a set of similar traces, the number of traces is intrinsically smaller regarding the configuration traces of the original and filtered configuration workflow. In conclusion, clusters enable to reduce the complexity of configuration workflow discovered by reducing the configuration traces involved in the same configuration workflow. However, the question is what level the quality of the obtained workflow is improved, and which distribution of cluster-entropy works better. In literature, several metrics are used to measure how "good" is a design of a business process model [ 10 , 43 , 48 ]. Discovered configuration workflows are also processes with features instead of activities, therefore, these metrics can be adapted to measure Traces Entropies Figure 6: Clustering for the ERP example using the Entropy-features. Traces Entropies Figure 7: Clustering for the ERP example using the Entropy-transitions. the quality of our obtained configuration workflows. The next set of metrics is adapted to measure the understandability and the complexity of the configuration workflows to compare the four discovered configuration workflows: • Density: the ratio of transitions divided by the maximum number of possible transitions. The lower the value of density, the higher the understandability and the lower complexity. • Cyclomatic number ( CC ): the number of paths needed to visit all features. The cyclomatic can be seen as a complexity metric, thus, the lower the value of CC , the lower the level of complexity. • Coefficient of connectivity ( CNC ): the ratio of transitions to features. The greater the value of CNC , the greater the complexity of configuration workflows. Although, the authors in [ 43 ] remark that models with the same CNC value might differ in complexity regarding this parameter. • Control Flow Complexity ( CFC ) enables to measure the complexity in terms of the potential transitions after a split depending on its type. The greater the value of the CFC, the greater the overall structural complexity of a workflow. 1652 170 3330 433 61 869 78 18 166 160 15 297 174 12 297 174 5 297 144 11 297 192 16 416 195 32 377 192 32 377 0 500 1000 1500 2000 2500 3000 3500 Features Configuration Traces Transitions Original Filtered C11 C12 C13 C14 C15 C21 C22 C23 Cluster -Features Cluster -Transitions Figure 8: Characteristics of the configurations logs. Table 3 shows the results of these metrics obtained from the discovered configuration workflows. As explained, for every metric the lower value they take, the better the understandability and less complexity. We shall highlight that the metrics associated with the different clusters are aggregated as arithmetic means for a better comparison. Config. Workflow Density CC CNC CFC Original 0,00123 1679 2,016 1677 Filtered 0,00464 437 2,069 434 Cluster-Features 0,01469 125,8 1,892 118,4 Cluster-Transitions 0,00632 118,8 1,213 156 Table 3: Metrics of the configuration workflows. It is interesting to note how configuration workflow for the original has the lowest density in comparison with the others. This is because the number of features is so high and it compensates the largest number of transitions. However, it has the highest complexity compared to CC and CFC . However, no conclusions can be achieved with regard to the complexity based on CNC . Therefore, although the density is the lowest, the other three metrics demonstrate that the workflow for the original is very complex and misunderstood. On the other hand, the workflow for the filtered version has the highest density, thus, it is the less understandable regarding to this metric. However, the complexity shown by the other metrics demonstrates that it is more understandable and less complex than the original configuration workflow. Comparing the workflow for the filtered version with both cluster versions, we can conclude that both clusters are more understandable due to lower values related to the four complexity metrics, (i.e., density, CC,CNC and CFC). In fact, these four metrics help us to know the complexity and understandability of the configuration workflows from the design perspective and the elements in the model. Nevertheless, these metrics used to measure the quality of the workflow are inconclusive to measure the real usefulness and quality of the discovered workflows applied to the context of the variability management. 3.3.2 Analysis of clustering. As introduced in previous sections, two different entropies are applied to infer the clustering (i.e., Entropy-features and Entropy-transitions). The two entropy formulas can help to understand the quality of the workflow. Thus, a lower value of entropy more quality of the cluster, hence, workflow has more quality. In this regard, Table 4 gives the values regarding the number of clusters and the entropy for each configuration workflow. In this case, the metrics associated with the clusters are aggregated as arithmetic means for better comparison. It is important to highlight that the clusters group a less number of configuration traces, the entropy of the clusters (as mean) are less in both cases than the original and the filtered solution. Config. Workflow N. of Clusters Entropy Entropy Quality features transitions (∆≡) Original 1 1 0,021 1598,84 Filtered 1 1 0,027 365,18 Cluster-Features 5 0,188 - 59,02 Cluster-Trans. 3 - 0,0052 115,037 Table 4: Comparison of the number of clusters, entropy and quality. In order to compare the clusters, the entanglement metric is determined as shown in Figure 9. The entanglement indicates the relation between two dendrograms charts, thus, two different distributions of clusters. The range of the entanglement is [ 0 .. 1 ] . The entanglement values closer to 0are better than to 1. Thus, the entanglement helps to understand how similar are the dendrograms, thereby, how similar clusters are with the independence of the entropy. In this case, the entanglement is 0 , 38 which indicates that both clusters are very similar, in other words, the entropy used to perform the clusters reach similar cluster distributions in this case. As previously mentioned, the uselessness of the quality metrics related to the workflows leads us to define a new custom metric which enables to establish the quality level of the workflow by relating the number of features and their occurrence within the discovered workflow of a cluster. Thus, a metric that enables us Figure 9: Comparative of entanglement between clusters (Entropy-features (right) and Entropy-transitions (left)). to measure how spaghetti is the workflow obtained. Our customquality metric is defined as follows: • Quality ( ∆≡ ) measures the difference between the total number of features and the ratio of the sum of the number of times that a feature is selected for each configuration trace and the number of configuration traces. Formally, given a workflow based on a set of configuration traces ( CT ) and a set of features ( Features ), the quality can be determined as the following formula: ∆≡=|Featur es | − Õ f∈W or k f low occur rences(f) |CT |(3) The range of the quality is [ 0 .. |Features|] , the lower value of quality indicates a better configuration workflow. The number of features and configuration traces are group into the more similar workflow, therefore, it brings about that the quality is near to 0. For instance, using the example in Figure 3 and the Cluster 2 in Figure 5, the number included in the rectangle of the feature corresponds to the number of times that the feature is selected regarding the configuration traces. Hence, the quality for the Cluster 2can be determined applying the formula as follows: ∆≡=8−3 3+3 3+1 3+1 3+3 3+1 3+2 3+2 3≈2,67 (4) Comparing the quality results in Table 4, the conclusion is that the original configuration workflow obtains the worst quality and the five clusters obtained using entropy-features have the best quality. Thus, the distribution of configuration traces in the five clusters (cf., Cluster-Features in Figure 8) achieves better results than the original, filtered, even another cluster regarding. In conclusion and according to the defined quality, the Cluster-Features are less complex, more understandable and less spaghetti than the other configuration workflows. 3.4 COLOSSI implementation COLOSSI is supported by the implementation of a tool which is composed of the next main components: (1) Configuration log extractor is a piece of software module which takes a set of raw configuration log (including timestamps) in a semi-structured format and returns a XES file. (2) Configuration log handler is another piece of software which takes a FM and a XES log as input. First, apply a set of operations over the FM as described in Section 2.3. Then, a data cleaning is carried out over the XES log to get a filtered configuration log. The output of this connector is a new XES log with the filtered configuration log. (3) Cluster generator is a Python/R module which takes a XES log file which is translated into a matrix. This matrix enables the entropy calculation and based on the analysis of certain parameters and the dendrogram, the number of clusters is determined. Using this information, a hierarchical agglomerative clustering algorithm is applied to determine the clusters. A new XES log file is generated for each cluster that composed the final output of this component. (4) Discovery connector is a piece of software which gets the XES file logs of each cluster and automatically feed the ProM to discover the process models by means of the Inductive Miner. The output of this component is a process model in Petri-net or BPMN format. All the resources, thus, configuration logs, the XES files, the workflows discovered, the source code of the COLOSSI tool (i.e., git repository), and a Jupyter notebook that are employed in this work are freely available at 2 http://www.idea.us.es/splc2019/. The notebook is self-explanatory and allows users to work interactively by executing step-by-step instructions to get the clusters. 3.5 Threats to validity Even though, the experiments presented in this paper provide evidences that the solution proposed is valid, there are some assumptions that we made that may affect their validity. In this section, we discuss the different threats to validity that affect the evaluation. External validity. The inputs used for the experiments presented in this paper were either realistic or designed to mimic realistic feature models. However, we do not control the development process and it may have errors and not encode all ERP configurations. The major threats to the external validity are: 2DOI:10.5281/zenodo.3236337