Comparing interactive evolutionary multiobjective optimization methods with an artificial decision maker

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY 4.0 https://creativecommons.org/licenses/by/4.0/ Comparing interactive evolutionary multiobjective optimization methods with an artificial decision maker © 2021 the Authors Published version Afsar, Bekir; Ruiz, Ana B.; Miettinen, Kaisa Afsar, B., Ruiz, A. B., & Miettinen, K. (2023). Comparing interactive evolutionary multiobjective optimization methods with an artificial decision maker. Complex & Intelligent systems, 9(2), 1165-1181. https://doi.org/10.1007/s40747-021-00586-5 2023 Complex & Intelligent Systems https://doi.org/10.1007/s40747-021-00586-5 ORIGINAL ARTICLE Comparing interactive evolutionary multiobjective optimization methods with an artificial decision maker Bekir Afsar1 ·AnaB.Ruiz 2 ·Kaisa Miettinen1 Received: 5 July 2021 / Accepted: 12 October 2021 © The Author(s) 2021 Abstract Solving multiobjective optimization problems with interactive methods enables a decision maker with domain expertise to direct the search for the most preferred trade-offs with preference information and learn about the problem. There are different interactive methods, and it is important to compare them and find the best-suited one for solving the problem in question. Comparisons with real decision makers are expensive, and artificial decision makers (ADMs) have been proposed to simulate humans in basic testing before involving real decision makers. Existing ADMs only consider one type of preference information. In this paper, we propose ADM-II, which is tailored to assess several interactive evolutionary methods and is able to handle different types of preference information. We consider two phases of interactive solution processes, i.e., learning and decision phases separately, so that the proposed ADM-II generates preference information in different ways in each of them to reflect the nature of the phases. We demonstrate how ADM-II can be applied with different methods and problems. We also propose an indicator to assess and compare the performance of interactive evolutionary methods. Keywords Decision making ·Preferences ·Performance comparison ·Many-objective optimization ·Interactive methods Introduction Multiobjective optimization problems refer to optimizing multiple conflicting objectives and arise in many application areas such as engineering, economy, or industry. Usually, no solution exists in which all objectives achieve their individual optima at the same time. Instead, there exists a set of the socalled Pareto optimal solutions, at which an improvement in one objective is only possible at the expense of getting worse values in, at least, one of the others. All Pareto optimal solutions form the Pareto optimal set. Pareto optimal solutions are mathematically incomparable, so additional preference information, usually coming BBekir Afsar bekir.b[email protected] Ana B. Ruiz [email protected] Kaisa Miettinen [email protected] 1University of Jyvaskyla, Faculty of Information Technology, FI-40014 University of Jyvaskyla, Finland 2Department of Applied Economics (Mathematics), Universidad de Málaga, C/ Ejido 6, 29071 Málaga, Spain from a decision maker (DM), who is an expert in the problem domain, is required to identify the most preferred solution (MPS) as the final solution. The role of the DM in the solution process varies depending on the type of multiobjective optimization method [19]. In a priori methods, the DM expresses her/his preferences before the solution process starts, while in a posteriori methods, preferences are used in selection once a representative set of Pareto optimal solutions has been generated. On the contrary, solution processes with interactive methods consist of iterations, where the DM is actively involved by directing the search with preference information. (S)he iteratively sees information about the solutions available, and expresses and fine-tunes or even changes her/his preference information at each iteration until (s)he is satisfied with some of the solutions. The main benefit of interactive methods is that the DM can learn which types of solutions are feasible without dealing with large amounts of data at once. At the same time, (s)he can progressively adjust one’s preferences based on the gained insight into the problem. Actually, in an interactive solution process, we can often distinguish two phases [21]: a learning phase, where the DM explores different solutions to find a region of interest (ROI) formed by the Pareto optimal solutions that satisfy her/him the most; and a decision phase, 123 Complex & Intelligent Systems where (s)he fine-tunes the search within this ROI to select her/his MPS. Over the years, many interactive methods have been proposed [18–20]. Among them, interactive evolutionary multiobjective optimization (EMO) methods [4] are suitable for problems with, e.g., discontinuous and non-differentiable functions. EMO methods incorporate preference information into an evolutionary process to generate a population of solutions approximating the ROI best by reflecting the given preferences [18,29]. To find a suitable method among the many alternatives available, we must test and compare different interactive methods to understand their potential to fulfill different needs of the solution process successfully. However, the quantitative assessment of interactive methods involving real DMs is not a trivial task due to several reasons [2,16]. It is expensive to involve many DMs with the appropriate domain expertise to run a sufficient amount of tests. Naturally, DMs learn about the problem during the interactive solution process, and thus, the order in which methods are applied affects their performance. To compensate for this, we would need an even higher number of DMs. A survey of published comparisons of interactive methods is given in [2], where the need for characterizing desirable properties of interactive methods is emphasized. Overall, the survey supports the need for improved means for comparing interactive methods. Overall, it is hard and time-consuming to design experiments with human DMs to compare different interactive methods due to their subjectivity, learning of the problem, human fatigue, and other limiting factors. However, to some degree, comparisons can be conducted without humans. According to [26], we can divide interactive methods into non ad-hoc and ad-hoc ones depending on whether a value function can be used to replace the DM or not, respectively. If the preference information used in the method cannot be derived from a value function, we need different means for assessing interactive methods, and this is the focus of this paper. Socalled artificial DMs (ADMs) have been introduced in the literature to replace the DM and run ad-hoc methods conveniently. However, they are suited for methods involving a single type of preference information. To the best of our knowledge, there are only a few ADMs to compare interactive methods: [1,3,13,24]. The one suggested in [13] simulates the learning of a DM by progressively narrowing the angle of a cone, which is defined based on a pre-defined MPS. In [3,24], ADMs are proposed for comparing reference point-based interactive methods (the former directed at EMO methods). Both of them consist of a predefined steady part that includes an aspiration point initially set (formed by aspiration levels for the objectives), which the solution process must converge to and which remains unchanged, and a current context that evolves based on the knowledge gained about the problem during the solution process. Note that these ADMs are based on a goal point set initially (an MPS in [13] and an aspiration point in [3,24]) and their performance highly relies on this point. These ADMs run each of the methods individually, which means that the preferences used at each iteration with each of them are different, since they are generated based on the output of every single method. We focus here on ADMs for comparing interactive EMO methods of ad-hoc type. We extend our previous ADM [1], which was tailored to methods applying reference points, to compare interactive EMO methods applying different types of preference information. We call it ADM-II. Both ADMs are designed to run all the methods to be compared simultaneously using similar preference information at each iteration. Furthermore, these are the first ADMs that generate preferences depending on the phase (learning or decision) of the interactive solution process to allow a better analysis of the performance in each phase. As said, the main novelty of ADM-II is its ability to generate different types of preference information, thus clearly extending the scope of the existing ADMs. Besides a reference point, the following types of preferences can be generated: selecting one or several preferred solution(s) or non-preferred solution(s) among a set of alternatives, specifying preferred ranges for the objective functions, and performing pairwise comparisons among solutions. The further novelty lies in the way preference information is generated in the decision phase (compared to [1]). At each iteration, ADM-II generates preference information based on the solutions obtained so far by all methods that are compared. In this way, we adapt the preferences to the insight gained during the solution process. To perform a fair comparison, the same computational resources (i.e., number of function evaluations or generations per iteration) are internally assigned to each interactive EMO method compared. To evaluate the performance of a method, we must measure the quality of the solutions produced at each iteration taking into account the preferences, and we propose a performance indicator for this purpose. This indicator counts the number of nondominated solutions which are in a composite front consisting of the nondominated solutions of populations of all compared methods. This composite front is updated, while ADM-II is performing iterations. Our indicator measures the number of nondominated solutions with which each method has contributed to building the composite front. It indicates the exploratory potential of each method and its adaptation capacity to the changes in the preference information in each phase. Furthermore, in the decision phase, the quality (i.e., Pareto optimality) of the final MPS reached can be evaluated using, e.g., an achievement scalarizing function [28], which provides a measurement of each method’s convergence capability. It is important to note that the quality 123 Complex & Intelligent Systems of the results obtained depends on the methods themselves, not on ADM-II. To summarize, our main contribution is proposing ADMII to provide a computational tool to gain deeper knowledge about the performance of different interactive EMO methods without involving human DMs. This means that many repetitions can be done in stable conditions. Unlike [1], our ADM is able to compare methods using different types of preferences, which has not been done earlier in the literature. Thus, it can be used to test several interactive methods. We do not claim that ADM-II can investigate all human biases that can affect decision making, but ADMs provide good means for finding viable candidate methods that can be further tested with humans or directly applied to solve the problem in question. In addition, we demonstrate how ADM-II can be applied by comparing some interactive EMO methods with a set of benchmark problems with up to nine objectives. Besides the new indicator, we also report the number of function evaluations used and apply a quality indicator developed for a priori EMO methods. The rest of the paper is organized as follows. We present the background concepts of multiobjective optimization in “Background concepts”. “Artificial decision maker for interactive EMO” gives a detailed description of the proposed ADM-II while, in “Computational experiments”, we demonstrate the performance of ADM-II comparing several interactive EMO methods applying different types of preference using benchmark problems. Finally, we conclude and mention future research directions in “Conclusions”. Background concepts In general, a multiobjective optimization problem can be formulated in the following form: minimize {f1(x), f2(x),..., fk(x)} subject to x∈S,(1) where fi:S→Rare the kconflicting objective functions (with k≥2) to be optimized at the same time. The decision vectors x=(x1,x2,...,xn)Tbelong to the feasible set S⊂ Rn, whose images in the objective space, denoted by f(x)= (f1(x), f2(x),..., fk(x))T, are called objective vectors. Usually, the conflict degree among the objectives makes it impossible to find a solution where all the objectives can reach their individual optimum. Therefore, we are interested in the so-called Pareto optimal solutions, at which no objective function value can be improved without impairing, at least, one of the others. Given z1,z2∈Rk, we say that z1dominates z2if z1 i≤z2 ifor all i=1,2,...,kand z1 j<z2 jfor, at least, one index j.Ifz1and z2do not dominate each other, they are (mutually) nondominated. Furthermore, a decision vector x∗∈Sis Pareto optimal if there does not exist another x∈S, such that f(x)dominates f(x∗). The corresponding objective vector f(x∗)is called a Pareto optimal objective vector. The set formed by all Pareto optimal solutions is called the Pareto optimal set, denoted by E, and its image in the objective space is referred to as the Pareto optimal front, denoted by PF. Since we deal here with EMO methods, they cannot guarantee Pareto optimality, but we deal with nondominated solutions approximating Pareto optimal ones. The ranges of the objective function values in the PF are defined by the ideal and the nadir points. The ideal point z=(z 1,...,z k)Tis obtained by z i= minx∈Sfi(x)=minx∈Efi(x)(i=1,...,k) and contains the lowest objective function values. The nadir point znad = (znad 1,...,znad k)Tcan be defined as znad i=maxx∈Efi(x) (i=1,...,k) and is formed by the highest (i.e., the worst) objective function values between Pareto optimal solutions. In practice, the nadir point is usually approximated, since its computation is difficult as the set Eis unknown (see, e.g., [9,19,27] and references therein). Alternatively, the DM can also be asked for the worst possible objective function values and consider them as the components of the nadir point. There are different ways of expressing preferences [17, 19,25]. The options available for expressing preferences in the ADM-II that we propose here are the following: •Giving a reference point q=(q1,...,qk)T, where each qiis a desirable aspiration value for the objective function fi(i=1,...,k). •Selecting p(with p≥1) solutions as the most preferred ones among a set of solutions. Let us denote them by PS1,...,PSp. •Selecting np (with np ≥1) solutions as the most nonpreferred (unacceptable) ones among a set of solutions. Let us denote them by NPS1,...,NPSnp. •Specifying preferred ranges with desirable values for the objective functions. We denote by [fl i,fu i]the preferred range for the objective function fi(i=1,...,k). As a result, the preferences are determined by a k-dimensional hyper-box [fl 1,fu 1]×···×[fl k,fu k]in the objective space. •Performing pairwise comparisons of solutions, i.e., given two solutions, the DM decides which one satisfies her/him the most. In the literature, there exists a plethora of interactive methods for solving multiobjective optimization problems (surveyed, e.g., in [4,18–20,29]). As mentioned in the introduction, the solution process with interactive methods can often be observed to have two phases aimed at different purposes [21]. First, in the learning phase, the DM explores the problem to learn about the conflict degree among the 123 Complex & Intelligent Systems objectives and what kind of solutions are feasible reflecting different preferences. At the end of this phase, an ROI is identified according to the DM’s desires. Second, in the decision phase, (s)he further explores this ROI by progressively fine-tuning her/his preferences to finally converge to her/his MPS. Artificial decision maker for interactive EMO In this section, we describe the new ADM-II for comparing the performance of interactive EMO methods. As mentioned, ADM-II can handle various types of preference information such as providing a reference point, selecting either the most preferred or the most non-preferred solution(s) among a set of alternative solutions, specifying desirable objective function ranges, or performing pairwise comparisons. To compare the methods in a meaningful way, all of them are run simultaneously using the same computational resources (i.e., number of function evaluations or generations per iteration). If all of the interactive methods being compared use the same preference type, ADM-II produces the preference information in the same way for all methods. In case methods utilizing different types of preferences are compared, ADM-II generates the preferences accordingly and produces the type of preference information each method expects, but the philosophy underlying the generating procedure for the different types is similar. Internally, our ADM-II follows a different strategy to generate the preferences at the iterations of each of the two phases of the interactive solution process. In the learning phase, it simulates an exploratory search in the objective space to inspect possible solutions. To this aim, the preference information for each iteration of this phase is generated in a way that the search is oriented toward the least explored region of the PF. At the last iteration of this phase, an ROI is found. In the decision phase, the behavior of ADM-II pursues a finer search within this ROI to find an MPS. Thus, at each iteration of this phase, the preferences produced by ADM-II are generated within the ROI to refine the solutions in it. The number of iterations carried out in each phase is set at the beginning. In what follows, we refer to the number of iterations in the learning and the decision phases by Land D, respectively. They are parameters of ADM-II. To generate new preferences depending on the responses of all the methods, ADM-II makes use of the solutions found so far by all methods included in the comparison. At each iteration, the solutions generated by the methods are combined, and a composite front is formed by deleting the dominated ones, as can be seen in Fig. 1a. To be more specific, at each iteration, the composite front is constituted by the nondominated solutions generated so far by all the methods. It is important to note that no information about the true PF is required to generate new preferences. The generation of preferences in ADM-II is based on dividing the objective space into sub-areas. This is done following the philosophy of the so-called decomposition-based EMO methods (like [5]), although any other procedure allowing us to have information about sub-areas of the PF can also be used in ADM-II. EMO methods of this type usually decompose the original problem into several sub-problems. We use this idea to make a distinction between sub-areas of the PF that have already been explored more or less (i.e., the exploration degree of the different parts of the PF). Based on this, we decide how to generate the preferences at each iteration to guide the search for new nondominated solutions toward a specific part of the PF. Let us describe the algorithm designed to divide the composite front into several sub-areas to identify the regions to be explored at each iteration. Initially, ADM-II creates a set of reference vectors uniformly distributed along the PF. To do this, the canonical simplex-lattice design method [7] is used, as suggested in [5,6]. In this method, the number of reference vectors that are created is controlled by a lattice resolution, which is given by l+k−1 k−1, where lis a pre-fixed parameter. Subsequently, at each iteration, ADM-II calculates the angles between each solution in the composite front and each reference vector. Then, each solution is assigned to the reference vector with the smallest angle. Figure 1b depicts an example for a bi-objective problem, where two reference vectors (V1and V2) and three solutions (S1,S2and S3) are shown. Solution S1is assigned to V1, given that the angle between S1and V1(denoted by β) is smaller than the angle between S1and V2(denoted by α). In the same way, S2is assigned to V1and S3is assigned to V2. Once all solutions in the composite front have been assigned to the reference vectors, ADM-II gets information about the exploration degree of each region of the PF at the current iteration by counting the number of assigned solutions to each reference vector. The more solutions assigned to a reference vector, the best explored the sub-area is, where the reference vector lies. This information is conveniently used in ADM-II to generate new preferences at each iteration of the learning and decision phases depending on the needs, as described in “Preference generation in the learning and decision phases”. ADM-II also considers how the interactive EMO methods compared reflect the preference information in the solutions they produce. As described in “Performance evaluation”, it employs performance indicators that internally consider the preferences used along the iterations. Furthermore, cumulative indicator values are also computed separately in the learning and decision phases, since they allow us to evaluate the performances of the methods in both phases. In addition, we propose a new performance indicator, called contribution 123 Complex & Intelligent Systems Fig. 1 Division of the PF internally performed in ADM-II to CF, which is obtained as the number of nondominated solutions each method has contributed to build/extend the composite front along the iterations. Algorithm 1contains the main steps of ADM-II. Algorithm 1 Main steps of ADM-II Step 0: Initialize all methods and provide the first preferences randomly. Step 1: Run all methods with the same computational budget (number of generations or function evaluations) and the previously generated preferences. Step 2: Build or update the composite front using the solutions obtained by each method until this iteration. Step 3: Evaluate the methods’ performances taking into account the preferences used. Step 4: Generate new preferences for the next iteration based on the composite front and phase (learning or decision) of the iteration being performed, according to the strategy designed for each preference type: a) In the learning phase, generate the new preferences for the least explored area of the composite front. At the end of the learning phase, identify the best explored area as the ROI. b) In the decision phase, generate the new preferences within the ROI identified at the end of the learning phase. Step 5: If a termination criterion is met, terminate the process and calculate cumulative indicator values for each phase. Else, continue with Step 1. Preference generation in the learning and decision phases As previously mentioned, in the learning phase, the main purpose of ADM-II is to explore the whole set of Pareto optimal solutions. Therefore, it progressively inspects the sub-areas of the PF that have been poorly covered so far. At each iteration of this phase, a set of uniformly distributed reference vectors on the composite front is first obtained, and the solutions of the composite front are assigned to reference vectors, as described before. Then, the least explored area of the composite front is determined based on the reference vector that has the lowest number of assigned solutions. ADM-II generates preferences for the next iteration to direct the search for new nondominated solutions toward this region. In the example of Fig. 2a, V2would be selected as the vector defining the least explored area. It should be noted that ADM-II does not consider the vectors with no solutions assigned. This prevents algorithms from being compelled to search for solutions in areas where no solutions exist in the true PF. Once the iterations corresponding to the learning phase have been completed (until iteration L), ADM-II finds the reference vector with the highest number of assigned solutions. Let us refer to this vector as VD. The part of the PF where the vector VDis located can be assumed to be the best explored area of the composite front. Then, the ROI to be explored in detail at the next Diterations (corresponding to the decision phase) is formed by the solutions assigned to VD, and the preference information is obtained at each iteration based on this vector. As one can note, the way of generating the preference information in both phases depends on the reference vector selected. In what follows, we describe the strategy designed to generate different types of preference information for the two phases. Giving a reference point In the learning phase, the reference vector selected at each iteration (identifying the least explored area) and the solu- 123 Complex & Intelligent Systems Fig. 2 Reference point as preference information in ADM-II tions assigned to this vector are used to find the location of the new reference point. For this, the distances of these solutions to the ideal point of the current composite front are calculated, and the one with the minimum distance (denoted by |d|) is identified. The next reference point is then located on the selected reference vector according to this distance |d|. Figure 2a graphically shows how the new reference point is generated, which is denoted by q. At the iterations of the decision phase, reference points are generated to progressively converge toward the PF by performing a finer search in the ROI identified at the end of the learning phase to find an MPS. ADM-II finds the solution assigned to the reference vector VDwith the minimum distance |d|to the ideal point of the composite front. Then, |d| allows us to generate the next reference point, as shown in Fig. 2b. First, we obtain the point labeled as ¯ qalong the reference vector VDusing |d|, and then, the new reference point qis generated by applying a perturbation to each component of this point. For the perturbation, the distance |¯ d|between ¯ qand the nearest solution to ¯ qfrom the composite front is calculated, and then, each component of qis obtained as the corresponding component of ¯ qminus |¯ d|. This way of producing reference points in the decision phase assures that the new reference point always dominates the previously generated points. In practice, this means that the reference points generated in this phase get progressively closer to the ideal point as the iterations are performed, since the solutions provided by the methods, which are used to update the composite front, progressively converge to the PF. It is noteworthy that the way of generating reference points in the learning phase is similar to the procedure designed in the previous ADM [1]. Nevertheless, in the decision phase, the behavior of ADM-II for producing new reference points is totally different from [1] (where the reference points generated in this phase always lie on the reference vector VDof the ROI being explored). Selecting the preferred solution(s) At each iteration of the learning phase, the most preferred solution(s) are selected among the solutions assigned to the reference vector of the least explored area. First, the distance of each solution to the ideal point of the current composite front is calculated. Then, all the assigned solutions are ranked in descending order according to their distances to the ideal point, and the first psolutions are selected as the most preferred solutions PS1,...,PSp. It may happen that ADM-II has to find a higher number of preferred solutions than the number of assigned solutions to the selected reference vector. In this case, the reference vector of the second least explored area is found, and ADM-II selects the necessary number of preferred solutions from the solutions assigned to this vector, in a similar way, based on their distances to the ideal point. In the decision phase, ADM-II selects the most preferred solutions in a similar way, considering the solutions assigned to the reference vector VD(which represents the best explored area at the end of the learning phase). If the number of assigned solutions to VDis lower than the required number of preferred solutions, the remaining preferred solutions are the closest ones to VDbased on the angle values, even if these solutions are assigned to other reference vectors. Since the purpose of ADM-II is to refine the search for solutions in the ROI defined by VD, we provide the preference information within this ROI and near to it if needed. This preference generation in the learning phase is exemplified in Fig. 3a, where V3is the vector representing the least 123 Complex & Intelligent Systems Fig. 3 Selecting the pmost preferred solutions as preference information in ADM-II explored area (since it has the least number of assigned solutions). If only one preferred solution is required by a method, PS1is the one selected given that it is the closest one to the ideal point. If a method expects two preferred solutions, then PS1and PS2are selected. If, e.g., four preferred solutions have to be selected, as V3has only two assigned solutions, the reference vector V1is found as the second least explored area. Then, the remaining preferred solutions are chosen among the ones assigned to V1based on their distances to the ideal point. In this case, besides PS1and PS2, solutions PS3and PS4are chosen as preferred solutions. The behavior in the decision phase is shown in Fig. 3b. If a method expects, e.g., four preferred solutions, ADM-II selects the three solutions assigned to the reference vector of the best explored area (V2in this case), and the solution PS4 (even though it is assigned to V1), because it is the one with the smallest angle to V2. Selecting the non-preferred solution(s) For finding non-preferred solutions, we follow a similar procedure to the previous one, but select the most unwanted solutions, so that the methods would not converge to the regions where unwanted solutions are. At each iteration of the learning phase, ADM-II finds the reference vector of the best explored area (with the highest number of assigned solutions). Then, the non-preferred solutions are found among the solutions assigned to this vector. In this way, in the learning phase, ADM-II avoids producing more nondominated solutions in the best approximated area, so that the regions with fewer solutions are emphasized. On the other hand, at each iteration in the decision phase, ADM-II selects as the non-preferred ones the solutions with the largest angles to the reference vector VDrepresenting the ROI that is being explored. This means that ADM-II avoids selecting the solutions outside the ROI or the furthest ones from VD. Figure 4a represents a case where a method expects the two most non-preferred solutions in the learning phase. As shown, NPS1and NPS2are selected among the solutions assigned to the vector V2associated with the best explored area. With this, ADM-II avoids getting more solutions around V2by selecting NPS1and NPS2as non-preferred solutions, because the region where V2is has already been explored. In this way, ADM-II seeks to search for solutions from the least explored areas, which is the purpose of the learning phase. In Fig. 4b, the solutions selected in the decision phase are shown. In this case, V2is the vector that represents the ROI. NPS1and NPS2are the furthest solutions from V2, based on the angles to this vector, and ADM-II chooses them as the two most non-preferred solutions. Specifying preferred ranges To generate preferred ranges for objective functions, first, ADM-II generates a reference point qas indicated in “Giving a reference point”, depending on the phase in question. Then, the ranges are calculated at each iteration by perturbing the components of qusing the distance |¯ d|of the nearest solution of the composite front to q. That is, for every i=1,...,k, the desirable range for the objective function fiis defined as [qi−|¯ d|,qi+|¯ d|], where qiis the component iof q.This is illustrated in Fig. 5. 123 Complex & Intelligent Systems Fig. 4 Selecting the np most non-preferred solutions as preference information in ADM-II Fig. 5 Specifying desirable objective function ranges as preference information in ADM-II Performing pairwise comparisons For pairwise comparisons, ADM-II compares the solutions generated by each method in the following way. At each iteration in the learning phase, the solution that is finally chosen is the one with the minimum angle to the reference vector of the least explored area. On the other hand, at each iteration in the decision phase, from the two ones provided by the method, the solution with the minimum distance to the ideal point of the composite front is chosen. Figure 6a illustrates the comparison of solutions in the learning phase, where solution S1is selected rather than S2, since it has the smallest angle to V3(which represents the least explored area). In Fig. 6b, we show the behavior in the decision phase. Between S1and S2, ADM-II selects S2, since it is closer to the ideal point. Performance evaluation To evaluate the performance of methods aimed at generating solutions reflecting preferences, it is not enough to quantify the convergence (closeness to the PF) and diversity (spread over the PF and uniformity among solutions) among the nondominated solutions obtained. Basically, this is the information that is assessed by commonly used performance indicators of (a posteriori) EMO methods [15]. However, when preferences are considered, the performance should also be assessed regarding the ROI defined by the preferences. Moreover, ideally, aspects in relation to the interaction with the DM should also be considered to evaluate the performance of interactive methods. Indeed, besides evaluating how well each method obeys the preferences (i.e., the method’s ability to generate solutions reflecting the different preferences), the quality of the solutions generated should be measured differently in the two phases of the solution process, since they have different goals. In the learning phase, one should measure how well each method responses to the given preference information at different parts of the PF; and in the decision phase, how well the method converges when exploring solutions within a specific ROI. In the literature, we can find performance indicators for EMO methods that incorporate preferences given a priori (before the solution process starts) [12,14,23,30]. However, to the best of our knowledge, quality indicators for measuring the quality of the solutions found by interactive methods 123 Complex & Intelligent Systems Table 3 continued Problem kPhase R-IGD Contribution to CF FEs iRVEA iNSGAIII iRVEA iNSGAIII iRVEA iNSGAIII Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. DTLZ4 3 Learning 0.105 0.029 0.070 0.025 157 31 421 2 107,486 4066 127,200 0 Decision 0.085 0.047 0.077 0.040 222 29 307 6 89,427 3968 95,400 0 4 Learning 0.255 0.128 0.171 0.179 282 11 482 1 169,013 6052 192,000 0 Decision 0.636 0.437 0.617 0.430 336 8 356 2 142,216 2098 144,000 0 5 Learning 0.747 0.748 0.536 0.358 425 83 505 3 224,266 38,507 252,000 0 Decision 0.627 0.404 0.650 0.445 361 45 373 2 186,999 22,702 189,000 0 6 Learning 0.523 0.199 0.471 0.247 484 15 504 2 280,748 3146 302,400 0 Decision 0.554 0.462 0.519 0.460 385 2 376 3 230,480 1674 226,800 0 7 Learning 1.267 0.889 0.701 0.151 297 63 330 13 172,607 38,083 235,200 0 Decision 0.747 0.350 0.645 0.354 244 38 248 5 168,479 30,595 176,400 0 8 Learning 2.260 1.383 0.665 0.089 393 92 483 1 259,205 74,122 384,000 0 Decision 0.842 0.453 0.694 0.476 380 1 360 1 281,918 10,469 288,000 0 9 Learning 0.800 0.254 0.708 0.262 685 8 663 1 537,059 29,818 597,600 0 Decision 1.118 0.325 1.093 0.358 518 1 496 2 438,909 14,570 448,200 0 Table 4 iRVEA-RP vs. iRVEA-Ranges: numerical results with 100 generations per iteration for the objectives ranging from 3 to 9 Problem kPhase R-IGD ContributiontoCF FEs iRVEA-RP iRVEA-Ranges iRVEA-RP iRVEA-Ranges iRVEA-RP iRVEA-Ranges Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean std. DTLZ1 3 Learning 2.473 0.106 2.604 0.196 150 69 34 52 102,132 11,138 72,373 31,324 Decision 1.859 0.216 1.873 0.203 292 48 64 46 91,521 2117 95,012 20,800 4 Learning 2.762 0.096 2.896 0.152 210 114 177 108 157,746 6878 159,461 24,309 Decision 1.787 0.095 1.775 0.104 349 22 275 126 134,653 6757 167,615 3311 5 Learning 2.780 0.117 2.977 0.180 302 86 234 135 214,526 23,343 225,120 20,445 Decision 2.106 0.241 2.103 0.239 358 59 291 148 189,002 5388 212,499 46,312 6 Learning 2.805 0.127 3.170 0.154 331 75 376 173 262,012 9682 288,029 32,166 Decision 1.959 0.332 1.961 0.291 384 17 352 155 220,375 7395 247,406 96,321 7 Learning 2.916 0.225 3.505 0.336 211 107 312 134 216,973 5618 256,392 21,765 Decision 2.035 0.375 2.025 0.336 200 57 185 142 180,099 5427 187,227 108,020 8 Learning 2.964 0.163 3.627 0.412 347 112 350 244 337,488 26,067 388,893 39,496 Decision 1.953 0.342 2.011 0.277 348 62 203 206 287,031 8905 220,664 189,815 9 Learning 2.835 0.150 3.637 0.333 577 85 570 238 524,079 31,435 594,523 46099 Decision 1.983 0.407 1.980 0.367 514 3 449 253 452,580 10,169 460,364 241,346 DTLZ2 3 Learning 0.195 0.207 0.362 0.195 204 56 62 62 89747 17,216 24,349 22,056 Decision 0.319 0.481 0.881 0.597 192 79 14 5 70,330 28,593 15,228 21,819 4 Learning 0.564 0.421 0.710 0.476 257 99 208 97 100,852 34,106 102,171 43,516 Decision 0.997 0.792 0.879 0.838 131 135 59 59 62,446 61,676 59,605 69,507 5 Learning 0.280 0.190 0.560 0.148 396 59 301 117 169,164 30,901 140,704 66,235 Decision 0.530 0.517 0.751 0.614 309 124 126 96 157,756 61,041 153,731 81,323 6 Learning 0.529 0.337 0.566 0.090 394 118 372 118 188971 62,602 205,655 71,323 Decision 1.689 1.849 1.414 1.583 269 162 242 129 159,317 94,462 195,966 100,939 123 Complex & Intelligent Systems Table 4 continued Problem kPhase R-IGD ContributiontoCF FEs iRVEA-RP iRVEA-Ranges iRVEA-RP iRVEA-Ranges iRVEA-RP iRVEA-Ranges Mean Std. Mean Std. Mean Std. Mean Std. Mean Std. Mean std. 7 Learning 0.486 0.216 0.940 0.343 248 62 276 87 136,385 37,755 184,031 45,542 Decision 1.155 1.565 0.897 1.369 177 109 166 99 122,086 73,071 177,301 93,222 8 Learning 0.476 0.267 0.784 0.285 362 55 335 142 209,208 36,673 224,348 104,465 Decision 2.458 2.049 0.969 0.424 182 140 385 180 127,167 99,390 322,144 108,602 9 Learning 0.528 0.274 0.908 0.213 510 142 423 221 341,056 112,124 389,291 180654 Decision 2.909 2.381 1.562 1.444 222 215 397 231 172,533 162,562 358,663 204,173 DTLZ3 3 Learning 0.188 0.237 0.649 0.440 88 44 16 14 91,701 9474 53,486 27128 Decision 0.213 0.305 0.172 0.175 175 102 27 31 71,843 33,198 61,302 41,075 4 Learning 0.234 0.147 0.651 0.338 174 44 44 44 148,334 23,573 110,230 56,902 Decision 0.427 0.643 0.570 0.611 249 134 25 38 107,189 47,229 71,661 70,361 5 Learning 0.464 0.174 1.094 0.667 175 84 155 156 185,903 28,649 140,813 77,061 Decision 0.223 0.291 0.533 0.402 283 113 94 128 157,741 49,214 81,671 93,460 6 Learning 0.707 0.451 1.079 0.273 224 109 157 118 205,280 44,914 209,504 56,752 Decision 0.272 0.120 0.432 0.409 293 85 117 106 171,301 37,604 192,897 114,324 7 Learning 0.525 0.416 1.461 0.447 190 62 233 137 182,259 31,951 225,480 41,383 Decision 0.575 0.969 0.379 0.293 185 74 173 117 158,003 37,831 205,151 49,427 8 Learning 0.732 0.418 1.548 0.675 323 51 260 189 288,271 35,718 354,430 79,190 Decision 0.710 0.873 0.482 0.773 190 154 180 104 154,163 123,620 272,603 125,257 9 Learning 0.369 0.093 1.548 0.587 503 83 374 182 470,856 52,312 466,521 80,068 Decision 0.658 0.544 0.330 0.248 323 217 296 150 275,755 184,561 471,451 173,463 DTLZ4 3 Learning 0.065 0.035 0.634 0.723 258 46 99 97 107,643 2929 50,189 28,180 Decision 0.097 0.052 0.157 0.081 251 19 80 34 90,690 1392 88,773 6430 4 Learning 0.182 0.163 0.358 0.111 380 52 263 169 169,891 7496 116,161 56,912 Decision 0.268 0.342 0.313 0.364 318 13 127 40 141,123 3540 139,299 15,072 5 Learning 0.777 1.239 0.834 0.699 433 109 410 129 212,099 57,517 208,734 63783 Decision 0.577 0.626 0.628 0.537 332 78 134 69 174,246 41,674 146,119 52,942 6 Learning 0.552 0.169 1.101 0.667 500 12 511 97 286,270 13,417 291,382 57,995 Decision 0.657 0.423 0.635 0.489 381 8 319 96 226,506 7408 259,833 11,953 7 Learning 2.400 2.691 2.989 2.526 212 128 247 141 133,483 82,945 158,791 95112 Decision 1.469 1.458 1.875 2.312 168 94 241 117 119,533 65,213 199,386 92,390 8 Learning 1.446 2.097 1.561 1.054 457 141 561 85 343,166 103,225 423,696 70,082 Decision 0.885 0.359 0.664 0.518 368 32 407 113 281,463 27,964 353,865 69,693 9 Learning 0.931 0.323 1.618 0.467 684 6 724 148 570,285 11,818 642,717 70,788 Decision 0.743 0.346 0.539 0.509 515 3 462 155 446,762 13,336 510,922 59,213 References 1. Afsar B, Miettinen K, Ruiz AB (2021) An artificial decision maker for comparing reference point based interactive evolutionary multiobjective optimization methods. In: Ishibuchi H, Zhang Q, Cheng R, Li K, Li H, Wang H, Zhou A (eds) Evolutionary multi-criterion optimization, 11th international conference, EMO 2021, Proceedings. Springer, pp 619–631 2. Afsar B, Miettinen K, Ruiz F (2021) Assessing the performance of interactive multiobjective optimization methods: a survey. ACM Comput Surv 54(4):85 3. Barba-González C, Ojalehto V, García-Nieto J.M, Nebro AJ, Miettinen K, Aldana-Montes JF (2018) Artificial decision maker driven by PSO: an approach for testing reference point based interactive methods. In: Auger A, Fonseca CM, Lourenço N, Machado P, Paquete L, Whitley D (eds) Parallel problem solving from nature—PPSN XV, 15th international conference, Proceedings, Part I. Springer, pp 274–285 4. Branke J, Deb K, Miettinen K, Slowinski R (eds) (2008) Multiobjective optimization. Interactive and evolutionary approaches. Springer, Berlin 123 Complex & Intelligent Systems 5. Cheng R, Jin Y, Olhofer M, Sendhoff B (2016) A reference vector guided evolutionary algorithm for many-objective optimization. IEEE Trans Evol Comput 20(5):773–791 6. Chugh T, Jin Y, Miettinen K, Hakanen J, Sindhya K (2018) A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization. IEEE Trans Evol Comput 22(1):129–142 7. Cornell JA (2011) Experiments with mixtures: designs, models, and the analysis of mixture data. Wiley, New York 8. Deb K, Jain H (2013) An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Trans Evol Comput 18(4):577–601 9. Deb K, Miettinen K, Chaudhuri S (2010) Towards an estimation of nadir objective vector using a hybrid of evolutionary and local search approaches. IEEE Trans Evol Comput 14(6):821–841 10. Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multiobjective optimization test problems. In: 2002 congress on evolutionary computation, Proceedings, pp 825–830 11. Hakanen J, Chugh T, Sindhya K, Jin Y, Miettinen K (2016) Connections of reference vectors and different types of preference information in interactive multiobjective evolutionary algorithms. In: 2016 IEEE symposium series on computational intelligence, Proceedings. IEEE, pp 1–8 12. Hou Z, Yang S, Zou J, Zheng J, Yu G, Ruan G (2018) A performance indicator for reference-point-based multiobjective evolutionary optimization. In: 2018 IEEE symposium series on computational intelligence, Proceedings. IEEE, pp 1571–1578 13. Huber S, Geiger MJ, Sevaux M (2015) Simulation of preference information in an interactive reference point-based method for the bi-objective inventory routing problem. J Multi-Criteria Decis Anal 22(1–2):17–35 14. Li K, Deb K, Yao X (2018) R-metric: evaluating the performance of preference-based evolutionary multiobjective optimization using reference points. IEEE Trans Evol Comput 22(6):821–835 15. Li M, Yao X (2019) Quality evaluation of solution sets in multiobjective optimisation: a survey. ACM Comput Surv 52(2):26 16. López-Ibánez M, Knowles J (2015) Machine decision makers as a laboratory for interactive EMO. In: Gaspar-Cunha A, Henggeler- Antunes C, Coello CC (eds) Evolutionary multi-criterion optimization, 8th international conference, Proceedings, Part II. Springer, pp 295–309 17. Luque M, Ruiz F, Miettinen K (2011) Global formulation for interactive multiobjective optimization. OR Spectrum 33(1):27–48 18. Meignan D, Knust S, Frayret JM, Pesant G, Gaud N (2015) A review and taxonomy of interactive optimization methods in operations research. ACM Trans Interact Intell Syst 5(3):171–1743 19. Miettinen K (1999) Nonlinear multiobjective optimization. Kluwer Academic Publishers, Boston 20. Miettinen K, Hakanen J, Podkopaev D (2016) Interactive nonlinear multiobjective optimization methods. In: Greco S, Ehrgott M, Figueira J (eds) Multiple criteria decision analysis: state of the art surveys, 2 edn. Springer, pp 931–980 21. Miettinen K, Ruiz F, Wierzbicki AP (2008) Introduction to multiobjective optimization: interactive approaches. In: Branke J, Deb K, Miettinen K, Słowi´nski R (eds) Multiobjective optimization: interactive and evolutionary approaches. Springer, Berlin, pp 27– 57 22. Misitano G, Saini BS, Afsar B, Shavazipour B, Miettinen K (2021) DESDEO: The modular and open source framework for interactive multiobjective optimization. IEEE Access 9:148277–148295 23. Mohammadi A, Omidvar MN, Li X(2013) A new performance metric for user-preference based multi-objective evolutionary algorithms. In: 2013 IEEE congress on evolutionary computation, Proceedings. IEEE, pp 2825–2832 24. Ojalehto V, Podkopaev D, Miettinen K (2016) Towards automatic testing of reference point based interactive methods. In: Handl J, Hart E, Lewis PR, López-Ibánez M, Ochoa G, Paechter B (eds) Parallel problem solving from nature—PPSN XIV, 14th international conference, Proceedings. Springer, pp 483–492 25. Ruiz F, Luque M, Miettinen K (2012) Improving the computational efficiency in a global formulation (GLIDE) for interactive multiobjective optimization. Ann Oper Res 197(1):47–70 26. Steuer RE (1986) Multiple criteria optimization: theory, computation and application. Wiley, New York 27. Szczepanski M, Wierzbicki AP (2003) Application of multiple criteria evolutionary algorithm to vector optimization, decision support and reference-point approaches. J Telecommun Inf Technol 3(3):16–33 28. Wierzbicki AP (1980) The use of reference objectives in multiobjective optimization. In: Fandel G, Gal T (eds) Multiple criteria decision making, theory and applications. Springer, Berlin, pp 468– 486 29. Xin B, Chen L, Chen J, Ishibuchi H, Hirota K, Liu B (2018) Interactive multiobjective optimization: a review of the state-of-the-art. IEEE Access 6:41256–41279 30. Yu G, Zheng J, Li X (2015) An improved performance metric for multiobjective evolutionary algorithms with user preferences. In: 2015 IEEE congress on evolutionary computation, Proceedings. IEEE, pp 908–915 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 123