Supplementary material for TELO Paper "Anytime Benchmarking of Forty-Five PAES-25 Configurations on Multi- and Many-Objective Variants of Leading-Ones-Trailing-Zeros Functions"
Abstract
Content Supplementary materials (detailed results and figures) Reference Anonymous. Anytime Benchmarking of Forty-Five PAES-25 Configurations on Multi- and Many-Objective Variants of Leading-Ones-Trailing-Zeros Functions. In ACM Transactions on Evolutionary Learning and Optimization (under review) Abstract The Leading Ones Trailing Zeros (LOTZ) benchmark is a well-established bi-objective pseudo-Boolean function (of scalable size) that has both a linear-sized Pareto front and also a front that is "far away" from a random solution. As such, it is a useful function for understanding the behaviour of multi-objective optimization algorithms, especially regarding their ability to approach the front and expand and maintain solutions all along it. It has contributed to both algorithm design and theoretical analysis. In this paper, we make use of three variants of LOTZ, recently introduced, which generalize it to the many-objective case in different ways while aiming to maintain similar properties of the original. We add further detail to the mathematical description of the benchmarks and use them to analyse the performance of a simple multi-objective local search algorithm (PAES-25) (based on (1+1)-PAES) with plug-in modules for archiving, neighbourhood, selection method (from the archive or not), and acceptance function. We measure the anytime performance of PAES-25 in different variants by observing a suite of online measures that give insight into the dynamics of the algorithm and provide guidance as to choice of configuration depending on factors such as time available and size of Pareto approximation set desired. We believe that the benchmarks and the anytime, multidimensional methods used for analysis might motivate further theory (especially of many-objective optimization and archiving) and may also contribute usefully to algorithm design ideas.
Full text
Benchmarking a Modular Archive-Based Local Search Algorithm on Multiand Many-Objective Variants of Leading-Ones-Trailing-Zeros Supplementary material ANONYMOUS AUTHOR(S) This document contains supplementary material to the main paper. It reports detailed and additional experimental results obtained with several alternative variants of PAES-25. The experiments were conducted on the following benchmark functions: •LOTZ (Leading Ones Trailing Zeros) with 2objectives, •LITZ (Leading Integers Trailing Zeros) with 4and 8objectives, •sLITZ (shuffled LITZ) with 4and 8objectives, •FRITZ (First Run of Integers Trailing Zeros) with 4and 8objectives. Results are reported for two problem dimensions: 𝑛=24 and 𝑛=96 variables. The following statistics are reported — all being plotted over time: •the proportion of Pareto-optimal points in the current archive, •the cumulative proportion of encountered Pareto-optimal points, • the relative deviation of the current archive from the PF in terms of hypervolume (lower is better, with 0 indicating the PF is reached), •the relative deviation of cumulative solutions from the PF in terms of hypervolume, •the archive size, •the sum of objectives from the current solution, •the number of accepted moves, •the last accepted move. Each plot reports mean and 95% confidence intervals from 30 runs under different algorithm configurations.
2 Anon. 1 RESULTS FOR LOTZ WITH 𝑛=24 VARIABLES AND 𝑚=2OBJECTIVES UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 3 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations cumulative relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2
4 Anon. UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1 3 10 1 3 10 1 3 10 1 3 10 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0 5 10 15 20 25 0 5 10 15 20 25 0 5 10 15 20 25 0 5 10 15 20 25 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 5 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 24 m = 2
6 Anon. 2 RESULTS FOR LITZ WITH 𝑛=24 VARIABLES AND 𝑚=4OBJECTIVES UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 7 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations cumulative relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4
8 Anon. UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1 3 10 30 1 3 10 30 1 3 10 1 3 10 30 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 2.5 5.0 7.5 10.0 12.5 2.5 5.0 7.5 10.0 12.5 2.5 5.0 7.5 10.0 12.5 2.5 5.0 7.5 10.0 12.5 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 9 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 4
16 Anon. UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1 10 100 1 3 10 30 1 3 10 1 10 100 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 24 m = 4 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 4 6 8 10 12 4 6 8 10 12 5.0 7.5 10.0 12.5 4 6 8 10 12 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 24 m = 4
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 17 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 24 m = 4 UNB MGA (20) MGA (50) MGA (100) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 24 m = 4
18 Anon. 5 RESULTS FOR LITZ WITH 𝑛=24 VARIABLES AND 𝑚=8OBJECTIVES UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 19 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 evaluations cumulative relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8
20 Anon. UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1 3 10 30 1 3 10 30 1 3 10 30 1 3 10 30 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 2 4 6 8 2 4 6 8 2 4 6 8 2 4 6 8 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 21 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 24 m = 8
22 Anon. 6 RESULTS FOR SLITZ WITH 𝑛=24 VARIABLES AND 𝑚=8OBJECTIVES UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 23 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.25 0.50 0.75 1.00 0.2 0.4 0.6 0.8 1.0 0.4 0.6 0.8 1.0 0.25 0.50 0.75 1.00 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 evaluations cumulative relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8
24 Anon. UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1 10 100 1 10 100 1 3 10 30 1 10 100 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 2 4 6 8 2 4 6 8 2 4 6 8 2 4 6 8 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 25 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8 UNB MGA (50) MGA (100) MGA (200) 2025210 215 220 2025210 215 220 2025210 215 220 2025210 215 220 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 1e+01 1e+03 1e+05 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 24 m = 8
32 Anon. MGA (200) HVA (50) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+02 1e+03 1e+04 1e+05 1e+06 1e+02 1e+03 1e+04 1e+05 1e+06 1e+02 1e+03 1e+04 1e+05 1e+06 1e+02 1e+03 1e+04 1e+05 1e+06 1e+02 1e+03 1e+04 1e+05 1e+06 1e+02 1e+03 1e+04 1e+05 1e+06 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 96 m = 2 MGA (200) HVA (50) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LOTZ n = 96 m = 2
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 33 9 RESULTS FOR LITZ WITH 𝑛=96 VARIABLES AND 𝑚=4OBJECTIVES MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.2 0.4 0.6 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4
34 Anon. MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.4 0.6 0.8 1.0 0.25 0.50 0.75 1.00 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 evaluations cumulative relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 35 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 3 10 30 100 3 10 30 100 3 10 30 3 10 30 100 3 5 10 3 10 30 100 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4
36 Anon. MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 4
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 37 10 RESULTS FOR SLITZ WITH 𝑛=96 VARIABLES AND 𝑚=4OBJECTIVES MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.8 0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.2 0.4 0.6 0.0 0.1 0.2 0.3 0.0 0.2 0.4 0.6 0.8 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4
38 Anon. MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.2 0.4 0.6 0.8 1.0 0.25 0.50 0.75 1.00 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00 evaluations cumulative relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 39 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1 10 100 1 10 100 1 3 10 30 1 10 100 1 3 10 1 10 100 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4
40 Anon. MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 4
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 41 11 RESULTS FOR FRITZ WITH 𝑛=96 VARIABLES AND 𝑚=4OBJECTIVES MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 4 MGA (200) HVA (100) UNB MGA (20) MGA (50) MGA (100) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.00 0.02 0.04 0.0 0.2 0.4 0.6 0.00 0.01 0.02 0.03 0.00 0.05 0.10 0.15 0.000 0.005 0.010 0.015 0.020 0.00 0.02 0.04 0.06 0.08 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 4
48 Anon. MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 8 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive LITZ n = 96 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 49 13 RESULTS FOR SLITZ WITH 𝑛=96 VARIABLES AND 𝑚=8OBJECTIVES MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 8 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.00 0.05 0.10 0.15 0.00 0.04 0.08 0.12 0.0 0.1 0.2 0.3 0.000 0.025 0.050 0.075 0.0 0.1 0.2 0.3 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 8
50 Anon. MGA (500) MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.4 0.6 0.8 1.0 0.4 0.6 0.8 1.0 0.4 0.6 0.8 1.0 0.2 0.4 0.6 0.8 1.0 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 8 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1 10 100 1 10 100 1 10 100 1000 1 3 10 30 1 10 100 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 51 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 10 15 20 25 30 10 20 30 10 15 20 25 30 10 20 30 10 15 20 25 30 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 8 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 8
52 Anon. MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive sLITZ n = 96 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 53 14 RESULTS FOR FRITZ WITH 𝑛=96 VARIABLES AND 𝑚=8OBJECTIVES MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.0 0.2 0.4 0.6 0.8 0.00 0.25 0.50 0.75 0.0 0.2 0.4 0.6 0.00 0.25 0.50 0.75 0.0 0.2 0.4 0.6 evaluations pareto−optimal points in current archive 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 8 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.00000 0.00005 0.00010 0.00015 0.00000 0.00005 0.00010 0.00015 0e+00 1e−04 2e−04 3e−04 4e−04 0.0e+00 2.5e−05 5.0e−05 7.5e−05 1.0e−04 0e+00 1e−04 2e−04 3e−04 evaluations cumulative pareto−optimal points seen 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 8
54 Anon. MGA (500) MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 0.5 0.7 0.9 0.5 0.6 0.7 0.8 0.9 1.0 0.7 0.8 0.9 1.0 0.25 0.50 0.75 1.00 evaluations current relative hypervolume deviation 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 8 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 3 10 30 100 3 10 30 100 10 100 1000 10000 3 10 30 3 10 30 100 300 evaluations current archive size 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 8
Benchmarking PAES-25 on Variants of Leading-Ones-Trailing-Zeros — Supplementary material 55 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 15 20 25 30 15 20 25 30 15 20 25 30 15 20 25 30 15 20 25 30 evaluations current solution's sum of objectives 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 8 MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 1e+02 1e+04 1e+06 evaluations number of accepted moves 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 8
56 Anon. MGA (500) UNB MGA (50) MGA (100) MGA (200) 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 210 212 214 216 218 220 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 1e+03 1e+04 1e+05 1e+06 evaluations last move 1−flip last−strict 1−flip last−neutral 1−flip from−archive rate=1/n last−strict rate=1/n last−neutral rate=1/n from−archive fast last−strict fast last−neutral fast from−archive FRITZ n = 96 m = 8