Section: New Results

ML-assisted optimization

As pointed out in our research program 3.2, we investigate the ML-assisted optimization following two directions: building efficiently surrogates to deal with expensive black-box objective functions and automatically building and predicting/improving the performance of metaheuristics through landscape/problem structure analysis. Regarding surrogate-assisted optimization, we put the focus in [31] on mixed discrete-continuous optmization problems, one of the major challenges of this topic. In addition, we focused on the evolution control and batch parallelism to deal with another challenge which consists in efficiently integrating surrogate models (Bayesian neural networks in this case) in evolutionary algorithms [FGCS-Briffoteaux, à compléter]. From the application point of view, we applied our approaches to three different real-world simulation-based problems in the context of three collaborations: multi-stage optimal scheduling of virtual power plants (collaboration with electrical engineering department of UMONS University, Belgium) in [27], aerospace vehicle design (collaboration with ONERA, Paris) in [31], and resource allocation for the Tuberculosis edidemic control (Monash University) in [FGCS-Briffoteax, à compléter]. The two latter ones are summarized in the following sections. Regarding the second direction, we thoroughly study in [16] the impact of landscape characteristics on the performance of search heuristics for black-box multi-objective combinatorial optimization problems. We also introduce in [26] new insightful features for continuous exploratory landscape analysis and algorithm selection. In addition, as pointed out in 3.2, variable selection is highly important to deal with BOPs. In [28], in collaboration with our partners from Shinshu University we come out with an efficient method to classify variables influencing convergence and increase their recombination rate. The contributions are summarized in the following.

Surrogate-assisted optimization of constrained mixed variable problems: application to the design of a launch vehiclethrust frame.

Participants : El-Ghazali Talbi [contact person] , Julien Pelamatti, Loïc Brevault [ONERA] , Mathieu Balesdent [ONERA] , Yannick Guerin [CNES] .

Within the framework of complex systems design, such as launch vehicles, numerical optimization is an essential tool as it allows to reduce the design process time and costs. The inclusion of discrete variables in the design process allows to extend the applicability of numerical optimization methods to a broader number of systems and sub-systems. In [31], a recently proposed adaptation of the Efficient Global Optimization method (EGO) for constrained mixed-variable problems is applied to the design optimization of a launch vehicle thrust frame, which depends on both continuous sizing parameters and discrete variables characterizing the number of structural reinforcements. The EGO adaptation that is considered is based on a redefinition of the Gaussian Process kernel as a product between a standard continuous kernel and a second kernel representing the covariance between the discrete variable values. From the results obtained on an analytical test-case as well as on the launch vehicle thrust frame design optimization, it is shown that the use of the mixed-variable EGO algorithm allows to converge towards the neighborhoods of the problems optima with fewer function evaluations when compared to reference optimization algorithms.

Parallel Batched Bayesian Neural Network-assisted GA versus q-EGO

Participants : Guillaume Briffoteaux, Maxime Gobert, Jan Gmys, Nouredine Melab [contact person] , Romain Ragonnet [School of Public Health and Preventive Medicine, Monash University, Australia] , Mohand Mezmaz [University of Mons, Blegium] , Daniel Tuyttens [University of Mons, Blegium] .

Surrogate-based optimization has been widely used to deal with expensive black-box simulation-based objective functions. The use of a surrogate model allows to reduce the number of calls to the costly simulator. In (SWEVO, Briffoteaux et al., 2020), the Efficient Global Optimization (EGO) reference framework is challenged by a Bayesian Neural Network-assisted Genetic Algorithm, namely BNN-GA. The Bayesian Neural Network (BNN) surrogate provides an uncertainty measure of the prediction that allows to compute the Expected Improvement of a candidate solution in order to improve the exploration of the objective space. BNN is also more reliable than Kriging models for high-dimensional problems and faster to set up thanks to its incremental training. In addition, we propose a batch-based approach for the parallelization of BNN-GA that is challenged by a parallel version of EGO, called q-EGO. Parallel computing is a complementary way to deal with the computational burden of simulation-based optimization. The comparison of the two parallel approaches is experimentally performed through several benchmark functions and two real-world problems within the scope of Tuberculosis Transmission Control (TBTC). The results demonstrate the efficiency and scalability of the parallel batched BNN-GA for small budgets, outperforming it for larger budgets on the benchmark testbed. A significant improvement of the solutions is obtained for two TBTC problems. Finally, our study proves that parallel batched BNN-GA is a viable alternative to q-EGO approaches being more suitable for high-dimensional problems and parallelization impact.

Landscape-aware performance prediction and algorithm selection for single- and multi-objective evolutionary optimization

Participants : Arnaud Liefooghe, Bilel Derbel, Fabio Daolio [ASOS.com, UK] , Sébastien Verel [LISIC, Université du Littoral Côte d'Opale] , Hernan Aguirre [Shinshu University, Japan] , Kiyoshi Tanaka [Shinshu University, Japan] .

Extracting a priori knowledge informing about the landscape underlying an unknown optimization problem has been proved extremely useful for different purposes, such as designing finely-tuned algorithms, predicting algorithm performance and designing automated portfolio-based solving approaches. Considering black-box continuous single-objective optimization problems, in [26], we adopt an exploratory landscape analysis approach providing a unified methodology for integrating landscape features into sophisticated machine learning techniques. More precisely, we consider the design of novel informative and cheap landscape features on the basis of the search tree constructed by the so-called SOO (Simultaneous Optimistic Optimization) algorithm, which is arguably a global optimizer coming from the machine learning community and having its foundations in the multi-armed bandit theory. We thereby provide empirical evidence on the accuracy of the proposed features for both predicting high-level problem properties, and tackling the algorithm selection problem with respect to a given portfolio of available solvers and a broad range of optimisation problems taking from the specialized literature and exposing different degrees of difficulty.

Considering black-box combinatorial multi-objective optimization problems, in [16], we expose and contrast the impact of landscape characteristics on the performance of search heuristics for black-box multi-objective combinatorial optimization problems. A sound and concise summary of features characterizing the structure of an arbitrary problem instance is identified and related to the expected performance of global and local dominance-based multi-objective optimization algorithms. We provide a critical review of existing features tailored to multi-objective combinatorial optimization problems, and we propose additional ones that do not require any global knowledge from the landscape, making them suitable for large-size problem instances. Their intercorrelation and their association with algorithm performance are also analyzed. This allows us to assess the individual and the joint effect of problem features on algorithm performance, and to highlight the main difficulties encountered by such search heuristics. By providing effective tools for multi-objective landscape analysis, we highlight that multiple features are required to capture problem difficulty, and we provide further insights into the importance of ruggedness and multimodality to characterize multi-objective combinatorial landscapes.

Estimating Relevance of Variables for Effective Recombination

Participants : Arnaud Liefooghe, Bilel Derbel, Sébastien Verel [LISIC, Université du Littoral Côte d'Opale] , Taishi Ito [Shinshu University, Japan] , Hernan Aguirre [Shinshu University, Japan] , Kiyoshi Tanaka [Shinshu University, Japan] .

Dominance and its extensions, decomposition, and indicator functions are well-known approaches used to design MOEAs. Algorithms based on these approaches have mostly sought to enhance parent selection and survival selection. In addition, several variation operators have been developed for MOEAs. In [28], we focus on the classification and selection of variables to improve the effectiveness of solution search. We propose a method to classify variables that influence convergence and increase their recombination rate, aiming to improve convergence of the approximation found by the algorithm. We incorporate the proposed method into NSGA-II and study its effectiveness using three-objective DTLZ and WFG benchmark functions, including unimodal, multimodal, separable, non-separable, unbiased, and biased functions. We also test the effectiveness of the proposed method on a real-world bi-objective problem. Simulation results verify that the proposed method can contribute to achieving faster and better convergence in several kinds of problems, including the real-world problem.