Section: Overall Objectives

Overall Objectives

In the context of black-box numerical optimization previously described, the scientific positioning of the RandOpt ream is at the intersection between theory, algorithm design, and applications. Our vision is that the field of stochastic black-box optimization should reach the same level of maturity than gradient-based convex mathematical optimization. This entails major algorithmic developments for constrained, multiobjective and large-scale black-box optimization and major theoretical developments for analyzing current methods including the state-of-the-art CMA-ES.

The specificity in black-box optimization is that methods are intended to solve problems characterized by a non-property—non-convex, non-linear, non-smooth. This contrasts with gradient-based optimization and poses on the one hand some challenges when developing theoretical frameworks but also makes it compulsory to complement theory with empirical investigations.

Our ultimate goal is to provide software that is useful for practitioners. We see that theory is a means for this end (rather than an end in itself) and it is also our firm belief that parameter tuning is part of the designer's task.

This shapes, on the one hand, four main scientific objectives for our proposed team:

1. develop novel theoretical frameworks for guiding (a) the design of novel black-box methods and (b) their analysis, allowing to

2. provide proofs ofkey features of stochastic adaptive algorithms including the state-of-the-art method CMA-ES: linear convergence and learning of second order information.

3. develop stochastic numerical black-box algorithms following a principled design in domains with a strong practical need for much better methods namely constrained, multiobjective, large-scale and expensive optimization. Implement the methods such that they are easy to use. And finally, to

4. set new standards in scientific experimentation, performance assessment and benchmarking both for optimization on continuous or combinatorial search spaces. This should allow in particular to advance the state of reproducibility of results of scientific papers in optimization.

On the other hand, the above motivates our objectives with respect to dissemination and transfer:

1. develop software packages that people can directly use to solve their problems. This means having carefully thought out interfaces, generically applicable setting of parameters and termination conditions, proper treatment of numerical errors, catching properly various exceptions, etc.;

2. have direct collaborations with industrials;

3. publish our results both in applied mathematics and computer science bridging the gap between very often disjoint communities.