Section: Scientific Foundations

Scientific Foundations

Introduction

This section describes Tao's main research directions, first presented during Tao's evaluation in November 2007. Four strategic issues had been identified at the crossroad of Machine Learning and Evolutionary Computation:

Six Special Interest Groups (SIGs) have been defined in TAO, investigating the above complementary issues from different perspectives. The comparatively small size of Tao SIGs enables in-depth and lively discussions; the fact that all TAO members belong to several SIGs, on the basis of their personal interests, enforces the strong and informal collaboration of the groups, and the fast information dissemination.

Representations and Properties

The choice of the solution space is known to be the crux of both Machine Learning (model selection) and Evolutionary Computation (genotypic-phenotypic mapping).

The first research theme in TAO thus concerns the definition of an adequate representation, or search space $\mathscr{H}$, together with that of adequate navigation operators. $\mathscr{H}$ and its navigation operators must enforce flexible trade-offs between expressiveness and compacity on the one hand, and stability and versatility on the other hand.

Expressiveness/compacity trade-off (static property): $\mathscr{H}$ should simultaneously include sufficiently complex solutions $-$ i.e. good-enough solutions for the problem at hand $-$ and offer a short description for these solutions, thus making it feasible to find them.

Stability/versatility trade-off (dynamic property): while most modifications of a given solution in $\mathscr{H}$ should only marginally modify its behavior (stability), some modifications should lead to radically different behaviors (versatility). Both properties are required for efficient optimization in complex search spaces; stability, also referred to as “strong causality principle” [98] is needed for optimization to do better than random walk; versatility potentially speeds up optimization through creating short-cuts in the search space.

This research direction is investigated in:

• the Complex System SIG (section 6.2 ) focusing on developmental representations for Design and sequential representations for Temporal Planning;

• the Large and Deep Networks SIG (section 6.6 ) considering deep or stochastic Neural Network Topologies;

• the Continuous Optimization SIG (section 6.4 ), concerned with adaptive representations.

Optimal Decision Under Uncertainty

Benefiting from the MoGo expertise, TAO investigates several extensions of the Multi-Armed Bandit (MAB) framework and the Monte-Carlo tree search. Some main issues raised by optimal decision under uncertainty are the following:

• Regret minimization and any-time behavior.

  The any-time issue is tightly related to the scalability of Optimal Decision under Uncertainty; typically, MAB was found better suited than standard Reinforcement Learning to large-scale problems as its criterion (the regret minimization) is more amenable to fast approximations.

• Dynamic environments (non stationary reward functions).

  The dynamic environment issue, first investigated in TAO through the On-line Trading of Exploration vs Exploitation Challenge(The OTEE Challenge, funded by Touch Clarity Ltd and organized by the PASCAL Network of Excellence, models the selection of news to be displayed by a Web site as a multi-armed bandit, where the user's interests are prone to sudden changes; the OTEE Challenge was won by the TAO team in 2006.), is relevant to e.g. on-line parameter tuning (see section 6.3 ).

• Use of side information / Multi-variate MAB

  The use of side information by MAB is meant to exploit prior knowledge and/or complementary information about the reward. Typically in MoGo, the end of the game can be described at different levels of precision (e.g., win/lose, difference in the number of stones); estimating the local reward estimate depending on the available side information aims at a better robustness.

• Bounded rationality.

  The bounded rationality issue actually regards two settings. The first one considers a number of options which is large relatively to the time horizon, meaning that only a sample of the possible actions can be considered in the imparted time. The second one deals with a finite unknown horizon, as is the case for the Feature Selection problem.

• Multi-objective optimization.

  Many applications actually involve antagonistic criteria; for instance autonomous robot controllers might simultaneously want to explore the robot environment, while preserving the robot integrity. The challenge raised by Multi-objective MAB is to find the “Pareto-front” policies for a moderately increased computational cost compared to the standard mono-objective approach.

This research direction is chiefly investigated by the Optimal Decision Making SIG (section 6.5 ), in interaction with the Complex System and the Crossing the Chasm SIGs (sections 6.2 and 6.3 ).

Hardware-Software Bridges

Historically, the apparition of parallel architectures only marginally affected the art of programing; the main focus has been on how to rewrite sequential algorithms to make them parallelism-compliant. The use of distributed architectures however calls for a radically different programming style/computational thinking, seamlessly integrating:

• computation: aggregating the local information available with any information provided by other nodes;

• communication: building abstractions of the local node state, to be transmitted to other nodes;

• assessment: modeling other nodes in order to modulate the exploitation (respectively, the abstraction) of the received (resp. emitted) information.

Message passing algorithms such as Page Rank or Affinity Propagation [92] are prototypical examples of distributed algorithms. The analysis is shifted from the static properties (termination and computational complexity) to the dynamic properties (convergence and approximation) of the algorithms, after the guiding principles of complex systems.

Symmetrically, modern computing systems are increasingly viewed as complex systems of their own, due to their ever increasing resources and computational load. The huge need of scalable administration tools, supporting grid monitoring and maintenance of the job running process, paved the way toward Autonomic Computing [94] . Autonomic Computing (AC) Systems are meant to feature self-configuring, self-healing, self-protecting and self-optimizing skills [99] . A key milestone for Autonomic Computing is to provide the system with a phenomenological model of itself (self-aware system), built from the system logs using Machine Learning and Data Mining.

This research direction is investigated in the Complex System SIG (section 6.2 ) and in the Autonomic Computing SIG (section 6.1 ).

Crossing the chasm

This fourth strategic priority, inspired by Moore's book [97] , is motivated by the fact that many outstandingly efficient algorithms never make it out of research labs. One reason for it is the difference between editor's and programmer's view of algorithms. In the perspective of software editors, an algorithm is best viewed as a single “Go” button. The programmer's perspective is radically different: as he/she sees that various functionalities can be ented on the same algorithmic core, the number of options steadily increases (with the consequence that users usually master less than 10% of the available functionalities). Independently, the programmer gradually acquires some idea of the flexibility needed to handle different application domains; this flexibility is most usually achieved through defining parameters and tuning them. Parameter tuning thus becomes a barrier to the efficient use of new algorithms.

This research direction is chiefly investigated by the Crossing the Chasm SIG (section 6.3 ) and also by the Continuous Optimization SIG (section 6.4 ).