Section: New Results

Lifelong autonomy

Motion planning for robot audition

Participants : François Charpillet, Francis Colas, Van Quan Nguyen.

We collaborated on this subject with Emmanuel Vincent from the Multispeech team (Inria Nancy – Grand Est).

Robot audition refers to a range of hearing capabilities which help robots explore and understand their environment. Among them, sound source localization is the problem of estimating the location of a sound source given measurements of its angle of arrival with respect to a microphone array mounted on the robot. In addition, robot motion can help quickly solve the front-back ambiguity existing in a linear microphone array. In this work, we focus on the problem of exploiting robot motion to improve the estimation of the location of an intermittent and possibly moving source in a noisy and reverberant environment. We first propose a robust extended mixture Kalman filtering framework for jointly estimating the source location and its activity over time. Building on this framework, we then propose a long-term robot motion planning algorithm based on Monte Carlo tree search to find an optimal robot trajectory according to two alternative criteria: the Shannon entropy or the standard deviation of the estimated belief on the source location. Experimental results show the robustness of the proposed estimation framework to false angle of arrival measurements within ±20° and 10% false source activity detection rate. The proposed robot motion planning technique achieves an average localization error 48.7% smaller than a one-step-ahead method.

Publication: [10]

Addressing Active Sensing Problems through Monte-Carlo Tree Search (MCTS)

Participants : Vincent Thomas, Gabriel Belouze, Sylvain Geiser, Olivier Buffet.

The problem of active sensing is of paramount interest for building self awareness in robotic systems. It consists in planning actions in a view to gather information (e.g., measured through the entropy over certain state variables) in an optimal way. In the past, we have proposed an original formalism, $\rho$-POMDPs, and new algorithms for representing and solving such active sensing problems [24] by using point-based algorithms, assuming either convex or Lipschitz-continuous criteria. More recently, we have developed new approaches based on Monte-Carlo Tree Search (MCTS), and in particular Partially Observable Monte-Carlo Planning (POMCP), which provably converge only assuming the continuity of the criterion. We are now going towards algorithms more suitable to certain robotic tasks by allowing for continuous state and observation spaces.

Publication: [20]

Heuristic Search for (Partially Observable) Stochastic Games

Participants : Olivier Buffet, Vincent Thomas.

Collaboration with Jilles Dibangoye (INSA-Lyon, Inria team CHROMA) and Abdallah Saffidine (University of New South Wales (UNSW), Sydney, Australia).

Many robotic scenarios involve multiple interacting agents, robots or humans, e.g., security robots in public areas. We have mainly worked in the past on the collaborative setting, all agents sharing one objective, in particular through solving Dec-POMDPs by (i) turning them into occupancy MDPs and (ii) using heuristic search techniques and value function approximation [2]. A key idea is to take the point of view of a central planner and reason on a sufficient statistic called occupancy state. We are now working on applying similar approaches in the important 2-player zero-sum setting, i.e., with two competing agents. As a preliminary step, we have proposed and evaluated an algorithm for (fully observable) stochastic games, which does not require any problem transformation. Then we have proposed an algorithm for partially observable stochastic games, here turning the problem into an occupancy Markov game.

[This line of research will be pursued through Jilles Dibangoye's ANR JCJC PLASMA.]

Interpretable Action Policies

Participant : Olivier Buffet.

Collaboration with Iadine Chadès and Jonathan Ferrer Mestres (CSIRO, Brisbane, Australia), and Thomas G. Dietterich (Oregon State University, USA).

Computer-aided task planning requires providing user-friendly plans, in particular, plans that make sense to the user. In probabilistic planning (in the MDP formalism), such interpretable plans can be derived by constraining action policies (if $X$ happens, do $Y$) to depend on a reduced subset of (abstract) states or state variables. We have (i) formalized the problem of finding a set of at most $K$ abstract states (forming a partition of the original state space) such that any optimal policy of the induced abstract MDP is as close as possible to optimal policies of the original MDP, and (ii) proposed 3 solution algorithms with theoretical and empirical evaluations.

Perspective: hierarchical quality diversity, from materials to machines

Participant : Jean-Baptiste Mouret.

Collaboration with CSIRO (Australia) and Vrije Universiteit Amsterdam (Netherlands).

Natural lifeforms specialize to their environmental niches across many levels, from low-level features such as DNA and proteins, through to higher-level artefacts including eyes, limbs and overarching body plans. We propose ‘multi-level evolution’, a bottom-up automatic process that designs robots across multiple levels and niches them to tasks and environmental conditions. Multi-level evolution concurrently explores constituent molecular and material building blocks, as well as their possible assemblies into specialized morphological and sensorimotor configurations. Multi-level evolution provides a route to fully harness a recent explosion in available candidate materials and ongoing advances in rapid manufacturing processes. We outline a feasible architecture that realizes this vision, highlight the main roadblocks and how they may be overcome, and show robotic applications to which multi-level evolution is particularly suited. By forming a research agenda to stimulate discussion between researchers in related fields, we hope to inspire the pursuit of multi-level robotic design all the way from material to machine.

Publication: [5]

Improving Embodied Evolutionary Robotics

Participant : Amine Boumaza.

Multi-robots learning is a hard still unsolved problem. When framed into the machine learning theoretical setting, it suffers from a high complexity when seeking optimal solutions. On the other hand, when sub-optimal solutions are acceptable Embodied Evolutionary Robotics, can provide solutions that perform well in practice. Improving these algorithms in terms of run-time or solution quality is an important research question.

It has been long known from the theoretical work on evolution strategies, that recombination improves convergence towards better solution and improves robustness against selection error in noisy environment. We propose to investigate the effect of recombination in online embodied evolutionary robotics, where evolution is decentralized on a swarm of agents. We hypothesize that these properties can also be observed in these algorithms and thus could improve their performance. We introduce the $(\mu /\mu ,1)$-On-line Embedded Evolutionary Algorithm (EEA) which uses a recombination operator inspired from evolution strategies and apply it to learn three different collective robotics tasks, locomotion, item collection and item foraging. Different recombination operators are investigated and compared against a purely mutative version of the algorithm. The experiments show that, when correctly designed, recombination improves significantly the adaptation of the swarm in all scenarios.

Publication: [13] [12]

Multi-robot exploration of an unknown environment

Participants : Nicolas Gauville, François Charpillet.

Different approaches exist for multi-robot autonomous exploration. These include frontier approaches, where robots are assigned to unexplored areas of the map, which provide good performance but require sharing the map and centralizing decision-making. The Brick and Mortar approaches, on the other hand, use a ground marking with local decision-making, but give much lower performance. The algorithm developped by Nicolas Gauville during his pre-thesis period is a trade-off between these two approaches, allowing local decision-making and, surprisingly, performances are closed to centralized frontier approaches. We also propose a comparative study of the performance of the three different approaches : Brick & Mortar, Global Frontiers and Local Frontiers. Our local algorithm is also complete for the exploration problem and can be easily distributed on robots with a minor loss of performance. This work follows the Cart-O-Matic project in which our team participated, which aimed to explore and map a building while recognizing specific objects inside with a team of 5 mobile robots.

Publication: [16]