7.1.1 Explauto

• Name: an autonomous exploration library
• Keyword: Exploration
• Scientific Description:

  An important challenge in developmental robotics is how robots can be intrinsically motivated to learn efficiently parametrized policies to solve parametrized multi-task reinforcement learning problems, i.e. learn the mappings between the actions and the problem they solve, or sensory effects they produce. This can be a robot learning how arm movements make physical objects move, or how movements of a virtual vocal tract modulates vocalization sounds. The way the robot will collects its own sensorimotor experience have a strong impact on learning efficiency because for most robotic systems the involved spaces are high dimensional, the mapping between them is non-linear and redundant, and there is limited time allowed for learning. If robots explore the world in an unorganized manner, e.g. randomly, learning algorithms will be often ineffective because very sparse data points will be collected. Data are precious due to the high dimensionality and the limited time, whereas data are not equally useful due to non-linearity and redundancy. This is why learning has to be guided using efficient exploration strategies, allowing the robot to actively drive its own interaction with the environment in order to gather maximally informative data to optimize the parametrized policies. In the recent year, work in developmental learning has explored various families of algorithmic principles which allow the efficient guiding of learning and exploration.

  Explauto is a framework developed to study, model and simulate curiosity-driven learning and exploration in real and simulated robotic agents. Explauto’s scientific roots trace back from Intelligent Adaptive Curiosity algorithmic architecture 131, which has been extended to a more general family of autonomous exploration architectures by 70 and recently expressed as a compact and unified formalism 123. The library is detailed in 125. In Explauto, interest models are implementing the strategies of active selection of particular problems / goals in a parametrized multi-task reinforcement learning setup to efficiently learn parametrized policies. The agent can have different available strategies, parametrized problems, models, sources of information, or learning mechanisms (for instance imitate by mimicking vs by emulation, or asking help to one teacher or to another), and chooses between them in order to optimize learning (a processus called strategic learning 128). Given a set of parametrized problems, a particular exploration strategy is to randomly draw goals/ RL problems to solve in the motor or problem space. More efficient strategies are based on the active choice of learning experiments that maximize learning progress using bandit algorithms, e.g. maximizing improvement of predictions or of competences to solve RL problems 131. This automatically drives the system to explore and learn first easy skills, and then explore skills of progressively increasing complexity. Both random and learning progress strategies can act either on the motor or on the problem space, resulting in motor babbling or goal babbling strategies.

  • Motor babbling consists in sampling commands in the motor space according to a given strategy (random or learning progress), predicting the expected effect, executing the command through the environment and observing the actual effect. Both the parametrized policies and interest models are finally updated according to this experience.
  • Goal babbling consists in sampling goals in the problem space and to use the current policies to infer a motor action supposed to solve the problem (inverse prediction). The robot/agent then executes the command through the environment and observes the actual effect. Both the parametrized policies and interest models are finally updated according to this experience.It has been shown that this second strategy allows a progressive solving of problems much more uniformly in the problem space than with a motor babbling strategy, where the agent samples directly in the motor space 70.

   

  

• Functional Description:

  This library provides high-level API for an easy definition of:

  • Real and simulated robotic setups (Environment level),
  • Incremental learning of parametrized policies (Sensorimotor level),
  • Active selection of parametrized RL problems (Interest level).

  The library comes with several built-in environments. Two of them corresponds to simulated environments: a multi-DoF arm acting on a 2D plan, and an under-actuated torque-controlled pendulum. The third one allows to control real robots based on Dynamixel actuators using the Pypot library. Learning parametrized policies involves machine learning algorithms, which are typically regression algorithms to learn forward models, from motor controllers to sensory effects, and optimization algorithms to learn inverse models, from sensory effects, or problems, to the motor programs allowing to reach them. We call these sensorimotor learning algorithms sensorimotor models. The library comes with several built-in sensorimotor models: simple nearest-neighbor look-up, non-parametric models combining classical regressions and optimization algorithms, online mixtures of Gaussians, and discrete Lidstone distributions. Explauto sensorimotor models are online learning algorithms, i.e. they are trained iteratively during the interaction of the robot in theenvironment in which it evolves. Explauto provides also a unified interface to define exploration strategies using the InterestModel class. The library comes with two built-in interest models: random sampling as well as sampling maximizing the learning progress in forward or inverse predictions.

  Explauto environments now handle actions depending on a current context, as for instance in an environment where a robotic arm is trying to catch a ball: the arm trajectories will depend on the current position of the ball (context). Also, if the dynamic of the environment is changing over time, a new sensorimotor model (Non-Stationary Nearest Neighbor) is able to cope with those changes by taking more into account recent experiences. Those new features are explained in Jupyter notebooks.

  This library has been used in many experiments including:

  • the control of a 2D simulated arm,
  • the exploration of the inverse kinematics of a poppy humanoid (both on the real robot and on the simulated version),
  • acoustic model of a vocal tract.

  Explauto is crossed-platform and has been tested on Linux, Windows and Mac OS. It has been released under the GPLv3 license.

• URL: https://github.com/flowersteam/explauto
• Contacts: Clément Moulin-Frier, Pierre Rouanet, Sebastien Forestier

7.1.2 KidBreath

• Keyword: Machine learning
• Functional Description: KidBreath is a web responsive application composed by several interactive contents linked to asthma and displayed to different forms: learning activities with quiz, short games and videos. There are profil creation and personalization, and a part which describes historic and scoring of learning activities, to see evolution of Kidreath use. To test Kidlearn algorithm, it is iadapted and integrated on this platform. Development in PHP, HTML-5, CSS, MySQL, JQuery, Javascript. Hosting in APACHE, LINUX, PHP 5.5, MySQL, OVH.
• Contacts: Pierre-Yves Oudeyer, Manuel Lopes, Alexandra Delmas
• Partner: ItWell SAS

7.1.3 Kidlearn: money game application

• Functional Description: The games is instantiated in a browser environment where students are proposed exercises in the form of money/token games (see Figure 2). For an exercise type, one object is presented with a given tagged price and the learner has to choose which combination of bank notes, coins or abstract tokens need to be taken from the wallet to buy the object, with various constraints depending on exercises parameters. The games have been developed using web technologies, HTML5, javascript and Django.

  

  

  

  

• URL: https://flowers.inria.fr/research/kidlearn/
• Contact: Benjamin Clement

7.1.4 Kidlearn: script for Kidbreath use

• Keyword: PHP
• Functional Description: A new way to test Kidlearn algorithms is to use them on Kidbreath Plateform. The Kidbreath Plateform use apache/PHP server, so to facilitate the integration of our algorithm, a python script have been made to allow PHP code to use easily the python library already made which include our algorithms.
• URL: https://flowers.inria.fr/research/kidlearn/
• Contact: Benjamin Clement

7.1.5 KidLearn

• Keyword: Automatic Learning
• Functional Description: KidLearn is a software which adaptively personalize sequences of learning activities to the particularities of each individual student. It aims at proposing to the student the right activity at the right time, maximizing concurrently his learning progress and its motivation.
• URL: https://flowers.inria.fr/research/kidlearn/
• Authors: Benjamin Clement, Didier Roy, Pierre-Yves Oudeyer, Manuel Lopes
• Contacts: Benjamin Clement, Pierre-Yves Oudeyer
• Participants: Benjamin Clement, Didier Roy, Manuel Lopes, Pierre Yves Oudeyer

7.1.6 Poppy

• Keywords: Robotics, Education
• Functional Description:

  The Poppy Project team develops open-source 3D printed robots platforms based on robust, flexible, easy-to-use and reproduce hardware and software. In particular, the use of 3D printing and rapid prototyping technologies is a central aspect of this project, and makes it easy and fast not only to reproduce the platform, but also to explore morphological variants. Poppy targets three domains of use: science, education and art.

  In the Poppy project we are working on the Poppy System which is a new modular and open-source robotic architecture. It is designed to help people create and build custom robots. It permits, in a similar approach as Lego, building robots or smart objects using standardized elements.

  Poppy System is a unified system in which essential robotic components (actuators, sensors...) are independent modules connected with other modules through standardized interfaces:

  • Unified mechanical interfaces, simplifying the assembly process and the design of 3D printable parts.
  • Unified communication between elements using the same connector and bus for each module.
  • Unified software, making it easy to program each module independently.

  Our ambition is to create an ecosystem around this system so communities can develop custom modules, following the Poppy System standards, which can be compatible with all other Poppy robots.

• URL: https://www.poppy-project.org/
• Contacts: Matthieu Lapeyre, Pierre Rouanet, Pierre-Yves Oudeyer, Didier Roy, Stephanie Noirpoudre, Theo Segonds, Damien Caselli, Nicolas Rabault
• Participants: Jonathan Grizou, Matthieu Lapeyre, Pierre Rouanet, Pierre-Yves Oudeyer

7.1.7 Poppy Ergo Jr

• Name: Poppy Ergo Jr
• Keywords: Robotics, Education
• Functional Description:

  Poppy Ergo Jr is an open hardware robot developed by the Poppy Project to explore the use of robots in classrooms for learning robotic and computer science.

  It is available as a 6 or 4 degrees of freedom arm designed to be both expressive and low-cost. This is achieved by the use of FDM 3D printing and low cost Robotis XL-320 actuators. A Raspberry Pi camera is attached to the robot so it can detect object, faces or QR codes.

  The Ergo Jr is controlled by the Pypot library and runs on a Raspberry pi 2 or 3 board. Communication between the Raspberry Pi and the actuators is made possible by the Pixl board we have designed.

  

  The Poppy Ergo Jr robot has several 3D printed tools extending its capabilities. There are currently the lampshade, the gripper and a pen holder.

  

  With the release of a new Raspberry Pi board early 2016, the Poppy Ergo Jr disk image was updated to support Raspberry Pi 2 and 3 boards. The disk image can be used seamlessly with a board or the other.

• URL: https://github.com/poppy-project/poppy-ergo-jr
• Contacts: Theo Segonds, Damien Caselli

7.1.8 S-RL Toolbox

• Name: Reinforcement Learning (RL) and State Representation Learning (SRL) for Robotics
• Keywords: Machine learning, Robotics
• Functional Description: This repository was made to evaluate State Representation Learning methods using Reinforcement Learning. It integrates (automatic logging, plotting, saving, loading of trained agent) various RL algorithms (PPO, A2C, ARS, ACKTR, DDPG, DQN, ACER, CMA-ES, SAC, TRPO) along with different SRL methods (see SRL Repo) in an efficient way (1 Million steps in 1 Hour with 8-core cpu and 1 Titan X GPU).
• URL: https://github.com/araffin/robotics-rl-srl
• Contact: David Filliat
• Partner: ENSTA

7.1.9 Deep-Explauto

• Name: Deep-Explauto
• Keywords: Deep learning, Unsupervised learning, Learning, Experimentation
• Functional Description:

  Until recently, curiosity driven exploration algorithms were based on classic learning algorithms, unable to handle large dimensional problems (see explauto). Recent advances in the field of deep learning offer new algorithms able to handle such situations.

  Deep explauto is an experimental library, containing reference implementations of curiosity driven exploration algorithms. Given the experimental aspect of exploration algorithms, and the low maturity of the libraries and algorithms using deep learning, proposing black-box implementations of those algorithms, enabling a blind use of those, seem unrealistic.

  Nevertheless, in order to quickly launch new experiments, this library offers an set of objects, functions and examples, allowing to kickstart new experiments.

• Contact: Alexandre Pere

7.1.10 Orchestra

• Name: Orchestra
• Keyword: Experimental mechanics
• Functional Description:

  Ochestra is a set of tools meant to help in performing experimental campaigns in computer science. It provides you with simple tools to:

  + Organize a manual experimental workflow, leveraging git and lfs through a simple interface. + Collaborate with other peoples on a single experimental campaign. + Execute pieces of code on remote hosts such as clusters or clouds, in one line. + Automate the execution of batches of experiments and the presentation of the results through a clean web ui.

  A lot of advanced tools exists on the net to handle similar situations. Most of them target very complicated workflows, e.g. DAGs of tasks. Those tools are very powerful but lack the simplicity needed by newcomers. Here, we propose a limited but very simple tool to handle one of the most common situation of experimental campaigns: the repeated execution of an experiment on variations of parameters.

  In particular, we include three tools: + expegit: a tool to organize your experimental campaign results in a git repository using git-lfs (large file storage). + runaway: a tool to execute code on distant hosts parameterized with easy to use file templates. + orchestra: a tool to automate the use of the two previous tools on large campaigns.

• Contact: Alexandre Pere

7.1.11 Curious

• Name: Curious: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning
• Keywords: Exploration, Reinforcement learning, Artificial intelligence
• Functional Description: This is an algorithm enabling to learn a controller for an agent in a modular multi-goal environment. In these types of environments, the agent faces multiple goals classified in different types (e.g. reaching goals, grasping goals for a manipulation robot).
• Contact: Cedric Colas

7.1.12 teachDeepRL

• Name: Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
• Keywords: Machine learning, Git
• Functional Description:

  Codebase from our CoRL2019 paper https://arxiv.org/abs/1910.07224

  This github repository provides implementations for the following teacher algorithms: - Absolute Learning Progress-Gaussian Mixture Model (ALP-GMM), our proposed teacher algorithm - Robust Intelligent Adaptive Curiosity (RIAC), from Baranes and Oudeyer, R-IAC: robust intrinsically motivated exploration and active learning. - Covar-GMM, from Moulin-Frier et al., Self-organization of early vocal development in infants and machines: The role of intrinsic motivation.

• URL: https://github.com/flowersteam/teachDeepRL
• Author: Remy Portelas
• Contact: Remy Portelas

7.1.13 Automated Discovery of Lenia Patterns

• Keywords: Exploration, Cellular automaton, Deep learning, Unsupervised learning
• Scientific Description: In many complex dynamical systems, artificial or natural, one can observe selforganization of patterns emerging from local rules. Cellular automata, like the Game of Life (GOL), have been widely used as abstract models enabling the study of various aspects of self-organization and morphogenesis, such as the emergence of spatially localized patterns. However, findings of self-organized patterns in such models have so far relied on manual tuning of parameters and initial states, and on the human eye to identify “interesting” patterns. In this paper, we formulate the problem of automated discovery of diverse self-organized patterns in such high-dimensional complex dynamical systems, as well as a framework for experimentation and evaluation. Using a continuous GOL as a testbed, we show that recent intrinsically-motivated machine learning algorithms (POP-IMGEPs), initially developed for learning of inverse models in robotics, can be transposed and used in this novel application area. These algorithms combine intrinsically motivated goal exploration and unsupervised learning of goal space representations. Goal space representations describe the “interesting” features of patterns for which diverse variations should be discovered. In particular, we compare various approaches to define and learn goal space representations from the perspective of discovering diverse spatially localized patterns. Moreover, we introduce an extension of a state-of-the-art POP-IMGEP algorithm which incrementally learns a goal representation using a deep auto-encoder, and the use of CPPN primitives for generating initialization parameters. We show that it is more efficient than several baselines and equally efficient as a system pre-trained on a hand-made database of patterns identified by human experts.
• Functional Description: Python source code of experiments and data analysis for the paper " Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems" (Chris Reinke, Mayalen Echeverry, Pierre-Yves Oudeyer in Submitted to ICLR 2020). The software includes: Lenia environment, exploration algorithms (IMGEPs, random search), deep learning algorithms for unsupervised learning of goal spaces, tools and configurations to run experiments, and data analysis tools.
• URL: https://github.com/flowersteam/automated_discovery_of_lenia_patterns
• Contacts: Chris Reinke, Mayalen Etcheverry

7.1.14 ZPDES_ts

• Name: ZPDES in typescript
• Keywords: Machine learning, Education
• Functional Description: ZPDES is a machine learning-based algorithm that allows you to customize the content of training courses for each learner's level. It has already been implemented in the Kidlern software in python with other algorithms. Here, ZPDES is implemented in typescript.
• URL: https://flowers.inria.fr/research/kidlearn/
• Authors: Benjamin Clement, Pierre-Yves Oudeyer, Didier Roy, Manuel Lopes
• Contact: Benjamin Clement

7.1.15 GEP-PG

• Name: Goal Exploration Process - Policy Gradient
• Keywords: Machine learning, Deep learning
• Functional Description: Reinforcement Learning algorithm working with OpenAI Gym environments. A first phase implements exploration using a Goal Exploration Process (GEP). Samples collected during exploration are then transferred to the memory of a deep reinforcement learning algorithm (deep deterministic policy gradient or DDPG). DDPG then starts learning from a pre-initialized memory so as to maximize the sum of discounted rewards given by the environment.
• Contact: Cedric Colas

7.1.16 EpidemiOptim

• Name: EpidemiOptim: a toolbox for the optimization of control policies in epidemiological models
• Keywords: Epidemiology, Optimization, Dynamical system, Reinforcement learning, Multi-objective optimisation
• Functional Description: This toolbox proposes a modular set of tools to optimize intervention strategies in epidemiological models. The user can define or use a pre-coded epidemiological model to represent an epidemic. He/she can define a set of cost functions to define a particular optimization problem. Finally, given an optimization problem (epidemiological model and cost functions and action modalities), the user can define/reuse optimization algorithms to optimize intervention strategies that minimize the costs. Finally, the toolbox contains visualization and comparison tools. This allows to investigate various hypotheses easily.
• URL: https://github.com/flowersteam/EpidemiOptim
• Contacts: Cedric Colas, Clément Moulin-Frier, Melanie Prague

7.1.17 IMAGINE

• Keywords: Exploration, Reinforcement learning, Modeling language, Artificial intelligence
• Functional Description: This software provides: 1. An environment modelling the social interaction between an autonomous agent and a social partner. The social partner gives natural language descriptions when the agent performs something interesting in the environment. 2. A modular architecture allowing the autonomous agent to manipulate and to target goals expressed in natural language. This architecture is divided into three modules: 2.a. A goal achievement function mapping language descriptions and the agent's observations to a reward signal 2.b. A goal conditioned-policy that uses the reward signal in order to learn the behaviour required to reach the goal (expressed in natural language). This module is trained via Reinforcement Learning 2.c. A goal imagination module allowing the agent to compose known goals into new sentences in order to creatively explore new outcomes in its environment.
• URL: https://github.com/flowersteam/Imagine
• Contacts: Tristan Karch, Cedric Colas, Clément Moulin-Frier, Pierre-Yves Oudeyer

7.1.18 DECSTR

• Name: Grouding Language to Autonomously-Acquired Skills via Goal Generation
• Keywords: Reinforcement learning, Curiosity, Intrinsic motivations
• Functional Description: DECSTR is a learning algorithm that trains an agent to reach semantic goals made of predicates characterizing spatial relations between pairs of blocks. After this first skill learning phase, the agent trains a language generation module that converts linguistic inputs into semantic goals. This module enables efficient language grounding.
• Contact: Cedric Colas

7.1.19 holmes

• Name: IMGEP-HOLMES, an algorithm for meta-diversity search applied to the automated discovery of novel structures in complex dynamical systems
• Keywords: Exploration, Incremental learning, Unsupervised learning, Hierarchical architecture, Intrinsic motivations, Cellular automaton, Complexity
• Functional Description: Python source code to reproduce the experiments and data analysis for the paper "Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems" (Mayalen Echeverry, Clément Moulin-Frier and Pierre-Yves Oudeyer, published at NeurIPS 2020). The user can define a complex system he would like to explore, or use the Lenia environment which is already provided. He/she can select an explorer to explore this system (Random or IMGEP explorer). For the IMGEP explorer, many variants of goal space representations are provided in the source code: hand-defined descriptors of the Lenia system, unsupervisedly learned descriptors that can be trained online during the course of exploration (VAE variants and Contrastive Learning variants) and the hierarchical progressively-learned architecture presented in the paper (HOLMES). To this purpose, the software includes tools and configurations to run experiments and for data analysis and comparison of the results, as well as for running the scripts on super-computers (SLURM job manager).
• URL: https://github.com/flowersteam/holmes
• Contact: Mayalen Etcheverry

7.1.20 metaACL

• Name: Meta Automatic Curriculum Learning
• Keywords: Machine learning, Git
• Functional Description:

  Codebase from our arxiv paper https://arxiv.org/abs/2011.08463

  This github repository provides implementations for AGAIN (Alp-Gmm and Inferred Progress Niches), our proposed Meta automatic curriculum learning teacher algorithm.

• URL: https://github.com/flowersteam/meta-acl
• Contact: Remy Portelas

7.1.21 EmComPartObs

• Name: Studying the joint role of partial observability and channel reliability in emergent communication
• Keywords: Multi-agent, Reinforcement learning, Emergent communication
• Functional Description: This source code contains a new grid-world environment where two agents interact to solve a task, Multi-Agent Reinforcement algorithms that solve that task, as well as plotting utilities.
• URL: https://github.com/UnrealLink/emergent_communication
• Publication: hal-03100681
• Contacts: Valentin Villecroze, Clément Moulin-Frier

7.1.22 grimgep

• Name: GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning
• Keywords: Machine learning, Reinforcement learning, Artificial intelligence, Exploration, Intrinsic motivations, Git, Deep learning
• Functional Description: Source code for the GRIMGEP paper (https://arxiv.org/abs/2008.04388) Contains: - Implementation of the GRIMGEP framework on top of three different underlying imgeps (Skew-fit, CountBased, OnlineRIG). - image-based 2D environment (PlaygroundRGB)
• URL: https://gitlab.com/Grg/grimgep
• Contacts: Grgur Kovac, Adrien Laversanne-Finot, Pierre-Yves Oudeyer

7.1.23 flowers-OL

• Name: flowers-open-lab
• Keyword: Experimentation
• Functional Description: This web platform designed for planning and implementing remote behavioural studies provides the following features: - Registration and login of participants - Presentation of the instructions concerning the experience and get informed consent - Behavioural task and questionnaires - Automatic management of a participant's schedule (sends emails before the user's appointments) - Quick and easy addition of new experimental conditions
• URL: https://flowers-mot.bordeaux.inria.fr/
• Contacts: Alexandr Ten, Maxime Adolphe

8.1.1 Testing the Learning Progres Hypothesis in Curiosity-Driven explortion in Human Adults

Participants: Pierre-Yves Oudeyer, Alexandr Ten.

This project involves a collaboration between the Flowers team and the Cognitive Neuroscience Lab of J. Gottlieb at Columbia Univ. (NY, US), on the understanding and computational modeling of mechanisms of curiosity, attention and active intrinsically motivated exploration in humans.

It is organized around the study of the hypothesis that subjective meta-cognitive evaluation of information gain (or control gain or learning progress) could generate intrinsic reward in the brain (living or artificial), driving attention and exploration independently from material rewards, and allowing for autonomous lifelong acquisition of open repertoires of skills. The project combines expertise about attention and exploration in the brain and a strong methodological framework for conducting experimentations with monkeys, human adults and children together with computational modeling of curiosity/intrinsic motivation and learning.

Such a collaboration paves the way towards a central objective, which is now a central strategic objective of the Flowers team: designing and conducting experiments in animals and humans informed by computational/mathematical theories of information seeking, and allowing to test the predictions of these computational theories.

Results

We provide empirical evidence that humans are sensitive to variation learniing progress (LP) by means of a novel experimental paradigm and computational modeling. We showed that while humans rely on competence information to avoid easy tasks, models that include an LP component provide the best fit to task selection data. These results provide a new bridge between research on artificial and biological curiosity, reveal strategies that are used by humans but have not been considered in computational research, and provide new tools for probing how humans become intrinsically motivated to learn and acquire interests and skills on extended time scales. The results were submitted to the journal Nature Communications and are currently under revision.

Participants ($N=330$) performed an online task in which they could freely engage with a set of learning activities (Fig. 5a). Each trial started with a free-choice panel prompting the participant to choose one of 4 activities depicted as families of “monsters” (Fig. 5a, (1)) and, after making a choice, received a randomly drawn member from the chosen family, made a binary guess about which food that member liked to eat (Fig. 5a, (2)), and received immediate feedback regarding their guess (Fig. 5a, (3)). To understand how participants self-organized their learning curriculum, we required them to complete 250 trials but did not impose any other constraint on their choice of activity.

Our key questions were (1) how people self-organize their exploration over a set of activities of variable difficulty, and (2) whether they spontaneously adopt learning maximization objectives when they do not receive explicit instructions. To examine these questions, we manipulated the difficulty of the available activities as a within-participant variable, and the instructions participants received as an across-participant variable. Difficulty was controlled by the complexity of the categorization rule governing the food preferences in each activity. In the easiest activity (A1), individual monster-family members differed in only one feature and that feature governed their food preference (e.g., a short octopus liked steak and a tall octopus liked broccoli; 1-dimensional categorization). In the next easiest level (A2), family members varied along two features but only one feature determined preference (1-dimensional with an irrelevant feature). In the most difficult learnable activity (A3) food preferences were determined by a conjunction of 2 variable features (2-dimensional categorization). Finally, the 4th activity (A4) was random and unlearnable: individual monsters had two variable features, but their food preferences were assigned randomly each time a new monster was sampled.

Learning objectives were manipulated across two randomly selected participant groups. Participants in the “external goal” group (EG; $N=176$) were asked to maximize learning across all the activities and were told that they will be tested at the end of the session. In contrast, participants in the “internal goal” group (IG, $N=154$) were told to choose any activity they wished with no constraint except for completing 250 trials. Except for this difference in instructions (and the fact that the EG group received the announced test), the two groups received identical treatments. Each group started with 15 forced-choice familiarization trials on each activity, followed by a 250-trial free-play stage, and gave several subjective ratings of the activities before and after the free play stage.

As shown in Fig. 5, our difficulty manipulation worked as expected in both groups. Average accuracy on the more difficult activites was lower than on easier activities.

To investigate whether LP played a role in self-determined study curriculum we fit the participants’ activity choices with a simple softmax choice model in which the utility of an activity was a linear combination of PC and LP:

$$U_{i,t}=w_{PC}\times P{C}_{i,t}+w_{LP}\times L{P}_{i,t}$$ 1

PC and LP were dynamically evaluated for each activity $i$ at each trial $t$ based on the recent feedback history. PC was defined as the number of correct guesses over the last 15 trials of activity $i$, and LP was defined as the difference in PC between first versus second parts of the same interval. We fit each participants’ data as a probabilistic (softmax) choice over 4 discrete classes, using maximum likelihood estimation with 3 free parameters - the softmax temperature (capturing choice stochasticity) and weights $w_{PC}$, $w_{LP}$ indicating the extent to which each participant was sensitive to, respectively, PC and LP (REFERENCE NOT FOUND: FLOWERS-RA-2020/label/subsubsec:methods/analysis/modeling).

The bivariate form of the model that included both LP and PC (eq. 1) provided a superior fit to the data (average AIC score of $M=549.023,SD=128.010)$) relative to an alternative model based on random selection ($AIC=693.147$). Importantly, the bivariate model also outperformed univariate models that included only LP or only PC terms in each instruction group (Fig. 6a). Moreover, the bivariate models had the lowest AIC scores in 75.63% of participants, and at least a 2 point lead from the next-best model in 63.13% (average minimum pairwise difference in AIC scores of $21.018,SD=38.276;Z=78,p<.001$, Wilcoxon signed-rank test for repeated measurements). This provides direct evidence that participants are sensitive to LP – a heuristic for the temporal derivative of PC – above and beyond overall error rates.

The fitted bivariate models were successful at qualitatively reproducing time-allocation patterns across learning activities and groups of participants (Fig. 6b). As shown in Fig. 6b, the fitted models captured the behavioral tendencies of individual subgroups in our data.

Next, we examined the fitted model coefficients from the bivariate models. The normalized coefficients ${\widehat{w}}_{PC}$ coefficients were, on average, positive in the IG group and negative in the EG group (IG: $M=0.158,SD=0.730$; EG: $M=-0.325,SD=0.697$; 1-way ANOVA; main effect of instruction, $F(1,318)=36.566,p<.001$) consistent with the fact that the EG group were relatively more attracted to challenging activities with higher error rates. In contrast, the ${\widehat{w}}_{LP}$ coefficients did not differ by instruction (IG: $M=0.095,SD=0.664;EG:M=0.075,SD=0.640$; 1-way ANOVA; main effect of instruction, $F(1,318)=0.073,p=.787$) suggesting that they captured a different aspect of the participants’ choices.

These differences are consistent with computational studies suggesting that sensitivities to PC and LP play distinct roles. A sensitivity to PC may motivate people to learn by steering them away from overly easy activities, while a sensitivity to LP may steer learners away from overly difficult or impossible activities. To examine this hypothesis, we analyzed how the two coefficients correlated with individual preferences for challenging over easier activities when the more challenging activity was, respectively, learnable or unlearnable. Figure 6 shows that while PC helped participant in engaging with more challening activities, LP seemed to have this effect only in the context of A3 vs A1, but not A4 vs A3, suggesting that LP does not lead people to the unlearnability trap as PC does.

8.1.2 Examining the effect of time on subjective judgments of learning dynamics

Participants: Alexandr Ten, Pierre-Yves Oudeyer, Hélène Sauzéon, Maxime Balan.

Our previous work left open the question of what is an optimal time period for which humans can accurately estimate their progress in learning a skill that requires a prolonged exposure or practice time to be mastered. Thus, this ongoing project aims to investigate veridical time scales for estimating progress in competence in humans. For this, we designed a behavioral sensorimotor task that requires some extended amount of time to be learned. The task is presented in the form of a video game, similar to the arcade game Lunar Lander, where the objective is to control a spacecraft and safely land it on the surface. As participants practice the task, we will record their performance (e.g. time to complete a single trial, success rate etc.), which will allow us to examine the accuracy of their subjective judgments about their performance. The subjective judgments of performance, along with different control measures, will be obtained via verbal questionnaires.

Crucially, we control the time frames in which we sollicit the participant's subjective judgments of learning in order examine the link between how much time has passed (5 minutes, 2 days, 5 days) and the accuracy of participants' beliefs about their ongoing progress. We designed a tool which can be used to administer this experiment remotely via the internet. Participants will be asked to log in and complete part of the task on 3 different days, each day consisting the same 3 phases: (1) 5 minute of practice (2) questionnaire (3) optional practice. The last, optional practice phase can be engaged by participants but is not required as part of the experiment. This will let us measure behaviorally, the level of interest / enthusiasm participants have for playing the game, which is hypothesized to be a function of the learning progress they experience.

8.2.1 Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey

Building autonomous machines that can explore open-ended environments, discover possible interactions and autonomously build repertoires of skills is a general objective of artificial intelligence. Developmental approaches argue that this can only be achieved by autonomous and intrinsically motivated learning agents that can generate, select and learn to solve their own problems. In recent years, we have seen a convergence of developmental approaches, and developmental robotics in particular, with deep reinforcement learning (RL) methods, forming the new domain of developmental machine learning. Within this new domain, we review here a set of methods where deep RL algorithms are trained to tackle the developmental robotics problem of the autonomous acquisition of open-ended repertoires of skills. Intrinsically motivated goal-conditioned RL algorithms train agents to learn to represent, generate and pursue their own goals. The self-generation of goals requires the learning of compact goal encodings as well as their associated goal-achievement functions, which results in new challenges compared to traditional RL algorithms designed to tackle pre-defined sets of goals using external reward signals. This project 58 proposes a typology of these methods at the intersection of deep RL and developmental approaches, surveys recent approaches and discusses future avenues.

8.2.2 Intrinsically Motivated Exploration of Learned Goal Spaces

Participants: Adrien Laversanne-Finot, Pierre-Yves Oudeyer.

Finding algorithms that allow agents to discover a wide variety of skills efficiently and autonomously, remains a challenge of Artificial Intelligence. Intrinsically Motivated Goal Exploration Processes (IMGEPs) have been shown to enable real world robots to learn repertoires of policies producing a wide range of diverse effects. They work by enabling agents to autonomously sample goals that they then try to achieve. In practice, this strategy leads to an efficient exploration of complex environments with high-dimensional continuous actions. Until recently, it was necessary to provide the agents with an engineered goal space containing relevant features of the environment. In this article we show that the goal space can be learned using deep representation learning algorithms, effectively reducing the burden of designing goal spaces. Our results pave the way to autonomous learning agents that are able to autonomously build a representation of the world and use this representation to explore the world efficiently. We present experiments in two environments using population-based IMGEPs. The first experiments are performed on a simple, yet challenging, simulated environment. Then, another set of experiments tests the applicability of those principles on a real-world robotic setup, where a 6-joint robotic arm learns to manipulate a ball inside an arena, by choosing goals in a space learned from its past experience. This work was published in 28

8.2.3 GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning

Participants: Grgur Kovač, Adrien Laversanne-Finot, Pierre-Yves Oudeyer.

Autonomous agents using novelty based goal exploration are often efficient in environments that require exploration. However, they get attracted to various forms of distracting unlearnable regions. To solve this problem, Absolute Learning Progress (ALP) has been used in reinforcement learning agents with predefined goal features and access to expert knowledge. This work extends those concepts to unsupervised image-based goal exploration.

We present the GRIMGEP framework: it provides a learned robust goal sampling prior that can be used on top of current state-of-the-art novelty seeking goal exploration approaches, enabling them to ignore noisy distracting regions while searching for novelty in the learnable regions. It clusters the goal space and estimates ALP for each cluster. These ALP estimates can then be used to detect the distracting regions, and build a prior that enables further goal sampling mechanisms to ignore them.

We construct an image based environment with distractors, on which we show that wrapping current state-of-the-art goal exploration algorithms with our framework allows them to concentrate on interesting regions of the environment and drastically improve performances.

This work is available as a preprint in 60 and the source code is available at https://gitlab.com/Grg/grimgep.

8.2.4 Language Augmented Intrinsically Motivated Agents

Participants: Cédric Colas, Tristan Karch, Pierre-Yves Oudeyer.

Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration

In this project, we investigate how autonomous multi-goal reinforcement learning agents can use language as a cognitive tool in order to creatively explore their environment and grow repertoires of skills. We follow a developmental approach inspired by how children learn to manipulate language, using it as a way to represent goals and to make plans in their heads.

We develop an algorithm called IMAGINE 43 enabling an intrinsically motivated agent to build a repertoire of skills only from natural language descriptions given by a Social Partner. In our setup, the agent starts without knowing any potential goal and acts randomly. As it reaches outcomes that are meaningful for the social partner, the social partner provides descriptions of the scene in natural language. The agent then converts these natural descriptions into targetable goals and learns to reach them.

This new learning algorithm offers several benefits over previous intrinsically motivated multi-goal reinforcement learning agents that do not use language to describe goals.

First, using linguistic descriptions as sole supervision helps get rid of the need to define hand-crafted reward functions for each of the reachable goals in the environment. In curious, for instance, the agent needed to have access to the description of each of the goal types as well as their associated reward functions in order to reach them. In IMAGINE, the agent builds its own internal reward function mapping natural language descriptions to binary rewards and uses this signal to train a goal-conditioned policy.

Second, using language to represent goals enables the agent to leverage language compositionality so as to imagine new goals, assembling pieces of descriptions communicated by the social partner in order to form new targetable goals. For instance, consider an agent that received the following descriptions: “Grasp red cat”, “Grow red cat” and “Grasp red plant”. This agent can imagine the goal “Grow red plant” and use it as a target in order to discover new outcomes in its environment. We call this mechanism goal imagination. We argue that goal imagination is key to be able to make creative discoveries because the corresponding targeted behaviors are out of the distribution of the outcomes communicated by the social partner. This sort of out-of-distribution goal generation can only be achieved with goals represented as language.

We carried out experiments in order to evaluate the benefits from goal imagination in intrinsically motivated learning. Experiments are split into two phases. In the first one, the agent interacts with the social partners, collects descriptions of goals and stores them in a set of known goal descriptions. The agent uses these descriptions paired with its observations in order to learn an internal reward function that detects when the goal represented by the descriptions are achieved in a given scene. Once this internal reward function is obtained, the agent uses its output (the reward signal) in order to train a goal-conditioned policy enabling it to reach any goal.

In the second phase, the social partner disappears and the agent starts imagining new goals by composing the descriptions stored in the set of known goals. The agent then targets these new goals and by doing so, discovers new interactions. This creative goal exploration process can only be efficient if imagined goal descriptions have a sufficient probability to be meaningful in the environment. As a result, we leveraged the construction grammar framework used to model child language acquisition with discovery of word equivalence classes in order to make sure that imagined goals follow the same construction rules as the descriptions communicated by the social partner. It is also important to note, that in order for goal imagination to work, the internal reward function trained from the social partner’s description must generalize. In other words, it should be able to detect if imagined goals are reached without receiving any new description from the social partner. To this end, we developed an object-factored learning architecture coupled with attention mechanisms 47 that facilitates generalization to new descriptions.

Finally, we measured the success rate of agents on a wide set of different skills and observed that agents that do not imagine goals (that stop at phase 1) master a smaller set of skills than agents that do imagine goals.

Grounding Language to Autonomously-Acquired Skills via Goal Generation

We are interested in the autonomous acquisition of repertoires of skills. Language-conditioned reinforcement learning (lc-rl) approaches are great tools in this quest, as they allow us to express abstract goals as sets of constraints on the states. However, most lc-rl agents are not autonomous and cannot learn without external instructions and feedback. Besides, their direct language condition cannot account for the goal-directed behavior of pre-verbal infants and strongly limits the expression of behavioral diversity for a given language input. To resolve these issues, we propose a new conceptual approach to language-conditioned rl: the Language-Goal-Behavior architecture (lgb). lgb decouples skill learning and language grounding via an intermediate semantic representation of the world—see Figure 13. To showcase the properties of lgb, we present a specific implementation called decstr. decstr is an intrinsically motivated learning agent endowed with an innate semantic representation describing spatial relations between physical objects–see Figure 14. In a first stage (g$\to$b), it freely explores its environment and targets self-generated semantic configurations. In a second stage (l$\to$g), it trains a language-conditioned goal generator to generate semantic goals that match the constraints expressed in language-based inputs. We showcase the additional properties of lgb w.r.t. both an end-to-end lc-rl approach and a similar approach leveraging non-semantic, continuous intermediate representations. Intermediate semantic representations help satisfy language commands in a diversity of ways, enable strategy switching after a failure and facilitate language grounding. This project led to a publication in the ICLR conference proceeding 56, 32.

8.2.5 Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy

Participants: Nicolas Duminy, Sao Mai Nguyen, Junshuai Zhu, Dominique Duhaut, Jerome Kerdreux.

In open-ended continuous environments, robots need to learn multiple parameterised control tasks in hierarchical reinforcement learning. We hypothesise that the most complex tasks can be learned more easily by transferring knowledge from simpler tasks, and faster by adapting the complexity of the actions to the task. We propose a task-oriented representation of complex actions, called procedures, to learn online task relationships and unbounded sequences of action primitives to control the different observables of the environment. Combining both goal-babbling with imitation learning, and active learning with transfer of knowledge based on intrinsic motivation, our algorithm self-organises its learning process. It chooses at any given time a task to focus on; and what, how, when and from whom to transfer knowledge. We show with a simulation and a real industrial robot arm, in cross-task and cross-learner transfer settings, that task composition is key to tackle highly complex tasks. Task decomposition is also efficiently transferred across different embodied learners and by active imitation, where the robot requests just a small amount of demonstrations and the adequate type of information. The robot learns and exploits task dependencies so as to learn tasks of every complexity.

This work lead to a publication in MDPI Applied Siences 45.

8.3.1 Relational inductive biases for recognizing configurations of objects

Since the structure "objects + relations" is naturally present in the world, a good idea is to implement it into the neural networks we are training. Set structures can be used for representing collections of objects, and the Deep Set architecture is well-suited for learning on sets. Graph structures can be used for representing collections of objects and their relations; the Graph Neural Network (GNN) family is well-suited for learning on graphs. Additionnally, we should observe differences between performance and sample efficiency of architectures having only the object-centered prior versus the ones that have the object-centered and relational priors in tasks that require processing of relational information.

We have tested this hypothesis in the case of learning to recognize spatial configurations of symbolic objects. For this purpose, we have created a benchmark dataset called SpatialSim that defines two tasks. The first task, called Identification, is learning to recognize a reference configuration of objects (up to an affine transformation) from a scene with the same objects but with their positions randomly reshuffled. The second task, called Comparison, consists in comparing two different configurations of objects and deciding if they are the same (up to an affine transformation).

In this context, we have trained architectures implementing increasing levels of relational computation: Deep Sets, Recurrent Deep Sets and Message-Passing GNNs. We have observed that the models with more relational computation perform better, especially in the Comparison task where Deep Set performance is very poor. This suggests that relational models are crucial for learning to compare configurations of objects.

This work has been presented as a spotlight talk at the Bridge Between Perception and Reasoning, Graph Neural Networks and Beyond workshop at ICLR 2020 63.

8.3.2 Extracting object representations from images

The previous work was concerned with symbolic objects described by their features such as position, orientation, etc. In a realistic setting we need to be able to learn to extract these object representations directly from raw images in an unsupervised representation learning scheme, and in a disentangled manner, such that each object is represented by a unique vector, and that each of that vector's coordinates represents a unique factor of variation (such as x or y position, color, etc). In the best case, this would recover the symbolic representations such as the ones used in the approach above.

Two architectures for object-centered unsupervised representation learning have been investigated: MONet 79 (an object-based variational autoencoder) and Contrastive-Structured World Models 108 (an architecture learning to extract objects from images by learning a world model expressed as an interaction graph). Integrating these approaches (along with mechanisms for object permanence) into an intrinsically motivated deep RL setting is still ongoing work.

8.3.3 In language-conditioned Deep RL agents

The impact of object-centered architectures in a deep RL setting has also been investigated. We have benchmarked their importance in the language-imagination deep rl setting given in 8.2.4. We have observed dramatic improvements in sample efficiency in this setting when we use Deep Sets as opposed to flat, unstructured architectures (such as regular Multi-Layer Perceptrons).

In addition to that, we observe increased generalization performance in this setting (see Figures 15 and 16), suggesting that the bias that all objects should be represented and processed in the same way (and the weight-sharing that is implied by this bias in the neural networks) is helpful for transferring skills across objects.

These object-based architectures are robust to the number of objects, contrary to their flat counterparts. Additionally, architectures that present biases for encoding relations between objects demonstrate increased performance in tasks that require interaction between objects, such as grasping objects that are identified by their position relative to another object.

This work was presented at the Beyond Tabula Rasa in RL ICLR 2020 workshop 47.

8.4.1 Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments

Participants: Remy Portelas, Katja Hoffman, Pierre-Yves Oudeyer.

In this work we considered the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments. To do so, we studied how a teacher algorithm can learn to generate a learning curriculum, whereby it sequentially samples parameters controlling a stochastic procedural generation of environments. Because it does not initially know the capacities of its student, a key challenge for the teacher is to discover which environments are easy, difficult or unlearnable, and in what order to propose them to maximize the efficiency of learning over the learnable ones. To achieve this, this problem is transformed into a surrogate continuous bandit problem where the teacher samples environments in order to maximize absolute learning progress of its student. We presented ALP-GMM (see figure 17), a new algorithm modeling absolute learning progress with Gaussian mixture models. We also adapted existing algorithms and provided a complete study in the context of DRL. Using parameterized variants of the BipedalWalker environment, we studied their efficiency to personalize a learning curriculum for different learners (embodiments), their robustness to the ratio of learnable/unlearnable environments, and their scalability to non-linear and high-dimensional parameter spaces. Videos and code are available at https://github.com/flowersteam/teachDeepRL.

Overall, this work demonstrated that LP-based teacher algorithms could successfully guide DRL agents to learn in difficult continuously parameterized environments with irrelevant dimensions and large proportions of unfeasible tasks. With no prior knowledge of its student's abilities and only loose boundaries on the task space, ALP-GMM, our proposed teacher, consistently outperformed random heuristics and occasionally even expert-designed curricula (see figure 18). This work was presented at CoRL 2019 138.

ALP-GMM, which is conceptually simple and has very few crucial hyperparameters, opens-up exciting perspectives inside and outside DRL for curriculum learning problems. Within DRL, it could be applied to previous work on autonomous goal exploration through incremental building of goal spaces 111. In this case several ALP-GMM instances could scaffold the learning agent in each of its autonomously discovered goal spaces. Another domain of applicability is assisted education, for which current state of the art relies heavily on expert knowledge 83 and is mostly applied to discrete task sets.

8.4.2 Meta Automatic Curriculum Learning

Participants: Remy Portelas, Clement Romac, Katja Hoffman, Pierre-Yves Oudeyer.

In this work we identified that a major challenge in the Deep RL (DRL) community is to train agents able to generalize their control policy over situations never seen in training. Training on diverse tasks has been identified as a key ingredient for good generalization, which pushed researchers towards using rich procedural task generation systems controlled through complex continuous parameter spaces. In such complex task spaces, it is essential to rely on some form of Automatic Curriculum Learning (ACL) to adapt the task sampling distribution to a given learning agent, instead of randomly sampling tasks, as many could end up being either trivial or unfeasible. Since it is hard to get prior knowledge on such task spaces, many ACL algorithms explore the task space to detect progress niches over time, a costly tabula-rasa process that needs to be performed for each new learning agents, although they might have similarities in their capabilities profiles.

To address this limitation, we introduced the concept of Meta-ACL (see fig. 19, and formalized it in the context of black-box RL learners, i.e. algorithms seeking to generalize curriculum generation to an (unknown) distribution of learners. We then presented AGAIN (see fig. 10), a first instantiation of Meta-ACL, and showcased its benefits for curriculum generation over classical ACL in multiple simulated environments including procedurally generated parkour environments with learners of varying morphologies. Videos and code are available at https://sites.google.com/view/meta-acl

This work is available as preprint 62 and will be submitted to ICML 2021. In future work, AGAIN could be improved by using adaptive approaches to build compact pre-test sets, e.g. using decision tree based test pruning methods, or by combining curriculum priors from multiple previously trained learners. While AGAIN is built on top of an existing ACL algorithm, developing an end-to-end Meta-ACL algorithm that generates curricula using a DRL teacher policy trained across multiple students is also a promising line of work to follow. Additionally, this work opens-up exciting new perspectives in transferring Meta-ACL methods to educational data-mining, e.g. in MOOC scenarios, given a previously trained pilot classroom, one could use Meta-ACL to infer adaptive curricula for new students.

8.4.3 Automatic Curriculum Learning for Deep RL in Procedural Task Spaces: a Benchmark

Participants: Clement Romac, Remy Portelas, Katja Hoffman, Pierre-Yves Oudeyer.

Training autonomous agents able to generalize to a multiplicity of environments/tasks is a key desiderata in current Deep Reinforcement Learning (DRL) research. In parallel to searching for DRL architectures able to learn sets of tasks, many works on Automatic Curriculum Learning (ACL) studied how to emancipate from the usual random task curriculum and instead use teacher algorithms to adapt task selection to the evolving abilities of black-box DRL agents. While multiple standard benchmarks exist to compare DRL agents, there is currently no such thing for ACL algorithms, which makes comparing existing approaches difficult, as too many experimental parameters differ from paper to paper.

In this work, we identified the following key challenges faced by ACL algorithms:

• Mostly unfeasible task spaces - Given the task space is large, as in PCG-encoding task spaces, there might be a predominant amount of unfeasible tasks (or at least initially unfeasible). A good teacher algorithm must have the ability to quickly detect and exploit promising task subspaces for its learner.
• Mostly trivial task spaces - On the contrary, w.r.t. a given student, the task space might be mostly trivial. In that case the teacher has to efficiently detect and exploit the small portion of subspaces of relevant difficulty.
• Forgetting students - DRL learners are prone to catastrophic forgetting, i.e. to overwrite important skills while training new ones. This has to be detected and dealt with by the teacher for optimal curriculum generation.
• Robustness to diverse students - Being able to adapt curriculum generation to diverse students is an important desiderata to ensure a given ACL mechanism has good chances to transfer to novel scenarios.
• Rugged difficulty landscapes - Another important property for ACL algorithms is to be able to deal with task spaces for which the optimal curriculum is not a smooth task distribution sampling drift across the space but rather a series of distribution jumps, e.g. as in complex PCG-task spaces.
• Working with no or few expert knowledge - Prior knowledge over a task space w.r.t. a given student is a costly information gathering process that needs to be repeated for each new problem/student. Relying on as little expert knowledge as possible is therefore a desirable property for ACL algorithms (especially if aiming for out-of-the-lab applications).

Based on these, we presented TeachMyAgent (TA), a benchmark of current ACL algorithms including 1) skill-specific unit-tests using variants of a procedural Box2D bipedal walker environment, and 2) a new procedural Parkour environment combining most ACL challenges, making it ideal for global performance assessment. We then leveraged TeachMyAgent to conduct a comparative study of existing approaches, showcasing the competitiveness of expert-knowledge-free ACL approaches, and showing that our Parkour environment remains an open problem.

Challenge-specific comparison with Stump Tracks

In order to propose a comparison of the different ACL methods on each of the challenges introduced above, we leveraged the Stump Tracks environment introduced in 138 to create five experiments, each of them designed to highlight the ability of a teacher in one the first five ACL challenges. Additionnaly, in order to analyse separately the expert knowledge challenge, we performed each of the experiments above in three prior knowledge conditions:

• No expert knowledge
• Low expert knowledge
• High expert knowledge

Using these 15 experiments, we compared 7 ACL methods in addition of a baseline teacher sampling tasks uniformly random over the task space. In each experiment, a DRL student is trained for 20 millions steps using an ACL algorithm to set the procedural generation of the environment at every episode. We monitored the DRL student's performances on a pre-defined test set composed of 100 tasks every 500000 steps and reported the average percentage of mastered tasks (i.e. task on which the agent obtained an episodic reward greater than 230). Results are gathered in figure 21 and presented as an order of magnitude of the Random teacher (using 32 seeds per experiment).

While these results highlighted the strenghs and weaknesses of each method (e.g. the inability of ADR to deal with rugged difficulty landscapes or the inertia of GoalGAN in adapting the curriculum to forgetting students), it also showed how competitive expert-knowledge-free methods like ALP-GMM are, even when compared to methods having access to a high amount of prior knowledge.

Global performances analysis using the Parkour

We then assessed the different ACL methods' performances on our Box2D Parkour track environment (see figure 22) which features most of the previously discussed ACL challenges: 1) most tasks are unfeasible, 2) Before each run, unknown to the teacher, the student's embodiment is uniformly sampled among three morphologies (bipedal walker, fish and chimpanzee), requiring the teacher to adapt curriculum generation to its current student's abilities, and 3) tasks are generated through a CCPN-based PCG, creating a rich task space with rugged difficulty landscape and hardly-definable prior knowledge.

We trained a DRL student for 20 millions steps with 48 different seeds (16 per morphology) and monitored the percentage of mastered tasks as in our Stump Tracks experiments. As few expert knowledge is accessible, our results (figure 23) showed to poor performances from expert-knowledge-depend teachers (e.g. ADR or SPDL). Additionally, overall results awerere quite low, especially for the seeds using our chimpanzee embodiment, where none of the ACL methods managed to master more than $1\%$ of the test set. This thus leaves our Parkour track as an open challenge for future design of ACL methods.

8.4.4 Other

8.5.1 Grounding Artificial Intelligence in the Origins of Human Behavior

Participants: Clément Moulin-Frier, Eleni Nisioti, Julius Taylor.

An inter-disciplinary dialogue between AI and HBE

As a first step in our project, we conducted a targeted yet extensive literature review on HBE, in particular works studying the effect that climate complexity has had on the emergence of adaptability, cooperation and cultural repertoire in human evolution. In parallel, we have reviewed the state-of-the-art in the study of open-ended skill acquisition in, in particular, the AI sub-fields of multi-agent reinforcement and meta reinforcement learning. We have compiled our review in a position paper that summarizes the project's objectives 61. An important objective at this stage was to justify the proposed exchange of ideas between the two fields by identifying their commonalities in terms of research challenges. In Figure 24, we introduce a conceptual framework that recognizes important ecological components, as well as the feedforward and feedback links that relate them. This figure was presented in our preprint 61.

8.6.1 Machine Learning for Adaptive Personalization in Intelligent Tutoring Systems

Participants: Pierre-Yves Oudeyer, Benjamin Clément, Didier Roy, Hélène Sauzeon.

8.6.2 Machine learning for adaptive cognitive training

Participants: Pierre-Yves Oudeyer, Hélène Sauzéon, Masataka Sawayama, Benjamin Clément, Maxime Adolphe.

Because of its cross-cutting nature to all cognitive activities such as learning tasks, attention is a hallmark of good cognitive health throughout life and more particularly in the current context of societal crisis of attention. Recent works have shown the great potential of computerized attention training for an example of attention training, with efficient training transfers to other cognitive activities, and this, over a wide spectrum of individuals (children, elderly, individuals with cognitive pathology such as Attention Deficit and Hyperactivity Disorders). Despite this promising result, a major hurdle is challenging: the high inter-individual variability in responding to such interventions. Some individuals are good responders (significant improvement) to the intervention, others respond variably, and finally some respond poorly, not at all, or occasionally. A central limitation of computerized attention training systems is that the training sequences operate in a linear, non-personalized manner: difficulty increases in the same way and along the same dimensions for all subjects. However, different subjects require in principle a progression at a different, personalized pace according to the different dimensions that characterize attentional training exercises.

To tackle the issue of inter-individual variability, the present project proposes to apply some principles from intelligent tutorial systems (ITS) to the field of attention training. In this context, we have already developed automatic curriculum learning algorithms such as those developed in the KidLearn project, which allow to customize the learner's path according to his/her progress and thus optimize his/her learning trajectory while stimulating his/her motivation by the progress made. ITS are widely identified in intervention research as a successful way to address the challenge of personalization, but no studies to date have actually been conducted for attention training. Thus, whether ITS, and in particular personalization algorithms, can optimize the number of respondents to an attention training program remains an open question.

To investigate this question, a web platform has been designed for planning and implementing remote behavioural studies. This tool provides means for registering recruited participants remotely and executing complete experimental protocols: from presenting instructions and obtaining informed consents, to administering behavioural tasks and questionnaires, potentially throughout multiple sessions spanning days or weeks. As a result, several studies using this tool will be conducted during the following months.

8.6.3 Interactive systems that foster curiosity for education

Participants: Pierre-Yves Oudeyer, Hélène Sauzéon, Mehdi Alami, Rania Abdelghani, Didier Roy, Edith Law.

Since 2019 via the renewal of the Idex cooperation fund (between the University of Bordeaux and the University of Waterloo, Canada) led by the Flowers team and also involving F. Lotte from the Potioc team, we continue our work on the development of new curiosity-driven interaction systems. Although experimentations have been slowed down by sanitary conditions, progress has been made in this area of application of FLOWERS works. In particular, three studies have been completed.

The first study regards a new interactive educational application to foster curiosity-driven question-asking in children. This study has been performed during the Master 2 internship of Mehdi Alaimi co-supervised by H. Sauzéon, E. Law and PY Oudeyer. It addresses a key challenge for 21st-century schools, i.e., teaching diverse students with varied abilities and motivations for learning, such as curiosity within educational settings. Among variables eliciting curiosity state, one is known as « knowledge gap », which is a motor for curiosity-driven exploration and learning. It leads to question-asking which is an important factor in the curiosity process and the construction of academic knowledge. However, children questions in classroom are not really frequent and don’t really necessitate deep reasoning. Determined to improve children’s curiosity, we developed a digital application aiming to foster curiosity-related question-asking from texts and their perception of curiosity. To assess its efficiency, we conducted a study with 95 fifth grade students of Bordeaux elementary schools. Two types of interventions were designed, one trying to focus children on the construction of low-level question (i.e. convergent) and one focusing them on high-level questions (i.e. divergent) with the help of prompts or questions starters models. We observed that both interventions increased the number of divergent questions, the question fluency performance, while they did not significantly improve the curiosity perception despite high intrinsic motivation scores they have elicited in children. The curiosity-trait score positively impacted the divergent question score under divergent condition, but not under convergent condition. The overall results supported the efficiency and usefulness of digital applications for fostering children’s curiosity that we need to explore further. The overall results are published in CHI'20 33.

The second study investigates the neurophysiological underpinnings of curiosity and the opportunities of their use for Brain-computer interactions 34. Understanding the neurophysiological mechanisms underlying curiosity and therefore being able to identify the curiosity level of a person, would provide useful information for researchers and designers in numerous fields such as neuroscience, psychology, and computer science. A first step to uncovering the neural correlates of curiosity is to collect neurophysiological signals during states of curiosity, in order to develop signal processing and machine learning (ML) tools to recognize the curious states from the non-curious ones. Thus, we ran an experiment in which we used electroencephalography (EEG) to measure the brain activity of participants as they were induced into states of curiosity, using trivia question and answer chains. We used two ML algorithms, i.e. Filter Bank Common Spatial Pattern (FBCSP) coupled with a Linear Discriminant Algorithm (LDA), as well as a Filter Bank Tangent Space Classifier (FBTSC), to classify the curious EEG signals from the non-curious ones. Global results indicate that both algorithms obtained better performances in the 3-to-5s time windows, suggesting an optimal time window length of 4 seconds to go towards curiosity states estimation based on EEG signals.

Finally, the third study investigates the role of intrinsic motivation in spatial learning in children (paper in progress). In this study, the state curiosity is manipulated as a preference for a level of uncertainty during the exploration of new environments. To this end, a series of virtual environments have been created and is presented to children. During encoding, participants explore routes in environments according the three levels of uncertainty (low, medium, and high), thanks to a virtual reality headset and controllers and, are later asked to retrace their travelled routes . The exploration area and the wayfinding. ie the route overlap between encoding and retrieval phase, (an indicator of spatial memory accuracy) are measured. Neuropsychological tests are also performed. Preliminary results showed that there are better performances under the medium uncertainty condition in terms of exploration area and wayfinding score. These first results supports the idea that curiosity states are a learning booster.

At the end of 2020, we started an industrial collaboration project with EvidenceB on this topic (CIFRE contract of Rania Abdelghani currently submitted to the ANRT). The overall objective of the thesis is to propose new educational technologies driven by epistemic curiosity, and allowing children to express themselves more and learn better. To this end, a central question of the work will be to specify the impact of self-questioning aroused by states of curiosity about student performance. Another objective will be to create and study the pedagogical impact of new educational technologies in real situations (schools) promoting an active education of students based on their curiosity.

8.6.4 A computer science and robotics integration model for primary school

Participants: Didier Roy.

Integrating computer science (CS) into school curricula has become a worldwide preoccupation. Therefore, we present a CS and Robotics integration model and its validation through a large-scale pilot study in the administrative region of the Canton Vaud in Switzerland. Approximately 350 primary school teachers followed a mandatory CS continuing professional development program (CPD) of adapted format with a curriculum scaffolded by instruction modality. This included CS Unplugged activities that aim to teach CS concepts without the use of screens, and Robotics Unplugged activities that employed physical robots, without screens, to learn about robotics and CS concepts. Teachers evaluated positively the CPD and their representation of CS improved. Voluntary adoption rates reached 97 percent during the CPD and 80 percent the following year. These results combined with the underpinning literature support the generalisability of the model to other contexts. This work was published in 23 and led by our colleagues at EPFL.

8.6.5 How An Automated Gesture Imitation Game Can Improve Social Interactions With Teenagers With ASD

Participants: Linda Nanan Vallée, Sao Mai Nguyen, Christophe Lohr, Ioannis Kanellos, Olivier Asseu.

With the outlook of improving communication and social abilities of people with ASD, we propose to extend the paradigm of robot-based imitation games to ASD teenagers. In this paper 52, we present an interaction scenario adapted to ASD teenagers, propose a computational architecture using the latest machine learning algorithm Openpose for human pose detection, and present the results of our basic testing of the scenario with human caregivers. These results are preliminary due to the number of session (1) and participants (4). They include a technical assessment of the performance of Openpose, as well as a preliminary user study to confirm our game scenario could elicit the expected response from subjects.

8.7.1 Curiosity-driven Learning for Automated Discovery of Physico-Chemical Structures

Participants: Chris Reinke, Mayalen Etcheverry, Pierre-Yves Oudeyer.

Introduction

Intrinsically motivated goal exploration algorithms (IMGEPs) enable machines to discover repertoires of action policies that produce a diversity of effects in complex environments. In robotics, these exploration algorithms have been shown to allow real world robots to acquire skills such as tool use 9170. In other domains such as chemistry and physics, they open the possibility to automate the discovery of novel chemical or physical structures produced by complex dynamical systems 137. However, they have so far assumed that self-generated goals are sampled in a specifically engineered feature space, limiting their autonomy. Recent work has shown how unsupervised deep learning approaches could be used to learn goal space representations 136 but they have used precollected data to learn the representations. This project studies how IMGEPs can be extended and used for automated discovery of behaviours of dynamical systems in physics or chemistry without using assumptions or knowledge about such systems.

As a first step towards this goal we choose Lenia 80, a simulated high-dimensional complex dynamical system, as a target system. Lenia is a continuous cellular automaton where diverse visual structures can self-organize (Fig.26, c). It consists of a two-dimensional grid of cells $A\in {[0,1]}^{256\times 256}$ where the state of each cell is a real-valued scalar activity $A^t\left(x\right)\in [0,1]$. The state of cells evolves over discrete time steps $t$. The activity change is computed by integrating the activity of neighbouring cells. Lenia's behavior is controlled by its initial pattern $A^{t=1}$ and several settings that control the dynamics of the activity change. Lenia can produce diverse patterns with different dynamics. Most interesting, spatially localized coherent patterns that resemble in their shapes microscopic animals can emerge. Our goal was to develop methods that allow to explore a high diversity of such animal patterns.

We could successfully accomplish this goal 38 based on two key contributions of our research: 1) the usage of compositional pattern producing networks (CPPNs) for the generation of initial states for Lenia, and 2) the development of a novel IMGEP algorithm that learns goal representations online during the exploration of the system.

Results

The diversity of identified patterns in the analytic behavior space show that IMGEP approaches with learned goal spaces via VAEs (PGL, OGL) could identify the highest diversity of patterns overall (Fig. 27, a). They were followed by the IMGEP with a hand-defined goal space (HGS). The lowest performance had the random exploration and the IMGEP with a random goal space (RGS). The advantage of learned goals space approaches (PGL, OGL) over all other approaches was even stronger for the diversity of animal patterns, i.e. the main goal of our exploration (Fig. 27, b).

(a) Diversity of all identified Patterns	(b) Diversity of Animal Patterns

8.7.2 Hierarchically Organized Latent Modules for Meta-Diversity Search in Morphogenetic Systems

Participants: Mayalen Etcheverry, Clément Moulin-Frier, Pierre-Yves Oudeyer.

In the previous paper 38, the problem of automated diversity-driven discovery in morphogenetic systems was introduced, highlighting that two key ingredients are autonomous exploration and unsupervised representation learning to describe "relevant" degrees of variations in the patterns. Yet, standard diversity-driven approaches assume that the intuitive notion of diversity can be captured within a single behavioral characterization (BC) space.

In this project, we follow the proposed experimental testbed of Reinke et al.(2020) 38 on a continuous game-of-life system (Lenia, 80). We provide empirical evidence that the discoveries of an IMGEP operating in a monolithic BC space are highly-diverse in that space, yet tend to be poorly-diverse in other potentially-interesting BC spaces (see Figure 28). This draws several limitations when it comes to applying such system as a tool for assisting discovery in morphogenetic system, as the suggested discoveries are unlikely to align with the interests of a end-user.

To address these limits, the contributions of this project are threefold. First, we formulate the problem of meta-diversity search as follows: an artificial “discovery assistant” incrementally learns a set of diverse BC spaces in an outer loop; and searches to discover diverse patterns within each of them in an inner loop. With minimal external feedback, a successful discovery assistant should be able to efficiently specialize the exploration strategy toward a particular type of diversity, corresponding to the initially unknown preferences of the human evaluator.

Second, we present HOLMES , a dynamic and modular model architecture for unsupervised learning of diverse representations where a hierarchy of module embedding networks is actively expanded. Additionally, we present IMGEP-HOLMES (see Figure 29) which extends the standard IMGEP framework by replacing the monolithic representation with the proposed hierarchy. We show that the hierarchical structure allows the IMGEP agent to target goals in the different nodes in order to achieve diversity in each BC space.

Finally, we show how this architecture can easily be leveraged to drive exploration, opening interesting perspectives for the integration of a human in the loop.

To conclude, this work shows that integrating flexible modular representation learning with intrinsically-motivated goal exploration processes for meta-diversity search are very promising directions in the context of automated discovery in morphogenetic systems. As an example, IMGEP-HOLMES was able to discover many types of solitons including unseen pattern-emitting lifeforms in less than 15000 training steps without guidance, when their existence remained an open question raised in the original Lenia paper 80.

Initial version of this work was presented at ICLR 2021 workshop "Beyond tabula rasa in Reinforcement Learning" 46. The final version of this work is published at NeurIPS 2020 35. The project website with videos and additional results can be found at http://mayalenE.github.io/holmes/, and the source code is available at http://mayalenE.github.io/holmes/.

8.7.3 Automated exploration of neuro-mechanical models or arms using goal exploration algorithms

Participants: Pierre-Yves Oudeyer.

This work was led by Daniel Cattaert, Aymar de Rugy and their collaborators at Incia, with contributions from Pierre-Yves Oudeyer.

Objective. Neuro-mechanical models are essential to increase our understanding of the fundamental mechanisms underlying natural sensorimotor control, and to foster robotic designs using them. Yet, the complexity of those models is such that current optimization methods are unsuited to establish the range of useful behaviors they could produce, and their associated parameter settings. Our goal is to provide both using recent advances in developmental machine learning. Approach. We designed a simplified neuro-mechanical model that nevertheless has the complexity that make current optimization fail. This model consists of a single (elbow) joint actuated by two muscles and their associated spindles, alpha and gamma motoneurons, receiving simple (non-dynamic) step commands. To establish the range of movements this system is capable of doing, a goal exploration process was used that built a repertoire of valid actions through iterative sampling of target behaviors, combined with stochastic variation on the parameter settings that elicited their closest behaviors in this repertoire. Results obtained with this process were compared to those obtained with alternative optimization methods. Main results. The goal exploration was found to widely outperform optimization methods in terms of its capacity to rapidly establish a repertoire of valid actions, and to find a large range of behaviors not otherwise found. The resulting repertoire also provides diverse parameter sets for any given actions, akin to what is observed in natural control. Families of solutions originating from few initial seeds should also be exploitable to generate novel behaviors through interpolation. Significance. The proposed method provides rich perspectives to explore the structure and settings of lower-level neural circuitry, and their associated descending commands, to produce a wide range of useful behaviors. Comparison of behavioral space obtained after selective manipulation of various elements of neuro-mechanical models should also help understand natural control, and promote its emulation in robotics. We have written an article under review.

8.7.4 Design of an Interactive Software for Automated Discovery in Complex Systems

Participants: Clément Romac, Mayalen Etcheverry, Clément Moulin-Frier, Pierre-Yves Oudeyer.

Additionally, 35 also showed that adding human in the exploration loop can be a key to obtain interesting mappings. Designing interactive algorithms is thus an important step towards the adoption of automated exploration and discovery of complex systems, as users previously using hand-made heuristics would still need to add their expert knowledge in the exploration process.

Following these, we propose to design an interactive software which aims to provide tools to easily use exploration algorithms (e.g. curiosity-driven) in various systems. Several challenges are to be faced in this project among the possibility to use any complex system (numeric or physical), the need of a scalable architecture or having an user-friendly interface with efficient and modular visualisation tools.

We propose to use a microservice architecture and leverage Docker to make the software easily installable and modifiable by non-computer scientist users. We separate the front-end application on which the user will create experiments and interact with them from the automated discovery process (making the scalability issues easier to deal). We choose to use Python for Machine Learning code (as it offers a large community and efficient tools) as well as recent web tools (e.g. Angular) to provide user-friendly interfaces. See figure 30 for an overview of the functional architecture of our software.

In the context of the project 8.7, we started a collaboration with Bert Chan, an independant researcher on Artificial Life and author of the Lenia system 80. During this collaboration, Bert Chan will help us design versions of IMGEP usable by scientists (non ML-experts) end-users, which is the aim of project 8.7.4. Having himself created the Lenia system, he is highly-interested to use our algorithms to automatically explore the space of possible emerging structures and will provide valuable insights into end-user habits and concerns. Additionally, we will be working with him to expand the set of discoveries of possible structures in continuous CAs, as a continuation of the project 8.7.2.

8.8.1 Explainable Deep Learning

Participants: Natalia Díaz Rodríguez, Adrien Bennetot.

Together with Segula Technologies and Sorbonne Université, ENSTA Paris has been working on eXplainable Artificial Intelligence (XAI) in order to make machine learning more interpretable. While opaque decision systems such as Deep Neural Networks have great generalization and prediction skills, their functioning does not allow obtaining detailed explanations of their behaviour. The objective is to fight the trade-off between performance and explainability by combining connectionist and symbolic paradigms 74.

Broad consensus exists on the importance of interpretability for AI models. However, since the domain has only recently become popular, there is no collective agreement on the different definitions and challenges that constitute XAI. The first step is therefore to summarize previous efforts made in this field. We presented a taxonomy of XAI techniques in 19 and we are currently working on a prediction model that generates itself an explanation of its rationale in natural language while keeping performance as close as possible to the the state of the art 74.

8.8.2 Knowledge engineering tools for neural-symbolic learning

Participants: Natalia Díaz Rodríguez, Adrien Bennetot.

Symbolic artificial intelligence methods are experiencing a come-back in order to provide deep representation methods the explainability they lack. In this area, a survey on RDF stores to handle ontology-based triple databases has been contributed 106, as well as the use of neural-symbolic tools that aim at integrating both neural and symbolic representations 74.

8.8.3 Explainability in Deep Reinforcement Learning

Participants: Alexandre Heuillet, Fabien Couthouis, Natalia Díaz-Rodríguez.

A large set of the explainable Artificial Intelligence (XAI) literature is emerging on feature relevance techniques to explain a deep neural network (DNN) output or explaining models that ingest image source data. However, assessing how XAI techniques can help understand models beyond classification tasks, e.g. for reinforcement learning (RL), has not been extensively studied. With this project 26, we review recent works in the direction to attain Explainable Reinforcement Learning (XRL), a relatively new subfield of Explainable Artificial Intelligence, intended to be used in general public applications, with diverse audiences, requiring ethical, responsible and trustable algorithms. In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box. We evaluate mainly studies directly linking explainability to RL, and split these into two categories according to the way the explanations are generated: transparent algorithms and post-hoc explainaility. We also review the most prominent XAI works from the lenses of how they could potentially enlighten the further deployment of the latest advances in RL, in the demanding present and future of everyday problems. We published this review

8.8.4 Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI

Participants: Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, Raja Chatila, Francisco Herrera.

In the last years, Artificial Intelligence (AI) has achieved a notable momentum that may deliver the best of expectations over many application sectors across the field. For this to occur, the entire community stands in front of the barrier of explainability, an inherent problem of AI techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI. Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is acknowledged as a crucial feature for the practical deployment of AI models. This overview 19 examines the existing literature in the field of XAI, including a prospect toward what is yet to be reached. We summarize previous efforts to define explainability in Machine Learning, establishing a novel definition that covers prior conceptual propositions with a major focus on the audience for which explainability is sought. We then propose and discuss about a taxonomy of recent contributions related to the explainability of different Machine Learning models, including those aimed at Deep Learning methods for which a second taxonomy is built. This literature analysis serves as the background for a series of challenges faced by XAI, such as the crossroads between data fusion and explainability. Our prospects lead toward the concept of Responsible Artificial Intelligence, namely, a methodology for the large-scale implementation of AI methods in real organizations with fairness, model explainability and accountability at its core. Our ultimate goal is to provide newcomers to XAI with a reference material in order to stimulate future research advances, but also to encourage experts and professionals from other disciplines to embrace the benefits of AI in their activity sectors, without any prior bias for its lack of interpretability.

8.8.5 Interdisciplinary Research in Artificial Intelligence: Challenges and Opportunities

Participants: Remy Kusters, Dusan Misevic, Hugues Berry, Antoine Cully, Yann Le Cunff, Loic Dandoy, Natalia Díaz-Rodríguez, Marion Ficher, Jonathan Grizou, Alice Othmani, Themis Palpanas, Matthieu Komorowski, Patrick Loiseau, Clément Moulin-Frier, Santino Nanini, Daniele Quercia, Michele Sebag, Françoise Soulié Fogelman, Sofiane Taleb, Liubov Tupikina, Vaibhav Sahu, Jill-Jênn Vie, Fatima Wehbi.

The use of artificial intelligence (AI) in a variety of researchfields is speeding up multiple digital revolutions, from shifting paradigms in healthcare, precision medicine and wearable sensing, to public services and education offered to the masses around the world, to futurecities made optimally efficient by autonomous driving. When a revolution happens, the consequences are not obvious straight away, and to date, there is no uniformly adaptedframework to guide AI research to ensure a sustainable societal transition. To answer this need, here we analyze three key challenges to interdisciplinary AI research, and deliver three broad conclusions 27: 1) future development of AI should not only impact other scientific domains but should also take inspiration and benefit from other fields of science, 2) AI research must be accompanied by decision explainability, dataset bias transparency aswell as development of evaluation methodologies and creation of regulatory agencies toensure responsibility, and 3) AI education should receive more attention, efforts and innovation from the educational and scientific communities. Our analysis is of interest notonly to AI practitioners but also to other researchers and the general public as it offers ways to guide the emerging collaborations and interactions toward the most fruitful outcomes.

8.8.6 Accessible Cultural Heritage through Explainable Artificial Intelligence

Participants: Natalia Díaz-Rodríguez, Galena Pisoni.

Ethics Guidelines for Trustworthy AI advocate for AI technology that is, among other things, more inclusive. Explainable AI (XAI) aims at making state of the art opaque models more transparent, and defends AI-based outcomes endorsed with a rationale explanation, i.e., an explanation that has as target the non-technical users. XAI and Responsible AI principles defend the fact that the audience expertise should be included in the evaluation of explainable AI systems. However, AI has not yet reached all public and audiences , some of which may need it the most. One example of domain where accessibility has not much been influenced by the latest AI advances is cultural heritage. In this project 44, we propose including minorities as special user and evaluator of the latest XAI techniques. In order to define catalytic scenarios for collaboration and improved user experience, we pose some challenges and research questions yet to address by the latest AI models likely to be involved in such synergy.

8.9.1 Machine Learning Optimization of Intervention Strategies for Epidemics

Participants: Cédric Colas, Clément Moulin-Frier, Pierre-Yves Oudeyer.

This project is a collaboration with the SISTM team from Inria Bordeaux. Modelling the dynamics of epidemics helps proposing control strategies based on pharmaceutical and non-pharmaceutical interventions (contact limitation, lock down, vaccination, etc). Hand-designing such strategies is not trivial because of the number of possible interventions and the difficulty to predict long-term effects. This task can be cast as an optimization problem where state-of-the-art machine learning algorithms such as deep reinforcement learning, might bring significant value. However, the specificity of each domain – epidemic modelling or solving optimization problem – requires strong collaborations between researchers from different fields of expertise. This is why we introduce EpidemiOptim, a Python toolbox that facilitates collaborations between researchers in epidemiology and optimization. EpidemiOptim turns epidemiological models and cost functions into optimization problems via a standard interface commonly used by optimization practitioners ( OpenAI Gym )—see Figure 32. Reinforcement learning algorithms based on Q-Learning with deep neural networks ( dqn ) and evolutionary algorithms ( nsga-ii ) are already implemented. We illustrate the use of EpidemiOptim to find optimal policies for dynamical on-off lock-down control under the optimization of death toll and economic recess using a Susceptible-Exposed-Infectious-Removed ( seir ) model for COVID-19. Using EpidemiOptim and its interactive visualization platform in Jupyter notebooks, epidemiologists, optimization practitioners and others (e.g. economists) can easily compare epidemiological models, costs functions and optimization algorithms to address important choices to be made by health decision-makers. Trained models can be explored by experts and non-experts via a web interface. This led to a submission at the journal JAIR (under review) 57. This project also led to a web interface where users can interact with trained lock-down intervention strategies, look at their effects on a models of the COVID-19 epidemics and design their own intervention strategy: https://epidemioptim.bordeaux.inria.fr/.

8.9.2 Applications in Robotic myoelectric prostheses

Participants: Pierre-Yves Oudeyer, Aymar de Rugy, Daniel Cattaert, Mick Sebastien.

Together with the Hybrid team at INCIA, CNRS (Sébastien Mick, Daniel Cattaert, Florent Paclet, Aymar de Rugy) and Pollen Robotics (Matthieu Lapeyre, Pierre Rouanet), the Flowers team continued to work on a project related to the design and study of myoelectric robotic prosthesis. The ultimate goal of this project is to enable an amputee to produce natural movements with a robotic prosthetic arm (open-source, cheap, easily reconfigurable, and that can learn the particularities/preferences of each user). This will be achieved by 1) using the natural mapping between neural (muscle) activity and limb movements in healthy users, 2) developing a low-cost, modular robotic prosthetic arm and 3) enabling the user and the prosthesis to co-adapt to each other, using machine learning and error signals from the brain, with incremental learning algorithms inspired from the field of developmental and human-robot interaction.

8.9.3 Traffic agent motion prediction

Participants: David Filliat, Vyshakh Palli Thazha.

For a vehicle to navigate autonomously, it needs to perceive its surroundings and estimate the future state of the relevant traffic-agents with which it might interact as it navigates across public road networks. Predicting the future state of the perceived entities is a challenge, as these might appear to move in a stochastic manner. However, their motion is constrained to an extent by context, in particular the road network structure. Conventional machine learning methods are mainly trained using data from the perceived entities without considering roads, as a result trajectory prediction is difficult. In this paper, the notion of maps representing the road structure are included into the machine learning process. For this purpose, 3D LiDAR points and maps in the form of binary masks are used. These are used on a recurrent artificial neural network, the LSTM encoder-decoder based architecture to predict the motion of the interacting traffic agents. A comparison between the proposed solution with one that is only sensor driven (LiDAR) is included. For this purpose, NuScenes dataset is utilised, that includes annotated 3D point clouds. The results have demonstrated the importance of context to enhance our prediction performance as well as the capability of our machine learning framework to incorporate map information.

Our results were published at the 2020 VTC conference 49

8.9.4 Egoshots, an ego-vision life-logging dataset and semantic fidelity metric to evaluate diversity in image captioning models

Participants: Pranav Agarwal, Alejandro Betancourt, Vana Panagiotou, Natalia Díaz-Rodríguez.

Image captioning models have been able to generate grammatically correct and human understandable sentences. However most of the captions convey limited information as the model used is trained on datasets that do not caption all possible objects existing in everyday life. Due to this lack of prior information most of the captions are biased to only a few objects present in the scene, hence limiting their usage in daily life. In this paper 39, we attempt to show the biased nature of the currently existing image captioning models and present a new image captioning dataset, Egoshots, consisting of 978 real life images with no captions. We further exploit the state of the art pre-trained image captioning and object recognition networks to annotate our images and show the limitations of existing works. Furthermore , in order to evaluate the quality of the generated captions, we propose a new image captioning metric, object based Semantic Fidelity (SF). Existing image captioning metrics can evaluate a caption only in the presence of their corresponding annotations; however, SF allows evaluating captions generated for images without annotations, making it highly useful for real life generated captions.

8.9.5 RobotDrlSim: A real time robot simulation platform for reinforcement learning and human interactive demonstration learning

Participants: Te Sun, Liang Gong, Xvdong Li, Shenghan Xie, Zhaorun Chen, Qizi Hu, David Filliat.

Deep reinforcement learning (DRL) techniques give robotics research an AI boost in many applications. In order to simultaneously accommodate the complex robotic behaviour simulation and DRL algorithm verification, a new simulation platform, namely the RobotDrlSim, is proposed 51. First, we design a standardized API interfacing mechanism for coordinating diverse environments on RobotDrlSim platform, where PyBullet simulator is equipped with an API to form a physical engine for robotics simulation. Second, benchmark DRL models are included in the baseline library for evaluation. Third, real-time human-robot interactions can be captured and imported to drive the RobotDrlSim tasks, which provide big data-stream for reinforcement learning. Experimentations show that cutting-edge DRL algorithms developed in python can be seamlessly deployed to the robots, and human interactions can be availed in training the robots. RobotDrlSim is valid for efficiently developing DRL algorithms for artificial intelligence models of robots, and it is especially suitable for the robot educational purposes.

Autonomous Driving Commuter Car (Renault)

Participants: David Filliat, Emmanuel Battesti.

We developed planning algorithms for a autonomous electric car for Renault SAS in the continuation of the previous ADCC project. We improved our planning algorithm in order to go toward navigation on open roads, in particular with the ability to reach higher speed than previously possible, deal with more road intersection case (roundabouts), and with multiple lane roads (overtake, insertion...).

Curiosity-driven Learning Algorithms for Exploration of Video Game Environments (Ubisoft)

Participants: Pierre-Yves Oudeyer.

Financing of a postdoc grant for a 2 year project with Ubisoft and Région Aquitaine.

Intrinsically Motivated Exploration for Lifelong Deep Reinforcement Learning in the Malmo Environment (Microsoft)

Participants: Pierre-Yves Oudeyer, Remy Portelas.

Financing of the PhD grant of Rémy Portelas by Microsoft Research.

Explainable continual learning for autonomous driving (Segula Technologies)

Participants: Natalia Díaz Rodríguez, Adrien Bennetot.

Financing of the CIFRE PhD grant of Adrien Bennetot by Segula Technologies.

Automated Discovery of Self-Organized Structures (Poïetis)

Participants: Pierre-Yves Oudeyer, Mayalen Etcheverry.

Financing of the CIFRE PhD grant of Mayalen Etcheverry by Poietis.

Machine learning for adaptive cognitive training (OnePoint)

Participants: Hélène Sauzéon, Pierre-Yves Oudeyer, Maxime Adolph.

Financing (in progress) of the CIFRE PhD grant of Maxime Adolph by Onepoint.

Curiosity-driven interaction system for learning (evidenceB)

Participants: Hélène Sauzéon, Pierre-Yves Oudeyer, Rania Abdelghani.

Financing of the CIFRE PhD grant of Rania Adolph by EvidenceB.

Perception Techniques and Sensor Fusion for Level 4 Autonomous Vehicles (Renault)

Participants: David Filliat, Vyshakh Palli-Thazha.

Financing of the CIFRE PhD grant of Vyshakh Palli-Thazha by Renault.

Exploration of reinforcement learning algorithms for drone visual perception and control (CEA)

Participants: David Filliat, Florence Carton.

Financing of the CIFRE PhD grant of Florence Carton by CEA.

Incremental learning for sensori-motor control (Softbank Robotics)

Participants: David Filliat, Hugo Caselles Dupré.

Financing of the CIFRE PhD grant of Hugo Caselles-Dupré by Softbank Robotics.

School+ project (FIRAH)

Participants: Hélène Sauzéon, Cécile Mazon.

Financing of one year-postdoctoral position (recruitment in progress) and the app. development by the International Foundation for Applied Research on Disability (FIRAH). The School+ project consists of a set of educational technologies to promote inclusion for children with Autism Spectrum Disorder (ASD). School+ primary aims at encouraging the acquisition of socio-adaptive behaviours at school while promoting self-determination (intrinsic motivation), and has been created according to the methods of the User-Centred Design (UCD). Requested by the stakeholders (child, parent, teachers, and clinicians) of school inclusion, Flowers team works to the adding of an interactive tool for a collaborative and shared monitoring of school inclusion of each child with ASD. This new app will be assessed in terms of user experience (usability and elicited intrinsic motivation), self-efficacy of each stakeholder and educational benefit for child. This project includes the Academie de Bordeaux –Nouvelle Aquitaine, the CRA (Health Center for ASD in Aquitania), and the ARI association.

10.1.1 Inria international partners

10.2.1 Visits of international scientists

Kevvyn Collins-Thompson, Univ. Michigan.

10.2.2 Invited talks

Germán Kruszewski (Facebook AI Research, Title: "The quest for compositional learning"), Guillermo Valle (Univ. Oxford, UK; Title: "Simplicity bias and generalization in deep neural networks"), Ferran Alet (MIT, US, Title: "Meta-learning curiosity algorithms"), Solande Denerveaud (Univ. Geneva, Switzerland; Title: "Error monitoring during learning: Neural and behavioral comparison studies of Montessori and traditionally-schooled students"), Hugo Cisneros (CIIRC, CTU in Prague, Title: "Artificial evolution and emergence in complex systems"), Remy van Trijt (Sony CSL Paris, France, Title: "Fluid Construction Grammar").

Myoelectric prosthesis - PEPS CNRS

PY Oudeyer collaborated with Aymar de Rugy, Daniel Cattaert, Mathilde Couraud, Sébastien Mick and Florent Paclet (INCIA, CNRS/Univ. Bordeaux) about the design of myoelectric robotic prostheses based on the Poppy platform, on the design of algorithms for co-adaptation learning between the human user and the prosthesis, and on the use of goal exploration algorithm to study the behaviour of models of neuromuscular systems. This was funded by a PEPS CNRS grant.

ANR JCJC ECOCURL

C Moulin-Frier obtained an ANR JCJC grant. The project is entitled "ECOCURL: Emergent communication through curiosity-driven multi-agent reinforcement learning". The project starts in Feb 2021 for a duration of 48 months. It will fund a PhD student (36 months) and a Research Engineer (18 months) as well as 4 Master internships (one per year).

Inria Exploratory Action ORIGINS

Clément Moulin-Frier obtained an Exploratory Action from Inria. The project is entitled "ORIGINS: Grounding artificial intelligence in the origins of human behavior". The project starts in October 2020 for a duration of 24 months. It funds a post-doc position (24 months). Eleni Nisioti has been recruited on this grant.

Inria Exploratory Action AIDE

Didier Roy is collaborator of the Inria Exploratory Action AIDE "Artificial Intelligence Devoted to Education", ported by Frédéric Alexandre (Inria Mnemosyne Project-Team), Margarida Romero (LINE Lab) and Thierry Viéville (Inria Mnemosyne Project-Team, LINE Lab). The aim of this Exploratory Action consists to explore to what extent approaches or methods from cognitive neuroscience, linked to machine learning and knowledge representation, could help to better formalize human learning as studied in educational sciences. AIDE is a four year project started middle 2020 until 2024. https://team.inria.fr/mnemosyne/aide/

Poppy Station structure

• Poppy Station Project : D. Roy, P.-Y. Oudeyer. This project aim to perpetuate the Poppy robot ecosystem by creating an external structure from outside Inria, with various partners. After the Poppy Robot Project, the Poppy Education Project has ended and Poppy Station structure is born. PerPoppy is the project which is building the new structure, and Poppy Station is the name of the new structure. Poppy Station, which includes Poppy robot ecosystem (hardware, software, community) from the beginning, is a place of excellence to build future educational robots and to design pedagogical activities to teach computer science, robotics and Artificial Intelligence. https://www.poppy-station.org
• Partners of Poppy Station : Inria, La Ligue de l’Enseignement, HESAM Université, IFÉ-ENS Lyon, MOBOTS – EPFL, Génération Robots, Pollen Robotics, KONEXInc, Mobsya, CERN Microclub, LINE Lab (Université Nice), Stripes, Canopé Martinique, Rights Tech Women, Editions Nathan.

10.3.1 Adaptiv'Math

• Adaptiv'Math
• Program: PIA
• Duration: 2019 - 2020
• Coordinator: EvidenceB

• Partners:

  • EvidenceB
  • Nathan
  • APMEP
  • LIP6
  • INRIA
  • ISOGRAD
  • Daesign
  • Schoolab
  • BlueFrog

The solution Adaptiv'Math comes from an innovation partnership for the development of a pedagogical assistant based on artificial intelligence. This partnership is realized in the context of a call for projects from the Ministry of Education to develop a pedagogical plateform to propose and manage mathematical activities intended for teachers and students of cycle 2. The role of Flowers team is to work on the AI of the proposed solution to personalize the pedagogical content to each student. This contribution is based on the work done during the Kidlearn Project and the thesis of Benjamin Clement 82, in which algorithms have been developed to manage and personalize sequence of pedagogical activities. One of the main goal of the team here is to transfer technologies developed in the team in a project with the perspective of industrial scaling.

Inria Cordi PhD grant

Julius Taylor, supervised by Clément Moulin-Frier, obtained an Inria Cordi PhD grant. He started his PhD thesis in November 2020 on model-based emergent communication in multi-agent reinforcement learning.

Inria - Region Post-doctoral grant - Call 2020 Region of New Aquitania

Hélène Sauzéon and Clément Moulin-Frier obtained an post-doctoral grant for the project entitled " Personalized Intelligent Tutorial Systems (ITS) for attention training: Modelling of personalization algorithms and effectiveness study" Masataka Sawayama started his postdoctoral position in Janvier 2021

11.1.1 Scientific events: organisation

11.1.2 Scientific events: selection

11.1.3 Journal

11.1.4 Invited talks

Cédric Colas has given an invited talk on the EpidemiOptim project at DeepMind, in the context of an internal seminar.

PY Oudeyer gave a keynote talk at the EGC conference in Brussels, on developmental machine learning, Jan. 2020, https://www.egc.asso.fr/non-classe/conferences-invitees-egc-2020.html,

PY Oudeyer gave an invited talk at the Deep RL Workshop of Neurips 2020, on intrinsically motivated goal-conditioned reinforcement learngin, Dec. 2020, https://slideslive.com/38938095/machines-that-invent-their-own-problems?ref=account-folder-62083-folders.

PY Oudeyer gave an invited seminar at the MIT embodied AI seminar, on developmental machine learning, Deep RL and artificial curiosity, April 2020, https://www.youtube.com/watch?v=Jx6-DKXgAKU;

PY Oudeyer gave a keynote talk at the Crossmodal Learning Center Autumn School, on Developmental Machine Learning, Curiosity and Deep RL, Dec. 2020. https://www.crossmodal-learning.org/home.html

PY Oudeyer gave an invited talk at the Brain and Cognition seminar at the University of Geneva, on Curiosity-driven learning in humans and mahcines, Oct. 2020. https://listes.unige.ch/sympa/arc/brain-and-cognition/2020-09/msg00000/BC_SEPT_OCT_NOV_DEC_2020.pdf

Didier Roy has given invited talks at Adaptiv'math project webinar, Class'code AI webinar, EPFL learning sciences conference, on Flowers researches, on AI for education and education to AI.

Didier Roy has given invited talks at Réseau Canopé "Mardis du numérique" at Toulon, on computer science basics and activities to teach computer science, robotics and AI.

Didier Roy has given invited talks at IFE ENS Lyon RNRE Conference.

Didier Roy was invited to participate to the CIDREE European Expert Meeting at IFE ENS Lyon. http://www.cidree.org/cidree-expert-meeting-lyon-january-13-14-2020/. The CIDREE is CONSORTIUM OF INSTITUTIONS FOR DEVELOPMENT AND RESEARCH IN EDUCATION IN EUROPE.

11.1.5 Leadership within the scientific community

PY Oudeyer was editor of the Cognitive and Developmental Systems newsletter of the Cognitive and Developmental Systems Technical Committee of the IEEE CIS Society

PY Oudeyer was elected as Distinguised speaker of the IEEE Computational Ingelligence Society

11.1.6 Scientific expertise

PY Oudeyer was a reviewer for the European Commission (FET program), and the ANR.

11.1.7 Research administration

PY Oudeyer has been member of piloting committees of consortium projects Adaptiv'Maths and Perseverons (eFran) on educational technologies.

11.2.1 Teaching

PY Oudeyer gave a course on developmental reinforcement learning at ENSEIRB master on AI and machine learning (3h), nov. 2019.

PY Oudeyer gave a course on developmental learning at CogMaster cognitive science master (8h), nov. 2019.

PY Oudeyer gave a course on developmental learning at ENSC/ENSEIRB "option robot" master (3h), dec. 2019.

During the latest academic year, Hélène Sauzéon teached 96h in the BS. and master degrees in cognitive science (Department of Mathematics & interaction, University of Bordeaux). She was (co-)responsible of 9 teaching units (3 in BS et 6 in Master).

N Díaz Rodríguez taught, at ENSTA, a total of 3.25 h in ROB313, 27h at IN104, 10.5 at IN102, 21h at IA301. She also gave 42h at IG.2410 at the engineering school ISEP, and 3h course on Continual Learning and State Representation Learning at the reinforcement learning course at ENSEIRB master on AI and machine learning (3h), nov. 2019.

Rémy Portelas and Tristan Karch gave a first year introductory course on programming at Université de Bordeaux (64h), sep. 2020 to jan. 2021.

Didier Roy gave courses on computer science basics, and on computer science, robotics and AI activities for education at Canton de Vaud teachers.

Clément Moulin-Frier gave courses on Robotics and AI at University Pompeu Fabra (Barcelona, Spain, Jan 2020, 10 hours) and Centre de Recherches Interdisciplaires (Paris, France, Apr 2020, 12 hours).

Maxime Adolphe gave courses on basics of AI (18h) at Ecole Nationale Supérieure de Cognitique (ENSC), sep. 2020 to jan.2021.

11.2.2 Supervision

• PhD defended: Cécile Mazon, "Des Technologies Numériques Pour L’inclusion Scolaire Des Collégiens Avec TSA : des approches individuelles aux approches écosystémiques pour soutenir l’individu et ses aidants ", University of Bordeaux (supervised by H. Sauzéon).
• PhD defended: Pierre-Antoine Cinquin, "Conception, intégration et validation de systèmes numériques d'enseignement accessibles aux personnes en situation de handicap cognitif  ", University of Bordeaux (supervised by H. Sauzéon & P. Guitton).
• PhD in progress : Rémy Portelas, "Teacher algorithms for curriculum learning in Deep RL", beg. in sept. 2018 (supervisors: PY Oudeyer and K Hoffmann)
• PhD in progress: Cédric Colas, "Intrinsically Motivated Deep RL", beg. in sept. 2017 (supervisors: PY Oudeyer and O Sigaud)
• PhD in progress: Tristan Karch, "Language acquisition in curiosity-driven Deep RL", beg. in sept. 2019 (supervisors: PY Oudeyer and C Moulin-Frier)
• PhD in progress: Alexander Ten, "Models of human curiosity-driven learning and exploration", beg. in sept. 2018 (supervisors: PY. Oudeyer and J. Gottlieb)
• PhD in progress: Laetitia Teodorescu, "Graph Neural Networks in Curiosity-driven Exploring Agents", beg. in sept. 2020 (supervisors: PY. Oudeyer and K. Hoffman)
• PhD in progress: Maxime Adolphe, "Adaptive personalization in attention training systems", beg. in sept. 2020 (supervisors: H. Sauzéon and PY. Oudeyer)
• PhD in progress: Rania Abdelgani, "Fostering curiosity and meta-cognitive skills in educational technologies", beg. in dec. 2020 (supervisors: H. Sauzéon and PY. Oudeyer.
• PhD in progress: Julius Taylor, "Emergent communication through curiosity-driven multi-agent reinforcement learning", beg. in nov. 2020 (supervisor: C Moulin-Frier and PY Oudeyer)
• PhD in progress: Mayalen Etcheverry, "Automated Discovery of Self-Organized Structures", beg. in sept. 2020 (supervisor: PY Oudeyer)
• PhD in progress: Adrien Bennetot, "Explainable continual learning for autonomous driving", Sorbonne University and ENSTA Paris (supervised by N Díaz Rodríguez & R Chatila).
• Master thesis defended: Anouche Banikyian "Curiosity, intrinsic motivation and spatial learning in children", University of Bordeaux (supervised by H. Sauzéon).
• Master thesis defended: Mehdi Alaimi "New educational application for fostering curiosity-related question-asking in children", University of Bordeaux (supervised by H. Sauzéon & PY Oudeyer).
• Master thesis defended: Juewan Wang "Can an accessible MOOC player improve the retention of disabled students? A MOOC accessibility assessment based on analytic method ", University of Bordeaux (supervised by H. Sauzéon & P. Guitton).
• Master thesis defended: Valentin Villecroze "Emergence of communication in multi-agent systems", Ecole Polytechique (supervised by Clément Moulin-Frier).
• Master thesis defended: Younès Rabii "Conception d’un environnement de simulation écologiquement valide pour agents autonomes", ENS Cognitique Bordeaux (supervised by Clément Moulin-Frier).
• Master thesis defended: Clément Romac "Automated Curriculum Learning: a Benchmark" (co-supervised by R. Portelas and PY. Oudeyer)
• Master thesis defended: Laetitia Teodorescu, "SpatialSim: learning to recognize spatial configurations with graph-neural networks", Telecom ParisTech (supervisor: PY. Oudeyer)
• Cédric Colas supervised two ENSC students for their master 1 project on Artificial Intelligence .

11.2.3 Juries

PY Oudeyer was a member of the admissibility jury of the CR1 competition at Inria Bordeaux Sud-Ouest

PY Oudeyer was a reviewer in the PhD juries of Shoko Ota (OIST, Okinawa, Japant, Title: "Intrinsic Motivation in Creative Activity"), Japan; Benoit Choffin (Univ. Paris Saclay, Title: "Algorithmes d'espacement adaptatif de l'apprentissage pour l'optimisation de la maitrise a long terme de composante de connaissance"); Alexis Jacq (EPFL, Switzerland, Title: "Mutual understanding in educational human-robot collaborations"); Thomas Moerlan (Univ, Delft, Holland, Title: "The intersection of planning and learning").

PY Oudeyer was in the PhD "comité de suivi" of Ahmed Akazia (univ. Paris VI), Alexandre ChenuU (Univ. Paris VI), Sylvia Pagliarni (Univ. Bordeaux), Arash Rashidi (Univ. Bordeaxu), Effie Segas (Univ. Bordeaux)

Hélène Sauzéon organized a selection commitee for recruitment of Assistant professor in Rehabilitative science (University of Bordeaux).

Hélène Sauzéon was external member of a selection commitee for recruitment of Assistant professor in cognitive psychology (University of Toulouse – LeMirail).

Hélène Sauzéon performed several scientific expertises for application requests such as HDR (ED SP2, University of Bordeaux) or local careers advancement (University of Bordeaux).

N. Díaz Rodríguez was invited jury (President) of the PhD thesis "Deep Learning for Abnormal Movement Detection using Wearable Sensors: Case Studies on Stereotypical Motor Movements in Autism and Freezing of Gait in Parkinson's Disease" in the University of Trento, Italy May 2019.

C Moulin-Frier was in the PhD "comité de suivi" of Marc-Antoine Georges (Université Grenoble Alpes).

C Moulin-Frier was reviewer of the PhD of Sock Ching Low, entitled "Giving Centre Stage to Top-Down Inhibitory Mechanisms for Selective Attention", University Pompeu Fabra, Spain, Dec. 2020.

11.3.1 Internal or external Inria responsibilities

Didier Roy was manager editor of a 370-pages computer science school textbook for kindergarten and elementary schools (collaboration Inria/EPFL/Canton de Vaud, Switzerland)

Didier Roy was manager editor of the EPFL MOOC "E-NUM", major contribution to train people, especially teachers, in computer science and digital sociology (collaboration Inria/EPFL/Canton de Vaud, Switzerland)

11.3.2 Articles and contents

Didier Roy and PY Oudeyer published an illustrated educational book on artificial intelligence and robotics for 7-8 years old children, Nathan, see https://www.nathan.fr/catalogue/fiche-produit.asp?ean13=9782092593295 and https://dproy.wordpress.com/.

Didier Roy was one of the authors of the Inria White Paper "Education and Digital, Challenges and Issues" https://hal.inria.fr/hal-03051329

Didier Roy was interviewed by Jérémy Dres, which was reported in a chapter of his comic "Les défis de l'intelligence artificielle".

PY. Oudeyer and S. Forestier were interviewed and appeared in a video documentary on Netflix, called "Babies", on models of infant development and curiosity-driven learning, https://en.wikipedia.org/wiki/Babies_(TV_series).

PY Oudeyer was interviewed by S. Paoli, which was reported in a chapter of the book "Ce qui vient", http://www.editionslesliensquiliberent.fr/livre-Ce_qui_vient-9791020908940-1-1-0-1.html.

H. Sauzéon was interviewed to present her research activitivies on a large-audience blog post on https://www.inria.fr/fr/helene-sauzeon-psychologie-realite-virtuelle.

H. Sauzéon was interviewed to present the results of the AIANA educational software to a large public on https://www.inria.fr/fr/bilan-du-logiciel-aiana-des-resultats-dapprentissage-ameliores

C Moulin-Frier and L Chevillot wrote a large-audience web article on the new Exploratory Action ORIGINS: Grounding Artificial Intelligence in the Origins of Human Behavior. https://www.inria.fr/fr/origins-ancrer-lintelligence-artificielle-dans-les-origines-des-comportements-humains

Cédric Colas helped design a web interface for the EpidemiOptim project. Users can interact with lock-down intervention strategies trained with machine learning to mitigate health and economic costs in the context of simulated COVID-19 epidemics https://epidemioptim.bordeaux.inria.fr/. Users can see the effect of various intervention strategies, can observe how they react to different parameters (sensitivity towards health vs economic costs) and can design their own intervention strategies.

Mayalen Etcheverry wrote an interactive blogpost on the paper of Reinke et al. (2020) "Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems" published at ICLR 2020. https://developmentalsystems.org/intrinsically_motivated_discovery_of_diverse_patterns.

11.3.3 Education

Didier Roy has reviewed contents of the Class'code IAI MOOC.

11.3.4 Interventions

P.-Y. Oudeyer, B. Clément and L. Teodorescu made interventions as part of the "Le Procès du robot" animation at Cap Sciences. The goal was to present in layman's terms the research done at the lab for an audience of junior high school students and to foster discussion among them around an imagined scenario, about the legal responsibility of a domestic robot having caused a minor accient in a home. The web page of the intervention can be found there: https://www.cap-sciences.net/vous-etes/espace-enseignants/proces-robot.

Pierre-Yves Oudeyer made several popular science interventions in Ecole Primaire AygueMarine (Ayguemorte-les-Graves), College de Cadillac (of which he is "parrain scientifique" in the context of "Maison des sciences").

International journals

• 19 article A.Alejandro Barredo Arrieta, N.Natalia Díaz-Rodríguez, J.Javier Del Ser, A.Adrien Bennetot, S.Siham Tabik, A.Alberto Barbado, S.Salvador García, S.Sergio Gil-López, D.Daniel Molina, R.Richard Benjamins, R.Raja Chatila and F.Francisco Herrera. Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI Information Fusion 58 June 2020
  HAL DOI back to text back to text back to text
• 20 articleL.Loïc Caroux, C.Charles Consel, M.Margaux Merciol and H.Hélène Sauzéon. Acceptability of notifications delivered to older adults by technology-based assisted living servicesUniversal Access in the Information Society192020, 675-683
  HALDOI
• 21 article P.-A.Pierre-Antoine Cinquin, P.Pascal Guitton and H.Hélène Sauzéon. Designing accessible MOOCs to expand educational opportunities for persons with cognitive impairments Behaviour and Information Technology March 2020
  HAL DOI
• 22 articleL.Lucile Dupuy and H.Hélène Sauzéon. Effects of an assisted living platform amongst frail older adults and their caregivers: 6 months vs. 9 months follow-up across a pilot field studyGerontechnology191March 2020, 16-27
  HALDOI
• 23 article L.Laila El-Hamamsy, F.Frédérique Chessel-Lazzarotto, B.Barbara Bruno, D.Didier Roy, T.Tereza Cahlikova, M.Morgane Chevalier, G.Gabriel Parriaux, J.-P.Jean-Philippe Pellet, J.Jacques Lanarès, J.Jessica Dehler Zufferey and F.Francesco Mondada. A computer science and robotics integration model for primary school: evaluation of a large-scale in-service K-4 teacher-training program Education and Information Technologies November 2020
  HAL DOI back to text
• 24 article M.Manfred Eppe and P.-Y.Pierre-Yves Oudeyer. Intelligent Behavior Depends on the Ecological Niche KI - Künstliche Intelligenz January 2021
  HAL DOI back to text
• 25 articleI.Ismael Freire, C.Clément Moulin-Frier, M.Marti Sanchez-Fibla, X.Xerxes Arsiwalla and P.Paul Verschure. Modeling the formation of social conventions from embodied real-time interactionsPLoS ONE156June 2020, e0234434
  HALDOIback to text
• 26 article A.Alexandre Heuillet, F.Fabien Couthouis and N.Natalia Díaz-Rodríguez. Explainability in Deep Reinforcement Learning Knowledge-Based Systems February 2021
  HAL back to text
• 27 article R.Remy Kusters, D.Dusan Misevic, H.Hugues Berry, A.Antoine Cully, Y.Yann Le Cunff, L.Loic Dandoy, N.Natalia Díaz-Rodríguez, M.Marion Ficher, J.Jonathan Grizou, A.Alice Othmani, T.Themis Palpanas, M.Matthieu Komorowski, P.Patrick Loiseau, C.Clément Moulin-Frier, S.Santino Nanini, D.Daniele Quercia, M.Michele Sebag, F.Françoise Soulié Fogelman, S.Sofiane Taleb, L.Liubov Tupikina, V.Vaibhav Sahu, J.-J.Jill-Jênn Vie and F.Fatima Wehbi. Interdisciplinary Research in Artificial Intelligence: Challenges and Opportunities Frontiers in Big Data 3 November 2020
  HAL DOI back to text
• 28 article A.Adrien Laversanne-Finot, A.Alexandre Péré and P.-Y.Pierre-Yves Oudeyer. Intrinsically Motivated Exploration of Learned Goal Spaces Frontiers in Neurorobotics 14 January 2021
  HAL DOI back to text
• 29 articleS.Sébastien Mick, A.Arnaud Badets, P.-Y.Pierre-Yves Oudeyer, D.Daniel Cattaert and A.Aymar De Rugy. Biological Plausibility of Arm Postures Influences the Controllability of Robotic Arm TeleoperationHuman FactorsAugust 2020, 001872082094161
  HALDOI
• 30 article V.Vieri Giuliano Santucci, P.-Y.Pierre-Yves Oudeyer, A.Andrew Barto and G.Gianluca Baldassarre. Editorial: Intrinsically Motivated Open-Ended Learning in Autonomous Robots Frontiers in Neurorobotics 13 January 2020
  HAL DOI back to text
• 31 article P.Patricia Shaw, M.Mark Lee, Q.Qiang Shen, K.Kathy Hirsh-Pasek, K.Karen E Adolph, P.-Y.Pierre-Yves Oudeyer and J.Jill Popp. Editorial: Modeling Play in Early Infant Development Frontiers in Neurorobotics 14 August 2020
  HAL DOI back to text

National journals

Invited conferences

International peer-reviewed conferences

• 32 inproceedings A.Ahmed Akakzia, C.Cédric Colas, P.-Y.Pierre-Yves Oudeyer, M.Mohamed Chetouani and O.Olivier Sigaud. Grounding Language to Autonomously-Acquired Skills via Goal Generation ICLR 2021 - Ninth International Conference on Learning Representation Vienna / Virtual, Austria May 2021
  HAL back to text
• 33 inproceedings M.Mehdi Alaimi, E.Edith Law, K.Kevin Daniel Pantasdo, P.-Y.Pierre-Yves Oudeyer and H.Hélène Sauzéon. Pedagogical Agents for Fostering Question-Asking Skills in Children CHI '20 - CHI Conference on Human Factors in Computing Systems Honolulu / Virtual, United States April 2020
  HAL DOI back to text back to text
• 34 inproceedings A.Aurélien Appriou, J.Jessy Ceha, S.Smeety Pramij, D.Dan Dutartre, E.Edith Law, P.-Y.Pierre-Yves Oudeyer and F.Fabien Lotte. Towards measuring states of epistemic curiosity through electroencephalographic signals IEEE SMC 2020 - IEEE International conference on Systems, Man and Cybernetics Toronto / Virtual, Canada October 2020
  HAL back to text
• 35 inproceedings M.Mayalen Etcheverry, C.Clément Moulin-Frier and P.-Y.Pierre-Yves Oudeyer. Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems NeurIPS 2020 - 34th Conference on Neural Information Processing Systems Vancouver / Virtual, Canada December 2020
  HAL back to text back to text back to text back to text
• 36 inproceedingsN.Nicolas Lair, C.Clement Delgrange, D.David Mugisha, J.-M.Jean-Michel Dussoux, P.-Y.Pierre-Yves Oudeyer and P.Peter Ford Dominey. User-in-the-loop Adaptive Intent Detection for Instructable Digital AssistantIUI '20: 25th International Conference on Intelligent User InterfacesCagliari, ItalyMarch 2020, 116-127
  HALDOI
• 37 inproceedings R.Rémy Portelas, K.Katja Hofmann and P.-Y.Pierre-Yves Oudeyer. Trying Again Instead of Trying Longer: Prior Learning for Automatic Curriculum Learning ICLR 2020 BeTR-RL (Beyond “Tabula Rasa” in Reinforcement Learning ) workshop Addis Abeba / Virtual, Ethiopia April 2020
  HAL
• 38 inproceedings C.Chris Reinke, M.Mayalen Etcheverry and P.-Y.Pierre-Yves Oudeyer. Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems International Conference on Learning Representations (ICLR) Addis Ababa, Ethiopia April 2020
  HAL back to text back to text back to text back to text back to text back to text

National peer-reviewed Conferences

Conferences without proceedings

• 39 inproceedings P.Pranav Agarwal, A.Alejandro Betancourt, V.Vana Panagiotou and N.Natalia Díaz-Rodríguez. Egoshots, an ego-vision life-logging dataset and semantic fidelity metric to evaluate diversity in image captioning models ICLR 2020 - 8th International Conference on Learning Representations Addis Ababa / Virtual, Ethiopia April 2020
  HAL back to text
• 40 inproceedings A.Adrien Bennetot, V.Vicky Charisi and N.Natalia Díaz-Rodríguez. Should artificial agents ask for help in human-robot collaborative problem-solving? Brain-PIL Workshop - ICRA2020 Paris, France June 2020
  HAL back to text
• 41 inproceedings H.Hugo Caselles-Dupré, M.Michael Garcia Ortiz and D.David Filliat. Object Detection for Embodied Agents using Sensory Commutativity of Action Sequences NeurIPS 2020 Workshop on BabyMind Vancouver / Virtual, Canada December 2020
  HAL back to text
• 42 inproceedings H.Hugo Caselles-Dupré, M.Michael Garcia Ortiz and D.David Filliat. On the Sensory Commutativity of Action Sequences for Embodied Agents Workshop on Learning in Artificial Open Worlds at ICML20 Online, France July 2020
  HAL back to text
• 43 inproceedings C.Cédric Colas, T.Tristan Karch, N.Nicolas Lair, J.-M.Jean-Michel Dussoux, C.Clément Moulin-Frier, P.Peter Ford Dominey and P.-Y.Pierre-Yves Oudeyer. Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration NeurIPS 2020 - 34th Conference on Neural Information Processing Systems Vancouver / Virtual, Canada February 2020
  HAL back to text back to text back to text back to text
• 44 inproceedings N.Natalia Díaz-Rodríguez and G.Galena Pisoni. Accessible Cultural Heritage through Explainable Artificial Intelligence PATCH 2020 - 11th Workshop on Personalized Access to Cultural Heritage Genova / Virtual, Italy July 2020
  HAL back to text
• 45 inproceedings N.Nicolas Duminy and S.Sao Mai Nguyen. Découverte et exploitation de la hiérarchie des tâches par motivation intrinsèque Réunion "Apprentissage et Robotique" Visioconférence, France http://www.gdr-isis.fr/index.php?page=reunion&idreunion=424 June 2020
  HAL back to text
• 46 inproceedings M.Mayalen Etcheverry, P.-Y.Pierre-Yves Oudeyer and C.Chris Reinke. Progressive growing of self-organized hierarchical representations for exploration ICLR 2020 workshop: Beyond tabula rasa in Reinforcement Learning Addis Ababa / Virtual, Ethiopia April 2021
  HAL back to text
• 47 inproceedings T.Tristan Karch, C.Cédric Colas, L.Laetitia Teodorescu, C.Clément Moulin-Frier and P.-Y.Pierre-Yves Oudeyer. Deep Sets for Generalization in RL Beyond Tabula Rasa in Reinforcement Learning: agents that remember adapt and generalize, Workshop at ICLR Addis Ababa, Ethiopia March 2020
  HAL back to text back to text
• 48 inproceedings C.Clément Moulin-Frier and P.-Y.Pierre-Yves Oudeyer. Multi-Agent Reinforcement Learning as a Computational Tool for Language Evolution Research: Historical Context and Future Challenges COMARL AAAI 2020-2021 - Challenges and Opportunities for Multi-Agent Reinforcement Learning, AAAI Spring Symposium Series Palo Alto, California / Virtual, United States February 2021
  HAL back to text
• 49 inproceedings V.Vyshakh Palli-Thazha, D.David Filliat and J.Javier Ibañez-Guzmán. Trajectory Prediction of Traffic Agents: Incorporating context into machine learning approaches VTC2020-Spring- 2020 IEEE 91st Vehicular Technology Conference Antwerp / Virtual, Belgium May 2020
  HAL back to text
• 50 inproceedings R.Rémy Portelas, C.Cédric Colas, L.Lilian Weng, K.Katja Hofmann and P.-Y.Pierre-Yves Oudeyer. Automatic Curriculum Learning For Deep RL: A Short Survey IJCAI 2020 - International Joint Conference on Artificial Intelligence Kyoto / Virtuelle, Japan January 2021
  HAL
• 51 inproceedings T.Te Sun, L.Liang Gong, X.Xvdong Li, S.Shenghan Xie, Z.Zhaorun Chen, Q.Qizi Hu and D.David Filliat. RobotDrlSim: A real time robot simulation platform for reinforcement learning and human interactive demonstration learning MSOTA 2020 - 3rd International Conference on Modeling, Simulation and Optimization Technologies and Applications Beijing / Virtual, China November 2020
  HAL back to text
• 52 inproceedings L.Linda Nanan Vallée, S.Sao Mai Nguyen, C.Christophe Lohr, I.Ioannis Kanellos and O.Olivier Asseu. How An Automated Gesture Imitation Game Can Improve Social Interactions With Teenagers With ASD IEEE ICRA Workshop on Social Robotics for Neurodevelopmental Disorders Paris, France June 2020
  HAL back to text
• 53 inproceedings V.Valentin Villecroze and C.Clément Moulin-Frier. Studying the joint role of partial observability and channel reliability in emergent communication 1st SMILES (Sensorimotor Interaction, Language and Embodiment of Symbols) workshop, ICDL 2020 Valparaiso / Virtual, Chile https://sites.google.com/view/smiles-workshop/ November 2020
  HAL back to text

Scientific books

• 54 bookG.Gérard Giraudon, P.Pascal Guitton, M.Margarida Romero, D.Didier Roy and T.Thierry Viéville. Éducation et numérique, Défis et enjeuxLivre Blanc Inriahttps://medsci-sites.inria.fr/education-et-numeriqueDecember 2020, 137
  HAL

Scientific book chapters

• 55 inbook C.Cécile Mazon and H.Hélène Sauzéon. Use of mobile technologies with children with ASD Numérique et Autisme Les éditions INSHEA 2021
  HAL back to text

Edition (books, proceedings, special issue of a journal)

Doctoral dissertations and habilitation theses

Reports & preprints

• 56 misc C.Cédric Colas, A.Ahmed Akakzia, P.-Y.Pierre-Yves Oudeyer, M.Mohamed Chetouani and O.Olivier Sigaud. Language-Conditioned Goal Generation: a New Approach to Language Grounding in RL January 2021
  HAL back to text
• 57 misc C.Cédric Colas, B.Boris P. Hejblum, S.Sébastien Rouillon, R.Rodolphe Thiébaut, P.-Y.Pierre-Yves Oudeyer, C.Clément Moulin-Frier and M.Mélanie Prague. EpidemiOptim: a Toolbox for the Optimization of Control Policies in Epidemiological Models 2020
  HAL back to text
• 58 misc C.Cédric Colas, T.Tristan Karch, O.Olivier Sigaud and P.-Y.Pierre-Yves Oudeyer. Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey January 2021
  HAL back to text
• 59 misc S.Stephane Doncieux, N.Nicolas Bredeche, L.Léni Kenneth Le Goff, B.Benoît Girard, A.Alexandre Coninx, O.Olivier Sigaud, M.Mehdi Khamassi, N.Natalia Díaz-Rodríguez, D.David Filliat, T.Timothy Hospedales, A.A. E. Eiben and R.Richard Duro. DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics May 2020
  HAL
• 60 misc G.Grgur Kovač, A.Adrien Laversanne-Finot and P.-Y.Pierre-Yves Oudeyer. GRIMGEP: Learning Progress for Robust Goal Sampling in Visual Deep Reinforcement Learning January 2021
  HAL back to text
• 61 misc E.Eleni Nisioti and C.Clément Moulin-Frier. Grounding Artificial Intelligence in the Origins of Human Behavior January 2021
  HAL back to text back to text
• 62 misc R.Rémy Portelas, C.Clément Romac, K.Katja Hofmann and P.-Y.Pierre-Yves Oudeyer. Meta Automatic Curriculum Learning January 2021
  HAL back to text
• 63 misc L.Laetitia Teodorescu, K.Katja Hofmann and P.-Y.Pierre-Yves Oudeyer. SpatialSim: Recognizing Spatial Configurations of Objects with Graph Neural Networks January 2021
  HAL back to text

Other scientific publications

• 64 misc P.Pascal Guitton and H.Hélène Sauzéon. Aïana, le lecteur de Mooc qui offre une accessibilité sur mesure May 2020
  HAL

Scientific popularization

• 65 miscP.-Y.Pierre-Yves Oudeyer, B.Benjamin Clément, D.Didier Roy and H.Hélène Sauzéon. Projet KidLearn : Vers une personnalisation motivante des parcours d’apprentissageApril 2020,
  HAL

Educational activities

Patents

Softwares

Cited publications

• 66 articleB.Brian A Anderson, P.Patryk A Laurent and S.Steven Yantis. Value-driven attentional captureProceedings of the National Academy of Sciences108252011, 10367--10371
  back to text
• 67 articleB.Brenna Argall, S.Sonia Chernova and M.Manuela Veloso. A Survey of Robot Learning from DemonstrationRobotics and Autonomous Systems5752009, 469--483
  back to text
• 68 articleM.M Asada, S.S Noda, S.S Tawaratsumida and K.K Hosoda. Purposive Behavior Acquisition On A Real Robot By Vision-Based Reinforcement LearningMachine Learning231996, 279-303
  back to text
• 69 inproceedingsB.Bowen Baker, I.Ingmar Kanitscheider, T.Todor Markov, Y.Yi Wu, G.Glenn Powell, B.Bob McGrew and I.Igor Mordatch. Emergent Tool Use From Multi-Agent Autocurriculatex.ids: Baker2019 arXiv: 1909.075282020, URL: https://openreview.net/forum?id=SkxpxJBKwS
  back to text
• 70 articleA.Adrien Baranes and P.-Y.Pierre-Yves Oudeyer. Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in RobotsRobotics and Autonomous Systems611January 2013, 69-73
  HALDOIback to textback to textback to text
• 71 article A.Andrew Barto, M.Marco Mirolli and G.Gianluca Baldassarre. Novelty or surprise? Frontiers in psychology 4 2013
  back to text
• 72 inproceedings A.A.G. Barto, S.S Singh and N.N Chentanez. Intrinsically Motivated Learning of Hierarchical Collections of Skills Proceedings of the 3rd International Conference on Development and Learning (ICDL 2004) Salk Institute, San Diego 2004
  back to text
• 73 misc P.Peter W. Battaglia, J.Jessica B. Hamrick, V.Victor Bapst, A.Alvaro Sanchez-Gonzalez, V.Vinicius Zambaldi, M.Mateusz Malinowski, A.Andrea Tacchetti, D.David Raposo, A.Adam Santoro, R.Ryan Faulkner, C.Caglar Gulcehre, F.Francis Song, A.Andrew Ballard, J.Justin Gilmer, G.George Dahl, A.Ashish Vaswani, K.Kelsey Allen, C.Charles Nash, V.Victoria Langston, C.Chris Dyer, N.Nicolas Heess, D.Daan Wierstra, P.Pushmeet Kohli, M.Matt Botvinick, O.Oriol Vinyals, Y.Yujia Li and R.Razvan Pascanu. Relational inductive biases, deep learning, and graph networks 2018
  back to text
• 74 unpublishedA.Adrien Bennetot, J.-L.Jean-Luc Laurent, R.Raja Chatila and N.Natalia Díaz-Rodríguez. Towards Explainable Neural-Symbolic Visual ReasoningNovember 2019, Accepted at IJCAI19 Neural-Symbolic Learning and Reasoning Workshop (https://sites.google.com/view/nesy2019/home)
  HALback to textback to textback to text
• 75 book D.D. Berlyne. Conflict, Arousal and Curiosity McGraw-Hill 1960
  back to text back to text
• 76 inbookM.Monique Borgerhoff Mulder and R.Ryan Schacht. Human Behavioural EcologyeLSAmerican Cancer Society2012, URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/9780470015902.a0003671.pub2
  DOIback to text
• 77 book C.C.L. Breazeal. Designing sociable robots The MIT Press 2004
  back to text
• 78 inproceedingsR.Rodney Brooks, C.Cynthia Breazeal, R.Robert Irie, C.Charles C. Kemp, B.Brian Scassellati and M.Matthew Williamson. Alternative essences of intelligenceProceedings of 15th National Conference on Artificial Intelligence (AAAI-98)AAAI Press1998, 961--968
  back to text
• 79 misc C.Christopher P. Burgess, L.Loic Matthey, N.Nicholas Watters, R.Rishabh Kabra, I.Irina Higgins, M.Matt Botvinick and A.Alexander Lerchner. MONet: Unsupervised Scene Decomposition and Representation 2019
  back to text
• 80 articleB.-C.Bert Wang-Chak Chan. Lenia: Bbiology of artificial lifeComplex Systems2832019, 251-286
  back to textback to textback to textback to textback to text
• 81 book A.Andy Clark. Mindware: An Introduction to the Philosophy of Cognitive Science Oxford University Press 2001
  back to text
• 82 phdthesis B.Benjamin Clément. Adaptive Personalization of Pedagogical Sequences using Machine Learning Université de Bordeaux December 2018
  HAL back to text back to text back to text
• 83 articleB.Benjamin Clément, D.Didier Roy, P.-Y.Pierre-Yves Oudeyer and M.Manuel Lopes. Multi-Armed Bandits for Intelligent Tutoring SystemsJournal of Educational Data Mining (JEDM)72June 2015, 20--48
  HALback to textback to text
• 84 articleD.D Cohn, Z.Z Ghahramani and M.M Jordan. Active learning with statistical modelsJournal of artificial intelligence research41996, 129--145
  back to text
• 85 book W.W Croft and D.D.A. Cruse. Cognitive Linguistics Cambridge Textbooks in Linguistics Cambridge University Press 2004
  back to text
• 86 book M.M Csikszenthmihalyi. Flow-the psychology of optimal experience Harper Perennial 1991
  back to text back to text
• 87 articleP.P Dayan and W.W Belleine. Reward, motivation and reinforcement learningNeuron362002, 285--298
  back to text
• 88 book E.E.L. Deci and R.R.M Ryan. Intrinsic Motivation and Self-Determination in Human Behavior Plenum Press 1985
  back to text
• 89 articleJ.J.L. Elman. Learning and development in neural networks: The importance of starting smallCognition481993, 71--99
  back to text
• 90 articleS.Shelly B Flagel, H.Huda Akil and T.Terry E Robinson. Individual differences in the attribution of incentive salience to reward-related cues: Implications for addictionNeuropharmacology562009, 139--148
  back to text
• 91 inproceedings S.Sébastien Forestier, Y.Yoan Mollard, D.Damien Caselli and P.-Y.Pierre-Yves Oudeyer. Autonomous exploration, active learning and human guidance with open-source Poppy humanoid robot platform and Explauto library The Thirtieth Annual Conference on Neural Information Processing Systems (NIPS 2016) 2016
  back to text
• 92 articleW.Willem E. Frankenhuis, K.Karthik Panchanathan and A.Andrew G. Barto. Enriching behavioral ecology with reinforcement learning methodsBehavioural Processes1612019, 94--100URL: http://www.sciencedirect.com/science/article/pii/S0376635717303637
  DOIback to textback to text
• 93 articleT.Todd M. Freeberg, R.Robin I. M. Dunbar and T.Terry J. Ord. Social complexity as a proximate and ultimate factor in communicative complexityPhilosophical Transactions of the Royal Society B: Biological Sciences3671597July 2012, 1785--1801URL: https://royalsocietypublishing.org/doi/10.1098/rstb.2011.0213
  DOIback to text
• 94 articleJ.Jacqueline Gottlieb, P.-Y.Pierre-Yves Oudeyer, M.Manuel Lopes and A.Adrien Baranes. Information-seeking, curiosity, and attention: computational and neural mechanismsTrends in Cognitive Sciences1711November 2013, 585-93
  HALDOIback to text
• 95 articleJ.Jacqueline Gottlieb, P.-Y.Pierre-Yves Oudeyer, M.Manuel Lopes and A.Adrien Baranes. Information-seeking, curiosity, and attention: computational and neural mechanismsTrends in cognitive sciences17112013, 585--593
  back to text
• 96 articleJ.Jonathan Grizou, L.Laurie J Points, A.Abhishek Sharma and L.Leroy Cronin. A curious formulation robot enables the discovery of a novel protocell behaviorScience advances652020, eaay4237
  back to text
• 97 inproceedingsT.Tuomas Haarnoja, A.Aurick Zhou, P.Pieter Abbeel and S.Sergey Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic ActorProceedings of the 35th International Conference on Machine Learning80Proceedings of Machine Learning ResearchStockholmsmässan, Stockholm SwedenPMLR10--15 Jul 2018, 1861--1870URL: http://proceedings.mlr.press/v80/haarnoja18b.html
  back to text
• 98 articleS.S Harnad. The symbol grounding problemPhysica D401990, 335--346
  back to text
• 99 bookM.M Hasenjager and H.H Ritter. Active learning in neural networksHeidelberg, Germany, GermanyPhysica-Verlag GmbH2002, 137--169
  back to text
• 100 book J.J Haugeland. Artificial Intelligence: the very idea Cambridge, MA, USA The MIT Press 1985
  back to text
• 101 articleJ.-C.J-C Horvitz. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward eventsNeuroscience9642000, 651-656
  back to text
• 102 inproceedingsX.X Huang and J.J Weng. Novelty and reinforcement learning in the value system of developmental robotsProceedings of the 2nd international workshop on Epigenetic Robotics : Modeling cognitive development in robotic systemsLund University Cognitive Studies 942002, 47--55
  back to text
• 103 inproceedingsS.Serena Ivaldi, N.Natalya Lyubova, D.Damien Gérardeaux-Viret, A.Alain Droniou, S.Salvatore Anzalone, M.Mohamed Chetouani, D.David Filliat and O.Olivier Sigaud. Perception and human interaction for developmental learning of objects and affordancesProc. of the 12th IEEE-RAS International Conference on Humanoid Robots - HUMANOIDSforthcomingJapan2012, URL: http://hal.inria.fr/hal-00755297
  back to text
• 104 book M.Mark Johnson. Developmental Cognitive Neuroscience Blackwell publishing 2005
  back to text
• 105 inproceedingsJ.J. Johnson, B.B. Hariharan, L.L. van der Maaten, L.L. Fei-Fei, C.C. L. Zitnick and R.R. Girshick. CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)2017, 1988-1997
  DOIback to text
• 106 incollectionP.Petteri Karvinen, N.Natalia Díaz-Rodríguez, S.Stefan Grönroos and J.Johan Lilius. RDF Stores for Enhanced Living Environments: An OverviewEnhanced Living Environments: Algorithms, Architectures, Platforms, and SystemsSpringerJanuary 2019, 19-52
  HALDOIback to text
• 107 article C.C. Kidd and B.B.Y Hayden. The psychology and neuroscience of curiosity Neuron (in press) 2015
  back to text
• 108 misc T.Thomas Kipf, E.Elise van der Pol and M.Max Welling. Contrastive Learning of Structured World Models 2020
  back to text
• 109 inproceedingsW.W. Bradley Knox and P.Peter Stone. Combining manual feedback with subsequent MDP reward signals for reinforcement learningProceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS'10)Toronto, Canada2010, 5--12
  back to text
• 110 misc R.Robert Tjarko Lange and H.Henning Sprekeler. Learning not to learn: Nature versus nurture in silico 2020
  back to text
• 111 inproceedings A.Adrien Laversanne-Finot, A.Alexandre Péré and P.-Y.Pierre-Yves Oudeyer. Curiosity Driven Exploration of Learned Disentangled Goal Spaces CoRL 2018 - Conference on Robot Learning Zürich, Switzerland October 2018
  HAL back to text
• 112 article J.Joel Z Leibo, E.Edward Hughes, M.Marc Lanctot and T.Thore Graepel. Autocurricula and the emergence of innovation from social interaction: A manifesto for multi-agent intelligence research arXiv preprint arXiv:1903.00742 2019
  back to text
• 113 articleG.George Loewenstein. The psychology of curiosity: A review and reinterpretationPsychological bulletin11611994, 75
  back to textback to text
• 114 inproceedingsM.Manuel Lopes, T.Thomas Cederborg and P.-Y.Pierre-Yves Oudeyer. Simultaneous Acquisition of Task and Feedback ModelsDevelopment and Learning (ICDL), 2011 IEEE International Conference onGermany2011, 1 - 7URL: http://hal.inria.fr/hal-00636166/en
  DOIback to text
• 115 inproceedingsM.Manuel Lopes, T.Tobias Lang, M.Marc Toussaint and P.-Y.Pierre-Yves Oudeyer. Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning ProgressNeural Information Processing Systems (NIPS)Lake Tahoe, United StatesDecember 2012, URL: http://hal.inria.fr/hal-00755248
  back to text
• 116 articleM.M Lungarella, G.G. Metta, R.R Pfeifer and G.G Sandini. Developmental Robotics: A SurveyConnection Science1542003, 151-190
  back to text
• 117 inproceedingsN.Natalya Lyubova and D.David Filliat. Developmental Approach for Interactive Object DiscoveryNeural Networks (IJCNN), The 2012 International Joint Conference onAustraliaJune 2012, 1-7
  HALDOIback to text
• 118 inproceedings J.J Marshall, D.D Blank and L.L Meeden. An Emergent Framework for Self-Motivation in Developmental Robotics Proceedings of the 3rd International Conference on Development and Learning (ICDL 2004) Salk Institute, San Diego 2004
  back to text
• 119 inproceedingsM.Martin Mason and M.Manuel Lopes. Robot Self-Initiative and Personalization by Learning through Repeated Interactions6th ACM/IEEE International Conference on Human-RobotSwitzerland2011, URL: http://hal.inria.fr/hal-00636164/en
  DOIback to text
• 120 articleC.Cécile Mazon, C.Charles Fage and H.Hélène Sauzéon. Effectiveness and usability of technology-based interventions for children and adolescents with ASD: A systematic review of reliability, consistency, generalization and durability related to the effects of interventionComputers in Human Behavior932019, 235--251
  back to text
• 121 book P.P.H. Miller. Theories of developmental psychology New York: Worth 2001
  back to text back to text back to text
• 122 incollectionM.Marco Mirolli and G.Gianluca Baldassarre. Functions and mechanisms of intrinsic motivationsIntrinsically Motivated Learning in Natural and Artificial SystemsSpringer2013, 49--72
  back to text
• 123 inproceedings C.Clément Moulin-Frier and P.-Y.Pierre-Yves Oudeyer. Exploration strategies in developmental robotics: a unified probabilistic framework ICDL-Epirob - International Conference on Development and Learning, Epirob Osaka, Japan August 2013
  HAL back to text
• 124 inproceedings C.Clement Moulin-Frier and P.-Y.Pierre-Yves Oudeyer. Multi-Agent Reinforcement Learning as a Computational Tool for Language Evolution Research: Historical Context and Future Challenges Challenges and Opportunities for Multi-Agent Reinforcement Learning (COMARL), AAAI Spring Symposium Series, Stanford University, Palo Alto, California, USA 2020
  back to text
• 125 inproceedings C.Clément Moulin-Frier, P.Pierre Rouanet and P.-Y.Pierre-Yves Oudeyer. Explauto: an open-source Python library to study autonomous exploration in developmental robotics ICDL-Epirob - International Conference on Development and Learning, Epirob Genoa, Italy October 2014
  HAL back to text
• 126 inproceedingsS.Sao Mai Nguyen, A.Adrien Baranes and P.-Y.Pierre-Yves Oudeyer. Bootstrapping Intrinsically Motivated Learning with Human DemonstrationsIEEE International Conference on Development and LearningFrankfurt, Germany2011, URL: http://hal.inria.fr/hal-00645986/en
  back to text
• 127 inproceedingsS.Sao Mai Nguyen, A.Adrien Baranes and P.-Y.Pierre-Yves Oudeyer. Constraining the Size Growth of the Task Space with Socially Guided Intrinsic Motivation using Demonstrations.IJCAI Workshop on Agents Learning Interactively from Human Teachers (ALIHT)Barcelona, Spain2011, URL: http://hal.inria.fr/hal-00645995/en
  back to text
• 128 articleS.Sao Mai Nguyen and P.-Y.Pierre-Yves Oudeyer. Socially Guided Intrinsic Motivation for Robot Learning of Motor SkillsAutonomous Robots363March 2014, 273-294
  HALDOIback to text
• 129 incollectionP.-Y.Pierre-Yves Oudeyer. L'auto-organisation dans l'évolution de la paroleParole et Musique: Aux origines du dialogue humain, Colloque annuel du Collège de FranceOdile Jacob2009, 83-112URL: http://hal.inria.fr/inria-00446908/en/
  back to text
• 130 articleP.-Y.Pierre-Yves Oudeyer, F.Frederic Kaplan and V.V. Hafner. Intrinsic Motivation Systems for Autonomous Mental DevelopmentIEEE Transactions on Evolutionary Computation1112007, 265--286URL: http://www.pyoudeyer.com/ims.pdf
  back to textback to textback to textback to text
• 131 articleP.-Y.Pierre-Yves Oudeyer, F.Frédéric Kaplan and V.Véréna Hafner. Intrinsic Motivation for Autonomous Mental DevelopmentIEEE Transactions on Evolutionary Computation112January 2007, 265-286
  HALDOIback to textback to text
• 132 inproceedingsP.-Y.Pierre-Yves Oudeyer and F.Frederic Kaplan. Intelligent adaptive curiosity: a source of self-developmentProceedings of the 4th International Workshop on Epigenetic Robotics117Lund University Cognitive Studies2004, 127--130
  back to textback to text
• 133 article P.-Y.Pierre-Yves Oudeyer and F.Frederic Kaplan. What is intrinsic motivation? A typology of computational approaches Frontiers in Neurorobotics 1 1 2007
  back to text back to text
• 134 incollectionP.-Y.Pierre-Yves Oudeyer. Sur les interactions entre la robotique et les sciences de l'esprit et du comportementInformatique et Sciences Cognitives : influences ou confluences ?Presses Universitaires de France2009, URL: http://hal.inria.fr/inria-00420309/en/
  back to text
• 135 inproceedings M.Madeline Pelz, S.Steven T Piantadosi and C.Celeste Kidd. The dynamics of idealized attention in complex learning environments IEEE International Conference on Development and Learning and on Epigenetic Robotics 2015
  back to text
• 136 inproceedings A.Alexandre Péré, S.Sébastien Forestier, O.Olivier Sigaud and P.-Y.Pierre-Yves Oudeyer. Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration ICLR2018 - 6th International Conference on Learning Representations Vancouver, Canada April 2018
  HAL back to text
• 137 articleL.Laurie J Points, J.James Ward Taylor, J.Jonathan Grizou, K.Kevin Donkers and L.Leroy Cronin. Artificial intelligence exploration of unstable protocells leads to predictable properties and discovery of collective behaviorProceedings of the National Academy of Sciences2018, 201711089
  back to text
• 138 inproceedings R.Rémy Portelas, C.Cédric Colas, K.Katja Hofmann and P.-Y.Pierre-Yves Oudeyer. Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments CoRL 2019 - Conference on Robot Learning Osaka, Japan October 2019
  HAL back to text back to text back to text
• 139 inbookA.A. Revel and J.J. Nadel. Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative DimensionsK.Kerstin DautenhahnC.C. NehanivCambridge University Press2004, How to build an imitator?
  back to text
• 140 articleE.Evan F Risko, N.Nicola C Anderson, S.Sophie Lanthier and A.Alan Kingstone. Curious eyes: Individual differences in personality predict eye movement behavior in scene-viewingCognition12212012, 86--90
  back to text
• 141 article V.Vieri G Santucci, G.Gianluca Baldassarre and M.Marco Mirolli. Which is the best intrinsic motivation signal for learning multiple skills? Frontiers in neurorobotics 7 2013
  back to text
• 142 inproceedingsP.-Y.Pierre-Yves Schatz. Learning motor dependent Crutchfield's information distance to anticipate changes in the topology of sensory body mapsIEEE International Conference on Learning and DevelopmentChine Shangai2009, URL: http://hal.inria.fr/inria-00420186/en/
  back to text
• 143 articleM.M Schembri, M.M Mirolli and G.G Baldassarre. Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot IEEE 6th International Conference on Development and Learning, 2007. ICDL 2007.July 2007, 282-287URL: http://dx.doi.org/10.1109/DEVLRN.2007.4354052
  back to text
• 144 inproceedingsJ.J Schmidhuber. Curious Model-Building Control SystemsProceedings of the International Joint Conference on Neural Networks, Singapore2IEEE press1991, 1458--1463
  back to text
• 145 articleW.W Schultz, P.P Dayan and P.P.R. Montague. A neural substrate of prediction and rewardScience2751997, 1593-1599
  back to text
• 146 articleD.David Silver, T.Thomas Hubert, J.Julian Schrittwieser, I.Ioannis Antonoglou, M.Matthew Lai, A.Arthur Guez, M.Marc Lanctot, L.Laurent Sifre, D.Dharshan Kumaran, T.Thore Graepel, T.Timothy Lillicrap, K.Karen Simonyan and D.Demis Hassabis. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-playScience36264192018, 1140--1144URL: https://science.sciencemag.org/content/362/6419/1140
  DOIback to text
• 147 inproceedingsK.Kenneth O Stanley. Exploiting regularity without developmentProceedings of the AAAI Fall Symposium on Developmental SystemsAAAI Press Menlo Park, CA2006, 37
  back to text
• 148 book L.Luc Steels R.Rodney Brooks The Artificial Life Route to Artificial Intelligence: Building Embodied, Situated Agents Hillsdale, NJ, USA L. Erlbaum Associates Inc. 1995
  back to text
• 149 inproceedings E.Emily Sumner, E.Erika DeAngelis, M.Mara Hyatt, N.Noah Goodman and C.Celeste Kidd. Toddlers Always Get the Last Word: Recency biases in early verbal behavior Proceedings of the 37th Annual Meeting of the Cognitive Science Society 2015
  back to text
• 150 book E.Esther Thelen and L.Linda B. Smith. A dynamic systems approach to the development of cognition and action Cambridge, MA MIT Press 1994
  back to text
• 151 articleA.Andrea L. Thomaz and C.Cynthia Breazeal. Teachable robots: Understanding human teaching behavior to build more effective robot learnersArtificial Intelligence Journal1722008, 716-737
  back to textback to text
• 152 articleA.A Turing. Computing machinery and intelligenceMind591950, 433-460
  back to text
• 153 articleM.Melina R Uncapher, M.Monica K Thieu and A.Anthony D Wagner. Media multitasking and memory: Differences in working memory and long-term memoryPsychonomic bulletin & review2015, 1--8
  back to text
• 154 book F.F.J. Varela, E.E Thompson and E.E Rosch. The embodied mind : Cognitive science and human experience Cambridge, MA MIT Press 1991
  back to text
• 155 misc R.Rui Wang, J.Joel Lehman, J.Jeff Clune and K.Kenneth O. Stanley. Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions 2019
  back to text
• 156 articleJ.J Weng, J.J McClelland, A.A Pentland, O.O Sporns, I.I Stockman, M.M Sur and E.Esther Thelen. Autonomous mental development by robots and animalsScience2912001, 599-600
  back to text