EN FR
EN FR

2024Activity reportProject-TeamLARSEN

RNSR: 201521241C
  • Research center Inria Centre at Université de Lorraine
  • In partnership with:Université de Lorraine, CNRS
  • Team name: Lifelong Autonomy and interaction skills for Robots in a Sensing ENvironment
  • In collaboration with:Laboratoire lorrain de recherche en informatique et ses applications (LORIA)
  • Domain:Perception, Cognition and Interaction
  • Theme:Robotics and Smart environments

Keywords

Computer Science and Digital Science

  • A5. Interaction, multimedia and robotics
  • A5.1. Human-Computer Interaction
  • A5.10. Robotics
  • A5.10.1. Design
  • A5.10.2. Perception
  • A5.10.3. Planning
  • A5.10.4. Robot control
  • A5.10.5. Robot interaction (with the environment, humans, other robots)
  • A5.10.6. Swarm robotics
  • A5.10.7. Learning
  • A5.10.8. Cognitive robotics and systems
  • A5.11. Smart spaces
  • A5.11.1. Human activity analysis and recognition
  • A8.2. Optimization
  • A8.2.2. Evolutionary algorithms
  • A9.2. Machine learning
  • A9.5. Robotics
  • A9.7. AI algorithmics
  • A9.9. Distributed AI, Multi-agent

Other Research Topics and Application Domains

  • B1.1.9. Biomechanics and anatomy
  • B2.1. Well being
  • B2.5.3. Assistance for elderly
  • B5.1. Factory of the future
  • B5.2.4. Aerospace
  • B5.6. Robotic systems
  • B7.2.1. Smart vehicles
  • B9.6. Humanities
  • B9.6.1. Psychology

1 Team members, visitors, external collaborators

Research Scientists

  • Francis Colas [Team leader, INRIA, Researcher, HDR]
  • Olivier Buffet [INRIA, Researcher, HDR]
  • Serena Ivaldi [INRIA, Senior Researcher, HDR]
  • Pauline Maurice [CNRS, Researcher]
  • Enrico Mingo Hoffman [INRIA, ISFP]
  • Jean-Baptiste Mouret [INRIA, Senior Researcher, HDR]
  • Quentin Rouxel [INRIA, Starting Research Position]
  • Bruno Scherrer [INRIA, Researcher, from Jul 2024, HDR]

Faculty Members

  • Amine Boumaza [UL, Associate Professor]
  • Mohamed Boutayeb [UL, Professor Delegation, until Aug 2024, HDR]
  • Sophie Lemonnier [UL, Associate Professor Delegation]
  • Alexis Scheuer [UL, Associate Professor]
  • Vincent Thomas [UL, Associate Professor]

Post-Doctoral Fellows

  • Anna Bucchieri [UL, Post-Doctoral Fellow, from Aug 2024]
  • Jessica Colombel [CNRS, Post-Doctoral Fellow, until Aug 2024]

PhD Students

  • Timothee Anne [INRIA, until Mar 2024]
  • Mathis Antonetti [INRIA, from Dec 2024]
  • Raphael Boige [UL, from Nov 2024]
  • Abir Bouaouda [sans employeur, from Sep 2024]
  • Abir Bouaouda [UL, ATER, until Aug 2024]
  • Raphael Bousigues [sans employeur, until Feb 2024]
  • Baptiste Buchi [UL, ATER, from Feb 2024 until Aug 2024]
  • Salome Lepers [UL]
  • Raphael Lorenzo [INRIA, from Oct 2024]
  • Thomas Martin [INRIA]
  • Nima Mehdi [sans employeur, from Mar 2024]
  • Nima Mehdi [UL, until Mar 2024]
  • Alexandre Oliveira Souza [SAFRAN, CIFRE]
  • Dionis Totsila [INRIA]
  • Konstantinos Tsakonas [INRIA, from Oct 2024]
  • Ioannis Tsikelis [INRIA, from Oct 2024]
  • Aya Yaacoub [CNRS]
  • Jacques Zhong [sans employeur, from Mar 2024 until Jul 2024]
  • Jacques Zhong [INRIA, until Feb 2024]

Technical Staff

  • Fabio Amadio [INRIA, Engineer, from Sep 2024]
  • Nicolas Beaufort [INRIA, Engineer]
  • Alexis Biver [INRIA, Engineer]
  • Raphael Bousigues [INRIA, Engineer, from Oct 2024]
  • Clemente Donoso [INRIA, Engineer, from Sep 2024]
  • Raphael Lartot [INRIA, Engineer, from Oct 2024]
  • Olivier Rochel [INRIA, Engineer]
  • Jefferson Roman Blanco [UL, Engineer, from Feb 2024 until Apr 2024]
  • Jacques Zhong [CNRS, Engineer, until Jan 2024]

Interns and Apprentices

  • Romain Chevret [CNRS, Intern, from Jun 2024 until Aug 2024]
  • Manon Cordone [UL, Intern, from Mar 2024 until May 2024]
  • Elissa El Khalil [CNRS, Intern, from Feb 2024 until Jul 2024]
  • Andrea Ferrari [INRIA, Intern, from Apr 2024 until Sep 2024]
  • Maelle Juilliot [UL, Intern, from Jun 2024 until Aug 2024]

Administrative Assistants

  • Véronique Constant [INRIA]
  • Antoinette Courrier [CNRS]

Visiting Scientist

  • Anna Bucchieri [Univ Milano, from Feb 2024 until Jul 2024]

External Collaborator

  • Timothee Anne [ENS Rennes, from Apr 2024 until Jul 2024]

2 Overall objectives

The goal of the Larsen team is to move robots beyond the research laboratories and manufacturing industries: current robots are far from being the fully autonomous, reliable, and interactive robots that could co-exist with us in our society and run for days, weeks, or months. While there is undoubtedly progress to be made on the hardware side, robotic platforms are quickly maturing and we believe the main challenges to achieve our goal are now on the software side. We want our software to be able to run on low-cost mobile robots that are therefore not equipped with high-performance sensors or actuators, so that our techniques can realistically be deployed and evaluated in real settings, such as in service and assistive robotic applications. We envision that these robots will be able to cooperate with each other but also with intelligent spaces or apartments which can also be seen as robots spread in the environment. Like robots, intelligent spaces are equipped with sensors that make them sensitive to human needs, habits, gestures, etc., and actuators to be adaptive and responsive to environment changes and human needs. These intelligent spaces can give robots improved skills, with less expensive sensors and actuators enlarging their field of view of human activities, making them able to behave more intelligently and with better awareness of people evolving in their environment. As robots and intelligent spaces share common characteristics, we will use, for the sake of simplicity, the term robot for both mobile robots and intelligent spaces.

Among the particular issues we want to address, we aim at designing robots that are able to:

  • handle dynamic environments and unforeseen situations;
  • cope with physical damage;
  • interact physically and socially with humans;
  • collaborate with each other;
  • exploit the multitude of sensor measurements from their surroundings;
  • enhance their acceptability and usability by end-users without robotics background.

All these abilities can be summarized by the following two major objectives:

  • life-long autonomy: continuously perform tasks while adapting to sudden or gradual changes in both the environment and the morphology of the robot;
  • natural interaction with robotics systems: interact with both other robots and humans for long periods of time, taking into account that people and robots learn from each other when they live together.

3 Research program

3.1 Lifelong autonomy

Scientific context

So far, only a few autonomous robots have been deployed for a long time (weeks, months, or years) outside of factories and laboratories. They are mostly mobile robots that simply “move around” (e.g., vacuum cleaners or museum “guides”) and data collecting robots (e.g., boats or underwater “gliders” that collect data about the water of the ocean).

A large part of the long-term autonomy community is focused on simultaneous localization and mapping (SLAM), with a recent emphasis on changing and outdoor environments 49, 57. A more recent theme is life-long learning: during long-term deployment, we cannot hope to equip robots with everything they need to know, therefore some things will have to be learned along the way. Most of the work on this topic leverages machine learning and/or evolutionary algorithms to improve the ability of robots to react to unforeseen changes 49, 54.

Main challenges

The first major challenge is to endow robots with a stable situation awareness in open and dynamic environments. This covers both the state estimation of the robot by itself as well as the perception/representation of the environment. Both problems have been claimed to be solved but it is only the case for static environments 53.

In the Larsen team, we aim at deployment in environments shared with humans which imply dynamic objects that degrade both the mapping and localization of a robot, especially in cluttered spaces. Moreover, when robots stay longer in the environment than for the acquisition of a snapshot map, they have to face structural changes, such as the displacement of a piece of furniture or the opening or closing of a door. The current approach is to simply update an implicitly static map with all observations but without attempt at distinguishing the suitable changes. For localization in not-too-cluttered or not-too-empty environments, this is generally sufficient since a significant fraction of the environment should remain stable. But for life-long autonomy, and in particular for navigation, the quality of the map, and especially the knowledge of the stable parts, is primordial.

A second major obstacle to moving robots outside of labs and factories is their fragility: Current robots often break in a few hours, if not a few minutes. This fragility mainly stems from the overall complexity of robotic systems, which involve many actuators, many sensors, and complex decisions, and from the diversity of situations that robots can encounter. Low-cost robots exacerbate this issue because they can be broken in many ways (high-quality material is expensive), because they have low self-sensing abilities (sensors are expensive and increase the overall complexity), and because they are typically targeted towards non-controlled environments (e.g., houses rather than factories, in which robots are protected from most unexpected events). More generally, this fragility is a symptom of the lack of adaptive abilities in current robots.

Angle of attack

To solve the state estimation problem, our approach is to combine classical estimation filters (Extended Kalman Filters, Unscented Kalman Filters, or particle filters) with a Bayesian reasoning model in order to internally simulate various configurations of the robot in its environment. This should allow for adaptive estimation that can be used as one aspect of long-term adaptation. To handle dynamic and structural changes in an environment, we aim at assessing, for each piece of observation, whether it is static or not.

We also plan to address active sensing to improve the situation awareness of robots. Literally, active sensing is the ability of an interacting agent to act so as to control what it senses from its environment with the typical objective of acquiring information about this environment. A formalism for representing and solving active sensing problems has already been proposed by members of the team 48 and we aim to use it to formalize decision-making problems for improving situation awareness.

Situation awareness of robots can also be tackled by cooperation, whether it be between robots or between robots and sensors in the environment (deployed in sensorized environments) or between robots and humans. This is in rupture with classical robotics, in which robots are conceived as self-contained. But, in order to cope with as diverse environments as possible, these classical robots use precise, expensive, and specialized sensors, whose cost prohibits their use in large-scale deployments for service or assistance applications. Furthermore, when all sensors are on the robot, they share the same point of view on the environment, which is a limit for perception. Therefore, we propose to complement a cheaper robot with sensors distributed in a target environment. This is an emerging research direction that shares some of the problematics of multi-robot operation and we are therefore collaborating with other teams at Inria that address the issue of communication and interoperability.

To address the fragility problem, the traditional approach is to first diagnose the situation, then use a planning algorithm to create/select a contingency plan. But, again, this calls for both expensive sensors on the robot for the diagnosis and extensive work to predict and plan for all the possible faults that, in an open and dynamic environment, are almost infinite. An alternative approach is then to skip the diagnosis and let the robot discover by trial and error a behavior that works in spite of the damage with a reinforcement learning algorithm 62, 54. However, current reinforcement learning algorithms require hundreds of trials/episodes to learn a single, often simplified, task 54, which makes them impossible to use for real robots and more ambitious tasks. We therefore need to design new trial-and-error algorithms that will allow robots to learn with a much smaller number of trials (typically, a dozen). We think the key idea is to guide online learning on the physical robot with dynamic simulations. For instance, in our recent work, we successfully mixed evolutionary search in simulation, physical tests on the robot, and machine learning to allow a robot to recover from physical damage 55, 1.

A final approach to address fragility is to deploy several robots or a swarm of robots or to make robots evolve in an active environment. We will consider several paradigms such as (1) those inspired from collective natural phenomena in which the environment plays an active role for coordinating the activity of a huge number of biological entities such as ants and (2) those based on online learning 52. We envision to transfer our knowledge of such phenomenon to engineer new artificial devices such as an intelligent floor (which is in fact a spatially distributed network in which each node can sense, compute and communicate with contiguous nodes and can interact with moving entities on top of it) in order to assist people and robots (see the principle in 60, 52, 47).

3.2 Natural interaction with robotic systems

Scientific context

Interaction with the environment is a primordial requirement for an autonomous robot. When the environment is sensorized, the interaction can include localizing, tracking, and recognizing the behavior of robots and humans. One specific issue lies in the lack of predictive models for human behavior and a critical constraint arises from the incomplete knowledge of the environment and the other agents.

On the other hand, when working in the proximity of or directly with humans, robots must be capable of safely interacting with them, which calls upon a mixture of physical and social skills. Currently, robot operators are usually trained and specialized but potential end-users of robots for service or personal assistance are not skilled robotics experts, which means that the robot needs to be accepted as reliable, trustworthy and efficient 65. Most Human-Robot Interaction (HRI) studies focus on verbal communication 61 but applications such as assistance robotics require a deeper knowledge of the intertwined exchange of social and physical signals to provide suitable robot controllers.

Main challenges

We are here interested in building the bricks for a situated HRI addressing both the physical and social dimension of the close interaction, and the cognitive aspects related to the analysis and interpretation of human movement and activity.

The combination of physical and social signals into robot control is a crucial investigation for assistance robots 63 and robotic co-workers 59. A major obstacle is the control of physical interaction (precisely, the control of contact forces) between the robot and the human while both partners are moving. In mobile robots, this problem is usually addressed by planning the robot movement taking into account the human as an obstacle or as a target, then delegating the execution of this “high-level” motion to whole-body controllers, where a mixture of weighted tasks is used to account for the robot balance, constraints, and desired end-effector trajectories 46.

The first challenge is to make these controllers easier to deploy in real robotics systems, as currently they require a lot of tuning and can become very complex to handle the interaction with unknown dynamical systems such as humans. Here, the key is to combine machine learning techniques with such controllers.

The second challenge is to make the robot react and adapt online to the human feedback, exploiting the whole set of measurable verbal and non-verbal signals that humans naturally produce during a physical or social interaction. Technically, this means finding the optimal policy that adapts the robot controllers online, taking into account feedback from the human. Here, we need to carefully identify the significant feedback signals or some metrics of human feedback. In real-world conditions (i.e., outside the research laboratory environment) the set of signals is technologically limited by the robot's and environmental sensors and the onboard processing capabilities.

The third challenge is for a robot to be able to identify and track people on board. The motivation is to be able to estimate online either the position, the posture, or even moods and intentions of persons surrounding the robot. The main challenge is to be able to do that online, in real-time and in cluttered environments.

Angle of attack

Our key idea is to exploit the physical and social signals produced by the human during the interaction with the robot and the environment in controlled conditions, to learn simple models of human behavior and consequently to use these models to optimize the robot movements and actions. In a first phase, we will exploit human physical signals (e.g., posture and force measurements) to identify the elementary posture tasks during balance and physical interaction. The identified model will be used to optimize the robot whole-body control as prior knowledge to improve both the robot balance and the control of the interaction forces. Technically, we will combine weighted and prioritized controllers with stochastic optimization techniques. To adapt online the control of physical interaction and make it possible with human partners that are not robotics experts, we will exploit verbal and non-verbal signals (e.g., gaze, touch, prosody). The idea here is to estimate online from these signals the human intent along with some inter-individual factors that the robot can exploit to adapt its behavior, maximizing the engagement and acceptability during the interaction.

Another promising approach already investigated in the Larsen team is the capability for a robot and/or an intelligent space to localize humans in its surrounding environment and to understand their activities. This is an important issue to handle both for safe and efficient human-robot interaction.

Simultaneous Tracking and Activity Recognition (STAR) 64 is an approach we want to develop. The activity of a person is highly correlated with his position, and this approach aims at combining tracking and activity recognition to make one benefit from the other. By tracking the individual, the system may help infer its possible activity, while by estimating the activity of the individual, the system may make a better prediction of his/her possible future positions (especially in the case of occlusions). This direction has been tested with simulator and particle filters 51, and one promising direction would be to couple STAR with decision making formalisms like partially observable Markov decision processes (POMDPs). This would allow us to formalize problems such as deciding which action to take given an estimate of the human location and activity. This could also formalize other problems linked to the active sensing direction of the team: how should the robotic system choose its actions in order to better estimate the human location and activity (for instance by moving in the environment or by changing the orientation of its cameras)?

Another issue we want to address is robotic human body pose estimation. Human body pose estimation consists of tracking body parts by analyzing a sequence of input images from single or multiple cameras.

Human posture analysis is of high value for human robot interaction and activity recognition. However, even though the arrival of new sensors like RGB-D cameras has simplified the problem, it still poses a great challenge, especially if we want to do it online, on a robot and in realistic world conditions (cluttered environment). This is even more difficult for a robot to bring together different capabilities both at the perception and navigation level 50. This will be tackled through different techniques, going from Bayesian state estimation (particle filtering), to learning, active and distributed sensing.

4 Application domains

4.1 Personal assistance

During the last fifty years, many medical advances as well as the improvement of the quality of life have resulted in a longer life expectancy in industrial societies. The increase in the number of elderly people is a matter of public health because although elderly people can age in good health, old age also causes embrittlement, in particular on the physical plan which can result in a loss of autonomy. That will lead us to re-think the current model regarding the care of elderly people.1 Capacity limits in specialized institutes, along with the preference of elderly people to stay at home as long as possible, explain a growing need for specific services at home.

Ambient intelligence technologies and robotics could contribute to this societal challenge. The spectrum of possible actions in the field of elderly assistance is very large, going from activity monitoring services to mobility or daily activity aids, medical rehabilitation, and social interactions. This will be based on the experimental infrastructure we have built in Nancy (Smart apartment platform) as well as the deep collaboration we have with OHS 2 and the company Pharmagest and its subsidiary Diatelic,created in 2002 by a member of the teams and others.

At the same time, these technologies can be beneficial to address the increasing development of musculo-skeletal disorders and diseases that is caused by the non-ergonomic postures of the workers, subject to physically stressful tasks. Wearable technologies, sensors and robotics, can be used to monitor the worker's activity, its impact on their health, and anticipate risky movements. Two application domains have been particularly addressed in the last years: industry, and more specifically manufacturing, and healthcare.

4.2 Civil robotics

Many applications for robotics technology exist within the services provided by national and local government. Typical applications include civil infrastructure services 3 such as: urban maintenance and cleaning; civil security services; emergency services involved in disaster management including search and rescue; environmental services such as surveillance of rivers, air quality, and pollution. These applications may be carried out by a wide variety of robots and operating modalities, ranging from single robots to small fleets of homogeneous or heterogeneous robots. Often robot teams will need to cooperate to span a large workspace, for example in urban rubbish collection, and operate in potentially hostile environments, for example in disaster management. These systems are also likely to have extensive interaction with people and their environments.

The skills required for civil robots match those developed in the Larsen project: operating for a long time in potentially hostile environment, potentially with small fleets of robots, and potentially in interaction with people.

5 Social and environmental responsibility

5.1 Footprint of research activities

The team is engaged in reducing its carbon footprint by taking actions to reduce the number of travels. Project meetings are carried out remotely, when possible, and trains are the most preferable form of travel when possible.

5.2 Engagement for women

The team promotes equal opportunities for women and minorities. Female researchers of the team are frequently invited to present their research and career to girls and young women, and are engaged in the many activities organized by the institutes (Inria, CNRS) and the University of Lorraine. Some examples:

  • Serena Ivaldi participated to the online panel discussion “Women in AI” organized by CLAIRE and the EU Network of Excellence (March 6, 2024).
  • Anna Bucchieri and Jessica Colombel participated to the online event “SheCodesMeetup” organized by VISION and the EU Network of Excellence (June 19, 2024).
  • Anna Bucchieri and Jessica Colombel co-organized the first IEEE RAS Women in Engineering (WiE) Lunch at HUMANOIDS 2024 – the first ever at a Humanoids conference, with more than 50 attendees!

5.3 Impact of research results

Hospitals- The research in the ExoTurn project led to the deployment of a total of four exoskeletons (Laevo) in the Intensive Care Unit Department of the Hospital of Nancy (CHRU). They have been used by the medical staff since April 2020 to perform Prone Positioning on COVID-19 patients with severe ARDS. To the best of our knowledge, other hospitals (in France, Belgium, Holland, Italy and Switzerland) are following on our footsteps and purchased Laevo exoskeletons for the same use. At the same time, the positive feedback from the CHRU of Nancy has motivated us to continue investigating if exoskeletons could be beneficial for the medical staff involved in other type of healthcare activities. A new study on bed bathing of hospitalized patients started in February 2021 in the department of vascular surgery. For sanitary reasons, preliminary experiments investigating the use of the Laevo for assisting nurses were conducted in the team's laboratory premises in summer 2021. An article presenting the findings is in preparation.

Ageing and health- This research line has the objective to propose technological solutions to the difficulties of elderly people in an ageing population (due to the increase of life expectancy). The placement of older people in a nursing home (EHPAD) is often only a choice of reason and can be rather poorly experienced by people. One answer to this societal problem is the development of smart home technologies that assist the elderly to stay in their homes longer than they can do today. With this objective we have a long term cooperation with Pharmagest which has been supported in recent years through a PhD thesis (Cifre) between June 2017 and August 2021. The objective is to enhance the CareLib solution developed by Diatelic (a subsidiary of the Wellcoop-Pharmagest group) and Larsen team through a previous collaboration (Satelor project). The Carelib offer is a solution consisting of (1) a connected box (with touch screen), (2) a 3D sensor that is able (i) to measure characteristics of the gait such as the speed and step length, (ii) to identify activities of daily life and (iii) to detect emergency situation such as a fall, and (3) universal sensors (motion, ...) installed in each part of the housing. A software licence has been granted by Inria to Pharmagest.

Environment- The new project TELEMOVTOP, in collaboration with the company Isotop, aims at automating the processes of disposal of metal sheets contaminated with asbestos from roofs. This procedure has a high environmental impact and is also a risk for the health of the workers. Robotics can be a major technology innovation in this field. With this project, the team aims at both helping to reduce the workers' risk of exposure to asbestos, and accelerating the disposal project to reduce environmental pollution.

Firefighters- The project POMPEXO, in collaboration with the SDIS 54 (firefighters from Meurthe-et-Moselle) and two laboratories from the Université de Lorraine (DevAH: biomechanics, and Perseus: Cognitive ergonomics), aims at investigating the possibility to assist firefighters with an exoskeleton during the car cutting maneuver. This frequent maneuver is physically very demanding, and due to a general trend in ageing and decreasing physical condition among firefighter crews, less and less firefighters are able to perform it. Hence the SDIS 54 is looking for a solution to increase the strength and reduce the fatigue of firefighters during this maneuver. Occupational exoskeletons have the potential to alleviate the physical load on the workers, and hence may be a solution. However, feasibility and benefits of exoskeletons are task-dependent. Hence we are currently analyzing the car cutting maneuver based on data collected on-site with professional firefighters, to identify what kind of exoskeleton may be suitable.

6 Highlights of the year

6.1 Awards

  • Our team was 4th in the Personal Robotics Challenge of the 1st euROBIN Coopetition 2024
  • Enrico Mingo Hoffman was appointed Senior IEEE Member

6.2 HUMANOIDS 2024

  • Serena Ivaldi was General Chair of the IEEE/RAS International Conference on Humanoid Robots (HUMANOIDS 2024), taking place on 22-24 November 2024 in Nancy at the Centre Prouvé. Many members of the team were involved in the organization.
  • Scientific conference: more than 500 conference registrations and 700 individual participants (conference, exhibitors, one-day pass), 30 exhibitors, 2 competitions (Humanoids walking competition and euRobin competition).
  • Outreach program: more than 900 people (adults and children) from the general public, coming to the conference for free guided tours of the exhibition and competition area, or for robotics workshops and classes, or attending a conference by Christian Duriez (PEPR O2R).

6.3 euROBIN Week 2024

  • Serena Ivaldi was also General Chair / Local Organizer of the euROBIN Week, taking place on 25-28 November 2024 in Nancy at the Centre Prouvé, hosting the 1st euROBIN Coopetition.
  • 150 participants, visit of a delegation of the European Commission, led by Cécile Huet (head of robotics and AI at the EC).
  • Outreach program: about 1000 kids from the schools (visits, robotics workshop, theater play, ...).

6.4 JNRH 2024

  • Pauline Maurice and Jessica Colombel were the organizers of the “Journées Nationales de la Robotique Humanoïde”' (JNRH) 2024, that took place on November 20-21 in Loria - Centre Inria de l'Université de Lorraine, in Nancy. JNRH is a national event that is organized every year by a different laboratory.
  • JNRH 2024 gathered 54 researchers from the French robotics community, well in line with the expected attendance (about 50 people). Thirty attendees were junior researchers (PhD students and postdocs). The scientific program included 2 invited talks from prominent international speakers, 16 oral presentations of selected contributions, and 9 posters.

7 New software, platforms, open data

  • OpenSoT: The Open Stack of Tasks (OpenSoT) library is a framework for instantaneous whole-body planning and control based on Quadratic Programming (QP) optimization. It has been developed in the last 10 years by Enrico Mingo Hoffman . OpenSoT is now maintained and extended as a joint collaboration between Inria (LARSEN Team) and the Italian Institute of Technology (HHCM Lab). GitHub
  • CartesI/O: CartesI/O is a framework to interface high-level controllers and simplify the interaction between user and Inverse Kinematics or Inverse Dynamics algorithms. Enrico Mingo Hoffman is one of the main developers. CartesI/O is now maintained and extended as a joint collaboration between Inria (LARSEN Team) and the Italian Institute of Technology (HHCM Lab). GitHub

7.1 New software

7.1.1 Prescyent

  • Name:
    Prescyent
  • Keywords:
    Machine learning, Robotics, Telerobotics, Timeseries Prediction
  • Functional Description:

    PreScyent is a trajectory forecasting library, built upon pytorch_lightning. It comes with datasets such as:

    • AndyData-lab-onePerson
    • AndyData-lab-onePersonTeleoperatingICub
    • Human3.6M
    • And synthetics dataset to test simple properties of our predictors.

    It come also with baselines to run over theses datasets, referred in the code as Predictors, such as:

    • SiMLPe a MultiLayer Perceptron (MLP) with Discrete Cosine Transform (DCT), shown as a strong baseline achieving SOTA results against bigger and more complicated models.
    • Seq2Seq, an architecture mapping an input sequence to an output sequence, that originated from NLP and grew in popularity for time series predictions. Here we implemented an RNN Encoder and RNN Decoder.
    • Probabilistic Movement Primitives (ProMPs), an approach commonly used in robotics to model movements by learning from demonstrations and generating smooth, adaptable trajectories under uncertainty.
    • Some simple ML Baselines such as a Fully Connected MultiLayer Perceptron and an auto-regressive predictor with LSTMs.
    • Non machine learning baselines, maintaining the velocity or positions of the inputs, or simply delaying it.
  • News of the Year:
    - First public release - Support for many datasets - Support for many baselines - Full documentation, including tutorials
  • URL:
  • Contact:
    Jean-Baptiste Mouret
  • Participants:
    Jean-Baptiste Mouret, Alexis Biver

7.2 New platforms

Participants: Jean-Baptiste Mouret, Serena Ivaldi, Olivier Rochel, Fabio Amadio, Clemente Donoso.

  • We upgraded our Tiago robot (PAL Robotics) to a dual-arm version with an omnidirectional mobile base. We modified the robot to include new cameras and computing capabilities. This is the main platform for the euROBIN project.
  • We created a teleoperation station with 5 screens and all the required hardware (2 master arms with 6 degrees of freedom, networking, etc.)
  • We acquired a second Tiago omnidirectional base (PAL Robotics) to design a new mobile robot for the CPER O2R.

8 New results

8.1 Lifelong autonomy

8.1.1 Planning, decision, and control

Heuristic search for (partially observable) stochastic games

Participants: Olivier Buffet, Aurélien Delage.

Collaboration with Jilles Dibangoye (University of Groningen) and Johan Peralez (INSA-Lyon, INRIA team CHROMA). This line of research is pursued through Jilles Dibangoye's ANR JCJC PLASMA.

Many robotic scenarios involve multiple interacting agents, robots or humans, e.g., security robots in public areas. After addressing in the past the collaborative setting, where all agents share one objective 2, we have applied a similar approach in the important 2-player zero-sum setting, i.e., with two competing agents, and proposed an algorithm for partially observable Stochastic Games (POSGs), turning the problem into an occupancy Markov game, and deriving bounding approximators that build on two types of continuity properties: Lipschitz-continuity, and convexity and concavity properties.

This year, we have proposed an approach that exploits a vertical information-sharing hierarchy among a group of agents (i.e., an agent i knows what all its subordinates (i+1 ...N) do and perceive) to speed up convergence time. The resulting algorithm relies on two nested dynamic programming processes, the outer one reasoning about time steps, and the inner one working with the hierarchy of agents. This work has been published in 24

Also, the ANR project was concluded with the PhD defense of Aurélien Delage on June 28 33.

Interpretable Solutions for Stochastic Dynamic Programming

Participants: Olivier Buffet.

Work in collaboration with Iadine Chadès (CSIRO), Jonathan Ferrer-Mestres (CSIRO), and Thomas Dietterich (Oregon State University).

We have a long-standing collaboration with Iadine Chadès on developing algorithms for sequential decision-making under uncertainty, in particular in the context of conservation biology. In recent years we have proposed a method to derive more interpretable action policies, in the fully observable setting of Markov decision processes, by aggregating states post hoc (i.e., after solving the problem and deriving a first policy). This year, we have prepared a publication presenting this approach to biologists, including experimental results on three case studies 36.

Multi-contact whole-body control for humanoid robots

Participants: Serena Ivaldi, Jean-Baptiste Mouret, Quentin Rouxel, Dionis Totsila.

Humans often use additional contact points to enhance their stability, for instance, by using a handrail or a wall when walking, or to extend their reach, for instance, when grasping a distant object. While humanoid robots would benefit from a similar strategy, current robots minimize the number of contacts and use them only for feet and required interactions with the environment, such as pushing a button.

In 2024, we worked on multi-contact control and contact choice along three axes:

  • We introduced a new model-based whole-body controller that can regulate the contact forces even when the robot is position-controlled; this controller leverages the flexibility introduced by the position controllers (PIDs). Publication: 12
  • Exploiting this controller, we introduced a flow matching algorithm designed to learn autonomous multi-contact policies or multi-contact assistance (e.g., the operator moves the left arm and the robot decides to add a contact with the right arm) from demonstrations. Publication: 25
  • Also using the same multi-contact controller, we introduced a pipeline that makes it possible to choose a contact from voice interaction (e.g., "put your hand next to the red book"). It leverages a Visual Language Model (VLM) and Large Language Model (LLM). Publication: 26

All these contributions have been demonstrated on the humanoid robot Talos.

Modeling and control of floating-base systems including hybrid serial-parallel kinematics chains

Participants: Enrico Mingo Hoffman, Dionis Totsila.

Work in collaboration with PAL Robotics.

Recent examples of humanoid bipedal systems feature novel mechanical designs based on hybrid serial-parallel kinematic chains. These designs improve mass and inertia distribution by relocating actuators closer to the base link and transferring motion to the joints, resulting in enhanced equivalent Cartesian inertia at the end-effector. However, modeling such robotic systems necessitates additional considerations in both kinematics and dynamics.

In our work presented in 10, we introduced a theoretical framework for modeling the kinematics and dynamics of these systems with a minimal number of Degrees of Freedom. The target platform for this study is Kangaroo from PAL Robotics. The proposed methodology also enables the derivation of algorithms for calibration, which are often essential for robots of this type. The algorithms for forward and inverse differential kinematics have also been implemented using the recent mjx library (MuJoCo implemented in Brax) and have undergone preliminary testing.

This work is part of a series focused on whole-body control and methodologies for formulating and solving whole-body control problems in highly articulated systems. Recently, we presented a paper on a software library designed to address such problems 8. This library has been used in the laboratory to control the Tiago robot, including its omnidirectional mobile base, while ensuring the avoidance of self-collisions.

8.1.2 Learning and optimization

Epistemic Reinforcement Learning

Participants: Olivier Buffet.

On-going collaboration with Geoffrey Laforest, Alexandre Niveau and Bruno Zanuttini (GREYC, Université de Caen Normandie) as part of the ANR project EpiRL.

Dynamic epistemic logic allows reasoning about an agent's knowledge and its evolution given the occurrence of events. Recent work has developed epistemic planning, i.e., seeking for a sequence of actions that leads to some state of knowledge (e.g., knowing the value of some variable, or whether some fact is true or not). The EpiRL ANR project aims at performing a similar task through (possibly deep) reinforcement learning, i.e., learning a behavior by trial and error. In this context, the PhD thesis of Geoffrey Laforest in Caen (supervised by Bruno Zanuttini and Alexandre Niveau) has recently started (November 1st, 2023), and looks in particular at the choice of suitable representations for the state of knowledge (which should be compact, and ideally embed probabilities, due to the stochastic dynamics).

Quality diversity algorithms

Participants: Timothée Anne, Jean-Baptiste Mouret.

Quality diversity algorithms are black-box optimization algorithms that search for a large set of high-quality solution that all behave differently; we co-introduced them in 2015 56. They have numerous uses, from chemistry to video games, but we mostly use them in the team for machine learning and robotics. In 2024, we obtained results on two topics:

  • a data-efficient quality diversity algorithm inspired by Bayesian optimization with Gaussian processes, with a budget of 1000-2000 evaluations (versus 50k to 100k for MAP-Elites), in collaboration with the University of Warwick and Autodesk Research; publication: 9.
  • an extension of the MAP-Elites algorithm to black-box parametric task optimization, that is, when a parametrized function has to be optimized for all the possible values of the parameters simultaneously. More formally, we search for G(θ):

    θΘ,G(θ)=xθ*=argmaxx𝒳(f(x,θ)) 1

    where 𝒳 is the solution space, Θ the task parameter space, f:𝒳×Θ the function to optimize, and G:Θ𝒳 a function that outputs the optimal solution xθ* that maximizes f for the task parameter θ. Publication: 17

Addressing Reachability and Discrete Component Selection in Manipulator Design through Kineto-Static Bi-Level Optimization

Participants: Enrico Mingo Hoffman.

Collaboration with PAL Robotics, University of Trento (Prof. Andrea del Prete), and ETH (Dr. Gabriele Fadini).

Designing robotic manipulators for generic tasks while meeting specific requirements is a complex, iterative process involving mechanical design, simulation, control, and testing. New computational design tools are needed to simplify and speed up such processes.

In 7, I presented an original formulation of the computational design problem, tailored to help design generic manipulators with strong reachability requirements. The primary challenges addressed in this work are twofold. First, the necessity to consider the design of both continuous quantities (e.g., link lengths, joint placements) and discrete components (e.g., motor sizes and reduction ratios). Second, the ability to guide the design using high-level requirements, like the robot's workspace, without needing a specific manipulation task, unlike other co-design frameworks. These two challenges are addressed by employing a novel kineto-static formulation, resulting in a Mixed Integer Nonlinear Programming problem, which is solved using bi-level optimization.

A compelling use case from a real industrial application is also presented to highlight the practical effectiveness of the proposed method.

Learning in multi-robot and swarm contexts

Participants: Amine Boumaza.

Designing behaviors in the multi-robots context is a challenging open problem which we address using embodied evolutionary robotics (EER). These are algorithms in which optimization is carried out in a decentralized way, where each mobile robot runs an evolutionary algorithm on board and exchanges genetic material with other agents when they meet. In this context, we were interested in analyzing the features of the environment that favors evolution of certain behaviors. We hypothesize that certain types of perceivable artifacts in the environment provide affordances that exert a selection pressure toward the fixation of certain behaviors. We test this hypothesis in simulation in two different settings an open-ended environment without any selection pressure and compare it to one in which originality of behaviors is enforced. The experiments showed that in the open-ended environment agents evolved a certain type of behavior that exploited affordances, hinting at the existence of a selection pressure from the affordances as it is claimed from ecological psychology in the case of natural systems.

This year, this line of work led to a publication 18.

8.2 Natural interaction with robotics systems

8.2.1 Planning for Human-Robot collaboration

Task-planning for human robot collaboration

Participants: Yang You, Francis Colas, Olivier Buffet, Vincent Thomas.

This work has been done as part of the ANR project Flying CoWorker.

This work is part of the ANR project Flying Co-Worker (FCW) and focuses on high-level decision making for collaborative robotics. When a robot has to assist a human worker, it has no direct access to the worker's current intention or preferences but has to adapt its behavior to help the human complete his task.

Human-robot collaboration often necessitates the robot to adapt to the uncertainty of human objectives and their induced behaviors. This may require the robot to have a human model to anticipate human partners' objectives and predict their actions, which is typically learned by the robot through available human data. However, in complex collaboration tasks, a chicken-and-egg problem arises because human data cannot be collected without a collaborative robot policy in the first place. We had previously proposed to describe the human-robot collaboration task with Markov decision models and to solve the chicken-and-egg problem through a probabilistic planning algorithm. This year, we have contributed an online version of this approach. This online framework can automatically derive a human model without real human data and plan robust robot actions to support human partners with respect to their uncertain objectives and behaviors. Through experiments with a human-robot co-working scenario, we demonstrate that our online method outperforms the previous offline approach in terms of scalability and the ability to plan robot actions within a bounded time.

This work has been presented in 28.

Adaptive control of collaborative robots for preventing musculoskeletal disorders

Participants: Aya Yaacoub, Francis Colas, Vincent Thomas, Pauline Maurice.

This work is part of Pauline Maurice's ANR JCJC ROOIBOS project.

The use of collaborative robots in direct physical collaboration with humans constitutes a possible answer to musculoskeletal disorders: not only can they relieve the worker from heavy loads, but they could also guide them towards more ergonomic postures. In this context, one objective of the ROOIBOS Project is to build adaptive robot strategies that are optimal regarding productivity but also the long-term health and comfort of the human worker, by adapting the robot behavior to the human's physiological state.

To do so, in a previous work (published in 2023), we proposed to use Partially Observable Markov Decision Processes (POMDP) to compute a robot policy taking into account the long-term consequences of the biomechanical demands on the human worker's joints (joint loading) and to distribute the efforts among the different joints during the execution of a repetitive task. The proposed platform merges within the same framework several works conducted in the LARSEN team, namely virtual human modeling and simulation, fatigue estimate and decision making in the face of uncertainties.

However, one important challenge for computing a cobot policy lies in the continuous nature of the action and the state space. This year, we focused on how to automatically extract a small discrete set of relevant actions from the continuous action space. We proposed to combine a digital human simulation to estimate the fatigue induced by possible actions, with a repeated short-term planning (greedy-based selection approach) phase that explores the fatigue space and simultaneously identifies optimal actions from a large space for each visited state. By repeating this process starting from a diversity of initial belief states, and retaining actions used in the short-term planning, this process allows us to extract a subset of relevant actions.

We are now designing an experiment to validate the effectiveness of the proposed POMDP-based planning approach for fatigue mitigation with human subjects, in a human-robot comanipulation task.

This year, this line of work led to several publications: 27, 43.

Explicability and interpretability in probabilistic planning

Participants: Salomé Lepers, Maëlle Juilliot, Olivier Buffet, Vincent Thomas.

Part of this work is a collaboration with Shuwa Miura and Shlomo Zilberstein from University of Massachusetts (UMass) at Amherst.

In a human-agent collaboration scenario, some properties of the agent behavior can be useful for the human and sometimes allow a better collaboration. These properties include, for instance, legibility (legible behaviors convey intentions, i.e., actual task at hand, via action choices), explicability (explicable behaviors conform to observers' expectations, i.e., they appear to have some purpose), and predictability (a behavior is usually considered predictable if it is easy to guess the end of an on-going trajectory). Recent theoretical frameworks allow formalizing such properties and proposing algorithms to enforce them. In the framework of Salomé Lepers' PhD thesis, we build in particular on Miura and Zilberstein's OAMDP framework (observer aware Markov decision process), where an agent interacts with a stochastic environment while trying to optimize a criterion that depends on an external observer's belief. Note: The agent often has a type that the observer may only estimate through a belief (probability distribution).

We have first looked in particular at predictability, where the end of the current trajectory may highly depend on the outcome of each action, and thus proposed that predictability should be about minimizing the number of errors when an external observer is asked repeatedly to guess the next action or next state. This has been formalized in a variant of the observer-aware Markov Decision Process (OAMDP) formalism, naturally coming with simple algorithms that efficiently find optimal solutions. This work, which includes both in silico and in vivo experiments (where actual humans observe the behavior of artificial agents), is presented in 37.

We have also started informally collaborating with Shuwa Miura and Shlomo Zilberstein themselves on OAMDPs. This led in particular to demonstrating that the optimal value function of an OAMDP—which may exhibit discontinuities in some areas, and proposing an approximate solving algorithm that can be applied when specific Lipschitz-continuity assumptions hold 22.

The main direction we have been pursuing as part of Salomé Lepers' PhD thesis is to allow for an observer with partial and noisy observability (the agent knowing exactly what the observer perceives), which led to introducing the PO-OAMDP formalism (partially observable OAMDPs). We have shown that this formalism is a strict generalization of OAMDPs, and that the current state of the system, along with the observer's belief about that state, make for a sufficient statistic for optimal planning. This allowed proposing a variant of the heuristic search value iteration algorithm that relies on a pointwise approximator. We also demonstrated that, in stochastic shortest-path problems, some information-oriented criteria may not induce policies that reach a terminal state with probability 1, what can be fixed by adding a per-step cost. These results, with illustrations of the resulting behaviors on various problems, are presented in 29.

8.2.2 Exoskeleton and ergonomics

Identifying human movement strategies for human-robot collaboration

Participants: Jessica Colombel, Pauline Maurice, Vincent Thomas, Francis Colas.

This work is part of the ANR project ROOIBOS, coordinated by Pauline Maurice

In human-robot physical collaboration, it is necessary that the robot can anticipate the whole-body posture of the human co-worker to enable a seamless and efficient collaboration. When co-manipulating an object, the human posture is partly guided by the pose of the robot end-effector. However owing to the high kinematic redundancy of the human body, an infinity of postures can in theory be adopted for a same hand pose. In practice, human movements are largely stereotyped, which drastically reduces the number of observed solutions. Yet some diversity remains, which we refer to as “movement strategies”. The objective of this work is to develop a methodology to identify human movement strategies in a manual task, and explore the relations between movement strategies annd factors such as anthropometry and physical fatigue. During the postdoctoral work of Jessica Colombel, we designed an experiment and conducted a large data collection campaign, to acquire human motion data in a repetitive manual task to work on. We started to analyze the data, and explored diffusion methods to cluster the data in different movement strategies. Inverse optimal control is also a method that we plan to explore. Jessica Colombel has now left, but we plan to continue working on the data, and hire new people to continue exploring the topic, in 2025.

This line of work led to a preliminary publication as an abstract and presentation in a French biomechanics conference 19, 5.

Motion prediction for active exoskeleton control

Participants: Alexandre Oliveira Souza, Raphaël Bousigues, Francois Charpillet, Pauline Maurice, Serena Ivaldi.

This work is a joint PhD thesis with Safran (supervisors: Jordane Grenier and Christophe Guettier).

Occupational exoskeletons are a promising solution to physically assist people in strenuous tasks, such as load carrying. Compared to passive exoskeletons, active exoskeletons are more powerful and more versatile, so they can offer a better assistance for a wide variety of tasks. However, their interaction with the user remains a problem currently because there is usually a delay in the assistance, and the selection of the assistance remains often manual. Hence motion prediction could be a promising way to improve exoskeleton control by anticipating the required assistance.

In a previous work (published in 2023), we proposed to use physics-based simulation to generate synthetic human-exoskeleton motion data, in order to train a prediction model. This year, we conducted an experiment with human participants, to evaluate the benefit of including motion prediction in active exoskeleton control, with respect to exoskeleton transparency. We designed a pointing task, that participants had to perform with an active elbow exoskeleton, either turned off, or controlled with 4 different controllers: two state-of-the-art non-predictive controllers, and 2 predictive controllers that we designed (based on a multi-layer-perceptron). The experiment showed that adding motion prediction leads to an increased transparency measured by sensors, even though it was not always perceived by the human user. We are currently extending this work to a load manipulation task, closer to a real use-case. We are also developing further the human-exoskeleton simulation, to explore using it to pre-train the prediction models before transferring them to human subject experiment.

It should be noted that the PhD work of Alexandre Oliveira Souza was initially planned with using an active upper-limb exosksleton prototype provided by Safran. However, the development of the prototype (not part of the PhD work) was significantly delayed, and will not be available during the PhD timeframe. As such, and because no suitable exoskeleton was commercially available for testing our controllers, we (Larsen team) developed a dedicated exoskeleton prototype (elbow assistance). This engineering work was lead by Raphaël Bousigues, who was hired for this purpose.

This year, this line of work led to a publication 23. The work was also presented by the PhD student at the Journée des Jeunes Chercheurs en Robotique in Paris.

Effect of Induced Postural Changes on Cognitive Load

Participants: Elissa El Khalil, Sophie Lemonnier, Pauline Maurice.

This work is part of the ANR project ROOIBOS, coordinated by Pauline Maurice

Collaborative robots can be used to alleviate physical load on the human co-worker. One recent line of research is to use cobots to induce postural changes in the human during repetitive task, because movement variability has been suggested to be beneficial to reduce the risk of developing musculoskeletal disorders. This can be done by changing the position in space where the robot co-worker presents the object to work on to the human. While such changes may be beneficial to reduce physical fatigue, they might induce an increase in cognitive load for the human user, compared to an entirely predictive task. Thus we designed a human subject experiment to study the effect of imposed postural changes on both physical and cognitive loads, in a repetitive manual task. This experiment was conducted without a cobot, since the objective was to focus on the human postural changes, and not on the cognitive effects of the robot presence. We compared 5 different conditions: fixed working position, predictable changes (small and large) and random changes (small and large). We showed that postural changes were indeed beneficial to reduce perceived physical fatigue, but at the cost of increased cognitive load.

This line of work led to a publication 40, and a full paper containing more in-depth analyses of the results is in preparation.

9 Bilateral contracts and grants with industry

9.1 Bilateral grants with industry

PhD grant with SAFRAN

Participants: François Charpillet, Pauline Maurice, Serena Ivaldi, Alexandre Oliveira Souza.

Collaboration with Jordane Grenier (Safran) and Christophe Guettier (Safran).

The thesis is funded by Safran to develop the AI-based control of their hybrid exoskeleton, based on the one developed in the DGA-Rapid project ASMOA. It consists in developing methods to predict the amount of assistance that is needed by the human in tasks invovling payload manipulation.

PhD work co-advised with CEA-LIST

Participants: Jacques Zhong, Francis Colas, Pauline Maurice.

Collaboration with Vincent Weistroffer (CEA-LIST) and Claude Andriot (CEA-LIST)

This PhD work started in October, 2020. The objective is to develop a software tool that allows taking into account the diversity of workers' morphology when designing an industrial workstation. The developed tool will enable us to test the feasibility and ergonomics of a task for any morphology of workers, based on a few demonstrations of the task in virtual reality by one single worker. The two underlying scientific questions are i) the automatic identification of the task features from a few virtual reality (VR) demonstrations, and ii) the transfer of the identified task to digital human models of various morphologies.

PhD work co-advised with CEA-LIST

Participants: Quentin Rolland, Jean-Baptiste Mouret.

Collaboration with Fabrice Mayran de Chamisso (CEA-LIST) and Claude Andriot (CEA-LIST)

This PhD work started in October, 2024. The objective is to leverage anomaly detection “one-class” learning algorithms to improve imitation learning in robotics.

10 Partnerships and cooperations

10.1 International research visitors

10.1.1 Visits of international scientists

Other international visits to the team
Nicola Scianca
  • Status:
    researcher
  • Institution of origin:
    University of Rome “La Sapienza”'
  • Country:
    Italy
  • Dates:
    2024-06-15–2024-07-15
  • Context of the visit:
    work with Enrico Mingo Hoffman on optimal control and model scheduling for legged systems
  • Mobility program/type of mobility:
    invited researcher

10.2 European initiatives

10.2.1 Horizon Europe

euROBIN

Participants: Serena Ivaldi, Jean-Baptiste Mouret, Enrico Mingo Hoffman, Quentin Rouxel, Dionis Totsila, Nicolas Beaufort.

euROBIN project on cordis.europa.eu

  • Title:
    European ROBotics and AI Network
  • Duration:
    From July 1, 2022 to June 30, 2026
  • Partners:
    • INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE (INRIA), France
    • C.R.E.A.T.E. CONSORZIO DI RICERCA PER L'ENERGIA L AUTOMAZIONE E LE TECNOLOGIE DELL'ELETTROMAGNETISMO (C.R.E.A.T.E.), Italy
    • PAL ROBOTICS SLU (PAL ROBOTICS), Spain
    • KUNGLIGA TEKNISKA HOEGSKOLAN (KTH), Sweden
    • INSTITUT JOZEF STEFAN (JSI), Slovenia
    • FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV (Fraunhofer), Germany
    • FUNDACION TECNALIA RESEARCH & INNOVATION (TECNALIA), Spain
    • TECHNISCHE UNIVERSITAET MUENCHEN (TUM), Germany
    • DHL EXPRESS SPAIN SL, Spain
    • COMMISSARIAT A L ENERGIE ATOMIQUE ET AUX ENERGIES ALTERNATIVES (CEA), France
    • INTERUNIVERSITAIR MICRO-ELECTRONICA CENTRUM (IMEC), Belgium
    • TEKNOLOGISK INSTITUT (DANISH TECHNOLOGICAL INSTITUTE), Denmark
    • UNIVERSITEIT TWENTE (UNIVERSITEIT TWENTE), Netherlands
    • ASEA BROWN BOVERI SA (ABB), Spain
    • ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE (EPFL), Switzerland
    • MATADOR INDUSTRIES AS, Slovakia
    • DEUTSCHES ZENTRUM FUR LUFT - UND RAUMFAHRT EV (DLR), Germany
    • IST-ID ASSOCIACAO DO INSTITUTO SUPERIOR TECNICO PARA A INVESTIGACAO E O DESENVOLVIMENTO (IST ID), Portugal
    • UNIVERSITA DI PISA (UNIPI), Italy
    • FUNDINGBOX ACCELERATOR SP ZOO (FBA), Poland
    • UNIVERSITAET BREMEN (UBREMEN), Germany
    • FONDAZIONE ISTITUTO ITALIANO DI TECNOLOGIA (IIT), Italy
    • KARLSRUHER INSTITUT FUER TECHNOLOGIE (KIT), Germany
    • EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH (ETH Zürich), Switzerland
    • CESKE VYSOKE UCENI TECHNICKE V PRAZE (CVUT), Czechia
    • OREBRO UNIVERSITY (ORU), Sweden
    • CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE CNRS (CNRS), France
    • VOLKSWAGEN AKTIENGESELLSCHAFT (VW AG), Germany
    • SIEMENS AKTIENGESELLSCHAFT, Germany
    • SORBONNE UNIVERSITE, France
    • UNIVERSIDAD DE SEVILLA, Spain
  • Inria contact:
    Serena Ivaldi
  • Coordinator:
    Prof. Dr. Alin Albu-Schäffer (DLR)
  • Summary:

    As robots are entering unstructured environments with a large variety of tasks, they will need to quickly acquire new abilities to solve them. Humans do so very effectively through a variety of methods of knowledge transfer – demonstration, verbal explanation, writing, the Internet. In robotics, enabling the transfer of skills and software between robots, tasks, research groups, and application domains will be a game changer for scaling up the robot abilities.

    euROBIN therefore proposes a threefold strategy: First, leading experts from the European robotics and AI research community will tackle the questions of transferability in four main scientific areas: 1) boosting physical interaction capabilities, to increase safety and reliability, as well as energy efficiency 2) using machine learning to acquire new behaviors and knowledge about the environment and the robot and to adapt to novel situations 3) enabling robots to represent, exchange, query, and reason about abstract knowledge 4) ensuring a human-centric design paradigm, that takes the needs and expectations of humans into account, making AI-enabled robots accessible, usable and trustworthy.

    Second, the relevance of the scientific outcomes will be demonstrated in three application domains that promise to have substantial impact on industry, innovation, and civil society in Europe. 1) robotic manufacturing for a circular economy 2) personal robots for enhanced quality of life 3) outdoor robots for sustainable communities. Advances are made measurable by collaborative competitions.

    Finally, euROBIN will create a sustainable network of excellence to foster exchange and inclusion. Software, data and knowledge will be exchanged over the EuroCore repository, designed to become a central platform for robotics in Europe.

    The vision of euROBIN is a European ecosystem of robots that share their data and knowledge and exploit their diversity to jointly learn to perform the endless variety of tasks in human environments.

10.3 National initiatives

10.3.1 PEPR O2R: AS3

Participants: Serena Ivaldi, Jean-Baptiste Mouret, Enrico Mingo Hoffman, Pauline Maurice.

  • Program:
    PEPR
  • Project acronym:
    AS3
  • Project title:
    Decision, Apprentissage et Interaction Sociale
  • Duration:
    Janvier 2024 – Decembre 2031
  • Coordinator:
    Serena Ivaldi
  • Local coordinator:
    Serena Ivaldi
  • Abstract:
    Le défi scientifique majeur de cette action structurante est de jeter les bases de nouveaux algorithmes de décision, d’apprentissage et d’interaction centrés sur la société. Nous avons identifié 4 défis clés qui seront au cœur du développement scientifique de ce projet : l’anticipation et la prédiction humaine, l’interaction multimodale, l’apprentissage au cours des interactions et la confiance. Notre approche pour relever ces défis consiste à concevoir et à mener des études d’observation conjointes sur le terrain, par des experts en robotique et en sciences sociales. L’analyse conjointe des études de terrain aura un impact sur la conception de nouvelles théories, de nouveaux modèles et de nouveaux algorithmes, en tenant compte des aspects humains et sociétaux de ces défis. Dans la première partie du projet, le consortium se concentrera sur les manipulateurs mobiles, en utilisant des plateformes facilement disponibles dans le consortium pour mener des expériences avec des humains en public et sur les lieux de travail. Dans la seconde partie du projet, le consortium élargira son champ d’investigation aux robots portables, avec un degré plus élevé d’incarnation et d’interaction physique. L’objectif est d’informer le développement des plateformes dans les PI1, PI2 et PI3 et d’identifier les liens bidirectionnels avec AS1, AS2 et AS4.

10.3.2 PEPR O2R: PI3

Participants: Serena Ivaldi, Pauline Maurice.

  • Program:
    PEPR
  • Project acronym:
    PI3
  • Project title:
    ASSISTMOV
  • Duration:
    Janvier 2024 – Decembre 2031
  • Coordinator:
    Franck Geffard (CEA)
  • Local coordinator:
    Serena Ivaldi
  • Abstract:
    Le projet intégré PI3 “ASSISTMOV”, composé d’une équipe pluridisciplinaire ingénierie et de Sciences Humaines et Sociales (SHS), cible le cas d’usage de l’utilisation de la robotique d’assistance au mouvement de personnes en situation de handicap (PSH). A travers le développement d’une gamme d’exosquelettes (membres inférieurs et supérieurs), ce projet vise une technologie de rupture pour une interaction fluide et robuste à la variété d’environnements et d’usages (de la rééducation à la vie quotidienne). Le projet suivra la philosophie proposée dans ce PEPR dont le but est de repenser la conception des robots depuis le matériel jusqu’au logiciel, de manière à favoriser l’adaptation sociale et l’inclusion. Centrée sur une vision globale de l’usage dans son écosystème, cette approche innovante intégrant les SHS, questionnera la pertinence des orientations technologiques existantes et projetées que ce soit par rapport au membre supérieur (MS) ou inférieur (MI). Les objectifs sont de proposer des démonstrateurs robotiques socialement adaptés (Challenge 1) en garantissant une interaction fluide(Challenge 3) basée sur une architecture matérielle et logicielle robuste à une variété d’environnements et d’usages (Challenge 2).

10.3.3 ANR : OSTENSIVE

Participants: Serena Ivaldi, Jean-Baptiste Mouret.

  • Program:
    ANR
  • Project acronym:
    OSTENSIVE
  • Project title:
    Ostensive Human-Robot Interaction
  • Duration:
    April 2025 – April 2028
  • Coordinator:
    Mohamed Chetouani (Sorbonne Université)
  • Local coordinator:
    Serena Ivaldi
  • Abstract:
    When humans demonstrate a task to another human or agent, they go beyond merely manipulating the targeted object (instrumental action) to add to their actions ostensive communicative cues such as eye gaze and/or modulations of the demonstrations in the space–-time dimensions (belief–directed action). This modulation results in behaviors that might appear to be sub–optimal, such as pause, repetition and exaggeration, but they are provided to communicate additional information. Recent research in Cognitive Science addressed this challenge of communication in action. Similarly, when robots have to perform actions, there is a need for mechanisms of communication in action allowing to combine instrumental and belief-directed actions. In robotics, this is known as the instantiation of legible robot motion (transparency through motion) by which a robot communicates its intent to a human observer. OSTENSIVE will provide novel solutions to study and develop human-robot interaction systems that are conceptually human centric by explicitly combining instrumental and belief-directed dimensions at key stages such as human behavior perception, human/robot motion representations, robot motion synthesis and simulation to real transfer. We will consider several approaches for (de)coupling instrumental and belief-directed actions by leveraging research in cognitive science and exploiting multi-task learning in order to explicitly consider dual components of human and robot actions.

10.3.4 ANR : MeRLin: Multi-limbed Robots empowered by whole-body Loco-manipulation.

Participants: Enrico Mingo Hoffman, Ioannis Tsikelis, Serena Ivaldi, Jean-Baptiste Mouret.

  • Program:
    ANR
  • Project title:
    MeRLin
  • Duration:
    April 2025 – September 2028
  • Coordinator:
    Enrico Mingo Hoffman (INRIA, Nancy)
  • Abstract:

    The MeRLin project addresses the limitations of current robotics in performing hazardous and strenuous tasks, which are still predominantly carried out by human workers in industries like construction and heavy manufacturing. Such tasks often expose workers to significant safety and health risks, including long-term musculoskeletal disorders. Motivated by the need to create safer working environments and enhance productivity, the project focuses on developing advanced robotics capable of operating in unstructured and dynamic environments.

    The project targets the development of a robot-agnostic framework that enables multi-limbed robots to perform complex loco-manipulation tasks by coordinating their entire bodies.

10.3.5 ANR: ROOIBOS

Participants: Pauline Maurice, Francis Colas, Vincent Thomas.

  • Program:
    ANR JCJC
  • Project acronym:
    ROOIBOS
  • Project title:
    User-Specific Adaptation of Collaborative Robot Motion for Improved Ergonomics
  • Duration:
    March 2021 – August 2025
  • Coordinator:
    Pauline Maurice (CNRS)
  • Summary:
    Collaborative robots have the potential to reduce work-related musculoskeletal disorders not only by decreasing the workers' physical load, but also by modifying and improving their postures. Imposing a sudden modification of one's movement can however be detrimental to the acceptance and efficacy of the human-robot collaboration. In ROOIBOS, we will develop a framework to plan user-specific trajectories for collaborative robots, to gradually optimize the efficacy of the collaboration and the long-term occupational health of the user. We will use machine learning and probabilistic methods to perform user-specific prediction of whole-body movements. We will define dedicated metrics to evaluate the movement ergonomic performance and intuitiveness. We will integrate those elements in a digital human simulation to plan a progressive adaptation of the robot motion accounting for the user's motor preferences. We will then use probabilistic decision-making to adapt the plan on-line to the user's motor adaptation capabilities. This will enable a smooth deployment of collaborative robots at work.

10.3.6 ANR : EpiRL

Participants: Olivier Buffet.

  • Program:
    ANR
  • Project title:
    Apprentissage par renforcement épistémique
  • Duration:
    February 2023 – February 2027
  • Coordinator:
    François Schwarzentruber (IRISA, École normale supérieure de Rennes)
  • Abstract:

    EpiRL project aims at investigating the combination of epistemic planning and reinforcement learning (RL), by proposing new algorithms that are efficient, adaptive, and capable of computing decisions relying on theory of knowledge and belief. We expect from this approach an efficiency in the generation of epistemic plans, while decisions made by RL algorithms will be explainable. Moreover, the algorithms of EpiRL will be tested and evaluated within a real application that exploits autonomous agents.

    The project will address the weaknesses of both epistemic planning and RL: on the one hand, existing epistemic planning algorithms are costly, do not adapt to the environment, and concepts are hand-crafted and are not learned; on the other hand, in reinforcement learning, agents adapt to their environments but are unable to reason about beliefs of other agents. The newly developed algorithms will combine the strengths of both fields.

    We propose four workpackages:

    1. Study representations of states
    2. Develop RL algorithms
    3. Study representations of policies
    4. Validating our algorithms with our industrial partner DAVI. In particular, we aim at developing a debunking chatbot whose use case will apply to raising awareness about environmental issues.

10.3.7 ANR: BUCOLYC

Participants: Jean-Baptiste Mouret, Thomas Martin.

  • Program:
    ANR
  • Project acronym:
    BUCOLYC
  • Project title:
    Papillons et drones en conditions de vol confiné : aérodynamique, biomimétisme et IA au service du contrôle et de la stabilisation
  • Duration:
    Sept. 2023 – Aug. 2027
  • Coordinator:
    Mickaël Bourgoin (Laboratoire de Physique de l'ENS Lyon)
  • Local coordinator:
    Jean-Baptiste Mouret
  • Abstract:

    While drone technology has matured for open-air flight, confined flight remains a major challenge, due to the aerodynamic interference induced by the complex couplings between the drone itself and the surrounding walls, which cause severe flight disturbances. This renders the usual UAV stabilization controls inoperative, and considerably reduces maneuverability. Our project aims to address this challenge by combining aerodynamic, biomimetic and machine learning approaches to improve UAV control and stability in confined, near-wall environments.

    To achieve this ambitious goal, our multidisciplinary consortium brings together experts in robotics and biorobotics, fluid mechanics and entomologists, as well as an industrial partner (XTim) recognized for its leadership in the market for biomimetic flapping-wing drones.

10.3.8 COMS@N: EXOCODESIM

Participants: Serena Ivaldi, Pauline Maurice.

  • Program:
    COMS@N Appel à Pre-Maturation
  • Project acronym:
    EXOCODESIM
  • Project title:
    Exoskeleton Co-design by Simulation
  • Duration:
    Octobre 2024 – Septembre 2025
  • Coordinator:
    Serena Ivaldi
  • Local coordinator:
    Serena Ivaldi
  • Abstract:
    Suite aux projet ExoTurn et ExoSim, nous souhiatons developper nos propres solutions exosquelettes à destination du CHRU de Nancy/ Cette demande de co-financement permettra de financer le co-développement de prototypes d'exosquelettes, validés par les outils de simulation devéloppées dans l'équipe, ainsi que par des tests utilisateurs sur site répondant aux objectifs d’une médecine 5P. Les résultats de ce projet ont vocation a être transferés vers une startup liée à l'équipe, qui serait porté par Raphael Bousigues et Raphael Lartot .

10.3.9 AID: ATOR (Assisted TeleOeration of Robots)

Participants: Jean-Baptiste Mouret, Serena Ivaldi.

  • Program:
    Inria-AID convention (Agence Innovation Defense)
  • Project acronym:
    ATOR
  • Project title:
    Assisted TeleOeration of Robots
  • Duration:
    01/01/2024 – 31/12/2028
  • Coordinator:
    Jean-Baptiste Mouret
  • Local coordinator:
    Jean-Baptiste Mouret
  • Abstract:
    The ATOR project aims at leveraging artificial intelligence algorithms to make it easier to teleoperate robots (typically mobile manipualtors). The main idea is to exploit a dataset of expert demonstrations to guide the hand of a non-expert, helping to understand in real-time “what would an expert do in that situation”. The project will propose novel, uncertainty-aware trajectory prediction algorithms, as well as demonstrations with the robots of the team.

10.4 Regional initiatives

10.4.1 LUE: EXOSIM

Participants: Serena Ivaldi, Pauline Maurice.

  • Program:
    LUE
  • Project acronym:
    EXOSIM
  • Project title:
    Simulation de tâches pénibles couramment réalisées en milieu hospitalier pour guider la recherche automatique de solutions exosquelettes pour assister les soignants
  • Duration:
    July 2024 – June 2026
  • Coordinator:
    Serena Ivaldi
  • Local coordinator:
    Serena Ivaldi
  • Abstract:
    Suite aux projet ExoTurn et ExoCare coordonnés par les deux porteuses de ce projet, nous souhaitons pérenniser la collaboration entre Loria et CHRU de Nancy sur l’utilisation d’outils numériques pour aider l’introduction, expérimentation et mise en œuvre d’exosquelettes dans le milieu sanitaire. Dans ce projet, nous souhaitons développer un nouveau pro- gramme de développement d’outils numériques pour 1) aider l’hôpital à identifier les exos- quelettes existants qui seraient potentiellement pertinents pour aider les soignants , et à défaut déterminer le cahier de charge d’un nouveau exosquelette à acquérir , à travers un logiciel de simulation physique de l’humain virtuel et d’évaluation ergonomique développé par le Loria ; 2) faire valider le logiciel par les ergonomes du CHRU de Nancy, ainsi que le choix du dispositif ; 3) doter l’hôpital d’instruments numériques intégrant évaluation subjec- tive (par questionnaires) et objective (par capteurs portés) pour suivre la campagne expéri- mentale de test des exosquelettes sur le court/moyen/longue terme, en utilisant ces don- nées pour guider le processus d’adoption de l’exosquelette.

11 Dissemination

11.1 Promoting scientific activities

General chair, scientific chair

11.1.1 Scientific events: organisation

Member of the organizing committees
  • Jean-Baptiste Mouret and Enrico Mingo Hoffman were local chair of the IEEE-RAS Conference on Humanoid Robots (Humanoids)
  • Pauline Maurice was publication chair of the IEEE-RAS Conference on Humanoid Robots (Humanoids)
  • Enrico Mingo Hoffman was part of the program committee of the Human Friendly Robotics (HFR) Conference 2024 .
  • Amine Boumaza was in the organizing commitee of the winter school "XIHO24 : Expérimenter l'interaction humain-objet : théories, méthodologies, analyses 2024" (site).
Workshop organization
  • Jean-Baptiste Mouret co-organized a half-day workshop at the IEEE Humanoids 2024 conference, with the support of the ADRA association – “Towards general-purpose robots: connecting generative artificial intelligence to humanoids”
  • Pauline Maurice was the main organizer of the Journées Nationales de la Robotique Humanoïde (JNRH) 2024 at Loria - Centre Inria de l'Université de Lorraine.
  • Pauline Maurice was co-organizer of a workshop on “Human Movement Modeling for Human-Robot Interation” at IEEE-RAS Humanoids 2024 (Nancy, France), and of a workshop on “Innovations and Applications of Human Modeling in Physical Human-Robot Interaction” at IEEE-RAS ICRA 2024 (Yokohama, Japan).
  • Enrico Mingo Hoffman organized the 2nd Edition of the Workshop on Advancements in Trajectory Optimization and Model Predictive Control for Legged Systems at the IEEE-RAS ICRA 2024 conference.

11.1.2 Scientific events: selection

Journals - editorial roles
  • Serena Ivaldi is Editor in Chief of the Springer International Journal of Social Robotics (until 09/2024)
  • Jean-Baptiste Mouret is associate editor for ACM Transactions on Evolutionary Learning and Optimization (TELO)
  • Jean-Baptiste Mouret is a member of the editorial board of Nature Robotics
  • Pauline Maurice is associate editor for IEEE Transactions on Neural Systems and Rehabilitation Engineering (TNSRE)
  • Enrico Mingo Hoffman serves as an Associate Editor for the IEEE Robotics and Automation Letters (RA-L). 2024 marks the final year of his 4-year term.
  • Enrico Mingo Hoffman serves as an Associate Editor for International Journal on Robotics Research (IJRR)
  • Enrico Mingo Hoffman serves as moderator for the arXiv category cs.RO
Conferences - editorial roles
  • Jean-Baptiste Mouret was an associate editor for the IEEE-RAS Conference on Humanoid Robots (Humanoids)
  • Enrico Mingo Hoffman serves as an Associate Editor for the IEEE-RAS Conference on Humanoid Robots (Humanoids), IEEE-RAS International Conference on Robotics and Automation (ICRA), IEEE-RAS International Conference on Ubiquitous Robots (UR), and IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
  • Pauline Maurice was an Associate Editor for IEEE-RSJ IROS 2024 (conference).
  • Serena Ivaldi was an Associate Editor for the IEEE-RAS International Conference on Robotics and Automation (ICRA).
Reviewer - reviewing activities
  • Jean-Baptiste Mouret reviewed an article for Nature Robotics
  • Enrico Mingo Hoffman reviewed articles for IEEE-RAS T-RO, IEEE-RAS RA-L , IEEE Access, 1 project for the Swiss National Science Foundation (SNSF), projects in the Cascade Funding RAISE Spoke 4 proposals 2024 (Machine Learning and real-time data) of the Italian Piano Nazionale di Ripresa e Resilienza (PNRR).
  • Pauline Maurice reviewed articles for IEEE Transactions on Robotics, Scientific Report, IEEE Transactions on Human Machine Systems, Int. J. of Industrial Ergonomics, Robotics and Autonomous Systems.
  • Olivier Buffet reviewed articles for the J. of Artificial Intelligence Research (JAIR), the Int. Journal on Approximate Reasoning (IJAR), and was a program committee member or reviewer for the European Conference on Artificial Intelligence (ECAI), the European Workshop on Reinforcement Learning (EWRL), the International Conference on Automated Planning and Scheduling (ICAPS), and the Journées Intelligence Artificielle Fondamentale - Journées Francophone sur la Planification, la Décision et l'apprentissage (JIAF-JFPDA).
  • Francis Colas reviewed articles for the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), the IEEE-RAS Conference on Humanoid Robots (Humanoids), the International Conference on Artificial Neural Networks (ICANN), and was a program committee member of the International Joint Conference on Artificial Intelligence (IJCAI) and the European Conference on Artificial Intelligence (ECAI).
  • Vincent Thomas reviewed articles for the International Conference on Cognitive Aircraft Systems (ICCAS), and was a program comittee member of the European Conference on Artificial Intelligence (ECAI), and the Journées Intelligence Artificielle Fondamentale - Journées Francophone sur la Planification, la Décision et l'apprentissage (JIAF-JFPDA).
  • Amine Boumaza was Program Committee Member both of the ACM Genetic and Evolutionary Computation Conference (GECCO'24) and of the Artificial Life Conference (ALIFE'24)
  • Serena Ivaldi reviewed for the Robotics & Automation Letters, IEEE Transactions on Robotics; she reviewed for the ERC Starting Grant call, the Inria Quadrant program, Innoviris Bruxelles, Prix de Thèse GDR Robotique, the ANR AAPG 2024 in the CE23 Artificial Intelligence call.

11.1.3 Invited talks

  • Jean-Baptiste Mouret was invited to the French-German-Japanese symposium “Generative AI: Pathways to democratization, transparency and sustainability” (Tokyo, Japan)
  • Pauline Maurice gave an invited talk in 2 French laboratories: LAAS-CNRS (robotics, Toulouse) and LBMC (biomechanics, Lyon)
  • Enrico Mingo Hoffman was invited to the Workshop on Real World Physical and Social Human-Robot Interaction held at IEEE-RAS HUMANOIDS 2024 (Nancy, France), JNRH 2024 (Nancy, France), Workshop on Human Movement Understanding, Whole-Body Control, and Human-Robot Interfaces held at IEEE CASE 2024 (Bari, Italy, online), Workshop Advancements in High Performance Humanoid & Legged Robot Functionalities held at ERF 2024 (Rimini, Italy)
  • Serena Ivaldi was invited to give presentations at the Club ORION of University of Lorraine, the ICRA 2024 Workshops on Human Modeling and PHRI, the ICRA 2024 Workshop on Teleoperation, the CORL 2024 Workshop on Whole-body Control and Bimanual Manipulation.

11.1.4 Leadership within the scientific community

  • Enrico Mingo Hoffman is serving as Corresponding Chair and representative for the primary region of Europe for the IEEE Technical Committee on Whole-Body Control (TC-WBC), with responsibility for the TC-WBC funds' management.
  • Pauline Maurice is a co-chair of the technical committee on “Humans and Robots” of the French GdR Robotique.
  • Serena Ivaldi is Vice-President of the Members Activities Board of the IEEE Robotics & Automation Society and member of the IEEE RAS HUMANOIDS Steering Committee.
  • Serena Ivaldi participated to the writing of the SRA (Strategic Research Agenda) of the euROBIN Network of Excellence in Robotics & AI, a strategic document for research in robotics and AI for the European Commission.

11.1.5 Research administration

  • Jean-Baptiste Mouret is Head of Science (Délégué Scientifique) of the Inria Center at Université de Lorraine (since March 1st, 2024)
  • Amine Boumaza is member of the Comité de Centre of Inria Centre at Université de Lorraine.
  • Francis Colas is member of the Comité de Centre and Comipers committees of Inria Centre at Université de Lorraine as well as the ComiDoc PhD hiring commission of the Loria laboratory.
  • Serena Ivaldi was Member of Bureau de Comité de Projets of the Inria Center at Université de Lorraine until 03/2024.

11.2 Teaching - Supervision - Juries

11.2.1 Teaching

  • Master: Vincent Thomas is co-responsible for the parcours “Intelligence Artificielle et ses Applications en Vision et Robotique” of the Computer Science Master, Univ. Lorraine and co-conducted the evolution of this parcours for the new training offer of Univ. Lorraine.
  • Master: Amine Boumaza , “Méta-heuristiques et recherche locale stochastique”, 30h eq. TD, M1 informatique, Univ. Lorraine, France.
  • Master: Amine Boumaza , “Modélisation de Phénomènes Biologiques ”, 12h eq. TD, M2 Sciences Cognitives IACH, Univ. Lorraine, France.
  • Master: Amine Boumaza , “Robotique collective”, 24h eq. TD, M2 Informatique IA²VR, Univ. Lorraine.
  • Tutorial: Olivier Buffet , “Planification dans l'incertain”, 2h CM, CNRS Formation Entreprise.
  • Master: Francis Colas , “Planification de trajectoires”, 17h eq. TD, M2 Informatique “Intelligence Artificielle et ses Applications en Vision et Robotique”, Univ. Lorraine, France.
  • Master: Francis Colas , “Ingénierie Logicielle pour l'Intelligence Artificielle et la Robotique”, 15h eq. TD, M2 Informatique “Intelligence Artificielle et ses Applications en Vision et Robotique”, Univ. Lorraine, France.
  • Master: Francis Colas , “Situation Intégratrice”, 36h eq. TD, M2 Informatique “Intelligence Artificielle et ses Applications en Vision et Robotique”, Univ. Lorraine, France.
  • Master: Serena Ivaldi , “Analyse Comportementale”, 16h CM/TD, M2 “Sciences Cognitives”, Univ. Lorraine, France.
  • Master: Pauline Maurice , “Analyse Comportementale”, 16h CM/TP, M2 “Sciences Cognitives”, Univ. Lorraine, France.
  • Master: Pauline Maurice , “Robotic assistive devices for occupational applications: From research to deployment”, 6h CM/TP, 3rd year (M2) in “Control Engineering”, Centrale-Supelec, France.
  • Master: Pauline Maurice , “Human motion analysis for human-robot interaction”, 6h CM, M2 Robotics and Biomechanics (joint class), Univ. of Lyon, France.
  • Master: Jean-Baptiste Mouret , “Quality Diversity and Creative Design”, 3h (M2 Innovation, Mines Paristech)
  • Master: Alexis Scheuer , “Introduction à la robotique autonome”, 30h eq. TD, M1 informatique, Univ. Lorraine, France.
  • Master: Alexis Scheuer , “Modélisation et commande en robotique”, 16h eq. TD, M2 “Apprentissage, Vision, Robotique”, Univ. Lorraine, France.
  • Master: Alexis Scheuer , “Éléments de robotique”, 4h eq. TD, Master MEEF 2d degré, INSPÉ, Univ. Lorraine, France.
  • Master: Vincent Thomas , “Apprentissage et raisonnement dans l'incertain”, 15h eq. TD, M2 M2 “Apprentissage, Vision, Robotique”, Univ. Lorraine, France.
  • Master: Vincent Thomas , “Planification Apprentissage et Controle pour la Robotique”, 15h eq. TD, M2 M2 “Intelligence Artificielle et ses Applications en Vision et Robotique”, Univ. Lorraine, France.
  • Master: Vincent Thomas , “Game Design”, 30h eq. TD, M1 Sciences Cognitives, Univ. Lorraine, France.
  • Master: Vincent Thomas , “Agent intelligent”, 15h eq. TD, M1 Sciences Cognitives, Univ. Lorraine, France.

11.2.2 Supervision

  • PhD: Timothée Anne , “L'optimisation multi-tâche et ses applications à la robotique : d'abord résoudre, ensuite généraliser', started in September 2020, defended 2024-06-06, Jean-Baptiste Mouret (advisor) 32.
  • PhD: Jacques Zhong , “Prise en compte de la variabilité morphologique d'opérateurs sur des chaines de montage en réalité virtuelle”', started in October 2020, defended 2024-06-25, Francis Colas (advisor), Pauline Maurice (co-advisor), and Vincent Weistroffer (co-advisor, CEA-LIST) 34.
  • PhD: Abir Bouaouda , “Apprentissage par renforcement pour le contrôle des robots parallèles à câbles”', started in October 2020, defended on 2024-12-13, Mohamed Boutayeb (advisor, CRAN), Dominique Martinez (co-advisor).
  • PhD: Nima Mehdi , “Approches probabilistes pour la perception et l'interprétation de l'activité humaine”', started in May 2020, defended on 2024-12-17, Francis Colas (advisor), Serena Ivaldi (co-advisor), and Vincent Thomas (co-advisor).
  • PhD in progress: Aya Yaacoub , “User-specific planning of a collaborative robot behavior to help prevent musculoskeletal disorders”, started in December 2021, Francis Colas (advisor), Pauline Maurice (co-advisor), Vincent Thomas (co-advisor), ROOIBOS project.
  • PhD in progress: Alexandre Oliveira Souza , “Intelligence Artificielle et contrôle de systèmes interactifs : Application aux exosquelettes”, started in May 2022, François Charpillet ( advisor), Pauline Maurice (co-advisor), Serena Ivaldi (co-advisor), CIFRE with Safran.
  • PhD in progress Salomé Lepers , “Explicabilité et interprétabilité en planification probabiliste”, started in October 2022, Olivier Buffet (advisor), Vincent Thomas (co-advisor).
  • PhD in progress: Dionis Totsila , “Learning Bimanual robot skills from human demonstrations and natural language”', started in November 2023, Serena Ivaldi (advisor) and Jean-Baptiste Mouret (co-advisor), euROBIN project.
  • PhD in progress: Thomas Martin , “Utilisation de l'apprentissage automatique pour le vol de drones en milieu très confiné”, started in January 2024, Jean-Baptiste Mouret (advisor) and Thibaut Rahajiraona (co-avisor)
  • PhD in progress: Konstantinos Tsakonas , “Intelligence artificielle pour la prédiction de trajectoires en robotique téléopérée”, started in October 2024, Serena Ivaldi (advisor) and Jean-Baptiste Mouret (co-advisor)
  • PhD in progress: Ioannis Tsikelis , “Model-base and learning methods for whole-body motion planning and control” (tentative) started in October 2024, Serena Ivaldi (advisor) and Enrico Mingo Hoffman (co-advisor)
  • PhD in progress: Raphael Lorenzo , “Multimodal prediction of human intention for human-robot collaboration”, started in October 2024, Serena Ivaldi (advisor) and Bertrand Luvison (co-advisor)
  • PhD in progress: Raphaël Boige , "Algorithmes de recherche dans les arbres : au-delà de Monte-Carlo Tree Search", started in November 2024, Bruno Scherrer (advisor) and Amine Boumaza (co-advisor)
  • PhD in progress: Mathis Antonetti , “Diffusion pour l'apprentissage de politiques de manipulation en robotique”, started in December 2024, Serena Ivaldi (advisor) and Jean-Baptiste Mouret (co-advisor)

11.2.3 Juries

  • Jean-Baptiste Mouret was :
    • President of the jury of the PhD of Quentin Le Lidec (Paris Sciences Lettres / Willow)
    • Reviewer of the PhD of Côme Perrot (INSA Toulouse / LAAS)
    • Reviewer of the Phd of Valetin Macé (Sorbonne Université / ISIR)
  • Pauline Maurice was :
    • Examinator of the PhD of Océane Dubois (ISIR / Sorbonne Université)
    • Examinator of the PhD of Arthur Favennec (DevAH / Université de Lorraine)
    • Examinator of the PhD of Erwan Landais (Inria Bordeaux / Université de Bordeaux)
    • Examinator of the PhD of Bianca Lento (INCIA / Université de Bordeaux)
    • Examinator of the PhD of Anna Bucchieri (Politecnico di Milano, Italy)
  • Serena Ivaldi was:
    • President of the jury of PhD of Sewade Olaolu Ogun (Université de Lorraine)
    • Reviewer of the PhD of Jianling Zou (Université Paris 8)
    • Examiner of the PhD of Lucas Quesada (Université Paris Saclay)
    • Examiner of the PhD of Anand Ballou (Université Grenoble Alpes)
    • Examiner of the HDR of Stéphane Caron (ENS & Sorbonne Université)
    • Member of the Jury of Selection of Chargé de Recherche & ISFP at Centre Inria de l'Université de Lorraine (jury NGE-CRCN-2024)
    • Examiner for the Academic Senate of Scuola Superiore Sant'Anna for the Promotion to Associate Professor of Assistant Professor Dr. Marco Controzzi
    • Member of Comité de Suivi de Thèse of Clelie Amiot (Université de Lorraine), Augustin Chartouny (Sorbonne Université), Alessia Fusco (Université Paul Sabatier III), Sebastien Muraccioli (Université de Montpellier)

11.3 Popularization

11.3.1 Specific official responsibilities in science outreach structures

  • Amine Boumaza is a member of the editorial board of Interstices.

11.3.2 Productions (articles, videos, podcasts, serious games, ...)

  • As a phd student, Aya Yaacoub participated in a formation on popularization and, in particular, fiction. This led to a publication in a book collection 45.
  • Serena Ivaldi and Jean-Baptiste Mouret produced a leaflet (4 pages) about robotics for the general public, distributed at the Semaine de la Robotique 44.
  • Serena Ivaldi , Jean-Baptiste Mouret and Anna Bucchieri translated the robotics activities sheets of IEEE Robots Guide from English to Italian and French.

11.3.3 Participation in Live events

  • Serena Ivaldi organized (with the help of the communication service) the outreach program of IEEE Humanoids (Friday-Sunday, November 22nd-24th 2024):
    • robotics workshop for 6-12 year old kids (building and programming a robot)
    • childcare program for the children of the conference's attendees (about 10 children, 3-12 year old)
    • robotics workshop for high-school students (robot soccer / robocup)
    • robotics workshop for university students (Nao humanoid robots with ChatGPT)
    • visit of the exhibition part of the conference (robot demonstrations, humanoid competition, ...)
    • general audience conference by Christian Duriez (Inria, Lille) about soft robot
    • also involved: Jean-Baptiste Mouret , Anna Bucchieri , Ioannis Tsikelis , Alexandre Oliveira Souza , Fabio Amadio , Clemente Donoso , Raphael Lorenzo , Raphael Lartot , Raphael Bousigues , Nima Mehdi , Alexis Biver , Dionis Totsila , Konstantinos Tsakonas , Olivier Rochel , Manon Cordone
    • Total: about 900 visitors
  • Serena Ivaldi organized (with the help of the communication service) the outreach program of the euROBIN coopetition (Monday-Thursday, November 25nd-28th 2024), focused on schools (6-12 year old):
    • visit of the competition area, with the robots of the euROBIN coopetition
    • robotic workshops
    • scientific play about artificial intelligence for high-school students
    • Total: about 700 visitors
    • also involved: Jean-Baptiste Mouret
  • Serena Ivaldi presented at the Féru des Sciences (Nancy) on September 21 in the context of the Journées du Matrimoine/Patrimoine.
  • Alexis Scheuer participated at the "fêtes de la science" by organizing and co-animating an introductory robotics workshop for pupils.
  • Vincent Thomas and Olivier Buffet participated at the "fêtes de la science" by co-animating an introductory robotics workshop for pupils.
  • Vincent Thomas was a member of the organizing comittee of a Game Jam event at Université de Lorraine (June 26th-27th 2024), aimed to assist professors in designing serious games enhancing the students' learning experience.
  • Vincent Thomas is a member of the organizing comittee of the pedagogy seminar "Apprendre et enseigner à l'Université de Lorraine " (planned in 2025).
  • Serena Ivaldi appeared on the Italian television PrimoCanale with a "live" from the lab; she was on TF1 news on automated factories; she was live on France 3 during the news from the Centre Prouvé in Nancy during the euROBIN coopetition.

11.3.4 Other science outreach relevant activities

  • Pauline Maurice gave a short conference and lab visit on occupational exoskeletons within the context of the Université de Lorraine outreach program “A votre santé” (about 15 persons).
  • Enrico Mingo Hoffman was interviewed by the Science & Vie newspaper for an article entitled “Le mécha : un robot géant vite encombrant
  • Serena Ivaldi was interviewed and appeared in articles by: L'Express, Les Tablettes Lorraine, Sciences et Avenir, L'Est Republicain, La Semaine, 20 Minutes, L'Usine Nouvelle, Reuters (the interview and Reuter's article appeared then on many other media, such as Rainews – the full list is here). She was on the radio France Inter (La Tech la Premiere), Direct FM and France Bleue.

12 Scientific production

12.1 Major publications

  • 1 articleA.Antoine Cully, J.Jeff Clune, D.Danesh Tarapore and J.-B.Jean-Baptiste Mouret. Robots that can adapt like animals.Nature5217553May 2015, 503-507HALDOIback to text
  • 2 articleJ. S.Jilles Steeve Dibangoye, C.Christopher Amato, O.Olivier Buffet and F.François Charpillet. Optimally Solving Dec-POMDPs as Continuous-State MDPs.Journal of Artificial Intelligence Research55February 2016, 443-497HALDOIback to text

12.2 Publications of the year

International journals

International peer-reviewed conferences

  • 17 inproceedingsT.Timothée Anne and J.-B.Jean-Baptiste Mouret. Parametric-Task MAP-Elites.Proceedings of the Genetic and Evolutionary Computation Conference (GECCO)GECCO 2024Melbourne, AustraliaACM2024HALDOIback to text
  • 18 inproceedingsA.Amine Boumaza. Exploring Affordances in Open-Ended Online Collective Robotics.Proceedings of the 2024 Artificial Life ConferenceCopenhagen, DenmarkMIT PressJuly 2024, 317-325HALDOIback to text
  • 19 inproceedingsJ.Jessica Colombel, V.Vincent Thomas, F.Francis Colas and P.Pauline Maurice. Identifying human movement strategies for human-robot collaboration.Multidisciplinary Biomechanics Journal49ème congrès de la Société de BiomécaniqueCompiègne, FranceOctober 2024HALback to text
  • 20 inproceedingsV.Véronique Cortier, P.Pierrick Gaudry, A.Anselme Goetschmann and S.Sophie Lemonnier. Belenios with cast-as-intended: towards a usable interface.SpringerEVote-ID 2024 - 9th International Joint Conference on Electronic VotingTerragona, SpainSpringerOctober 2024HAL
  • 21 inproceedingsT.Thomas Martin, J. R.Jefferson Roman Blanco, J.-B.Jean-Baptiste Mouret and T.Thibaut Raharijaona. Compact docking station for sub-150g UAV indoor precise landing.2024 International Conference on Unmanned Aircraft Systems (ICUAS) : 4-7 June 20242024 International Conference on Unmanned Aircraft Systems (ICUAS 2024)Chania - Crete, GreeceIEEEJune 2024, 1267-1274HALDOI
  • 22 inproceedingsS.Shuwa Miura, O.Olivier Buffet and S.Shlomo Zilberstein. Approximation Algorithms for Observer Aware MDPs.Proceedings of the 40th Conference on Uncertainty in Artificial Intelligence (UAI)40th Conference on Uncertainty in Artificial Intelligence (UAI)Barcelone, SpainJuly 2024HALback to text
  • 23 inproceedingsA.Alexandre Oliveira Souza, J.Jordane Grenier, F.François Charpillet, S.Serena Ivaldi and P.Pauline Maurice. Enhancing Exoskeleton Transparency with Motion Prediction: An Experimental Study.IEEE-RAS International Conference on Humanoid RobotsNancy, FranceNovember 2024HALback to text
  • 24 inproceedingsJ.Johan Peralez, A.Aurélien Delage, O.Olivier Buffet and J.Jilles Dibangoye. Solving Hierarchical Information-Sharing Dec-POMDPs: An Extensive-Form Game Approach.Proceedings of the 41st International Conference on Machine LearningICML 2024 - 41st International Conference on Machine Learning235Proceedings of Machine Learning ResearchVienne, Austria2024, 63010HALback to text
  • 25 inproceedingsQ.Quentin Rouxel, A.Andrea Ferrari, S.Serena Ivaldi and J.-B.Jean-Baptiste Mouret. Flow Matching Imitation Learning for Multi-Support Manipulation.2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids)Nancy, FranceJuly 2024HALback to text
  • 26 inproceedingsD.Dionis Totsila, Q.Quentin Rouxel, J.-B.Jean-Baptiste Mouret and S.Serena Ivaldi. Words2Contact: Identifying Support Contacts from Verbal Instructions Using Foundation Models.2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids)Nancy, FranceIEEENovember 2024, 9-16HALDOIback to text
  • 27 inproceedingsA.Aya Yaacoub, V.Vincent Thomas, F.Francis Colas and P.Pauline Maurice. Fatigue Mitigation through Planning in Human-Robot Repetitive Co-Manipulation: Automatic Extraction of Relevant Action Sets.2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids)2024 IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids)Nancy, FranceIEEENovember 2024, 795-802HALDOIback to text
  • 28 inproceedingsY.Yang You, V.Vincent Thomas, F.Francis Colas, R.Robert Skilton and O.Olivier Buffet. Online Robust Robot Planning for Human-Robot Collaboration.Lecture Notes in Artificial IntelligenceTAROS 2024 - 25th Toward Autonomous Robotic Systems Conference15052Lecture Notes in Computer ScienceLondres, United KingdomSpringer Nature SwitzerlandDecember 2025, 11HALDOIback to text

National peer-reviewed Conferences

  • 29 inproceedingsS.Salomé Lepers, V.Vincent Thomas and O.Olivier Buffet. Un cadre pour la planification consciente d'un observateur sous observabilité partielle.Actes des 18es Journées d’Intelligence Artificielle Fondamentale et des 19es Journées Francophones sur laPlanification, la Décision et l’Apprentissage pour la conduite de systèmes18es Journées d’Intelligence Artificielle Fondamentale et 19es Journées Francophones sur la Planification, la Décision et l’Apprentissage pour la conduite de systèmesLa Rochelle, FranceJune 2024HALback to text

Conferences without proceedings

Doctoral dissertations and habilitation theses

  • 32 thesisT.Timothee Anne. Multi-task optimization and its application to robotics : first solve, then generalize.Université de LorraineJune 2024HALback to text
  • 33 thesisA.Aurélien Delage. Theoretical foundations for planning in partially observable stochastic games.INSA de LyonJune 2024HALback to text
  • 34 thesisJ.Jacques Zhong. Assessing the variability of operators' morphology for assembly tasks using virtual reality.Université de LorraineJune 2024HALback to text

Reports & preprints

Other scientific publications

Scientific popularization

12.3 Cited publications

  • 46 articleA.Andrea Del Prete, F.Francesco Nori, G.Giorgio Metta and L.Lorenzo Natale. Prioritized Motion-Force Control of Constrained Fully-Actuated Robots: "Task Space Inverse Dynamics". Robotics and Autonomous Systems 2014, URL: http://dx.doi.org/10.1016/j.robot.2014.08.016back to text
  • 47 inproceedingsM.Mihai Andries and F.François Charpillet. Multi-robot taboo-list exploration of unknown structured environments.2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015)Hamburg, GermanySeptember 2015HALback to text
  • 48 inproceedingsM.Mauricio Araya-López, O.Olivier Buffet, V.Vincent Thomas and F.François Charpillet. A POMDP Extension with Belief-dependent Rewards.Advances in Neural Information Processing Systems (NIPS)Vancouver, CanadaMIT PressDecember 2010HALback to text
  • 49 articleT.Tim Barfoot, J.Jonathan Kelly and G.Gabe Sibley. Special Issue on Long-Term Autonomy.The International Journal of Robotics Research32142013, 1609--1610back to textback to text
  • 50 inproceedingsA.Abdallah Dib and F.François Charpillet. Pose Estimation For A Partially Observable Human Body From RGB-D Cameras.IEEE/RJS International Conference on Intelligent Robots and Systems (IROS)Hamburg, GermanySeptember 2015, 8HALback to text
  • 51 inproceedingsA.Arsène Fansi Tchango, V.Vincent Thomas, O.Olivier Buffet, F.Fabien Flacher and A.Alain Dutech. Simultaneous Tracking and Activity Recognition (STAR) using Advanced Agent-Based Behavioral Simulations.ECAI - Proceedings of the Twenty-first European Conference on Artificial IntelligencePragues, Czech RepublicAugust 2014HALback to text
  • 52 inproceedingsI.Iñaki Fernández Pérez, A.Amine Boumaza and F.François Charpillet. Comparison of Selection Methods in On-line Distributed Evolutionary Robotics.ALIFE 14: The fourteenth international conference on the synthesis and simulation of living systemsArtificial Life 14New York, United StatesJuly 2014HALDOIback to textback to text
  • 53 articleU.Udo Frese. Interview: Is SLAM Solved?KI - Künstliche Intelligenz2432010, 255-257URL: http://dx.doi.org/10.1007/s13218-010-0047-xDOIback to text
  • 54 articleJ.J. Kober, J. A.J. A. Bagnell and J.J. Peters. Reinforcement Learning in Robotics: A Survey.The International Journal of Robotics ResearchAugust 2013back to textback to textback to text
  • 55 articleS.Sylvain Koos, A.Antoine Cully and J.-B.Jean-Baptiste Mouret. Fast damage recovery in robotics with the t-resilience algorithm.The International Journal of Robotics Research32142013, 1700--1723back to text
  • 56 articleJ.-B.Jean-Baptiste Mouret and J.Jeff Clune. Illuminating search spaces by mapping elites.arXiv preprint arXiv:1504.049092015back to text
  • 57 inproceedingsF.François Pomerleau, P.Philipp Krüsi, F.Francis Colas, P.Paul Furgale and R.Roland Siegwart. Long-term 3D map maintenance in dynamic environments.Robotics and Automation (ICRA), 2014 IEEE International Conference onIEEE2014, 3712--3719back to text
  • 58 techreportSPARC. Robotics 2020 Multi-Annual Roadmap.2014, URL: http://www.eu-robotics.net/ppp/objectives-of-our-topic-groups/back to textback to text
  • 59 inproceedingsJ.J. Shah, J.J. Wiken, B.B. Williams and C.C. Breazeal. Improved human-robot team performance using Chaski, A human-inspired plan execution system. ACM/IEEE International Conference on Human-Robot Interaction (HRI)2011, 29-36back to text
  • 60 inproceedingsO.Olivier Simonin, T.Thomas Huraux and F.François Charpillet. Interactive Surface for Bio-inspired Robotics, Re-examining Foraging Models.23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI)Boca Raton, United StatesIEEENovember 2011HALback to text
  • 61 inproceedingsN.N. Stefanov, A.A. Peer and M.M. Buss. Role determination in human-human interaction.3rd Joint EuroHaptics Conf. and World Haptics2009, 51-56back to text
  • 62 bookR. S.R. S. Sutton and A. G.A. G. Barto. Introduction to Reinforcement Learning.MIT Press1998back to text
  • 63 articleA.A. Tapus, M.M.J. Matarić and B.B. Scassellati. The grand challenges in Socially Assistive Robotics.IEEE Robotics and Automation Magazine - Special Issue on Grand challenges in Robotics1412007, 1-7back to text
  • 64 articleD.D.H. Wilson and C.C. Atkeson. Simultaneous Tracking and Activity Recognition (STAR) Using Many Anonymous, Binary Sensors.34682005, 62-79URL: http://dx.doi.org/10.1007/11428572_5DOIback to text
  • 65 articleG.Gregor Wolbring and S.Sophya Yumakulov. Social Robots: Views of Staff of a Disability Service Organization.International Journal of Social Robotics632014, 457-468back to text
  1. 1See the Robotics 2020 Multi-Annual Roadmap 58.
  2. 2OHS (Office d'Hygiène Sociale) is an association managing several rehabilitation or retirement home structures.
  3. 3See the Robotics 2020 Multi-Annual Roadmap 58, section 2.5.