2023Activity reportProject-TeamCHROMA

Project-team positioning

The team carries out research across the full (software) stack of an autonomous driving system. This goes from perception of the environment to motion forecasting and trajectory optimization. While most of the techniques that we develop can stand on their own and are typically tested on our experimental platforms, we slowly move towards their complete integration. This will result in a functioning autonomous driving software stack. Beyond that point, we envision to focus our research on developing new methods that improve the individual performance of each of the stack’s components, as well as the global performance. This is alike to how autonomous driving companies operate (e.g. Waymo, Cruise, Tesla). The research topics that we explore are addressed by a very long list of international research laboratories and private companies. However, in terms of scope, we find the research groups of Prof. Dr.-Ing. Christoph Stiller (KIT), Prof. Marcelo Ang (NUS), Prof. Daniela Rus (MIT) and Prof. Wolfram Burgard (TRI California) to be the closest to us; we have active interconnections with these research groups (e.g. in the scope of the IEEE RAS Technical Committee AGV-ITS we are co-chairing any of the related annual Workshops). At the national level, we are cooperating in the framework of several R&D projects with UTC, Institut Pascal Clermont-Ferrand, University Gustave Eiffel, and the Inria Teams ASTRA, Acentauri and Convecs. Our main research originality relies in the combination of Bayesian approaches, AI technologies, Human-Vehicle interactions models, Robust State Estimation formalism, and a mix of Real experiments and Formal methods for validating our technologies in complex real world dynamic scenarios.

Our research on visual-inertial sensor fusion is closely related to the research carried out by the group of Prof. Stergios Roumeliotis (University of Minnesota) and the group of Prof. Davide Scaramuzza (University of Zurich). Our originality in this context is the introduction of closed-form solutions instead of filter-based approaches.

Scientific achievements

1. Bayesian & AI based Environment Perception and Forecasting

In order to be able to operate in a dynamic environment surrounded by humans or other robots, an autonomous system needs to Perceive, Understand and Forecast any changes of its surroundings. We have investigated the perception task from different angles, most of them with the shared nexus of involving both Bayesian approaches and Deep Learning models.

Bayesian Perception: The team's work on Bayesian approaches has given rise over the last decade to numerous publications and to some patents and licensed software8. Over the past 4 years, we have focused on collision risk assessment, safe navigation and semantic processing issues, with a strong emphasis on the integration and the validation of our technologies on real-time experimental platforms. The new results gave rise to the filing of two patents and to a transfer of technology with a French SME and with a Japanese international company (R&D work still ongoing). The main advances relate to (1) the generalization of the principle of Bayesian fusion in order to be able to integrate connected sensors located outside the ego-vehicle (infrastructure or other vehicles), (2) new real-time methods allowing to better estimate the risks of collision and make navigation safer (including path planning), and (3) experimental validation principle combining real tests, synchronized simulated environments (augmented reality principle) and formal methods 55, 94, 102, 54. A new PhD thesis addressing the subject of validating AI-based perception components in autonomous vehicles, with a particular focus on formal validation methods, has started in 2021 in cooperation with the Inria Convecs team (funding: French project PRISSMA).

Situation awareness & Environment Forecasting: In addition to the above-mentioned work, we have developed AI-based methods for extracting the semantic information of observed scenes 91, 88, 92, 120. In particular, we have developed hybrid methods for 3D object detection 104, 123105, semantic mapping of the environment from a top-view 72, 73, 74 and ground plane estimation and segmentation 103. We have likewise investigated the use of domain adaptation techniques to mitigate the degradation of some of the previous algorithms across different weather conditions 70, 68 and sensor modalities 120. In terms of environment forecasting, two PhD theses were dedicated to the task of predicting the future behavior of surrounding traffic 53, 125. The first of them targeted road intersections; the second focused on highway scenarios and leveraged a behavioral model learned from driving demonstrations 121 to cope with the long-term prediction horizon necessary for highways. Both theses also explored how to integrate the forecasting models into a behavioral planning framework 122. In addition to these forecasting models, we have also carried out research on localization 51, tracking 71, detection of abnormal behaviors 129, and contextualization of complex risk mitigation scenarios 119. A new PhD thesis addressing the subject of “Hybrid Sensor fusion” was launched in 2021 (funding: EU CPS4EU).

2. Modeling and simulation of dynamic environments

Modeling pedestrian's behaviors and recognizing intentions When Autonomous vehicles and social robots evolve in shared space where other cars, humans, cyclists, electrical scooters are present, they must recognize their intention in term of trajectory and collaboration. In the scope of several PhD, we proposed to model and predict various kinds of agents in the scene (P. Vasishta's Phd (2019), M. Kabtoul's (2021), K. Bhowmik's PhD started in may 2023) using maching learning (HMM, multi-criteria optimisation or Deeplearning models). Pedestrians simulation. We proposed the simulator SPACiSS, "Simulator for Pedestrians and an Autonomous Car in Shared Spaces", an open source simulator based on the PedSim framework that can simulate interactions between pedestrians and vehicles in different shared space scenarios. With the integration in the ROS framework, commonly used in robotics, the software is designed as an environment to test autonomous navigation systems 107, 108109, 110.

3. Robust State Estimation

Our investigations have been in the framework of the unknown input observability problem. This problem was introduced and firstly investigated during the sixties and has remained unsolved for 50 years. During the last 8 years, we have studied this fundamental open problem. Due to its complexity, the general solution was published in a book 98. The main result of this year is the extension of the above solution. Specifically, in 98, we provided the solution by restricting our investigation to systems that satisfy a special assumption that is called canonicity with respect to the unknown inputs. In 2022, after an exhaustive characterization of the concept of canonicity, we also accounted for the case when this assumption is not satisfied and we obtained the most general solution of the problem. These results have been published by the Journal of Information Fusion 97. A further result obtained in 2022, is the extension of the well known observability rank condition to time-varying systems. This extension has been published by the Transaction on Automatic Control 95.

Collaborations

• Main long-term industrial collaborations: IRT Nanoelec & CEA $\sim$10 years), Toyota Motor Europe ($\sim$12 years, R&D center Zaventen-Brussels), Renault ($\sim$10 years), SME Easymile ($\sim$3 years), Sumitomo Japan (started in 2019), Iveco (3 years in the framework of STAR project). These collaborations are funded either by Industry or by National, Regional or European projects. The main outputs are common publications, softwares development, patents or technological transfers. Two former doctoral students of our team were recruited some years ago by respectively TME Zaventen and Renault R&D centers.
• Main academic collaborations: Institut Pascal Clermont-Ferrand, University Compiegne (UTC), University Gustave Eiffel, several Inria Teams (ASTRA, Acentauri, Convecs, Rainbow). These collaborations are most of the time conducted in the scope of various National R&D projects (FUI, PIA, ANR, etc.). The main outputs are scientific and software exchanges. In addition, we had a fruitful collaboration with B. Mourrain from the Inria-Aromath team. The goal was to find the analytical solution of a polynomial equation system that fully characterizes the cooperative visual-inertial sensor fusion problem with 2 agents.
• International cooperation for scientific exchanges and IEEE scientific events organization, e.g. NUS & NTU Singapore, Peking University, TRI Mountain View, KIT Karlsruhe, Coimbra University. We have constantly interacted with Prof. D. Scaramuzza from the university of Zurich to make our findings in visual inertial sensor fusion more usable in a realistic context (e.g., for the autonomous navigation of drones).

Project-team positioning

In his reference book Planning algorithms86, S. LaValle discusses the different dimensions that make the motion-planning a complex problem, which are the number of robots, the obstacle regions, the uncertainty of perception and action, and the allowable velocities. In particular, it is emphasized that multiple robot planning in complex environments are NP-hard problems which implies that exact approaches have exponential time complexities in the worst case (unless P=NP). Moreover, dynamic and uncertain environments, as human-populated ones, expand this complexity. In this context, we aim at scaling up decision-making in human-populated environments and in multi-robot systems, while dealing with the intrinsic limits of the robots and machines (computation capacity, limited communication). To address these challenges, we explore new algorithms and AI architectures following three directions: online (or real-time) planning, machine learning to adapt to the complexity of uncertain environments, and combinatorial optimization to deal with offline constrained problems. Combining these approaches, seeking to scale up them, and also evaluating them with real platforms are also elements of originality of our work.

We share these goals with other laboratories/teams in the world, such as the Mobile Robotics Laboratory and Intelligent Systems "IntRoLab" at Sherbrooke University (Montreal) led by Prof. F. Michaud, the Learning Agents Research Group (LARG) within the AI Lab at the University of Texas (Austin) led by Prof. P. Stone, the Autonomous Systems Lab at ETH Zurich (Switzerland) led by R. Siegwart, the Robotic Sensor Networks Lab at the University of Minnesota (USA) led by Prof. V. Isler, and the Autonomous Robots Lab at NTNU (Norway) to cite a few. At Inria, we share some of the objectives with the LARSEN team in Nancy (multi-agent machine learning, multi-robot planning) and the RAINBOW team in Rennes (multi-UAV planning). We have also collaborations with the ACENTAURY team, in Sophia Antipolis, about autonomous navigation among humans. In France, among other labs involved on similar subjects, we can cite the LAAS (CNRS, Toulouse), the ISIR lab (CNRS, Paris Sorbonne Univ.) and the MAD team in Caen University. In the more generic domain of problem solving, we can mention the TASC team at LS2N (Nantes), the CRIL lab (Lens) and, at the international level, we can cite the Insight Centre for Data Analytics (Cork, Ireland).

Scientific achievements

1. Online Planning

In this theme, we address online planning when robots/vehicles/UAVs have to navigate or work in complex and generally unknown environments. The challenge comes from the necessity for entities to decide actions in real time and with limited information. Hereafter we present our main results following different problem contexts.

Social and ethic navigation

We investigated new methods to build safe and socially compliant trajectories among humans using models of behaviors presented in Axis I. A focus on navigation of an autonomous vehicle among crowds has been done were there is no free space to plan a safe trajectory (M. Kabtoul's PhD, ANR Hianic Project). We proposed a solution that is a proactive social navigation framework. The system is based on the idea of the coupled navigation behavior between the pedestrian and the vehicle in shared spaces. The system takes into account the cooperative nature of human behavior and exploits it to explore new navigation options, and navigate the shared space “proactively" 83. We obtained promising results using the SPACISS simulator previously described. Walese demonstrate how this vehicle can navigate crowds with a limited pedestrian disturbance 85. Some experiments have been done using the real autonomous Zoe car and simulated crowds perceived by the car using augmented reality (T. Genevois's PhD). We also investigated ethic navigation when an autonomous vehicle cannot avoid a collision with an object or a vulnerable road user (L. Serafim Guardini's PhD). We exploit accidentology data where each class of object or agent presents an injury probability with respect to the impact speed and ethical/economical/political factors. Our method generates a cost map containing a collision probability along with its associated risk of injury, which are used to plan trajectories with the lowest risk 119.

We also investigated another navigation problem, which is NAMO (Navigation Among Movable Obstacles). We extended NAMO in two directions considering social constraints when robots move obstacles (B. Renault's PhD). First, we examined where obstacles should be optimally moved with regards to space access (called Social-NAMO). We derived new spatial cost functions and NAMO algorithms preserving accesses 111. Second, we generalized the problem to $n$ robots (called MR-NAMO). We proposed a local coordination strategy exploiting Social-NAMO rearrangement properties and offering online efficient plans (under review). For evaluation, we developped a simulator (S-NAMO-SIM) that we share with the communauty.

Multi-robot exploration

An important part of our research is focused on designing new online planning solutions for multi-robot systems to accomplish exploration and observation gathering tasks in complex and dynamic environments. Problems related to computational and communication constraints are here crucial, and distributed approaches are fundamental to overcome them and obtain robust solutions.

Following these goals, a particular attention has been given to the exploration/mapping of 3D environments with a team of aerial vehicles, for which we presented a new decentralized solution based on the combination of a stochastic optimization approach with a more classic strategy that exploits frontier points 114. In 113, we then showed how this approach is part of a more general formulation that can deal with the optimal deployment coverage as well, for which we also studied the impact of an offline initialization step based on very partial information on the environment. Stochastic-based planning methods have been also exploited to propose a new source seeking strategy 112. This solution is based on robust estimations of the signal gradient obtained by the robots while exploring the environment in predefined symmetric formations 61 and that are then used to bias a correlated random walk that guides the robots.

Connected to this subject, we also investigated multi-robot strategies to cover complex scenes/ environments, by introducing multi-resolution maps. This concerned human activity observation by a set of ground robots (99) and more recently inspection of large structures with aerial robots (in the framework of the European project BugWright2, in collaboration with LSL, Austria).

Swarm navigation

In the context of self-reorganized systems, or swarm robotics, we explore how collective navigation can be defined and adapted to fleets of communicating robots. In A. Bonnefond's PhD, we studied flocking models, where each agent communicates locally its speed and position, and by simulating radio propagation we examined how obstacles impact the flocking robustness. Then we extended and combined two standard models, Olfati-Saber and Vásárhelyi, to improve their ability to stay connected while evolving in environments with different obstacle distributions 59. This work is supported by the Inria/DGA Dynaflock project and by the national Equipex+ TIRREX infrastructure. Related to this work, we address Controled Mobility where navigation must take into account the need of maintaining the fleet connectivity. In R. Grunblatt's PhD, we proposed a distributed algorithm that optimizes communication in a swarm of UAVs by changing their antenna orientation online 80. This work is continued in the ANR CONCERTO project.

2. Machine Learning

Machine learning techniques for decision-making are autonomous algorithms for computing behaviors, also called policies or strategies, which describe what action an agent should take when at a given state. Learning to act in complex environments requires knowledge about the structure of underlying decision-making problems. We contribute in revealing insightful underlying structures, exploiting them within existing or customized algorithms, and showing they improve performances in a wide variety of decision-making problems. For instance, optimal policies may have concise representations of their mental states, e.g., sufficient statistics, ego-centric neural-network architectures.

In the context of POMDPs9, dec-POMDPs, zs-POSGs10, and st-POSGs, we have identified a unified framework for solving these problems 69. Our methodological framework splits into five steps. First, we have shown that all these problems are convertible into a single problem called a common-knowledge game 69. Then, we recast the (possibly discrete) problem into a continuous one. Furthermore, we establish uniform continuity properties for general and hierarchical dec-POMDPs. We proved that optimal value functions for general and hierarchical dec-POMDPs are PWLC 69, 128. We also established optimal value functions for general zs-POSGs that are Lipschitz continuous 62, 63. Finally, we designed planning algorithms for games against Nature 76, pure competitive games 64, as well as for pure cooperative games 69. We extended exact and deep reinforcement learning algorithms for general dec-POMDPs 69. We finally conducted applications of deep reinforcement along two main directions: autonomous transportation systems 60; and automatic control systems 130.

Creating agents capable of high-level reasoning based on structured memory was the main topic of C. Wolf during its Inria "delegation" in Chroma (2017-19). In the PhD thesis of E. Beeching we proposed EgoMap, a spatially structured metric neural memory architecture in deep reinforcement learning. Through visualizations, the agent learns to map relevant objects in its spatial memory without any supervision purely from reward, see 58 (ICPR), 57 (ECCV) and 56 (ECML). In late 2019, C. Wolf obtained the AI Chair "REMEMBER", involving O. Simonin and J. Dibangoye from Chroma and L. Matignon from LIRIS lab/Univ Lyon 1. In this context, the PhD of P. Marza proposes to use an implicit representation of the environment to navigate towards targets. Neural fields are a recent category of approaches where the geometry, and eventually the semantics, of a 3D scene are encapsulated into the weights of a trained neural network.

As mentioned in Axis I, making appropriate decisions for safe robot / vehicle navigation is highly dependent on the results of the perception step. Indeed, the critical elements of a complex dynamic scene must first be extracted, classified and tracked in real time, by combining Bayesian and AI approaches to build "semantic grids" 9093, 72, 70. Two types of decision-making based on RL were developed for autonomous driving on highways and road junctions respectively, as part of the doctoral theses of David Sierra-Gonzalez 125 and Mathieu Barbier 53.

A last activity concerns our involvement in the RoboCup11 competition. In 2017 we founded the LyonTech team12 to participate to the RoboCup@Home challenge within the Pepper league. We worked on learning and integrating high-level functionalities such as object and people recognition, mapping and planning in indoor environments (see 82). We reached the 5th place in 2018, the 3rd place in 2019, the 2d place in 2021, and awarded the best scientific paper at the RoboCup conference in 2019 118.

3. Offline constrained optimisation problems

In this theme, we study and design algorithms for solving NP-hard constrained optimization problems (COPs) such as vehicle routing problems, or multi-agent path finding problems. We more particularly study approaches based on Constraint Programming (CP), that provides declarative languages for describing these problems by means of constraints, and generic algorithms for solving them by means of constraint propagation.

In the context of the European project BugWright2 and the PhD thesis of Xiao Peng (started in Nov. 2020), we study a new multi-agent path finding problem for tethered robots, that are attached with cables to anchor points. In 106, we show how to compute bounds for this problem by solving well known assignment problems, and we introduce a CP model for solving the problem to optimality. In 101, we extend this approach to the case of non point-sized robots.

In the context of the PhD thesis of Romain Fontaine (started in Oct. 2020), we study vehicle routing problems with time-dependent cost functions (such that travel times between locations to visit may change along the day), and we design new solving approaches based on an anytime extension of A* 78, 15. We study the impact of time-window constraints on the satisfiability and the difficulty of routing problems in 115.

Besides planning and routing problems which are at the core of Chroma topics, we also investigate the interest of using CP solvers for solving cryptanalysis problems 116.

Collaborations

• Social navigation: collaboration with Julie Dugdale and Dominique Vaufreydaz, Asso. Prof. at LIG Laboratory and with Prof. Philippe Martinet from Acentauri Inria team (common PhD students and projects: ANR VALET, ANR HIANIC, ANR ANNAPOLIS).
• Multi-robot exploration: Prof. Cedric Pradalier from GeorgiaTech Metz/CNRS (France) leading the European project BugWright2 — Prof. Luca Schenato from University of Padova (Italy).
• Swarm of UAVs: Prof. Isabelle Guérin-Lassous from LIP lab/Inria DANTE team (common PhD students and projects: ANR CONCERTO, INRIA/DGA DynaFlock) — Research Director Isabelle Fantoni from CNRS/LS2N Lab in Nantes (common projects: ANR CONCERTO, Equipex TIRREX) — Dr. Melanie Schranz, senior researcher at LakeSide Lab, Austria (BugWright2 project).
• Machine Learning: strong collaborations with Christian Wolf, Naver Labs Europe & LIRIS lab/INSA13 (common PhD students and projects: ANR DELICIO, ANR AI Chair REMEMBER) and with Laetitia Matignon, Asso. Prof. at LIRIS lab/Univ Lyon 1 (common projects and Phd/Master students) — Akshat Kumar, Asso. Prof. at School of Computing and Information Systems (Singapore) — Abdallah Saffidine, Research Asso. at University of New South Wales (Sydney) — Prof. Stéphane Canu at INSA Rouen — Olivier Buffet, Vincent Thomas and François Charpillet, Inria researchers in LARSEN team, Nancy (common publications and project ANR PLASMA).
• Semantic grids for autonomous navigation: Collaboration with Toyota Motor Europe (TME), R&D center Zaventen (Brussels). R&D contracts, common publications and patents.
• Constrained optimization: Omar Rifki (Post-doc) and Thierry Garaix (Ass. Prof) at LIMOS Ecole des Mines de Saint Etienne — Pascal Lafourcade and François Delobel, Asso. Prof. at LIMOS Université Clermont-Ferrand.

7.1.1 CMCDOT

• Functional Description:
  CMCDOT is a Bayesian filtering system for dynamic occupation grids, allowing parallel estimation of occupation probabilities for each cell of a grid, inference of velocities, prediction of the risk of collision and association of cells belonging to the same dynamic object. Last generation of a suite of Bayesian filtering methods developed in the Inria eMotion team, then in the Inria Chroma team (BOF, HSBOF, ...), it integrates the management of hybrid sampling methods (classical occupancy grids for static parts, particle sets for parts dynamics) into a Bayesian unified programming formalism , while incorporating elements resembling the Dempster-Shafer theory (state "unknown", allowing a focus of computing resources). It also offers a projection system of the estimated scene in the near future, to reference potential collisions with the ego-vehicle or any other element of the environment, as well as very low cost pre-segmentation of coherent dynamic spaces (taking into account speeds). It takes as input instantaneous occupation grids generated by sensor models for different sources, the system is composed of a ROS package, to manage the connectivity of I / O, which encapsulates the core of the embedded and optimized application on GPU Nvidia (Cuda), allowing real-time analysis of the direct environment on embedded boards (Tegra X1, X2). ROS (Robot Operating System) is a set of open source tools to develop software for robotics. Developed in an automotive setting, these techniques can be exploited in all areas of mobile robotics, and are particularly suited to highly dynamic and uncertain environment management (e.g. urban scenario, with pedestrians, cyclists, cars, buses, etc.).
• Authors:
  Amaury Nègre, Lukas Rummelhard, Jean-Alix David, Christian Laugier
• Contact:
  Christian Laugier

7.1.2 Ground Elevation and Occupancy Grid Estimator (GEOG - Estimator)

• Keywords:
  Robotics, Environment perception
• Functional Description:
  GEOG-Estimator is a system of joint estimation of the shape of the ground, in the form of a Bayesian network of constrained elevation nodes, and the ground-obstacle classification of a pointcloud. Starting from an unclassified 3D pointcloud, it consists of a set of expectation-maximization methods computed in parallel on the network of elevation nodes, integrating the constraints of spatial continuity as well as the influence of 3D points, classified as ground-based or obstacles. Once the ground model is generated, the system can then construct a occupation grid, taking into account the classification of 3D points, and the actual height of these impacts. Mainly used with lidars (Velodyne64, Quanergy M8, IBEO Lux), the approach can be generalized to any type of sensor providing 3D pointclouds. On the other hand, in the case of lidars, free space information between the source and the 3D point can be integrated into the construction of the grid, as well as the height at which the laser passes through the area (taking into account the height of the laser in the sensor model). The areas of application of the system spread across all areas of mobile robotics, it is particularly suitable for unknown environments. GEOG-Estimator was originally developed to allow optimal integration of 3D sensors in systems using 2D occupancy grids, taking into account the orientation of sensors, and indefinite forms of grounds. The ground model generated can be used directly, whether for mapping or as a pre-calculation step for methods of obstacle recognition or classification. Designed to be effective (real-time) in the context of embedded applications, the entire system is implemented on Nvidia graphics card (in Cuda), and optimized for Tegra X2 embedded boards. To ease interconnections with the sensor outputs and other perception modules, the system is implemented using ROS (Robot Operating System), a set of opensource tools for robotics.
• Authors:
  Amaury Nègre, Lukas Rummelhard, Lukas Rummelhard, Jean-Alix David, Christian Laugier
• Contact:
  Christian Laugier

7.1.3 Zoe Simulation

• Name:
  Simulation of INRIA's Renault Zoe in Gazebo environment
• Keyword:
  Simulation
• Functional Description:
  This simulation represents the Renault Zoe vehicle considering the realistic physical phenomena (friction, sliding, inertia, ...). The simulated vehicle embeds sensors similar to the ones of the actual vehicle. They provide measurement data under the same format. Moreover the software input/output are identical to the vehicle's. Therefore any program executed on the vehicle can be used with the simulation and reciprocally.
• Authors:
  Christian Laugier, Nicolas Turro, Thomas Genevois
• Contact:
  Christian Laugier

7.1.4 Hybrid-state E*

• Name:
  Path planning with Hybrid-state E*
• Keywords:
  Planning, Robotics
• Functional Description:

  Considering a vehicle with the kinematic constraints of a car and an environment which is represented by a probabilistic occupancy grid, this software produces a path from the initial position of the vehicle to its destination. The computed path may include, if necessary, complex maneuvers. However the suggested path is often the simpler and the shorter.

  This software is designed to take benefit from bayesian occupancy grids such as the ones computed by the CMCDOT software.

• URL:
  https://­team.­inria.­fr/­chroma/­en/
• Authors:
  Christian Laugier, Thomas Genevois
• Contact:
  Christian Laugier
• Partner:
  CEA

7.1.5 Pedsim_ros_AV

• Name:
  Pedsim_ros_AV
• Keywords:
  Simulator, Multi-agent, Crowd simulation, Autonomous Cars, Pedestrian
• Scientific Description:

  These ROS packages are useful to support robotic developments that require the simulation of pedestrians and an autonomous vehicle in various shared spaces scenarios. They allow: 1. in simulation, to pre-test autonomous vehicle navigation algorithms in various crowd scenarios, 2. in real crowds, to help online prediction of pedestrian trajectories around the autonomous vehicle.

  Individual pedestrian model in shared space (perception, distraction, personal space, pedestrians standing, trip purpose). Model of pedestrians in social groups (couples, friends, colleagues, family). Autonomous car model. Pedestrian-autonomous car interaction model. Definition of shared space scenarios: 3 environments (business zone, campus, city centre) and 8 crowd configurations.

• Functional Description:
  Simulation of pedestrians and an autonomous vehicle in various shared space scenarios. Adaptation of the original Pedsim_ros model to simulate heterogeneous crowds in shared spaces (individuals, social groups, etc.). The car model is integrated into the simulator and the interactions between pedestrians and the autonomous vehicle are modeled. The autonomous vehicle can be controlled from inside the simulation or from outside the simulator by ROS commands.
• URL:
  https://­github.­com/­maprdhm/­pedsim_ros_AV
• Publications:
  hal-02194735, hal-02514963
• Contact:
  Manon Predhumeau
• Participants:
  Manon Predhumeau, Anne Spalanzani, Julie Dugdale, Lyuba Mancheva
• Partner:
  LIG

7.1.6 S-NAMO-SIM

• Name:
  S-NAMO Simulator
• Keywords:
  Simulation, Navigation, Robotics, Planning
• Functional Description:
  2D Simulator of NAMO algorithms (Navigation Among Movable Obstacles) and MR-NAMO (Multi-Robot NAMO), ROS compatible
• Release Contributions:
  Creation
• Authors:
  Benoit Renault, David Brown, Jacques Saraydaryan, Olivier Simonin
• Contact:
  Olivier Simonin

7.1.7 SimuDronesGR

• Name:
  Simultion of UAV fleets with Gazebo/ROS
• Keywords:
  Robotics, Simulation
• Functional Description:
  The simulator includes the following functionality : 1) Simulation of the mechanical behavior of an Unmanned Aerial Vehicle : * Modeling of the body's aerodynamics with lift, drag and moment * Modeling of rotors' aerodynamics using the forces and moments' expressions from Philppe Martin's and Erwan Salaün's 2010 IEEE Conference on Robotics and Automation paper "The True Role of Accelerometer Feedback in Quadrotor Control". 2) Gives groundtruth informations : * Positions in East-North-Up reference frame * Linear velocity in East-North-Up and Front-Left-Up reference frames * Linear acceleration in East-North-Up and Front-Left-Up reference frames * Orientation from East-North-Up reference frame to Front-Left-Up reference frame (Quaternions) * Angular velocity of Front-Left-Up reference frame expressed in Front-Left-Up reference frame. 3) Simulation of the following sensors : * Inertial Measurement Unit with 9DoF (Accelerometer + Gyroscope + Orientation) * Barometer using an ISA model for the troposphere (valid up to 11km above Mean Sea Level) * Magnetometer with the earth magnetic field declination * GPS Antenna with a geodesic map projection.
• Release Contributions:
  Initial version
• Authors:
  Vincent Le Doze, Johan Faure, Olivier Simonin, Vincent Dufour
• Contact:
  Olivier Simonin
• Partner:
  Insa de Lyon

7.1.8 spank

• Name:
  Swarm Protocol And Navigation Kontrol
• Keyword:
  Protocoles
• Functional Description:
  Communication and distance measurement in an uav swarm
• URL:
  https://­gitlab.­inria.­fr/­dalu/­spank
• Contact:
  Stéphane D'Alu
• Participant:
  Stéphane D'Alu

7.2.1 Chroma Aerial Robots platform

Participants: Olivier Simonin, Alessandro Renzaglia, Johan Faure.

This platform is composed of :

• Four quadrirotor PX4 Vision UAVs (Unmaned Aerial Vehicles), acquired 4 in 2021 and 2022. This platform is funded and used in the projects "Dynaflock" (Inria-DGA) and ANR "CONCERTO" (the team also owns 5 Parrot Bebop UAVs).
• Two outdoor inflatable aviaries of 6m(L) x 4m(l) x 5m(H) each.
• Two indoor inflatable aviaries of 5m(L) x 3m(l) x 2.5m(H) each.

7.2.2 Barakuda

Participants: Lukas Rummelhard, Thomas Genevois, Robin Baruffa, Hugo Bantignies, Nicolas Turro.

Within the scope of the ANR Challenge MOBILEX, a 4-wheel all-terrain Barakuda robot from Shark Robotics is provided, and will be equipped with sensors and compute capabilities to achieve increasing level of autonomy.

8.1.1 Evolution of the CMCDOT framework

Participants: Robin Baruffa, Hugo Bantignies, Thomas Genevois, Nicolas Turro, Lukas Rummelhard.

Recognized as one of the core technologies developed within the team over the years (see related sections in previous activity report of Chroma, and previously e-Motion reports), the CMCDOT framework is a generic Bayesian Perception framework, designed to estimate a dense representation of dynamic environments and the associated risks of collision, by fusing and filtering multi-sensor data. This whole perception system has been developed, implemented and tested on embedded devices, incorporating over time new key modules. Now extended to 3D for grid fusion and incorporating semantic states in the whole process, this framework, and the corresponding software, has continued in 2023 to be the core of many important industrial partnerships and academic contributions, and to be the subject of important developments, both in terms of research and engineering. Most work on the core technology consisted this year on adaptations for the deployment on new experimental platforms and environments, and software evolutions.

8.1.2 Cooperative Perception through communication with Dynamic Occupancy Grids

Participants: Hugo Bantignies, Nicolas Turro, Lukas Rummelhard.

In order to reduce the cost of deploying a fleet of automated mobile robots, or to enrich the perception capacity of a mobile element in certain particularly accident-prone areas, the option of deploying different sensors on an instrumented and communicating infrastructure is particularly attractive. Perception methods based on Bayesian occupancy grid are well-suited for the fusion of heterogeneous and even remote or mobile sensors, but the size of these structures makes their efficient communication a real challenge. Such an option is studied and tested within the CMCDOT framework, using V2X communication (Vehicle to everything) to communicate dynamic probabilistic occupancy grids estimated by intelligent infrastructure to a mobile robot. Recently, following the installation of a fixed infrastructure perception station, a transmission of objects following a standard communication protocol (ETSI TR 103 562) was implemented, using industrial V2X communication systems (e.g. Lacroix), while developing methods for compressing and sending occupancy grids. After analysis of the geometric primitives supported by the communication protocol, methods of decomposition of occupancy grids into simple transmissible elements under constraint, composed of weighted objects and poligons, and their reconstruction and adaptation after radio communication, was developed and tested, the objective being to reconstruct the occupancy as faithfully as possible, with the smallest possible dataflow.

8.1.3 Generation of proper probabilistic occupancy grids by neural network

Participants: Robin Baruffa, Lukas Rummelhard.

In the CMCDOT framework, modules based on neural networks for the semantic analysis of the environment in the image plane of the cameras had previously been developed and integrated, making it possible to distinguish different types of occupancy ( pedestrians, vehicles, etc.) after reprojection and fusion with probabilistic occupancy grids generated by other sensors. These modules were limited to this semantic information, relying on other sensors for precise position estimation. New modules was developed, integrating state-of-the-art neural networks (Lift Splat Shoot), allowing inference from camera images of semantic occupancy grids, therefore directly the position of the different types of semantic occupancy. The performance of the model was evaluated through various experiments, notably on the Transpolis test space. In parallel, the integration of neural networks within a proper Bayesian probabilistic framework were studied. If the generated semantic grids are particularly rich and interesting to merge, offering a different modality from the other sensors (thus a decorrelated potential error), the provided grid values should be reworked to properly correspond to same probilistic state estimations as the other sensors. Statistical analysis encapsulating probabilistic models, and associated software, have been initiated.

8.1.4 From Probabilistic Occupancy Grids to versatile Collision Avoidance using Predictive Collision Detection

Participants: Thomas Genevois, Lukas Rummelhard, Anne Spalanzani, Christian Laugier.

As robots are spreading in diverse environments, new collision avoidance challenges arise. Real world collision avoidance often includes dynamic, complex, unstructured and uncertain environments. While these constraints are rarely considered all together in perception and navigation research works, a broad integration from perception to navigation in a versatile collision avoidance approach was developped, designed to operate under these constraints. The proposed solution relies on a new Predictive Collision Detector, an interface between state-of-art grid-based perception and sampling-based planners. Unlike most other approaches, ours operates only on elementary spatial occupancy, without object segmentation, capturing the richness and versatility of modern occupancy grid perception.

Relying on the embedded Bayesian perception bricks developed within the CMCDOT framework, these original navigation methods on adapted Bayesian occupancy grid allowed the automation of different robotic platforms (automated Zoe, robot of industrial transit, light transport vehicle), in different types of environments generally considered difficult (uncertain, noisy, unknown, complex, dynamic), fully relying on the advantages provided by the representation in the form of a probabilistic occupancy grid. This approach was the subject of a paper presented at the IEEE IV 2023 international conference 29.

8.1.5 Validation of AI-based algorithms in autonomous vehicles

Participants: Jean-Baptiste Horel, Alessandro Renzaglia, Radu Mateescu [Convecs], Wendelin Serwe [Convecs], Christian Laugier.

In the last years, there has been an increasing demand for regulating and validating intelligent vehicle systems to ensure their correct functioning and build public trust in their use. Yet, the analysis of safety and reliability poses a significant challenge. More and more solutions for autonomous driving are based on complex AI-based algorithms whose validation is particularly challenging to achieve. An important part of our work has been recently devoted to tackle this problem, finding new suitable approaches to validate probabilistic algorithms for perception and decision making in autonomous driving, and investigating how simulations and experiments in controlled environment can help to solve this challenge. This activity, started with our participation in the European project Enable-S3 (2016-2019), is now continuing with the PRISSMA project, started in April 2021, where we participate in collaboration the the Inria Convecs team. This project, funded by the French government in the framework of the “Grand Défi: Sécuriser, certifier et fiabiliser les systèmes fondés sur l'intelligence artificielle”, regroups several companies and public institutes in France and tackles the challenge of the validation and certification of artificial intelligence based solutions for autonomous mobility.

Within this context, our main effort has been focused on proposing a new approach specifically designed for perception component verification that, given a high-level and human-interpretable description of a critical situation, generates relevant AV scenarios and uses them for automatic verification. To achieve this goal, we integrate two recently proposed methods for the generation and the verification that are based on formal verification tools. First, we proposed the use of formal conformance test generation tools to derive, from a verified formal model, sets of scenarios to be run in a simulator. This work was based on the results published in 81. Second, we model check the traces of the simulation runs to validate the probabilistic estimation of collision risks. Using formal methods brings the combined advantages of an increased confidence in the correct representation of the chosen configuration (temporal logic verification), a guarantee of the coverage and relevance of automatically generated scenarios (conformance testing), and an automatic quantitative analysis of the test execution (verification and statistical analysis on traces). accepted for pubblication in the Journal of Intelligent and Robotic Systems 16.

8.1.6 Navigation-Based Evaluation Metric for Probabilistic Occupancy Grids

Participants: Jean-Baptiste Horel, Robin Baruffa, Lukas Rummelhard, Alessandro Renzaglia, Christian Laugier.

With the goal of testing and validating perception algorithms, a critical step is to define appropriate metrics to evaluate the system behavior. Despite the importance and broad use of probabilistic Occupancy Grids (OG) in autonomous driving, most existing approaches to evaluate their reliability are based on general-purpose metrics derived from the computer vision literature. In 31, we propose a new metric that evaluates the similarity of two OGs: a Ground Truth (GT) of the environment and the inference of the environment made by the perception system. This metric is specifically designed to assess the similarity between OGs by considering the behavior of an ego vehicle navigating through the grids. The main postulate is that if a navigation algorithm generates similar trajectories using two OGs, then the two are similar for navigation purposes.

Simulating and comparing the navigation behavior instead of doing cell-wise comparisons directly on the OGs gives this metric relevant properties. It is able to evaluate topological errors, it measures how occupancy errors on the inference change the cost of the paths and how this affects the global topology of the OG. It puts emphasis on cells that are most crossed by paths since their occupancy is incorporated in more costs. In other words, these cells are topologically more relevant for navigation (e.g., areas closer to the ego vehicle or bottlenecks). Furthermore, this metric is well suited to evaluate uncertainty: paths tend to avoid uncertainty whenever possible or cross it otherwise, but in both cases, the navigation cost is increased.

8.2.1 Situation Understanding and Motion Forecasting

Participants: Kaushik Bhowmik, Anne Spalanzani.

Forecasting the motion of surrounding traffic is one of the key challenges in the quest to achieve safe autonomous driving technology. Current state-of-the-art deep forecasting architectures are capable of producing impressive results. However, in many cases, they also output completely unreasonable trajectories, making them unsuitable for deployment. In 2022, we have developed a deep forecasting architecture that leverages the map lane centerlines available in recent datasets to predict sensible trajectories; that is, trajectories that conform to the road layout, agree with the observed dynamics of the target, and react to the presence of surrounding agents. To model such sensible behavior, the proposed architecture first predicts the lane or lanes that the target agent is likely to follow. Then, a navigational goal along each candidate lane is predicted, allowing the regression of the final trajectory in a lane- and goal-oriented manner. Our experiments in the Argoverse dataset show that our architecture achieves performance on-par with lane-oriented state-of-the-art forecasting approaches and not far behind goal-oriented approaches, while consistently producing sensible trajectories. This work was published at IEEE ITSC 2022 124.

In 2023 we started to work on predicting the motion of heterogeneous agents in urban environment (K. Bhowmik's PhD). The goal is to predict the behavior of pedestrians, cyclists, and even electrical scooters. Using Deeplearning methods a existing big datasets of urban scenes, we plan to build robusts models of various agents. When their motion is highly unpredictable, we aim at warning the autonomous vehicle that a specific zone around it could be dangerous.

8.2.2 Semantic Grid Generation from Camera and LiDAR Sensor Fusion

Participants: Manuel Alejandro Diaz-Zapata, David Sierra-González, Christian Laugier, Jilles Dibangoye, Olivier Simonin.

Following with the work towards semantic grid generation, during 2023, the LAPTNet and LAPTNet-FPN-PP methods 67 were further fine-tuned for vehicle detection, these results were presented at IEEE ICRA 2023 27. Later, these models were also trained to perform Panoptic Segmentation, achieving interesting results over the state of the art, keeping a good segmentation performance at an improved inference speed compared to the state of the art models.

After this, a cross-dataset training and evaluation framework was developed as a manner to address the gap in performance that occurs when deploying a model in a real platform. This framework allows for the use of two or more datasets, for training and/or evaluation in the tasks of semantic grid and panoptic grid generation. Experiments across state of the art methods showed that across datasets the segmentation performance decreased, where the models that use LiDAR data showed the worst cross-dataset perfomance. Interestingly, these experiments also showed that performance can be improved on one dataset when adding extra training information from another dataset.

8.2.3 Lidar-RGB Sensor Fusion with Transformers for Semantic Grid Prediction

Participants: Gustavo Salazar-Gomez, David Sierra-González, Manuel Alejandro Diaz-Zapata, Wenqian Liu, Christian Laugier.

The ability of perception systems in autonomous vehicles to accurately interpret and respond to the surrounding environment is prime important for ensuring safety and efficiency. One significant challenge that these systems face is the presence of occlusions, where objects or obstacles obstruct the line of sight. Tackling occlusion scenarios is both crucial and challenging. Achieving robust environmental perception is vital for effectively managing occlusions and assisting in navigation. Cutting-edge models integrate data from LiDAR and cameras to deliver remarkable perception outcomes; however, the identification of obstructed objects remains a persistent challenge. We emphasize the vital significance of temporal cues in enhancing robustness against occlusions within the birds-eye-view (BEV) semantic grid segmentation task. We propose a framework that enables the processing of temporal multi-step inputs, where the input at each time step comprises the spatial information encoded from fusing LiDAR and camera sensor readings, using cross-attention and self-attention as means to fuse the features from these.

Furthermore, by achieving forecasting of multiple agents in the scene and handling the occlusions, we propose an additional architecture to predict the trajectory of the ego vehicle, based on the interaction with the obstacles in the environment, using as input semantic grids. This was completed by experimenting in the cutting-edge nuPlan dataset, a new framework proposed to be the first benchmark for autonomous vehicle planning. In the next steps, we explore the options and needs for training an end-to-end framework, capable of receiving as input the raw data from the LiDAR and camera sensors, and output the trajectory planning of the autonomous vehicle to avoid obstacles in the scene.

8.2.4 Dynamic Scene Prediction for Urban Traffic Scene Using Occupancy Grid Maps

Participants: Rabbia Asghar, Lukas Rummelhard, Anne Spalanzani, Christian Laugier.

Prediction of dynamic environment is crucial to safe navigation for an autonomous vehicle. This, however, is a challenging task due to uncertainty in the sensor data, the non-deterministic nature of future, and complex behavior of agents. Initial work in 2022 proposed to represent the traffic scene in a fixed frame as allo-centric occupancy grid and incorporate deep learning video prediction networks to improve scene prediction. This work was presented at the ICARCV22 conference 52 and won the best paper prize.

Carrying forward the work on allo-centric grid prediction, in 2023, we continue to tackle the prediction problem by representing the scene as dynamic occupancy grid maps (DOGMs), associating semantic labels to the occupied cells and incorporating map information. We propose a novel framework that combines deep-learning-based spatio-temporal and probabilistic approaches to predict vehicle behaviors. Contrary to the conventional OGM prediction methods, evaluation of our work is conducted against the ground truth annotations. We experiment and validate our results on real-world NuScenes dataset and show that our model shows superior ability to predict both static and dynamic vehicles compared to OGM predictions. This work was presented to the IROS23 conference 24. Moreover, Rabbia Asghar presented a poster at Naverlab workshop and won the price of the best poster 44. In next steps, we continue to work on a real-time analysis of complex traffic situations in the occupancy grid framework in order to predict all the scene. (see Fig. 3)

complex traffic scene forcasting

8.2.5 Temporal Multi-Modality Fusion Towards Occlusion-Aware Semantic Segmentation

Participants: Wenqian Liu, Manuel Alejandro Diaz-Zapata, David Sierra-González, Christian Laugier.

In the context of our cooperation with Toyota Motor Europe, we study new deep learning approaches to understand the surrounding scenes of the autonomous vehicle. More particularly, we focus on occlusion-aware bird's-eye view (BEV) semantic segmentaion by proposing a novel sptio-temporal approach.

Our main output in 2023 is an innovative model designed to process sequences of multi-modal sensor data inputs, capturing the temporal memory of the scene and objects to enhance occlusion-aware semantic bird's-eye-view (BEV) grid segmentation. At the input stage, our approach extracts spatial features from fused LiDAR and camera sensor readings at each time step throughout the temporal sequence. We also use this model to experiment on downstream tasks such as to predict multi-step future bird's-eye view (BEV) grids in a one-shot manner and to forecast the future trajectory of the ego-vehicle. This model also paths our way into end-to-end trainable framework for full-stack driving tasks for our future research. This work is currently pending for publication.

8.4.1 Motion-planning in dense pedestrian environments

Participants: Thomas Genevois, Anne Spalanzani.

Under the coordination of Anne Spalanzani, we study new motion planning algorithms to allow robots/vehicles to navigate in human populated environment while predicting pedestrians' motions. We model pedestrians and crowd behaviors using notions of social forces and cooperative behavior estimations. We investigate new methods to build safe and socially compliant trajectories using theses models of behaviors. We propose proactive navigation solutions as well as deep learning ones.

A scene of an Autonomous Vehicle with pedestrians

A scene of an Autonomous Vehicle with pedestrians

since 2018 we've been working on modelling crowds and autonomous vehicles using Extended Social Force Models (in the scope of Manon Predhumeau's PhD) and Proactive Navigation for navigating dense human populated environments (in the scope of Maria Kabtoul's PhD). The focus of the first work has been on the realistic simulation of crowds in shared spaces with an autonomous vehicle (AV), see Fig. 4.a. We proposed an agent-based model for pedestrian reactions to an AV in a shared space, based on empirical studies and the state of the art. The model includes an AV with Renault Zoé car's characteristics, and pedestrians' reactions to the vehicle. The model extends the Social Force Model with a new decision model, which integrates various observed reactions of pedestrians and pedestrians groups. The SPACiSS simulator is an opensource simulator still used by the team. The focus of the second work was a pedestrian-vehicle interaction behavioral model and a proactive navigation algorithm. The model estimates the pedestrian's cooperation with the vehicle in an interaction scenario by a quantitative time-varying function. Using this cooperation estimation the pedestrian's trajectory is predicted by a cooperation-based trajectory planning model. We then used this cooperation based behavioral model to design a proactive longitudinal velocity controller. In 2020, we integrated the previous navigation system with the SPACiSS simulator under ROS (see Fig. 4.b). Next, we worked on a proactive and human-like maneuvering system to deploy the steering control part of the navigation system. This work was published in ICRA'22 84 and in 2023 17.

In 2023, Thomas Genevois worked on improving the previous works done in this topic. He proposed some improvement on the SPACiSS simulator, proposed a version of a huamn and interaction aware collision avoidance that can be used in the global architecture of the zoe car. He then linked the SPACiSS simulator to his Augmented Reality simulator so that the navigation strategies among crowds can be tested using the real zoé car 30.

8.4.2 Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning

Participants: Jacques Saraydaryan, Laetitia Matignon (Lyon1/LIRIS), Erwan Escudie (Master internship), Olivier Simonin.

Learning robot navigation strategies among pedestrian is crucial for domain based applications. Combining perception, planning and prediction allows us to model the interactions between robots and pedestrians, resulting in impressive outcomes especially with recent approaches based on deep reinforcement learning (RL). However, these works do not consider multi-robot scenarios.

In this work, we define MultiSoc, a new method for learning multi-agent socially aware navigation strategies using RL (see illustration fig. 5.a). Inspired by recent works on multi-agent deep RL, our method leverages graph-based representation of agent interactions, combining the positions and fields of view of entities (pedestrians and agents). Each agent uses a model based on two Graph Neural Network combined with attention mechanisms. First an edge-selector produces a sparse graph, then a crowd coordinator applies node attention to produce a graph representing the influence of each entity on the others. This is incorporated into a model-free RL framework to learn multi-agent policies. We evaluated our approach on simulation and provide a series of experiments in a set of various conditions (number of agents/pedestrians). Empirical results show that our method learns faster than social navigation deep RL mono-agent techniques, and enables efficient multi-agent implicit coordination in challenging crowd navigation with multiple heterogeneous humans. Furthermore, by incorporating customizable meta-parameters, we can adjust the neighborhood density to take into account in our navigation strategy. This work has been accepted for publication in AAMAS 2024 28.

8.4.3 Deep-Learning for vision-based navigation

Participants: Pierre Marza (PhD. student CITI & LIRIS), Christian Wolf (NaverLabs europe), Olivier Simonin, Laetitia Matignon (LIRIS/Lyon 1).

a. Multisoc architecture. b. NN with two implicit representations.

a. Multisoc architecture. b. NN with two implicit representations.

Our goal is the automatic learning of robot navigation in complex environments based on specific tasks and from visual input. The robot automatically navigates in the environment in order to solve a specific problem, which can be posed explicitly and be encoded in the algorithm (e.g. find all occurences of a given object in the environment, or recognize the current activities of all the actors in this environment) or which can be given in an encoded form as additional input, like text. Addressing these problems requires competences in computer vision, machine learning/AI, and robotics (navigation and paths planning).

A critical part for solving these kind of problems involving autonomous agents is handling memory and planning. Many control problems in partially observed 3D environments involve long term dependencies and planning. Solving these problems requires agents to learn several key capacities: spatial reasoning (exploring the environment and learning spatio-temporal regularities) and semantic mapping (discovering semantics from interactions). While solutions exist for semantic mapping and semantic SLAM 65, 126, a more interesting problem arises when the semantics of objects and their affordances are not supervised, but defined through the task and thus learned from reward.

PhD thesis of Pierre Marza (since 2021). In the context of the REMEMBER AI Chair, Pierre Marza's PhD work is focusing on high-level navigation skills for autonomous agents in 3D environments (supervised by C. Wolf, L. Matignon and O. Simonin). A key question is the ability to learn representations with minimal human intervention and annotation. In 2023, we proposed to structure neural networks with two neural implicit representations, which are learned dynamically during each episode and map the content of the scene: (i) the Semantic Finder predicts the position of a previously seen queried object; (ii) the Occupancy and Exploration Implicit Representation encapsulates information about explored area and obstacles. Both representations are leveraged by an agent trained with Reinforcement Learning (RL) and learned online during each episode. We evaluated the agent on Multi-Object Navigation and showed the high impact of using neural implicit representations as a memory source. This work has been published in ICCV 2023 32, cf. fig. 5.b.

8.4.4 Navigation Among Movable Obstacles: extension of NAMO to multi-robot and social constraints

Participants: Benoit Renault (PhD. stud.), Jacques Saraydaryan, Olivier Simonin, David Brown (Ing. Inria).

S-NAMO paths and Pepper robot

S-NAMO paths and Pepper robot

In 2018 we started to study the NAMO planning problem (Navigation Among Movable Obstacles). Since 2019, the PhD thesis of Benoit Renault aims to extend NAMO in two directions:

• Defining Social-NAMO, by considering where obstacles should be optimally moved with regards to space access. We have defined a new spatial cost function based on social heuristics and the topology of the environment. It allows to extended existing NAMO algorithms with the ability to maintain area accesses (connectivity), see IROS publication 111.
• Defining Multi-robot NAMO (MR-NAMO). It consists in generalizing the problem to $n$ robots that can move the objects of the environment in order to reach their individual goal positions (see Fig. 6.b). We have proposed a first coordination strategy based on local heuristics, able to detect robots conflicts and to manage deadlock situations. If this approach is not complete we showed experimentlay that it offers a significant gain for solving MR-NAMO problems compared to naive coordination rules and standard NAMO algorithms. Publication in preparation.

During his PhD thesis, Benoit Renault developped a simulator for NAMO, and its extensions, called S-NAMO-SIM (see Fig. 6.a). As this tools soon started to be used by the community, we asked for an Inria ADT project. In 2023, we obtained this ADT (NAMOEX) which allowed to hire David Brown (Ing.), in order to develop the simulator, by adding new functionnalisties, including its connexion with 3D simulators and the monitoring of experiments with real robots.

On December, 19th, 2023, Benoit Renault defended his Phd (Navigation Among Movable Obstacles (NAMO) Extended to Social and Multi-Robot Constraints), see 40.

8.5.1 Multi-UAV Exploration and Visual Coverage of 3D Environments

Participants: Alessandro Renzaglia, Olivier Simonin, Benjamin Sportich.

Multi-robot teams, especially when involving aerial vehicles (UAVs 19), are extremely efficient systems to help humans in collecting information on large and complex environments. In these scenarios, cooperative coverage and exploration are two fundamental problems with numerous important applications, such as mapping, inspection of complex structures, and search and rescue missions. In both cases, the robots' goal is to navigate through the environment and cooperate to acquire new information to accomplish the mission in the minimum possible time.

Following our recent results on these problems 114, 113, in the last year we focused on a strictly related path planning problem where a fleet of UAVs must use both prior information and online gathered data to efficiently inspect large surfaces such as ship hulls and water tanks 79. This work is conducted in the framework of the H2020 European project BugWright2, where one central use-case is focused on corrosion detection and inspection on cargo ships. In our case, UAVs can detect corrosion patches and other defects on the surface from low-resolution images. If defects are detected, they get closer to the surface for a high-resolution inspection. The prior information provides expected defects locations and is affected by both false positives and false negatives. The mission objective is to prioritize the close-up inspection of defected areas while keeping a reasonable time for the coverage of the entire surface. In this work, we propose two solutions to this problem: a coverage algorithm that divides the problem into a set of Traveling Salesman Problems and a cooperative frontier approach that introduces frontier utilities to incorporate the prior information.

A new ANR project (ANR JCJC project "AVENUE"), started in January 2023, aims now to further investigate these research lines by focusing on the cooperative exploration and 3D reconstruction of complex unknown environments with a fleet of aerial vehicles. Within this framework, a new PhD student, Benjamin Sportich, began his thesis in November 2023, co-supervised by Alessandro Renzaglia and Olivier Simonin.

8.5.2 Decentralized control of swarm of UAVs: Flocking models

Participants: Alexandre Bonnefond (Phd. Inria/CITI), Isabelle Guerrin-Lassous (Lyon 1/LIP), Olivier Simonin, Johan Faure.

(a) simulation of a flocking (b) outdoor inflatable viary

(a) simulation of a flocking (b) outdoor inflatable viary

This work was mainly conduced through the Inria/DGA "AI" project "DYNAFLOCK" (2019-23). It funded the PhD thesis of Alexandre Bonnefond.

In this project, we studied communication-based flocking models (decentralized swarm navigation) and proposed extensions to improve their abilities to deal with environments having obstacles. Often depicted as robust systems, there is yet a lack of understanding how flocking models compare and how they are impacted by the communication quality when they exchange control data.

After extending and combining two standard models, Olfati-Saber 100 and Vásárhelyi et al. 127, we showed their limits (see IROS 2021 59 and Fig. 7.a). In 2022/2023, we proposed a new model, called APR (Asymetric Pressure Regulation) 39. This model introduces an individual measure of pressure allowing to define a new flocking control able to anticipate dangerous/close agents interactions. Alexandre Bonnefond defended his PhD thesis on June 14th, 2023.

In 2023, we also continued on developing flocking simulations with the Gazebo 3D simulator (mainly supported by Johan Faure). We also conducted real experiments with PX4 Vision UAVs, embedding UWB-based distance measurement, and allowing to test flocking/formation experiments (see Fig. 7.b).

8.5.3 Communication and localization in swarm of UAVs

Participants: Olivier Simonin, Thierry Arrabal, Théotime Balaguer (PhD student), Stephane d'Alu, Johan Faure, Isabelle Guerin-Lassous (Lyon 1/Lip), Nicolas Valle, Hervé Rivano (Agora team).

In 2023, we continued the work within the CONCERTO ANR project : study of efficient communications in fleet of UAVs (aerial robots). With the arrival of Thierry Arrabal (post-doc/engineer), we built up an experimental platform that aim to be as "plug-n-play" as possible. The first step was the deployment of a network infrastructure to connect any needed device (computer, UAV, camera, etc..). Then, UAVs have been configured to allow them to fly both indoor and outdoor (in aviaries). An experimental pipeline have been develop (and still under active improvement) to facilitate the conduction of regular experimental studies.

This platform is used to study the limitations of the networking capacity in UAV communications and how these limitations mights affect their control. For instance, we conducted batches of experiments consisting in sending ROS2 messages to pilot UAVs throught a Wi-Fi network while actively perturbing the Wi-Fi channel. Some of these tests have been done in association with the ARMEN team from the LS2N in Nantes to benefits from their MOCAP system to get precise data about UAV poses. In figure 8, we compare the UAV behavior without network perturbation (in blue) against the UAV behavior with network perturbation (in red) while a remote computer sends periodical command to perform a rotation on the Z-axis (carousel behavior).

Traffic perturbation on UAV remote control

Traffic perturbation on UAV remote control

In 2023, we also continued to study the Ultra-Wide Band (UWB) technology to estimate distance between UAVs. In particular, in the paper 35, we explored the possibility to embed several UWB emitter/receiver per UAV in order to estimate distances and the relative angle.

In the following of our early work 131, we designed supports to embed UWB chips on our PX4 Vision UAVs. In 2023, we conducted experiments with 3 PX4 UAVs, to test autonomous leader-follower formation flights. This work is continuing by experimenting flocking models.

8.6.1 Global Min-Max Computation for alpha-Hölder Zero-Sum Games

Participants: Aurelien Delage [PhD Student], Jilles S. Dibangoye, Olivier Buffet [Inria Nancy, Larsen].

Min-max optimization problems recently arose in various settings. From Generative Adversarial Networks (GANs) to aerodynamic optimization through Game Theory, the assumptions on the objective function vary. Motivated by the applications to deep learning and especially GANs, most recent works assume differentiability to design local search algorithms such as Gradient Descend Ascent (GDA). In contrast, this work will only require $\alpha$-Hölder properties to tackle general game-theoretic problems with poor continuity as- sumptions. Focusing on the example of problems in which max and min optimization variables live in simplices, we provide a simple algorithm, based on Deterministic Optimistic Optimization (DOO), relying on an outer min-optimization using the solutions of an inner max-optimization. The algorithm is shown to converge in finite time to an $ϵ$-global optimum. Experimental validations are given and the time complexity of our algorithm is studied. This work led to a publication 25.

8.6.2 Optimally Solving Decentralized POMDPs Under Hierarchical Information Sharing

Participants: Johan Peralez [Post-doc], Aurélien Delage [PhD Student], Olivier Buffet [Inria Nancy, Larsen], Jilles S. Dibangoye.

This work is the result of an effort to provide a strong theory for optimally solving decentralized decision-making problems, represented as decentralized partially observable Markov decision processes (Dec-POMDPs), with tractable structures 128. This work describes core ideas of multi-agent reinforcement learning algorithms for decentralized partially observable Markov decision processes (Dec-POMDPs) under hierarchical information sharing. That is the dominant management style in our society—each agent knows what her immediate subordinate knows. In this case, we prove backups to be linear due to (normal-form) Bellman optimality equations being for the first time specified in extensive form. Doing so, one can sequence agent decisions at every point in time while still preserving optimality; hence paving the way for the first optimal $Q$-learning algorithm with linear-time updates for an important subclass of Dec-POMDPs. That is a significant improvement over the double-exponential time required for normal-form Bellman optimality equations, in which updates of agent decisions are performed in sync every point in time. Experiments show algorithms utilizing extensive instead of normal-form Bellman optimality equations can be applied effectively to much larger problems. This work is under review for a publication in an international conference.

8.6.3 Optimally Solving Decentralized POMDPs: The Sequential Central Learning Approach

Participants: Johan Peralez [Post-doc], Aurélien Delage [PhD Student], Olivier Buffet [Inria Nancy, Larsen], Jilles S. Dibangoye.

The centralized learning for simultaneous decentralized execution paradigm emerged as the state-of-the-art approach to $ϵ$-optimally solving a simultaneous- move decentralized partially observable Markov decision process, but the scalability remains a significant issue. This paper presents a novel—yet more scalable—alternative, namely sequential central learning for simultaneous decentralized execution. This methodology pushes the applicability of Bellman’s principle of optimality further, raising three new properties. First, it allows a central learner to reason upon sufficient sequential statistics instead of prior simultaneous ones. Next, it proves that $ϵ$-optimal value functions are piecewise linear and convex in sufficient sequential statistics. Finally, it drops the time complexity of the backup operators from double exponential to linear at the expense of longer planning horizons. Besides, it makes it easy to use single-agent methods—e.g., a reinforcement learning algorithm enhanced with these findings applies while still preserving its asymptotic convergence guarantees. Experiments on standard 2- as well as $n$-agent benchmarks from the literature against state-of-the-art $ϵ$-optimal solvers confirm the superiority of the novel paradigm.

8.6.4 Learning To Act Optimally in Sequential-Move Dec-POMDPs: Quantized Q-learning Algorithms

Participants: Jacopo Castellini [Post-doc], Johan Peralez [Post-doc], Jilles S. Dibangoye.

Policy optimization is currently the dominant approach to solve sequential-move $n$-agent Dec-POMDPs. Methods of this family demonstrated an impressive ability to scale up on large-scale entertainment applications, yet they often fail to provide provable guarantees in the face of critical applications with significant consequences on humans or wildlife, e.g. search and rescue, wildlife protection or autonomous vehicles. This work introduces core ideas for the design of optimal multi-agent value-based reinforcement learning algorithms for sequential-move $n$-agent Dec-POMDPs. It leverages the piece-wise linearity and convexity (PWLC) property of the optimal $Q$-value functions in sequential-move $n$-agent Dec-POMDPs. In particular, it introduces a temporal-different update rule that ensures asymptotic convergence toward an optimal solution with a linear-time complexity. The paper also describes quantized $Q$-learning algorithms, featuring a tabular representation of PWLC $Q$-value functions. These algorithms succeed where state-of-the-art policy- and value-based methods fail. They eventually find optimal solutions in small to medium-sized domains and scale up to much larger problems from the literature. This algorithmic scheme also lays the foundation for further work on tabular and neural value-based multi-agent reinforcement learning methods.

8.6.5 Stable Reinforcement Learning Methods

Participants: Jilles S. Dibangoye, Vincent Andrieu [LAGEPP, Lyon], Daniele Astolfi [LAGEPP, Lyon], Giacomo Casadei [LAGEPP, Lyon], Madiha Nadri [LAGEPP, Lyon], Samuele Zoboli [PhD Student, LAGEPP, Citi lab].

This recent research project is at the intersection of three closely related research fields: optimal control, machine learning, and mobile robotics. In this project, we try to find the best compromise between control methodologies to provide stability guarantees and machine learning methodologies that can scale up and converge towards a locally optimal solution. Applications of this hybrid methodology concern mobile robotics, including the driving of drones at very high speed. Two PhD. theses have been started on this problem the last year.

PhD thesis of Samuele Zoboli (since Jul. 2020, ANR DELICIO). In control theory, a natural approach for the design of stabilizing control is to consider a linear approximation around the operating point of the dynamic system. In this way, it is possible to design control laws that are optimal and which guaranteed stability for the linearized system. However, this behavior is only guaranteed for the original system around this operating point (local approach). The PhD thesis of S. Zoboli developped new real time algorithms (contol law and estimation algorithms for dynamical systems) that combines tools coming from control theory and reinforcement learning. He defended his PhD thesis on 28/09/2023.

8.6.6 Event-based Neural Learning for Quadrotor Control

Participants: Estéban Carvalho [PhD Student, GIPSA lab, Citi lab], Pierre Susbielle [GIPSA lab, Grenoble], Ahmad Hably [GIPSA lab, Grenoble], Nicolas Marchand [GIPSA lab, Grenoble], Jilles S. Dibangoye.

The design of a simple and adaptive flight controller is a real challenge in aerial robotics. A simple flight controller often generates a poor flight tracking performance. Furthermore, adaptive or learning algorithms are costly in time and resources, and may cause instability problems, for instance in the presence of disturbances. In this paper, we propose an event-based neural learning strategy that combines the use of a standard cascaded flight controller enhanced using a deep neural network that learns the disturbances to improve the tracking performance. The strategy relies on two events: one allowing the improvement of tracking errors and a second to ensure the stability of the closed-loop system. After a validation of the proposed strategy in a ROS/Gazebo simulation environment, its effectiveness is confirmed on real experiments, in the presence of wind disturbance. In 2023, this work has been published in Autonomous Robots journal 13, and Esteban Carvalho defended his Phd thesis on 20/04/2023.

8.7.1 Constraint Programming

Participants: Jean-Yves Courtonne [Inria STEEP], François Delobel [LIMOS], Patrick Derbez [Inria Capsule], Léon Fauste, Arthur Gontier [Inria Capsule], Mathieu Mangeot [Inria STEEP], Loïc Rouquette, Christine Solnon.

In many of our applications, we have to solve constrained optimization problems (COPs) such as vehicle routing problems, or multi-agent path finding problems, for example. These COPs are usually NP-hard, and we study and design algorithms for solving these problems. We more particularly study approaches based on Constraint Programming (CP), that provides declarative languages for describing these problems by means of constraints, and generic algorithms for solving them by means of constraint propagation. In 33, we introduce canonical codes for describing benzenoids, and we introduce CP models for efficiently generating all benzenoids that satisfy some given constraints.

Post-doc of Loïc Rouquette (Dec. 2022 - Jul. 2023). This post-doc was funded by the ANR project DeCrypt, that aims at designing declarative languages (based on CP) for solving cryptanalysis problems, and it is in the continuation of the PhD thesis of Loïc. During his PhD thesis, Loïc has designed new propagation algorithms that are integrated in the Choco CP library 117, new CP models for mounting related-key differential attacks of Rijndael 116 and Boomerang attacks on Feistel ciphers 87. During his post-doc, he has integrated these algorithms in Tagada, a tool for automatically generating CP models for mounting related-key differential attacks given a graph-based specification of the cipher 26

PhD thesis of Léon Fauste (since Sept. 2021). This thesis is funded by Ecole Normale Supérieure Paris Saclay and it is co-supervised by Mathieu Mangeot, Jean-Yves Courtonne (from Inria STEEP team) and Christine Solnon. The goal of the thesis is to design new tools for choosing the best geographical scale when relocating productive activities, and the context of this work is described in 66. During the first months of his thesis, Léon has designed a first Integer Linear Programming (ILP) model described in 75. In 2023/2024, Léon is on leave (césure) for one year in order to follow a Master in Geography.

8.7.2 Multi-Robot Path Planning with cable constraints

Participants: Xiao Peng [PhD. student CITI], Christine Solnon, Olivier Simonin.

The research is done within the context of the european project BugWright2 that aims at designing an adaptable autonomous multi-robot solution for servicing ship outer hulls.

PhD thesis of Xiao Peng (since Nov. 2020). The goal of this PhD thesis is to design algorithms for planning a set of mobile robots, each attached to a flexible cable, to cover an area with obstacles. In 106, we formally define the Non-Crossing MAPF (NC-MAPF) problem and show how to compute lower and upper bounds for this problem by solving well known assignment problems, we introduce a Variable Neighbourhood Search approach for improving the upper bound, and we introduce a CP model for solving the problem to optimality. In 18, we extend this approach to the case of non point-sized robots. In 2023, we also extended the approach to deal with the coverage of a whole area using a single tethered robot, then we examined different algotihms to divide the environment in order to distribute the coverage to several robots.

8.7.3 Vehicle routing problems

Participants: Romain Fontaine [PhD Student CITI], Jilles S. Dibangoye, Christine Solnon.

The research is done within the context of the transportation challenge and funded by INSA Lyon.

9.1.1 Toyota Motor Europe (2006 - 2023)

Participants: Christian Laugier, David Sierra González, Özgür Erkent, Jilles Dibangoye, Christian Wolf, Wenqian Liu, Gustavo Salazar-Gomez.

The contract with Toyota Motors Europe is a joint collaboration involving Toyota Motors Europe, Inria and ProbaYes. It follows a first successful short term collaboration with Toyota in 2005. This contract aims at developing innovative technologies in the context of automotive safety. The idea is to improve road safety in driving situations by equipping vehicles with the technology to model on the fly the dynamic environment, to sense and identify potentially dangerous traffic participants or road obstacles, and to evaluate the collision risk. The sensing is performed using sensors commonly used in automotive applications such as cameras and lidar.

This collaboration has been extended in 2018 for 4 years (period 2018-2021) and Toyota provides us with an experimental vehicle Lexus equipped with various sensing and control capabilities. Several additional connected technical contracts have also been signed, and an exploitation license for the CMCDOT software has been bought by Toyota in 2018.

9.1.2 Sumitomo Electric Industries (2020-23)

Participants: Lukas Rummelhard, Robin Baruffa, Hugo Bantignies, Nicolas Turro, Christian Laugier.

R&D project to develop and demonstrate multi-modal perception using communicating infrastructures, data exchange and fusion between such infrastructure and mobile autonomous vehicles, and the added value of such architecture. Based on embedded Bayesian perception systems (CMCDOT framework), the project aims to develop long-term prediction and risk assessment.

9.2.1 IRT Nanoelec PULSE – Security of Autonomous Mobile Robotics (2023-2025)

Participants: Lukas Rummelhard, Thomas Genevois, Robin Baruffa, Hugo Bantignies, Nicolas Turro, Christian Laugier.

Chroma participates in a project supported by PIA in the scope of the program PULSE of IRT Nanoelec. The objective of this project is to integrate, develop and promote technological bricks of context capture, for the safety of the autonomous vehicle. Building on Embedded Bayesian Perception for Dynamic Environment, Bayesian data fusion and filtering technologies from sets of heterogeneous sensors, these bricks make it possible to secure the movements of vehicles, but also provide them with an enriched and useful representation for autonomy functions themselves. In this context, various demonstrators embedding those technology bricks are developed in cooperation with industrial partners. Current work is mainly focused on distributed perception using realistic communication systems and Bayesian occupancy grids, and integration of Neural-Network-based systems into probabilistic frameworks.

9.2.2 MeanWhile (2021 - 2023)

Participants: Fabrice Jumel, Jacques Saraydaryan.

This contract is a BPI France funding between MeanWhile compagny and CPE Lyon. The project concerns the development of an indoor geolocation device for third-party equipment for a fleet of autonomous mobile robots. It funds an engineer position (18 months CPE/Meanwhile).

10.1.1 Visits of international scientists

10.2.1 H2020 projects

10.3.1 ANR

10.3.2 PEPR

10.3.3 Grand Defis

10.3.4 DGA/INRIA AI projects

R&D Booster "Moov-IT" project (2020 – 2023)

Participants: Thomas Genevois, Lukas Rummelhard, Christian Laugier.

The objective of the project is to develop an autonomous system to be integrated in production chains to provide modernization, flexibility and productivity, including fixed and mobile transitic systems. In that perspective, Chroma's CMCDOT framework will be adapted, extended and integrated on a mobile robot from Akeoplus, and tested in real environments.

11.1.1 Scientific events: organisation

11.1.2 Scientific events: selection

11.1.3 Journal

11.1.4 Leadership within the scientific community

• Christine Solnon is member of the "comité de direction" of the GDR IA (www.gdria.fr)
• Christine Solnon is member of the "Conseil d'Administration" of the TUBA (lyonurbandata.sfr-monsitebusiness.fr)
• Anne Spalanzani is Co-animator of the GDR Robotique TS5 "Robotics and Data".
• Olivier Simonin is member of the Executive Committee of the PEPR O2R (Organic Robotics) 2023-2030 (www.cnrs.fr/en/pepr/pepr-exploratoire-o2r-robotique).
• Olivier Simonin and Fabrice Jumel are members of Robocup France committee: (www.robocup.fr/le-comite)
• Fabrice Jumel is a member of the organization committee of the Robocup@Home competition (since 2016).
• Christian Laugier is a founding member and co-chair of the IEEE RAS Technical Committee on "Autonomous Ground Vehicles and Intelligent Transportation Systems"

11.1.5 Scientific expertise

• Anne Spalanzani was a member of the jury of the “Georges Giralt Ph.D. Award” which showcases PhD theses defended during 2023 in European Universities from all areas of robotics.
• Anne Spalanzani was a member of the ANR CE33 "robotics and interaction" committee in 2023.

11.1.6 Research administration

• O. Simonin is member of the Inria Lyon COS (comité d'orientation scientifique du Centre).

11.2.1 Supervision

Phd in progress :

• Benjamin Sportich, Next-Best-View Planning for 3D Reconstruction with Cooperative Multi-UAV Systems, A. Renzaglia, O. Simonin (ANR AVENUE).
• Théotime Balaguer, Contrôle efficace de flotte de drones soumis à des contraintes de communication hétérogènes, O. Simonin, I. Guérin Lassous, I. Fantoni (Lyon1/LIP), (CNRS/LS2N Nantes) (INSA/DGA).
• Rabbia Asghar, Real-time analysis of complex traffic situations by Predicting/Characterizing Abnormal Behaviors and Dangerous Situations, A. Spalanzani, C. Laugier, L. Rummerhard.
• Manuel Diaz Zapata, AI-based Framework for Scene Understanding and Situation Awareness for Autonomous Vehicles, J. Dibangoye, C. Laugier, O. Simonin.
• Jean-Baptiste Horel, Validation des composants de perception basés sur l’IA dans les véhicules autonomes, R. Mateescu (Inria Convecs), A. Renzaglia, C. Laugier (funded by PRISSMA project).
• Pierre Marza, Apprentissage large-échelle d'agents autonomes capables de naviguer dans des environnements réels, C. Wolf (NaverLabs Europe), L. Matignon (Lyon1/LIRIS), O. Simonin (funded by Chaire IA "REMEMBER").
• Aurelien Delage, Zero-Sum Partially Observable Stochastic Games, Jilles Dibangoye and Olivier Buffet (Larsen Inria Nancy), funded by ANR JCJC PLASMA.
• Romain Fontaine, Optimisation de la mobilité en ville en tenant compte des conditions de traffic, J. Dibangoye and C. Solnon.
• Léon Fauste, Aide à la décision pour le choix de l'échelle lors de la relocalisation d'activités industrielles, M. Mangeot (STEEP, Inria Montbonnot), J.-Y. Courtonne (STEEP, Inria Montbonnot), and C. Solnon.
• Xiao Peng, Planning Problems in a Multi-Tethered-Robot System, O. Simonin, C. Solnon, European H2020 BugWright2 project funding. To be defended on 19/2/24.
• Thomas Genevois, Navigation de véhicule autonome à l'aide de grilles d'occupations probabilistes ». Thèse en validation d'acquis, Université Grenoble Alpes. A. Spalanzani, C. Laugier. To be defended in april 2024.
• Gustavo Salazar Gomez, AI-driven safe motion planning and driving decision-making for autonomous driving. Thèse de l'Université Grenoble Alpes. A. Spalanzani, C. Laugier.

Phd defended in 2023 :

• Benoit Renault, NAvigation en milieu MOdifiable (NAMO) étendue à des contraintes sociales et multi-robots, O. Simonin and J. Saraydaryan, defended on 19/12/23.
• Alexandre Bonnefond, Flocking models based on local communications : From theory to simulations, O. Simonin and I. Guerrin-Lassous (Lyon 1), defended on 14/6/23
• Mihai Popescu, Static and Dynamic Multi-Robot Routing with Periodic Connectivity Maintenance, Patrolling and Network Data Delivery, O. Simonin, A. Spalanzani, F. Valois (CITI/Inria Agora), defended on 11/7/23.
• Zoboli Samuel, Learning globalization of locally stabilizable policies for nonlinear systems, Jilles Dibangoye, Vincent Andrieu and Daniele Astolfi (LAGEPP,Lyon), funded by ANR PRC Delicio. Defended on 28/09/23.
• Estéban Carvalho, Amélioration de l'efficacité du contrôle du vol des quadrirotors par apprentissage neuronal, Ahmad Hably (Gipsa-Lab), Nicolas Marchand (Gipsa-Lab), Jilles S. Dibangoye.

11.2.2 Juries

• Anne was president of several PhD juries: Rotanna Ly (Université Grenoble Alpes, November 2023), Charles-Olivier Artizzu (Université Côte d'Azur, June 2023)
• Anne was member of several PhD juries: Houssem Eddine Boulahbal (Université Côte d’Azur, September 2023), Lina Mezghani (Université Grenoble Alpes, July 2023), Thomas Gilles (Ecole des mines Paris, April 2023).
• Anne was examiner of Eric Lucet's HDR, Sorbonne Université, october 2023.
• Olivier Simonin was president of the PhD thesis jury of Quentin Bruel (Centrale Lyon/CEA, 13/06/23) and Jamy Chahal (LIP6/Sorbonne Univ., 30/11/23).
• Olivier Simonin was member, as reviewer, of the PhD thesis jury of Nicolas Fontbonne (Univ. Sorbonne, 14/03/23), Jad Bassil (FEMTO/U. Franche-Comté, 17/09/23), Félix Quinton (ISAE/ONERA Toulouse, 1//11/23) and Aurélien Merci (Univ. Toulon, 15/12/23).

11.2.3 Education

Olivier Simonin:

• INSA Lyon 5th year / Master - Robotics option (20 students) : AI for Robotics (plannning, RL, bio-inspiration), Multi-Robot Systems, Robotics Projects, 80h, Resp., Telecom Dept.
• INSA Lyon 4th year / Master. : AI for telecom (Intro IA, Metaheuristics), 10h (90 students), Telecom Dept.
• INSA Lyon 3rd year / Lic. : Algorithmics, 50h (90 students), L3, Resp., Telecom Dept.

Jilles S. Dibangoye :

• INSA Lyon 4th year / Master : Introduction to AI, 20h (90 students), Resp., Telecom Dept.
• INSA Lyon 3rd year / Lic. & Master : Operating Systems, 56h L3 and 16h Master, Telecom Dept.

Christine Solnon :

• INSA Lyon 3rd year / Lic. : Advanced algorithms for artificial intelligence and graphs, 40h (130 students), Resp., Computer Science Dept.
• INSA Lyon 4th year / Master : Object Oriented and Agile software development, 62h (130 students), Resp., Computer Science Dept.
• INSA Lyon 5th year / Master : Fundamental Computer Science, 20h (60 students), Resp., Computer Science Dept.
• INSA Lyon 5th year / Master : Prescriptive Data Analytics, 6h (30 students), Computer Science Dept.

Fabrice Jumel :

• CPE Lyon 4-5th year / Master : Robotic vision, cognitive science, HRI, deeplearning, robotic platforms, Kalman Filter, 280h, SN Dept.
• Workshop Manager, Autonomous Mobile Robotics, "worldkills 2024", Lyon

Jacques Saraydaryan :

• CPE Lyon 5th year / Master : Autonomous Robot Navigation 24h, Particle Filter 12h, SN Dept.
• CPE Lyon 4-5th year / Master : Software Design and Big Data, 400h, Resp., SN Dept.
• CPE Lyon 4-5th year / Master : Software Architecture, 160h, SN Dept.
• CPE Lyon 4-5th year / Master : Introduction to Cyber Security, 20h, SN Dept.

Anne Spalanzani :

• INPG Ense3 Grenoble, Master MARS: Autonomous Robot Navigation 6h.

Agostino Martinelli:

• Master (M2R) MoSIG: Autonomous Robotics, 12h, ENSIMAG Grenoble.

Alessandro Renzaglia:

• INPG Ense3 Grenoble, Master MARS: AI and Autonomous Systems, 12h.
• INSA Lyon 5th year / Master - Robotics option: Multi-Robot Systems, 2h, Telecom Dept.
• INSA Lyon 3rd year / Lic.: supervision of research initiation projects (groups of 5 students) along one semester, 15h, Telecom Dept.

11.2.4 Interventions

• Anne Spalanzani gave a conference to the "University Inter-âge" of Grenoble presenting recent advances on autonomous cars
• F. Jumel, J. Saraydaryan and O. Simonin are members and founders of Lyontech Team (robocup@home) since 2017.

International journals

• 13 articleE.Esteban Carvalho, P.Pierre Susbielle, N.Nicolas Marchand, A.Ahmad Hably and J.Jilles Dibangoye. Event-based neural learning for quadrotor control.Autonomous RobotsJune 2023HALDOIback to text
• 14 articleA.Aurélien Delage, O.Olivier Buffet, J.Jilles Dibangoye and A.Abdallah Saffidine. HSVI Can Solve Zero-Sum Partially Observable Stochastic Games.Dynamic Games and ApplicationsSeptember 2023, 58HALDOIback to text
• 15 articleR.Romain Fontaine, J.Jilles Dibangoye and C.Christine Solnon. Exact and Anytime Approach for Solving the Time Dependent Traveling Salesman Problem with Time Windows.European Journal of Operational Research2023HALDOIback to textback to text
• 16 articleJ.-B.Jean-Baptiste Horel, P.Philippe Ledent, L.Lina Marsso, L.Lucie Muller, C.Christian Laugier, R.Radu Mateescu, A.Anshul Paigwar, A.Alessandro Renzaglia and W.Wendelin Serwe. Verifying Collision Risk Estimation using Autonomous Driving Scenarios Derived from a Formal Model.Journal of Intelligent and Robotic Systems1074April 2023, 1-45HALDOIback to text
• 17 articleM.Maria Kabtoul, M.Manon Prédhumeau, A.Anne Spalanzani, J.Julie Dugdale and P.Philippe Martinet. How To Evaluate the Navigation of Autonomous Vehicles Around Pedestrians?IEEE Transactions on Intelligent Transportation SystemsOctober 2023, 1-11HALDOIback to text
• 18 articleX.Xiao Peng, O.Olivier Simonin and C.Christine Solnon. Non-Crossing Anonymous MAPF for Tethered Robots.Journal of Artificial Intelligence Research78October 2023, 357-384HALDOIback to text
• 19 articleA.Aleksandra Savina, T.Thierry Jaffredo, F.Frederic Saldmann, C.Chris Faulkes, P.Philippe Moguelet, C.Christine Leroy, D. D.Delphine Del Marmol, P.Patrice Codogno, L.Lucy Foucher, A.Antoine Zalc, M.Mélanie Viltard, G.Gérard Friedlander, S.Selim Aractingi and R.Romain Fontaine. Single-cell transcriptomics reveals age-resistant maintenance of cell identities, stem cell compartments and differentiation trajectories in long-lived naked mole-rats skin.Aging (Albany N.Y.)149February 2023, 3728-3756HALDOI
• 20 articleP. T.Phani Teja Singamaneni, P.Pilar Bachiller-Burgos, L. J.Luis J. Manso, A.Anaís Garrell, A.Alberto Sanfeliu, A.Anne Spalanzani and R.Rachid Alami. A Survey on Socially Aware Robot Navigation: Taxonomy and Future Challenges.The International Journal of Robotics Research2024, 1-33HAL

International peer-reviewed conferences

• 21 inproceedingsN.Naveed Ahmad, M.Malcolm Egan, J.-M.Jean-Marie Gorce, J.Jilles Dibangoye and F.Frédéric Le Mouël. Optimization of Sensor Configurations for Fault Identification in Smart Buildings.ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)Rhodes Island, United StatesIEEE2023, 1-5HALDOI
• 22 inproceedingsS.Safuriyawu Ahmed, F.Frédéric Le Mouël, N.Nicolas Stouls and J.Jilles Dibangoye. R-MDP: A Game Theory Approach for Fault-Tolerant Data and Service Management in Crude Oil Pipelines Monitoring Systems.MobiQuitous 2022 - 19th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and ServicesPittsburgh, United StatesJanuary 2023HAL
• 23 inproceedingsT.Thierry Arrabal, M.Marija Stojanova, I.Isabelle Guérin-Lassous and J.Joris Picot. Experiment-driven platform for link quality estimation in IEEE 802.11 WLANs.HPSR 2023 - IEEE 24th International Conference on High-Performance Switching and RoutingAlbuquerque, NM, United States2023, 1-6HAL
• 24 inproceedingsR.Rabbia Asghar, M. A.Manuel Alejandro Diaz-Zapata, L.Lukas Rummelhard, A.Anne Spalanzani and C.Christian Laugier. Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps.awaiting publication on IEEE XploreIROS 2023 - IEEE/RSJ International Conference on Intelligent Robots and SystemsDetroit, Michigan, United StatesIEEEOctober 2023, 1-6HALback to text
• 25 inproceedingsA.Aurélien Delage, O.Olivier Buffet and J.Jilles Dibangoye. Global min-max Computation for α-Hölder Games.Proceedings of the 35th IEEE International Conference on Tools with Artificial Intelligence (ICTAI)ICTAI 2023 - 35th IEEE International Conference on Tools with Artificial IntelligenceAtlanta (Georgia), United StatesIEEE Computer SocietyNovember 2023, 518-525HALback to text
• 26 inproceedingsF.François Delobel, P.Patrick Derbez, A.Arthur Gontier, L.Loïc Rouquette and C.Christine Solnon. A CP-based Automatic Tool for Instantiating Truncated Differential Characteristics ⋆.LNCSINDOCRYPT 2023 - 24th International Conference on Cryptology in IndiaGoa, IndiaSpringerDecember 2023, 1-23HALback to text
• 27 inproceedingsM. A.Manuel Alejandro Diaz-Zapata, D.David Sierra González, Ö.Özgür Erkent, J.Jilles Dibangoye and C.Christian Laugier. LAPTNet-FPN: Multi-scale LiDAR-aided Projective Transform Network for Real Time Semantic Grid Prediction.Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA 2023)ICRA 2023 - IEEE International Conference on Robotics and AutomationLondres, United KingdomIEEEMay 2023, 1-6HALback to text
• 28 inproceedingsE.Erwan Escudie, L.Laëtitia Matignon and J.Jacques Saraydaryan. Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning.Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent SystemsAAMAS 2024 - International Conference on Autonomous Agents and Multi-Agent SystemsAuckland, New Zealand2024HALback to text
• 29 inproceedingsT.Thomas Genevois, L.Lukas Rummelhard, A.Anne Spalanzani and C.Christian Laugier. From Probabilistic Occupancy Grids to versatile Collision Avoidance using Predictive Collision Detection.2023 IEEE International Conference on Intelligent Transportation Systems (ITSC)ITSC 2023 - IEEE International Conference on Intelligent Transportation SystemsBilbao, SpainIEEESeptember 2023, 1-7HALback to text
• 30 inproceedingsT.Thomas Genevois, A.Anne Spalanzani and C.Christian Laugier. Interaction-aware Predictive Collision Detector for Human-aware Collision Avoidance.2023 IEEE Intelligent Vehicles Symposium (IV)IV 2023 - IEEE Intelligent Vehicles SymposiumAnchorage, United StatesIEEEJune 2023, 1-7HALDOIback to text
• 31 inproceedingsJ.-B.Jean-Baptiste Horel, R.Robin Baruffa, L.Lukas Rummelhard, A.Alessandro Renzaglia and C.Christian Laugier. A Navigation-Based Evaluation Metric for Probabilistic Occupancy Grids: Pathfinding Cost Mean Squared Error.IEEE International Conference on Intelligent Transportation SystemsITCS 2023 - 26th IEEE International Conference on Intelligent Transportation SystemsBilbao, Spain, SpainIEEE2023, 1-6HALback to text
• 32 inproceedingsP.Pierre Marza, L.Laetitia Matignon, O.Olivier Simonin and C.Christian Wolf. Multi-Object Navigation with dynamically learned neural implicit representations.ICCV 2023 - International Conference on Computer VisionParis, France2023, 1-21HALback to text
• 33 inproceedingsX.Xiao Peng and C.Christine Solnon. Using Canonical Codes to Efficiently Solve the Benzenoid Generation Problem with Constraint Programming.International Conference on Principles and Practice of Constraint Programming (CP)CP 2023 - 29th International Conference on Principles and Practice of Constraint ProgrammingToronto (CA), CanadaLIPIcs2023HALDOIback to text
• 34 inproceedingsJ.Jacques Saraydaryan, F.Fabrice Jumel and O.Olivier Simonin. Human Presence Probability Map (HPP): a Probability propagation based on Human Flow Grid.RoboCup 2023 - 26e symposium international RoboCupRoboCup 2023: Robot World Cup XXVBordeaux, France2023HAL
• 35 inproceedingsS.Stéphane d'Alu, H.Hervé Rivano and O.Olivier Simonin. TDoA for in-flight relative localization in UAV swarm using Ultra-Wide Band.CEUR Workshop ProceedingsIPIN 2023 - 13th International Conference on Indoor Positioning and Indoor NavigationIPIN-WiP 2023 Indoor Positioning and Indoor Navigation - Work-in-Progress Papers 2023Nuremberg, GermanySeptember 2023HALback to text

National peer-reviewed Conferences

• 36 inproceedingsT.Théotime Balaguer, O.Olivier Simonin, I. G.Isabelle Guérin Lassous and I.Isabelle Fantoni. Etat de l’art sur la co-simulation robotique et réseau des systèmes multi-robots.Journées Francophones des Systèmes Multi-AgentsJFSM 2023 - 31èmes Journées Francophones des Systèmes Multi-AgentsStrasbourg, France2023, 1-10HAL

Conferences without proceedings

• 37 inproceedingsA.Aurélien Delage, O.Olivier Buffet and J.Jilles Dibangoye. Global Min-Max Computation for α-Hölder Zero-Sum Games.GAIW 2023 - 5th Games, Agents, and Incentives WorkshopLondres (London), United Kingdom2023, 1-9HAL
• 38 inproceedingsA.Aurélien Delage, O.Olivier Buffet, J.Jilles Dibangoye and A.Abdallah Saffidine. Heuristic Search Value Iteration can solve zero-sum Partially Observable Stochastic Games.MSDM 202311th Multiagent Sequential Decision Making under Uncertainty Workshop ; Held as part of the Workshops at the IFAAMAS 2023 - 21st International Conference on Autonomous Agents and Multiagent SystemsLondres, United KingdomMay 2023HAL

Doctoral dissertations and habilitation theses

• 39 thesisA.Alexandre Bonnefond. Flocking models based on local communications : From theory to simulations.INSA de LyonJune 2023HALback to text
• 40 thesisB.Benoit Renault. Navigation Among Movable Obstacles (NAMO) Extended to Social and Multi-Robot Constraints.Insa LyonDecember 2023HALback to text

Reports & preprints

• 41 miscA.Agostino Martinelli. Detection of a serious error in the paper: "On identifiability of nonlinear ODE models and applications in viral dynamics".January 2024HALback to textback to textback to text
• 42 miscA.Agostino Martinelli. General analytical condition to nonlinear identifiability and its application in viral dynamics.January 2024HALback to text
• 43 miscA.Agostino Martinelli. Identifiability of nonlinear ODE Models with Time-Varying Parameters: the General Analytical Solution and Applications in Viral Dynamics.December 2023HALback to text

Other scientific publications

• 44 inproceedingsR.Rabbia Asghar, M. A.Manuel Alejandro Diaz-Zapata, L.Lukas Rummelhard, A.Anne Spalanzani, A.Anne Spalanzani and C.Christian Laugier. Vehicle Motion Forecasting using Prior Information and Semantic-assisted Occupancy Grid Maps.NAVER LABS 2023 - Europe International Workshop on AI for RoboticsGrenoble, FranceNovember 2023HALback to textback to text
• 45 miscT.Thomas Genevois, A.Anne Spalanzani and C.Christian Laugier. Video supplement for : "Interaction-aware Predictive Collision Detector for Human-aware Collision Avoidance", T. Genevois, A. Spalanzani and C. Laugier, in IEEE Intelligent Vehicles Symposium (IV), 2023.April 2023HAL

Scientific popularization

• 46 articleC.Christine Solnon. Déconstruire le mythe de l'immatérialité du numérique.La RechercheJuly 2023HAL
• 47 articleC.Christine Solnon. Des algorithmes pour expliquer les algorithmes.La RechercheApril 2023HAL
• 48 articleC.Christine Solnon. L'art d'écrire des programmes.La RechercheJanuary 2023HAL
• 49 articleC.Christine Solnon. Le meilleur des deux mondes.La Recherche2023HAL
• 50 article C.Christine Solnon. Optimiser, mais pour qui ? La Recherche January 2024 HAL