Project-team positioning

The team carries out research across the full (software) stack of an autonomous driving system. This goes from perception of the environment to motion forecasting and trajectory optimization. While most of the techniques that we develop can stand on their own and are typically tested on our experimental platforms, we slowly move towards their complete integration. This will result in a functioning autonomous driving software stack. Beyond that point, we envision to focus our research on developing new methods that improve the individual performance of each of the stack’s components, as well as the global performance. This is alike to how autonomous driving companies operate (e.g. Waymo, Cruise, Tesla). The research topics that we explore are addressed by a very long list of international research laboratories and private companies. However, in terms of scope, we find the research groups of Prof. Dr.-Ing. Christoph Stiller (KIT), Prof. Marcelo Ang (NUS), Prof. Daniela Rus (MIT) and Prof. Wolfram Burgard (TRI California) to be the closest to us; we have active interconnections with these research groups (e.g. in the scope of the IEEE RAS Technical Committee AGV-ITS we are co-chairing any of the related annual Workshops). At the national level, we are cooperating in the framework of several R&D projects with UTC, Institut Pascal Clermont-Ferrand, University Gustave Eiffel, and the Inria Teams RITS, Acentauri and Convecs. Our main research originality relies in the combination of Bayesian approaches, AI technologies, Human-Vehicle interactions models, Robust State Estimation formalism, and a mix of Real experiments and Formal methods for validating our technologies in complex real world dynamic scenarios.

Our research on visual-inertial sensor fusion is closely related to the research carried out by the group of Prof. Stergios Roumeliotis (University of Minnesota) and the group of Prof. Davide Scaramuzza (University of Zurich). Our originality in this context is the introduction of closed-form solutions instead of filter-based approaches.

Scientific achievements

1. Bayesian & AI based Environment Perception and Forecasting

In order to be able to operate in a dynamic environment surrounded by humans or other robots, an autonomous system needs to Perceive, Understand and Forecast any changes of its surroundings. We have investigated the perception task from different angles, most of them with the shared nexus of involving both Bayesian approaches and Deep Learning models.

Bayesian Perception: The team's work on Bayesian approaches has given rise over the last decade to numerous publications and to some patents and licensed software8. Over the past 4 years, we have focused on collision risk assessment, safe navigation and semantic processing issues, with a strong emphasis on the integration and the validation of our technologies on real-time experimental platforms. The new results gave rise to the filing of two patents and to a transfer of technology with a French SME and with a Japanese international company (R&D work still ongoing). The main advances relate to (1) the generalization of the principle of Bayesian fusion in order to be able to integrate connected sensors located outside the ego-vehicle (infrastructure or other vehicles), (2) new real-time methods allowing to better estimate the risks of collision and make navigation safer (including path planning), and (3) experimental validation principle combining real tests, synchronized simulated environments (augmented reality principle) and formal methods 50, 80, 92, 49. A new PhD thesis addressing the subject of validating AI-based perception components in autonomous vehicles, with a particular focus on formal validation methods, has started in 2021 in cooperation with the Inria Convecs team (funding: French project PRISSMA).

Situation awareness & Environment Forecasting: In addition to the above-mentioned work, we have developed AI-based methods for extracting the semantic information of observed scenes 78, 77, 79, 105. In particular, we have developed hybrid methods for 3D object detection 94, 10824, semantic mapping of the environment from a top-view 66, 67, 68 and ground plane estimation and segmentation 93. We have likewise investigated the use of domain adaptation techniques to mitigate the degradation of some of the previous algorithms across different weather conditions 65, 62 and sensor modalities 105. In terms of environment forecasting, two PhD theses were dedicated to the task of predicting the future behavior of surrounding traffic 48, 109. The first of them targeted road intersections; the second focused on highway scenarios and leveraged a behavioral model learned from driving demonstrations 106 to cope with the long-term prediction horizon necessary for highways. Both theses also explored how to integrate the forecasting models into a behavioral planning framework 107. In addition to these forecasting models, we have also carried out research on localization 47, tracking 20, detection of abnormal behaviors 118, and contextualization of complex risk mitigation scenarios 104. A new PhD thesis addressing the subject of “Hybrid Sensor fusion” was launched in 2021 (funding: EU CPS4EU).

2. Modeling and simulation of dynamic environments

Modeling pedestrian's behaviors with a limited amount of data is a challenge. In Pavan Vasishta's PhD and the ANR Valet project, we proposed to learn typical pedestrian behavior using HMM, starting from prior knowledge itself derived from the JJ Gibson’s sociological principles of “Natural Vision” and “Natural Movement”. The work assumes that there are some parts of the observed scene which are more attractive to pedestrians and some areas, repulsive 113.

Pedestrians simulation. We studied the behavior of pedestrians in a shared space with an autonomous vehicle by modeling and simulating realistic pedestrian behavior. Our approach integrates empirical observations and concepts from social science into an agent-based model and simulator for an application in robotics. At each step, the proposed model has been evaluated and validated through simulations of many scenarios and comparisons with real-world data. Then we proposed the simulator SPACiSS, "Simulator for Pedestrians and an Autonomous Car in Shared Spaces", an open source simulator based on the PedSim framework that can simulate interactions between pedestrians and vehicles in different shared space scenarios. With the integration in the ROS framework, commonly used in robotics, the software is designed as an environment to test autonomous navigation systems 95, 9627, 14.

Robots in human flows. Another work, led by J. Saraydaryan, was conduced on simulating pedestrians and mobile robots in indoor environments, by extending PedSim. This work aimed to simulated scenarios where robots perceive and map human flows and compute paths by exploiting such information (in the cost-function of A* algorithm) 10272.

3. Robust State Estimation

Our investigations have been in the framework of the following two fundamental problems: (i) the visual and inertial sensor fusion problem, and (ii) the unknown input observability problem. Regarding the former, our activity was the follow-up of our previous research on this domain, published in 88, 81, and 75. We extended the above results to the cooperative case (in particular, we studied the case of two agents). First, we proved the equivalence between the cooperative visual-inertial sensor fusion problem and a Polynomial Equation System (PES) 87. The power of this equivalence is twofold. From one side, it allowed us to obtain its analytical solution 86. In other words, we obtained the closed-form solution of this important problem and, as a result, we were able to determine the state without the need of an initialization. The analytical solution was obtained by using the method based on the Macaulay resultant matrices developed by the AROMATH team at INRIA Sophia Antipolis 111. This solution, that is the exact solution of the cooperative visual and inertial sensor fusion problem in the case of two agents, was published in 86. From another side, the theoretical study of the PES, allowed us to obtain fundamental insights into all the structural properties of the problem 82. The usefulness of this analysis has also been illustrated with simulations.

Regarding the second problem, i.e., the unknown input observability problem, we provided a fundamental new result. This problem was introduced and firstly investigated during the sixties and has remained unsolved for 50 years. During the last five years, we have obtained the analytic solution of this fundamental open problem. Very surprisingly, the complexity of this solution is comparable to the complexity of the standard method to check the state observability in the case without unknown inputs (i.e., the observability rank condition). On the other hand, the analytic derivations and all the proofs necessary to analytically derive the new solution and its convergence properties and to prove its general validity are very complex and they are extensively based on the Ricci calculus with tensors. The analytic criterion becomes much easier for driftless systems and in the presence of a single unknown input. The solution of this simple case (driftless systems and in the presence of a single unknown input) is given in 84. Due to its complexity, the general solution was published in a book 85.

Collaborations

• Main long-term industrial collaborations: IRT Nanoelec & CEA $\sim$10 years), Toyota Motor Europe ($\sim$12 years, R&D center Zaventen-Brussels), Renault ($\sim$10 years), SME Easymile ($\sim$3 years), Sumitomo Japan (started in 2019), Iveco (3 years in the framework of STAR project). These collaborations are funded either by Industry or by National, Regional or European projects. The main outputs are common publications, softwares development, patents or technological transfers. Two former doctoral students of our team were recruited some years ago by respectively TME Zaventen and Renault R&D centers.
• Main academic collaborations: Institut Pascal Clermont-Ferrand, University Compiegne (UTC), University Gustave Eiffel, several Inria Teams (RITS, Acentauri, Convecs, Rainbow). These collaborations are most of the time conducted in the scope of various National R&D projects (FUI, PIA, ANR, etc.). The main outputs are scientific and software exchanges. In addition, we had a fruitful collaboration with B. Mourrain from the Inria-Aromath team. The goal was to find the analytical solution of a polynomial equation system that fully characterizes the cooperative visual-inertial sensor fusion problem with 2 agents.
• International cooperation for scientific exchanges and IEEE scientific events organization, e.g. NUS & NTU Singapore, Peking University, TRI Mountain View, KIT Karlsruhe, Coimbra University. We have constantly interacted with Prof. D. Scaramuzza from the university of Zurich to make our findings in visual inertial sensor fusion more usable in a realistic context (e.g., for the autonomous navigation of drones).

Project-team positioning

In his reference book Planning algorithms76, S. LaValle discusses the different dimensions that make the motion-planning a complex problem, which are the number of robots, the obstacle regions, the uncertainty of perception and action, and the allowable velocities. In particular, it is emphasized that multiple robot planning in complex environments are NP-hard problems which implies that exact approaches have exponential time complexities in the worst case (unless P=NP) Moreover, dynamic and uncertain environments, as human-populated ones, expand this complexity. In this context, we aim at scaling up decision-making in human-populated environments and in multi-robot systems, while dealing with the intrinsic limits of the robots and machines (computation capacity, limited communication). To address these challenges, we explore new algorithms and AI architectures following three directions: online (or real-time) planning, machine learning to adapt to the complexity of uncertain environments, and combinatorial optimization to deal with offline constrained problems. Combining these approaches, seeking to scale up them, and also evaluating them with real platforms are also elements of originality of our work.

We share these goals with other laboratories/teams in the world, such as the Mobile Robotics Laboratory and Intelligent Systems "IntRoLab" at Sherbrooke University (Montreal) led by Prof. F. Michaud, the Learning Agents Research Group (LARG) within the AI Lab at the University of Texas (Austin) led by Prof. P. Stone, the Autonomous Systems Lab at ETH Zurich (Switzerland) led by R. Siegwart, the Robotic Sensor Networks Lab at the University of Minnesota (USA) led by Prof. V. Isler, and the Autonomous Robots Lab at NTNU (Norway) to cite a few. At Inria, we share some of the objectives with the LARSEN team in Nancy (multi-agent machine learning, multi-robot planning) and the RAINBOW team in Rennes (multi-UAV planning). We have also collaborations with the ACENTAURY team, in Sophia Antipolis, about autonomous navigation among humans. In France, among other labs involved on similar subjects, we can cite the LAAS (CNRS, Toulouse), the ISIR lab (CNRS, Paris Sorbonne Univ.) and the MAD team in Caen University. In the more generic domain of problem solving, we can mention the TASC team at LS2N (Nantes), the CRIL lab (Lens) and, at the international level, we can cite the Insight Centre for Data Analytics (Cork, Ireland).

Scientific achievements

1. Online Planning

In this theme, we address online planning when robots/vehicles/UAVs have to navigate or work in complex and generally unknown environments. The challenge comes from the necessity for entities to decide actions in real time and with limited information. Hereafter we present our main results following different problem contexts.

Social and ethic navigation

We investigated new methods to build safe and socially compliant trajectories among humans using models of behaviors presented in Axis I. A focus on navigation of an autonomous vehicle among crowds has been done were there is no free space to plan a safe trajectory (M. Kabtoul's PhD, ANR Hianic Project). We proposed a solution that is a proactive social navigation framework. The system is based on the idea of the coupled navigation behavior between the pedestrian and the vehicle in shared spaces. The system takes into account the cooperative nature of human behavior and exploits it to explore new navigation options, and navigate the shared space “proactively" 74, 73. We obtained promising results using the SPACISS simulator previously described. We also investigated ethic navigation when an autonomous vehicle cannot avoid a collision with an object or a vulnerable road user (L. Serafim Guardini's PhD). We exploit accidentology data where each class of object or agent presents an injury probability with respect to the impact speed and ethical/economical/political factors. Our method generates a cost map containing a collision probability along with its associated risk of injury, which are used to plan trajectories with the lowest risk 104.

We also investigated another navigation problem, which is NAMO (Navigation Among Movable Obstacles). We extended NAMO in two directions considering social constraints when robots move obstacles (B. Renault's PhD). First, we examined where obstacles should be optimally moved with regards to space access (called Social-NAMO). We derived new spatial cost functions and NAMO algorithms preserving accesses 97. Second, we generalized the problem to $n$ robots (called MR-NAMO). We proposed a local coordination strategy exploiting Social-NAMO rearrangement properties and offering online efficient plans (under review). For evaluation, we developped a simulator (S-NAMO-SIM) that we share with the communauty.

Multi-robot exploration

An important part of our research is focused on designing new online planning solutions for multi-robot systems to accomplish exploration and observation gathering tasks in complex and dynamic environments. Problems related to computational and communication constraints are here crucial, and distributed approaches are fundamental to overcome them and obtain robust solutions.

Following these goals, a particular attention has been given to the exploration/mapping of 3D environments with a team of aerial vehicles, for which we presented a new decentralized solution based on the combination of a stochastic optimization approach with a more classic strategy that exploits frontier points 100. In 99, we then showed how this approach is part of a more general formulation that can deal with the optimal deployment coverage as well, for which we also studied the impact of an offline initialization step based on very partial information on the environment. Stochastic-based planning methods have been also exploited to propose a new source seeking strategy 98. This solution is based on robust estimations of the signal gradient obtained by the robots while exploring the environment in predefined symmetric formations 56 and that are then used to bias a correlated random walk that guides the robots.

Connected to this subject, we also investigated multi-robot strategies to cover complex scenes/ environments, by introducing multi-resolution maps. This concerned human activity observation by a set of ground robots (89) and more recently inspection of large structures with aerial robots (in the framework of the European project BugWright2, in collaboration with LSL, Austria).

Swarm navigation

In the context of self-reorganized systems, or swarm robotics, we explore how collective navigation can be defined and adapted to fleets of communicating robots. In A. Bonnefond's PhD, we studied flocking models, where each agent communicates locally its speed and position, and by simulating radio propagation we examined how obstacles impact the flocking robustness. Then we extended and combined two standard models, Olfati-Saber and Vásárhelyi, to improve their ability to stay connected while evolving in environments with different obstacle distributions 17. This work is supported by the Inria/DGA Dynaflock project and by the national Equipex+ TIRREX infrastructure. Related to this work, we address Controled Mobility where navigation must take into account the need of maintaining the fleet connectivity. In R. Grunblatt's PhD, we proposed a distributed algorithm that optimizes communication in a swarm of UAVs by changing their antenna orientation online 13. This work is continued in the ANR CONCERTO project.

2. Machine Learning

Machine learning techniques for decision-making are autonomous algorithms for computing behaviors, also called policies or strategies, which describe what action an agent should take when at a given state. Learning to act in complex environments requires knowledge about the structure of underlying decision-making problems. We contribute in revealing insightful underlying structures, exploiting them within existing or customized algorithms, and showing they improve performances in a wide variety of decision-making problems. For instance, optimal policies may have concise representations of their mental states, e.g., sufficient statistics, ego-centric neural-network architectures.

In the context of POMDPs9, dec-POMDPs, zs-POSGs10, and st-POSGs, we have identified a unified framework for solving these problems 64. Our methodological framework splits into five steps. First, we have shown that all these problems are convertible into a single problem called a common-knowledge game 64. Then, we recast the (possibly discrete) problem into a continuous one. Furthermore, we establish uniform continuity properties for general and hierarchical dec-POMDPs. We proved that optimal value functions for general and hierarchical dec-POMDPs are PWLC 64, 117. We also established optimal value functions for general zs-POSGs that are Lipschitz continuous 57, 58. Finally, we designed planning algorithms for games against Nature 69, pure competitive games 12, as well as for pure cooperative games 64. We extended exact and deep reinforcement learning algorithms for general dec-POMDPs 64. We finally conducted applications of deep reinforcement along two main directions: autonomous transportation systems 55; and automatic control systems 30.

Creating agents capable of high-level reasoning based on structured memory was the main topic of C. Wolf during its Inria "delegation" in Chroma (2017-19). In the PhD thesis of E. Beeching we proposed EgoMap, a spatially structured metric neural memory architecture in deep reinforcement learning. Through visualizations, the agent learns to map relevant objects in its spatial memory without any supervision purely from reward, see 53 (ICPR), 52 (ECCV) and 51 (ECML). In late 2019, C. Wolf obtained the AI Chair "REMEMBER", involving O. Simonin and J. Dibangoye from Chroma and L. Matignon from LIRIS lab/Univ Lyon 1. In this context, the PhD of P. Marza proposes to use an implicit representation of the environment to navigate towards targets. Neural fields are a recent category of approaches where the geometry, and eventually the semantics, of a 3D scene are encapsulated into the weights of a trained neural network.

As mentioned in Axis I, making appropriate decisions for safe robot / vehicle navigation is highly dependent on the results of the perception step. Indeed, the critical elements of a complex dynamic scene must first be extracted, classified and tracked in real time, by combining Bayesian and AI approaches to build "semantic grids" 3637, 66, 65. Two types of decision-making based on RL were developed for autonomous driving on highways and road junctions respectively, as part of the doctoral theses of David Sierra-Gonzalez 109 and Mathieu Barbier 48.

A last activity concerns our involvement in the RoboCup11 competition. In 2017 we founded the LyonTech team12 to participate to the RoboCup@Home challenge within the Pepper league. We worked on learning and integrating high-level functionalities such as object and people recognition, mapping and planning in indoor environments (see 71). We reached the 5th place in 2018, the 3rd place in 2019, the 2d place in 2021, and awarded the best scientific paper at the RoboCup conference in 2019 103.

3. Offline constrained optimisation problems

In this theme, we study and design algorithms for solving NP-hard constrained optimization problems (COPs) such as vehicle routing problems, or multi-agent path finding problems. We more particularly study approaches based on Constraint Programming (CP), that provides declarative languages for describing these problems by means of constraints, and generic algorithms for solving them by means of constraint propagation.

In the context of the European project BugWright2 and the PhD thesis of Xiao Peng (started in Nov. 2020), we study a new multi-agent path finding problem for tethered robots, that are attached with cables to anchor points. In 26, we show how to compute bounds for this problem by solving well known assignment problems, and we introduce a CP model for solving the problem to optimality. In 91, we extend this approach to the case of non point-sized robots.

In the context of the PhD thesis of Romain Fontaine (started in Oct. 2020), we study vehicle routing problems with time-dependent cost functions (such that travel times between locations to visit may change along the day), and we design new solving approaches based on an anytime extension of A* 70. We study the impact of time-window constraints on the satisfiability and the difficulty of routing problems in 28.

Besides planning and routing problems which are at the core of Chroma topics, we also investigate the interest of using CP solvers for solving cryptanalysis problems 23.

Collaborations

• Social navigation: collaboration with Julie Dugdale and Dominique Vaufreydaz, Asso. Prof. at LIG Laboratory and with Prof. Philippe Martinet from Acentori Inria team (common PhD students and projects: ANR VALET and ANR HIANIC).
• Multi-robot exploration: Prof. Cedric Pradalier from GeorgiaTech Metz/CNRS (France) leading the European project BugWright2 — Prof. Luca Schenato from University of Padova (Italy).
• Swarm of UAVs: Prof. Isabelle Guérin-Lassous from LIP lab/Inria DANTE team (common PhD students and projects: ANR CONCERTO, INRIA/DGA DynaFlock) — Research Director Isabelle Fantoni from CNRS/LS2N Lab in Nantes (common projects: ANR CONCERTO, Equipex TIRREX) — Dr. Melanie Schranz, senior researcher at LakeSide Lab, Austria (BugWright2 project).
• Machine Learning: strong collaborations with Christian Wolf, Naver Labs Europe & LIRIS lab/INSA13 (common PhD students and projects: ANR DELICIO, ANR AI Chair REMEMBER) and with Laetitia Matignon, Asso. Prof. at LIRIS lab/Univ Lyon 1 (common projects and Phd/Master students) — Akshat Kumar, Asso. Prof. at School of Computing and Information Systems (Singapore) — Abdallah Saffidine, Research Asso. at University of New South Wales (Sydney) — Prof. Stéphane Canu at INSA Rouen — Olivier Buffet, Vincent Thomas and François Charpillet, Inria researchers in LARSEN team, Nancy (common publications and project ANR PLASMA).
• Semantic grids for autonomous navigation: Collaboration with Toyota Motor Europe (TME), R&D center Zaventen (Brussels). R&D contracts, common publications and patents.
• Constrained optimization: Omar Rifki (Post-doc) and Thierry Garaix (Ass. Prof) at LIMOS Ecole des Mines de Saint Etienne — Pascal Lafourcade and François Delobel, Asso. Prof. at LIMOS Université Clermont-Ferrand.

7.1.1 CMCDOT

• Keywords:
  Robotics, Environment perception
• Functional Description:
  CMCDOT is a Bayesian filtering system for dynamic occupation grids, allowing parallel estimation of occupation probabilities for each cell of a grid, inference of velocities, prediction of the risk of collision and association of cells belonging to the same dynamic object. Last generation of a suite of Bayesian filtering methods developed in the Inria eMotion team, then in the Inria Chroma team (BOF, HSBOF, ...), it integrates the management of hybrid sampling methods (classical occupancy grids for static parts, particle sets for parts dynamics) into a Bayesian unified programming formalism , while incorporating elements resembling the Dempster-Shafer theory (state "unknown", allowing a focus of computing resources). It also offers a projection system of the estimated scene in the near future, to reference potential collisions with the ego-vehicle or any other element of the environment, as well as very low cost pre-segmentation of coherent dynamic spaces (taking into account speeds). It takes as input instantaneous occupation grids generated by sensor models for different sources, the system is composed of a ROS package, to manage the connectivity of I / O, which encapsulates the core of the embedded and optimized application on GPU Nvidia (Cuda), allowing real-time analysis of the direct environment on embedded boards (Tegra X1, X2). ROS (Robot Operating System) is a set of open source tools to develop software for robotics. Developed in an automotive setting, these techniques can be exploited in all areas of mobile robotics, and are particularly suited to highly dynamic and uncertain environment management (e.g. urban scenario, with pedestrians, cyclists, cars, buses, etc.).
• Authors:
  Amaury Nègre, Lukas Rummelhard, Jean-Alix David, Christian Laugier
• Contact:
  Christian Laugier
• Partners:
  CEA, CNRS

7.1.2 Ground Elevation and Occupancy Grid Estimator (GEOG - Estimator)

• Keywords:
  Robotics, Environment perception
• Functional Description:
  GEOG-Estimator is a system of joint estimation of the shape of the ground, in the form of a Bayesian network of constrained elevation nodes, and the ground-obstacle classification of a pointcloud. Starting from an unclassified 3D pointcloud, it consists of a set of expectation-maximization methods computed in parallel on the network of elevation nodes, integrating the constraints of spatial continuity as well as the influence of 3D points, classified as ground-based or obstacles. Once the ground model is generated, the system can then construct a occupation grid, taking into account the classification of 3D points, and the actual height of these impacts. Mainly used with lidars (Velodyne64, Quanergy M8, IBEO Lux), the approach can be generalized to any type of sensor providing 3D pointclouds. On the other hand, in the case of lidars, free space information between the source and the 3D point can be integrated into the construction of the grid, as well as the height at which the laser passes through the area (taking into account the height of the laser in the sensor model). The areas of application of the system spread across all areas of mobile robotics, it is particularly suitable for unknown environments. GEOG-Estimator was originally developed to allow optimal integration of 3D sensors in systems using 2D occupancy grids, taking into account the orientation of sensors, and indefinite forms of grounds. The ground model generated can be used directly, whether for mapping or as a pre-calculation step for methods of obstacle recognition or classification. Designed to be effective (real-time) in the context of embedded applications, the entire system is implemented on Nvidia graphics card (in Cuda), and optimized for Tegra X2 embedded boards. To ease interconnections with the sensor outputs and other perception modules, the system is implemented using ROS (Robot Operating System), a set of opensource tools for robotics.
• Authors:
  Amaury Nègre, Lukas Rummelhard, Lukas Rummelhard, Jean-Alix David, Christian Laugier
• Contact:
  Christian Laugier

7.1.3 Zoe Simulation

• Name:
  Simulation of INRIA's Renault Zoe in Gazebo environment
• Keyword:
  Simulation
• Functional Description:
  This simulation represents the Renault Zoe vehicle considering the realistic physical phenomena (friction, sliding, inertia, ...). The simulated vehicle embeds sensors similar to the ones of the actual vehicle. They provide measurement data under the same format. Moreover the software input/output are identical to the vehicle's. Therefore any program executed on the vehicle can be used with the simulation and reciprocally.
• Authors:
  Christian Laugier, Nicolas Turro, Thomas Genevois
• Contact:
  Christian Laugier

7.1.4 Hybrid-state E*

• Name:
  Path planning with Hybrid-state E*
• Keywords:
  Planning, Robotics
• Functional Description:

  Considering a vehicle with the kinematic constraints of a car and an environment which is represented by a probabilistic occupancy grid, this software produces a path from the initial position of the vehicle to its destination. The computed path may include, if necessary, complex maneuvers. However the suggested path is often the simpler and the shorter.

  This software is designed to take benefit from bayesian occupancy grids such as the ones computed by the CMCDOT software.

• URL:
  https://­team.­inria.­fr/­chroma/­en/
• Authors:
  Christian Laugier, Thomas Genevois
• Contact:
  Christian Laugier
• Partner:
  CEA

7.1.5 Pedsim_ros_AV

• Name:
  Pedsim_ros_AV
• Keywords:
  Simulator, Multi-agent, Crowd simulation, Autonomous Cars, Pedestrian
• Scientific Description:

  These ROS packages are useful to support robotic developments that require the simulation of pedestrians and an autonomous vehicle in various shared spaces scenarios. They allow: 1. in simulation, to pre-test autonomous vehicle navigation algorithms in various crowd scenarios, 2. in real crowds, to help online prediction of pedestrian trajectories around the autonomous vehicle.

  Individual pedestrian model in shared space (perception, distraction, personal space, pedestrians standing, trip purpose). Model of pedestrians in social groups (couples, friends, colleagues, family). Autonomous car model. Pedestrian-autonomous car interaction model. Definition of shared space scenarios: 3 environments (business zone, campus, city centre) and 8 crowd configurations.

• Functional Description:
  Simulation of pedestrians and an autonomous vehicle in various shared space scenarios. Adaptation of the original Pedsim_ros model to simulate heterogeneous crowds in shared spaces (individuals, social groups, etc.). The car model is integrated into the simulator and the interactions between pedestrians and the autonomous vehicle are modeled. The autonomous vehicle can be controlled from inside the simulation or from outside the simulator by ROS commands.
• URL:
  https://­github.­com/­maprdhm/­pedsim_ros_AV
• Publications:
  hal-02194735, hal-02514963
• Contact:
  Manon Prédhumeau
• Participants:
  Manon Prédhumeau, Anne Spalanzani, Julie Dugdale, Lyuba Mancheva
• Partner:
  LIG

7.1.6 S-NAMO-SIM

• Name:
  S-NAMO Simulator
• Keywords:
  Simulation, Navigation, Robotics, Planning
• Functional Description:
  2D Simulator of NAMO algorithms (Navigation Among Movable Obstacles) ROS compatible
• Release Contributions:
  Creation
• Author:
  Benoit Renault
• Contact:
  Benoit Renault

7.1.7 SimuDronesGR

• Name:
  Simultion of UAV fleets with Gazebo/ROS
• Keywords:
  Robotics, Simulation
• Functional Description:
  The simulator includes the following functionality : 1) Simulation of the mechanical behavior of an Unmanned Aerial Vehicle : * Modeling of the body's aerodynamics with lift, drag and moment * Modeling of rotors' aerodynamics using the forces and moments' expressions from Philppe Martin's and Erwan Salaün's 2010 IEEE Conference on Robotics and Automation paper "The True Role of Accelerometer Feedback in Quadrotor Control". 2) Gives groundtruth informations : * Positions in East-North-Up reference frame * Linear velocity in East-North-Up and Front-Left-Up reference frames * Linear acceleration in East-North-Up and Front-Left-Up reference frames * Orientation from East-North-Up reference frame to Front-Left-Up reference frame (Quaternions) * Angular velocity of Front-Left-Up reference frame expressed in Front-Left-Up reference frame. 3) Simulation of the following sensors : * Inertial Measurement Unit with 9DoF (Accelerometer + Gyroscope + Orientation) * Barometer using an ISA model for the troposphere (valid up to 11km above Mean Sea Level) * Magnetometer with the earth magnetic field declination * GPS Antenna with a geodesic map projection.
• Release Contributions:
  Initial version
• Author:
  Vincent Le Doze
• Contact:
  Vincent Le Doze
• Partner:
  Insa de Lyon

7.1.8 spank

• Name:
  Swarm Protocol And Navigation Kontrol
• Keyword:
  Protocoles
• Functional Description:
  Communication and distance measurement in an uav swarm
• URL:
  https://­gitlab.­inria.­fr/­dalu/­spank
• Author:
  Stéphane d'Alu
• Contact:
  Stéphane d'Alu

8.1.1 Extension of the CMCDOT framework to 3D Semantic Occupancy Grids

Participants: Andres Gonzalez Moreno, Lukas Rummelhard, Christian Laugier.

The CMCDOT perception framework has been reworked and extended:

• To better fuse data from different sensors, of different type and with different view points, probabilistic fusion of occupancy grids have been extended in 3 dimensions.
• To enrich perception and prediction capabilities, the whole framework now represents, fuses and filters semantic states instead of sole occupancy.
• To observe such semantic states in 3D grids, methods to generate probabilistic estimates in 3D grids from semantic estimation from neural networks in camera views, have been developed

These framework evolutions led to important software evolutions, including the development of numerous new software bricks, software factorizations and class abstractions, preparing for future upgrades. The first prototype has been developed and is being tested in simulation and with real data, with promising results.

The image shows a 2D representation of the 3D the probabilistic semantic state grid generated from the fusion of the grid generated by lidars, and the grid generated by projection of the semantic image, before temporal filtering

8.1.2 Cooperative Perception with Dynamic Occupancy Grids

Participants: Andres Gonzalez Moreno, Rabbia Asghar, Jerome Lussereau, Nicolas Turro (SED), Lukas Rummelhard, Christian Laugier.

In order to expand coverage and overall perception capabilities of an embedded perception system, data exchange between sensing agents is a valuable solution. If occupancy grids are a well-suited tool to fuse data from heterogeneous sources with different viewpoints, the mere size of these structures makes their efficient communication a real challenge. On an experimental plateform consisting of a infrastructure unit equipped with 2 lidars, a Nvidia TX2 card executing the CMCDOT framework and VTX capabilities, and a connected autonomous Zoe plateform (with its own sensors and CMCDOT framework running), preliminary work on such communication has been conducted. Two main methods of grid transmission from the infrastructure to the autonomous vehicle have been developed and tested:

• a first version which extracts mobile objects from the dynamic occupancy grid of the infrastructure, sends object information to the vehicle using VTX communication, then reconstructs the corresponding occupancy grid based on these (taking into account added uncertainties) and fuses it with local perception.
• a second version which directly compresses the infrastructure occupancy grid, sends it via UDP, and fuses it with the vehicle grid.

Raising many technical challenges, both methods provided promising results, leading to further works next year.

2 partly-superimposed occupancy grids generated by the infrastructure and the vehicle

8.1.3 Integration of environment motion prediction in the collision avoidance system

Participants: Thomas Genevois, Christian Laugier.

For several years, CHROMA has been developing navigation and collision avoidance software (DWA Planner), and applied it in various applications (ZOE navigation, ADAS emergency avoidance system, navigation of the MoovIT project). Among its latest developments, GPU optimizations allowed for collision risk estimation module improvements.

The DWA Planner samples possible trajectories over a short time horizon (approximately 2500 trajectories, each projected over a horizon of 6 seconds) and estimates the probable positions of the controlled vehicle according to a control-command uncertainty model. The occupancy grid is then read at the given positions to estimate the collision risks.

A new module has been developed to extrapolate the occupancy grid from the estimated motion of cell from the CMCDOT. Each moving occupancy particle is projected across multiple cells, representing the occupancy at the next instants. An uncertainty model in acceleration and steering is applied.

By reading the predicted occupancy grids, the DWA can anticipate the movements of other actors to plan trajectories.

Occupancy grid prediction

8.1.4 Augmented Reality tools for advanced automotive software testing

Participants: Thomas Genevois, Christian Laugier.

CHROMA has also developed an Augmented Reality framework for testing and validation of software on the Renault Zoe experimental vehicle. This framework provides a flexible way to introduce any virtual element in real time in the data of the LiDAR sensors of the vehicle. Our Augmented Reality accurately handles all possible occlusions between real and virtual elements. The representability of test scenes generated by the augmented reality framework has been experimentally proven. It is then possible to easily and safely place the whole vehicle and all its software, from perception to control, in hybrid but realistic test scenes. This new testing methodology is intended to be a bridge between Vehicle-in-the-Loop and real-world testing.

Pointcloud fusion between real and virtual sensors

8.1.5 Development of an industrial AIV, light perception and navigation systems

Participants: Thomas Genevois, Amrita Suresh, Christian Laugier.

CHROMA has been working with the company AKEOPLUS, in a new use case, in the scope of the regional R&D project Booster MoovIT, to continue the development of the embedded version of perception and navigation software. This project involves developing a versatile AIV (Autonomous Intelligent Vehicle) dedicated to the logistics of warehouses and industrial workshops. While AKEOPLUS develops the mechanics, the electronics, the basic automatisms of the robot as well as the supervision system and the user interface, CHROMA develops the functions of perception, path planning and local navigation. The jointly developed AIV should be tested in the workshop of a third-party company, Spanninga. If this specific use case implies new developments, these will also benefit the other applications of the software suite built around the CMCDOT. Such developments included:

• multi-instance CMCDOT for multimodal perception: a new grid-based data fusion method, instancing different CMCDOT for each sensor modality, then fusion the filtered grids, offering promising results
• Bayesian-Occupancy-Grid-based planning without kinematic constraints: a simplified version of the Zoe-adapted Hybrid-State-E* previously-developed method, allowing in these simplified circumstances for quicker computations. Also include added functionalities, like path-smoothing and a partial-replanning service
• Adaptation of the navigation software interface: to allow for management by a supervision system, including multiple online parameters, options, error and status messages, etc.

Planning and navigation through several rooms in simulated environment

8.1.6 Validation of AI-based algorithms in autonomous vehicles

Participants: Jean-Baptiste Horel, Alessandro Renzaglia, Anshul Paigwar, Radu Mateescu (Convecs), Christian Laugier.

In the last years, there has been an increasing demand for regulating and validating intelligent vehicle systems to ensure their correct functioning and build public trust in their use. Yet, the analysis of safety and reliability poses a significant challenge. More and more solutions for autonomous driving are based on complex AI-based algorithms whose validation is particularly challenging to achieve. An important part of our work has been recently devoted to tackle this problem, finding new suitable approaches to validate probabilistic algorithms for perception and decision making in autonomous driving, and investigating how simulations and experiments in controlled environment can help to solve this challenge. This activity, started with our participation in the European project Enable-S3 (2016-2019), is now continuing with the PRISSMA project, started in April 2021, where we participate in collaboration the the Inria Convecs team. This project, funded by the French government in the framework of the “Grand Défi: Sécuriser, certifier et fiabiliser les systèmes fondés sur l’intelligence artificielle”, regroups several companies and public institutes in France and tackles the challenge of the validation and certification of artificial intelligence based solutions for autonomous mobility. A new PhD thesis (J.-B. Horel), co-supervised between Chroma and Convecs teams, started in April 2021 in the framework of PRISSMA.

Within this context, our main work in 2021 has been focused on the automatic generation of large numbers of relevant critical scenarios for autonomous driving simulators 21. This approach is based on the generation of behavioral conformance tests from a formal model (specifying the ground truth configuration with the range of vehicle behaviors) and a test purpose (specifying the critical feature to focus on). The obtained abstract test cases cover, by construction, all possible executions exercising a given feature, and can be automatically translated into the inputs of autonomous driving simulators. We illustrate our approach by generating thousands of behavior trees for the CARLA simulator for several realistic configurations. An extension of this work, where we model check the traces of the simulation runs, combining the advantages of temporal logic verification and quantitative analyses to validate a probabilistic collision risk estimation, is currently ongoing.

8.2.1 Situation Understanding and Motion Forecasting

Participants: Özgür Erkent, David Sierra-González, Wenqian Liu, Christian Laugier.

In the context of our cooperation with Toyota Motor Europe, we study new approaches to understand the environment around an autonomous vehicle. More particularly, we focus on generating semantic grid representations of the environment by fusing information from multiple sensors (e.g. RGB cameras and LIDAR), and on tracking and forecasting the behavior of any detected dynamic obstacles.

Our main output in 2021 is a Multiple Object Detection and Tracking algorithm that works in the above-mentioned semantic grid representation of the environment 20. In this work, an object detector detects the objects 108 and a Grid Track Network (GTN) takes the information from occupancy 101 and semantic 68 grids, and combines it to track and detect the objects in bird's eye view. The results are evaluated on the Kitti MOT dataset and show comparable results with the state-of-the-art.

In terms of motion forecasting, we have developed in 2021 a novel method to forecast the lane intentions of surrounding vehicles using Graph Neural Networks. This work is currently pending for publication.

Future work will further explore deep learning sensor-fusion techniques for semantic grid generation.

8.2.2 Fusing Event-based and RGB camera for Robust Object Detection in Adverse Conditions

Participants: Abhishek Tomy, Anshul Paigwar, Khushdeep Singh Mann, Alessandro Renzaglia, Christian Laugier.

The ability to detect objects, under image corruptions and under different weather conditions is vital for deep learning models especially when applied to real-world applications such as autonomous driving. Traditional RGB-based detection fails under these conditions and it is thus important to design a sensor suite that is redundant to failures of the primary frame-based detection. Event-based cameras can complement frame-based cameras in low-light conditions and high dynamic range scenarios that an autonomous vehicle can encounter during navigation. Accordingly, we propose a redundant sensor fusion model of event-based and frame-based cameras that is robust to common image corruptions. The method utilizes a voxel grid representation for events as input and proposes a two-parallel feature extractor network for frames and events. Our sensor fusion approach is more robust to corruptions by over 30% compared to only frame-based detections and outperforms the only event-based detection. The model is trained and evaluated on the publicly released DSEC dataset.

This work has been accepted for presentation at the 2022 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, USA 29.

8.2.3 Frustum-PointPillars: A Multi-Stage Approach for 3D Object Detection using RGB Camera and LiDAR

Participants: Anshul Paigwar, David Sierra-González, Özgür Erkent, Christian Laugier.

A better understanding of the objects in 3D facilitates better decision-making and path planning. RGB Cameras and LiDAR are the most commonly used sensors in autonomous vehicles for environment perception. Many approaches have shown promising results for 2D detection with RGB Images, but efficiently localizing small objects like pedestrians in the 3D point cloud of large scenes has remained a challenging area of research. We propose a novel method, Frustum-PointPillars, for 3D object detection using LiDAR data. Instead of solely relying on point cloud features, we leverage the mature field of 2D object detection to reduce the search space in the 3D space. Then, we use the Pillar Feature Encoding network for object localization in the reduced point cloud. On the KITTI test set our method outperforms other multi-sensor SOTA approaches for 3D pedestrian localization (Bird’s Eye View) and ranked among the top 5 at the time of submission while achieving a significantly faster runtime of 14 Hz.

This work has been published and presented at the 2021 IEEE/CVF International Conference on Computer Vision (ICCVW) Workshop on Autonomous Vehicle Vision 24.

8.2.4 STADIE-Net: Stagewise Disparity Estimation from Stereo Event-based Cameras

Participants: Abhishek Tomy, Anshul Paigwar, Alessandro Renzaglia, Christian Laugier.

Event-based cameras complement the frame-based cameras in low-light conditions and high dynamic range scenarios that a robot can encounter for scene understanding and navigation. Apart from that, the comparatively cheaper cost in relation to a LiDAR sensor makes this a viable candidate when designing a sensor suite for a robot designed to operate in a dynamic environment. One of the challenges that the sensor suite needs to address is the ability to provide a 3D scene understanding of the environment that would enable the robot to localize obstacles in the environment. This work evaluates the accuracy with which an event-based camera can support this task by providing the disparity estimate between the left and right camera frames which can be utilized to calculate the depth of surroundings. A new deep network architecture, named STADIE-Net is proposed that takes advantage of stagewise refinement and prediction of disparity using events from 2 neuromorphic cameras in a stereo setup. The method utilizes voxel grid representation for events as input and proposes a 4 stage network going from coarse to finer disparity prediction. The model is trained and evaluated on the publicly released DSEC dataset that has data recorded from multiple cities using event-based and frame-based cameras mounted on a moving vehicle. Experimental results show comparable accuracy with the baseline method provided for the DSEC dataset.

This work was presented at Conference on Computer Vision and Pattern Recognition, Workshop on Event-based Vision (CVPRW), 2021, Nashville, United States 45.

8.2.5 Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning

Participants: Khushdeep Singh, Abhishek Tomy, Anshul Paigwar, Alessandro Renzaglia, Christian Laugier.

Reliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other vehicles, modeling the dynamics of vehicles subjected to different traffic conditions, and vanishing surrounding objects. To tackle these challenges, we propose a Spatio-temporal prediction network pipeline that takes the past information from the environment and semantic labels separately for generating future occupancy predictions. Compared to the current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds and in a relatively complex environment from the nuScenes dataset. Our experimental results demonstrate the ability of Spatio-temporal networks to understand scene dynamics without the need for HD-Maps and explicit modeling of dynamic objects. We publicly release our occupancy grid dataset based on nuScenes to support further research.

This work has been submitted for review at the 2022 IEEE Intelligent Vehicle Symposium (IV).

8.2.6 Panoptic Grid generation from monocular camera images

Participants: Manuel Alejandro Diaz-Zapata, Özgür Erkent, David Sierra-Gonzalez, Nicolas Turro, Lukas Rummelhard, Jilles Dibangoye, Christian Laugier.

During the year 2021, given our work on segmentation of camera images 19 we continued the work towards the panoptic segmentation of occupancy grids given camera images. As a first approach, we used information from a stereo pair to perform panoptic segmentation given the corresponding orthogonal projection of the RGB image based on the estimated depth. With the RGB information in the grid space, a modified SOLOv2 116 network was used to perform both instance and semantic segmentation on the projected images. This network was first trained on a mix of the projection of the ground truth labels available in the Cityscapes Dataset 61 and a small subset of densely annotated samples from it.

As future work, we plan to use other datasets such as the NuScenes dataset 59 and move to a sensor fusion approach where LiDAR and camera images can be used together to exploit their corresponding features for a better generation of semantic grids.

8.2.7 Real-time Collision Risk Estimation based on Stochastic Reachability Spaces

Participants: Unmesh Patil, Anshul Paigwar, Alessandro Renzaglia, Christian Laugier.

To guarantee the safety of autonomous vehicles and the robustness of driver assistance systems, it is essential to have a reliable evaluation of the risk of collision with the traffic participants. To estimate the risk, the system needs to perceive the environment, track the objects and predict their future actions. However, all the uncertainties involved in both sensing and prediction make this problem particularly challenging. In this work 25, we studied new probabilistic models to obtain stochastic reachability spaces for vehicles and pedestrians detected in the scene. We then exploit these probabilistic predictions of the road-users' future positions, along with the expected ego-vehicle trajectory, to estimate the probability of collision risk in real-time. The proposed stochastic models only depend on the velocity, acceleration, tracked bounding box, and the class of the detected object. This information can easily be obtained through off-the-shelf 3D object detection frameworks. As a result, the proposed approach for collision risk estimation is widely applicable to a variety of autonomous vehicle platforms. To validate our approach, we tested the stochastic motion prediction on the KITTI dataset. Further experiments were also carried out in the CARLA simulator, by reproducing realistic collision scenarios, with the goal of demonstrating the effectiveness of our risk prediction.

8.4.1 Modelling crowds and autonomous vehicles using Extended Social Force Models

Participants: Manon Predhumeau (LIG Grenoble & Chroma), Anne Spalanzani, Julie Dugdale (LIG Grenoble).

The focus of this work has been on the realistic simulation of crowds in shared spaces with an autonomous vehicle (AV), see Fig. 8. We proposed an agent-based model for pedestrian reactions to an AV in a shared space, based on empirical studies and the state of the art. The model includes an AV with Renault Zoé car’s characteristics, and pedestrians’ reactions to the vehicle. The model extends the Social Force Model with a new decision model, which integrates various observed reactions of pedestrians and pedestrians groups. We performed a qualitative and quantitative evaluation of the proposed model, through comparisons of the simulated trajectories with ground truth trajectories. The proposed model reproduces well the various pedestrians behaviors: running to cross, stopping to wait without deviating, staying in social group to avoid the AV. The model can be used for predicting pedestrians’ displacement, linear velocity, orientation, and approach distance around the AV in fundamental interactions (AAMAS'21 27).

We implemented this model using Pedsim_ros and proposed an open source simulation tool to help the robotic community with AV developments. The simulator integrates various observed pedestrians behaviours, with individual pedestrians as well as pedestrian groups with four different types of relationships (SCS'20 96). Complex shared spaces situations have been modeled, which require the interaction of a large number of pedestrians responding to the AV movement. The simulated AV can be controlled by an external ROS module, so that AV navigation algorithms can be tested in various realistic, crowded shared space environments, before real-world experiments. Moreover, simulations can reproduce real world scenes and can run faster than real-time to help live prediction of pedestrian trajectories around the AV.

A scene of an Autonomous Vehicle with pedestrians

8.4.2 Proactive Navigation for navigating dense human populated environments

Participants: Maria Kabtoul, Anne Spalanzani, Philippe Martinet [Chorale, Inria Sophia Antipolis].

Developing autonomous vehicles capable of navigating safely and socially around pedestrians is a major challenge in intelligent transportation. This challenge cannot be met without understanding pedestrians’ behavioral response to an autonomous vehicle, and the task of building a clear and quantitative description of the pedestrian to vehicle interaction remains a key milestone in autonomous navigation research. As a step towards safe proactive navigation in a space shared with pedestrians, we started to introduce in 2018 a pedestrian-vehicle interaction behavioral model. The model estimates the pedestrian’s cooperation with the vehicle in an interaction scenario by a quantitative time-varying function. Using this cooperation estimation the pedestrian’s trajectory is predicted by a cooperation-based trajectory planning model (see Fig. 9.a). Both parts of the model are tested and validated using real-life recorded scenarios of pedestrian-vehicle interaction. The model is capable of describing and predicting agents’ behaviors when interacting with a vehicle in both lateral and frontal crossing scenarios. In early 2019, we used this cooperation based behavioral model to design a proactive longitudinal velocity controller. The suggested velocity control strategy enabled the vehicle to cooperate and proactively interact with its surrounding rather than merely reacting to the behavior of the pedestrians. This resulted in more natural driving patterns, avoiding the sub-optimal navigation solutions which result in over-reactive methods. The method is tested and validated in lateral interactions with a flow of pedestrians. The method showed a high gain in the time to reach the goal in comparison with reactive navigation, while ensuring the pedestrians safety. In 2020, we integrated the previous navigation system with the PedSim simulator under ROS. Next, we worked on a proactive and human-like maneuvering system to deploy the steering control part of the navigation system. We adopted a two-step approach. In the first step, the navigation space is explored proactively and an optimal navigation channel is found using a fuzzy logic model. This proactive dynamic channel is guaranteed to exist even in dense scenarios where obstacle free paths do not exist. In the second step, a human-like steering is adapted using a Quintic transition path and a sliding mode control approach. The resulting system is tested under ROS in combination with the longitudinal velocity controller and the cooperation-based behavioral model. The resulting system is validated in both frontal and lateral interactions with dense crowds (see Fig. 9.b). The system results in major advantage in the travel time avoiding the freezing of the vehicle, while ensuring the pedestrian’s safety. This work has been published in ITSC'20 73, ICRA'20 74 and ICRA'22 22

Proactive navigation through crowds

8.5.1 Deep-Learning for task-based motion planning

Participants: Edward Beeching, Pierre Marza (PhD. student LIRIS), Christian Wolf (LIRIS/INSA Lyon), Jilles Dibangoye, Olivier Simonin, Laetitia Matignon (LIRIS/Lyon 1).

RGB observations and the attention heat map

Our goal is the automatic learning of robot navigation in complex environments based on specific tasks and from visual input. The robot automatically navigates in the environment in order to solve a specific problem, which can be posed explicitly and be encoded in the algorithm (e.g. find all occurences of a given object in the environment, or recognize the current activities of all the actors in this environment) or which can be given in an encoded form as additional input, like text. Addressing these problems requires competences in computer vision, machine learning and AI, and robotics (navigation and paths planning).

A critical part for solving these kind of problems involving autonomous agents is handling memory and planning. Many control problems in partially observed 3D environments involve long term dependencies and planning. Solving these problems requires agents to learn several key capacities: spatial reasoning (exploring the environment and learning spatio-temporal regularities) and semantic mapping (discovering semantics from interactions). While solutions exist for semantic mapping and semantic SLAM 60, 110, a more interesting problem arises when the semantics of objects and their affordances are not supervised, but defined through the task and thus learned from reward.

PhD thesis of Edward Beeching (since 2018, Inria grant): After showing that through visualizations an agent can learn to map relevant objects in its spatial memory without any supervision purely from reward (see Fig. 10.a and 53, 52, 51) we proposed a 3D training framework. E. Beeching developed the Godot RL Agents, which is an interface between DeepRL algorithms and the Godot Game Engine (Linietsky 2014). This new framework allows to create 2D & 3D games to learn behaviors with automated testing and multi-process parallel training (see Fig. 10.b). In the fall of 2021, E. Beeching wrote his doctoral thesis, which he will defend on May 3, 2022.

PhD thesis of Pierre Marza (since Dec. 2020). In the context of the REMEMBER AI Chair, P. Marza started a PhD on the subject "Large-scale learning of high-level navigation skills for autonomous agents in 3D environments" (supervised by C. Wolf, L. Matignon and O. Simonin). We propose to extend the advances to the sequential decision-making of agents in a context of planning and control in complex 3D environments. A key question is the ability to learn representations with minimal human intervention and annotation. From the Egocentric Map approach, we introduced supplementary supervision in the form of auxiliary tasks designed to favor the emergence of spatial perception capabilities in agents trained for a goal-reaching downstream objective. This allowed us to win the MultiON Challenge at CVPR2021 Embodied AI Workshop. A paper presenting our architecture with the auxiliary tasks is under review.

8.5.2 Navigation Among Movable Obstacles: extension to multi-robot and social constraints

Participants: Benoit Renault, Olivier Simonin, Jacques Saraydaryan, Fabrice Jumel.

S-NAMO paths and Pepper robot

In 2018 we started to study the NAMO planning problem (Navigation Among Movable Obstacles). Since 2019, the PhD thesis of Benoit Renault aims to extend NAMO in two directions:

• Defining Social-NAMO, by considering where obstacles should be optimally moved with regards to space access. We have defined new spatial cost functions and extended existing NAMO algorithms with the ability to maintain area accesses (connectivity) for humans and robots, see IROS publication 97. We also developed a simulator for NAMO, and its extensions, called S-NAMO-SIM (see Fig. 11.a).
• Defining Multi-robot NAMO (MR-NAMO). It consists in generalizing the problem to $n$ robots that can move the objects of the environment in order to reach their individual goal positions. We have proposed a first coordination strategy based on local rules. We have shown that considering the social cost we introduced with S-NAMO offers a significant gain for solving MR-NAMO because of the rearranging it provides (see Fig. 11.b).

In 2021, a particular effort has been made to create varied and complex multi-robot NAMO scenarios. This allowed to extend the evaluation of the proposed approaches. In addition, we identifed the different deadlocks that can occurs between robots. We then proposed some local rules (heuristics) allowing to solve certain deadlocks, by exploiting the social cost map computed in S-NAMO (a conference paper is in preparation). Since the end of 2021, B. Renault is writting his doctoral thesis .

8.6.1 Multi-UAV Exploration and Visual Coverage of 3D Environments

Participants: Alessandro Renzaglia, Olivier Simonin, Jilles Dibangoye, Vincent Dufour.

Multi-robot teams, especially when involving aerial vehicles (UAVs 20), are extremely efficient systems to help humans in acquiring information on large and complex environments. In these scenarios, cooperative coverage and exploration are two fundamental problems with numerous important applications, such as mapping, inspection of complex structures, and search and rescue missions. In both cases, the robots' goal is to navigate through the environment and cooperate to maximize the observed area, either by finding optimal static configurations that provides the best global view in the case of the coverage or by maximizing the new observed areas at every step until the environment becomes completely known in the case of the exploration.

Following our recent results on these problems 100, 99, in the last year we focused on a strictly related path planning problem where a fleet of UAVs must use both prior information and online gathered data to efficiently inspect large surfaces such as ship hulls and water tanks (work submitted to IROS22, currently under review.). This work is conducted in the framework of the H2020 European project BugWright2, where one central use-case is focused on corrosion detection and inspection on cargo ships. In our case, UAVs can detect corrosion patches and other defects on the surface from low-resolution images. If defects are detected, they get closer to the surface for a high-resolution inspection. The prior information provides expected defects locations and is affected by both false positives and false negatives. The mission objective is to prioritize the close-up inspection of defected areas while keeping a reasonable time for the coverage of the entire surface. In this work, we propose two solutions to this problem: a coverage algorithm that divides the problem into a set of Traveling Salesman Problems (Part-TSP) and a cooperative frontier approach that introduces frontier utilities to incorporate the prior information (Coop-Frontier). We extensively tested our solutions in simulation and compare them with alternative solutions. Additional work on this subject, which involves a Master internship currently ongoing, aims at leveraging the probabilistic prior information to optimally partition the environment and equally distribute the expected inspection time among the robots.

8.6.2 Controlled mobility & communication optimization in swarm of UAVs

Participants: Remy Grunblatt [PhD. student LIP & CITI], Isabelle Guerin-Lassous [Inria/Lyon 1 Dante team], Olivier Simonin.

At the same time Remy Grunblatt defended his PhD on beginning of 2021 38, 13, we started the CONCERTO ANR ("mobilité Contrôlée et cOmmunicatioNs effiCaces dans une flottE de dRones auTo-Organisée"). The first year was dedicated to build a state of the art on controlled mobility of swarm of drones, to discover partner experimental platforms, to select a common UAV platform (the PX4 Vision) and to define a common scenario/model to study and experiment. We chosen to study a leader-follower flocking model (based on the Olfati-Saber model) that we aim at measuring and controling the communication flows in outdoor deployment. As a first stage, we prepared indoor experiments in Nantes LS2N platform.

8.6.3 Decentralized control of swarm of UAVs: Flocking models

Participants: Alexandre Bonnefond [Phd. Inria/CITI],, Isabelle Guerrin-Lassous [LIP/Inria/Dante Lyon 1],, Olivier Simonin, Vincent Dufour.

Decreasing funciton of the received power, simulation of the Olfati-Saber flocking

This work is conduced through the Inria/DGA "AI" project "DYNAFLOCK" (2019-22). It funds the PhD thesis of Alexandre Bonnefond.

In this work, we study existing flocking models (decentralized swarm navigation) and propose extensions to improve their abilities to deal with environments having obstacles impacting the communication quality. Often depicted as robust systems, there is yet a lack of understanding how flocking models compare and how they are impacted by the communication quality when they exchange control data. In 2020, we first modeled the obstacles’ impact on communications in a simulator that we use to optimize flocking parameters (see. Fig. 12 a,b). Second, we extended and combined two standard models, Olfati-Saber 90 and Vásárhelyi et al. 112 (see Fig. 12 b,c), to improve their ability to stay connected while evolving in environments with different obstacles distributions. This work has been published in IROS 2021 17 and JFSMA 2021 31 conferences.

In 2021, the work continued in two directions : 1) by simulating more accuratly these models in the Gazebo 3D simulator, in order to prepare real experiments with the PX4 Vision platfom, 2) by modeling the agent's pressure inside the flock. From this new individual measure we derived an asymetric repulsion law, which allows to reduce agent collisions and the fragmentation of the flock. A publication is in preparation.

8.7.1 Optimally solving zero-sum games using centralized planning for decentralized control theory

Participants: Aurelien Delage [PhD Student], Jilles S. Dibangoye, Olivier Buffet [Inria Nancy, Larsen], Vincent Thomas [Inria Nancy, Larsen], Abdallah Saffidine [Univ. New South Whales], Christopher Amato [Univ. New Hampshire], François Charpillet [Inria Nancy, Larsen].

During the last two years, we investigated deep and standard reinforcement learning for solving systems with multiple agents and different information structures. Our preliminary results include:

1. (Theoretical) – As an extension of 63 in the competitive cases, we characterize the optimal solution of two-player fully and partially observable stochastic games.
2. (Theoretical) – We further exhibit new underlying structures of the optimal solution for both non-cooperative two-player settings with information asymmetry, one agent sees what the other does and sees.
3. (Algorithmic) – We extend a non-trivial procedure for computing such optimal solutions.

This work aims at reinforcing a recent theory and algorithms to optimally solving a two-person zero-sum POSGs (zs-POSGs). That is, a general framework for modeling and solving two-person zero-sum games (zs-Games) with imperfect information. Our theory builds upon a proof that the original problem is reducible to a zs-Game—but now with perfect information. In this form, we show that the dynamic programming theory applies. In particular, we extended Bellman equations 54 for zs-POSGs, and coined them maximin (resp. minimax) equations. Even more importantly, we demonstrated Von Neumann & Morgenstern’s minimax theorem 115114 holds in zs-POSGs. We further proved that value functions—solutions of maximin (resp. minimax) equations—yield special structures. More specifically, the optimal value functions are Lipchitz-continuous. Together these findings allow us to extend planning techniques from simpler settings to zs-POSGs. To cope with high-dimensional settings, we also investigated low-dimensional (possibly non-convex) representations of the approximations of the optimal value function. In that direction, we extended algorithms that apply for convex value functions to lipschitz value functions. This research direction lead to many publications in different avenues 58 along with technical reports 12.

PhD thesis of Aurelien Delage (since Sept. 2020, ANR JCJC PLASMA). This thesis aims at extending ideas conducted on cooperative multi-agent partially observable systems to pure competitive ones.

8.7.2 HSVI fo zs-POSGs using Concavity, Convexity and Lipschitz Properties

Participants: Delage Aurelien [PhD Student], Jilles S. Dibangoye, Olivier Buffet [Inria Nancy, Larsen].

Dynamic programming and heuristic search are at the core of state-of-the-art solvers for sequential decision-making problems. In partially observable or collaborative settings (e.g., POMDPs and Dec-POMDPs), this requires introducing an appropriate statistic that induces a fully observable problem as well as bounding (convex) approximators of the optimal value function. This approach has succeeded in some subclasses of 2-player zero-sum partially observable stochastic games (zs-POSGs) as well, but failed in the general case despite known concavity and convexity properties, which only led to heuristic algorithms with poor convergence guarantees. We overcome this issue, leveraging on these properties to derive bounding approximators and efficient update and selection operators, before deriving a prototypical solver inspired by HSVI that provably converges to an $ϵ$-optimal solution in finite time, and which we empirically evaluate. This opens the door to a novel family of promising approaches complementing those relying on linear programming or iterative methods 43.

8.7.3 Optimally Solving Decentralized POMDPs Under Hierarchical Information Sharing

Participants: Sayil Barish [MS Intern], Albert David [BS Intern], Xie Yuxuan [BS Intern], Olivier Buffet [Inria Nancy, Larsen], Jilles S. Dibangoye.

This work is the result of an effort to provide a strong theory for optimally solving decentralized decision-making problems, represented as decentralized partially observable Markov decision processes (Dec-POMDPs), with tractable structures 117. This work describes core ideas of multi-agent reinforcement learning algorithms for decentralized partially observable Markov decision processes (Dec-POMDPs) under hierarchical information sharing. That is the dominant management style in our society—each agent knows what her immediate subordinate knows. In this case, we prove backups to be linear due to (normal-form) Bellman optimality equations being for the first time specified in extensive form. Doing so, one can sequence agent decisions at every point in time while still preserving optimality; hence paving the way for the first optimal $Q$-learning algorithm with linear-time updates for an important subclass of Dec-POMDPs. That is a significant improvement over the double-exponential time required for normal-form Bellman optimality equations, in which updates of agent decisions are performed in sync every point in time. Experiments show algorithms utilizing extensive instead of normal-form Bellman optimality equations can be applied effectively to much larger problems.

8.7.4 Stable Reinforcement Learning Methods

Participants: Jilles S. Dibangoye, Nicolas Marchand [GIPSA lab, Grenoble], Ahmad Hably [GIPSA lab, Grenoble], Estéban Carvalho [PhD Student, GIPSA lab, Citi lab], Vincent Andrieu [LAGEPP, Lyon], Daniele Astolfi [LAGEPP, Lyon], Giacomo Casadeic [LAGEPP, Lyon], Madiha Nadri [LAGEPP, Lyon], Samuele Zoboli [PhD Student, LAGEPP, Citi lab].

This recent research project is at the intersection of three closely related research fields: optimal control, machine learning, and mobile robotics. In this project, we try to find the best compromise between control methodologies to provide stability guarantees and machine learning methodologies that can scale up and converge towards a locally optimal solution. Applications of this hybrid methodology concern mobile robotics, including the driving of drones at very high speed. Two PhD. theses have been started on this problem the last year.

PhD thesis of Estéban Carvalho (since Oct. 2019). The first thesis focuses on high-speed navigation for UAVs in collaboration with the laboratory GIPSA. The goal of this PhD is to develop safe and aggressive piloting algorithms for UAVs. Piloting algorithms will be designed on event-switched control coupled with AI routines to cope with unmodeled and uncertain part of the system to reach as much as possible the limits of aggressive flight.

PhD thesis of Samuele Zoboli (since Jul. 2020, ANR DELICIO). In control theory, a natural approach for the design of stabilizing control is to consider a linear approximation around the operating point of the dynamic system. In this way, it is possible to design control laws that are optimal and which guaranteed stability for the linearized system. However, this behavior is only guaranteed for the original system around this operating point (local approach). The aim of this PhD thesis is to develop new real time algorithms (contol law and estimation algorithms for dynamical systems) that combines tools coming from control theory and reinforcement learning.

8.7.5 Neural Enhanced Control for Quadrotor Linear Behavior Fitting

Participants: Estéban Carvalho [PhD Student, GIPSA lab, Citi lab], Pierre Susbielle [GIPSA lab, Grenoble], Ahmad Hably [GIPSA lab, Grenoble], Jilles S. Dibangoye, Nicolas Marchand [GIPSA lab, Grenoble].

Designing an efficient autopilot for a quadrotor can be a very long and tedious process. This comes from the complex nonlinear dynamics that rule the flying robot behavior as battery discharge, blade flapping, gyroscopic effect, frictions, etc. In this paper we proposed to use a traditional cascaded control architecture enhanced with Deep Neural Network (DNN). The idea is to easily setup a control algorithm using linear cascaded laws and then correct unmodelled dynamics and approximations made during the linear control design with the DNN. The tuning process is reduced to choice of proportional and derivative gains of each control loop. The approach is tested in the ROS/Gazebo simulation environment and experimentally in a motion capture room. Results confirm that the methodology significantly improves the performance of linear approaches on nonlinear quadrotor system.

8.7.6 Neural Network Assisted Feedback Linearization Control for Quadrotor Flight with Unknown Suspended Load

Participants: Estéban Carvalho [PhD Student, GIPSA lab, Citi lab], Pierre Susbielle [GIPSA lab, Grenoble], Ahmad Hably [GIPSA lab, Grenoble], Nicolas Marchand [GIPSA lab, Grenoble], Jilles S. Dibangoye.

Load transportation problem for quadrotor is a very emerging application. The presence of a suspended load is known to generate problematic disturbances. Knowledge of the latter is often used for the design of the controller and is often based on simplifying assumptions. In order to follow a given trajectory while minimizing the perturbations of an unknown load we propose a Deep Neural Network (DNN) Assisted Feedback Linearization Controller. The DNN is used to learn load dynamics using flight data. Online estimation of these perturbation dynamics is then used to perform exact feedback linearization. The proposed approach is tested experimentally and demonstrates the effectiveness of this hybrid control architecture.

8.7.7 Locally Asymptotically Stable Deep Actor-Critic Algorithms For Quadratic Cost Setpoint Optimal Control Problems

Participants: Samuele Zoboli [PhD Student, LAGEPP, Citi lab], Vincent Andrieu [LAGEPP, Lyon], Daniele Astolfi [LAGEPP, Lyon], Giacomo Casadeic [LAGEPP, Lyon], Madiha Nadri [LAGEPP, Lyon], Jilles S. Dibangoye.

Deep reinforcement learning (DRL) algorithms showed surprising capabilities in solving complex nonlinear control problems. However, these capabilities come at a price. For instance, they construct controllers with only practical stability properties. On the contrary, Lyapunov-based controllers provide strong stability guarantees by exploiting the knowledge about the dynamic model. A recent work proved DRL algorithms, enhanced with such dynamic model information, locally guarantee desirable properties. The present work circumvents the requirement of local dynamic model's knowledge by learning it from data. The resulting DRL algorithms achieve the best of both worlds: local strong stability properties and improved domain of attraction compared to classical Lyapunov-based methods. Experiments over three benchmarks support our claims.

8.7.8 Reinforcement Learning Policies With Local LQR Guarantees For Nonlinear Discrete-Time Systems

Participants: Samuele Zoboli [PhD Student, LAGEPP, Citi lab], Vincent Andrieu [LAGEPP, Lyon], Daniele Astolfi [LAGEPP, Lyon], Giacomo Casadeic [LAGEPP, Lyon], Madiha Nadri [LAGEPP, Lyon], Jilles S. Dibangoye.

Optimal control of nonlinear systems is a difficult problem which has been addressed by both the Control Theory (CT) and Reinforcement Learning (RL) communities. Frequently, the former relies on the linearization of the system thus obtaining only local guarantees. The latter relies on data to build model-free controllers, focused solely on performances. In this paper we proposed a methodology to combine the advantages of both approaches, casting the formulation of an optimal local Linear Quadratic Regulator (LQR) into a Deep RL problem. Our solution builds on the linear framework to derive a learnt nonlinear controller showing local stability properties and global performances.

8.8.1 Constraint Programming

Participants: Jean-Yves Courtonne [Inria STEEP], François Delobel [LIMOS], Léon Fauste [LIMOS], Pascal Lafourcade [LIMOS], Luc Libralesso [Inria STEEP], Mathieu Mangeot [Inria STEEP], Loïc Rouquette, Christine Solnon.

In many of our applications, we have to solve constrained optimization problems (COPs) such as vehicle routing problems, or multi-agent path finding problems, for example. These COPs are usually NP-hard, and we study and design algorithms for solving these problems. We more particularly study approaches based on Constraint Programming (CP), that provides declarative languages for describing these problems by means of constraints, and generic algorithms for solving them by means of constraint propagation. In 23, we show how to automatically generate efficient CP models for solving differential cryptanalysis problems given the description of a symmetric block cipher.

PhD thesis of Loïc Rouquette (since Oct. 2019). This thesis is funded by the ANR project DeCrypt, that aims at designing declarative languages (based on CP) for solving cryptanalysis problems. During the first year of his thesis, Loïc has designed new propagation algorithms that are integrated in the Choco CP library. During the second year, he has designed CP models for mounting related-key differential attacks of Rijndael (paper submitted to AfricaCrypt) and Boomerang attacks of WARP (paper submitted to FSE).

PhD thesis of Léon Fauste (since Sept. 2021). This thesis is funded by Ecole Normale Supérieure Paris Saclay and it is co-supervised by Mathieu Mangeot, Jean-Yves Courtonne (from Inria STEEP team) and Christine Solnon. The goal of the thesis is to design new tools for choosing the best geographical scale when relocating productive activities, and the context of this work is described in 32. During the first months of his thesis, Léon has designed a first Integer Linear Programming (ILP) model described in 35.

8.8.2 Multi-Robot Path Planning with cable constraints

Participants: Xiao Peng [PhD. student CITI], Christine Solnon, Olivier Simonin.

The research is done within the context of the european project BugWright2 that aims at designing an adaptable autonomous multi-robot solution for servicing ship outer hulls.

PhD thesis of Xiao Peng (since Nov. 2020). The goal of this PhD thesis is to design algorithms for planning a set of mobile robots, each attached to a flexible cable, to cover an area with obstacles. In 26, we formally define the Non-Crossing MAPF (NC-MAPF) problem and show how to compute lower and upper bounds for this problem by solving well known assignment problems, we introduce a Variable Neighbourhood Search approach for improving the upper bound, and we introduce a CP model for solving the problem to optimality. In 91, we extend this approach to the case of non point-sized robots.

8.8.3 Vehicle routing problems

Participants: Romain Fontaine [PhD Student CITI], Thierry Garaix [LIMOS, Saint Etienne], Omar Rifki [LIMOS, Saint Etienne], Jilles S. Dibangoye, Christine Solnon.

The research is done within the context of the transportation challenge and funded by INSA Lyon.

9.1.1 Toyota Motor Europe (2006 - 2022)

Participants: Christian Laugier, David Sierra González, Özgür Erkent, Jilles Dibangoye, Christian Wolf.

This collaboration has been extended in 2018 for 4 years (period 2018-2021) and Toyota provides us with an experimental vehicle Lexus equipped with various sensing and control capabilities. Several additional connected technical contracts have also been signed, and an exploitation license for the CMCDOT software has been bought by Toyota in 2018.

9.1.2 Sumitomo Electric Industries (2020-22)

Participants: Christian Laugier, Lukas Rummelhard (SED), Andres Gonzalez Moreno, Rabbia Asghar, Jerome Lussereau, Nicolas Turro (SED).

9.2.1 Renault (2019 - 2022)

Participants: Luiz Alberto Guardini Serafim, Christian Laugier, Anne Spalanzani, Philippe Martinet.

9.2.2 IRT Nanoelec – Security of Autonomous Vehicles project (2018 - 2020), Security of Industrial Robotics (2021-2023)

Participants: Christian Laugier, Lukas Rummelhard (SED), Jerome Lussereau, Andres Gonzalez Moreno, Thomas Genevois, Nicolas Turro (SED).

10.1.1 FP7 & H2020 projects

10.2.1 ANR

10.2.2 Grand Defis

10.2.3 FUI Projects

10.2.4 DGA/INRIA AI projects

WIFI-Drones project, FIL (Federation d'Informatique de Lyon), 2019-21

Participants: Isabelle Guerin-Lassous (Lip/Lyon1, Inria Dante), Olivier Simonin, Remy Grunblatt (Telecom Paris Sud).

Project between two teams of two laboratories from Lyon : DANTE (LIP) and CHROMA (CITI), entitled "Performances des communications Wi-Fi dans les réseaux de drones : une approche expérimentale", 2019-2021. Leader : I. Guerin-Lassous & O. Simonin.

The project aims to experimentally evaluate the Wireless communication in UAVs fleet scenarios. We consider the recent version of Wi-Fi based on 802.11n and 802.11 ac. Experimental measures will be used to build propagation models in order to be integrated in UAVs fleet simulations (in particular with Gazebo and NS3 simulators).

Region-funded R&D Booster "Moov-IT" project, 2020-2023

Participants: Amrita Suresh, Thomas Genevois, Christian Laugier.

The objective of the project is to develop an autonomous system to be integrated in production chains to provide modernization, flexibility and productivity, including fixed and mobile transitic systems. In that perspective, Chroma's CMCDOT framework will be adapted, extended and integrated on a mobile robot from Akeoplus, and tested in real environments.

11.1.1 Scientific events: organisation

11.1.2 Scientific events: selection

11.1.3 Journal

11.1.4 Invited talks

• C. Solnon gave an invited talk at 23ème édition du congrès annuel de la Société Française de Recherche Opérationnelle et d'Aide à la Décision (ROADEF)

11.1.5 Leadership within the scientific community

• C. Laugier is regularly involved in various IEEE Awards Committees and evaluation committees: Best conference paper (ICRA, IROS, ICARCV), IEEE RAS Awards (DSA, chapter award), IROS awards (Fellow, Harashima, DSA, Young Professional), European projects applications.
• C. Laugier is member of the scientific committee of the French GDR Robotics.
• C. Solnon is member of the executive committee and the scientific committee of the AI GDR (national research group).

11.1.6 Scientific expertise

• O. Simonin was a member of the committee of the Robotics ASTRID ANR project selection in 2021.
• C. Solnon was member of the Hcéres evaluation committee of the GREYC (Caen), 2021.
• C. Solnon was expert for the Automn University of the A2U Alliance (alliance between "Université du Littoral et Cote d'Opale", "Université d’Artois" and "Université de Picardie Jules Verne")
• C. Solnon was member of the scientific committee of the "Prix de thèse du GT2L", 2021

11.1.7 Research administration

• O. Simonin is member of the Inria Lyon COS (comité d'orientation scientifique du Centre).

11.2.1 Supervision

Phd in progress :

• Benoit Renault, Navigation coopérative et sociale de robots mobiles en environnement modifiable, O. Simonin and J. Saraydaryan.
• Edward Beeching, Large-scale automatic learning of autonomous agent behavior with structured deep reinforcement learning, C. Wolf, O. Simonin and J. Dibangoye.
• Luiz Serafim-Guardini, Conduite Automobile Autonome : Utilisation de grilles d'occupation probabilistes dynamiques pour la planification contextualisée de trajectoire d'urgence à criticité minimale, A. Spalanzani, C. Laugier, P. Martinet (Inria acentauri).
• Alexandre Bonnefond, Large-scale automatic learning of autonomous agent behavior with structured deep reinforcement learning, O. Simonin and I. Guerrin-Lassous (Inria Dante).
• Mihai Popescu, Planification multi-agents pour la patrouille d'un réseau de cibles découvertes dynamiquement, O. Simonin, A. Spalanzani, F. Valois (CITI/Inria Agora) (on break in 2020).
• Estéban Carvalho, Safe and aggressive piloting of UAVs, 2019, Ahmad Hably (Gipsa-Lab), Nicolas Marchand (Gipsa-Lab), Jilles S. Dibangoye.
• Xiao Peng, Planification sous contraintes d’une flotte de robots mobiles pour des tâches de couverture spatiale, O. Simonin, C. Solnon, European H2020 BugWright2 project funding.
• Pierre Marza, Apprentissage large-échelle d'agents autonomes capables de naviguer dans des environnements réels, C. Wolf, L. Matignon, O. Simonin, funded by Chaire IA "REMEMBER".
• Loic Rouquette, Programmation par contraintes pour la cryptanalyse symétrique, M. Minier (LORIA) and C. Solnon
• Romain Fontaine, Optimisation de la mobilité en ville en tenant compte des conditions de traffic, J. Dibangoye and C. Solnon
• Léon Fauste, Aide à la décision pour le choix de l'échelle lors de la relocalisation d'activités industrielles, M. Mangeot (STEEP, Inria Montbonnot), J.-Y. Courtonne (STEEP, Inria Montbonnot), and C. Solnon

11.2.2 Juries

PhD thesis :

• O. Simonin was reviewer and member of the defense committee of the PhD thesis of
  • Bruno Dato, IRIT Univ. Toulouse 3, July 9th, 2021.
  • Cristino de Souza Junior, Heudiasyc UTC, April 19th, 2021.
  • Rémi Dromnelle, ISIR Sorbonne Univ., July 8th, 2021.
  • Guilhem Buisan, LAAS INSA Toulouse, July 6th, 2021.
• C. Solnon was president of the defense committee of the PhD thesis of Clément Hénin, INSA Lyon, October, 2021.
• C. Solnon was external reviewer of the defense committee of the PhD thesis of
  • Arthur Godet, Ecole des Mines de Nantes, September, 2021;
  • Matthieu Py, Université Aix Marseille, December, 2021;
  • Gustav Bjordal, Uppsala University (Sweeden), April, 2021;
  • Lucas Groleaz, INSA Lyon, June, 2021.
• A. Spaalanzani was reviewer and member of the defense committee of the PhD thesis of
  • Fabien Grzeskowiak, Univ. Rennes 1, June 2021.
  • Guillaume Vailland, Univ. Rennes 1, Décember 2021.

11.3.1 Articles and contents

• C. Laugier participated as a speaker in two round tables broadcast on the internet, to popularize the issues and approaches on which the concept of “Autonomous Driving” is based. These round tables were organized by IRT (Institute for Technological Research) Nanoelec and the Minalogic competitiveness cluster (respectively in Sept. 2020 and Dec. 2021).
• C. Solnon has written an article on “Where are the hard problems?” for the Tangente journal 15, 2021

International journals

• 11 articleG.Guillaume Bono, J.Jilles Dibangoye, O.Olivier Simonin, L.Laëtitia Matignon and F.Florian Pereyron. Solving Multi-Agent Routing Problems Using Deep Attention Mechanisms.IEEE Transactions on Intelligent Transportation SystemsJuly 2021, 1-10
  HALDOI
• 12 articleO.Olivier Buffet, J.Jilles Dibangoye, A.Abdallah Saffidine and V.Vincent Thomas. Heuristic Search Value Iteration for zero-sum Stochastic Games.IEEE Transactions on Games1332021, 1-10
  HALDOIback to textback to text
• 13 articleR.Rémy Grünblatt, I.Isabelle Guérin-Lassous and O.Olivier Simonin. A distributed antenna orientation solution for optimizing communications in a fleet of UAVs.Computer Communications181January 2022, 102-115
  HALDOIback to textback to text
• 14 articleM.Manon Prédhumeau, A.Anne Spalanzani and J.Julie Dugdale. Pedestrian Behavior in Shared Spaces with Autonomous Vehicles: An Integrated Framework and Review.IEEE Transactions on Intelligent Vehicles2021, 1-21
  HALDOIback to text

National journals

• 15 articleC.Christine Solnon. Où sont les problèmes difficiles.Bibliothèque Tangente752021, 50-53
  HALback to text

International peer-reviewed conferences

• 16 inproceedingsB.Benedikt Bollig, C.Cinzia Di Giusto, A.Alain Finkel, L.Laetitia Laversa, E.Etienne Lozes and A.Amrita Suresh. A Unifying Framework for Deciding Synchronizability.CONCUR 2021 - 32nd International Conference on Concurrency TheoryParis, FranceAugust 2021, 1-33
  HAL
• 17 inproceedingsA.Alexandre Bonnefond, O.Olivier Simonin and I.Isabelle Guérin-Lassous. Extension of Flocking Models to Environments with Obstacles and Degraded Communications.IROS 2021 - IEEE/RSJ International Conference on Intelligent Robots and SystemsProceedings IROS 2021Prague / Virtual, Czech RepublicIEEEJuly 2021, 9139-9145
  HALDOIback to textback to text
• 18 inproceedingsN.Niranjan Deshpande, D.Dominique Vaufreydaz and A.Anne Spalanzani. Navigation In Urban Environments Amongst Pedestrians Using Multi-Objective Deep Reinforcement Learning.ITSC 2021 - 24th IEEE International Conference on Intelligent Transportation SystemsIndianapolis, United StatesSeptember 2021, 1-7
  HALDOI
• 19 inproceedingsM. A.Manuel Alejandro Diaz Zapata, Ö.Özgür Erkent and C.Christian Laugier. YOLO-based Panoptic Segmentation Network.COMPSAC 2021 - Intelligent and Resilient Computing for a Collaborative World 45th Anniversary ConferenceMadrid, SpainJuly 2021, 1-5
  HALDOIback to text
• 20 inproceedingsÖ.Özgür Erkent, D.David Sierra Gonzalez, A.Anshul Paigwar and C.Christian Laugier. GridTrack: Detection and Tracking of Multiple Objects in Dynamic Occupancy Grids.ICVS - International Conference on Vision SystemsVirtual Conference, AustriaOctober 2021, 1-14
  HALDOIback to textback to text
• 21 inproceedingsJ.-B.Jean-Baptiste Horel, C.Christian Laugier, L.Lina Marsso, R.Radu Mateescu, L.Lucie Muller, A.Anshul Paigwar, A.Alessandro Renzaglia and W.Wendelin Serwe. Using Formal Conformance Testing to Generate Scenarios for Autonomous Vehicles.DATE/ASD 2022 - Design, Automation and Test in Europe - Autonomous Systems DesignAntwerp, BelgiumIEEEMarch 2022
  HALback to text
• 22 inproceedingsM.Maria Kabtoul, A.Anne Spalanzani and P.Philippe Martinet. Proactive And Smooth Maneuvering For Navigation Around Pedestrians.ICRA 2022 - International Conference on Robotics and AutomationPhiladelphia, United StatesMay 2022, 1-7
  HALback to text
• 23 inproceedingsL.Luc Libralesso, F.François Delobel, P.Pascal Lafourcade and C.Christine Solnon. Automatic Generation of Declarative Models for Differential Cryptanalysis.CP 2021 - 27th International Conference on Principles and Practice of Constraint ProgrammingLIPIcs–Leibniz International Proceedings in InformaticsMontpellier, FranceOctober 2021
  HALDOIback to textback to text
• 24 inproceedingsA.Anshul Paigwar, D.David Sierra-Gonzalez, Ö.Özgür Erkent and C.Christian Laugier. Frustum-PointPillars: A Multi-Stage Approach for 3D Object Detection using RGB Camera and LiDAR.2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)2nd Autonomous Vehicle Vision (AVVision) Workshop - ICCV 2021California, United StatesOctober 2021, 1-9
  HALDOIback to textback to text
• 25 inproceedingsU.Unmesh Patil, A.Alessandro Renzaglia, A.Anshul Paigwar and C.Christian Laugier. Real-time Collision Risk Estimation based on Stochastic Reachability Spaces.ICAR 2021 - International Conference on Advanced RoboticsLjubljana, SloveniaDecember 2021, 1-6
  HALDOIback to text
• 26 inproceedingsX.Xiao Peng, C.Christine Solnon and O.Olivier Simonin. Solving the Non-Crossing MAPF with CP.CP 2021 - 27th International Conference on Principles and Practice of Constraint ProgrammingLIPIcs: Leibniz International Proceedings in InformaticsMontpellier (on line), FranceOctober 2021, 1-17
  HALDOIback to textback to text
• 27 inproceedingsM.Manon Prédhumeau, L.Lyuba Mancheva, J.Julie Dugdale and A.Anne Spalanzani. An Agent-Based Model to Predict Pedestrians Trajectories with an Autonomous Vehicle in Shared Spaces.Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021)AAMAS 2021 - 20th International Conference on Autonomous Agents and Multiagent SystemsOnline, FranceMay 2021, 1-9
  HALback to textback to textback to text
• 28 inproceedingsO.Omar Rifki, T.Thierry Garaix and C.Christine Solnon. An asymptotic approximation of the traveling salesman problem with uniform non-overlapping time windows.CASE 2021 - IEEE 17th International Conference on Automation Science and EngineeringLyon, FranceIEEEAugust 2021, 1-6
  HALDOIback to textback to text
• 29 inproceedingsA.Abhishek Tomy, A.Anshul Paigwar, K. S.Khushdeep Singh Mann, A.Alessandro Renzaglia and C.Christian Laugier. Fusing Event-based and RGB camera for Robust Object Detection in Adverse Conditions.2022 IEEE International Conference on Robotics and Automation (ICRA 2022)2022 IEEE International Conference on Robotics and Automation (ICRA 2022)Philadelphia, United StatesIEEEMay 2022
  HALback to text
• 30 inproceedingsS.Samuele Zoboli, V.Vincent Andrieu, D.Daniele Astolfi, G.Giacomo Casadei, J. S.Jilles S Dibangoye and M.Madiha Nadri. Reinforcement Learning Policies With Local LQR Guarantees For Nonlinear Discrete-Time Systems.2021 60th IEEE Conference on Decision and Control (CDC)CDC 2021 - 60th IEEE Conference on Decision and ControlTexas, United StatesIEEEDecember 2021
  HALDOIback to text

National peer-reviewed Conferences

• 31 inproceedingsA.Alexandre Bonnefond, O.Olivier Simonin and I.Isabelle Guérin-Lassous. Extension des Modèles de Flocking aux Environnements avec Obstacles et Communications Dégradées.JFSMA 2021: 29èmes Journées Francophones sur les Systèmes Multi-AgentsProceedings JFSMA 2021Bordeaux, FranceJune 2021
  HALback to text
• 32 inproceedingsJ.-Y.Jean-Yves Courtonne, M.Mathilde Boissier, A.Alexandre Borthomieu, L.Léon Fauste, V.Vincent Jost, E.Emmanuel Krieger, G.Guillaume Mandil, M.Matthieu Mangeot, O.Olivier Mauviel, C.Christine Solnon and P.Peter Sturm. Des outils numériques d'aide à la décision pour la conception et l'évaluation participative d'organisations alternatives de l'économie.ROADEF 2022 - 23ème congrès annuel de la Société Française de Recherche Opérationnelle et d'Aide à la DécisionVilleurbanne - Lyon, FranceFebruary 2022, 1-2
  HALback to textback to text
• 33 inproceedingsA.Aurélien Delage, O.Olivier Buffet and J.Jilles Dibangoye. HSVI pour zs-POSG usant de propriétés de convexité, concavité, et Lipschitz-continuité.JFPDA 2021 - Journées Francophones Planification, Décision et ApprentissageBordeaux (virtuel), FranceJune 2021, 1-14
  HAL
• 34 inproceedingsA.Aurélien Delage, O.Olivier Buffet and J.Jilles Dibangoye. Max-Min Optimization for Lipschitz-Continuous Functions.ROADEF 2022 - 23ème congrès annuel de la Société Française de Recherche Opérationnelle et d'Aide à la DécisionVilleurbanne - Lyon, FranceFebruary 2022, 1-2
  HAL

Conferences without proceedings

• 35 inproceedingsL.Léon Fauste. Support for the choice of geographic scale in the manufacturing industry through constrained optimization.23ème congrès annuel de la Société Française de Recherche Opérationnelle et d'Aide à la DécisionLyon, FranceFebruary 2022, 1-2
  HALback to text
• 36 inproceedingsC.Christian Laugier. Combining Bayesian and AI approaches for Autonomous Driving.IROS 2021 - IEEE/RSJ International Conference on Intelligent Robots and Systems - Workshop "Perception and Navigation for Autonomous Robotics in Unstructured and Dynamic Environments"Prague, Czech RepublicIEEESeptember 2021
  HALback to text
• 37 inproceedingsC.Christian Laugier. The challenge of on-board Perception and Decision-making for Autonomous Driving in Mixed traffic conditions.IROS 2021 - IEEE/RSJ International Conference on Intelligent Robots and Systems - Interactive session on "Future advances and research directions for autonomous robotics"Prague, Czech RepublicIEEESeptember 2021
  HALback to text

Doctoral dissertations and habilitation theses

• 38 thesisR.Rémy Grünblatt. From WiFi Performance Evaluation to Controlled Mobility in Drone Networks.Université Claude Bernard Lyon 1January 2021
  HALback to text
• 39 thesisM.Maria Kabtoul. Proactive and social navigation of autonomous vehicles in shared spaces.Universite Grenoble AlpesDecember 2021
  HAL
• 40 thesisM.Manon Prédhumeau. Modelling and simulation of realistic pedestrian behaviour in a shared space with an autonomous vehicle.Université Grenoble Alpes [2020-....]December 2021
  HAL

Reports & preprints

• 41 reportR.Rabbia Asghar and C.Christian Laugier. System Architecture for Autonomous Driving on Route du Bray.Inria Grenoble - Rhône-Alpes; Inria ChromaFebruary 2021
  HAL
• 42 miscE.Edward Beeching, M.Maxim Peter, P.Philippe Marcotte, J.Jilles Debangoye, O.Olivier Simonin, J.Joshua Romoff and C.Christian Wolf. Graph augmented Deep Reinforcement Learning in the GameRLand3D environment.December 2021
  HAL
• 43 miscA.Aurélien Delage, O.Olivier Buffet and J.Jilles Dibangoye. HSVI fo zs-POSGs using Concavity, Convexity and Lipschitz Properties.October 2021
  HALback to text
• 44 reportC.Christian Laugier. Bayesian & AI driven Embedded Perception and Decision-making. Application to Autonomous Navigation in Complex, Dynamic, Uncertain and Human-populated Environments.Synoptic of Research Activity, Period 2004-20 and beyond.INRIA Grenoble - Rhone-Alpes; LIG (Laboratoire informatique de Grenoble)February 2021
  HALback to text

Other scientific publications

• 45 inproceedingsA.Abhishek Tomy, A.Anshul Paigwar, A.Alessandro Renzaglia and C.Christian Laugier. STADIE-Net: Stagewise Disparity Estimation from Stereo Event-based Cameras.Conference on Computer Vision and Pattern Recognition, Workshop on Event-based Vision (CVPRW), 2021Nashville, United StatesINRIA GrenobleDecember 2021, 1-4
  HALback to text

Scientific popularization

• 46 inproceedingsC.Christian Laugier. La sécurité du véhicule autonome: technologies, enjeux et perspectives.IRT Nanoelec 2021 - Annual Meeting Nanoelec Institut de recherche technologiqueGrenoble, FranceNovember 2021
  HAL