Keywords
Computer Science and Digital Science
- A1.5. Complex systems
- A1.5.1. Systems of systems
- A1.5.2. Communicating systems
- A2.3. Embedded and cyber-physical systems
- A3.4. Machine learning and statistics
- A3.4.1. Supervised learning
- A3.4.2. Unsupervised learning
- A3.4.3. Reinforcement learning
- A3.4.5. Bayesian methods
- A3.4.6. Neural networks
- A3.4.8. Deep learning
- A5.3. Image processing and analysis
- A5.3.3. Pattern recognition
- A5.3.4. Registration
- A5.4. Computer vision
- A5.4.1. Object recognition
- A5.4.2. Activity recognition
- A5.4.4. 3D and spatio-temporal reconstruction
- A5.4.5. Object tracking and motion analysis
- A5.4.6. Object localization
- A5.5.1. Geometrical modeling
- A5.9. Signal processing
- A5.10. Robotics
- A5.10.2. Perception
- A5.10.3. Planning
- A5.10.4. Robot control
- A5.10.5. Robot interaction (with the environment, humans, other robots)
- A5.10.6. Swarm robotics
- A5.10.7. Learning
- A6. Modeling, simulation and control
- A6.1. Methods in mathematical modeling
- A6.2.3. Probabilistic methods
- A6.2.6. Optimization
- A6.4.1. Deterministic control
- A6.4.3. Observability and Controlability
- A6.4.4. Stability and Stabilization
- A6.4.5. Control of distributed parameter systems
- A8.6. Information theory
- A8.9. Performance evaluation
- A9.2. Machine learning
- A9.3. Signal analysis
- A9.5. Robotics
- A9.6. Decision support
- A9.7. AI algorithmics
Other Research Topics and Application Domains
- B5.2.1. Road vehicles
- B5.6. Robotic systems
- B6.6. Embedded systems
- B7.1.2. Road traffic
- B7.2. Smart travel
- B7.2.1. Smart vehicles
- B7.2.2. Smart road
- B9.5.6. Data science
1 Team members, visitors, external collaborators
Research Scientists
- Fawzi Nashashibi [Team leader, Inria, Senior Researcher, DR, HDR]
- Zayed Alsayed [Valeo, Researcher, from Feb 2022]
- Alexandre Boulch [Valeo, from Jul 2022]
- Benazouz Bradaï [Valeo, from Feb 2022]
- Andrei Bursuc [Valeo, Researcher, from Jul 2022]
- Guy Fayolle [Inria, Emeritus]
- Fernando Garrido [Valeo, Researcher, from Feb 2022]
- Jean-Marc Lasgouttes [Inria, Researcher, CR]
- Gérard Le Lann [Inria, Emeritus]
- Renaud Marlet [Valeo, Researcher, from Jul 2022, HDR]
- Gilles Puy [Valeo, Researcher, from Jul 2022]
- Patrick Pérez [Valeo, Researcher, from Jul 2022, HDR]
- Anne Verroust-Blondet [Inria, Researcher, CR]
- Tuan-Hung Vu [Valeo, Researcher, from Jul 2022]
- Raoul de Charette [Inria, Researcher, CR, HDR]
Post-Doctoral Fellows
- Nelson De Moura [Inria, from Aug 2022]
- Tiago Rocha Goncalves [Inria, from May 2022]
PhD Students
- Yacine Ben Ameur [Inria]
- Anh Quan Cao [Inria]
- Mohammad Fahes [Inria, from Oct 2022]
- Amina Ghoul [Inria]
- Ivan Lopes [Inria]
- Noël Nadal [Inria, from Oct 2022]
- Fabio Pizzati [Vislab, until Oct 2022]
- Renaud Poncelet [Sorbonne Université, until Aug 2022]
- Jiahao Zhang [Inria]
Technical Staff
- Emmanuel Doucet [Valeo, from Feb 2022 until Jun 2022]
- Axel Jeanne [Valeo, Engineer, from Feb 2022]
- Kathia Melbouci [Inria]
- Kaouther Messaoud [Inria, Engineer, until Apr 2022]
- Paulo Resende [Valeo, Engineer, from Feb 2022]
Interns and Apprentices
- Mohammad Fahes [Inria, Intern, from Apr 2022 until Sep 2022]
Administrative Assistant
- Anne Mathurin [Inria]
2 Overall objectives
Context
SAE International1 recently unveiled a new visual chart 88 that is designed to define the six levels of driving automation, from SAE Level 0 (no automation) to SAE Level 5 (full vehicle autonomy). It serves as the industry’s most-cited reference for automated-vehicle (AV) capabilities.
Fully autonomous cars (Level 5 of automation according to SAE J3016), which can work everywhere in all conditions, are not yet on the roads. Nevertheless, major advances are making vehicle automation a reality. Systems exist on serial vehicles with Level 2/2+ (assisted driving) and even Level 3 (high automation, driving only upon system request) since 2021 on privately owned vehicles as well as on public transport driverless vehicles are offered to passengers and goods around the world. Recent demonstrators (automated shuttles and robotaxis) have the merit of proving the feasibility of automated driving as a solution for improving mobility, comfort, safety and energy efficiency.
Current regulation (UN 157 – adopted in June 2020 and voted by 60 countries) allows today vehicles to drive in L3 up to 60 km/h on carriageway roads. Original Equipment Manufacturers (OEMs) are pushing for the extension of this regulation up to 130 km/h including automated lane changes. To allow that (L3/L4 on the highway), many challenges are still to be taken up; technical challenges of course, but also non-technical challenges which are not the easiest to deal with (legal, liability, ethical, monopoly, acceptance, economical...) and that are not in the scope of this document even though some intersect with some technical considerations 77, 95, 119.
In this context, the official ambition of France was previously recalled by the President of France, who reaffirmed his willingness to deploy these solutions, to extend transport services based on the autonomous vehicle by 2021 whenever this is possible.
For public transportation, on-road experiments are conducted around the world in specific Operational Design Domains (ODDs) and first commercial services are being deployed. For example, in Russia, Yandex has launched the first commercial service in Europe in 2019 in the city of Innopolis and Waymo. One ride-hailing service using highly automated vehicles in the Phoenix metropolitan area (US). These systems are operating in geofenced controlled environments due to the lack of technology maturity that are able to deal with all road types (missing lines, construction areas, reckless road users behaviour like scooters, etc.).
Therefore, the development of alternative solutions at a large scale needs other scientific foundations and technological breakthroughs. Car makers, suppliers, infrastructure operators and academics across the world are working today on ways to make driving safer, more comfortable, more efficient and more inclusive through automation, and the race is on to bring the technology to the mass market.
In this context Inria and Valeo are internationally distinguished players especially thanks to their R&D activities on automated unmanned vehicles, Cybercars and more generally on the development of advanced intelligent sensors-based decision systems.
Motivation
Partners in numerous collaborative research projects and bilateral projects, Inria and Valeo have also collaborated in the supervision of doctoral and post-doctoral students. Many Inria researchers have also joined Valeo's R&D teams for several years. Finally, numerous technology transfer actions and joint patent applications have taken place. Motivated by this very strong collaboration for over 15 years, Inria and Valeo wanted to formalize this synergy by strengthening their links, both in the fields of research and technology transfer.
What could be better than to create a joint research team to share the same visions on mobility and transport automation? And what could be better than working together upstream on breakthrough research topics? This naturally resulted in the creation of a joint research team: the ASTRA team. This team brings together talents from three entities: the former RITS team at Inria (Paris), members of the DAR team at Valeo (Créteil) and members at Valeo.ai (Paris). Beyond the strategic vision assumed by the management of these three entities, the France Relance national plan was an important incentive for the creation of this unusual joint entity.
3 Research program
Today, there are still many challenges facing the development and deployment of autonomous vehicles to reach an exploitable and commercially viable solution. This is due equally to technical and non-technical challenges. In particular, the challenges include aspects related to the performance of the systems, their efficiency, their integrability and their costs, not to mention the legal, social and ethical aspects.
A classic robust autonomous navigation architecture should take into account additional aspects related to real-time implementation, functional redundancy, durability, certification and purely technical aspects related to the design and development of functional bricks as well.
As part of this project-team we focus mainly on developments related to automated sensor-based navigation. The other aspects are be dealt with in the framework of collaborations and exchanges with other academic, industrial and institutional partners. Therefore, we focus on four research topics that are central to autonomous navigation and a major focus point for the scientific and technical communities. These components are: perception and understanding of the scene, decision systems and vehicle control, cooperative driving and system modeling. These components are linked one another through a complex but straightforward architecture depicted in Fig. 1.
Obviously, the ability to perceive and understand the scene is the starting point of any navigation architecture since it represents the first step of processing sensory data, capturing the world state, and creating the internal digital representations of the decision system. The latter relies on these representations, on the ego vehicle localization and the positions of other road users and on contextual data to build decision schemes which include maneuvers planning and trajectory generation. The control-command loop is then responsible of the execution of the trajectories by the generation of control laws that control the vehicle's actuators.
All these modules interact as shown in Fig. 1 and ensure an autonomous but individual navigation of a vehicle. However, it is important to study the behavior of these vehicles and their performance when the penetration rate (i.e., their ratio to total traffic) of these vehicles becomes critical. It is also very interesting to study the interactions between these vehicles and their potential cooperation. This is called cooperative driving; it can only take place in the presence of connectivity. The latter also ensures interaction and cooperation between autonomous vehicles and infrastructure. The benefits of this type of cooperation are significant, both in terms of the individual performance of each vehicle but also of the overall performance of the vehicle fleet and traffic in general.
3.1 Research Axis 1: Vision and 3D Perception for Scene Understanding
Navigation for mobile robotics requires a robust understanding of the environment from 2D or 3D sensors. Recent learning-based vision algorithms are now able to operate in highly cluttered environments, and tasks which were considered challenging — such as semantic segmentation or object detection — are soon to be solved to a certain extent. Still, the classical supervision paradigm, which relies on large annotated datasets, cannot encompass in practice all outdoor conditions and scenarios. There is therefore a need both to relax the requirement of massive annotations and to extend the perception capability to situations unseen or rarely seen in the training data.
To that aim, in this research axis, we investigate several broad topics. First, we transversely investigate learning with less supervision with applications to various perception tasks. Focusing on outdoor vision, we conduct research relying on data-driven or physics-guided paradigms to hallucinate complex lighting/weather conditions and compensate for missing data in the training sets. Because mobile robots evolve in the physical world we also investigate how vision algorithms can provide in-depth 3D understanding of the scene from images and/or LiDAR scans.
To evaluate our research as well as to foster reproducibility, we rely on relevant recent public datasets (nuScenes 48, Waymo Open 139, Woodscapes 150, SemanticKITTI 39, CADCD 126, etc.) and intend to openly share our research results.
3.1.1 Learning with less supervision
It is now widely accepted that supervised learning is a long-term dead end for computer vision. It relies on costly human- biased annotations, which will soon be unbearable with regard to the ever-increasing size of datasets, trying to cover data diversity. To circumvent the need for labels, strategies have been developed where a trained model is either (almost) directly applicable to unseen conditions (i.e., zero-/few-shot learning) or finetuned on a target domain (i.e., domain adaptation). On the need of data, we investigate automatic generation of data with Generative Adversarial Networks (GANs). Following recent work from the group members 91,6, 142, 143, 129, 127, 148, 118, we contribute to these research directions, investigating the remaining scientific locks that are detailed below.
Regarding zero-shot learning, we observe that current methods are limited by the low amount of geometric information featured in the embeddings that are used as auxiliary information; we therefore boost this geometric information in the embeddings, for example by jointly using text and images. As for few-shot learning, we use high-contrast dictionary-based approaches where generalization is controlled by the level of sparsity. We are also interested in category-agnostic models that can operate on (e.g., detect, segment) arbitrary objects, or that can adapt online to information retrieved from databases of rare objects. We build upon recent progress in representation learning to enforce separable features representations 92 while enforcing orthogonality of features 141. Besides, we investigate both zero- and few-shot learning in the context of a complete perception pipeline, instead of focusing on individual vision tasks as commonly done. In both cases, we will also investigate the use of multiple views and multiple modalities (using both images and LiDAR scans).
Concerning domain adaptation, common unsupervised strategy exploits resemblance between a source and a target domain using a self-supervised signal (e.g., pseudo labels 102) to discover statistics in the target domain. However, when the domain gap is too big, the model adaptation leads to sub-optimal minima 151, 51. To accommodate bigger domain gaps, we investigate the discovery of new statistics with the support of several modalities (e.g., both 2D and 3D), for a variety of tasks (e.g., semantics, depth and normal estimation). Regarding representation learning, we focus on disentangling latent space representations, working towards domain-invariant features by enforcing orthogonality of the domain features while enabling the discovery of exclusive task/domain features. We study bridging zero-/few-shot to the domain adaptation paradigm, investigating the open domain adaptation setting that accounts for novel unseen domains such as 110, 45.
Finally, to relax the need of training data we investigate automatic data generation with image-to-image (i2i) translations and style-transfer techniques, which both can help training in self-supervision settings 40, 127, 101. We observe that GANs commonly lack diversity and controllability in the generated data. To that aim, we study multi-domain setups 53 and automatic discovery of domain attributes 84 to foster controllable latent representations. We fight the lack of diversity in the generated datasets 40 with continuous 145 and multi-modal 127 strategies. Besides standard metrics, we also evaluate the quality of our generated data by training proxy vision tasks.
3.1.2 Vision in complex conditions
The wide variety and continual physical nature of physics prevent any dataset to encompass all lighting and weather conditions. Most outdoor datasets account exclusively for data recorded in clear weather daytime while only a handful of them include adverse conditions. In fact, regardless of the recording complexity some conditions are unlikely to be included in any dataset due to their inherent rarity (e.g., snow storm at sunset). Because they lead to drastically varying appearances we focus here on changing weathers, seasons and lighting conditions; with the complimentary goals to improve robustness of vision algorithms and to automatically assess failures cases.
Rather than agnostic data-driven models, we study training with a priori knowledge, with the ultimate goal to get representations invariant to these conditions. To compensate for the scarcity of data as well as to generalize training to unseen conditions, we rely on physics-guided learning to ease and accommodate the discovery of statistics. We rely here on physical guidance to discover the continuous underlying manifold where data lives 11. Using physical models to guide the training helps vision algorithms to accommodate better to partial or imbalanced distribution in the training set, as well as to better extrapolate to unseen conditions. We are focusing on invariant representations that can improve both the image translation setup and proxy vision tasks (segmentation, objects, etc.); relying on prior works from group members 11, 131, 14, 12.
Sometimes, weather conditions go even beyond the sensing capabilities of sensors, e.g., sun glare or very dark scenes can reduce dramatically the perception of standard cameras. In such cases, robustness is difficult to attain and the system should rather trigger an alert or fail gracefully. Unseen weather conditions encountered at runtime can be regarded as a dataset/distribution shift and can be addressed with predictive uncertainty estimation methods 123. Through a Bayesian lens we study and devise strategies for automatic assessment and detection of dataset drifts by leveraging approximate ensembles 112, 34, 67, observer networks 59, 85, and complementary information from other sensors 41. We rely on prior findings and works from group members 59, 67, 66,14, 131.
On application, we evaluate robustness of the proposed methods on core vision tasks of recent adverse weather datasets 135, 152, 139, 48, 41.
3.1.3 3D scene understanding
Robots still commonly lack the natural ability of humans to estimate the fine-grained geometry of a scene while understanding object interactions and reasoning beyond their field of view. To provide accurate geometry, 3D active sensors such as LiDARs are commonly used in autonomous driving 89, but they only provide a sparse sensing of the scene. In this third topic, we seek a fine-grained geometrical/semantics 3D understanding of the scene with or without 3D sensing, while also relying on frugal supervision. This topic benefits from prior work of group members 130, 43, 42, 91,13, 90, 149, 96, 49.
Building up on recent methods 43, 42, 140, 105, 79 that efficiently convolve point clouds, we look forward at improving 3D tasks (detection, segmentation, etc.) relying on contextual priors. Furthermore, we address 3D generative tasks like point cloud up-sampling, completion and generation, as well as surface reconstruction, which provides important navigation cues for robotics, and can also assist the human driver in augmented reality scenarios, particularly in adverse conditions. Temporally consecutive point clouds will also be leveraged to disambiguate occlusions and provide denser scene sensing 130, 49. Regarding richer scene representations, we study the intertwined relation of geometry and semantics 137 through the semantic scene completion task 13, 133, 132, which gained growing interest lately 39.
Another line of study is the interaction between modalities of different nature like for scene understanding, in particular the complementarity of 2D images and 3D scans. We study how multi-modal features can jointly improve performance of core tasks, but also how it can lead to improving the performance of single modalities by exploiting cross-modal features as self-supervision 91,6.
Besides the use of 3D devices, we also investigate 3D understanding from 2D images. As they originate from passive sensors, images carry less obvious geometrical cues but humans are still able to estimate depth and understand 3D from a photograph, heavily reasoning on learned priors. We study here challenging tasks like scene reconstruction or 6-DOF localization, which can be conveniently self-supervised from either 3D sensing or sequential data.
3.2 Research Axis 2: Localization & Mapping
Vehicle localization and environmental mapping are pillars of the perception task for an autonomous vehicle. While vehicle localization ensures the global positioning of the vehicle in its environment and local positioning with regard to the road and to the close road features, environment mapping contributes in building a useful internal representation that is exploited by the decision system.
Inria and Valeo teams have been working - separately and jointly - on the localization and mapping solutions for over the past 15 years. Many algorithms have been developed and showed their effectiveness in terms of accuracy, precision and safety expectations for autonomous driving. However, the integrity, safety, data size and costs are still challenging points that ASTRA wants to address.
3.2.1 Localization and Map Integrity
Many localization methods were developed mainly based on Particle Filter and GraphSLAM together with a point cloud representation of the environment. These solutions mainly focus on the accuracy and precision requirements of the pose estimations. Yet, the integrity of localization and integrity of maps used for localization are critical to ensure a safe use of the localization system for autonomous driving. State-of-the-art methods on localization integrity usually proceed by: 1. employing Fault Detection and Isolation algorithms (FDI) to remove outliers from input data. 2. computing Protection Levels (PL) to qualify the integrity zone 9983100 or by calculating the Protection Levels (without FDI) such as in 10836. Maps integrity is highly related to the feasibility to find a distinctive matching when using the map for localization. Indeed the map can be explored by an algorithm that aims to identify the zones or sections that represent a potential ambiguity for matching algorithms such as in 86.
3.2.2 Online Alignment of Multiple Map Layers
A wide diversity of maps that are dedicated to vehicle’s localization are nowadays available. These maps are different from each other regarding different key localization features. The most important aspects may be: the structure of the representation (e.g., grid, graph etc.), the underlying theory to represent the information of the environment (e.g., occupancy probabilities, landmarks, etc), and the sensor used to collect information (LiDAR, camera, etc). Map providers, such as Here and TomTom, usually provide maps with different layers to encode different information that are relevant to ADS features (Road model, lanes, and road features). Valeo, having the advantage of being the leader of automotive LiDAR sensor, wants to enhance his ADS solutions arsenal as a map provider by providing a map service based on the laser point clouds and potentially other information layers that are relevant to ADS. For this purpose it is important to find correspondences and align different map layers with other maps from maps providers. This subject is addressed by considering semantic information that can be extracted from heterogeneous sensors and maps data such as in [9] and [10].
3.2.3 Georeferencing of maps without RTK GNSS and IMU
Highly accurate maps that are used for AD localization are usually built using a very expensive Fusion box that includes a very precise RTK_GPS receiver and a first grade IMU. These solutions for map building are very expensive and require deployment of RTK bases in the environment to receive the corrections which imply extra cost. The idea of this subject is to be able to use available sensors (such as standard GNSS, IMU, CAN, LiDAR, Camera) and possibly maps from other providers to build a highly accurate (in the global reference) map using point clouds. Different inputs from sensors and maps can be considered together with an asynchronous fusion method to build an accurate estimation [11]. The method to achieve this goal constitutes the subject of this study.
3.3 Research Axis 3: Decision making, motion Planning & vehicle Control
Decision-making, maneuver and motion planning, and vehicle control are extremely vital components of the intelligent vehicle. These modules act as a bridge, connecting the perception subsystem of the environment and the bottom-level control subsystem in charge of the execution of the motion. We address these issues covering various strategies of designing the decision-making, trajectory planning, and tracking control, as well as shared driving of the human-automation to adapt to different levels of the automated driving system accounting with the driver profile.
The challenges related to decision making and path planning are mainly related to four distinct elements:
- Errors and uncertainties introduced by the perception subsystems
- Environment static and dynamic occlusions
- Lack of understanding and prediction of other road users behaviors
- Simultaneous consideration of several constraints related to: vehicles dynamics, energy consumption, passengers comfort, offense to driving rule...
Different approaches are investigated in the state of the art addressing one or several issues but, to our knowledge, none are capable of addressing all of them simultaneously. More specifically in most approaches decision and planning are dealt separately or in a way that favors one of them. Approaches based on Markov decision process (MPD, POMDP,...), path-speed profiles, ontologies, artificial potential fields coupled to MPC controllers are able to show interesting results in dedicated environments or in specific situations, however most of them do not tackle properly specific issues such as intention and behavior predictions, interactions or multi-criteria real time optimal maneuver decision.
While continuing the investigation of end-to-end driving approaches based (inverse-)reinforcement learning decision-making approaches, we keep on improving current path-planning methods already developed by both teams at RITS and DAR: Reachable Interaction Sets 38, Artificial Potentials Fields (coupled to MPC control) which are designed for obstacle avoidance, as well as traditional path planning methods. Optimal methods based on the convex optimization and cubic splines are investigated at DAR to design optimized and robust trajectories. More specifically, we are mainly focusing on the following three scientific topics (detailed in the next sections):
- Maneuvers and trajectories prediction of surrounding road users
- Schemes for ego-vehicle actions and maneuvers decision making and motion planning
- Motion planning and trajectories generation
3.3.1 Maneuver and trajectory prediction
To achieve a safe and comfortable driving, an autonomous driving system must have an accurate knowledge of the future motions of all other traffic agents surrounding the autonomous vehicle, such as cars, pedestrians, cyclists, etc. Motion prediction is thus a key task in autonomous vehicles. Several methods of motion prediction have been studied in the literature. Lefèvre et al 103 propose their classification in three levels with an increasing degree of abstraction: Physics-based models, Maneuver-based models and interaction-based models.
- Physics-based motion models. They consider that the motion of vehicles only depends on the laws of physics. The future motion is predicted using dynamic and kinematic models linking some control inputs car properties and external conditions. These models are limited to short term prediction and are unable to anticipate any change in the motion of the car caused by the execution of a particular maneuver.
- Maneuver-based motion models. They consider that the future motion of a vehicle also depends on the maneuver that the driver intends to perform. The future motion of a vehicle on the road network corresponds to a series of maneuvers executed independently from the other vehicles. These models are Unadaptable to different road layouts.
-
Interaction-aware motion models. They take into account the inter-dependencies between vehicles’ maneuvers. These models require computing all the potential trajectories of the vehicles which is computationally expensive and no compatible with real-time risk assessment. Valeo has filed a patent to overcome this issue 146. This patented method is being developed in order to be tested in the automated driving prototypes.
Fig. 2 shows a comparison of the different models including their challenges and the used algorithms.
Valeo has considered these categories in its development of the automated driving prototypes Cruise4U and Drive4U. The physical-based model is used in situations when their is no knowledge about the route geometry (for example in a big roundabout without lanes), the maneuver-based in highway and urban environment when the road topology is available from HD Map or valeo Drive4U Locate map.
In the few last years, machine learning based algorithms and particularly deep learning are used in order to solve the limits of the current prediction methods. Human motion trajectory prediction has been addressed in the literature 44, 134. A large amount of naturalistic road user trajectories in different contexts (highways 57, 58, 93 or urban 48, 50) needed to train and evaluate deep learning methods are now available. Our first works 10, 117,9, 115, taking as input the track history of a target vehicle and of its surrounding moving road users, obtained accurate prediction results of the target vehicle motion on highways and an extension 116, including the static scene structure, has been proposed for an urban context. Valeo is involved in this research area with activities in prediction of other road users and ego-vehicle trajectory. Different approaches have been implemented and tested in simulation and on test cars 47, 46.
However, work has still to be done in this domain in terms of performance, robustness and generalization before being used in real autonomous driving applications. In fact, the behavior of a human driver depends also on the contextual knowledge of the environment (speed limits, traffic density, day of the week, visibility, road equipment, driver's country, etc.) and on its goal 154. We plan to include these contextual cues in a prediction method, which should also compute multiple plausible trajectories representing the driver's diverse possible behaviors, give uncertainties estimations on the predictions, carry out multi-agents trajectory forecasts and should be usable in any environment. It will necessitate the use of a more complete dataset 153 composed of various driving scenarios collected from different countries, which may be completed by our own dataset collected with the help of Valeo if necessary. This work will be done in collaboration with Itheri Yahiaoui from Reims University and within the starting PhD thesis of Amina Ghoul funded by the SAMBA project.
3.3.2 Ego-vehicle actions and maneuvers decision making
The most important component of an autonomous vehicle navigation system is the decision system that elaborates the coming tactical actions and maneuvers to be executed. The selection of the optimal maneuver should be the result of relevant and simultaneous consideration of several factors. These factors are mainly: safety and risk assessment, respect of the dynamic constraints of the vehicle and its controllability, uncertainties related to the perception outputs, nearby uncertain interactions with/between close road users, and finally the criterion related to the navigation objectives such as journey duration minimization, driver/passenger comfort, fuel/energy consumption minimization, respect of driving rules, etc. The latter being expressed in terms of kinematics constraints.
In the literature, there are very few approaches describing unified decision architectures capable of taking into account all of the considerations mentioned above. Most approaches are developing planning schemes which separate motion generation and decision making. In these approaches, motion planning (including reactive planning) usually exploits geometry, configuration spaces and other optimization techniques. Decision making schemes rely on AI logic based approaches such as rule based 122, decision trees 55, 106, Finite Set Machines 155, Bayesian Networks and Markov Decision Processes like approaches (MDP, POMDP…), AI heuristics algorithms (SVM’s and evolutionary methods) or AI approximate reasoning methods (fuzzy logic) and Artificial Neural Networks (CNN’s, Reinforcement Learning…) 107, 144, 52. In 56 propose an architecture that provides an optimization of the motion generation using the decision making function as the evaluation function, the aggregation of fuzzy logic and belief theory allowing decision making on heterogeneous criteria and uncertain data.
In the coming period we will work on unified architectures, that tackles simultaneously decision making and motion planning. Very likely, we will focus on deep learning techniques based on reinforcement learning and inverse reinforcement learning where we deem a (dense) reward function that is suitable for a large class of behavioural planning tasks. More generally, we will investigate model-free and model-based approaches where some interesting approaches have already been initiated and showed interesting results such as in 104. In particular, in order to better evaluate safety costs, we will take as input the output of the maneuvers and trajectories prediction system described in the previous section, which has the advantage to better estimate the road users trajectories thanks to attention mechanisms that encode interactions and behaviors. This work is done within the PhD of Yacine Ben Ameur funded by the SAMBA project.
3.3.3 Trajectory planning
State of the art on motion planning techniques have been mainly focusing on methods generating the geometric path first, and then applying a speed profile to the generated path. To mention just a few, this approach has been tackled by the following methods (or combinations): interpolating curve-based 75, 76, graph-search based 109, sampling-based 94 and optimization-based 80.
From the motion planning point of view, the inclusion of human factors is a key element for increasing the acceptance of the automated vehicle behavior and for providing a more human-like response. For that purpose, the use of data from real drivers should be envisaged to better define the motion constraints in dynamic environments, allowing to adapt the trajectories to any specific road scenario (intersections, roundabouts, merging, overtaking, lane driving, etc). For instance, motion constraints such as longitudinal and lateral accelerations as well as jerks should be properly taken into account in the generation of a human-like speed-profile, as introduced in 35.
Furthermore, the inclusion of driving factors such as energy consumption or the traffic occupancy should be considered in the multi-criteria optimization for better adapting to any driving situation. This would help to reduce the driving time (such as the commute time) or even reduce the energetic consumption and the stress of both driver and car passengers by reducing the traffic jams and the corresponding repetitive acceleration and braking maneuvers.
Finally, this planning module must fit to the time constraints for its execution in real-time to ensure safety. Thus, a complete and rapid motion planning approach is needed; it should consider the functional safety to generate real-time collision-free trajectories considering the different interactions with the surrounding vehicles to be tracked by the control. For that purpose, works presented at 37 will be extended in order to consider the interaction among the several surrounding road users as one and not as individual interactions, investigating the risk assessment metric that is the most appropriate for each specific scenario.
3.3.4 Robust control of automated vehicles
In order to execute safely a planned trajectory or a reactive maneuver, it is essential that the vehicle executes these trajectories taking into account the vehicle dynamics while ensuring safe, stable and comfortable maneuvers. A tremendous effort was deployed the last 10 years by the team partners in the area of motion planning and intelligent control. Seven PhD thesis were dedicated to the important problem of path and motion planning as well as on corresponding control-command. All are addressing the navigation of autonomous vehicles in structured but complex environments. Harsh configurations such as intersections and roundabouts need specific planning approaches taking into account the geometry and the topology of the places, but also the dynamic and kinematic constraints of each ego-vehicle and as the safety and comfort constraints.
Previously, RITS team (Inria) also implemented specific control algorithms dedicated to specific road maneuvers such as overtaking 124 and parking maneuvers 125. Control laws were designed with the theoretical proof of stability and optimality. Very interesting results were obtained in two major domains, mainly related to the controllability and stability of dynamic complex systems which are key aspects when it comes to design intelligent control algorithms for vehicles:
- Plug&Play control for highly non-linear systems: Stability analysis of autonomous vehicles. The developed Plug&Play control is able to provide stability responses for autonomous vehicles under uncontrolled circumstances, including modifications on the input/output sensors. Former RITS team was among the very first to investigate these theories for automotive applications. They were Investigated in the PhD thesis of Mr. F. Navas 120 and I. Mahtout 111. The approach deals with: the reconfiguration of existing controllers whenever changes are introduced in the system (gain scheduling), online closed loop identification of the vehicle and its components, and Automatic control reconfiguration to achieve optimal performance 1218.
- Fractional Calculus for Cooperative Car Following Control A Car-Following gap regulation controller using fractional order calculus, has been developed and has been proven to yield a more accurate description of real processes and ensure string stability of the platoons or the vehicles involved in a Cooperative Autonomous Cruise Control 63. In an effort to combine fractional calculus robust control with plug&play control, a multi-model adaptive control (MMAC) algorithm based on Youla-Kucera (YK) theory to deal with heterogeneity in cooperative adaptive cruise control (CACC) systems was proposed 64.
ASTRA will evolve by introducing intelligent cooperation between vehicles and, at the same time, autonomously driving the vehicle in a human driver way (increasing driver acceptability) but with the safety and accuracy of optimized control algorithms. To achieve this, we will rely on the existing approaches developed so far but no further research will be conducted in the lifetime of the joint team. This is mainly due to the absence of a senior researcher at ASTRA capable of carrying this topic independently at a high level. This also motivates the team to seek to recruit a new confirmed researcher in the field of the control of dynamic systems, a crucial domain for a team willing to develop and deploy advanced control architectures on real mobile platforms. In the meanwhile it would be very interesting to envisage collaborations with other Inria teams working on similar topics. A perfect example is DISCO team (Inria Saclay Research Center, head: Mrs. Catherine Bonnet). Among others, the research interests of DISCO cover: the realization and reduction of infinite-dimensional systems, Robust and BIBO parametrization and stabilization of infinite-dimensional systems, stabilization by finite-dimensional controllers (PID control), delay systems and fractional systems.
This research direction comprises a big interaction with the research axis: Large scale modeling and deployment of mobility systems in Smart Cities. The former will be essential when developing control algorithms that rely on a very small communication delay for getting a stable latency, designing stable systems. The latter will serve to analyze the effect over the traffic flow from a developed algorithm, moving from the validation of a proposed controller in a limited number of vehicles to a its study from a macroscopic perspective.
3.4 Research Axis 4: Large scale modeling and deployment of mobility systems in Smart Cities
While axes 1 to 3 deal with subjects related to the on-board intelligence of an “individual” intelligent vehicle and its autonomous navigation, axis 4 intervenes when it comes to many communicating, autonomous or automated vehicles but also when it comes to the cooperation with the static environment (infrastructure). The latter may contain and integrate: roadside and monitoring sensors (Cooperative Perception Services), signaling, communication infrastructures, cloud... The research concerns in particular the deployment of equipped vehicles on a large scale in a road or urban environment.
The research objectives are twofold.
-
First, the focus is on the modeling of systems comprising a large number of vehicles, often seen as random entities.
The methodology is mainly to explore the links between large random systems and statistical physics. This approach proves very powerful, both for macroscopic (fleet management 62) and microscopic (car-level description of traffic, formation of jams 69, 136, 74, 73) analysis. The general setting is mathematical modelling of large systems (typically in the so-called thermodynamical limit), without any a priori restriction: networks, random graphs, etc. One often aims at establishing a classification based on criteria of a twofold nature: quantitative (performance, throughput, etc) and qualitative (stability, asymptotic behavior, phase transition, complexity).
- The second objective concerns the cooperation of these communicating entities in order to address the efficiency and safety of mobility. This cooperation takes several forms. Direct or indirect communications (V2X) are dedicated to maneuver coordination, taming and improving traffic efficiency (cf. section 4.4.2), platooning, safety critical distributed coordination (cf. 4.4.3)... Crowdsourcing is another aspect that could be used for traffic modeling and prediction (cf. 4.4.1), environment augmented mapping, or global vehicles localization. A Phd student will be hired this year to work on this precise subject (cf. 4.5).
Beside this core methodology, other past activities of interest include discrete event simulation 54, 98 and resource allocation for ITS 97, 81, 82.
Finally, axis 4 does not represent a structural unit like the other axes. Its objective is to deal with the problem of scaling, deployment and multi-vehicle cooperation in a global and systemic way. On the substance, methods and theories of modeling will be investigated and the design of secure telecommunication systems will be elaborated. These models and systems are intended to be implemented in more global systems and architectures. They will interact with the other axes through these architectures and will respond in a targeted way to needs; for example, whenever a need for probabilistic modeling is expressed (e.g. section 4.5).
3.4.1 Traffic prediction in urban settings: detecting extreme events
A probabilistic forecasting method that can provide predictions of urban traffic at city level, accurate at short term and meaningful for a horizon of up to several hours, has been devised in the team 72, 68, 71, 113, 114, 703, in collaboration with C. Furtlehner (TAU, Inria Saclay). It is designed to leverage spatial and temporal dependency and can deal with missing data, both for training and running the model. The method consists in learning a sparse Gaussian copula of traffic variables, compatible with the Gaussian belief propagation algorithm. Results of tests performed on three urban datasets show a very good ability to predict flow variables and reasonably good performances on speed variables.
When investigating the output of the model, some rare but large errors are noticeable. It turns out that this corresponds to detectors which, for a long period, send values completely at odds with the ones observed during training. These badly behaving detectors may either correspond to corrupted ones, or to drastic changes of the traffic conditions on the corresponding segment, because of road work or accidents for instance.
One way of examining these events has been proposed in 87, and we plan to investigate whether it can be used to improve models. Separating sensor failure from extremal events is even more important, and this is what we plan to investigate in a PhD thesis, by careful analysis of the correlation structure of the model.
3.4.2 Taming highway traffic using cooperative automated vehicles
Several authors 65, 60, 147, 78 have suggested that it is possible to use a small proportion of automated vehicles to regulate highway traffic. These studies are set in a traffic regime which exhibits string instability, which means in terms of transfer function that any excitation of a frequency below a certain limit is amplified. We are interested here in a slightly different setting, where reaction time is taken into account for human drivers. We have shown 17 that the introduction of this delay involves a non rational transfer function, implying in particular that the system is not always stable. We have proposed a complete self-contained proof of stability conditions, based on classical complex analysis. Moreover, we bring to light a phase transition with a new propagation regime, named partial string stability, situated between string stability and string instability.
With these foundations established, the next steps are to devise a traffic stabilization scheme by means of a fleet of cooperative automated vehicles. However, contrary to the work in 65, our approach is based on a car-following model with reaction-time delay, rather than on a first order fluid model. The continuation of these studies will concern shock wave analysis and adequate traffic-stabilizing control strategies.
3.4.3 Crowdsourced mapping
The deployment of intelligent and connected vehicles, equipped with increasingly sophisticated equipment, and capable of sharing accurate positions and trajectories, is expected to lead to a substantial improvement of road safety and traffic efficiency. Nevertheless, in order to guarantee accurate positioning in all conditions, including in dense zones where GNSS signals can get degraded by multi-path effects, it is expected that sensory equipped vehicles will need to use precise maps of the environment to support their localization algorithms. Crowdsourced mapping represents a cost-effective solution to this problem, consisting in making use of measurements retrieved by multiple production vehicles equipped with standard sensors in order to build an accurate map of landmarks and maintain it up-to-date in realistic, long-term scenarios. Existing SoA crowdsourced mapping solutions rely on triangulation optimization or graph-based optimization where trade-offs between the map quality and computational scalability are still to be investigated. We propose to extend the work of 138 to improve scalability. One possible approach is to rely on a Gaussian Belief algorithm to estimate and update the position of landmarks and of the the vehicles, along with their corresponding uncertainties.
3.4.4 Cooperative automated driving involving V2X communications
Automated driving in a complex shared road requires cooperation among road entities in terms of cooperative control, cooperative perception, and cooperative path planning. This poses new research challenges that did not exist in the domain of vehicular communications e.g., communications for cooperative automated driving and intention-aware communications. Based on our experiences and know-how on mobile telecommunications, networking, and robotics domains, the ASTRA team will conduct research activities within the following domains:
- Safety critical V2V communications.
- Safety critical distributed coordination.
- Safety and performance guided V2X communication and data processing
- Vehicles' behaviors and intention-aware communications
4 Application domains
The aim of the project team is to tackle the challenges and provide breakthrough solutions for the autonomous and connected mobility. It covers the improvement of the safety, the availability and the performances of ADAS “Advanced Driver Assistance Systems” and the L3 automated systems (Traffic Jam Pilot and Highway Pilot) for privately owned vehicles as well as L4 automated systems including Robotaxi and automated transportation systems like autonomous shuttles. Enabled by 5G and the V2X connectivity in general, the extension to cooperative Automated driving and the Smart city will also be considered. There are more and more cities and highways equipping their infrastructures with sensors that can enable extended and shared perception. During the project, the developed solutions are tested for these applications. Valeo Automated Driving roadmap is addressing them through 3 programs. Cruise4U Program for multiple carriageway/highways, Drive4U for Urban environment including autonomous shuttles and eDeliver4U for last mile delivery as shown in Fig. 3.
The Cruise4U and Drive4U programs allowed to Valeo to perform open roads experiments around the world with more than 200,000 km accumulated in real conditions with plenty of use cases.
Fig. 4 shows a part of the Cruise4U experiments, while Fig. 5 shows world premieres: Drive4U open road experiments with only Valeo serial production sensors operating in Paris, Las Vegas and Tokyo.
A dedicated Automated Driving platform for the project team is under discussion in order to allow quick and easy integration, tests and validations of the Joint team developments.
5 Highlights of the year
- This year was the last year of activity for our dear colleague Anne Verroust-Blondet who retired at the end of the calendar year. We salute an inspiring and admirable colleague with a rich and fruitful professional life. This departure is an even more pressing opportunity to recruit at least one new researcher in the team, in order to ensure scientific continuity and even to reinforce the supervising team. The profile sought is that of a talented young researcher in the field of AI, capable of collaborating with the members of the team while developing striking scientific activities in rupture with the existing ones.
- Raoul de Charette defended his Habilitation à diriger les recherches “Vision for Scene Understanding” 29 on Jan 25, 2022.
- Fabio Pizzati and Raoul de Charette received the best paper award of VISAPP 2022 for “Leveraging Local Domains for Image-to-Image Translation” 23.
- The Inria members of ASTRA team team wish to highlight that its 2022 production was affected by the current Inria situation. Administrative malfunctions and new processes (e.g. hiring restrictions) inevitably led to an overload of the support services, both at the Inria and team level, which impacted our ability to recruit talented scientists, manage projects, etc. More generally, we are, to say the least, puzzled by what seems to be a shift in some missions of the institute. The subsequent rising tensions between parties altered our working conditions. We hope that clear direction and lighter and less stringent rules will allow team members to refocus on their original scientific mission.
6 New software and platforms
New softwares
-
MonoScene
Monocular 3D Semantic Scene Completion
- Url: MonoScene
- Contribution: leader
- Software Family: research
- Audience: community
- Evolution and maintenance: basic
- Duration of the Development:
-
-
SceneRF
Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields
- Url: SceneRF
- Contribution: leader
- Software Family: research
- Audience: community
- Evolution and maintenance: basic
- Duration of the Development:
-
-
PODA
Prompt-driven Zero-shot Domain adaptation
- Url: PODA
- Contribution: leader
- Software Family: research
- Audience: community
- Evolution and maintenance: basic
- Duration of the Development:
-
-
ManiFest
Few-shot image translation method working on unstructured environments
- Url: ManiFest
- Contribution: leader
- Software Family: research
- Audience: community
- Evolution and maintenance: basic
- Duration of the Development:
-
-
DenseMTL
Cross-task Attention Mechanism for Dense Multi-task Learning
- Url: DenseMTL
- Contribution: leader
- Software Family: research
- Audience: community
- Evolution and maintenance: basic
- Duration of the Development:
-
-
COARSE3D
Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation
- Url: COARSE3D
- Contribution: leader
- Software Family: research
- Audience: community
- Evolution and maintenance: basic
- Duration of the Development:
-
6.1 New software
6.2 New platforms
Participants: Paulo Resende, Benazouz BRADAI, Gaëtan Le Gall, Fawzi Nashashibi.
The creation of the team has resulted in the strengthening ofthe experimental side of the team which has always had among its objectives the validation of work on real instrumented platforms. The creation of the team has resulted in the strengthening of the experimental side of the team which has always had among its objectives the validation of work on real instrumented platforms. Thus, the team is equipped with at least four road vehicles (Cruise4U and Drive4U from Valeo, a C1 and a Zoé from Inria), 3 shuttles (2 Cybus from Inria and a NAVYA from Valeo) and 2 Cybercars (Inria) ( 6).
7 New results
7.1 Learning vision with less supervision
Participants: Andrei Bursuc, Anh-Quan Cao, Raoul de Charette, Mohammad Fahes, Ivan Lopes, Patrick Pérez, Tuan-Hung Vu.
Supervision is an evident bottleneck for computer vision in the real-word. In this research axis we have explored new paradigms for relaxing the need of human supervision. We explored the use of multi-modal inputs (here, image and 3D data) to discover statistical correlations between 2D and 3D which we have shown is beneficial for a wide variety of transfer learning scenarios in semantic segmentation, leading to a T-PAMI publication 20. Multi-tasks learning, where a single network solves tasks of different natures, also alleviate supervision burdens because it lowers the need of task-specific architecture. In that sense, we proposed a novel task-exchange mechanism which helps knowledge distillation 26, and demonstrates state-of-the-art results on several datasets. To reduce the amount of supervision, in the 3D axis, we employed constrative learning to build rich and compact representations of semantic classes for 3D semantic segmentation which led to 25, and demonstrated performance to full-supervision (i.e., 100% labels) using only 0.01% labels. Finally, in the latest months we have explored interaction of language and vision models to perform zero-shot adaptation from textual prompt which drives the adaptation of the image features and our experiments show exciting state-of-the-art results 32.
7.2 Vision in complex conditions
Participants: Raoul de Charette, Fabio Pizzati.
Because the physical world is a continuum, encompassing all conditions in a single dataset is impossible. In this direction, and also related to weakly supervised algorithms, we investigated how human-informed knowledge can benefit source-to-target transfer learning, for example to perform on adverse lighting/weather conditions unseen during training. With minimal geometric human-guidance and a patch-wise learning, we demonstrated state-of-the-art results on multiple scenarios 23, which was rewarded Best Paper Award at VISAPP 2022. In a separate work, we also investigated the realistic translation of images (image-to-image translation or, i2i) using of a subsidiary domain, which we call anchor, to learn i2i from only few-shots. In particular, this was applied to the complex day-to-night, or clear-to-rain translations 27. When target domain includes `occlusions' such as raindrop in clear-to-rain, the translations lack realism, and as a continuation of prior works we explored the use of physics models to disentangle visual representations and obtain realistic and controllable translations with GANs 33 (journal submission). In this axis, most results were within the thesis of Fabio Pizzati 128 (defended on Nov 29th 2022).
7.3 3D scene understanding
Participants: Anh-Quan Cao, Raoul de Charette.
Mobile robots require a holistic understanding of the scene, in particular of its geometry. In this research axis, we investigated how 3D perception is improved. Correlated to the weakly-supervised axis, we explored the learning of compact feature representations with just a tiny split of the complete labels in the context of 3D semantic segmentation of point cloud 25. An other important aspect, is the ability to complete the scene geometry and semantics beyond the inevitable scene occlusions. We studied how complete 3D scene semantics can be extracted from a single 2D image, in particular by proposing a new 2D-3D network bridging strategy, leading to MonoScene – the first monocular semantic scene completion method – published in 22. Inspired by these results, in our work, SceneRF, we studied 3D scene reconstruction from a monocular image but this time without any human supervision – building on Neural Radiance Fields (NeRF) to hallucinate novel depth views of a scene from image sequences, leading to 31.
7.4 Trust Management Framework for Misbehavior Detection in Collective Perception Services
Participants: Jiahao Zhang, Fawzi Nashashibi.
This work has been conducted in collaboration with external partners from IRT-SystemX Institute: Ines Ben-Jemaa and Francesca Bassi.
Cooperative Intelligent Transportation System (C-ITS) is a new technology that aims to reduce traffic accidents and improve road safety. The technology relies on wireless communications in the form of safety messages. Collective Perception Messages (CPM) enable vehicles to share their perceived objects with their neighbors in V2X network. These perception data extend local vehicles’ perception and consequently improve road safety awareness. However, attacks on perception data are challenging and require advanced and efficient misbehavior detection mechanism especially in specific road scenarios where contradictory information need to be analyzed. In this work, we introduce a trust management framework 28 to detect misbehaving nodes through transmitted CPM messages. Our framework is based on trust assessment built through several processing steps. It addresses conflict situation when contradictory data are received using the Subjective Logic mechanism. The results show that our solution is effective in detecting misbehaving nodes based on their attributed trust scores. In addition, we show the impact of our solution and some CPM configuration parameters on safety services and especially on risk anticipation in intersection scenarios. As a future work, we plan to evaluate our solution taking into consideration more advanced parameters such as perception uncertainty and more complex attacks on new road scenarios.
7.5 Motion planning and prediction
Participants: Nelson De Moura, Fernando Garrido, Paulo Resende.
Guided by the findings in the recent survey of some team members 19, we have investigated dynamic motion planner as well as studied generation of speed profiles in a path-speed decoupled fashion. With the PostDoc Nelson De Moura (started in Aug. 2022), we also studied: 1) interaction-aware motion predictions with behavior clustering, 2) high-level decision making with Partially Observable Markov Decision Process (POMDP). Initial results on the inD datasets, using k-means clustering, show emergence of category-specific patterns, which could be later employed for fine-grained prediction of traffic users. Some of the results led to experiments on Valeo prototypes (eDeliver4U) on test sites, as show in 7.
7.6 Control and Human Factors
Participants: Nelson De Moura, Tiago Rocha.
Following prior member works 119, 61, we studied a risk prediction system to evaluate the trade-off between ethical aspects and fuel-efficiency improvements in platooning systems. Leveraging the harm concept from 119, an MDP approach is used to adjust dynamically controllers during platooning operation.
7.7 Risk-averse reinforcement learning algorithm for autonomous driving
Participants: Yacine Ben Ameur, Fawzi Nashashibi, Anne Verroust-Blondet.
In a different effort to contribute to autonomous vehicles decision making, a new Reinforcement Learning-based approach has been investigated using surrogate-based optimization to prioritize safety by maximizing performance in worst-case scenarios. The approach is compared to the commonly used risk-neutral PPO algorithm and shows improved results in avoiding collisions while still managing to navigate dense traffic. The algorithm uses a quantile utility network to predict the sum of rewards and compute the gradient w.r.t. policy parameters. Policy parameters and observed utility are stored in a replay buffer and used to train the utility network. This risk-averse approach is a step forward in ensuring the safety of autonomous vehicles. This work has been recently submitted for publication in IEEE ARSO conference.
7.8 A Lightweight Goal-Based model for Trajectory Prediction
Participants: Amina Ghoul, Kaouther Messaoud, Itheri Yahiaoui, Anne Verroust-Blondet, Fawzi Nashashibi.
Predicting the future motion of a dynamic agent knowing its past trajectory is crucial in many fields such as advanced surveillance systems and autonomous vehicles. However this task is challenging as it depends on various factors such as the agent’s intention, the static environment around the agent, the interaction with other agents and its kinematics. Because of these uncertainties, future motion of agents are inherently multimodal. We present a lightweight goal-based model for multimodal, probabilistic trajectory prediction for urban driving. Previous conditioned-on-goal methods have used map information in order to establish a set of potential goals and then complete the corresponding full trajectory for each goal. We instead propose two original representations, based on the agent's states and its kinematics, to extract the potential goals. We conduct a comparative study between the two representations. We also evaluate our approach on the nuScenes dataset, and show that it outperforms a wide array of state-of-the-art methods. The results can be found at the ITSC 2022 Conference 24.
7.9 Stability and String Stability of Car-following Models with Reaction-time Delay
Participants: Guy Fayolle, Jean-Marc Lasgouttes.
In 17, in collaboration and C. Flores (Institute of Transportation Studies, UCB), we investigate the transfer function emanating from the linearization of a car-following model for human drivers, when taking into account their reaction time, which is known to be a cause of the stop-and-go traffic phenomenon. This leads to a non rational transfer function, implying nontrivial stability conditions which are explicitly given. They are in particular satisfied whenever string stability holds. It is also shown how this reaction time can introduce a regime of partial string stability, where the transfer function modulus remains smaller than 1, up to some critical frequency. Conditions are explored in the parameter space discriminating between 3 different regimes (stability, string stability, partial string stability).
7.10 Reflected brownian motion in a non convex cone
Participant: Guy Fayolle.
In collaboration with S. Franceschi (LMO, Paris-Saclay University) and K Raschel (CNRS, Tours University), we study the stationary reflected Brownian motion in a non-convex wedge, which, compared to its convex analogue model, has been much rarely analyzed in the probabilistic literature. Two approaches are proposed.- In 15, we prove that the stationary distribution can be found by solving a two dimensional vector boundary value problem (BVP) on a single curve (an hyperbola) for the associated Laplace transforms. The reduction to this kind of vector BVP seems to be quite new in the literature. As a matter of comparison, one single boundary condition is sufficient in the convex case. When the parameters of the model (drift, reflection angles and covariance matrix) are symmetric with respect to the bisector line of the cone, the model is reducible to a standard reflected Brownian motion in a convex cone. Finally, we construct a one-parameter family of distributions, which surprisingly provides, for any wedge (convex or not), one particular example of stationary distribution of a reflected Brownian motion.
- In 16, the main result is to show that the stationary distribution can indeed be obtained by solving a boundary value problem of the same kind as the one encountered in the quarter plane, up to various dualities and symmetries. The idea is to start from Fourier (and not Laplace) transforms, allowing to get a functional equation for a single function of two complex variables.
7.11 Random walks in orthants and lattice path combinatorics
Participant: Guy Fayolle.
In the revised version of the second edition of the book 2 (see also ), original methods were proposed to determine the invariant measure of random walks in the quarter plane with small jumps (size 1), the general solution being obtained via reduction to boundary value problems. Among other things, an important quantity, the so-called group of the walk, allows to deduce theoretical features about the nature of the solutions. In particular, when the order of the group is finite and the underlying algebraic curve is of genus 0 or 1, necessary and sufficient conditions have been given for the solution to be rational, algebraic or -finite (i.e. satisfying a linear differential equation). In this framework, number of difficult open problems related to lattice path combinatorics are currently being explored, both from theoretical and computer algebra points of view: concrete computation of the criteria, utilization of differential Galois theory, genus greater than 1 (i.e. when some jumps are of size ), etc. A recent topic (mentioned in 2019) deals with the connections between simple product-form stochastic networks (so-called Jackson networks) and explicit solutions of functional equations for counting lattice walks. Some partial extensions of are still under development.8 Bilateral contracts and grants with industry
8.1 Bilateral contracts with industry
Participants: Fawzi Nashashibi, Anne Verroust-Blondet.
Valeo Group: a very strong partnership is ongoing between Valeo and Inria. Several bilateral contracts were signed to conduct joint works on Driving Assistance, some of which Valeo is funding. This joint research includes:-
Several CIFRE like PhD thesis are under construction between Valeo and Inria.
Mr. Karim ESSALMI will join ASTRA in 2023 as a new PhD student working on Maneuver decision and Motion planning. Another PhD student will be working on Localization. A third student student will be working in the vision group within ASTRA.
- Othe PhD students and post-docs are jointly funded by Valeo and Inria while Mr. Nelson de Moura is hired as a 2-years post-doc thanks to the national Plan de relance Programme.
- Valeo is currently a major financing partner of the “GAT” international Chaire/JointLab in which Inria is a partner. The other partners are: UC Berkeley, Shanghai Jiao-Tong University, EPFL, IFSTTAR, Stellantis and SAFRAN.
- Technology transfer is also a major collaboration topic between ASTRA and Valeo as well as the development of a road automated prototype.
- Finally, Inria and Valeo are partners of the French project SAMBA (Sécurité Active et MoBilités Autonomes) including SAFRAN Group, Inria Paris, TwinswHeel, Soben, Stanley Robotics and EXPLEO.
9 Partnerships and cooperations
Participants: Yacine Ben Ameur, Ahn-Quan Cao, Raoul de Charette, Amina Ghoul, Yvan Lopes, Kathia Melbouci, Fawzi Nashashibi, Anne Verroust-Blondet.
9.1 International initiatives
9.1.1 Participation in other International Programs
Samuel de Champlain Québec-France collaboration program: “Vision par ordinateur en conditions difficiles”, 2018-2022, cooperation between Raoul de Charette and Jean-François Lalonde from Université Laval.
9.2 National initiatives
9.2.1 ANR
SIGHT
- Title: viSIon throuGH weaTher
- Instrument: ANR JCJC
- Duration: January 2021- December 2024
- Coordinator: Raoul de Charette
- Partners: Inria Paris, Université Laval, Mines ParisTech
- Inria contact: Raoul de Charette
- Abstract: SIGHT investigates invariant algorithms for complex weather conditions (rain, snow, hail). The project leverages un-/self-supervised algorithms with physic-guidance to model physically realistic weather, and learn weather-invariant features to improve vision algorithms.
9.2.2 ADEME – Bpifrance
SAMBA
- Title: Sécurité Active et MoBilités Autonomes (SAMBA2022)
- Instrument: Plan de soutien R&D automobile France
- Duration: September 2020 – January 2023
-
Grant:
902 302 €
- Partners: Valeo Group, SAFRAN Group, Inria Paris, TwinwHeel, Soben, Stanley Robotics, EXPLEO.
- Inria contact: Fawzi Nashashibi
- Abstract: The project aims to design active safety and autonomous mobility solutions that are affordable and can be deployed quickly, particularly on private vehicles. Technological solutions for new mobility services are proposed.
9.2.3 AMI – EquipEx+
TIRREX
- Inria is a major partner and beneficiary of the new EquipEx+ national initiative TIRREX (Infrastructure technologique pour la recherche d’excellence en robotique). RASTRA is an active participant of the “Autonomous Land Robotics” axis.
- Project start: Dec. 18, 2021
- Kick-off: Jan. 14, 2022
9.2.4 Competitivity Clusters
NextMove
(prev. MOV'EO): we are particularly involved in several technical committees like the DAS SMI (Systèmes de Mobilité Intelligents), for example.
Vedecom
(IEED): main Inria contributor and active participant to the CD2 domain dedicated to automated driving.
SystemX Institute
: close partnership, with the jointly supervised PhD thesis of Jiahao Zhang.
10 Dissemination
Participants: Patrick Pérez, Raoul de Charette, Guy Fayolle, Jean-Marc Lasgouttes, Gérard Le Lann, Fawzi Nashashibi, Fabio Pizzati, Patrick Pérez, Tiago Rocha Gonçalves, Anne Verroust-Blondet, Tuan-Hung Vu, Itheri Yahiaoui.
10.1 Promoting scientific activities
10.1.1 Scientific events: organisation
General chair, scientific chair
- Raoul de Charette, Fabio Pizzati, Patrick Pérez, Tuan-Hung Vu and Andrei Bursuc organized a Workshop on “Weakly Supervised Computer Vision” at the Deep Learning Indaba summer school, Tunis, Tunisia. Aug. 25th 2022. Approx. 80 participants.
Member of the organizing committees
- Raoul de Charette organized the ASTRA-vision group-retreat, Les Molières, France. Oct. 14-15th 2022.
- Raoul de Charette organized an Inria/Valeo.ai workshop, Paris, France. July 13th 2022.
10.1.2 Scientific events: selection
Member of the conference program committees
- Fawzi Nashashibi was a member of program committee of SMART 2022, IEEE ICCP 2022, MT-ITS 2023, PSIVT 2022 and VEHITS 2022.
- Anne Verroust-Blondet was a member of the program committee of CGI 2022.
Reviewer
- Raoul de Charette: CVPR, IROS, WACV, BMVC, MAIS.
- Fernando Garrido: ITSC, IV.
- Jean-Marc Lasgouttes: IV.
- Fawzi Nashashibi: regular reviewer of the following conferences: IEEE ICRA, IEEE/ISJ IROS, IEEE IV, IEEE ITSC, IEEE ICARCV, IEEE ICVES, ROBOVIS, VEHITS.
- Tiago Rocha Gonçalves: IEEE ICC, IFAC AAC, IEEE VTC.
- Anne Verroust-Blondet: ICRA, ITSC, IV.
- Tuan-Hung Vu: CVPR, NeurIPS.
10.1.3 Journal
Member of the editorial boards
- Guy Fayolle: associate editor of the journal Markov Processes and Related Fields.
- Fawzi Nashashibi: Associate editor of the journals IEEE Transactions on Intelligent Vehicles (T-IV), IEEE Transactions on Intelligent Transportation Systems (T-ITS); guest Editor of the IEEE Sensors journal. Special Issue on “The Application of Sensors in Autonomous Vehicles”.
- Anne Verroust-Blondet: associate editor of the journal The Visual Computer.
Reviewer - reviewing activities
- Guy Fayolle: reviewed several papers and books submitted for publication in some majors journals, e.g. Transactions of the American Mathematical Society, Markov Processes and Related Fields, Journal of Statistical Physics, Physica A, etc.
- Fernando Garrido: IEEE Transactions on Vehicular Technologies.
- Jean-Marc Lasgouttes: IEEE Transactions on Intelligent Transportation Systems, Physica A.
- Raoul de Charette was Associate Editor of International Conference on Intelligent Robots and Systems (IROS) 2022.
- Fawzi Nashashibi: Regular Associate Editor of IEEE-ICRA, IEEE-IROS, IEEE-IV, IEEE-ITSC, IEEE ICARCV, Transportation research Part C, Sensors
- Fawzi Nashashibi: regular reviewer of: IEEE Transactions on ITS, IEEE Transactions on IV, IEEE Transactions on Image Processing, IEEE Transactions on Robotics, IEEE Transactions on Vehicular Technologies, IEEE Transactions on Instrumentation and Measurement, Sensors, Transportation Research Part C, TRB...
- Tiago Rocha Gonçalves: IEEE Transactions on Intelligent Transportation Systems, IEEE Transactions on Vehicular Technology.
- Anne Verroust-Blondet: IEEE Transactions on Intelligent Vehicles.
- Tuan-Hung Vu: T-PAMI, IJCV.
10.1.4 Invited talks
- Raoul de Charette: talk at the Deep Learning Indaba workshop on Francophone Artificial Intelligence, Tunis, Tunisia. Aug. 28th 2022
- Raoul de Charette: talk at the CVPR Workshop on Vision For All Seasons, New Orleans, USA. June 22th 2022
- Raoul de Charette: talk at Université Laval, Quebec city, Canada. June 14th 2022
- Raoul de Charette: talk at “30 min of science”, Inria, Paris, France. June 2nd 2022
- Guy Fayolle: presentation of the article 17 at the LAREMA (Laboratoire Angevin de Recherche en Mathématiques), UMR CNRS 6093.
- Tiago Rocha Gonçalves: talk on “Robust control of platooning systems over imperfect wireless channels” at Labex DigiCosme C2-SyDiC (Contrôle et Certification de Performance de Systèmes Distribués Communicants), May 20, 2022.
10.1.5 Scientific expertise
- Guy Fayolle is a member of the MATHEXP team (Centre Inria de Saclay)
- Guy Fayolle is scientific advisor and associate researcher at the Robotics Laboratory of Mines ParisTech.
- Guy Fayolle is a member of the working group IFIP WG 7.3.
- Jean-Marc Lasgouttes: member of the Commission d'Évaluation of Inria.
- Jean-Marc Lasgouttes: member of the CRCN recruiting commissions of Inria research centers of Bordeaux.
- Fawzi Nashashibi: scientific reviewer of a FNR project (Luxembourg) under the CORE 2022 Programme.
- Fawzi Nashashibi: represents Inria at the Board of Governors of the VEDECOM Institute and at the Board of Governors of NextMove Competitiveness cluster.
- Fawzi Nashashibi: member of the international Automated Highway Board Committee of the TRB conference (AHB30).
- Fawzi Nashashibi: member of the Global Partnership on AI’s (GPAI) working group on Innovation and Commercialization.
- Fawzi Nashashibi: member of the SMI (Intelligent Mobility Systems) Working Group (NextMove).
- Fawzi Nashashibi: reviewer at the European Commission for Horizon Europe projects.
- Anne Verroust-Blondet is a reviewer for a FET-Open project (Horizon Europe programme for research and innovation).
10.1.6 Research administration
- Jean-Marc Lasgouttes: member of the Comité Technique Inria, of the Comité National Hygiène Sécurité et Conditions de Travail and of the Comité Local Hygiène Sécurité et Conditions de Travail of Inria Paris.
- Raoul de Charette is Principal Investigator of ANR JCJC SIGHT on Vision Through Weather.
10.2 Teaching - Supervision - Juries
10.2.1 Teaching
- Mastère : Raoul de Charette, “Scene Understanding with Computer Vision”, 20h, post master, Mines ParisTech, France.
- Seminar: Fernando Garrido, Paulo Resende, “decision-making and planning for automated driving”, 16 hours, Valeo Créteil, France.
- Engineering: Fernando Garrido, Paulo Resende, “decision-making and planning for automated driving”, 24 hours, École d'ingénieurs ESME Sudria, France.
- Engineering: Fernando Garrido, Paulo Resende, “decision-making and planning for autoamted driving”, 24 hours, Institut Supérieur de l'Automobile et des Transports (ISAT) à Nevers, France.
- Mastère: Jean-Marc Lasgouttes, “Introduction au Boosting”, 10.5h, Mastère Spécialisé Expert en sciences des données, INSA-Rouen, France.
- Engineering, 2nd year: Fawzi Nashashibi, “Image synthesis and 3D Infographics”, 12h, M2, INT Télécom SudParis, IMA4503 “Virtual and augmented reality for autonomy”.
- Master: Fawzi Nashashibi, “Perception and Image processing for Mobile Autonomous Systems”, 12h, M2, University of Evry.
- Engineering, 2nd year: Tiago Rocha Gonçalves, “Véhicule intelligent et communicant,”, 6h (TP), CentraleSupélec, France.
10.2.2 Supervision
- PhD in progress: Yacine Ben Ameur, UPMC Sorbonne University, “Trajectories planning and decision systems for autonomous vehicle navigation in complex environments”, October 2021, supervisor Fawzi Nashashibi, co-supervisor: Anne Verroust-Blondet.
- PhD in progress: Ahn Quan Cao, PSL Research University, “Unsupervised 3D scene understanding from image(s)”, March 2021, supervisor: Raoul de Charette.
- PhD in progress: Mohammad Fahes, Mines-ParisTech, “Crowdsourced Unsupervised Learning in Adverse Conditions”, October 2022, supervisor: Raoul de Charette, co-supervisors: Tuan-Hung Vu, Andrei Bursuc, Patrick Pérez.
- PhD in progress: Amina Ghoul, UPMC Sorbonne University, “Trajectory prediction in an urban environment”, May 2021, supervisor Fawzi Nashashibi, co-supervisors: Anne Verroust-Blondet, Itheri Yahiaoui.
- PhD in progress: Ivan Lopes, PSL Research University, “Physic-guided learning for vision in adverse weather conditions”, November 2021, supervisor: Raoul de Charette.
- PhD in progress: Noël Nadal, “Cartographie et localisation crowdsourcées pour la conduite autonome en environnement urbain”, October 2022, co-supervisors: Fawzi Nashashibi and Jean-Marc Lasgouttes.
- PhD: Fabio Pizzati, “Style transfer and domain adaptation for semantic segmentation”, PSL Research University and University of Bologna, November 2022, co-supervisors: Andrea Prati and Raoul de Charette.
- PhD: Renaud Poncelet, “Navigation autonome en présence d'obstacles fortement dynamiques au mouvement incertain”, UPMC Sorbonne University, December 2022, supervisor: Anne Verroust-Blondet, co-supervisor: Fawzi Nashashibi.
- PhD in progress: Jiahao Zhang, "Misbehavior detection for collective perception in Intelligent Transportation System", October 2021, UPMC Sorbonne University, supervisor Fawzi Nashashibi, co-supervisor: Ines Ben Jemaa.
10.2.3 Juries
- Raoul de Charette: reviewer of PhD thesis of Michaël Ramamonjisoa, “3D Scene Reconstruction from Images”, Ecole Nationale des Ponts et Chaussées, Nov. 22nd 2022.
- Raoul de Charette: reviewer of mid-term PhD for Antonin Vobecky, “Weakly supervised learning for visual recognition in the autonomous driving context”, Czech Technical University in Prague, March 31st 2022.
- Raoul de Charette: member of the researcher recruitment committee, Telecom Paris, France. May 11th 2022.
- Fawzi Nashashibi: president in the jury of the HdR thesis of Redouane Khemmar, “Contribution à la perception d'environnement pour la smart mobilité”, ESIGELEC-Normandie Université, November 28, 2022, Saint-Etienne du Rouvray.
- Fawzi Nashashibi: reviewer in the jury of the HdR thesis of Nicolas Rivière, “Interactions matière-rayonnement et nouveaux concepts d’imageurs laser 3D”, ONERA-Université de Toulouse, November 16, 2022, Toulouse.
- Fawzi Nashashibi: examiner in the jury of the HdR thesis of Raoul de Charette, “Vision for Scene Understanding”, Inria-Sorbonne Université, January 25, 2022, Paris.
- Fawzi Nashashibi: president in the jury of the PhD thesis of Alexis Stoven-Dubois, “Robust Crowdsourced Mapping for Landmark-based Vehicle Localization”, VEDECOM-Université Clermont Auvergne, March 2, 2022, Versailles.
- Fawzi Nashashibi: president of the jury of the PhD thesis of Mr. Robin Condat, “Contribution to the improvement of the robustness of perception systems based on multimodal neural networks”, LITIS-Normandie Université, July 8, 2022, Saint-Etienne du Rouvray.
- Fawzi Nashashibi: reviewer in the jury of the PhD thesis of Mr. Hussam Atoui, “Switching/Interpolating LPV Control based on Youla-Kucera Parameterization: Application to Autonomous Vehicles”, Grenoble INP-Université Grenoble Alpes, October 20, 2022, Grenoble.
- Fawzi Nashashibi: reviewer in the jury of the PhD thesis of Mr. Tristan Klempka, “Prédiction de déplacements de véhicules connectés en interaction pour la caractérisation de situations à risques”, LAAS-Université de Toulouse, November 14, 2022, Toulouse.
- Fawzi Nashashibi: reviewer in the jury of the PHD thesis of Mrs. Andra Petrovai, “Deep Learning-based Visual Perception for Autonomous Driving”, Technical University of Cluj-Napoca, December 20, 2022, Cluj (Romania).
- Anne Verroust-Blondet: jury member of the PhD thesis of Paul Guelorget, “Apprentissage actif pour la détection d’objets d’intérêt opérationnel dans les contenus multimédia”, December 2022, Institut Polytechnique de Paris.
- Anne Verroust-Blondet: jury member of the PhD thesis of Yanis Marchand, “Scaling up and evaluating surface reconstruction from point clouds of open scenes”, November 2022, Université Gustave Eiffel.
- Tuan-Hung Vu: reviewer of PhD thesis of Antoine Saporta, “Domain Adaptation for Urban Scene Segmentation”, Sorbonne Université, April 14th 2022.
10.3 Popularization
10.3.1 Education
- Raoul de Charette: “Introduction à l'intelligence artificielle”, 10h, primary school, Puck et Ribambelle, Castelnau-le-Lez, France.
10.3.2 Interventions
- Raoul de Charette: “Nvidia introduit les modèles Transformer dans l'auto”, Industrie & Technologies. November 2022 (journalist: Frédéric Monflier)
- Raoul de Charette, Fawzi Nashashibi: “Voitures autonomes: est-ce bien raisonnable de les faire rouler ?”, Epsiloon. July 2022 (journalist: Muriel Valin)
- Fawzi Nashashibi: “Voitures autonomes sur les routes françaises cet été : révolution ou désillusion”, Challenges. July 2022 (journalist: Quentin Halbout).
- Fawzi Nashashibi: public presentation during the NUMOK Festival in Paris “Les nouvelles mobilités du 21ème siècles et leurs enjeux”, Bibliothèque Arthur Rimbaud, April 15, 2022, Paris.
11 Scientific production
11.1 Major publications
- 1 inproceedings2D SLAM Correction Prediction in Large Scale Urban Environments.ICRA 2018 - International Conference on Robotics and Automation 2018Brisbane, AustraliaMay 2018
- 2 bookRandom Walks in the Quarter Plane: Algebraic Methods, Boundary Value Problems, Applications to Queueing Systems and Analytic Combinatorics.40Probability Theory and Stochastic ModellingSpringer International PublishingFebruary 2017, 255
- 3 articleShort-term Forecasting of Urban Traffic using Spatio-Temporal Markov Field.IEEE Transactions on Intelligent Transportation Systems2382022, 10858-10867
- 4 articleA Review of Motion Planning Techniques for Automated Vehicles.IEEE Transactions on Intelligent Transportation SystemsApril 2016
- 5 inproceedingsPhysics-Based Rendering for Improving Robustness to Rain.ICCV 2019 - International Conference on Computer VisionSeoul, South KoreaOctober 2019
- 6 articleCross-Modal Learning for Domain Adaptation in 3D Semantic Segmentation.IEEE Transactions on Pattern Analysis and Machine Intelligence452March 2022, 1533-1544
- 7 reportCyberphysical Constructs and Concepts for Fully Automated Networked Vehicles.RR-9297INRIA Paris-RocquencourtOctober 2019
- 8 articleAdvances in Youla-Kucera parametrization: A Review.Annual Reviews in ControlJune 2020
- 9 articleAttention Based Vehicle Trajectory Prediction.IEEE Transactions on Intelligent Vehicles612021, 175-185
- 10 inproceedingsNon-local Social Pooling for Vehicle Trajectory Prediction.Intelligent Vehicles Symposium (IV)Paris, FranceJune 2019
- 11 inproceedingsCoMoGAN: continuous model-guided image-to-image translation.CVPR 2021 - IEEE Conference on Computer Vision and Pattern RecognitionIEEE Conference on Computer Vision and Pattern RecognitionOnline, FranceJune 2021
- 12 inproceedingsModel-based occlusion disentanglement for image-to-image translation.ECCV 2020 - European Conference on Computer VisionECCV 2020Glasgow / Virtual, United KingdomAugust 2020
- 13 article3D Semantic Scene Completion: a Survey.International Journal of Computer Vision2021
- 14 articleRain Rendering for Evaluating and Improving Robustness to Bad Weather.International Journal of Computer VisionSeptember 2020
11.2 Publications of the year
International journals
National journals
International peer-reviewed conferences
Conferences without proceedings
Doctoral dissertations and habilitation theses
Reports & preprints
11.3 Cited publications
- 34 inproceedingsDeep evidential regression.Advances in Neural Information Processing Systems (NeurIPS)2020
- 35 miscSelf-Driving like a Human driver instead of a Robocar: Personalized comfortable driving experience for autonomous vehicles.2020
- 36 articleIntegrity Monitoring of Multimodal Perception System for Vehicle Localization.Sensors2020
- 37 phdthesisPlanification de trajectoire dans un environnement peu contraint et fortement dynamique.Sorbonne Université2019
- 38 inproceedingsRIS: A Framework for Motion Planning Among Highly Dynamic Obstacles.International Conference on Control, Automation, Robotics and Vision (ICARCV)2018
- 39 inproceedingsSemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences.International Conference~on Computer Vision (ICCV)2019
- 40 inproceedingsThis dataset does not exist: training models from generated images.International Conference on Acoustics, Speech and Signal Processing (ICASSP)2020
- 41 inproceedingsSeeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather.Conference on Computer Vision and Pattern Recognition (CVPR)2020
- 42 articleConvPoint: Continuous convolutions for point cloud processing.Computers & Graphics2020
- 43 inproceedingsFKAConv: Feature-Kernel Alignment for Point Cloud Convolution.Asian Conference on Computer Vision (ACCV)2020
- 44 articleA Taxonomy and Review of Algorithms for Modeling and Predicting Human Driver Behavior.CoRR2020
- 45 articleBUDA: Boundless Unsupervised Domain Adaptation in Semantic Segmentation.arXiv preprint arXiv:2004.011302020
- 46 inproceedingsPLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving.Conference on Robot Learning (CoRL)2020
- 47 miscConditional Vehicle Trajectories Prediction in CARLA Urban Environment.2019
- 48 inproceedingsnuScenes: A Multimodal Dataset for Autonomous Driving.Conference on Computer Vision and Pattern Recognition (CVPR)2020
- 49 articlePCAM: Product of Cross-Attention Matrices for Rigid Registration of Point Clouds.Submitted for publication2021
- 50 inproceedingsArgoverse: 3D Tracking and Forecasting with Rich Maps.Conference on Computer Vision and Pattern Recognition (CVPR)2019
- 51 articleMind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation.arXiv preprint arXiv:2003.037872020
- 52 articleDeepDriving: Learning Affordance for Direct Perception in Autonomous Driving.CoRR2015
- 53 inproceedingsStargan v2: Diverse image synthesis for multiple domains.Conference on Computer Vision and Pattern Recognition (CVPR)2020
- 54 articlePioneering Driverless Electric Vehicles in Europe: The City Automated Transport System (CATS).Transportation Research Procedia13Towards future innovative transport: visions, trends and methods, 43rd European Transport Conference Selected Proceedings2016, 30--39URL: http://www.sciencedirect.com/science/article/pii/S2352146516300047
- 55 inproceedingsA path planner for autonomous driving on highways using a human mimicry approach with Binary Decision Diagrams.European Control Conference (ECC)2015
- 56 inproceedingsMulti-Criteria Decision Making for Autonomous Vehicles using Fuzzy Dempster-Shafer Reasoning.Intelligent Vehicles Symposium (IV)2018
- 57 inproceedingsUs highway 101 dataset..Federal Highway Administration (FHWA), Tech. Rep. FHWA-HRT07-0302007
- 58 inproceedingsUs highway i-80 dataset..Federal Highway Administration (FHWA), Tech. Rep. FHWA-HRT-06-1372006
- 59 incollectionAddressing Failure Prediction by Learning Model Confidence.Advances in Neural Information Processing Systems (NeurIPS)Curran Associates, Inc.2019
- 60 inproceedingsStabilizing traffic flow via a single autonomous vehicle: possibilities and limitations.Intelligent Vehicles Symposium (IV)2017
- 61 articleReducing fuel consumption in platooning systems through reinforcement learning.IFAC-PapersOnLine55156th IFAC Conference on Intelligent Control and Automation Sciences ICONS 20222022, 99-104URL: https://www.sciencedirect.com/science/article/pii/S2405896322010266
- 62 articleAsymptotics and scalings for large closed product-form networks via the Central Limit Theorem.Markov Processes and Related Fields221996, 317-348
- 63 phdthesisControl architecture for adaptive and cooperative car-following.Université Paris sciences et lettresDecember 2018
- 64 inproceedingsUsing Fractional Calculus for Cooperative Car-Following Control.Intelligent Transportation Systems Conference 2016IEEERio de Janeiro, BrazilNovember 2016
- 65 inproceedingsA Cooperative Advanced Driver Assistance System to mitigate vehicular traffic shock waves.INFOCOM - Conference on Computer Communications2014
- 66 articleEncoding the latent posterior of Bayesian Neural Networks for uncertainty quantification.arXiv preprint arXiv:2012.028182020
- 67 inproceedingsTRADI: Tracking deep neural network weight distributions.European Conference on Computer Vision (ECCV)2020
- 68 inproceedingsSpatial and Temporal Analysis of Traffic States on Large Scale Networks.Intelligent Transportation Systems Conference (ITSC)2010
- 69 inproceedingsA queueing theory approach for a multi-speed exclusion process..Traffic and Granular Flow '07Springer2009, 129-138URL: http://hal.archives-ouvertes.fr/hal-00175628/en/
- 70 techreportSpatio-temporal Probabilistic Short-term Forecasting on Urban Networks.INRIA2018
- 71 articleLearning Multiple Belief Propagation Fixed Points for Real Time Inference.Physica A: Statistical Mechanics and its Applications2010
- 72 inproceedingsA belief propagation approach to traffic prediction using probe vehicles.Intelligent Transportation Systems Conference (ITSC)2007
- 73 articleOne-dimensional Particle Processes with Acceleration/Braking Asymmetry.Journal of Statistical Physics1476June 2012, 1113-1144
- 74 inproceedingsThe Fundamental Diagram on the Ring Geometry for Particle Processes with Acceleration/Braking Asymmetry.TGF'11 - Traffic and Granular FlowMoscowDecember 2011, URL: http://hal.inria.fr/hal-00646988
- 75 phdthesisTwo-staged local trajectory planning based on optimal pre-planned curves interpolation for human-like driving in urban areas.Université Paris sciences et lettres2018
- 76 articleA Two-Stage Real-Time Path Planning : Application to the Overtaking Manuever.IEEE AccessJuly 2020
- 77 inproceedingsImplementable Ethics for Autonomous Vehicles.Autonomes Fahren: Technische, rechtliche und gesellschaftliche AspekteBerlin, HeidelbergSpringer Berlin Heidelberg2015, 87--102URL: https://doi.org/10.1007/978-3-662-45854-9_5
- 78 inproceedingsOn a weaker notion of ring stability for mixed traffic with human-driven and autonomous vehicles.Conference on Decision and Control (CDC)2019
- 79 inproceedings3D Semantic Segmentation with Submanifold Sparse Convolutional Networks.Conference on Computer Vision and Pattern Recognition (CVPR)2018
- 80 inproceedingsOn-Road Motion Planning for Autonomous Vehicles.Intelligent Robotics and Applications - International Conference, ICIRALecture Notes in Computer ScienceSpringer2012
- 81 inproceedingsPlatoon Route Optimization for Picking up Automated Vehicles in an Urban Network.21st IEEE International Conference on Intelligent Transportation Systems2018 IEEE 21th International Conference on Intelligent Transportation Systems (ITSC)Maui, United StatesNovember 2018
- 82 articleA game theory-based route planning approach for automated vehicle collection.Concurrency and Computation: Practice and ExperienceFebruary 2021
- 83 articleLocalization Integrity for Intelligent Vehicles Through Fault Detection and Position Error Characterization.Transactions on Intelligent Transportation Systems (T-ITS)2020
- 84 articleAttgan: Facial attribute editing by only changing what you want.Transactions on Image Processing2019
- 85 inproceedingsFailure prediction for autonomous driving.Intelligent Vehicles Symposium (IV)2018
- 86 inproceedingsOn Ambiguities in Feature-Based Vehicle Localization and their A Priori Detection in Maps.Intelligent Vehicles Symposium (IV)2019
- 87 articleRobust Tensor Recovery with Fiber Outliers for Traffic Events.Trans. Knowl. Discov. Data2021
- 88 miscSAE Standards: J3016 automated-driving graphic..
- 89 articleComputer vision for autonomous vehicles: Problems, datasets and state of the art.Foundations and Trends in Computer Graphics and Vision2020
- 90 inproceedingsSparse and dense data with CNNs: Depth completion and semantic segmentation.International Conference on 3D Vision (3DV)2018
- 91 inproceedingsxmuda: Cross-modal unsupervised domain adaptation for 3D semantic segmentation.Conference on Computer Vision and Pattern Recognition (CVPR)2020
- 92 articleSupervised contrastive learning.arXiv preprint arXiv:2004.113622020
- 93 inproceedingsThe highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems.Intelligent Transportation Systems Conference (ITSC)2018
- 94 inproceedingsMotion planning for urban driving using RRT.International Conference on Intelligent Robots and Systems (IROS)2008
- 95 miscMoral Machine - Human Perspectives on Machine Ethics.
- 96 inproceedingsSurface reconstruction from 3D line segments.International Conference on 3D Vision (3DV)2019
- 97 inproceedingsGlobal On-line Optimization for Charging Station Allocation.Intelligent Transportation Systems Conference, ITSC 2015IEEE2015
- 98 inproceedingsToward Efficient Simulation Platform for Platoon Communication in Large Scale C-ITS Scenarios.IEEE International Symposium on Networks, Computers and CommunicationsRoma, ItalyJune 2018
- 99 inproceedingsAutomotive localization integrity using proprioceptive and pseudo-ranges measurements.Accurate Localization for Land TransportationLes Collections de l'INRETS2009
- 100 inproceedingsVehicle Localization Integrity Based on Trajectory Monitoring.Intelligent Robots and Systems (IROS)2009
- 101 inproceedingsSemantic Palette: Guiding Scene Generation with Class Proportions.Conference on Computer Vision and Pattern Recognition (CVPR)2021
- 102 inproceedingsPseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.Workshop on challenges in representation learning (ICML)2013
- 103 articleA survey on motion prediction and risk assessment for intelligent vehicles. ROBOMECH Journal112014, URL: http://www.robomechjournal.com/content/1/1/1
- 104 phdthesisSafe and Efficient Reinforcement Learning for Behavioural Planning in Autonomous Driving.Université de Lille2020
- 105 articlePointcnn: Convolution on x-transformed points.Advances in Neural Information Processing Systems (NeurIPS)2018
- 106 inproceedingsAn Explicit Decision Tree Approach for Automated Driving.ASME 2017 Dynamic Systems and Control ConferenceDynamic Systems and Control Conference2017
- 107 articleGame-Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems.CoRR2016
- 108 articleVisual Measurement Integrity Monitoring for UAV Localization.CoRR2019
- 109 articlePlanning Long Dynamically Feasible Maneuvers for Autonomous Vehicles.International Journal of Robotics Research2009
- 110 inproceedingsOpen compound domain adaptation.Conference on Computer Vision and Pattern Recognition (CVPR)2020
- 111 phdthesisYoula-Kucera based multi-objective controllers : Application to autonomous vehicles.Université Paris sciences et lettresDecember 2020
- 112 inproceedingsPredictive uncertainty estimation via prior networks.Advances in Neural Information Processing Systems (NeurIPS)2018
- 113 inproceedingsGMRF Estimation under Topological and Spectral Constraints.ECML PKDD Proceedings,Part II2014
- 114 articleLatent binary MRF for online reconstruction of large scale systems.Annals of Mathematics and Artificial Intelligence2015
- 115 phdthesisDeep Learning based Trajectory Prediction for Autonomous Vehicles.Sorbonne UniversitéJune 2021
- 116 articleMulti-Head Attention with Joint Agent-Map Representation for Trajectory Prediction in Autonomous Driving.CoRR2020
- 117 inproceedingsRelational Recurrent Neural Networks For Vehicle Trajectory Prediction.Intelligent Transportation Systems Conference (ITSC)IEEE2019
- 118 articleGenerative Zero-Shot Learning for Classification and Semantic Segmentation of 3D Point Clouds.Submitted for publication2021
- 119 inproceedingsEthical decision making for autonomous vehicles.IEEE Symposium on Intelligent VehicleLas Vegas (virtual), United StatesOctober 2020
- 122 inproceedingsRule-Based Highway Maneuver Intention Recognition.International Conference on Intelligent Transportation Systems (ITSC)2015
- 123 inproceedingsCan you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift.Advances in Neural Information Processing Systems (NeurIPS)2019
- 124 articleModeling and Nonlinear Adaptive Control for Autonomous Vehicle Overtaking.IEEE Transactions on Intelligent Transportation Systems154August 2014, 14
- 125 inproceedingsSaturated Feedback Control for an Automated Parallel Parking Assist System.13th International Conference on Control, Automation, Robotics and Vision (ICARCV'14)Singapore, SingaporeDecember 2014
- 126 articleCanadian adverse driving conditions dataset.International Journal of Robotics Research2020
- 127 inproceedingsDomain bridge for unpaired image-to-image translation and unsupervised domain adaptation.Winter Conference on Applications of Computer Vision (WACV)2020
- 128 phdthesisExploring domain-informed and physics-guided learning in image-to-image translation.Mines ParisNovembre 2022
- 129 articleManiFest: Manifold Deformation for Few-shot Image Translation.arXiv2021
- 130 inproceedingsFLOT: Scene Flow on Point Clouds Guided by Optimal Transport.European Conference on Computer Vision (ECCV)2020
- 131 articleStyleLess layer: Improving robustness for real-world driving.arXiv preprint arXiv:2103.139052021
- 132 phdthesis3D Scene Reconstruction and Completion for Autonomous Driving.Sorbonne UniversitéJuly 2021
- 133 inproceedingsLMSCNet: Lightweight Multiscale 3D Semantic Completion.International Conference on 3D Vision (3DV)2020
- 134 articleHuman motion trajectory prediction: a survey.The International Journal of Robotics Research2020
- 135 articleACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding.arXiv preprint arXiv:2104.133952021
- 136 inproceedingsExactly Solvable Stochastic Processes for Traffic Modelling.25th International Symposium on Computer and Information Sciences - ISCIS 2010Erol Gelenbe, Ricardo Lent, Georgia SakellariLondres, Royaume-UniSeptember 2010, 75-78URL: http://hal.inria.fr/inria-00533154
- 137 inproceedingsSemantic scene completion from a single depth image.Conference on Computer Vision and Pattern Recognition (CVPR)2017
- 138 inproceedingsA Collaborative Framework for High-Definition Mapping.2019 IEEE Intelligent Transportation Systems Conference (ITSC)2019, 1845-1850
- 139 inproceedingsScalability in perception for autonomous driving: Waymo open dataset.Conference on Computer Vision and Pattern Recognition (CVPR)2020
- 140 inproceedingsKpconv: Flexible and deformable convolution for point clouds.International Conference on Computer Vision (ICCV)2019
- 141 inproceedingsUnsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings.Winter Conference on Applications of Computer Vision (WACV)2021
- 142 inproceedingsAdvent: Adversarial entropy minimization for domain adaptation in semantic segmentation.Conference on Computer Vision and Pattern Recognition (CVPR)2019
- 143 inproceedingsDADA: Depth-Aware Domain Adaptation in Semantic Segmentation.International Conference on Computer Vision (ICCV)2019
- 144 articleA Reinforcement Learning Based Approach for Automated Lane Change Maneuvers.CoRR2018
- 145 inproceedingsDeep network interpolation for continuous imagery effect transition.Conference on Computer Vision and Pattern Recognition (CVPR)2019
- 146 miscAssistance á la conduite d'un véhicule automobile.May 2020
- 147 inproceedingsStabilizing Traffic with Autonomous Vehicles.International Conference on Robotics and Automation (ICRA)2018
- 148 inproceedingsFew-shot object detection and viewpoint estimation for objects in the wild.European Conference on Computer Vision (ECCV)2020
- 149 inproceedingsPose from shape: Deep pose estimation for arbitrary 3D objects.British Machine Vision Conference (BMVC)2019
- 150 inproceedingsWoodscape: A multi-task, multi-camera fisheye dataset for autonomous driving.International Conference on Computer Vision (ICCV)2019
- 151 inproceedingsUniversal domain adaptation.Conference on Computer Vision and Pattern Recognition (CVPR)2019
- 152 inproceedingsBdd100k: A diverse driving video database with scalable annotation tooling.Conference on Computer Vision and Pattern Recognition (CVPR)2020
- 153 articleINTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps.arXiv:1910.03088 [cs, eess]2019
- 154 articleTNT: Target-driveN Trajectory Prediction.CoRR2020
- 155 articleMaking Bertha Drive—An Autonomous Journey on a Historic Route.Intelligent Transportation Systems Magazine (ITS Magazine)2014