The focus of the project-team is to develop the technologies linked to
Intelligent Transportation Systems (ITS) with the objective to achieve
sustainable mobility by the improvement of the safety, the efficiency of road transport
according to the recent “Intelligent Vehicle Initiative” launched by
the DG Information Society of the European Commission (for “Smarter,
Cleaner, and Safer Transport”).
More specifically, we want to develop, demonstrate and test some
innovative technologies under the framework of LaRA,
“La Route Automatisée
These developments are all based on the sciences and technologies of information and communications (STIC) and have the objective to bring significant improvements in the road transport sector through incremental or breakthrough innovations. The project-team covers fundamental R&D work on key technologies, applied research to develop techniques that solve specific problems, and demonstrator activities to evaluate and disseminate the results.
The scientific approach is focused on the analysis and optimization of road transport systems through a double approach:
the control of individual road vehicles to improve locally their efficiency and safety,
the design and control of large transportation systems.
The first theme on vehicle control is broadly based on signal processing and data fusion in order to have a better machine understanding of the situation a vehicle may encounter, and on robotics techniques to control the vehicle in order to help (or replace) the driver to avoid accidents while improving the performance of the vehicle (speed, comfort, mileage, emissions, noise...). The theme also includes software techniques needed to develop applications in a real-time distributed and complex environment with extremely high safety standards. In addition, data must be exchanged between the vehicles; communication protocols have thus to be adapted to and optimized for vehicular networks characteristics (e.g. mobility, road safety requirements, heterogeneity, density), and communication needs (e.g. network latency, quality of service, network security, network access control).
The second theme on modeling and control of large transportation systems is also largely dependent on STIC. The objective, there, is to improve significantly the performance of the transportation system in terms of throughput but also in terms of safety, emissions, energy while minimizing nuisances. The approach is to act on demand management (e.g. through information, access control or road charging) as well as on the vehicles coordination. Communications technologies are essential to implement these controls and are an essential part of the R&D, in particular in the development of technologies for highly dynamic networks.
In order to address those issues simultaneously, RITS is organized into three research axes, each of which being driven by a separate sub-team. The first axis addresses the traditional problem of vehicle guidance and autonomous navigation. The second axis focuses on the large scale deployment and the traffic analysis and modeling. The third axis deals with the problem of telecommunications from two points of view:
Technical: design certified architectures enabling safe vehicle-to-vehicle and vehicle-to-vehicle communications obeying to standards and norm;
Fundamental, design and develop appropriate architectures capable of handling thorny problems of routing and geonetworking in highly dynamic vehicular networks and high speed vehicles.
Of course, these three research sub-teams interact to build intelligent cooperative mobility systems.
There are three basic ways to improve the safety of road vehicles and these ways are all of interest to the project-team. The first way is to assist the driver by giving him better information and warning. The second way is to take over the control of the vehicle in case of mistakes such as inattention or wrong command. The third way is to completely remove the driver from the control loop.
All three approaches rely on information processing. Only the last two involve the control of the vehicle with actions on the actuators, which are the engine power, the brakes and the steering. The research proposed by the project-team is focused on the following elements:
perception of the environment,
planning of the actions,
real-time control.
Either for driver assistance or for fully automated guided vehicle purposes, the first step of any robotic system is to perceive the environment in order to assess the situation around itself. Proprioceptive sensors (accelerometer, gyrometer,...) provide information about the vehicle by itself such as its velocity or lateral acceleration. On the other hand, exteroceptive sensors, such as video camera, laser or GPS devices, provide information about the environment surrounding the vehicle or its localization. Obviously, fusion of data with various other sensors is also a focus of the research.
The following topics are already validated or under development in our team:
relative ego-localization with respect to the infrastructure, i.e. lateral positioning on the road can be obtained by mean of vision (lane markings) and the fusion with other devices (e.g. GPS);
global ego-localization by considering GPS measurement and proprioceptive information, even in case of GPS outage;
road detection by using lane marking detection and navigable free space;
detection and localization of the surrounding obstacles (vehicles, pedestrians, animals, objects on roads, etc.) and determination of their behavior can be obtained by the fusion of vision, laser or radar based data processing;
simultaneous localization and mapping as well as mobile object tracking using laser-based and stereovision-based (SLAMMOT) algorithms.
Scene understanding is a large perception problem. In this research axis we have decided to use only computer vision as cameras have evolved very quickly and can now provide much more precise sensing of the scene, and even depth information. Two types of hardware setups were used, namely: monocular vision or stereo vision to retrieve depth information which allow extracting geometry information.
We have initiated several works:
estimation of the ego motion using monocular scene flow. Although in the state of the art most of the algorithms use a stereo setup, researches were conducted to estimate the ego-motion using a novel approach with a strong assumption.
bad weather conditions evaluations. Most often all computer vision algorithms work under a transparent atmosphere assumption which assumption is incorrect in the case of bad weather (rain, snow, hail, fog, etc.). In these situations the light ray are disrupted by the particles in suspension, producing light attenuation, reflection, refraction that alter the image processing.
deep learning for object recognition. New works are being initiated in our team to develop deep learning recognition in the context of heterogeneous data.
deep learning for vehicle motion prediction.
From the understanding of the environment, thanks to augmented perception, we have either to warn the driver to help him in the control of his vehicle, or to take control in case of a driverless vehicle. In simple situations, the planning might also be quite simple, but in the most complex situations we want to explore, the planning must involve complex algorithms dealing with the trajectories of the vehicle and its surroundings (which might involve other vehicles and/or fixed or moving obstacles). In the case of fully automated vehicles, the perception will involve some map building of the environment and obstacles, and the planning will involve partial planning with periodical recomputation to reach the long term goal. In this case, with vehicle to vehicle communications, what we want to explore is the possibility to establish a negotiation protocol in order to coordinate nearby vehicles (what humans usually do by using driving rules, common sense and/or non verbal communication). Until now, we have been focusing on the generation of geometric trajectories as a result of a maneuver selection process using grid-based rating technique or fuzzy technique. For high speed vehicles, Partial Motion Planning techniques we tested, revealed their limitations because of the computational cost. The use of quintic polynomials we designed, allowed us to elaborate trajectories with different dynamics adapted to the driver profile. These trajectories have been implemented and validated in the JointSystem demonstrator of the German Aerospace Center (DLR) used in the European project HAVEit, as well as in RITS's electrical vehicle prototype used in the French project ABV. HAVEit was also the opportunity for RITS to take in charge the implementation of the Co-Pilot system which processes perception data in order to elaborate the high level command for the actuators. These trajectories were also validated on RITS's cybercars. However, for the low speed cybercars that have pre-defined itineraries and basic maneuvers, it was necessary to develop a more adapted planning and control system. Therefore, we have developed a nonlinear adaptive control for automated overtaking maneuver using quadratic polynomials and Lyapunov function candidate and taking into account the vehicles kinematics. For the global mobility systems we are developing, the control of the vehicles includes also advanced platooning, automated parking, automated docking, etc. For each functionality a dedicated control algorithm was designed (see publication of previous years).
Wireless communications are expected to play an essential role in ensuring road safety, road efficiency, and driving comfort o. Road safety applications often require relatively short response time and reliable information exchange between neighboring vehicles and road-side units in any road density condition. Because of the performance of the existing radio communications technology largely degrades with the increase of the traffic density, the challenge of designing wireless communications solution suitable for safety applications is enabling reliable communications in highly dense scenarios.
To investigate this open problem and trade-off situations, RITS has been working on medium access control design for the IEEE 802.11p radio communication and the deployment of supportive solutions such as visible light communications and testing the use-cases for extreme traffic conditions and highly dense scenarios. The works have been carried out considering the vehicle behavior such as autonomous and connected vehicles merging, sharing, and convoy forming as platoon scenarios with considering the hard-safety requirements.
Unlike many of the road safety applications, the applications regarding road efficiency and comfort of road users, often require connectivity to the Internet. Based on our expertise in both Internet-based communications in the mobility context and in ITS, we are investigating the use of IPv6 (Internet Protocol version 6 which is going to replace the current version, IPv4, IoT) for vehicular communications, in a combined architecture supporting both V2V and V2I.
Communication contributions at RITS team have been working on channel modeling for both radio and visible light communications, and design of communications mechanisms, especially for security, service discovery, multicast, and Geo-Cast message delivery, and access point selection.
RITS-team has one of the latest certified standard communication hardware and tools supported by the partnership with the YoGoKo Company. All platforms (connected and autonomous vehicles) are equipped with state-of-art communication units On-Board-Units (OBU), where the Rocquencourt site equipped with two stationary Road-Side-Units (RSU) enabling all kind of tests and projects requirements
Below follows a more detailed description of the related research issues.
The technological advances of autonomous and connected road vehicles have been shown an accelerating pace in the recent years. On the other hand, the regulations for autonomous, or driverless, road vehicles across Europe still deserve much attention and discussion
Therefore, RITS-Inria team plays a key element in one of the European demonstration-based projects (AUTOC-ITS), which aims to contribute to the regulation study for interoperability in the adoption of autonomous driving in European urban nodes. The regulation study done by RITS team and project partners meant to conduct a deployment of Cooperative Intelligent Transport Systems (C-ITS) in Europe by enhancing interoperability for autonomous vehicles . The project activities and RITS contributions will also boost the role of C-ITS as the primary catalyst for any future implementation of autonomous driving scenarios in Europe. The final demonstration of different European partners will require the implementation and preparations of three pilots sites in three major European cities: Paris, Madrid, and Lisbon. Pilot locations in these major cities are chosen to be located along the European Atlantic Corridor for interoperability evaluation.
RITS-Inria is coordinating the French contribution by evaluating the deployment of C-ITS services in the A13-Paris, which belongs to the French part of the Atlantic Corridor.
Team Core contributions:
Provide up to date feedback to contribute to the present EU and international regulations on autonomous vehicles.
Build and evaluate the pilots experimentally by deploying fully autonomous vehicles and a Cooperative Intelligent Transport Systems (C-ITS).
Define and evaluate a safety autonomous driving services, such as:
Roadworks warning.
Weather conditions.
Other hazardous notifications.
Define and perform communication interoperability tests between deferent partners for different scenarios, messaging and hardware to ensure the compatibility in using the IEEE 802.11p standard.
Study the extension of the results on large-scale deployment in other European countries.
Contribute to the European standards organizations such as C-Roads, C-ITS platforms.
AUTOC-ITS project brings the road authorities from France, Span, and Portugal (DGT, ANSR, SANEF) and C-ITS experts from research institutes and universities (Inria, INDRA, UPM, UC, IPN) to carry out a cooperative work and contributes to the C-ITS Platform by bringing answers to the field of automation driving.
The development work and generating proper components to facilitate communication requirements and to be deployed in different projects scenarios is one of the main ongoing activities by all RITS team members.
There are continuous activities on both theoretical modeling and experimental evaluation of the radio channel characteristics in vehicular networks, especially the radio quality, channel congestion, load allocations, congestion, and bandwidth availability.
Based on our previous expertise and studies, we develop mechanisms for efficient and reliable V2X communications, access point selection, handover algorithms which are especially dedicated to road safety and autonomous driving applications.
Intelligent vehicular networks (IVNs) are constituents of ITS. IVNs range from platoons with a lead vehicle piloted by a human driver to fully ad-hoc vehicular networks, a.k.a. VANETs, comprising autonomous/automated vehicles. Safety issues in IVNs appear to be the least studied in the ITS domain. The focus of our work is on safety-critical (SC) scenarios, where accidents and fatalities inevitably occur when such scenarios are not handled correctly. In addition to on-board robotics, inter-vehicular radio communications have been considered for achieving safety properties. Since both technologies have known intrinsic limitations (in addition to possibly experiencing temporary or permanent failures), using them redundantly is mandatory for meeting safety regulations. Redundancy is a fundamental design principle in every SC cyber-physical domain, such as, e.g., air transportation. (Optics-based inter-vehicular communications may also be part of such redundant constructs.) The focus of our on-going work is on safety-critical (SC) communications. We consider IVNs on main roads and highways, which are settings where velocities can be very high, thus exacerbating safety problems acceptable delays in the cyber space, and response times in the physical space, shall be very small. Human lives being at stake, such delays and response times must have strict (non-stochastic) upper bounds under worst-case conditions (vehicular density, concurrency and failures). Consequently, we are led to look for deterministic solutions.
Rationale
In the current ITS literature, the term safety is used without being given a precise definition. That must be corrected. In our case, a fundamental open question is: what is the exact meaning of SC communications? We have devised a definition, referred to as space-time bounds acceptability (STBA) requirements. For any given problem related to SC communications, those STBA requirements serve as yardsticks for distinguishing acceptable solutions from unacceptable ones with respect to safety. In conformance with the above, STBA requirements rest on the following worst-case upper bounds:
Via discussions with foreign colleagues, notably those active in the IEEE 802 Committee, we have comforted our early diagnosis regarding existing standards for V2V/V2I/V2X communications, such as IEEE 802.11p and ETSI ITS-G5: they are totally inappropriate regarding SC communications. A major flaw is the choice of CSMA/CA as the MAC-level protocol. Obviously, there cannot be such bounds as
Reliance on V2I communications via terrestrial infrastructures and nodes, such as road-side units or WiFi hotspots, rather than direct V2V communications, can only lead to poorer results. First, reachability is not guaranteed: hazardous conditions may develop anywhere anytime, far away from a terrestrial node. Second, mixing SC communications and ordinary communications within terrestrial nodes is a violation of the very fundamental segregation principle: SC communications and processing shall be isolated from ordinary communications and processing. Third, security: it is very easy to jam or to spy on a terrestrial node; moreover, terrestrial nodes may be used for launching all sorts of attacks, man-in-the-middle attacks for example. Fourth, delays can only get worse than with direct V2V communications, since transiting via a node inevitably introduces additional latencies. Fifth, the delivery of every SC message must be acknowledged, which exacerbates the latency problems. Sixth, availability: what happens when a terrestrial node fails?
Trying to tweak existing standards for achieving SC communications is vain. That is also unjustified. Clearly, medium-range omnidirectional communications are unjustified for the handling of SC scenarios. By definition, accidents can only involve vehicles that are very close to each other. Therefore, short-range directional communications suffice. The obvious conclusion is that novel protocols and inter-vehicular coordination algorithms based on short-range direct V2V communications are needed. It is mandatory to check whether these novel solutions meet the STBA requirements. Future standards specifically aimed at SC communications in IVNs may emerge from such solutions.
Naming and privacy
Additionally, we are exploring the (re)naming problem as it arises in IVNs. Source and destination names appear in messages exchanged among vehicles. Most often, names are IP addresses or MAC addresses (plate numbers shall not be used for privacy reasons). A vehicle which intends to communicate with some vehicle, denoted
At any time, a vehicle can assign itself a name that is unique within a geographical zone centered on that vehicle (no third-party involved),
No linkage may exist between a name and those identifiers (plate numbers, IP/MAC addresses, etc.) proper to a vehicle,
Different (unique) names can be computed at different times by a vehicle (names can be short-lived or long-lived),
name(V) at UTC time
We have solved the (re)naming problem in string/cohort formations . Ranks (unique integers in any given string/cohort) are privacy-preserving names, easily computed by every member of a string, in the presence of string membership changes (new vehicles join in, members leave). That problem is open when considering arbitrary clusters of vehicles/strings encompassing multiple lanes.
This activity concerns the modeling of random systems related to ITS, through the identification and development of solutions based on probabilistic methods and more specifically through the exploration of links between large random systems and statistical physics. Traffic modeling is a very fertile area of application for this approach, both for macroscopic (fleet management , traffic prediction) and for microscopic (movement of each vehicle, formation of traffic jams) analysis. When the size or volume of structures grows (leading to the so-called “thermodynamic limit”), we study the quantitative and qualitative (performance, speed, stability, phase transitions, complexity, etc.) features of the system.
In the recent years, several directions have been explored.
Large random systems are a natural part of macroscopic studies of traffic, where several models from statistical physics can be fruitfully employed. One example is fleet management, where one main issue is to find optimal ways of reallocating unused vehicles: it has been shown that Coulombian potentials might be an efficient tool to drive the flow of vehicles. Another case deals with the prediction of traffic conditions, when the data comes from probe vehicles instead of static sensors.
While the widely-used macroscopic traffic flow models are well adapted to highway traffic, where the distance between junction is long (see for example the work done by the NeCS team in Grenoble), our focus is on a more urban situation, where the graphs are much denser. The approach we are advocating here is model-less, and based on statistical inference rather than fundamental diagrams of road segments. Using the Ising model or even a Gaussian Random Markov Field, together with the very popular Belief Propagation (BP) algorithm, we have been able to show how real-time data can be used for traffic prediction and reconstruction (in the space-time domain).
This new use of BP algorithm raises some theoretical questions about the ways the make the belief propagation algorithm more efficient:
find the best way to inject real-valued data in an Ising model with binary variables ;
build macroscopic variables that measure the overall state of the underlying graph, in order to improve the local propagation of information ;
make the underlying model as sparse as possible, in order to improve BP convergence and quality .
The focus here is on road traffic modeled as a granular flow, in order to analyze the features that can be explained by its random nature. This approach is complementary to macroscopic models of traffic flow (as done for example in the Opale team at Inria), which rely mainly on ODEs and PDEs to describe the traffic as a fluid.
One particular feature of road traffic that is of interest to us is the spontaneous formation of traffic jams. It is known that systems as simple as the Nagel-Schreckenberg model are able to describe traffic jams as an emergent phenomenon due to interaction between vehicles. However, even this simple model cannot be explicitly analyzed and therefore one has to resort to simulation.
One of the simplest solvable (but non trivial) probabilistic models for road traffic is the exclusion process. It lends itself to a number of extensions allowing to tackle some particular features of traffic flows: variable speed of particles, synchronized move of consecutive particles (platooning), use of geometries more complex than plain 1D (cross roads or even fully connected networks), formation and stability of vehicle clusters (vehicles that are close enough to establish an ad-hoc communication system), two-lane roads with overtaking.
The aspect that we have particularly studied is the possibility to let
the speed of vehicle evolve with time. To this end, we consider models
equivalent to a series of queues where the pair (service rate, number
of customers) forms a random walk in the quarter plane
Having in mind a global project concerning the analysis of complex systems, we also focus on the interplay between discrete and continuous description: in some cases, this recurrent question can be addressed quite rigorously via probabilistic methods.
We have considered in some classes of models dealing with
the dynamics of discrete curves subjected to stochastic deformations.
It turns out that the problems of interest can be set in terms of
interacting exclusion processes, the ultimate goal being to derive
hydrodynamic limits after proper scaling. A seemingly new method is
proposed, which relies on the analysis of specific partial
differential operators, involving variational calculus and functional
integration. Starting from a detailed analysis of the Asymmetric Simple Exclusion Process
(ASEP) system on
the torus
This field remains one of the important violon d'Ingres in our research activities in stochastic processes, both from theoretical and applied points of view. In particular, it is a building block for models of many communication and transportation systems.
One essential question concerns the computation of stationary measures (when they exist). As for the answer, it has been given by original methods formerly developed in the team (see books and related bibliography). For instance, in the case of small steps (jumps of size one in the interior of
where the unknown generating functions
The given function
Group of the random walk. In several studies, it has been
noticed that the so-called group of the walk governs the
behavior of a number of quantities, in particular through its
order, which is always even. In the case of small jumps, the
algebraic curve
Nature of the counting generating functions. Enumeration of planar lattice walks is a classical topic in combinatorics. For a given set of allowed jumps (or steps), it is a matter of counting the number of paths starting from some point and ending at some arbitrary point in a given time, and possibly restricted to some regions of the plane. A first basic and natural question arises: how many such paths exist? A second question concerns the nature of the associated counting generating functions (CGF): are they rational, algebraic, holonomic (or D-finite, i.e. solution of a linear differential equation with polynomial coefficients)?
Let
satisfies the functional equation
where
Some exact asymptotics in the counting of walks in
We have worked on various simulation tools to study and evaluate the performance of different transportation modes covering an entire urban area.
Discrete event simulation for collective taxis, a public transportation system with a service quality comparable with that of conventional taxis.
Discrete event simulation a system of self-service cars that can reconfigure themselves into shuttles, therefore creating a multimodal public transportation system; this second simulator is intended to become a generic tool for multimodal transportation.
Joint microscopic simulation of mobility and communication, necessary for investigation of cooperative platoons performance.
These two programs use a technique allowing to run simulations in batch mode and analyze the dynamics of the system afterward.
While the preceding section focused on methodology, in connection with automated guided vehicles, it should be stressed that the evolution of the problems which we deal with remains often guided by the technological developments. We enumerate three fields of application whose relative importance varies with time and which have strong mutual dependencies: driving assistance, cars available in self-service mode and fully automated vehicles (cybercars).
Several techniques will soon help drivers. One of the first immediate goal is to improve security by alerting the driver when some potentially dangerous or dangerous situations arise, i.e. collision warning systems or lane tracking could help a bus driver and surrounding vehicle drivers to more efficiently operate their vehicles. Human factors issues could be addressed to control the driver workload based on additional information processing requirements. Another issue is to optimize individual journeys. This means developing software for calculating optimal (for the user or for the community) paths. Nowadays, path planning software is based on a static view of the traffic: efforts have to be done to take the dynamic component in account.
The problems related to the abusive use of the individual car in large cities led the populations and the political leaders to support the development of public transport. A demand exists for a transport of people and goods which associates quality of service, environmental protection and access to the greatest number. Thus the tram and the light subways of VAL type recently introduced into several cities in France conquered the populations, in spite of high financial costs. However, these means of mass transportation are only possible on lines on which there is a keen demand. As soon as one moves away from these “lines of desire” or when one deviates from the rush hours, these modes become expensive and offer can thus only be limited in space and time. To give a more flexible offer, it is necessary to plan more individual modes which approach the car as we know it. However, if one wants to enjoy the benefits of the individual car without suffering from their disadvantages, it is necessary to try to match several criteria: availability anywhere and anytime to all, lower air and soils pollution as well as sound levels, reduced ground space occupation, security, low cost. Electric or gas vehicles available in self-service, as in the Praxitèle system, bring a first response to these criteria. To be able to still better meet the needs, it is however necessary to re-examine the design of the vehicles on the following points:
ease empty car moves to better distribute them;
better use of information systems inboard and on ground;
better integrate this system in the global transportation system.
These systems are now operating. The challenge is to bring them to an industrial phase by transferring technologies to these still experimental projects.
The long term effort of the project is to put automatically guided vehicles (cybercars) on the road. It seems too early to mix cybercars and traditional vehicles, but data processing and automation now make it possible to consider in the relatively short term the development of such vehicles and the adapted infrastructures. RITS aims at using these technologies on experimental platforms (vehicles and infrastructures) to accelerate the technology transfer and to innovate in this field. Other application can be precision docking systems that will allow buses to be automatically maneuvered into a loading zone or maintenance area, allowing easier access for passengers, or more efficient maintenance operations. Transit operating costs will also be reduced through decreased maintenance costs and less damage to the braking and steering systems. Regarding technical topics, several aspects of Cybercars have been developed at RITS this year. First, we have stabilized a generic Cycab architecture involving Inria SynDEx tool and CAN communications. The critical part of the vehicle is using a real-time SynDEx application controlling the actuators via two Motorola’s MPC555. Today, we have decided to migrate to the new dsPIC architecture for more efficiency and ease of use. This application has a second feature, it can receive commands from an external source (Asynchronously to this time) on a second CAN bus. This external source can be a PC or a dedicated CPU, we call it high level. To work on the high level, in the past years we have been developing a R&D framework called (Taxi) which used to take control of the vehicle (Cycab and Yamaha) and process data such as gyro, GPS, cameras, wireless communications and so on. In order to rely on a professional and maintained solution, we use the RTMaps SDK development platform for our developments and demonstrations. These demonstrations include: reliable SLAMMOT algorithm using 2 to 4 laser sensors simultaneously, automatic line/road following techniques, PDA remote control, multi sensors data fusion, collaborative perception via ad-hoc network. The second main topic is inter-vehicle communications using ad-hoc networks. We have worked with the EVA team for setting and tuning OLSR, a dynamic routing protocol for vehicles communications. Our goal is to develop a vehicle dedicated communication software suite, running on a specialized hardware. It can be linked also with the Taxi Framework for getting data such GPS information’s to help the routing algorithm.
Keyword: Localization
Scientific Description: Simultaneous Localization and Mapping method based on 2D laser data.
Participants: Fawzi Nashashibi and Zayed Alsayed
Contact: Fawzi Nashashibi
Vehicle-to-Pedestrian
Functional Description: It is a software developed for the Vehicle-to-Pedestrian (V2P) communications, risk calculation, and alarming pedestrians of collision risk. This software is made of an Android application dedicated to pedestrians and RtMaps modules for the vehicles.
On the pedestrian side, the application is relying on GPS data to localize the user and Wi-Fi communications are used to receive messages about close vehicles and send information about the pedestrian positioning. Besides, a service has been developed to evaluate the collision risk with the vehicles near the pedestrian and an HMI based on OpenStreetMap displays all the useful information such as pedestrian and vehicles localization and, collision risk.
On the vehicle side, RtMaps modules allowing V2X communications have been developed. These modules contain features such as TCP/UDP socket transmissions, broadcast, multicast, unicast communications, routing, forwarding algorithms, and application specific modules. In the V2ProVu software, a particular application module has been implemented to create data packets containing information about the vehicle state (position, speed, yaw rate,...) and the V2X communication stack is used to broadcast these packets towards pedestrians. Moreover, the V2proVu application can also receive data from pedestrians and create objects structures that can be shared with the vehicle perception tools.
Contact: Fawzi Nashashibi
Connected Autonomous Vehicles Simulator
Functional Description: The software provides an interface between the network simulator ns-3 (https://www.nsnam.org/) and the modular prototyping framework RTMaps (https://intempora.com/).
This code allows to create an RTMaps component which activates and controls the ns-3 simulator. The component handles the sending and reception of data packets between ns-3 and RTMaps for each vehicle. It also handles the mobility of vehicles in ns-3 using their known position in RTMaps.
Authors: Pierre Merdrignac, Oyunchimeg Shagdar and Jean-Marc Lasgouttes
Contact: Jean-Marc Lasgouttes
We want to solve the problem of multi-task pedestrian protection system (PPS) including not only pedestrian classification, detection and tracking,but also pedestrian action-unit classification and prediction, and finally pedestrian risk estimation. The goal of our research work is to develop an intelligent pedestrian protection component based only on single stereo vision system using an optimal cross-modality deep learning architecture in order to fulfill the prior requirements.
The system has to be able not only to detect all the pedestrians with high precision but also to track all the pedestrian paths, to classify the current pedestrian action and to predict their next actions and, finally, to estimate the pedestrian risk by the time to crossing for each pedestrian.
First, we investigate the classification component where we analyzed how learning representations from one modality would enable recognition for other modalitie(s) within various deep learning, which one terms as cross-modality learning. Second, we study how the cross modality learning improves an end-to-end the pedestrian action detection. Third, we analyze the pedestrian action prediction and the estimation of time to cross the street.
This work has been done in collaboration with Alexandrina Rogozan and Abdelaziz Bensrhair of INSA Rouen. More detail can be fund in , , , and in the PhD manuscript of Danut Ovidiu Pop .
Following the works initiated in past years, we have emphasized the need of developing for outdoor-applications to be robust to adverse weather conditions.
Three works were developed this year: two in the the context of the Samuel de Champlain Québec-France collaboration with Jean-François Lalonde from Univ. Laval (Canada) and another in the context of the new co-tutelle PhD thesis of Fabio Pizzati.
We have first proposed a physically-based rain rendering pipeline for realistically inserting rain into clear weather images. Our research was published at ICCV'19 and relies on a physical particle simulator, an estimation of the scene lighting and an accurate rain photometric modeling to augment images with arbitrary amount of realistic rain or fog. We validate our rendering with a user study, proving our rain is judged 40% more realistic that state-of-the-art. Using our generated weather augmented Kitti and Cityscapes dataset, we conduct a thorough evaluation of deep object detection and semantic segmentation algorithms and show that their performance decreases in degraded weather, on the order of 15% for object detection and 60% for semantic segmentation. Furthermore, we show refining existing networks with our augmented images improves the robustness of both object detection and semantic segmentation algorithms. We experiment on the popular nuScenes dataset and measure an improvement of 15% for object detection and 35% for semantic segmentation compared to original rainy performance.
Along with the research we have released the full augmented dataset on our project page
An alternative proposal is to use generative networks (GANs) to learn the translation of clear weather images to rainy images. This was achieved in the thesis of Fabio Pizzati and led to an accepted conference paper at WACV'20. To overcome the limitation of publicly available annotated datasets, we propose to learn the clear to rain mapping from datasets of different sources. Standard image-to-image translation architectures have limited effectiveness in such case due to the large source / target domain gap and usually fail to model typical traits of rain as water drops, which ultimately impacts the synthetic images realism. We proposed here a new type of domain bridge, that benefits from web-crawled data to reduce the domain gap.
To circumvent the limitation of physics-based rendering and GANs rendering, we are currently working on extensions of with Maxime Tremblay, PhD student at Univ. Laval. In this work, we are combining data-driven GAN approaches and physics-based driven learning.
There is an evident dead end to the paradigm of supervised learning, as it requires costly human labeling of millions of data frames to learn the appearance models of objects. As of today, the databases are recorded in very narrow conditions (e.g. only clear weather, only USA, only daytime). Adjusting to unseen conditions such as snow, hail, nighttime or unseen cities, require supervised algorithms to be retrained. Conversely, as humans we're capable of generalizing prior knowledge to new tasks. During this year, we initiated two works on transfer learning, typically Unsupervised Domain Adaptation (UDA) which is crucial to tackle the lack of annotations in a new domain. We have conducted two parallel projects on UDA: the first one in the scope of Maximilian Jaritz' thesis (submitted), and the second one in the scope of Fabio Pizzati's work on rainy scenarios:
xMUDA: In the first work, we explore how to learn from multi-modality and proposed cross-modal UDA (xMUDA) where we assumed the presence of 2D images and 3D point clouds for 3D semantic segmentation. This is challenging as the two input spaces are heterogeneous and can be impacted differently by domain shift. In xMUDA, modalities learn from each other through mutual mimicking, disentangled from the segmentation objective, to prevent the stronger modality from adopting false predictions from the weaker one. We evaluated on new UDA scenarios including day-to-night, country-to-country and dataset-to-dataset, leveraging recent autonomous driving datasets. xMUDA brings large improvements over uni-modal UDA on all tested scenarios, and is complementary to state-of-the-art UDA techniques.
Weighted Pseudo Labels: The second work focus specifically on semantic segmentation in rainy scenarios. We benefited from our other work on GANs clear to rain translation to apply a self-supervised domain adaptation (aka UDA) that learns from the use of pseudo labels. Using pseudo labels enables the self-supervision of the learning reinforcing the network belief in its own predictions. To circumvent the use of hard-coded threshold, which is a common practice for pseudo labels, we proposed new Weighted Pseudo Labels that actively learn the ad-hoc threshold in a sort of region-growing techniques.
Depth sensors (LiDARs, Time-of-flight cameras, stereo) gather geometrical knowledge about the scene which are rich and may be beneficial for many tasks. However, the depth information is usually sparse in nature and do not recover volumes and surfaces of objects.
This year we have conducted three works on the topic: one work to densify the 3D point clouds generated from LiDAR sensors, another work to fuse 2D images and 3D point clouds, and finalized another work to reconstruct 3D deformable objects.
The first work is in spirit a 3D point completion and was initiated with intern Manohar Kv. We developed a 3D pipeline to process point cloud and densify existing point clouds. It uses a modified version of the popular PointNet++ and it is thus able to reconstruct highly occluded 3D point clouds. The work is not yet published.
In we introduce a framework to fuse 2D multi-view images and 3D point clouds in an effective way by computing image features in 2D first, lifting them to 3D, and then fuse complementary geometry and image information in canonical 3D space. This work has been done while Maximilian Jaritz was visiting San Diego University.
In , we propose a new algorithm to reconstruct 3D deformable objects heavily occluded. It uses an automatic registration of multiple depth sensors and Gaussian Mixture Modeling in the radial domain to detect and reconstruct object from their symmetrical properties. This research was applied in the context of pottery wheel for the preservation of the cultural heritage and conducted in collaboration with Mines ParisTech. It resulted that our method enabled reconstruction of challenging deformable objects with an average precision of 7.6mm.
To achieve fully autonomous navigation, vehicles need to compute an accurate model of their direct surroundings. In fact, imprecise representations may lead to unexpected situations that could endanger the passengers. This year, we have proposed an algorithm capable to perform a fine and accurate 3D surface reconstruction of the environment from depth sensors. This representation keeps a high level of detail on the reconstruction, while maintaining a high density in the areas close to the vehicle.
Existing methods used for surface reconstruction from 3D data struggle to accommodate to the heterogeneous density of the input data while keeping the reconstruction accuracy. Conversely, our method is capable of handling this variable density by using an adaptive neighborhood kernel that perform local approximations of the data at different levels. This also permit to gain robustness against noise and output a smoother reconstruction. We also introduce a Gaussian confidence function capable to select the most adequate kernel for the local surface estimation. A Truncated Signed Distance Function (TSDF) is then globally estimated from the local surfaces to obtain the final mesh that represents the input scan.
The proposed method was evaluated in both simulated and real data. Reconstruction results show an improvement on the representation when compared with popular methods such as Implicit Moving Least Squares (IMLS), as the average error of our reconstruction is often 50% lower. Furthermore, almost 80% of vertices from our output mesh present an error below 0.2m, while only 40% of vertices lie below the same threshold for IMLS. Our method is capable to output a higher level of detail on the reconstruction, while keeping a high density in vehicle surroundings, the mesh can be of special interest for both the robotics and the graphics community to perform different tasks, such as terrain traversability assessment or physical modeling.
More details can be found in . This research is partially funded by AKKA Technology.
Scene understanding and future motion prediction of surrounding vehicles are crucial to achieve safe and reliable decision-making and motion planning for autonomous driving in a highway environment. This is a challenging task considering the correlation between the drivers behaviors. Two methods using attention mechanisms have been introduced in this context:
In , we present a new approach based on an LSTM encoder-decoder that uses a social pooling mechanism to model the interactions between all the neighboring vehicles. This social pooling module combines both local and non-local operations: the non-local multi-head attention mechanism captures the relative importance of each vehicle despite the inter-vehicle distances to the target vehicle, while the local blocks represent nearby interactions between vehicles.
Evaluations have been performed using two naturalistic driving datasets: Next Generation Simulation (NGSIM) and the highD Dataset
In we propose an RRNNs based encoder-decoder architecture where the encoder analyzes the patterns underlying in the past trajectories and the decoder generates the future trajectory sequence. The originality of this network is that it combines the advantages of the LSTM blocks in representing the temporal evolution of trajectories and the attention mechanism to model the relative interactions between vehicles. The proposed method outperforms LSTM encoder decoder in terms of RMSE values of the predicted trajectories on the large scaled naturalistic driving highD dataset.
Enhancing the outdoor mapping with SLAM based approaches is still an active research area. The main reason is that a consistent map of the vehicle's surrounding is one of the prerequisites for an effective vehicle interaction with this environment. In this context, and for the VALET project purpose, we have extended the PML-SLAM framework to handle 2D and 3D Lidars by replacing the localization module and designing a sparse pose graph optimizer. The sparse pose graph jointly optimizes the poses of the submaps generated by the local SLAM, which are already used for the mapping task, and the poses of the scans estimated following the scan matching process. This optimization is formulated as a non linear least square problem, and runs online in a background thread. The optimized poses are used to correct the vehicle's trajectory and to update the environment map. Furthermore, the graph-based PML-SLAM can deal with different sensors (IMU, GPS), that is, a sensor fusion "Kalman-filter" based is available to provide a good pose estimate for the local SLAM.
Self-vehicle localization is one of the fundamental tasks for autonomous driving. Most of current techniques for global positioning are based on the use of GNSS (Global Navigation Satellite Systems). However, these solutions do not provide a localization accuracy that is better than 2-3 m in open sky environments. Alternatively, the use of maps has been widely investigated for localization since maps can be pre-built very accurately. State of the art approaches often use dense maps or feature maps for localization. This year, we tackled to problems:
In we proposed a road sign perception system for vehicle localization within a third party map. This is challenging since third party maps are usually provided with sparse geometric features, which makes the localization task more difficult in comparison to dense maps. Experiments have been conducted on a Highway-like test track using GNSS/INS with RTK corrections as ground truth (GT).
In High Reflective Landmarks (HRL) - such as lane markings, road signs and guard rail reflectors (GRR) - are detected from a 3D point cloud. A particle filtering algorithm estimates the position of the vehicle by matching observed HRLs with HD map attributes. Experiments have been conducted on a highway-like test track using GNSS/INS with RTK corrections as a ground truth (GT). Error evaluations are given as cross-track (CT) and along-track (AT) errors defined in the curvilinear coordinates related to the map. The obtained accuracies of the localization system is 18 cm for the cross-track error and 32 cm for the along-track error.
Safe motion planning in a dynamic environment is of great importance in many robotics applications. This year, we have worked in two directions:
The work on reachable interaction sets introduced in has been extended to the case of dynamic obstacles with uncertain motions. We consider that the obstacles have stochastic motions and we use a probabilistic formulation to compute the RIS at each time step. Our approach improves existing methods in such a context (cf. Pierre de Beaucorps PhD thesis ).
Focusing on autonomous vehicles, we begun to study scenarios with occluded dynamic obstacles.
The ability to identify malfunctions on autonomous vehicles is critical for their deployment. As a matter of fact, systems able to identify when the positioning systems are not providing accurate data, or the perception algorithms are not properly detecting the environment, are extremely important to assure a certain safety level for automated vehicles. This is especially true since these systems are finally connected to the control module that provides the adequate commands to vehicle’s actuators. For control algorithms to work properly, proper inputs are necessary to reduce noise, increase controllability and avoid system’s malfunctions and instability. It is also critical for these algorithms to identify/consider vehicle physical limits for determining when is the automated system still capable of handling the vehicle. From the above, it is clear that automated vehicles are in need of proper inputs to control the vehicle, but also it is necessary to detect critical situations where the nominal control behavior is no longer assured, in order to take the vehicle to a safe state. Slide slip state is an example of a critical situation where the vehicle is no longer able to correct its trajectory. Thus, this part of my thesis work consists on developing a module for providing smooth signals to the controller and, at the same time, detect side slip situations. The lateral controller implemented in our automated driving (AD) system is based on the yaw error minimization between the desired yaw rate (obtained from the road layout information in function of the curvature) and the current vehicle yaw rate. From this, the first step is to provide a proper current yaw rate measurement. The proposed device compares the measured yaw rate value (coming from the vehicle sensor) with a model-based estimated yaw rate value. The idea is to identify vehicle model mismatches, correcting the model in real time. This permits to extend the nominal vehicle planar model to a road layout-independent model, where roll and pitch variations are considered. This first stage consists of a vehicle model compensator that includes unmodeled vehicle dynamics and parameter incertitude in real time when the vehicle is operating in autonomous mode. The vehicle lateral controller is then fed by the compensator output to allow robust performance in all road conditions. Once lateral control is fed with proper inputs, the second stage is the one detecting that vehicle handling physical limits are surpassed. The proposed system based on Youla-Kucera parametrization identifies the vehicle physical limits by estimating front and rear lateral forces, using as input the previous corrected model and on-board vehicle info. This permits to provide an accurate identification of slide slip vehicle states without adding any additional sensor to production vehicle’s on-board sensors (see Fig. ).
Without loss of generality, let us consider a single camera for detecting a preceding vehicle in the road. It is clear that there are two parameters that impact the performance of the perception system:
1) the specific algorithm developed to detect and track the objects providing accurate measurement; and
2) the physical limitation of the sensor itself. For a camera, the number of pixels limits the resolution of the image so the farther away the vehicle is, the lower the accuracy in its detection. This implies a more inaccurate measurement that will degrade the ego-vehicle performance. From the vehicle response point of view, we cannot expect that a single control device can handle for example a camera-based car-following system for all detected vehicle distances.
For the sake of clarity, Figure shows the speed of a preceding vehicle measured from a ground truth (solid blue line); and the measured speed from an on-board perception system. The speed of the preceding vehicle was computer controlled so we can assure a given response for it. In this example, it follows four consecutive reference speed changes from 0 to 5m/s, and finally to 8m/s. The ego-vehicle equipped with the on-board perception system was following that preceding vehicle at a speed dependent distance (i.e. as any on-the-market ACC system), meaning than the higher the speed, the higher the inter-vehicle distance. One can clearly see how the higher the speed, the higher the inaccuracy of the perception system, the more degraded the measurement.
We worked on a novel control device able to adapt its response to the perception system capabilities, modifying vehicle response accordingly to the level of accuracy of the perception system. This novel idea redefines and extends the capabilities of any ADAS or autonomous vehicle technology, not only because it improves vehicle’s performance but also because we can, a-priori, understand the limitations of the full vehicle performance with a complete closed loop analysis from perception to vehicle control. As additional remark, there are ways of dealing with these problems by filtering the perception signal. This causes two problems: control response stability is no longer guaranteed, and it smooths the response but it’s not possible to link the full system performance to that filtering. Figure presents an overview of the method. The block Perception set-of-sensor represents the specific embedded perception system. It can consist either in a single sensor or a combination of them. The characterization of the specific perception setup can be done offline, calibrating the system performance accordingly to the on-board sensors. The module Offline calibration will contain a 3-D look-up table with object distance, preceding-vehicle speed, and as a third parameter the desired measurement inaccuracy (it can be either the speed as shown in Figure or any other relevant parameter as the distance, yaw rate…). This offline calibration allows defining specific design parameters for the vehicle performance. Current control systems linked to perception don’t consider this inaccuracy when designing the vehicle performance. We here include two different control design criteria blocks. Assuming that we keep the regular controller design that considers perfect measurement from the perception system; the First control design criteria block includes the current production system controller. On the contrary side, we have also included a Second control design criteria block that can be adapted in function of the specific interest of each application. Following with the example on Figure , let us assume that we are interested on developing an application between 0 and 10m/s and the inaccuracy of the perception system is the one presented in the plots. Having this in mind, we can design the second controller with the goal of minimizing the impact of that inaccuracy in the vehicle performance. The system also uses as input the real-time perception value coming from the Perception system measurement block (in the case of Figure would provide the speed of the preceding vehicle in real-time). Then, this measurement feds the Perception-adapted controller block and the Performance degradation block. This last, accordingly to the information from the Offline calibration block, determines the status of the on-board perception system. The output of the Performance degradation block with the output of the first and second design criteria blocks fed the core module of this work: the real-time vehicle performance adaptation module. It is composed by two main blocks: the response corrector block to adapt the vehicle performance and the Perception-adapted controller block that merges both designed controller in a single stable structure.
Safety, privacy, efficiency, and cybersecurity (SPEC) properties are key to the advent of self-forming and self-healing networks of fully automated (driverless) terrestrial vehicles. Such vehicles are referred to as Next-Gen Vehicles (NGVs) in order to avoid confusion with Connected Autonomous Vehicles (CAVs). NGVs prefigure SAE level 5 vehicles. CAVs and NGVs rest on robotics capabilities (sensors, motion control laws, actuators, onboard systems, etc.). CAVs are equipped with V2X (vehicle-to-everything) functionalities based on medium range WiFi radio communications. NGVs will be equipped with CMX (coordinated mobility for X) functionalities, X standing for S, P, E, and C, based on very short range communications (cellular radio and optics).
Work in 2019 has been devoted to defining the CMX framework and to comparing V2X and CMX functionalities. The outputs of this work have been published in .
Highest SE (safety and efficiency) is one of the most fundamental goals set to designers of onboard systems. It is surmised that onboard robotics must be supplemented with inter-vehicular communication (IVC) capabilities in order to achieve highest SE properties. Thus the question: which IVC capabilities? In the V2X framework, two distinct sets of IVC capabilities are considered, namely DSRC-V2X (WiFi radio) and C-V2X (4G LTE, 5G, cellular radio). IVC capabilities in the CMX framework encompass cellular radio, VLC and passive optics.
Since V2X functionalities rest on medium range radio communications, they are vulnerable to remote and local cyberattacks (message falsification, masquerading, Sybil attacks, injection of bogus messages, DDoS attacks, etc.). It has been amply demonstrated that such cyberattacks can compromise safety (collisions caused by remote and/or local attackers) as well as efficiency (congested roadways and cities). Furthermore, V2X functionalities break down when radio channels are noisy (messages get lost) or/and jammed (intentional remote and local cyberattacks). Finally, owing to decade-old design decisions, there are no privacy properties with V2X functionalities. For example, every CAV must periodically broadcast messages that carry vehicle-centric characteristics and unencrypted current GNSS space coordinates (referred to as beaconing, frequencies ranging between 1 Hz and 10 Hz. Despite certificate-based pseudonymisation, routes followed by vehicles can be tracked and communications can be eavesdropped and recorded. Linkage with passengers) personal data is straightforward.
Therefore, in addition to degrading safety and efficiency properties achieved by onboard robotics, V2X functionalities do not meet elementary requirements regarding privacy and cybersecurity. Some proponents of the V2X approach assert that it is impossible to deliver road safety without breaching passengers' privacy. To be valid, that statement should be backed with an impossibility proof. Such a proof has not appeared yet and will never appear for the simple reason that safety and privacy properties can be achieved jointly, by design, proofs given, as demonstrated with the CMX approach.
From a more theoretical perspective, the V2X and the CMX frameworks can be contrasted as follows. Unquestionably, full asynchrony is the appropriate model for representing the vehicular network universe faithfully. Vehicles are started or stopped at arbitrary times, velocities change unpredictably, ditto for lane changes, on-ramp merging, concurrent traversals of intersections and roundabouts, and so on. Onboard processes that are life/safety critical are run in the presence of fortuitous failures, cyberattacks, and concurrency (due to resource sharing). It follows that even if one postulates the existence of finite bounds for process execution durations, it is impossible to assume any a priori knowledge of values taken by those bounds. That is precisely the definition of full asynchrony.
Numerous impossibility results relative to fully asynchronous systems have been published since the late-1970s. For example, problems akin to distributed consensus (terminating reliable broadcast, consistent multi-copied data structures, exact agreement, leader election, etc.) have no solutions in the presence of a single failure, even when communications are assumed to be perfect (no message losses). Since mobile wireless communications are unreliable, those results hold a fortiori in vehicular networks. Obviously, problems that involve termination in computable/predictable time bounds (a real-time property) have no solutions either.
The above-mentioned problems shall be solved in order to provide vehicles and vehicular networks with the SPEC properties. Knowing that solutions exist when considering synchrony models -such as e.g. partial synchrony, timed asynchrony, full synchrony- the challenge is to show how synchrony models could emerge from full asynchrony. This challenge is ignored in the V2X framework. Conclusion: since V2X designs are conducted considering full asynchrony, none of the SPEC properties may hold true.
The CMX framework results from addressing this challenge. NGVs are endowed with CMX functionalities which are based on specific cyberphysical constructs (cells, cohorts, flocks). These constructs serve to instantiate synchrony models within which it is possible to design protocols and algorithms (e.g., deterministic MAC protocols, time-bounded distributed algorithms for message dissemination, approximate agreement, and consensus) that are needed for establishing and proving the SPEC properties, while matching the real vehicular networks universe.
Concepts at the core of the CMX framework (cyberphysical levels, unfalsifiable vehicle profiles, proactive security modules, privacy-preserving naming, etc.) are detailed in . Regarding SE properties, we show how to achieve theoretical absolute safety (no fatalities, no severe injuries) while keeping smallest safe gaps (highest efficiency) in cohort-structured vehicular networks, under assumptions of high coverage. As for PC properties, we show that passengers' privacy cannot be compromised via cyber eavesdropping and/or physical tracking of vehicles. This is due to the fact that messages do not carry vehicle-centric characteristics or GNSS space coordinates. CMX functionalities are shown to be immune to remote cyberattacks. Thanks to optical communications (in addition to very short range cellular radio), they can withstand radio channel jamming. Owing to controlled cohort admission, external local cyberattacks aimed at cohort members are inoperative. Local cyberattacks launched from the inside of a cohort, i.e. by cohort members themselves, can be thwarted. In the unlikely case of success, dishonest members would be involved in those collisions which they create. Conclusion: the only cyberattacks that may compromise safety in cohort-structured vehicular networks are due to irrational attackers.
This work , , in collaboration with Cyril Furtlehner (TAU, Inria), deals with real-time prediction of traffic conditions in a urban setting with incomplete data. The main focus is on finding a good way to encode available information (flow, speed, counts,...) in a Markov Random Field, and to decode it in the form of real-time traffic reconstruction and prediction. Our approach relies in particular on the Gaussian belief propagation algorithm.
Through our collaboration with PTV Sistema, we obtained extensive results on large-scale datasets containing 250 to 2000 detectors. The results show very good ability to predict flow variables and a reasonably good performance on speed or occupancy variables. Some element of understanding of the observed performance are given by a careful analysis of the model, allowing to some extent to disentangle modelling bias from intrinsic noise of the traffic phenomena and its measurement process.
This year we worked on code optimization and submitted our work to Transportation Research: Part C.
We investigate in the transfer function emanating from the linearization of a car-following model, when taking into account a driver reaction time. This leads to stability conditions, which are explicitly given. We also show how this reaction time can introduce a weak string instability.
This paper is intended as a foundation of a larger work on traffic stabilization by means of a fleet of cooperative automated vehicles. Contrary to some earlier works, our approach is based on a car-following model with reaction-time delay, rather than on a first order fluid model. The continuation of these studies will concern schockwave analysis and adequate traffic-stabilizing control strategies.
In the second edition of the book ,
original methods were proposed to determine the invariant measure of
random walks in the quarter plane with small jumps (size 1), the
general solution being obtained via reduction to boundary value
problems. In this framework, number of difficult open problems related
to lattice path combinatorics are currently being explored, in
collaboration with A. Bostan and F. Chyzak (project-team SPECFUN,
Inria-Saclay), both from theoretical and computer algebra points of
view: concrete computation of the criteria, utilization of
differential Galois theory, genus greater than 1 (i.e. when some
jumps are of size
Validating Advanced Driver Assistance Systems (ADAS) is a strategic issue, since such systems are becoming increasingly widespread in the automotive field.
But ADAS validation is a complex issue, particularly for camera based systems, because these functions may be facing a very high number of situations that can be considered as infinite. Building at a low cost level a sufficiently detailed campaign is thus very difficult. Indeed, test case generation faces the crucial question of inherent combinatorial explosion. An important constraint is to generate almost all situations in the most economical way. This task can be considered from two points of view: deterministic via binary search trees, or stochastic via Markov chain Monte Carlo (MCMC) sampling. We choose the latter probabilistic approach described below, which in our opinion seems to be the most efficient one. Typically, the problem is to produce samples of large random vectors, the components of which are possibly dependent and take a finite number of values with some given probabilities.The following flowchart is proposed.
In a first step, starting from the simulation graph generated by the toolboxes of MATLAB, we construct a so-called Markov Random Field (MRF). When the parameters are locally dependent, this can be achieved from the user's specifications and by a systematic application of Bayes' formula.
Then, to cope with the combinatorial explosion, test cases are produced by implementing (and comparing) various Gibbs samplers, which are fruitfully employed for large systems encountered in physics. In particular, we strive to make a compromise between the convergence rate toward equilibrium, the percentage of generated duplicates and the path coverage, keeping in mind that the speed of convergence is exponential, a classical property deduced from the general theory of Markov chains.
The generation of rare events by mixing Gibbs samplers, large deviation techniques (LDT) and cross-entropy method is a work in progress.
The French car manufacturer Groupe PSA shows a great interest in these methods and has established a contractual collaboration involving ARMINES-Mines ParisTech (Guy Fayolle as associate researcher) and Can Tho University in Vietnam (Pr. Van Ly Tran).
Valeo Group: a very strong partnership is under reinforcement between Valeo and Inria. Several bilateral contracts were signed to conduct joint works on Driving Assistance, some of which Valeo is funding. This joint research includes:
The PhD thesis of Pierre de Beaucorps under the framework of Valeo project “Daring”,
A CIFRE like PhD thesis is ongoing between Valeo and Inria (Maximilian JARITZ), dealing with multisensor processing and learning techniques for free navigable road detection.
Valeo is currently a major financing partner of the “GAT” international Chaire/JointLab in which Inria is a partner. The other partners are: UC Berkeley, Shanghai Jiao-Tong University, EPFL, IFSTTAR, MPSA (Peugeot-Citroën) and SAFRAN.
Technology transfer is also a major collaboration topic between RITS and Valeo as well as the development of a road automated prototype.
Finally, Inria and Valeo are partners of the PIA French project CAMPUS ( Connected Automated Mobility Platform for Urban Sustainability) including SAFRAN, Invia and Gemalto. The aim of the project is the development of autonomous vehicles and the realization of two canonical uses-cases on highways and urban like environments.
Renault Group: Collaboration between Renault and RITS re-started in 2016. Different research teams in Renault are now working separately with RITS on different topics.
A CIFRE like PhD thesis is ongoing between Renault and Inria (Farouk GHALLABI) The thesis deals with the accurate localization of an autonomous vehicle on a highway using mainly on-board low-cost perception sensors.
Another CIFRE PhD thesis is ongoing since November 2017 (Imane MAHTOUT).
AKKA Technologies: Collaboration with AKKA since 2012 (for the Link & Go prototype).
Inria and AKKA Technologies are partners in the VALET projects (ANR projects).
A CIFRE PhD thesis (Luis ROLDAO) dealing with 3D-environment modeling for autonomous vehicles begun in October 2017.
Title: Redistribution automatique d’une flotte de véhicules en partage et valet de parking
Instrument: ANR
Duration: January 2016 - September 2019
Coordinator: Fawzi Nashashibi
Partners: Inria, Ecole Centrale de Nantes (IRCCyN), AKKA Technologies
Inria contact: Fawzi Nashashibi
Abstract: The VALET project proposes a novel approach for solving car-sharing vehicles redistribution problem using vehicle platoons guided by professional drivers. An optimal routing algorithm is in charge of defining platoons drivers’ routes to the parking areas where the followers are parked in a complete automated mode. The main idea of VALET is to retrieve vehicles parked randomly on the urban parking network by users. These parking spaces may be in electric charging stations, parking for car sharing vehicles or in regular parking places. Once the vehicles are collected and guided in a platooning mode, the objective is then to guide them to their allocated parking area or to their respective parking lots. Then each vehicle is assigned a parking place into which it has to park in an automated mode.
Title: navigation autonome dans les foules inspirée par les humains (Human Inspired Autonomous Navigation In Crowds)
Instrument: ANR
Duration: January 2018 - December 2020
Coordinator: Anne Spalanzani (Inria Rhône-Alpes, Chroma research team)
Partners: Inria Rhône-Alpes, Inria Paris, LIG Laboratoire d'Informatique de Grenoble, LS2N - ECN Laboratoire des Sciences du Numérique de Nantes
Inria contact: Fawzi Nashashibi
Abstract: The HIANIC project will try to address some problems that will arise when these cars are mixed with pedestrians. The HIANIC project will develop new technologies in term of autonomous navigation in dense and human populated traffic. It will explore the complex problem of navigating autonomously in shared-space environments, where pedestrians and cars share the same environment.
Such a system will contribute both to urban safety and intelligent mobility in “shared spaces”. Negotiation will help to avoid frozen situations increasing the vehicle’s reactivity and optimizing the navigable space. Negotiation, Human-Aware Navigation and Communication will contribute to a better public acceptance of such autonomous systems and facilitate their penetration in the transportation landscape.
Title: Perception augmentée par coopération véhicule avec l'infrastructure routière
Instrument: FUI
Duration: September 2016 - May 2020
Coordinator: SIGNATURE Group (SVMS)
Partners: DigiMobee, LOGIROAD, MABEN PRODUCTS, SANEF, SVMS, VICI, Inria, VEDECOM
Inria contact: Raoul de Charette
Abstract: The objective of the project is to integrate two technologies currently being deployed in order to significantly increase the time for an automated vehicle to evolve autonomously on European road networks. It is the integration of technologies for the detection of fixed and mobile objects such as radars, lidars, cameras ... etc. And local telecommunication technologies for the development of ad hoc local networks as used in cooperative systems.
RITS team is a very active partner in the competitivity clusters, especially MOV'EO and System@tic. We are involved in several technical committees like the DAS SUR of MOV'EO for example.
RITS is also the main Inria contributor in the VEDECOM institute (IEED).
Title: AUTOCITS Regulation Study for Interoperability in the Adoption of Autonomous Driving in European Urban Nodes
Program: CEF- TRANSPORT Atlantic corridor
Duration: November 2016 - March 2019
Coordinator: Indra Sistemas S.A. (Spain)
Partners: Indra Sistemas S.A. (Spain); Universidad Politécnica de Madrid (UPM), Spain; Dirección General de Tráfico (DGT), Spain; Inria (France); Instituto Pedro Nunes (IPN), Portugal; Autoridade Nacional de Segurança Rodoviária (ANSR), Portugal; Universidade de Coimbra (UC), Portugal.
Inria contact: Fawzi Nashashibi, Mohammad Abualhoul
Abstract: The aim of the Study is to contribute to the deployment of C-ITS in Europe by enhancing interoperability for autonomous vehicles as well as to boost the role of C-ITS as catalyst for the implementation of autonomous driving. Pilots will be implemented in 3 major Core Urban nodes (Paris, Madrid, Lisbon) located along the Core network Atlantic Corridor in 3 different Member States. The Action consists of Analysis and design, Pilots deployment and assessment, Dissemination and communication as well as Project Management and Coordination.
RITS is member of the euRobotics AISBL (Association Internationale Sans But Lucratif) and the Leader of “People transport” Topic. This makes from Inria one of the rare French robotics representatives at the European level.
See also: http://
RITS is a full partner of VRA – Vehicle and Road Automation, a support action funded by the European Union to create a collaboration network of experts and stakeholders working on deployment of automated vehicles and its related infrastructure. VRA project is considered as the cooperation interface between EC funded projects, international relations and national activities on the topic of vehicle and road automation. It is financed by the European Commission DG CONNECT and coordinated by ERTICO – ITS Europe. See also: http://
RITS has signed 3 MoU with the following international laboratories:
Vehicle Dynamics and Control Laboratory, Seoul National University (SNU), S. Korea: international cooperation agreement for Graduate-Level Academic and Research Collaboration
MICA Lab, Hanoi University of Science and Technology, Vietnam: cooperation agreement for research collaboration and PhD students co-supervision
Integrated Industrial Design Lab (INDEL) of the Department of Product and Systems Design Engineering, University of the Aegean, Greece: international cooperation agreement for Graduate-Level Academic and Research Collaboration
Samuel de Champlain Québec-France collaboration program: "Vision par ordinateur en conditions difficiles", cooperation between Raoul de Charette and Jean-François Lalonde from Laval University.
Plamen Petrov from Technical University of Sofia, from July to September 2019.
Pranav Agarwal, from August 2019.
Fares Bessam, Master student, April-September 2019.
Manuel Gonzalez and Leonardo Ward, from Simon Bolivar University, Venezuela, from September 2019.
Manohar KV, May-July 2019.
Maximilian Jaritz was at UC San Diego, visiting SU Lab directed by Hao Su, from October 1st 2018 to February 15th 2019.
Raoul de Charette: BMVC 2019, CVPR 2019, ICCV 2019, AAAI 2019, IEEE ITSC 2019, IEEE IV 2019.
Jean-Marc Lasgouttes: IEEE ITSC 2019, ICVES 2019.
Fawzi Nashashibi : IEEE ICRA 2019, IEEE IROS 2019, IEEE IV 2019, IEEE ITSC 2019, IEEE VTC 2018, IEEE ICVES 2019.
Anne Verroust-Blondet: IEEE ICRA 2020, IEEE IV 2019, IEEE ITSC 2019, ICVES 2019.
Guy Fayolle: associate editor of the journal Markov Processes and Related Fields.
Fawzi Nashashibi: associate editor of the journal IEEE Transactions on Intelligent Vehicles, associate editor of the journal IEEE Transactions on Intelligent Transportation Systems.
Fawzi Nashashibi: currently Guest Editor of the IEEE Sensors journal (ISSN 1424-8220), Special Issue on “The Application of Sensors in Autonomous Vehicles”.
Anne Verroust-Blondet: associate editor of the journal The Visual Computer.
Raoul de Charette: IEEE Transactions on Intelligent Transportation Systems
Guy Fayolle: AAP, MPRF, PTRF, QUESTA, European Journal of Combinatorics, JSP, Physica A, Springer Science.
Jean-Marc Lasgouttes: IEEE Transactions on Intelligent Transportation Systems, Probability in the Engineering and Informational Sciences.
Fawzi Nashashibi: IEEE Transactions on Intelligent Transportation Systems, IEEE Transactions on Intelligent Vehicles, Transportation Research Part C, IEEE ACCESS.
Anne Verroust-Blondet: IEEE Transactions on Intelligent Vehicles, The Visual Computer.
Raoul de Charette: Sparse and dense data in computer vision, Universite Laval, Québec, Canada, July 5th 2019.
Guy Fayolle was invited participant at the conference Transient Transcendence in Transylvania, Brașov, Romania, May 13–17, 2019.
Gérard Le Lann: Automated Driving: Ethical/Moral Quandaries, invitation by Comité d'éthique du CNRS (COMETS), Paris, 26 February 2019.
Gérard Le Lann: Ethical and Privacy Issues in Autonomic Vehicular Networks, invited talk, Symposium on Networking Research Topics: Past, Present and Future inspired by Mario Gerla, Politecnico di Milano (Italy), 3 June 2019.
Gérard Le Lann: Une mise en perspective - Qu'avons-nous appris de l'aventure Arpanet/Internet qui puisse servir aujourd'hui ?, Colloque SIF Les 50 ans d'Internet, CNAM, Paris, 29 October 2019.
Fawzi Nashashibi: Connected autonomous vehicles: objectives and challenges, invited plenary talk, at the JNRR 2019 (national days on robotics research), Vittel, October 18, 2019.
Fawzi Nashashibi: Connected Autonomous Vehicles: selected challenges, Keynote speaker, at the IEEE ITSC 2019 conference - workshop, Auckland (New Zealand), October 27, 2019.
Fawzi Nashashibi: The role of the intelligent infrastructure in the development and deployment of autonomous vehicles, Keynote speaker, at the SMIV international workshop, organised by Vedecom institute, Paris, November 13, 2019.
Fawzi Nashashibi: Connected Autonomous Vehicles: next challenges, Keynote speaker, at the IEEE CIS-RAM conference, Bangkok (Thailand), November 18-20, 2019.
Guy Fayolle is scientific advisor and associate researcher at the Robotics Laboratory of Mines ParisTech. He is also collaborating member of the research-team SPECFUN at Inria-Saclay.
Jean-Marc Lasgouttes is member of the Inria Commission d'Evaluation of Inria and of the Conseil Académique of Université Paris-Saclay.
Gérard Le Lann: Contribution to White Book on Cybersecurity: current challenges and Inria's research directions, Section 6.3.2 on Robotics and connected autonomous vehicles, February 2019.
Fawzi Nashashibi is an associate researcher at the Robotics Laboratory, Mines ParisTech. He is an evaluator/reviewer of European H2020 projects.
Anne Verroust-Blondet was a member of the CRCN Recruiting Commission of the CRI Paris in 2019.
Jean-Marc Lasgouttes is a member of the Comité Technique Inria and of the Comité Local Hygiène Sécurité et Conditions de Travail of Inria Paris.
Guy Fayolle is a member of the working group IFIP WG 7.3.
Fawzi Nashashibi is a member of the international Automated Highway Board Committee of the TRB (AHB30).
He is a member of the Board of Governors of the VEDECOM Institute representing Inria and of the Board of Governors of MOV'EO Competitiveness cluster representing Inria. He is also member of the DAS-SMI of MOV'EO cluster.
Anne Verroust-Blondet is the scientific correspondent of the European affairs and of the International Partnerships for Inria Paris. She was a member of the COST-GTRI committee at Inria (Committee in charge of the evaluation of international projects) and a member of the "emplois scientifiques" committee of Inria Paris in 2019.
Primary school: Raoul de Charette, “Introduction à l'intelligence artificielle”, 10h, -, Puck et Ribambelle, Castelnau-le-Lez, France.
Licence: Fawzi Nashashibi, “Programmation avancée”, 84h, L1, Université Paris-8 Saint-Denis, France.
Engineering, 2nd year: Fawzi Nashashibi, “Image synthesis and 3D Infographics”, 12h, M2, INT Télécom SudParis, IMA4503 “Virtual and augmented reality for autonomy”.
Master: Jean-Marc Lasgouttes, “Analyse de données”, 54h, second year of Magistère de Finance (M1), University Paris 1 Panthéon Sorbonne, France.
Master: Jean-Marc Lasgouttes, “Analyse de données”, 52.5h, Master 1 SIC APP, University Paris 1 Panthéon Sorbonne, France.
Master: Kaouther Messaoud, "Machine learning 1", 22h (TP), M1, ESIEE, Paris.
Master: Fawzi Nashashibi, “Obstacle detection and Multisensor Fusion”, 4h, M2, INSA de Rouen.
Master: Fawzi Nashashibi, “Perception and Image processing for Mobile Autonomous Systems”, 12h, M2, University of Evry.
Mastere : Raoul de Charette, “Scene Understanding with Computer Vision”, 20h, post master, Mines ParisTech, France.
Doctorat: Jean-Marc Lasgouttes, “Introduction au Boosting”, 10.5h, Mastère Spécialisé “Expert en sciences des données”, INSA-Rouen, France.
PhD: Pierre de Beaucorps, “Planification de trajectoire dans un environnement peu contraint et fortement dynamique”, UPMC Sorbonne University, July 2019, supervisor: Anne Verroust-Blondet, co-supervisor: Fawzi Nashashibi.
PhD: Danut Ovidiu Pop, "Multi-Task Cross-Modality Deep Learning for Pedestrian Risk Estimation", INSA Rouen, November 2016, supervisor: Abdelaziz Bensrhair, co-supervisor: Fawzi Nashashibi.
PhD in progress: Farouk Ghallabi, “Environment modeling and simultaneous localization of a mobile vehicle on motorways: a multi-sensor approach”, Mines ParisTech, PSL Research University, October 2016, supervisor: Fawzi Nashashibi.
PhD in progress: Maximilian Jaritz, “Perception multi-capteur pour la conduite autonome grâce à l'apprentissage profond”, Mines ParisTech, PSL Research University, January 2017, supervisor: Fawzi Nashashibi, co-supervisor: Raoul de Charette.
PhD in progress: Imane Matout, “Estimation de l'intention des véhicules pour la prise de décision et le contrôle sans faille en navigation autonome”, Mines ParisTech, PSL Research University, December 2017, supervisor: Fawzi Nashashibi, co-supervisor: Vicente Milanés.
PhD in progress: Kaouther Messaoud, “Détermination des manoeuvres et des intentions des véhicules avoisinant un véhicule autonome”, UPMC Sorbonne University, October 2017, supervisor: Anne Verroust-Blondet, co-supervisors: Fawzi Nashashibi, Itheri Yahiaoui.
PhD in progress: Fabio Pizzati, “Style transfer and domain adaptation for semantic segmentation, PSL Research University and University of Bologna, November 2019, co-supervisors: Andrea Prati, Stefano Selleri, Fawzi Nashashibi and Raoul de Charette.
PhD in progress: Renaud Poncelet, “Navigation autonome en présence d'obstacles fortement dynamiques au mouvement incertain”, UPMC Sorbonne University, September 2018, supervisor: Anne Verroust-Blondet, co-supervisor: Fawzi Nashashibi.
PhD in progress: Luis Roldao, “Modélisation 3D de l’environnement et de la manoeuvrabilité d’un véhicule”, UPMC Sorbonne University, October 2017, supervisor: Anne Verroust-Blondet, co-supervisor: Raoul de Charette.
Fawzi Nashashibi was a reviewer of the HDR thesis of Mr. Romuald Aufrère - Systèmes de perception bayésienne pour la robotique mobile : application à la localisation de véhicules, Université Clermont-Ferrand, July 10, 2019.
Fawzi Nashashibi was a reviewer of the PhD thesis of Mrs. Laurène Claussmann - Motion Planning for Autonomous Highway Driving: A Unified Architecture for Decision-Maker and Trajectory Generator, Université Versailles, September 27, 2019.
Fawzi Nashashibi was a reviewer of the PhD thesis of Mr. Mohamed-Taha Boudali - Contributions au guidage d’un véhicule autonome en situations non conventionnelles de conduite, Université Mulhouse, September 25, 2019.
Fawzi Nashashibi was a jury member of the PhD thesis of Mr. David Pérez Morales - Multi-sensor-based control in Intelligent Parking applications, Université Nantes, December 6, 2019.
Fawzi Nashashibi was a reviewer of the PhD thesis of Mr. Mathieu Barbier - Crossing of Road Intersections: Decision-Making Under Uncertainty for Autonomous Vehicles, Université Grenoble Alpes, December 11, 2019.
Anne Verroust-Blondet was a jury member of the PhD thesis of Mr. David Sierra Gonzalez - Towards Human-Like Prediction and Decision-Making for Automated Vehicles in Highway Scenarios, Université Grenoble Alpes, 1 April 2019.
Anne Verroust-Blondet was a reviewer of the PhD thesis of Mr. Hernan Gonzalez - Complex Dynamic scene analysis through multi-body motion segmentation - application to intelligent vehicles, Université Paris-Saclay, 13 December 2019.
Fawzi Nashashibi was audited by a joint committee from the Parliament-Senat and cited in a OPECST report titled “End of thermal vehicles in 2040: what realistic scenarios”. Authors: Mrs. Huguette Tiegna (Parliament Deputy, Vice President) and Mr. Stéphane Piednoir (Senator). ISSN: 1249-3872.
Participation of several team members to PAC V2X “Forum des acteurs” through oral scientific presentations - Paris, September 11, 2019.
Participation of several team members in oral presentations and demonstrations with real mobile platforms during the Journées du Patrimoine (Heritage Days), Rocquencourt, September 21, 2019.
Fawzi Nashashibi was interviewed by Eric Gibory in Le Monde
Fawzi Nashashibi was interviewed by Muriel Jaoën in Les Echos
Raoul de Charette: What is the limitation of artificial intelligence ?, Arbre des Connaissances, Paris, France, February 12th 2019.
Raoul de Charette: Dangers de l'intelligence artificielle ?, Crégy-lès-Meaux, France, March 2019.
Raoul de Charette: Qu'est-ce que l'intelligence artificielle ?, Oissery, France, May 28th 2019.
Raoul de Charette: Intelligence artificielle, Arbre des Connaissances, Paris, France, June 17th 2019.
Fawzi Nashashibi: panelist, round table on the “Vehicle of the Future”, Maison de la Recherche, Paris, April 2019.
Fawzi Nashashibi: panelist, round table on the “Autonomous Vehicle”, Centrales-Supélec, Paris, September 16, 2019.