**Main challenge:**The project-team
*e-Motion*aims at developing models and algorithms allowing us to build
*``artificial systems''*including advanced sensori-motors loops, and exhibiting sufficiently efficient and robust behaviors for being able to operate in
*open and dynamic environments*(i.e. in partially known environments, where time and dynamics play a major role), while leading to
*various types of interaction with humans*. Recent technological progress on embedded computational power, on sensor technologies, and on miniaturised mechatronic systems, make the
required technological breakthroughs potentially possible (including from the scalability point of view).

**Approach and research themes:**In order to try to reach the previous objective, we combine the respective advantages of
*computational geometry*and of the
*theory of probabilities*. We are also working in cooperation with neurophysiologists on sensory-motor systems, for trying to apply and experiment some
*biological models*. This approach leads us to study, under these different points of view, three strongly correlated fundamental research themes:

*Perception and multimodal modelling of space and motion*. The basic idea consists in continuously building (using preliminary knowledge and current perceptive data) several types of
models having complementary functional specialisations (as suggested by neurophysiologists). This leads us to address the following questions : how to model the various aspects of the
real world ? how to consistently combine a priori knowledge and flows of perceptive data ? how to predict the motions and behaviors of the sensed object ?

*Motion planning and autonomous navigation in the physical world*. The main problem is to simultaneously take into account various constraints of the physical world such as
non-collision, environment dynamicity, or reaction time, while mastering the related algorithmic complexity. Our approach for solving this problem consists in addressing two main
questions : how to construct incrementally efficient and reliable space-time representations for both motion planning and navigation ? how to define an iterative motion planning
paradigm taking into account kinematics, dynamics, time constraints, and safety issues ?

*Learning, decision, and probabilistic inference*. The main problem to solve is to be able to correctly reason about prior and learned knowledge, while taking explicitely into account
the related uncertainty. Our approach for addressing this problem is to use and develop our bayesian programming paradigm, while collaborating with neurophysiologists on some particular
topics such as the modeling of human navigation mechanisms or of biological sensory-motors loops. The main questions we are addressing are the followings : how to model sensory-motor
systems and related behaviors ? how to take safe navigation decisions under uncertainty ? What kind of models and computational tools are required for implementing the related bayesian
inference paradigms ?

In spite of the significant technological advances made during the last decade, Robotics is still blocked by the problems of
*scalability*and of the
*real integration of robotised systems in our everyday life*. The reason for this stems primarily from the fact that models and technologies developed in the past (e.g. approaches based on
logics, geometrical based methods, randomized search techniques, reactive architectures ...) have mainly reached their limits, and cannot be directly used for crossing the complexity gap
introduced by the physical environments in which we are living. Indeed, such environments include complex multimodal data, are continuously changing and partially unpredictable, and generate
complex interactions with humans. This means that
*appropriate decision-making processes*taking into account these characteristics have to be designed, implemented and experimented in real situations. Such processes have to be efficient
and robust enough for making it possible to meet the required reactivity characteristics, while being able to make appropriate decisions from complex and incomplete data and knowledge, i.e. by
reasoning about a combination of partial a priori knowledges, of some incremental experimental data (including sensory data), and of some hidden variables. This means that new models and
algorithms have to be designed for beeing able to formalise the intrinsic
*``incompleteness''*of the problem, and to better model the intricate
*``complexity''*of the real world.

The objective of the
*e-Motion*project-team is to find formal and practical solutions to the previous unsolved problems, still very little addressed by our scientific community. Our bet is that our approach
based on both geometry and bayesian programming, will allow us to achieve the following technological breakthroughs:

*Motion autonomy in a dynamic complex world.*We are especially interested in the problems arising from the richness of the environments considered (i.e. how to model them efficiently),
from their dynamicity (i.e. taking explicitly into account the ``space-time'' dimension), and from the large variety of possible interactions (e.g. estimation and prediction of the
behaviors of the potential obstacles).

*Increased robustness of the navigation processes.*We put the emphasis on the problems of
*incompleteness*(factors not taken into account or hidden variables) inherent to the representation of any physical phenomenon. This dimension of the problem is generally empirically
and approximatively taken into account in traditional approaches, leading the related systems to be poorly reliable. Our approach for dealing with this problem is to convert the
``incompleteness'' into numerically quantifiable data, coded in terms of probability distributions and referred to as
*``uncertainties''*. Then, such random variables can be combined, evaluated, and used in various decision-making processes.

*Intuitive programming of artificial systems.*Our approach consists in using as far as possible
*learning*processes (supervised or not), in order to be able to combine the a priori knowledge (called ``preliminary data'') and the past experience of the system (called
``experimental data''). This approach, which is consistent with our
*Bayesian Programming*paradigm, should permit us to gradually obtain systems more robust and better adapted to the problems at hand.

As previously mentionned, the main applications of our research are those aiming at introducing advanced and secured robotized systems into the human ``living space''. In this context, we are focussing onto the following application domains :

*Future cars and transportation systems.*This application domain should quickly change under the effects of both new technologies and current economical and security requirements of
our modern society. Various technologies are currently studied and developed by research laboratories and industry. Among these technologies, we are interested in
*ADAS**Automatic Driving*functions allowing fully automatic displacements of private or public vehicles in particular driving situations and/or in some equipped areas (e.g. automated car
parks or captive fleets in downtown centres).

*Service and intervention robotics.*This application domain should really explode as soon as robust industrial products, easily usable by non-specialists, and of a reasonable cost will
appear on the market. One can quote in this field of application, home robots (such as for example current vacuum cleaning robots which are both too expensive and poorly efficient), active
surveillance systems (e.g. surveillance mobile robots, civilian or military safety, etc.), entertainment robots, or robotised systems for assisting elderly and/or to disabled people. The
technologies we are developing should obviously be of a major interest for such types of applications.

*Potential spin-offs in some other application domains.*The software technologies we are developing (for bayesian programming) should also have a potential impact on a large spectrum
of application domains, covering fields as varied as the interaction with autonomous agents in a virtual world (e.g.. in the video games), the modelling of some biological sensory-motor
systems for helping neurophysiologists to understand living systems, or applications in economic sectors far away from robotics like those of finance or plant maintenance (applications
currently covered by our start-up
*Probayes*commercializing products based on Bayesian programming).

*ProBT.*

*People involved : Juan-Manuel Ahuactzin, Kamel Mekhnacha, Pierre Bessière, Emmanuel Mazer, Manuel Yguel, Olivier Aycard.*

Formerly known as
*OPL*,
*ProBT*is a C++ library for developing efficient Bayesian software. It is available for Linux, Unix, PC Windows (Visual C++), MacOS9, MacOSX and Irix systems. The ProBT library has two
main components: (i) a friendly Application Program Interface (API) for building Bayesian models, and (ii) a high-performance Bayesian Inference Engine (BIE) allowing to execute all the
probability calculus in exact or approximate way.
*ProBT*is now commercialized by our start-up
*Probayes*; it represents the main Bayesian programming tool of the
*e-Motion*project-team, and it is currently used in a variety of external projects both in the academic and industrial field (e.g. for the European project BACS and for some industrial
applications such as Toyota or Denso future driving assistance systems).

*Cycab Simulator and programming toolbox.*

*People involved : Cédric Pradalier, Christophe Braillon, David Raulo, participation of the SED team.*

In order to perform pre-test and to provide help for Cycab developers, a Cycab simulator has been developed. This simulator is intended to simulate hardware and low-level drivers, in order to produce a temporal behaviour (refresh frequency, scheduling...) similar to what can be found on the Cycab. Furthermore, a hierarchy of C++ classes has been developed in order to keep a consistent interface between the simulated Cycab and the real one. Applications written and tested on the simulated robot can then be settled to the real one with one minor modifications (instantiating one class or the other). Sensors and environment are also simulated, so that complete applications can be developed on this test bed. Finally, we also provide developers an TCP/IP controllable Cycab, consistent with simulated and real Cycab in term of C++ interface.

*Markov models toolbox.*

*People involved : Olivier Aycard.*

This toolbox is a C++ library for prototyping applications for interpretation of temporal sequences of noisy data. It is available for Linux and PC Windows (Visual C++). The Markov models toolbox has two main components: (i) a definition of Markov models and learning of its parameters component. This component permits to manually define the topology of a Markov model, and to automatically learns the parameters of the defined model. Original learning algorithms have also been developped to automatically build the topology of the model and estimate its parameters. The result of this part is a set of Markov models, where each model is trained (ie, estimated) to recognize a particular type of temporal sequence of noisy data. (ii) an interpretation component. Its goal is to interpret a temporal sequence of noisy data and to determine the most probable corresponding Markov models. This Markov models toolbox is patented under the french APP patent #IDDN.FR.001.280011.000.S.P.2004.000.10000 and has been used to perform a preliminay study of recognition of behaviours of a car driver in cooperation with TOYOTA and also to interpret sequence of noisy sensor data of mobile robots

*Grid Occupany Wavelets (GROW).*

*People involved : Manuel Yguel, Francis Colas, David Raulo.*

These software components are C++ libraries for designing applications that build dense representation of the occupancy function of a environment from telemetric sensor measurements either 2D or 3D. It is available for Linux. The GROW library suit has several components:

definition of telemetric sensor models from their intrinsec parameters such as failure rate or sensor uncertainty and the world parameter of the occupancy prior. It is the goal of the telemSensMod library.

definition of change of coordinate systems from sensor own coordinate system to world coordinate system. It is the goal of the dualGrids library.

definition of the sampling functions to sample both of the grids of the coordinate systems. It is the goal of the telemSample library.

definition of Haar wavelet transform and wavelet representation through a sparse tree representation. It is the goal of the WTree library.

definition of the hierarchical rasterization of a telemetric range measurement set, matching function between tree world representation and bundle of telemetric measurements for a certain sensor pose and definition of the intersection queries functions for the special case of a laser range-finder. It is the goal of the scanWT library.

Other libraries are part of GROW as display, sensor driver, concurrent algorithms or debug and trace tools: MY_errorHandler, MY_s11n, colorMapper,glImageTextureMapper,glLaser,laserDriver, laserRangeFinderAPI, occupancyGrid. This Grid Occupany Wavelets software components are declared under the french APP declaration and has been used to scientific experiments described in a scientific article, submitted to ICRA 2007.

*Bayesian Occupation Filter (BOF) Toolbox.*

*People involved: Kamel Mekhnacha, Tay Meng Keat Christopher*

The BOF toolbox is a C++ library that implements the Bayesian Occupation Filter. It is often used for modelling dynamic environments. It contains the relevant functions for performing bayesian filtering in grid spaces. The output from the BOF toolbox are the estimated probability distributions of each cell's occupation and velocity. Some basic sensor models such as the laser scanner sensor model or gaussian sensor model for gridded spaces are also included in the BOF toolbox. The sensor models and BOF mechanism in the BOF toolbox provides the necessary tools for modelling dynamic environments in most robotic applications. This toolbox is patented under two patents : ``Procédé d'assistance à la conduite d'un véhicule et dispositif associé'' n° 0552735 (9 september 2005) and ``Procédé d'assistance à la conduite d'un véhicule et dispositif associé amélioré'' n° 0552736 (9 september 2005)

Related to close field of research of the e-Motion team-project, these softwares are not used anymore by the researchers of our research team.

*VisteoPhysic.*

*People involved : Cesar Mendoza, Kenneth Sundaraj, Christian Laugier.*

This library provides efficient tools for deformable object simulation. It is patented under the french APP patent #IDDN.FR.001.210025.000.S.P.2004.000.10000.

*ColDetect.*

*People involved : Christian Laugier, Kenneth Sundaraj.*

This library has been implemented for providing robust and efficient collision detection, exact distance computation, and contact localisation of three-dimensional polygonal objects. It is patented under the french APP patent #IDDN.FR.001.280011.000.S.P.2004.000.10000.

Perceiving and understanding the surroundings dynamic environment of an intelligent vehicle or of an autonomous mobile robot is an important step towards safe navigation or driving. In this context, we are focusing on two major complementary points: (1) efficient reconstruction of stationary components of the environments, and (2) detection and robust tracking of the mobile components (cars, pedestrians, bicycles, etc.).

Our approach to solve this problem is to use sensory data fusion mechanisms, to map the related information into 2D or 3D Occupancy Grid models associated when necessary to some velocity distributions, and to simultaneously track the observed entities. This approach allows us to apply an unified framework for processing uncertainty in the reconstruction, the data fusion, the estimation/prediction, and the tracking processes. The environment reconstruction problem is addressed in the sections , and ; the interpretation of the sensed dynamic scene is addressed in section ; the multi-target tracking problem is addressed in section ; the prediction of the future probable motions of the observed moving entities is addressed in section .

The expected output of this step is an evolutive ``dynamic map´´ updated on-line, related to a continuous assessment of the risk of collision.

In a second step, to extract the moving objects and track them, we identify the actual dynamic parts of the environment by building an occupancy grid of the actual perceived environment (ie, a snapshoot of the current environment) and a second one corresponding to the static environment and compare these two grids (section ). While the vehicle acquires its map, it has to simultaneously localize itself relative to this map. This problem is known as the Simultaneous Localization And Mapping problem (section ).

Once moving objects are detected and localized, we need to track them. To achieve this task, we associate this list of extracted moving objects with the list of objects previously present in the environment using filtering (section ), association and objects management techniques.

An implementation of the complete architecture could be found in .

Finally, to be able to predict the future motions of the vehicle or compute the risk of collision with other objects, we need to predict the future motion of the other objects (section ).

In 2005, We have started to develop a new representation using multiscaled time-space representations like wavelets. The challenge with this new data structure was to make the fusion process available even if the information is compressed in the wavelet space. This technique has been implemented and tested and produces good results . With this framework we have reduced the size of occupancy grids by 80%. Moreover with our experiment the accuracy of the map at different scales seems relevant and it leads in the future to possible multilevel path-planning algorithms.

However, this first work has highlighted two major issues that need to be studied more deeply:

The proposed algorithm required a ``local intermediate representation'' to perform the wavelet transform on, uncompatible with a 3D instance of the problem,

The rate of compression in the wavelet occupancy grids is strongly dependant on the smoothness of the sensor models and of the accuracy of their usage for map building.

Therefore, we have formalized the equations of telemetric sensors for occupancy grids, such as the resulting sensor models is smooth even in two dimensions. This formalism relies on a general switching model of discrete coordinate systems. The algorithm we have presented is exact and leads to anytime precision procedures for any kind of 2D switching of discrete coordinate systems. This framework is useful because the issue of switching of discrete coordinate systems is fundamental in most of bayesian modelling. Moreover, the implementation is useful as a reference benchmark for any other implementation and especially real-time one using graphical processors. From that start point, we have obtained results that allows to fuse up to fifty telemetric sensors in the same map with a GPU. This works led to two publications one conference paper and one journal paper (submitted to ICRA 07).

In other hand, we have designed a totally new algorithm that solved point 1. The results give the possibility to construct a 3d occupancy grid at 30Hz with a laser
rang-finder and were validated in simulation with a virtual rotating laser range-finder (fig:
). The memory size of the 3D wavelet occupancy grid
obtained for the INRIA car park was 23MB, the tree representing the structure contains 178304 nodes i.e. 5MB of data. The map covered a volume of 120m x 120m x 120m with cells of 0.1m x 0.1m
x 0.1m (1.728 billions of cells to compare with the number of nodes in the wavelet tree structure
*i.e.*
0.01032%of data is sufficient for representing the whole occupancy volumic function). The INRIA car park buildings occupy a volume of 50m x 50m x 16m in
the volume map (This work has been submitted to ICRA 07).

*Perspectives:*In the very close future we intent to produce grid based fast-slam for the wavelet grid representation. That involves designing a good localisation algorithm for the 3d
case. In the further future we plan to give object extraction for the wavelet grid representation and finally integrate our world modelling in the tracking algorithms framework of the
references given at the begining of this section.

To navigate in open environments, that is to say where there is a lot of moving obstacles (pedestrians, bicycles, cars, ...), we need to detect obstacles (moving or not) and to estimate their state (position, speed, ...). The current state of the art does not allow to estimate obstacles motions.

In this work, we focus on obstacle detection using fusion of several vision techniques. At first we focus on fusion between stereo-vision and optical flow. Both approches are model-based: we try to extract outliers of a ground plane model. The resulting objects are obtacles (i.e. not located into the ground plane).

The first part of this work has been carried at CSIRO (Brisbane) with K. Usher and C. Pradalier during a 3 months stay of C. Braillon at CSIRO. It consists in extracting the ground plane in a 3D point cloud given by a stereo camera. The model used is a basic cartesian plane equation (4 parameters). After computing the parameters, it becomes possible to extract the outliers by measuring their height with respect to the computed plane. The output of this algorithm is a 2D occupancy grid (projection of the obstacles on the ground plane using probabilistic models of occupancy) (fig: ).

In a second step, we have extracted obstacles from optical flow. Since experiments show that detection of obstacles in an optical flow field using a ground model is not accurate enough to be usable in pratical applications, we have proposed to use a reverse model. The reverse model consists in computing the ``theoretical optical flow field'' from odometry, and in trying to match this field on the camera image. The result of this technique is a kind of disparity image (like in stereo vision). An occupancy grid is then computed from this map by projecting it in a special way (fig: ).

This projection take into account the incompleteness of the inverse projection from the image to a 3D world. In this work, we proposed a way to project the image back to the 3D world using a priori knowledge on the height of obstacles. The result of this back projection is a 2D occupancy grid.

In a third step, we have decided to make the fusion between the two previous occupancy grids in order to obtain more robust results. This is performed using classical occupancy grid fusion (fig: ) .

These results have been published in , , , ,

At the same time, methods based on intrinsic scale variation have been developed. It consists in detecting objects using natural interest points and ridges. These objects are tracked to measure their intrinsic scale variation, which we have shown to be in linear relation to the time to collision.

In order to increase the computation speed, most of algorithms have been adapted to work on the graphic card processor (GPU).

Experiments have been performed on the Cycab platform on front of which is fixed a color camera. Papers about this work has been presented in .

For moving object detection, we developed an online algorithm for simultaneous localization and mapping of dynamic environments. First we used scan matching combined with grid correlation to correct the vehicle position from odometry data. Then the new measurement with corrected position is integrated into the grid map using the grid-base fusion framework. The core notion to detect moving objects is the inconsistencies between observed free space and observed occupied space. If free space appears where a static object was observed, then it probably moved. If measures appear in the areas previously seen as free, then this measures probably correspond to moving objects.

First experimental results tested on Cycab platform moving around in parking lot of INRIA demonstrated our algorithm is able to detect pedestrians in outdoor environments using laser scanner and odometry (Fig. ).

*Perspectives:*We are now testing our algorithm on new datasets from industrial partners (DaimlerChrysler, Volvo) in the framework of ProFusion project. The data is collected with
demonstrator vehicles traveling at high speed in various environments, such as: city streets, rural roads, highways and is measured from multiple sensors: multilayer laser, short range radar
and infrared-red camera, vehicle dynamics. Studying models for these new sensors and developing a suitable grid-based fusion method then a robust and reliable moving object detection will
constitute an underlying part of our future work.

Besides, we are investigating object classification methods using information about detected objects (e.g. size, geometric shape, velocity...) combined with machine learning algorithms. Detected objects can be recognized as one of some predefined categories such as: pedestrians, animals, bicycles/motorcycles, cars/buses/trucks, etc. This information will facilitate observation to observation association step as well as help to reduce significantly the size of MHT hypothesis tree.

Moreover, algorithms of road detection and road type classification based on constructed grid map are being considered. The motivation is that road detection can help object detection step to filtering out noisy and irrelevant data off-the-road and focus more on road-likeky regions. In all, the fusion of road detection, object recognition and object tracking certainly will enable a better interpretation of driving situations.

The BOF is represented as a two dimensional planar grid based decomposition of the environment. Each cell of the grid contains two probability distributions. A probability distribution on the occupancy of the cell, and the probability distribution on the velocity of the cell occupancy.

This model of the dynamic grid is different from the approach adopted in the original BOF formulation by Coué et al. . The grid model in is in 4 dimensional space wherereas the new version models the grid in 2 dimensional space. The essential difference albeit subtle is that the original model allows the representation for overlapping objects but the new model proposed does not.

In the BOF models, concepts such as
*objects*or
*tracks*do not exist; they are replaced by more useful properties such as occupancy or risk, which are directly estimated for each cell of the grid using both sensor observations and
some prior knowledge.

When an object level representation is pertinent, we demonstrate that a general clustering tracking approach makes it possible to recover objects and perform tracking at the object level despite having no object representation in the BOF (fig: )

The BOF method has been tested using Lidar data. The challenge was to construct an ``occupancy probability map'' synchronized with the video image of a real traffic scene. This occupancy probability map will be constructed using the BOF approach and Lidar data. The Lidar data will be transformed into one 2D Bayesian occupancy grids, allowing robust tracking of obstacles over space and time.

Reliable and efficient perception in dynamic environments especially when densely cluttered is still a challenge today. To work in these environments, the mobile robot needs to perceive through its sensors, which measurements often are uncertain. Most of the systems used for target tracking are based on object models. However these existing techniques require an explicit handling of data association and occlusion problems.

The lidar is a laser range finder. It features multiple planes in which many range measures are taken. It is designed to detect the reflectors. During this project, we adapted the BOF library for the use of the lidar. We used a range finder sensor model with the column-occupancy assumption, in order to integrate the data from the three recorded planes of the lidar. We developped an interface to the data as well as a graphical visualization of the BOF. This visualization allows for the projection of the BOF onto an image. This way, we can have the BOF information as an overlay over a video stream.

Figure shows an example of this visualization.

We tested the BOF on 10 datasets. These dataset consisted of a set of images and the associated lidar scans. The durations span from 4 to 11 minutes. The data were recorder either in a urban environment or on a highway and either in a fine weather or in a rainy weather. Among these sequences, we did a more in depth analysis of 13 short sequences (from 20 to 60 seconds).

This analysis has shown that the BOF was a adequate tool for representation of the obstacle in the environment. By our choice of parameters, we were able to tune the BOF for an more precise estimation of obstacle with a small relative velocity, while filtering out static obstacles or obstacles moving in the opposite direction.

To move autonomously in an unknown and dynamic environment, a robot must first perceive the different obstacle on its way. In particular dynamic obstacles, named targets, must be recognized and tracked to permit the robot to avoid them. So, the target tracking is a pre-requisite to address the autonomous motion problem.

Classical approaches of Multi-Target Tracking are designed for military applications and therefore do not take into account urban driving specificities and constrints. Numerous methods (JPDA, PMHT, etc.) consider a previously known and constant number of targets. Other methods (MHT) allow the creation of new tracks, but they are intractable in situations involving numerous appearances, disappearances and occlusions of a large number of rapidly manoeuvering objects. New methods have recently been proposed for dealing with such issues , , ; nevertheless all these methods strongly rely on a good target extraction framework.

In maneuvering target tracking we must deal with motion uncertainty of the observed target. Indeed, a target could move unpredictly in different ways with different speeds. So it's very difficult to define a model which fits all the possible motions of a target.

To deal with these motion uncertainties, Interacting Multiple Models (IMM) have been successfully applied in several applications. The IMM approach overcomes the difficulty due to motion uncertainty by using more than one dynamic model. The principle is to assume a set of models as possible candidates of the true displacement model of the object at one time. To do so, a bank of elemental filters (Kalman filter for instance) is ran at each time, each corresponding to a specific dynamic model, and the final state estimation is obtained by merging the results of all elemental filters. Also, the probability the target changes of displacement models is encoded in a transition probability matrix (TPM), i.e the transition between models which is assumed Markovian.

Nevertheless, to apply IMM on a real application a number of critical parameters have to be defined for instance the set of motion models and the transition probability matrix(TPM). In practice, the TPM is often assumed known and is chosen a priori. Even if the design of TPM for different applications have been studied, its definition and construction do not rely on the real on-line data and so such TPMs can not be adapted to the real application. Therefore it is an important issue to automatically adapt the TPM to fit the application of the IMM algorithms.

In our work, we are interested in this TPM adaptation problem. In particular we focus on defining an adaptive IMM filter suitable to the problem of tracking pedestrians. We define an on-line method to automatically adapt the TPM of an IMM according to the real observation data. The use of IMM filters is directly induced by the uncertainty on the pedestrian behavior. Indeed, a pedestrian could have various motions and could suddenly change their current motion. To add to this, a car park's configuration is not static and pedestrian trajectories vary according to this configuration. Thus, the TPM has to be continuously updated to allow an effective filtering by the IMM.

In 2006, we have developed a fast method which adapts on-line the TPM according to pedestrian trajectories and so we obtain a suitable and robust filtering. Moreover, the effectiveness of our method has been validated on a real car park environment .

To navigate or plan motions for a robotic system placed in an environment with moving objects, reasoning about the future behaviour of the moving objects is required. In most cases, this future behaviour is unknown and one has to resort to predictions. Most prediction techniques found in the literature are limited to short-term prediction only (a few seconds at best) which is not satisfactory especially from a motion planning point of view.

In 2003, we have started to explore the problem of medium-term motion prediction for moving objects. As a result, we have proposed a novel cluster-based technique that learns typical motion patterns using pairwise clustering and use those patterns to predict future motion .

During 2004 and 2005, we developed a number of libraries and utilities (environment modelling, sensor modelling, camera calibration, communications) that will be used for further experiments. From 2005 to 2006, we have developed a new learn and predict approach which addresses a number of issues that were not solved by our first proposal:

*Prediction of unobserved patterns:*our first approach was not able to address a situation where a moving object would ``change its mind'' and switch from one typical motion pattern
to another one. The new approach which relies upon the notion of goals rather than typical motion patterns will be able to address this kind of problem.

*On-line/adaptive learning:*the clustering technique used in our first approach is an off-line process taking as input a data set of observed trajectories. The new learning technique
is able to learn on-line,
*ie*to take into account newly observed trajectories while carrying out the prediction.

Our new approach represents motion on the basis of a proposed extension to the well-known Hidden Markov Models framework, that we have named Growing Hidden Markov Models (GHMM). Basically, the extension allows incremental, adaptive, real time learning of the model's parameters and structure. The key idea is to use a special kind of self-organizing maps known as topology representing networks to learn the model structure in an incremental and adaptive manner.

In order to apply this extension to the prediction problem, we have augmented the GHMM state by including in it the coordinates of the final position reached by the object, which are available during learning. This allows to representing several motion patterns with a single GHMM, on the assumption that motion is mainly motivated by the place that the object intends to reach.

Most of our early experimental work has been focused on testing our approach with real data coming from a vision system located in INRIA's entry hall. The obtained results were very encouraging and are consistent with the ones previously obtained in simulations . Further work in 2006 has been performed to apply the technique to both real and simulated parking lot environments. Our results (see fig. ) seem to confirm the viability of our approach for incremental real-time motion pattern learning and long-term motion prediction.

The problem of autonomous navigation for a mobile platform has been considered with particular attention to the task of simultaneous localization and mapping (SLAM). This activity has been developed in the framework of the European project BACS in collaboration with the ETHZ in Zurich. Two main contributions have been introduced. The first one is a new method able to improve the convergence of the metric map built during the navigation , . This method is based on the concept of the relative map approaches. The second contribution is an approach able to self calibrate the robot's sensors during the navigation , , , , .

The basic idea of the proposed approach consists in introducing a map state, which only contains quantities invariant under shift and rotation and in carrying out the estimation of this relative map through a Kalman filter. The shift and rotation invariance of the state allows significantly reducing the computational burden. In particular, the computational requirement is independent of the number of features. Furthermore, since the estimation process is local, it is not affected by the linearization introduced by the EKF.

With respect to the conventional
EKFbased
SLAMthere are two advantages:

the computational requirement is independent of the number of features;

the estimation process is not affected by the linearization, since it is local.

However, in order to have these two advantages, the relative quantities referring to distant features are not correlated. This approximation does not imply a lost of information if and only if the robot does not make a loop. Indeed, as soon as the robot accomplishes a loop, all the relative quantities become correlated. The proposed approach is able to exploit the information coming from only local dependencies among the relative quantities .

The approach was validated through real experiments carried out on a real platform in our laboratory and by using the Victoria park dataset (see fig ).

We are extending our approach in order to improve its performance. Possible directions to be explored are:

the approach has to be able to deal with the case of loop closure

the estimation process can be improved by adopting other Bayesian methods which outperform the EKF in non-linear systems. This could be done by applying for the first time methods previously developed from the control community to the SLAM. A possible method is the Polynomial Extended Kalman Filter, recently developed.

the approach can be adapted in order to deal with a dynamic environment. To this end, the first step is the introduction of a method able to predict the movement of the objets recognized as dynamic.

A new method, which allows simultaneously estimating of the robot configuration and the systematic components of the sensor error, has been introduced. The method performs during the navigation. It has been developed both for proprioceptive sensors , and exteroceptive sensors , , .

In the future, we want to find optimal controls (i.e. optimal trajectory) in order to minimize the final error on the parameters characterizing the systematic error. Preliminary results, which only regards the odometry self calibration in the case of holonomic vehicles, are already obtained .

In order to perform pre-test and to provide help for Cycab developers, a Cycab simulator has been developed. The Cycab Simulator is currently widely used by the researchers of the
*e-Motion*project-team; it has also been used in a collaboration with the RIA team (LAAS, Toulouse), and with the LAG laboratory (IMAG, Grenoble). A recent extension of the system is
also used for student home-work in the scope of robotics courses (Summer school on image & robotics 2004-2006, and Master IVR at ENSIMAG Grenoble 2006-07). The simulator project (called
CycabTK) has been published on the INRIA Forge (gforge.inria.fr). This publication aims at federating Cycab users by providing a generic toolkit. The Cycab simulator will be tested in several
INRIA Research unit (IRISA, LORIA, Rocquencourt) and in LASMEA laboratory (Université Blaise Pascal, Clermont Ferrand).

The simulator provides several simulated sensors: B&W cameras, color cameras, stereo cameras, Sick range finder, rotating Sick. It will soon include a GPS simulator.

When placed in a dynamic environment,
*ie*an environment with moving objects, a robotic system cannot stand still since it might be collided by one of the moving objects. In a situation like this, a
*real-time constraint*is imposed upon the navigation or motion planning system of the robot considered: it has a limited time only to determine its future course of action. The time
available is a function of what is called the
*dynamicity*of the environment which is directly related to the dynamics of both the moving objects and the robotic system.

Oddly enough, this real-time constraint is in most cases overlooked by previous works addressing motion planning in dynamic environments. Few recent works do attempt to take it into
account however,
*eg*
,
. These works rely upon the most efficient
motion planning techniques available today, namely randomised techniques such as the Probabilistic Roadmap
or the Rapidly Exploring Random Tree
. The average running time of these motion
planning algorithms is low enough so that they can be used on-line. This type of approach is very interesting since it offers an answer to the lack of lookahead of the regular reactive
approaches. It also seems to address the real-time constraint mentioned earlier. Unfortunately, the real-time constraint is not satisfied since the running time of a randomised technique
cannot be upper bounded.

Given the intrinsic complexity of motion planning in dynamic environments (see the complexity results established in
and
), it seems unlikely that a hard real-time
constraint could ever be met in realistic situations.
*Partial Motion Planning*(PMP) is our own answer to the problem at hand. PMP is a planning scheme that take into account the real-time constraint explicitly. PMP has an anytime flavor:
when the time available is over, PMP is interrupted and it returns a partial motion,
*ie*a motion that may not necessarily reach the goal. This partial motion is then passed along to the navigation system of the robot for execution. Of course, since only a partial motion
is computed, it is necessary to iterate the partial motion planning process until the goal is reached.

The iterative nature of PMP is doubly required when the robotic system at hand is placed in a uncertain dynamic environment,
*ie*an environment for which everything is not known in advance (in particular, the future behaviour of the moving objects). Indeed, motion planning means reasoning about the future.
When the future is unknown, one has to resort to predictions, predictions whose validity duration is limited in most cases. An iterative planning scheme permits to take into account the
unexpected changes of the environment by updating the predictions at a given frequency (which is also determined by the environment dynamicity).

Like reactive approaches, PMP faces two issues, namely the convergence and the safety issues:

What guarantee do we have that the goal will ever be reached?

What guarantee do we have that the robotic system will never found itself in a dangerous situations?

As for the convergence issue, the unrealistic convergence conditions established in
leave little hope (it is hardly surprising
if the system is placed in an environment with no a priori information about the moving obstacles). To address the safety issue, two different solutions are explored: the first one
relying upon the concept of
*Inevitable Collision States*and the second one upon the concept of
*Non Linear Velocity Obstacles*(
*cf*sections
and
respectively).

In 2006, the integration of the PMP module within the software architecture of a Cycab experimental vehicle has been furthered. Coupling with the control, perception and environment modelling modules have been achieved. Live experiments have been carried out (Fig. ). This work has yielded a number of papers , , .

In 2002, we introduced the novel concept of
*inevitable collision states*(ICS). An ICS for a given robotic system is a state for which, no matter what the future trajectory followed by the system is, a collision with an object
eventually occurs. For obvious safety reasons, a robotic system should never ever end up in an ICS.

The ICS concept has already been used in a number of applications. The first one concerns the safe motion of a mobile robot subject to sensing constraints,
*ie*a limited field of view, and moving in a partially known static environment
. The second one concerns a car-like vehicle
moving in a roadway-like environment
In both cases, the future motion of the
robotic system at hand is guaranteed never to take the system in an ICS. Although the ICS concept offers a theoretical answer to the motion safety issue, using it in practise raises a major
problem: that of the characterisation of the set of ICS for a given robotic system. In theory, determining whether a given state is an ICS or not requires to check for collision
*all possible future trajectories of infinite duration*that the robotic system can follow from this particular state!

In practise, it is fortunately possible to use the approximation property established in . This property says that a conservative approximation of the ICS set of a given robotic system can be obtained by considering only a finite subset of the whole set of possible future trajectories. What the approximation property does not say is how to select this subset. This is unfortunate because the quality of the approximation largely depends on the subset considered. If the approximation is too coarse, one might end up with most states being labelled as ICS (when in fact they are not).

In 2006, we have proposed a general principle to select the subset of trajectories that can be used to determine whether a state is an ICS or not. By introducing the concept of
*imitating manoeuvres*,
*ie*trajectories leading the robotic system to duplicate the behaviour of the environment objects (fixed or moving), it is shown how a good approximation of the ICS set can be
obtained.

We also have designed an ICS-Checker for a car-like vehicle moving in a dynamic environment (Fig. ). It is an extension of the algorithm used in that considered static environments only. This ICS-Checker integrates the above-mentioned selection principle. It is efficient (it has a polynomial complexity) and could be used in practise to compute truly safe motions for a car-like vehicle amidst moving objects. A paper presenting these two results has been submitted to the 2007 Int. Conf. on Robotics and Automation.

Designing an autonomous robotic system requires to solve a number of challenging problems in domains as different as perception, localisation, environment modelling, reasoning and
decision-making, control,
*etc*.. However, whatever the robotic system and whatever the kind of tasks it is expected to carry out, at some point, it has to move. Motion determination is therefore a fundamental
issue that has been widely addressed by the Robotics community and a large number of autonomous navigation architectures have been proposed (
*cf*the recent review established in
).

From the motion determination perspective, these navigation architectures can be broadly classified into
*deliberative*(
*aka*
*motion planning-based*) versus
*reactive*approaches: deliberative approaches aim at computing a complete motion all the way to the goal using motion planning techniques, whereas reactive approaches determine the
motion to execute during the next time-step only.

Deliberative approaches have to solve a motion planning problem
: they require a model of the environment as
complete as possible and their intrinsic complexity is such that it may preclude their application in dynamic environments (
*cf* §
). Reactive approaches on the other hand can operate
on-line using local sensor information: they can be used in any kind of environment whether unknown, changing or dynamic. This accounts for the large number of reactive approaches that have
been developed over the years. Reactive approaches however face a challenge: the convergence towards the goal is not guaranteed (local minima problem).

In an attempt to bridge the gap between deliberative and reactive approaches, a complementary approach has been proposed based upon
*motion deformation*. The principle is simple: a complete motion to the goal is computed first using a priori information. It is then passed on to the robotic system for execution.
During the course of the execution, the still-to-be-executed part of the motion is continuously deformed in response to sensor information acquired on-line, thus accounting for the
incompleteness and inaccuracies of the a priori world model. Deformation usually results from the application of constraints both external (imposed by the obstacles) and internal (to
maintain motion feasibility and connectivity). Provided that the motion connectivity can be maintained, convergence towards the goal is achieved.

It appears that the different motion deformation techniques that have been proposed
,
,
,
all performs
*path deformation*. In other words, what is deformed is a geometric curve,
*ie*the sequence of positions that the robotic system is to take in order to reach its goal.

The problem with path deformation techniques is that, by design, they cannot take into account the time dimension of a dynamic environment. For instance in a scenario such as the one
depicted in Fig.
, if the obstacle moves fast, it may be better to modify
the velocity of the robotic system along the path so as to avoid collision with the moving obstacle (by slowing down or accelerating). But if the obstacle stops or moves slowly at the robot's
path, then it may be better to deform both the path and velocity of the robot simultaneously. To achieve this, it is necessary to depart from the path deformation paradigm and resort to
*trajectory deformation*instead.

Unlike path deformation wherein spatial deformation only takes place, trajectory deformation features both
*spatial and temporal*deformation meaning that the planned velocity of the robotic system can be altered thus permitting to handle gracefully situations such as the one depicted in
Fig.
.

In 2006, we have proposed the first trajectory deformation scheme. It operates in the state
×time space of the robotic system considered. It assumes that a model of the future evolution of the environment is available and updated regularly. At
each update, the model of the future is used to determine whether the current trajectory is collision-free. If not, it is deformed accordingly (Fig.
). The trajectory deformation scheme operates in two
steps: a collision avoidance step and a connectivity maintenance step. In the collision avoidance step, repulsive forces generated by the obstacles deform the trajectory so that it remains
collision-free. The purpose of the connectivity maintenance step is to ensure that the deformed trajectory remains feasible,
*ie*that it satisfies the robotic system's kinematic and/or dynamic constraints.

A paper presenting this technique has been submitted to the 2007 Int. Conf. on Robotics and Automation.

Automatic navigation in dynamic real world environments is an open field of research. The main problems rise from the integration of existing path planning methods with the incomplete and uncertain information of the world provided by real sensors. Among the strategies proposed in literature, the velocity obstacle approach (LVO, ) takes into account the dynamics of the obstacles and allows to compute the time to collision for all admissible movements of the robot. Various extension have been proposed, to generalize the approach to non linear velocity of the obstacles (NLVO, ) and to uncertain knowledge over the velocity of the obstacles (PVO, ). This year we focused our work on the integration of probabilistic velocity obstacle with a dynamic occupancy grid. The sensors observations are elaborated to give a dynamic occupancy grid, inspired to the Bayesian Occupancy Filter concept (BOF, ).

The occupancy grid carries the information about the occupation of the space and an histogram of probabilities over velocities for each cell; the velocity obstacles are computed for each cell of the space and for the probable velocities represented in the grid. Provided a threshold on the probability of collision, the time to collision is estimated and a safe control and a time of action is chosen. The advantage of this new discretized approach is that we can easily represent in the grid the uncertain knowledge about both the obstacles and the non-observed and occluded space, taking into account in a unified probabilistic framework not only the uncertainty on motion, position and shape of observed obstacles but also the danger coming from eventually hidden obstacles. We tested the algorithm simulating different environments and sensor inputs.. Preliminary results have been submitted to ICRA'07.

In this example the robot faces a crossing. Figure shows the complete simulated environment: the robot is the circle at the bottom and has a positive velocity in the y direction. Static obstacles delimit the environment and two other moving obstacles, respectively at the right and left of the image move toward the center of the crossing. The robot goal location is on the upper part of the image. Different sensor inputs have been simulated. In the first experiment (Fig (a)) the input grid represents the whole environment: the velocity of the obstacles is known with certainty and there are not occluded zones. The robot performs the obstacle avoidance passing near the obstacles and reaching the goal with a short trajectory. In the second experiment (Fig (b)) a Gaussian uncertainty on obstacles velocity is simulated .The robot tries to keep its trajectory further away from obstacles; the path results longer as the robot performs wider curves to avoid collisions. In the third experiment (Fig (c)) a distance sensor input is simulated. The visible distance is limited by a short range and the occluded zone hide obstacles and their shape. The obstacle's velocities are known with the same Gaussian uncertainty of the second experiment. The robot goes slower and the motion controls are re-planned more often. The robot reacts appropriately to the unexpected obstacles and successfully reaches the goal.

Last year, we addressed the problem of controlling autonomous sensory-motor systems situated in dynamic environments, in order to achieve autonomous behaviour .

Due to the natural complexity of this problem, it was proposed to exploit partial independence between different domains of interest in the environment, as well as use behaviour selection and attention focusing mechanisms to simplify programmations and reduces overhead when processing raw sensor data.

This year, the implementation on a mobile robot was presented as evidence of the feasibility of the proposal when solving practical problems . The presented experiment consisted in robot interacting with subjects in the environment and obstacle avoidance, as shown in figure .

During last decade, the bayesian point of view has gained much popularity in the machine learning community due to its clean theoretical foundations, and to the apparition of numerous efficient inference algorithms. Bayesian learning consists in inferring a set of parameters of a predefined model from the information contained in some data. The application of this general approach to robot programming is called the Bayesian Robot Programming framework . It consists in representing the sensory-motor space of a robot with a probability distribution. This distribution can then be used to take motor decision in order to accomplish a predefined task.

In this framework, a Bayesian programmer starts to describe the task by specifying some
**preliminary knowledge**to the robot. This is the model definition stage.

Establishing preliminary knowledge can be divided into three parts:

Define relevant variables for the problem;

Assume some conditional independencies;

Choose prior distributions on those variables.

Usually this preliminary knowledge comes from the human programmer, but thanks to the bayesian formalism, it's also possible to express the model as a variable and to infer it from data.

Last year we proposed an algorithm able to discover the ``relevant variables'' part of a new program. This has been achieved under the supervision of another Bayesian Program, and has been published in ICRA'05 .

Bayesian programming techniques used in robotics can be applied to synthetic characters such as those in video games. Games actually provide for rich virtual worlds in need of intelligent autonomous creatures.

We have worked on a technique called enhanced fusion by coherence, which allows building elementary tasks from multiple motor commands, each expressed as a probability distribution. We combine these tasks into complete behaviours by a method of sequencing called inverse programming.

The combination of these techniques allows building hierarchical models of behaviour, that rival finite state machines in both simplicity and efficiency. Wihtin this framework, the difficulty of manual specification remains under control when complexity grows. Moreover, learning behaviours by demonstration becomes possible. An adaptation of the Baum-Welch algorithm permits to learn sequenced behaviours incrementally, in real time. This opens up two possibilities : the human player can teach a behaviour to an autonomous character, by just showing her how to play ; the developper can adjust behaviours using learning, without the need to specify all parameters by hand.

Relative motion between an observer and the objects in a visual scene leads to a deformation of the image on the retina, called optic flow. Optic flow depends heavily on the three-dimensional (3-D) shapes and motions of the objects in a scene, and can therefore be used to extract 3-D information about scene geometry. The capacity to extract 3-D shape or structure from motion ( sfm) is also known as motion parallax or the kinetic depth effect ( , ). Although it is simple to derive the optic flow corresponding to given 3-D geometry and motion, perception faces the inverse problem, to derive 3-D shape and motion from optic flow. Because an infinite number of combinations of geometry and motion can lead to the same optic flow, sfmis an ill-posed inverse problem.

It is commonly believed that it is at least partly solved by a constraint called the
*rigidity assumption*, the hypothesis that optic flow is due to 3D translations and rotations of a rigid body. This drastically reduces the number of degrees of freedom associated with
motion, and it can be shown that under this assumption, both structure and motion can theoretically be recovered from very little optic flow information (
). Psychophysical results show that human
performance on some
sfmtasks is at least broadly consistent with predictions based on the rigidity assumption (
,
). Although most of these studies involve an
immobile observer, it is known that
sfmis also effective when optic flow is generated by the observer's own head movement about a stationary 3D scene (
). Until recently, it has been thought that
3D shapes perceived in subject-motion
sfmare the same as those perceived in object-motion
sfm, as long as the optic flow is the same (
,
). However, in some cases this turns out to
be false: even when optic flow is kept constant, the observer's movement does influence perceived 3D shape
.

We had formerly proposed a Bayesian model of the perception of planes from optic flow. This model was able to reproduce the results from six different psychophysics experiments and has been submitted to an interdisciplinary journal ( ).

In accordance with our theoretical framework, we developed a Bayesian process for the reparametrization of optical flow. Indeed, the model we proposed formerly is based on high level optical flow representation. The aim of this work was to compute the input of the perception model using more low level features.

More precisely, we consider a set of points with their velocity in the image, and we compute the derivatives of the global optical flow up to the second order. This can be done based on the naive Bayesian fusion structure. However, the complexity of this solution is very high as the output space—the joint set of all possible derivatives of the optic flow—is large. Therefore we developed a set of separated filters for each component of the optic flow based on subset of 1, 2, or 3 points at a time for respective order of derivation of 0, 1, or 2. Each subset voted for its own evaluation of the component.

This algorithm was implemented and we have shown that its statistical properties were compatible with our perception model. Indeed a large uncertainty on the outcoming evaluation of the optic flow by the reparametrization algorithm would have negatively impacted the performance of the perception model. Figure shows an example of outcoming distribution. The sharp peak illustrates the precision over the evaluated optic flow components.

This work is detailed in Colas' PhD thesis, which has been defended late january ( ).

This project is a collaboration between France and Singapore, in the context of the STIC-ASIE program. It was carried out at
C^{2}i(Centre for Computational Intelligence), a laboratory located at NTU, Singapore, under the direction of Michel Pasquier.

The aim of our project was to implement intelligent speed adaptation (ISA, see ) for a car to anticipate curves by adapting its speed according to the degree of curvature of the road .

In order to implement ISA, we used the GenSoYagerFNN, a fuzzy neural network developped in the laboratory, which has shown good performances for autonomous driving .

The information about the environment is given by a camera put in front of the car, and image processing is used for lane detection and to extract necessary data. These data are then fed into the network, which gives in output the vehicle controls (i.e. steering, brake and throttle). The GenSoYagerFNN must first learn from a training set, given by collecting data from a human driver, in order to be able to drive correctly the vehicle.

During the project, we observed the need to anticipate the upcoming curves, when these one were too sharp. So an anticipation system was added to our longitudinal control, adding a memory to our system.

Experimental results have shown that the GenSoYagerFNN was able to adapt correctly its speed according to the curvature of the road, in the same way that human do. The learning process from a human training set has been a success for some simple tests, and results are encouraging for the continuation of the project. Results obtained will be submitted in 2007.

This work concerns sensorimotor systems and the modelling of their perception using the Bayesian formalism. Given a phenomenon and some sensors, we focus on how to get information on the phenomenon by putting together the information of the different sensors. In robotics, one of the most common Bayesian models proposed in this area is the sensor fusion model.

We are interested in verifying whether this robotic sensor fusion model can be applied to modelling human perception. Indeed, one of the starting point of the analysis of human perception is that it is highly multimodal, with fusion of different sources of information like vision, touch, etc.

We proposed to apply Bayesian modelling to the visuoacoustic fusion of information for vowel recognition . The first step was to propose several Bayesian models. All of them apply the sensor fusion schema but differ with respect to the nature of the internal space in which the fusion is carried out.

Then, we compare models to experimental data in two cases : matching data (the vowel seen corresponds to the vowel heard) and conflicintg data (the subject is presented discongruent stimuli). Thanks to Bayesian model selection, we can quantitatively compare the models. It is an original method because previous works usually confine to defining only one model.

This work was done in collaboration with Jean-Luc Schwartz of the ICP laboratory (CNRS, Grenoble), and under the joint supervision of Julien Diard of the LPNC laboratory (CNRS, Grenoble).

ProBT is a C++ library for developing efficient Bayesian software. This library has two main components: (i) a friendly Application Program Interface (API) for building Bayesian models and (ii) a high-performance Bayesian inference and learning engine allowing execution of the probability calculus in exact or approximate ways. This paper is not intended to describe ProBT. It will only focus on its underlying inference algorithms.

The aim of ProBT is to provide a programming tool that facilitates the creation of Bayesian models and their reusability. Its main idea is to use ``probability expressions'' as basic bricks to build more complex probabilistic models. The numerical evaluation of these expressions is accomplished just-in-time: computation is done when numerical representations of the corresponding target distributions are required. This property allows designing advanced features such as submodel reuse and distributed inference. Therefore, constructing symbolic representations of expressions is a central issue in ProBT.

During this year the main development of ProBT has been carried out by Probayes: a spin-off born from the e-motion project. Both, e-motion and Probayes, are part of the European project Bayesian Approach to Cognitive Systems (BACS). The development of ProBT has been effectuated taking into account the goals of the BACS project partners. In 2006 the main subjects of development and research has been:

The ProBT objects serialization: saving or transmitting ProBT objects, for example, onto a file, or a memory buffer. The serialization allows the reconstruction of ProBT object that
has been constructed in the past by the execution of another program. The serialization is very useful in divers applications such as the parallel computation where
nprocessors shares the same object to execute different computations.

ProBT interface with applications, libraries and languages such as (Excel, PHP, Python, and Graphical tools).

ProBT Graphical Interface.

Other subjects of development and research were launched this year such as:

Specialized probability calculus algorithms.

Learning (e.g. Maximum likelihood, Maximum Entropy, Expectation-Maximization, Structures Learning)

Probabilistic calculus based on a Graphical Process Unit (GUP) architecture.

Cybercars 2 [January 2006-December 2008]

European project IST Cybercars 2, ``Close Communications for Cooperation between Cybercars''.

This Project is driven by the vision that, in the short term future, Cybernetic Transport Systems (CTS) based on fully automated urban vehicles (the cybercars) will be seen on city roads
and on new dedicated infrastructures. Such systems have been developed and evaluated in the scope of the CyberCars (www.cybercars.org) and CyberMove (www.cybermove.org) projects of the 5th
FWP and are now being deployed. However, presently these CTS can only operate in low demand environments where little interaction between vehicles is anticipated. In order for these systems
to address high demands, more cooperation between vehicles is needed. This is the topic of this Project, based on vehicle-vehicle and vehicle-infrastructure communications and vehicles
coordination. We will address in particular the cooperation between vehicles running at close range (platooning) and at intersections (merging, crossing). The contribution of
*e-Motion*to Cybercars 2 focuses on cooperative driving.

Profusion [February 2004-February 2008]

European project, PreVENT Programme (Preventive and Active Safety Applications) Profusion, ``Project for Robust and Optimized Perception by Sensor Data Fusion''

The second phase of ProFusion started in April 2005. The goal of this second phase is to develop new concepts, methods and theories for sensor data fusion for Automotive Industry. These solutions will permit to perceive the environment surrounding a car and automatically build a model of this environment. This model will be interpreted it in order to assist the car driver. The resulting prototypes will be tested and validated on European car demonstrators. The specification and design of the system for sensor data fusion is still in development until the beginning of 2006. In 2006, the development of algorithms and methods will start, and the integration of the resulting prototypes on european car manufacturers demonstrators is planned for 2007. Our group plans to integrate its prototype on a Volvo truck and a Mercedes-DaimlerChrysler car.

The Profusion phase2 consortium is constitued of 10 european car manufacturers, suppliers and research institutes: (BMW, CRF, DaimlerChrysler, Delphi, Forwiss, ICCS, INRIA, Sagem, TUC, Volvo).

Puvame [October 2003-April 2006]

National project, Predit Programme Puvame ``Protection des Usagers Vulnérables par alarme et Manoeuvre d'Evitement''

An important number of accidents between vulnerable road users and moving traffic could be avoided by improving the abilities of visibility and estimation of the situation by the driver,
and by putting in action an alarm system addressed to the driver and the road user in danger. This project will contribute to reduce the number of accidents of this type, by developing the
principal following functionalities: (1) Improvement of the abilities of perception of the driver in close and average distance environments by dated fusion; (2) Detection and estimation of
the dangerous situations, by analyzing current data relating on the "behavior of the driver" and to the estimation result of the "dangerosity" of the operations in progress; (3) Activation
of alert actions associated to vehicle and vulnerable users; (4) Integration and experiments on vehicles and preliminary study on bus and/or trams. INRIA Rhône-Alpes is coordinator of the
project. The partners are:
*e-Motion*project (Inria Rhône-Alpes) and Imara project (Inria Rocquencourt), Ecole des Mines de Paris (EMP), INRETS, Intempora, Probayes, Robosoft, Connex.

Mobivip [November 2003-June 2007]

National project Predit Mobivip ``Véhicules Individuels Publics pour la Mobilité en centre ville''.

( http://www-sop.inria.fr/mobivip).

Mobivip is a research project involving 5 research laboratories and 6 industrial partners in order to implement, evaluate and demonstrate the impact of Information Technologies on a
novel mobility service for downtown areas, such mobility service being based upon a fleet of small public urban vehicles integrated in the existing public transport system. The contribution
of
*e-Motion*to Mobivip focuses on autonomous driving in open and dynamic environments.

LOVe [December 2005-December 2008]

National project, Predit Programme LOVe ``Logiciel d'Observation des Vulnérables''. The goal of this project is to develop new methods and prototypes for improving vulnerable security and car driver's confort. Artificial perception algorithms will be developped for the localization of the road and for the detection, localization, recognition and tracking of mobile objects using different kind of sensors (lidar, radar, cameras). This project is constituted of 14 french partners (Renault, Valéo, INRETS/LCPC/LIVIC, INRIA (emotion, icare, imara), CEA List, Université Paris Sud, CNRS/heudiasyc, LASMEA, Armines (CNN, CAOR)).

Toyota Motors Europe [Feb 2006 - Jan 2009]

The contract with Toyota Motors Europe began its first phase in the beginning of Feb 2006. It is a joint collaboration involving Toyota Motors Europe, INRIA and ProBayes.

This contract aims to develop innovative technologies in the context of automotive safety. The idea is to improve road safety in driving situations by equipping vehicles with the technology to model the dynamic environment and sense incoming danger. This is with the aid of sensors commonly used in automotive applications such as cameras and lidar.

Denso [june 2006-September 2006]

Industrial contract ``Application of Bayesian Occupancy Filter technology to DENSO LIDAR data'' involving INRIA, Denso and Probayes.

The objective of the collaboration was to evaluate how the ``Bayesian Occupancy Filtering'' (BOF) INRIA technology works with DENSO's Lidar. The BOF has initially been developed by INRIA (see C. Coué Thesis and related INRIA publications); then, it has been improved in collaboration with the Probayes company.

*BIBA (Bayesian Inspired Brain and Artifacts - IST-2001-32115) [November 2001-March 2006]*

We initiated the BIBA project in year 2000. BIBA finished finished in march 2006. We think that over the next decade, probabilistic reasoning will provide a new paradigm for understanding neural mechanisms and the strategies of animal behaviour at a theoretical level, and will raise the performance of engineering artefacts to a point where they are no longer easily outperformed by the biological examples they are imitating. The BIBA project has been motivated by this conviction and aimed to advance in this direction. The twin scientific objectives of the BIBA project were:

To reconsider in the light of Bayesian probabilistic reasoning our methodology models, algorithms and techniques for building artefacts for the "real world", gaining inspiration from the way living beings have evolved and adapted to the properties of their natural environments, and constructing robots that use these principles.

To provide a firm Bayesian basis for understanding how biological systems use probabilistic logic to exploit the statistical properties of their environ- ments, both at the level of neural mechanisms, and at the level of strategy, and to use robots to test the validity of these ideas.

To reach these objectives, the project was organized along 3 axes of research and development:

*Neural basis of probabilistic inference*: The objective of this first axe of research was to identify how the nervous system does (or at least may) implement probabilistic
inference. We constructed well-defined models of how probabilities may be represented and manipulated, and test predictions with psychophysical performance measures and studies of
regional brain activation. We improved our understanding of neural mechanisms and derive new ideas for the implementation of probabilistic inference in engineering systems.

*New probabilistic models and algorithms for perception and action*: The main goal was to illustrate how probabilistic computation, and more specifically, Bayesian programming, may
account for global behaviours of organisms in interaction with their environment. We focused on specific questions concerning multi-sensory perception and motion control. We developed
new probabilistic models that explain the observed behaviours in human or animals and to implement them on autonomous artefacts.

*New probabilistic methodology and techniques for artefact conception and development*: We used the Bayesian and learning paradigm to develop robots that acquire repertoires of
reactive probabilistic behaviours (synergies), builds combinations, hierarchies and temporal sequences of these behaviours (strategies) and can be trained for different tasks. We
developed a formalism and a programming methodology called Bayesian Programming and an API to automate the probabilistic computation called ProBT®.

The partners of the BIBA project are: Laboratoire GRAVIR - INRIA - CNRS (ccordinator), Laboratoire de Physiologie de la Perception et de l'Action (LPPA) - Collège de France, Gatsby Computational Neuroscience Unit - University College of London, Department of Physiology - University of Cambridge, Laboratoire de Système Autonomes - Ecole Polytechnique Fédérale de Lausanne, Non-Linear System Laboratory - Massachusetts Institute of Technology

*BACS (Bayesian Approach to Cognitive Systems - FP6-IST-027140) [january 2006-february 2011]*

As the BIBA project gave very interesting results, the European Commission decided to finance a new and extended project on the same thematic.

The BACS project started at the very beginning of 2006 for five years. It is an IP project including a number of new partners and especially some industrial partners to develop applications of the bayesian technology.

Despite very extensive research efforts contemporary robots and other cognitive artifacts are not yet ready to autonomously operate in complex real world environments. One of the major reasons for this failure in creating cognitive situated systems is the difficulty in the handling of incomplete knowledge and uncertainty.

In this project we will investigate and apply Bayesian models and approaches in order to develop artificial cognitive systems that can carry out complex tasks in real world environments. We will take inspiration from the brains of mammals including humans and apply our findings to the developments of cognitive systems.

The Bayesian approach will be used to model different levels of brain function, from neural functions up to complex behaviors. This will enable us to show that neural functions and higher-level cognitive activities can coherently be modeled within the Bayesian framework. The Bayesian models will be validated and adapted as necessary according to neuro-physiological data from rats and humans and through psychophysical experiments on human.

The Bayesian approach will also be used to develop four artificial cognitive systems concerned with (i) autonomous navigation, (ii) multi-modal perception and reconstruction of the environment, (iii) Semantic facial motion tracking, and (iv) human body motion recognition and behavior analysis.

The conducted research shall result in a consistent Bayesian framework offering enhanced tools for probabilistic reasoning in complex real world situations. The performance will be demonstrated through its applications to drive assistant systems and 3D mapping, both very complex real world tasks.

The partners of the BACS project are: Laboratoire de Système Autonomes - Ecole Polytechnique Fédérale de Zurich (coordinator), Laboratoire Gravir - INRIA - CNRS, Laboratoire de Physiologie de la Perception et de l'Action (LPPA) - Collège de France, Département d'étude Cognitive - Ecole Normale Supérieure, Institute for Biological Cybernetics - Max-Planck society, Institut Dalle Molle d'Intelligence Artificielle, Les Hopitaux Universitaires de Genève, Faculdade de Ciencas et Technologia - Universidade de Coimbra, ProBAYES S.A.S, Bluebotics SA, Electricité de France (EDF)

*France-Singapore "Neurophysiology and robotics" [June 2004-June 2008]*

This CNRS-PICS project involving the Collège de France (LPPA), INRIA
*e-Motion*, the University of Singapore (NUS), and the
*IPAL*joint CNRS-NUS laboratory in Singapore. The objective is to study some aspects of the physiology of human vision, and to try to develop robotics models inspired from biological
systems. An application of this research we will be to control a wheelchair from natural human control channels. This research involves a co-directed PhD student located in singapore.

*ICT-Asia "FACT" [October 2005-December 2007]*

The Fact project is a joint research project in the scope of the ICT-Asia programme founded by the French Ministry of foreign affairs, the CNRS and INRIA. It aims at conducting common research activities in the area of Intelligent Transportation Systems (ITS). The main objective is to develop new technologies related to the concept of "Cybercar". The project involves the following research teams : e-Motion project at INRIA Rhône-Alpes (leader), Imara project at INRIA Rocquencourt, LASMEA Laboratory at Clermont-Ferrand, SungKyunKwan University (Korea), Shangai Giao Tong University (China), Nanyong Technological University (Singapore) and Tokyo University (Japan).

Since October 1997,
*e-Motion*has a collaboration with Riken Institute in Tokyo in the multi-robots systems field . Crossed visits have occured for the last three years. In 2002, Thierry Fraichard spent 4
months in Riken Institute and Igor Paromtchik spent 4 months at INRIA. Informal exchanges are still going on.

The common laboratory, named
*Intelligent Vehicle Lab*, between Nanyang Technological University of Singapore (NTU) and Inria has started in November 1998, in the framework of the scientific collaboration in the
field of autonomous vehicles. This collaboration has brought: (a) an important number of crossed visits and stays (one week to several months) of researchers, (b) Singaporeans
students in Inria (level undergraduate to graduate), and (c) organization of workshops (1999-2002). in 2003, Julien Diard has been a Postdoc student in NUS and a co-directed PhD
(Brice Rebsamen) will begin in January 2004 in NUS. In addition, a PICS CNRS project has been accepted in 2004 with NUS and LPPA(College de France, Alain Berthoz).

In the scope of the France-Korea Safemove project,
*e-Motion*has a collaboration with the SungKyunKwan University at Suwon (Korea).

see the description of the ICT-ASIA ``Fact'' project above.

Collaboration in the field of dextral handling begun with the stay of professor K. Gupta at Inria Rhône-Alpes in 1995, continued by several long stays of Moez Cherif and Juan Manuel Ahuactzin, and with crossed visits. Common publications has been done in 2000 and 2001. Informal exchanges are still going on.

The thematic network "Image et Robotique" has been implemented from the French-Mexican symposium in Computer Sciences and Control (JFMIA'99) which has been held in Mexico in March 1999. The main goal of this network is to promote and increase the french-mexican cooperations in Image and Robotics in scientific, academic and industrial fields. This network has been effectively settled in 2000. It supports a yearly school, students exchange and crossed visits since 2000.

The dissemination of results and the active participation to international scientific events (see bibliography) are two essential activities of the
*e-Motion*project-team.

Some members of
*e-Motion*participate to some international committees:

C. Laugier is a member of the steering-advisory committee of IEEE/RSJ IROS (Intelligent Robots and Systems) international conference since 1997. He is also a member of the advisory committee of the ICARCV International conference on Control, Automation, Robotics and Vision.

C. Laugier is a member of the steering committee of the European Network EURON.

C. laugier is co-chair of the IEEE Technical Committee on ``Intelligent Transportation Systems'' (since 2005).

C. laugier is coordinator of the ICT-Asia Network on ITS named FACT (including partners from France, Singapore, Japan, Korea, and China).

C. laugier is a member of the permanent organization committee of the Field and Service Robotics Conference (since 1997).

C. laugier is a member of the editorial board of the journal ``Intelligent Service Robotics'' (since 2005). He is also a member of the editorial board of the national journal ``Revue d'Intelligence Artificielle'' (since 1987).

Th. Fraichard is a regular member of the programme committees of the ICRA and IROS conferences. In 2006, he was also a member of the following programme committees:

Int. Conf. on Robotics and Biomimetics (ROBIO), Kunming (CN),
*December 2006*.

Workshop on Robotics of the Mexican Encounters in Computer Science (TRENC), San Luis Potosi (MX),
*September 2006*.

Int. Conf. Robotics: Science and Systems (RSS), Philadelphia, PA (US),
*August 2006*.

In addition to ponctual academic lectures, the members of
*e-Motion*have tought the following lectures:

Lecture ``Introduction to robotics and current research issues'' (every year): France-Mexico Summer school on ``Image and Robotics'' (every year).
*Teacher: C. Laugier.*

Lecture ``Robotics and motion autonomy'' (every year): DEA ``Imagerie, Vision, Robotique'' INPG, Grenoble, (FR).
*Teacher: C. Laugier*.

Lecture ``Basic tools and models for Robotics'' (every year): Cnam Grenoble.
*Teachers: C. Laugier and J. Troccaz.*

Advanced Motion Planning, Invited course, Univ. of Zaragoza (ES),
*June 2006*.
*Teacher: Th. Fraichard.*

Motion Planning course, LAFMI Summer School on Image and Robotics: Montpellier (FR),
*July 2006*.
*Teacher: Th. Fraichard.*

Advanced Motion Planning'', Post-Master course, Mathematics Informatics INPG-UJF Doctoral School, Grenoble.
*Teacher: Th. Fraichard.*

Lecture ``Outils probabilistes et statistiques pour l'Informatique'': (every year): Licence d'Informatique de l'Université Joseph Fourier 3 ème année, Grenoble, (FR).
*Teacher: O. Aycard*.

Lecture ``Inférence et apprentissage bayesien": (every year): Master d'Informatique de l'Université Joseph Fourier 1ère année, Grenoble, (FR).
*Teacher: O. Aycard*.

Lecture ``Inférence et apprentissage bayesien": (every year): Ecole Polytechnique Universitaire de Grenoble, Filière Technologies de l'Information pour la Santé 3ème année, Grenoble,
(FR).
*Teacher: O. Aycard*.

Lecture ``Raisonnement bayesien'': (every year): Master 2ème année ``Imagerie, Vision, Robotique'' de l'INPG, Grenoble, (FR).
*Teachers: P. Bessière and O. Aycard*.

Lecture ``Bayesian techniques in vision and perception'': France-Mexico Summer school on ``Image and Robotics'' (every year).
*Teachers: O. Aycard, E. Sucar*.

Lecture on Bayesian Approach to Process Control, Grenoble, France, Compagnie Générale de Chauffage, Janvier 2006.
*Teacher: P. Bessière.*

Lecture on Bayesian Programming, BACS school 2006, Coimbra, Portugal, May 2006.
*Teacher: P. Bessière.*

Lecture on Bayesian Approach to Banking Operational Risk, Paris, France, June 2006.
*Teacher: P. Bessière.*

Lecture on Bayesian Programming, SSIR 2006 (Summer School on Image and Robotics), Montpellier, France, juillet 2006.
*Teacher: P. Bessière.*

Lecture on Bayesian model of perception, PSFMR 2006 (Perception and Sensor Fusion in Mobile Robotics, Fermo, Italy, September 2006.
*Teacher: P. Bessière.*

Some members of
*e-Motion*participates to various international conferences committees and to the organization of summer schools :

C. Laugier and J.M. Ahuactzin participated to the organizing committee of the 2005 summer school on ``Image and Robotics'' at the University of Montpellier, Montpellier (July 2006).

C. Laugier participates every year to the organization committees of the major international conference on Robotics, in particular : IEEE International Conference on Robotics and Automation (ICRA), IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), International Conference on Field and Service Robotics (FSR). He was general chair of IROS'97, Regional programm chair of IROS'00, Program chair of IROS'02. He will be program chair of IEEE/RSJ IROS 2008 to be held in Nice (France).

C. Laugier will be the general Chair of the International conference on Field and Service Robotics 2007, to be held in Chamonix (France) in July 2007.

C. Laugier has organized a workshop on ``Safe navigation in Dynamic environments; application to driving assistance'' in the scope of the IEEE/RSJ IROS'06 conference in Beijing (Oct. 2006).

C. Laugier has organized a workshop on ``Intelligent Transportation Systems'' related to the IEEE Intelligent Vehicle conference 2006 in Tokyo (June 2006); this workshop has been organized in the scope of the ICT-Asia Network FACT.

Th. Fraichard organised the France-Korea workshop on Advanced Driver Assistance Systems (ADAS), Paris,
*December 2005*.

Th. Fraichard gave a number of invited talks:

*Motion Safety in Dynamic Environments*, Safe Navigation in Open and Dynamic Environments - Autonomous Systems versus Driving Assistance Systems workshop, Beijing (CN),
*October 2006*.

*Safe Motion in Dynamic Environments*, Univ. of Zaragoza (ES),
*June 2006*.

*Safety in Dynamic Environments*, Autonomous Sensor-Based Motion for Complex Robots in Complex Environments workshop, Orlando, FL (US),
*May 2006*.

O. Aycard is member of the programme committees of the IEEE IV'2006 conference.

O. Aycard is member of the programme committees of the IEEE ITSC'2006 conference.

A. Spalanzani is part of the editorial commitee of the
*In Cognito*cognitive sciences journal.

Pierre Bessière organized the
*Bayesian Cognition*international workshop on probabilistic models of perception, inference, reasoning, decision, action, learning and neural processing (
http://bayesian-cognition.org). This workshop was organized in collaboration with Alain Berthoz from Collège de France, Peter Dayan from the Gatsby Computational Neuroscience unit of
the University College of London, Roland Siegwart from EPFL and Jean- Jacques Slotine from MIT. It gathered 21 prestigious invited speakers of the field and an audience of 250 persons. It
was held at the French ministry of Science in Paris, from January 16th to 18th 2006.

P. Bessière is a member of the editorial committees of the following conferences :

Conference ESANN (European Symposium on Artificial Neural Networks)

Conference RFIA (Reconnaissance des Formes et Intelligence Artificielle)

Conference IEEE/ICRA (International Conference on Robotics and Automation)

Conference IEEE/IROS (International Conference on Intelligent Robots and Systems)

Conference EA (International Conference on Artificial Evolution)

P. Bessière reviews regularly in the IEEE Transactions on Evolutionary Computation and Autonomous Robots journals.