**Main challenge:**The project-team
*e-Motion*aims at developing models and algorithms allowing us to build
*“artificial systems”*including advanced sensori-motors loops, and exhibiting sufficiently efficient and robust behaviors for being able to operate in
*open and dynamic environments*(i.e. in partially known environments, where time and dynamics play a major role), while leading to
*various types of interaction with humans*. Recent technological progress on embedded computational power, on sensor technologies, and on miniaturised mechatronic systems, make the
required technological breakthroughs potentially possible (including from the scalability point of view).

**Approach and research themes:**In order to try to reach the previous objective, we combine the respective advantages of
*computational geometry*and of the
*theory of probabilities*. We are also working in cooperation with neurophysiologists on sensory-motor systems, for trying to apply and experiment some
*biological models*. This approach leads us to study, under these different points of view, three strongly correlated fundamental research themes:

*Perception and multimodal modelling of space and motion*. The basic idea consists in continuously building (using preliminary knowledge and current perceptive data) several types of
models having complementary functional specialisations (as suggested by neurophysiologists). This leads us to address the following questions : how to model the various aspects of the
real world ? how to consistently combine a priori knowledge and flows of perceptive data ? how to predict the motions and behaviors of the sensed object ?

*Motion planning and autonomous navigation in the physical world*. The main problem is to simultaneously take into account various constraints of the physical world such as
non-collision, environment dynamicity, or reaction time, while mastering the related algorithmic complexity. Our approach for solving this problem consists in addressing two main
questions : how to construct incrementally efficient and reliable space-time representations for both motion planning and navigation ? how to define an iterative motion planning
paradigm taking into account kinematics, dynamics, time constraints, and safety issues ?

*Learning, decision, and probabilistic inference*. The main problem to solve is to be able to correctly reason about prior and learned knowledge, while taking explicitely into account
the related uncertainty. Our approach for addressing this problem is to use and develop our bayesian programming paradigm, while collaborating with neurophysiologists on some particular
topics such as the modeling of human navigation mechanisms or of biological sensory-motors loops. The main questions we are addressing are the followings : how to model sensory-motor
systems and related behaviors ? how to take safe navigation decisions under uncertainty ? What kind of models and computational tools are required for implementing the related bayesian
inference paradigms ?

In spite of the significant technological advances made during the last decade, Robotics is still blocked by the problems of
*scalability*and of the
*real integration of robotized systems in our everyday life*. The reason for this stems primarily from the fact that models and technologies developed in the past (e.g. approaches based on
logics, geometrical based methods, randomized search techniques, reactive architectures ...) have mainly reached their limits, and cannot be directly used for crossing the complexity gap
introduced by the physical environments in which we are living. Indeed, such environments include complex multimodal data, are continuously changing and partially unpredictable, and generate
complex interactions with humans. This means that
*appropriate decision-making processes*taking into account these characteristics have to be designed, implemented and experimented in real situations. Such processes have to be efficient
and robust enough for making it possible to meet the required reactivity characteristics, while being able to make appropriate decisions from complex and incomplete data and knowledge, i.e. by
reasoning about a combination of partial a priori knowledges, of some incremental experimental data (including sensory data), and of some hidden variables. This means that new models and
algorithms have to be designed for beeing able to formalise the intrinsic
*“incompleteness”*of the problem, and to better model the intricate
*“complexity”*of the real world.

The objective of the
*e-Motion*project-team is to find formal and practical solutions to the previous unsolved problems, still very little addressed by our scientific community. Our bet is that our approach
based on both geometry and bayesian programming, will allow us to achieve the following technological breakthroughs:

*Motion autonomy in a dynamic complex world.*We are especially interested in the problems arising from the richness of the environments considered (i.e. how to model them efficiently),
from their dynamicity (i.e. taking explicitly into account the “space-time” dimension), and from the large variety of possible interactions (e.g. estimation and prediction of the behaviors
of the potential obstacles).

*Increased robustness of the navigation processes.*We put the emphasis on the problems of
*incompleteness*(factors not taken into account or hidden variables) inherent to the representation of any physical phenomenon. This dimension of the problem is generally empirically
and approximatively taken into account in traditional approaches, leading the related systems to be poorly reliable. Our approach for dealing with this problem is to convert the
“incompleteness” into numerically quantifiable data, coded in terms of probability distributions and referred to as
*“uncertainties”*. Then, such random variables can be combined, evaluated, and used in various decision-making processes.

*Intuitive programming of artificial systems.*Our approach consists in using as far as possible
*learning*processes (supervised or not), in order to be able to combine the a priori knowledge (called “preliminary data”) and the past experience of the system (called “experimental
data”). This approach, which is consistent with our
*Bayesian Programming*paradigm, should permit us to gradually obtain systems more robust and better adapted to the problems at hand.

As previously mentionned, the main applications of our research are those aiming at introducing advanced and secured robotized systems into the human “living space”. In this context, we are focussing onto the following application domains :

*Future cars and transportation systems.*This application domain should quickly change under the effects of both new technologies and current economical and security requirements of
our modern society. Various technologies are currently studied and developed by research laboratories and industry. Among these technologies, we are interested in
*ADAS**Automatic Driving*functions allowing fully automatic displacements of private or public vehicles in particular driving situations and/or in some equipped areas (e.g. automated car
parks or captive fleets in downtown centres).

*Service and intervention robotics.*This application domain should really explode as soon as robust industrial products, easily usable by non-specialists, and of a reasonable cost will
appear on the market. One can quote in this field of application, home robots (such as for example current vacuum cleaning robots which are both too expensive and poorly efficient), active
surveillance systems (e.g. surveillance mobile robots, civilian or military safety, etc.), entertainment robots, or robotised systems for assisting elderly and/or to disabled people. The
technologies we are developing should obviously be of a major interest for such types of applications.

*Potential spin-offs in some other application domains.*The software technologies we are developing (for bayesian programming) should also have a potential impact on a large spectrum
of application domains, covering fields as varied as the interaction with autonomous agents in a virtual world (e.g.. in the video games), the modelling of some biological sensory-motor
systems for helping neurophysiologists to understand living systems, or applications in economic sectors far away from robotics like those of finance or plant maintenance (applications
currently covered by our start-up
*Probayes*commercializing products based on Bayesian programming).

*ProBT.*

*People involved : Juan-Manuel Ahuactzin, Kamel Mekhnacha, Pierre Bessière, Emmanuel Mazer, Manuel Yguel, Olivier Aycard.*

Formerly known as
*OPL*,
*ProBT*is a C++ library for developing efficient Bayesian software. It is available for Linux, Unix, PC Windows (Visual C++), MacOS9, MacOSX and Irix systems. The ProBT library has two
main components: (i) a friendly Application Program Interface (API) for building Bayesian models, and (ii) a high-performance Bayesian Inference Engine (BIE) allowing to execute all the
probability calculus in exact or approximate way.
*ProBT*is now commercialized by our start-up
*Probayes*; it represents the main Bayesian programming tool of the
*e-Motion*project-team, and it is currently used in a variety of external projects both in the academic and industrial field (e.g. for the European project BACS and for some industrial
applications such as Toyota or Denso future driving assistance systems).

*Cycab Simulator and programming toolbox.*

*People involved : Christophe Braillon, Amaury Nègre, Mehdi Rabah, participation of the SED team.*

In order to perform pre-test and to provide help for Cycab developers, a robot simulator has been developed. This simulator is intended to simulate hardware and low-level drivers, in order to produce a temporal behaviour (refresh frequency, scheduling...) similar to what can be found on real robots with real sensors.

A middleware called Hugr has been developed to allow easy switching between simulated and real platform. Application that uses this middleware do not need to be recompiled when going from the simulator to the real hardware. Moreover Hugr makes it easy to design distributed application. It uses network to share data between applications that are not located on the same machine as easily as if they were on the same one.

Several sensors and robots have been simulated, among them the most original ones are catadioptric and fisheye cameras. Realistic models have been developed for laser sensors, GPS, cameras, ... All these models rely on state-of-the-art GPU techniques. Computing most of the simulated data on the GPU means that the CPU is free for applications. Therefore there is practically no difference between simulation and real sensors or robots.

Applications written and tested on the simulated robot can then be settled to the real one without any modification. Sensors and environment are also simulated, so that complete applications can be developed on this test bed.

The simulator project has been published on the INRIA Forge (gforge.inria.fr/projects/cycabtk).

*Markov models toolbox.*

*People involved : Olivier Aycard.*

This toolbox is a C++ library for prototyping applications for interpretation of temporal sequences of noisy data. It is available for Linux and PC Windows (Visual C++). The Markov models toolbox has two main components: (i) a definition of Markov models and learning of its parameters component. This component permits to manually define the topology of a Markov model, and to automatically learns the parameters of the defined model. Original learning algorithms have also been developped to automatically build the topology of the model and estimate its parameters. The result of this part is a set of Markov models, where each model is trained (ie, estimated) to recognize a particular type of temporal sequence of noisy data. (ii) an interpretation component. Its goal is to interpret a temporal sequence of noisy data and to determine the most probable corresponding Markov models. This Markov models toolbox is patented under the french APP patent #IDDN.FR.001.280011.000.S.P.2004.000.10000 and has been used to perform a preliminay study of recognition of behaviours of a car driver in cooperation with TOYOTA and also to interpret sequence of noisy sensor data of mobile robots

*Grid Occupany Wavelets (GROW).*

*People involved : Manuel Yguel, Francis Colas, David Raulo.*

These software components are C++ libraries for designing applications that build dense representation of the occupancy function of a environment from telemetric sensor measurements either 2D or 3D. It is available for Linux. The GROW library suit has several components:

definition of telemetric sensor models from their intrinsec parameters such as failure rate or sensor uncertainty and the world parameter of the occupancy prior. It is the goal of the telemSensMod library.

definition of change of coordinate systems from sensor own coordinate system to world coordinate system. It is the goal of the dualGrids library.

definition of the sampling functions to sample both of the grids of the coordinate systems. It is the goal of the telemSample library.

definition of Haar wavelet transform and wavelet representation through a sparse tree representation. It is the goal of the WTree library.

definition of the hierarchical rasterization of a telemetric range measurement set, matching function between tree world representation and bundle of telemetric measurements for a certain sensor pose and definition of the intersection queries functions for the special case of a laser range-finder. It is the goal of the scanWT library.

Other libraries are part of GROW as display, sensor driver, concurrent algorithms or debug and trace tools: MY_errorHandler, MY_s11n, colorMapper,glImageTextureMapper,glLaser,laserDriver, laserRangeFinderAPI, occupancyGrid. This Grid Occupany Wavelets software components are declared under the french APP declaration and has been used to scientific experiments described in a scientific article, submitted to ICRA 2007.

*Bayesian Occupation Filter (BOF) Toolbox.*

*People involved: Kamel Mekhnacha, Tay Meng Keat Christopher, C. Laugier, M. Yguel*

The BOF toolbox is a C++ library that implements the Bayesian Occupation Filter. It is often used for modelling dynamic environments. It contains the relevant functions for performing bayesian filtering in grid spaces. The output from the BOF toolbox are the estimated probability distributions of each cell's occupation and velocity. Some basic sensor models such as the laser scanner sensor model or gaussian sensor model for gridded spaces are also included in the BOF toolbox. The sensor models and BOF mechanism in the BOF toolbox provides the necessary tools for modelling dynamic environments in most robotic applications. This toolbox is patented under two patents : “Procédé d'assistance à la conduite d'un véhicule et dispositif associé” n° 0552735 (9 september 2005) and “Procédé d'assistance à la conduite d'un véhicule et dispositif associé amélioré” n° 0552736 (9 september 2005)

Related to close field of research of the e-Motion team-project, these softwares are not used anymore by the researchers of our research team.

*VisteoPhysic.*

*People involved : Cesar Mendoza, Kenneth Sundaraj, Christian Laugier.*

This library provides efficient tools for deformable object simulation. It is patented under the french APP patent #IDDN.FR.001.210025.000.S.P.2004.000.10000.

*ColDetect.*

*People involved : Christian Laugier, Kenneth Sundaraj.*

This library has been implemented for providing robust and efficient collision detection, exact distance computation, and contact localisation of three-dimensional polygonal objects. It is patented under the french APP patent #IDDN.FR.001.280011.000.S.P.2004.000.10000.

Perceiving and understanding the surroundings dynamic environment of an intelligent vehicle or of an autonomous mobile robot is an important step towards safe navigation or driving. In this context, we are focusing on two major complementary points: (1) efficient reconstruction of stationary components of the environments, and (2) detection and robust tracking of the mobile components (cars, pedestrians, bicycles, etc.).

Our approach to solve this problem is to use sensory data fusion mechanisms, to map the related information into 2D or 3D Occupancy Grid models associated when necessary to some velocity distributions, and to simultaneously track the observed entities. This approach allows us to apply an unified framework for processing uncertainty in the reconstruction, the data fusion, the estimation/prediction, and the tracking processes. An evolutive “dynamic map´´ is updated on-line, related to a continuous assessment of the risk of collision. Moving objects are extracted and tracked by building an occupancy grid of the actual perceived environment (ie, a snapshot of the current environment) and a second one corresponding to the static environment and compare these two grids. While the vehicle acquires its map, it has to simultaneously localize itself relative to this map. This problem is known as the Simultaneous Localization And Mapping problem. To achieve the tracking of the detected moving objects, we associate this list of extracted moving objects with the list of objects previously present in the environment using filtering, association and objects management techniques. Finally, to be able to predict the future motions of the vehicle or compute the risk of collision with other objects, future motion of the other objects is predicted.

In 2005 and 2006 we have developed a wavelet-based representation with the objective to solve the dense mapping issue in simultaneous localisation and mapping problem in 3D. We have published three papers on the subject this year , , .

The previous work of the subject had lead to a mapping algorithm. Given a localisation, the robot was able to build a map of its environment in real time and in 3D. To be complete a SLAM algorithm is required to give a localisation. Such an algorithm based on our wavelet based representation was designed and presented in . This algorithm was real-time in 2D but not in 3D. The amount of data even with the current representation was still to huge to be processed in real-time.

The grid-based framework of this approach limits, in fact, the possibilities of map compressions. In practice we have observed that only the empty space is compressed interestingly, which is however the main part of an environment. For example, a plane in 3D cut a lot of voxels at each different scales and then cost a lot of memory. Therefore we have decided to change our approach and to use a point-based representation that is able to compress the structure of the map. The first results of this approach are visible in (fig: ).

*Perspectives:*The next steps of this work are:

to improve the precision of the point based mapping,

to give a theoretical framework for the point based map we have designed and especially to see how it will fit into the hybrid map theory.

to design a localisation algorithm for this kind of map and especially a multi-scale one.

This work is implied in the BACS project.

To navigate in open environments, that is to say where there is a lot of moving obstacles (pedestrians, bicycles, cars, ...), we need to detect obstacles (moving or not) and to estimate their state (position, speed, ...). The current state of the art does not allow to estimate obstacles motions.

In this work, we focus on obstacle detection using fusion of several vision techniques. At first we focus on fusion between stereo-vision and optical flow. Both approches are model-based: we try to extract outliers of a ground plane model. The resulting objects are obtacles (i.e. not located into the ground plane).

The first part of this work has been carried at CSIRO (Brisbane) with K. Usher and C. Pradalier during a 3 months stay of C. Braillon at CSIRO. It consists in extracting the ground plane in a 3D point cloud given by a stereo camera. The model used is a basic cartesian plane equation (4 parameters). After computing the parameters, it becomes possible to extract the outliers by measuring their height with respect to the computed plane. The output of this algorithm is a 2D occupancy grid (projection of the obstacles on the ground plane using probabilistic models of occupancy) In a second step, we have extracted obstacles from optical flow. Since experiments show that detection of obstacles in an optical flow field using a ground model is not accurate enough to be usable in pratical applications, we have proposed to use a reverse model. The reverse model consists in computing the “theoretical optical flow field” from odometry, and in trying to match this field on the camera image. The result of this technique is a kind of disparity image (like in stereo vision). An occupancy grid is then computed from this map by projecting it in a special way.

This projection take into account the incompleteness of the inverse projection from the image to a 3D world. In this work, we proposed a way to project the image back to the 3D world using a priori knowledge on the height of obstacles. The result of this back projection is a 2D occupancy grid.

In a third step, we have decided to make the fusion between the two previous occupancy grids in order to obtain more robust results. This is performed using classical occupancy grid fusion.

Another part of this work consist in evaluating the TTC (time to collision). The computation of the TTC is based on an original and very efficient use of Stokes' theorem:

Knowing that the TTC can be computed using the divergence of optical flow:

Stokes' theorem gives us a good hint on how to compute the TTC of a surface: we only need to compute the integral of the normal flow on the contour of this surface. Using the gradient constraint:

that links the optical flow vector to the partial derivative of the image intensity
I, we can compute the normal flow
and integrate it in Stokes' equation. This gives us a strong relationship between the mean TTC on the surface and the partial derivatives of the image intensity:

This strong theoretical result will lead to very interesting result on efficient implementation of TTC computation.

C. Braillon's phD thesis will be defended the 15th of january.

Part of this work is implied in the LOVe project.

To navigate safely in a dynamic environment (with moving pedestrians, vehicles or other obstacles), it is necessary to detect and characterize dangerous objects. A moving unsafe object can be define as an object which go near to the robot. For these objects, we can characterize the time to collision (TTC) as the time remaining before contact. In we showed it is possible to measure the time-to-collision with an uncalibrated monocular camera by tracking interest points and ridges lines as their intrinsic scale is in direct relation with the time to collision.

If we know the ego-motion of the robot and the TTC of an obstacle we can predict the position of the impact with the obstacle. By observing the motion of the object in the camera image, we measure the angular velocity and we predict the trajectory of the moving obstacle. Our current works consists in using these parameters to apply some reactive commands to the robot in order to avoid detected obstacles.

To increase the computation speed, all vision based algorithms are executed on the graphic processing unit (GPU) : we first compute a Gaussian and Laplacian pyramid, next we detect interest points and we track these points using a particle filters (fig. ) . The GPU permits to obtain video-rate computation.

This work is implied in the LOVe project.

In collaboration with the CSIRO ICT Group (Australia), we designed a novel scale and rotationally invariant visual target and recognition routine based on Self-Similar Landmarks (SSL), these landmarks were first used in a robotic context by Scharstein and Briggs. They designed targets easily detectable by a standard camera at any distance. We modified this target to and some interesting properties such as rotationally invariance, size-less point and motion blur robustness. We also designed a 3-Points Target for pose and range estimation (fig. ).

These algorithm have been applied to an Autonomous Underwater Vehicle (AUV) for vision-based docking with the target and have been proved robust in all conditions . (fig. )

Another application of these landmarks is the camera parameter identification : it consists on estimate the external parameters of a set of cameras using fixes SSL (fig. ).

Perceiving or understanding the environment surrounding of a vehicle is a very important step in driving assistant systems or autonomous vehicles. The task involves both simultaneous localization and mapping (SLAM) and detection and tracking of moving objects (DATMO). It is believed that if we are able to accomplish SLAM and DATMO reliably in real time, we can detect every critical situations to warn the driver in advance and this will certainly improve driving safety and can prevent traffic accidents.

In this context, we design and develop a generic architecture to solve SLAM and DATMO in dynamic outdoor environments. The architecture (Fig. ) is divided into two main parts: the first part where the vehicle environment is mapped and moving objects are detected; and the second part where previously detected moving objects are verified and tracked.

In the first part of the architecture, to model the environment surrounding the vehicle, we use the Occupancy Grid framework. Compared with feature-based approaches, grid maps can represent any environment and are specially suitable for noisy sensors in outdoor environments where features are hard to define and extract. Grid-based approaches also provide an interesting mechanism to integrate different kinds of sensors in the same framework taking the inherent uncertainty of each sensor reading into account.

In general, in order to perform mapping or modeling the environment from a moving vehicle, a precise vehicle localization is essential. To correct vehicle locations from odometry, we introduce a new fast laser-based incremental localization method that can work reliably in dynamic environments. When good vehicle locations are estimated, by integrating laser measurements we are able to build a consistent grid map surrounding of the vehicle. Finally by comparing new measurements with the previously constructed local vehicle map, based on discrepancies between them dynamic objects then can be detected. The more detail published results can be found in .

In the second part, detected moving objects in the vehicle environment are tracked. Since some objects may be occluded or some are false alarms, multi object tracking helps to identify occluded objects, recognize false alarms and reduce mis-detections. In general, the multi object tracking problem is complex: it includes the definition of filtering methods, but also association methods and maintenance of the list of objects. Regarding tracking techniques, Kalman filters or particle filters are generally used. These filters require the definition of a specific dynamic model of tracked objects. However, defining a suitable motion model is a real difficulty. To deal with this problem, Interacting Multiple Models have been successfully applied in several applications. In previous works , we have developed a fast method to adapt on-line IMM according to trajectories of detected objects and so we obtain a suitable and robust tracker. To deal with the association and maintenance problem, we extend our approach to multiple objects tracking using the Multiple Hypothesis Tracker.

The proposed algorithm for solving SLAM and DATMO is tested on data collected from DaimlerChrysler demonstrator car equipped with a laser scanner, two short range radar and a camera. In addition, vehicle odometry information such as velocity and yaw rate are provided. Images from camera are only for visualization purpose. The experimental results on different scenarios such as: urban streets, country roads and highways demonstrate the efficiency of the proposed algorithm (Fig. ). The complete results have been submitted to Information Fusion Journal.

*Perspectives:*Future works include incorporating road models and object models that give a more meaningful representation of detected objects with specific shapes and sizes instead of
only sets of contour points as in our current work. Also, predictions computed in the tracking part can be introduced as motion models in the detection part to increase robustness of the
system.

Moreover, algorithms of road detection and road type classification based on constructed grid map are being considered. The motivation is that road detection can help object detection step to filtering out noisy and irrelevant data off-the-road and focus more on road-likely regions. In all, the fusion of road detection, object recognition and object tracking certainly will enable a better interpretation of driving situations.

This work is implied in the Prevent project.

Previously we focused on dynamic environment modelling. The result was the formulation of the Bayesian Occupancy Filter (BOF) , , . The BOF provides a model for dynamic objects only on a short time scale. We have thus commenced on the modelling of dynamic objects by extending it to encompass a longer temporal scale.

A model of dynamic objects at a longer term (usually several seconds) are especially useful in environments where movements of objects exhibit certain structures or motion patterns. In these cases, it is possible to construct motion exemplaires that characterizes the motion patterns which are then used for performing longer term motion prediction. This is performed by construction models of each motion exemplaires and learning the parameters of the models based on data collected from a certain scene. The ability to perform motion prediction on a longer time scale is beneficial for robotic applications such as target tracking or collision avoidance.

This problem has been addressed in a previous work within e-Motion . The general idea is to construct a network using the growing neural gas algorithm that discretizes space adaptively. A markov transition model can be combined with this adaptive network to represent the probability distribution of movements in a certain scene.

To avoid issues related with discretization, we proposed a novel motion model based on Gaussian Processes (GP). A GP is a generalization of the gaussian distribution to function spaces where a gaussian distribution is placed on functions. Not only discretization issues are avoided in using GP, it provides a theoretically sound bayesian framework in which one can obtain a probability measure on the predictions.

The GP model has been tested against simulated data. The simulated data comprises of movement paths taken by people in the entry hall modelled after the entry hall of INRIA Rhône-Alpes (fig. ).

We model the scene using a mixture of GP where each GP represents a motion pattern. The mixture of GP then corresponds to the different motion patterns in a certain scene. An example is shown in figure where the shaded bars represent the covariances of motion patterns and the mean of the of the motion pattern as a red line.

Another advantage of using GP to represent exemplaire paths is the representation of uncertainty in when performing motion prediction. The uncertainty in motion prediction can be simply calculated by marginalizing the guassian distribution representing the path, given a set of observations. An illustration of the prediction is in figure . The figure shows the various GP components corresponding to the predicted paths starting from a certain point.

We are currently working on online learning and applying it to domains and applications such as to aid path planning in collision avoidance.

This work is implied in a project with TOYOTA.

To navigate or plan motions for a robotic system placed in an environment with moving objects, reasoning about the future behaviour of the moving objects is required. In most cases, this future behaviour is unknown and one has to resort to predictions.

Most prediction techniques found in the literature are limited to short-term prediction only (a few seconds at best) which is not satisfactory especially from a motion planning point of view.

In 2003, we have started to explore the problem of medium-term motion prediction for moving objects. As a result, we have proposed a novel cluster-based technique that learns typical motion patterns using pairwise clustering and use those patterns to predict future motion.

During 2004 and 2005, we developed a number of libraries and utilities (environment modelling, sensor modelling, camera calibration, communications) that will be used for further experiments. From 2005 to 2006, we have developed a new learn and predict approach which addresses issues that were not solved by our first proposal: (a) Prediction of unobserved patterns and (b) On-line/adaptive learning.

The new approach represents motion on the basis of a proposed extension to the well-known Hidden Markov Models framework, that we have named Growing Hidden Markov Models (GHMM). Basically, the extension allows incremental, adaptive, real time learning of the models parameters and structure. Incorporating final positions, objects' goals, in the GHMM state several motion patterns can be represented with a single GHMM.

From 2006 to 2007 new issues became target:

*Model optimization:*Is it possible to improve the model-data fitness and speed up the model convergence?

*Integration of high level features:*How higher level features, as objects' and environments' particularities (
*e.g.*objects' nature, doors' positions, occupied areas, etc) can be taken in account to improve the prediction task?

Concerning the
*model optimization*, we are investigating the model learning techniques and the training data feature set. As a short term result (until the end of 2007), we expect to run tests with
new learning algorithms comparing it against the actual one, focusing on complexity, model-data fitness and quality of prediction aspects.

With respect to the
*integration of new features*, we are investigating methods to merge the
*general motion model*(represented by a GHMM) to
*local models*(
*e.g.*object interaction in collision avoidance scenarios). The key idea is to keep the
*general motion model*as sober and generic as possible, based on a low dimensional feature set (
*e.g.*objects' position, velocities and goals) and treat categorized features and sub-features issuing from categorized ones in independent specialized
*local models*. First results on merging general and local models is expected for the middle of 2008.

This work is implied in the BACS project.

Image segmentation is an important and challenging problem in vision. Its goal is to identify homogeneous regions in images as distinct from the background and belonging to different
objects. A common approach is to classify pixels on the basis of local features (
*e.g.*color, position, texture), and then grouping them together according to their class in order to identify different objects in the scene.

For the specific problem of finding moving objects from static cameras, the traditional segmentation approach is to separate pixels into two classes: background and foreground. This is
called
*Background Substraction*
and constitutes an active research domain (see
).

The output of most background segmentation techniques consists of a bitmap image, where values of 0 and 1 correspond to background and foreground, respectively (
*e.g.*
,
,
). Having such a bitmap, the next processing step consists of merging foreground pixels to form bigger groups
corresponding to candidate objects, this process is known as
*object extraction*.

In previous work, we have proposed a novel clustering approach for object extraction based on Self Organizing Networks (SON). We have applied this algorithm to images and occupancy grids , and shown that it is able to produce good results in real time.

However, this approach suffered of two drawbacks: a) first, it needed a binary input, which implied the necessity of applying a threshold to the input grid, thus losing information; and b)
its complexity depended from the number of input cells and the size of the SON, thus, when the number of input cells (
*i.e.*foreground pixels) was high, the performance of the algorithm degraded.

Recent improvements in the learning stage of the algorithm addressed the drawbacks mentioned above, and they permit fast processing of continuous input values (
*e.g.*grayscale images).

The changes in the algorithm represents a complexity change from
O(
N_{f}M)to
O(
N), where
N_{f}is the number of foreground pixels,
Nis the total number of pixels in the image and
M, the size of the SON.

About the computational costs, the object extraction from a
384×288grayscale image, using a
20×20SON consumes about
30
m
s
e
c(mean obtained from 1044 frames processed), running on a machine equipped with an Intel Pentium IV processor. To observe the algorithm complexity property, we
repeated the procedure above using a
100×100SON and the increase in the processing time kept around
1
m
s
e
c.

This work is implied in the BACS project.

This activity has been carried out in collaboration with the ETHZ in Zurich in the frame work of the European project BACS. The general problem here considered is the autonomous navigation of a mobile platform with special focus on the localization task and the mapping task.

A new strategy was introduced in 2006 by our lab in the framework of the BACS project. However, two issues remained open and have been successfully addressed during this year. The former regards the applicability of the suggested strategy. Indeed, the case when the robot accomplishes one or more loop was not considered before. New solutions to deal with this case have been introduced , , . The latter regards the improvement of the estimation of the robot pose by adopting other Bayesian methods which outperform the EKF in non-linear systems. Also in this case, results have been obtained .

Furthermore, a great effort has been spent to improve the feature extraction task , . Finally, the case of several communicating robots has been considered for the first time and preliminary and promising results have been obtained .

Two methods able to exploit the information at the loop closure in the
SLAMproblem have been introduced. Both methods have three fundamental advantages. The first one is that to apply the loop closure constraint they do not require to compute any
correlation among the features which are not observed simultaneously. The second advantage is that the loop closure constraint can be applied only once at the end (even after more than one
loop) in a single step with low computational complexity. Hence, the computational complexity during the robot exploration is independent of the number of features. Finally, the third
advantage is that the linearization does not affect the estimation process. This especially holds for the second method, which is based on the Levenberg-Marquardt algorithm. The first
method is based on the Extended Kalman Filter. Simulations show that these approaches significantly outperform the conventional
EKFbased
SLAMboth in the computational cost and in the map precision (see fig
).

A new analytical algorithm to perform the localization of a mobile robot has been introduced. The algorithm is based on a polynomial extended Kalman Filter and avoids the problems coming from the system non-linearities. Very accurate simulations showed that this approach significantly outperforms other localization approaches, especially in critical situations where the system non-linearities often lead to divergence .

A new and robust method for extracting and matching visual vertical features between images taken by an omnidirectional camera has been introduced. Matching robustness is achieved by creating descriptor, which is unique and distinctive for each feature. Furthermore, the proposed descriptor is invariant to rotation. The robustness of the approach was validated through real experiments with a wheeled robot equipped with an omnidirectional camera. We showed that vertical lines are very well extracted and tracked during the robot motion .

A new approach to the problem of simultaneously localizing a team of mobile robots equipped with proprioceptive sensors able to monitor their motion and with exteroceptive sensors able of sensing one another has been introduced. The method is based on a series of extended Kalman filters hierarchically distributed. In particular, the team is decomposed in several groups and, for each group, an extended Kalman filter estimates the configurations of all the members of the group in a local frame attached to one robot, the group leader. Finally, at the highest level of the hierarchy, one single filter estimates the locations of all the group leaders. The key advantage of this approach is its ability to distribute the computation necessary to perform the multi robot localization under limited computation and communication capabilities. In particular, the approach significantly outperforms an optimal approach based on a single estimator. This was shown by analytically computing the precision on the localization of each robot in the case of one single degree of freedom. In particular, the best hierarchy was analytically determined by deriving the dependency of the localization precision on the communication and computation capabilities and on the sensors accuracy. This work is implied in the BACS project.

In order to perform pre-test and to provide help for Cycab developers, a Cycab simulator has been developed. The Cycab Simulator is currently widely used by the researchers of the
*e-Motion*project-team; it has also been used in a collaboration with the RIA team (LAAS, Toulouse), and with the LAG laboratory (IMAG, Grenoble). A recent extension of the system is
also used for student home-work in the scope of robotics courses (Summer school on image & robotics 2004-2006, and Master IVR at ENSIMAG Grenoble 2006-07). The simulator project (called
CycabTK) has been published on the INRIA Forge (gforge.inria.fr). This publication aims at federating Cycab users by providing a generic toolkit. The Cycab simulator will be tested in several
INRIA Research unit (IRISA, LORIA, Rocquencourt) and in LASMEA laboratory (Université Blaise Pascal, Clermont Ferrand).

The simulator provides several simulated sensors: B&W cameras, color cameras, stereo cameras, Sick range finder, rotating Sick. It will soon include a GPS simulator.

This simulator is implied in the BACS project.

Dynamic environments,
*ie*environments with moving objects, impose a strict upper-bound on the time available to a robotic system in order to determine its future course of action. This constraint is
henceforth called the
**decision time constraint**. The decision time available is a function of what is called the
*dynamicity*of the environment which is directly related to the dynamics of both the moving objects and the robotic system. Failure to meet the decision time constraint may put a robotic
system in a situation where a collision eventually occurs.

Lately, a number of motion planning-based navigation schemes dealing with dynamic environments have been proposed,
*eg*
,
,
,
. Unlike earlier reactive navigation schemes (that seeks to determine the motion to execute during the next
time-step only), they aim at computing a complete motion all the way to the goal. To do so, they rely upon the most efficient motion planning techniques available today, namely randomised
techniques such as the Probabilistic Roadmap
or the Rapidly Exploring Random Tree
, whose average running times are low enough so that they can be used on-line. This type of approach is very
enticing since it makes up for the lack of lookahead of reactive navigation schemes. However, it is argued that the decision time constraint can be such that it prevents the use of such
schemes since the running time of a randomised technique cannot be upper bounded.

Given the intrinsic complexity of motion planning in dynamic environments (see the complexity results established in
and
), it seems unlikely that a hard decision time constraint could ever be met in realistic situations. The
*Partial Motion Planning*principle
is advocated instead as a general way to handle the decision time constraint and to address autonomous
navigation in dynamic environments. Partial Motion Planning (PMP) is an interruptible planning scheme: when the time available is over, it returns either a complete motion to the goal or a
partial motion only,
*ie*a motion that may not necessarily reach the goal. This partial motion is then passed along to the navigation system of the robot for execution. Of course, since only a partial motion
is computed, it is necessary to iterate the partial motion planning process until the goal is reached.

The iterative nature of PMP is doubly required when the robotic system at hand is placed in a uncertain dynamic environment,
*ie*an environment for which everything is not known in advance (in particular, the future behaviour of the moving objects). Indeed, motion planning means reasoning about the future.
When the future is unknown, one has to resort to predictions, predictions whose validity duration is limited in most cases. An iterative planning scheme permits to take into account the
unexpected changes of the environment by updating the predictions at a given frequency (which is also determined by the environment dynamicity).

Like reactive approaches, PMP faces two issues, namely the convergence and the safety issues:

Convergence. What guarantee do we have that the goal will ever be reached?

Safety. What guarantee do we have that the robotic system will never found itself in a dangerous situations?

As for the convergence issue, the unrealistic convergence conditions established in
leave little hope (it is hardly surprising if the system is placed in an environment with no a priori
information about the moving obstacles). To address the safety issue, a solution relying upon the concept of
*Inevitable Collision States*is explored (
*cf*sections
). To address the safety issue, two different solutions are explored: the first one relying upon the concept of
*Inevitable Collision States*and the second one upon the concept of
*Velocity Obstacles*(
*cf*sections
and
respectively).

In 2007, the integration of the PMP module within the software architecture of a Cycab experimental vehicle has been furthered. Coupling with the control, perception and environment modelling modules have been achieved. Simulated and live experiments have been carried out (Fig. ). This work has yielded a number of papers detailing the overall architecture , , .

This work is implied in the Cybercars and Mobivip projects.

In 2002, we introduced the novel concept of
*inevitable collision states*(ICS). An ICS for a given robotic system is a state for which, no matter what the future trajectory followed by the system is, a collision with an object
eventually occurs. For obvious safety reasons, a robotic system should never ever end up in an ICS.

We have already used the ICS concept in a number of applications,
*eg*
,
. In all cases, the robotic system is guaranteed never to enter an ICS.

Although the ICS concept offers a theoretical answer to the motion safety issue, using it in practise raises a major problem: that of the characterisation of the set of ICS for a given
robotic system. In theory, determining whether a given state is an ICS or not requires to check for collision
*all possible future trajectories of infinite duration*that the robotic system can follow from this particular state!

In practise, it is fortunately possible to use the approximation property established in . This property says that a conservative approximation of the ICS set of a given robotic system can be obtained by considering only a finite subset of the whole set of possible future trajectories.

Early 2007, we have presented at the 2007 Int. Conf. on Robotics and Automation (ICRA) two papers respectively emphasizing the importance of the ICS concept in the design of safe
navigation schemes
, and detailing an ICS checking algorithm,
*ie*an algorithm that determines whether a given state is an ICS or not, for a car-like vehicle moving in a dynamic environments
. This algorithm exploits the approximation property and introduces the concept of
*imitating manoeuvres*,
*ie*trajectories leading the robotic system to duplicate the behaviour of the environment objects (fixed or moving), it is shown how a good approximation of the ICS set can be obtained
by considering imitating manoeuvres to build the conservative approximation of the ICS set.

Later on, we have made further progress and have proposed a
*generic*and
*efficient*ICS-Checker for planar robotic systems with arbitrary dynamics moving in dynamic environments. The efficiency is obtained by applying the following principles: (a) reasoning
on 2D slices of the state space of the robotic system, (b) precomputing off-line as many things as possible, and (c) exploiting graphic hardware processing performances. The efficiency of the
resulting algorithm has been demonstrated in the case of a car-like system with complex dynamics.

A paper presenting these latest results has been submitted to the 2008 Int. Conf. on Robotics and Automation.

Designing an autonomous robotic system requires to solve a number of challenging problems in domains as different as perception, localisation, environment modelling, reasoning and
decision-making, control, etc. However, whatever the robotic system and whatever the kind of tasks it is expected to carry out, at some point, it has to move. Motion determination is
therefore a fundamental issue that has been widely addressed by the Robotics community and a large number of autonomous navigation architectures have been proposed (
*cf*the recent review established in
).

From the motion determination perspective, these navigation architectures can be broadly classified into
*deliberative*(
*aka*
*motion planning-based*) versus
*reactive*approaches: deliberative approaches aim at computing a complete motion all the way to the goal using motion planning techniques, whereas reactive approaches determine the
motion to execute during the next time-step only.

Deliberative approaches have to solve a motion planning problem
: they require a model of the environment as complete as possible and their intrinsic complexity is such that
it may preclude their application in dynamic environments (
*cf* §
). Reactive approaches on the other hand can operate on-line using local sensor information: they can be used in any kind of
environment whether unknown, changing or dynamic. This accounts for the large number of reactive approaches that have been developed over the years. Reactive approaches however face a
challenge: the convergence towards the goal is not guaranteed (local minima problem).

In an attempt to bridge the gap between deliberative and reactive approaches, a complementary approach has been proposed based upon
*motion deformation*. The principle is simple: a complete motion to the goal is computed first using a priori information. It is then passed on to the robotic system for execution.
During the course of the execution, the still-to-be-executed part of the motion is continuously deformed in response to sensor information acquired on-line, thus accounting for the
incompleteness and inaccuracies of the a priori world model. Deformation usually results from the application of constraints both external (imposed by the obstacles) and internal (to
maintain motion feasibility and connectivity). Provided that the motion connectivity can be maintained, convergence towards the goal is achieved.

It appears that the different motion deformation techniques that have been proposed
,
,
,
all performs
*path deformation*. In other words, what is deformed is a geometric curve,
*ie*the sequence of positions that the robotic system is to take in order to reach its goal. The problem with path deformation techniques is that, by design, they cannot take into
account the time dimension of a dynamic environment. To achieve this, it is necessary to depart from the path deformation paradigm and resort to
*trajectory deformation*instead. Unlike path deformation wherein spatial deformation only takes place, trajectory deformation features both
*spatial and temporal*deformation meaning that the planned velocity of the robotic system can also be altered.

In 2007, we have finalized our first trajectory deformation scheme. It operates a two steps: a collision avoidance step and a connectivity maintenance step. In the collision avoidance
step, repulsive forces generated by the obstacles deform the trajectory so that it remains collision-free. The purpose of the connectivity maintenance step is to ensure that the deformed
trajectory remains feasible,
*ie*that it satisfies the robotic system's kinematic and/or dynamic constraints. This work has been presented at the 2007 Int. Conf. on Intelligent Robots and Systems (IROS)
.

In parallel, in the scope of the Master's thesis of Vivien Delsart, we have developed an alternate version of the trajectory deformation scheme that operates in a single step only . This new algorithm is more efficient and is currently applied to a broader class of robotic systems (holonomic and nonholonomic ones). Preliminary results are depicted in Fig. .

The problem of autonomous navigation has been deeply studied in literature and several techniques have been developed. In a deterministic environment, static or dynamic, global path planning techniques are suitable: a sequence of control inputs from the initial state to the goal state can be computed and safely executed. In other real world application, for example navigation among moving people or other vehicles, we need to put in relation the decision about motion and the on-line perception of the world: the uncertainty on the representation of the static and dynamic world can't be ignored and is not possible to compute a full plan in advance.

Last year we focused our attention on a reactive method based on the velocity obstacle approach . We developed a novel technique to integrate the probabilistic velocity obstacles with a dynamic occupancy grid: the occupancy grid estimates the occupation of the space and a probability distribution over velocities for each object in the environment so that the velocity obstacles can be computed. By providing a threshold on the probability of collision, the time to collision was estimated and a safe control was chosen. Our results were published at ICRA 2007 .

This year we worked for the purpose of integrate this algorithm in a more complex navigation structure, to achieve better and safer performance in highly dynamic environments. The developed approach combines a medium term prediction of dynamic obstacles based on pre-learned models and partial motion planning .

The moving obstacles considered are supposed to be intelligent agents moving with the intention to reach a specific goal. With this hypothesis it is possible to observe the environment in order to learn the typical behavior of the obstacles and predict their future motion with a certain confidence. We consider that the system has already learnt a set of typical patterns and that they are described by HMM graphs where the nodes represent the position of the obstacles and the edges represent the probability to pass from one node to another in a time step. Each graph represents all the typical patterns in the considered environment leading to one particular goal. With respect to classical tracking algorithms, this technique allows to obtain a non-linear, more reliable prediction, that takes into account the intentions of the obstacle and the structure of the environment.

The robot observes the environment with its on board sensors and estimates the position of the moving obstacles, in our case, simulated pedestrians; the goal of each pedestrian is estimated using subsequent observations and a probabilistic prediction of his future position is obtained letting the HMM graphs evolve from the observed state. For each time step an occupation grid is built so that a safe path is searched on a state-time grid. A rapidly exploring random tree (RRT , ) algorithm computes a safe partial path. During the execution, the planner continues to search for a path to the goal while the reactive algorithm checks for unexpected events: if necessary it suggests a safer control and reinitializes the path planner. The algorithm has been tested on the Cycabtk simulator and real and simulated trajectories acquired in collaboration with Dizan Vasquez.

This work is supported by a grant from the European Community under the Marie-Curie project VISITOR and implied in the LOVe project.

During last two decades, machine learning methods gained much visibility in the artificial intelligence community, with new algorithms and new results. This can be explained by three advances:

the intensive use of the bayesian framework,

the convenience of graphical models (such as bayesian networks and factor graphs),

and the discovery of new and efficient approximate algorithms for bayesian inference.

The bayesian framework, applied to learning tasks, is basically the use of Bayes' rule to infer the posterior probability of an hypothesis, after the observation of some data. It's based on the mathematical notion of probability, interpreted as a subjective degree of belief of an agent living in an uncertain environment . This philosophical foundations present the advantages to be clean and rigourous, and allow us to derive an automatic method for doing inductive inference. This is also well adapted to robotics, a robot beeing an autonomous reasoning agent.

In order to apply the bayesian learning method to robotics, we use the “Bayesian Robot Programing” framework . It consists in representing the sensory-motor space of a robot with a probability distribution. This distribution is then used to take motor decision in order to accomplish a predefined task.

By studying the nature of different learning problems, we proposed to classify them regarding their “difficulty”. The difficulty of a learning problem can have several sources. Firstly, it can come from the lack of prior knowledge concerning the problem itself, and the way to model it. For instance, it's an issue to define what are the relevant variables, what are their conditional independences, and what are their a-priori parametric distributions. The other difficulty of learning comes from the bayesian inference itself; which can be intractable in several ways. The problem can be non-analytic, can present a huge (super exponential) configuration space, and can presents some loops in its dependence structure. In these cases, some clever approximations are required.

In a thesis which will be defended during December 2007, we present the application of the bayesian learning method, to different problems, ranked by their intrinsic difficulty.

The thesis is divided into three parts. Firstly, we study the mathematical, philosophical ans semantic foundations of the bayesian framework. Then we apply this method to two kind of learning problems. The first and easier one concerns the problem of parameter learning, when relevant variables and probabilistic structure are known. We propose two algorithms for two different applications.

The first one consist in ranking objects from paired comparisons across time, and is used to derive historical ratings of Chess players. The resulting algorithm is a deterministic approximation based on the Expectation Propagation framework; the true posterior distribution is approximated by a fully-factorized product of gaussians. We applied it to the largest top-class annoted chess database in the world, ChessBase which covers more than 3.5 million games from 1560 to 2006. Our final probabilistic model contains almost 20 million variables organized in a big and loopy factor graph. Thanks to the fully factorized gaussian approximation, and due to a clever semi-dynamic scheduling algorithm, our message passing method converges in less than 20 minutes on a desktop computer. Hyper-parameters are point-estimated thanks to the empirical bayes process, also called evidence maximisation or type II maximum likelihood. The resulting ranks make sense to chess historians. Our method has been proven better than Pr. Elo's method, both in terms of relevance of the generative model, and in terms of predictive accuracy. This work will be presented during the forthcoming NIPS (Dec. 2007) conference .

The second application of parameter learning is related to the Netflix challenge. This is a collaborative filtering problem. Netflix, a US based DVD rental company, has collected data regarding user preferences about movies. The goal is to predict if a user will like a movie, based only on his past tastes and on the past preferences of other users. For this problem we proposed a fully generative bayesian model. Our model clusters users into user groups, and movies into movie types. Inference is performed with Expectation Maximization, with a variational approximation during the E-step. Because the training set contains 100 million ratings from over 480 thousand users on nearly 18 thousand movie titles, a hierarchical optimization of this clustering method has been introduced to speed up convergence. Results are interesting, the found clusters make sense, and predictive accuracy is better than the one obtained by memory based methods, but lower than the one reported by Netflix challenge leaders.

The last part of the thesis tackles more difficult learning tasks: the selection and creation of relevant variables.

A comparison of different feature selection algorithms for a robotic experiment has been performed two years ago and has been published in ICRA 2005 . This year we finalized an incremental algorithm for creating a discrete variable. We chose to focus on a sub class of problems, restricting the searched space to the set of dynamic bayesian networks. In this subclass of models, our algorithm is able to discover that the introduction of a discrete hidden variable with an unknown cardinality simplifies the search process. We are applying this algorithm to spatial clustering in a simple vision task.

Navigation a mobile vehicle is a great challenge, and controlling the robot with simple algorithms often leads to mistakes, as the environment can't be expressed in its whole. So, a better method would be to learn skills from human expertise, as humans usually know how to drive vehicles but are not able to explain how they do.

This method has been applied for car driving on highways, using fuzzy neural networks, which was the project of my master's thesis, carried out at
C^{2}i(Centre for Computational Intelligence), a laboratory located at NTU, Singapore, under the direction of Michel Pasquier
.

It can be extended to controlling a wheelchair for a high disabled person. the challenge is to provide the person a wheelchair able to understand his intentions. The robot, with its representation of the world need to communicate with the person who has his own representation of the same world. More over the communication can be limited to a poor interface.

To communicate with the wheelchair, Brice Rebsamen has developped a Brain Control Wheelchair, especially for indoor environment, when the topology of the environment is known. This method could be applied to our project, to help communication between the disabled person and the wheelchair.

In order to learn to navigate, the robot will first have to learn basic behaviors using a supervised learning method, and then to learn the temporal sequence of these simple behaviors, to have an emergence of a global behavior. The learning of the sequence of behaviors will be done by adpating Growing Hidden Markov Model, a method developped by Dizan Vasquez during his Phd in eMotion team .

During this year, the differents behaviors to be learnt have been identified, and clustered in three groups: reflex (collision, wandering...), reactive (obstacle avoidance, wall following...) and cognitive (human tracking, path following ...) behaviors. Some of these behaviors have been implemented using Fuzzy Neural Networks.

This work is implied in the BACS project.

This work has been done at the NTU of Singapore and in cooperation with the NUS Singapore. It is also implied in the BACS project. A brain-computer interface (BCI) is a system that allows direct control of a computer by thought. BCIs have the potential to help people with various disabilities, e.g. individuals suffering from amyotrophic lateral scleroses (ALS), severe cerebral palsy, head trauma, multiple sclerosis, and muscular dystrophies, to communicate or perform ordinary tasks. BCI applications range from typing words, moving a cursor over the screen, gaming, or controlling a TV.

Robotic applications are a different story: controlling a robotic device, such as a wheelchair or a prosthesis, requires continuous and accurate commands. However, due to the poor and noisy signal, information from BCIs can typically be extracted only at a very slow pace (up to several seconds), or with a very high uncertainty. A solution to this conflicting situation is to endow the system with enough autonomy to avoid dangerous situations.

Our solution relies on motion guidance and destination selection: virtual guiding paths connect locations of interest in the environment (see figure ). These paths can be traced automatically if an accurate map of the environment is available, or manually by simply pushing the wheelchair once along the desired trajectory. Once a network of guiding paths is available, navigating in the environment simply consists in selecting the desired destination.

For destination selection we use a BCI based on the P300 signal. The user focuses his or her attention on an item within a list of 20 or more while the items are presented in random order (see figure ). A peak of potential appears in the EEG about 300ms after the item of interest was presented (see figure ). Upon detection of the P300 signal, the target is traced back as the item that was presented 300ms earlier. For greater accuracy and reliability the process can be repeated several times. Typically, this kind of interface can achieve a response time in between 10 to 20 seconds with an accuracy close to 100%.

This strategy has the important benefit of requiring minimal input from the subject, therefore minimizing concentration effort, thus fatigue. Moreover, since the trajectories are repeated over time, the predictability of the motion adds confidence in the machine. Lastly, because no complex sensor is required to perceive and interpret the environment, the price of the robotic system is very low, thus allowing a larger number of disabled to afford it.

This work has been done in collaboration with LPNC of Grenoble (J. Diard). Is is implied also in the BACS project. During the first year of PhD of Estelle Gilet, we studied the state- of-the-art of the modeling of sensorimotor systems, focusing more precisely on handwriting movements.

We are interested in how the central nervous system represents the sensorimotor plans associated with writing movements. Given the structure of the internal space, we are interested in studying the perception, the learning and the production of letters. In the motor theories of perception, the perception and the production share the same set of invariants and they must be linked.

We focused on the state-of-the-art of the perception of hand trajectory and we examined studies of the kinematic and dynamic aspects of human arm movements. These studies identify common kinematic features or patterns of muscle activation like the two- thirds power law, the isochrony motor principle, etc. For example, the two-thirds power law describes the relationship between the tangential hand speed and the curvature of its trajectory during curved motion.

The production of movement is commonly modelled mathematically by defining an objective function, which is a measure of performance for any possible movements. The performance is measured as the time integral of some cost like jerk (rate of change of acceleration), energy, torque change, variance, etc. This class of works assume that, out of all possible solutions for producing the desired trajectory, the central nervous systems selects the one minimizing the chosen measure of performance: this covers the minimum-variance, minimum-jerk, minimum-torque models.

This literature review allowed us to define our next step of research. In Estelle Gilet's PhD project, we aim to implement a simulation of handwriting learning. This model will be based on three points: the perception, the learning and the production of letters.

This work is done under the joint supervision of Pierre Bessiere and Julien Diard of the LPNC laboratory (Laboratoire de Psychologie et NeuroCognition, CNRS, Grenoble).

This work has been done in collaboration with the Start-up Probayes. ProBT is a C++ library for developing efficient Bayesian software. This library has two main components: (i) a friendly Application Program Interface (API) for building Bayesian models and (ii) a high-performance Bayesian inference and learning engine allowing execution of the probability calculus in exact or approximate ways. This paper is not intended to describe ProBT. It will only focus on its underlying inference algorithms.

The aim of ProBT is to provide a programming tool that facilitates the creation of Bayesian models and their reusability. Its main idea is to use “probability expressions” as basic bricks to build more complex probabilistic models. The numerical evaluation of these expressions is accomplished just-in-time: computation is done when numerical representations of the corresponding target distributions are required. This property allows designing advanced features such as submodel reuse and distributed inference. Therefore, constructing symbolic representations of expressions is a central issue in ProBT.

During this year the main development of ProBT has been carried out by Probayes: a spin-off born from the e-motion project. Both, e-motion and Probayes, are part of the European project Bayesian Approach to Cognitive Systems (BACS). The development of ProBT has been effectuated taking into account the goals of the BACS project partners. In 2006 the main subjects of development and research has been:

The ProBT objects serialization: saving or transmitting ProBT objects, for example, onto a file, or a memory buffer. The serialization allows the reconstruction of
ProBT object that has been constructed in the past by the execution of another program. The serialization is very useful in divers applications such as the parallel computation where
nprocessors shares the same object to execute different computations.

ProBT interface with applications, libraries and languages such as (Excel, PHP, Python, and Graphical tools).

ProBT Graphical Interface.

Other subjects of development and research were launched this year such as:

Specialized probability calculus algorithms.

Learning (e.g. Maximum likelihood, Maximum Entropy, Expectation-Maximization, Structures Learning)

Probabilistic calculus based on a Graphical Process Unit (GUP) architecture.

Cybercars 2 [January 2006-December 2008]

European project IST Cybercars 2, “Close Communications for Cooperation between Cybercars”.

(
http://

This Project is driven by the vision that, in the short term future, Cybernetic Transport Systems (CTS) based on fully automated urban vehicles (the cybercars) will be seen on city roads
and on new dedicated infrastructures. Such systems have been developed and evaluated in the scope of the CyberCars (www.cybercars.org) and CyberMove (www.cybermove.org) projects of the 5th
FWP and are now being deployed. However, presently these CTS can only operate in low demand environments where little interaction between vehicles is anticipated. In order for these systems
to address high demands, more cooperation between vehicles is needed. This is the topic of this Project, based on vehicle-vehicle and vehicle-infrastructure communications and vehicles
coordination. We will address in particular the cooperation between vehicles running at close range (platooning) and at intersections (merging, crossing). The contribution of
*e-Motion*to Cybercars 2 focuses on cooperative driving.

Profusion [February 2004-February 2008]

European project, PreVENT Programme (Preventive and Active Safety Applications) Profusion, “Project for Robust and Optimized Perception by Sensor Data Fusion”

The second phase of ProFusion started in April 2005. The goal of this second phase is to develop new concepts, methods and theories for sensor data fusion for Automotive Industry. These solutions will permit to perceive the environment surrounding a car and automatically build a model of this environment. This model will be interpreted it in order to assist the car driver. The resulting prototypes will be tested and validated on European car demonstrators. The specification and design of the system for sensor data fusion is still in development until the beginning of 2006. In 2006, the development of algorithms and methods will start, and the integration of the resulting prototypes on european car manufacturers demonstrators is planned for 2007. Our group plans to integrate its prototype on a Volvo truck and a Mercedes-DaimlerChrysler car.

The Profusion phase2 consortium is constitued of 10 european car manufacturers, suppliers and research institutes: (BMW, CRF, DaimlerChrysler, Delphi, Forwiss, ICCS, INRIA, Sagem, TUC, Volvo).

Mobivip [November 2003-June 2007]

National project Predit Mobivip “Véhicules Individuels Publics pour la Mobilité en centre ville”.

(
http://

Mobivip is a research project involving 5 research laboratories and 6 industrial partners in order to implement, evaluate and demonstrate the impact of Information Technologies on a
novel mobility service for downtown areas, such mobility service being based upon a fleet of small public urban vehicles integrated in the existing public transport system. The contribution
of
*e-Motion*to Mobivip focuses on autonomous driving in open and dynamic environments.

LOVe [December 2005-December 2008]

National project, Predit Programme LOVe “Logiciel d'Observation des Vulnérables”.

(
http://

The goal of this project is to develop new methods and prototypes for improving vulnerable security and car driver's confort. Artificial perception algorithms will be developped for the localization of the road and for the detection, localization, recognition and tracking of mobile objects using different kind of sensors (lidar, radar, cameras). This project is constituted of 14 french partners (Renault, Valéo, INRETS/LCPC/LIVIC, INRIA (emotion, icare, imara), CEA List, Université Paris Sud, CNRS/heudiasyc, LASMEA, Armines (CNN, CAOR)).

Toyota Motors Europe [Feb 2006 - Jan 2009]

The contract with Toyota Motors Europe began its first phase in the beginning of Feb 2006. It is a joint collaboration involving Toyota Motors Europe, INRIA and ProBayes.

This contract aims to develop innovative technologies in the context of automotive safety. The idea is to improve road safety in driving situations by equipping vehicles with the technology to model the dynamic environment and sense incoming danger. This is with the aid of sensors commonly used in automotive applications such as cameras and lidar.

Denso [Sept 2007 - Sept 2008]

Industrial contract “Application of Bayesian Occupancy Filter technology to DENSO LIDAR data” involving INRIA, Denso and Probayes.

This contract is the follow up of a previous contract signed in 2006 by Denso and INRIA for evaluating the applicability of the “Bayesian Occupancy Filtering” (BOF)

*BACS (Bayesian Approach to Cognitive Systems - FP6-IST-027140) [january 2006-february 2011]*

As the BIBA project gave very interesting results, the European Commission decided to finance a new and extended project on the same thematic.

The BACS project started at the very beginning of 2006 for five years. It is an IP project including a number of new partners and especially some industrial partners to develop applications of the bayesian technology.

Despite very extensive research efforts contemporary robots and other cognitive artifacts are not yet ready to autonomously operate in complex real world environments. One of the major reasons for this failure in creating cognitive situated systems is the difficulty in the handling of incomplete knowledge and uncertainty.

In this project we will investigate and apply Bayesian models and approaches in order to develop artificial cognitive systems that can carry out complex tasks in real world environments. We will take inspiration from the brains of mammals including humans and apply our findings to the developments of cognitive systems.

The Bayesian approach will be used to model different levels of brain function, from neural functions up to complex behaviors. This will enable us to show that neural functions and higher-level cognitive activities can coherently be modeled within the Bayesian framework. The Bayesian models will be validated and adapted as necessary according to neuro-physiological data from rats and humans and through psychophysical experiments on human.

The Bayesian approach will also be used to develop four artificial cognitive systems concerned with (i) autonomous navigation, (ii) multi-modal perception and reconstruction of the environment, (iii) Semantic facial motion tracking, and (iv) human body motion recognition and behavior analysis.

The conducted research shall result in a consistent Bayesian framework offering enhanced tools for probabilistic reasoning in complex real world situations. The performance will be demonstrated through its applications to drive assistant systems and 3D mapping, both very complex real world tasks.

The partners of the BACS project are: Laboratoire de Système Autonomes - Ecole Polytechnique Fédérale de Zurich (coordinator), Laboratoire Gravir - INRIA - CNRS, Laboratoire de Physiologie de la Perception et de l'Action (LPPA) - Collège de France, Département d'étude Cognitive - Ecole Normale Supérieure, Institute for Biological Cybernetics - Max-Planck society, Institut Dalle Molle d'Intelligence Artificielle, Les Hopitaux Universitaires de Genève, Faculdade de Ciencas et Technologia - Universidade de Coimbra, ProBAYES S.A.S, Bluebotics SA, Electricité de France (EDF)

*France-Singapore "Neurophysiology and robotics" [June 2004-June 2007]*

This CNRS-PICS project involving the Collège de France (LPPA), INRIA
*e-Motion*, the University of Singapore (NUS), and the
*IPAL*joint CNRS-NUS laboratory in Singapore. The objective is to study some aspects of the physiology of human vision, and to try to develop robotics models inspired from biological
systems. An application of this research we will be to control a wheelchair from natural human control channels. This research involves a co-directed PhD student located in singapore.

*ICT-Asia "FACT" [October 2005-December 2007]*

The Fact project is a joint research project in the scope of the ICT-Asia programme founded by the French Ministry of foreign affairs, the CNRS and INRIA. It aims at conducting common research activities in the area of Intelligent Transportation Systems (ITS). The main objective is to develop new technologies related to the concept of "Cybercar". The project involves the following research teams : e-Motion project at INRIA Rhône-Alpes (leader), Imara project at INRIA Rocquencourt, LASMEA Laboratory at Clermont-Ferrand, SungKyunKwan University (Korea), Shangai Giao Tong University (China), Nanyong Technological University (Singapore) and Tokyo University (Japan).

Since October 1997,
*e-Motion*has a collaboration with Riken Institute in Tokyo in the multi-robots systems field . Crossed visits have occured for the last three years. In 2002, Thierry Fraichard spent 4
months in Riken Institute and Igor Paromtchik spent 4 months at INRIA. Informal exchanges are still going on.

The common laboratory, named
*Intelligent Vehicle Lab*, between Nanyang Technological University of Singapore (NTU) and Inria has started in November 1998, in the framework of the scientific collaboration in the
field of autonomous vehicles. This collaboration has brought: (a) an important number of crossed visits and stays (one week to several months) of researchers, (b) Singaporeans
students in Inria (level undergraduate to graduate), and (c) organization of workshops (1999-2002). in 2003, Julien Diard has been a Postdoc student in NUS and a co-directed PhD
(Brice Rebsamen) will begin in January 2004 in NUS. In addition, a PICS CNRS project has been accepted in 2004 with NUS and LPPA(College de France, Alain Berthoz).

In the scope of the France-Korea Safemove project,
*e-Motion*has a collaboration with the SungKyunKwan University at Suwon (Korea).

see the description of the ICT-ASIA “Fact” project above.

The thematic network "Image et Robotique" has been implemented from the French-Mexican symposium in Computer Sciences and Control (JFMIA'99) which has been held in Mexico in March 1999. The main goal of this network is to promote and increase the french-mexican cooperations in Image and Robotics in scientific, academic and industrial fields. This network has been effectively settled in 2000. It supports a yearly school, students exchange and crossed visits since 2000.

The dissemination of results and the active participation to international scientific events (see bibliography) are two essential activities of the
*e-Motion*project-team.

Some members of
*e-Motion*participate to some international committees:

C. Laugier has published with R. Chatila in 2007 a book in the springer STAR on “Safe Navigation in Dynamic Environment”

C. Laugier is a member of the steering-advisory committee of IEEE/RSJ IROS (Intelligent Robots and Systems) international conference since 1997. He is also a member of the advisory committee of the ICARCV International conference on Control, Automation, Robotics and Vision.

C. Laugier is a member of the steering committee of the European Network EURON.

C. laugier is co-chair of the IEEE Technical Committee on “Intelligent Transportation Systems” (since 2005).

C. laugier is coordinator of the ICT-Asia Network on ITS named FACT (including partners from France, Singapore, Japan, Korea, and China).

C. laugier is a member of the permanent organization committee of the Field and Service Robotics Conference (since 1997).

C. laugier is a member of the editorial board of the journal “Intelligent Service Robotics” (since 2005). He is also a member of the editorial board of the national journal “Revue d'Intelligence Artificielle” (since 1987).

A Martinelli is a member of the permanent organization committee of the Field Service Robotics (since 2007)

A. Martinelli organized a workshop in the framework of the BACS project in February 2007.

C. Laugier and A. Martinelli are guest editors for a special issue on IJRR journal (to be published in 2008)

C. Laugier and A. Martinelli are guest editors for a special issue on JFR journal (to be published in 2008)

In addition to ponctual academic lectures, the members of
*e-Motion*have tought the following lectures:

Lecture “Introduction to robotics and current research issues” (every year): France-Mexico Summer school on “Image and Robotics” (every year).
*Teacher: C. Laugier.*

Lecture “Robotics and motion autonomy” (every year): DEA “Imagerie, Vision, Robotique” INPG, Grenoble, (FR).
*Teacher: C. Laugier*.

Lecture “Basic tools and models for Robotics” (every year): Cnam Grenoble.
*Teachers: C. Laugier and J. Troccaz.*

Motion Planning course, LAFMI Summer School on Image and Robotics: Mexico City (MX),
*July 2007*.
*Teacher: Th. Fraichard.*

Motion Planning course, INPG-UJF “Image, Vision & Robotics” Master course, Grenoble.
*Teacher: Th. Fraichard.*

Lecture “Outils probabilistes et statistiques pour l'Informatique”: (every year): Licence d'Informatique de l'Université Joseph Fourier 3 ème année, Grenoble, (FR).
*Teacher: O. Aycard*.

Lecture “Inférence et apprentissage bayesien": (every year): Master d'Informatique de l'Université Joseph Fourier 1ère année, Grenoble, (FR).
*Teacher: O. Aycard*.

Lecture “Inférence et apprentissage bayesien": (every year): Ecole Polytechnique Universitaire de Grenoble, Filière Technologies de l'Information pour la Santé 3ème
année, Grenoble, (FR).
*Teacher: O. Aycard*.

Lecture “Raisonnement bayesien”: (every year): Master 2ème année “Imagerie, Vision, Robotique” de l'INPG, Grenoble, (FR).
*Teachers: P. Bessière and O. Aycard*.

Lecture “Bayesian techniques in vision and perception”: France-Mexico Summer school on “Image and Robotics” (every year).
*Teachers: O. Aycard, E. Sucar*.

A. Martinelli held a course on SLAM for PhD students at EPFL in Lausanne (Switzerland)

Some members of
*e-Motion*participates to various international conferences committees and to the organization of summer schools :

C. Laugier and J.M. Ahuactzin participated to the organizing committee of the 2005 summer school on “Image and Robotics” at the University of Montpellier, Montpellier (July 2006).

C. Laugier participates every year to the organization committees of the major international conference on Robotics, in particular : IEEE International Conference on Robotics and Automation (ICRA), IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), International Conference on Field and Service Robotics (FSR). He was general chair of IROS'97, Regional programm chair of IROS'00, Program chair of IROS'02. He will be program chair of IEEE/RSJ IROS 2008 to be held in Nice (France).

C. Laugier was the general Chair of the International conference on Field and Service Robotics 2007, held in Chamonix (France), July 2007.

C. Laugier has organized a workshop on “Safe navigation in Dynamic environments; application to driving assistance” in the scope of the IEEE/RSJ IROS'06 conference in Beijing (Oct. 2006).

C. Laugier has organized a workshop on “Intelligent Transportation Systems” related to the IEEE Intelligent Vehicle conference 2006 in Tokyo (June 2006); this workshop has been organized in the scope of the ICT-Asia Network FACT.

C. Laugier has organized a workshop on “Navigation in Dynamic Environments” in the scope of the IEEE ICRA'07 conference in Rome (Apr. 2007).

C. Laugier has organized a workshop on “Navigation in Dynamic Environments” in the scope of the IEEE/RSJ IROS'07 conference in San Diego (Oct. 2007).

Th. Fraichard is a regular member of the programme committees of the ICRA and IROS conferences. In 2007, he was also a member of the following programme committee:

Workshop on Planning, Perception and Navigation for Intelligent Vehicles, Roma (IT),
*Apr. 2007*.

Th. Fraichard gave a number of invited talks:

*Safely Navigating Dynamic Environments*, Rice Univ., Houston, TX (US)
*Nov. 2007*.

*Safely Navigating Dynamic Environments*, “Algorithmic Motion Planning for Autonomous Robots in Challenging Environments” workshop, San Diego, CA (US)
*Oct. 2007*.

*Safe Motion in Dynamic Environments*, Carnegie Mellon Univ., Pittsburgh, PA (US),
*June 2007*.

*Motion Safety for Mobile Robots*, “Bayesian Approach to Cognitive Systems” workshop, Grenoble,
*Feb. 2007*.

O. Aycard is member of the programme committees of the FSR'2007 conference.

O. Aycard is member of the programme committees of the IEEE ITSC'2007 conference.

A. Spalanzani is part of the editorial commitee of the
*In Cognito*cognitive sciences journal.

Pierre Bessière organized the
*Bayesian Cognition*international workshop on probabilistic models of perception, inference, reasoning, decision, action, learning and neural processing (
http://

P. Bessière is a member of the programme committees of the following conferences :

Conference ESANN (European Symposium on Artificial Neural Networks)

Conference RFIA (Reconnaissance des Formes et Intelligence Artificielle)

Conference IEEE/ICRA (International Conference on Robotics and Automation)

Conference IEEE/IROS (International Conference on Intelligent Robots and Systems)

Conference EA (International Conference on Artificial Evolution)

P. Bessière reviews regularly in the IEEE Transactions on Evolutionary Computation and Autonomous Robots journals.