Research activities of the Lagadic team are concerned with visual servoing and active vision. Visual servoing consists in using the information provided by a vision sensor to control the movements of a dynamic system. This system can be real within the framework of robotics, or virtual within the framework of computer animation or augmented reality. This research topic is at the intersection of the fields of robotics, automatic control, and computer vision. These fields are the subject of profitable research since many years and are particularly interesting by their very broad scientific and application spectrum. Within this spectrum, we focus ourselves on the interaction between visual perception and action. This topic is significant because it provides an alternative to the traditional Perception-Decision-Action cycle. It is indeed possible to link more closely the perception and action aspects, by directly integrating the measurements provided by a vision sensor in closed loop control laws.
This set of themes of visual servoing is the central scientific topic of the Lagadic group. More generally, our objective is to design strategies of coupling perception and action from images for applications in robotics, computer vision, virtual reality and augmented reality.
This objective is significant, first of all because of the variety and the great number of potential applications to which can lead our work. Secondly, it is also significant to be able to raise the scientific aspects associated with these problems, namely modeling of visual features representing in an optimal way the interaction between action and perception, taking into account of complex environments and the specification of high level tasks. We also work to treat new problems provided by imagery systems such as those resulting from an omnidirectional vision sensor or echographic probes. We are finally interested in revisiting traditional problems in computer vision (3D localization, structure and motion) through the visual servoing approach.
Amaury Dame and Eric Marchand received the Best Paper Runner-Up Award at IEEE ISMAR'2010 for their paper about visual tracking using mutual information.
Basically, visual servoing techniques consist in using the data provided by one or several cameras in order to control the motions of a dynamic system . Such systems are usually robot arms, or mobile robots, but can also be virtual robots, or even a virtual camera. A large variety of positioning tasks, or mobile target tracking, can be implemented by controlling from one to all the degrees of freedom of the system. Whatever the sensor configuration, which can vary from one on-board camera on the robot end-effector to several free-standing cameras, a set of visual features has to be selected at best from the image measurements available, allowing to control the desired degrees of freedom. A control law has also to be designed so that these visual features reach a desired value , defining a correct realization of the task. A desired planned trajectory can also be tracked. The control principle is thus to regulate to zero the error vector . With a vision sensor providing 2D measurements, potential visual features are numerous, since 2D data (coordinates of feature points in the image, moments, ...) as well as 3D data provided by a localization algorithm exploiting the extracted 2D features can be considered. It is also possible to combine 2D and 3D visual features to take the advantages of each approach while avoiding their respective drawbacks.
More precisely, a set
of
kvisual features can be taken into account in a visual
servoing scheme if it can be written:
where
describes the pose at the instant
tbetween the camera frame and the target frame,
the image measurements, and
a set of parameters encoding a potential additional
knowledge, if available (such as for instance a coarse
approximation of the camera calibration parameters, or the 3D
model of the target in some cases).
The time variation of can be linked to the relative instantaneous velocity between the camera and the scene:
where is the interaction matrix related to . This interaction matrix plays an essential role. Indeed, if we consider for instance an eye-in-hand system and the camera velocity as input of the robot controller, we obtain when the control law is designed to try to obtain an exponential decoupled decrease of the error:
where is a proportional gain that has to be tuned to minimize the time-to-convergence, is the pseudo-inverse of a model or an approximation of the interaction matrix, and an estimation of the features velocity due to a possible own object motion.
From the selected visual features and the corresponding interaction matrix, the behavior of the system will have particular properties as for stability, robustness with respect to noise or to calibration errors, robot 3D trajectory, etc. Usually, the interaction matrix is composed of highly non linear terms and does not present any decoupling properties. This is generally the case when is directly chosen as . In some cases, it may lead to inadequate robot trajectories or even motions impossible to realize, local minimum, tasks singularities, etc. It is thus extremely important to design adequate visual features for each robot task or application, the ideal case (very difficult to obtain) being when the corresponding interaction matrix is constant, leading to a simple linear control system. To conclude in few words, visual servoing is basically a non linear control problem. Our Holy Grail quest is to transform it to a linear control problem.
Furthermore, embedding visual servoing in the task function approach allows solving efficiently the redundancy problems that appear when the visual task does not constrain all the degrees of freedom of the system. It is then possible to realize simultaneously the visual task and secondary tasks such as visual inspection, or joint limits or singularities avoidance. This formalism can also be used for tasks sequencing purposes in order to deal with high level complex applications.
Elaboration of object tracking algorithms in image sequences is an important issue for researches and applications related to visual servoing and more generally for robot vision. A robust extraction and real-time spatio-temporal tracking process of visual cues is indeed one of the keys to success of a visual servoing task. To consider visual servoing within large scale applications, it is mandatory to handle natural scenes without any fiducial markers but with complex objects in various illumination conditions. If fiducial markers may still be useful to validate theoretical aspects of visual servoing in modeling and control, non cooperative objects have to be considered to address realistic applications.
Most of the available tracking methods can be divided into two main classes: feature-based and model-based. The former approach focuses on tracking 2D features such as geometrical primitives (points, segments, circles,...), object contours, regions of interest...The latter explicitly uses a model of the tracked objects. This can be either a 3D model or a 2D template of the object. This second class of methods usually provides a more robust solution. Indeed, the main advantage of the model-based methods is that the knowledge about the scene allows improving tracking robustness and performance, by being able to predict hidden movements of the object, detect partial occlusions and acts to reduce the effects of outliers. The challenge is to build algorithms that are fast and robust enough to meet our applications requirements. Therefore, even if we still consider 2D features tracking in some cases, our researches mainly focus on real-time 3D model-based tracking, since these approaches are very accurate, robust, and well adapted to any class of visual servoing schemes. Furthermore, they also meet the requirements of other classes of application, such as augmented reality.
The natural applications of our research are obviously in robotics. In the past, we mainly worked in the following fields:
grasping and manipulating tools in hostile environments such as nuclear environment typically;
underwater robotics for the stabilization of images and the positioning of uninstrumented robot arm;
agro-industry for the positioning of a vision sensor in order to ensure an improvement of the quality controls of agro-alimentary products; and
video surveillance by the control of the movements of a pan-tilt camera to track mobile natural objects.
More recently, we addressed the field of mobile robotics through activities around the Cycab vehicle (see Section ): detection and tracking of mobile objects (pedestrians, other vehicles), control by visual servoing of the movements of the vehicle.
In fact, researches undertaken in the Lagadic group can apply to all the fields of robotics implying a vision sensor. They are indeed conceived to be independent of the system considered (and the robot and the vision sensor can even be virtual for some applications).
Currently, we are interested in using visual servoing for robot arms in space, micromanipulation, autonomous vehicle navigation in large urban environments, and underactuated flying robots such as miniature helicopters and aircrafts.
We also address the field of medical robotics. The applications we consider turn around new functionalities of assistance to the clinician during a medical examination: visual servoing on echographic images, active perception for the optimal generation of 3D echographic images, compensation of organ motions, etc.
Robotics is not the only possible application field to our researches. In the past, we were interested in applying visual servoing in computer animation, either for controlling the motions of virtual humanoids according to their pseudo-perception, or for controlling the point of view of visual restitution of an animation. In both cases, potential applications are in the field of virtual reality, for example for the realization of video games, or virtual cinematography.
Applications also exist in computer vision and augmented reality. It is then a question of carrying out a virtual visual servoing for the 3D localization of a tool with respect to the vision sensor, or for the estimation of its 3D motion. This field of application is very promising, because it is in full rise for the realization of special effects in the multi-media field or for the design and the inspection of objects manufactured in the industrial world.
Lastly, our work in visual servoing and active perception could be related with those carried out in cognitive science, in particular in the field of psychovision (for example on the study of eye motion in the animal and human visual system, on the study of the representation of perception, or on the study of the links between action and perception).
Visual servoing is a very active research area in robotics. A software environment that allows fast prototyping of visual servoing tasks is then of prime interest. The main difficulty is that it usually requires specific hardware (the robot and, most of the time, dedicated image framegrabbers). The consequence is that the resulting applications are often not portable and cannot be easily adapted to other environments. Today's software design allows proposing elementary components that can be combined to build portable high-level applications. Furthermore, the increasing speed of micro-processors allows developing real-time image processing algorithms on an usual workstation. We have thus developed a library of canonical vision-based tasks for eye-in-hand and eye-to-hand visual servoing that contains the most classical visual features that are used in practice. The ViSP software environment features all the following capabilities: independence with respect to the hardware, simplicity, extendability, portability. Moreover, ViSP involves a large library of elementary positioning tasks with respect to various basic visual features (points, straight lines, circles, spheres, cylinders, frames, ...) that can be combined together, and an image processing library that allows tracking of visual cues (dots, segments, ellipses,...). Simulation capabilities are also available. ViSP and its full functionalities are presented Fig. and described in .
This year, we continued to improve the software and
documentation quality. A new open source versions was
released in September 2010. It is available at
http://
To ensure the stability of the software, daily builds were tighten on Inria's porting platform (Pipol) to test ViSP on new materials but also different software architectures (Linux Fedora, Linux Ubuntu, Windows, Mac OS). Moreover, new functionnalities were introduced like 3D model-based tracker, wireframe simulator and robot simulator, planar object detector, and real-time data plotting.
ViSP last release code has been downloaded more than 200 times since end of September. It is used in research labs in France, USA, Japan, Korea, India, China, Lebanon, Italy, Spain, Portugal, Hungary, Canada. For instance, it is used as a support in a graduate course delivered at MIT, at IFMA Clermont-Ferrand and ESIR Rennes engineer schools.
We exploit several experimental platforms to validate our researches in visual servoing and active vision. More precisely, we have two industrial robotic systems built by Afma Robots in the nineties. The first one is a Gantry robot with six degrees of freedom, the other one is a cylindrical robot with four degrees of freedom (see Fig. a). These robots are equipped with cameras mounted on their end effector. These equipments require specific hardware, but also software maintenance actions and new developments in order to make them evolve. Training and assistance of the users, presentation of demonstrations also form part of the daily activities.
This year, a new electric two parallel fingers gripper was installed. Thanks to tool changers, depending on the application, it is possible to plug easily one of our electric or pneumatic grippers to the end-effector of the Gantry robot.
This platform is by far the most-used one by Lagadic members (5 papers published in 2010 enclose results validated on it). This year, it was opened and used by a researcher from I3S at Sophia-Antipolis.
To improve the panel of demonstrations and to highlight our research activities, we are developing new robotic vision-based applications. This year we designed a new demonstration that combines 3D model-based visual tracking and visual servoing techniques provided in ViSP (see Section ) to pick up cubes in order to build a tower. One of the challenges was to automize the object localization requested to initialize the tracker. At this end we have developed a generic template pose estimation algorithm based on Surf or Ferns points of interest. Since this algorithm requires a learning stage that could be heavy depending on the objects to consider, other techniques based on object detection like MSER is under development.
To validate our researches in medical robotics, we exploit since 2004 a six degrees of freedom Hippocrate medical arm designed by the Sinters company (see Fig. .c). Last year, this platform was extended with a new and more accurate Adept Viper S850 arm (see Fig. .d). A force torque sensor is mounted on their end-effector, which holds an 2D ultrasound probe connected to a SonoSite 180 Plus imaging system. Each robot is connected to a PC running Linux where high level ultrasound image processing and visual servoing is performed. These materials are shared betweeen the Lagadic and Visages teams.
This year, as described in Section , a multi-plane approach for ultrasound visual servoing applied to a registration task was performed with the Viper arm equiped with the 2D US probe . To be able to extend this work, a new Ultrasonix SonixTouch system connected to a 3D motorized probe or a biplane probe was installed. It will allow us to consider 3D ultrasound volumes to ease the automatic positioning of an US probe in order to reach a desired B-scan image of an object of interest.
The Cycab is a four wheel drive autonomous electric car dedicated to vision-based mobile robotic applications (see Fig. .b). A pan-tilt head (Biclops PTM) equipped with a firewire Marlin camera with about 70 degrees field of view is mounted on the front bumper. This year a new Sick LDMRS laser rangefinder was installed. Concerning the computer units, the Cycab is equipped with two microprocessors dedicated to the low level control of the vehicle actuators and a laptop dedicated to high level visual servoing applications. They are connected through an internal CAN bus. The camera, the pan-tilt head and the laser rangefinder are connected to the laptop.
This year, with the engineer staff in charge of the Cycab at Inria Grenoble - Rhônes-Alpes, the low level controller installed last year was improved and extended to support the Aroccam framework. Aroccam is the software platform developed by Lasmea and Cemagref at Clermond-Ferrand and adopted by all the partners of the ANR CityVIP project (see Section ).
Our vision-based navigation scheme in outdoor urban environments is under modification to improve the accuracy in navigation from a visual memory, but also to handle obstacle avoidance by exploiting the new laser rangefinder (see Section ).
This long-term study is directly related to the search of adequate visual features, as described in Section . The approach we currently develop is based on the spherical projection model since it provides interesting geometrical properties. It also allows considering the same modeling for classical perspective cameras and omnidirectional vision sensors such as fish-eye and catadioptric sensors. In collaboration with Romeo Tatsambon, we have been interested this year in determining the best visual features from a set of three points . Decoupling properties have been obtained by using the distances between the points observed on the sphere, which are invariant to any rotation. The three other features used to control the robot orientation are based on a particular rotation matrix. This particular choice of features allowed to revisit the classical singular configurations exhibited a long time ago, but with a very complex demonstration. Furthermore, experimental results obtained with the Afma 6 robot have shown a wide convergence domain even in the presence of points depth estimation errors.
Finally, our previous works about the use of spherical moments have been published in and . These works were realized in collaboration with Omar Tahri (ISR Coimbra), Youcef Mezouar (Lasmea), and Peter Corke (CSIRO, Melbourn).
One of the main problems in visual servoing is to extract and track robustly the image measurements that are used to build the visual features involved in the control scheme (see equation ( )). This may lead to complex and time consuming image processing algorithms, and may increase the effect of image noise. To cope with this problem, we proposed to use directly photometric features as input of the control scheme. More precisely, the luminance of all pixels in the image, that is , is used as input of the control scheme .
This year, in collaboration with Christophe Collewet who is now a member of the Fluminance team, we proposed a way to perform visual servoing tasks from image gradient . This approach can be seen as an extension of our previous works based on the luminance . Indeed, considering that most of the useful information in an image is located in its high frequency areas (that are contours), we have considered various possible combinations of visual features based on luminance and gradient. Gradients are directly used in the control law avoiding therefore any complex image processing as features extraction or matching.
We also extended this approach to omnidirectional photometric visual servoing (see section ).
Recently, new approaches have been proposed that do not require any visual feature extraction or matching to solve the eye-in-hand visual servoing problem (see Section ). One problem that remains with these approaches is the robustness with respect to illumination variations and occlusions. To solve this problem, we focused on mutual information (MI), an alignment function that is known to be very robust to these perturbations. In a first work, we showed how the visual servoing problem can be formulated as maximizing the MI between the desired and the current images. Using classical optimization approaches, promising preliminary results have shown a very good positioning accuracy, even under strong illumination variations. It has also been possible to achieve first multimodal visual servoing experiments. Nevertheless, a further study of the MI function and its derivatives has shown that classical optimization of MI was performed using false approximations. Taking the correct derivatives into account, a new approach has been proposed in , that makes the whole optimization more simple, more efficient, and that increases the convergence domain of the task. Finally, this work has been summarized in .
This study is devoted to the design of new kinematics control schemes . We have proposed a new projection operator for the redundancy framework. Classically, this operator allows considering the components of any secondary task that do not perturb the regulation to zero of the primary task. This means that some degrees of freedom have to remain available, which is very restrictive in practice. The new projection operator does not consider all the components of the main task, but its norm, which involves only one scalar constraint, instead of several, while preserving the stability properties of the system. The new projection operator has been validated using as main task a visual homing that induces all the six degrees of freedom of the system and trajectory following as secondary task . It has also been validated by considering joints limits avoidance as a supplementary secondary task .
Finally, our previous works realized in collaboration with Guillaume Allibert and Estelle Courtial about the application of predictive cntrol in visual servoing has been published in and .
This study is devoted to the automatic landing on an aircraft carrier by visual servoing. This year we have been interested by the image processing and computer vision parts of the process. First, an initialization procedure has been developed. It is based on a template matching whose search area is defined from the available sensors on the aircraft. Then, we have used the 3D model-based tracker developed in the past to localize the carrier with respect to the aircraft .
This study was realized within the ANR Psirob Scuav project (see Section ). We are interested in fusing the data provided by several sensors directly in the control law, instead of estimating the state vector. For that, we have first considered autocalibration methods to estimate the intrinsic and extrinsic parameters of sensors. The method we have developed is based on the simultaneous measure of the robot velocity and features velocity in the sensor space. It has been validated through experimental results using a classical camera . We have also developed new control schemes to fuse the data provided by multiple sensors.
This long-term study is devoted to appearance-based navigation from an image database. It is carried out in the scope of the ANR Tosa Cityvip project (see Section ). The navigation relies on a monocular camera, and the navigation path is represented as a set of reference images, acquired during a preliminary teaching phase. This year, we have developed a method to avoid potential obstacles during the navigation. For that, we use a pan-tilt camera so that it is able to observe the environment along the path while avoiding the obstacles. A new control scheme based on the redundancy framework has been developed and validated thanks to simulation results . Promising experimental results have also been obtained recently.
This work has been done in collaboration with FEMTO-ST/AS2M in Besançon. Robotic microassembly is a promising way to build micro-metric components based on 3D compound productse: structures, devices, MEMS, MOEMS,... In this work the relevance of real-time 3D visual tracking and servoing has been demonstrated. The poses of the MEMS are supplied in real time by the 3D model-based tracking algorithm we developed few years ago . The assembly of 400 m × 400 m × 100 m parts by their 100 m × 100 m × 100 m notches with a mechanical play of 3 m is achieved with a rate of 41 seconds per assembly. Assembly of a complex compound has also been demonstrated . We have also considered an original method for microscope autofocusing based on photometric visual servoing .
We continue our works dedicated to controlling the motion of a 2D ultrasound probe actuated by a medical robot in order to reach and to track a desired cross-section image. The visual features are based on the moments computed on a region segmented in the image, which is detected and tracked in practice using an active contour. The method allows the control of both in-plane and out-of-plane probe motions . In order to endow the system with the capability of automatically interacting with objects of unknown shape, a model-free visual servoing was developed. It is based on a local and on-line estimation of the object surface normal obtained by considering successive images. This general approach has been validated in simulations, experiments on an ultrasound phantom, and ex-vivo experiments using a motionless kidney immersed in a water-filled tank. Rafik Mebarki defended his Ph.D. thesis related to this work in March 2010 .
We have studied a new approach for rigid registration of an intra-operative ultrasound image with a pre-operative volume by formulating it as a virtual visual servoing problem . In this case, the desired image is provided by the ultrasound probe used in the operating room and the current one is obtained by the interaction of a virtual probe with the pre-operative volume. The visual servoing strategy is then used to move this virtual probe to minimize the error between the current and desired image features. In the proposed approach, moments based image features are extracted from three orthogonal ultrasound images to servo in-plane and out-of-plane motions of the system. Experimental results demonstrate that this approach improves upon techniques based on a single 2D US image in term of probe positioning. Multimodal registration experiments have been performed with an ultrasound phantom containing an egg-shaped object to provide a first experimental validation of the proposed method.
This study is realized within the ANR Contint Prosit project (see Section ). It consists in developing several autonomous control modes based on ultrasound visual servoing that will assist the physician during a robotized and teleoperated ultrasound examination (tele-echography). This year, a new method of active contour (snake) based on Fourier descriptors has been developed. It allows to detect non-holomorphic contour of an anatomical element of interest and can also detect the topology change of the observed object. The tracking part of the snake was implemented on GPU using the CUDA language in order to provide real-time performance. We also studied a first autonomous control mode to guarantee the visibility of an anatomical element of interest while the physician is teleoperating the probe. In this mode, the physician can control the rotation around the Y-axis of the probe (main axis). The two other axes (the rotation around X-axis and the rotation around Z-axis) are automatically controled by visual servoing to keep the anatomical element of interest in the center of the image with a maximum cross-section.
This study is realized within the ANR Contint USComp project (see Section ). In the context of this project, a new 3D ultrasound imaging system has been acquired by the Lagadic team. As described in Section , it is composed of the Ultrasonix SonixTouch system equipped with a 3D motorized probe. To access to the volumetric ultrasound images we first developed an algorithm that performs in real time the 3D scan conversion from a set of 2D pre-scan data captured by the motorized ultrasound 3D probes. This algorithm was implemented on GPU using the CUDA language in order to reduce the processing time and to insure a 3D image streaming in real-time. Until now, we considered only rigid motion in our compensation tracking tasks. However, in practice soft tissue may deform due to the physiological motion of the patient. Therefore we recently developed a 3D deformable motion model using TPS (thin-plate spline) model. The motion parameters of the motion model, which are the 3D control points of the TPS equation, are estimated from the variation of the voxels intensity between successive 3D ultrasound images. This method has been implemented with parallel computing on GPU. First simulation results show us the effectiveness of the proposed method to track deformation in a sequence of 3D ultrasound images.
This study focuses on real-time augmented reality (AR) for mobile devices. Its goal is to enable AR on mobile devices like GSM or PDA used by pedestrians in urban environments.
We have proposed a method for camera pose tracking that uses a partial knowledge on the scene. The method is based on a monocular vision localization method that uses previously known information about the environment (that is, the map of walls) and takes advantages from the various available databases and blueprints to constrain the problem. We have extended this approach in order to consider both a camera and an inertial sensor (IMU) .
In the scope of the ANR AM Gamme project (see Section ), we also examined how an AR guide can enrich museum visits. An ergonomic experimentation has been conducted where real visitors used our AR prototype . We collected feedback from these users, helping us to identify the usefulness of AR for museum visits or appreciation of art work.
The motivation of this work is to develop tracking algorithms that are suitable for the control of small UAV (X4 flyers). In the work carried out this year, the model-based tracking problem has been considered. Assuming a 3D model of the edges of an object is known, the tracking then consists in finding the camera pose which best aligns the projection of this model with the edges of the image.
The existing deterministic approaches (virtual visual servoing, Newton's minimization,...) usually suffer from possible ambiguities when tracking edges, since different edges may show very similar appearances leading to tracking errors. In order to handle these ambiguities, an optimisation method has been designed in which several hypotheses are maintained at the edge-tracking level, to retrieve several possible camera poses . This process is then used to optimize the best particles of a particle filter. Particle filtering framework allows the tracking to be robust to occlusions and large displacements that are expected in the considered application .
Work has also been done to fuse inertial data from the UAV with the visual tracking, to improve the prediction of the filter and build an estimate of the UAV's velocity . Complete visual servoing experiment have been done using the X4 flyer UAV developed at CEA List to validate positioning tasks and tracking tasks .
This study with Guillaume Caron and El Mustapha Mouaddib started when they were still in MIS lab at the Université Picardie Jules Verne in Amiens. The motivation of this work is to take advantage of the wide field of view induced by catadioptric cameras.
This year we focused on omnidirectional visual servoing techniques taking into account the photometric information of the entire image . This approach was previously tackled with images of perspective cameras (see Section ). We adapted this technique to central cameras. We also proposed to adapta method for gradient computation to take into account distortions of such cameras.
In an other study, we have been interested by the redundancy brought by stereovision for omnidirectional vision sensors. This has been obtained by combining a single camera and multiple mirrors. Within this framework we proposed a method to calibrate the stereo-catadioptric sensor .
In this work, we take advantage of the robustness of MI for visual tracking. Some approaches have already been proposed in this area. However, all of them were based on a very coarse approximation of the derivatives involved in the optimization process, which limited the efficiency and accuracy of the tracking task. In , we used the new optimization approach that was proposed for the visual servoing problem (see Section ) and adapted it to the tracking problem. This optimization approach is perfectly suited to solve the forward compositional and inverse compositional tracking approaches. By using our new optimization scheme in the inverse compositional formulation, we showed that an accurate real-time estimation of the displacement parameters is possible. Experiments on reference benchmarks have shown that the convergence domain of MI is as wide as the one of SSD, competing with the results obtained using the ESM algorithm with classical sequences, and having better robustness in case of illumination variations. The paper won the Best Paper Runner-Up Award at ISMAR'2010 conference.
no. Inria 5140, duration : 36 months.
This contract supports Laurent Coutard's Ph.D. about automatic aircraft landing on carrier by visual servoing (see Section ). Dassault Aviation provided us a realistic flight model.
no. Inria 4759, duration : 6 months.
This contract was devoted to test our 3D model-based tracker for autonomous satellites rendezvous.
duration : 36 months.
This contract is devoted to support the Cifre convention between France Telecom R&D and Université de Rennes 1 regarding Pierre Martin's Ph.D. (see Section ). The goal of the Ph.D. is to enable augmented reality on mobile devices like GSM used by pedestrians in urban environments. More precisely, its aim is to compute the absolute pose of the camera to show to the end-user geolocalized information in an explicit way.
no. Inria 2435, duration: 42 months.
This project, led by Tarek Hamel from I3S, ended on November 2010. It was realized in collaboration with I3S, the EPI ARobAS at Inria Sophia Antipolis-Méditerranée, Heudiasyc in Compiègne, the CEA-List and the Bertin company. It was devoted to the sensor-based control of small helicopters for various applications (stabilization landing, target tracking, etc.). In this project, we have been interested in sensor autocalibration and in multi-sensor based control (see Section ).
no. Inria 2861, duration: 36 months.
This project started in March 2008. It is realized in collaboration with Orange Labs, CEA Leti, Movea, Polymorph, and the Museum of fine arts in Rennes.
The Augmented Reality (AR) concept aims to enhance our real world perception, combining it with fictitious elements. AR research is concerned with the different methods used to augment live video imagery with coherent computer generated graphics. The combination of mobile technologies and AR will allow the design of a video rendering system with an augmentation of the real world depending on user localisation and orientation.
This project is focused on indoor environments with the implementation of AR technologies on mobile devices as main objective. The experimental field proposed is the Museum, a controlled environment (constant lighing and location of objects) without some of the perturbations of outdoor environments.
Within this project we are involved in visual tracking and sensor fusion parts of the AR process (see Section ).
no. Inria 3208, duration: 42 months.
This project, managed by Lasmea, started in June 2008. It involves eight partners, including Lagadic. The project consists of enhancing the autonomy of urban vehicles by integrating sensor-based techniques with a geographical database. Within CityVIP, Lagadic provides its expertise in vision-based localization and vision-based navigation, including safe navigation in the presence of obstacles. The work that we have realized within this project is described in Section .
no. Inria 3005, duration: 36 months.
This project started in March 2008. It is realized in collaboration with BA Systèmes, CEA List, and Université de Caen. RobM@rket project aims at developing automated applications for order picking in a fast-expanding business which mainly includes manual tasks. The system would apply to packaging before dispatching items ordered on a website through an online catalogue including more than 1000 references or to order picking with orders dedicated to kitting.
The robotic system is made of a PLC mobile platform of AGV type (Automatic Guided Vehicles, by BA Systèmes) and of an industrial robot arm. This platform is used to integrate several algorithms allowing picking up selected items in a warehouse through a command file and bringing them back for dispatching or assembling them. The items can be either methodically stored or jumbled in the boxes. Our work in this project consists in developing vision-based objects localization and visual servoing techniques for grasping them.
no. Inria 3585, duration: 36 months.
This project is a multidisciplinary industrial research type project led by the Prisme lab in Bourges. It started in December 2008 in collaboration with Lirmm in Montpellier, LMS in Poitiers, CHU of Tours, and the Robosoft company. The goal of this project is to develop an interactive master-slave robotic platform for medical diagnosis applications (tele-echography) and to develop a cluster of interactive functionalities combining visual servoing, force control, haptic feedback, virtual human interface, and 3D representation of organs. Within this project, we study and develop autonomous control modes that directly make use of visual data extracted from the 2D ultrasound image and force measurement to move the ultrasound probe. The work that we have realized within this project is described in Section .
no. Inria 3560, duration: 36 months.
This project, led by Alexandre Krupa, started in December 2008. It involves a collaboration with the Visages team in Rennes, LSIIT in Strasbourg and Lirmm in Montpellier. Its goal is to provide methodological solutions for real-time compensation of soft tissues motion during ultrasound imaging. The approach consists in synchronizing the displacement of a 2D or 3D ultrasound transducer to stabilize the observed image by the use of a robotic arm actuating the ultrasound probe. The problematic concerns more specifically the use in the control scheme of the per-operative ultrasound image, the interaction force between the probe and the soft tissues and the measurements of external signals providing the breathing state of the patient. The work that we have realized within this project is described in Sections and .
no. Inria 4549, duration: 36 months.This project started in January 2010. Combination of broadcast technologies and augmented reality should allow to propose interactive systems for which the augmented contents will depend on the localization and orientation of the user. The FUI project ReV-TV proposes to develop such an augmented reality system suitable for TV studio. The goal of this project is to provide tools to develop new TV programs allowing the final user to interact within an immersive and convivial interface. Within this project, we focus on the development of tracking algorithms (3D localization) and on visual servoing techniques for camera localization.
This project aims to handle the problem of monocular real-time 3D object tracking targeting augmented reality and visual servoing applications. This is a collaboration with the computer science center of Federal University of Pernambuco in Recife, Brazil and with the Robotics ans automation division, mining technology center at the Universidad de Chile in Santiago, Chile. Veronica Teichrieb (Recife) and Javier Ruiz del Solar (Santiago) had a one-week visit in Rennes in September 2010. Amaury Dame and Celine Teulière had a two-week visit in University of Recife in November 2010.
Eric Marchand and Hideaki Ushiyama organized a workshop in Rennes with the Keyo University on augmented reality on OCtober 2010.
Short visit by Pierre Dupont (Children's Hospital Boston), Olivier Stasse (JRL/AIST Tsukuba), Sambhunath Biswas (Indian Statistical Institute, Kolkata), Seth Hutchinson (UIUC), Vincent Lepetit (EPFL), Ekrem Misri (Sintef, Norway).
É. Marchand and F. Chaumette were scientific experts for the DGRI (Direction Générale de la Recherche et de l'Innovation) of the French ministry of research till August 2010.
E. Marchand was a member of the evaluation committee of the ANR Contint projects submitted in 2010.
E. Marchand was a member of the ICRA'10 “Best vision paper award” committee.
F. Chaumette is a member of the evaluation committee of the ANR Psirob projects that started in 2007, 2008 and 2009.
F. Chaumette was a member of the evaluation committee of the FP7 Echord projects submitted in March 2010.
F. Chaumette is a member of the Scientific Council of the GdR on Robotics.
F. Chaumette was the Head of the CUMIR of the Inria centre Rennes-Bretagne Atlantique ( Commission des Utilisateurs des Moyens Informatiques) till April 2010. He was also a member of the Inria Cost in charge of the evaluation of the ARC and ADT till June 2010.
F. Chaumette is a member of the executive board of the project committee of Inria centre Rennes-Bretagne Atlantique.
F. Chaumette is a member of the animation committee of Inria's thematic domain “Perception, cognition, interaction”.
F. Spindler is a member of the “Comité de Centre” of the Inria centre Rennes-Bretagne Atlantique.
A. Krupa is a member of the “Commission des Utilisateurs des Moyens Informatiques” of the Inria centre Rennes-Bretagne Atlantique.
Editorial boards of journals
E. Marchand is Associate Editor of the IEEE Trans. on Robotics from June 2010.
F. Chaumette is in the Editorial Board of the Int. Journal of Robotics Research.
F. Chaumette was in charge with Peter Corke (CSIRO Brisbane) and Paul Newman (Univ. Oxford) of a special issue on robot vision appeared in the Int. Journal of Robotics Research in Feb 2010 .
F. Chaumette is Associate Editor of the Int. Journal of Optomechatronics.
Technical program committees of conferences
F. Chaumette: ICRA'10, ETFA'10, PCV'10, PGAI'10, Robotica'10, ISOT'10, IROS'10, ICRA'11.
E. Marchand: CORESA'10, ECCV'10, CVPR'10, ISMAR'10, ORASIS'11, CVPR'11
Ph.D. and HdR jury
F. Chaumette: Rogério de Almeida Richa (Ph.D., Lirmm, reviewer and president), Pierre Lothe (Ph.D., CEA-List and Lasmea, reviewer), Bruno Hérissé (Ph.D., CEA-List and I3S, reviewer), Tiago Goncalves (Ph.D., IST Lisbon, reviewer), Florent Le Bras (Ph.D., I3S, reviewer), Patrick Danès (HdR, Laas, reviewer).
E. Marchand: Thierry Chateau (HdR, Lasmea, reviewer), Iman Mayssa Zendjebil (Ph.D,. Université d'Evry, reviewer), Julien Michot (Ph.D., CEA-List and Lasmea, reviewer), Redwan Dahmouche (Ph.D, Lasmea, reviewer), Guillaume Caron (Ph.D., Université de Picardie-Jules Vernes), Mohammed Marey (Ph.D., Irisa, president), Jean-François Layerle (Ph.D., Université de Picardie-Jules Vernes, president).
Master M2RI of Computer Science, Ifsic, Université de Rennes 1 (É. Marchand): 3D computer vision, augmented reality.
Master SIBM (Signals and Images in Biology and Medicine), Université de Rennes 1, Brest and Angers (A. Krupa): medical robotics for physician students.
Master Erasmus Mundus Emaro at Ecole Centrale de Nantes (F. Chaumette: visual servoing ; E. Marchand: computer vision)
ESIR, Université de Rennes 1 (E. Marchand: 3D vision, image recognition, color images, programming tools for image processing; F. Chaumette: visual servoing).
Undergraduate student interns: Antoine Petit (Supelec Paris), Bertrand Delabarre (ESIR, Université de Rennes 1).
A. Dame and E. Marchand received the Best Paper Runner-Up Award at IEEE ISMAR'2010 for their paper about visual tracking using mutual information.
F. Chaumette gave a plenary talk at the international conference ISIVC'2010 .
Alexandre Krupa was invited to give a presentation at the 2nd GMSI International Symposium organized by the Global Center of Excellence for Mechanical Systems Innovation in the University of Tokyo