The basic concept of Augmented Reality (AR) is to place information correctly registered with the environment into the user's perception. What makes AR stand out is that this new technology offers the potential for big changes in many application fields such as industrial maintenance, creative technologies, image guided medical gestures, entertainment...
Augmented reality technologies have made major advancements recently, both in terms of capability, mobile development and integration into current mobile devices. Most applications are dedicated to multimedia and entertainment, games, lifestyle and healthcare and use rough localization information provided by the sensors of the mobile phones. Cutting-edge augmented reality applications which take place in complex environments and require high accuracy in augmentation are less prevalent. There are indeed still technological barriers that prevent applications from reaching the robustness and the accuracy required by such applications.
The aim of the MAGRIT project is to develop vision based methods which allow significant progress of AR technologies in terms of ease of implementation, reliability and robustness. An expected consequence is the widening of the current application field of AR.
The team is active in both medical and classical applications of augmented reality for which accurate integration of the virtual objects within the scene is essential. Key requirements of AR systems are the availability of registration techniques, both rigid and elastic, that allow the virtual objects to be correctly aligned with the environment, as well as means to build 3D models which are appropriate for pose computation and for handling interactions between the virtual objects and the real scene. Considering the common needs for tracking, navigation, advanced modeling and visualization technologies in both medical and industrial applications, the team focuses on three main objectives: matching, localization and modeling. Methods are developed with a view to meet the expected robustness and accuracy over time and to provide the user with a realistic perception of the augmented scene, while satisfying the real-time achievements required by these procedures.
One of the most basic problems currently limiting AR applications is the registration problem. The objects in the real and virtual worlds must be properly aligned with respect to each other, or the illusion that the two worlds coexist will be compromised.
As a large number of potential AR applications are interactive, real time pose computation is required. Although the registration problem has received a lot of attention in the computer vision community, the problem of real-time registration is still far from being a solved problem, especially for unstructured environments. Ideally, an AR system should work in all environments, without the need to prepare the scene ahead of time, independently of the variations in experimental conditions (lighting, weather condition,...) which may exist between the application and the time the model of the scene was acquired.
For several years, the MAGRIT project has been aiming at developing on-line and marker-less methods for camera pose computation. The main difficulty with on-line tracking is to ensure robustness of the process over time. For off-line processes, robustness is achieved by using spatial and temporal coherence of the considered sequence through move-matching techniques. To get robustness for open-loop systems, we have investigated various methods, ranging from statistical methods to the use of hybrid camera/sensor systems. Many of these methods are dedicated to piecewise-planar scenes and combine the advantage of move-matching methods and model-based methods. In order to reduce statistical fluctuations in viewpoint computation, which lead to unpleasant jittering or sliding effects, we have also developed model selection techniques which allow us to noticeably improve the visual impression and to reduce drift over time. Another line of research which has been considered in the team to improve the reliability and the robustness of pose algorithms is to combine the camera with another form of sensor in order to compensate for the shortcomings of each technology.
The success of pose computation over time largely depends on the quality of the matching at the initialization stage. Indeed, the current image may be very different from the appearances described in the model both on the geometrical and the photometric sides. Research is thus conducted in the team on the use of probabilistic methods to establish robust correspondences of features. The use of a contrario methods has been investigated to achieve this aim . We especially addressed the complex case of matching in scenes with repeated patterns which are common in urban scenes. We are also investigating the problem of matching images taken from very different viewpoints which is central for the re-localization issue in AR. Within the context of a scene model acquired with structure from motion techniques, we are currently investigating the use of viewpoint simulation in order to allow successful pose computation even if the considered image is far from the positions used to build the model .
Recently, the issue of tracking deformable objects has gained importance in the team. This topic is mainly addressed in the context of medical applications through the design of bio-mechanical models guided by visual features . We have successfully investigated the use of such models in laparoscopy, with a vascularized model of the liver and with a hyper-elastic model for tongue tracking in ultrasound images. However, these results have been obtained so far in relatively controlled environments, with non pathological cases. When clinical routine applications are to be considered, many parameters and considerations need to be taken into account. Among the problems that need to be addressed are more realistic model representations, the specification of the range of physical parameters and the need to enforce the robustness of the tracking with respect to outliers, which are common in the interventional context.
Modeling the scene is a fundamental issue in AR for many reasons. First, pose computation algorithms often use a model of the scene or at least some 3D knowledge on the scene. Second, effective AR systems require a model of the scene to support interactions between the virtual and the real objects such as occlusions, lighting reflections, contacts...in real-time. Unlike pose computation which has to be computed in a sequential way, scene modeling can be considered as an off-line or an on-line problem depending on the requirements of the targeted application. Interactive in-situ modeling techniques have thus been developed with the aim to enable the user to define what is relevant at the time the model is being built during the application. On the other hand, we also proposed off-line multimodal techniques, mainly dedicated to AR medical applications, with the aim to obtain realistic and possibly dynamic models of organs suitable for real-time simulation.
In-situ modeling
In-situ modeling allows a user to directly build a 3D model of his/her surrounding environment and verify the geometry against the physical world in real-time. This is of particular interest when using AR in unprepared environments or building scenes that either have an ephemeral existence (e.g., a film set) or cannot be accessed frequently (e.g., a nuclear power plant). We have especially investigated two systems, one based on the image content only and the other based on multiple data coming from different sensors (camera, inertial measurement unit, laser rangefinder). Both systems use the camera-mouse principle (i.e., interactions are performed by aiming at the scene through a video camera) and both systems have been designed to acquire polygonal textured models, which are particularly useful for camera tracking and object insertion in AR.
Multimodal modeling for real-time simulation
With respect to classical AR applications, AR in medical context differs in the nature and the size of the data which are available: a large amount of multimodal data is acquired on the patient or possibly on the operating room through sensing technologies or various image acquisitions . The challenge is to analyze these data, to extract interesting features, to fuse and to visualize this information in a proper way. Within the MAGRIT team, we address several key problems related to medical augmented environments. Being able to acquire multimodal data which are temporally synchronized and spatially registered is the first difficulty we face when considering medical AR. Another key requirement of AR medical systems is the availability of 3D (+t) models of the organ/patient built from images, to be overlaid onto the users' view of the environment.
Methods for multimodal modeling are strongly dependent on the image modalities and the organ specificities. We thus only address a restricted number of medical applications –interventional neuro-radiology, laparoscopic surgery– for which we have a strong expertise and close relationships with motivated clinicians. In these applications, our aim is to produce realistic models and then realistic simulations of the patient to be used for surgeon's training or patient's re-education/learning.
One of our main applications is about neuroradiology. For the last 20 years, we have been working in close collaboration with the neuroradiology laboratory (CHU-University Hospital of Nancy) and GE Healthcare. As several imaging modalities are now available in an intraoperative context (2D and 3D angiography, MRI, ...), our aim is to develop a multi-modality framework to help therapeutic decision and treatment.
We have mainly been interested in the effective use of a multimodality framework in the treatment of arteriovenous malformations (AVM) and aneurysms in the context of interventional neuroradiology. The goal of interventional gestures is to guide endoscopic tools towards the pathology with the aim to perform embolization of the AVM or to fill the aneurysmal cavity by placing coils. We have proposed and developed multimodality and augmented reality tools which make various image modalities (2D and 3D angiography, fluoroscopic images, MRI, ...) cooperate in order to help physicians in clinical routine. One of the successes of this collaboration is the implementation of the concept of augmented fluoroscopy, which helps the surgeon to guide endoscopic tools towards the pathology. Lately, in cooperation with the team MIMESIS, we have proposed new methods for implicit modeling of the vasculature with the aim of obtaining near real-time simulation of the coil deployment in the aneurysm . These works open the way towards near real-time patient-based simulations of interventional gestures both for training and for planning.
Many problems in computer vision or image analysis can be formulated in terms of parameter estimation from image-based measurements. This is the case of many problems addressed in the team such as pose computation or image-guided estimation of 3D deformable models. Often traditional robust techniques which take into account the covariance on the measurements are sufficient to achieve reliable parameter estimation. However, depending on their number, their spatial distribution and the uncertainty on these measurements, some problems are very sensitive to noise and there is a considerable interest in considering how parameter estimation could be improved if additional information on the noise were available. Another common problem in our field of research is the need to estimate constitutive parameters of the models, such as (bio)-mechanical parameters for instance. Direct measurement methods are destructive and elaborating image based methods is thus highly desirable. Besides designing appropriate estimation algorithms, a fundamental question is to understand what group of parameters under study can be reliably estimated from a given experimental setup.
This line of research is relatively new in the team. One of the challenges is to improve image-based parameter estimation techniques considering sensor noise and specific image formation models. In a collaboration with the Pascal Institute (Clermont Ferrand), metrological performance enhancement for experimental solid mechanics has been addressed through the development of dedicated signal processing methods . In the medical field, specific methods based on an adaptive evolutionary optimization strategy have been designed for estimating respiratory parameters . In the context of designing realistic simulators for neuroradiology, we are now considering how parameters involved in the simulation could be adapted to fit real images.
We have a significant experience in AR that allowed good progress in building usable, reliable and robust AR systems. Our contributions cover the entire process of AR: matching, pose initialization, 3D tracking, in-situ modeling, handling interaction between real and virtual objects....
For 15 years, we have been working in close collaboration with University Hospital of Nancy and GE Healthcare in interventional neuroradiology. Our common aim is to develop a multimodality framework to help therapeutic decisions and interventional gestures. Contributions of the team focus on the developments of AR tools for neuro-navigation as well as the development of simulation tools for training or planning. Laparoscopic surgery is another field of interest with the development of methods for tracking deformable organs based on bio-mechanical models. Some of these projects are developed in collaboration with the MIMESIS project team.
In experimental solid mechanics, an important problem is to characterize properties of specimen subject to mechanical constraints, which makes it necessary to measure tiny strains. Contactless measurement techniques have emerged in the last few years and are spreading quickly. They are mainly based on images of the surface of the specimen on which a regular grid or a random speckle has been deposited. We are engaged since June 2012 in a transdisciplinary collaboration with Institut Pascal (Clermont-Ferrand Université). The aim is to characterize the metrological performances of these techniques limited by, e.g., the sensor noise, and to improve them by several dedicated image processing tools.
The Inria development action (ADT) LTrack aims at developing an Android platform in order to facilitate the transfer of some of our algorithms onto mobile devices. For the moment, the tracking-by-synthesis algorithm has been implemented (up to our knowledge, for the first time on a mobile device) in order to rigidly track a real object in real time assuming that a CAD model of this object is available. The design and implementation of the platform have been guided by the need to enable easy integration of any tracking algorithm based on combining video data and other sensor information.
Contact: Marie-Odile Berger, Gilles Simon.
PoLAR (Portable Library for Augmented Reality) is a software library that offers powerful and state of the art visualization solutions under an API that is adapted and easy to use for a computer vision scientist. An ADT, also named PoLAR, started in October, 1st 2014 to sustain its development: a software engineer, Pierre-Jean Petitprez, was hired for two years. His contract ended at the end of September, 2016.
This year, the library was ported on Android, and Qt 5.7. Various diffusion media were also built: demos, e.g. linked with OpenCV; web site http://polar.inria.fr; detailed documentation with tutorials; and a paper was published at ISMAR'2016 conference .
PoLAR was made available to the public in October 2015, and can be used under Linux, Windows, MacOS and Android.
Contact: Erwan Kerrien, Pierre-Frédéric Villard.
Fast>VP is a fast and effective tool to detect vanishing points in uncalibrated images of man-made environments and automatically orthorectify the involved planes. It is a Matlab implementation of the algorithm described in our Eurographics'2016 paper .
Contact: Gilles Simon
This Matlab toolbox implements several efficient and state-of-the art algorithms to estimate displacement and strain fields from grid images deposited on the surface of a specimen submitted to a loading or tensile test.
Contact: Frédéric Sur
Pose initialization
Estimating the pose of a camera from a model of the scene is a challenging problem when the camera is in a position not covered by the views used to build the model, because feature matching is difficult in such a situation. Several viewpoint simulation techniques have been recently proposed in this context. They generally come with a high computational cost, are limited to specific scenes such as urban environments or object-centered scenes, or need an initial guess for the pose. In , we have proposed a viewpoint simulation method well suited to most scenes and query views. Two major problems have been addressed: the positioning of the virtual viewpoints with respect to the scene, and the synthesis of geometrically consistent patches. Experimental results showed that patch synthesis dramatically improves the accuracy of the pose in case of difficult registration, with a limited computational cost.
Facade detection and matching
Planar building facades are semantically meaningful city-scale landmarks. Such landmarks are essential for localization and guidance tasks in GPS-denied areas which are common in urban environments. Detection of facades is also key in augmented reality systems that allow for the annotation of prominent features in the user's view. We introduced several “facadeness” measures of image regions and showed how to combine them to generate building facade proposals in images of urban environments . We demonstrated the interest of this procedure through two applications. First, a convolutional neural network (CNN) was proposed to detect facades from a restricted list of facade proposals. We showed that this method outperforms the state-of-the-art techniques in term of adequation of the detected facades with a ground truth. In addition, the computational time is compatible with the navigation requirements. Second, we investigated image matching based on facade proposals. Considering a large set of data extracted from Google Street View, we showed that matching based on Euclidean distances between CNN descriptors outperforms the classical SIFT matching based on RANSAC-homography calculation. This work has been submitted to IEEE ICRA'2017.
A preliminary step in facade detection is the image rectification process. For that purpose, we introduced a simple and effective method to detect orthogonal vanishing points in Manhattan scenes. A key element of this approach is to explicitly detect the horizon line before detecting the vanishing points, which is done by exploiting accumulations of oriented segments around the horizon line. This results in a significant reduction in computation times, while keeping an accuracy comparable or superior to more complex approaches. A paper reporting on this work was published and an oral presentation was made at Eurographics'2016 .
Simultaneous pose estimation and augmentation of elastic surface
We have proposed an original method to estimate the pose of a monocular camera while simultaneously modeling and capturing the elastic deformation of the object to be augmented . Our method tackles a challenging problem where ambiguities between rigid motion and non-rigid deformation are present. This issue represents a major barrier for the establishment of an efficient surgical augmented reality where the endoscopic camera moves and organs deform. Using an underlying physical model to estimate the low stressed regions our algorithm separates the rigid body motion from the elastic deformations using polar decomposition of the strain tensor. Following this decomposition, a constrained minimization, that encodes both the optical and the physical constraints, is resolved at each frame. Results on real and simulated data proved the effectiveness of our approach.
Fusing US and CT data
3D ultrasound (3D US) is an ideal imaging modality for hepatic image-guided interventions. Yet, its limited field of view and poor in-depth image quality reduce its usefulness. Within J. Guevara's PhD thesis, we propose to reduce these limitations by augmenting the intraoperative 3D US view with a preoperative image. Our approach is automatic and does not require manual initialization or a tracking device for the 3D US probe. Moreover, by using an underlying biomechanical model, the proposed method handles significant liver deformation, even when it occurs outside the 3D US field of view. The method relies on the segmentation of a vascular tree from the preoperative and intraoperative images, and their transformation into graphs. The preoperative and partial intraoperative graphs are then matched using an algorithm based on a combined Gaussian Process regression approach and biomechanical model. The model is used to robustly select a correct match from several hypotheses generated by the Gaussian Process. Once the two graphs are matched, a deformation of the preoperative liver is driven by the local displacement field computed from the partial graph match.
Individual-specific heart valve modeling
We developed a method to semi-automatically build a mitral valve computational model from micro CT (computed tomography) scans: after manually picking fiducial points on the chordae, the leaflets were segmented and the boundary conditions as well as the loading conditions were automatically defined. Fast Finite Element Method (FEM) simulation was carried out using Simulation Open Framework Architecture (SOFA) to reproduce leaflet closure at peak systole. We developed three metrics to evaluate simulation results. We validated our method on three explanted porcine hearts and showed that our model performs well. We evaluated the sensitivity of our model to changes in various parameters. We also measured the influence of the positions of the chordae tendineae on simulation results.
Tools reconstruction for interventional neuro-radiology Minimally invasive techniques impact surgery in such ways that, in particular, an imaging modality is required to maintain a visual feedback. Live X-ray imaging, called fluoroscopy, is used in interventional neuroradiology. Such images are very noisy, and cannot show any brain tissue except the vasculature. In particular, since at most only two projective fluoroscopic views are available, containing absolutely no depth hint, the 3D shape of the micro-tool (guidewire, micro-catheter or micro-coil) can be very difficult, if not impossible to infer, which may have an impact on the clinical outcome of the procedure.
In collaboration with GE Healthcare, we aim at devising ways to
reconstruct the micro-tools in 3D from fluoroscopy images. Charlotte
Delmas has been working as a PhD CIFRE student on this subject since
April 2013. A setup was designed in a view to reconstruct in 3D a
deploying coil in as little X-ray dose and time as possible. It
combines a fast rotation of both X-ray planes around the patient's
head and a tomographic reconstruction combining an
Image driven simulation We consider image-driven simulation, applied to interventional neuroradiology as a coupled system of interactive computer-based simulation (interventional devices in blood vessels) and on-line medical image acquisitions (X-ray fluoroscopy). The main idea is to use the live X-ray images as references to continuously refine the parameters used to simulate the blood vessels and the interventional devices (micro-guide, micro-catheter, coil).
Raffaella Trivisonne started her PhD thesis in November 2015 (co-supervised by Stéphane Cotin, from MIMESIS team in Strasbourg) to address this research topic. Both projective and mechanical constraints were integrated in an augmented Lagrangian framework to solve the dynamical system. Experiments based on synthetic and phantom data were indicative that the shape from template problem could be solved without the need for considering collisions with the vessel surface, if an efficient tracking of the catheter in the X-ray images is available. These results were submitted for publication at a conference.
Progress was made during this year on several aspects of our collaboration with Institut Pascal on experimental mechanics. As mentioned in Section 4.3, the surface of the specimens under study are marked either by a regular grid, or by a random speckle. Displacement and strain maps are estimated by comparing images taken before and after deformation: through spectral methods (named here "the grid method") in the first case and through digital image correlation (DIC) in the latter.
Our contributions to the grid method are twofolds. First, we carefully analyzed the effect of digital sampling which causes aliasing . We have proposed simple guidelines to minimize the effect of aliasing on strain maps. Second, we have mathematically characterized the properties of the analysis windows commonly used for processing grid images through the grid method . It turns out that a Gaussian window has to be used, mainly because of its good concentration in both spatial and spectral domains in the sense of the Wigner-Ville transform. We eventually published a comprehensive review paper on the use of grid methods in experimental mechanics
We also contributed to DIC-based methods. We have proposed new predictive formulas for the resolution of the displacement maps provided by DIC, which is mainly limited by sensor noise. These formulas take interpolation into account . Indeed, displacement amplitude being often much smaller than one pixel, it is crucial to analyze the effect of the interpolation scheme. We have also proposed an experimental validation of these formula. This requires to take into account the heteroscedastic nature of sensor noise and rigid body motions caused by unavoidable vibrations .
The partnership with GE Healthcare started in 1993. In the past few years, it bore on the supervision of CIFRE PhD fellows on the topic of using a multi-modal framework and augmented reality in interventional neuroradiology. A new PhD thesis -Charlotte Delmas- started in April 2013 with the aim to perform 3D reconstruction of tools in interventional neuroradiology. Our goal is to help clinical gesture by providing the physician with a better understanding of the relative positions of the tools and of the pathology.
Lorraine regional project about AR for liver surgery (2015-2018)
The MAGRIT and the MIMESIS teams have been working for several years on the use of augmented reality for deformable organs and especially on liver surgery. The PhD of Jaime Garcia started in October 2015 and is funded by the Région Lorraine. It follows on from our past works and aims at improving the reliability and the robustness of AR-based clinical procedures.
Participant: M.-O. Berger, V. Gaudillière, G. Simon.
This 3-year project is supported by DGA/DGE and led by the SBS-Interactive company. The objective is to develop a prototype for location and object recognition in large-scale industrial environments (factories, ships...), with the aim to enrich the operator's field of view with digital information and media. The main issues concern the size of the environment, the nature of the objects (often non textured, highly specular...) and the presence of repeated patterns. Use cases will be provided by industrial partners such as DCNS and Areva. A class of officer cadets and professors of the Merchant Marine School will also be associated to judge the pedagogical interest of such a tool. A PhD student, Vincent Gaudillière, has been recruited to work on this project and his contract started on 1rst December 2016.
Participant: F. Sur.
Since June 2012, we have been engaged in a collaboration with Pr. Michel Grédiac. The aim is to give a mathematical analysis and to help improving the image processing tools used in experimental mechanics at Institut Pascal.
The TIMEX project (2014-2016) is funded by GDR ISIS ("Appel à projet exploratoire, projet interdisciplinaire"). It aims at investigating image processing tools for enhancing the metrological performances of contactless measurement systems in experimental mechanics.
Participants: R. Anxionnat, M.-O. Berger, E. Kerrien.
The SOFA-InterMedS large-scale Inria initiative is a research-oriented collaboration across several Inria project-teams, international research groups and clinical partners. Its main objective is to leverage specific competences available in each team to further develop the multidisciplinary field of Medical Simulation research. Our action within the initiative takes place in close collaboration with both the MIMESIS team and the Department of diagnostic and therapeutic interventional neuroradiology of Nancy University Hospital. We aim at providing in-vivo models of the patient's organs, and in particular a precise geometric model of the arterial wall. Such a model is used by the MIMESIS team to simulate the coil deployment within an intracranial aneurysm. The associated medical team in Nancy, and in particular our external collaborator René Anxionnat, is in charge of validating our results. For three years, we have also been collaborating with the MIMESIS team about real-time augmentation of deformable organs.
Pierre-Frederic Villard has a "Harvard Affiliate" status through his collaboration with the Harvard Biorobotics Lab led by Professor Robert D. Howe. It follows a one year and a half sabbatical years (2014-2016) that Pierre-Frederic Villard spent in Harvard University in Cambridge (USA) working on heart valve modeling.
M.-O. Berger was member of the program committee of: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2016), International Conference on Pattern recognition (ICPR 2016), International Conference on Robotic and automation (ICRA 2016), International Conference on Information Processing in Computer assited interventions (IPCAI 2017)
Erwan Kerrien was a member of the program committee of MICCAI 2016 and of IPCAI 2017.
F. Sur was a member of the program committe of RFIA 2016 (Reconnaissance des formes et intelligence artificielle).
P.-F. Villard was member of the program committe of MICCAI 2016, International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing 2016, Eurographics Workshop on Visual Computing for Biology and Medicine 2016.
M.-O. Berger was a reviewer for IROS 2016.
The members of the team reviewed articles in Computers in Biology and Medicine (Elsevier), Digital Signal Processing, IEEE Transactions on Biomedical Engineering.
Pierre-Frédéric Villard gave a seminar on "Mitral Valve Biomechanical Model Construction" at the Department of Cardiac Surgery at Boston Children's Hospital, Boston (USA) on the 08/01/2016
Marie-Odile Berger was a member of the HCERES visiting commitee of LTSI (Rennes)
Marie-Odile Berger was president of the AFRIF thesis prize and member of the GDR Robotique thesis prize.
Marie-Odile Berger is president of the Association française pour la reconnaissance et l'interprétation des formes (AFRIF)
Marie-Odile Berger is a member of the Inria evaluation committee.
Gilles Simon is Chargé de Mission Loria to take part in an EIT's KIC (Knowledge and Innovation Communities) proposal in the thematic of manufacturing (KIC Added-value Manufacturing).
The four associate professors of the MAGRIT team actively teach at Université de Lorraine with an annual number of around 200 teaching hours in computer sciences, some of them being accomplished in the field of image processing. Inria researchers have ponctual teaching activities in computer vision and shape recognition mainly in the computer science Master of Nancy and in several Engineering Schools near Nancy (ENSMN Nancy, SUPELEC Metz, ENSG). Our goal is to attract Master students with good skills in applied mathematics towards the field of computer vision. The list of courses given by staff members which are tightly related to image processing and computer vision is detailed below:
Licence: Graphic and haptic rendering, 30h, IUT Saint-Dié des Vosges.
Licence: Image processing, 30h, IUT Saint-Dié des Vosges.
Licence: 3D programming, 30h, IUT Saint-Dié des Vosges.
Game design with Unity3D , 15h, IUT Saint-Dié des Vosges.
Introduction to augmented reality, 6h, IUT Saint-Dié des Vosges.
Master: Signal analysis, 50 h, Université de Lorraine.
Master: Augmented reality, 24 h, Télécom-Nancy, Université de Lorraine.
Master : Introduction to computer vision, 12h, Université de Lorraine.
Master : Shape recognition, 15 h, Université de Lorraine.
Master : Computer vision: foundations and applications, 15 h, Université de Lorraine.
Master : Introduction to image processing, 21 h, École des Mines de Nancy
Master : Image processing for Geosciences, ENSG, 12h.
Master : Introduction to signal processing and applications, 21 h, Ecole des Mines de Nancy
Master : Augmented reality, 24h, M2 IHM Metz
Master : Augmented reality, 3 h, SUPELEC Metz.
In addition, G. Simon was interim director of the CESS (Centre d'Études Supérieures et Scientifiques) in Epinal, a branch of the science faculty of Nancy, from january to april 2016. He is also head of the Licence professionnelle Infographie Paysagère of the faculty of Nancy.
A software, named artEoz, has been co-designed by B. Wrobel-Dautcourt . It aims at supporting students in learning computer programming.
artEoz original design stems from the author's long term experience in teaching object oriented programming. It offers
the students a pedagogical view of the memory state, that is dynamically updated while the user's program runs. artEoz can be freely
downloaded for academic use only. This year, we have proposed an on-line version of the software. Documentation, tutorials, on-line tools and
download are available on the website http://
PhD in progress: Pierre Rolin, Calcul de pose par simulation de points de vue pour la réalité augmentée, octobre 2013, Marie-Odile Berger, Frédéric Sur.
PhD in progress: Charlotte Delmas, Reconstruction 3D des outils chirurgicaux en radiologie interventionnelle, avril 2013, Marie-Odile Berger, Erwan Kerrien.
PhD in progress: Antoine Fond, Introduction de sémantique dans la modélisation urbaine dans un contexte de calcul du point de vue, octobre 2014, Marie-Odile Berger, Gilles Simon.
PhD in progress: Jaime Garcia Guevara, Vers une utilisation clinique de la réalité agmentée pour la chirurgie hépatique, octobre 2015, Marie-Odile Berger, Stéphane Cotin (MIMESIS).
PhD in progress: Raffaella Trivisonne, Image-guided real-time simulation using stochastic filtering, novembre 2015, Erwan Kerrien, Stéphane Cotin (MIMESIS).
PhD in progress: Vincent Gaudillière, Reconnaissance de lieux et d'objets pour la réalité augmentée en milieux complexes, décembre 2016, Marie-Odile Berger, Gilles Simon.
Marie-Odile Berger was external reviewer of the PhD of A. Bauer and A. Agustinos and of the HDR of R. Grompone. Gilles Simon was a member of the PhD committee of Liming Yang.
Members of the team participate on a regular basis, to scientific awareness and mediation actions.
Marie-Odile Berger and Gilles Simon wrote an article about augmented reality in Interstices .
Erwan Kerrien is Chargé de Mission for scientific mediation at
Inria Nancy-Grand Est. As such, he is a member of the steering
committee of "la Maison pour la Science de
Lorraine"
Gilles Simon participated to the “Fête de la science” (unplugged activities in computer science).
Pierre-Frédéric Villard participated to open days and science festival in the IUT of Saint-Dié des Vosges. He presented augmented and virtual reality demos and their link to the high school mathematics program.
Pierre-Frédéric Villard has been involved in the conception of the virtual visit for the Homo Numericus Exhibition (http://