At the end of 2011, a part of Inria team SHACRA moved from Lille to Strasbourg to join the newly created Institute of Image-guided Surgery (IHU) whose main objective is to develop novel clinical technologies at the crossroads of laparoscopic surgery, flexible endoscopy, and interventional radiology. Similar institutes have been created in the past decade around the world with the same global objective: to create a synergy between clinicians and scientists to develop new technologies that can redefine healthcare with a strong emphasis on clinical translation.
The global objective of research team MIMESIS is to create a synergy between clinicians and scientists to develop new technologies that can redefine healthcare, with a strong emphasis on clinical translation. To achieve this goal, we have joined IHU and we collaborate with numerous partners in both academic and private domain.
The scientific objectives of the team MIMESIS are related to this ambitious objective. Over the past years we have developed new approaches supporting advanced simulations in the context of simulation for training. We now propose to focus our research on the use of real-time simulation for per-operative guidance. The underlying objectives include numerical techniques for real-time computation and data-driven simulation dedicated to patient-specific modeling. This last topic is a transversal research theme and raises several open problems, ranging from non-rigid registration to augmented reality.
The core research topics of the MIMESIS project-team essentially aim at improving the realism and fidelity of interactive simulations of medical procedures. This increase in realism makes it possible to envisage new clinical applications, in particular per-operative guidance, that currently rely on imaging techniques, but could greatly benefit from our expertise in real-time numerical simulation.
To reach these objectives we have identified several challenges that lie at the intersection of several scientific domains. Our research projects are currently organized around three main axes:
Real-time Patient-Specific Computational Models
Adaptive Meshing and Advanced Simulation Techniques
Image-driven Simulation
The SOFA framework is used to integrate our various contributions as a means to facilitate validation and technology transfer.
Accounting for the biomechanics and physiology of organs under various stimuli requires employment of biomechanical models. These must describe biophysical phenomena such as soft-tissue deformation, fluid dynamics, electrical propagation, or heat transfer. We aim at simulating the impact of certain therapies (such as cryosurgery or radio-frequency ablation) and representing the behavior of complex organs such as the brain, the liver or the heart.
An important part of our research is dedicated to the development of new accurate models that remain compatible with real-time computation . Such advanced models do not only permit to increase the realism of future training systems, but they act as a bridge toward the development of patient-specific preoperative planning as well as augmented reality tools for the operating room , . Yet, patient-specific planning or per-operative guidance also requires the models to be parametrized with patient-specific biomechanical data. Our objective is the study of hyper-elastic models and their validation for a range of tissues. Preliminary work has been done through two collaborations, one with the biomechanical lab in Lille (LML) , and the biomechanics group from the Icube laboratory in Strasbourg on the development and validation of liver and kidney models .
Another important research topic is related to model reduction through various approaches, such as Proper Generalized Decomposition (PGD) . Similar approaches, such as the use of Krylov spaces, have already been studied in our group recently .
We continue our work on cardiac electro-physiology simulation , with a focus on patient-specific adaptation of the model. We also study a similar problem, related to the modeling of the electrical conduction in the brain, in the context of Deep Brain Stimulation (DBS) , . In this neurosurgical procedure, electrodes are implanted deep into the brain and, connected to a brain pacemaker, send electrical impulses to specific regions. A final objective is to solve optimization problems in the context of heat diffusion. This is a key element of the development of a planning system that can estimate the locations of the electrodes leading to an optimal therapeutic effect .
Most simulations in the field of biomechanics, physiological modeling, or even computer graphics, are performed using finite element approaches. Such simulations require a discretization of the domain of interest, and this discretization is traditionally made of tetrahedral or hexahedral elements. The topology defined by these elements is usually considered as being invariant. However, this is not a realistic assumption if the model is to be employed during a real surgical intervention.
The first objective of this work is to jointly develop advanced topological operations and new finite element approaches that can leverage the use of dynamic topologies. In particular we focus our research on multi-resolution meshes where elements are subdivided in areas where numerical errors need to be kept small , .
Our second objective is to improve, at the numerical level, the efficiency, robustness, and quality of the simulations. To reach these goals, we essentially rely on two main directions: adaptive meshing to allow mesh transformations during a simulation and support cuts, local remeshing or dynamic refinement in areas of interest; and numerical techniques, such as asynchronous solvers, domain decomposition and model order reduction , , , .
We also work on mixed Finite Element Modeling where both tetrahedra and hexahedra can be used at the same time, allowing an ideal compromise between numerical efficiency and mesh adaptation to complex geometries. This research also includes the study of domain decomposition techniques and other coupling techniques for multi-domain multi-physics simulations.
Once the problem, as defined in the previous challenge, has been discretized, we need to solve a large system of linear or nonlinear equations. In both cases, it is necessary to employ numerical solvers repeatedly to construct the solution representing the state of the simulated system. In the past years, we have contributed to this topic through our work on asynchronous preconditioning . We would like to pursue this area of research exploiting the relevant advances in hierarchy-based topologies (e.g. the multi-grid methods). We will also consider advanced non-linear solvers which are necessary for correct resolution of hyper-elastic models and composite models.
Finally, to improve computational times from a programming stand-point, we have started a collaboration with the CAMUS team at Inria. This collaboration aims at using smart code analysis and on-the-fly parallelism to automatically speed-up computation times. In a typical scenario, the modeled organ or tissue is surrounded by its environment represented by other organs, connective tissues or fat. Further, during the intervention, the tissues are manipulated with instruments. Therefore, the interaction will also be an important aspect of our research. We have already developed methods for modeling of advanced interactions between organs, tissues and tools , . We will continue exploiting novel methods such as partial factorization and integrate our approach with other techniques such as augmented Lagrangian.
Image-driven simulation is a recent area of research that has the potential to bridge the gap between medical imaging and clinical routine by adapting pre-operative data to the time of the procedure. Several challenges are related to image-guided therapy but the main issue consists in aligning pre-operative images onto the patient and keep this alignment up-to-date during the procedure. As most procedures deal with soft-tissues, elastic registration techniques are necessary to perform this step.
Recently, registration techniques started to account for soft tissue biomechanics using physically-based methods, yet several limitations still hinder the use of image-guided therapy in clinical routine. First, as registration methods become more complex, their computation times increase, thus lacking responsiveness. Second, techniques used for non-rigid registration or deformable augmented reality only borrow ideas from continuum mechanics but lack some key elements (such as identification of the rest shape, or definition of the boundary conditions). Also, these registration or augmented reality problems are highly dependent on the choice of image modality and require investigating some aspects of computer vision or medical image processing.
However, if we can properly address these challenges, the combination of a real-time simulation and regular acquisitions of image data during the procedure opens up very interesting possibilities by using data assimilation to better adapt the model to the intra-operative data , .
In the area of non-rigid registration and augmented reality, we have already demonstrated the benefit of our physics-based approaches. This was applied in particular to the problem of organ tracking during surgery (Figure ) and led to several key publications , , and awards (best paper ISMAR 2013, second best paper at IPCAI 2014). We continue this work with an emphasis on robustness to uncertainty and outliers in the information extracted in real-time from image data and by improving upon our current computer vision techniques, in particular to guarantee a very accurate initial registration of the pre-operative model onto the per-operative surface patch extracted from monocular or stereo laparoscopic cameras. This work will finally benefit from advances in the challenges listed previously, in particular real-time hyper-elastic models of behavior.
The use of simulation in the context of image-guided therapy can be extended in several other ways. A direction we are addressing is the combined use of simulation and X-ray imaging during interventional radiology procedures. Whether it is for percutaneous procedures or catheterization, the task of the simulation is to provide a short-term (1 to 5 seconds) prediction of the needle or catheter position. Using information extracted from the image, the parameters of the simulation can be assimilated (using methods such as unscented Kalman filters and its reduced-order versions ), so that the simulation progressively matches the real data in order to reduce uncertainties. We have already started to create a flexible framework integrating the real-time soft-tissue simulation and state-of-the-art methods of data assimilation and filtering .
Virtual training prevents medical students from early manipulation of real patients. The development of simulation used for medical training usually requires important computational power, since realistic behaviours are key to deliver a high-fidelity experience to the trainee. Further, the quality of interaction with the simulator (usually via visual and haptic rendering) is also of a paramount importance. All these constraints make the development of training systems time-consuming thus limiting the deployment of virtual simulators in standard medical curriculum.
Beyond training, clinicians ask for innovative tools that can assist them in the pre-operative planning of an intervention. Using the patient information acquired before the operation, physics-based simulations allow to simulate the effect of a therapy with no risk to the patient. The clinicians can thus virtually assess different strategies and select the optimal procedure. Compared to a training simulation, a planning system requires a high accuracy to ensure reliability. Constrained by the time elapsed between the preoperative acquisition and the intervention, the computation must also be efficient.
Besides the surgery training and planning, another major need from clinicians is surgical guidance. While the practician is performing the operation, a guidance system provides enriched visual feedback. This is especially useful with the emergence of minimally invasive surgery (MIS) where the visual information is often strongly limited. It can be used for example to avoid critical area such as vessels or to highlight the position of a tumour during its resection. In the MIS technique, the clinician does not interact with organs directly as in the open surgery, but manipulates instruments inserted through trocars placed in small incisions in the wall of the abdominal cavity. The surgeon can observe these instruments on a display showing a video stream captured by an endoscopic camera inserted through the navel. The main advantage of the method resides in reducing pain and time recovery, in addition to reducing bleeding and risks of infection. However, from a surgical standpoint, the procedure is quite complex since the field of view is considerably reduced and the direct manipulation of organs is not possible.
Prix de thèse 2016 en Génie Biologique et Médical attributed to Rosalie Plantefève for her thesis Augmented Reality and Numerical Simulation for Resection of Hepatic Tumor. The award is attributed by three scientific bodies: IEEE EMBS, Société Française de Génie Biologique et Médical, and Alliance pour le Génie Biologique et Médical. In this context, R. Plantefève was invited to submite a paper to the Journal on Innovation and Research in BioMedical Engineering and the manuscript was accepted for publication .
Runner up for the best poster award at the IEEE International Symposium on Mixed and Augmented Reality 2017 with the poster Deformed Reality: Proof of concept and Preliminary Results . The poster introduced a new paradigm to interactively manipulate objects in a scene in a deformable manner. Using the core principle of augmented reality to estimate a rigid pose over time, the method enables the user to deform the targeted object while it is being rendered with its natural texture, giving the sense of a real-time object editing in the user environment. The results show that the method is capable of opening new ways of not only augmenting the scene but also to interact with it in real by imposing possibly non-linear transformations to selected entities.
The physics-based image and video editing tool Calipso was resumed in Two-minutes papers on YouTube. At the end of 2017, the video has more that 35k views. Calipso is an interactive method for editing images and videos in a physically-coherent manner. The main idea is to perform physics-based manipulations by running a full physics simulation on proxy geometries given by non-rigidly aligned CAD models. Running these simulations allows us to apply new, unseen forces to move or deform selected objects, change physical parameters such as mass or elasticity, or even add entire new objects that interact with the rest of the underlying scene.
Simulation Open Framework Architecture
Keywords: Real time - Multi-physics simulation - Medical applications
Functional Description: SOFA is an Open Source framework primarily targeted at real-time simulation, with an emphasis on medical simulation. It is mostly intended for the research community to help develop new algorithms, but can also be used as an efficient prototyping tool. Based on an advanced software architecture, it allows : the creation of complex and evolving simulations by combining new algorithms with algorithms already included in SOFA, the modification of most parameters of the simulation (deformable behavior, surface representation, solver, constraints, collision algorithm, etc. ) by simply editing an XML file, the building of complex models from simpler ones using a scene-graph description, the efficient simulation of the dynamics of interacting objects using abstract equation solvers, the reuse and easy comparison of a variety of available methods.
Participants: Christian Duriez, François Faure, Hervé Delingette and Stéphane Cotin
Partner: IGG
Contact: Stéphane Cotin
Keywords: Simulation - Linear Systems Solver - Direct solvers - Collision - Numerical simulations
Scientific Description: The SofaPardisoSolver allows for fast direct solution of sparse systems of linear equations, using a decomposition (such as LU, LDL and Cholesky) according to the type of the matrix. Moreover, the wrapper allows for employing a partial factorization which brings a significant improvement when solving augmented systems, usually resulting in problems involving collisions and/or domain decomposition.
Functional Description: The SofaPardisoSolver plugin contains a wrapper allowing for an efficient direct solution of a system of linear equations. It also contains an advanced feature which exploits an algorithm of partial decomposition available in Pardiso. This feature significantly accelerates the computation of Schur complement, typically needed to solve linear complementarity problems (LCP). Example of use: collision and contacts.
Author: Igor Peterlik
Contact: Igor Peterlik
Keywords: Simulation - Realistic rendering - Real-time rendering - Medical imaging - Medical applications
Functional Description: This work allows to emulate a X-ray scan image within the simulation platform SOFA. By defining the position of an emitter and receptor in the 3D space, an image is rendered. A realistic medical image of organs can thus be obtained from surface meshes (triangulated or quadrangulated) in real-time.
Version compatible with SOFA v17.06
Authors: Stéphane Cotin and Frédérick Roy
Contact: Stéphane Cotin
Participants: Y. Adagolodjo, R. Trivisonne, H. Courtecuisse, S. Cotin
A contribution focusing on intra-operative organ pose estimation was publihsed at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017) . A novel method for semi-automatic registration of 3D deformable models using 2D shape outlines (silhouettes) extracted from a monocular camera view was introduced. The proposed framework is based on the combination of a biomechanical model of the organ with a set of projective constraints influencing the deformation of the model. To enforce convergence towards a global minimum for this ill-posed problem we interactively provide a rough (rigid) estimation of the pose. We show that our approach allows for the estimation of the non-rigid 3D pose while relying only on 2D information. The method is evaluated experimentally on a soft silicone gel model of a liver, as well as on real surgical data, providing augmented reality of the liver and the kidney using a monocular laparoscopic camera. Results show that the final elastic registration can be obtained in just a few seconds, thus remaining compatible with clinical constraints. We also evaluate the sensitivity of our approach according to both the initial alignment of the model and the silhouette length and shape.
Participants: N. Haouchine, I. Peterlik, S. Cotin
A novel method was proposed in the context of image-driven stochastic simulation employed in the intra-operative navigation . In the proposed approach, the boundary conditions are modeled as stochastic parameters. The method employs the reduced-order unscented Kalman filter to transform in real-time the probability distributions of the parameters, given observations extracted from intra-operative images. The method is evaluated using synthetic, phantom and real data acquired in vivo on a porcine liver. A quantitative assessment is presented and it is shown that the method significantly increases the predictive power of the biomechanical model employed by a framework implemented the augmented reality for surgical navigation.
Participants: A. Mendizabal, C. Paulus, R. Bessard-Duparc, I. Peterlik, S. Cotin
A method based on face-based smoothed finite element method was proposed and applied in the context of modeling of brain shift in . This numerical technique has been introduced recently to overcome the overly stiff behavior of the standard FEM and to improve the solution accuracy and the convergence rate in solid mechanics problems. In this paper, a face-based smoothed finite element method (FS-FEM) using 4-node tetrahedral elements is presented. We show that in some cases, the method allows for reducing the number of degrees of freedom, while preserving the accuracy of the discretization. The method is evaluated on a simulation of a cantilever beam loaded at the free end and on a simulation of a 3D cube under traction and compression forces. Further, it is applied to the simulation of the brain shift and of the kidney's deformation. The results demonstrate that the method outperforms the standard FEM in a bending scenario and that has similar accuracy as the standard FEM in the simulations of brain shift and kidney deformation.
Participants: C. Paulus, S. Cotin
Although the finite element method is widely used as a numerical approach in this area, it is often hindered by the need for an optimal meshing of the domain of interest. The derivation of meshes from imaging modalities such as CT or MRI can be cumbersome and time-consuming. In our contribution , we employed the Immersed Boundary Method (IBM) to bridge the gap between these imaging modalities and the fast simulation of soft tissue deformation on complex shapes represented by a surface mesh directly retrieved from binary images. A high resolution surface, that can be obtained from binary images using a marching cubes approach, is embedded into a hexahedral simulation grid. The details of the surface mesh are properly taken into account in the hexahedral mesh by adapting the Mirtich integration method. In addition to not requiring a dedicated meshing approach, our method results in higher accuracy for less degrees of freedom when compared to other element types. Examples on brain deformation demonstrate the potential of our method.
Participants: H. Courtecuisse, S. Cotin
A contribution presents the first real-time a posteriori error-driven adaptive finite element approach for real-time simulation and demonstrates the method on a needle insertion problem.
We use corotational elasticity and a frictional needle–tissue interaction model. The problem is solved using finite elements and the refinement strategy relies upon a hexahedron-based finite element method, combined with a posteriori error estimation driven local h-refinement, for simulating soft tissue deformation. We proposed to control the local and global error level in the mechanical fields (e.g. displacement or stresses) during the simulation. We show the convergence of the algorithm on academic examples, and demonstrate its practical usability on a percutaneous procedure involving needle insertion in a liver. For the latter case, we compare the force displacement curves obtained from the proposed adaptive algorithm with that obtained from a uniform refinement approach. Error control guarantees that a tolerable error level is not exceeded during the simulations. Local mesh refinement accelerates simulations. The work provides a first step to discriminate between discretization error and modeling error by providing a robust quantification of discretization error during simulations.
Participants: R. Plantefève, I. Peterlik, S. Cotin
Different aspects of model-based registration in the context of surgical navigation employing the augmented reality were analyzed in an invited contribution published in the context of the attributed Prix de thèse de former Ph.D. student Rosalie Plantefève. Preoperative images such as computed tomography scans or magnetic resonance imaging contain lots of valuable information that are not easily available for surgeons during an operation. To help the clinicians better target the structures of interest during an intervention, many registration methods that align preoperative images onto the intra-operative view of the organs have been developed. For important organ deformation, biomechanical model-based registration has proven to be a method of choice. Using an existing model-based registration algorithm for laparoscopic liver surgery we investigated the influence of the heterogeneity of the liver on the registration result. It was found that the use of an heterogeneous model does not improve significantly the registration result but increases the computation time necessary to perform the registration.
Participants: I. Peterlik
A contribution Registration of Cell Nuclei in 2D Live Cell Microscopy was published in a collaboration with Centre of Biomedical Image Analysis at Masaryk University, Czech Republic . The analysis of the pure motion of sub-nuclear structures without influence of the cell nucleus motion and deformation is essential in live cell imaging. We proposed a 2D contour-based image registration approach for compensation of nucleus motion and deformation in fluorescence microscopy time-lapse sequences. The proposed approach extends our previous approach which uses a static elasticity model to register cell images. Compared to that scheme, the new approach employs a dynamic elasticity model for forward simulation of nucleus motion and deformation based on the motion of its contours. The contour matching process is embedded as a constraint into the system of equations describing the elastic behavior of the nucleus. This results in better performance in terms of the registration accuracy. Our approach was successfully applied to real live cell microscopy image sequences of different types of cells including image data that was specifically designed and acquired for evaluation of cell image registration methods.
Participants: R. Plantefève, I. Peterlik
An algorithm of an automatic skeletonization of vascularization based on Dijkstra minimum-cost spanning tree was published in . The result is an extension of an existing graph-based method where the vascular topology is constructed by computation of shortest paths in a minimum-cost spanning tree obtained from binary mask of the vascularization. We suppose that the binary mask is extracted from a 3D CT image using automatic segmentation and thus suffers from important artifacts and noise. When compared to the original algorithm, the proposed method (i) employs a new weighting measure which results in smoothing of extracted topology and (ii) introduces a set of tests based on various geometric criteria which are executed in order to detect and remove spurious branches. The method is evaluated on vascular trees extracted from abdominal contrast-enhanced CT scans and MR images. The method is quantitatively compared to the original version of the algorithm showing the importance of proposed modifications. Since the branch testing depends on parameters, the para-metric study of the proposed method is presented in order to identify the optimal parametrization.
Participants: N. Haouchine, S. Cotin
A method of template-based 3D recovery of elastic shapes using Lagrange multiplied was presented at a top computed-vision conference . By exploiting the object's elasticity, in contrast to isometric methods that use inextensibility constraints, a large range of deformations can be handled. Our method is expressed as a saddle point problem using Lagrangian multipliers resulting in a linear system which unifies both mechanical and optical constraints and integrates Dirichlet boundary conditions, whether they are fixed or free. We experimentally show that no prior knowledge on material properties is needed, which exhibit the generic usability of our method with elastic and inelastic objects with different kinds of materials. Comparison with existing techniques are conducted on synthetic and real elastic objects with strains ranging from 25% to 130% resulting to low errors.
Participants: N. Haouchine, F. Roy, S. Cotin
DejaVu, a novel surgical simulation approach for intra-operative surgical gesture rehearsal was published in in collaboration with UCL London. With DejaVu we aim at bridging the gap between pre-operative surgical simulation and crucial but not yet robust intra-operative surgical augmented reality. By exploiting intra-operative images we produce a simulation that faithfully matches the actual procedure without visual discrepancies and with an underlying physical modeling that performs real-time deformation of organs and surrounding tissues, surgeons can interact with the targeted organs through grasping, pulling or cutting to immediately rehearse their next gesture. We present results on different in vivo surgical procedures and demonstrate the feasibility of practical use of our system.
MIMESIS has active bilateral collaborations with following industrial partners:
InSimo: A startup providing biomedical simulation software which are able to reproduce the behavior of organs, tissues and surgical procedures in a realistic and interactive way. Created in January 2013 as a spin-off forces by former members of team SHACRA (the predecessor of MIMESIS). Currently, we collaborate on simulations of eye surgery as well as on preparation of projects aiming at validation of algorithms and codes of simulation framework SOFA.
Altran: A global leader in innovation and high-tech engineering consulting, Altran accompanies supports its clients in the creation and development of their new products and services. We have a common history of successful collaboration via CIFRE Ph.D. thesis of Rosalie Plantefève. A new CIFRE Ph.D. will start on 01/01/2018 focusing on fusion of multisensor data in the context of intra-operative navigation of catheters.
Siemens: A global leader in healthcare industry. Via IHU, we collaborate with Siemens in the context of the IHU project CIOS Alpha Fusion dealing with augmentation of the intra-operative image provided by a fluoroscopic imaging modality with pre-operative data.
Renumics: A German startup focusing on automation of computer aided engineering (CAE) using artificial intelligence in general and machine learning techniques in particular. In close collaboration with SOFA Consortium, MIMESIS is involved in preparation of projects aiming at validation of SOFA.
Naviworks: A South Korean company specialized in ICT convergence simulation/IoT smart controlling. We collaborate on simulation and visualization in the context of interventional radiology.
The Institute of Image-Guided Surgery of Strasbourg develops innovative surgery to deliver personalized patient care, combining the most advanced minimally invasive techniques and the latest medical imaging methods.
Project CIOS Alpha Fusion funded by IHU Strasbourg has started at the beginning of 2017. The goal of the project is to develop a solution for real-time, accurate, image fusion between 3D anatomical data and 2D X-ray images. This requires to spatially align these two imaging datasets with each other, knowing that a deformation has occurred between the 2 acquisitions. We consider two different cases, of increasing scientific complexity: static image fusion using 2 fluoroscopic images taken at 2 different angles, and dynamic image fusion using a single fluoroscopic image. We also consider two additional scenarios: in the first one, a 3D image or a 3D model has been obtained from a preoperative CTA or MRA while in the second scenario it has been acquired using an intra-operative contrast-enhanced CBCT. In the second case, tissue deformation between the 2D and 3D data is significantly reduced.
The project team involves scientists from the MIMESIS team at Inria, engineers from Siemens as industrial partner, and clinicians from the NHC hospital and IHU.
At the regional level, the MIMESIS team collaborates with
ICube Automatique Vision et Robotique (AVR): We have been collaborating with the medical robotics team on percutaneous procedures, in particular robotized needle insertion (with Prof. Bernard Bayle), and needle tracking in medical images (with Elodie Breton). We are also collaborating with Jonathan Vappou on elastography.
ICube Informatique Géométrique et Graphqiue (IGG): MIMESIS joined the IGG team and develops collaboration in the domain of dynamic topologies, mainly through the use of the CGoGN framework. CGoGN is a C++ library for the manipulation of meshes. It implements combinatorial maps and their multiresolution extensions and has been used in various high level application like the simulation of crowds of autonomous agents and the simulation of cuts, tears and fractures in the context of surgical simulations.
Nouvel Hôpital Civil, Strasbourg: since 2014 we have been working with Prof. David Gaucher, an ophthalmologist and expert in retina surgery. This led to the submission of the ANR project RESET which started in March 2015. We also collaborate with Prof. Patrick Pessaux, a surgeon who helps us in the context of the SOFA-OR project.
Team MIMESIS received a support for the development of the SOFA framework through two ADTs:
DynMesh (Sep 2015 – Aug 2017): The objectives of the ADT was the coupling of SOFA, the physical simulation platform supported by Inria, and CGoGN, the mesh management library developed within the ICube lab at Strasbourg. The goal is to extend the physical engine SOFA with the topological kernel of CGoGN that supports a wide variety of mesh and many local remeshing operations. The coupling of both software libraries will provide users of physical engines with new tools for the development of simulations involving topological changes like cutting, fracturing, adaptation of the resolution or improving contact management or collision detection. The impacts are numerous and will be operated directly within the MIMESIS Team, with our partners or through the establishment of new collaborations.
ASNAP (Accélération des Simulations Numériques pour l'Assistance Peropératoire, Jan 2017 – Dec 2018). We are partners of ADT ASNAP with principal investigator being Inria team CAMUS. The goal of the project is a significant acceleration of physics-based simulations developed by MIMESIS. The technologies such as Apollo, XFOR, ORWL, developed by team CAMUS are used to optimize the execution of different components of framework SOFA, taking into account the possibilities provided by modern CPUs and GPGPUs. Since team CAMUS is also located in Strasbourg, the project benefits from the geographical location: an engineer Maxim Mogé was recruited, starting from 01/01/2017 and he shares his time between the two teams.
MIMESIS participates in the following ANR projects:
RESET: This project started in March 2015 and will end in May 2017. Its objective is to develop a high-fidelity training system for retinal surgery. Retinal surgery is an increasingly performed procedure for the treatment of a wide spectrum of retinal pathologies. Yet, as most micro-surgical techniques, it requires long training periods before being mastered. This simulator is built upon our scientific expertise in the field of real-time simulation, and our success story for technology transfer in the field of cataract surgery simulation (MSICS simulation developed for the HelpMeSee foundation).
Coordinator: MIMESIS
Partners: the InSimo company, the AVR team of the ICube lab.
EVEREST: The overall objective of the EVEREST project is thus to bring a leap forward in factorization of large sparse tensors in order to improve the accessibility, completeness and reliability of real-world KBs. This line of research could have a huge impact in industry (Semantic Web, biomedical applications, etc.). For that reason, Xerox Research Center Europe is supporting this project and will supply data, provide expertise and ease industrial transfer. This proposal is also consistent with the long-term research direction of its principal partner, Heudiasyc, since it contributes in several aspects of the 10 years LabEx program on Technological Systems of Systems started in 2011.
Coordinator: IHU Strasbourg
Partners: Inria, IRCAD, University of Strasbourg, Siemens Healthcare, Karl Storz GmbH., University of Twente
MIMESIS is closely connected to the SOFA Consortium, created by Inria in November 2015 with the objective to support the SOFA community and encourage contributions from new SOFA users. The Consortium should also be a way to better answer to the needs of academic or industrial partners. MIMESIS actively participates at the development of SOFA and contributed to the evolution of the framework. Moreover, MIMESIS also participates in an initiative aiming at verification and validation of codes and algorithms of SOFA.
Further, MIMESIS actively collaborates with the following Inria teams:
MAGRIT: The team at Inria Grand Est focuses on research in computer vision and is also actively involved in computer-based solutions for the planning or the simulation of interventional radiology procedures, with a strong collaboration with the CHU in Nancy. We collaborate with MAGRIT in the area of interventional radiology and augmented reality. Currently, two PhD thesis are co-supervised by researcher from Magrit: the PhD thesis of Jaime Garcia Guevara and Raffaella Trivisonne.
CAMUS: The team focuses on developing, adapting and extending automatic parallelizing and optimizing techniques, as well as proof and certification methods, for the efficient use of current and future multi-core processors. Currently, we collaborate with team CAMUS on parallelization of framework SOFA in ADT project ASNAP.
DEFROST: The team conducts research in soft robotics. We continue mutual interaction with DEFROST mainly in the context of contact modeling.
At the national level, the MIMESIS team collaborates with:
The TIMC laboratory(Techniques de l’Ingénierie Médicale et de la Complexité) in Grenoble: this large research group has a strong background in computer-aided surgery, medical imaging, registration, statistical and bio-mechanical modeling. We have regular interactions with various members of this group. We are collaborating with Yohan Payan (DR CNRS) on the modeling and simulation of the brain shift. A common PhD thesis started on that topic in late 2014. Other areas of interest are in the field of advanced soft tissue modeling and computer aided surgery.
The LML laboratory(Laboratoire de Mécanique de Lille): a French research laboratory (UMR CNRS 8107) part of the Carnot institute ARTS. With more than two hundred researchers, LML focuses on the following research areas: mechanical reliability and Tribology, fluid mechanics, civil engineering and soil mechanics.
Hôpital Paul-Brousse: a hospital in South Paris. We collaborate with Centre Hépato-Biliaire via the co-supervision of the Ph.D. thesis of Nicolas Golse, MD, who is a surgeon at the center.
Program: H2020, Innovative Training Network, MSCA
Project acronym: HiPerNav
Project title: High performance soft tissue navigation
Coordinator: Oslo University Hospital
Other partners: SINTEF Trondheim, University of Bern
Abstract: HiPerNav is an Innovative Training Network (ITN) funded through a Marie Skłodowska-Curie grant. There will be 14 fully funded and 2 partially funded PhD’s working on the project. The project aims to improve soft tissue navigation through research and development, to improve several bottleneck areas:
Creating effective pre-operative model(s) and planning
Faster and more accurate intra-operative model updates
Faster and more accurate model-to-patient registration
More intuitive user-interaction and effective work flow
Usage of high performance computing (e.g. GPU)
University of Twente: Thanks to our clinical partner IHU, we collaborate with Prof. Stefano Stramigioli, head of a group at Robotics and Mechatronics laboratory.
Faculty of Informatics, Masaryk University, Czech Republic: We collaborate on simulation of living cells in fluorescent microscopy. The collaboration resulted in a presentation at an international conference and a journal paper .
Team Legato, University of Luxembourg: we have an active collaboration with Prof. Stéphane Bordas on error estimation in real-time simulations of deformable objects. The collaboration resulted in a common publication .
The MIMESIS team actively collaborates with following international partners:
CIMIT & Harvard Medical School, Boston, USA: We collaborate on a project REBOASim in the contect of interventional radiology, , in particular the design and development of a hardware interface for tracking catheters and guidewires. The common DoD project REBOASim focuses on development of the physics-based models for catheter and guidewire motion, blood flow and graphical rendering towards a novel simulator for REBOA that will include physical vascular access, simulated passage of the IR instruments into the aorta with accompanying training/educational content, device withdrawal and closure: Duration of the project: Feb 2017 – Feb 2019.
From Feb 2017 to July 2017, Prof. Adam Wittek joined team MIMESIS as a visiting scientist. Prof. Wittek is with Intelligent Systems for Medicine Laboratory, School of Mechanical and Chemical Engineering at the University of Western Australia, Perth. His research focuses on patient-specific biomechanical modeling and he has published an important number of high-quality publications on this topic with more than 2,000 citations.
During his stay, Prof. Wittek provided his highly valuable expertise in various domains of patient-specific simulations and advanced techniques of modeling of deformations in soft tissues such as meshless methods. He was also involved in projects related to insertions of flexible needles into soft tissues.
From Jul 2017 to Dec 2017, Vincent Magnoux, a Canadian PhD student from École polytechnique de Montréal, joined MIMESIS as an international intern. During his stay, he has worked on implementing and validating a meshless method for computing organ deformation. This work also involved exploring methods to accelerate these computations on multi-core systems for an interactive simulation.
David Cazier contributed to the organization of the Annual workshop of the Animation & Simulation group of the GDR IGRV of the CNRS in Strasbourg.
Stephane Cotin provided reviews for: Int. Conf. on Information Processing in Computer-Assisted Interventions, Workshop on Virtual Reality Interaction and Physical Simulation
Igor Peterlik provided reviews for: Int. Conf. of Medical Image Computing and Computer Assisted Intervention Society
Antoine Petit provided reviews for: International Conference on Robotics and Automation
Stephane Cotin provided reviews for: International Journal of Computer Assisted Radiology and Surgery
David Cazier provided reviews for: Computer-Aided Design, Visual Computer, Computer & Graphics, Int. Journal on Virtual Reality
Igor Peterlik provided reviews for: Computer and Graphics
Antoine Petit provided reviews for: International Journal Of Robotics Research, Robotics and Automation Letters
Christoph Paulus provided reviews for: MDPI Journal Symmetry
Lionel Untereiner provided reviews for: Special Issue on Parallel and Distributed Algorithms of Concurrency and Computation
Keynote lecture by S. Cotin at 10th Medical Korea conference (Seoul, South Korea)
Invited lecture by S. Cotin FMTS conference (Strasbourg, France)
Invited lecture by S. Cotin B.E.S.T. symposium (Strasbourg, France)
Invited talk by S. Cotin at Fraunhofer MEVIS lab (Bremen, Germany)
Invited lecture by S. Cotin at European Computer-Assisted Liver Surgery Society (Mainz, Germany)
Invited lecture by S. Cotin at 127th annual meeting of the French Ophthalmology Association (Paris, France)
Invited lecture by S. Cotin at the French Academy of Surgery (Paris, France)
Igor Peterlik has been providing a scientific expertise at Masaryk University, Czech Republic as a consultant and co-investigator of a project unded by Grant Agency of the Czech Republic: Development of Reliable Methods for Automated Quantitative Characterization of Cell Motility in Fluorescence Microscopy.
David Cazier is a member and local coordinator for a CITEPH project Paleo GTM: A Paleo Geological and Topological Modeler. Subject and expected contributions: multiresolution meshing and visualization for handling of massive geological data.
The project started in Sep 2017 (duration 2 years) and involves following partners:
GEOSIRIS SAS (StartUp)
Laboratoire ICUBE, UMR 7357 (Université de Strasbourg)
Laboratoire XLIM, UMR 7352 (Université de Poitiers)
Laboratoire LSIS, UMR 7296 (Université d’Aix-Marseille)
Master: Igor Peterlik, Modélisation des systèmes vivants, 17h, M2, University of Strasbourg
Master: Igor Peterlik, Visualisation des données et simulation, 10h, M1, University of Strasbourg
Master: Hadrien Courtecuisse, Real time simulation, 30h, M2, University of Strasbourg
Master: Hadrien Courtecuisse, Visualisation des données et simulation, 10h, M2, University of Strasbourg
Master: Hadrien Courtecuisse, Visualisation des données et simulation, 10h, M1, University of Strasbourg
Licence: David Cazier, Web technologies and programming, 96h, L3, University of Strasbourg
PhD: Christoph Paulus, Modeling and real-time simulation of topological changes in soft tissue, University of Strasbourg, 03/04/2017
PhD: Fanny Morin, Non linear simulation for intra-operative guidance for neurosurgery, Université Grenoble Alpes, 05/10/2017
PhD in progress: Jaime Garcia Guevara, Augmented ultrasound imaging for hepatic surgery, 01/09/2015, supervised by Stéphane Cotin, Marie-Odile Berger
PhD in progress: Raffaella Trivisonne, Computer-aided vascular interventions, 01/09/2015, Stéphane Cotin, Erwan Kerrien
PhD in progress: Yinoussa Adagolodjo, Coupling between robotics and medical simulation for automated procedures, 01/02/2015, supervised by Hadrien Courtecuisse
PhD in progress: Nicolas Golse, Navigation using the augmented reality during hepatic surgery, 01/09/2016, supervised by Stéphane Cotin
PhD in progress: Lukáš Ručka, Validation and verification of soft tissue models, 01/09/2016, co-supervised by Igor Peterlik, supervised by Prof. Ludek Matyska at Masaryk University, Czech Republic
PhD in progress: Sergei Nikolaev, Characterization of boundary conditions for biomechanical modeling of liver, 01/05/2017, supervised by Stéphane Cotin, co-supervised by Igor Peterlik and Hadrien Courtecuisse
PhD in progress: Jean-Nicolas Brunet, Characterization of boundary conditions for biomechanical modeling of liver, 01/09/2017, supervised by Stéphane Cotin
PhD in progress: Andrea Mendizabal, Numerical simulation of soft tissues and machine learning, 01/09/2017, supervised by Stéphane Cotin
Stéphane Cotin was a member of jury of HDR of Christian Herlin (MD): Imagerie et simulation pour la chirurgie plastique et reconstructrice. Université de Montpellier. Nov 2017
Demonstration at 50 Years Inria on Nov 10 in Paris, attended by Mr. Mounir Mahjoubi, secrétaire d'état auprès du premier ministre, chargé du numérique.