Section: New Results
Physics-based Deep Neural Network for Augmented Reality
Participants : Jean-Nicolas Brunet, Andrea Mendizabal, Antoine Petit, Nicolas Golse, Eric Vibert, Stéphane Cotin.
We propose an approach combining a finite element method and a deep neural network to learn complex elastic deformations with the objective of providing augmented reality during hepatic surgery. Derived from the U-Net architecture, our network is built entirely from physics-based simulations of a preoperative segmentation of the organ (see figure 5). These simulations are performed using an immersed-boundary method, which offers several numerical and practical benefits, such as not requiring boundary-conforming volume elements. We perform a quantitative assessment of the method using synthetic and ex vivo patient data. Results show that the network is capable of solving the deformed state of the organ using only a sparse partial surface displacement data and achieve similar accuracy as a FEM solution, while being about 100x faster. When applied to an ex vivo liver example, we achieve the registration in only with a mean target registration error (TRE) of . This results were presented at MICCAI 2019 .