Computational vision is one of the most challenging research domains in engineering sciences. The aim is to reproduce human visual perception through intelligent processing of visual data. The application domains span from computer aided diagnosis to industrial automation & robotics. The most common mathematical formulation to address such a challenge is through mathematical modeling. In such a context, first the solution of the desired vision task is expressed in the form of a parameterized mathematical model. Given such a model, the next task consists of associating the model parameters with the available observations, which is often called the model-to-data association. The aim of this task is to determine the impact of a parameter choice to the observations and eventually maximize/minimize the adequacy of these parameters with the visual observations. In simple words, the better the solution is, the better it will be able to express and fit the data. This is often achieved through the definition of an objective function on the parametric space of the model. Last, but not least given the definition of the objective function, visual perception is addressed through its optimization with respect to the model parameters. To summarize, computation visual perception involves three aspects, a task-specific definition of a parametric model, a data-specific association of this model with the available observations and last the optimization of the model parameters given the objective and the observations.

Such a chain processing inherits important shortcomings. The curse of dimensionality is often used to express the importance of the model complexity. In simple words, the higher the complexity of the model is, the better its expressive power will be with counter effect the increase of the difficulty of the inference process. Non-linearity is another issue to be addressed which simply states that the association between the model and the data is a (highly) non-linear function and therefore direct inference is almost infeasible. The impact of this aspect is enforced from the curse of non-convexity that characterizes the objective function. Often it lives in high-dimensional spaces and is ill posed making exact inference problematic (in many cases not possible) and computationally expensive. Last, but not least modularity and scalability is another important concern to be addressed in the context of computational vision. The use of task-specific modeling and algorithmic solutions make their portability infeasible and therefore transfer of knowledge from one task to another is not straightforward while the methods do not always scale well with respect either to the dimensionality of the representation or the data.

GALEN aims at proposing innovative techniques towards automatic structuring, interpretation and longitudinal modeling of visual data. In order to address these fundamental problems of computational perception, GALEN investigates the use of discrete models of varying complexity. These methods exhibit an important number of strengths such as their ability to be modular with respect to the input measurements (clinical data), the nature of the model (certain constraints are imposed from computational perspective in terms of the level of interactions), and the model-to-data association while being computational efficient.

A general framework for the fundamental problems of image segmentation, object recognition and scene analysis is the interpretation of an image in terms of a set of symbols and relations among them. Abstractly stated, image interpretation amounts to mapping an observed image, *optimally explain the underlying image*, as measured by a scoring function

Applying this framework requires (a) identifying which symbols and relations to use (b) learning a scoring function

One of the main themes of our work is the development of methods that jointly address (a,b,c) in a shape-grouping framework in order to reliably extract, describe, model and detect shape information from natural and medical images. A principal motivation for using a shape-based framework is the understanding that shape- and more generally, grouping- based representations can go all the way from image features to objects. Regarding aspect (a), image representation, we cater for the extraction of image features that respect the shape properties of image structures. Such features are typically constructed to be purely geometric (e.g. boundaries, symmetry axes, image segments), or appearance-based, such as image descriptors. The use of machine learning has been shown to facilitate the robust and efficient extraction of such features, while the grouping of local evidence is known to be necessary to disambiguate the potentially noisy local measurements. In our research we have worked on improving feature extraction, proposing novel blends of invariant geometric- and appearance- based features, as well as grouping algorithms that allow for the efficient construction of optimal assemblies of local features.

Regarding aspect (b) we have worked on learning scoring functions for detection with deformable models that can exploit the developed low-level representations, while also being amenable to efficient optimization. Our works in this direction build on the graph-based framework to construct models that reflect the shape properties of the structure being modeled. We have used discriminative learning to exploit boundary- and symmetry-based representations for the construction of hierarchical models for shape detection, while for medical images we have developed methods for the end-to-end discriminative training of deformable contour models that combine low-level descriptors with contour-based organ boundary representations.

Regarding aspect (c) we have developed algorithms which implement top-down/bottom-up computation both in deterministic and stochastic optimization. The main idea is that ‘bottom-up’, image-based guidance is necessary for efficient detection, while ‘top-down’, object-based knowledge can disambiguate and help reliably interpret a given image; a combination of both modes of operation is necessary to combine accuracy with efficiency. In particular we have developed novel techniques for object detection that employ combinatorial optimization tools (A* and Branch-and-Bound) to tame the combinatorial complexity, achieving a best-case performance that is logarithmic in the number of pixels.

In the long run we aim at scaling up shape-based methods to 3D detection and pose estimation and large-scale object detection. One aspect which seems central to this is the development of appropriate mid-level representations. This is a problem that has received increased interest lately in the 2D case and is relatively mature, but in 3D it has been pursued primarily through ad-hoc schemes. We anticipate that questions pertaining to part sharing in 3D will be addressed most successfully by relying on explicit 3D representations. On the one hand depth sensors, such as Microsoft’s Kinect, are now cheap enough to bring surface modeling and matching into the mainstream of computer vision - so these advances may be directly exploitable at test time for detection. On the other hand, even if we do not use depth information at test time, having 3D information can simplify the modeling task during training. In on-going work with collaborators we have started exploring combinations of such aspects, namely (i) the use of surface analysis tools to match surfaces from depth sensors (ii) using branch-and-bound for efficient inference in 3D space and (iii) groupwise-registration to build statistical 3D surface models. In the coming years we intend to pursue a tighter integration of these different directions for scalable 3D object recognition.

The foundation of statistical inference is to learn a function that minimizes the expected loss of a prediction with respect to some unknown distribution

where

A key problem is that the distribution

which makes an implicit assumption that the training samples

where

Equation Eq. is very well studied in classical statistics for the case that the output,

Many tasks in artificial intelligence are solved by building a model whose parameters encode the prior domain knowledge and the likelihood of the observed data.
In order to use such models in practice, we need to estimate its parameters automatically using training data. The most prevalent paradigm of parameter estimation is supervised learning, which requires the collection of the inputs

In order to address the deficiencies of supervised learning, researchers have started to focus on the problem of parameter estimation with data that contains hidden
variables. The hidden variables model the missing information in the annotations. Obtaining such data is practically more feasible: image-level labels (`contains car',`does not contain person') instead of tight bounding boxes; partial segmentation of medical images. Formally, the parameters **w** of the model are learned by minimizing the following objective:

Here,

Previous attempts at minimizing the above objective function treat all the training samples equally. This is in stark contrast to how a child learns: first focus on easy samples (`learn to add two natural numbers') before moving on to more complex samples (`learn to add two complex numbers'). In our work, we capture this intuition using a novel, iterative algorithm called self-paced learning (spl). At an iteration

Here, samples with

Here, **w**. The use of a more general loss function will allow us to better exploit the freely available data with missing information. For example, consider the case where

A wide variety of tasks in medical image analysis can be formulated as discrete labeling problems. In very simple terms, a discrete optimization problem can be stated as follows: we are given a discrete set of variables *singleton* potential function *pairwise* potential function

Our goal is then to choose a labeling which will allow us to pay the smallest total price. In other words, based on what we have mentioned above, we want to choose a labeling that minimizes the sum of all the MRF potentials, or equivalently the MRF energy. This amounts to solving the following optimization problem:

The use of such a model can describe a number of challenging problems in medical image analysis. However these simplistic models can only account for simple interactions between variables, a rather constrained scenario for high-level medical imaging perception tasks. One can augment the expression power of this model through higher order interactions between variables, or a number of cliques

where

**Paticipants:** Eugene Belilovsky, Matthew Blaschko
Collaboration with Inria Parietal: Gael Varoquaux

Proposed a new algorithm for determining the differences in functional brain connectivity between two populations. The aim of our work was to leverage assumptions and show a method that can efficiently provide significance results in the form of (p-values). We demonstrated that our approach works well in practice and simulation and can provide faithful p-values on complicated fMRI data.

**Paticipants:** Evgenios Kornaropoulos, Evangelia Zacharaki, Nikos Paragios

The use of Diffusion Weighted MR Imaging (DWI) is investigated as an alternative tool to radiologists for tumor detection, tumor characterization, distinguishing tumor tissue from non-tumor tissue, and monitoring and predicting treatment response. In collaboration with Hôpitaux Universitaires Henri-Mondor in Paris, France and Chang Gung Memorial Hospital – Linkou in Taipei, Taiwan we investigate the use of model-based methods of 3D image registration, clustering and segmentation towards the development of a framework for automatic interpretation of images, and in particular extraction of meaningful biomarkers in aggressive lymphomas . In we combine deformable group-wise registration with a physiological model in order to better estimate diffusion in Diffusion-Weighted MRI, whereas in we explicitly model the diffusion coefficients by a high-order MRF-based joint deformable registration and labeling scheme.

**Paticipants:** Evangelia Zacharaki, Nikos Paragios
(in collaboration with D. Vlachakis, University of Patras, Greece)

The massive expansion of the worldwide Protein Data Bank (PDB) provides new opportunities for computational approaches which can learn from available data and extrapolate the knowledge into new coming instances. The aim of our work in was to exploit experimentally acquired structural information of enzymes through machine learning techniques in order to produce models that predict enzymatic function.

**Paticipants:** Guillaume Chassagnon, Evangelia Zacharaki, Nikos Paragios

Diagnosis and staging of chronic lung diseases is a major challenge for both patient care and approval of new treatments. Among imaging techniques, computed tomography (CT) is the gold standard for in vivo morphological assessment of lung parenchyma currently offering the highest spatial resolution in chronic lung diseases. Although CT is widely used its optimal use in clinical practice and as an endpoint in clinical trials remains controversial. Our goal is to develop quantitative imaging biomarkers that allow (i) severity assessment (based on the correlation to functional and clinical data) and (ii) monitoring the disease progression. In the current analysis we focus on scleroderma and cystic fibrosis as models for restrictive and obstructive lung disease, respectively. Two different approaches are investigated: disease assessment by histogram or texture analysis and assessment of the regional lung elasticity through deformable registration. This work is in collaboration with the Department of Radiology, Cochin Hospital, Paris.

**Paticipants:** Enzo Ferrante, Nikos Paragios, Iasonas Kokkinos

New algorithms to perform co-segmentation and co-registration of subcortical brain structures on MRI images were investigated in collaboration with Ecole Polytechnique de Montreal and the Sainte-Justine Hospital Research Center from Montreal . Brain subcortical structures are involved in different neurodegenerative and neuropsychiatric disorders, including schizophrenia, Alzheimers disease, attention deficit, and subtypes of epilepsy. Segmenting these parts of the brain enables a physician to extract indicators, facilitating their quantitative analysis and characterization. We are investigating how estimated maps of semantic labels (obtained using machine learning techniques) can be used as a surrogate for unlabelled data. We are exploring how to combine them with multi-population deformable registration to improve both alignment and segmentation of these challenging brain structures.

Wacha Bounliphone and Eugène Belilovsky received the Université Paris-Saclay STIC Doctoral School Best Scientific Contribution Award

Eugène Belilovsky received the MITACS-Inria Globalink Award

Prof. Iasonas Kokkinos was invited as keynote speaker in Astronomical Data Analysis Summer School, Chania, Greece, May 2016

Prof. Iasonas Kokkinos was invited as keynote speaker in Local features workshop, held in conjunction with ECCV, October 2016

Dr. Evangelia Zacharaki was appointed as guest associate editor for the Medical Physics journal

2nd place at the 2016 IEEE GRSS Data Fusion Contest for the paper: Simultaneous Registration, Segmentation and Change Detection from Multisensor, Multitemporal Satellite Image Pairs .

Finalists (not-awarded) of the Best Papers award at the IEEE conference ICIP'16 for the paper: A Block Parallel Majorize-Minimize Memory Gradient Algorithm .

Oral presentation in the Neural Information Processing Systems (NIPS), 2016 conference of the paper: Testing for Differences in Gaussian Graphical Models: Applications to Brain Connectivity (oral presentations: only 1% of more than 2000 submitted papers).

Oral presentation in the British Machine Vision Conference (BMVC), 2016, of the paper: Efficient Learning for Discriminative Segmentation with Supermodular Losses (oral presentations: 7% of submitted papers).

Oral presentation in the International Conference on Artificial Intelligence and Statistics (AISTATS), 2016, of the paper: A Convex Surrogate Operator for General Non-Modular Loss Functions (oral presentations: 11% of submitted papers).

Acceptance of the project entitled «Predicteurs performants de l’efficacite des agents anticancereux par apprentissage profond (deep learning) de donnees radiomiques et genomiques» as part of the program *Imagerie Médicale Computationnelle*. PI: Dr. Charles Ferte, Gustave Roussy, 94805 Villejuif.

Dense Image and Surface Descriptors

Functional Description Scale-Invariant Descriptor, Scale-Invariant Heat Kernel Signatures DISD implements the SID, SI-HKS and ISC descriptors. SID (Scale-Invariant Descriptor) is a densely computable, scale- and rotation- invariant descriptor. We use a log-polar grid around every point to turn rotation/scalings into translation, and then use the Fourier Transform Modulus (FTM) to achieve invariance. SI-HKS (Scale-Invariant Heat Kernel Signatures) extract scale-invariant shape signatures by exploiting the fact that surface scaling amounts to multiplication and scaling of a properly sampled HKS descriptor. We apply the FTM trick on HKS to achieve invariance to scale changes. ISC (Intrinsic Shape Context) constructs a net-like grid around every surface point by shooting outwards and tracking geodesics. This allows us to build a meta-descriptor on top of HKS/SI-HKS that takes neighborhood into account, while being invariant to surface isometries.

Participants: Iasonas Kokkinos and Eduard Trulls

Contact: Iasonas Kokkinos

URL: http://

Functional Description DPMS implements branch-and-bound object detection, cutting down the complexity of detection from linear in the number of pixels to logarithmic.

Participant: Iasonas Kokkinos

Contact: Iasonas Kokkinos

Part-Based Object Detection

Functional Description A DPM model gives rise to the energy function being optimized during inference and has to be learned offline beforehand. All functionality is accessible via a ROS interface. The work follows closely Felzenszwalb's "Object Detection with Discriminatively Trained Part Based Models", using the FFLD-library in the implementation and facilitating communication in a setup with NAO robots.

Participants: Haithem Boussaid, Stefan Kinauer, Iasonas Kokkinos

Contact: Stefan Kinauer

Keywords: Health - Merging - Registration of 2D and 3D multimodal images - Medical imaging

Functional Description Drop is a software programme that registers images originating from one or more modes by quickly and efficiently calculating a non-rigid / deformable field of deformation. Drop is a new, quick and effective registration tool based on new algorithms that do not require a cost function derivative.

Partner: Centrale Paris

Contact: Nikos Paragios

Keyword: Medical imaging

Functional Description FastPD is an optimization platform in C++ for the computer vision and medical imaging community.

Contact: Nikolaos Paragyios

Grammar Parser for Shapes

Functional Description It is a software for parsing facade images using shape grammars. Grapes implement a parsing methods based on Reinforcement Learning principles. It optimizes simultaneously the topology of the parse tree as well as the associated parameters. GraPeS comes along with predefined shape grammars as XML files and defines three kinds of rewards. However, it also offers the possibility to create new grammars and to provide custom rewards in text files, widening the scope of potential applications. The name of the software comes from the aspect of the parse tree of the binary split grammars involved in the process.

Participant: Iasonas Kokkinos

Contact: Iasonas Kokkinos

URL: http://

High-Order Average Precision SVM

Scientific Description

We consider the problem of using high-order information (for example, persons in the same image tend to perform the same action) to improve the accuracy of ranking (specifically, average precision). We develop two learning frameworks. The high-order binary SVM (HOB-SVM) optimizes a convex upper bound of the surrogate 0-1 loss function. The high-order average precision SVM (HOAP-SVM) optimizes a difference-of-convex upper bound on the average precision loss function. Authors of the research paper: Puneet K. Dokania, A. Behl, C. V. Jawahar and M. Pawan Kumar

Functional Description The software provides a convenient API for learning to rank with high-order information. The samples are ranked according to a score that is proportional to the difference of max-marginals of the positive and the negative class. The parameters of the score function are computed by minimizing an upper bound on the average precision loss. The software also provides an instantiation of the API for ranking samples according to their relevance to an action, using the poselet features. The following learning algorithms are included in the API:

(1) Multiclass-SVM (2) AP-SVM (3) High Order Binary SVM (HOB-SVM) (4) High Order AP-SVM (HOAP-SVM) (5) M4 Learning (unpublished work)

The API is developed in C/C++ by Puneet K. Dokania.

Participants: Puneet Dokania and Pawan Kumar

Contact: Puneet Dokania

URL: http://

Learning-Based Symmetry Detection

Functional Description LBSD implements the learning-based approach to symmetry detection. It includes the code for running a detector, alongside with the ground-truth symmetry annotations that we have introduced for the Berkeley Segmentation Dataset (BSD) benchmark.

Participant: Stavros Tsogkas

Contact: Stavros Tsogkas

Functional Description Texture, modulation, generative models, segmentation, TeXMeG is a front-end for texture analysis and edge detection platform in Matlab that relies on Gabor filtering and image demodulation. Includes frequency- and time- based definition of Gabor- and other Quadrature-pair filterbanks, demodulation with the Regularized Energy Separation Algorithm and Texture/Edge/Smooth classification based on MDL criterion.

Participant: Iasonas Kokkinos

Contact: Iasonas Kokkinos

Keywords: Health - Medical imaging

Functional Description Deformable image and volume registration, is a deformable registration platform in C++ for the medical imaging community. This is the first publicly available platform which contains most of the existing metrics to perform registration under the same concept. The platform is used for clinical research from approximately 3,000 users worldwide.

Participant: Nikos Paragios

Contact: Nikos Paragios

Functional Description MAP inference in MRFs can be performed through the LP relaxation approach. This project was a study of the feasibility and benefit of Newton-type methods for solving the optimization problem that is obtained by smoothing the dual of the LP relaxation. The project TRN-MRF is a trust-region Newton method that can address inference in higher order MRFs, for cases in which decomposition according to individual cliques leads to practical convergence. The project QN-MRF works for problems that need decomposition according to larger sub-problems (like chains of higher order cliques) and it works as long as a node is shared by exactly two sub-problems.

Participants: Hariprasad Kannan, Nikos Paragios, Nikos Komodakis

Contact: Hariprasad Kannan

URL: https://

Functional Description This software can be used to compare two distributions to a reference distribution. Applications include model selection and exploratory data analysis.

Participants: Eugene Belilovsky, Wacha Bounliphone, Matthew Blaschko (in collaboration with researchers at UCL and Deepmind)

Contact: Eugene Belilovsky

Functional Description Novel approach to graph structure discovery using neural network. This software provides a much faster approach to discover underlying conditional independence structure in small sample data.

Participants: Eugene Belilovsky

Contact: Eugene Belilovsky

**Paticipants:** Nikos Paragios (in collaboration with researchers from Université Paris-Est, LIGM, ENPC )

**Paticipants:** Dimitris Samaras, Nikos Paragios

**Paticipants:** Maxim Berman, Stefan Kinauer, Iasonas Kokkinos

**Paticipants:** Jiaqian Yu, Matthew Blaschko

We have proposed an alternating direction method of multipliers (ADMM) based decomposition method loss augmented inference, that only depends on two individual solvers for the loss function term and for the inference term as two independent subproblems. In this way, we can gain computational efficiency and achieve more flexibility in choosing our non-modular loss functions of interest. We have proposed a novel supermodular loss function that empirically achieved better performance on the boundary of the objects, finding elongated structure . We also introduced a novel convex surrogate operator for general non-modular loss functions, which provides for the first time a tractable solution for loss functions that are neither supermodular nor submodular, e.g. Dice loss. This surrogate is based on a canonical submodular-supermodular decomposition for which we have demonstrated its existence and uniqueness. It is further proven that this surrogate is convex, piecewise linear, an extension of the loss function, and for which subgradient computation is polynomial time .

**Paticipants:** Eugene Belilovsky, Wacha Bounliphone, Matthew Blaschko (in collaboration with researchers at UCL and Deepmind)

Kernel mean embeddings allow for comparisons of complex distributions. They have been recently heavily used in hypothesis testing to compare distributions as well as in the nascent field of deep generative modeling. In this work we derived the asymptotic variance of the MMD and the cross covariance between joint MMD. We showed how this can be used effectively for model selection in complex Deep Generative Models where the likelihood metric is not accessible. Our results on the asymptotic variance of the MMD have already been used by other researchers to propose an efficient method for optimal testing and improved training of generative models.

**Paticipants:** Emilie Chouzenoux and Jean-Christophe Pesquet
(in collaboration with F. Abboud, PhD student, J.-H. Chenot and L. Laborelli, research engineers, Institut National de l’Audiovisuel)

Optimization methods play a central role in the solution of a wide array of problems encountered in various application fields, such as signal and image processing. Especially when the problems are highly dimensional, proximal methods have shown their efficiency through their capability to deal with composite, possibly non smooth objective functions. The cornerstone of these approaches is the proximity operator, which has become a quite popular tool in optimization. In this work, we propose new dual forward-backward formulations for computing the proximity operator of a sum of convex functions involving linear operators. The proposed algorithms are accelerated thanks to the introduction of a block coordinate strategy combined with a preconditioning technique. Numerical simulations emphasize the good performance of our approach for the problem of jointly deconvoluting and deinterlacing video sequences.

**Paticipants:** Emilie Chouzenoux and Jean-Christophe Pesquet
(in collaboration with Y. Marnissi, PhD student at Univ. Paris-Est Marne la Vallée and Y. Zheng, IBM Research China)

In this work, a methodology is investigated for signal recovery in the presence of non-Gaussian noise. In contrast with regularized minimization approaches often adopted in the literature, in our algorithm the regularization parameter is reliably estimated from the observations. As the posterior density of the unknown parameters is analytically intractable, the estimation problem is derived in a variational Bayesian framework where the goal is to provide a good approximation to the posterior distribution in order to compute posterior mean estimates. Moreover, a majorization technique is employed to circumvent the difficulties raised by the intricate forms of the non-Gaussian likelihood and of the prior density. We demonstrate the potential of the proposed approach through comparisons with state-of-the-art techniques that are specifically tailored to signal recovery in the presence of mixed Poisson-Gaussian noise. Results show that the proposed approach is efficient and achieves performance comparable with other methods where the regularization parameter is manually tuned from an available ground truth.

**Paticipants:** Emilie Chouzenoux and Jean-Christophe Pesquet
(in collaboration with S. Cadoni, Master student at Univ. Paris-Est Marne la Vallée and Dr C. Chaux, Univ. Aix-Marseille)

State-of-the-art methods for solving smooth optimization problems are nonlinear conjugate gradient, low memory BFGS, and Majorize-Minimize (MM) subspace algorithms. The MM subspace algorithm which has been introduced more recently has shown good practical performance when compared with other methods on various optimization problems arising in signal and image processing. However, to the best of our knowledge, no general result exists concerning the theoretical convergence rate of the MM subspace algorithm. The paper aims at deriving such convergence rates both for batch and online versions of the algorithm and, in particular, discusses the influence of the choice of the subspace. We also propose a Block Parallel Majorize-Minimize Memory Gradient (BP3MG) algorithm for solving large scale optimization problems in . This algorithm combines a block coordinate strategy with an efficient parallel update. The proposed method is applied to a 3D microscopy image restora- tion problem involving a depth-variant blur, where it is shown to lead to significant computational time savings with respect to a sequential approach.

**Paticipants:** Jean-Christophe Pesquet
(In collaboration with Pr. P. L. Combettes, North Carolina State university)

Stochastic approximation techniques have been used in various contexts in data science. We propose a stochastic version of the forward-backward algorithm for minimizing the sum of two convex functions, one of which is not necessarily smooth. Our framework can handle stochastic approximations of the gradient of the smooth function and allows for stochastic errors in the evaluation of the proximity operator of the nonsmooth function. The almost sure convergence of the iterates generated by the algorithm to a minimizer is established under relatively mild assumptions. We also propose a stochastic version of a popular primal-dual proximal splitting algorithm, establish its convergence, and apply it to an online image restoration problem.

**Paticipants:** Jean-Christophe Pesquet
(in collaboration with Pr. G. Chierchia, Univ. Paris-Est Marne la Vallée, and Dr. N. Pustelnik, ENS Lyon)

Sparsity-inducing penalties are useful tools in variational methods for machine learning. In this paper, we propose two block-coordinate descent strategies for learning a sparse multiclass support vector machine. The first one works by selecting a subset of features to be updated at each iteration, while the second one performs the selection among the training samples. These algorithms can be efficiently implemented thanks to the flexibility offered by recent randomized primal-dual proximal methods. Experiments carried out for the supervised classification of handwritten digits demonstrate the interest of considering the primal-dual approach in the context of block-coordinate descent. The efficiency of the proposed algorithms is assessed through a comparison of execution times and classification errors.

**Paticipants:** Emilie Chouzenoux
(in collaboration with Prof. M.-A. Delsuc, IGBMC, Strasbourg, and A. Cherni, PhD student, Univ. Strasbourg)

NMR is a tool of choice for the measure of diffusion coefficients of species in solution. The DOSY experiment, a 2D implementation of this measure, has proven to be particularly useful for the study of complex mixtures, molecular interactions, polymers, etc. However, DOSY data analysis requires to resort to inverse Laplace transform, in particular for polydisperse samples. This is a known difficult numerical task, for which we present here a novel approach. A new algorithm based on a splitting scheme and on the use of proximity operators is introduced. Used in conjunction with a Maximum Entropy and

**Paticipants:** Maria Vakalopoulou, Nikos Paragios

We proposed a scalable, modular, metric-free, single-shot change detection/registration method for remote sensing image pairs . The framework exploits a decomposed interconnected graphical model formulation where in the presence of changes the iconic similarity constraints are relaxed. We employ a discretized, grid-based deformation space. State-of-the-art linear programming and duality principles have been used to optimize the joint solution space where local consistency is imposed on the deformation and the detection space. The proposed framework is working both in a unsupervised and supervised manner depending on the application. The developed method has been validated through large scale experiments on several multi-temporal very high resolution optical satellite datasets. Also a novel generic framework has been designed, developed and validated for addressing simultaneously the tasks of image registration, segmentation and change detection from multisensor, multiresolution, multitemporal satellite image pairs . Our approach models the inter-dependencies of variables through a higher order graph. A patch-based deep learning strategy has been employed and used for segmentation likelihoods. The evaluation of the developed framework was performed on the ’2016 IEEE GRSS Data Fusion Contest’ dataset and indicate very promising results for all three different tasks.

**Paticipants:** Nikos Komodakis, M. Pawan Kumar, Stavros Alchatzidis, Enzo Ferrante, Evangelia Zacharaki, Nikos Paragios

Computer vision tasks are often reformulated as mathematical inference problems where the objective is to determine the set of parameters corresponding to the lowest potential of a task-specific objective function. Graphical models have been the most popular formulation in the field over the past two decades. In we focus on the inference component of the problem and in particular we discuss in a systematic manner the most commonly used optimization principles in the context of graphical models. In we briefly review hyper-graph representations as prominent tools in the casting of perception as a graph optimization problem. We discuss their strength and limitations, provide appropriate strategies for their inference and present their application to address a variety of problems in biomedical image analysis.

Multi-atlas segmentation has emerged in recent years as a simple yet powerful approach in medical image segmentation. It commonly comprises two steps: (1) a series of pairwise registrations that establish correspondences between a query image and a number of atlases, and (2) the fusion of the available segmentation hypotheses towards labeling objects of interest. In , we introduce a novel approach that solves simultaneously for the underlying segmentation labels and the multi-atlas registration. We propose a pairwise Markov Random Field approach,where registration and segmentation nodes are coupled towards simultaneously recovering all atlas deformations and labeling the query image.

**Paticipants:** Evangelia Zacharaki
(in collaboration with Prof. M. Megalooikonomou, University of Patras and M. Koutroumanidis, King’s College, London)

We have addressed the needs of epileptic patients and healthcare professionals, aiming at the design and development of a non-intrusive personal health system for the monitoring and analysis of epilepsy-relevant multi-parametric data and the documentation of the epilepsy related symptoms. Specifically, we investigated the classification of epileptic and non-epileptic events from EEG based on temporal and spectral analysis and different fusion schemes . We also studied the EEG brain activity during whole night sleep, since sleep is recognized as a major precipitator of epileptic activity .

Program: ANR Blanc International

Project acronym: ADAMANTIUS

Project title: Automatic Detection And characterization of residual Masses in pAtients with lymphomas through fusioN of whole-body diffusion-weighTed mrI on 3T and 18F-flUorodeoxyglucoSe pet/ct

Duration: 9/2012-8/2015

Coordinator: CHU Henri Mondor - FR

Program: ANR JCJC

Project acronym: HICORE

Project title: HIerarchical COmpositional REpresentations for Computer Vision

Duration: 10/2010-9/2014

Coordinator: ECP - FR

Program: ANR JCJC

Project acronym: LearnCost

Project title: Learning Model Constraints for Structured Prediction

Duration: 2014-2018

Coordinator: Inria Saclay - FR

Program: ITMOs Cancer & Technologies pour la santé d'Aviesan / INCa

Project acronym: CURATOR

Project title: Slice-to-Image Deformable Registration towards Image-based Surgery Navigation & Guidance

Duration: 12/2013-11/2015

Coordinator: ECP - FR

Title: ICT-Supported Bath Robots

Programm: FP7

Duration: March 2015 - March 2018

Coordinator: Robotnik Automation S.L.L.

Partners:

Bethanien Krankenhaus - Geriatrisches Zentrum - Gemeinnutzige GMBH (Germany)

Fondazione Santa Lucia (Italy)

Institute of Communication and Computer Systems (Greece)

Karlsruher Institut für Technologie (Germany)

Theofanis Alexandridis Kai Sia Ee (OMEGATECH) (Greece)

Robotnik Automation Sll (Spain)

Scuola Superiore di Studi Universitari E di Perfezionamento Sant'Anna (Italy)

Frankfurt University of Applied Sciences (Germany)

Inria contact: Iasonas Kokkinos

The I-SUPPORT project envisions the development and integration of an innovative, modular, ICT-supported service robotics system that supports and enhances older adults’ motion and force abilities and assists them in successfully, safely and independently completing the entire sequence of bathing tasks, such as properly washing their back, their upper parts, their lower limbs, their buttocks and groin, and to effectively use the towel for drying purposes. Advanced modules of cognition, sensing, context awareness and actuation will be developed and seamlessly integrated into the service robotics system to enable the robotic bathing system to adapt to the frail elderly population’ capabilities and the frail elderly to interact in a master-slave mode, thus, performing bathing activities in an intuitive and safe way. Adaptation and integration of state-of-the-art, cost-effective, soft-robotic manipulators will provide the hardware constituents, which, together with advanced human-robot force/compliance control that will be developed within the proposed project, will form the basis for a safe physical human-robot interaction that complies with the most up-to-date safety standards. Human behavioural, sociological, safety, ethical and acceptability aspects, as well as financial factors related to the proposed service robotic infrastructure will be thoroughly investigated and evaluated so that the I-SUPPORT end result is a close-to-market prototype, applicable to realistic living settings.

Title: Intelligent Active MObility Aid RoBOT integrating Multimodal Communication

Programm: FP7

Duration: February 2013 - January 2016

Coordinator: Technische Universität München

Partners:

Bartlomiej Marcin Stanczyk (Poland)

Athena Research and Innovation Center in Information Communication & Knowledge Technologies (Greece)

Bethanien Krankenhaus - Geriatrisches Zentrum - Gemeinnutzige (Germany)

Diaplasis Rehabilitation Center (Greece)

Ecole Centrale des Arts et Manufactures (France)

Technische Universitaet Muenchen (Germany)

Ruprecht-Karls-Universitaet Heidelberg (Germany)

Inria contact: Iasonas Kokkinos

Mobility disabilities are prevalent in our ageing society and impede activities important for the independent living of elderly people and their quality of life. The MOBOT project aims at supporting mobility and thus enforcing fitness and vitality by developing intelligent active mobility assistance robots for indoor environments that provide user-centred, context-adaptive and natural support. Our driving concept envisions cognitive robotic assistants that act (a) proactively by realizing an autonomous and context-specific monitoring of human activities and by subsequently reasoning on meaningful user behavioural patterns, as well as (b) adaptively and interactively, by analysing multi-sensory and physiological signals related to gait and postural stability, and by performing adaptive compliance control for optimal physical support and active fall prevention. Towards these targets, a multimodal action recognition system will be developed to monitor, analyse and predict user actions with a high level of accuracy and detail. The main thrust of our approach will be the enhancement of computer vision techniques with modalities such as range sensor images, haptic information as well as command-level speech and gesture recognition. Data-driven multimodal human behaviour analysis will be conducted and behavioural patterns will be extracted. Findings will be imported into a multimodal human-robot communication system, involving both verbal and nonverbal communication and will be conceptually and systemically synthesised into mobility assistance models taking into consideration safety critical requirements. All these modules will be incorporated in a behaviour-based and context-aware robot control framework. Direct involvement of end-user groups will ensure that actual user needs are addressed. Finally, user trials will be conducted to evaluate and benchmark the overall system and to demonstrate the vital role of MOBOT technologies for Europe's service robotics.

Type: FP7

Defi: Cognitive Systems and Robotics

Instrument: Specific Targeted Research Project

Objectif: Cognitive Systems and Robotics

Duration: February 2013 - January 2016

Coordinator: Dimos Dimarogonas

Partner: KTH (SE)

Inria contact: Iasonas Kokkinos

The RECONFIG project aims at exploiting recent developments in vision, robotics, and control to tackle coordination in heterogeneous multi-robot systems. Such systems hold promise for achieving robustness by leveraging upon the complementary capabilities of different agents and efficiency by allowing sub-tasks to be completed by the most suitable agent. A key challenge is that agent composition in current multi-robot systems needs to be constant and pre-defined. Moreover, the coordination of heterogeneous multi-agent systems has not been considered in manipulative scenarios. We propose a reconfigurable and adaptive decentralized coordination framework for heterogeneous multiple & multi-DOF robot systems. Agent coordination is held via two types of information exchange: (i) at an implicit level, e.g., when robots are in contact with each other and can sense the contact, and (ii) at an explicit level, using symbols grounded to each embodiment, e.g, when one robot notifies one other about the existence of an object of interest in its vicinity.

Title: Statistically Efficient Structured Prediction for Computer Vision and Medical Imaging

Programm: FP7

Duration: January 2014 - December 2017

Coordinator: Inria

Inria contact: Matthew Blaschko

'Inference in medical imaging is an important step for disease diagnosis, tissue segmentation, alignment with an anatomical atlas, and a wide range of other applications. However, imperfections in imaging sensors, physical limitations of imaging technologies, and variation in the human population mean that statistical methods are essential for high performance. Statistical learning makes use of human provided ground truth to enable computers to automatically make predictions on future examples without human intervention. At the heart of statistical learning methods is risk minimization - the minimization of the expected loss on a previously unseen image. Textbook methods in statistical learning are not generally designed to minimize the expected loss for loss functions appropriate to medical imaging, which may be asymmetric and non-modular. Furthermore, these methods often do not have the capacity to model interdependencies in the prediction space, such as those arising from spatial priors, and constraints arising from the volumetric layout of human anatomy. We aim to develop new statistical learning methods that have these capabilities, to develop efficient learning algorithms, to apply them to a key task in medical imaging (tumor segmentation), and to prove their convergence to optimal predictors. To achieve this, we will leverage the structured prediction framework, which has shown impressive empirical results on a wide range of learning tasks. While theoretical results giving learning rates are available for some algorithms, necessary and sufficient conditions for consistency are not known for structured prediction. We will consequently address this issue, which is of key importance for algorithms that will be applied to life critical applications, e.g. segmentation of brain tumors that will subsequently be targeted by radiation therapy or removed by surgery. Project components will address both theoretical and practical issues.'

Title: Development of a system helping in optimizing and tailoring computed tomography protocols

Program: Collaboration et encadrement de these CIFRE

Partners: GE Medical Systems, Université Paris-Est Créteil Val de Marne

Duration: December 2014 - November 2017

The purpose of the PhD is to create a method of CT dose and IQ optimization taking into account clinical indication, patient's characteristics and previous exams. The method will consist (i) in linking a clinical indication to IQ defined by quantitative physical metrics and (ii) in quantifying the patient's morphology from the analysis of local scans.

Title: Développement d' outils de Quantification en Imagerie dans les maladies respiratoires chroniques fibrosantes et obstructives

Program: Fondation de Coopération Scientifique

Partners: GE Medical Systems SCS

Duration: April 2016 - March 2019

We aim to test two different approaches for the development of new imaging biomarkers in interstitial lung disease (scleroderma): assessment of the regional lung elasticity through deformable registration and tissue characterization by textural analysis. We will also study structural changes in obstructive lung disease (cystic fibrosis) in order to develop new imaging biomarkers based on elastic registration, histogram and textural analysis.

Title: Identification and prediction of salient brain states through probabilistic structure learning towards fusion of imaging and genomic date

Partners: Fondation de Coopération Scientifique Digiteo

Duration: January 2012 - December 2016 (extended by 1 year)

UCL – Collaboration with Professor Arthur Gretton. Topic: non-parametric hypothesis testing

University of Toronto – Collaboration with Professors Raquel Urtasun and Richard Zemel. Topic: Representation of Scene Graphs for Image Retrieval and Visual Question Answering

MILA, Universite de Montreal - Collaboration with Kyle Kastner from the lab of Professor Yoshua Bengio. Topic: Efficient Inference of Graph Structures using Deep Learning

Sup’Com Tunis – Collaborative research with Amel Benazza-Benhayia. Collaboration Topic: Multispectral imaging

University of Patras, Greece – Collaborative research with V. Megalooikonomou. Collaboration Topic: Biosignal analysis.

University of Oxford – Collaborative research with Andrea Vedaldi. Collaboration Topic: Deep Learning for Texture Recognition.

Google Research – Collaborative research with George Papandreou. Collaboration Topic: Deep Learning for Semantic Segmentation.

Universitad Polytecnical de Catalunia – Collaborative research with Francesc Moreno. Collaboration Topic: Deep Learning for Image Descriptors.

Ecole Polytechnique Federale de Lausanne (EPFL) – Collaborative research with Eduard Trulls. Collaboration Topic: Deep Learning for Image Descriptors.

University of California at Los Angeles – Collaborative research with Alan Yuille. Collaboration Topic: Deep Learning for Semantic Segmentation.

University of Massachusetts at Amherst – Collaborative research with Subhransu Maji. Collaboration Topic: Deep Learning for Texture Recognition.

Ryerson University – Collaborative research with Kostas Derpanis. Collaboration Topic: Deep Learning for Learning Segmentation.

University of Pennsylvania - Collaborative research with Aristeidis Sotiras. Collaboration Topic: Higher Order Graphs in biomedical image analysis.

Pesquet, Jean-Christophe (co-organizer): Special session on Advanced Optimization Methods for Online Signal Processing at the European Signal Processing Conference (EUSIPCO) 2016

Kokkinos, Iasonas: European Conference on Computer Vision (ECCV) 2016, Asian Conference on Computer Vision (ACCV), IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Pesquet, Jean-Christophe: European Signal Processing Conference (EUSIPCO), IEEE International Conference on Image Processing (ICIP)

The members of the team reviewed numerous papers for several international conferences, such as for the annual conferences on Computer Vision and Pattern Recognition (CVPR), Medical Image Computing and Computer Assisted Intervention (MICCAI), Neural Information Processing Systems (NIPS) and International Conference on Learning Representations (ICLR), IEEE International Conference and Acoustics Speech and Signal Processing (ICASSP), IEEE International Conference on Image Processing (ICIP), IEEE Statistical Signal Processing workshop (SSP), European Signal Processing Conference.

Paragios, Nikos: Medical Image Analysis Journal (MedIA), SIAM Journal on Imaging Sciences

Kokkinos, Iasonas: Image and Vision Computing Journal (IVC), Computer Vision and Image Understanding Journal (CVIU), Special Issue on Deep Learning for Computer Vision (guest editor)

Zacharaki, Evangelia: Medical Physics (guest editor), International Journal of Radiology, Dataset Papers in Science (Radiology)

Pesquet, Jean-Christophe: IEEE Trans. on Signal Processing, IEEE Trans. on Image Processing, IEEE Trans. on Information Theory (IEEE-TI), Signal Processing, SIAM Journal on Optimization, SIAM Journal on Imaging Sciences, Journal of Mathematical Imaging and Vision, Journal of Optimization Theory and Applications

Kokkinos, Iasonas: International Journal of Computer Vision, IEEE Trans. on Pattern Analysis and Machine Intelligence, Computer Vision and Image Understanding (CVIU)

Chouzenoux, Emilie: IEEE Trans. on Image Processing, IEEE Trans. Image Processing, Journal of Mathematical Imaging and Vision (JMIV), Journal of Optimization Theory and Applications, Journal of Global Optimization

Zacharaki, Evangelia: IEEE Trans. on Medical Imaging (T-MI), Medical Image Analysis (MedIA), Trans. on Biomedical Engineering, Neuroimage, Artificial Intelligence in Medicine, Expert Systems with Applications

Ferrante, Enzo: IEEE Trans. on Medical Imaging (T-MI), Medical Image Analysis (MedIA), Computarized Medical Imaging and Graphics (CMIG)

Pesquet, Jean-Christophe: Modena (Optimization Techniques for Inverse Problems workshop), Mathematics Faculty of Wien University,

Polytechnic University of Warsaw.

Kokkinos, Iasonas: Local features workshop held in conjunction with ECCV, October 2016 (keynote speech),

Astronomical Data Analysis Summer School, Chania, Greece, May 2016 (keynote speech),

September 2016, Qualcomm-UvA Deep Vision Seminar

Perceptual Organization in Computer Vision Workshop (CVPR), Jun 2016,

Facebook Artificial Intelligence Research (FAIR) Paris, May 2016,

ETH,May 2016,

Oxford University, March 2016,

Univerity College London, March 2016,

Center for Machine Perception, Prague, March 2016,

Simon Fraser University, February 2016.

Chouzenoux, Emilie: Séminaire Parisien des Mathématiques Appliquées à l'Imagerie, IHP, Paris, November 2016,

Cavalieri workshop on Optimization and Optimal Transport for Imaging, Inria Paris, October 2016.

Zacharaki, Evangelia: LIGM, Université Paris Est, November 2016

**Masters**

Kokkinos, Iasonas

Master: Machine Learning for Computer Vision, 24, M2, Ecole Normale Superieure-Cachan, FR

Master: Introduction to Deep Learning, 24, M2, CentraleSupélec, FR

Zacharaki, Evangelia

Master: Foundations in Machine Learning, 36, M2 DataScience, CentraleSupélec , FR

Corbineau, Marie-Caroline and Pesquet, Jean-Christophe

Master: Advanced course on Optimization, 30h, M1, CentraleSupélec,FR

Chouzenoux Emilie and Pesquet, Jean-Christophe

Master: Foundations of Distributed and Large Scale Computing, 24h, M2 DataScience, CentraleSupélec,FR

PhD in progress : Eugène Belilovsky, Structured Output Prediction on Large Scale Neuroscience Data, Universite Paris-Saclay & KU Leuven, 2014-2017, M. Blaschko

PhD in progress : Jiaqian Yu, Structured Prediction Methods for Computer Vision and Medical Imaging, Universite Paris-Saclay, 2014-2017, M. Blaschko

PhD in progress : Wacha Bounliphone, Statistical tools for Imaging-Genetics data integration, 2013-2016, Universite Paris-Saclay & KU Leuven, M. Blaschko & A. Tenenhaus

PhD in progress : Diane Bouchacourt, Large Scale Diverse Learning for Structured Output Prediction, 2014-2017, M. Pawan Kumar

PhD in progress: Siddhartha Chandra, Efficient Learning and Optimization for 3D Visual Data, 2013-2016, I. Kokkinos, Pawan Kumar

PhD in progress: Stefan Kinauer, Surface-based representations for high-level vision tasks, 2013-2016, I. Kokkinos.

PhD in progress: Alp Guler, Learning 3D representations for high-level vision, 2016-2018, I. Kokkinos.

PhD in progress : Stavros Alchatzidis, Message Passing Methods, Parallel Architectures & Visual Processing, 2011-2014 (extended), Nikos Paragios

PhD in progress : Enzo Ferrante, 2D-to-3D Multi-Modal Deformable Image Fusion, 2012-2015 (extended), N. Paragios

PhD in progress : Vivien Fecamp, Linear-Deformable Multi-Modal Deformable Image Fusion, 2012-2015 (extended), N. Paragios

PhD in progress : Evgenios Kornaropoulos, Diffusion Coefficient: a novel computer aided bio-marker, 2013-2016, N. Paragios

PhD in progress : Maxim Berman, Learning Higher Order Graphical Models, 2014-2017, N. Paragios, I. Kokkinos

PhD in progress : Hariprasad Kannan, Efficient Inference on Higher Order Graphs, 2014-2017, N. Paragios

PhD in progress : Huu Dien Khue Le, Graph-based Visual Perception : Theories and Applications, 2014-2017, N. Paragios

PhD in progress : Marie-Caroline Corbineau, Fast online optimization algorithms for machine learning and medical imaging, 2016-2019, supervised by Emilie Chouzenoux and J.-C. Pesquet

PhD in progress : Loubna El Gueddari, Parallel proximal algorithms for compressed sensing MRI reconstruction - Applications to ultra-high magnetic field imaging, 2016-2019, supervised by Emilie Chouzenoux and J.-C. Pesquet

PhD in progress : Azar Louzi, Fast online optimization algorithms for machine learning and computer vision, 2016-2019, supervised by Emilie Chouzenoux and J.-C. Pesquet

The faculty members of the team (N. Paragios, J.-C. Pesquet, I. Kokkinos, E. Chouzenoux) participated in several PhD Thesis Committees, HDR Committees and served as Grant Reviewers.