3.1 Research program

The recent advent of an increasing number of new microscopy techniques giving access to high throughput screenings and micro or nano-metric resolutions provides a means for quantitative imaging of biological structures and phenomena. To conduct quantitative biological studies based on these new data, it is necessary to develop non-standard specific tools. This requires using a multi-disciplinary approach. We need biologists to define experiment protocols and interpret the results, but also physicists to model the sensors, computer scientists to develop algorithms and mathematicians to model the resulting information. These different expertises are combined within the Morpheme team. This generates a fecund frame for exchanging expertise, knowledge, leading to an optimal framework for the different tasks (imaging, image analysis, classification, modeling). We thus aim at providing adapted and robust tools required to describe, explain and model fundamental phenomena underlying the morphogenesis of cellular and supra-cellular biological structures. Combining experimental manipulations, in vivo imaging, image processing and computational modeling, we plan to provide methods for the quantitative analysis of the morphological changes that occur during development. This is of key importance as the morphology and topology of mesoscopic structures govern organ and cell function. Alterations in the genetic programs underlying cellular morphogenesis have been linked to a range of pathologies.

Biological questions we will focus on include:

1. what are the parameters and the factors controlling the establishment of ramified structures? (Are they really organize to ensure maximal coverage? How are genetic and physical constraints limiting their morphology?),
2. how are newly generated cells incorporated into reorganizing tissues during development? (is the relative position of cells governed by the lineage they belong to?)

Our goal is to characterize different populations or development conditions based on the shape of cellular and supra-cellular structures, e.g. micro-vascular networks, dendrite/axon networks, tissues from 2D, 2D+t, 3D or 3D+t images (obtained with confocal microscopy, video-microscopy, photon-microscopy or micro-tomography). We plan to extract shapes or quantitative parameters to characterize the morphometric properties of different samples. On the one hand, we will propose numerical and biological models explaining the temporal evolution of the sample, and on the other hand, we will statistically analyze shapes and complex structures to identify relevant markers for classification purposes. This should contribute to a better understanding of the development of normal tissues but also to a characterization at the supra-cellular scale of different pathologies such as Alzheimer, cancer, diabetes, or the Fragile X Syndrome. In this multidisciplinary context, several challenges have to be faced. The expertise of biologists concerning sample generation, as well as optimization of experimental protocols and imaging conditions, is of course crucial. However, the imaging protocols optimized for a qualitative analysis may be sub-optimal for quantitative biology. Second, sample imaging is only a first step, as we need to extract quantitative information. Achieving quantitative imaging remains an open issue in biology, and requires close interactions between biologists, computer scientists and applied mathematicians. On the one hand, experimental and imaging protocols should integrate constraints from the downstream computer-assisted analysis, yielding to a trade-off between qualitative optimized and quantitative optimized protocols. On the other hand, computer analysis should integrate constraints specific to the biological problem, from acquisition to quantitative information extraction. There is therefore a need of specificity for embedding precise biological information for a given task. Besides, a level of generality is also desirable for addressing data from different teams acquired with different protocols and/or sensors. The mathematical modeling of the physics of the acquisition system will yield higher performance reconstruction/restoration algorithms in terms of accuracy. Therefore, physicists and computer scientists have to work together. Quantitative information extraction also has to deal with both the complexity of the structures of interest (e.g., very dense network, small structure detection in a volume, multiscale behavior, $...$) and the unavoidable defects of in vivo imaging (artifacts, missing data, $...$). Incorporating biological expertise in model-based segmentation methods provides the required specificity while robustness gained from a methodological analysis increases the generality. Finally, beyond image processing, we aim at quantifying and then statistically analyzing shapes and complex structures (e.g., neuronal or vascular networks), static or in evolution, taking into account variability. In this context, learning methods will be developed for determining (dis)similarity measures between two samples or for determining directly a classification rule using discriminative models, generative models, or hybrid models. Besides, some metrics for comparing, classifying and characterizing objects under study are necessary. We will construct such metrics for biological structures such as neuronal or vascular networks. Attention will be paid to computational cost and scalability of the developed algorithms: biological experimentations generally yield huge data sets resulting from high throughput screenings. The research of Morpheme will be developed along the following axes:

• Imaging: this includes i) definition of the studied populations (experimental conditions) and preparation of samples, ii) definition of relevant quantitative characteristics and optimized acquisition protocol (staining, imaging, $...$) for the specific biological question, and iii) reconstruction/restoration of native data to improve the image readability and interpretation.
• Feature extraction: this consists in detecting and delineating the biological structures of interest from images. Embedding biological properties in the algorithms and models is a key issue. Two main challenges are the variability, both in shape and scale, of biological structures and the huge size of data sets. Following features along time will allow to address morphogenesis and structure development.
• Classification/Interpretation: considering a database of images containing different populations, we can infer the parameters associated with a given model on each dataset from which the biological structure under study has been extracted. We plan to define classification schemes for characterizing the different populations based either on the model parameters, or on some specific metric between the extracted structures.
• Modeling: two aspects will be considered. This first one consists in modeling biological phenomena such as axon growing or network topology in different contexts. One main advantage of our team is the possibility to use the image information for calibrating and/or validating the biological models. Calibration induces parameter inference as a main challenge. The second aspect consists in using a prior based on biological properties for extracting relevant information from images. Here again, combining biology and computer science expertise is a key point.

5.1 Awards

• Luca Calatroni was awarded the Prix d'Excellence 2020 from Univ. Côte d'Azur for the H2020 RISE NoMADS project.

5.2 Publication

• Grégoire Malandain co-authored an article in the journal Science 14. This article was mentionned in newpapers (see section 10.3.1).

6.1 New software

7.1 A continuous relaxation of the ${\ell }_2-{\ell }_0$ problem

Participants: Gilles Aubert, Arne Henrik Bechensteen, Laure Blanc-Féraud.

This work focuses on the minimization of the least square loss function under a $k$-sparse constraint with a ${\ell }_0$ pseudo-norm. This is a non-convex, non-continuous and NP-hard problem. Recently, for the penalized form (sum of the least square loss function and a ${\ell }_0$ penalty term) a relaxation has been introduced which has strong results in terms of minimizers. This relaxation is continuous and does not change the global minimizers, among other favorable properties. The question that has driven this project is the following: can a continuous relaxation of the $k$-sparse constraint problem be developed following the same idea and same steps from the penalized${\ell }_2-{\ell }_0$ problem? We calculate the convex envelope of the constrained problem when the observation matrix is orthogonal and propose a continuous non-smooth, non-convex relaxation of the $k$-sparse constraint functional. We give some equivalence of minimizers between the original and the relaxed problems. The subgradient is calculated as well as the proximal operator of the relaxation, and we propose an algorithm that ensures convergence to a critical point of the $k$-sparse constraint problem. We apply the algorithm to the problem of single-molecule localization microscopy and compare the results with well-known sparse minimization schemes. The results of the proposed algorithm are as good as the state-of-the-art results for the penalized form, while fixing the constraint constant is usually more intuitive than fixing the penalty parameter (see Figure 1).

This work is published in Journal of Mathematical Imaging and Vision 10.

Keywords: inverse problems, ${\ell }_0$ problem, sparse modelling, non-convex, non-smooth, relaxation.

7.2 Molecule localisation in super-resolution microscopy with Poisson data

Participants: Luca Calatroni, Marta Lazzaretti.

This work has been carried out in collaboration with C. Estatico (DIMA, University of Genova).

We propose a continuous non-convex variational model for Single Molecule Localisation Microscopy (SMLM) super-resolution in order to overcome light diffraction barriers. Namely, inspired by previous work 32, 26 we consider a variation of the Continuous Exact ${\ell }_0$ (CEL0) penalty recently introduced to relax the ${\ell }_2$-${\ell }_0$ problem where a weighted-${\ell }_2$ data fidelity is considered to model signal-dependent Poisson noise. For the numerical solution of the associated minimisation problem, we consider an iterative reweighted ${\ell }_1$ (IRL1) strategy for which we detail efficient parameter computation strategies. We report qualitative and quantitative molecule localisation results showing that the proposed weighted-CEL0 (wCEL0) model improves the results obtained by CEL0 and state-of-the-art deep learning approaches for the high-density SMLM ISBI 2013 dataset (see Figure 2).

This will appear in the conference proceedings of the IEEE ISBI 2021 conference.

7.3 Covariance-based ${\ell }_0$ super-Resolution Microscopy with intensity Estimation

Participants: Luca Calatroni, Laure Blanc-Féraud, Vasiliki Stergiopoulou.

We exploit molecules fluctuations for the design of a super-resolution live-cell imaging approach relying on common microscopes equipment and conventional fluorescent dyes. We call the proposed method COL0RME for COvariance-based ${\ell }_0$ super-Resolution Microscopy with intensity Estimation. It codifies the assumption of sparse distribution of the fluorescent molecules as well as the temporal and spatial independence between emitters via a non-convex optimization problem formulated in the covariance domain, similarly as for the SPARCOM approach proposed in 31 and based on the ${\ell }_1$ regularization term. In order to deal with real data, the proposed approach also estimates background and noise statistics. It also includes a final estimation step where intensity information is retrieved, which is valuable for biological interpretation and future applications to super-resolution imaging (see Figure. 3).

This will appear in the proceedings of the IEEE ISBI 2021 conference.

Vasiliki Stergoipoulou's work is supported by the French Government through the 3IA Côte d’Azur Investments in the Future project managed by the National Research Agency (ANR) with the reference number ANR-19-P3IA-0002.

7.4 Statistical framework for the parametric study of Fibronectin Variants in healthy and disease state

Participants: Anca-Ioana Grapa, Laure Blanc-Féraud, Xavier Descombes.

This work is done in collaboration with Ellen Van Obberghen-Schilling and Georgios Efthymiou (iBV).

We were interested in the numerical analysis and modeling of the Fibronectin (FN) networks, a major extracellular matrix (ECM) molecule, expressed in pathological states (fibrosis, cancer, etc). Since the functions of these variants are not well defined in the context of their role within the tumor microenvironment, we hope that an analytic framework for the study of FN parametric maps can differentiate between the tumoral and the normal-like state for one specific FN variant, and statistically characterize the differences, with respect to fibrillar features.

Fiber networks representation

In previous work, we derived a graph-based representation of FN fibers, built on top of Gabor features (fiber scale, orientation, etc). More specifically, Gabor filters were used to enhance the fibrillar structures, followed by a morphological skeletonization of the maximum response of the Gabor filter set. We then derived the corresponding fiber graph networks.

Statistical analysis of deformation maps

FN graph representation enabled the study of parametric deformation maps (e.g. fiber length deformation maps) in a statistical framework for the comparison of "normal" and "tumour-like" state FN variants. Inspired from the methodologies found in the statistical mapping of the functional images, we performed this analysis relying on two different approaches, the first one based on the theory of Gaussian Random Fields (GRF), and the second one, relying on the computation of empirical distributions, as described in the following paragraphs.

The normal FN deformation maps were first modelled as realizations of the GRF, and thus represented the learning set for deriving the model parameters. We were then able to determine a set of probabilities that characterize a degree of belonging to the Gaussian field, of certain regions from the tumoral deformation maps taken at various thresholds. In other words, the current statistical analysis can identify the foreign regions with respect to the GRF within both types of maps under the null hypothesis, with a view to compare their properties (see Figure 4).

Additionally, we considered deformation maps between pairs of registered (i.e. matched) FN networks, normal vs tumor, (see Figure 5 for a example of a deformation map in terms of fiber length - which can be regarded as a local stretching of the fibers that should be applied to first graph in order to obtain the second one), and applied a similar statistical analysis. The matching (alignment) between the images and the relative analysis are important to mitigate the effect of non-stationarity in different tissues (variability within the tissues).

The methodology described above was applied for the quantitative and qualitative comparison of differences with respect to the fiber length for one variant FN B-A+ in both normal and tumoral-like state. Here we illustrate one example of the application under the following conditions (P value p = 0.05):

• Learning dataset : 50 parametric maps (Normal)
• Test set: 70 parametric maps (Tumoral) and 20 parametric maps (Normal)
• Thresholds of intensity T = [95 105 115 120 127]

The results in Table 1 illustrate for every threshold, the average number of identified foreign clusters per image as well as the average cluster area. By comparison, within the normal parametric maps of the test dataset, the method does not identify any foreign cluster when $p=0.05$.

The statistical analysis shows that the differences with respect to the fiber length, between "normal" and "tumoral" variants are statistically significant.

Future perspectives

The results of the statistical analysis of parametric maps (fiber length) were encouraging with respect to both qualitative and quantitative differences among FN variants in normal and tumor-like cases. A more comprehensive analysis can include the study of larger samples, and the analysis of various fiber parameters.

Anca Grapa's work is supported by the French Government (National Research Agency, ANR) through the ”Investments for the Future” LABEX SIGNALIFE: program reference ANR-11-LABX-0028-01.

7.5 Estimation of the total bio-volume of copepods from 2-dimensional views

Participants: Cédric Dubois, Eric Debreuve.

This work was made in collaboration with Jean-Olivier Irisson (LOV, Villefranche-sur-mer, France).

We proposed a method to correct the total bio-mass of copepods estimated with the Equivalent Sphere Diameter method (ESD) or the Ellipse method (ELL) for individual volume inference from two-dimensional images (see Figure 6). The copepod body is modeled with an ellipsoid, which allows to mathematically derive the projected silhouette resulting from the 3D-to-2D acquisition process (typical of UVPs (Underwater Vision Profiler) or the ZooScan for example). This framework is further developed to generate realistic samples of copepods bodies with random orientations, compute their projected silhouette, and estimate their volume with ESD or ELL. We can then compute a correction factor for the total volume. This factor can be subsequently applied to correct total volume estimations obtained with ESD or ELL on real data. The proposed method was validated on a large database (+$150000$ images) of UVP images extracted from the EcoTaxa project. As an auxiliary task, a more general version of the proposed random copepod generator was used to check that estimation errors made by ESD or ELL on individual volumes have no significant influence on the volume spectrum. Incidentally, we also present alternative surface estimation and ellipse fitting approaches for copepod body to make ESD and ELL more reliable.

Top or bottom view of a copepod with the antennas visible	Side view of a copepod with the antennas not visible

7.6 Particle detection using ObjMPP

Participants: Somia Rahmoun, Eric Debreuve, Xavier Descombes.

This work is made in collaboration with Tamas Matuzek and Pascal Thérond, iBV, within the ANR project HMOVE

We consider confocal microscopy 3D stacks of drosophila wing disks on which we aim to detect Hedgehog granules. We employ the marked point process approach thanks to our software Objmpp. The image intensity is severely heterogeneous and can be approximately split in two areas (see Figure 7). Due to the differences in the resolutions in Z and in the X-Y plane, the granules appear as "smarties shaped" flat objects . We have two channels (GFP and Antibody) corresponding to markers of different cell compartments. The goal is to compare the granule populations in both conditions. In this study we have compared different setting of Objmpp (2D vs 3D, mask or no mask) and applied several pre-processing steps on the central part of the tissue. As a conclusion, the best results were obtained by applying a spatial normalisation and a top hat filter to the initial data. Then the image is split in two parts on which Objmpp is run with two different values of the threshold parameters. The next step will consists in analyzing the robustness of this setting along the whole tissue.

7.7 Organoids segmentation and classification

Participants: Jérôme Emonet, Xavier Descombes.

This work is made in collaboration with Stephan Clavel and Victor Tiroille (C3M)

In this project we consider confocal microscopy images of prostate organoids. Prostate cancer is one of the most prevalent cancer in men. The C3M team has the technology to produce organoid from cells. This allows molecules testing in vitro on biological material issued from a given patient. Organoids are therefore a promising technology for reaching a personalised medicine. We have proposed an algorithm to automatically segment organoids from images. In a first step we estimate a texture parameter defined by the conditional variance associated to a Gaussian Markov random field. We then estimate a geometrical approximation of the objects by fitting ellipse using a marked point process model. The segmentation is then adjusted using a marker controlled watershed (see Figure 8). From the segmented object we computed several geometric parameters such as the size and the circularity as well as an average intensity profile from the center to the object border. Using a k-means algorithm we were able to classify the different objects as debris, compact or cystic organoids. In the next steps we will enrich the phenotypic description of organoids to study the influence of endocrinian disruptor on the organoid growth.

7.8 Uveitis detection from OCT using deep learning

Participants: Fatima Zahra El Barche, Alin Achim, Xavier Descombes.

Uveitis is an auto-immune disease that induces retina inflammation. Early prognostic is important to avoid long term visual deficiency. In this project we consider OCT (Optical Coherence Tomography) images to test the ability of convolutional neural networks (CNN) to detect uveitis from such images, Bristol University has collected a database of mouse eyes images. The database contains wild type retina (WT) and retina affected by Uveitis respectively 2, 7 and 14 days after the disease injection. Each dataset is a 3D volume where only part of the slices may be affected by the disease. We consider the classification on 2D slices after removing those that are not concerned by the disease (see Figure 9). After testing several CNN we have retained the VGC16 architecture to perform a binary classification between wild type and affected mice. We obtained an accuracy of $98\%$ between WT and day 14, $97\%$ between WT and day 7 and only $68\%$ between WT and day 2. This first study thus shows that a satisfactory detection of the uveitis can be obtained after 7 days. To reduce this delay we plan to consider a GAN like approach for "predicting" image after 7 day from an image after 2 days and thus expect to improve the classification after 2 days. Unfortunately, images of the same retina cannot be obtained both after 2 and 7 days. We therefore have considered cycleGAN networks. The first results are encouraging.

7.9 Non-convex Super-resolution of OCT images via sparse representation

Participants: Luca Calatroni, Gabriele Scrivanti, Alin Achim.

This project has been done in collaboration with S. Morigi (Università di Bologna).

We propose a non-convex variational model for the super-resolution of Optical Coherence Tomography (OCT) images of the murine eye, by enforcing sparsity with respect to suitable dictionaries learned from high-resolution OCT data. The statistical characteristics of OCT images motivate the use of $\alpha$-stable distributions for learning dictionaries, by considering the non-Gaussian case, $\alpha =1$. The sparsity-promoting cost function relies on a non-convex penalty - Cauchy-based or Minimax Concave Penalty (MCP) - which makes the problem particularly challenging. We propose an efficient algorithm for minimizing the function based on the forward-backward splitting strategy which guarantees at each iteration the existence and uniqueness of the proximal point. Comparisons with standard convex L1-based reconstructions show the better performance of non-convex models, especially in view of further OCT image analysis (see Figure 10).

This will appear in the conference proceedings of the IEEE ISBI 2021 conference.

7.10 Estimation of a rejection class in machine learning algorithms

Participants: Sarah Laroui, Eric Debreuve, Xavier Descombes.

This work is made in collaboration with Aurelia Vernay (Bayer) as part of a contract with Bayer

We propose a learning strategy for supervised classification that is able, by design, to reject a sample as not belonging to any of the known classes 18. It follows a general strategy defined by three main steps: model learning independently for each class, threshold learning based on class interactions, and prediction procedure.

Each class model (PDF) consists of a GMM (Gaussian Mixture Model). We propose to learn a threshold on the associated PDF by analyzing how it responds to “its own” sample set, but also to the sample sets of the other classes (see Figure 11). We consider two approaches. In the 1-vs-all approach, the threshold is directly learned from the class samples and the samples from the other classes. In the 1-vs-1 approach we learn a threshold for each of the other classes by considering all pairs. The final threshold corresponds to the maximum of these values between. To estimate this threshold, we consider two methods. Considering the logistic regression, the derived sigmoid moves from the zero-tag position up to the one-tag one and the threshold is defined as the inverse image of the fictitious tag value 0.5, whereas for the misclassification method, the threshold corresponds to the value leading to the same number of false negatives and false positives.

In order to recognize known and new fungi phenotypes (see Figure 12), images are acquired by transmitted light microscopy. We take advantage of the feature learning capacity of Convolutional Neural Networks (CNN). Once MobileNet trained on the four known phenotypes, each image is represented by the feature vector produced by the CNN from its rescaled version. We proposed two options for feature vector definition. The output of the last layer before the classification part of the network (called Bottlenecks features) and the vector of the output layer (Output features). Using those two type of feature vectors, the proposed method provided very good classification accuracies for both the rejection class and the known classes on our data.

7.11 A classification scheme based on optimal transport

Participants: Sarah Laroui, Eric Debreuve, Xavier Descombes.

This work is made in collaboration with Aurelia Vernay (Bayer) as part of a contract with Bayer

Optimal transport provides a distance between two distributions defined as the minimum cost for transforming one distribution into another. It is a powerful tool for the comparison of distributions (continuous or discrete), or directly of point clouds.

As an alternative to GMM which seeks to "stick to the complexity" of the distribution of the data, we have proposed to transform the data so that they follow a simple model (in practice, Gaussian), the complexity of the data then being "hidden" in the transformation. This method is based on the calculation of an optimal transport plan between the data and samples generated according to a Gaussian law of the same mean and co-variance matrix as the data. The PDF of each class is therefore estimated via a transport to the Gaussian distribution chosen in the target space. Once in this space, the classification method with rejection class is the same as for the Gaussian mixture model. On the other hand, the observations whose class is to be predicted must be transported to the target space before being classified.

In this work in progress we have obtained promising results on synthetic data in 2D (see figure 13).

7.12 RCC Classification by Vascular Morphology and A new RCC Histopathological Dataset

Participants: Rudan Xiao, Xavier Descombes.

This work is made in collaboration with Damien Ambrosetti, Nice CHU

RCC is a highly malignant tumor in the urinary system, $90\%$ of kidney cancers are RCC  29, which is mainly divided into clear cell RCC (ccRCC) with $75\%$ and papillary RCC (pRCC) with $10\%$  28. RCC classification is a challenging task. Most of the current classification research is focused on the number, distribution and properties of cells. There is also another crucial factor for RCC classification is that the tumoral architecture highly relies on the structure of the vascular network. For example, ccRCC is characterized by a “fishnet” structure while the pRCC has a tree-like structure. However, for machine learning model, if vascular network structure is crucial and essential for RCC classification is still an open question. We are focusing on the importance role of the vascular network in the RCC diagnosis. Currently, there is no public benchmark dataset for RCC classification based on vascular network, so we provide a new well-labeled vascular-based RCC histopathological image dataset (VRCC200 dataset) to answer the question. We annotate the vascular network segment of each ccRCC and pRCC patch images. The new dataset contains the data of seven patients, totally 200 well-labeled tumor patch images. This new dataset is extracted from a larger RCC histopathological image dataset (BigRCC dataset), which is labeled with RCC categories. The larger dataset contains the data of 68 patients totally (107 WSIs (Whole Slide Images) for ccRCC and 51 WSIs for pRCC), which include 39986 patch images of ccRCC and 18254 patch images of pRCC respectively (see Figure 14). Next, we build benchmark results of various algorithms, including traditional methods and deep learning, on VRCC200 dataset. To make the traditional algorithms achieve the best results, we further propose two features, called “skeleton feature” and “Lattice feature”, which are extracted from skeletal vascular network. Specifically, we transform the vascular network into skeleton. The skeleton composed with three parts: junction, non-end branch, end branch. Firstly, under our former works, we define some new meaningful describers from the skeleton to form the skeleton feature, such as the small NE, big NE, and Density. Then, we perform a series of operations on the skeleton map, including watershed, minima imposition, removing surrounding, resulting in the lattice feature, such as the mean area, median area, mean perimeter, median perimeter, mean eccentricity and median eccentricity. The obtained classification results using both deep learning or features based methods showed that the vascular networks properties allows a perfect discrimination between pRCC and ccRCC without any additional information.

7.13 Hierarchical classification using Convolutional Neural Networks (CNNs)

Participants: Abed Zaoui, Cédric Dubois, Eric Debreuve.

This work was made in collaboration with Jean-Olivier Irisson, LOV, Villefranche-sur-mer, France

CNN-based models have been used successfully in many computer vision tasks, and particularly for image classification. The Branch CNN model (B-CNN) has a specific architecture allowing to exploit the hierarchical relationships between the classes of a problem within the natural hierarchical structure of CNNs (see Figure 15). By constraining the optimization of the early layers so that they can provide appropriate features to classify the inputs into high-level classes, it is expected to get a higher accuracy on the finer classes at the network output. We tailored a network following this strategy in order to improve the classification of images of zooplankton individuals by using the zooplankton taxonomy established by marine ecologists. Rather than a higher accuracy, we obtained results on par with an equivalent, non-hierarchical CNN, but with a faster learning phase. Although this is encouraging, more work on the network architecture and the learning procedure is needed to indeed reach better performances.

7.14 Computational detection, characterization, and clustering of mouse glial cells to unravel the implication of hypothalamic inflammatory response in obesity

Participants: Morgane Nadal, Eric Debreuve.

This work was made in collaboration with Clara Sanchez and Carole Rovère, IPMC, Sophia Antipolis, France

Overweight and obesity are major public health issues affecting respectively 39% and 13% of the world population (From World Health Organization, 2016). They constitute prominent risks for numerous chronic diseases, including diabetes, cardiovascular diseases, and cancer. Studies in animal models and humans reveal that excess fat diets promote both a peripheral chronic inflammation and a hypothalamic neuroinflammation, which possibly leads to feeding behavior deregulation. Ascertaining whether the inhibition of early activation of two major brain cells involved in feeding behavior (glial cells, more specifically astrocytes and microglia) in the hypothalamus could prevent obesity would offer new prospects for therapeutic treatments. To understand the mechanisms pertaining to obesity-related neuroinflammatory response, we developed a fully automated algorithm, NutriMorph (see Figure 16). Although some algorithms were developed in the past decade to detect and segment neural cells, they are highly specific, not fully automatic, and do not provide the desired morphological analysis. Our algorithm cope with these issues and performs the analysis of cells images (here, microglia of the hypothalamic arcuate nucleus), and the morphological clustering of these cells through statistical analysis and machine learning. Using the k-Means algorithm, it clusters the microglia of the control condition (healthy mice) and the different states of neuroinflammation induced by high-fat diets (obese mice) into subpopulations. Here we show that early hypothalamic inflammation could be already set on within a few hours through modification of microglia subpopulation proportions, instead of a couple of months as previously hypothesized and that the activated microglia show specific morphological characteristics.

7.15 Cortical-inspired Wilson-Cowan models for vision. Applications to visual illusions

Participants: Luca Calatroni.

This work has been supported by the iCODE institute in Paris and has been carried out in collaboration with Dario Prandi (L2S, Paris), Valentina Franceschi (Univàrsita di Padova, Italy), Alexander Gomez-Villa and Marcelo Bertalmìo (UPF, Spain).

In this project, we reproduced suprathreshold perception phenomena, specifically visual illusions, by Wilson–Cowan (WC)-type models of neuronal dynamics. Our findings show that the ability to replicate the illusions considered is related to how well the neural activity equations comply with the efficient representation principle. Our first contribution consists in showing that the WC equations can reproduce a number of brightness and orientation-dependent illusions. Then we formally prove that there cannot be an energy functional that the WC dynamics are minimizing. This leads us to consider an alternative, variational modeling, which has been previously employed for local histogram equalization (LHE) tasks. To adapt our model to the architecture of V1, we perform an extension that has an explicit dependence on local image orientation. Finally, we report several numerical experiments showing that LHE provides a better reproduction of visual illusions than the original WC formulation, and that its cortical extension is capable also to reproduce complex orientation-dependent illusions, such as grating induction and a modified version of the Poggendorff illusion (see Figure 17). For this latter example, we empirically show the existence of a set of threshold parameters differentiating from inpainting to perception-type reconstructions and describing long-range connectivity between different hypercolumns in V1.

This work has been published in 13, 12

The software is available at the webpage https://github.com/dprn/WCvsLHE

7.16 Variational osmosis for cultural heritage digital restoration

Participants: Luca Calatroni.

This project has been done in collaboration with Simone Parisotto (Fitzwilliam Musem, Cambridge), Nicolas Papadakis (IMB, Bordeaux) Aurelie Bougeau (LABRI, Bordeaux), Carola-Bibiane Schönlieb (DAMTP, Cambridge).

In this project, we proposed a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. and Weickert et al. The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision and parameter tuning and can encode prior information on the structure of the images to be fused. For the numerical solution of the proposed model, we develop a primal-dual algorithm and we apply the resulting minimization scheme to solve multi-modal face fusion, color transfer and cultural heritage conservation problems (see Figure 18). Visual and quantitative comparisons to state-of-the-art approaches prove the out-performance and the flexibility of our method.

These results are described in the paper 15.

The code is available at the webpage https://github.com/simoneparisotto/Variational-Osmosis-for-Non-Linear-Image-Fusion

7.17 Fast optimisation via inertial dual diagonal descent

Participants: Luca Calatroni.

This work is in collaboration with L. Rosasco, S. Villa (MaLGA, Genova) and G. Garrigos (LPSM, Paris).

We propose and analyze an accelerated iterative dual diagonal descent algorithm for the solution of linear inverse problems with general regularization and data-fit functions. In particular, we develop an inertial approach of which we analyze both convergence and stability. Using tools from inexact proximal calculus, we prove early stopping results with optimal convergence rates for additive data-fit terms as well as more general cases, such as the Kullback-Leibler divergence, for which different type of proximal point approximations hold.

7.18 Direction-of-Arrival Estimation through Exact Continuous ${\ell }_{2,0}$ -Norm Relaxation problem

Participants: Laure Blanc-Féraud.

This work is made in collaboration with Emmanuel Soubies (IRIT, Toulouse), Adilson Chinatto and Joã Romano (University of Campinas, Brasil) and Pascal Larzabal (University Paris-Saclay).

Direction of Arrival (DOA) estimation is of fundamental importance in array signal processing. It refers to the process of retrieving the incident angles of signals reaching an antenna array. On-grid methods make the assumption that the incident angles belong to a prescribed grid. DOA estimation is then transformed into a challenging linear group-sparse optimization problem involving the ${\ell }_{2,0}$ pseudo-norm. We show that the challenging groupsparse optimization problem that defines on-grid DOA estimation methods can be exactly relaxed by replacing the ${\ell }_{2,0}$ term by a group minimax concave penalty (group-MCP). More precisely, we prove that for a suitable choice of the group-MCP parameters the relaxation preserves the global minimizers of the ${\ell }_{2,0}$ penalized least-squares criteria while removing some of its local minimizers. Moreover, we propose a new dimensionality reduction technique to decrease the computational burden of the estimation when the number of snapshots is larger than the number of antennas. Finally, we deploy an iteratively reweighted ${\ell }_{2,1}$ algorithm to minimize the proposed relaxation and compare its performance against previously proposed on-grid methods (see Figure 19).

This work has been published in IEEE Signal Processing Letters 17.

7.19 New Insights on the Optimality Conditions of the ${\ell }_2-{\ell }_0$ Minimization Problem

Participants: Laure Blanc-Féraud, Gilles Aubert.

This work is made in collaboration with Emmanuel Soubies (IRIT, Toulouse).

This work is devoted to the analysis of necessary (not sufficient) optimality conditions for the ${\ell }_0$-regularized least-squares minimization problem. The initial problem is the minimization of the so-called ${\ell }_2-{\ell }_0$ problem which consists in minimizing ${𝐅}_0\left(𝐱\right)=\frac{1}{2}{\parallel 𝐀𝐱-𝐲\parallel }^2+\lambda {\parallel 𝐱\parallel }_0$ with respect to $𝐱\in {ℝ}^n$ where $𝐲\in {ℝ}^m$ are the observed data, $𝐀\in {ℝ}^{m\times n}$. Usually we are dealing with problems such that $m<<n$ which are under-determined.The sparsity condition reduces the number of solutions. As this is a combinatorial NP-hard optimization problem, non-convex and non-continuous, it is usually replaced by the relaxed continuous problem which consists in minimizing $\widetilde{𝐅}\left(𝐱\right)=\frac{1}{2}{\parallel 𝐀𝐱-𝐲\parallel }^2+\lambda \Phi \left(𝐱\right)$, where $\Phi$ is a continuous function, nor necessarily convex. The necessary optimality conditions on minimizers of $\widetilde{𝐅}\left(𝐱\right)$ are the roots of the plethora of algorithms that have been designed to cope with this NP-hard problem.

Indeed, as global optimality is, in general, intractable, these algorithms only ensure the convergence to suboptimal points that verify some necessary optimality conditions. The degree of restrictiveness of these conditions is thus directly related to the performance of the algorithms. Within this context, our first goal is to provide a comprehensive review of commonly used necessary optimality conditions as well as known relationships between them. Then, we complete this hierarchy of conditions by proving new inclusion properties between the sets of candidate solutions associated to them. Moreover, we go one step further by providing a quantitative analysis of these sets. Finally, we report the results of a numerical experiment dedicated to the comparison of several algorithms with different optimality guaranties. In particular, this illustrates the fact that the performance of an algorithm is related to the restrictiveness of the optimality condition verified by the point it converges to. The results can be synthesized in Fig. 20.

This work has been published in Journal of mathematical Imaging 16.

7.20 Quantification of confidence for SMLM localization via optimal transport

Participants: Bastien Laville, Laure Blanc-Féraud, Gilles Aubert.

Species of the fungal genus Candida are etiological agents responsible for life threatening infections, particularly in immunocompromised patients. For instance, candida albicans is carried by 70$\%$ of the population and can transform itself into an infectious agent, resulting in the transition from an ovoid to a filamentous form of yeast. The study of this transformation is carried out using different fluorescence microscopy techniques, which produce sequences of acquisitions that can be processed to obtain an estimate of the position of the fluorescent molecules. The question of the confidence that can be given to these localizations can be addressed by a brand-new technique, the WIF estimator [Mazidi et al.]. By synthesizing the state of the art on this very recent method, the internship gave an insight of the compilation of several papers, the definition of the mathematical framework, the design of an algorithm illustrating the interest of the estimator and the search for improvements of this criterion.

7.21 Cell lineage calculation

Participants: Manuel Petit, Christophe Godin, Grégoire Malandain.

This work is made within the IPL Naviscope.

In recent years, techniques to image the development of biological organisms have made spectacular progresses. Researchers are now able to observe the trajectories corresponding to the development of 3D-plant tissues or animal embryo with cellular resolution. However, such observations yield a large amount of data, which, in turn, require fast and robust analysis tools to extract information while minimizing user interaction. The goal of M. Petit is first to propose new lineage extraction schemes, and then analysis tools over a population of lineages.

7.22 Morphogenesis of the sea urchin embryo

Participants: Angie Moullet, Grégoire Malandain.

This work is made in collaboration with Barthélémy Delorme and Matteo Rauzi (iBV, Nice).

The goal of the project is to understand how biophysical forces are generated and how they work to produce exquisitely precise and controlled tissue shape changes in embryo development. Tissue morphogenesis is a process by which the embryo is reshaped into the final form of a developed animal. Tissues are constituted by cells that are interconnected one another: local changes of cell mechanical properties and shape drive consequent tissue shape change. Nevertheless, the knowledge per se of the mechanisms and mechanics at the cell level which drive cell shape changes is insufficient to explain how tissues change their shape. Emerging properties arise at higher scales resulting from the interaction of cells within tissues and of tissues coordinating and interacting with one another.

To study the embryo evolution at a cellular scale, temporal series have been acquired by a multi-view light-sheet microscope. We will use the Mediterranean sea urchin embryo species Paracentrotus lividus as a model system and focus on the process of tissue folding.

The Astec software toolbox, originally developed for 14, has been adapted to urchins, and enriched with measurements of new morphological parameters (e.g. urchin embryos have a blastocoel that ascidian embryos do not have)(see Figure 21). A. Moullet succeeded in measuring and studying the archenteron length evolution over a few sea urchin embryos.

7.23 Image processing pipeline and statistical analysis for the ANR project RNAGRIMP

Participants: Florence Besse, Eric Debreuve, Xavier Descombes, Fabienne de Graeve, Somia Rahmoun, Xuchun Zhang.

From a methodological point of view, the ANR project RNAGRIMP deals with the detection of particles (RNP-IMP granules) inside cell cytoplasms, and their analysis in terms of number, properties, and spatial repartition. Two series of images have been acquired using fluorescence microscopy: one where the cell cytoplasm has been stained with GFP (Green Fluorescent Protein), the second where the nuclei have been stained with DAPI (4',6-diamidino-2-phenylindole). The experimental conditions were varied in order to target different genes potentially implied in the regulation of the RNP-IMP granules. In total, this screening generated 3 millions of images. In the first step of the image processing pipeline, the nuclei are detected on the DAPI images and a classifier into living or dead cell is learned based on morphological and radiometric nuclei properties (average intensity, area, granularity, circularity...). Then, the living cell cytoplasms are segmented on the GFP images. This task is particularly difficult due to the heterogeneity of the cytoplasmic intensity, both within a cell and across cells. Besides, cells regularly form clusters inside which there is no clear separation between adjacent cells. Next, small particles are detected inside the segmented cytoplasms. Finally, a database of particles is fed with the characteristics of the detected particles: corresponding cell and experimental condition, shape properties, radiometric properties, and indicators of position within the cell cytoplasm.

In order to identify which genes are indeed implied in the regulation of the RNP-IMP granules and how, specific statistical tools are being developed to analyze the particle database. The average number of particles per cell, or the cytoplasmic particle density if all cells are roughly the same size, is the primary factor to study. Preliminary results already allow to ensure that candidate regulatory genes can be identified using the proposed image processing pipeline and database analysis (see Fig. 22).

8.1 Bilateral contracts with industry

• Bayer CropScience, Lyon: 39-month companion contract (Aug 2018 – Oct 2021) for the Cifre PhD thesis of Sarah Laroui.

9.1 Informal international partners

• Xavier Descombes is collaborating with Franscesco Ponzio, PhD student at Politecnico di Torino, on deep learning approaches for analyzing histopathological images. This work is part of a collaboration with Nice Hospital on histopathology for kidney cancer.
• Luca Calatroni is collaborating with Prof. Fiorella Sgallari, Prof. Alessandro Lanza, Dr. Monica Pragliola, Prof. Elena Loli Piccolomini, Pasquale Cascarano and Prof. Serena Morigi (University of Bologna, Italy) on variational modesl for image reconstruction.
• Luca Calatroni is collaborating with Dr. Simone Rebegoldi (University of Firenze) on optimisation approaches for imaging.
• Luca Calatroni has collaborated with Dr. Emre Baspinar (MathNeuro team) on neural-field models for vision.
• Luca Calatroni has collaborated with Dr. Guillaume Garrigos (LPSM, Paris), Prof. Lorenzo Rosasco and Prof. Silvia Villa (MaLGA, University of Genova) on inertial optimisation models for imaging and machine learning.
• Laure Blanc-Féraud has collaborated with A Chinato and J. Romano from university of Campinas Brasil on sparse optimization for the DOA (Direction of Arrival) problem in antennas processing.

9.2 International research visitors

9.3 European initiatives

9.4 National initiatives

9.5 Regional initiatives

10.1 Promoting scientific activities

10.2 Teaching - Supervision - Juries

10.3 Popularization

11.1 Major publications

• 1 articleMarceloM. Bertalmio, LucaL. Calatroni, ValentinaV. Franceschi, BenedettaB. Franceschiello, AlexanderA. Gomez-Villa and DarioD. Prandi. Visual illusions via neural dynamics: Wilson-Cowan-type models and the efficient representation principleJournal of Neurophysiology1235May 2020, 1606-1618
  HALDOI
• 2 article FabienneF. De Graeve, EricE. Debreuve, SomiaS. Rahmoun, SzilviaS. Ecsedi, AliaA. Bahri, ArnaudA. Hubstenberger, XavierX. Descombes and FlorenceF. Besse. Detecting and quantifying stress granules in tissues of multicellular organisms with the Obj.MPP analysis tool Traffic July 2019
  HAL DOI
• 3 article XavierX. Descombes. Multiple objects detection in biological images using a marked point process framework Methods 2016
  HAL DOI
• 4 articleR. Fernandez, PradeepP. Das, V. Mirabet, E. Moscardi, JanJ. Traas, J.-L. Verdeil, GrégoireG. Malandain and ChristopheC. Godin. Imaging plant growth in 4D : robust tissue reconstruction and lineaging at cell resolution.Nature Methods72010, 547-553
  HALDOI
• 5 article LéoL. Guignard, Ulla-MajU.-M. Fiuza, BrunoB. Leggio, JulienJ. Laussu, EmmanuelE. Faure, GaëlG. Michelin, KilianK. Biasuz, LarsL. Hufnagel, GrégoireG. Malandain, ChristopheC. Godin and PatrickP. Lemaire. Contact area-dependent cell communication and the morphological invariance of ascidian embryogenesis Science July 2020
  HAL DOI
• 6 articleAlejandroA. Mottini, XavierX. Descombes and FlorenceF. Besse. From Curves to Trees: A Tree-like Shapes Distance Using the Elastic Shape Analysis FrameworkNeuroinformatics132April 2015, 175-191
  HALDOI
• 7 articleAgustinaA. Razetti, CarolineC. Medioni, GrégoireG. Malandain, FlorenceF. Besse and XavierX. Descombes. A stochastic framework to model axon interactions within growing neuronal populationsPLoS Computational Biology1412December 2018, e1006627
  HAL
• 8 articleEmmanuelE. Soubies, LaureL. Blanc-Féraud and GillesG. Aubert. A Continuous Exact l0 penalty (CEL0) for least squares regularized problemSIAM Journal on Imaging Sciences83July 2015, pp. 1607--1639 (33 pages)
  HALDOI
• 9 article EmmanuelE. Soubies, AgataA. Radwanska, DominiqueD. Grall, LaureL. Blanc-Féraud, EllenE. Van Obberghen-Schilling and SébastienS. Schaub. Nanometric axial resolution of fibronectin assembly units achieved with an efficient reconstruction approach for multi-angle-TIRF microscopy Scientific Reports 9 1 December 2019
  HAL DOI

11.2 Publications of the year

11.3 Cited publications

• 26 inproceedingsSimonS. Gazagnes, EmmanuelE. Soubies and LaureL. Blanc-Féraud. High density molecule localization for super-resolution microscopy using CEL0 based sparse approximationBiomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium onIEEE2017, 28--31
  back to text
• 27 articleN. Gustafsson, S. Culley, G. Ashdown, D. M.D. Owen, P. M.P. Pereira and R. Henriques. Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuationsNature communications, 7(1)2016, 12471-12471
  back to text
• 28 articleA. Lopez-Beltran and et al. 2004 WHO classification of the renal tumors of the adultsEuropean urology492006, 798-805
  back to text
• 29 articleR.J.R. Motzer, H. B.H. Neil and DavidD. M.N.. Renal-cell carcinomaNew England Journal of Medicine3351996, 865-875
  back to text
• 30 articleDanielD. Sage, HagaiH. Kirshner, ThomasT. Pengo, NicoN. Stuurman, JunhongJ. Min, SulianaS. Manley and MichaelM. Unser. Quantitative evaluation of software packages for single-molecule localization microscopyNature methods1282015, 717--24
  back to text
• 31 articleO. Solomon, Y. C.Y. Eldar, M. Mutzafi and M. Segev. SPARCOM: Sparsity Based Super-resolution Correlation MicroscopySIAM Journal on Imaging Sciences, 12 (1)2019, 392-419
  back to textback to text
• 32 articleEmmanuelE. Soubies, LaureL. Blanc-Féraud and GillesG. Aubert. A Unified View of Exact Continuous Penalties for ${}_2-{}_0$ Minimization'SIAM Journal on Optimization2732017, 2034--2060
  back to text