(Fluorescence Lifetime Microscopy Imaging): imaging of fluorescent molecule lifetimes.
(Photo-Activated Localization Microscopy): high-resolution microscopy using stochastic photo-activation of fluorophores and adjustment of point spread functions .
(Structured Illumination Microscopy): high-resolution light microscopy using structured patterns and interference analysis .
(Total Internal Reflectance): 2D optical microscopy using evanescent waves and total reflectance .
(Cryo-Electron Tomography): 3D representation of sub-cellular and molecular objects of 5-20 nanometres, frozen at very low temperatures, from 2D projections using a transmission electron microscope.
Light microscopy, especially fluorescence microscopy, has taken a prominent role in life science research due to its ability to investigate the 3D interior of cells and organisms. It enables to visualize, in vitro and in vivo, particular biomolecules and proteins (gene expression) with high specificity through fluorescent labeling (GFP - Green Fluorescence Protein probes) both at the microscopic and nanoscopic scales. Nevertheless, the mechanisms of life are very complex and driven by multimolecular interactions: mitotic spindle, cell signaling complexes, intracellular transport, cell morphogenesis and motility... A dynamical quantitative and integrated description of molecular interactions and coordination within macromolecular complexes at different scales appears essential today for the global understanding of live mechanisms. A long-term research consists in inferring the relationships between the dynamics of macromolecules and their functions. This constitutes one of the challenges of modern biology. The proposed mathematical models and algorithms are mainly developed to identify molecular processes in fundamental biology but they have also a strong potential for applications in biotechnology and medicine: disease diagnosis, detection of genomic instabilities, deterioration of cell cycle, epigenetic mechanisms and cancer prevention.
Facing the amount of information provided by high-throughput multidimensional microscopy, the serpico team investigates computational and statistical models to better elucidate the role of specific proteins inside their multiprotein complexes and to help to decipher the dynamic coordination and organization of molecular complexes at the single cell level. We investigate image processing methods, mathematical models, and algorithms to build an integrated imaging approach that bridges the resolution gaps between the molecule and the whole cell, in space and time . We address the following topics:
Image superresolution/image denoising required to preserve cell integrity (photo-toxicity versus exposure time) and image analysis in multidimensional microscopy;
Motion analysis and computation of molecule trajectories in live-cell imaging to study molecular interactions in space and time);
Computational simulation and modelling of molecule trafficking at different spatial and temporal scales (e.g. biophysical model assimilation for dynamic representation in video-microscopy and prediction in biology).
We focus on the cellular and molecular mechanisms involved in membrane transport and trafficking at the scale of a single cell.
In most cases, modern microscopy in biology is characterized by a large number of dimensions that fits perfectly with the complexity of biological features: two or three spatial dimensions, at macro to nano-scales, and one temporal dimension, sometimes spectrally defined and often corresponding to one particular bio-molecular species. Dynamic microscopy is also characterized by the nature of the observable objects (cells, organelles, single molecules, ...), by the large number of small size and mobile elements (chromosomes, vesicles, ...), by the complexity of the dynamic processes involving many entities or group of entities sometimes interacting, by particular phenomena of coalescence often linked to image resolution problems, finally by the association, dissociation, recomposition or constitution of those entities (such as membrane fusion and budding). Thus, the corpus of data to be considered for a comparative analysis of multiple image series acquisitions is massive (up to few GigaBytes per hour). Therefore, it becomes necessary to facilitate and rationalize the production of those multidimensional data, to improve post acquisition analysis (i.e. image processing) which are limiting factors in front of the data, and to favor the organization and the interpretation of the information associated to this data corpus. It motivates and requires innovative mathematical tools and concepts: data fusion, image registration, superresolution, data mining, life dynamics modelling, ...
In collaboration with UMR 144 CNRS-Institut Curie (“Space Time imaging of Endomembranes and organelles Dynamics” (STED) team) and PICT-IBiSA (Cell and Tissue Imaging Facilities), the members of the serpico team participate in several projects (PhD and post-doc supervision, contracts...) with biologists in the field of cell biology and microscopy. We have promoted and designed non-parametric methods since prior knowledge cannot be easily taken into account for extracting unattended but desired information from image data. We have proposed user-friendly algorithms for processing 3D or 4D data.
The scientific projects of the serpico team are complementary to the other on-going and planned projects of the UMR 144 CNRS-Institut Curie Unit. A subset of projects is related to instrumentation in electronic and photonic microscopy (PICT-IBiSA platform) including computational aspects on the reconstruction and enhancement of images related to sub-diffraction light microscopy and multimodal approaches. Our projects rely partially on the results and advances of these instrumental projects and a positive synergy is foreseen.
Many live-cell fluorescence imaging experiments are limited in time to prevent phototoxicity and photobleaching. The amount of light and time required to observe entire cell divisions can generate biological artifacts. In order to produce images compatible with the dynamic processes in living cells as seen in video-microscopy, we study the potential of denoising, superresolution, tracking, and motion analysis methods in the Bayesian and the robust statistics framework to extract information and to improve image resolution while preserving cell integrity.
In this area, we have already demonstrated that image denoising allows images to be taken more frequently or over a longer period of time . The major advantage is to preserve cell integrity over time since spatio-temporal information can be restored using computational methods , , , . This idea has been successfully applied to wide-field, spinning-disk confocal microscopy , TIRF , fast live imaging and 3D-PALM using the OMX system in collaboration with J. Sedat and M. Gustafsson at UCSF . The corresponding nd-safir denoiser software (see Section ) has been licensed to a large set of laboratories over the world. New information restoration and image denoising methods are currently investigated to make SIM imaging compatible with the imaging of molecular dynamics in live cells. Unlike other optical sub-diffraction limited techniques (e.g. STED , PALM ) SIM has the strong advantage of versatility when considering the photo-physical properties of the fluorescent probes . Such developments are also required to be compatible with “high-throughput microscopy” since several hundreds of cells are observed at the same time and the exposure times are typically reduced.
The main challenge is to detect and track xFP tags with high precision in movies representing several GigaBytes of image data. The data are most often collected and processed automatically to generate information on partial or complete trajectories. Accordingly, we address both the methodological and computational issues involved in object detection and multiple objects tracking in order to better quantify motion in cell biology. Classical tracking methods have limitations as the number of objects and clutter increase. It is necessary to correctly associate measurements with tracked objects, i.e. to solve the difficult data association problem . Data association even combined with sophisticated particle filtering techniques or matching techniques is problematic when tracking several hundreds of similar objects with variable velocities. Developing new optical flow and robust tracking methods and models in this area is then very stimulating since the problems we have to solve are really challenging and new for applied mathematics. In motion analysis, the goal is to formulate the problem of optical flow estimations in ways that take physical causes of brightness constancy violations into account , . The interpretation of computed flow fields enables to provide spatio-temporal signatures of particular dynamic processes (e.g. Brownian and directed motion) and could help to complete the traffic modelling.
Protein complexes in living cells undergo multiple states of local concentration or dissociation, sometimes associated with diffusion processes. These events can be observed at the plasma membrane with TIRF microscopy. The difficulty arises when it becomes necessary to distinguish continuous motions due to trafficking from sudden events due to molecule concentrations or their dissociations. Typically, plasma membrane vesicle docking, membrane coat constitution or vesicle endocytosis are related to these issues.
Several approaches can be considered for the automatic detection of appearing and vanishing particles (or spots) in wide-field and TIRF microscopy images. Ideally this could be performed by tracking all the vesicles contained in the cell , . Among the methods proposed to detect particles in microscopy images , , none is dedicated to the detection of a small number of particles appearing or disappearing suddenly between two time steps. Our way of handling small blob appearances/dis-appearances originates from the observation that two successive images are redundant and that occlusions correspond to blobs in one image which cannot be reconstructed from the other image (see also ). Furthermore, recognizing dynamic protein behaviors in live cell fluorescence microscopy is of paramount importance to understand cell mechanisms. In our studies, it is challenging to classify intermingled dynamics of vesicular movements, docking/tethering, and ultimately, plasma membrane fusion of vesicles that leads to membrane diffusion or exocytosis of cargo proteins. Our aim is then to model, detect, estimate and classify subcellular dynamic events in TIRF microscopy image sequences. We investigate methods that exploits space-time information extracted from a couple of successive images to classify several types of motion (directed, diffusive (or Brownian) and confined motion) or compound motion.
Mathematical biology is a field in expansion, which has evolved into
various branches and paradigms to address problems at various scales
ranging from ecology to molecular structures. Nowadays, system
biology , aims at
modelling systems as a whole in an integrative perspective instead of
focusing on independent biophysical processes. One of the goals of
these approaches is the cell in silico as investigated at Harvard
Medical School (http://
In this area, we focus on the analysis of transport intermediates (vesicles) that deliver cellular components to appropriate places within cells. We have already investigated the concept of Network Tomography (NT) mainly developed for internet traffic estimation. The idea is to determine mean traffic intensities based on statistics accumulated over a period of time. The measurements are usually the number of vesicles detected at each destination region receiver. The NT concept has been investigated also for simulation since it can be used to statistically mimic the contents of real traffic image sequences. In the future, we plan to incorporate more prior knowledge on dynamics to improve representation. An important challenge is to correlate stochastic, dynamical, one-dimensional in silico models studied at the nano-scale in biophysics, to 3D images acquired in vivo at the scale of few hundred nanometers. A difficulty is related to the scale change and statistical aggregation problems (in time and space) have to be handled.
In the past recent years, research carried at UMR 144 CNRS-Institut Curie (“Space Time imaging of Endomembranes and organelles Dynamics” (STED) team) contributed to a better understanding of the intracellular compartimentation of specialized model cells such as melanocytes and Langerhans cells, the components and structural events involved in the biogenesis of their specialized organelles: melanosomes and Birbeck granules, respectively. These studies have started to highlight: i) multiple sorting and structural events involved in the biogenesis of these organelles; ii) complexity of the endo-melanosomal network of these highly specialized cells; iii) complex molecular architecture organizing and coordinating their dynamics; iv) intracellular transport steps affected in genetic diseases, among which the Hermansky Pudlak syndrome (HPS) or involved in viral infection (HIV and Langerin in Langerhans cells).
In this context, the central aim of serpico is to understand how the different machineries of molecular components involved are interconnected and coordinated to generate such specialized structures. We need to address the following topics:
developing new bioimaging approaches to observe and statistically analyze such coordinated dynamics in live material;
correlating this statistically relevant spatiotemporal organization of protein networks with the biological architectures and at the ultrastructural level;
modeling intracellular transport of those reference biological complex systems and proposing new experimental plans in an iterative and virtuous circle;
managing and analyzing the workflow of image data obtained along different multidimensional microscopy modalities.
These studies are essential to unravel the complexity of the endomembrane system and how different machineries evolve together (e.g. see Fig. ). They help to control cell organization and function at different scales through an integrative workflow of methodological and technological developments.
At long term, these studies will shed light on the cellular and molecular mechanisms underlying antigen presentation, viral infection or defense mechanisms, skin pigmentation, the pathogenesis of hereditary genetic disorders (lysosomal diseases, immune disorders) and on the mechanisms underlying cell transformation. Our methodological goal is also to link dynamics information obtained through diffraction limited light microscopy, eventually at a time regime compatible with live cell imaging. The overview of ultrastructural organization will be achieved by complementary electron microscopical methods. Image visualization and quantitative analysis are of course important and essential issues in this context.
.
Motion2d: Parametric motion model estimation
The Motion2D software written in c++ (APP deposit number: FR.001.520021.001.S.A.1998.000.21000 / release 1.3.11, January 2005) and java (plug-in ImageJ (http://
Free academic software distribution: Motion2D Free Edition is the version of Motion2D available for development of Free and Open Source software only. More information on Motion2D can be found at http://
On-line demo: Mobyle@SERPICO http://
Collaborator: Fabien Spindler (Inria Lagadic team).
ND-Safir and Fast2D-SAFIR: Image denoising software
The nD-Safir software (APP deposit number: IDDN.FR.001.190033.002.S.A.2007.000.21000 / new release 3.0 in 2013) written in c++, java and matlab, removes additive Gaussian and non-Gaussian noise in still 2D or 3D images or in 2D or 3D image sequences (without any motion computation) . The method is unsupervised and is based on a pointwise selection of small image patches of fixed size (a data-driven adapted way) in spatial or space-time neighbourhood of each pixel (or voxel). The main idea is to modify each pixel (or voxel) using the weighted sum of intensities within an adaptive 2D or 3D (or 2D or 3D + time) neighbourhood and to use image patches to take into account complex spatial interactions. The neighbourhood size is selected at each spatial or space-time position according to a bias-variance criterion. The algorithm requires no tuning of control parameters (already calibrated with statistical arguments) and no library of image patches. The method has been applied to real noisy images (old photographs, jpeg-coded images, videos, ...) and is exploited in different biomedical application domains (time-lapse fluorescence microscopy, video-microscopy, mri imagery, x-ray imagery, ultrasound imagery, ...).
The fast-2d-safir software (APP deposit number: IDDN.FR.001.190033.001.S.A.2007.000.21000) written in c++ removes mixed Gaussian-Poisson noise in large 2D images, typically
On-line demo: Mobyle@SERPICO http:// |
Free download binaries: Binaries of the software nD-safir are freely and electronically distributed. Developed in standard C/C++ under Linux using the CImg library, it has been tested over several platforms such as Linux/Unix, Windows XP and Mac OS. |
Academic licence agreements: Institut Curie, CNRS, ENS Ulm, Oxford University, Weizmann Institute, UCSF San-Francisco, Harvard University, Berkeley University, Stanford University, Princeton University, Georgia-Tech, Kyoto UNiversity, IMCB Singapore ... |
Commercial licence agreements: Innopsys, Roper Scientfic, Photmetrics, Nikon (2015). |
Collaborators: Jérôme Boulanger and Jean Salamero (UMR 144 CNRS-Institut Curie, STED team), Peter Elbau (RICAM Linz, Austria) and Jean-Baptiste Sibarita (UMR 5091, University of Bordeaux 2). |
HullkGround: Background subtraction by convex hull estimation
The HullkGround software (APP deposit number: IDDN.FR.001.400005.000.S.P.2009.000.21000) written in java (plug-in ImageJ)
decomposes a fluorescence microscopy image sequence into two dynamic
components: i) an image sequence showing mobile objects; ii) an image
sequence showing the slightly moving background. Each temporal signal
of the sequence is processed individually and analyzed with
computational geometry tools. The convex hull is estimated
automatically for each pixel and subtracted to the original
signal. The method is unsupervised, requires no parameter tuning and
is a simplified version of the
On-line demo: Mobyle@SERPICO http://
Collaborators: Anatole Chessel and Jean Salamero (UMR 144 CNRS-Institut Curie, STED team).
.
TubuleJ: Straightening of microtubule cryo-EM projection views
The TubuleJ software (APP deposit number: IDDN.FR.001.240023.000.S.P.2011.000.21000) written in java (plug-in ImageJ) is devoted to the analysis of microtubules and helical structures in 2D
cryo-electron microscope images. The software straightens curved microtubule images by estimating automatically points locations on the microtubule axis. The estimation of microtubule principal axis relies on microtubule cylindrical shape analyzed in the Fourier domain. A user-friendly interface enables to filter straight fiber images by selecting manually the layer lines of interest in the Fourier
domain. This software can be used to generate a set of 2D projection views from a single microtubule projection view and a few parameters of this microtubule structure. These projection views are
then back projected, by using the imod plug-in (http://
On-line demo: see http://
Collaborators: Sophie Blestel and Denis Chrétien (UMR 6290, CNRS, University of Rennes 1).
Cryo-Seg: Segmentation of tomograms in cryo-electron microscopy
The Cryo-Seg software written in c++ and java (plug-in mageJ) has been developed to detect microtubule structures and helical structures in 2D cryo-electron microscope images. Cryo-electron tomography allows 3D observation of biological specimens in their hydrated state. Segmentation is formulated as Maximum A Posteriori estimation problem and exploits image patches to take into account spatial contexts (Markov Random Fields). Because of the contrast anisotropy in the specimen thickness direction, the whole tomogram is segmented section by section, with an automatic update of reference patches. This algorithm has been evaluated on synthetic data and on cryo-electron tomograms of in vitro microtubules . On real data, this segmentation method extracts the most contrasted regions of microtubules, and 3D visualization is improved.
Collaborators: Sophie Blestel and Denis Chrétien (UMR 6290, CNRS-University of Rennes 1).
.
The objective is to disseminate the distribution of serpico image processing software for biologist users:
Free binaries: software packages have been compiled for the main operating systems (Linux, MacOS, Windows) using CMake (see http://
Mobyle@SERPICO web portal: An on-line version of the image processing algorithms has been developped using the Mobyle framework (Institut Pasteur, see http://
ImageJ plug-ins: ImageJ (see http://
Institut Curie CID iManage database: The microscopy facility of Institut Curie has co-developped a commercial database system (CID iManage/Strand Avadis company). The database can be searched via meta-data and includes menu selections that enable to run remote processing from a cluster. We have integrated nD-Safir and Hullkground in the interface environment to allow the database users to process their images easily, and store associated results and parameters used.
Collaborators: Charles Deltel (Inria Rennes SED) and Perrine Paul-Gilloteaux (UMR 144 CNRS-Institut Curie, STED team and PICT-IBiSA).
Inspired form the non-local means , we developed a stochastic NL-means-based denoising algorithm for generalized non-parametric noise models , . First, we provided a statistical interpretation to current patch-based neighborhood filters and justify the Bayesian inference that needs to explicitly account for discrepancies between the model and the data. Furthermore, we investigated the Approximate Bayesian Computation (ABC) rejection method , combined with density learning techniques for handling situations where the posterior is intractable or too prohibitive to calculate. This is particularly relevant for images contaminated by heterogeneous sources of noise. A major difference with previous methods is that we directly handle the structure of the noise, without precise parametric modeling of the noise. We demonstrated the flexibility of our stochastic Gamma non-local means (SGNL-means) by showing how it can be adapted to tackle noise in frequency domain fluorescence lifetime imaging microscopy (FD-FLIM) and cryo-electron tomography (see Fig. ).
Moreover, we also proposed a general statistical aggregation method which combines image patches denoised with several commonly-used algorithms . We showed that weakly denoised versions of the input image obtained with standard methods, can serve to compute an efficient patch-based aggregated estimator. In our approach, we evaluate the Stein’s Unbiased Risk Estimator (SURE) of each denoised candidate image patch and use this information to compute the exponential weighted aggregation (EWA) estimator. The aggregation method is flexible enough to combine any standard denoising algorithm and has an interpretation with Gibbs distribution. The denoising algorithm (PEWA) is based on an MCMC sampling and is able to produce results that are comparable to the state of the art (, ). In this range of work, we have also introduced in a general method to combine estimators in order to produce a better estimate. From a theoretical point of view, we proved that this method is optimal in some sense. It is illustrated on standard statistical problems in parametric and semi-parametric models where the averaging estimator outperforms the initial estimators in most cases. As part of an on-going work, we are applying this method to improve patch-based image denoising algorithms.
Collaborators: Philippe Roudot (UT Southwestern Medical Center, Dallas (TX))
Francois Waharte (UMR 144 CNRS-Institut Curie, STED team and PICT-IBiSA)
Paul Rochet (Laboratoire de Mathématiques Jean Leray (LMJL), university of Nantes)
.
Fluorescence imaging is popular in cell biology research due to its high contrast imaging capability. In microscopy imaging under low exposure conditions, the image quality is limited by out-of-focus blur and high noise. As a result a preprocessing stage known as deconvolution is needed to estimate a good quality version of the observed image. We proposed to design an efficient deconvolution algorithm for fluorescence microscopy under low exposure conditions by using the Poisson noise model. The result of deconvolution depends heavily on the choice of the regularization term. The regularization functional should be designed to remove noise while retaining the image structure. The choice of Poisson noise model and new regularization functional demands the design of a new and efficient optimization algorithm. We proposed to use a complex non quadratic regularization functional along with Poisson noise assumption for the first time. The use of non quadratic regularization makes the resulting optimization problem a complex one. This demanded the development of a problem-specific optimization algorithm which is fast as well as robust enough to minimize a non quadratic cost function. The use of non quadratic regularization together with Poisson noise model ensures that finer details of underlying structures are well restored in the presence of large amount of noise.
Collaborator: Muthuvel Arigovindan (Imaging Systems Lab, Department of Electrical Engineering, Indian
Institute of Science, Bangalore, India).
.
In live cell fluorescence microscopy images, the moving tagged structures of interest, such as vesicles, often appear as bright spots with intensity that varies along time over a time-varying and cluttered background. Localization and morphology assessment of these small objects over time is then crucial to provide valuable information for quantitative traffic analysis. In this study, we have focused on the Rab6 protein as a typical intracellular membrane-associated protein. Rab6 is known to promote vesicle trafficking from Golgi to Endoplasmic Reticulum or to plasma membrane. In our study, micro-fabricated patterns have been used to enforce cells to have circular or crossbow normalized shape. Micro-patterns impose constraints on the cytoskeleton and the location of organelles (e.g. Golgi apparatus) is thus better controlled. These micro-patterns also influence the spatial distribution of Rab6 transport carriers. However, the direct influence of the micro-patterns on the spatial dissemination of these trafficking vesicles has so far not been completely characterized. In this work, we have considered a statistical Bayesian approach in the framework of conditional random fields (CRF) for background estimation and vesicle segmentation . Within this approach, we have designed a robust detection measure for fluorescence microscopy based on the distribution of neighbor patch similarity. We formulate the vesicle segmentation and background estimation as a global energy minimization problem. An iterative scheme to jointly segment vesicles and background is proposed for 2D-3D fluorescence image sequences. We have conducted a quantitative comparison with state-of-the-art methods on a large set of synthetic image sequences with a cluttered time-varying background and achieved a quantitative validation of the vesicle segmentation method on 2D and 3D micro-patterned cells expressing GFP-Rab6.
Collaborators: Jean Salamero (UMR 144 CNRS-Institut Curie, STED team and PICT-IBiSA)
Jérôme Boulanger (UMR 144 CNRS-Institut Curie, STED team)
.
Rab6 proteins are trafficking from the Golgi apparatus at the cell center to Endoplasmic Reticulum or to plasma membrane located at the periphery of the cell. The cell shape influences Rab6 trafficking but no study has ever quantified the effect of the cell shape on the trafficking orientation. In this study , we compare Rab6 trafficking orientation constrained by two different micropatterns (circular and crossbow-shaped cells) from fluorescence video-microscopy. Object/background separation is first applied to 3D+T image sequences to extract Rab6 spatio-temporal coordinates. The bandwidth of the von Mises kernel is automatically estimated using the rule of thumb and leads to two different densities for the two different micropatterns. We propose to quantitatively compare these densities by computing the Wilcoxon rank sum paired test between inter- and intra-micropattern distances. We considered the circular earth mover’s distance (also known as the Wasserstein metric) to compare traffic densities. Our quantitative study on micro-patterned cells concludes that the Rab6 transport carriers destinations concentrate at the three corner points of the crossbow-shaped cells corresponding to the main adhesion sites, while the vesicle destination distribution is somewhat uniform for circular-shaped cells.
Collaborators: Jean Salamero (UMR 144 CNRS-Institut Curie, STED team and PICT-IBiSA)
Jérôme Boulanger (UMR 144 CNRS-Institut Curie, STED team)
.
Accurately detecting subcellular particles in fluorescence microscopy is of primary interest for further quantitative analyses such as counting, tracking or classification. Our primary goal was to segment vesicles in fluorescence microscopy images. In we proposed a first spot detection method with automatic scale selection. We have now dramatically improved the precision of the scale selection step, yielding to a more reliable detection of the spots . The method relies on a Laplacian of Gaussian (LoG) filter to first enhance the spots while reducing noise. To obtain good detection results, the scale of the Gaussian filter must be precisely set, according to the spots size . In order to cope with very small spots, we rely on the discrete analog of the Gaussian filter , instead of the previously used sampled Gaussian filter. With this filter, we can find the optimal Gaussian scale with an arbitrary precision by minimizing a statistical criterion. We have introduced two criteria for this purpose and compared them. Once the optimal scale is selected, we threshold the lowest values of the LoG-filtered image, which correspond to spots. To cope with inhomogeneous background, thresholding must be adapted to local statistics so that a single probability of false alarm (PFA) setting can be defined for the whole image or even the collection of images to be processed. In short, we automatically infer from image data the optimal parameters usually left to the user guidance in other methods, that is, spot scale and detection threshold. We have carried out an extensive comparative evaluation, which demonstrates that our new scale selection approach improves detection performances, and that our spot detection method outperforms state-of-the-art detectors .
Collaborators: Jean Salamero (UMR 144 CNRS-Institut Curie, STED team and PICT-IBiSA)
Jérôme Boulanger (UMR 144 CNRS-Institut Curie, STED team)
.
Characterizing the spatial repartition of interacting moving proteins is a fundamental step for co-localization and co-expression. Based on the segmentation algorithm , , this challenge amounts to characterizing the repartition or spatial distribution of spots (see Fig. ). This is part of the more general statistical analysis of random geometrical objects, and in particular of random points. Gibbs models form a large class of point process models, that can be used to characterize either complete randomness or attraction or repulsion between points depending on the Gibbs potential at hand.
First in , we focused on infinite range potentials that include the most famous interaction potential arising from statistical physics, namely the Lennard Jones potential. To fit this kind of models to a dataset, the standard inference methods are not applicable. We introduced in a modification of the pseudolikelihood method, with a specific border correction, and we prove that this provides consistent and asymptotically normal estimators. Second, in , we studied an alternative class of models, the determinantal point processes (DPP). They are designed to model repulsion between points and are thus adapted to regular point patterns. These models are becoming very popular in the spatial statistics community due to many appealing properties. We quantified the possible repulsiveness that a DPP can model . In particular, we determined the most repulsive stationary DPP. We finally introduced new parametric families of DPPs that cover a large range of DPPs, from the homogeneous Poisson process (for no interaction) to the most repulsive DPP.
An application of these models to the problem of co-localization between proteins is part of an on-going project. In each protein, the set of vesicles is modeled by a union of random balls, possibly overlapping, and a Gibbs interaction is introduced to take into account the possible interaction in the location of vesicles between two proteins. Our first concern is to test whether the two proteins actually interact, i.e. co-localization occurs, or in other words whether the Gibbs interaction is empty or not. If there is co-localization, the further step is to characterize it through the estimation of the strength of the Gibbs interaction.
Collaborators: Christophe Ange Napoléon Biscio (LMJL, University of Nantes)
Jean-François Coeurjolly (Laboratoire Jean Kuntzmann, Grenoble Alpes University)
.
Assessing the dynamics of plasma membrane diffusion processes in live cell fluorescence microscopy is of paramount interest to understand cell mechanisms. We investigated methods to detect vesicle fusion events, and estimate the associated diffusion coefficients in TIRFM image sequences . In contrast to classical approaches, a diffusion coefficient is locally estimated for each detected fusing vesicle. We first detect the membrane fusion events and then select the diffusion configurations among them with a correlation test. To estimate the diffusion coefficient, a geometric model is fitted to the detected spot directly in the 2D+T subvolume. This recent estimation approach produced more satisfying results when compared to . Diffusion events are reliably recognized, and the diffusion coefficient is accurately estimated for each diffusion event. This work will be integrated in a broader study, spanning from transport phase to membrane fusion, and non-diffusion events will be analyzed.
Collaborators: Jean Salamero (UMR 144 CNRS-Institut Curie, STED team and PICT-IBiSA)
Jérôme Boulanger (UMR 144 CNRS-Institut Curie, STED team)
.
Automatic analysis of the dynamic content in fluorescence video-microscopy is crucial for understanding molecular mechanisms involved in cell functions. We have proposed an original approach for analyzing particle trafficking in these sequences. Instead of individually tracking every particle, we only locally count particles crossing boarders between regions over time and minimize a global energy function. Three methods to determine the particle flow have been considered. We have conducted comparative experiments on synthetic and real fluorescence image sequences. We have shown that adding a sparsity constraint on the number of detected events allows us to reduce the number of false alarms. Compared to usual tracking methods, our approach is simpler and the results are very stable. This estimation method needs the adjustment of only two parameters. (see Fig. ).
Collaborators: Jean Salamero (UMR 144 CNRS-Institut Curie, STED team and PICT-IBiSA)
Jérôme Boulanger (UMR 144 CNRS-Institut Curie, STED team)
.
In this study, we are interested in the influence of the mechanical properties of astral microtubules in the centering mechanisms of the mitotic spindle, giving it a robust positioning. In their previous studies, the CeDRE group (IGDR Rennes) identified two subpopulations of astral microtubules that either push or pull the cell cortex. To better understand these mechanisms, image sequences are acquired at the cortex level where extremities of astral microtubules come to exert forces. In order to characterize the two subpopulations of astral microtubules during the mitosis in the unicellular embryos of C. Elegans, life span, that is the period during which the microtubule is touching the cell cortex, for every single microtubule has to be measured. A short life span corresponds to a pulling force while a longer life span corresponds to a pushing force. Detecting and tracking microtubules at the cell cortex has to be done to collect these measures. As the signal-to-noise ratio is low, a denoising step is needed to detect the microtubule extremities. Several detection methods were tested but we need to further investigate this step to find the most suited methods for this particular application. Finally, the U-track algorithm is applied to track the microtubules extremities to measure their life span.
Collaborators: Jacques Pécréaux (CeDRE group, IGDR Rennes, CNRS UMR 6290)
Hélène Bouvrais (CeDRE group, IGDR Rennes, CNRS UMR 6290)
.
Tissue core de-arraying is one of the most important steps in tissue microarray (TMA) image analysis. A very first task of TMA (Tissue MicroArray) image analysis is to accurately localize spots (separate tissue core) representing arrays of
Collaborator: Vincent Paveau (Innopys company)
.
The design of data costs is one of the main research issue for variational optical flow estimation. The aim is to improve discriminative power by integrating appropriate neighborhood information, while preserving computational efficiency. Most previous works define features on patches with predefined sizes and shapes, or filter pixelwise costs with fixed filtering parameters. We proposed a novel approach estimating spatially varying parameters of filters used to define the data term . More specifically, our model considers Gaussian filtering of the pixelwise brightness constancy equation and imposes smoothness constraints on motion and convolution filter size (bandwith). The energy encoding these assumptions is alternatively minimized over flow field and the spatially varying bandwidth in a variational framework. Experimental results on the Middlebury database demonstrated clear improvements yielded by our method over the spatially constant case of (see Fig. ).
Collaborator: Denis Fortun (UMR 144 CNRS-Institut Curie, STED team, Paris)
(EPFL, Lausanne, Switzerland)
.
Assessing crowd behaviors from videos is a difficult task while of interest in many applications. We have defined a novel approach which identifies from two successive frames only, crowd behaviors expressed by simple image motion patterns. It relies on the estimation of a collection of sub-affine motion models in the image, a local motion classification based on a penalized likelihood criterion, and a regularization stage involving inhibition and reinforcement factors . The apparent motion in the image of a group of people is assumed to be locally represented by one of the three following motion types: translation, scaling or rotation. The three motion models are computed in a collection of predefined windows with the robust estimation method . At every point, the right motion model is selected owing to the corrected (for small sample size) Akaike information criterion (AICc). To classify the local motion type, the three motion models are further subdivided into a total of eight crowd motion classes. Indeed, scaling refers either to gathering (Convergence) or dispersing people (Divergence). Rotation can be either Clockwise or Counterclockwise. Since our classification scheme is view-based, four image-related translation directions are distinguished: North, West, South, East. Then, to get the final crowd classification, a regularization step is performed, based on a decision tree and involving inhibition for opposed classes such as convergence and divergence. We have also developed an original and simple method for recovering the dominant paths followed by people in the observed scene. It involves the introduction of local paths determined from the space-time average of the parametric motion subfields selected in image blocks. Starting from one given block in the image, we straightforwardly reconstruct a global path by concatenating the local paths from block to block. Experiments on synthetic and real scenes have demonstrated the performance of our method, both for motion classification and principal paths recovery.
.
We have developed a new and generic method to detect and localize abnormal events in videos of crowd scenes. The algorithm consists first in determining the flow vector and crowd motion class for every moving pixel from a set of affine motion models estimated on a collection of windows. Then, the observed scene is subdivided in blocks to compute crowd motion class histograms weighted by the motion vector magnitudes. A very simple training step enables to get the reference histograms per block accounting for the normal behaviours. For each block, we can automatically set by means of statistical arguments the threshold on the distance between the histogram in the current image and the reference histogram that decides the presence of an abnormal event in that block. Results of extensive experimentation on different types of anomaly datasets show that our method is competitive with respect to methods relying on far more elaborated models on both appearance and motion and thus involving a significant learning stage. It outperforms any other existing purely motion-based anomaly localization method.
In collaboration with Magellium company and Institut Gustave Roussy, Innopsys plans to develop new image analysis software to be included in the INGRID platform developed by Megellium company. New statistical methods and algorithms will be investigated by serpico for:
segmentation and detection of deformable cell contours and cell nuclei in 2D fluorescence tissue microarray images;
deconvolution and superresolution of fluorescence microarray imaging.
The three-year contract supports the PhD thesis of Hoai Nam Nguyen (2013-2016).
ENSAI-CREST: Statistical methods and models for image registration, Vincent Briane PhD thesis is co-funded by Inria and ENSAI-CREST and co-supervised by Myriam Vimond (ENSAI-CREST)
BioGenOuest: Advisory committee of the Biogenouest engineer S. Prigent in charge of the organization of image processing services for Biogenouest bio-imaging facilities.
.
Biofilms are composed of spatially organized microorganisms (possibly including pathogens) embedded in an extracellular polymeric matrix. A direct time-lapse confocal microscopic technique was recently developed to enable the real-time visualization of biocide activity within the biofilm. It can provide information on the dynamics of biocide action in the biofilm and the spatial heterogeneity of bacteria-related susceptibilities that are crucial for a better understanding of biofilm resistance mechanisms. The approach is here to characterize the spatial and temporal exploration of the biofilm by microorganisms.
In this project, serpico develop methods and software for the computation of mean velocity as well as other descriptors of swimmers bacteria dynamics inside biofilm image sequences. We investigate spatio-temporal features and descriptors for comparison, classification, indexing and retrieval.
Funding: ANR, duration: 24 months |
Partners: INRA, AgroParisTech, Naturatech company |
.
The goal of the project is to build a distributed coordinated French infrastructure for photonic and electronic cellular bioimaging dedicated to innovation, training and technology transfer. High computing capacities are needed to exhaustively analyse image flows. We address the following problems: i) exhaustive analysis of bioimaging data sets; ii) deciphering of key steps of biological mechanisms at organ, tissular, cellular and molecular levels through the systematic use of time-lapse 3D microscopy and image processing methods; iii) storage and indexing of extracted and associated data and metadata through an intelligent data management system. serpico is co-head of the IPDM (Image Processing and Data Management) node of the FBI network composed of 6 nodes.
Funding: Investissement d'Avenir - Infrastructures Nationales en Biologie et Santé, ANR (2011-2016) |
Partners: CNRS, Institut Jacques Monod, Institut Pasteur, Institut Curie, ENS Ulm, Ecole Polytechnique, INRA, INSERM |
ESFRI Euro-BioImaging initiative: serpico participates in the ESFRI Euro-BioImaging project, one of the four new biomedical science projects in the roadmap of the European Strategic Forum on Research Infrastructures (ESFRI). The mission of Euro-BioImaging is to provide access, service and training to state-of-the-art imaging technologies and foster the cooperation and networking at the national and European level including multidisciplinary scientists, industry regional, national and European authorities. serpico also participates in the French counterpart, the so-called “France-BioImaging” (FBI) network which gathers several outstanding cellular imaging centers (microscopy, spectroscopy, probe engineering and signal processing) as described in Section .
Collaboration with UT Southwestern Medical Center, Dallas (TX), Prof. G. Danuser, on object tracking in video-microscopy.
Collaboration with University of California - San Francisco (USA), J. Sedat and D. Agard, on image deconvolution in light microscopy.
Collaboration with Imaging Systems Lab, Department of Electrical Engineering, Indian Institute of Science, Bangalore, India (Prof. Muthuvel Arigovindan) on image deconvolution in fluorescence imaging.
Deepak George Skariah: Internship, Imaging Systems Lab, Department of Electrical Engineering, Indian Institute of Science, Bangalore, India.
Charles Kervrann was member of the organizing committee of the international Quantitative BioImaging 2015 (QBI) conference, Institut Pasteur, January 2015 (180 participants).
Perrine Paul-Gilloteaux was member of the organizing committees:
Second edition of the European Bio-Image analyst Symposium: EUBIAS Taggathon workshop organized on the 8th and 9th of December 2014 for the creation of a webtool biii.info referencing bio image analysis workflows (20 participants invited over 2 days), and of the EuBIAS community meeting for bio-image analysts on the 5th and 6th of January (118 participants).
Microscopy school MiFoBio'2014 (Seignosse, October 2014): Organization of advanced modules and round tables with Alain Dieterlen (Laboratoire MIPS-uha Mulhouse).
Patrick Bouthemy was member of the “commité de pilotage” for the organization of RFIA'2014.
Frédéric Lavancier is head of the workshop “Spatio-temporal models and statistics”, IRMAR University of Rennes 1, LMJL University of Nantes, ENSAI, University of Rennes 2, INRA Rennes, Inria Rennes.
Charles Kervrann: Associated editor for the conference isbi'2015, PC member for isbi'2014, member of scientific committee of “Journées d'Imagerie Optique Non-Conventionnelle” (JIONC'2014).
Patrick Bouthemy: Area chair for the conference icip’2014, PC member for icpram'2014.
Thierry Pécot: Associated editor for the conference isbi'2015.
Charles Kervrann: reviewer for icip'2014, icassp'2014, ssvm'2015, rfia'2014, eusipco'2014, emmcprv'2015.
Patrick Bouthemy: reviewer for eccv’2014, isbi'2014, isbi'2015, rfia'2014.
Perrine Paul-Gilloteaux: reviewer for isbi'2015, expert for the project evaluation in the framework of France-Brazil cooperation COFECUB (Comité Français d’Évaluation de la Coopération Scientifique et Universitaire avec le Brésil).
Charles Kervrann is guest editor of the special issue entitled “Advanced Signal Processing in Microscopy and Cell Imaging” of the IEEE Selected Topics in Signal Processing Journal (publication in 2015).
Charles Kervrann: reviewer in 2014 for Bioinformatics, Digital Signal Processing, IEEE Transactions on Image Processing, Journal Mathematical Imaging and Vision, Medical Image Analysis, Traitement du Signal.
Patrick Bouthemy: reviewer in 2014 for IEEE Transactions on Image Processing, IEEE Transactions on Medical Imaging, Medical Image Analysis.
Frédéric Lavancier : reviewer since September 2014 for Bernoulli, Climate Dynamics, Metrika, Statistics and Probability Letters.
Perrine Paul-Gilloteaux: reviewer for PLoS One.
Charles Kervrann was invited to give a talk entitled “Tracking and motion analysis in fluorescence microscopy” at the microscopy school MiFoBio'2014 (Seignosse, October 2014), “Localization, classification and estimation of membrane dynamics in TIRFM image sequences” at the Quantitative BioImaging 2015 (QBI 2015) (Institut Pasteur, Paris January 2015), “Patch-based methods and algorithms for breaking the signal-to-noise ratio in fluorescence microscopy” at the Max-Planck Institute Munich (Biochemistry Department, Martinsried, Germany, December 2014), “Conditional Random Fields for tubulin-microtubule segmentation in cryo-electron tomography” at the special session on Electron Microscopy, Image Processing Problems and Applications in Biology: From Structure to Dynamics of the IEEE International Conference on Image Processing (ICIP'2014) (Paris, October 2014), “Approximate Bayesian computation, stochastic algorithms and non-local means for complex noise models” at the special session on Photon-Limited Image Reconstruction of the IEEE International Conference on Image Processing (ICIP'2014) (Paris, October 2014).
Frédéric Lavancier was invited to give a talk entitled “Inference for union of interacting discs” at JSTAR, Rennes, October 2014.
Thierry Pécot was invited to give a talk entitled “Space-time representation imaging and cellular dynamics of molecular complexes and Mobyle platform”, at EuBIAS meeting, Institut Curie, Paris in January 2015 and at the BioGenouest meeting - Imaging Platforms on February 2014.
Thierry Pécot and Charles Kervrann organized a practical on “Image processing methods for motion analysis of particles” for microscopy school MiFoBio'2014 (Seignosse, October 2014).
Perrine Paul-Gilloteaux was invited to give a talk entitled “Microscopy images life cycle management in biology: knowledge mining from an image database” at the INRA seminar “Open Data” on the Curie image data base including development realized in collaboration with SERPICO for automatic processing on clusters from the database (Saint Martin des Combes, December 2014).
Perrine Paul-Gilloteaux (with F. Waharte) organized a practical on “Molecular dynamics in microscopy based on fluorescence image correlation” for microscopy school MiFoBio'2014 (Seignosse, October 2014).
Charles Kervrann:
Member of the IEEE BISP “Biomedical Image and Signal Processing” committee.
Member of executive board of the GdR MIV (2588 - Microscopie Fonctionnelle du Vivant) CNRS,
Member of the scientific committee of the Interdisciplinary MiFoBio School CNRS (http://
Member of the executive board of the project committee of the Inria Rennes - Bretagne Atlantique centre.
Member of the Scientific Council of the INRA Rennes Research Centre.
Patrick Bouthemy:
Head of Excellence Lab CominLabs since April 2014.
Deputy member of the board of directors and member of the Selection and Validation Committee of the Images & Réseaux competitivity cluster.
Deputy member of the board of directors of IRT (Technological Research Institute) B-com.
President of AFRIF (Association Française pour la Reconnaissance et l’Interprétation des Formes) and member of the board of the GRETSI (Groupement de Recherche en Traitement du Signal et des Images).
Charles Kervrann:
Master: From BioImage Processing to BioImage Informatics, 5 hours, coordinator of the module (30 hours), Master 2 Research IRIV, Telecom-Physique Strasbourg & University of Strasbourg.
Master: Geometric Modeling for Shapes and Images, 6 hours, Master 2 Research SISEA, University of Rennes 1.
Engineer Degree and Master 2 Statistics and Mathematics: Statistical Models and Image Analysis, 37 hours + 15 hours (TP, Hoai Nam Nguyen), 3rd year, Ecole Nationale de la Statistique et de l'Analyse de l'Information (ENSAI), Rennes.
Patrick Bouthemy:
Master: Analysis of Image Sequences, 18 hours, Master 2 Research SISEA, ISTIC & University of Rennes 1.
Master: Video Indexing, 9 hours, Master 2 Research Computer Science, ISTIC & University of Rennes 1.
Engineer Degree and Master 2 Research IRIV: Motion Analysis, 12 hours, Telecom-Physique Strasbourg & University of Strasbourg.
PhD defense: Philippe Roudot (May 2014), Lifetime estimation of moving vesicles in FLIM microscopy, started in October 2010, supervised by Charles Kervrann and Francois Waharte (UMR 144 CNRS-Institut Curie, STED team) (ese ).
PhD defense: Denis Fortun (July 2014), Optical flow computing, aggregation methods and statistical methods: application to time-lapse fluorescence microscopy, started in October 2010, supervised by Charles Kervrann and Patrick Bouthemy (see ).
PhD in progress: Antoine Basset, Event detection and recognition in video-microscopy and applications in cell biology, started in October 2012, supervised by Patrick Bouthemy and Charles Kervrann in collaboration with Jérôme Boulanger and Jean Salamero (UMR 144 CNRS-Institut Curie, STED team).
PhD in progress: Hoai Nam Nguyen, Methods and algorithms for tissue microarrays image analysis, started in October 2013, supervised by Charles Kervrann and Vincent Paveau (Innopsys company).
PhD in progress: Vincent Briane, Statistical methods and models for image registration, started in October 2014, supervised by Charles Kervrann and Myriam Vimond (ENSAI-CREST)
PhD in progress: Bertha Mayela Toledo Acosta, Methods and algorithms for 3D image registration, started in October 2014, supervised by Patrick Bouthemy.
PhD in progress : Christophe Biscio, Statistical aspects of determinantal point processes, started in October 2012, supervised by Frédéric Lavancier.
Member of a jury for the recruitment of an assistant professor: University of Paris Descartes (Section CNU 26) [C. Kervrann].
Referee of Habilitation thesis: B.M. Jedynak (University of Lille) [Patrick Bouthemy].
Chair of Habilitation jury: O. Le Meur (University of Rennes 1) [Patrick Bouthemy].
Referee of PhD thesis: F.Z. Benamar (University of Picardie and University Mohammed V Agdal Rabat, supervised by D. Aboutajdine and E.M. Mouaddib [Patrick Bouthemy], P R. Kumar (University of Nice Sophia-Antipolis, supervised by M. Thonnat and G. Charpiat) [P. Bouthemy], S. Rigaud (Universiy of Pierre et Marie Curie, supervised by D. Rococeanu and L.J. Hwee) [P. Bouthemy], M. Maggioni (Tampere University of Technology, supervised by A. Foi) [C. Kervrann], J. Gul-Mohammed (Universiy of Pierre et Marie Curie, supervised by T. Boudier) [C. Kervrann], G. Trigui (University of Paris Sud, supervised by B. Dubreucq and A. Trubuil) [C. Kervrann].
Chair of PhD thesis juries: P.-H. Conze (Insa Rennes, supervised by L. Morin and P. Robert) [C. Kervrann], B. Delabarre (PhD, committee president, University of Rennes 1, supervised by E. Marchand) [P. Bouthemy], N. Morsli (Université de Grenoble, supervised by J.-F. Coeurjolly) [F. Lavancier]