(Fluorescence Lifetime Microscopy Imaging): imaging of fluorescent molecule lifetimes.
(Photo-Activated Localization Microscopy): high-resolution microscopy using stochastic photo-activation of fluorophores and adjustment of point spread functions .
(Structured Illumination Microscopy): high-resolution light microscopy using structured patterns and interference analysis .
(Total Internal Reflectance): 2D optical microscopy using evanescent waves and total reflectance .
(Cryo-Electron Tomography): 3D representation of sub-cellular and molecular objects of 5-20 nanometres, frozen at very low temperatures, from 2D projections using a transmission electron microscope.
Light microscopy, especially fluorescence microscopy, has taken a prominent role in life science research due to its ability to investigate the 3D interior of cells and organisms. It enables to visualize, in vitro and in vivo, particular biomolecules and proteins (gene expression) with high specificity through fluorescent labeling (GFP - Green Fluorescence Protein probes) both at the microscopic and nanoscopic scales. Nevertheless, the mechanisms of life are very complex and driven by multimolecular interactions: mitotic spindle, cell signaling complexes, intracellular transport, cell morphogenesis and motility... A dynamical quantitative and integrated description of molecular interactions and coordination within macromolecular complexes at different scales appears essential today for the global understanding of live mechanisms. A long-term research consists in inferring the relationships between the dynamics of macromolecules and their functions. This constitutes one of the challenges of modern biology. The proposed mathematical models and algorithms are mainly developed to identify molecular processes in fundamental biology but they have also a strong potential for applications in biotechnology and medicine: disease diagnosis, detection of genomic instabilities, deterioration of cell cycle, epigenetic mechanisms and cancer prevention.
Facing the amount of information provided by high-throughput multidimensional microscopy, the serpico team investigates computational and statistical models to better elucidate the role of specific proteins inside their multiprotein complexes and to help to decipher the dynamic coordination and organization of molecular complexes at the single cell level. We investigate image processing methods, mathematical models, and algorithms to build an integrated imaging approach that bridges the resolution gaps between the molecule and the whole cell, in space and time . We address the following topics:
Image superresolution/image denoising required to preserve cell integrity (photo-toxicity versus exposure time) and image analysis in multidimensional microscopy;
Motion analysis and computation of molecule trajectories in live-cell imaging to to study molecular interactions in space and time);
Computational simulation and modelling of molecule trafficking at different spatial and temporal scales (e.g. biophysical model assimilation for dynamic representation in video-microscopy and prediction in biology).
We focus on the cellular and molecular mechanisms involved in membrane transport and trafficking at the scale of a single cell.
In most cases, modern microscopy in biology is characterized by a large number of dimensions that fits perfectly with the complexity of biological features: two or three spatial dimensions, at macro to nano-scales, and one temporal dimension, sometimes spectrally defined and often corresponding to one particular bio-molecular species. Dynamic microscopy is also characterized by the nature of the observable objects (cells, organelles, single molecules, ...), eventually by the large number of small size and mobile elements (chromosomes, vesicles, ...), by the complexity of the dynamic processes involving many entities or group of entities sometimes interacting, by particular phenomena of coalescence often linked to image resolution problems, finally by the association, dissociation, recomposition or constitution of those entities (such as membrane fusion and budding). Thus, the corpus of data to be considered for a comparative analysis of multiple image series acquisitions is massive (up to few GigaBytes per hour). Therefore, it becomes necessary to facilitate and rationalize the production of those multidimensional data, to improve post acquisition analysis (i.e. image processing) which are limiting factors in front of the data, and to favor the organization and the interpretation of the information associated to this data corpus. It motivates and requires innovative mathematical tools and concepts: data fusion, image registration, superresolution, data mining, life dynamics modelling, ...
In collaboration with UMR 144, CNRS-Institut Curie (“Subcellular Structure and Cellular Dynamics” Unit) and PICT IBiSA (Cell and Tissue Imaging Facilities), the members of the serpico team participate in several projects (PhD and post-doc supervision, contracts...) with biologists in the field of cell biology and microscopy. We have promoted and designed non-parametric methods since prior knowledge cannot be easily taken into account for extracting unattended but desired information from image data. We have proposed user-friendly algorithms for processing 3D or 4D data.
The scientific projects of the serpico team are complementary to the other on-going and planned projects of the UMR 144, CNRS-Institut Curie Unit. A subset of projects is related to instrumentation in electronic and photonic microscopy (PICT IBiSA platform) including computational aspects on the reconstruction and enhancement of images related to sub-diffraction light microscopy and multimodal approaches. Our projects rely partially on the results and advances of these instrumental projects and a positive synergy is foreseen.
Serpico is an Inria Team-Project from July 2013.
New computing (7 nodes with 2 CPU x 8 cores, 64 GigaBytes and 128 GigaBytes of RAM including in the IGRIDA computing grid) and storage (207 TeraBytes controlled by a server with 2 CPU x 6 cores, 32 GigaBytes of RAM) facilities dedicated to calculations and algorithm runs.
Finalist for Best Student Paper Award : D. Fortun et al. Aggregation of patch-based estimations for illumination-invariant optical flow in live cell imaging. IEEE Int. Symp. Biomedical Imaging (ISBI'13), San-Francisco CA, April 2013.
Many live-cell fluorescence imaging experiments are limited in time to prevent phototoxicity and photobleaching. The amount of light and time required to observe entire cell divisions can generate biological artifacts. In order to produce images compatible with the dynamic processes in living cells as seen in video-microscopy, we study the potential of denoising, superresolution, tracking, and motion analysis methods in the Bayesian and the robust statistics framework to extract information and to improve image resolution while preserving cell integrity.
In this area, we have already demonstrated that image denoising allows images to be taken more frequently or over a longer period of time, while preserving image quality . The major advantage of the nD-Safir software is to acquire images at very low SNR while recovering denoised 2D+T(ime) and 3D+T(ime) images , , , . This approach has been successfully applied to wide-field, spinning-disk confocal microscopy , TIRF , fast live imaging and 3D-PALM using the OMX system in collaboration with J. Sedat and M. Gustafsson at UCSF . The nD-Safir software (see Section ) has been licensed to a large set of laboratories over the world (see Figure ). New information restoration and image denoising methods are currently investigated to make SIM imaging compatible with the imaging of molecular dynamics in live cells. Unlike other optical sub-diffraction limited techniques (e.g. STED , PALM ) SIM has the strong advantage of versatility when considering the photo-physical properties of the fluorescent probes . Such developments are also required to be compatible with “high-throughput microscopy” since several hundreds of cells are observed at the same time and the exposure times are typically reduced.
The main challenge is to detect and track xFP tags with high precision in movies representing several GigaBytes of image data. The data are most often collected and processed automatically to generate information on partial or complete trajectories. Accordingly, we address both the methodological and computational issues involved in object detection and multiple objects tracking in order to better quantify motion in cell biology. Classical tracking methods have limitations as the number of objects and clutter increase. It is necessary to correctly associate measurements with tracked objects, i.e. to solve the difficult data association problem . Data association even combined with sophisticated particle filtering techniques or matching techniques is problematic when tracking several hundreds of similar objects with variable velocities. Developing new optical flow and robust tracking methods and models in this area is then very stimulating since the problems we have to solve are really challenging and new for applied mathematics. In motion analysis, the goal is to formulate the problem of optical flow estimations in ways that take physical causes of brightness constancy violations into account , . The interpretation of computed flow fields enables to provide spatio-temporal signatures of particular dynamic processes (e.g. Brownian and directed motion) and could help to complete the traffic modelling.
Protein complexes in living cells undergo multiple states of local concentration or dissociation, sometimes associated with diffusion processes. These events can be observed at the plasma membrane with TIRF microscopy. The difficulty arises when it becomes necessary to distinguish continuous motions due to trafficking from sudden events due to molecule concentrations or their dissociations. Typically, plasma membrane vesicle docking, membrane coat constitution or vesicle endocytosis are related to these issues.
Several approaches can be considered for the automatic detection of appearing and vanishing particles (or spots) in wide-field and TIRF microscopy images (see Fig. ). Ideally this could be performed by tracking all the vesicles contained in the cell , . Among the methods proposed to detect particles in microscopy images , , none is dedicated to the detection of a small number of particles appearing or disappearing suddenly between two time steps. Our way of handling small blob appearances/dis-appearances originates from the observation that two successive images are redundant and that occlusions correspond to blobs in one image which cannot be reconstructed from the other image (see also ). Furthermore, recognizing dynamic protein behaviors in live cell fluorescence microscopy is of paramount importance to understand cell mechanisms. In our studies, it is challenging to classify intermingled dynamics of vesicular movements, docking/tethering, and ultimately, plasma membrane fusion of vesicles that leads to membrane diffusion or exocytosis of cargo proteins. Our aim is then to model, detect, estimate and classify subcellular dynamic events in TIRF microscopy image sequences. We investigate methods that exploits space-time information extracted from a couple of successive images to classify several types of motion (directed, diffusive (or Brownian) and confined motion) or compound motion.
Mathematical biology is a field in expansion, which has evolved into
various branches and paradigms to address problems at various scales
ranging from ecology to molecular structures. Nowadays, system
biology , aims at
modelling systems as a whole in an integrative perspective instead of
focusing on independent biophysical processes. One of the goals of
these approaches is the cell in silico as investigated at Harvard
Medical School (http://
In this area, we focus on the analysis of transport intermediates (vesicles) that deliver cellular components to appropriate places within cells. We have already investigated the concept of Network Tomography (NT) mainly developed for internet traffic estimation. The idea is to determine mean traffic intensities based on statistics accumulated over a period of time. The measurements are usually the number of vesicles detected at each destination region receiver. The NT concept has been investigated also for simulation since it can be used to statistically mimic the contents of real traffic image sequences. In the future, we plan to incorporate more prior knowledge on dynamics to improve representation. An important challenge is to correlate stochastic, dynamical, one-dimensional in silico models studied at the nano-scale in biophysics, to 3D images acquired in vivo at the scale of few hundred nanometers. A difficulty is related to the scale change and statistical aggregation problems (in time and space) have to be handled.
In the past recent years, research carried at UMR 144, CNRS-Institut Curie contributed to a better understanding of the intracellular compartimentation of specialized model cells such as melanocytes and Langerhans cells, the components and structural events involved in the biogenesis of their specialized organelles: melanosomes and Birbeck granules, respectively. These studies have started to highlight:
multiple sorting and structural events involved in the biogenesis of these organelles;
complexity of the endo-melanosomal network of these highly specialized cells;
complex molecular architecture organizing and coordinating their dynamics;
intracellular transport steps affected in genetic diseases, among which the Hermansky Pudlak syndrome (HPS) or involved in viral infection (HIV and Langerin in Langerhans cells).
In this context, the central aim of serpico is to understand how the different machineries of molecular components involved are interconnected and coordinated to generate such specialized structures. We need to address the following topics:
developing new bioimaging approaches to observe and statistically analyze such coordinated dynamics in live material;
correlating this statistically relevant spatiotemporal organization of protein networks with the biological architectures and within the overall biological environment as seen at the ultrastructural level;
modeling intracellular transport of those reference biological complex systems and proposing new experimental plans in an iterative and virtuous circle;
managing and analyzing the workflow of image data obtained along different multidimensional microscopy modalities.
These studies are essential to unravel the complexity of the endomembrane system and how different machineries evolve together (e.g. see Figs. -). They help to control cell organization and function at different scales through an integrative workflow of methodological and technological developments.
At long term, these studies will shed light on the cellular and molecular mechanisms underlying antigen presentation, viral infection or defense mechanisms, skin pigmentation, the pathogenesis of hereditary genetic disorders (lysosomal diseases, immune disorders) and on the mechanisms underlying cell transformation. Our methodological goal is also to link dynamics information obtained through diffraction limited light microscopy, eventually at a time regime compatible with live cell imaging. The overview of ultrastructural organization will be achieved by complementary electron microscopical methods. Image visualization and quantitative analysis are of course important and essential issues in this context.
In cell and molecular biology , new challenges arise to acquire a complete and quantified view from the scale of a “single” cell to the scale of a multi-cellular structure, within the whole organism. In the near future, image analysis will be central to the successful use of optical microscopy in post-genomics biology. Nevertheless, one major difficulty lies in correlating and/or fusing multi-modalities, now routinely used in biology laboratories: optical imaging (spinning-disk confocal, TIRF, SIM, PALM, STED, FLIM-FRET, MP, SPIM/DSLM), ionic imaging (NanoSIMS), atomic force imaging (AFM) and electron imaging (Cryo-EM, Tomo EM).
Moreover, in the emerging era of high-throughput microscopy (biochemical screens, cell-based screening), systematic and accurate correlation and analysis of these data cannot be performed manually, since the image sequences are composed of several hundred of 3D stacks. Consequently, data to manipulate range from few to tens of TeraBytes. From the experimental perspective, molecular (drugs, RNA interference), mechanical (micro-patterning...), and optical (FRAP, photoactivation, optogenetic, ...) functional modulations allow one to quantify the importance of molecular linkage into macrocomplexes within a single cell. We are now able to limit shape variability between cells during an exposed period , . Consequently, efficient storage, fast retrieval and secure sharing of microscopy images are crucial challenges. Even with high-speed computers, the processing step will considerably slow down the whole analysis process.
We propose to address several important issues in this area and to adapt the proposed methodologies and algorithms to face a deluge of data. Our goal is also to participate to the technical specifications of an image database with a built-in query system to annotate, retrieve, process and integrate analysis from different imaging modalities. The combination of complementary skills (image processing and analysis software, image data management) will yield a full integration of the image and data life-cycle, from image acquisition and analysis, to statistical analysis and mathematical modeling in systems biology.
.
Motion2d: parametric motion model estimation
The Motion2D software written in c++ (APP deposit number: FR.001.520021.001.S.A.1998.000.21000 / release 1.3.11, January 2005) and java (plug-in ImageJ (http://
Free academic software distribution: Motion2D Free Edition is the version of Motion2D available for development of Free and Open Source software only. More information on Motion2D can be found at http://
On-line demo: Mobyle@SERPICO http://
Partner: Fabien Spindler (Inria Lagadic team).
ND-Safir and Fast2D-SAFIR: Image denoising software
The nD-Safir software (APP deposit number: IDDN.FR.001.190033.002.S.A.2007.000.21000 / new release 3.0 in 2013) written in c++, java and matlab, removes additive Gaussian and non-Gaussian noise in still 2D or 3D images or in 2D or 3D image sequences (without any motion computation) (see Figure ) . The method is unsupervised and is based on a pointwise selection of small image patches of fixed size (a data-driven adapted way) in spatial or space-time neighbourhood of each pixel (or voxel). The main idea is to modify each pixel (or voxel) using the weighted sum of intensities within an adaptive 2D or 3D (or 2D or 3D + time) neighbourhood and to use image patches to take into account complex spatial interactions. The neighbourhood size is selected at each spatial or space-time position according to a bias-variance criterion. The algorithm requires no tuning of control parameters (already calibrated with statistical arguments) and no library of image patches. The method has been applied to real noisy images (old photographs, jpeg-coded images, videos, ...) and is exploited in different biomedical application domains (time-lapse fluorescence microscopy, video-microscopy, mri imagery, x-ray imagery, ultrasound imagery, ...).
The fast-2d-safir software (APP deposit number: IDDN.FR.001.190033.001.S.A.2007.000.21000) written in c++ removes mixed Gaussian-Poisson noise in large 2D images, typically
On-line demo: Mobyle@SERPICO http:// |
Free download binaries: Binaries of the software nD-safir are freely and electronically distributed. Developed in standard C/C++ under Linux using the CImg library, it has been tested over several platforms such as Linux/Unix, Windows XP and Mac OS. |
Academic licence agreements: Institut Curie, CNRS, ENS Ulm, Oxford University, Weizmann Institute, UCSF San-Francisco, Harvard University, Berkeley University, Stanford University, Princeton University, Georgia-Tech, Kyoto UNiversity, IMCB Singapore ... |
Partners: J. Boulanger, J. Salamero (UMR 144, CNRS-Institut Curie), P. Elbau (RICAM Linz, Austria), J.B. Sibarita (UMR 5091, University of Bordeaux 2). |
HullkGround: Background subtraction by convex hull estimation
The HullkGround software (APP deposit number: IDDN.FR.001.400005.000.S.P.2009.000.21000) written in java (plug-in ImageJ, see Fig. )
decomposes a fluorescence microscopy image sequence into two dynamic
components: i/ an image sequence showing mobile objects; ii/ an image
sequence showing the slightly moving background. Each temporal signal
of the sequence is processed individually and analyzed with
computational geometry tools. The convex hull is estimated
automatically for each pixel and subtracted to the original
signal. The method is unsupervised, requires no parameter tuning and
is a simplified version of the
On-line demo: Mobyle@SERPICO http://
Partners: A. Chessel and J. Salamero (UMR 144, CNRS-Institut Curie)
.
TubuleJ: Straightening of microtubule cryo-EM projection views
The TubuleJ software (APP deposit number: IDDN.FR.001.240023.000.S.P.2011.000.21000) written in java (plug-in ImageJ) is devoted to the analysis of microtubules and helical structures in 2D
cryo-electron microscope images. The software straightens curved microtubule images by estimating automatically points locations on the microtubule axis. The estimation of microtubule principal axis relies on microtubule cylindrical shape analyzed in the Fourier domain. A user-friendly interface enables to filter straight fiber images by selecting manually the layer lines of interest in the Fourier
domain. This software can be used to generate a set of 2D projection views from a single microtubule projection view and a few parameters of this microtubule structure. These projection views are
then back projected, by using the imod plug-in (http://
On-line demo: see http://
Partners: S. Blestel and D. Chrétien (UMR 6290, CNRS, University of Rennes 1)
Cryo-Seg: Segmentation of tomograms in cryo-electron microscopy
The Cryo-Seg software written in c++ and java (plug-in mageJ) has been developed to detect microtubule structures and helical structures in 2D cryo-electron microscope images. Cryo-electron tomography allows 3D observation of biological specimens in their hydrated state. Segmentation is formulated as Maximum A Posteriori estimation problem and exploits image patches to take into account spatial contexts (Markov Random Fields). Because of the contrast anisotropy in the specimen thickness direction, the whole tomogram is segmented section by section, with an automatic update of reference patches. This algorithm has been evaluated on synthetic data and on cryo-electron tomograms of in vitro microtubules. On real data, this segmentation method extracts the most contrasted regions of microtubules, and 3D visualization is improved.
Partners: S. Blestel and D. Chrétien (UMR 6290, CNRS-University of Rennes 1)
.
The objective is to disseminate the distribution of serpico image processing software for biologist users:
Free binaries: software packages have been compiled for the main operating systems (Linux, MacOS, Windows) using CMake (see http://
Mobyle@SERPICO web portal: An on-line version of the image processing algorithms has been developped using the Mobyle framework (Institut Pasteur, see http://
ImageJ plug-ins: ImageJ (see http://
Institut Curie CID iManage database: Institut Curie is currently acquiring a commercial database system (CID iManage / Strand Avadis company) to store mass of data. The database can be searched via meta-data and includes menu selections that enable to run remote processing. We have integrated nD-Safir in the interface environment to allow the database users to denoise images easily.
Partners: C. Deltel (Inria Rennes SED) and Perrine Paul-Gilloteaux (UMR 144, PICT IBiSA, CNRS-Institut Curie)
.
In this study, we investigated a Maximum Likelihood (ML) framework for photon counting-based fluorescence lifetime estimation in Fluorescence Lifetime Imaging Microscopy (FLIM). Data collected at a given pixel consist of photon counts exponentially decreasing along the time and are assumed to follow Poisson statistics (see Fig. ). A careful analysis of the biophysical phenomenon and instrument models are used to derive a proper ML framework for lifetime estimation. Unlike usual pointwise approaches, a neighborhood-wise approach is proposed to take explicitly into account the spatial correlation of data . The application to real biological data allowed us to prove the spatial localisation of interactions, a new result which was not achievable with conventional methods. For future work, the main challenge is to extend the framework to deal with multi-exponential decay estimate and adaptive neighborhoods, a challenge we need to address for a large class of biological studies.
Partners: A. Chessel (University of Cambridge, UK), F. Waharte and J. Boulanger (UMR 144, PICT IBiSA, CNRS-Institut Curie)
.
Accurately detecting cellular structures in fluorescence microscopy is of primary interest for further quantitative analysis such as counting, tracking or classification. We aimed at segmenting vesicles in Total Internal Reflection Fluorescence (TIRF) microscopy images.
In this study, we have proposed an original and efficient method – called SLT-LoG – for vesicle segmentation with fewer parameters than the state-of-the-art methods. It exploits the Laplacian of Gaussian (LoG) of the images at several scales. Since the vesicles size is almost constant in space and time, a prominent mode is expected in the empirical distribution of the scales at which the minima of LoG values are detected. It precisely corresponds to the optimal sought scale. The vesicle segmentation map is then derived by thresholding the LoG values obtained at this optimal scale. To set the threshold, we assume that the values of the LoG locally follow a normal distribution (see Fig. ). For each point, we estimate the local mean and variance, and the threshold is deduced from a user-selected probability of false alarm.
We have evaluated our method on classical synthetic sequences for which the performances of many detection methods are available , . The comparative results on the dataset demonstrated that our method outperforms well-known unsupervised methods. We have also obtained very satisfactory results on real complex TIRF sequences.
Partners: Jean Salamero, J. Boulanger (UMR 144, PICT IBiSA, CNRS-Institut Curie)
.
Image analysis applied to fluorescence live cell microscopy has become a key tool in molecular biology since it enables to characterize biological processes in space and time at the subcellular level. In fluorescence microscopy imaging, the moving tagged structures of interest, such as vesicles, appear as bright spots over a static or non-static background. In this work, we consider the problem of vesicle segmentation and time-varying background estimation at the cellular scale. The main idea is to formulate the joint segmentation-estimation problem in the general Conditional Random Field (CRF) framework. Furthermore, segmentation of vesicles and background estimation are alternatively performed by energy minimization using a min cut-max flow algorithm. The proposed approach relies on a detection measure computed from intensity contrasts between neighboring patches in fluorescence microscopy images. We have demonstrated the competitiveness of the proposed method through an experimental comparison with state-of-the-art methods in fluorescence videomicroscopy, for single cell studies. We have also characterized the density of Rab6 transport carriers spatially dispersed at the cell periphery, for two different specific adhesion geometries.
Partners: Jean Salamero, J. Boulanger (UMR 144, PICT IBiSA, CNRS-Institut Curie)
.
Handling large displacements, motion details and occlusions all together remains an open issue for reliable computation of optical flow in a video sequence. Our recently investigated aggregation paradigm is an attractive approach supplying motion candidates at every pixel in a first step, and combining them in a second step to determine the global optical flow field . We experimentally demonstrate that simple and purely local parametric estimations combined with patch correspondences are sufficient to produce highly accurate motion candidates. Nevertheless, the performances are limited by the presence of large occlusion areas. Therefore we have proposed an exemplar-based occlusion handling scheme integrated in the two steps of the aggregation process. At the first stage, local motion candidates sets are extended at the detected occluded pixels with candidates from non-occluded pixels, and specific occlusions due to camera motion are handled by estimating the dominant motion in the image. Local occlusion cues are extracted from this first step. Then, we define a global energy function which cooperatively selects the best motion candidates for each point while recovering the occlusion areas and ensuring smoothness properties. Results on small displacement sequences are competitive with state-of-the-art methods, and great improvements are observed in the case of large displacements and occlusions (Fig. ).
Alternatively to the discrete aggregation based on graph cut optimization, a new continuous aggregation model has been designed. In accordance with the demonstrated evidence that the set of candidates always contains at least one accurate motion vector, the aggregation is formulated in a sparse framework restricting the number of non negligible weights associated to the candidates. The continuous framework is less dependent on the quality of the candidates and thus allows us to considerably reduce the computational cost of both aggregation and candidates estimation.
.
Diffusion coefficient estimation in live cell fluorescence imaging is usually achieved with correlation-based methods related to Image Correlation Spectroscopy (ICS) . This approach requires a high computational cost and the spatial resolution of the resulting diffusion map is limited by the inherent block-based principle of the method. To overcome these drawbacks, we propose a novel diffusion estimation method in a variational framework providing dense and discontinuity-preserving diffusion fields. The diffusion equation is integrated in a global energy via a neighborhood-wise data term, positivity constraint and temporal integration. The performances of the variational and ICS approaches were compared on simulated sequences. We have demonstrated the accuracy of ICS in stationarity conditions, and we pointed out the advantages of dense variational estimation to accurately recover spatial and temporal discontinuities (Fig. ).
Partners: Perrine Paul-Gilloteaux, Francois Waharte and Chen Chen (UMR 144, PICT IBiSA, CNRS-Institut Curie)
.
Recognizing dynamic protein behaviors in live cell fluorescence microscopy is of paramount importance to understand cell mechanisms. In the case of membrane traffic, cargo molecules are transferred from a donor to an acceptor compartments . At each step, dedicated molecular platforms are acting to form, transport and address selected proteins. In microscopy imaging, this sequence of processes leads to a series of heterogeneous dynamics, which need to be untangled in order to understand the spatiotemporal coordination of the molecular actors. In this study, we aim at locating and recognizing temporal events in TIRF microscopy image sequences related to membrane dynamics. After segmenting the time-varying vesicles in the image, we exploit space-time information extracted from three successive images only to model, locate and recognize the two dynamic configurations of interest: translational motion or local fluorescence diffusion (see Fig. ). A likelihood ratio test is defined to solve this issue. Results on synthetic sequences and real TIRF sequences demonstrated the accuracy and efficiency of the proposed method.
Partners: Jean Salamero, J. Boulanger (UMR 144, PICT IBiSA, CNRS-Institut Curie)
.
Important research efforts have been devoted to crowd analysis for several years , . We are interested in this topic for two main reasons. First, views of crowded scenes are not that different of light microscopy intracellular images. Second, the addressed problem, i.e. motion understanding, is common, and we are investigating similar data-driven methodological approaches. This a way to cross-fertilize two domains.
We address the problem of classifying coherent crowd motions in videos recorded by a fixed camera. In contrast to most existing methods, which are based on trajectories or tracklets, our approach for crowd motion analysis provides a crowd motion classification on a frame-by-frame basis. Indeed, we only compute affine motion models from pairs of two consecutive video images. The classification itself relies on simple rules on the coefficients of the computed affine motion models, and therefore does not imply any prior learning stage. The overall method proceeds in three steps: we first compute a set of motion model candidates on a collection of windows of different sizes in the image, then we select the motion model at each point owing to a ML criterion, finally we determine the crowd motion class map with a hierarchical classification tree regularized by majority votes. The algorithm is almost parameter-free, and is extremely efficient in terms of memory and computation load. Experiments on computer-generated sequences and real video sequences demonstrate that our method is accurate, and can successfully handle complex situations (see Fig. ).
.
Automatic analysis of the dynamic content in fluorescence video-microscopy is crucial for understanding molecular mechanisms involved in cell functions. We have proposed an original approach for analyzing particle trafficking in these sequences. Instead of individually tracking every particle, we only need to locally count particles on regions over time and minimize a global energy function. We have specified three methods to determine the particle flow. We especially compared the NNLS algorithm and the PPXA algorithm known as well suited to non differentiable convex minimization problem . We have conducted comparative experiments on synthetic and real fluorescence image sequences. We have shown that adding a sparsity constraint on the number of detected events allows us to reduce the number of false alarms. Compared to usual tracking methods, our approach is simpler and the results are very stable with respect to the only two parameters involved (see Fig. ).
Partners: Jean Salamero, J. Boulanger (UMR 144, PICT IBiSA, CNRS-Institut Curie)
.
Image tracking of fluorescent objects, from labeled molecules to organelles and entire cells, is an essential task in the analysis of cellular functions. During the last decade, several algorithms have been tailored to cope with different types of cellular and subcellular motion down to Brownian single molecule behavior . One of the remaining big challenges in this area of technology development has been the tracking of extremely heterogeneous movements of objects in crowded scenes. We tested several state-of-the-art algorithms , to follow dense populations of diffusing particles, which suddenly change to directed motion. A frequent cellular scenario with this property is the jerky motion of vesicles and viruses switching between cytoplasmic diffusion and motor-mediated, fast displacements (see Fig. ).
These switches are particularly challenging to detect because they occur rarely. The presence of numerous detected objects in the expected range of particle displacements makes the tracking ambiguous and induces wrong associations. Lowering the ambiguity by reducing the search range, on the other hand, is not an option, as this would increase the rate of false negatives.
We first explored the existing methods in the literature to analyze their strenghts and weakness for tracking objects with heterogeneous motion and high density. Based on the conclusion we draw, we proposed a new method build on the U-track platform . More specifically, we propose an interacting multiple state model that exploits recursive tracking in multiple rounds in forward and backward temporal directions. As a result, it achieves convergence of the instantaneous speed estimate time-point-by-time-point. This allows us to predict and recover abrupt transitions from freely or confined diffusive to directed motion. To address the issue of a particle that disappears as a neighboring particle appears in the same image and thus to better detect track termination, we also exploit this recursive tracking by proposing a locally adaptive on-line estimation of the search window radius for assignment (a.k.a. gating), while most of state-of-the-art algorithms propose only a global search window radius or weak per-track search radius estimations. We have shown on simulated data that our method outperforms state-of-the-art algorithms that model motion heterogeneity on different scenarios, e.g. heterogeneous motion type (see Figure ) and speed heterogeneity (see Fig. ), while keeping the computational cost of a deterministic method (10% overhead with respect to U-track).
Partners: Gaudenz Danuser (Harvard Medical School, Boston, USA)
.
Microtubules (MT) are highly dynamic tubulin polymers that are involved in many cellular processes such as mitosis, intracellular cell organization and vesicular transport. Nevertheless, the modeling of cytoskeleton and MT dynamics based on physical properties is difficult to achieve. We proposed to model microtubules as rigid and growing cylinders alike (Nedelec and Foethk 2007) but including Newtonian dynamics. Using the Euler-Bernoulli beam theory, we have proposed then to model the rigidity of microtubules on a physical basis using forces, mass and acceleration. In addition, we linked microtubules growth and shrinkage to the presence of molecules (e.g. GTP-tubulin) in the cytosol. The overall model enables linking cytosol to microtubules dynamics in a constant state space, thus allowing usage of data assimilation techniques (see Fig. ).
.
A very first task of TMA (Tissue MicroArray) image analysis is to accurately localize spots (separate tissue core) representing arrays of 512 x 512 pixels each, in very large images of several thousands of pixels. For this purpose, we have investigated a three-stage methodological approach. First, since tissue cores are separately assembled in array (grid structure). We started to design a graphical model to eliminate image defects due to the presence of dusts or the imperfection of TMA blocks fabrication. In the second stage, a wavelet-like transform is currently used to recognize interested features (spots) given the size of spots a prori. Third, we started to investigate the superpixel-based image representation (SLIC) , to handle very large images and biological details inside each spot.
Partners: V. Paveau (Innopys company)
In collaboration with Magellium company and Institut Gustave Roussy, Innopsys plans to develop new image analysis software to be included in the INGRID platform developed by Megellium company. New statistical methods and algorithms will be investigated by serpico for:
segmentation and detection of deformable cell contours and cell nuclei in 2D fluorescence tissue microarray images;
deconvolution and superresolution of fluorescence microarray imaging.
The three-year contract supports the PhD thesis of Nam-Hoai Nguyen (2013-2016).
.
In this project, we developed new statistical restoration algorithms for fluorescence and electron imaging and PSF (point-spread function) and CTF (contrast transfer function) correction, respectively. An integrated highly focused approach combing the efforts of three teams in image processing (Serpico), in-vivo light microscopy (IGDR-CeDRE) and cryo-electron tomography (IGDR-TIPs) has been studied to produce novel computational strategies for biological imaging.
Funding: RTR Syscom, European University of Brittany (UEB): 12 months |
Partners: UMR 6290 – IGDR (Institut de Génétique et Développement de Rennes) |
.
Quaero is a European collaborative research and development program with the goal of developing multimedia and multi-lingual indexing and management tools for professional and public applications. serpico team participates in the Work Package 9 on Video Processing (WP9) of QUAERO Core Technology Cluster Project (CTC). Within WP9, former Vista project-team leaded three tasks: “Motion Recognition”, “Object Tracking” and “Event Recognition”. Since October 2010, serpico has conducted activities in object tracking and indexing for video-microscopy analysis (Denis Fortun PhD grant) and Thibault Geffroy (Master 1 INSA Rouen).
Funding: Quaero (no. Inria Alloc 3184), duration: 60 months |
Partners: 24 academic and industrial partners leaded by Technicolor |
.
Biofilms are composed of spatially organized microorganisms (possibly including pathogens) embedded in an extracellular polymeric matrix. A direct time-lapse confocal microscopic technique was recently developed to enable the real-time visualization of biocide activity within the biofilm. It can provide information on the dynamics of biocide action in the biofilm and the spatial heterogeneity of bacteria-related susceptibilities that are crucial for a better understanding of biofilm resistance mechanisms. The approach is here to characterize the spatial and temporal exploration of the biofilm by microorganisms.
In this project, serpico develop methods and software for the computation of mean velocity as well as other descriptors of swimmers bacteria dynamics inside biofilm image sequences. We investigate spatio-temporal features and descriptors for comparison, classification, indexing and retrieval.
Funding: ANR, duration: 24 months |
Partners: INRA, AgroParisTech, Naturatech company |
.
The goal of the project is to build a distributed coordinated French infrastructure for photonic and electronic cellular bioimaging dedicated to innovation, training and technology transfer. High computing capacities are needed to exhaustively analyse image flows. We address the following problems: i/ exhaustive analysis of bioimaging data sets; ii/ deciphering of key steps of biological mechanisms at organ, tissular, cellular and molecular levels through the systematic use of time-lapse 3D microscopy and image processing methods; iii/ storage and indexing of extracted and associated data and metadata through an intelligent data management system. serpico is co-head of the IPDM (Image Processing and Data Management) node of the FBI network composed of 6 nodes.
Funding: Investissement d'Avenir - Infrastructures Nationales en Biologie et Santé (2011-2016) |
Partners: CNRS, Institut Jacques Monod, Institut Pasteur, Institut Curie, ENS Ulm, Ecole Polytechnique, INRA, INSERM |
ESFRI Euro-BioImaging initiative: serpico participates in the ESFRI Euro-BioImaging project, one of the four new biomedical science projects in the roadmap of the European Strategic Forum on Research Infrastructures (ESFRI). The mission of Euro-BioImaging is to provide access, service and training to state-of-the-art imaging technologies and foster the cooperation and networking at the national and European level including multidisciplinary scientists, industry regional, national and European authorities. (3-year Preparatory Phase / start: December 2010). serpico also participates in the French counterpart, the so-called “France-BioImaging” (FBI) network which gathers several outstanding cellular imaging centers (microscopy, spectroscopy, probe engineering and signal processing) as described in Section .
Collaboration with Harvard Medical School (Boston, MA), Prof. G. Danuser, on object tracking in video-microscopy (P. Roudot's visit in 2012-2013, 3 months, Inria & CNRS grant).
Collaboration with University of California - San Francisco (USA), J. Sedat and D. Agard, on image denoising in cryo-electron microscopy.
Technical program committees of conferences
Charles Kervrann: PC member for isbi'2013, isbi'2014, reviewer for icassp'2013, icassp'2014, icip'2013, icip'2014, emmcprv'2013, Gretsi'2013, member of scientific committee of “Journées d'Imagerie Optique Non-Conventionnelle” (JIONC'2014).
Patrick Bouthemy: PC member for icpram'2013, icpram'2014, mldm'2013, taima'2013, reviewer for isbi'2013, isbi'2014, icra'2013, member of scientific committee of ELMI'13.
Journal reviewing
Charles Kervrann: reviewer in 2013 for IEEE Transactions on Image Processing, IEEE Transactions on Medical Imaging, SIAM Journal Imaging Sciences, PLoS One, Image and Vision Computing, Journal of Signal Image and Video Processing.
Patrick Bouthemy: reviewer in 2013 for International Journal of Computer Vision, IEEE Transactions on Circuits and Systems for Video Technology, IEEE Transactions on Image Processing, IEEE Transactions on Medical Imaging.
Project reviewing
Charles Kervrann: reviewer in 2013 for ERC (Consolidator Grant), ANR and IDEX Paris-Saclay.
Participations in seminars, invitations, awards
Charles Kervrann was invited to give a talk entitled “Non parametric change detection methods in fluorescence life cell imaging for subcellular trafficking and exocytosis analysis” at the GdR 2588 “BioImage Informatics” Days (Institut Curie, Paris, July 2013) and the ANR MOTIMO workshop (University of Nice, September 2013).
Responsibilities
Charles Kervrann:
Member of the IEEE BISP “Biomedical Image and Signal Processing” committee.
Member of executive board of the GdR MIV (2588 - Microscopie Fonctionnelle du Vivant) CNRS, member of the scientific committee of the Interdisciplinary MiFoBio School CNRS (http://
Member of the executive board of the project committee of the Inria Rennes - Bretagne Atlantique centre.
Member of the Scientific Council of the INRA Rennes Research Centre.
Member of the Steering Committee of ESFRI EuroBioImaging (Inria representative).
Patrick Bouthemy:
Deputy member of the board of directors and member of the Selection and Validation Committee of the Images & Réseaux competitivity cluster.
Deputy member of the board of directors of IRT (Technological Research Institute) B-com.
President of AFRIF (Association Francaise pour la Reconnaissance et l'Interprétation des Formes) and member of the board of the GRETSI (Groupement de Recherche en Traitement du Signal et des Images).
Other activities
serpico is involved in the CNRS French networks GdR MIV (2588 - Microscopie Fonctionnelle du Vivant) and the GdR ISIS.
serpico is member of the regional BioGenOuest GIS.
serpico is member of the France-BioImaging Infrastructure in Biological Imaging.
Charles Kervrann:
Master: From BioImage Processing to BioImage Informatics, 5 hours, coordinator of the module (30 hours), Master 2 Research IRIV, Telecom-Physique Strasbourg & University of Strasbourg.
Master: Geometric Modeling for Shapes and Images, 6 hours, Master 2 Research SISEA, University of Rennes 1.
Engineer Degree and Master 2 Statistics and Mathematics: Statistical Models and Image Analysis, 37 hours + 15 hours (TP, Nam-Hoai Nguyen), 3rd year, Ecole Nationale de la Statistique et de l'Analyse de l'Information ENSAI Bruz & University of Rennes 1.
Course on “Introduction to Markov Random Fields” (2 hours), “Image Denoising in Live Cell Imaging” (2 hours) and “Object tracking in Video-Microscopy” (2 hours) for BIAT'2013 (BioImaging Advanced Training), Campus of CNRS, Gif-sur-Yvette, November 2013.
Course on “Biological Imaging” (4 hours) for CIMI (Centre International de Mathematiques et d'Informatique de Toulouse) Image Processing Thematic School, Saint-Lary, June 2013.
Patrick Bouthemy:
Master: Analysis of Image Sequences, 18 hours, Master 2 Research SISEA, ISTIC & University of Rennes 1.
Master: Video Indexing, 9 hours, Master 2 Research Computer Science, ISTIC & University of Rennes 1.
Engineer Degree and Master 2 Research IRIV: Motion Analysis, 12 hours, Telecom-Physique Strasbourg & University of Strasbourg.
Course on “Motion Analysis” (2 hours) for BIAT'2013 (BioImaging Advanced Training), Campus of CNRS, Gif-sur-Yvette, November 2013 and for the CNRS formation “Comprendre les différentes méthodes d'analyse de la dynamique en biologie et leurs paramètres”, Institut Jacques Monod, Paris, novembre 2013.
PhD in progress: Philippe Roudot, Lifetime estimation of moving vesicles in FLIM microscopy, started in October 2010, supervised by Charles Kervrann and Francois Waharte (UMR 144, PICT IBiSA, CNRS-Institut Curie).
PhD in progress: Denis Fortun, Optical flow computing, aggregation methods and statistical methods: application to time-lapse fluorescence microscopy, started in October 2010, supervised by Charles Kervrann and Patrick Bouthemy.
PhD in progress: Antoine Basset, Event detection and recognition in video-microscopy and applications in cell biology, started in October 2012, supervised by Patrick Bouthemy and Charles Kervrann in collaboration with Jérôme Boulanger (UMR 144, CNRS-Institut Curie).
PhD in progress: Nam-Hoai Nguyen, Methods and algorithms for tissue microarrays image analysis, started in October 2013, supervised by Charles Kervrann and Vincent Paveau (Innopsys company).
Chair of a jury for the recruitment of an assistant professor: University of Rennes 1 (Section CNU 26) [Charles Kervrann].
Member of a jury for the recruitment of an assistant professor: University of Caen - Basse Normandie (Section CNU 27) [Patrick Bouthemy].
Referee of Habilitation thesis: N. Komodakis (ENPC) [Patrick Bouthemy], J.-B. Sibarita (University of Bordeaux 2) [Charles Kervrann].
Referee of PhD thesis: X.S. N'Guyen (University of Paris 6) supervised by S. Dubuisson et C. Gonzales [Patrick Bouthemy], M. Souded (University of Nice Sophia-Antipolis) supervised by F. Bremond [Patrick Bouthemy], K. Haas (University of Bordeaux 2) supervised by D. Choquet [Charles Kervrann, Chair of the jury], M.A. Kechkar (University of Bordeaux 2) supervised by J.-B. Sibarita [Charles Kervrann], L. Genin (University of Paris 13) supervised by F. Champagnat and G. Le Besnerais [Charles Kervrann].
Chair of PhD thesis juries: C. Maumet (University of Rennes 1) supervised by C. Barillot [Patrick Bouthemy], A. Petit (University of Rennes 1) supervised by E. Marchand [Patrick Bouthemy].