The AYIN team is devoted to the modeling of spatio-temporal structures, for use in the analysis of high-resolution image data, with particular application to images arising in remote sensing, broadly interpreted, and skin care.
The latest and upcoming generations of imaging sensors, for example, in remote sensing (Pleiades, EnMAP, Sentinel) and medicine (Philips, Christie Medical), result in large volumes of heterogeneous data with high spatial, spectral, and temporal resolution. High resolution imagery (this may refer to spatial, spectral, or temporal resolutions) is a rich source of information about the imaged scene, information that is unavailable in lower resolution data. In particular, spatial and spatio-temporal structures abound, and frequently constitute the information of greatest interest in practice. As a result, such imagery is vital to advances in a range of applications (urban monitoring, precision agriculture, skin disease diagnosis, etc.). The high resolution and high volume of the imagery presents new challenges, however, that must be overcome if the potential of the data is to be realized. Extracting the available information requires the development of new modeling techniques adapted to the nature and profusion of structures, and the design of corresponding algorithms, which must in turn be implemented in a time- and space-efficient way if the techniques are to be made operational.
The overall scientific objective of the AYIN team is precisely to advance the state of theory and practice in this area by the development of such modeling techniques and the design of such algorithms. We make use of a variety of methodologies in order to achieve this goal, taking a broadly Bayesian point of view. This point of view suggests dividing the modeling task into two parts: modeling of the scene, i.e. describing the scenes to be expected in any given application; and modeling of the image, i.e. describing the images to be expected from any given scene. AYIN focuses on spatio-temporal and spectral structure, leading to the modeling of geometrical properties on the one hand, and large, coherent structures in images and image sequences on the other. The new models also require new algorithms, for dealing with the nuisance parameters they contain, and for extracting the desired information. This forms a third major component of AYIN's research. The models and algorithms are developed in parallel with their application to information extraction from very high resolution images, in particular data arising in remote sensing and skin care.
One of the grand challenges of computer vision and image processing is the expression and use of prior geometric information via the construction of appropriate models. For very high resolution imagery, this problem becomes critically important, as the increasing resolution of the data results in the appearance of a great deal of complex geometric structure hitherto invisible. AYIN studies various approaches to the construction of models of geometry and shape.
One of the most promising approaches to the inclusion of this type of information is stochastic geometry, which is an important research direction in the AYIN team. Instead of defining probabilities for different types of image, probabilities are defined for configurations of an indeterminate number of interacting, parameterized objects located in the image. Such probability distributions are called `marked point processes'. New models are being developed both for remote sensing applications, and for skin care problems, such as wrinkle and acne detection.
An alternative approach to shape modeling starts with generic `regions' in the image, and adds constraints in order to model specific shapes and objects. AYIN investigates contour, phase field, and binary field representations of regions, incorporating shape information via highly-structured long-range interactions that constrain the set of high-probability regions to those with specific geometric properties. This class of models can represent infinite-dimensional families of shapes and families with unbounded topology, as well as families consisting of an arbitrary number of object instances, at no extra computational cost. Key sub-problems include the development of models of more complex shapes and shape configurations; the development of models in more than two spatial dimensions; and understanding the equivalences between models in different representations and approaches.
AYIN is concerned with spectral and spatio-temporal structures. To deal with
the latter, the above scene modeling approaches are extended into the time
dimension, either by modeling time dependence directly, or, in the
field-based approaches, by modeling spacetime structures, or, in the
stochastic geometry approach, by including the time
The key issue that arises in modeling the high-resolution image data generated in AYIN's applications, is how to include large-scale spatial, temporal, and spectral dependencies. AYIN investigates approaches to the construction of image models including such dependencies. A central question in the use of such models is how to deal with the large data volumes arising both from the large size of the images involved, and the existence of large image collections. Fortunately, high dimensionality typically implies data redundancy, and so AYIN investigates methods for reducing the dimensionality of the data and describing the spatial, temporal, and spectral dependencies in ways that allow efficient data processing.
One way to achieve large-scale dependencies is via explicit long-range interactions. MRFs with long-range interactions are also used in AYIN to model geometric spatial and temporal structure, and the techniques and algorithms developed there will also be applied to image modeling. In modeling image structures, however, other important properties, such as control of the relative phase of Fourier components, and spontaneous symmetry breaking, may also be required. These properties can only be achieved by higher-order interactions. These require specific techniques and algorithms, which are developed in parallel with the models.
Another way to achieve long-range dependencies is via shorter range interactions in a hierarchical structure. AYIN works on the development of models defined as a set of hierarchical image partitions represented by a binary forest structure. Key sub-problems include the development of multi-feature models of image regions as an ensemble of spectral, texture, geometrical, and classification features, where we search to optimize the ratio between discrimination capacity of the feature space and dimensionality of this space; and the development of similarity criteria between image regions, which would compute distances between regions in the designed feature space and would be data-driven and scale-independent. One way to proceed in the latter case consists in developing a composite kernel method, which would seek to project multi-feature data into a new space, where regions from different thematic categories become linearly or almost linearly separable. This involves developing kernel functions as a combination of basis kernels, and estimating kernel-based support vector machine parameters.
Computational techniques are necessary in order to extract the information of interest from the models. In addition, most models contain `nuisance parameters', including the structure of the models themselves, that must be dealt with in some way. AYIN is interested in adapting and developing methods for solving these problems in cases where existing methods are inadequate.
In order to render the models operational, it is crucial to find some way to deal with nuisance parameters. In a Bayesian framework, the parameters must be integrated or marginalized out. Unfortunately, this is usually very difficult. Fortunately, Laplace's method often provides a good approximation, in many cases being equivalent to classical maximum likelihood parameter estimation. Even these problems are not easy to solve, however, when dealing with complex, structured models. This is particularly true when it is necessary to estimate simultaneously both the information of interest and the parameters. AYIN is developing a number of different methods for dealing with nuisance parameters, corresponding to the diversity of modeling approaches.
Extracting the information of interest from any model involves making estimates based on various criteria, for example MAP, MPM, or MMSE. Computing these estimates often requires the solution of hard optimization problems. The complexity of many of the models to be developed within AYIN means that off-the-shelf algorithms and current techniques are often not capable of solving these problems. AYIN develops a diversity of algorithmic approaches adapted to the particular models developed.
Josiane Zerubia is IEEE Signal Processing Society Distinguished
Lecturer for 2016 and 2017, see
(http://
Josiane Zerubia received the Excellency Prize of UCA (Université Cote d'Azur) for her outstanding research work in December 2016.
Nazre Batool who was an Inria post-doc in AYIN till May 2015 received
the IEEE R8 Women in Engineering Clementina Saduwa 2016 award, see
(http://
Josiane Zerubia, in collaboration with Gabriele Moser from University of Genoa (Italy), edited a book of more than 400 pages on mathematical models for remotely sensed image processing which was submitted to Springer Verlag in December 2016 and will be published early 2017. They also contributed to two chapters of this book.
Josiane Zerubia is a scientific consultant for the Galderma company (http://
This work was carried out in collaboration with Prof. Gabriele Moser and Prof. Sebastiano Serpico from DITEN departement (http://
In this work we address the problem of constructing statistical models of images using Hierarchical Hidden Markov modeling techniques for high resolution remotely sensed image classification of urban areas. The main difficulty is to develop a classifier that jointly utilizes the benefits of multi-band and multi-resolution input data while maintaining a good trade-off between accuracy and computation time. In this framework, Markov random field (MRF) models are widely used in classification problems since they provide a convenient and consistent way of integrating contextual information into the classification scheme. Furthermore, MRF models defined according to hierarchical structures exhibit good methodological and application-oriented properties including causality, thanks to the use of appropriate graphs such as a quadtree structure . The input satellite images are inserted in a hierarchical structure on the basis of their spatial resolution. This approach is aimed at both exploiting multi-scale information, which is known to play a crucial role in high-resolution image analysis, and supporting contextual information at each scale. However, hierarchical MRFs on quad-trees rely on a causality concept captured by the factorization of the prior distribution in terms of causal transition probabilities . In practice, this structure tends to generate "blocky" effects in the final classification map. Due to this disadvantage, a new hierarchical MRF based on a Symmetric Markov Mesh Random Field (SMMRF) is proposed in this work, to overcome these limitations from both mathematical and practical points of view, and to establish a causal and symmetrical model. This can be accomplished by scanning the lattice at each level of the hierarchical model based on the visiting scheme shown in Fig . Then, for each scale of the quad-tree, the causal SMMRF is integrated into the hierarchical structure. Accordingly, each node
We applied the developed hierarchical classification approach to a multi-resolution dataset that consists of a panchromatic and a multi-spectral Pléiades images acquired over Port-au-Prince (Haiti). Experimental results with HR satellite imagery of a very high-resolution urban scene suggest that the method allows to effectively incorporate spatial information in the hierarchical classification process and provides higher accuracies than previous techniques. Indeed, it is confirmed experimentally (see Fig. ) that MMRFs and their lattice models are corner-dependent, and that the proposed approach is effective in circumventing this drawback by using a Symmetric Markov Mesh Random Field. The proposed method, in the application to a challenging urban area classification problem, is able to combine the computational and modeling benefits of hierarchical and symmetric mesh MRF models, while preventing their individual artifacts.
This work was carried out in collaboration with Dr. Vladimir Krylov, Prof. Gabriele Moser and Prof. Sebastiano Serpico from DITEN departement (http://
Multitemporal change detection on image sequences is one of the fundamental image processing problems and multiple detection, monitoring and tracking applications rely on its accurate and timely performance. To address this problem we develop an approach that gives a unified statistical thresholding procedure to perform change detection based on statistical features that have a known distribution under the no-change hypothesis. The proposed False Discovery Rate (FDR) formulation is based on the control of the proportion of false alarms among all detections . This efficient technique for large scale hypotheses testing allows to use the wide range of statistical tests developed in the state-of-the-art by adjusting to the dependence structure present in the images and the patch-based samples. The developed approach involves only a few parameters and is highly parallelizable. We propose several rank-based statistical features that report accurate experimental results and the corresponding detectors positively compare with benchmark techniques in three different applications. Further features can be easily constructed to elaborate application-specific change detectors.
In Fig. we demonstrate a typical result of the FDR-based change detector on a semi-synthetic 3-image synthetic aperture radar sequence based on COSMO-SkyMed (©ASI) images of Haiti (April, May and August 2011). In this experiments the Levene multisample statistic with a 9-by-9 local window is employed. A comparison of the omnibus test (formulating the hypothesis for all three images simultaneously) with pairwise tests, demonstrates that the latter are more sensitive to changes. This sensitivity can be a disadvantage as is the case with the detection noise in (e) and (f). The omnibus test on the other hand did not suffer from the same mistake due to a generally higher level of tolerance to pairwise fluctuations. Hence, from the SAR-change detection point of view, the results reported by omnibus version of Levene-statistic are considered more adequate.
This work has been conducted in collaboration with Prof. Farzad Kamalabadi, Dr. Jianqi Qin and Dr. Mark Butala from Coordinated Science Laboratory (CSL, http://
ICON (Ionospheric Connection Explorer) is a satellite which is part of
the NASA Explorer missions (see http://
There will be 4 instruments on board. One of them is the FUV: Far UltraViolet spectrographic imager. Prof Kamalabadi is responsible to process the FUV data from this instrument. During Josiane Zerubia's stay at CSL, UIUC, she worked with Prof. Farzad Kamalabadi team on defining a proper energy function using Bayesian theory (i.e. defining a data term + various priors for regularizing the solution) in order to be able to take into account the geometry of the information and also to deal with optical transmission function. This inverse problem is highly non linear. We will continue in the future to work on the problem of the error estimation (or bound derivation) as far as the estimation of distribution parameters is concerned.
This work was carried out in collaboration with Ali Madooei (Simon Fraser University (https://
This study focused on detection and grading of skin erythema using hyperspectral image processing. The ultimate objective is to build a system for monitoring radiation response in individuals using hyperspectral imaging technology and image processing. The present project was to investigate the possibility of monitoring the degree of skin erythema. To this aim, we proposed an image processing pipeline and conducted controlled experiments to demonstrate the efficacy of the proposed approach for (1) reproducing clinical assessments, and (2) outperforming RGB imaging data (see Fig. ). We combined the problem of erythema detection and grading into a multi-class classification problem where each pixel is classified as one of the four erythema classes or a non-erythema class. We used a weighted LDA (linear discriminant analysis) classifier to deal with noisy labels. Moreover, we devised pre-processing steps to deal with noisy measurements. We evaluate the system against the clinical assessment of an experienced clinician. We also compare the performance to that of using digital photography (instead of hyperspectral images) . The results from this preliminary study are encouraging and indicate that hyperspectral image data contain relevant information, and indeed outperform imaging photography. In future, we want to extend the technique to further detect other skin responses to radiation (such as dry/moist desquamation, skin necrosis, etc.) and also to experiment with real patients undergoing radiotherapy.
Josiane Zerubia has been in contact with Dr. Sandrine Mathieu, image processing quality expert
at Thales Alenia Space in Cannes (https://
Josiane Zerubia is part of the FAULTS-R-GEMS project funded by Academy 3 of Idex UCA-Jedi (http://
Josiane Zerubia has a strong collaboration with University of Genoa, Italy, for more than 20 years .
Another collaboration in Canada with Mc Master University, Hamilton, started in 2012 , and has been extended to Juravinski Cancer Center (http://
Josiane Zerubia was invited to spend 2 months, from late August to late October, at the Coordinated Science Laboratory (CSL, http://
Josiane Zerubia in collaboration with Farzad Kamalabadi from UIUC
organized a special session at the Symposium URSI-France (http://
Josiane Zerubia is part of the organizing committee and will be co-Chair of the EARTHVISION workshop at CVPR'17 (http://
Josiane Zerubia is part of the organizing committee of the 50th GRETSI
conference (http://
Josiane Zerubia was part of the conference program committee of
SYNASC'16 (http://
Josiane Zerubia was part of the conference program committee of ISPRS/SPIE Remote Sensing'16 (http://
Josiane Zerubia was a reviewer for the conferences IEEE ICASSP'16, IEEE ICIP'16, ISPRS-SPIE Remote Sensing'16, ECCV'16, IEEE CVPR'16, IAPR ICPR'16, IEEE EMBC'16, IEEE-EURASIP EUSIPCO'16 and SYNASC'16.
Josiane Zerubia is an Associate Editor of the collection "Foundation and Trends in Signal Processing" (http://
Josiane Zerubia is a member of the Editorial Board of the "Revue Française de Photogrammétrie et
de Télédétection of SFPT" (http://
Josiane Zerubia is an Associate Editor of the electronic journal Earthzine (http://
Ihsen Hedhli was a reviewer for IEEE TGRS, TGRL and TIP journals and Elsevier Pattern Recognition Letters.
As IEEE SPS Distinguished Lecturer, Josiane Zerubia gave in 2016 ten
invited talks: at DITEN Dept., University of Genoa, Italy, at the Faculty
of Mathematics and Computer Science from the West University of
Timisoara, Romania, at the Faculty of Electronics, Telecommunications &
IT and the Faculty of Medical Engineering, University Politehnica of
Bucharest, Romania, at the Georgia Tech Research Institute in Atlanta,
USA, at the SenSIP Center, School of Electrical, Computer & Energy
Engineering, Arizona State University, Tempe, USA, at both the Sine
Seminars and the Remote Sensing Seminars, within the Coordinated Science Laboratory,
University of Illinois at Urbana Champaign, USA, at the Signal
and Communications laboratory in the Dept. of Engineering, University of
Cambridge, UK, at the Faculty of Engineering, University of Bristol, UK,
and finally a plenary talk at the international symposium ISIVC2016 in
Tunis, organized by Sup'Com (see
http://
Josiane Zerubia is IEEE Fellow
(http://
Josiane Zerubia was a reviewer for the Israeli Ministry of Science, Technology and Space (http://
Josiane Zerubia is part of the Scientific Council of Academy 3 of UCA-Jedi (http://
Josiane Zerubia is part of the Board of Directors of the French Photogrammetry and Remote Sensing
Society (http://
Josiane Zerubia is a member of the Inria-SAM Center Committee since 2016.
Masters: Josiane Zerubia, Advanced Techniques in Signal and Image Processing, 30h eq. TD (20h of lectures), ISAE/SUPAERO, France. Josiane Zerubia is also director of this course (total: 30h of lectures and 10h of TD). This course was given to third-year students at ISAE/SUPAERO and was also validated by the M2 of Applied Mathematics at the University Toulouse III Paul Sabatier, France.
Masters: Josiane Zerubia, Deconvolution and Denoising in Confocal Microscopy, 18h eq. TD (12h of lectures), M2 BCC, Université de Nice Sophia-Antipolis, France. Josiane Zerubia is also director of this course (total: 24h of lectures).
Summer School (PhD and Post-doc level): Josiane Zerubia gave a presentation at the Summer School SSMIA'16 in Budapest, Romania.
PhD (defended with best honors): Ihsen Hedhli, Change detection methods for multisensor and multiresolution remote sensing images for applications to environmental disaster management, University of Genoa and Université Nice Sophia Antipolis, started in January 2013, defended in March 2016, Gabriele Moser and Josiane Zerubia.
Josiane Zerubia was a member of a PhD defense committee at University of Genoa, in Italy, in March 2016.
Josiane Zerubia presented Inria and Inria-SAM to Masters, PhDs and Post-docs of the following foreign universities during 2016: University of Genoa, West University of Timisoara, Georgia Tech, Arizona State University, University of Illinois at Urbana Champaign, University of Cambridge, University of Bristol.
Josiane Zerubia was interviewed in August with two other French women
scientists working in remote sensing by a journalist for Earthzine (webzine
supported by IEEE and NASA) see
https://
In September, Josiane Zerubia actively participated to a debate about
Women in Signal Processing at the IEEE International Conference ICIP'16
(http://
Josiane Zerubia, with Gabriele Moser and Sebastiano Serpico from
University of Genoa, wrote in December an article about natural disaster monitoring for ERCIM News, see
http://