Section: New Results

Image processing

Participants : Gilles Simon, Fabien Pierre, Frédéric Sur.

Variational methods for image processing

In the previous decade, variational methods in image processing have been widely used with a huge number of applications. The convex hypothesis generally makes the proof of convergence easier, whereas it is not fulfilled by the most interesting problems in imaging. The non-convexity may appear in some applications such as image colorization with multiple candidates selection [27] or in the case of M-estimators computation, in particular with an assumption of Cauchy noise [16]. These two points of view of the non-convex variational methods bring two different mathematical challenges to ensure the convergence of the numerical scheme. The choice of one candidate among a collection of possible ones implies bi-convex functions (functions with multiple variables, convex with respect to each ones). The computation of M-estimators with Cauchy noise hypothesis implies smooth but non-convex functions.

Our contributions concern both types of non-convexity. For bi-convex functions, we have demonstrated in [27] the convergence of alternate gradient descent numerical scheme with inertial relaxation of the iterates. Moreover, an application to image colorization has been proposed. In [16], a fixed-point algorithm has been studied to solve the problem of the Myriad filters. The particularity of this work is the convergence of the numerical scheme to a local minimum with probability 1, which is, up to our best knowledge, a novelty in the optimization community.

Computational photomechanics

In computational photomechanics, mainly two methods are available for estimating displacement and strain fields on the surface of a material specimen subjected to a mechanical test, namely digital image correlation (DIC) and localized spectrum analysis (LSA). With both methods, a contrasted pattern marks the surface of the specimen: either a random speckle pattern for DIC or a regular pattern for LSA, this latter method being based on Fourier analysis. It is a challenging problem since strains are tiny quantities giving deformations often not visible to the naked eye. This year's outcomes of our collaboration with Institut Pascal (Clermont-Ferrand) focus on three areas.

We have proposed an algorithm to render synthetic speckle images deformed under a predetermined deformation fixed by the user [17]. The goal is to generate ground truth datasets in order to assess the performance of the numerous variants of DIC and also the influence of extrinsic factors such as the noise or the marking pattern. It is required to carefully design the rendering algorithm in order to ensure that any measurement bias is caused by DIC estimation and not by the rendering algorithm itself. We have proposed to render speckle images based on a Boolean model, a standard model of stochastic geometry, a Monte Carlo estimation giving the gray level at any pixel. A software library and datasets are publicly available.

We have also investigated the optimization of the pattern marking the specimen [15], which is the topic of various recent papers. Checkerboard is the optimized pattern in terms of sensor noise propagation when the signal is correctly sampled, but its periodicity causes convergence issues with DIC. The consequence is that checkerboards are not used in DIC applications although they are optimal in terms of sensor noise propagation. We have shown that it is possible to use LSA to estimate displacement and strain fields from checkerboard images, although LSA was originally designed to process 2D grid images. A comparative study of checkerboards and grids shows that, under similar lighting conditions, the noise level in displacement and strain maps obtained with checkerboards is lower than that obtained with classic 2D grids.

Another scientific contribution concerns the restoration of displacement and strain maps. DIC and LSA both provide displacement fields equal to the actual one convolved by a kernel known a priori. The kernel indeed corresponds to the Savitzky-Golay filter in DIC, and to the analysis window of the windowed Fourier transform used in LSA. While convolution reduces noise level, it also gives a systematic measurement error. We have proposed a deconvolution method to retrieve the actual displacement and strain fields from the output of DIC or LSA [14]. The proposed algorithm can be considered as a variant of Van Cittert deconvolution, based on the small strain assumption. It is demonstrated that it allows enhancing fine details in displacement and strain maps, while improving the spatial resolution.

Cartoon-texture image decomposition

Decomposing an image as the sum of geometric and textural components is a popular problem of image analysis. In this problem, known as cartoon and texture decomposition, the cartoon component is piecewise smooth, made of the geometric shapes of the images, and the texture component is made of stationary or quasi-stationary oscillatory patterns filling the shapes. Microtextures being characterized by their power spectrum, we propose to extract cartoon and texture components from the information provided by the power spectrum of image patches. The contribution of texture to the spectrum of a patch is detected as statistically significant spectral components with respect to a null hypothesis modeling the power spectrum of a non-textured patch. The null-hypothesis model is built upon a coarse cartoon representation obtained by a basic yet fast filtering algorithm of the literature. The coarse decomposition is obtained in the spatial domain and is an input of the proposed spectral approach. We thus design a “dual domain” method. The statistical model is also built upon the power spectrum of patches with similar textures across the image. The proposed approach therefore falls within the family of non-local methods. Compared to variational methods or fast filers, the proposed non-local dual-domain approach [18] is shown to achieve a good compromise between computation time and accuracy. Matlab code is publicly available.