Section: Overall Objectives

Analysis and modeling for compact representation

Analysis and modeling of the visual data are crucial steps for a number of video processing problems: navigation in the 3D scene, compression, loss concealment, denoising, inpainting, editing, content summarization and navigation. The focus is on the extraction of different cues such as scene geometry, edge, texture and motion, on the extraction of high-level features (GIST-like or epitomes), and on the study of computational models of visual attention, useful for different visual processing tasks.

In relation to the above problems, the project-team places a special focus on 3D and multi-view content. 3D displays, going from stereo solutions with glasses to multi-view auto-stereoscopic displays which do not require glasses, are starting to appear for home environments. Depth perception on such displays requires transmitting two or more views, i.e. very large volumes of data. The TEMICS project-team focuses on several algorithmic problems to analyze, represent, compress and render multi-view video content. The team first addresses the problem of depth information extraction. The depth information is associated with each view as a depth map, and transmitted in order to perform virtual view generation. The design of algorithmic solutions for the end-to-end capturing, compression, transmission and rendering of 3D and multi-view content actually raises a number of unresolved theoretical and practical issues related to depth or scene geometry estimation, analysis of multi-view video, efficient compression of video plus depth data, and high quality rendering (view generation process). Given one view with its depth information, depth image-based rendering techniques have the ability to render views in any other spatial positions. However, the issue of intermediate view reconstruction remains a difficult ill-posed problem. Most errors in the view synthesis are caused by incorrect geometry information, inaccurate camera parameters, and occlusions/disocclusions. efficient inpainting techniques are necessary to restore disocclusions areas.