Previous | 
Home
Section: New Software and Platforms
Platforms
Light field editor
Participants : Pierre Allain, Laurent Guillo, Christine Guillemot.
As part of the ERC Clim project, the EPI Sirocco is developing a light field editor, a tool analogous to traditional image editors such as the GNU image manipulation program Gimp or the raster graphic editor Photoshop but dedicated to light fields. As input data, this tool accepts for instance sparse light fields acquired with High Density Camera Arrays (HDCA) or denser light fields captured with microlens array (MLA). Two kinds of features are provided. Traditional features such as changing the angle of view, refocusing or depth map extraction are or will be soon supported. More advanced features are being integrated in our tool as libraries we have developed, such as inpainting to support light field manipulations like object removal, and denoising in the 4D ray space. The next steps are to integrate libraries enabling scene depth estimation and view synthesis. The tool and libraries are developed in C++ and the graphical user interface relies on Qt.
Acquisition of multi-view sequences for Free viewpoint Television
Participants : Cédric Le Cam, Laurent Guillo, Thomas Maugey.
The scientific and industrial community is nowadays exploring new multimedia applications using 3D data (beyond stereoscopy). In particular, Free viewpoint Television (FTV) has attracted much attention in the recent years. In those systems, the user can choose in real time its view angle from which he/she wants to observe the scene. Despite the great interest for FTV, the lack of realistic and ambitious datasets penalizes the research effort. The acquisition of such sequences is very costly in terms of hardware and working effort, which explains why no multi-view videos suitable for FTV has been proposed yet.
In the project ATeP (funded by Inriahub), we have developed a novel acquisition procedure relying on forty synchronized omnidirectional cameras. The captured content allows an omni-directional visualization of the scene at a set of discrete viewpoints corresponding to the pre-defined camera positions. We also propose a calibration technique to estimate the position and orientation of each camera with respect to a same reference. This solution relies on a calibration of each individual camera, and a graph-based synchronization of all the estimated parameters.
Based on these developed tools, we have built a complete dataset that we share on the following website https://project.inria.fr/ftv360. Our dataset is made of two different captures (indoor and outdoor), with, in total 8 different sequences (each of them having 40 synchronized videos of 1 to 4 min long). The calibration parameters are shared with the calibration toolkit that was developed during the project. These data can serve for the development of new tools for FTV, such as: view synthesis, depth estimation, super resolution, inpainting, etc.
Light fields datasets
Participants : Pierre Allain, Christine Guillemot, Laurent Guillo.
The EPI Sirocco makes extensive use of light field datasets with sparse or dense contents provided by the scientific community to run tests. However, it has also generated its own natural and synthetic contents.
Natural content has been created with Lytro cameras (the original first generation Lytro and the Lytro Illum). The team also owns a R8 Raytrix plenoptic cameras with which still and video contents have been captured. Applications taking advantage of the Raytrix API have been developed to extract views from the Raytrix lightfield. The number of views per frame is configurable and can be set for instance to 3x3 or 9x9 according to the desired sparsity. A dataset of video light fields captured by our raytrix R8 camera has been proposed to the MPEG-I standardization group and retained for test purposes [24].
Synthetic content exists for dense light fields with small baselines. To address issues of scene depth estimation and of view synthesis in more difficult configurations like in the case of large baselines, we have produced two datasets that we use for training neural networks for scene depth estimation from light fields with small and large baselines. Most of our rendered light field scenes are indoor scenes, with light reflection and diffusion on the object surfaces to make them more realistic. Both dense and sparse light fields of views of pixels have been rendered from the input 3D models, with a disparity range of for sparse light fields and for dense light fields. The dense and sparse light fields datasets contains 43 and 53 scene respectively. They are provided together with the ground truth depth maps.
Similarly, as no publicly available dataset exist for video light fields, we have produced our own data set from the Sintel film (https://durian.blender.org/download/), which is a short computer animated film by the Blender institute, part of the Blender Foundation. A specific Blender add-on is used to extract views from a frame. As previously, the number of views is configurable. Synthetic contents present the advantage to provide a ground truth useful to evaluate how accurate our algorithms are to compute, for instance, the depth maps and the scene flows. At the moment, the dataset contains two synthetic video light fields of 50 frames.
All these contents are made available via the project web site: http://clim.inria.fr/DataSoftware.html