Section: New Results

Aggregation methods for optical flow computation

Participants : Charles Kervrann, Denis Fortun.

We address the problem of optical flow estimation, that is recovering the dense apparent motion of the pixels in a sequence of images. It is a fundamental computer vision task at the basis of a large variety of applications: object tracking, video compression, motion segmentation, movement detection, 3D reconstruction . . . Most of state-of-the art methods rely on a common global variational framework [22] , [31] . Computing optical flow amounts to minimizing a global energy. Our experiments demonstrated that the restriction of the minimization to local regions yields significant improvements of the estimation. Motivated by this fact, we developed a novel method to take advantage of this local approach by deriving the global estimate of the flow field from an aggregation of several local estimates. Our work can thus be seen as a general semi-local framework which can be used to improve the performance of any global variational method (Figure 3 ). We evaluated the performance of our approach on real and synthetic sequences. We investigate adaptations of this methodology to time-lapse fluorescence microscopy and we have recently performed comparisons with usual correlation techniques.

Figure 3. Average Angular Error (AAE) of vector flows obtained with the global variational method and three aggregation methods applied to three image sequences of the Middlebury (http://vision.middlebury.edu/flow/ ) database: row #1: first frame of the sequence; row #2: global variational method; row #3: mean aggregation; row #4: confidence-based weighted average; row #5: graph cut aggregation.