Section: Partnerships and Cooperations

European Initiatives

FP7 & H2020 Projects

ERC Advanced grant Allegro

Participants : Cordelia Schmid, Pavel Tokmakov, Nicolas Chesneau, Vicky Kalogeiton, Konstantin Shmelkov, Daan Wynen, Xiaojiang Peng.

The ERC advanced grant ALLEGRO started in April 2013 for a duration of five years. The aim of ALLEGRO is to automatically learn from large quantities of data with weak labels. A massive and ever growing amount of digital image and video content is available today. It often comes with additional information, such as text, audio or other meta-data, that forms a rather sparse and noisy, yet rich and diverse source of annotation, ideally suited to emerging weakly supervised and active machine learning technology. The ALLEGRO project will take visual recognition to the next level by using this largely untapped source of data to automatically learn visual models.  We will develop approaches capable of autonomously exploring evolving data collections, selecting the relevant information, and determining the visual models most appropriate for different object, scene, and activity categories. An emphasis will be put on learning visual models from video, a particularly rich source of information, and on the representation of human activities, one of today's most challenging problems in computer vision.

EU Marie Curie project: Egovision4health

Participants : Grégory Rogez, Cordelia Schmid.

After the 2-year outgoing phase hosted by the University of California, Irvine, G. Rogez spent the return (and final) phase of the project in the team. In 2015, he analyzed functional object manipulations focusing on fine-grained hand-object interactions and created a large dataset of 12000 RGB-D images covering 71 everyday grasps in natural interactions. This Grasp UNderstanding dataset (GUN-71) has been made publicly available in 2016 (http://www.gregrogez.net/research/egovision4health/gun-71/). In the last period of the fellowship, G. Rogez and C. Schmid addressed the more general problem of full-body 3D pose estimation in third-person images. They developed a new data synthesis technique to generate large-scale (2 millions images) training data that were later used to train Deep Convolutional Neural Networks. The collaboration resulted in a publication [19]. Dataset, code and models will be released soon.