Section: Software

Tracking Focus of Attention for Large Screen Interaction

Participants : Claudine Combe, John Alexandre Ruiz Hernandez, Varun Jain, James Crowley [correspondant] .

Large multi-touch screens may potentially provide a revolution in the way people can interact with information in public spaces. Technologies now exist to allow inexpensive interactive displays to be installed in shopping areas, subways and urban areas. Thesis displays can provide location aware access to information including maps and navigation guidance, information about local businesses and and commercial activities. While location information is an important component of a users context, information about the age and gender of a user, as well as information about the number of users present can greatly enhance the value of such interaction for both the user and for local commerce and other activities.

The objective of this task is to leverage recent technological advances in real time face detection developed for cell phones and mobile computing to provide a low-cost real time visual sensor for observing users of large multi-touch interactive displays installed in public spaces. The initial requirements for this system were expressed by the recent INRIA start-up HiLabs, created in 2008. By the end of 2010, HiLabs had installed over 100 interactive displays in public spaces, mostly in the form of interactive shop windows for travel agents, real-estate agents and banks. HiLabs customers indicated a potential important gain in market if such displays could be made aware of the number, gender and age of users.

The software developed for this activity builds on face detections software that has recently been developed by INRIA for the French OSEO project MinImage. MinImage was a five year, multi-million euro project to develop next generation technologies for integrated digital imaging devices to be used in cellphones, mobile and lap-top computing devices, and digital cameras, that has begun in February of 2007. The project scope included research on new forms of retinas, integrated optics, image formation and embedded image processing. INRIA was responsible for embedded algorithms for real time applications of computer vision.

Within MinImage, INRIA developped embedded image analysis algorithms using image descriptors that are invariant to position, orientation and scale and robust to changes in viewing angle and illumination intensity. INRIA proposed use of a simple hardware circuit to compute a scale invariant Gaussian pyramid as images acquired by the retina. Sums and differences of image samples from the pyramid provide invariant image descriptors that can be used for a wide variety of computer vision applications including detection, tracking and recognition of visual landmarks, physical objects, commercial logos, human bodies and human faces. Detection and tracking of human faces was selected as benchmark test case. This work has been continued with support from EIT ICTlabs, to provide context information for interaction with large multi-touch interactive displays installed in public spaces.

Multitouch interactive displays are increasingly used in outdoor and public spaces. This objective of this task is to provide a visual observation system that can detect and count users of a multitouch display and to estimate information such as the gender, and age category of each user. us rendering the system sensitive to environmental context.

SuiviDeCiblesCouleur locates individuals in a scene for video communications. FaceStabilsationSystem renormalises the position and scale of images to provide a stabilised video stream. SuiviDeCiblesCouleur has been declared with the APP "Agence pour la Protection des Programmes" under the Interdeposit Digital number IDDN.FR.001.370003.000.S.P.2007.000.21000.

A revised APP is under preperation for new versions of this software for face detection, face tracking, gender and age estimation, and orientation estimation.