<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1 plus MathML 2.0 plus SVG 1.1//EN" "http://www.w3.org/2002/04/xhtml-math-svg/xhtml-math-svg.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="application/xhtml+xml; charset=utf-8"/>
    <title>Project-Team:PRIMA</title>
    <link rel="stylesheet" href="../static/css/raweb.css" type="text/css"/>
    <meta name="description" content="Research Program - Perception for Social Interaction"/>
    <meta name="dc.title" content="Research Program - Perception for Social Interaction"/>
    <meta name="dc.subject" content=""/>
    <meta name="dc.publisher" content="INRIA"/>
    <meta name="dc.date" content="(SCHEME=ISO8601) 2015-01"/>
    <meta name="dc.type" content="Report"/>
    <meta name="dc.language" content="(SCHEME=ISO639-1) en"/>
    <meta name="projet" content="PRIMA"/>
    <!-- Piwik -->
    <script type="text/javascript" src="/rapportsactivite/piwik.js"></script>
    <noscript><p><img src="//piwik.inria.fr/piwik.php?idsite=49" style="border:0;" alt="" /></p></noscript>
    <!-- End Piwik Code -->
  </head>
  <body>
    <div class="tdmdiv">
      <div class="logo">
        <a href="http://www.inria.fr">
          <img style="align:bottom; border:none" src="../static/img/icons/logo_INRIA-coul.jpg" alt="Inria"/>
        </a>
      </div>
      <div class="TdmEntry">
        <div class="tdmentete">
          <a href="uid0.html">Project-Team Prima</a>
        </div>
        <span>
          <a href="uid1.html">Members</a>
        </span>
      </div>
      <div class="TdmEntry">
        <a href="./uid3.html">Overall Objectives</a>
      </div>
      <div class="TdmEntry">Research Program<ul><li><a href="uid8.html&#10;&#9;&#9;  ">Situation Models for Context Aware Systems and Services</a></li><li><a href="uid14.html&#10;&#9;&#9;  ">Service Oriented Architectures for Intelligent Environments</a></li><li><a href="uid15.html&#10;&#9;&#9;  ">Robust view-invariant Computer Vision</a></li><li class="tdmActPage"><a href="uid21.html&#10;&#9;&#9;  ">Perception for Social Interaction</a></li><li><a href="uid27.html&#10;&#9;&#9;  ">End User Programming of Smart Environments</a></li></ul></div>
      <div class="TdmEntry">
        <a href="./uid33.html">Highlights of the Year</a>
      </div>
      <div class="TdmEntry">New Software and Platforms<ul><li><a href="uid37.html&#10;&#9;&#9;  ">OMiSCID</a></li><li><a href="uid41.html&#10;&#9;&#9;  ">AppsGate</a></li><li><a href="uid46.html&#10;&#9;&#9;  ">SPOK</a></li><li><a href="uid48.html&#10;&#9;&#9;  ">DomiCube</a></li><li><a href="uid52.html&#10;&#9;&#9;  ">EmoPRAMAD</a></li><li><a href="uid55.html&#10;&#9;&#9;  ">MobileRGBD</a></li><li><a href="uid58.html&#10;&#9;&#9;  ">Online Movie Director</a></li><li><a href="uid61.html&#10;&#9;&#9;  ">PALGate</a></li><li><a href="uid64.html&#10;&#9;&#9;  ">PERFECT MATCH STEREO</a></li><li><a href="uid67.html&#10;&#9;&#9;  ">PrimaCV</a></li><li><a href="uid70.html&#10;&#9;&#9;  ">STEREO VIEWFINDER</a></li><li><a href="uid73.html&#10;&#9;&#9;  ">SmartEnergy</a></li><li><a href="uid76.html&#10;&#9;&#9;  ">SmartServoFramework</a></li></ul></div>
      <div class="TdmEntry">New Results<ul><li><a href="uid81.html&#10;&#9;&#9;  ">Attention-Based Navigation</a></li><li><a href="uid82.html&#10;&#9;&#9;  ">SPOK: End User Programming for Smart Homes</a></li><li><a href="uid83.html&#10;&#9;&#9;  ">Qualitative approaches for building energy management</a></li><li><a href="uid84.html&#10;&#9;&#9;  ">Situation Aware Services on Mobile Devices </a></li><li><a href="uid85.html&#10;&#9;&#9;  ">Perceiving mass in mixed reality </a></li><li><a href="uid86.html&#10;&#9;&#9;  ">Pseudo-haptic feedback </a></li></ul></div>
      <div class="TdmEntry">Bilateral Contracts and Grants with Industry<ul><li><a href="uid88.html&#10;&#9;&#9;  ">Bilateral Contracts with Industry</a></li></ul></div>
      <div class="TdmEntry">Partnerships and Cooperations<ul><li><a href="uid92.html&#10;&#9;&#9;  ">Nationally Funded Projects</a></li><li><a href="uid104.html&#10;&#9;&#9;  ">International Initiatives</a></li><li><a href="uid107.html&#10;&#9;&#9;  ">European Projects</a></li><li><a href="uid112.html&#10;&#9;&#9;  ">International Research Visitors</a></li></ul></div>
      <div class="TdmEntry">Dissemination<ul><li><a href="uid119.html&#10;&#9;&#9;  ">Promoting Scientific Activities</a></li><li><a href="uid175.html&#10;&#9;&#9;  ">Teaching - Supervision - Juries</a></li></ul></div>
      <div class="TdmEntry">
        <div>Bibliography</div>
      </div>
      <div class="TdmEntry">
        <ul>
          <li>
            <a id="tdmbibentmajor" href="bibliography.html">Major publications</a>
          </li>
          <li>
            <a id="tdmbibentyear" href="bibliography.html#year">Publications of the year</a>
          </li>
          <li>
            <a id="tdmbibentfoot" href="bibliography.html#References">References in notes</a>
          </li>
        </ul>
      </div>
    </div>
    <div id="main">
      <div class="mainentete">
        <div id="head_agauche">
          <small><a href="http://www.inria.fr">
	    
	    Inria
	  </a> | <a href="../index.html">
	    
	    Raweb 
	    2015</a> | <a href="http://www.inria.fr/en/teams/prima">Presentation of the Project-Team PRIMA</a></small>
        </div>
        <div id="head_adroite">
          <table class="qrcode">
            <tr>
              <td>
                <a href="prima.xml">
                  <img style="align:bottom; border:none" alt="XML" src="../static/img/icons/xml_motif.png"/>
                </a>
              </td>
              <td>
                <a href="prima.pdf">
                  <img style="align:bottom; border:none" alt="PDF" src="IMG/qrcode-prima-pdf.png"/>
                </a>
              </td>
              <td>
                <a href="../prima/prima.epub">
                  <img style="align:bottom; border:none" alt="e-pub" src="IMG/qrcode-prima-epub.png"/>
                </a>
              </td>
            </tr>
            <tr>
              <td/>
              <td>PDF
</td>
              <td>e-Pub
</td>
            </tr>
          </table>
        </div>
      </div>
      <!--FIN du corps du module-->
      <br/>
      <div class="bottomNavigation">
        <div class="tail_aucentre">
          <a href="./uid15.html" accesskey="P"><img style="align:bottom; border:none" alt="previous" src="../static/img/icons/previous_motif.jpg"/> Previous | </a>
          <a href="./uid0.html" accesskey="U"><img style="align:bottom; border:none" alt="up" src="../static/img/icons/up_motif.jpg"/>  Home</a>
          <a href="./uid27.html" accesskey="N"> | Next <img style="align:bottom; border:none" alt="next" src="../static/img/icons/next_motif.jpg"/></a>
        </div>
        <br/>
      </div>
      <div id="textepage">
        <!--DEBUT2 du corps du module-->
        <h2>Section: 
      Research Program</h2>
        <h3 class="titre3">Perception for Social Interaction</h3>
        <p>Affective Computing, Perception for social interaction.
</p>
        <p>Current research on perception for interaction primarily focuses on recognition and communication of linguistic signals. However, most human-to-human interaction is non-verbal and highly dependent on social context. A technology for natural interaction requires abilities to perceive and assimilate non-verbal social signals, to understand and predict social situations, and to acquire and develop social interaction skills.</p>
        <p>The overall goal of this research program is to provide the scientific and technological foundations for systems that observe and interact with people in a polite, socially appropriate manner.
We address these objectives with research activities in three interrelated areas:</p>
        <ul>
          <li>
            <p class="notaparagraph"><a name="uid22"> </a>Multimodal perception for social interactions.</p>
          </li>
          <li>
            <p class="notaparagraph"><a name="uid23"> </a>Learning models for context aware social interaction, and</p>
          </li>
          <li>
            <p class="notaparagraph"><a name="uid24"> </a>Context aware systems and services.</p>
          </li>
        </ul>
        <p>Our approach to each of these areas is to draw on models and theories from the cognitive and social sciences, human factors, and software architectures to develop new theories and models for computer vision and multi-modal interaction. Results will be developed, demonstrated and evaluated through the construction of systems and services for polite, socially aware interaction in the context of smart habitats.</p>
        <a name="uid25"/>
        <h4 class="titre4">Detailed Description</h4>
        <p>First part of our work on perception for social interaction has concentrated on measuring the physiological parameters of Valence, Arousal and Dominance using visual observation form environmental sensors as well as observation of facial expressions.</p>
        <p>People express and feel emotions with their face. Because the face is both externally visible and the seat of emotional expression, facial expression of emotion plays a central role in social interaction between humans. Thus visual recognition of emotions from facial expressions is a core enabling technology for any effort to adapt systems for social interaction.</p>
        <p>Constructing a technology for automatic visual recognition of emotions requires solutions to a number of hard challenges. Emotions are expressed by coordinated temporal activations of 21 different facial muscles assisted by a number of additional muscles. Activations of these muscles are visible through subtle deformations in the surface structure of the face. Unfortunately, this facial structure can be masked by facial markings, makeup, facial hair, glasses and other obstructions. The exact facial geometry, as well as the coordinated expression of muscles is unique to each individual. In additions, these deformations must be observed and measured under a large variety of illumination conditions as well as a variety of observation angles. Thus the visual recognition of emotions from facial expression remains a challenging open problem in computer vision.</p>
        <p>Despite the difficulty of this challenge, important progress has been made in the area of automatic recognition of emotions from face expressions. The systematic cataloging of facial muscle groups as facial action units by Ekman <a href="./bibliography.html#prima-2015-bid15">[34]</a>  has let a number of research groups to develop libraries of techniques for recognizing the elements of the FACS coding system <a href="./bibliography.html#prima-2015-bid16">[27]</a> . Unfortunately, experiments with that system have revealed that the system is very sensitive to both illumination and viewing conditions, as well as the difficulty in interpreting the resulting activation levels as emotions. In particular, this approach requires a high-resolution image with a high signal-to-noise ratio obtained under strong ambient illumination. Such restrictions are not compatible with the mobile imaging system used on tablet computers and mobile phones that are the target of this effort.</p>
        <p>As an alternative to detecting activation of facial action units by tracking individual face muscles, we propose to measure physiological parameters that underlie emotions with a global approach. Most human emotions can be expressed as trajectories in a three dimensional space whose features are the physiological parameters of Pleasure-Displeasure, Arousal-Passivity and Dominance-Submission. These three physiological parameters can be measured in a variety of manners including on-body accelerometers, prosody, heart-rate, head movement and global face expression.</p>
        <p>In our work, we address the recognition of social behaviours multimodal information.
These are unconscious inmate cognitive processes that are vital to human communication and interaction.
Recognition of social behaviours enables anticipation and improves the quality of interaction between humans.
Among social behaviours, we have focused on engagement, the expression of intention for interaction.
During the engagement phase, many non-verbal signals are used to communicate the intention to engage to the partner <a href="./bibliography.html#prima-2015-bid17">[54]</a> . These include posture, gaze, spatial information, gestures, and vocal cues.</p>
        <p>For example, within the context of frail or elderly people at home, a companion robot must also be able to detect the engagement of humans in order to adapt their responses during interaction with humans to increase their acceptability.
Classical approaches for engagement with robots use spatial information such as human position and speed, human-robot distance and the angle of arrival.
Our believe is that uni-modal methods may be suitable for static display <a href="./bibliography.html#prima-2015-bid18">[55]</a>  and robots in wide space area <a href="./bibliography.html#prima-2015-bid19">[43]</a>  but not for home environments.
In an apartment, relative spatial information of people and robot are not as discriminative as in an open space.
Passing by the robot in a corridor should not lead to an engagement detection, and possible socially inappropriate behaviour by the robot.</p>
        <p>In our experiments, we used a kompai robot from Robosoft <a href="./bibliography.html#prima-2015-bid20">[26]</a> .
As an alternative to wearable physiological sensors (such as pulse bracelet Cardiocam, etc.) we integrate multimodal features using a Kinect sensor (see figure <a title="Perception for Social Interaction" href="./uid21.html#uid26">1</a> ).
In addition of the spatial cues from the laser telemeter, one can use new multimodal features based on persons and skeletons tracking, sound localization, etc.
Some of these new features are inspired from results in cognitive science
domain <a href="./bibliography.html#prima-2015-bid21">[51]</a> .</p>
        <div align="center" style="margin-top:10px">
          <a name="uid26">
            <!--...-->
          </a>
          <table title="" class="objectContainer">
            <caption align="bottom"><strong>Figure
	1. </strong>On the left image, one can see the telemeter range in red, the foot detection (blue spot) and the angle view from the Kinect (in green). the middle and right image represent RGB camera in depth view from the Kinect.</caption>
            <tr align="center">
              <td>
                <table>
                  <tr>
                    <td xmlns="" style="height:3px;" align="center">
                      <img xmlns="http://www.w3.org/1999/xhtml" style="width:427.0pt" alt="IMG/Features.png" src="IMG/Features.png"/>
                    </td>
                  </tr>
                </table>
              </td>
            </tr>
          </table>
        </div>
        <p>Our multimodal approach has been confronted to a robot centered dataset for
multimodal social signal processing recorded in a home-like environment
<a href="./bibliography.html#prima-2015-bid22">[32]</a> . The evaluation on our corpus highlights its
robustness and validates use of such technique in real
environment. Experimental validation shows that the use of multimodal
sensors gives better results than only spatial features (50% of error
reduction). Our experimentations also confirm results from
<a href="./bibliography.html#prima-2015-bid21">[51]</a> : relative shoulder rotation, speed and facing
visage are among crucial features for engagement detection.</p>
      </div>
      <!--FIN du corps du module-->
      <br/>
      <div class="bottomNavigation">
        <div class="tail_aucentre">
          <a href="./uid15.html" accesskey="P"><img style="align:bottom; border:none" alt="previous" src="../static/img/icons/previous_motif.jpg"/> Previous | </a>
          <a href="./uid0.html" accesskey="U"><img style="align:bottom; border:none" alt="up" src="../static/img/icons/up_motif.jpg"/>  Home</a>
          <a href="./uid27.html" accesskey="N"> | Next <img style="align:bottom; border:none" alt="next" src="../static/img/icons/next_motif.jpg"/></a>
        </div>
        <br/>
      </div>
    </div>
  </body>
</html>
