Section: Research Program

Objective 1: Understanding humans interacting with the digital world

Our first objective is centered on the human side. Our finality is not to enhance the general knowledge about the human being as a research team in psychology would do. Instead, we focus on human skills and behaviors during interaction processes. To this end, we conduct experiments that allow us to better understand what the users like, where and why they have difficulties. Thanks to these investigations, we are able to design interaction techniques and systems (described in Objective 2) that are well suited to the targeted users. We believe that this fundamental piece of work is the first step that is required for the design of usable popular interactions. We are particularly interested in 3D interaction tasks for which we design dedicated experiments. We also propose a new approach based on physiological and brain (ElectroEncephaloGraphy - EEG) signals for the evaluation of these interactions.

Interacting with 3D

In the scope of the national project InSTInCT (ANR), we have studied how users tend to interact with a touchscreen for interacting with 3D content. Indeed, whereas such kind of interaction has been extensively studied for 2D contexts, it has been little explored in 3D. However, we believe that it is fundamental to understand users' strategies and preferences well in order to promote 3D interaction on touch screens. We conducted a set of experiments to investigate such kind of interaction. We proposed guidelines to help designers in the creation of more user friendly tools. Such kind of study led to the design of tBox. We also conducted experiments to better understand how users manage to control finger pressure, and how they tend to use this input modality. In another work, we have studied the impact of directness when manipulating 3D content on multitouch screens. This allowed us to gain knowledge about users performance in touch-based interaction.

Evaluating 3DUIs with physiological signals

We recently started to explore a new approach to HCI evaluations: using various physiological signals, and notably EEG signals, as a new complementary tool to assess objectively and more precisely the ergonomic quality of a given 3DUI. In particular we aim at using physiological signals to identify where and when the pros and cons of this interface are, based on the user's mental state during interaction. For instance, estimating the user's mental workload during interaction can give insights about where and when the interface is cognitively difficult to use. Such tools could prove very promising to improve evaluations by complementing existing tools (e.g., questionnaires or interviews) that can suffer from reporting bias, can disturb the user, or only provide an a-posteriori global (but undetailled) evaluation of the interaction. So far, we studied the different kinds of mental states that can be estimated from EEG signals and that are valuable for HCI and user evaluations. We also obtained promising first results suggesting that the level of comfort during stereoscopic visualization could be estimated from EEG signals, hence opening the way to faster, more objective and more individualized stereoscopic display design and calibration. Still with the objective of estimating various users' mental states to refine system evaluations and users' understanding, we explored mental stress (a.k.a., mental workload) and social stress (pressure due to a social evaluation) estimation from brain and physiological signals. To this end, we first had to design a protocol to induce mental stress and social stress, which we did successfully. Then, we were able to calibrate stress recognition from EEG and physiological signals as well as to assess the accuracy of the stress estimators. Then, we managed to robustly estimate mental stress levels from EEG and physiological signals (EEG being the most robust modality), even accross different contexts, here accross different levels of social stress. This is an interesting step towards robust estimation of mental stress in realistic conditions. Finally, we also studied and reviewed emotion recognition from EEG signals, which, again, is another interesting mental state to consider during an HCI evaluation.

Interacting with Brain-Computer Interfaces

Finally, we also studied how humans interact with a specific HCI: Brain-Computer Interfaces (BCI). Indeed, although EEG-based BCIs are very promising for numerous applications, e.g., rehabilitation or gaming, they mostly remain prototypes not used outside laboratories, due to their low reliability. Poor BCI performances are partly due to imperfect EEG signal processing algorithms but also to the user, who may not be able to produce reliable EEG patterns. Indeed, BCI use is a skill, requiring the user to be properly trained to achieve BCI control. If he/she cannot perform the desired mental commands, no signal processing algorithm could identify them. Therefore, rather than improving EEG signal processing alone (which is what most current BCI research is about), we proposed to also guide users to learn BCI control mastery. We actually studied some theoretical models and guidelines from psychology and cognitive sciences about human learning, which revealed the many theoretical limitations of current standard BCI training approaches. We also conducted some actual experiments to further illustrate some limitations of current BCI training protocols and try to understand and analyse them. Finally, we explored new feedback types and new EEG visualization techniques in order to help users to learn BCI control skills more efficiently. These new feedback and visualizations notably aim at providing BCI users with more information about their EEG patterns using, in order to identify more easily relevant BCI control strategies, as well as motivating and engaging them in the learning task. This was achieved using augmented reality displays of the activity on the whole cortex - using an approach entitled the "Mind-Mirror", or by using multiplayer video game-based BCI training. Overall, this line of research seem largely unexplored but promising, and we are currently investing increasingly more research efforts into it.