Section: New Results

Brain-Computer Interfaces

A new reference book in Brain-Computer Interfaces

Participants : Maureen Clerc, Laurent Bougrain [Neurosys project-team] , Fabien Lotte [POTIOC project-team] , All Ipl Bci-Lif Members.

Most of the results in Brain-Computer Interfaces are conducted in the context of the Inria Project-Lab BCI-LIFT (see contracts section). Researchers of this Inria Project-Lab (Maureen Clerc, Laurent Bougrain and Fabien Lotte) have edited a reference book in 2016 on the topic of Brain-Computer Intefaces. It consists of two volumes, “Foundations and Methods” (in French [54] and in English [1]) and “Technology and Applications” (in French [55] and in English [53]).

A Separability Marker Based on High-Dimensional Statistics for Classification Confidence Assessment

Participants : Nathalie Gayraud, Maureen Clerc, Nathanaël Foy, Alain Rakotomamonjy [University of Rouen] .

This work provides a theoretical analysis framework for features that belong to the high dimensional Riemannian manifold of symmetric positive definite matrices. In non-invasive EEG-based Brain Computer Interfaces, such as the P300 speller, these are sample covariance matrices of the epoched EEG signal that are classified into two classes. An analysis of the class shape on the manifold is performed, and the separability level of the two classes is evaluated. The main contribution is the Separability Marker (SM)-confidence method, a method that appends a confidence marker to the prediction of a binary classifier whose decision function is based on the comparison of Riemannian distances.

This work has been published in [23].

Comparison of Hierarchical and Non-Hierarchical Classification for Motor Imagery Based BCI Systems

Participants : Nathalie Gayraud, Maureen Clerc, Cecilia Lindig-León [Neurosys project-team] , Laurent Bougrain [Neurosys project-team] .

Motor imagery (MI) based BCI systems record and analyze the brain activity to determine users' intentions while imagining moving some parts of their body. In order to build systems that are able to detect several commands, multiclass schemes need to be applied. Hierarchical methods allow solving multiclass problems by using a tree of binary classifiers, whose root discriminates between two groups, each one containing a half of the classes. Each succeeding node includes again only one half of the classes from the selected group, and the process is recursively repeated until each node contains a single class, from which the final decision can be inferred. In this study we compare a series of multiclass approaches to assert the benefits of hierarchical classification. The compared methods are based on two effective techniques for MI-discrimination, namely, Common Spatial Patterns (CSP) and Riemannian geometry, for which the hierarchical and non-hierarchical approaches have been considered. We include the CSP by Joint Diagonalization method (CSP by JAD), which corresponds with a non-hierarchical approach; and its hierarchical counterpart, namely, Binary CSP. In addition, the non-hierarchical Minimum Distance to Riemannian Mean method (MDRM) [4] is also evaluated, together with its analogous hierarchical approach; a contribution of the present work called Hierarchical MDRM algorithm (HMDRM). All these methods have been applied on dataset 2a of the BCI competition IV to facilitate their comparison. The highest accuracies were reached by the BCSP and HMDRM methods, confirming the effectiveness of hierarchical algorithms.

This work has been published in [36].

Motor imagery learning using Functional Electrical Stimulation

Participants : Maureen Clerc, Saugat Bhattacharyya [CAMIN project-team] , Mitsuhiro Hayashibe [CAMIN project-team] .

Functional Electrical Stimulation (FES) stimulates the affected region of the human body thus providing a neu-roprosthetic interface to non-recovered muscle groups. FES in combination with Brain-computer interfacing (BCI) has a wide scope in rehabilitation because this system can directly link the cerebral motor intention of the users with its corresponding peripheral mucle activations. Such a rehabilitative system would contribute to improve the cortical and peripheral learning and thus, improve the recovery time of the patients. In this paper, we examine the effect of electrical stimulation by FES on the electroencephalography (EEG) during learning of a motor imagery task. The subjects are asked to perform four motor imagery tasks over six sessions and the features from the EEG are extracted using common spatial algorithm and decoded using linear discriminant analysis classifier. Feedback is provided in form of a visual medium and electrical stimulation representing the distance of the features from the hyperplane. Results suggest a significant improvement in the classification accuracy when the subject was induced with electrical stimulation along with visual feedback as compared to the standard visual one.

This work has been published in [13] and [22].

Brain training with neurofeedback

Participants : Maureen Clerc, Lorraine Perronnet [Hybrid project-team] , Anatole Lécuyer [Hybrid project-team] , Fabien Lotte [Potioc project-team] , Christian Barillot [Visages project-team] .

Neurofeedback is a promising tool for brain rehabilitation and peak performance training. Neurofeedback approaches usually rely on a single brain imaging modality such as EEG or fMRI. Combining these modalities for neurofeedback training could allow to provide richer information to the subject and could thus enable him/her to achieve faster or more specific self-regulation. Yet unimodal and multimodal neurofeedback have never been compared before. In the present work, we introduce a simultaneous EEG-fMRI experimental protocol in which participants performed a motor-imagery task in unimodal and bimodal NF conditions. With this protocol we were able to compare for the first time the effects of unimodal EEG-neurofeedback and fMRI-neurofeedback versus bimodal EEG-fMRI-neurofeedback by looking both at EEG and fMRI activations. We also propose a new feedback metaphor for bimodal EEG- fMRI-neurofeedback that integrates both EEG and fMRI signal in a single bi-dimensional feedback (a ball moving in 2D). Such a feedback is intended to relieve the cognitive load of the subject by presenting the bimodal neurofeedback task as a single regulation task instead of two. Additionally, this integrated feedback metaphor gives flexibility on defining a bimodal neurofeedback target. Participants were able to regulate activity in their motor regions in all NF conditions. Moreover, motor activations as revealed by offline fMRI analysis were stronger during EEG-fMRI-neurofeedback than during EEG-neurofeedback. This result suggests that EEG-fMRI-neurofeedback could be more specific or more engaging than EEG-neurofeedback. Our results also suggest that during EEG-fMRI-neurofeedback, participants tended to regulate more the modality that was harder to control. Taken together our results shed light on the specific mechanisms of bimodal EEG-fMRI-neurofeedback and on its added- value as compared to unimodal EEG-neurofeedback and fMRI-neurofeedback.

This work has been published in [51] and [50].

Out-of-the-lab P300 speller

Participants : Maureen Clerc, Théodore Papadopoulo, Nathanaël Foy, Federica Turi, Étienne Guerlais.

New developments have been performed in the context of ADT OpenViBE-X to robustify the P300 speller system, correcting some timing issues (in OpenViBE), and making the system easier to use and install. This has been validated by the use of our system out-of-the-lab, by a patient in Chambéry (see article  [104]).

This work has been published in [48]and [49].

Clinical study with the CoAdapt P300 speller

Participants : Maureen Clerc, Théodore Papadopoulo, Marie-Hélène Soriani [Nice University Hospital] , Claude Desnuelle [Nice University Hospital] , Violaine Guy [Nice University Hospital] , Mariane Bruno [Nice University Hospital] .

Amyotrophic Lateral Sclerosis (ALS) is a progressive neurodegenerative disease which, a few years after onset, restricts patients’ communication capacity. In this study, the usability by disabled patients with ALS of a visual Brain-Computer Interface (BCI), the P300-speller, is evaluated as a viable alternative to existing assistive communication tools. After clinical evaluation of their physical, cognitive and language capacities, 20 patients with ALS were included. The study consisted of two 3-block sessions, at 2 to 4-week interval, using a P300-speller BCI in several modes of operation in view of evaluating its usability in its 3 components: effectiveness, efficiency and satisfaction. Over all participants, the system is effective (100% of participants successfully achieved copy spelling and free spelling tasks). It is also efficient (over 95% of correct symbols were selected by 65% of participants). The average number of correct symbols/min ranged from 3.6 (without word prediction) to 5.04 (with word prediction). Participants expressed satisfaction through an average of 8.7/10 measuring comfort, ease of use and utility. Patients quickly learned how to operate this system and were able to use it with good performance without much learning effort. Word prediction and other algorithmic optimizations improve the information transfer rate and make such systems competitive with existing solutions of alternative communication such as eye trackers.

This work was published in [25].