Section: Research Program

Scientific methodology

In this section we briefly describe the scientific methods we use to achieve our research goals.

Adaptive image processing

An impressive range of techniques have been developed in the fields of image processing, computer vision and computer graphics to manipulate and interpret image content for a variety of applications. So far, only a few of these techniques have been applied in the context of vision aid systems and even less have been carefully evaluated with patients. However, it is worth noticing a recent gain of interest from the artificial vision side to low-vision applications  (See, e.g., the Special issue on Assistive Computer Vision and Robotics - "Assistive Solutions for Mobility, Communication and HMI" from Computer Vision and Image Understanding (August 2016) or the International Workshop on Assistive Computer Vision and Robotics (ECCV 2016 Satelite workshop)). A possible explanation for this comes from the technological side, as new hardware are now available (e.g., head-mounted platforms, depth cams), which make it easier to design prototypes that patients could test.

Enhancing and manipulating image content is the first natural type of image processing to consider. This covers a variety of approaches such as equalization, gamma correction, tone mapping, edge enhancement, image decomposition or cartoonization. Some of these methods have already been tested with low-vision patients  [44], [58], [59] or even in retina prosthesis systems as pre-processing steps  [43]. More sophisticated approaches have also been tested to help patients focus on the most relevant information, such as scene retargeting  [62] and seam carving  [48], [45]. Using depth information can be another way to highlight specific features of a scene. Depth can be obtained from stereo head systems or RGB-D cameras. It can be used to make 3D-based enhancements  [67], [66] and in the context of assisting visually impaired people, depth information can help them navigating in cluttered environments  [64], [52].

Our goal is to investigate which image processing could bring a real advantage to the patients in the design of vision aid systems. We study how to combine them and how to make them adapted to patients pathologies, so that they can not only "see" an image but understand it more efficiently. We work on doing this using virtual and augmented reality technology (Sec. 3.2.2).

Virtual, mixed and augmented reality

Virtual, mixed and augmented reality technology (VR/MR/AR) is becoming an important technology in many areas. It is based on the idea of combining digital words with physical realities in different ways, and it encompasses a wide spectrum of hardware. A new term is increasingly being used, which is cross reality (XR), to refer to this continuum of immersive experiences. Initially pushed by game and film industries, it appears that this technology could change our experiences in many domains such as education, journalism, media, training and healthcare.

To start experiencing XR, some cheap solutions work in conjunction with mobile phones like the Samsung GR or the simple Google Cardboard. Integrated solutions are also the market such as Oculus, OSVR (open source). New solutions appear on the market with increased capacities. Just to cite a few, let us mention the Oculus Go, the Oculus Rift or the HTC vive. For many of them, it is also planed to have eye tracking inside, as done for example in the FOVE.

Given this recent evolution and the promising perspectives, it is our conviction that this technology will play a major role in the domain of low-vision. Not only it can be useful to design novel vision aid systems and rehabilitation programs, but also this technology can help us to study the behaviour of low-vision people more precisely. Following these goals, we develop new immersive experiences to help low-vision people in their daily tasks.

Biophysical modeling

Modeling in neuroscience has to cope with several competing objective. On one hand describing the biological realm as close as possible, and, on the other hand, providing tractable equations at least at the descriptive level (simulation, qualitative description) and, when possible, at the mathematical level (i.e., affording a rigorous description). These objectives are rarely achieved simultaneously and most of the time one has to make compromises. In Biovision team we adopt the point of view of physicists: try to capture the phenomenological description of a biophysical mechanism, removing irrelevant details in the description, and try to have a qualitative description of equations behaviour at least at the numerical simulation level, and, when possible, get out analytic results. We do not focus on mathematical proofs, instead insisting on the quality of the model in predicting, and, if possible proposing new experiments. This requires a constant interaction with neuroscientists so as to keep the model on the tracks, warning of too crude approximation, still trying to construct equations from canonical principles [4], [5],[36].

Methods from theoretical physics

Biophysical models mainly consist of differential equations (ODEs or PDEs) or integro-differential equations (neural fields). We study them using dynamical systems and bifurcation theory as well as techniques coming from nonlinear physics (amplitude equations, stability analysis, Lyapunov spectrum, correlation analysis, multi-scales methods).

For the study of large scale populations (e.g., when studying population coding) we use methods coming from statistical physics. This branch of physics gave birth to mean-field methods as well statistical methods for large population analysis. We use both of them. Mean-field methods are applied for large scale activity in the retina and in the cortex [7], [11], [46].

For the study of retina population coding we use the so-called Gibbs distribution, initially introduced by Boltzmann and Gibbs. This concept includes, but is not limited to, maximum entropy models  [63] used by numerous authors in the context of the retina (see, e.g.,  [71], [73], [61], [60], [74]). These papers were restricted to a statistical description without memory neither causality: the time correlations between successive times is not considered. A paradigmatic example of this is the Ising model, used to describe the retinal activity in, e.g.,  [71], [73]. However, maximum entropy extends to spatio-temporal correlations as we have shown in, e.g., [5] [75], [33].

More generally, while maximum entropy models rely heavily on the questionable assumption of stationarity, the concept of Gibbs distribution does not need this hypothesis. Beside, it allows to handle models with large memory; it also provides a framework to model anticipation [17]. It includes as well existing models to explain retina statistics such as the Generalized Linear Model (GLM)  [51].