Section: New Results

Assistance

Smart Environment for Human Behaviour Recognition

Participants : Mohamed Hedi Amri, Alain Coulbois, Artem Melnyk, Aurélien Massein, Yves Papegay, Odile Pourtallier [correspondante] .

The general aim of this research activity focuses on long term indoor monitoring of frail persons. In particular we are interested in early detection of daily routine and activity modifications. These modifications may indicate health condition alteration of the person and may require further medical or family care. Note that our work does not aim at detecting brutal modifications such as faintness or fall.

In our research we envisage both individual and collective housing such as rehabilitation center or retirement home.

Our work relies on the following leading ideas :

• We do not base our monitoring system on wearable devices since it appears that they may not be well accepted and worn regularly,

• Privacy advocates adequacy between the monitoring level needed by a person and the detail level of the data collected. We therefore strive to design a system fitted to the need of monitoring of the person.

• In addition to privacy concern, intrusive feature of video led us not to use it.

This year we have concentrated our effort on the first step of this research that consists in being able to locate the person in his/her indoor environment.

A natural way of being able to adapt the accuracy of localization (and consequently accuracy of monitoring), is to use a partition of the monitoring area in a finite number of elementary zones ; the number of zones together with their geometry being closely related with the pursued level of monitoring. In practice these zones will be materialized by sensors barriers that detect the passage of a person from one zone to the other. Henceforth each zone are polygonal.

Several directions have been followed this year.

• monitoring system design,

• material development,

• data gathering and analysis,

• experimentation.

Monitoring system design

We aim at designing the partition of the monitoring space. Given the geometry of the monitoring area, the admissible position of the sensors barriers and a set of points of interest, the objective is to determine the positions of a minimal number of barriers such that each zone therefore defined includes at most one point of interest. The crossing of a given secession of barriers therefore allows to determine the trajectory of a person from one point of interest to another. An algorithm for solving this problem has been developed.

Material development

We initially used commercialized Infra Red barriers to detect the crossing time from one zone to an other. Nevertheless although the collected data is sufficient for the monitoring of a single person it prove not to be sufficient in a environment where there may be several persons, which is typically the case when considering retirement home for example.

Hence we have developed a multi-sensor barrier, a box containing two infra red distance sensors and two motion sensors (passive infrared type). It has been designed and created by 3D fast prototyping printer. The box is light, cheap and discreet. In addition to detecting the crossing time, it also gives the direction of crossing together with information about the speed and the size of the crossing person or object. This last information is helpful to differentiate for example a person using a wheelchair, a valid person (e.g medical staff), or an elderly.

We use phidget interface kits connected to a fit-pc for data acquisition and recording.

Data gathering and analysis

The aim of this data processing is to transform the raw data provided by the sensor barriers in a higher level data composed by the time and direction of crossing and rough estimation of the speed and size of the object or person crossing the barrier. This information can be deduced using only the data given by the distance sensors after processing. Nevertheless in real situations the barrier may be hidden by an object (food or cleaning trolley for example), and the redundant information from PIR sensors of an other closed barrier may be useful to recover the missing information.

The data are intended to be collected on large period of time (typically months). Inline filtering and averaging techniques were used to transform large and noisy raw data to get reasonable dataset size. Figure 3 shows in blue or red the general direction of the measurement of the stations (that create detection zones) and in each zone the current estimation of the number of people in each zone (a cross indicates 0, a black circle represents one person). For example the lower left zone has between 1 and 2 people.

Experimentation

A monitoring system has been installed in the first floor of EHPAD Valrose in Nice. The area of monitoring was restricted to the hallway that leads to the individual rooms of six residents. Residents are proposed several activities (social or cultural activities, physical activities, meals) and have to use the hallway when participating to those activities. In addition to residents medical and service staff also use this hallway. The aim of this experiment is to determine an activity measure for each resident and to study its evolution with time. In that case the sensor placement is designed in such way that individual information may be obtained (e.g. by having stations on both side of the door of the individual room).

The installed system is composed of 10 multi sensor barriers installed on the wall of the hallway and 7 additional PIR sensors installed on the ceiling of the hallway. The data are transmitted by phidget interface kits and are processed by a fit-PC that store the daily data sets. A similar setup will be installed at the very beginning of 2017 in the Institut Claude Pompidou to monitor the activity in the corridors and in the waiting room. Here the medical community is more interested in statistical analysis than in individual analysis.

Sensors placement

Both economic motivations due to demographic evolution and willingness of people to live independently at home when aging, facing physical impairment or recovering from injuries has raised the need for activity monitoring at home, in rehabilitation center or in retirement home. Monitoring systems provide informations that can range from a broad measure of the daily activity to a precise analysis of the ability of a person performing a task (cooking, dressing, ...) and its evolution.

The broad range of needs and contexts, together with the large variety of available sensors implies the necessity to carefully think the design of the monitoring system. An appropriate system should be inexpensive and forgettable for the monitored person, should respect privacy but collect necessary data, and should easily adapt to stick to new needs. We aim to provide an assisting tool for designing appropriate monitoring systems.

As a second year of a PhD work, metrics have been defined to evaluate quality of sensors solutions and placement to infer people behaviors inside a smart environments. Based on these metrics, a methodology for optimal design of smart environments has been developed.

Rehabilitation

Participants : Alain Coulbois, Artem Melnyk, Jean-Pierre Merlet [correspondant] .

We have developed the specific walking aid ANG-med to be used to monitor rehabilitation exercises beside performing analysis of walking pattern as any walker of the ANG family. The main addition for this walker are two rear looking distance sensors and two of such sensor mounted on a pan-tilt head (figure 4). These sensors have been placed under the guidance of the medical community in order to monitor and assess rehabilitation exercise such as leg flexion/extension/abduction and plantar flexion.

The walker is since on year in test in the MATIA fundation in Spain. The software that is used to for this walker has been developed with the European RAPP project (see section 9.3.1.1) so that new exercise may easily be programmed and downloaded through a message passing system [9].