Section: New Results

Natural Interaction with Robotics Systems

Control of Interaction

Because of the AnDy project, we are currently focused on interaction in industrial contexts, in particular to encourage ergonomic motions.

Robust Real-time Whole-Body Motion Retargeting from Human to Humanoid

Participants : Serena Ivaldi, Luigi Penco, Brice Clement, Jean-Baptiste Mouret.

Transferring the motion from a human operator to a humanoid robot is a crucial step to enable robots to learn from and replicate human movements. The ability to retarget in real-time whole-body motions that are challenging for the humanoid balance is critical to enable human to humanoid teleoperation. In this work, we design a retargeting framework that allows the robot to replicate the motion of the human operator, acquired by a wearable motion capture suit, while maintaining the whole-body balance. We introduce some dynamic filter in the retargeting to forbid dangerous motions that can make the robot fall. We validate our approach through several experiments on the iCub robot, which has a significantly different body structure and size from the one of the human operator.

Publication: [24]

Prediction of Human Whole-Body Movements with AE-ProMPs

Participants : Serena Ivaldi, Oriane Dermy, Francis Colas, François Charpillet.

The ability to predict intended movements is crucial for collaborative robots to anticipate the human actions and for assistive technologies to alert if a particular movement is non-ergonomic and potentially dangerous for humans. In this paper, we address the problem of predicting the future human whole-body movements given early observations. We propose to predict the continuation of the high-dimensional trajectories mapped into a reduced latent space, using autoencoders (AE). The prediction is based on a probabilistic description of the movement primitives (ProMPs) in the latent space, which notably reduces the computational time for the prediction to occur, and hence enables to use the method in real-time applications. We evaluate our method, named AE-ProMPs, for predicting future movements belonging to a dataset of 7 different actions performed by a human, recorded by a wearable motion tracking suit.

Publication: [13]

Publications: [28],

Generating Assistive Humanoid Motions for Co-Manipulation Tasks with a Multi-Robot Quadratic Program Controller

Participants : Karim Bouyarmane, Serena Ivaldi.

Human-humanoid collaborative tasks require that the robot takes into account the goals of the task, interaction forces with the human, and its own balance. We present a formulation for a real-time humanoid controller which allows the robot to keep itself stable, while also assisting the human in achieving their shared objectives. This is achieved with a multi-robot quadratic program controller, which solves for human motion reconstruction and optimal robot controls in a single optimization problem. Our experiments on a simulated robot platform demonstrate the ability to generate interactions motions and forces that are similar to what a human collaborator would produce.

Publication: [21]

Activity Recognition With Multiple Wearable Sensors for Industrial Applications

Participants : Francis Colas, Serena Ivaldi, Adrien Malaisé, Pauline Maurice, François Charpillet.

We address the problem of recognizing the current activity performed by a human operator, providing an information useful for automatic ergonomic evaluation for industrial applications. While the majority of research in activity recognition relies on cameras observing the human, here we explore the use of wearable sensors, which are more suitable in industrial environments. We use a wearable motion tracking suit and a sensorized glove. We describe our approach for activity recognition with a probabilistic model based on Hidden Markov Models, applied to the problem of recognizing elementary activities during a pick-and-place task inspired by a manufacturing scenario. We show that our model is able to correctly recognize the activities with 96% of precision if both sensors are used.

Publication: [19],[19]

Activity Recognition for monitoring eldely people at home

Participants : Yassine El Khadiri, François Charpillet.

Early detection of frailty signs is important for senior people who prefer to keep living in their homes instead of moving to a nursing home. Sleep quality is a good predictor for frailty monitoring. Thus we are interested in tracking sleep parameters like sleep wake patterns to predict and detect potential sleep disturbances of the monitored senior residents. We use an unsupervised inference method based on actigraphy data generated by ambient motion sensors scattered around the senior’s apartment. This enables our monitoring solution to be flexible and robust to the different types of housings it can equip while still attaining accuracy of 0.94 for sleep period estimates.

Publication: [15]

Ethics

Ethical and Social Considerations for the Introduction of Human-Centered Technologies at Work

Participants : Serena Ivaldi, Adrien Malaisé, Pauline Maurice, Ludivine Allienne.

Human-centered technologies such as collaborative robots, exoskeletons, and wearable sensors are rapidly spreading in industry and manufacturing because of their intrinsic potential at assisting workers and improving their working conditions. The deployment of these technologies, albeit inevitable, poses several ethical and societal issues. Guidelines for ethically aligned design of autonomous and intelligent systems do exist, however we argue that ethical recommendations must necessarily be complemented by an analysis of the social impact of these technologies.

In a recent paper[20], we report on our preliminary studies on the opinion of factory workers and of people outside this environment on human-centered technologies at work. In light of these studies, we discuss ethical and social considerations for deploying these technologies in a way that improves acceptance.

Publication: [20]