Previous | 
Home
Section: New Results
Natural Interaction with Robotics Systems
Thanks to the arrival of Pauline Maurice and the AnDy H2020 project, our activities about interaction are currently focused on ergnonomic interaction, which requires good foundations in motion analysis.
Digital human modeling for collaborative robotics
Participant : Pauline Maurice.
Collaboration with Vincent Padois (Inria Bordeaux and Sorbonne Université), Yvan Measson (CEA-LIST) and Philippe Bidaud (ONERA and Sorbonne Université).
Work-related musculoskeletal disorders in industry represent a major and growing health problem in many developed countries. Collaborative robotics, which allows the joint manipulation of objects by both a robot and a person, is a possible solution provided that it is possible to assess the ergonomic benefit they offer. Using a digital human model (DHM) can cut down the development cost and time by replacing the physical mock-up by a virtual one easier to modify. The first part of this work details the challenges of digital ergonomic assessment for collaborative robotics. State-of-the-art work on DHM simulations with collaborative robots is reviewed to identify which questions currently remain open. The second part of this work focuses on a specific use case and presents a DHM-based method to optimize design parameters of a collaborative robot for an industrial task.
Publication: [21]
Probabilistic decision making for collaborative robotics
Participants : Yang You, Vincent Thomas, Olivier Buffet, François Charpillet, Francis Colas.
Collaboration with Rachid Alami (LAAS, France).
This work is part of the ANR Flying Co-Worker project and focuses on high-level decision making for collaborative robotics. When a robot has to assist a human worker, it does not have direct access to his current intention or his preferences but has to adapt its behaviour to help the human completing his task. To achieve this, we followed what has been proposed by [31] to model a situation of interaction as a Partially Observable Markov Decision Process (POMDP) by assuming that (i) the robot and the human act sequentially, one after another, and that (ii) the human is rational and makes his decision without considering the future robot's action.
Ativity recognition and prediction
Participants : François Charpillet, Francis Colas, Serena Ivaldi, Niyati Rawal, Vincent Thomas.
This work is part of the ANR Flying Co-Worker project and focuses on activity recognition and long-term prediction for collaborative robotics. Recognizing and predicting human activities is fundamental for a robot to help a human. Previous work in the team on activity recognition [6] rely on Hidden Markov Models (HMM) with, in particular, the Markov assumption stating that the distribution on the next state is independent from former states given the current state. This assumption, at the heart of the recurrent expression of the inference in HMM, has the unfortunate consequence to constrain the a priori distribution on the duration in each state to exponential distributions. However, it can be observed in datasets that this is not the case for many activities, which have a typical duration. This discrepancy is negligeable for recognition where HMM models achieve good performance thanks to the observations, but prevents longer-term activity prediction.
In the master project of Niyati Rawal, we investigated a slightly different model, Explicit Duration Hidden Markov Model (EDHMM), in which the duration of the activity can be modeled more finely. Preliminary results show that the recognition performance was similar to HMM but with a better prediction performance.
Humanoid Whole-Body Movement Optimization from Retargeted Human Motions
Participants : Waldez Azevedo Gomes Junior, Vishnu Radhakrishnan, Luigi Penco, Valerio Modugno, Jean-Baptiste Mouret, Serena Ivaldi.
Motion retargeting and teleoperation are powerful tools to demonstrate complex whole-body movements to humanoid robots: in a sense, they are the equivalent of kinesthetic teaching for manipulators. However, retargeted motions may not be optimal for the robot: because of different kinematics and dynamics, there could be other robot trajectories that perform the same task more efficiently, for example with less power consumption. We propose to use the retargeted trajectories to bootstrap a learning process aimed at optimizing the whole-body trajectories w.r.t. a specified cost function. To ensure that the optimized motions are safe, i.e., they do not violate system constraints, we used constrained optimization algorithms. We compared both global and local optimization approaches, since the optimized robot solution may not be close to the demonstrated one. We evaluated our framework with the humanoid robot iCub on an object lifting scenario, initially demonstrated by a human operator wearing a motion-tracking suit. By optimizing the initial retargeted movements, we can improve robot performance by over 40%.
Publication: [14]
Tele-operation of Humanoids
Participants : Luigi Penco, Waldez Gomes, Valerio Modugno, Serena Ivaldi.
We envision a world where robots can act as physical avatars and effectively replace humans in hazardous scenarios by means of teleoperation, which we see as a particular way of interacting with a robot. However, teleoperating humanoids is a challenging task because of differences in kinematics (e.g., structure and joint limits) and dynamics (e.g., mass distribution, inertia) are still significant. Another crucial issue is ensuring the dynamic balance of the robot while trying to imitate the human motion. We propose a multi-mode teleoperation framework for controlling humanoid robots for loco-manipulation tasks that address the aforementioned challenges by using two levels of teleoperation: a low-level for manipulation, realized via whole-body teleoperation, and a high-level for locomotion, based on the generation of reference velocities that are then tracked by the humanoid. We believe that this combination of different modes of teleoperation will considerably ease the burden of controlling humanoids, ultimately increasing their adaptability to complex situations which cannot be handled satisfactorily by fully autonomous systems.
Publication: [11]
Activity Recognition for Ergonomics Assessment of Industrial Tasks with Automatic Feature Selection
Participants : Adrien Malaisé, Pauline Maurice, Francis Colas, Serena Ivaldi.
In industry, ergonomic assessment is currently performed manually based on the identification of postures and actions by experts. We aim at proposing a system for automatic ergonomic assessment based on activity recognition. In this work, we define a taxonomy of activities, composed of four levels, compatible with items evaluated in standard ergonomic worksheets. The proposed taxonomy is applied to learn activity recognition models based on Hidden Markov Models. We also identify dedicated sets of features to be used as input of the recognition models so as to maximize the recognition performance for each level of our taxonomy. We compare three feature selection methods to obtain these subsets. Data from 13 participants performing a series of tasks mimicking industrial tasks are collected to train and test the recognition module. Results show that the selected subsets allow us to successfully infer ergonomically relevant postures and actions.
Publication: [6]
Human movement and ergonomics: An industry-oriented dataset for collaborative robotics
Participants : Pauline Maurice, Adrien Malaisé, Serena Ivaldi.
With the participation of Clélie Amiot, Nicolas Paris and Guy-Junior Richard, interns from Université de Lorraine during the summer 2018.
Improving work conditions in industry is a major challenge that can be addressed with new emerging technologies such as collaborative robots. Machine learning techniques can improve the performance of those robots, by endowing them with a degree of awareness of the human state and ergonomics condition. The availability of appropriate datasets to learn models and test prediction and control algorithms, however, remains an issue. This work presents a dataset of human motions in industry-like activities, fully labeled according to the ergonomics assessment worksheet EAWS, widely used in industries such as car manufacturing. Thirteen participants performed several series of activities, such as screwing and manipulating loads under different conditions, resulting in more than 5 hours of data. The dataset contains the participants’ whole-body kinematics recorded both with wearable inertial sensors and marker-based optical motion capture, finger pressure force, video recordings, and annotations by three independent annotators of the performed action and the adopted posture following the EAWS postural grid. Sensor data are available in different formats to facilitate their reuse. The dataset is intended for use by researchers developing algorithms for classifying, predicting, or evaluating human motion in industrial settings, as well as researchers developing collaborative robotics solutions that aim at improving the workers' ergonomics. The annotation of the whole dataset following an ergonomics standard makes it valuable for ergonomics-related applications, but we expect its use to be broader in the robotics, machine learning, and human movement communities.
Publication: [8]
Objective and Subjective Effects of a Passive Exoskeleton on Overhead Work
Participants : Pauline Maurice, Serena Ivaldi.
Collaboration with Jernej Čamernik, Daša Gorjan and Jan Babič (Jozef Stefan Institute, Ljubljana, Slovenia), with Benjamin Schirrmeister and Jonas Bornmann (Otto Bock SE & Co. KGaA, Duderstadt, Germany), with Luca Tagliapietra, Claudia Latella and Daniele Pucci (Istituto Italiano di Tecnologia, Genova, Italy), and with Lars Fritzsche (IMK Automotive, Chemitz, Germany).
Overhead work is a frequent cause of shoulder work-related musculoskeletal disorders. Exoskeletons offering arm support have the potential to reduce shoulder strain, without requiring large scale reorganization of the workspace. Assessment of such systems however requires to take multiple factors into consideration. This work presents a thorough in-lab assessment of PAEXO, a novel passive exoskeleton for arm support during overhead work. A list of evaluation criteria and associated performance metrics is proposed to cover both objective and subjective effects of the exoskeleton, on the user and on the task being performed. These metrics are measured during a lab study, where 12 participants perform an overhead pointing task with and without the exoskeleton, while their physical, physiological and psychological states are monitored. Results show that using PAEXO reduces shoulder physical strain as well as global physiological strain, without increasing low back strain nor degrading balance. These positive effects are achieved without degrading task performance. Importantly, participant' opinions of PAEXO are positive, in agreement with the objective measures. Thus, PAEXO seems a promising solution to help prevent shoulder injuries and diseases among overhead workers, without negatively impacting productivity.
Assessing and improving human movements using sensitivity analysis and digital human simulation
Participant : Pauline Maurice.
Collaboration with Vincent Padois (Inria Bordeaux and Sorbonne Université), Yvan Measson (CEA-LIST) and Philippe Bidaud (ONERA and Sorbonne Université).
Enhancing the performance of technical movements aims both at improving operational results and at reducing biomechanical demands. Advances in human biomechanics and modeling tools allow to evaluate human performance with more and more details. Finding the right modifications to improve the performance is, however, still addressed with extensive time consuming trial-and-error processes. This work presents a framework for easily assessing human movements and automatically providing recommendations to improve their performances. An optimization-based whole-body controller is used to dynamically replay human movements from motion capture data, to evaluate existing movements. Automatic digital human simulations are then run to estimate performance indicators when the movement is performed in many different ways. Sensitivity indices are thereby computed to quantify the influence of postural parameters on the performance. Based on the results of the sensitivity analysis, recommendations for posture improvement are provided. The method is successfully validated on a drilling activity.
Publication: [9]
Human Motion analysis for assistance
Participants : François Charpillet, Jessica Colombel.
Collaboration with David Daney (Inria Bordeaux, Auctus Team)
Different sort of sensors can be used for rehabilitation at home. This year we have evaluated the usabily of a Kinect 2. The proposed approach is to improve joint angle estimates. It is based on a constrained extended Kalman Filter that tracks inputted measured joint centers. Since the proposed approach uses a biomechanical model, it allows to obtain physically consistent constrained joint angles and constant segment lengths. A practical method, that is not sensor specific, for the optimal tuning of the extended Kalman filter covariance matrices is provided. It uses reference data obtained from a stereophotogrammetric system but it has to be tuned only once since it is task specific only. The improvement of optimal tuning over classical methods for setting the covariance matrices is shown with a statistical parametric mapping analysis. The proposed approach was tested with six healthy subjects performing 4 rehabilitation tasks. Joint estimates accuracy was assessed with a reference stereophotogrammetric system. Even if some joints such as the internal/external rotations were not well estimated, the proposed optimized algorithm reached a satisfactory average root mean square difference of 9.7deg and a correlation coefficient 0.86 of for all joints. Our results show that affordable RGB-D sensor can be used for simple in-home rehabilitation when using a constrained biomechanical model.
A work carried out this year, takes the search for a sensor for personal assistance a step further with the study of the new Kinect Azure. Human-robot interaction requires a robust estimate of human motion in real-time. This work presents a fusion algorithm for joint center positions tracking from multiple depth cameras to improve human motion analysis accuracy. The proposed algorithm is based on body tracking measurements fusion with an extended Kalman filter and anthropomorphic constraints. However, the effectiveness and robustness of such algorithm depends on the A direct comparison of joint center positions estimated with a reference stereophotogrammetric system and the ones estimated with the new Kinect 3 (Azure Kinect) sensor and its older version the Kinect 2 (Kinect for Windows) has been made. The proposed approach improves body tracker data even for Kinect 3 which has not the same characteristics than Kinect 2. This study shows also the importance of defining good heuristics to merge data depending on how the body tracking works. Thus, with proper heuristics, the joint center position estimates are improved by at least 14.6 %. Finally, we propose an additional comparison between Kinect 2 and Kinect 3 exhibiting the pros and cons of the two sensors. This study is now in submission for an international conference.
Finally, a state of the art on biological motion was realized. The purpose of this study is to understand and develop methods for decomposing motion. The EWalk dataset (http://gamma.cs.unc.edu/GAIT/#EWalk) will allow us to test emotion recognition from simple decompositions and classifiers. Then, we will extend the methods to other cognitive parameters.
Reliable localization of pedestrians in a smart home using multi-sensor data fusion
Participants : François Charpillet, Lina Achaji.
Collaboration with Maan Badaoui EL Najjar(Cristal Laboratory Lille, DiCOT Team), Mohamad Daher (the Lebanese University Faculty of technology, Tripoli)
One objective of the Larsen team is to develop technologies allowing older people to live independently as long as possible in their own homes instead of in specialized institutions. However, elderly people face physical problems that reduce their autonomy, and consequently their capacity to achieve daily activities. The integration of environmental or body sensors in what is called nowadays smart habitats is a solution that is appealing to provide a better quality of life with safer conditions. Localization and tracking of people in indoor environments are one of the primary services to be developed to follow them up at home, permitting to evaluate their physical states through the observation of their Activities of Daily Living (ADL). We proposed during the internship of Lina Achaji to localize and track the center of pressure (CoP) of people (one or two) in a smart home using a load sensing floor equipped with around 400 load sensors as well as wearable sensors. The data fusion is made using an informational filter where an inverted pendulum bio-mechanical model is introduced. The obtained results are very promising and were validated using a motion tracking system and force plates.
Publication: [4]
Ambient assisting living
Participants : François Charpillet, Yassine El Khadiri.
Collaboration with Cedric Rose from Diatelic compagny.
The ageing of the population confronts modern societies with an unprecedented demographic transformation. These include the imbalance in our pension systems and the cost of caring for the elderly. On this last point, apart from the economic aspects, the placement of elderly people is often only a choice of reason and can be quite badly experienced by people. One response to this societal problem is the development of technologies that make it easier to keep elderly people at home. The state of the art in this field abounds with upstream projects that are moving in this direction. Many of them are seeking to develop home monitoring systems. Their objectives are to detect and even prevent the occurrence of worrying or critical situations and to assess the physical condition or even fragility of the people being monitored. It is within this framework that this contribution is made. In this work, we have focused on the particular problem of monitoring the quality of sleep as well as the detection of nocturnal waking of a person living alone at home. The home is equipped with simple ambient sensors such as binary motion detectors. We have developed a Bayesian inference method that allows our solution to be flexible and robust enough for different types of installations and apartment configurations while maintaining a prediction accuracy of 0.94. This solution is currently being deployed on several dozen apartments in Lorraine by Diatelic and Pharmagest compagnies.
Publication: [15]