- A2.3. Embedded and cyber-physical systems
- A5.1. Human-Computer Interaction
- A5.6. Virtual reality, augmented reality
- A5.10. Robotics
- A5.11. Smart spaces
- A6.1. Methods in mathematical modeling
- A6.2. Scientific computing, Numerical Analysis & Optimization
- A6.4. Automatic control
- A8.4. Computer Algebra
- A8.11. Game Theory
- A9.5. Robotics
- B2.1. Well being
- B2.5. Handicap and personal assistances
- B2.7. Medical devices
- B2.8. Sports, performance, motor skills
- B3.1. Sustainable development
- B3.5. Agronomy
- B5.2. Design and manufacturing
- B5.6. Robotic systems
- B5.7. 3D printing
- B8.1. Smart building/home
- B8.4. Security and personal assistance
- B9.1. Education
- B9.2. Art
- B9.9. Ethics
1 Team members, visitors, external collaborators
- Jean-Pierre Merlet [Team leader, Inria, Senior Researcher, HDR]
- Yves Papegay [Inria, Researcher, HDR]
- Odile Pourtallier [Inria, Researcher]
- Eric Wajnberg [Institut national de recherche pour l'agriculture, l'alimentation et l'environnement, Senior Researcher, HDR]
- Toufik Bentaleb [Inria, from May 2021 until Oct 2021]
- Romain Tissot [Inria, from Sep 2021]
Interns and Apprentices
- Lila Azzaz [Inria, from Mar 2021 until Jul 2021]
- Ichrak Ben Yahia [Inria, from Mar 2021 until Aug 2021]
- Sophie Hean [Inria, from Mar 2021 until Jul 2021]
- Meriem Belinda Naamani [Inria, from Apr 2021 until Jul 2021]
- Clara Thomas [Inria, from Apr 2021 until Aug 2021]
- Jane Desplanques [Inria]
- Eric Sejor [Centre hospitalier universitaire de Nice, from Mar 2021]
- Ting Wang [ESIEE, until Oct 2021]
2 Overall objectives
HEPHAISTOS has been created as a team on January 1st, 2013 and as a project team in 2015.
The goal of the project is to set up a generic methodology for the design and evaluation of an adaptable and interactive assistive ecosystem for the elderly and the vulnerable persons that provides furthermore assistance to the helpers, on-demand medical data and may manage emergency situations. More precisely our goals are to develop devices with the following properties:
- they can be adapted to the end-user and to its everyday environment
- they should be affordable and minimally intrusive
- they may be controlled through a large variety of simple interfaces
- they may eventually be used to monitor the health status of the end-user in order to detect emerging pathology
Assistance will be provided through a network of communicating devices that may be either specifically designed for this task or be just adaptation/instrumentation of daily life objects.
The targeted population is limited to frail people 1 and the assistive devices will have to support the individual autonomy (at home and outdoor) by providing complementary resources in relation with the existing capacities of the person. Personalization and adaptability are key factor of success and acceptance. Our long term goal will be to provide robotized devices for assistance, including smart objects, that may help disabled, elderly and handicapped people in their personal life.
Assistance is a very large field and a single project-team cannot address all the related issues. Hence HEPHAISTOS will focus on the following main societal challenges:
- mobility: previous interviews and observations in the HEPHAISTOS team have shown that this was a major concern for all the players in the ecosystem. Mobility is a key factor to improve personal autonomy and reinforce privacy, perceived autonomy and self-esteem.
- managing emergency situations: emergency situations (e.g. fall) may have dramatic consequences for elderly. Assistive devices should ideally be able to prevent such situation and at least should detect them with the purposes of sending an alarm and to minimize the effects on the health of the elderly.
- medical monitoring: elderly may have a fast changing trajectory of life and the medical community is lacking timely synthetic information on this evolution, while available technologies enable to get raw information in a non intrusive and low cost manner. We intend to provide synthetic health indicators, that take measurement uncertainties into account, obtained through a network of assistive devices. However respect of the privacy of life, protection of the elderly and ethical considerations 4 impose to ensure the confidentiality of the data and a strict control of such a service by the medical community.
- rehabilitation and biomechanics: our goals in rehabilitation are 1) to provide more objective and robust indicators, that take measurement uncertainties into account to assess the progress of a rehabilitation process 2) to provide processes and devices (including the use of virtual reality) that facilitate a rehabilitation process and are more flexible and easier to use both for users and doctors. Biomechanics is an essential tool to evaluate the pertinence of these indicators, to gain access to physiological parameters that are difficult to measure directly and to prepare efficiently real-life experiments.
Addressing these societal focus induces the following scientific objectives:
design and control of a network of connected assistive
assistance devices suffer from a lack of essential functions
(communication, monitoring, localization,...) and their acceptance and
efficiency may largely be improved. Furthermore essential functions
(such as fall detection, knowledge sharing, learning, adaptation to
the user and helpers) are missing. We intend to develop new
devices, either by adapting existing systems or developing brand-new
one to cover these gaps. Their performances, robustness and
adaptability will be obtained through an original design
process, called appropriate design, that takes uncertainties
into account to determine almost all the nominal values of the
design parameters that guarantee to obtain the required
The development of these devices covers our robotics works
(therefore including robot analysis, kinematics, control, ...)
but is not limited to them. These devices will be present in the three
elements of the ecosystem (user, technological helps and
environment) and will be integrated in a common network.
The study of this robotic network and of its element is
therefore a major focus point of the HEPHAISTOS
project. In this field our
- to develop methods for the analysis of existing robots, taking into account uncertainties in their modeling that are inherent to such mechatronic devices
- to propose innovative robotic systems
- evaluation, modeling and programming of assistive ecosystem: design of such an ecosystem is an iterative process which relies on different types of evaluation. A large difference with other robotized environments is that effectiveness is not only based on technological performances but also on subjectively perceived dimensions such as acceptance or improvement of self-esteem. We will develop methodologies that cover both evaluation dimensions. Technological performances are still important and modeling (especially with symbolic computation) of the ecosystem will play a major role for the design process, the safety and the efficiency, which will be improved by a programming/communication framework than encompass all the assistance devices. Evaluation will be realized with the help of clinical partners in real-life or by using our experimental platforms.
- uncertainty management: uncertainties are especially present in all of our activities (sensor, control, physiological parameters, user behavior, ...). We intend to systematically take them into account especially using interval analysis, statistics, game theory or a mix of these tools.
- economy of assistance: interviews by the HEPHAISTOS team and market analysis have shown that cost is a major issue for the elderly and their family. At the opposite of other industrial sectors manufacturing costs play a very minor role when fixing the price of assistance devices: indeed prices result more from the relations between the players and from regulations. We intend to model these relations in order to analyze the influence of regulations on the final cost.
The societal challenges and the scientific objectives will be supported by experimentation and simulation using our development platforms or external resources.
In terms of methodologies the project will focus on the use and mathematical developments of symbolic tools (for modeling, design, interval analysis), on interval analysis (for design, uncertainties management, evaluation), on game theory (for control, localization, economy of assistance) and on control theory. Implementation of the algorithms will be performed within the framework of general purpose software such as Scilab, Maple, Mathematica and the interval analysis part will be based on the existing library ALIAS, that is still being developed mostly for internal use.
Experimental work and the development of our own prototypes are strategic for the project as they allow us to validate our theoretical work and to discover new problems that will feed in the long term the theoretical analysis developed by the team members.
Dissemination is also an essential goal of our activity as its background both on the assistance side and on the theoretical activities as our approaches are not sufficiently known in the medical, engineering and academic communities.
In summary HEPHAISTOS has as major research axes assistance robotics, modeling (see section 9.1.2), game theory, interval analysis and robotics (see section 8.1). The coherence of these axis is that interval analysis is a major tool to manage the uncertainties that are inherent to a robotized device, while assistance robotics provides realistic problems which allow us to develop, test and improve our algorithms. Our overall objectives are presented in this document and in a specific page on assistance page on assistance.
3 Research program
3.1 Interval analysis
We are interested in real-valued system solving (, ), in optimization problems, and in the proof of the existence of properties (for example, it exists such that or it exist two values , such that and ). There are few restrictions on the function as we are able to manage explicit functions using classical mathematical operators (e.g. as well as implicit functions (e.g. determining if there are parameter values of a parametrized matrix such that the determinant of the matrix is negative, without calculating the analytical form of the determinant).
Solutions are searched within a finite domain (called a box) which may be either continuous or mixed (i.e. for which some variables must belong to a continuous range while other variables may only have values within a discrete set). An important point is that we aim at finding all the solutions within the domain whenever the computer arithmetic will allow it: in other words we are looking for certified solutions. For example, for 0-dimensional system solving, we will provide a box that contains one, and only one, solution together with a numerical approximation of this solution. This solution may further be refined at will using multi-precision.
The core of our methods is the use of interval analysis that allows one to manipulate mathematical expressions whose unknowns have interval values. A basic component of interval analysis is the interval evaluation of an expression. Given an analytical expression in the unknowns and ranges for these unknowns we are able to compute a range , called the interval evaluation, such that
In other words the interval evaluation provides a lower bound of the minimum of and an upper bound of its maximum over the box.
For example if and , then , meaning that for any in [0.5,1.6] we guarantee that .
The interval evaluation of an expression has interesting properties:
- it can be implemented in such a way that the results are guaranteed with respect to round-off errors i.e. property 1 is still valid in spite of numerical errors induced by the use of floating point numbers
- if or , then no values of the unknowns in their respective ranges can cancel
- if (), then is positive (negative) for any value of the unknowns in their respective ranges
A major drawback of the interval evaluation is that may be overestimated i.e. values of such that may not exist. This overestimation occurs because in our calculation each occurrence of a variable is considered as an independent variable. Hence if a variable has multiple occurrences, then an overestimation may occur. Such phenomena can be observed in the previous example where while the real maximum of is approximately 0.9144. The value of is obtained because we are using in our calculation the formula with having the same interval value as .
Fortunately there are methods that allow one to reduce the overestimation and the overestimation amount decreases with the width of the ranges. The latter remark leads to the use of a branch-and-bound strategy in which for a given box a variable range will be bisected, thereby creating two new boxes that are stored in a list and processed later on. The algorithm is complete if all boxes in the list have been processed, or if during the process a box generates an answer to the problem at hand (e.g. if we want to prove that , then the algorithm stops as soon as for a certain box ).
- exclusion operators: these operators determine that there is no solution to the problem within a given box. An important issue here is the extensive and smart use of the monotonicity of the functions
- filters: these operators may reduce the size of the box i.e. decrease the width of the allowed ranges for the variables
- existence operators: they allow one to determine the existence of a unique solution within a given box and are usually associated with a numerical scheme that allows for the computation of this solution in a safe way
- bisection: choose one of the variable and bisect its range for creating two new boxes
- storage: store the new boxes in the list
The scope of the HEPHAISTOS project is to address all these steps in order to find the most efficient procedures. Our efforts focus on mathematical developments (adapting classical theorems to interval analysis, proving interval analysis theorems), the use of symbolic computation and formal proofs (a symbolic pre-processing allows one to automatically adapt the solver to the structure of the problem), software implementation and experimental tests (for validation purposes).
Important note: We have insisted on interval analysis because this is a major component or our robotics activity. Our theoretical work in robotics is an analysis of the robotic environment in order to exhibit proofs on the behavior of the system that may be qualitative (e.g. the proof that a cable-driven parallel robot with more than 6 non-deformable cables will have at most 6 cables under tension simultaneously) or quantitative. In the quantitative case as we are dealing with realistic and not toy examples (including our own prototypes that are developed whenever no equivalent hardware is available or to very our assumptions) we have to manage problems that are so complex that analytical solutions are probably out of reach (e.g. the direct kinematics of parallel robots) and we have to resort to algorithms and numerical analysis. We are aware of different approaches in numerical analysis (e.g. some team members were previously involved in teams devoted to computational geometry and algebraic geometry) but interval analysis provides us another approach with high flexibility, the possibility of managing non algebraic problems (e.g. the kinematics of cable-driven parallel robots with sagging cables, that involves inverse hyperbolic functions) and to address various types of issues (system solving, optimization, proof of existence ...). However whenever needed we will rely as well on continuation, algebraic geometry, geometry or learning.
HEPHAISTOS, as a follow-up of COPRIN, has a long-standing tradition of robotics studies, especially for closed-loop robots 2, especially cable-driven parallel robots. We address theoretical issues with the purpose of obtaining analytical and theoretical solutions, but in many cases only numerical solutions can be obtained due to the complexity of the problem. This approach has motivated the use of interval analysis for two reasons:
- the versatility of interval analysis allows us to address issues (e.g. singularity analysis) that cannot be tackled by any other method due to the size of the problem
- uncertainties (which are inherent to a robotic device) have to be taken into account so that the real robot is guaranteed to have the same properties as the theoretical one, even in the worst case. This is a crucial issue for many applications in robotics (e.g. medical or assistance robot)
Our field of study in robotics focuses on kinematic issues such as workspace and singularity analysis, positioning accuracy, trajectory planning, reliability, calibration, modularity management and, prominently, appropriate design, i.e. determining the dimensioning of a robot mechanical architecture that guarantees that the real robot satisfies a given set of requirements. The methods that we develop can be used for other robotic problems, see for example the management of uncertainties in aircraft design 5.
Our theoretical work must be validated through experiments that are essential for the sake of credibility. A contrario, experiments will feed theoretical work. Hence HEPHAISTOS works with partners on the development of real robots but also develops its own prototypes. In the last years we have developed a large number of prototypes and we have extended our development to devices that are not strictly robots but are part of an overall environment for assistance. We benefit here from the development of new miniature, low energy computers with an interface for analog and logical sensors such as the Arduino or the Phidgets. The web pages presents all of our prototypes and experimental work.
4 Application domains
While the methods developed in the project can be used for a very broad set of application domains (for example we have an activity in CO2 emission allowances, it is clear that the size of the project does not allow us to address all of them. Hence we have decided to focus our applicative activities on mechanism theory, where we focus on modeling, optimal design and analysis of mechanisms. Along the same line our focus is robotics and especially service robotics which includes rescue robotics, rehabilitation and assistive robots for elderly and handicapped people. Although these topics were new for us when initiating the project we have spent two years determining priorities and guidelines by conducting about 200 interviews with field experts (end-users, doctors, family and caregivers, institutes), establishing strong collaboration with them (e.g. with the CHU of Nice-Cimiez) and putting together an appropriate experimental setup for testing our solutions.
It must be reminded that we are able to manage a large variety of problems in totally different domains only because interval analysis, game theory and symbolic tools provides us the methodological tools that allow us to address completely a given problem from the formulation and analysis up to the very final step of providing numerical solutions. Hence although we mainly focus on medical and assistance robotics we address also a large number of applications: agriculture, biology, arts, system design to name a few.
5 Social and environmental responsibility
5.1 Footprint of research activities
Clearly our activities may have an impact on the environment, especially through travels. We have (an we plan to continue) reduced our travel activities while having reduced our own energy consumption at work in different manners. Still we must emphasize that all aspects of our impact have to be taken into account before coercitive measures are taken. For example when we travel to a retirement house to install assistive devices the footprint impact has to be balanced with the social impact (and this is not an easy task). Furthermore human relationships are essential for initiating new research areas and collaborations and virtual conferences are not very effective for that purpose.
5.2 Impact of research results
Our works on assistance clearly may have a social impact and we are deeply aware of our ethical responsibilities as illustrated by the activity of the team in ethical committees, our collaboration with the academic law community and our large dissemination effort toward the general audience.
Regarding environmental responsibility energy is an important topic in our research. Indeed our assistance/health monitoring devices require additional energy source and elderly people may have some difficulties to deal with battery charging. Consequently since the beginning of the project we have focused on low consumption electronic components and most our devices use mechanical energy converter to produce a large part of the energy they need. However the intended benefits of these devices on health, self-esteem and dignity (all issues that are difficult to measure/compare in pure financial terms or environmental impact) have to be taken into account. We have not had a direct activity regarding Covid as our medical research areas are not related to medical modeling or virus. Still to contribute to the collective protection effort we have installed at the end of december 2020 a home-made, low-cost but very accurate CO2 measurement station in our laboratory connect to an aeration system that is still in use after one year of full-time monitoring.
6 Highlights of the year
The pandemic has had a severe impact on our research activity especially as we have had planned since a long time numerous medical experiments for 2020 and 2021 that have all been postponed and are currently on stand-by with no visibility on a new scheduling. Such experiments are difficult to mount as several actors are involved and their preparation is time and funds consuming. The pandemics has also had an impact on our publications with many postponed conferences. Still we have been able to
- initiate new research activities (e.g. see section 8.4.1)
- develop new assistance devices (see section 8.2)
- consolidate our theoretical work on Cable-Driven Parallel Robots CDPR (see section 8.1.1) especially the management of sagging cables
- identify major issues in our activity on Human Activity Recognition (HAR) which will become a major focus for the team in the future (see section 8.2.5)
- prepare new experiments in the medical and artistic fields (e.g. with a 3-months exhibition scheduled in a museum during summer 2022)
7 New software and platforms
We present in this section major evolutions of existing platforms or brand new devices.
7.1 New software
Algorithms Library of Interval Analysis for Systems
The ALIAS library whose development started in 1998, is a collection of procedures based on interval analysis for systems solving and optimization.
ALIAS is made of two parts:
ALIAS-C++ : the C++ library (87 000 code lines) which is the core of the algorithms
ALIAS-Maple : the Maple interface for ALIAS-C++ (55 000 code lines). This interface allows one to specify a solving problem within Maple and get the results within the same Maple session. The role of this interface is not only to generate the C++ code automatically, but also to perform an analysis of the problem in order to improve the efficiency of the solver. Furthermore, a distributed implementation of the algorithms is available directly within the interface.
Jean-Pierre Merlet, Odile Pourtallier
7.2 New platforms
Participants: Lila Azzaz, Sophie Hean, Jean-Pierre Merlet, Yves Papegay, Clara Thomas.
. We have used the modularity of our MARIONET-CRANE prototype to design a new model for the art exhibition "Vitrifications". The first prototype was used for a 2 months exhibition in 2019. The second prototype was intended to be displayed during the robotics conference ICRA that should have been held in Paris and has been postponed because of the pandemic. Several new sensors were added: lidars for pose estimation and accelerometers for measuring the cable angles allowing use to design more efficient control laws. In 2021 we start preparing a new version for a second three months exhibition that is scheduled to take place at the Espace de l'Art Concret, a museum close to Sophia, during the summer of 2022. This prototype will benefit from new improvements, see section 8.1.1 but its development has been severely hindered by the pandemic. This highly modular platform is ready to be used for medical purpose, namely walking assistance or medical monitoring.
REVMED: virtual reality and rehabilitation.
INRIA and Université Côte d'Azur have agreed to fund us for developing the platform REVMED whose purpose is to introduce end-user motion and their analysis in a virtual reality environment in order to make rehabilitation exercises more attractive and more appropriate for the rehabilitation process. The main idea is to have a modular rehabilitation station allowing to manage various exercise devices with a very low set-up time (typically 10 mn), that will be actuated in order to allow ergotherapists to favor the work of various muscles groups and the difficulty of the exercise, while monitoring the rehabilitation process with various external sensors, providing an objectification of the evaluation. Version 2 has been completed in 2019 and we were planning to proceed in 2020 to the first trials. Unfortunately the Covid crisis has led to postpone these experiments. In spite of the pandemic which has created heavy problems when dealing with hardware we have been able to elaborate version 3 in 2021 with the main change that the use of kinects was rejected because of the lack of accuracy and have been substituted by 5 planar lidars at various altitudes, see section 8.2.
Suturing robot RSUR
We have also used our parallel robot "left hand" to develop an automated suturing robot, see section 8.2.
In 2021 we have re-activated a project for developing an instrumented glove which allows to evaluate the motricity of patient before, right after and 6 months after a surgical operation, see section 8.2.
Self verticalizing cane.
Fall is a major problem for elderly and may have dramatic consequences. To minimize the risk of fall it is necessary to avoid situations with imbalance. Such a situation may happen if the elderly lean to grasp his cane that has fallen down. In 2021 we have developed a proof of concept of a cane that is able to detect that is is down and uses an umbrella-like mechanism to come back to a stable vertical position, see section 8.2.
8 New results
8.1.1 Analysis of Cable-driven parallel robots
Participants: Jean-Pierre Merlet [correspondant], Yves Papegay, Romain Tissot.We have continued the analysis of suspended CDPRs for control and design purposes. This analysis is heavily dependent on the behavior of the cable. Three main models can be used: ideal (no deformation of the cable due to the tension, the cable shape is a straight line between the attachments points), elastic (cable length changes according to the tension to which it is submitted, straight line cable shape) and sagging (cable shape is not a line as the cable is submitted to its own mass). The different models leads to very different analysis with a complexity increasing from ideal to sagging. All cables exhibit sagging but the sagging effect is most often neglected, especially for small-sized CDPR. But this is an error for suspended CDPRs as the mechanical equilibrium of the system may lead to cable(s) being slack. A configuration is the set of cables that are under tension at a given pose. The problem is that for a given pose there may be different possible configurations that will exhibit different kineto-static properties. For example cable tensions will be different but more importantly the relationship between the platform velocities and the cable velocities will differ. As this relationship is used for control under the assumption that all cables are under tension a change in the velocities relationship will induce a positioning error on the platform. By definition the slackness cannot be deduced from the cable length measurements. Hence even for low mass, high stiffness cables we must take into account cable slackness whatever the CDPR size and consequently we need a realistic cable model that takes into account the elasticity and mass of the cable material.
The most used sagging model is the Irvine model 17. This is a non algebraic planar model with the upper attachment point of the cable is supposed to be grounded: it provides the coordinates of the lowest attachment point of the cable if the cable length at rest and the horizontal and vertical components of the force applied at this point are known.
where is the Young modulus of the cable material, its linear density and the area of a cross-section of the cable. This model takes into account both the elasticity and deformation of the cable due to its own mass. This model must be taken into account to solve the inverse kinematics problem (IK, i.e. find the cable lengths to reach a given platform pose) and the direct kinematics problem (DK, i.e. finding the possible platform poses being given cable lengths). With the sagging cables both problems may have several solutions. But an advantage of the Irvine model is that being given an IK solution we are able to determine the cable shape. Hence if we are able to measure the slope of the cable at a given point, then we are able to determine which cable(s) is slack and the amount of its slackness. The slope measurement may be obtained with an accelerometer, a method we are using in our latest prototype, see section 8.1.2. Hence we are mastering the real-time IK and DK (i.e. solving the current state of the robot being given their solution at the previous sampling time).
But the IK and DK are important not only for control but also for design. In the design phase we have to consider all IK/DK solutions over a given workspace for example for determining the maximal cable tensions allowing us to determine what should be the cable diameter and the winch motor torque. For that purpose it is necessary calculate all the IK/DK solutions for a large number of poses. But if we are one of the very few world teams that is able to calculate these solutions it appears that our solving methods such as interval analysis 3, 16 and continuation are computer intensive in the general case (a few hours for the IK and a few days for the DK for CDPR with 6 cables). Such computation time is prohibitive for the design phase.
This year we have experimented a new approach based on the AI Multi-Layer Perceptron (MLP). For that purpose we have considered CDPRs with 2 and 3 cables. For a planar CDPR with 3 cables the IK has 9 unknowns (the for each cable) and 9 equations (6 from the Irvine equations and 3 for the mechanical equilibrium) and the DK is of same size. We have then solved both IK and DK for a very limited set of inputs (either the pose for the IK and the cable lengths for the DK) and we have used a continuation method to obtain a large training set together with a large verification set. Then we have solved the IK/DK under the assumptions of having rigid legs and possibly having legs not playing a role (the equivalent of slack cable). This leads to a set of specific IK/DK solutions that has been used as guess for the Newton scheme using the sagging cable model. In some cases this strategy has allowed us to always determine all the solutions for the verification set but for other cases this strategy failed and hence we have focused on solving these cases with MLP. Note that Compared to the main topics that are considered in AI (text and images) we have the advantage that we can assess objectively the quality of a MLP prediction as we have exact data in the verification set.
As we are managing a very limited number of data the MLP training time is low (between 3 and 20 minutes). Consequently we have adopted a systematic approach where we have considered all MLPs with 2 to 6 layers, a number of neurons per layer in a set of 10 integers ranging from 10 to 200 and all combinations of activation functions (AF) in a set of 7 usually used AF. We have then run all these MLPs for the verification set and have performed a statistics analysis of the prediction error. The result of this analysis is that none of the MLP has produced a prediction that is close enough to the exact solution. In the design phase we may tolerate a small maximal error say 0.1% for the force not more and the best MLPs have at best a 10% rate. But it appears that in some cases using one of the MLP prediction as a guess for the Newton method leads to an exact solution. Furthermore it was noticed that for some MLP some of the unknowns were predicted with a relatively small error. Consequently we have elaborated a solving strategy that consists in establishing a set of guess for the Newton method based on the good MLP predictions and their combination and testing if any guess will make Newton convergent and therefore provide an exact solution. We have then tested this strategy on a very large verification set and it has proven to be very effective as we obtain a success rate of 100% 14, 15. Hence this strategy is promising but we are confronted to a difficult problem for generalizing it. The CDPR examples we have considered have a single solution for both the IK and DK while, in general, both IK/DK will have multiple solutions and we cannot predict what will be the maximal number of solutions when considering all poses in the workspace for the IK or an hypyer-cube in the space for the DK. This is a major problem for the MLP as the size of the output should be provided. This is the problem we are currently working on.
8.1.2 Cable-Driven Parallel Robots for large scale additive manufacturing
Participants: Jean-Pierre Merlet [correspondant], Yves Papegay [correspondant].
Easy to deploy and to reconfigure, dynamically efficient in large workspaces even with payloads, cable-driven parallel robots are very attractive for solving displacement and positioning problems in architectural building at large scale seems to be a good alternative to crane and industrial manipulators in the area of additive manufacturing. We have co-founded in 2015 the XtreeE start-up company that is currently one of the leading international actors in large-scale 3D concrete printing.
We have been contacted in 2018 by the artist Anne-Valérie Gasc that is interested in mimicking the 3D additive manufacturing process on large scale for a live art performance. She was interested in a mean for widespreading glass micro-beads on a given trajectory over a 21 9m large platform located at the contemporary art center Les Tanneries, located close to Montargis. She was especially interested in using a CDPR for that purpose because of the low visual intrusivity of the cables and its ability to move large load. After a few month of discussions we agree to recycle our old MARIONET-CRANE prototype (2009) that was at the heart of the artistic exhibition "Les Larmes du Prince - Vitrifications", that was run during July and August 2019 under the control of a local student.
After this first successful operation we were planning to exhibit this robot as a demonstration during the largest robotics conference ICRA which was planned to be held at Paris in May 2020. The exhibition place was slightly different but our robot is designed to be modular in order to adapt his performance to the available size and requested performance. We have also analyzed from the large amount of data that has been retrieved during the 2019 experiments to improve our CDPR. Our first purpose was to manage cable sagging and for that purpose we have added accelerometer at specific location close to the platform for measuring the cable slope at this point, thereby providing the amount of slackness of each cable. Then we have improved the lidar system we have used in 2019. For the 2019 exhibition these lidars were used to determine the pose of the platform. The vertical lidar give us a very accurate estimation of the platform height but the estimation in the plane was based on the detection of small pillars sparsely distributed around the robot that were often not detected because of the short range of the lidars (10m). For the 2020 we have put calibrated vertical tarpaulins on 3 sides of the place. The horizontal lidars now provide between 100 to 200 hundred points on the tarpaulin and we have implemented a robust algorithm that calculate the platform pose based on these points. This method enables to locate the moving platform with an accuracy of less than 2mm in the horizontal direction and 0.5 mm in the vertical direction at a frequency of about 5Hz, this approach being unique in the CDPR community.
Our prototype uses 4 cables to control the 3 d.o.f of the platform ad it can be proven that in general there will be at most 3 taught cables, the other one being slack. But the length of this cable should not be too far away from the the length it will have under tension as in another pose the cable will have to be taught for satisfying the mechanical equilibrium condition. As we have now a measurement of the amount of slackness we have designed a control strategy that always limits the amount of slackness of any slack cable. A side effect is that we avoid the "loop effect": when uncoiling a slack cable which has almost a zero tension it appears that the cable is not moving after the winch while it moves at the winch level. Consequently the cable makes a loop that may be caught by the winch on its outside part, thereby reverting the coiling/uncoiling action of the winch: although the control is trying to increase the cable length to reduce the height of the platform the loop effect leads to reduce the cable length, the platform going up and not down. Most of the failure of the 2019 prototype were caused by this effect.
Our prototype, figure 1, in its ICRA configuration has been tested at INRIA in a configuration that was identical to the assigned place in Paris and has shown impressive performances and was ready to be shipped to Paris. Unfortunately ICRA has been canceled because of the Covid.
This adventure, as the show, will go on as a new exhibition is scheduled for 3 months during the summer of 2022 at the Espace de l'Art Concret, a museum close to Sophia-Antipolis. The logs of the 2020 prototype tests have been analyzed and has provided us some new ideas that will be tested in 2022.
8.1.3 Pedagogical CDPR
Participants: Yves Papegay.
For pedagogical purposes Y.Papegay have designed and build during the third lockdown a simple 3-cables driven parallel robot in crane configuration based on steppers and low-cost electronic. It could serve as a basis for an upcoming project in the TerraNumerica initiative.
8.2 Medical activities
As all of our clinical trials scheduled for 2020-2021 have been either canceled or postponed because of the pandemic we have used 2021 to develop new devices that were planned for later on. Still we have had restriction for on-site access that have penalized these developments.
8.2.1 Suturing robot
Participants: Jean-Pierre Merlet.
Eric Sejor, a surgeon at Nice hospital, has contacted us about developing a robotized system for realizing sutures in an autonomous way. Suturing is a lengthy process while in many cases this is not a complex operation. Eric Sejor mentions that developing an autonomous system allowing to manage standard wounds may be extremely interesting, especially for emergency service that are under-staffed. A prototype of such system has been developed in 2020 and clinical trials were scheduled for 2021. Unfortunately the pandemic has prohibited to perform these tests.
8.2.2 Instrumented Gloves
Prior to a limb operation a surgeon proceed with an assessment of the patient motricity with an isometric test: for example for an arm operation the surgeon will put the palm of his hand on the hand of the patient and will ask him to flex its arm with the highest possible force while opposing this motion until an equilibrium position is reached, the surgeon hand being immobile. At this point the surgeon exerts with his hand the same force than the one exerted by the patient (figure 2). Based on his feeling the surgeon ranks the patient motricity with an integer indicator from 0 to 5 (0=no motricity, 5= normal motricity). This procedure is used before the operation, right after it and then 6 months later. But this procedure is not completely satisfactory: the motricity is only roughly evaluated, the medical literature shows an interoperator variability (for the same patient two surgeons provide a different ranking) and the procedure is time consuming. We were approached by the institute IULTS from CHU Nice to determine if it was possible to provide a device that will assess the motricity more objectively and speed up the process. We end up with a thin gloves on which we have 5 force sensors one one side while on the other side we have an accelerometer and a small ARM processor with integrated bluetooth communication that manages the measurements of the force sensors and of the accelerometer. Our test have shown that for each test a specific combination of the measurements of the force sensors reflects the force exerted by the patient. At the equilibrium this combination remains constant and the accelerometer shows the immobility of the surgeon's hand as well: hence we are able to automatically detect the equilibrium. A request of the doctors was that their phone must be interfaced with the system. With the support of our engineers service an application has been developed for that purpose. First the surgeon selects a series of tests that will be run in sequence. During each test the force is continuously displayed on the phone and as soon as equilibrium is reached the force measurement is stored and the system get ready for the next test. When the sequence ends a PDF file with the results is created and can be transferred to any host computer.
8.2.3 Self verticalizing cane
Mobility analysis is a major topic of the team and walking analysis is a major tool in the medical community to assess the state of health of frail people. In the past we have developed several rollators for assistance and walking analysis and a cane for walking analysis. But we are also concerned by fall which is a major issue for frail people (it is estimated that yearly in France 10 000 frail people die after a fall). For preventing fall it is necessary to avoid imbalance situations and typically for a cane that the frail people lean to retrieve its fallen down cane. We typically use a Tango cane that is able to stand vertically, instrumented with an accelerometer for walking analysis that allows also for detecting that the cane is down. It is theoretically possible to retrieve the cane by pushing with the foot on its tip but this maneuver requires a very good coordination. Our objective was to design a mechanism that will lift the cane automatically back to a stable vertical position. As a proof of concept we have used an umbrella-like mechanism with two motors which allows the cane to reach a stable vertical position in about 7 seconds (figure 3)
8.2.4 Modular rehabilitation station
Rehabilitation is also a major topic of the team. The medical community has indicated to us several major issues with commercially available rehabilitation devices: difficulty to configure the device for training a given musculo-squelettic group which impose a persistent action to the ergotherapeuthe and therefore divert him from observing the patient, the lack of synthetic indicators to assess the rehabilitation efficiency, the lack of modularity and of mobility help that complicates the ergotherapeuthe task when the patient motricity is not yet sufficient, the lack of motivation stimulus for the patient, a large setup time for installing the patient in the device and finally the cost of the devices. To address these problems we start developing a modular rehabilitation station which is first able to manage different types of training devices (illustrated with a treadmill in figure 4 but a rower or an equilibrium station are also available). The training devices of the station are actuated: for example we can change the slope and inclination of the treadmill which allow to adjust the difficulty level of the exercise but also to favor the work of specific musculo-squelettic groups. Furthermore a CDPR (see next section) is able to exert a vertical lifting force that allows to start a rehabilitation process even with a patient that is not yet able to stand up on his own. Regarding the patient motivation the patient is immersed in a virtual environment that makes the patient more comfortable but also allows for a replay of an exercise: on the figure the patient is walking in a mountainous environment whose slope is completely or partially reflected by the treadmill. Finally the patient body configuration during the exercise is measured by using several external sensors (lidars, accelerometers, ...) while, if necessary, the support force that he requires is measured by the CDPR and/or by force sensors in the handle (which allow also to estimate his stress). Using this external measurements allows for a minimal setup time and pertinent medical indicators are deduced from these measurements (for example for the treadmill we measure the number of steps, the leg velocities and poses, the patient trajectory deviation,...). Consequently the main task for the ergotherapeuthe is to adjust the difficulty level and to observe the patient during the exercise while a complete set of medical indicators will be provided at the end of the exercises.
8.2.5 Human activity recognition
Participants: Jean-Pierre Merlet, Yves Papegay, Odile Pourtallier, Eric Wajnberg.
Human activity recognition (HAR) is a major topic in the team. We are focusing on monitoring mobility and displacements (we are not interested in recognizing the individual action of our subject) using a sensor-based approach, excluding vision which is intrusive and even prohibited in some places for legal reasons. For that purpose we have developed a smart barrier combining redundant passive infrared motion detectors and infrared distance sensors. Smart barriers have been implemented in Ehpad Valrose, a new retirement house in which a specific infrastructure has been put in place to accommodate research works and in Institut Claude Pompidou, a Alzheimer day care hospital from 2019 to 2020. These two long term experiments has allowed us to determine that essential points in HAR are to determine what is possible to measure, the sensor types, how to retrieve and process sensor data, how effective are the quality of measurements on a long term basis and the level of monitoring that is acceptable for frail peoples and their helpers while providing significant and reliable data for the medical community in spite of the uncertainties both in the measurements and in the system modeling. These samples of questions will become central in our work. Unfortunately the pandemic has prohibited us to improve our smart barriers and to test a new version.
8.3 Biology activities
Participants: Eric Wajnberg, Odile Pourtallier.
8.3.1 Optimized flower visiting strategy of bees
In 2020, through an international scientific cooperation with Israeli scientists located at the University of Haifa, we developed an optimization deterministic model trying to understand what should be the optimized flower visiting strategy of bees foraging for nectar. This work was continued in 2021. Bees are actually visiting either simple or complex flowers. The reward acquired on simple flowers is easy to obtain and do not need a learning ability. On complex flowers, however, bees need to learn how to forage on them to get a reward. Once the learning period has been done, however, the reward acquired on complex flowers is higher than on simple flowers. One question is how the beehive can sustain the time needed for the bees to learn how to forage on complex flowers. Are there bees keeping on foraging on simple flowers to get back sufficient food to the hive for the entire colony, and to enable others to take the needed time to learn ? In 2021, additional, more realistic features were added to the model (e.g., the ability of the bees to return to simple flowers if this is maximizing their overall foraging success, and the fact that bees have a limited lifetime duration). Also, a manuscript has been written and was submitted to a good international journal in the field of animal ecology (Philosophical Transactions of the Royal Society B), contributing to a Special Issue under preparation on "Evolutionary Transitions in light of Socio-Technological Evolution of the Human Species". The results we obtained are providing a couple of predictions that are now being tested on real animals in the University of Haifa in Israel.
8.4 Other activities
8.4.1 Guaranteed perception from fusion of sensors
Participants: Yves Papegay.
We are interested in mixing model-driven and data-driven approach for deriving high level guaranteed perception from fusion of low-level sensor data, submitted to uncertainties/failure. Study of several practical examples led us to develop a theoretical framework, as well as formalism and methodology in a generic way.
In 2021, we are studying two practical examples, the first one consist in analysis of human activity from signals of home automation devices, the second one consist in detecting potential spoofing of GPS signal during motion of autonomous vehicles, it is a starting collaboration with Technology Innovation Institute of Abu Dhabi.
8.4.2 Localization and target following
Participants: Yves Papegay.
This year we have got a new CIFRE PhD student, Alexandre Tran, in collaboration with the QENVI company which will work on the localization of subject in an uncertain environment.
9 Bilateral contracts and grants with industry
9.1 Bilateral contracts with industry
9.1.1 Pedagogical cable-driven parallel robot
Participants: Jean-Pierre Merlet.We are in the process of signing a contract with the company GenerationRobot for developing a self-contained pedagogical kit allowing to build a two degrees-of-freedom cable-driven parallel robot. This kit will allow students from the college to university level to build the CDPR.
We have started to write a pedagogical manual describing various experiments allowing to physically illustrate scientific concepts in various fields (mathematics, computer network, mechanics, robotics, ...). This kit will tested by teachers and then commercialized by GenerationRobot. The commercialization was scheduled for the end of 2021 but was postponed because of the pandemic.
9.1.2 Symbolic tools for modeling and simulation
Participants: Yves Papegay.
This activity is the main part of a long-term ongoing collaboration with Airbus whose goal is to directly translate the conceptual work of aeronautics engineers into digital simulators to accelerate aircraft design.
An extensive modeling and simulation platform - MOSELA - has been designed which includes a dedicated modeling language for the description of aircraft dynamics models in term of formulae and algorithms, and a symbolic compiler producing as target an efficient numerical simulation code ready to be plugged into a flight simulator, as well as a formatted documentation compliant with industrial requirements of corporate memory.
Technology demonstrated by our prototype has been transferred : final version of our modeling and simulation environment has been delivered to Airbus in November 2012 and developer level know-how has been transferred in 2013 to a software company in charge of its industrialization and maintenance.
Since 2014, we are working on several enhancements and extension of functionalities, namely to enhance the performances and the numerical quality of the generated C simulation code, ease the integration of our environment into the airbus toolbox, help improving the robustness of the environment and the documentation. In 2021, like previous year and to cope with aerospace industry crisis we focused on preparing demonstrations and showcasing dedicated problems and user stories to enlarge the panel of users of our platform with the aim to continue our collaboration after the crisis. Recently, new methods have been investigated to ease the use of neural networks to provide simplified and efficient versions of models allowing performing parameters optimisation during pre-design of new airplanes.
10 Partnerships and cooperations
Participants: Jean-Pierre Merlet, Yves Papegay, Odile Pourtallier, Eric Wajnberg.
10.1 European initiatives
10.1.1 FP7 & H2020 projects
A project called ROCABLE has been funded by the COVR project that received funding from the European Union’s Horizon 2020 research and innovation program. This project was intended to deal with the safety of cable-driven parallel robot with as partners LS2N, Hephaistos, IRT Jules Verne, CETIM and Eiffage- Clemessy.
10.2 National initiatives
- the project Craft on collaborative cable-driven parallel robot has been funded by ANR. It involves LS2N (Nantes) and the Cetim. This project has started beginning of 2019.
10.3 Other collaborations
- we have a fruitful collaboration with the artist A-V Gasc with which we have organized several exhibitions with our cable-driven parallel robots. The next exhibition is scheduled for the summer of 2022 at the Espace de l'Art Concret, a museum close to Sophia Antipolis
- we have a fruitful collaboration with the artist Linda Blanchet for which we have developed a robot for the theater piece Killing robots.
- we have an extensive collaboration with the local medical community (especially CHU Nice) and many of the devices described in the section 8.2 have been developed because of this collaborations
- we have a constant informal collaboration with various academic international partners. This leads to joint publications, conference organization, joint students that changes over the years. To name a few: University of Bologna, Italy, University Innsbruck, Austria, Fraunhofer IPA, Stuttgar, Germany, Duisburg-Essen University, Germany, University of New-Brunswick, Canada, University Laval, Québec, Canada, University of Tokyo, Japan, Tianjin University,
Participants: Jean-Pierre Merlet, Yves Papegay, Odile Pourtallier, Eric Wajnberg.
11.1 Promoting scientific activities
We will not mention our review activity which is quite large but part of the job.
11.1.1 Scientific events: organisation
- J-P. Merlet is a permanent member of the International Steering Committee of the IROS conference, of the CableCon conference and chairman of the scientific Committee of the Computational Kinematics workshop. He was also an advisor for the largest robotics conference ICRA 2020, that has finally being held virtually because of the Covid,
- Y. Papegay is a permanent member of the International Steering Committee of the International Mathematica Symposium conferences series. He is a member of the OpenMath Society, building an extensible standard for representing the semantics of mathematical objects.
- E. Wajnberg is Editor-in-Chief of the journal BioControl (published by Springer) and a board member of the journals Entomologia Experimentalis et Applicata (published by Wiley), Neotropical Entomology (published by Springer), and Applied Entomology and Zoology (published by Springer)
11.1.3 Leadership within the scientific community
- J-P. Merlet is a member of the IFToMM (International Federation for the Promotion of Mechanism and Machine Science) Technical Committees on History and on Computational Kinematics. He is a member of the IFToMM Executive Council Publication Advisory Board. He is a member of the scientific committee of the CNRS GDR robotique and a senior chair of 3IA Côte d’Azur.
11.1.4 Scientific expertise
- J-P. Merlet was invited to be a member of a HCERES panel to evaluate the PPRIME Institute in Poitiers in December 2021. He is a Nominator for the Japan’s Prize.
- E. Wajnberg was invited to be a member of a HCERES panel to evaluate the UMR INRAE 1349 « Institute of Genetics, Environment and Plant Protection » (IGEPP) of Rennes (headed by MARIA MANZANARES-DAULEUX) in January 2021.
11.1.5 Research administration
- J-P. Merlet is a corresponding member of INRIA ethical committee (COERLE) and member of the Research, Ethical Committees of UCA (CERNI). He is an elected member of INRIA Scientific Committee.
- Y.Papegay is the president of Comité des Utilisateurs des Moyens Informatiques.
- O. Pourtallier is a substitute board member of SeaTech, an Engineering School of University of Toulon, a member of the NICE committee (long term invited scientists and post-doctoral student selection) and a member of the Transform committee. She was a member of the local researcher recruitment jury.
11.2 Teaching - Supervision - Juries
- J-P. Merlet has taught 9 hours on parallel robots to Master ISC (M2) at University of Toulon.
- Y. Papegay has taught 6 hours on parallel robots to Master ISC (M2) at University of Toulon 40h at University of Polynésie (L3), 39h in the master class on Artificial Intelligence.
- O. Pourtallier lectured 6 hours on game theory to Master OSE (M2), at École des Mines de Paris, Sophia Antipolis, France.
- E. Wajnberg teaches 60h at The Hebrew University of Jerusalem and University of Sao Paulo - Piracicaba, Brazil.
Additionally the team has received 5 master students.
PhD in progress
- R. Tissot, 3IA PhD student; Using AI for the development of a transfer system with health monitoring.
- M. Tomé (2019-2022). PhD student at the ESALQ, Sao Paulo University, Brazil). Modeling fall armyworm resistance in Bt-corn areas during crop and off-seasons.
- J-P. Merlet gives talks in the Alpes-Maritimes in the framework of the Science pour Tous association. He is also working with the GenerationRobot company for developing a pedagogical kit allowing students from college to university to build a 2 dof cable-driven parallel robot, while teachers will benefit from a pedagogical manual to illustrate scientific concepts.
- Y. Papegay is developing several pedagogical resources based on small robotics devices at high-school level.
- E. Wajnberg gives talks in the Alpes-Maritimes in the framework of the Science pour Tous association.
- Because of the pandemic we have been obliged to cancel the usually numerous visits we have of our robotics hall.
- J-P. Merlet is involved in the robotics group of the Terra Numerica UCA/Inria initiative.
- Y.Papegay is actively participating to the Math.en.Jeans initiative for Mathematics teaching for undergraduate students. He is a member of the advisory board of this non-profit organisation concerning more than 5000 high school students. He is developing several pedagogical resources based on small robotics devices at high-school level. He is namely involved in the robotics group of the Terra Numerica UCA/Inria initiative.
- O. Pourtallier is corresponding researcher for two Math.en.Jeans workshops, an initiative for Mathematics teaching for undergraduate students.
12 Scientific production
12.1 Major publications
- 1 articleInterval Analysis and Reliability in Robotics.International Journal of Reliability and Safety32009, 104-130URL: http://hal.archives-ouvertes.fr/inria-00001152/en/
- 2 bookParallel robots, 2nd Edition.Springer2005
- 3 inproceedingsUsing interval analysis in robotics problems.SCANTokyo, JapanSeptember 2018
- 4 articleLes avancées en robotique d'assistance à la personne sous le prisme du droit et de l'éthique.Revue générale de droit médicaleDecember 2017
- 5 phdthesisDe la modélisation littérale à la simulation certifiée.Université de Nice Sophia-AntipolisNice, FranceJune 2012, URL: http://tel.archives-ouvertes.fr/tel-00787230
- 6 inproceedingsFrom Modeling to Simulation with Symbolic Computation: An Application to Design and Performance Analysis of Complex Optical Devices.Proceedings of the Second Workshop on Computer Algebra in Scientific ComputingMunichSpringer VerlagJune 1999
- 7 inproceedingsA Polynomial Time Local Propagation Algorithm for General Dataflow Constraint Problems.Proc. Constraint Programming CP'98, LNCS 1520 (Springer Verlag)1998, 432--446
12.2 Publications of the year
International peer-reviewed conferences
Conferences without proceedings
12.3 Cited publications
- 16 articleSolving the direct geometrico-static problem of underconstrained cable-driven parallel robots by interval analysis.Int. J. of Robotics Research2015
- 17 bookCable Structures.MIT Press1981