Keywords
Computer Science and Digital Science
- A2.3.3. Real-time systems
- A3.1.1. Modeling, representation
- A5.5. Computer graphics
- A5.6. Virtual reality, augmented reality
- A5.10. Robotics
- A6.2.1. Numerical analysis of PDE and ODE
- A6.2.6. Optimization
- A6.4.3. Observability and Controlability
- A6.4.4. Stability and Stabilization
- A9.2. Machine learning
- A9.5. Robotics
Other Research Topics and Application Domains
- B2.5.1. Sensorimotor disabilities
- B2.5.3. Assistance for elderly
- B2.7. Medical devices
- B5.1. Factory of the future
- B5.5. Materials
- B5.6. Robotic systems
- B5.7. 3D printing
- B9.2. Art
1 Team members, visitors, external collaborators
Research Scientists
- Christian Duriez [Team leader, Inria, Senior Researcher, HDR]
- Olivier Goury [Inria, Researcher]
- Gang Zheng [Inria, Researcher, HDR]
Faculty Members
- Jeremie Dequidt [Université de Lille, Associate Professor]
- Alexandre Kruszewski [École centrale de Lille, Professor, HDR]
Post-Doctoral Fellow
- Yinoussa Adagolodjo [Inria, from Jul 2020]
PhD Students
- Walid Amehri [Inria]
- Haihong Li [École centrale de Lille, from Oct 2020]
- Shijie Li [Université de Valenciennes et du Hainaut Cambrésis]
- Etienne Menager [École normale supérieure de Rennes, from Sep 2020]
- Pierre Schegg [Robocath, CIFRE]
- Felix Vanneste [Inria]
- Ke Wu [Inria]
- Lingxiao Xun [Inria, from Oct 2020]
Technical Staff
- Yinoussa Adagolodjo [Inria, Engineer, until Jun 2020]
- Stefan Escaida Navarro [Inria, Engineer]
- Meichun Lin [Université de Lille, Engineer, until Aug 2020]
- Damien Marchal [CNRS, Engineer]
- Bruno Marques [Inria, Engineer, until Oct 2020]
- Thor Morales Bieze [Inria, Engineer]
Interns and Apprentices
- Paul Chaillou [Inria, from Sep 2020]
- Arjeta Heta [Inria, from May 2020 until Sep 2020]
- Elie Khairallah [Université de Lille, from May 2020 until Sep 2020]
- Yueying Liang [Inria, from Oct 2020]
- Etienne Menager [École normale supérieure de Rennes, from Apr 2020 until Aug 2020]
- Fatimetou Mohamed El Hacen [Inria, from Apr 2020 until Sep 2020]
- Valentin Pouilly-Cathelain [Inria, from Apr 2020 until Sep 2020]
Administrative Assistant
- Anne Rejl [Inria]
2 Overall objectives
The team DEFROST aims to address the open problem of control and modelling methods for deformable robots by answering the following challenges:
- Providing numerical methods and software support to reach the real-time constraint needed by robotic systems: the numerical solutions for the differential equations governing the deformation generate tens of thousands degrees of freedom, which is three orders of magnitude of what is frequently considered in classical methods of robotic modelling and control.
- Integrating deformation models in the control methods of soft robot: In soft-robotics, sensing, actuation and motion are coupled by the deformations. Deformable models must be placed at the heart of the control algorithm design.
- Investigating predictable interaction models with soft-tissues and parameter estimation by visual feedback from medical imaging: on the contrary too many cases in surgical robotics, the contact of the soft robot with the anatomy is permitted and it creates additional deformations on the robot.
3 Research program
3.1 Introduction
Our research crosses different disciplines: numerical mechanics, control design, robotics, optimisation methods and clinical applications. Our organisation aims at facilitating the team work and cross-fertilisation of research results in the group. We have three objectives (1, 2 and 3) that correspond to the main scientific challenges. In addition, we have two transverse objectives that are also highly challenging: the development of a high performance software support for the project (Objective 4) and the validation tools and protocols for the models and methods (Objective 5).
3.2 Objective 1: Accurate model of soft robot deformation computed in finite time
The objective is to find concrete numerical solutions to the challenge of modelling soft robots with strong real-time constraints. To solve continuum mechanics equations, we will start our research with real-time Finite Element Method (FEM) or equivalent methods that were developed for soft-tissue simulation. We will extend the functionalities to account for the needs of a soft-robotic system:
- Coupling with other physical phenomenons that govern the activity of sensors and actuators (hydraulic, pneumatic, electro-active polymers, shape-memory alloys etc.).
- Fulfilling the new computational time constraints (harder than surgical simulation for training) and find better tradeoff between cost and precision of numerical solvers using reduced-order modelling techniques with error control.
- Exploring interactive and semi-automatic optimisation methods for design based on obtained solution for fast computation on soft robot models.
3.3 Objective 2: Model based control of soft robot behavior
The focus of this objective is on obtaining a generic methodology for soft robot feedback control. Several steps are needed to design a model based control from FEM approach:
- The fundamental question of the kinematic link between actuators, sensors, effectors and contacts using the most reduced mathematical space must be carefully addressed. We need to find efficient algorithms for real-time projection of non-linear FEM models in order to pose the control problem using the only relevant parameters of the motion control.
- Intuitive remote control is obtained when the user directly controls the effector motion. To add this functionality, we need to obtain real-time inverse models of the soft robots by optimisation. Several criteria will be combined in this optimisation: effector motion control, structural stiffness of the robot, reduce intensity of the contact with the environment etc.
- Investigating closed-loop approaches using sensor feedback: as sensors cannot monitor all points of the deformable structure, the information provided will only be partial. We will need additional algorithms based on the FEM model to obtain the best possible treatment of the information. The final objective of these models and algorithms is to have robust and efficient feedback control strategies for soft robots. One of the main challenge here is to ensure / prove stability in closed-loop.
3.4 Objective 3: Modeling the interaction with a complex environment
Even if the inherent mechanical compliance of soft robots makes them safer, more robust and particularly adapted to interaction with fragile environments, the contact forces need to be controlled by:
- Setting up real-time modelling and the control methods needed to pilot the forces that the robot imposes on its environment and to control the robot deformations imposed by its environment. Note that if an operative task requires to apply forces on the surrounding structures, the robot must be anchored to other structures or structurally rigidified.
- Providing mechanics models of the environment that include the uncertainties on the geometry and on the mechanical properties, and are capable of being readjusted in real-time.
- Using the visual feedback of the robot behavior to adapt dynamically the models. The observation provided in the image coupled with an inverse accurate model of the robot could transform the soft robot into sensor: as the robot deforms with the contact of the surroundings, we could retrieve some missing parameters of the environment by a smart monitoring of the robot deformations.
3.5 Objective 4: Soft Robotics Software
Expected research results of this project are numerical methods and algorithms that require high-performance computing and suitability with robotic applications. There is no existing software support for such development. We propose to develop our own software, in a suite split into three applications:
- The first one will facilitate the design of deformable robots by an easy passage from Computer-Aided Design (CAD) software (for the design of the robot) to the FEM based simulation.
- The second one is an anticipative clinical simulator. The aim is to co-design the robotic assistance with the physicians, thanks to a realistic simulation of the procedure or the robotic assistance. This will facilitate the work of reflection on new clinical approaches prior any manufacturing.
- The third one is the control design software. It will provide the real-time solutions for soft robot control developed in the project.
3.6 Objective 5: Validation and application demonstrations
The implementation of experimental validation is a key challenge for the project. On one side, we need to validate the model and control algorithms using concrete test case example in order to improve the modelling and to demonstrate the concrete feasibility of our methods. On the other side, concrete applications will also feed the reflexions on the objectives of the scientific program.
We will build our own experimental soft robots for the validation of Objectives 2 and 3 when there is no existing “turn-key” solution. Designing and making our own soft robots, even if only for validation, will help the setting-up of adequate models.
For the validation of Objective 4, we will develop “anatomical soft robot”: soft robot with the shape of organs, equipped with sensors (to measure the contact forces) and actuators (to be able to stiffen the walls and recreate natural motion of soft-tissues). We will progressively increase the level of realism of this novel validation set-up to come closer to the anatomical properties.
4 Application domains
4.1 Industry
Robotics in the manufacturing industry is already widespread and is one of the strategies put in place to maintain the level of competitiveness of companies based in France and to avoid relocation to cheap labor countries. Yet, in France, it is considered that the level of robotization is insufficient, compared to Germany for instance. One of the challenges is the high investment cost for the acquisition of robotic arms. In recent years, this challenge has led to the development of “generic” and “flexible” (but rigid) robotic solutions that can be mass produced. But their applicability to specific tasks is still challenging or too costly. With the development of 3D
printing, we can imagine the development of a complete opposite strategy: a “task-specific” design of robots. Given a task that needs to be performed by a deformable robot, we could optimize its shape and its structure to create the set of desired motions. A second important aspect is the reduction of the manufacturing cost: it is often predicted that the cost of deformable robots will be low compared to classical rigid robots. The robot could be built on one piece using rapid prototyping or 3D printers and be more adapted for collaborative work with operators. In this area, using soft materials is particularly convenient as they provide a mass/carried load ratio several orders of magnitude higher than traditional robots, highly decreasing the kinetic energy thus increasing the motion speed allowed in presence of humans. Moreover, the technology allows more efficient and ergonomic wearable robotic devices, opening the option for exo-skeletons to be used by human operators inside the factories and distribution centers. This remains to be put in place, but it can open new perspectives in robotic applications. A last remarkable property of soft robots is their adaptability to fragile or tortuous environments. For some
particular industry fields (chemistry, food industry etc.) this could also be an advantage compared to existing rigid solutions. For instance, the German company http://
4.2 Personal and service robotics
Personal and service robotics are considered an important source of economic expansion in the coming years. The potential applications are numerous and in particular include the challenge of finding robotic solutions for active and healthy aging at home. We plan to develop functional orthosis for which it is better not to have a rigid exoskeleton that is particularly uncomfortable. These orthosis will be ideally personalized for each patient and built using rapid prototyping. On this topic, the place of our team will be to provide algorithms for controlling the robots. We will find some partners to build these robots that would fall in the category of “wearable robots”. With this thematic we also connect with a strong pole of excellence of the region on intelligent textiles (see Up-Tex) and with the strategic plan of INRIA (Improving Rehabilitation and Autonomy).
4.3 Entertainment industry and arts
Robots have a long history with entertainment and arts where animatronics have been used for decades for cinematographic shootings, theater, amusement parks (Disney's audio-animatronic) and performing arts. This year, we obtained an award for an Art Installation at the Exhibit Panorama 22. The installation “L'Érosarbénus”, which was produced at Le Fresnoy for the exhibition Panorama 22, is a collaboration between visual artist Yosra Mojtahedi and DEFROST. This installation, whose aesthetics are inspired by rocky, plant and human forms, is brought to life with the help of Soft Robotics devices. It was awarded the prize “Révélation Art Numérique — Art Vidéo 2020” by the ADAGP. See the ARTE video.
4.4 Medical Applications
Soft robots have many medical applications as their natural compliance makes them safer than traditional robots when interacting with humans. Such robots can be used for minimally invasive surgery, to access and act on remote parts of the body through minimal incisions in the patient. Applications include laparascopic and brain surgery, treatment of several cancers including prostate cancer, and cardiology, for example percutaneous coronary interventions. As an example, we received an industry grant (CIFRE) with Robocath to work on autonomous catheter navigation. See Section 8.2.
Another application is cochlear implant surgery in the project ANR ROBOCOP.
5 Social and environmental responsibility
5.1 Footprint of research activities
This year, with the health situation, we have cancelled completely our trips. Even if this situation is temporary, it has enabled us to totally reduce the carbon footprint of our activities. In the future, even if we resume our activities, we will ensure that this carbon footprint is kept low.
6 Highlights of the year
6.1 Ernest Déchelle award
Christian Duriez is laureate of the Ernest Déchelle prize awarded by the French Academy of Sciences.
6.2 PhD thesis award
Eulalie Coevoet is laureate of the prize for the best PhD thesis in Robotics in 2019 awarded by the GDR Robotique.
6.3 Equipex+
The Tirrex project (Equipex+) has been selected by the ANR. The team will have an important role in this project, particularly in the digital infrastructure axis.
6.4 Révélation Arts Numériques — Art Vidéo award
Yosra Mojtahedi from Le Fresnoy collaborated with the team, especially Stefan Escaida Navarro, to create a soft robotic masterpiece called erosarbénus and was awarded the prize of Révélation Art Numérique — Art Vidéo by the ADAGP.
7 New software and platforms
7.1 New software
7.1.1 SOFA
- Name: Simulation Open Framework Architecture
- Keywords: Real time, Multi-physics simulation, Medical applications
- Functional Description: SOFA is an Open Source framework primarily targeted at real-time simulation, with an emphasis on medical simulation. It is mostly intended for the research community to help develop new algorithms, but can also be used as an efficient prototyping tool. Based on an advanced software architecture, it allows the creation of complex and evolving simulations by combining new algorithms with algorithms already included in SOFA, the modification of most parameters of the simulation (deformable behavior, surface representation, solver, constraints, collision algorithm etc.) by simply editing an XML file, the building of complex models from simpler ones using a scene-graph description, the efficient simulation of the dynamics of interacting objects using abstract equation solvers, the reuse and easy comparison of a variety of available methods.
- News of the Year: The new version v20.06 has been released including new elements on SoftRobots + ModelOrderReduction integration, in addition to an improved architecture and lots of cleans and bugfixes.
-
URL:
http://
www. sofa-framework. org - Contact: Hugo Talbot
- Participants: Christian Duriez, François Faure, Hervé Delingette, Stéphane Cotin, Hugo Talbot, Maud Marchal
- Partner: IGG
7.1.2 SoftRobots
- Name: SoftRobots plugin for Sofa
- Keywords: Numerical simulations, Problem inverse, Soft robotics
- Functional Description: This plugin allows the modeling of deformable robots in the Sofa platform. It allows the modeling of different actuators, such as cable, pneumatic pressure, hydraulics and other simpler types of actuation. It also contains useful tools for animation design or communication with the robot. Coupled with the SoftRobots.Inverse plugin, it also allows the control of these robots. More information can be found on the dedicated website.
-
URL:
https://
project. inria. fr/ softrobot/ - Contact: Christian Duriez
- Participants: Christian Duriez, Olivier Goury, Jérémie Dequidt, Damien Marchal, Eulalie Coevoet, Félix Vanneste
7.1.3 Model Order Reduction Plugin for SOFA
- Name: Model Order Reduction Plugin for SOFA
- Keywords: Model Order Reduction, Sofa, Finite element modelling
- Scientific Description: This plugin allows speed-up of SOFA simulations by providing tools to create a reduced version of the SOFA simulation that runs at much higher rates but remains accurate. Starting with a snapshot of the object deformations on a high-dimensional Finite Element mesh, Proper Orthogonal Decomposition (POD) is used to compute a reduced basis of small dimension representing correctly all the possible deformations of the object. The original system describing the object motion is then greatly reduced. To keep numerical efficiency, a hyper-reduction method is used to speed-up the construction of the reduced system.
- Functional Description: This plugin allows to dramatically reduce computational time in mechanical simulation in the SOFA framework. A reduced simulation, of much smaller dimension but still accurate is created in an automatic way by the plugin. Building the reduced model may take time, but this operation is made once only. The user can then benefit from a reduced and interactive version of his/her simulation without significant loss of accuracy.
- Release Contributions: This is the first version of the plugin.
- News of the Year: Publication using this plugin accepted in IEEE Robotics and Automation Letters (RA-L)
-
URL:
https://
project. inria. fr/ modelorderreduction/ - Publication: hal-01834483
- Contact: Olivier Goury
- Participants: Olivier Goury, Félix Vanneste, Christian Duriez, Eulalie Coevoet
7.1.4 SoftRobots.Inverse
- Name: SoftRobots.Inverse
- Keywords: Sofa, SoftRobots
- Scientific Description: This plugin implements a method to compute the inverse model of a robot in its environment. The input of the method is the desired position of the effector. The output is the force or the motion that needs to be applied to the actuators in order to minimize the distance with the effector position. This is found by minimizing the constraint equation using Quadratic Programming (QP), i.e. minimizing the violation of the defined constraints.
- Functional Description: This plugin builds on the plugin SoftRobots. Inside the plugin, there are some constraint components that are used to describe the robot (effectors, actuators, sensors). An optimisation algorithm is provided to find the efforts to put on actuators in order to place the robot in the closest possible configuration to the one described by “effectors”, or to a state described by “sensors”. This method used to control the soft robots in the task space is patented.
-
URL:
https://
project. inria. fr/ softrobot - Publication: hal-01649355
- Contact: Christian Duriez
- Participants: Christian Duriez, Eulalie Coevoet
- Partners: CNRS, Université de Lille, Ecole Centrale de Lille
7.1.5 SofaPython3
- Name: SofaPython3
- Keywords: Python, Numerical simulations, Sofa
- Functional Description: This plugin allows to use Sofa as a library from any python3 program. It also allows to write new mechanical component for a Sofa simulation in python3.
- Release Contributions: Python3 and SOFA are meeting in one plugin: SofaPython3. In 2020, the integration of this plugin within SOFA opens up new perspectives. Not only can a SOFA simulation be described using a Python script with the embedded Python interpreter, but one or several SOFA instances can now created from a Python environment. Even more powerful, you can create new components (e.g. forcefields) in Python using the bindings provided in the plugin. Python-scripted simulations have no limit anymore!
- News of the Year: Python3 and SOFA are meeting in one plugin: SofaPython3. In 2020, the integration of this plugin within SOFA opens up new perspectives. Not only can a SOFA simulation be described using a Python script with the embedded Python interpreter, but one or several SOFA instances can now created from a Python environment. Even more powerful, you can create new components (e.g. forcefields) in Python using the bindings provided in the plugin. Python-scripted simulations have no limit anymore!
-
URL:
https://
github. com/ sofa-framework/ SofaPython3/ - Contact: Hugo Talbot
- Participants: Jean-Nicolas Brunet, Damien Marchal, Bruno Marques, Thierry Gaugry, Frédérick Roy, Guillaume Paran, Hugo Talbot
7.1.6 SofaQtQuick
- Name: runSofa2
- Keywords: Sofa, GUI (Graphical User Interface), Modeling, Physical simulation
-
Functional Description:
Smooth the user experience with Sofa By integrating authoring features into runSofa so we can design simulation in an integrated environment. We should be able to model scenes, simulate & debug them.
This tool replaces the old “runSofa” interface, today deprecated but still in use by most SOFA users.
SofaQtQuick provides a fluid and dynamic user experience for SOFA, thanks to the integration of authoring tools to design complex simulations directly in the 3D environment, rather that scripting them as it is done today.
FEATURES: Scene graph editing Interactive modeling Project oriented approach Prefab as reusable and parametric object 2D Canvas Custom widgets per component Live coding Node base interface for data link debugging Everything with a non-linear workflow
Based on a code gift from Anatoscope, stringly inspired by Blender & Unity's workflow.
- Release Contributions: 1st Beta version, unstable, but testable
-
URL:
http://
github. com/ SofaDefrost/ SofaQtQuick - Contact: Christian Duriez
7.1.7 Cosserat plugin
- Keywords: Physical simulation, Finite element modelling, Soft robotics, Needle insertion, Frictional contact
- Functional Description: The cosserat model has been introduced in continuum robotics to simulate the deformation the robot body whose geometry and mechanical characteristics are similar to a rod. By extension, this model can be used to simulate needles and sutures or wires. The specificity of Cosserat's theory from the point of view of the mechanics of continuous media is to consider that the set of material points consists of rigid solids (3 translations, 3 rotations), where most other models of continuum media mechanics consider the material points as particles (3 translations). For the modeling of linear structures, it is therefore possible to find a framework very close to the articulated solids with a series of rigid solids whose relative position is defined by a strain state. This model can be used to model and control concentric tube robots, continuum robots actuated with cables or pneumatic soft robots with a constant cross-section. There are several formalism to model Cosserat rods. In this plugin, we have developed the Piecewise Constant Strain (PCS) presented by Federico Renda in the SOFA framework.
-
URL:
https://
github. com/ SofaDefrost/ plugin. Cosserat/ tree/ master/ src/ constraint - Authors: Yinoussa Adagolodjo, Christian Duriez, Federico Renda
- Contact: Christian Duriez
8 New results
8.1 Design, implementation and control of a deformable manipulator robot based on a compliant spine
This work, published in 2 includes the conception, the numerical modeling and the control of a dexterous, deformable manipulator bio-inspired by the skeletal spine found in vertebrate animals. Through the implementation of this new manipulator, we show a methodology based on numerical models and simulations, that goes from design to control of continuum and soft robots. The manipulator is modeled using Finite Element Method (FEM), using a set of beam elements that reproduce the lattice structure of the robot. The model is computed and inverted in real-time using optimisation methods. A closed-loop control strategy is implemented to account for the disparities between the model and the robot. This control strategy allows for accurate positioning, not only of the tip of the manipulator, but also the positioning of selected middle points along its backbone. In a scenario where the robot is piloted by a human operator, the command of the robot is enhanced by a haptic loop that renders the boundaries of its task space as well as the contact with its environment. The experimental validation of the model and control strategies is also presented in the form of an inspection task use case. (See the illustration in Figure 1.)
8.2 Modeling Novel Soft Mechanosensors based on Air-Flow Measurements
This work has been done in collaboration with Mimesis team. It has published in ICRA 2020 22. It introduces a method for robotic steering of a flexible needle inside moving and deformable tissues. The method relies on a set of objective functions allowing to automatically steer the needle along a predefined path. In order to follow the desired trajectory, an inverse problem linking the motion of the robot end effector with the objective functions is solved using a Finite Element simulation. The main contribution of the article is the new constraint-based formulation of the objective functions allowing to: 1) significantly reduce the computation time; 2) increase the accuracy and stability of the simulation-guided needle insertion. The method is illustrated, and its performances are characterized in a realistic framework, using a direct simulation of the respiratory motion generated from in vivo data of a pig. Despite the highly non-linear behavior of the numerical simulation and the significant deformations occurring during the insertion, the obtained performances enable the possibility to follow the trajectory with the desired accuracy for medical purpose. (See the illustration in Figure 2.)
8.3 Anisotropic soft robots based on 3D printed meso-structured materials: design, modeling by homogenization and simulation
In this work, made in the context of the PhD thesis of Félix Vanneste and published in Robotics and Automation Letters (RAL) 3, we propose to use new 3D-printed meso-structured materials to build soft robots and we present a modeling pipeline for design assistance and control. These meta-materials can be programmed before printing to target specific mechanical properties, in particular heterogeneous stiffness and anisotropic behaviour. Without changing the external shape, we show that using such meta-material can lead to a dramatic change in the kinematics of the robot. This highlights the importance of modeling. Therefore, to help the design and to control soft robots made of these meso-structured materials, we present a modeling method based on numerical homogenization and Finite Element Method (FEM) that captures the anisotropic deformations. The method is tested on a 3 axis parallel soft robot initially made of silicone. We demonstrate the change in kinematics when the robot is built with meso-structured materials and compare its behavior with modeling results. (See the illustration in Figure 3.)
8.4 A Model-based Sensor Fusion Approach for Force and Shape Estimation in Soft Robotics
This work was carried out in collaboration with several laboratories in Europe: Karlsruhe Institue of Technology (Germany), Hasselt University (Belgium), Carinthia University (Austria), University of Klagenfurt (Austria). It was published in 1. We address the challenge of sensor fusion in Soft Robotics for estimating forces and deformations. In the context of intrinsic sensing, we propose the use of a soft capacitive sensor to find a contact's location, and the use of pneumatic sensing to estimate the force intensity and the deformation. Using a FEM-based numerical approach, we integrate both sensing streams and model two Soft Robotics devices we have conceived: a Soft Pad and a Soft Finger. These devices include capacitive and pneumatic sensing. Capacitive sensing allows us to find contact locations and pneumatic sensing to measure the amount of deformation. Sensor fusion is mediated by a numerical model (FEM), which allows us to estimate the forces being applied and the deformation of the devices. We show in an evaluation that external forces on the Soft Pad can be estimated and that the shape of the Soft Finger can be reconstructed. (See the illustration in Figure 4.)
8.5 LPV framework for Non-Linear Dynamic Control of Soft Robots using Finite Element Model
This work published at IFAC 2020 29, presents a methodology to control soft robots using a reduced order nonlinear finite element model. The Linear Parameter-Varying (LPV) framework is used both to model the robot along a prescribed trajectory and to design its control law. Model reduction algorithms along with radial basis functions network are used to identify the nonlinear behavior of the robot. Finally, the method is validated through simulation experiments. (See the illustration in Figure 5.)
8.6 FEM Based Workspace Estimation for Soft Robots: a Forward-Backward Interval Analysis Approach
Given a configuration of a silicone soft robot, with the bounded installed actuators, this work, published in IEEE RoboSoft 2020 21, investigates the workspace estimation for a certain chosen points of interest for such a soft robot. For this, the Finite Element method is adopted to deduce the mathematical model of soft robots, based on which a forward-backward interval analysis approach is performed to estimate the workspace. Numerical simulations are provided to highlight the feasibility of the proposed methodology. (See the illustration in Figure 6.)
8.7 Robust control of a silicone soft robot using neural networks
This work, published in ISA Transactions 19, deals with the robust controller design problem to regulate the position of a soft robot with elastic behavior, driven by 4 cable actuators. In this work, we first used an artificial neural network to approximate the relation between these actuators and the controlled position of the soft robot, based on which two types of robust controllers (type of integral and sliding mode) are proposed. The effectiveness and the robustness of the proposed controllers have been analyzed both for the constant and the time-varying disturbances. The performances (precision, convergence speed and robustness) of the proposed method have been validated via different experimental tests. (See the illustration in Figure 7.)
8.8 Control of a Silicone soft tripod robot via uncertainty compensation
Unlike using different techniques (such as Beam theory, Cosserat theory or high dimensional finite element method) to model the dynamics of soft robots, this work, published in IEEE Robotics and Automation Letters 18, introduces a simplified nominal model with uncertainty to describe its dynamic behavior. The link between this simple model and the finite-element method has been established, and a robust controller is proposed, by compensating the uncertainty which is estimated in a finite time by applying different types of estimators. The experiments have been made for different scenarios, and the corresponding results show the efficiency of the proposed method. (See the illustration in Figure 8.)
8.9 FEM-based Gain-Scheduling Control of a Soft Trunk Robot
This work, published in IEEE Robotics and Automation Letters 4, demonstrates a gain-scheduling closed-loop method to control a soft trunk robot operating within its workspace by using Finite Element Method (FEM). The main idea of this method is to divide the workspace into several sub-workspaces where the most suitable gains are applied correspondingly in each sub-workspace. As a result, it becomes feasible to control the trunk by gain scheduling when crossing from one sub-workspace to another as well as considering its dynamic characteristics. The derivation of the method is presented accordingly. In the end, the proposed method is validated by experimental testing with convincing results provided afterwards. (See the illustration in Figure 9.)
9 Bilateral contracts and grants with industry
9.1 Bilateral grants with industry
We received an industry grant (CIFRE) with Robocath to work on autonomous catheter navigation. This grant funds the on-going PhD project carried out by Pierre Schegg for 3 years since February 2019.
We have an ongoing bilateral project with the company InSimo on the simulation of suture.
10 Partnerships and cooperations
10.1 International initiatives
10.1.1 Inria International Labs
AC/DC: A Charm lab/DEFROST Collaboration
Participants: Christian Duriez.
- Duration: since 2019
- Partners: Department of Mechanical Engineering, Stanford (United States)
- Summary: This project aims to strengthen the collaboration between the Defrost team and the Charm Lab (Collaborative HAptics and Robotics in Medicine Lab, Stanford University, USA) on the topic of soft robots. On this topic, these two entities are very complementary because the Charm Lab is interested in the new design, the realization, the planning and the experimentation and the Defrost team is more centered on mechanical modeling, simulation and the algorithms of control. In 2020, the collaboration has been focused on the control of new haptic rendering based on a parallel continuum robot, in particular with the visit of Margaret Koehler (PhD student at Stanford University) in January 2020.
10.1.2 Inria international partners
Informal international partners
We have a collaboration with the University of Naples Federico II (Italy), and, in particular, the ICAROS center. This project aims to reproduce in simulation the biopsy of the prostate; a probe and a needle are inserted in the rectum to come close to the prostate, once the needle is aligned with a target region in the prostate it is inserted (gunshot) inside the organ. With a 3D reconstruction of a patient's prostate and surrounding tissues (from MRI and US images), and the simulation framework SOFA, we reproduce the mechanical behaviour of the structures in simulation. Because there is a benefit to robotized the procedure, an articulated robot is also modelled and simulated to control the probe and needle insertion. The challenge is to solve the inverse kinematics of the robot to insert the probe in the rectum, and align the needle with a target. This is challenging because; (1) a soft part is placed between the robot and the probe, and (2) the probe may deform the prostate and thus move the target while trying to align the needle with it.
For the work of the paper entitled “A Model-based Sensor Fusion Approach for Force and Shape Estimation in Soft Robotics” published this year 1, the team relied on the collaboration with several partners. A stretchable touch-pad was provided by Hasselt University (Belgium), the electronics to perform capacitive measurements was provided by Karlsruhe Institute of Technology (Germany) and Carinthia University of Applied Sciences as well as University of Klagenfurt (both in Austria) were involved in the evaluation of the capacitive sensors.
10.2 International research visitors
10.2.1 Visits of international scientists
At the end of 2019, Yinoussa Adagolodjo from our team went to the university of Abu Dhabi and we were supposed to invite Assistant Professor Federico Renda for a second stay in our team on June 2020. Unfortunately, it has not been possible but we have worked remotely on a paper (conditionally accepted) on Cosserat Rod models. In particular, we have shown being able to couple Cosserat Rod and FEM. At that are particularly helpful for modeling continuum robots with a reduced number of degrees of freedom.
10.3 European initiatives
10.3.1 FP7 & H2020 Projects
We are participating in the H2020 project SimCardioTest (coordinated by Maxime Sermesan at Inria Sophia) that has been accepted in 2020. The project has started in January 2021.
10.3.2 Collaborations in European programs, except FP7 and H2020
COMOROS: Control of deformable robots for surgery
Participants: Christian Duriez.
- Duration: April 2017 to March 2020
- Program: FEDER
-
Summary: Surgical procedures are often carried out using instruments made of stiff materials that interact with delicate biological tissues such as internal organs, blood vessel walls and small cavities. This incompatibility of stiffness is one of the sources of danger in many surgical procedures. The use of robots made of soft materials, also called soft robots, would limit such risks by reducing contact pressures and stress concentrations. Their intrinsic deformability would also increase the ability to manoeuvre in confined spaces. However, the promising concept of using soft robots for surgical procedures cannot be practically implemented, due to the lack of precise modelling and control methods for soft robots. This scientific obstacle, identified as a pending issue by major surveys in this
field, becomes particularly challenging when interacting with an environment as complex as the human anatomy. Drawing on our background in soft tissue simulation, contact models, surgical applications and soft robotics, our ambition in this project is to:
- Develop accurate and generic numerical methods for continuum mechanics, adapted to strong real time constraints in order to demonstrate the ability to model soft mechatronics systems.
- Reconsider parametrization methodologies of digital models of the patient anatomy through the observation of mechanical interactions with soft robots via embedded sensors and medical imaging
- Rethink motion generation and teleoperation control with force feedback so as to be compatible with the large number of degrees of freedom of soft robots and be based on accurate, rapidly computed deformable models and interaction models.
CoBra: Cooperate Brachytherapy
Participants: Meichun Lin, Stefan Escaida Navarro.
- Duration: January 2018 to September 2022
- Program: Interreg 2 Seas Mers Zeeën
-
URL:
https://
cobra-2seas. eu/ - Summary: This 4 years project gathers the experts from the countries between English Channel and southern North Sea aiming to find an advanced method for treating prostate cancer. The project is divided by several fields: MR compatible robot design, radiation dose measurement, steerable needle design, mimic soft-tissue (phantom) design and virtual reality real-time training tool development, etc. Meichun Lin worked on developing virtual reality real-time training tool with the Defrost team. The aim is to have a interactive platform for human and the robot. By using SOFA framework to simulate the soft tissue's deformation and the interaction with needle insertion under the real-time, also with the Image modelling of MRI and soft-tissue modelling. In the last months of 2020, Stefan Escaida Navarro worked on the creation of a prostate phantom equipped with sensor that allow detecting needle interactions with it. This phantom can serve multiple purposes in the project, such as the generation of MRI-images, as a test-bed for robot-based needle insertion, etc.
10.4 National initiatives
- ROBOCOP: Robotization of Cochlear implant. This is a 4-year project, supported by the ANR (French National Agency for Research) in the framework of PRCE, starting from 1 October 2019 until 30 September 2023. ROBOCOP aims at creating a new prototype of cochlear implant, and robotize (i.e. actuate and control) its insertion process to facilitate the work of surgeon, to increase the success ratio, and to decrease the probability of trauma.
- Cosseroot: Cosserat Rod Theory for Slender Robots. This is a 4 year project, supported by the ANR (French National Agency for Research) in the framework of PRC, starting from 1 Novembre 2020 until 31 October 2024. The objective of COSSEROOTS project is to systematically investigate the relative parameterization modeling technique in order to create for the first time a toolbox dedicated to modeling and control of slender, flexible, continuous, bio-inspired robots that can undergo large, controlled deformations.
11 Dissemination
11.1 Promoting scientific activities
11.1.1 Scientific events: organisation
Member of the organizing committees
- Christian Duriez is member of the organizing committee of ICRA 2020 (in Paris), chair of Social Media and Community.
- Stefan Escaida Navarro was one of the main organizers of the 3rd Workshop on Proximity Perception in Robotics at IROS 2020, which was held online.
11.1.2 Scientific events: selection
Chair of conference program committees
- Gang Zheng is IPC member of APCRAS 2020, ICSRT 2020, RSV 2020, ETFA 2020
- Jeremie Dequidt is IPC member of International Symposium on Visual Computing 2019 and 2020
- Christian Duriez is IPC of Robosoft 2020 and Guest Editor for IEEE World Haptic Conference 2021 (Transaction on Haptics paper track)
Reviewer
Members of the team review several articles for the robotics conferences: Robosoft, ICRA and IROS.
11.1.3 Journal
Member of the editorial boards
Christian Duriez is associate editor of IEEE Robotics and Automation Letters.
Gang Zheng is associate editor of IET Cyber-Systems and Robotics.
11.1.4 Leadership within the scientific community
- Gang Zheng has the following scientific responsibilities:
- Deputy director of LAI with the Nanjing University of Science and Technology, China
- Member of IFAC TC4.3
- Christian Duriez is the coordinator of the CPER CornelIA (for research on IA at the regional scale)
11.1.5 Scientific expertise
- Christian Duriez is a scientific advisor of the InSimo company.
- Christian Duriez is monitor for the European Community, to evaluate the FET project HybridHeart. An evaluation report was written in May 2020
- Olivier Goury was member of the Section 61 hiring committee for an Assistant Professor (Maître de conférences) position at Université de Franche Comte.
- Jeremie Dequidt is member of the evaluation committee CE33 for ANR (AAPG 2020–2021)
11.1.6 Research administration
- Gang Zheng was Vice-President of the CRCN-ISFP jury at Inria Lille, 2020.
- Gang Zheng is member of Ecole Doctorale SPI 072 in the domain of AGITSI, from 2020
- Gang Zheng is member of the scientific bureau of Inria Lille – Nord Europe
- Christian Duriez was president of the research jobs commission at Inria Lille
- Alexandre Kruszewski was member of the lab council of CRIStAL UMR CNRS 9189
- Olivier Goury was member of the Center Committee (comité du centre) of the Inria Lille – Nord Europe research center until November 2020.
- Olivier Goury was member of the Technology Development Committee (Comission développement technologique; CDT) at Inria Lille – Nord Europe.
11.2 Teaching - Supervision - Juries
11.2.1 Teaching
Teaching administration
- Jérémie Dequidt is the head of the Embedded Systems Department at Polytech Lille
- Alexandre Kruszewski is in charge of the Embedded and Cyberphysical system major of Centrale Lille. He is also in charge of the coordination of the ‘automatic control' part of the EEA department of Centrale Lille Institute.
Teaching activities
- Engineering cycle: Walid Amehri, Start & Go Arts et Sciences, 24h, level L3, Centrale Lille.
- Engineering cycle: Walid Amehri, Start & Go Conception et Environement, 10h, level L3, Centrale Lille.
- Engineering cycle: Walid Amehri, Automatique IE3, 20h, level L3, Centrale Lille.
- Engineering cycle: Alexandre Kruszewski, 14 modules (automatic control, numeric control, embeded systems, robotics etc.), 250h, level (L3, M1, M2), Centrale Lille
- Engineering cycle: Jeremie Dequidt 6 modules (Programming, Software Engineering, Embedded Systems, Databases, Medical Simulation etc.) 350h, level (L3, M1, M2), Polytech Lille
- Master: Christian Duriez, Soft robotics, 24h, M2, Graduate degree en intelligence artificielle à l'Ecole Polytechnique (Palaiseau)
- Master: Christian Duriez, Interactive simulation, 20h, M2, Master Image Visualisation Interaction, Université de Lille
11.2.2 Supervision
- PhD in progress: Walid Amehri, Workspace analysis of soft robots, G. Zheng and A. Kruszewski
- PhD in progress: Ke Wu, Control of soft robot under constraints, G. Zheng
- PhD in progress: Pierre Schegg, Catheter Navigation using Reinforcement Learning, J. Dequidt and C. Duriez
- PhD in progress: Félix Vanneste, Design and simulation of Soft Robots made of mesostructured materials, since December 2018, C. Duriez and O. Goury
- PhD in progress: Etienne Ménager, High level control of soft robot, C.Duriez
- PhD in progress: Lingxiao Xun, Modeling and control of ionic polymers, A. Kruszewski and G. Zheng
- PhD in progress: Shijie Lin, Dynamic Control of soft robot, A. Kruszewski
11.2.3 Juries
- Christian Duriez was member of the following juries of PhD thesis:
- Camille Krewcun, Université de Clermont Auvergne, Novembre 2020 (Reviewer)
- Romain Lagneau, INSA Rennes, December 2020 (Reviewer)
- Juan Miguel Alvarez Palacio, Ecole Nationale Supérieure d'Arts et Métiers, March 2020 (Examinateur)
- Gang Zheng has participated the following juries of thesis:
- Lifei Wang, INSA Centre Val de Loire, (Reviewer)
11.3 Popularization
11.3.1 Articles and contents
- Christian Duriez wrote a paper in The Conversation. This article was reprinted in Usine Nouvelle and Numerama.
- Le Figaro has published a paper asking for expertise of C. Duriez: “Des arbres pour inspirer la robotique” (journal paper non visible online, journalist Sylvie Redon-Clauzard, published end of October 2020)
- La voix du Nord has published a 2 page paper on the work of the team. Visit the website.
- Le journal 20 minutes has republished a part of the content produced for The Conversation (see above) in the form of a shorter journal paper. Visit the website
11.3.2 Education
11.3.3 Interventions
Starting in late 2019, visual artist Yosra Mojtahedi started developing her installation “L'Erosarbénus” with the help of DEFROST. During 2020, there was an intense collaboration to develop and automate the Soft Robotics devices that would make the installation become alive. This collaboration allowed DEFROST to experiment in the domains of design, fabrication and automation and helped the artist in realizing her vision.
12 Scientific production
12.1 Major publications
- 2 article Design, implementation and control of a deformable manipulator robot based on a compliant spine The International Journal of Robotics Research May 2020
- 3 articleAnisotropic soft robots based on 3D printed meso-structured materials: design, modeling by homogenization and simulationIEEE Robotics and Automation Letters52January 2020, 2380-2386
- 4 article FEM-based Gain-Scheduling Control of a Soft Trunk Robot IEEE Robotics and Automation Letters 2021
12.2 Publications of the year
International journals
International peer-reviewed conferences
Conferences without proceedings
Scientific books
Scientific book chapters
Reports & preprints
12.3 Other
Patents