• The Inria's Research Teams produce an annual Activity Report presenting their activities and their results of the year. These reports include the team members, the scientific program, the software developed by the team and the new results of the year. The report also describes the grants, contracts and the activities of dissemination and teaching. Finally, the report gives the list of publications of the year.

• Legal notice
• Personal data

#### AUCTUS

##### AUCTUS - 2021

2021
Activity report
Project-Team
AUCTUS
RNSR: 201722256X
Research center
In partnership with:
Institut Polytechnique de Bordeaux
Team name:
Augmenting human comfort in the factory using cobots
Domain
Perception, Cognition and Interaction
Theme
Robotics and Smart environments
Creation of the Project-Team: 2020 April 01

# Keywords

• A2.1.5. Constraint programming
• A5.1.1. Engineering of interactive systems
• A5.1.2. Evaluation of interactive systems
• A5.1.7. Multimodal interfaces
• A5.4.4. 3D and spatio-temporal reconstruction
• A5.4.5. Object tracking and motion analysis
• A5.5.1. Geometrical modeling
• A5.10.1. Design
• A5.10.2. Perception
• A5.10.4. Robot control
• A5.10.5. Robot interaction (with the environment, humans, other robots)
• A5.10.8. Cognitive robotics and systems
• A5.11.1. Human activity analysis and recognition
• A6.2.5. Numerical Linear Algebra
• A6.2.6. Optimization
• A6.4.6. Optimal control
• A6.5.1. Solid mechanics
• A8.3. Geometry, Topology
• A9.5. Robotics
• A9.8. Reasoning
• B1.2.2. Cognitive science
• B2.8. Sports, performance, motor skills
• B5.1. Factory of the future
• B5.2. Design and manufacturing
• B5.6. Robotic systems
• B9.6. Humanities
• B9.9. Ethics

# 1 Team members, visitors, external collaborators

## Research Scientists

• David Daney [Team leader, Inria, Researcher, HDR]
• Vincent Padois [Inria, Senior Researcher, HDR]

## Faculty Members

• Nasser Rezzoug [Univ de Toulon et du Var, Associate Professor, in secondment with a 50% Teaching discharge from Toulon Univ., HDR]
• Jean Marc Salotti [Institut National Polytechnique de Bordeaux, Professor, HDR]

## Post-Doctoral Fellow

• Lucas Joseph [Inria, until Jun 2021]

## PhD Students

• Nassim Benhabib [Inria]
• Raphael Bousigues [inria, in collaboration LARSEN, located in Nancy]
• Benjamin Camblor [Inria]
• Pierre Laguillaumie [Univ de Poitiers]
• Gautier Laisne [Inria, from Oct 2021]
• Erwann Landais [Inria, from Sep 2021]
• Julien Passama [Akka Technologies, CIFRE, from Oct 2021]
• Nicolas Simonazzi [Orange, CIFRE, until May 2021]
• Antun Skuric [Inria]
• Nicolas Torres [Groupe PSA, CIFRE]

## Technical Staff

• Erwann Landais [Inria, Engineer, until Aug 2021]
• Guillaume de Mathelin de Papigny [inria, Engineer, Until Dec 2021]

## Interns and Apprentices

• Arthur Esquerre Pourtere [Inria, from Mar 2021 until Aug 2021]
• Pirewa Lagaza [Farm cube, from Mar 2021 until Jul 2021]
• Gianmarco Panzetta [Inria, from Sep 2021]

• Chrystel Plumejeau [Inria]

## External Collaborators

• Lucas Joseph [CNRS, from Aug 2021]
• Margot Vulliez [Institut Pprime de Poitiers, from Mar 2021]

# 2 Overall objectives

The project of the Auctus team is to design the collaborative robotics cells of the future.

The robotics community still tends to separate the cognitive (HRI) and physical (pHRI) aspects of human/robot interaction. One of the main challenges is to characterize the task as well as biomechanical, physiological and cognitive capacities of humans in the form of physical constraints or objectives for the design of cobotized workstations. This design is understood in a large sense: the choice of the robot's architecture (cobot, exoskeleton, etc.), the dimensional design (human/robot workspace, trajectory calculation, etc.), the coupling mode (comanipulation, teleoperation, etc.) and control. The approach then requires the contributions of the human and social sciences to be considered in the same way as those of exact sciences. The topics considered are broad, ranging from cognitive sciences, ergonomics, human factors, biomechanics and robotics.

The first challenge is to evaluate the hardship at work, the well-being of the operators and, further upstream, their cognitive state which impacts their sensorimotor strategy for performing a task. In the industry, the ergonomic analysis of the task is carried out by an ergonomist based on direct but often ad hoc observations. However, the context is changing: the digitization of factories, through the installation of on-site sensors, allows longitudinal observation of machines and humans. The information available can thus allow us to rethink the way in which the evaluation of activities is carried out. Currently, an emerging subdomain named ergonomic robotics adapts the available ergonomic evaluation criteria (RULA, REBA, etc.). However, they are related to the (quasi-static) posture of the operator, which limits the understanding of human motor strategies over a long period of time. Similarly, kinematic or biomechanical analysis may tend to see humans as a high-performance machine to be optimized. This may make sense for a top-level athlete, but repeating actions in the industry over a day, months or years of work means that a temporary change of posture, possibly poorly rated according to usual ergonomic criteria, can in fact be a good long-term strategy. These questions make a direct link between motor and cognitive aspects that can be reflected in particular strategies as the fatigue or the expertise (manual and cognitive). This approach has not been widely explored in robotics to determine the right criteria to adapt the behavior of a cobot.

The second challenge is to define a methodology to link the analysis of the task and the human movements it induces to the robot design. Indeed, as we have been able to verify on several occasions in the context of industrial projects, between the ergonomist, expert in task analysis and psychology, and the robotician, expert in mechanics, control and computer science, there is a significant conceptual distance that makes it very difficult to analyze needs and define the specifications of the technical solution. To fill these methodological gaps, it is necessary, on the basis of case studies, to better define the notion of tasks in the context of a human/robot coupling and to establish a typology of this type of interaction by taking into account, with as much details as possible, the different physical and cognitive constraints and their potential psychological, organizational or ethical impacts.

The third challenge is related to the need to think about the control laws of collaborative robots in terms of human/robot coupling. The effectiveness of this coupling requires an ability to predict future human actions. This prediction should make the interaction more intuitive but also aims at an optimal coupling from the point of view of “slow” phenomena such as fatigue. The major challenge is therefore to move from reactive to predictive control laws, integrating a human prediction model, both in terms of movement strategies and decision strategies. Beyond the great computational complexity of predictive approaches, obtaining prediction models is an ambitious challenge. It is indeed necessary to learn models that are quite complex in terms of the physical realities they can account for and quite simple from a computational point of view.

# 3 Research program

## 3.1 Analysis and modelling of human behavior

### 3.1.1 Scientific Context

The purpose of this axis is to provide metrics to assess human behavior. We place ourselves here from the point of view of the human being and more precisely of the industrial operator. We assume the following working hypotheses: the operator's task and environmental conditions are known and circumscribed; the operator is trained in the task, production tools and safety instructions; the task is repeated with more or less frequent intervals. We focus our proposals on assessing:

• the physical and cognitive fragility of operators in order to meet assistance needs;
• cognitive biases and physical constraints leading to a loss of operator safety;
• ergonomic, performance and acceptance of the production tool.

In the industrial context, the fields that best answer these questions are work ergonomics and cognitive sciences. Scientists typically work on 4 axes: physiological/biomechanical, cognitive, psychological and sociological. More specifically, we focus on biomechanical, cognitive and psychological aspects, as described by the ANACT 26, 27. The aim here is to translate these factors into metrics, optimality criteria or constraints in order to implement them in our methodologies for analysis, design and control of the collaborative robot.

To understand our desired contributions in robotics, we must review the current state of ergonomic workstation evaluation, particularly at the biomechanical level. The ergonomist evaluates the gesture through the observation of workstations and, generally, through questionnaires. This requires long periods of field observation, followed by analyses based on ergonomic grids (e.g. RULA 38, REBA 29, LUBA 34, OWAS 33, ROSA 47,...). Until then, the use of more complex measurement systems was reserved for laboratories, particularly biomechanical laboratories. The appearance of inexpensive sensors such as IMUs (Inertial Measurement Units) or RGB-D cameras makes it possible to consider a digitalized, and therefore objective, observation of the gesture, postures and more generally of human movement. Thanks to these sensors, which are more or less intrusive, it is now possible to permanently install observation systems on production lines. This completely changes paradigms and opens the door to longitudinal observations. It should be noted that this is comparable to the evolution of maintenance, which becomes predictive.

On the strength of this new paradigm, ergonomic robotics has recently taken an interest in this type of evaluation to adapt the robot's movements in order to reduce ergonomic risk scores. This approach complements the more traditional approaches that only consider the performance of the action produced by the human in interaction with the robot. However, we must go further. Indeed, the ergonomic criteria are based on the principle that the comfort positions are distant from the human joints articular limits. In addition, the notation must be compatible with an observation of the human being through the eye of the ergonomist. In practice, evaluations are inaccurate and subjective 50. Moreover, they are made for quasi-static human positions without taking into account the evolution of the person's physical, physiological and psychological state. The repetition of gestures, the solicitation of muscles and joints is one of the questions that must complete these analyses. One of the methods used by ergonomists to limit biomechanical exposures is to increase variations in motor stress by rotating tasks 48. However, this type of extrinsic method is not always possible in the industrial context 37.

One of Auctus' objectives is to show how, through a cobot, the operator's environment can be varied to encourage more appropriate motor strategies. To do so, we must focus on a field of biomechanics that studies the intrinsic variability of the motor system allowed by the joint redundancy of the human body. This motor variability refers to the natural alternation of postures, movements and muscle activity observed in the individual to respond to a requested task 48. This natural variation leads to differences between the motor coordinates used by individuals, which evokes the notion of motor strategy 30.

As shown by the cognitive dimension of ergonomics (see above), we believe that some of these motor strategies are a physically quantifiable reflection of the operator's cognitive state. For example, fatigue 45 and its anticipation or the manual expertise (dexterous and cognitive) of the operator which allows him to anticipate his movements over long periods of time in order to preserve his body, his performance and his pain.

### 3.1.2 Methodology

How can we observe, understand and quantify these human motor strategies to better design and control the behavior of the cobotic assistant? When we study the systems of equations considered (kinematic, static, dynamic, musculoskeletal), several problems appear and explain our methodological choices:

• the large dimensions of the problems to be considered, due to joint, muscle and placement redundancy;
• the variabilities of the parameters, for example: physiological (consider not an operator, but a set of operators), geometric (consider a set of possible placements of the operator) and static (consider a set of forces that the operator must produce);
• the uncertainties of measurement and models approximation.

The idea is to start from a description of redundant workspaces (geometric, static, dynamic...). To do this, we use set theory approaches, based on interval analysis 49, 41, which allow us to respond to the uncertainties and variability issues previously mentioned. In addition, one of the advantages of these techniques is that they allow the results to be certified, which is essential to address safety issues. Some members of the team has already achieved success in mechanical design for performance certification and robot design 40. The adaptation of these approaches allows us to obtain a mapping of ergonomic and efficient movements in which we can project the operators' motor strategies and thus define a metric quantifying the sensorimotor commands chosen with regard to the cognitive criteria studied.

It is therefore necessary to:

• to model human capabilities at the musculoskeletal level to allow for global, yet detailed, analysis as well as efficient integration of such knowledge at the control level of collaborative robots.
• propose new indices in the fields of ergonomics, biomechanics and robotics linking different types of performances and taking into account the influence of fatigue, stress, level of expertise;
• divide the gesture into homogeneous phases: this process is complex and depends on the type of index used and the techniques used. We are exploring several ways: inverse optimal control, learning methods, or the use of techniques from signal processing;
• develop interval extensions of the identified indices. These indices are not necessarily the result of a direct model, and algorithms need to be developed or adapted (calculation of manipulability, UCM, etc.);
• Aggregate proposals into a dedicated interval analysis library (use of and contribution to the existing ALIAS-Inria and the open source IBEX library).

The originality and contribution of the methodology is to allow an analysis taking into account in the same model the measurement uncertainties (important for on-site use of analytical equipment), the variability of tasks and trajectories, and the physiological characteristics of the operators.

Other avenues of research are being explored, particularly around the inverse optimal control 42 which allows us to project human movement on the basis of performance indices and thus to offer a possible interpretation in the analysis of behaviors.

We also use automatic classification techniques: 1) to propose cognitive models that will be learned experimentally 2) for segmentation or motion recognition, for example by testing Reservoir Computing 31 approaches.

## 3.2 Operator / robot coupling

This research axis is at the frontier between humans and robots and is focused on the question of coupling these two entities in the best possible way. This raises questions which relates both to human models and abilities (axis 1) and robot control (axis 3). One way to approach this research axis is to tackle the formal computation of joint human-robot capabilities, beyond the activities in Axis 1 focused on the ability to quantify aspects such as human fatigue and motor variability (see 12 and the Mover project) or force production abilities 16.

Taking into account the coupling between human and robotic constraints at the control level is also central to the team's objectives. We wish to understand how the robot influences the operator's work and thus how his mental model of the task evolves according to the interactions with the robot. The challenge is then to predict a behavior of the operator that takes into account his cognition. In the context of a manual task at work, we can rely on abilities clearly identified by the required know-how and instructions to perform it.

We then wish to demonstrate how a collaborative robot can be used to mediate between a control objective based on a globally ideal compromise between task performance, safety and comfort (what we consider as the expertise trinity) and the subjective action model of the human influenced by his level of expertise.

Manual expertise of highly skilled operators is also of paramount interest and needs to be analyzed respectively on its dynamic aspects and on the ability to synchronize with other operators in the environment. Better understanding expertise is envisioned as a way to alleviate the operators of repetitive and easy to handle tasks and decide when they should actually replace the robot based on the complexity of the task to perform. It can also be a way to get the operator far from the operating zone through teleoperation while correctly reflecting its expert abilities (and potentially correcting for its punctual weaknesses).

The subject of human-robot collaboration covers a wide spectrum of topics. We are strongly confident that the notion of expertise is central, especially in relation with the notion of fatigue and motor variability (axis 1). From the cognitive side of the coupling, one of the main challenges lies in the ability to map cognitive level constraints into control constraints. This relates much to the way the control problem is expressed (see axis 3) but also in an on-going effort to better modeling the cognitive aspects of human-robot collaboration (axis 1). Furthermore, this research axis raises the question of the modification of the work induced by collaborative robots for expert operators. While the overall goal is clearly to make use of robots to punctually or continuously improve the work conditions of these operators (and clearly not to replace them), the presence of these robots necessarily impacts the work referential and thus the expertise itself. One of the central questions, yet to be tackled, relates to the original and core part of the expertise that should remain unchanged.

## 3.3 Design of cobotic systems

### 3.3.1 Architectural design

Is it necessary to cobotize, robotize or assist the human being? Which mechanical architecture meets the task challenges (a serial cobot, a specific mechanism, an exoskeleton)? What type of interaction (H/R cohabitation, comanipulation, teleoperation)? These questions are the first requests from our industrial partners. For the moment, we have few comprehensive methodological answers to provide them. Choosing a collaborative robot architecture is a difficult problem 35. It is all the more when the questions are approached from both a cognitive ergonomics, boimechanical and robotics perspectives. There are indeed major methodological and conceptual differences in these areas. It is therefore necessary to bridge these representational gaps and to propose an approach that takes into consideration the expectations of the robotician to model and formalize the general properties of a cobotic system as well as those of the ergonomist to define the expectations in terms of an assistance tool.

To do this, we propose a user-centered design approach, with a particular focus on human-system interactions. From a methodological point of view, this requires first of all the development of a structured experimental approach aimed at characterizing the task to be carried out through a “system” analysis but also at capturing the physical markers of its realization: movements and efforts required, ergonomic stress. This characterization must be done through the prism of the systematic study of the exchange of information (and their nature) by humans in their performance of the considered task. On the basis of these analyses, the main challenge is to define a decision support tool for the choice of the robotic architecture and for the specifications of the role assigned to the robot and the operator as well as their interactions.

The evolution of the chosen methodology is for the moment empirical, based on the user cases regularly treated in the team (see sections on contracts and partnerships).

It can be summarized for the moment as:

• identify difficult jobs on industrial sites. This is done through visits and exchanges with our partners (manager, production manager, ergonomist...);
• select some of them, then observe the human in its ecological environment. Our tools allow us to produce a motion analysis, currently based on ergonomic criteria. In parallel we carry out a physical evaluation of the task in terms of expected performance and an evaluation of the operator by means of questionnaires;
• synthesize these first results to deduce the robotic architectures to be initiated, the key points of human-robot interaction to be developed, the difficulties in terms of human factors to be taken into account.

In addition, the different human and task analyses take advantage of the different expertise available within the team. We would like to gradually introduce the evaluation criteria presented above. Indeed, the team has already worked on the current dominant approach: the use of a virtual human to design the cobotic cell through virtual tools 3. However, the very large dimensions of the problems treated (modelling of the body's dofs and the constraints applied to it) makes it difficult to carry out a certified analysis. We then choose to go through the calculation of the body's workspace, representing its different performances, which is not yet done in this field. The idea here is to apply set theory approaches, using interval analysis already discussed in section 3.1.2. The goal is then to extend to intervals the constraints played in virtual reality during the simulation. This would allow the operator to check his trajectories and scenarios not only for a single case study but also for sets of cases. For example, it can be verified that, regardless of the bounded sets of simulated operator physiologies, the physical constraints of a simulated trajectory are not violated. Thus, the assisted design tools certify cases of use as a whole. Moreover, the intersection between the human and robot workspaces provides the necessary constraints to certify the feasibility of a task. This allows us to better design a cobotic system to integrate physical constraints. In the same way, we will look for ways in which human cognitive markers can be included in this approach.

Thus, we summarize here the contributions of the other research axes, from the analysis of human behavior in its environment for an identified task, to the choice of a mechanical architecture, via an evaluation of torque and interactions. All the previous analyses provide design constraints. This methodological approach is perfectly integrated into an Appropriate Design approach used for the dimensional design of robots, again based on interval analysis. Indeed, to the desired performance of the human-robot couple in relation to a task, it is sufficient to add the constraints limiting the difficulty of the operator's gesture as described above. The challenges are then the change of scale in models that symbiotically consider the human-robot pair, the uncertain, flexible and uncontrollable nature of human behavior and the many evaluation indices needed to describe them.

### 3.3.2 Control design

The control laws of collaborative robots from the major robot manufacturers differ little or not at all from the existing control laws in the field of conventional industrial robotics. Security is managed a posteriori, as an exception, by a security PLC / PC. It is therefore not an intrinsic property of the controller. This quite strongly restricts the possibilities of physical interaction 1 and collaboration and leads to sub-optimal operation of the robotic system. It is difficult in this context to envision real human-robot collaboration. Collaborative operation requires, in this case, a control calculation that integrates safety and ergonomics as a priori constraints.

The control of truly collaborative robots in an industrial context is, from our point of view, underpinned by two main issues. The first is related to the macroscopic adaptation of the robot's behaviour according to the phases of the production process. The second is related to the fine adaptation of the degree and/or nature of the robot's assistance according to the ergonomic state of the operator. If this second problem is part of a historical dynamic in robotics that consists in placing safety constraints, particularly those related to the presence of a human being, at the heart of the control problem 2839, 32, it is not approached from the more subtle point of view of ergonomics where the objective cannot be translated only in terms of human life or death, but rather in terms of long-term respect for their physical and mental integrity. Thus, the simple and progressive adoption by a human operator of the collaborative robot intended to assist him in his gesture requires a self-adaptation in the time of the command. This self-adaptation is a fairly new subject in the literature 43, 44.

At the macroscopic level, the task plan to be performed for a given industrial operation can be represented by a finite state machine. In order not to increase the human's cognitive load by explicitly asking him to manage transitions for the robot, we propose to develop a decision algorithm to ensure discrete transitions from one task (and the associated assistance mode) to another based on an online estimate of the current state of the human-robot couple. The associated scientific challenge requires establishing a link between the robot's involvement and a given working situation. To do so, we propose an incremental approach to learning this complex relationship. The first stage of this work will consist in identifying the general and relevant control variables to conduct this learning in an efficient and reusable way, regardless of the particular method of calculating the control action. Physically realistic simulations and real word experiments will be used to feed this learning process.

In order to handle mode transitions, we propose to explore the richness of the multi-tasking control formalism under constraints 36 in order to ensure a continuous transition from one control mode to another while guaranteeing compliance with a certain number ofrobotoocc control constraints. Some of these constraints are based on ergonomic specifications and are dependent on the state of the robot and of the human operator, which, by nature, is difficult to predict accurately. We propose to exploit the interval analysis paradigm to efficiently formulate ergonomic constraints robust to the various existing uncertainties.

Purely discrete or reactive adaptation of the control law would make no sense given the slow dynamics of certain physiological phenomena such as fatigue. Thus, we propose to formulate the control problem as a predictive problem where the impact of the control decision at a time $t$ is anticipated at different time horizons. This requires a prediction of human movement and knowledge of the motor variability strategies it employs. This prediction is possible on the basis of the supervision at all times of the operational objectives (task in progress) in the short term. However, it requires the use of a virtual human model and possibly a dynamic simulation to quantify the impact of these potential movements in terms of performance, including ergonomics. It is impractical to use a predictive command with simulation in the loop with an advanced virtual manikin model. We therefore propose to adapt the prediction horizon and the complexity of the corresponding model in order to guarantee a reasonable computational complexity.

# 4 Application domains

## 4.1 Factory 4.0

The 4th industrial revolution (factory 4.0) is characterized by the integration of digital technologies into the production process, in order to meet the challenge of customizing services and products. This agility requires making manufacturing and maintenance lines flexible and versatile. This capacity for adaptation is the characteristic of the human being, which puts him at the center of the production apparatus. However, this can no longer be done at the expense of their health and well-being. How then can we reconcile the enhancement of our manual and analytical expertise, the ever desired increase in productivity and manufacturing quality, while reducing the hardship at work? Collaborative robotics, which we are seeking to build, is one of the central solutions to meet this societal challenge. By assisting humans in their most dangerous and painful tasks, it complements or replaces them in their phases of physical and cognitive fragility.

More generally, we are interested in workstation cobotization, in the manufacturing and assembly industry but also in the construction and craft industries. The application areas are related to regional needs in aeronautics, including maintenance, water and waste treatment. In most of these cases, it is possible to define the tasks, evaluate the stakes and added value of our work.

# 5 Social and environmental responsibility

The scientific positioning of Auctus has an explicit social objective : assisting industrial workers to improve their working conditions through the appropriate limitations of physical solicitations and the improvement of their cognitive comfort. This has a direct societal impact on the health of the population and regarding the preservation of industrial skills and expertise in the local and national industrial ecosystem.

From an environmental point of view, the research goals of Auctus do not explicitly aim at improving the human footprint on the planet or at better understanding environmental related issues and processes. Yet, some of our projects can have a direct impact on these issues. This impact is for example directly related to the application context in the case of our collaboration in the domain of remote operation of technical skills with the Farm3 company. Indeed, this company aims at producing plants locally, with a reduced physical footprint while minimizing water consumption. We also envision a less direct but more fundamental impact of our work in the control domain where we aim at exploiting at best robot capabilities. In the long term the impact of this work should lead to a reduction of the size of robots and of the amount of energy they consume to achieve a given task. This is clearly in line with the general objective of saving energy as well as the natural resources.

# 6 Highlights of the year

## 6.1 Awards

Antun Skuric has received the 2021 Transactions on Automation Science and Engineering Best Paper Award for the paper “A Recursive Watermark Method for Hard Real-Time Industrial Control Systems Cyber Resilience Enhancement” 46, proposing an efficient real-time encryption algorithm for improving the safety of control systems.

## 6.2 Learned Societies

Nasser Rezzoug is Vice-President of the Society of Biomechanics since october 2021.

# 7 New software and platforms

This section lists the main software programs that have been developed by the team as well as the ones that evolved this year. It also describes the platforms developed this year.

## 7.1 New software

### 7.1.1 torque_qp

• Name:
torque_qp
• Keywords:
Robot Operating System (ROS), Robot Panda, Simulation, Robotics
• Scientific Description:
Formulation of a control problem as a constrained optimization problem at the joint torque level.
• Functional Description:
This software is a control architecture for the Franka Emika Panda robot. The control problem is formulated as a constrained optimization at the joint torque level. This software is interfaced with Gazebo for simulaton, and the franka_ros packages for sending the desired velocity to the real robot.
• URL:
• Authors:
• Contact:
Lucas Joseph

### 7.1.2 velocity_qp

• Name:
velocity_qp
• Keywords:
Robotics, Robot Operating System (ROS), Robot Panda, Simulation
• Scientific Description:
Formulation of a control problem as a constrained optimization problem at the joint velocity level.
• Functional Description:
This software is a control architecture for the Franka Emika Panda robot. The control problem is formulated as a constrained optimization at the joint velocity level. This software is interfaced with Gazebo for simulaton, and the franka_ros packages for sending the desired velocity to the real robot.
• URL:
• Authors:
• Contact:
Lucas Joseph

### 7.1.3 pycapacity

• Name:
Real-time capable task-space capability calculation module in python
• Keywords:
Computational geometry, Kinematics, Robotics, Biomechanics
• Scientific Description:

Many recent human-robot collaboration strategies, are promoting human-centered robot control, where the robot continuously adapts its assistance level based on the real-time need of its human counterpart. One of the fundamental assumptions of these approaches is the ability to measure or estimate the physical capacity of humans and robots in real-time. One of the most well known task-space capability metrics is manipulability ellipsoid, which has been developed the robotics community but is used for humans as well, for motion analysis and rehabilitation. Even though the ellipsoid measures have been used widely for many years they are proven to represent the underestimation of the real capabilities, which have a form of polytopes. However, the robot and human capability polytopes are much more inefficient to calculate and so far have not been used for real-time applications.

Therefore, pycapacity software package implements the set of algorithms that have been developed in two recent papers of the AUCTUS team that bring the polytope evaluation in the real-time spectrum.

• Functional Description:
Pycapacity is a python package which provides a framework for the generic task-space capability calculation of robotic serial manipulators and human musculoskeletal models, based on the ellipsoid and polytope metrics. Additionally, the package also provides a set of generic polytope evaluation algorithms for standard polytope formulations, that can be used as a standalone library.
• URL:
• Publications:
• Authors:
Antun Skuric, Vincent Padois, David Daney, Nasser Rezzoug
• Contact:
Antun Skuric

### 7.1.4 ForcePolySearch

• Name:
On-line force capability evaluation based on efficient polytope vertex search
• Keywords:
Robotics, Kinematics, SVD, Real-time application
• Scientific Description:
Ellipsoid-based manipulability measures are often used to characterize the force / velocity task-space capabilities of robots. While computationally simple, this approach largely approximate and underestimate the true capabilities. Force / velocity polytopes appear to be a more appropriate representation to characterize robot's task-space capabilities. However, due to the computational complexity of the associated vertex search problem, the polytope approach is mostly restricted to offline use, e.g. as a tool aiding robot mechanical design, robot placement in work-space and offline trajectory planning. In this paper, a novel on-line polytope vertex search algorithm is proposed. It exploits the parallelotop geometry of actuator constraints. The proposed algorithm significantly reduces the complexity and computation time of the vertex search problem in comparison to commonly used algorithms. In order to highlight the on-line capability of the proposed algorithm and its potential for robot control, a challenging experiment with two collaborating Franka Emika Panda robots, carrying a load of 12 kilograms, is proposed. In this experiment, the load distribution is adapted on-line, as a function of the configuration dependant task-space force capability of each robot, in order to avoid, as much as possible, the saturation of their capacity.
• Functional Description:
Software implementing new online capable polytope vertex search algorithm developed for force capacity evaluation of serial robotic manipulators, described in the paper: On-line force capability evaluation based on efficient polytope vertex search. The algorithm is capable of finding the vertices of force polytopes for 7DOF robot under 3 ms and 6DOF robot under 2 ms. Software repository consists of Matlab/Octave implementation intended for fast prototyping and benchmarking as well as Python module for easier integration in larger projects.
• URL:
• Publication:
• Authors:
Antun Skuric, Vincent Padois, David Daney
• Contact:
Antun Skuric

### 7.1.5 WoobotSim

• Keywords:
Robotics, Dynamic Analysis
• Functional Description:
WoobotSim is a simulator that reports the dynamics of the parties involved in an industrial task implying a strong interaction between a machine tool, an operator and a handled object, it also offers the possibility to add a cobot as an actor. Developed on Matlab, this simulator allows to visualize the efforts exchanged by the participants during the task, as well as the dynamics of the object being manipulated. For the specific case of woodworking shaper. It includes a wood cutting model. A model of task control by the craftsman and a model of the robot.
• URL:
• Contact:
Nassim Benhabib

### 7.1.6 WoobotCtrl

• Keywords:
Control, Robot Panda, Synchronization
• Functional Description:

WoobotCtrl is a set of ROS packages that provide control of a collaborative robot with 7 degrees of freedom (Franka's PANDA) in order to reproduce the cutting forces experienced by an operator during a wood shaping task.

Developed in C++, WoobotCtrl also manages the real-time acquisition and synchronization of data from multiple sensors (Kinect, force sensors, robot sensors). As well as the pre-shrinking of the data.

• URL:
• Contact:
Nassim Benhabib
• Participants:

### 7.1.7 PolyIOC

• Keyword:
Optimal control
• Functional Description:
Software for solving the Inverse Optimal Control problem with polynomials: - Direct Optimal Control - Inverse Optimal Control - Polynomial Direct Optimal Control - Polynomial Inverse Optimal Control - Truncated polynomial direct optimal control - Truncated polynomial inverse optimal control
• URL:
• Contact:
Jessica Colombel

### 7.1.8 rtt_panda

• Name:
rtt_panda
• Keywords:
OROCOS component, Robot Operating System (ROS), Robotics, Simulation
• Functional Description:

A set of tools to control a robot with the OROCOS toolchain software.

Implemented tools: a library using KDL to modelize the robot , a wrapper for qpOASES , a Cartesian trajectory generator using KDL , a cartesian PID controller , a library to publish OROCOS port to ROS topics , a driver for the Panda Franka Emika robot , a Gazebo plugin , a set of tool to launch the scripts for the panda

• URL:
• Author:
Lucas Joseph
• Contact:
Lucas Joseph

### 7.1.9 keyence_laser_tracker

• Name:
keyence_laser_tracker
• Keyword:
Driver
• Functional Description:
A ROS package interfacing the Keyence SZ-V32N device. It allows the configuration of the device and periodically get the information distance measured by the device through UDP. A ROS message is published with all the laser information.
• URL:
• Author:
Lucas Joseph
• Contact:
Lucas Joseph

### 7.1.10 kdl_trajectories

• Keyword:
Trajectory Generation
• Functional Description:
This tools simplifies the definition of trajectories in Cartesian space. It allows to define the trajectory velocity profile as well as the type of motion (Linear, Circular, ...). The trajectories can be defined off-line through CSV description files. They can also be update on-line.
• URL:
• Author:
Lucas Joseph
• Contact:
Lucas Joseph

• Name:
Kinematic Chain Appropriate Design Library
• Keywords:
Interval analysis, Uncertainty, Kinematics
• Functional Description:
Software for the modelling and analysis of imprecise serial kinematic chains. Chain objects are built by iteratively adding rigid-body segments with associated joint connections (e.g., fixed, revolute, prismatic). Several standard options are provided to model each segment (e.g., Denavit-Hartenberg parameters, transformation matrices, twists). Each option accepts interval and non-interval arguments, allowing to model the uncertainties and variabilities of imprecise serial kinematic chains and also the conventional precise serial kinematic chains. Forward Kinematic (FK) and Inverse Kinematic (IK) solvers are available for Chain objects. The FK solver computes an outer bound of the set of poses associated with a set of joint configurations. The IK solver computes an outer bound of the set of joint configurations associated with a set of poses.
• URL:
• Publication:
• Contact:
David Daney
• Participants:
Joshua Kevin Pickard, David Daney, Nasser Rezzoug, Vincent Padois

### 7.1.12 AUCTUS-MOVER

• Name:
AUCTUS panda MOVER project
• Keywords:
Automatic control, Variability
• Functional Description:
Software for controlling the Franka Emika Panda robot to study human motor variabilities. Consists of a torque-based controller for the Franka Emika Panda collaborative robot with the following actuation modes: auto mode - the robot follows a predefined trajectory, manual mode - the robot is in a constrained gravity compensation mode (the constraints may be adapted online). A graphical interface allows the operator to switch between modes. Software for calibrating the robot with respect to the Optitrack motion capture system is also included.
• URL:
• Contact:
Joshua Kevin Pickard

### 7.1.13 AUCTUS-RT

• Name:
AUCTUS - Redunancy Tubes
• Keyword:
Variability
• Functional Description:
Software for modelling and analyzing human motor variabilities along a path. Currently used to study the redundant motions associated with the upper limb. Anatomical constraints for the shoulder, elbow, and wrist may be customized for a human subject using: maximal and minimal bounding regression equations, spherical polygon constraints. A constrained imprecise kinematic model of the subject is obtained and the task redundancies and joint redundancies associated with the constrained imprecise kinematic model are able to be evaluated via branch-and-bound exploration to determine achievable and non-achievable redundant motions of the human. For n redundant degrees-of-freedom, this provides an n-dimensional redundant workspace describing the human's capabilities. Along a path, this provides an (n+1)-dimensional redundant workspace tube.
• URL:
• Authors:
Joshua Kevin Pickard, Nasser Rezzoug, David Daney
• Contact:
Joshua Kevin Pickard

### 7.1.14 pyosim

• Keywords:
3D, Biomechanics
• Functional Description:

Opensim is a motion modeling and simulation tool, mainly used in biomechanics. It addresses a number of issues related to the manipulation of biological bodies (calculation of muscular forces, accurate modeling of musculoskeletal models based on experimental data, management of data from motion capture systems).

This tool has an API developed in C++. Since this API can be difficult to use, the pyosim library was developed to easily access the most important functions (inverse kinematics, position and orientation of the model's body parts, manipulation of the model's anatomical markers).

• URL:
• Contact:
Erwann Landais

### 7.1.15 fosim

• Keywords:
Biomechanics, 3D
• Functional Description:

Opensim is a motion modeling and simulation tool, mainly used in biomechanics. It addresses a number of issues related to the manipulation of biological bodies (calculation of muscular forces, accurate modeling of musculoskeletal models based on experimental data, management of data from motion capture systems).

This tool has an API developed in C++. Since this API can be difficult to use, the fosim library was developed to easily access the most important functions (inverse kinematics, position and orientation of the model's body parts, manipulation of the model's anatomical markers).

• URL:
• Contact:
Erwann Landais

### 7.1.16 ospi2urdf

• Keywords:
3D, Robotics, Biomechanics
• Functional Description:

Opensim is a motion modelling and simulation tool, mainly used in biomechanics. It addresses a number of issues related to the manipulation of biological bodies (calculation of muscular forces, precise modelling of musculoskeletal models based on experimental data, management of data from motion capture systems).

A number of robotics projects linking robots and humans could benefit from the functionalities of this tool for modelling the upper body of a human being in real time. This is why the objective of this project is to combine some of the functionalities of Opensim with ROS packages dedicated to the modelling, control and analysis of the movement of polyarticulated robotic systems (torque_qp, Pinocchio).

One of the paths currently being explored is to use OpenSim and ROS packages together for the same model of a human being. As the modeling format of Opensim models (.osim) is not the same as the one classically used by the ROS packages (.urdf), it is first necessary to convert the .osim format to .urdf. A library is being created for this purpose.

Once this library is completed, it will be possible to create the framework allowing the use of OpenSim functionalities to the packages used for the management of polyarticulated systems in real time, in particular for motion capture problems.

• URL:
• Contact:
Erwann Landais

## 7.2 New platforms

### 7.2.1 Arcol

Participants: Erwann Landais, Lucas Joseph, David Daney, Vincent Padois, Jean-Marc Salotti, Nasser Rezzoug.

In 2021, Auctus has pursued his effort of developping its experimental platform of collaborative Robotics (Arcol).

The Arcol platform provides technical support for the short, medium and long term experimental developments carried out within the framework of Auctus' scientific and dissemination activities. These technological developments are essentially software related in the context of human motion capture and real-time control of collaborative robots. Arcol aims at easing their implementation, deployment, documentation and support.

New developpements in 2021 are mostly related to:

• the real-time capability of the software architecture dedicated to the control of our two 7 degrees of freedom Panda robot (from Franka Emika);
• the ongoing effort related to the generic integration of sensors and robots in the platform through the use of middlewares such as ROS and Orocos;
• the ongoing effort to converge towards an online motion capture system connected to model computation softwares allowing for the online estimation of human capabilities.

### 7.2.2 FarmCube

Participants: Pirewa Lagaza, David Daney, Margot Vulliez.

Farm3 develops a robotic and ultrasound-based farm for vertical off-ground cultivation: The Cube. Supported by French starred chefs, the company aims at countering intensive farming by locally producing nutritive and tasty veggies and herbs, without pesticides nor GMOs, and in respect of the environment. Through a collaboration with the AUCTUS team, at INRIA Bordeaux, and the RoBioSS team at Pprime Institute, Farm3 aims at smartly robotize the Cube, to enhance the cultivation conditions while facilitating the remote control of the farm. The first challenge of this project is to set up the dedicated robotic manipulator inside the Cube to autonomously perform simple tasks, such as grasping and placing plant buckets on the cultivation grid. The second objective is to preserve the farmers’ expertise as they should be able to control the actions through a Human-Robot interface. To remotely perform the task, the teleoperation interface should be augmented with relevant visual and haptic information, related to the plants and the Cube.

### 7.2.3 Tower of Hanoi

Participants: Auctus Team.

In 2021 Auctus has developed a platform to be used as a showcase of the team activities. This platform evolves around the problem of the tower of Hanoi which require both physical interaction with the world and thought process. One or several robots are used to assist a human in solving the problem. This plateform will allow the team to presents there contribution in both the cognitive (HRI) and physical (pHRI) aspects of human/robot interaction.

The platform is designed to be fully autonomous. This means that external sensors must be used to sense the environment and get the state of the problem in real time as well as the presence of humans. A visual solution based on the use of a monocular camera and Apriltag markers was chosen. After a study on the type of Apriltag markers best suited to the project, a framework based on ROS packages was developed. This framework includes the generation and recognition of the Apriltag markers, the calibration of the intrinsic and extrinsic parameters of the camera as well as the position of the camera with respect to the robot and the generation and processing of the camera video stream.Trajectory generation algorithm must be implemented to adapt online the next trajectory of the robot depending on the state of the game. Control algorithms being at the core of Auctus, our state of the art tools are being used inside the project. A state machine using PDDL is used to solve the tower of hanoi problem. A second state machine is implemeted to supervise the correct execution of the robot tasks.

In the first version of this platform the robot must solve the problem without any human intervention and without any knowledge of the initial state of the problem. Here is a video of this initial version.

Future development on this showcase will involve:

• use of an RGBD camera to refine the position of the Apriltag markers in space;
• accounting for a human in the scene:
• the robot could still solve the problem alone, as fast as possible, but reduce its velocity depending on the distance to human to ensure safety;
• it could also be used to assist the human into solving the problem;
• it could also change its behavior to actively solve the problem alongside the human;
• two robots working on the problem and sharing their tasks;
• add cognitive markers such as a breathing behavior on the redundant axes. Or external sensor (leds, pushbutton, ...) to enhance the interaction with the robot.

### 7.2.4 Woobot platform

Participants: Nassim Ben Habib, Benjamin Camblor, Lucas Joseph, Vincent Padois, David Daney.

The physically realistic experimental mock-up of a wood milling task, initially designed within the framework of the Woobot project, has been improved to allow for the measurement of interaction forces (2 6-axis F/T sensors) and human posture (Kinect based) during experiments. This set-up has both been used for testing adaptive control of collaborative robots (Panda robot used both as as a cutting force emulatore and assisitive robot) and signaling motions for improved situation awareness.

# 8 New results

## 8.1 Human Factors and cognitive approaches in human/system interactions

### 8.1.1 Accident report analysis

Participants: Benjamin Camblor, Jean-Marc Salotti, David Daney.

We analyzed industrial robotics accidents from the INRS database EPICEA. We adopted an approach based on interactions in a robotic workspace, coupled with Endsley’s model of Situation Awareness (SA) to identify factors that potentially degrade operators’ SA. We analyzed 45 industrial accident reports involving robotic. Our analysis showed that SA demons were mainly associated with the perception level of SA. Additionally, SA demons are generally involved in human-robot interaction and human-environment interaction. We have also been able to describe five patterns of SA demon occurrence. At the end, we could propose recommendations regarding workspace design.

Related publications:

### 8.1.2 Bayesian network for accident probability

Participants: Benjamin Camblor, Jean-Marc Salotti, David Daney.

Based on the work on Accident report analysis, we wanted to develop a Bayesian network capable of calculating the probability of an accident according to the probabilities of occurrence of the different SA demons. A first version of the network has been implemented but it is still in a preliminary stage. In the future, we would like to integrate additional factors into this network so that it calculates more accurate probabilities. Our long-term goal is to integrate this Bayesian network into a task planner that will schedule actions between a human and a robot according to these accident probabilities. This will allow to secure the workspaces and to minimize the risks of accidents during human-robot interaction scenarios.

### 8.1.3 Situation awareness modeling in human-robot interaction

Participants: Benjamin Camblor, Jean-Marc Salotti, David Daney.

Situation awareness (SA) is a cognitive process that is involved in decision making. It is defined as "the perception of the elements of the environment in a volume of time and space, the understanding of their meaning and the projection of their status in the near future". SA is one of the human factors to be taken into account in human-robot interaction scenarios and is itself influenced by other factors that may be cognitive, physical or environmental. Our goal was to represent a schema describing those factors that influence SA in human-robot interaction. This model was built with the help of readings of already existing works in SA and robotics. It takes into account the SA of the human in relation to its environment but also in relation to the robot and the task he has to perform. It will later allow us to synthesize and verify the influence of some of these factors on SA in our research. The construction of this model is continuous and directly linked to the progress of our work.

### 8.1.4 Signaling motion to reinforce situational awareness during a human-robot interaction

Participants: Benjamin Camblor, David Daney, Jean-Marc Salotti, Nassim Benhabib, Vincent Padois.

We have explored the concept of signaling motion in the context of the improvement of situation awareness. These motions consist in using the redundant degrees of freedom of a robot performing a task as new means of meaningful robot-human communication. They are generated through quasi-static torque control, in consistency with the main robot task. A double within-subject (N=16) study has been conducted to evaluate the effects of two signaling motions (a diffuse one - breathing - and an explicit one - postural encoding) on the performance of a task by participants and on their behavior towards the robot. Our results show a positive effect on both the task execution and the participants behavior. Additionally, both signaling motions seem to improve the situation awareness of the participants by fueling their mental model throughout the interaction.

Based on a systematic assessment of potential SA demons and this work, our long term goal is to explore a general way to superimpose SA angels to robot motions in an automatic, seamless and appropriate way. One challenge to achieve this goal is related to the understanding of the relation between the context as well as user specific characteristics and the perception of signaling motions.

This work has been accepted for publication at the international conference on Human Robot Intercation (HRI) in 2022 21.

### 8.1.5 Interface design of a robotized handling and transport platform in a factory

Participants: Jean-Marc Salotti, David Daney.

In the Portage project, one of the modus operandi of using a shared robotized platform is to allow the remote operation, by a human, of heavy loads. During this year, our task was to propose a design for the interface of the program that manages and controls the robotic platforms, paying attention to safety and contingencies. We also worked on the criteria that should be used to assess the efficiency and usability of the platform.

We have contributed on three main topics :

• System use scenarios (hybrid control of the robot);
• The design, the modeling and the realization of the interface itself;
• The assessment of the task.

### 8.1.6 Interactions with a chatbot

Participants: Nicolas Simonazzi, Jean-Marc Salotti.

In the context of the CIFRE Orange PhD work by Nicolas Simonazzi under the supervision of Jean-Marc Salotti and with the objective of analyzing and identifying emotions during interactions with a chatbot, a second experiment was conducted. It involved a user, the use of a smartphone, viewing videos, asking questions about the content of the video and the registration of data collected through the sensors or the smartphone. An analysis of the data was carried out with Russell's relatively simple emotional model as an explanatory framework based on two variables, the positive or negative valence of the emotion, and the degree of excitement. A paper has been published on the use of the Genova Emotion Wheel to better characterize the emotions (18). The experimental results showed that there was a slight correlation between the valence indicated by the user, the accuracy of the answers to the questions and the accelerations of the smartphone. The thesis was successfully defended in November 2021 and a publication is currently being drafted.

Related publications:

## 8.2 Human Behavior Analysis

### 8.2.1 On-line feasible wrench polytope evaluation based on human musculoskeletal models: an iterative convex hull method

Participants: Antun Skuric, Vincent Padois, David Daney, Nasser Rezzoug.

Many recent human-robot collaboration strategies, such as Assist-As-Needed (AAN), are promoting human-centered robot control, where the robot continuously adapts its assistance level based on the real-time need of its human counterpart. One of the fundamental assumptions of these approaches is the ability to measure or estimate the physical capacity of humans in real-time.

In this work, we propose an algorithm for the feasibility set analysis of a generic class of linear algebra problems. This novel iterative convex-hull method is applied to the determination of the feasible Cartesian wrench polytope associated to a musculoskeletal model of the human upper limb. The method is capable of running in real-time and allows the user to define the desired estimation accuracy. The algorithm performance analysis shows that the execution time has near-linear relationship to the considered number of muscles, as opposed to the exponential relationship of the conventional methods. Finally, real-time robot control application of the algorithm is demonstrated in a Collaborative carrying experiment, where a human operator and a Franka Emika Panda robot jointly carry a 7kg object. The robot is controlled in accordance to the AAN paradigm maintaining the load carried by the human operator at 30

Related publications: Related publication: 24

### 8.2.2 Musculoskeletal modelling for the study of personalized human force capacities through data-driven learning approach

Participants: Gautier Laisné, Nasser Rezzoug, Jean-Marc Salotti.

The biomechanical parameters of an operator are used as fundamental elements to describe static and dynamic properties of the body and its muscles. One of the mostly adopted parameter set is the one described by Hill. However, its parameters are numerous and difficult to retrieve in vivo: there is a need to find new methods to reduce their number and to obtain them in vivo. There are previous studies based on musculoskeletal model parameters reduction, using geometric properties of the model as well as parameter approximations using polynomials or artificial intelligence showing promising results. These are a starting point for the PhD thesis of Gautier Laisné (started in October 2021) whose goal is to derive and predict fundamental biomechanical parameters by studying the modelling of force capacities, known as the force polytope.

### 8.2.3 Set based representation of human arm motion workspace

Participants: Gianmarco Panzetta, David Daney, Nasser Rezzoug.

This work relates to motor variability assessment for fatigue detection, expertise modeling and musculoskeletal disorders reduction within the framework of the MOVER project. Due to the redundancy of the upper-limb, the motor variability can be expressed in a subspace of the human's workspace which is not constrained by the task. To characterize this redundant workspace for an operator's upper limb, we use interval analysis techniques, taking into account plausible biomechanical constraints, uncertainties on the upper limb geometry and the morphological variability of human operators. Basically, we propose modeling and filtering algorithms adapted to the equations representing the kinematic chain and to the mathematical properties of interval analysis.

By merging the experimental results of the MOVER project (see 8.3.5) and the proposed workspace modeling, the objective is then to relativize the measured motor variability with respect to the available workspace and the operator's capabilities. This will allow the development of new metrics of motor variability to characterize the adaptation of motor strategies due to expertise, fatigue and task constraints.

### 8.2.4 Development of a framework for the exploitation of human movement data

Participants: Erwann Landais, Nasser Rezzoug, Vincent Padois, Lucas Joseph.

A number of Auctus team projects could benefit from tools modeling the upper body of a human being to obtain precise data on the movement of individuals.

In this project, a complete framework has been developed. It first recovers the position of I.R markers placed on a human. An auto-labelling algorithm is used to get the correspondences between the I.R markers and the anatomical markers defined on an Opensim model. Then, not only the positions of the anatomical markers of the Opensim model can be obtained over time, but also the posture of the subjects thanks to the Inverse Kinematics algorithm proposed by Opensim. Finally, a filtering method is used to obtain velocity and acceleration of the subject's body parts from their position over time.

Although this framework is fully functional, the labeling algorithm presents a lot of difficulties to obtain the correspondences between real markers and markers of the model at each instant. However, this framework has been used to obtain interesting results (and exploitable by software classically used in robotics) on the movements of subjects over time.

### 8.2.5 Human motion decomposition

Participants: Jessica Colombel, David Daney.

The aim of this work is to find ways of representing human movement in order to extract meaningful physical and cognitive information. After the realization of a state of the art on human movement, several methods are compared: principal component analysis (PCA), Fourier series decomposition and inverse optimal control. These methods are used on a signal comprising all the angles of a walking human being. PCA makes it possible to understand the correlations between the different angles during the trajectory. Fourier series decomposition methods are used for a harmonic analysis of the signal. Finally, inverse optimal control sets up a modeling of the engine control to highlight qualitative properties characteristic of the whole motion. An in-depth study of the inverse optimal control method has been carried out in order to better understand its properties and to develop easier methods of resolution. However, in the context of these methods it is necessary to better pay attention to the reliability of the results. We wrote a paper on the Reliability of Inverse Optimal Control 22. This paper proposes an approach based on the evaluation of Karush-Kuhn-Tucker conditions relying on a complete analysis with Singular Value Decomposition. With respect to a ground truth, our simulations illustrate how the proposed analysis guarantees the reliability of the resolution. After introducing a clear methodology, the properties of the matrices are studied with different noise levels and different experimental model and conditions. We show how to implement the method step by step by explaining the numerical difficulties encountered during the resolution and thus how to make the results of the IOC problem reliable. To go further, we continued this work on the use of the inverse optimal control with a formalization of the data in polynomial form.

### 8.2.6 Human motor caracterization from hand movement tracing

Participants: Nasser Rezzoug.

In collaboration with the Research Groups in Intelligent Machines, (National School of Engineers of Sfax ENIS, University Sfax, Tunisia), we seek to apply modeling trajectory formalisms to identify the state of a subject or operator from hand movement tracing (writing, drawing or hand trajectory). Nowadays, motion capture (kinect, graphical tablets e.g.) associated with appropriate signal processing allow an objective analysis of these trajectories, 1) for documenting the effect of ageing and highlighting the neural, motor and sensory effects, 2) to develop new clinical tests for neurodegenerative disease early detection, 3) for more engineering-oriented applications such as signature automatic recognition, authentication for forensics, 4) to assess the state of an operator during hand movement execution (fatigue). The proposed model (the beta elliptic model) adequacy according to age and gender has been validated for detecting changes in hand movement during ellipse drawing on a database of 99 subjects with age ranging from 19 to 85 years.

Related publications:

## 8.3 Physic Human Robot Interaction

### 8.3.1 Graphical modelling framework for physical human-robot interaction scenarios

Participants: Antun Skuric, Vincent Padois, David Daney.

As the collaborative robotics grows momentum so does the number of potential scenarios of physical human-robot interaction. However, a proper model of the physics of their interaction is necessary in order to be able to truly exploit all the robots capabilities and provide, at the right place and at right time, the appropriate assistance to the human operator.

In this research therefore, we focused on the development of a generic modelling scheme for physical interaction scenarios. The proposed framework consists of blocks representing the dynamical systems such as the humans, robots, tools, objects and the environment. Furthermore, the dynamical systems (blocks) can exchange forces and wrenches with other blocks as well as have interaction constraints. This graphical framework enables relatively simple and intuitive characterization of the collaboration scenarios and can be used to determine the structure of the physical model.

As the future work this framework could potentially be used to automatically generate the physical equations of the system, as well as to choose the appropriate robot assistance strategy and the control algorithm.

### 8.3.2 Common wrench capability evaluation of a human-robot collaborative system

Participants: Antun Skuric, Vincent Padois, David Daney, Nasser Rezzoug.

Wrench capacity metrics are well known tools for the evaluation and the analysis of the performance of robotic manipulators. Arguably, the most widely used approximative capacity metrics are manipulability ellipsoids but the true capabilities have the form of polytopes. Ellipsoids and polytopes both express the robot's ability to resist and apply forces and torques in arbitrary directions.

Even though these metrics have been developed for robotics systems, they have been used for human upper limb (arm) analysis as well 9, in particular for rehabilitation and sports. Recently, with the development of collaborative robotics these metrics have been extended to evaluation of the common wrench capacity of the collaborative systems consisting of multiple robotic manipulators 15.

Therefore this research proposed the extension of these techniques to calculate the common wrench capacity of the human-robot collaboration system in the form of the wrench polytopes.

Related publication: 16, 9

### 8.3.3 Design of a analysis methodology of a manual task for the design of a collaborative robotics assistance adaptable to the level of expertise of an industrial operator

Participants: Nassim Benhabib, Vincent Padois, David Daney.

As a follow-up to the work in the Woobot project, an approach to quantify manual expertise has been proposed. The objective is to use the notion of expertise to improve the design of collaborative robots, both from a mechanical and control point of view. After a study of the existing literature and multiple exchanges with highly skilled manual operators, a definition of manual expertise has been proposed. Based on this definition, quantitative evaluation criteria for three important dimensions of manual task performance have been established: safety, discomfort and performance. Via the distribution of these criteria, manual expertise has been evaluated in experiments based on a physically realistic mock-up of a woodmilling task. This mock-up integrates a collaborative robot used both to reproduce the forces of the cutting tool and to provide measurements of the movement of the wood piece. This experimental device has been used in a protocol to analyse the manual expertise of three groups of 10 apprentice carpenters at different training levels. This protocol confirms that the proposed approach allows to observe and analyse the manipulation strategy developed by an operator during his apprenticeship and to correlate the experience of a craftsman with the distribution of the three scores of the established criteria.

Finally, a controller based on an LQP optimisation technique is proposed and implemented in a collaborative robotics assistant. The developed assistance is designed to adapt to the user's manual expertise via the previously calculated safety-perfomance-comfort evaluation scores. The objective of the latter is to improve these scores in proportion to the user's needs, the lower the score of a dimension, the more significant the robot's action on it. The contribution of the assistance is analysed using the previously presented model. The protocol consists of comparing the safety-performance-comfort evaluation scores on three groups of ten apprentices at different levels of training without and with assistance. This protocol confirms that the proposed approach can have a positive effect on the selected evaluation scores.

### 8.3.4 Collaborative robot assistance for the formulation of new solvents process

Participants: Erwann Landais, Nasser Rezzoug, Vincent Padois.

The preservation of the environment and human health leads to rethink the use of solvents in the chemical industry. New strategies for the production of chemical reactions using less toxic solvents are envisaged to meet these challenges. However, it is necessary to characterize the properties of solvents that can be used in green chemistry, including descriptors of empirical solvency that are essential for predicting mass phenomena.

Experimental determination of these properties is tedious, repetitive and sometimes dangerous. The use of collaborative robotics to assist operators in their solvent formulation operations is therefore being considered by Solvay. In this context, the objective of a thesis (began in Sept. 2021) is to develop a robotic approach for the identification and modeling of the manual and cognitive expertise of the operator during repetitive solvent characterization tasks.

The envisaged methodology can be decomposed into three steps: 1) the modeling and identification of the operator's manipulative abilities based on motion primitives, 2) modeling and identification of the perceptive elements of the characterization of the solvency, 3) the synthesis of the expert gesture for solvation.

### 8.3.5 Study of Motor Variability

Participants: Raphael Bousigues, Vincent Padois, David Daney, Nasser Rezzoug.

In the context of reducing the risk of musculoskeletal disorders (MSDs) in industry, the question of the motor variability (MV) of industrial operators must be raised. Indeed, MV could promote the distribution of effort and fatigue in the body through variations in motor strategies during movement. By optimizing this MV, it could be possible to spare the muscles, joints and skeleton of operators. A project, conducted through the thesis of Raphaël Bousigues in collaboration with Auctus, Larsen Inria NGE and INRS, aims to verify a set of hypotheses, such as, first of all, the fundamental existence of the VM. The reproduction of a repetitive manual task with the same operator allows to observe the kinematics of his arm and to quantify the presence and evolution of motor variability through the different repetitions of the task.

This year, an experimental set-up has been realized to simulate in a tangible way a repetitive task, and constrained by a robot, through a simple wire loop game. Particular attention has been given to design a controller based on an LQP using ROS. This pair controller allows the operator to move freely along and around the trajectory but constrains some degrees of freedom in orientation. The framework proposed by the team (8.4.1) allows a simple implementation on our robots.

### 8.3.6 Surgeon assistant work characterization

Participants: Julien Passama, Vincent Padois, Jean-Marc Salotti.

In the framework of our collaboration with Akka, a PhD work, funded by a CIFRE contract, started in october 2021, with the objective of determining the best options to use a collaborative robot to improve the interactions between the surgeon and his assistant during a surgical operation. The first study focussed on the instrumentalist/operating aid tasks such as tool grasping and tool passing to surgeon in order to decrease the occupied space around the patient, and decrease long and hard postures.

## 8.4 Robotics and control

### 8.4.1 Control architecture

Constant amelioration on the already available control architectures developed by the team (torque_qp and velocity_qp) the previous years have been conducted. The ROS packages have been enhanced with the library pinocchio, a state of the art robot modeling software jointly developped by the Gepetto team at LAAS and the Willow team at INRIA. Efforts have been placed on the improvement of trajectories and will lead to a new trajectory generation interface for the next year. New sensors using RGB cameras and QR codes have been integrated to the control architecture. Finally new features were added to fulfill some requests made either by the team or outside partners such as being able to constrain the robot joint jerk or access to specific parts of the robot model.

### 8.4.2 Instrumented glove box

Participants: Lucas Joseph, Vincent Padois, David Daney.

In chemistry, glove boxes are infrastructures designed to prepare chemical solutions in a controlled environment. The air pressure, quantity of oxygen, etc. can be monitored to preserve the chemical elements involved in the process. The space inside these boxes is constrained and manipulation of objects is not always practicle. The long term objective of this project is to put a robot inside a glove box to assist the technician. The first step is to build a demonstrator in a simplified environment. Three tasks are considered:

• The robot can bring tools/vials to the technician.
• The robot can act has a 3rd arm to hold tools/vials for the technician.
• The robot can bring the preparation to a device (stirrer, scale...).

Beyond the application itself, this project in collaboration with Solvay and the CNRS LOF provides a complex applicative context which raises fondamental issues regarding:

• safe robot control in cluttered environment;
• human-robot interactions;
• scene perception.

An initial demonstrator has been built and will allow the team to perform experiments to explore and validate new control and control architecture paradigms.

### 8.4.3 Dynamic synthesis and analysis of the shared workspace for safety in collaborative robotics

Participants: Nicolas Torres, Lucas Joseph, Vincent Padois, David Daney, Arthur Esquerre-Pourtère.

An industrial collaboration was initiated between INRIA and Stellantis in 2020 through a CIFRE thesis.

Collaborative robotics tries to answer the safety challenges arising from increasingly shared workspaces between humans and robots. Robotics safety standards impose artificial limits to performance and productivity, like the classic restrained static security zones and forced interruptions when operators get close. The objective of this project is to achieve a fluid human-robot collaboration through a dynamic representation of the robot capabilities in a given workspace and a control framework exploiting this representation.

After exploring constrained optimization for the formulation of reactive control problems during the first 6 months of the PhD, the focus has been placed this year on model predictive control formulations. MPC is seen as a step towards locally optimal control laws making use at best of the robot capabilities. In order to obtain a linear MPC formulation, compatible, from a computational point of view, with reactive control approaches, a linear formulation of motions in $se\left(3\right)$ is being explored. A publication is in preparation on this specific topic.

In this project, we have also explored through a 6 months internship the possibility to learn a braking control policy. The goal of this internship is line with the main focus of the project: exploit fully the robot actuation capabilities rather than using constant deceleration capabilities. While this type of approximation renders the planning problem simpler to solve, it does not reflect the true capabilities of the robot and leads to mostly suboptimal robot behaviours. The intern has explored the use of deep neural networks based on a trapezoidal acceleration profile with constant maximal deceleration to learn a control policy delta. The obtained results are encouraging. A publication is in preparation.

### 8.4.4 Impact aware robot control

Within the framework of a collaboration with A. Saccon (Associate professor in the department of Mechanical Engineering and the Group of Dynamics and Control) at TU Eindhoven, we have been working on the modeling of robots in impact with their environment. This fundamental modeling problem relates both to the efficient control of robots evolving in dynamic environments as well as the safety of robots physically interacting with human beings. Initial experimental results have been published in 13 and confirmed within the framework of the M. Sc. of Wouter Weekers. The work has then been pursued within the framework of the M. Sc. of Jelle Bevers on the control aspects, especially impact robustness at the control level. This work is now being pursued in collaboration with Franka Emika, one of the world leaders in collaborative robotics.

# 9 Bilateral contracts and grants with industry

## 9.1 Orange

Participants: Nicolas Simonazzi, Jean-Marc Salotti.

In 2019, a contract was signed with Orange for a PhD on emotions detection in chatbot interactions. The PhD student is Nicolas Simonazzi under the supervision of Jean-Marc Salotti. The objective is analyzing and identifying emotions during interactions with a chatbot. For the company, an important issue is indeed to be able to adapt the response of the chatbot on the website of the customer service. The study was further on focused on the use of smartphones and the identification of emotions based on accelerometers and move patterns. The associated grant was 10 kEuros per year during the 3-years period of the PhD.

Project in a nutshell:

• Consortium: AUCTUS@Ims, Orange
• Funding: Orange + ANRT (CIFRE), 30 kE
• Duration: 2018–2021 (PhD passed in nov. 2021)

## 9.2 Airbus

Participants: David Daney, Vincent Padois, Antun Skuric.

The collaboration aims to design a constellation of mini-satellites and one of the challenges is to rethink their production, in particular through robotic assistance of operators. In this project, we have developed a coupled model of human-robot physical capabilities.

see 10.3

Project in a nutshell:

• Consortium : AUCTUS@Inria, Airbus
• Funding : BPI
• Duration : 2020 – 2023

## 9.3 Stellantis

Participants: David Daney, Vincent Padois, Nicolas Torres.

The objective of this project is to work on the required principles to avoid the classic restrained static security zones, synthesizing a dynamic representation of the shared workspace to take advantage of state of the art control laws, allowing a fluid collaboration between human-robot. This dynamic synthesis requires knowledge of the robots state (geometric, cinematic and cognitive, such as fatigue, expertise and situation awareness), its tasks, capacities, state of humans that surround it and their tasks. Furthermore, it needs to achieve a formal and provable online algorithm that correctly estimates the state of the human and guarantee a safe shared workspace tackling ambitious scientific questions poorly addressed in literature.

Project in a nutshell:

• Consortium: AUCTUS@Inria, PSA Automobiles
• Funding: PSA Automobiles, ANRT (CIFRE)
• Duration: 2020–2023

## 9.4 Akka

Participants: Vincent Padois, Jean-Marc Salotti, Margot Vulliez, Julien Passama, David Daney.

In 2019-2022, we have established a recurrent collaboration with AKKA to transfer our know-how on collaborative robotics design. Two partnerships have been established. The first one is a bilateral contract (Portage) which allowed the user-centered design of a semi-autonomous vehicle for pulling heavy loads. In coordination with Akka’s engineers, we used methodologies from the field of Human Factors to highlight the psychological factors leading to the occurrence of human errors. We then deduced interfaces and interaction modes and tested and validated our solutions. The second collaboration started in September 2021 through the supervision of a PhD (CIFRE) on the design of a cobotic solution for the assistance of a surgeon in an operating room.

• Portage Project - see 10.4
• A contract is currently signed with Akka for a PhD aiming at determining the best option for assisting the assistant of the surgeon during a surgical operation, using a collaborative robot. Project in a nutshell:
• Company: Akka
• Funding: Akka and ANRT (CIFRE)
• Duration: 2021–2024
• PhD Student: Julien Passama
• Researchers involved: Jean-Marc Salotti, Vincent Padois, with the participation of Margot Vulliez (PPrime)

## 9.5 Solvay

Participants: Vincent Padois, Nasser Rezzoug, Lucas Joseph, David Daney, Erwan Landais.

Since 2020, we have developed a long term collaboration with the chemical company Solvay in order to help them in the digitalization and robotization of their productions. Our interlocutors are researchers of their Laboratory Of the Future (LOF) on the theme of collaborative robotics, seen as an important way to assist their operators and secure their potentially dangerous actions. The first objective is to develop a cobotic solution to follow an operators’ work step by step by proposing an assistance available at the user’s request in a constrained environment. This project has led to a bilateral contract and to the participation in the ANR Pacbot. In addition, in 2021, we are participating in the ”Miels project” in order to cobotize a handling task for the mixing of chemicals that requires human gestural expertise. The difficulty is first to learn from the human and then to synthesize this expertise.

• Miels Project - see 10.4
• Glovebox project Technicians at LOF handle chemicals elements that needs to be contained in specific environment. To that extent they use glove boxes. The space inside these boxes is constrained and manipulation of objects is not always practicle. To that extent, they want to introduce robots inside the glove boxes to assist the lab technician. A postdoctoral student, Lucas Joseph, has been recruited to build a demonstrator of such technology.

Project in a nutshell:

• Consortium: AUCTUS, Solvay, CNRS
• Funding: 1 year postdoc grant
• Duration: 2021–2022

# 10 Partnerships and cooperations

## 10.1 International initiatives

Participants: Jean-Marc Salotti.

Project in a nutshell:

• Consortium: AUCTUS@ims, OEWF, Austria (leader)
• Duration: 2021
• People involved: Jean-Marc Salotti

## 10.2 European initiatives

HARRY2

The objective of the HARRY2 project is to attain more advanced workspace sharing capabilities through fully exploiting the collaborative possibilities defined by ISO TS 15066. This is achieved by:

• Developing PLC software and motion controllers using robot-agnostic industrially-rated components to ease and standardize the development of safe robotic applications with workspace sharing.
• Integrating state-of-the-art energy-based control algorithms using these industrial hardware components, so that safety is no longer treated as an exception but considered as a constraint when computing the control solution in real-time.
• Enabling the use of high-level and intuitive teaching interfaces reducing robot programming time and difficulty.
• Developing a systematic and practical methodology for quantitative safety evaluation.

Project in a nutshell:

• Consortium: AUCTUS@Inria, RoBioSS@PPRIME, FuzzyLogicRobotics
• Funding: H2020 COVR, Région Nouvelle-Aquitaine
• Duration: 2019–2021
• People involved: Lucas Joseph, Vincent Padois
• Dissemination: Video, Harry2@COVR overview (pdf), Twitter

## 10.3 National initiatives

### 10.3.1 BPI LiChIE

Participants: David Daney, Vincent Padois, Antun Skuric.

The LiChIE project (funded by BPI) aims to design a constellation of mini-satellites for optical Earth observation. Among many other topics, this requires to rethink the way sattelites are being produced in order to ease this highly complex process. There is actually an unprecedented economical and societal demand for robots that can be used both as advanced and easily programmable tools for automatizing complex industrial operations in contexts where human expertise is a key factor to success and as assistive devices for alleviating the physical and cognitive stress induced by such industrial task. Unfortunately, the discrepancy between the expectations related to idealized versions of such systems and the actual abilities of existing so-called collaborative robots is large. Beyond the limitations of existing systems, especially from a safety point of view, there are very few methodological tools that can actually be used to quantify physical and cognitive stress. There is also a lack of formal approaches that can be used to quantify the contribution of collaborative robots to the realization of industrial tasks by expert operators. Of course, in the state-of-the-art, existing works in that domain do consider some aspects of the current state of the operator in order to propose an appropriate robot behaviour. One of their conceptual limitations is to consider an a priori defined human-robot collaboration scenario where the expertise of the human operator is of importance but limited to a single operation. The consideration of larger varieties of tasks is rarely considered and, when it is, only a strict separation of the tasks to be achieved by each member of the human-robot dyad is considered. In this project, we propose to develop a coupled model of human-robot physical abilities that does not make any a priori with respect to the type of assistance. This requires to develop a parameterisable generic model of the potential physical link and implied constraints between the human operator and the robot. This model should allow to describe the task to be achieved by the human alone or using a collaborative robot through different interaction modalities. Online simulation of these scenarios coupled with ergonomic and performance indicators should both allow for the discrete choice of the right assistance mode given the task currently being achieved as well as for the continuous modulation of the robot behaviour.

Project in a nutshell:

• Consortium : AUCTUS@Inria, Airbus, Erems, iXblue, TEAMS@Inria, Onera
• Funding : BPI
• Duration : 2020 – 2023
• Researchers involved : Antun Skuric (PhD Student), Vincent Padois (thesis advisor) and David Daney (thesis advisor)

### 10.3.2 ANR PACBOT

Participants: David Daney, Jean-Marc Salotti, Benjamin Camblor.

The general objective of the project is to design a semi-autonomous cobotic system for assistance, able to choose, synchronize and coordinate tasks distributed between humans and robots by adapting to different types of variability in professional gestures, all by anticipating dangerous situations. The orchestration of tasks between a man and a robot is difficult because it must answer the question of the distribution of roles within the couple according to physical and decision-making skills and constraints as well as the consequences of their interactions. However, we cannot put the two actors at the same level: the robot has to adapt its actions to the work of an operator and, more precisely, to its motor and cognitive strategies that materialize through the quantifiable variability of professional actions. On the other hand, the very interest of the robot is to assist the operator in his phases of fragility while preserving his physical and mental integrity, in particular considering that human error is inherent in operator action. These considerations are, for Pacbot, the conditions necessary for the joint achievement of efficient work.

Partners of the ANR project:

• Auctus (Leader), David Daney (principal investigator)
• Laboratoire Informatique de Grenoble
• Laboratoire Interuniversitaire de Psychologie (Lyon)

General information:

• ANR Funding for Auctus: 246,240 Euros
• Duration: 3 years
• Start: January 2021

### 10.3.3 JCJC Margot Vulliez

Auctus is accompanying the developments of the JCJC ANR grant obtained by Margot Vulliez in the RoBioSS team of the PPRIME lab in Poitiers. More particularly, this project aims at:

• study key features of HR perception-action links and to identify multisensory integration processes involved in HR interaction. This human-based approach will constitute the baseline of later developments and shape a shared-autonomy scheme,
• developping a shared perception between the different actors, according to their sensorial data and involvement in the task. This shared perception will be based on a multimodal feedback mixture conveying information about the task, the environment, and the collaborators,
• merging HR commands into a joint action toward the task goal. The human inputs will first be used to infer the operator intent and adapt the robot behavior. Then, the shared action will combine robot skills and human commands into a unified and consistent control objective.

In this work, Auctus brings its expertise in control architectures for collaborative robots.

## 10.4 Regional initiatives

### 10.4.1 Miels

Participants: Vincent Padois, Nasser Landais, David Daney, Erwann Landais.

The Miels project is co-financed by Solvay (cf. 9.5).

The main objective of the MIELS project is to develop innovative strategies to characterize and develop neoteric, non-toxic solvents through strategies that will enable to grasp the enormous quantity of required experimental tests all in insuring an absolute safety of the manipulator. For this purpose, we intend to work on two complementary routes, the development of solvent characterization methodologies and the integration of a cobotic approach in solvent handling and evaluation, with the ambition of merging these developments at the end of the project in order to draw as much synergy as possible. This project is built around teams with complementary competencies to achieve these objectives. UMR LOF and Solvay LOF have great experience acquired over several years of research the fields of solvent evaluation and robotics, whereas Auctus INRIA team has a strong expertise in collaborative robotics. By combining our competencies and expertise, the Miels project aims to merge all these fields in order to expand 1) the fundamental study of solvents and their characterization techniques including theoretical techniques, in particular for green solvents and 2) the development of the use of cobotics, in collaboration with Auctus INRIA team, for increasing the efficacy and safety of laboratory workers in industry, in particular those working on characterization of solvents.

Project in a nutshell:

• Consortium : AUCTUS@Inria, Solvay, LOF
• Funding : Région Nouvelle Aquitaine, Solvay
• Duration : 2020 – 2024

### 10.4.2 Portage

Participants: Jean-Marc Salotti, David Daney.

The Portage project is co-financed by Akka (cf. 9.4).

The global objective of this project is to develop a semi-autonomous carrier dedicated to the transport of heavy structures in industrial factories. The Auctus team has been assigned the role of task analysis and human systems interactions analysis in order to determine the best interface, to improve ergonomics, to reduce risks and to account for acceptability. A postdoctoral student, Charles Fage, has been recruited for the first year of the study (10/2019-10/2020). A 2-years contract (2019-2021) has been signed with AKKA Technologies as part of a consortium, which included two other companies, IIDRE and Ez-Wheel, and another research team from IMS laboratory. The total amount of the grant from Akka to Auctus is 110 kEuros.

Project in a nutshell:

• Consortium: AUCTUS@Ims, AKKA, EZ-WHEEL, IIDRE, IMS (productique team)
• Funding: Akka (Auctus is subcontractor)
• Duration: 2019–2021

### 10.4.3 Woobot

Participants: Vincent Padois, David Daney, Nassim Benhabib.

The main goal of the work is to develop a design methodology (control and physical architecture) of collaborative robotic systems dedicated to assisting and securing operator expert gestures in the context of industrial applications involving strong physical interaction between a human, a tool and a manipulated object. The collaborative system should improve ergonomics and safety while maintaining the level of industrial performance and preserving the user’s know-how. The application context is the carpentry, in particular wood shaping.

This project is conducted by Nassim Benhabib, David Daney and Vincent Padois. With the collaboration of BTP CFA de la Gironde. This allows to provide realistic information of the task studied and to collect the feedback from the craftsmen.

Project in a nutshell:

• Consortium: AUCTUS@Inria, BTP-CFA-Gironde
• Funding: Région Nouvelle-Aquitaine, Inria
• Duration: 2019–2022

# 11 Dissemination

Participants: All Members.

## 11.1 Promoting scientific activities

### 11.1.1 Scientific events: selection

#### Member of the conference program committees

• Nasser Rezzoug: Coordinator with Bruno Watier (LAAS CNRS) of the theme ”Biomechanics of human-system interaction, ergonomics and robotics” and scientific Program Committee member at the 46th congress of the "Societé de Biomécanique".
• David Daney and Vincent Padois were scientific Program Committee member at 46th Congresses of the Society of Biomechanics 2021.

#### Associate Editor

• Vincent Padois is associate editor for the IEEE IROS and ICRA 2021 conferences.

#### Reviewer

• Benjamin Camblor: IEEE HRI
• Jessica Colombel: IEEE ICRA
• David Daney: IEEE ICRA, IEEE IROS, 46th Congress of the "Société de Biomécanique"
• Lucas Joseph: IEEE IROS and ICRA
• Vincent Padois: IEEE IROS, ICRA, 46th Congress of the "Société de Biomécanique"
• Nasser Rezzoug: IEEE NER, 46th Congress of the "Société de Biomécanique"
• Antun Skuric: IEEE IROS

### 11.1.2 Journal

#### Reviewer - reviewing activities

• Nasser Rezzoug: BMC Musculoskeletal Disorders, Frontiers in Rehabilitation Sciences, Sensors, Applied Sciences, Ijerph, Computer Methods and Programs in Biomedicine, Heliyon.
• Jean-Marc Salotti: Journal of Intelligent Manufacturing, Acta Astronautica, Neural Computing and Applications, Frontiers in Astronomy and Space Sciences.
• David Daney: Mechanism and Machine Theory, IEEE Transactions on Robotics.
• Vincent Padois: IEEE Transactions on Robotics, IEEE/RAS Robotics and Automation Letters, Robotics and Computer Integrated Manufacturing.

### 11.1.3 Invited talks

• Vincent Padois. Robot Control A Personal View on Some Limits and Perspectives. Invited talk at the "1st workshop of the Regional Research Networ in Robotics". Bidart, France, 2021.
• Antun Skuric. Efficient calculation of human wrench capacity based on human musculoskeletal models Application in collaborative robot control. Invited Talk at GDR robotique GT1-GT6: Session "Exosquelettes pour l’assistance physique : quelles solutions optimales ?", Paris, 2021.
• David Daney. Understanding human behavior for robotic interaction. Invited Talk at GdR Robotique GT8/GT5, October, Paris, 2021.
• David Daney, Vincent Padois, Lucas Joseph and Antun Skuric gave scientific talks at the weekly seminars of the R4 network.
• Antun Skuric: "A coupled view of the physical abilities of human-robot dyad for the online quantitative evaluation of assistance needs" – Airbus Defence and Space, Oct 2021, Toulouse, France
• David Daney. Gestes et comportements humains en cobotique industrielle, Invited talk at NAIA.R, Forum Néo-Aquitain sur l'IA et la robotique, Dec. 2021.
• Benjamin Camblor: "Conscience de situation partagée pour la planification de tâches entre un humain et un robot" Presentation of Situation Awareness in Robotics for Pacbot project.
• Lucas Joseph, Jean-Marc Salotti and David Daney have presented our works in LOF senimars.

### 11.1.4 Leadership within the scientific community

• Nasser Rezzoug was elected Vice-President of the French-speaking Society of Biomechanics in ortober 2021.

### 11.1.5 Scientific expertise

• Nasser Rezzoug: Reviewer for the 2021 ANR generic project call, Technology for health.
• Vincent Padois: Expert for the ECOS NORD-MEXIQUE call.
• Vincent Padois: External auditor for IRT Jules Verne.

## 11.2 Teaching - Supervision - Juries

### 11.2.1 Teaching

• Master: Jean-Marc Salotti, Bases de l'intelligence artificielle (ENSC 2A), 40h eqTD.
• Master: Jean-Marc Salotti, Apprentissage Automatique (ENSC 2A), 25h eqTD.
• Master: Jean-Marc Salotti, Collaboration Homme Robot (ENSC 3A), 32h eqTD.
• Master: Jean-Marc Salotti, supervision of projects and internships (ENSC 1A, 2A, 3A), 100h eqTD.
• Master: David Daney, Interactions Hommes Robots, 2h eqTD, M2, Ecole Nationale Supérieure de Cognitique / Bordeaux INP, France.
• Master: David Daney, Mathématiques pour la robotique, 30h eqTD, M2, Enseirb/Ensc, Bordeaux INP, France.
• Master: Vincent Padois, Literature review - What, Why and How?, 20h éqTD, M2, Enseirb/Ensc, Bordeaux INP, France.
• Master: Vincent Padois, Student projects, 40h éqTD, M2, Enseirb/Ensc, Bordeaux INP, France.
• Master: Vincent Padois, Maintenance du futur - Cours introductif, 2h éqTD, M1, ENSPIMA, Bordeaux INP, France.
• Master: Vincent Padois, Maintenance du futur - Inrtroduction à la Robtique, 10h éqTD, M2, ENSPIMA, Bordeaux INP, France.
• Master: Vincent Padois, Introduction à la Robotique, 2h éqTD, M1, Université de Bordeaux, France.
• Master: Benjamin Camblor, Fonction cognitives en situation et Handicap, 9h éqTD, M1, Master de Sciences cognitives et Ergonomie, Université de Bordeaux, France.
• Licence : Benjamin Camblor, Facteurs Humains et Ergonomie, 22.5h éqTD, L2, Licence MIASHS, Université de Bordeaux, France.
• Master: Jessica Colombel, Traduction des langages (compilation), 18h éqTD, M1, TELECOM Nancy, France.
• Licence: Jessica Colombel, Web et Base de Données, 40h éqTD, L3, TELECOM Nancy, France.
• Master: Gautier Laisné, Interactions hommes-robots, 12h éqTD, M2, École Nationale Supérieure de Cognitique.
• Master: Gautier Laisné, Base de l'intelligence artificielle, 12h éqTD, M1, École Nationale Supérieure de Cognitique.
• Master: Lucas Joseph, Projet Robotique, 36.75h éqTD, M2, Campus d'Enseignement Superieur et de Formation Professionnelle, France.
• Licence: Nasser Rezzoug, Biomécanique, 60h éqTD, L1, UFR STAPS, Université de Toulon, France.
• Master: Nasser Rezzoug, Biomécanique, 25h éqTD, M1, UFR Sciences and Tech,Master ISC parcours Robotique et Objets Connectés, Université de Toulon, France.
• Master: Nasser Rezzoug, Applications biomédicales des objets connectés, 6h éqTD, M2, UFR Sciences and Tech, Master ISC parcours Robotique et Objets Connectés, Université de Toulon, France.
• Master: Antun Skuric, Mathématiques et Informatique (Intermédiaire), 30h éq TD, M1, ENSAM Bordeaux, Arts et Métiers ParisTech, France.
• Master: Antun Skuric, Mathématiques et Informatique (Avancé), 30h éq TD, M2, ENSAM Bordeaux, Arts et Métiers ParisTech, France.
• Master: Antun Skuric, Matériel Mobile Systèmes Embarqués, 16h éq TD, M2, ASPIC, Université de Bordeaux, France.

### 11.2.2 Supervision

• PhD in progress: Nassim Benhabib (Inria / Région NA – Woobot project), “Méthodologie de conception et de commande d’un système robotique collaboratif pour assister et sécuriser les gestes d’un opérateur”, November 2018 – , David Daney and Vincent Padois.
• PhD in progress: Benjamin Camblor, Situation Awareness in Collaborative Robotics, funding: ANR Pacbot, January 2020 - Jean-Marc Salotti.
• PhD in progress: Julien Passama, Assisting the assistant of the surgeon with collaborative robot, funding: Akka and ANRT, October 2020 - Jean-Marc Salotti
• PhD in progress: Nicolas Torres (Thèse ED SPI, Bordeaux / CIFRE PSA), “Synthesis and dynamic analysis of the shared workspace for safety in collaborative robotics”, April 2020 – Vincent Padois and David Daney.
• PhD in progress: Antun Skuric (Thèse ED SPI, Bordeaux / Financement projet Lichie Airbus), “A coupled view of the physical abilities of human-robot dyad for the online quantitative evaluation of assistance needs”, July 2020 – Vincent Padois and David Daney.
• PhD in progress: Raphaël Bousigues (Thèse ED SPI, Bordeaux / INRS-Inria), “Motor variability in human-robot collaboration”, December 2020 – Pauline Maurice, Vincent Padois, David Daney and Jonathan Savin.
• PhD in progress: Jessica Colombel (Inria), “Analyse du mouvement humain pour l’assistance à la personne”, February 2019 – , François Charpillet and David Daney (Inria Nancy Grand-Est).
• PhD in progress: Erwann Landais (MIELS project, Région NA), “Approche robotique pour la formulation de nouveaux solvants”,Sept 2021 –, Vincent Padois and Nasser Rezzoug.
• PhD in progress: Gautier Laisné (Inria), "Musculoskeletal models and data driven learning for personalized human force capacities evaluation", October 2021 -, Nasser Rezzoug and Jean-Marc Salotti.
• PhD passed: Nicolas Simonazzi (CIFRE Orange), “Analyse comportementale et détection des émotions dans le cadre de l’utilisation de chat-bots en ligne”, May 2018 – Dec 2021, Jean-Marc Salotti.
• MS Thesis in progress: Gianmarco Panzetta, " Human upper-limb workspace computation by interval analysis for ergonomics", Sept 2021 - Feb 2022, David Daney and Nasser Rezzoug.
• MS Thesis passed: Mathilde Harter, "Simulation des couples maximum d’actionnement du membre supérieur pour l’évaluation ergonomique de postes de travail", Feb 2021 - August 2021, Nasser Rezzoug and Jonathan Savin (INRS).
• Internship: Cléo Weber (1st year at ENSC), "cobotic and situation awareness", May 2021- July 2021, Jean-Marc Salotti, Nassim Benhabib and Benjamin Camblor.
• Internship: Lucas Bardisbanian (1st year at ENSC), "cobotic and situation awareness", May 2021- July 2021, Jean-Marc Salotti, Nassim Benhabib and Benjamin Camblor.
• Internship: Arthur Esquerre-Pourtère (M2 Androïde Sorbonne Université), "Learning an optimal breaking policy for robotic manipulators", Mar 2021- July 2021, Vincent Padois, Nicolas Torres.
• M. Sc. project: Wouter Weeker (TU Eindhoven), "Validation of Nonsmooth Impact Maps in Robot Impact Experiments", Graduated Apr. 2021, Vincent Padois, Alessandro Saccon.
• M. Sc. project: Jelle bevers (TU Eindhoven), "Robustness of QP Robot Control Against Constraint Violations", In progress, Vincent Padois, Alessandro Saccon.

### 11.2.3 Juries

• David Daney
• PhD: Wanda Zhao, reviewer, "Design of Robot End-Effector for Collaborave Robot Works", 02/12/2021, Docteur de l’École Centrale de Nantes, LS2N.
• CRCN : member of the "junior researcher" jury of Inria Rennes-Bretagne Altlantique Research Centre.
• PhD: Gomes Waldez, examiner, "Improving Ergonomics Through Physical Human-Robot Collaboration", 13/12/2021, Docteur de l'Université de Lorraine, Loria.
• PhD: Bolotnikova Anastasia, reviewer, "Frail human assistance by a humanoid robot using multi-contact planning and physical interaction", 25/03/2021, Docteure de l’Université de Montpellier, LIRMM.
• PhD: Randa Mallat, reviewer, "Toward an Affordable Multi-Modal Motion Capture System Framework for Human Kinematics and Kinetics Assessment", 01/2021, Docteure de l'Université Paris-Est Créteil, LISSI.
• Nasser Rezzoug
• PhD: Cyrille Grébonval, reviewer, "Automatisation des véhicules et nouveaux habitacles : postures de confort et risques associés en cas de choc", 17/12/2021, Université Claude-Bernard Lyon 1, Laboratoire de biomécanique et mécanique des chocs.
• PhD: Olfa Haj-Mahmoud, reviewer, "Monitoring de l'efficience gestuelle des opérateurs sur les postes de travail", 7/12/2021, Université Rennes 2, Laboratoire M2S.
• PhD: Alice Nicolai, reviewer, "Interpretable representations of human biosignals for individual longitudinal follow-up", 22/11/2021, Université de Paris, Centre Borelli.
• Jean-Marc Salotti
• PhD: Nicolas Simonazzi, supervisor, "Reconnaissance d’états émotionnels à partir des interactions avec un smartphone", 19/11/2021, Université de Bordeaux, IMS.

## 11.3 Popularization

### 11.3.1 Education

Vincent Padois (12), David Daney (2), Jessica Colombel (2) have several presentations in High Schools within the framework of the Chiche and "Fête de la Science" programs.

### 11.3.2 Interventions

• Jessica Colombel:
• Regional Final of "Ma Thèse en 180s";
• Scientifique mediation on "Ciné-débat à l'Est'ival des sciences";
• David Daney,
• ”Abolir les frontières entre science et fiction”, Festival Hypermondes, roundtable discussion, David Daney, Mérignac.

# 12 Scientific production

## 12.1 Major publications

• 1 inproceedingsN.Nassim Benhabib, V.Vincent Padois and D.David Daney. Securing Industrial Operators with Collaborative Robots: Simulation and Experimental Validation for a Carpentry task.ICRA 2020 - IEEE International Conference on Robotics and AutomationParis, FranceMay 2020
• 2 miscB.Benjamin Camblor, N.Nassim Benhabib, D.David Daney, V.Vincent Padois and J.-M.Jean-Marc Salotti. Task-Consistent Signaling Motions for Improved Understanding in Human-Robot Interaction and Workspace Sharing.January 2022
• 3 articleP.Pauline Maurice, V.Vincent Padois, Y.Yvan Measson and P.Philippe Bidaud. Human-oriented design of collaborative robots.International Journal of Industrial Ergonomics572017, 88--102
• 4 articleJ.Jonathan Savin, C.Clarisse Gaudez, M. A.Martine A. A Gilles, V.Vincent Padois and P.Philippe Bidaud. Evidence of movement variability patterns during a repetitive pointing task until exhaustion.Applied Ergonomics962021, 103464
• 5 miscA.Antun Skuric, V.Vincent Padois, N.Nasser Rezzoug and D.David Daney. On-line feasible wrench polytope evaluation based on human musculoskeletal models: an iterative convex hull method.October 2021

## 12.2 Publications of the year

### International journals

• 6 articleB.Benjamin Camblor, J.-M.Jean-Marc Salotti, C.Charles Fage and D.David Daney. Degraded situation awareness in a robotic workspace: accident report analysis.Theoretical Issues in Ergonomics ScienceFebruary 2021
• 7 articleA.Augustin Chevallier, S.Sylvain Pion and F.Frédéric Cazals. Improved polytope volume calculations based on Hamiltonian Monte Carlo with boundary reflections and sweet arithmetics.Journal of Computational Geometry2022
• 8 articleT.Thameur Dhieb, N.Nasser Rezzoug, H.Houcine Boubaker, M.Mounir Ben Ayed and A.Adel Alimi. Do individual characteristics influence the beta-elliptic modeling errors during ellipse drawing movements?Computer Methods in Biomechanics and Biomedical EngineeringSeptember 2021, 34
• 9 articleN.Nasser Rezzoug, V.Vincent Hernandez and P.Philippe Gorce. Upper-Limb Isometric Force Feasible Set: Evaluation of Joint Torque-Based Models.Biomechanics1April 2021, 102 - 117
• 10 articleJ.-M.Jean-Marc Salotti and J.Julien Doche. Preliminary study of rescue systems for Mars landing.Acta Astronautica188November 2021, 81-88
• 11 articleLauncher size optimization for a crewed Mars mission.Acta Astronautica191February 2022, 235-244
• 12 articleJ.Jonathan Savin, C.Clarisse Gaudez, M. A.Martine A. A Gilles, V.Vincent Padois and P.Philippe Bidaud. Evidence of movement variability patterns during a repetitive pointing task until exhaustion.Applied Ergonomics962021, 103464

### International peer-reviewed conferences

• 13 inproceedingsI.Ilias Aouaj, V.Vincent Padois and A.Alessandro Saccon. Predicting the Post-Impact Velocity of a Robotic Arm via Rigid Multibody Models: an Experimental Study.IEEE International Conference on Robotics and AutomationXi’an, ChinaMay 2021
• 14 inproceedingsC.Charles Fage and J.-M.Jean-Marc Salotti. Analyse de Risques en Robotique : Apports de la Conscience de Situation.SELF 2021 - 55ème congrès de la Société d'Ergonomie de Langue FrançaiseParis / Virtuelle, FranceJanuary 2021
• 15 inproceedingsOn-line force capability evaluation based on efficient polytope vertex search.ICRA 2021 - IEEE International Conference on Robotics and AutomationXi'an, ChinaMay 2021
• 16 inproceedingsA.Antun Skuric, N.Nasser Rezzoug, D.David Daney and V.Vincent Padois. Common wrench capability evaluation of a human-robot collaborative system.46th Congrès de la Société de BiomécaniqueSaint Etienne, FranceOctober 2021

### Conferences without proceedings

• 17 inproceedingsJ.-M.Jean-Marc Salotti and M.Michel Lamontagne. Phobos, Possible Refueling Station for a Single Stage to Orbit Vehicle.IAC 2021 - 72nd International Astronautical CongressDubaï, United Arab EmiratesOctober 2021
• 18 inproceedingsN.Nicolas Simonazzi, J.-M.Jean-Marc Salotti, M.Marie Morelle, C.Caroline Dubois and P.Philippe Le Goff. The Geneva Emotion Wheel Mobile Interface: an Instrument to Report Emotions on Android Devices.ERGO'IA 2021 - De l’Interaction Homme-Machine à la Relation Homme-Machine, comment concevoir des systèmes performants et éthiquesBidart, France2021

### Scientific book chapters

• 19 inbookN.Nicolas Simonazzi, J.-M.Jean-Marc Salotti, C.Caroline Dubois and D.Dominique Seminel. Emotion Detection Based on Smartphone Using User Memory Tasks and Videos.1253Advances in Intelligent Systems and Computing 1253, Human Interaction, Emerging Technologies and Future Applications III, proceedings of the 3rd International Conference on Human Interaction and Emerging Technologies: Future Applications (IHIET 2020), August27–29, 2020, Paris, FranceAISCSpringerAugust 2021, 244-249

## 12.3 Other

### Scientific popularization

• 25 inbookJ.-M.Jean-Marc Salotti, M.Michel Lamontagne, L.Laurent Gauthier, F.Florent Bednarek, C.Carl Vambert, O.Olivier Gourdon and R.Rémi Navarro. Chapter 15: Foundation.Mars City States, New Societies for a New WorldPolaris BooksApril 2021

## 12.4 Cited publications

• 26 bookJ.Jack Bernon, E.Evelyne Escriva and J. M.Jean Michel Schweitzer. Agir sur la prévention durable des TMS.Anact2011
• 27 bookP.Philippe Douillet. Agir sur Prévenir les risques psychosociaux.Anact2013
• 28 incollectionS.S. Haddadin and E.E. Croft. Physical Human-Robot Interaction.Handbook of RoboticsSpringer Verlag2016, 1835--1874
• 29 articleS.Sue Hignett and L.Lynn McAtamney. Rapid Entire Body Assessment (REBA).Applied Ergonomics312April 2000, 201--205
• 30 articleJ.Julien Jacquier-Bret, P.Philippe Gorce, G.Genaro Motti Lilian and N.Nadine Vigouroux. Biomechanical analysis of upper limb during the use of touch screen: motion strategies identification.Ergonomics603March 2017, 358--365
• 31 articleH.Herbert Jaeger. Using Conceptors to Manage Neural Long-Term Memories for Temporal Patterns.Journal of Machine Learning Research18132017, 1-43
• 32 inproceedingsL.L. Joseph, V.V. Padois and G.G. Morel. Towards X-ray medical imaging with robots in the open: safety without compromising performances.Proceedings of the IEEE International Conference on Robotics and AutomationBrisbane, AustraliaMay 2018, 6604--6610
• 33 articleO.Osmo Karhu, P.Pekka Kansi and I.Iikka Kuorinka. Correcting working postures in industry: A practical method for analysis.Applied Ergonomics84December 1977, 199--201
• 34 articleD.Dohyung Kee and W.Waldemar Karwowski. LUBA: an assessment technique for postural loading on the upper body based on joint motion discomfort and maximum holding time.Applied Ergonomics324August 2001, 357--366
• 35 phdthesisX.X. Lamy. Conception d'une Interface de Pilotage d'un Cobot.Université Pierre et Marie Curie - Paris VIMarch 2011
• 36 articleM.Mingxing Liu, Y.Yang Tan and V.Vincent Padois. Generalized hierarchical control.Autonomous Robots4012016, 17--31
• 37 articleS. E.Svend Erik Mathiassen. Diversity and variation in biomechanical exposure: What is it, and why would we like to know?Applied Ergonomics374Special Issue: Meeting Diversity in Ergonomics2006, 419-427
• 38 articleL.Lynn McAtamney and E. N.E Nigel Corlett. RULA: a survey method for the investigation of work-related upper limb disorders.Applied ergonomics2421993, 91--99
• 39 incollectionA.A. Meguenani, V.V. Padois, J.J. Da Silva, A.A. Hoarau and P.P. Bidaud. Energy-based control for safe Human-robot physical interactions.Springer Proceedings in Advanced Robotics - The 2016 International Symposium on Experimental RoboticsSpringer International Publishing AG2017,
• 40 inbookJ.-P.Jean-Pierre Merlet and D.D. Daney. Appropriate Design of Parallel Manipulators.Smart Devices and Machines for Advanced ManufacturingL.Lihui WangJ.Jeff XiLondonSpringer London2008, 1--25
• 41 articleD.D. Oetomo, D.D. Daney and J.J. Merlet. Design Strategy of Serial Manipulators With Certified Constraint Satisfaction.IEEE Transactions on Robotics251Feb 2009, 1-11
• 42 bookA.Adina Panchea. Inverse optimal control for redundant systems of biological motion.OrlÃ©ansDecember 2015,
• 43 articleL.Luka Peternel, T.Tadej Petrič, E.Erhan Oztop and J.Jan Babič. Teaching Robots to Cooperate with Humans in Dynamic Manipulation Tasks Based on Multi-modal Human-in-the-loop Approach.Autonomous Robots361-2January 2014, 123--136
• 44 articleL.Luka Peternel, N.Nikos Tsagarakis, D.Darwin Caldwell and A.Arash Ajoudani. Robot adaptation to human physical fatigue in human–robot co-manipulation.Autonomous Robots425June 2018, 1011--1021
• 45 inproceedingsJ.Jonathan Savin, M.Martine Gilles, C.Clarisse Gaudez, V.Vincent Padois and P.Philippe Bidaud. Movement Variability and Digital Human Models: Development of a Demonstrator Taking the Effects of Muscular Fatigue into Account.Advances in Applied Digital Human Modeling and SimulationChamSpringer International Publishing2017, 169--179
• 46 articleZ.Zhen Song, A.Antun Skuric and K.Kun Ji. A Recursive Watermark Method for Hard Real-Time Industrial Control System Cyber-Resilience Enhancement.IEEE Transactions on Automation Science and Engineering172©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.February 2020, 1030-1043
• 47 articleM.Michael Sonne, D. L.Dino L. Villalta and D. M.David M. Andrews. Development and evaluation of an office ergonomic risk checklist: ROSA - Rapid Office Strain Assessment.Applied Ergonomics431January 2012, 98--108
• 48 articleD.Divya Srinivasan and S. E.Svend Erik Mathiassen. Motor variability in occupational health and performance.Clinical Biomechanics27102012, 979-993
• 49 articleC.Carlos Viegas, D.David Daney, M.Mahmoud Tavakoli and A. T.Aníbal T. de Almeida. Performance analysis and design of parallel kinematic machines using interval analysis.Mechanism and Machine Theory1152017, 218-236
• 50 bookJ. R.John R. Wilson and S.Sarah Sharples. Evaluation of Human Work, Fourth Edition.Google-Books-ID: uXB3CAAAQBAJCRC PressApril 2015
1. 1In the ISO TS 15066 technical specification on collaborative robotics, human-robot physical interaction is allowed but perceived as a situation to be avoided.