- A5.1.1. Engineering of interactive systems
- A5.1.2. Evaluation of interactive systems
- A5.1.3. Haptic interfaces
- A5.1.5. Body-based interfaces
- A5.1.7. Multimodal interfaces
- A5.1.9. User and perceptual studies
- A5.4.2. Activity recognition
- A5.4.4. 3D and spatio-temporal reconstruction
- A5.4.5. Object tracking and motion analysis
- A5.4.8. Motion capture
- A5.5.1. Geometrical modeling
- A5.6.2. Augmented reality
- A5.10.1. Design
- A5.10.2. Perception
- A5.10.3. Planning
- A5.10.4. Robot control
- A5.10.5. Robot interaction (with the environment, humans, other robots)
- A5.10.8. Cognitive robotics and systems
- A6.2.5. Numerical Linear Algebra
- A6.2.6. Optimization
- A6.4.6. Optimal control
- A6.5.1. Solid mechanics
- A8.3. Geometry, Topology
- A9.5. Robotics
- A9.8. Reasoning
- B1.1.11. Plant Biology
- B1.2.2. Cognitive science
- B2.8. Sports, performance, motor skills
- B5.1. Factory of the future
- B5.2. Design and manufacturing
- B5.6. Robotic systems
- B9.5.5. Mechanics
- B9.6. Humanities
- B9.9. Ethics
1 Team members, visitors, external collaborators
- David Daney [Team leader, INRIA, Senior Researcher, HDR]
- Vincent Padois [INRIA, Senior Researcher, HDR]
- Margot Vulliez [INRIA, Researcher, from Oct 2022]
- Nasser Rezzoug [Toulon University, Associate Professor, until Sep 2022, in secondment with a 50% Teaching discharge, HDR]
- Jean-Marc Salotti [BORDEAUX INP, Professor, HDR]
- Vincent Fortineau [INRIA, from Nov 2022]
- Lucas Joseph [INRIA, from Aug 2022 until Nov 2022]
- Virgile Batto [LAAS CNRS, from Oct 2022 until Sep 2022, In collaboration with, ]
- Nassim Benhabib [INRIA, until Apr 2022]
- Alexis Boulay [FARM3 , CIFRE, from Jun 2022]
- Raphael Bousigues [INRIA, in collaboration EPI LARSEN, located in Nancy]
- Benjamin Camblor [INRIA]
- Jessica Colombel [INRIA, ATER, in collaboration EPI LARSEN, located in Nancy]
- Elio Jabbour [INRIA, from Oct 2022]
- Pierre Laguillaumie [UNIV POITIERS, teaching fellow]
- Gautier Laisne [INRIA]
- Erwann Landais [INRIA]
- Julien Passama [AKKA TECHNOLOGIES, CIFRE, until Sep 2022]
- Antun Skuric [INRIA]
- Nicolas Torres [GROUPE PSA, CIFRE]
- Alexis Boulay [BORDEAUX INP, Engineer, from Apr 2022 until May 2022]
- Guillaume De Mathelin De Papigny [INRIA, Engineer, in collaboration with Aerospline (industrial partner)]
- Sebastien Dignoire [INRIA, Engineer, from Oct 2022, in collaboration with Fuzzy Logic Robotics (industrial partner)]
- Lucas Joseph [INRIA, Engineer, from Dec 2022]
Interns and Apprentices
- Clemente Donoso [INRIA, Intern, from May 2022 until Aug 2022, International Mobility Program Inria-Chile]
- Simon Gervaises [INRIA, Intern, from May 2022 until Aug 2022]
- Gianmarco Panzetta [INRIA, until Feb 2022]
- Chrystel Plumejeau [INRIA, until Aug 2022]
- Nathalie Robin [INRIA, from Sep 2022]
- Nasser Rezzoug [Poitiers University, from Sep 2022]
- Margot Vulliez [Poitiers University, until Sep 2022]
2 Overall objectives
The project of the Auctus team is to design the collaborative robotics cells of the future.
The robotics community still tends to separate the cognitive (HRI) and physical (pHRI) aspects of human-robot interaction. One of the main challenges is to characterize the task as well as biomechanical, physiological and cognitive capabilities of humans in the form of physical constraints or objectives for the design of cobotized workstations. This design must be understood in a large sense: the choice of the robots' architecture (cobot, exoskeleton, human-robot interface, etc.), the dimensional design (human/robot workspace, trajectory calculation, etc.), the coupling mode (comanipulation, teleoperation, etc.) and the control strategy. The approach then requires the contributions of the human and social sciences to be considered in the same way as those of exact sciences. The topics considered are broad, ranging from cognitive sciences, ergonomics, human factors, biomechanics and robotics.
The first challenge is to evaluate the hardship at work, the well-being of the operators and, further upstream, their cognitive state which impacts their sensorimotor strategy while performing a task. In the industry, the ergonomic analysis of the task is carried out by an ergonomist based on direct but often ad hoc observations. However, the context is changing: the digitization of factories, through the installation of on-site sensors, allows longitudinal observation of machines and humans. The available information can thus allow us to rethink the way in which the evaluation of activities is carried out. Currently, an emerging subdomain, named ergonomic robotics, adapts available ergonomic criteria (RULA, REBA, etc.) to the evaluation of robotic cells. However, such criteria are related to the (quasi-static) posture of the operator, which limits the understanding of human motor strategies over a long period of time. Similarly, kinematic or biomechanical analyses may tend to see humans as a high-performance machine to be optimized. This can make sense for a top-level athlete, but repeating actions in the industry over a day, months or years of work means that a temporary change of posture, possibly poorly rated according to usual ergonomic criteria, can actually be a good long-term strategy. These questions directly link motor and cognitive aspects that can be reflected in particular strategies such as the fatigue or the expertise (manual and cognitive). This approach has not been widely explored in robotics while it could determine the right criteria to adapt the behavior of a cobot.
The second challenge is to define a methodology to link the analysis of the task and the induced human movements to the robot design. Several of our industrial projects have shown that there is a significant conceptual distance between the ergonomist, expert in task analysis and psychology, and the robotician, expert in mechanics, control and computer science, which makes it very difficult to analyze the needs and define the specifications of the technical solution. To fill these methodological gaps, it is necessary to better define the notion of tasks in the context of a human/robot coupling, on the basis of case studies. We also have to establish a typology of human/robot interaction by taking into account the different physical and cognitive constraints, in a very detailed fashion, and their potential psychological, organizational or ethical impacts.
The third challenge addresses the need to think about the control laws of collaborative robots in terms of human/robot coupling. The effectiveness of this coupling requires an ability to predict future human actions. This prediction should make the interaction more intuitive but also aims at optimizing the robot assistive behavior from the point of view of "slow" phenomena, such as fatigue. The major challenge is, therefore, to move from reactive to predictive control laws, by integrating a human prediction model, both in terms of movement strategies and decision strategies. Beyond the great computational complexity of predictive approaches, obtaining prediction models is an ambitious challenge. It is indeed necessary to learn models that are quite complex, due to the physical realities they can account for, and quite simple from a computational point of view.
3 Research program
3.1 Analysis and modeling of human behavior
3.1.1 Scientific Context
The purpose of this axis is to provide metrics to assess human behavior. Our human study specifically focuses on industrial operators. We assume the following working hypotheses: the operator's task and environmental conditions are known and circumscribed; the operator is trained in the task, production tools and safety instructions; the task is repeated with more or less frequent intervals. We aim at analyzing:
- the physical and cognitive fragility of operators in order to meet assistance needs;
- cognitive biases and physical constraints leading to a loss of the operator safety;
- ergonomics, performance and acceptance of the production tool.
In the industrial context, these questions are tackled through the fields of work ergonomics and cognitive sciences. Four main axes are typically addressed: physiological/biomechanical, cognitive, psychological and sociological studies. We particularly focus on the biomechanical, cognitive and psychological aspects, as described by the ANACT 27, 28. The aim is to translate these factors into metrics, optimality criteria or constraints which can be implemented in our methodologies to better analyze, design and control the collaborative robots.
A review of ergonomic workstation evaluation helps in positioning our desired contributions in robotics. Ergonomists evaluate the gesture through the observation of the workstations and, generally, through questionnaires. This requires long periods of field observation, followed by analyses based on ergonomic grids (e.g. RULA 44, REBA 36, LUBA 40, OWAS 39, ROSA 57,...). Until then, the use of more complex measurement systems was reserved for laboratories, particularly in biomechanical studies. The development of inexpensive sensors, such as IMUs (Inertial Measurement Units) or RGB-D cameras, makes it possible to consider a digitalized, and therefore objective, observation of the gesture, posture and more generally of human movement and at the same time collecting information on physiological states. Thanks to these sensors, which are more or less intrusive, observation systems can be permanently installed on production lines. This change of paradigm opens the door to longitudinal observations. It can be compared to the evolution of maintenance, which became predictive.
Ergonomic robotics has recently taken an interest in this new evaluation paradigm to adapt the robot behavior in order to reduce ergonomic risks. This ergonomic adaptation complements the conventional approaches that only consider the performance of the action produced by the human in interaction with the robot. However, ergonomic criteria are usually based on the principle that the comfort positions are distant from the human joint limits. The notation are compatible with an observation of the human operator through the eye of the ergonomist. In practice, such evaluations are inaccurate and subjective 60. Moreover, they only consider quasi-static human positions, without taking into account the evolution of the person's physical, physiological and psychological state. We aim at extending this approach to more reliable and comprehensive ergonomic metrics. The repetition of gestures, the solicitation of muscles and joints, are questions that must complete these analyses. A method used by ergonomists to limit biomechanical exposures is to increase variations in motor stress by rotating tasks 58. However, this type of extrinsic method is not always suitable in the industrial context 43 where we place our research efforts.
Through these human analyses, the Auctus team aims at revising the use of collaborative robots in the workplace to vary the operator's environment and encourage more appropriate motor strategies. This approach relies on biomechanical studies of the intrinsic variability of the motor system allowed by the joint redundancy of the human body. This motor variability refers to the alternation of postures, movements and muscle activity observed in the individual to perform a requested task 58. This variation leads to differences between the motor coordination used by each operator, and conveys the notion of motor strategy 37.
The cognitive dimension of ergonomics must also be addressed in our approach to reduce the mental workload and foster wellness of the worker. We believe that known sensorimotor strategies can be a physically quantifiable reflection of the operator's cognitive state. For example, human motion measurements can be used to predict fatigue 55 and, therefore, adjust the robot physical assistance. A key challenge here is to better analyze human manual expertise (dexterous and cognitive) to adapt the human-robot interaction. The expertise embodies the operators' decision-making process while perceiving, understanding, and anticipating their gestures to preserve their safety, comfort, and performance in the task. We particularly aim at adapting and refining known human cognitive models (multisensory perception 34, situation awareness 33) to infer the influence of the task context and environment on the operator behavior.
How can we observe, understand and quantify human sensorimotor and cognitive strategies to better design and control the behavior of the cobotic assistant?
When we study systems of equations (kinematic, static, dynamic, musculoskeletal, etc.) to model human behavior, several problems appear and explain our methodological choices:
- the large dimension of the problems to be considered, due to the human body complexity (eg. joint, muscle and placement redundancy);
- the variability of the parameters, for example, physiological (set of operators), geometric (set of possible trajectories, postures, and placements of the operator), and static parameters (set of forces that the operator must produce with respect to the task and context);
- the uncertainties in the measurement and the model approximations.
The idea is to start from a description of redundant workspaces (geometric, static, dynamic...). We use set-theory approaches, based on interval analysis 59, 49, which meet the variability requirement and cope with model and measurement uncertainties. Another advantage of such techniques is that they allow the results to be certified, which is essential to address safety issues. Some members of the team have already achieved success in mechanical design for performance certification and robot design 46. By extending these set-theory approaches to our problem, a mapping of ergonomic, efficient, and safe movements can be obtained, in which we project the operators' motor strategies. Biomechanical, ergonomic, and cognitive metrics can, then, be defined and evaluated to quantify the human behavior in specific work situations.
It is therefore necessary to:
- model human capabilities, both at the musculoskeletal and the perceptive/cognitive levels, to allow for global, yet detailed, analyses as well as efficient integration of such knowledge in the control of the collaborative robots.
- propose new ergonomic, biomechanical, robotic, and cognitive indices which will link different types of performances while taking into account the influence of fatigue, stress, level of expertise, etc.;
- divide the task and the gesture into homogeneous phases: this process is complex and depends on the type of studied index and the techniques being used. We are exploring several methods: inverse optimal control, learning methods, techniques from signal processing;
- develop interval extensions of the proposed indices. The indices are not necessarily the result of a direct model, and algorithms must be developed or adapted to compute them (calculation of manipulability, Uncontrolled Manifold, etc.);
- Aggregate proposals into a dedicated interval-analysis library for human behavior studies (use of and contribution to the existing ALIAS-Inria and the open source IBEX library).
The major contribution of the methodology is to embrace in the same model the measurement uncertainties (important for on-site use of measurement equipment), the variability of tasks and trajectories (proper to dexterous industrial operations), and the physiological characteristics of the operators (critical adaptability to every individual). The originality of the approach is to combine biomechanical, ergonomic and cognitive metrics with the usual performance indices to build a comprehensive and objective analysis of human behavior.
Other avenues of research are being explored, particularly around the inverse optimal control 50, to project human movements on the basis of the performance or ergonomic indices. Such a projection would offer new interpretations and enhance the analysis of human behaviors.
3.2 Operator / robot coupling
This research axis is at the frontier between humans and robots and focuses on optimal methods to couple together these two entities to perform joint activities. This raises questions which are directly related both to human models and abilities (axis 1) and robot control (axis 3). Two main concerns must be addressed to form an effective human-robot dyad.
3.2.1 Human-Robot interaction
The first step to couple the operator and cobot together at work is to provide interaction modalities through which the agents can communicate and coordinate. The interaction can be direct, where the robot and operator act together in the same shared environment, or the operator can remotely perform the task with the robot through a teleoperation system (which reflects the remote interaction and potentially corrects for punctual weaknesses). The level and type of human-robot interactions are chosen with respect to the task, the context, or other human factors. The challenge is, then, to predict the joint human-robot behavior and capabilities for each interaction situation and collaborative context.
The formal computation of joint human-robot capabilities can be given thanks to the models and evaluation indices built in the Axis 1. We can focus on quantifying how the interaction with the robot will impact the human sensorimotor strategy (changes in the posture, positions, forces, etc., induced by the robot) and recomputing metrics such as human fatigue and motor variability 54 and the Mover project. We can further use the biomechanical and robotic models to consider a unified operator-robot entity and to compute their joint abilities (e.g. common human-robot force capabilities 56).
Developing human cognitive and sensorimotor models to account for the effect of the human-robot interaction could provide a valuable tool to evaluate cobotic systems and collaborative works. However, the accuracy of these models must be addressed. We wish to understand how the robot influences the operator's work and thus how his mental model of the task evolves according to the interactions with the robot. The challenge is, then, to predict a behavior of the operator that takes into account his cognition in the interaction situations. Preliminary literature results have shown that key cognitive mechanisms in human teaming may transfer to human-robot collaboration, such as joint human-robot action representation 30 or coordination mechanisms 48. But the situation awareness of the operator is modified by the interaction with the robot 53. Developing a joint mental model which accurately capture the human-robot interaction can later guide the design of relevant interaction modalities to improve the team understanding 12.
3.2.2 Cobot adaptive assistance
Taking into account the coupling between the operator and the robot at the control level is also central to the team's objectives. We wish to demonstrate how a collaborative robot can be used to mediate between a control objective which optimizes task performances, safety and comfort (what we consider as the expertise trinity), on one hand, and the action model of the human interacting with the robot (the inferred human intent), on the other hand. Such an arbitration in the control law adapts the robot assistive behavior to better collaborate with the operator. This shared-autonomy concept is the focus of part of our research. It can range from a discrete task allocation between the agents to an effectively shared task 47.
We are strongly confident that the notion of expertise is central to adjust the cobot behavior. The robot controller is designed to increase the level of expertise in the operator/robot team: it optimizes the human-centered metrics (safety criteria, biomechanical and cognitive comfort, etc.) and provides a gain in performances (joint human-robot capabilities). But it also aims at preserving the operator particular expertise and know-how in the center of the activity. Manual expertise of highly skilled operators needs to be analyzed respectively on its dynamic aspects and on the ability to synchronize with other operators in the environment. Better understanding expertise is envisioned as a way to alleviate the operators of repetitive and easy operations while maintaining their ability to perform expert gestures based on the complexity of the task.
Furthermore, this research axis raises the question of the modification of the work induced by collaborative robots for expert operators. While the overall goal is to make use of robots to punctually or continuously improve the work conditions of these operators (and clearly not to replace them), the presence of these robots necessarily impacts the work referential and thus the expertise itself. One of the central questions, yet to be tackled, relates to the original and core part of the expertise that should remain unchanged. The proposed modeling of the operator/robot coupling and interaction is a first avenue to predict possible changes in the expertise. It can be input to the controller to constraint the robot to naturally let the operator take the expert decisions.
3.3 Design of cobotic systems
3.3.1 Architectural design
Is it necessary to cobotize, robotize or assist the human being? Which mechanical architecture meets the task challenges (a serial cobot, a specific mechanism, an exoskeleton)? What type of interaction (H/R cohabitation, comanipulation, teleoperation)? These questions are the first requests from our industrial partners. For the moment, we have few comprehensive methodological answers to these requests. Choosing a collaborative robot architecture is a difficult problem 41. It becomes even more complex when the architectural design is approached from concurrent cognitive ergonomic, biomechanical and robotic perspectives. There are major methodological and conceptual differences in these areas. It is, therefore, necessary to bridge these representational gaps and to propose a global and generic approach that takes into consideration the expectations of the robotician to model and formalize the general properties of a cobotic system as well as those of the ergonomist to define the expectations in terms of an assistance tool.
To do this, we propose a user-centered design approach, with a particular focus on human-system interactions. From a methodological point of view, this requires to develop a structured experimental approach. It aims at characterizing the task to be carried out ("system" analysis) but also at capturing the physical markers of its realization (required movements and efforts, ergonomic stress, etc). This specification must be done through the prism of a systematic study of the exchanged information (type and modality) needed by the humans to perform the considered task. On the basis of these analyses, the main challenge is to define a decision support tool for the choice of the robotic architecture and for the specifications of the role assigned to the robot and the operator as well as their interactions.
The evolution of the chosen methodology is for the moment empirical, based on the user cases regularly treated in the team (see sections on contracts and partnerships).
The process can be summarized through the following steps:
- identify expert or difficult jobs on industrial sites. This is done through visits and exchanges with our partners (manager, production manager, ergonomist...);
- select some challenging use cases to be studied and, then, observe the operator in its ecological environment. Our tools allow us to produce a force-motion analysis, based on previously defined ergonomic criteria, and a physical evaluation of the task in terms of expected performances, from experiments and simulations. In parallel, an evaluation of the operator expertise and cognitive strategy is done by means of questionnaires;
- synthesize these observatory results into design requirements to deduce: the robotic architectures to be initiated, the key points of human-robot interaction to be developed, and the difficulties in terms of human factors to be taken into account.
The different human and task analyses take advantage of the expertise available within the auctus team. The team has already worked on the current dominant approach: the use of human models to design the cobotic cell through virtual tools 3. We would like to gradually introduce the additional evaluation criteria presented above. However, the very large dimensions of the treated problems (modeling of the body's dofs and the constraints applied to it) makes it difficult to carry out a certified analysis. We choose to go through the calculation of the human workspace and performances, which is not yet done in this field. The idea here is to apply set theory approaches, using interval analysis as already discussed in section 3.1.2. The goal is, then, to extend the human constraints to intervals, which integrate the model variability, and to play them in virtual reality during simulations of the tasks. This would allow the operator to check his trajectories and scenarios not only for a single case study but for sets of cases. For example, it can be verified that, regardless of the bounded sets of simulated operator physiologies, the physical constraints of a simulated trajectory are not violated. Therefore, the assisted design tools certify cases of use as a whole. Moreover, the intersection between the human and robot workspaces/capabilities provides the necessary constraints to certify the feasibility of a task in the interaction situation. Overall, this integration of human and task-related physical constraints in the design process brings to better cobotic systems. In the future, we will similarly develop tools to include human cognitive markers into the design approach.
This research line merges the contributions of the other axes, from the analysis of the human behavior and capabilities in its environment for an identified task, the prediction of the effects of the interaction/coupling strategy with the robot, to the choice of a mechanical architecture from the resulting design constraints. The proposed task-oriented and human-centered methodology is perfectly integrated into an Appropriate Design approach. It can be used for the dimensional design and optimization of robots, again based on interval analysis. The challenges are the change of scale in models that symbiotically consider the human-robot pair, the uncertain, flexible and uncontrollable nature of human behavior, and the many evaluation indices needed to describe them.
It is worth noting that we aim at developing a global mechatronic design approach, which would build upon the design constraints to specify the robot hardware and controller at once. The chosen set-theory computational methodology is particularly appropriate to meet this objective since the interval-based representation of the design constraints can be directly and equally used to set the control constraints.
3.3.2 Control design
The control laws of collaborative robots from the major robot manufacturers differ little or not at all from the existing control laws in the field of conventional industrial robotics. Security is managed a posteriori, as an exception, by a security PLC / PC. It is therefore not an intrinsic property of the controller. This strongly restricts possible physical interactions 1 and leads to sub-optimal operation of the robotic system. It is difficult in this context to envision tangible human-robot collaboration. Collaborative operation requires, in this case, a control calculation that integrates safety and ergonomics as a priori constraints.
The control of truly collaborative robots in an industrial context is, from our point of view, underpinned by two main issues. The first one is related to the macroscopic adaptation of the robot's behaviour according to the phases of the production process. The second one is related to the fine adaptation of the degree and/or nature of the robot's assistance according to the ergonomic state of the operator. If this second problem is part of a historical dynamics in robotics that consists in placing safety constraints, particularly those related to the presence of a human being, at the heart of the control problem 3545, 38, it is not approached from the more subtle point of view of ergonomics where the objective cannot be translated only in terms of human life or death, but rather in terms of long-term respect for their physical and mental integrity. Thus, the simple and progressive adoption by a human operator of the collaborative robot intended to assist him in his gesture requires a self-adaptation in the time of the command. This self-adaptation is a fairly new subject in the literature 51, 52.
At the macroscopic level, the task plan to be performed for a given industrial operation can be represented by a finite state machine. To avoid increasing the human's cognitive load by explicitly asking him to manage transitions for the robot, we propose to develop a decision algorithm which would ensure discrete transitions from one task (and the associated assistance mode) to another based on an online estimate of the current state of the human-robot couple. The associated scientific challenge is to establish a link between the robot's involvement and a given working situation. We propose an incremental approach to learn this complex relationship. Its first stage will consist in identifying the general and relevant control variables to conduct this learning in an efficient and reusable way, regardless of the particular calculation method of the control action. Then, physically-realistic simulations and real-world experiments will be used to feed this learning process.
To handle mode transitions, we propose to explore the richness of the multi-tasking control formalism under constraints 42. It would ensure a continuous transition from one control mode to another while guaranteeing compliance with a certain number of robot control constraints. Some of these constraints convey the ergonomic specifications and are dependent on the state of the robot and of the human operator, which, by nature, is difficult to predict accurately. We propose, again, to exploit the interval-analysis paradigm to efficiently formulate ergonomic constraints robust to the various existing uncertainties.
Purely discrete or reactive adaptation of the control law would make no sense given the slow dynamics of certain physiological phenomena such as fatigue. Thus, we propose to formulate the control problem as a predictive problem where the impact of the control decision at a time is anticipated at different time horizons. This requires a prediction of human movement and knowledge of the motor variability strategies it employs. This prediction is possible on the basis of the supervision at all times of the operational objectives (task in progress) in the short term. However, it requires the use of a virtual human model and possibly a dynamic simulation to quantify the impact of these potential movements in terms of performances, including ergonomics. It is impractical to use a predictive command with simulation in the loop with an advanced virtual manikin model. We therefore suggest to adapt the prediction horizon and the complexity of the corresponding model in order to guarantee a reasonable computational complexity.
4 Application domains
4.1 Factory 4.0
The 4th industrial revolution (factory 4.0) is characterized by the integration of digital technologies into the production process, in order to meet the challenge of customizing services and products. This agility requires making manufacturing and maintenance lines flexible and versatile. This adaptation capacity is a characteristic of the human being, which puts him at the center of the production apparatus. However, this can no longer be done at the expense of the human operators' health and well-being. How can we reconcile the enhancement of our manual and analytical expertise, the ever desired increase in productivity and manufacturing quality, while reducing the hardship at work? Collaborative robotics, which we are seeking to build, is one of the key solutions to meet these societal challenges. By assisting humans while performing dangerous and painful tasks, the collaborative robot complements and helps them in their phases of physical and cognitive fragility.
More generally, we are interested in workstation cobotization, in the manufacturing and assembly industries but also in the construction and craft industries. The application areas are related to regional needs in aeronautics, maintenance, water and waste treatment. In most of these cases, it is possible to define the tasks and to evaluate the stakes and added value of our work.
5 Social and environmental responsibility
The scientific positioning of Auctus has an explicit social objective: assisting industrial workers to improve their working conditions through the appropriate limitations of physical solicitations and the improvement of their cognitive comfort. This has a direct societal impact on the health of the population and regarding the preservation of industrial skills and expertise in the local and national industrial ecosystem.
From an environmental point of view, the research goals of Auctus do not explicitly aim at improving the human footprint on the planet or at better understanding environmental related issues and processes. Yet, some of our projects can have a direct impact on these issues. This impact is for example directly related to the application context in the case of our collaboration in the domain of remote operation of technical skills with the Farm3 company. Indeed, this company aims at producing plants locally, with a reduced physical footprint while minimizing water consumption. We also envision a less direct but more fundamental impact of our work in the control, mechatronics and mechanical architecture domains where we aim at exploiting at best robot capabilities. In the long term, the impact of this work should lead to a reduction of the size of robots and of the amount of energy they consume to achieve a given task. This is clearly in line with the general objective of saving energy as well as the natural resources.
6 Highlights of the year
The highlight of this year is the reinforcement of our team by the recruitment of two people on permanent positions. First, Margot Vulliez joined us as a researcher. She completes our research proposals on shared autonomy, especially in the context of robotic teleoperation, and strengthens our axis on robotic design by her achievements in haptic systems with parallel mechanical structure. At the same time, Lucas Joseph has been recruited as a senior engineer and assigned to our team on a long-term basis. This will provide us with the necessary force to sustain our long term effort on the development of control software and architecture. It will also reinforce our ability to design experiments including human and robots to demonstrate the pertinence of our research contributions.
7 New software and platforms
7.1 New software
Real-time capable task-space capability calculation module in python
Computational geometry, Kinematics, Robotics, Biomechanics
Many recent human-robot collaboration strategies, are promoting human-centered robot control, where the robot continuously adapts its assistance level based on the real-time need of its human counterpart. One of the fundamental assumptions of these approaches is the ability to measure or estimate the physical capacity of humans and robots in real-time. One of the most well known task-space capability metrics is manipulability ellipsoid, which has been developed the robotics community but is used for humans as well, for motion analysis and rehabilitation. Even though the ellipsoid measures have been used widely for many years they are proven to represent the underestimation of the real capabilities, which have a form of polytopes. However, the robot and human capability polytopes are much more inefficient to calculate and so far have not been used for real-time applications.
Therefore, pycapacity software package implements the set of algorithms that have been developed in two recent papers of the AUCTUS team that bring the polytope evaluation in the real-time spectrum.
Pycapacity is a python package which provides a framework for the generic task-space capability calculation of robotic serial manipulators and human musculoskeletal models, based on the ellipsoid and polytope metrics. Additionally, the package also provides a set of generic polytope evaluation algorithms for standard polytope formulations, that can be used as a standalone library.
Antun Skuric, Vincent Padois, David Daney, Nasser Rezzoug
Robot Operating System (ROS), Robot Panda, Simulation, Robotics
Formulation of a control problem as a constrained optimization problem at the joint torque level.
This software is a control architecture for the Franka Emika Panda robot. The control problem is formulated as a constrained optimization at the joint torque level. This software is interfaced with Gazebo for simulaton, and the franka_ros packages for sending the desired velocity to the real robot.
Lucas Joseph, Vincent Padois
Robotics, Robot Operating System (ROS), Robot Panda, Simulation
Formulation of a control problem as a constrained optimization problem at the joint velocity level.
This software is a control architecture for the Franka Emika Panda robot. The control problem is formulated as a constrained optimization at the joint velocity level. This software is interfaced with Gazebo for simulaton, and the franka_ros packages for sending the desired velocity to the real robot.
Lucas Joseph, Vincent Padois
Robotics, Control, Optimization
A ROS package to control a Franka Emika panda robot using the Quadratic Programing controllers velocity_qp and torque_qp.
Moveit is a program that allows you to easily create trajectories avoiding obstacles. However, the control part of MoveIt is very basic and does not allow advanced control of the robot. Moreover, the trajectories calculated by MoveIt cannot be retrieved outside Moveit. moveit_trajectory_interface uses the Toppra library to generate a trajectory based on the path generated by MoveIt. This trajectory can then be used in any control algorithm.
Interfacing software to retrieve a geometric path determined by the MoveIt software (https://moveit.ros.org/) and to create a trajectory object.
Quadratic Optimization coNTROL
Robotics, Control, Optimisation
Qontrol is a tool for the generic formulation of robotic control problems in the form of constrained optimization problems. It is initially intended for fixed-base polyarticulated robots. It allows to easily create tasks and constraints in the control law.
Software for solving the Inverse Optimal Control problem with polynomials: - Direct Optimal Control - Inverse Optimal Control - Polynomial Direct Optimal Control - Polynomial Inverse Optimal Control - Truncated polynomial direct optimal control - Truncated polynomial inverse optimal control
7.2 New platforms
Participants: Erwann Landais, Lucas Joseph, David Daney, Vincent Padois, Jean-Marc Salotti, Nasser Rezzoug.
In 2022, Auctus members have pursued their effort of developping their experimental platform of collaborative Robotics (Arcol).
The Arcol platform provides technical support for the short, medium and long term experimental developments carried out within the framework of Auctus' scientific and dissemination activities. These technological developments are essentially software related in the context of human motion capture and real-time control of collaborative robots. Arcol aims at easing their implementation, deployment, documentation and support.
New developpements in 2022 are mostly related to:
- the integration of tools to perform obstacle free motion planning with state of the art algorithm (using MoveIt);
- the real-time capability of the software architecture dedicated to the control of fixed based serial robot (Franka Emika robot, and Universal robots);
- the ongoing effort related to the generic integration of sensors and robots in the platform through the use of the middleware ROS;
- the ongoing effort to converge towards an online motion capture system connected to model computation softwares allowing for the online estimation of human capabilities.
Participants: Alexis Boulay, David Daney, Margot Vulliez.
Farm3 develops an ultrasound-based farm for vertical off-ground cultivation: The Cube. Through the collaboration with the AUCTUS team, Farm3 aims at smartly robotizing the Cube, to enhance the cultivation conditions while facilitating the remote control of the farm in teleoperation. The experimental challenges in this project are: 1. Set up the dedicated robotic manipulator inside the Cube to autonomously perform simple tasks, such as grasping and placing plant buckets on the cultivation grid. 2. Preserve the farmers’ expertise as they should be able to remotely control the actions through a teleoperation interface augmented with relevant visual and haptic information.
In 2022, a simulation environment is being built to act has a digital twin of the real Cube environment. This digital environment includes the robot that will be used inside the Cube and the cultivation shelves. This simulation environment will be used to evaluate different control approaches and to help the user teleoperate the robot through visual feedback.
7.2.3 Instrumented glovebox
Participants: Lucas Joseph, Vincent Padois, David Daney.
Auctus is working in collaboration with the company Solvay. The technicians at Solvay handle chemical elements that must be contained in a specific environment. To do this, they use glove boxes. The space inside these boxes is limited and handling objects is not always practical. This is why they want to introduce robots in the glove boxes to help the lab technicians. The purpose of this platform is to build a demonstrator of this technology and test different forms of assistance. This demonstrator includes:
- voice recognition, to detect the user commands;
- detection of the environment, to detect objects of interest and obstacles inside the glovebox;
- motion generation, to plan obstacle free trajectories;
- control solutions, using the control tools developed by the team over the years;
The demonstrator is built around the three following tasks:
- to bring in unreachable objects;
- to use the robot as a "third arm" to hold objects;
- to bring preparations to other devices (scale, rheometer, ...).
The proof-of-concept experiment is available as a video here.
7.2.4 ATROCE : Attention Tunneling and RObotic Collaboration Experiments
Participants: Lucas Joseph, Benjamin Camblor, David Daney, Jean-Marc Salotti.
The platform consists in observing a human carrying out a task mobilizing his attention while a robot is working in its close environment. This platform will be used to study many cognitive aspects of a human carrying a task and how the robot can be used in such a situation.
In the first set of experiment the objective is to detect if the human is subject to an attentional tunneling effect and propose operating strategy for the robot to minimize the operator's entry into this incidental mental state. To monitor the human state, an eye tracking device and a heart rate monitoring device are used. While the human is doing a specific task, the robot must assemble several lego towers in a determined order. When the robot doesn't have the required lego to carry on its task, it must wait for the human to give it the correct lego block.
8 New results
8.1 Human Factors and cognitive approaches in human/system interactions
8.1.1 Experiment design for the study of human factors in human robot collaboration scenarios
Participants: Benjamin Camblor, Alexis Boulay, Simon Gervaise, Lucas Joseph, Jean-Marc Salotti, David Daney.
An experimental set-up was designed to study the factors that come into play during human-robot interaction. This set-up is intended to be adaptable and usable for several experiments, particularly on the cognitive side of interaction. It includes a Franka Emika Panda robot and a construction task using Legos which can be performed in collaborative way. It also includes an eye-tracking system to allow for objective measurements of some human features in an interaction scenario. For example, it can provide data related to a person's attention. The system was chosen after a detailed technical review of existing devices.
8.1.2 Human Factors in the space domain
Participants: Jean-Marc Salotti.
Jean-Marc Salotti has published several articles which are not directly related to the field of collaborative robotics, but which nevertheless enhance the general methodology adopted by the team. Thus, the article published in the Journal of Space Safety Engineering takes up essential elements of the risk analysis classically proposed in robotics with the integration of risks associated with the degradation of situational awareness. Similarly, in the article published in the journal Futures, the risk analysis takes into account human factors in human-system interactions.
8.2 Human Behavior Analysis
8.2.1 On the Reliability of Inverse Optimal Control
Participants: Jessica Colombel, David Daney, François Charpillet.
Inverse Optimal Control (IOC) is a popular method for human motion analysis. In the context of these methods it is necessary to pay attention to the reliability of the results. The approach developed on this topic is an approach based on the evaluation of Karush-Kuhn-Tucker conditions relying on a complete analysis with Singular Value Decomposition and provides a detailed analysis of reliability. With respect to a ground truth, simulations illustrate how the proposed method analyzes the reliability of the resolution. After introducing a clear methodology, the properties of the matrices are studied with different noise levels and different experimental models and conditions. The proposed work details the numerical difficulties encountered during the resolution and thus how to make the results of the IOC problem reliable.
Related publication: 13
8.2.2 Study of Motor Variability
Participants: Raphael Bousigues, David Daney, Vincent Padois.
This work focuses on the study otor variability of a human arm during a constrained task and the impact of a collaborative robot arm on human movement. To this end, a path following experiment by a human assisted by a robot is being conducted.
After finalizing the design of the experimental set-up (physical wire-loop to be tracked, tunable chair for the human subject, robot controller, motion capture system), a first experimental campaign was conducted with 11 subjects completing the wireloop course 50 times while being assisted by the collaborative robot. The movements of the human and the robot were captured using an optical motion capture system. The motion data captured during the experiment was identified and then filtered. The subjects' joint centers were computed using a virtual human model. The motor variability analysis data was calculated from the joint center positions for further analysis.
A preliminary analysis of the results highlights the motor variability of the subjects in the case of following a parabolic path in collaboration with a robot. No specific intra- or inter-individual behavior, that could characterize the subjects' experience of the task or the fatigue (muscular or cognitive), has been noticed.
For the second experiment, the path chosen is composed of alternated curved and straight portions. The robot controller needs to know the equation of the path, and its first and second derivatives. To determine these equations, a Gaussian equation fitting method was implemented and integrated into the robot controller. This second experimental campaign should start early 2023.
8.2.3 Observing and recording information on an expert task
Participants: Erwann Landais, Vincent Padois, Nasser Rezzoug.
Within the framework of the study of the possible contributions of a semi-autonomous system for the realization of an expert task in the chemical industry, the question arose of identifying the components of the expertise expressed by the operator carrying out this task. To this end, several contacts were made with different operators, in the form of interviews and video recordings of demonstrations. Thanks to these interviews, the components of the expertise expressed by the operators could be highlighted (manual, visual and cognitive expertise), as well as a more general definition of the problematic linked to this task (manipulation of objects in the context of a visual classification task). This information opened up different lines of research concerning possible semi-autonomous assistances to help experts to carry out a task.
Searching for best-fitting musculoskeletal models approximating an individual's upper limb force capacities
Participants: Gautier Laisné, Jean-Marc Salotti, Nasser Rezzoug.
Three main direction of research have been pursued this year on this topic related to the ability to accurately evaluate the musculo-skeletal capacities of human operators:
Genetic algorithm as an optimizer for searching a best-fitting musculoskeletal model
The force capacities of an individual in a static posture can either be approximately measured experimentally or computed if the subject's musculoskeletal model is known. What is computed is called a force polytope: it's the intersection of the torque zonotope and the joint configuration's null space.
The purpose of this research is to use measured data of an individual's force capacities at different postures and use them to find a musculoskeletal model that would produce the observed force polytopes. Preliminary results in simulation focused on personnalizing a generic upper limb musculoskeletal model (Holzbaur's model) which would reproduce the force capacities created by a specific MSK model with known parameters. These results were obtained using a genetic algorithm, due to the nature of the optimization problem which doesn't express naturally in a convenient way that would allow us to use more specific optimizers.
Analytical expression for force polytope via Minkowski sums of superquadrics
When measuring a subject's maximal force capacities in specific direction, we can suppose that we measure points at the surface of a polytope. The aim of this research was to express analytically these points, in order to parametrize a musculoskeletal model which would generate force polytopes fitting the measured points.
The surface of a force polytope is described as the surface of torque zonotope intersected with the joint configuration's null space. Besides, the torque zonotope has great geometrical properties, thus it is possible to describe its surface via Minkowski sum of superquadrics (a generalization of ellipsoids which have simple analytic expression). First results were in 2D and lead to a smooth analytic expression of the force polytope surface.
Geometric algebra as a new tool for force polytope description
One fundamental problem with computing force polytopes is its expression. Geometrically, it is represented as the intersection of two objects. The intersection operation is not well understood in its most common description. However, using geometric algebra and its novative geometric formalism, it is possible to partially describe the intersection as a bilinear operation. This research focused on rewriting the force polytope expression in a formalism which would remove a fundamental drawback brought by the intersection.
8.3 Human Robot Interaction
8.3.1 Approximating robot reachable space using convex polytopes
Participants: Antun Skuric, Vincent Padois, David Daney.
Human and robots are expected to work in close proximity in future, establishing a high degree of collaboration and physical interaction. In order to enable such scenarios, better design and performance evaluation tools are needed to convieve safer and more capable robots. In addition, new stategies are needed to inform the operator about the robot's current state and its physical abilities, both in real-time and ideally in future.
This work presents a new metric for quantifying and visualising robot's movement capacity in a form of the robot's reachable space. The proposed approach predicts the reachable space over a given time horizon based on the robot's actuation limits and kinematic constraints. The approach is furthermore extended to integrate the robot's environment, assuming it can be expressed in a form of linear constraints, and to account for the robot's link geometry. The accuracy of the proposed method is evaluated using simulations of robot's nonlinear dynamics and it is compared against the cartesian space limits, usually provided by manufacturers in standard datasheets. The accuracy analysis results show that the proposed method has good performance for the time horizons up to 250ms, encapsulating most of the simulated robot's reachable space while maintaining comparable volume. For a 7 dof robot, the method has an average execution time of 50ms, independent of the horizon time, potentially enabling real-time applications.
Related publication: 14
8.3.2 Evaluation of Cartesian error polytopes as a function of controller architecture and stiffness
Participants: Vincent Fortineau, Vincent Padois, David Daney.
During interactions with a physical environment, the human endpoint impedance provides modeling insights both for the field of human movement science and for the design of innovative controllers for collaborative robotics based on physical human-robot interaction 31, 29, 61.
The use of polytopes to qualify the endpoint behavior of a robot or human while subject to unmodeled external forces can define a metric that can be used for robotic command 11. The purpose of this research is to use the estimated Cartesian error polytopes of a robot or human while it interacts with another robot, to adapt the behavior of the later robot with regard to the polytopic metric. For example, while a robot and a human share a load with a desired coupled stiffness, the robot could adapt its endpoint behavior to the time dependent stiffness behaviour of the human, to maintain the targeted joint stiffness value. The preliminary work performed on this topic both consists in a literature review of the domain of robot and human impedance modeling as well as of a theoretical computation of the Cartesian error polytope for a manipulator subjected to three different control architectures: task space computed torque control, joint space computed torque control and Jacobian transpose control. Extension to the dual arm case is in preparation as well as experimental validations of the obtained theoretical results.
8.3.3 Real-time video feedback for virtual reality
Participants: Erwann Landais, Vincent Padois, Nasser Rezzoug.
One of the objectives of a teleoperation task is to ensure a telepresence feel; that is, to ensure that the use of the teleoperation system is as intuitive as possible for the user. In the context of a manipulation task, where a robotic arm has to reproduce the user's movements in front of a camera, it therefore seems appropriate to confuse the video feedback on the robot's performance of the task with the user's vision.
For this purpose, an implementation of real-time camera video feedback on a Hololens augmented reality headset was carried out. This implementation is based on Gstreamer and Unity software.
Although this implementation is functional, it still presents various software problems (quality of the displayed video stream, mandatory presence of a redundant Unity application for the transmission of the video stream which can cause connection difficulties) which must be corrected before being applied in an experiment.
8.4 Robotics and control
8.4.1 Online task-space trajectory planning using real-time estimations of robot motion capabilities
Participants: Antun Skuric, Nicolas Torres, Vincent Padois, Lucas Joseph, David Daney.
Planning a robot's task-space movement according to its actual capacity is a difficult problem because this capacity depends on its state and thus can evolve significantly during the execution of the movement. This work proposes a method for real-time trajectory planning based on time-optimal Trapezoidal acceleration profile (TAP) trajectories, that adapts to the real-time evolution of the robot's capacity. The method is based on an efficient approach for projecting the robot's kinematic limits in the trajectory direction, using the convex polytope algebra. The method is experimentally validated on a Franka Emika Panda collaborative robot and compared with the classical approach considering fixed robot's Cartesian space motion capacity. The results show that the proposed method is able to better exploit true robot's motion capacity by generating faster trajectories for the same level of tracking accuracy.
Related publication: 21
8.4.2 Linear Model Predictive Control in SE(3) for online trajectory planning in dynamic workspaces
Participants: Nicolas Torres, Antun Skuric, Vincent Padois, Lucas Joseph, David Daney.
Inria and Stelantis formed a cooperation project in the form of an Industrial PhD to tackle safety for workspaces shared between humans and robots. The project aims to avoid the classical solution of static security zones by proposing a dynamic formulation capable of exploting the knowledge of the robot's state and its capacities, the environment information (such as the humans around it), while also trying to estimate the evolution of the whole. In the end this approach aims to make the robot capable of adapting its behaviour according to the changing conditions around it, while offering safety guarantees for humans working in close collaboration.
Efficient workspace sharing of collaborative robots and human operators remains an unsolved problem in the industry. This problem goes beyond the use of a priori or a posteriori safety measures and has to be tackled at the control level. To address the need of adaptation to human presence as well as to endow the robot with the ability to adapt interactively to new Cartesian targets, a linear Model Predictive Controller has been developed. This controller computes acceleration bounded optimal Cartesian trajectories in SE(3) over a receding horizon. The pertinence of the proposed control architecture is demonstrated using experiments with the Franka Emika robots in different scenarios implying both adaptation of the maximum allowed velocity to comply with human presence and on-the-fly update of a Cartesian goal pose.
Related publication: 22
8.4.3 Teleoperation for remote tasks realization
Participants: Alexis Boulay, Margot Vulliez, David Daney.
Within the framework of the collaboration with the Farm3 company, where the development of a telerobotic system based on shared autonomy for the remote nurturing of plants in a controlled chamber is envisionned, several axis of developments have been launched:
- Simulation: Design of a simulation environment using ROS/Gazebo to allow for the testing of the shared-control strategies. Control (optimization based control such as 7.1.3) and motion planning algorithms (using MoveIt!) integration has also been started.
- State-of-the-art on tasks analysis: Before starting any reflection on which controllers to implement on a remote robotic systems, it is important to know its purpose. On which task will it be implemented? What are the important characteristics of this task? Which part of the task can be managed fully automatically? Which part of the task require a human expertise? There are many ways to describe a task but most through a review of the state of the art in the domain we noticed the existence of a consensus on the decomposition of a task into very simple sub-tasks (motion, force,...). This advocates for the future definition of a library of sub-tasks that can be used as building blocks for more complex tasks in the considered application domain. Conversely a task decomposition methodology could help converting complex tasks into an arrangement of generic sub-tasks.
- State-of-the-art on virtual fixtures and their limitations.
8.4.4 Shared Control in Human-Robot interaction
Participants: Elio Jabbour, Margot Vulliez, Vincent Padois.
The concept of shared control pertains to a methodology of human-robot interaction, in which the robot and human agents partake in a collaborative effort to accomplish a specified task. This methodology allows for a segregation of labor between the human and robot, with the latter assuming responsibility for physically taxing or perilous assignments, while both agents' capabilities are mutually supplementary. The emphasis is placed upon the transparent and intuitive interaction between the human and robot, which is vital to the efficacy of shared control with regards to communication and collaboration.
One essential question in shared control relates to what the robot and human agent should possess minimally for shared autonomy to benefit from the end performance/results?
A thorough literature review is currently being conducted with the aim of classifying and identifying correlations between existing shared control methodologies within the following categories: Human/Robot input, Blending method, Human intent, Environment, and Feedback types to/between the Human and Robot. The overarching objective of this endeavor is to formulate a general criterion for defining shared control for any robotic applications in the near future.
8.4.5 Modeling the impact of joint internal backlashes on robot capabilities, a specific study of robot legs comprising parallel mechanisms.
Participants: Virgile Batto, Margot Vulliez.
The legged-robot codesign PhD project, in collaboration with the Gepetto team at LAAS-CNRS, aims at developing AI-based tools to help in designing a new leg architecture for a dynamic walking robot. Dynamic bipeds often comprise parallel mechanisms in their leg kinematics to relocate the actuators close to the hip and reduce the weight of moving parts. Such parallel mechanisms must be finely optimized that the robot meets the required capabilities to perform different locomotion tasks.
In this context, an objective is to exploit the capacity of the available robotic library Pinocchio 32 to model the dynamics of such parallel legs and to compute its expected performances. The study of such closed-loop mechanisms requires few developments within the Pinocchio library, currently in progress. We further want to exploit the modeling library to optimize the robot parameters with respect to the task or specific hardware-actuation-control uncertainties. So far, we particularly focus on analyzing the impact of joint internal backlashes on the robot performances. We have proposed a preliminary geometric model of joint clearances which, combined with the Pinocchio library, can predict the behavior of the imperfect robot, in each task/configuration. This first simple model will be validated on several leg architectures, and later generalized to model any joint backlashes. We propose to integrate this model in a global design support tool to optimize the robot capabilities.
8.4.6 Elasto-Geometric Calibration of a Humanoid Robot
Participants: David Daney.
This result is obtained through a collaboration led by the LAAS, CNRS.
The whole-body elasto-geometrical calibration of humanoid robots is critical particularly for their control and accurate simulation. However, it is often not considered probably since it is a nontrivial task due to the mechanical complexity and inherent constraints of anthropomorphic structures. Also, humanoid robots have to sustain great efforts on their support legs, leading to link and joint being deformed, and are prone to auto-collision. Thus, elastic parameters have to be factored in in addition to the geometric ones and to improve the precision of the pose of all robot segments. This is much more cumbersome and time consuming than the classical calibration of serial manipulators that deals solely with the estimation of the pose of the end-effector. Finally, due to the complexity of the task, a manual intervention in several steps of the calibration is no longer possible and a thorough automation of the approach is needed. Therefore, we propose to use a stereophotogrammetric system along with embedded joint torque sensors to calibrate the pose of all robot links with a fully automatic procedure. The generation of the minimal set of optimal calibration postures is based on a new iterative optimization process that leads to a stable maximum of an observability index. Then full set of geometrical parameters but also joint and base elastic parameters were calibrated using a single least-square optimization program. The proposed method was validated on a TALOS humanoid robot allowing to obtain an accurate whole-body calibration in less than 10 minutes. The proposed approach was cross-validated experimentally and showed an average RMS error of the tracked markers of 2.2mm.
9 Bilateral contracts and grants with industry
Participants: David Daney, Vincent Padois, Antun Skuric.
The collaboration aims to design a constellation of mini-satellites and one of the challenges is to rethink their production, in particular through robotic assistance of operators. In this project, we have developed a coupled model of human-robot physical capabilities, see 10.2.
Project in a nutshell:
- Consortium : AUCTUS@Inria, Airbus
- Funding : BPI
- Duration : 2020 – 2023
Participants: David Daney, Vincent Padois, Nicolas Torres.
The objective of this project is to work on the required principles to avoid the classic restrained static security zones, synthesizing a dynamic representation of the shared workspace to take advantage of state of the art control laws, allowing a fluid collaboration between human-robot. This dynamic synthesis requires knowledge of the robots state (geometric, cinematic and cognitive, such as fatigue, expertise and situation awareness), its tasks, capacities, state of humans that surround it and their tasks. Furthermore, it needs to achieve a formal and provable online algorithm that correctly estimates the state of the human and guarantee a safe shared workspace tackling ambitious scientific questions poorly addressed in literature.
Project in a nutshell:
- Consortium: AUCTUS@Inria, PSA Automobiles
- Funding: PSA Automobiles, ANRT (CIFRE)
- Duration: 2020–2023
Participants: Vincent Padois, Jean-Marc Salotti, Margot Vulliez, Julien Passama, David Daney.
In 2019-2022, we have established a recurrent collaboration with AKKA to transfer our know-how on collaborative robotics design. Two partnerships have been established. The first one is a bilateral contract (Portage) which allowed the user-centered design of a semi-autonomous vehicle for pulling heavy loads. In coordination with Akka’s engineers, we used methodologies from the field of Human Factors to highlight the psychological factors leading to the occurrence of human errors. We then deduced interfaces and interaction modes and tested and validated our solutions. A demonstrator has been presented to industrial partners on December the 17th, 2022, in presence of all members of the consortium. A second collaboration started in September 2021 through the supervision of a PhD (CIFRE) on the design of a cobotic solution for the assistance of a surgeon in an operating room. Julien Passama, who was already working at Akka research, was recruited. However, the student stopped the PhD for personal reasons.
- A contract was signed with Akka for a PhD aiming at determining the best option for assisting the assistant of the surgeon during a surgical operation, using a collaborative robot.
Project in a nutshell:
- Company: Akka
- Funding: Akka and ANRT (CIFRE)
- Duration: 2021–2022 (stopped)
- PhD Student: Julien Passama
- Researchers involved: Jean-Marc Salotti, Vincent Padois, with the participation of Margot Vulliez
Participants: Vincent Padois, Nasser Rezzoug, Lucas Joseph, David Daney, Erwan Landais.
Since 2020, we have developed a long term collaboration with the chemical company Solvay in order to help them in the digitalization and robotization of their productions. Our interlocutors are researchers of their Laboratory Of the Future (LOF) on the theme of collaborative robotics, seen as an important way to assist their operators and secure their potentially dangerous actions. The first objective is to develop a cobotic solution to follow an operators’ work step by step by proposing an assistance available at the user’s request in a constrained environment. This project has led to a bilateral contract and to the participation in the ANR Pacbot. In addition, in 2021, we are participating in the ”Miels project” in order to cobotize a handling task for the mixing of chemicals that requires human gestural expertise. The difficulty is first to learn from the human and then to synthesize this expertise.
- Miels Project - see 10.3.1
Glovebox project Technicians at LOF handle chemicals elements that needs to be contained in specific environment. To that extent they use glove boxes. The space inside these boxes is constrained and manipulation of objects is not always practicle. To that extent, they want to introduce robots inside the glove boxes to assist the lab technician. A postdoctoral student, Lucas Joseph, has been recruited to build a demonstrator of such technology.
Project in a nutshell:
- Consortium: AUCTUS, Solvay, CNRS
- Funding: 1 year postdoc grant
- Duration: 2021–2022
Participants: Alexis Boulay, David Daney, Margot Vulliez.
In 2020, the team has begun a collaboration with Farm3, a start-up company specialized in ultraponic vertical farming. The company develops a robotized vertical farm, the Cube, to grow plants in a controlled environment through ultrasound-based techniques. Agronomists and farmers can remotely act on the plants through a teleoperation system, to perform expert tasks (seedlings, pollinating flowers, measuring data...) without polluting the sensitive growth environment.
After preleminary results of a master's internship in 2021, a contract was signed with Farm3 in 2022 to start a PhD project on assisting the human by transferring skills to the robot in teleoperated vertical agriculture. The research interest of the thesis is to develop and implement co-control approaches to assist the agronomists/farmers in their work. The first scientific axis will focus on analyzing specific farming tasks and automating the simpler ones (moving a pot, taking pictures of the plants, ...). The second axis will consist in determining the information and actions needed to preserve the expertise of the agronomists in the teleoperation context. The interaction modalities, in which the perceptive/action data could be communicated via a haptic and visual system, will then be defined. Once the whole shared-autonomy teleoperation system will be established, a study of skill-transfer solutions is planned, to increase the robot autonomy during its interaction and to better help and correct of the gestures of the agronomist.
Project in a nutshell:
- Consortium: AUCTUS, Farm3, CNRS (Institut Pprime)
- Funding: Farm3, ANRT (CIFRE)
- Duration: 2022–2025
Participants: Guillaume De Mathelin de Papigny, Vincent Padois, Lucas Joseph.
Started in 2021, the partnership between AUCTUS and the robotics company Aerospline has two main objectives. On the one hand, the collaboration brings to Aeropline new scientific research knowledge in Robotics in order to develop new robotic methods and tools for their projects. On the other hand, it brings to AUCTUS elements for the formulation of new scientific problems.
Project in a nutshell:
- Consortium: AUCTUS, Aerospline
- Funding: Plan de Relance
- Duration: 2021-2023
9.7 Fuzzy Logic Robotics
Participants: Sébastien Dignoire, Vincent Padois.
Started in 2022, the partnership between AUCTUS and the robotics company Fuzzy Logic Robotics has two main objectives. On the one hand, the collaboration provides Fuzzy Logic Robotics with a way to evaluate the pertinence of state of the art knowledge in robotics in their daily robotics practice. On the other hand, it brings to AUCTUS elements for the formulation of new scientific problems.
Project in a nutshell:
- Consortium: AUCTUS, Aerospline
- Funding: Plan de Relance
- Duration: 2021-2023
10 Partnerships and cooperations
10.1 International research visitors
The team is strengthening its collaborations with international teams.
10.1.1 Visits of international scientists
The Auctus team welcomed two students:
- Clemente Donoso
Institution of origin:Universidad Técnica Federico Santa María
Dates:from May 2022 until Aug 2022
Context of the visit:International Mobility Program Inria-Chile
Mobility program/type of mobility:internship
- Gianmarco Panzetta
Institution of origin:University of Bologna
Dates:from Sept 2021 until Feb 2022
Context of the visit:Project of collaboration with Prof. Marco Carricato on robot workspace analysis.
Mobility program/type of mobility:internship
10.1.2 Visits to international teams
Context of the visit:
Collaboration research mission
Mobility program/type of mobility:
collaborative project IITD-Inria-Naval Group
Mila - Institut québécois d'intelligence artificielle ; CRVI (CCTT), IVADO, Kinova
Context of the visit:
Collaboration research mission
Mobility program/type of mobility:
Regional Initiative, R4 delegation
MIT - CSAIL, Northeastern University, MassRobotics, Barrett technology
Context of the visit:
Collaboration research mission
Mobility program/type of mobility:
Regional Initiative, R4 delegation
10.2 National initiatives
10.2.1 BPI LiChIE
Participants: David Daney, Vincent Padois, Antun Skuric.
The LiChIE project (funded by BPI) aims to design a constellation of mini-satellites for optical Earth observation. Among many other topics, this requires to rethink the way sattelites are being produced in order to ease this highly complex process. There is actually an unprecedented economical and societal demand for robots that can be used both as advanced and easily programmable tools for automatizing complex industrial operations in contexts where human expertise is a key factor to success and as assistive devices for alleviating the physical and cognitive stress induced by such industrial task. Unfortunately, the discrepancy between the expectations related to idealized versions of such systems and the actual abilities of existing so-called collaborative robots is large. Beyond the limitations of existing systems, especially from a safety point of view, there are very few methodological tools that can actually be used to quantify physical and cognitive stress. There is also a lack of formal approaches that can be used to quantify the contribution of collaborative robots to the realization of industrial tasks by expert operators. Of course, in the state-of-the-art, existing works in that domain do consider some aspects of the current state of the operator in order to propose an appropriate robot behaviour. One of their conceptual limitations is to consider an a priori defined human-robot collaboration scenario where the expertise of the human operator is of importance but limited to a single operation. The consideration of larger varieties of tasks is rarely considered and, when it is, only a strict separation of the tasks to be achieved by each member of the human-robot dyad is considered. In this project, we propose to develop a coupled model of human-robot physical abilities that does not make any a priori with respect to the type of assistance. This requires to develop a parameterisable generic model of the potential physical link and implied constraints between the human operator and the robot. This model should allow to describe the task to be achieved by the human alone or using a collaborative robot through different interaction modalities. Online simulation of these scenarios coupled with ergonomic and performance indicators should both allow for the discrete choice of the right assistance mode given the task currently being achieved as well as for the continuous modulation of the robot behaviour.
Project in a nutshell:
- Consortium : AUCTUS@Inria, Airbus, Erems, iXblue, TEAMS@Inria, Onera
- Funding : BPI
- Duration : 2020 – 2023
- Researchers involved : Antun Skuric (PhD Student), Vincent Padois (thesis advisor) and David Daney (thesis advisor)
10.2.2 ANR PACBOT
Participants: David Daney, Jean-Marc Salotti, Benjamin Camblor.
The general objective of the project is to design a semi-autonomous cobotic system for assistance, able to choose, synchronize and coordinate tasks distributed between humans and robots by adapting to different types of variability in professional gestures, all by anticipating dangerous situations. The orchestration of tasks between a man and a robot is difficult because it must answer the question of the distribution of roles within the couple according to physical and decision-making skills and constraints as well as the consequences of their interactions. However, we cannot put the two actors at the same level: the robot has to adapt its actions to the work of an operator and, more precisely, to its motor and cognitive strategies that materialize through the quantifiable variability of professional actions. On the other hand, the very interest of the robot is to assist the operator in his phases of fragility while preserving his physical and mental integrity, in particular considering that human error is inherent in operator action. These considerations are, for Pacbot, the conditions necessary for the joint achievement of efficient work.
Partners of the ANR project:
- Auctus (Leader), David Daney (principal investigator)
- Laboratoire Informatique de Grenoble
- Laboratoire Interuniversitaire de Psychologie (Lyon)
- ANR Funding for Auctus: 246 240 Euros
- Duration: 3 years
- Start: January 2021
10.2.3 ANR JCJC ASAP-HRC
Participants: Elio Jabbour, Vincent Padois, Margot Vulliez.
A young-researcher ANR grant was obtained in 2021 to start a collaborative project between the AUCTUS team at Inria, the RoBioSS team at the Pprime Institute (CNRS), and the CeRCA laboratory (CNRS). It aims at rethinking autonomy for shared action and perception in Human-Robot Collaboration, through transverse studies in robotics and cognitive sciences. More particularly, three scientific axes must be addressed to develop a human-centered and generic shared-autonomy framework:
- study key features of Human-Robot perception-action mechanisms and identify multisensory integration processes involved in Human-Robot interaction. These human studies should constitute the baseline of robotic developments and shape the shared-autonomy scheme,
- develop a shared perception between the different actors (humans and collaborative robots), according to their sensory data and involvement in the task. This shared perception will be based on a multimodal (haptic, visual) feedback mixture conveying information about the task, the environment, and the collaborators,
- combine Human-Robot commands into a joint action toward the task goal. The human inputs will first be used to infer the operator intent and adapt the robot behavior. Then, the shared action will combine robot skills and human commands into a unified and consistent control objective.
Partners of the ANR project
- AUCTUS (coordinator: Margot Vulliez)
- Institut Pprime CNRS, RoBioSS team (Poitiers)
- Centre de Recherches sur la Cognition et l'Apprentissage, CeRCA CNRS (Poitiers)
- ANR Funding: 287 840 Euros
- Duration: 4 years
- Start: October 2021
10.2.4 LAAS-AUCTUS collaborations
Participants: Virgile Batto, Margot Vulliez, David Daney.
We have built a close scientific relationship with the Gepetto team at LAAS CNRS (Toulouse) these past few years, through several collaborative projects.
- Humanoid robot calibration A collaboration on problems of calibration of humanoid robots is underway. - see 8.4.6.
- IOC A collaboration on Inverse Optimal Control problem has started in pursuit of the results obtained in 18 - see 8.2.1.
- Legged-robot codesign - This PhD project aims at developing a generic codesign approach which will cover the hardware specification and dimensioning and the control strategy and requirements at once. We propose to leverage mastered AI-based methods (simulation, planning, optimization) to guide the mechatronic design cycles and to provide tools to assist designers. The transversal approach will be applied to the codesign of a new dynamic legged robot, as a balance between versatile but heavy robots (Atlas, Talos) and light robots limited to walking (Digit).
10.3 Regional initiatives
Participants: Erwann Landais, Nasser Rezzoug, Vincent Padois, David Daney.
The main objective of the MIELS project is to develop innovative strategies to characterize and develop neoteric, non-toxic solvents through strategies that will enable to grasp the enormous quantity of required experimental tests all in insuring an absolute safety of the manipulator. For this purpose, we intend to work on two complementary routes, the development of solvent characterization methodologies and the integration of a cobotic approach in solvent handling and evaluation, with the ambition of merging these developments at the end of the project in order to draw as much synergy as possible. This project is built around teams with complementary competencies to achieve these objectives. UMR LOF and Solvay LOF have great experience acquired over several years of research the fields of solvent evaluation and robotics, whereas Auctus INRIA team has a strong expertise in collaborative robotics. By combining our competencies and expertise, the Miels project aims to merge all these fields in order to expand 1) the fundamental study of solvents and their characterization techniques including theoretical techniques, in particular for green solvents and 2) the development of the use of cobotics, in collaboration with Auctus INRIA team, for increasing the efficacy and safety of laboratory workers in industry, in particular those working on characterization of solvents.
Project in a nutshell:
- Consortium : AUCTUS@Inria, Solvay, LOF
- Funding : Région Nouvelle Aquitaine, Solvay
- Duration : 2020 – 2024
Participants: Nassim Benhabid, David Daney, Vincent Padois.
The main goal of the work is to develop a design methodology (control and physical architecture) of collaborative robotic systems dedicated to assisting and securing operator expert gestures in the context of industrial applications involving strong physical interaction between a human, a tool and a manipulated object. The collaborative system should improve ergonomics and safety while maintaining the level of industrial performance and preserving the user’s know-how. The application context is the carpentry, in particular wood shaping.
This project is conducted by Nassim Benhabib, David Daney and Vincent Padois. With the collaboration of BTP CFA de la Gironde. This allows to provide realistic information of the task studied and to collect the feedback from the craftsmen.
Project in a nutshell:
- Consortium: AUCTUS@Inria, BTP-CFA-Gironde
- Funding: Région Nouvelle-Aquitaine, Inria
- Duration: 2019–2022
10.3.3 AAPR Perception-HRI
Participants: Margot Vulliez.
This regional project completes the ASAP-HRC ANR objectives with additional cognitive studies to improve the exchange of perceptive information during Human-Robot interactions. Such an exchange of information between the agents is required to communicate and coordinate together. We particularly focus on visual and haptic feedbacks, related to the task, the context, or the robot assistance, and given through a teleoperation device to perform an industrial task.
Only a fine analysis and modeling of the human multisensory perception and integration processes can provide practical guidelines to determine the optimal mixture of feedbacks to implement in the human-robot interface. The project therefore aims at developing personalized mathematical models of the perceptive and sensorimotor integration of visuo-haptic informations, in interaction scenarios with a robot.
Partners of the project
- CeRCA CNRS (Poitiers)
- Institut Pprime CNRS (Poitiers)
- Funding: 151 000 Euros
- Duration: 4 years
- Start: December 2022
11.1 Promoting scientific activities
11.1.1 Scientific events: organisation
General chair, scientific chair
- Nasser Rezzoug was the scientific chair of the 47th congress of the french speaking Society of Biomechanics, Monastir, Tunisia, 26-28 october 2022
Member of the organizing committees
- David Daney organized a seminar on collaborative robotics for the Aquitaine Robotics cluster on December 06, 2022, Talence, France.
- Antun Skuric and Lucas Joseph organized a demonstration on the work "Online task-space trajectory planning using real-time estimations of robot motion capabilities" 21 during the seminar on collaborative robotics for the Aquitaine Robotics cluster on December 06, 2022, Talence, France.
11.1.2 Scientific events: selection
Chair of conference program committees
- Vincent Padois was an Associate Editor for the IROS 2022 International Conference.
List of conferences for which Auctus members have review activities :
- Robotics: IEEE ICRA, IEEE IROS
- Biomechanics: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, 11th International IEEE EMBS Conference on Neural Engineering, 47th congress of the french speaking Society of Biomechanics
Invited talks and other presentations
- David Daney and Antun Skuric participated in a networking trip in Montreal and Boston organized by R4, regional robotics reserach network, in october 2022. Within the framework of this trip, Antun Skuric gave 2 talks on his research on the evaluation of human and robot capabilities using polytopes respectively at the Institute for Experiential Robotics (professor Takin Padir) and at the Action Lab (professor Dagmar Sternad), Northeastern University, Boston, USA.
- Jessica Colombel – Guest seminar "Biological Motion and Human-Robot Interaction", Journées Psyphine, november 2022, Nancy
- Jessica Colombel – Guest seminar "Biological Motion and Human-Robot Interaction", Séminaire Cognition & Langage, Institut des sciences du Digital, Management Cognition, october 2022, Nancy
- Vincent Padois – Invited seminar "Collaborative Robotics - Myths, Legends, Facts", Journées Nationales de la Recherche en Robotique Humanoïdes, LARIS, july 2022, Angers
- Vincent Padois – Invited seminar "Évaluation des performances et des capacités pour la commande appropriée des robots collaboratifs", Journée GDR GT1 "Robotique & Ergonomie", ISIR - Sorbonne Université, october 2022, Paris
- Vincent Padois – Invited seminar "Auctus, Robots for Humans at Work – A rapid introduction of our research activities", Matinale Aquitaine Robotics – "Robotique Collaborative", Inria, Centre de l’université de Bordeaux, december 2022, Talence
Reviewer - reviewing activities
List of journals for which Auctus members have review activities:
- Robotics: IEEE Robotics and Automation Letters, Mechanism and Machine Theory, IEEE Transactions Robotics, IEEE Transactions on Haptics, Journal of intelligent manufacturing, International Journal of Social Robotics
- Biomechanics: Journal of Biomechanics
- Software: Journal of Open Source Software
- Space: Microgravity Science and Technology, Journal of Aerospace Information Systems, Acta Astronautica, Journal of Space Safety Engineering
11.1.4 Leadership within the scientific community
Nasser Rezzoug is vice-President of the french speaking Society of Biomechanics since 10/2021
11.1.5 Scientific expertise
Vincent Padois was a reviewer for two submissions "Défis Clés" of the regional initiative "Robotique centrée sur l'humain" of the Occitanie region.
Jean-Marc Salotti is participating as an expert in the "Ecosystème d'Excellence Economie de la Donnée / IA (eDIA)" du pôle Aerospace Valley, which gathers all regional actors concerned with artificial intelligence mainly in the aerospace sector, but not limited to that domain.
Jean-Marc Salotti is member of the Commission des Emplois de Recherche de Inria Bordeaux Sud-Ouest (CER-BSO). He reviewed several PhD, Post-doc and "délégation" Applications in preparation of deliberations.
The Auctus team is involved in the “Aquitaine robotics” cluster, which brings together robotics players in Nouvelle-Aquitaine. David Daney and Jean-Marc Salotti respectively represent Inria and Ensc on the board of directors. David Daney is a member of the executive board. David Daney and Jean-Marc Salotti are respectively president and vice-president of the labelling committee which promotes all robotics projects for the Nouvelle-Aquitaine region.
11.1.6 Research administration
The Auctus team is involved in R4, a regional robotics network involving 12 research entities in the region of Nouvelle-Aquitaine, France. David Daney is a member of the executive board.
Vincent Padois is since september 2022 head of the “Commission des Développements Technologiques” for the Inria Bordeaux research center.
Jean-Marc Salotti is member of the thesis and postdoc comittee for the Inria Bordeaux research center.
A new website has been build to help disseminating the news related to the team. It is build around the software HUGO which simplifies the way content is added to the website. It is used to publish general information about the team, but also new scientific results and job offers.
11.2 Teaching - Supervision - Juries
- Master: Jean-Marc Salotti , Bases de l'intelligence artificielle (ENSC 2A), 40h eqTD.
- Master: Jean-Marc Salotti , Apprentissage Automatique (ENSC 2A), 25h eqTD.
- Master: Jean-Marc Salotti , Interactions Humains Robots (ENSC 3A), 32h eqTD.
- Master: Jean-Marc Salotti , Facteurs Humains et Ingénierie Cognitique (ENSC 3A), 28h eqTD.
- Master: Jean-Marc Salotti , supervision of projects and internships (ENSC 1A, 2A, 3A)and jury for oral presentations, 100h eqTD.
- Licence: Nasser Rezzoug , Biomécanique, 60h éqTD, L1, UFR STAPS, Université de Toulon, France.
- Licence: Nasser Rezzoug , Biomécanique, 10.5h éqTD, L1, Faculté des Sciences du Sport (FSS), Université de Poitiers, France.
- Licence: Nasser Rezzoug , Biomécanique, 34.5 h éqTD, L2, FSS, Université de Poitiers, France.
- Master: Nasser Rezzoug , Biomécanique, 25h éqTD, M1, UFR Sciences and Tech,Master ISC parcours Robotique et Objets Connectés, Université de Toulon, France.
- Master: Nasser Rezzoug , Ergonomie et interaction homme poste de travail, 19h éqTD, M1, FSS Master APAS/IRHPM et UFR SFA Master Ingénierie biomécanique, Université de Poitiers, France.
- Master: Nasser Rezzoug , Modalités de prescription de l'activité physique, 16h éqTD, M1, FSS Master APAS/IRHPM, Université de Poitiers, France.
- Master: Nasser Rezzoug , Applications biomédicales des objets connectés, 6h éqTD, M2, UFR Sciences and Tech, Master ISC parcours Robotique et Objets Connectés, Université de Toulon, France.
- Master: David Daney , Interactions Humains Robots, 6h eqTD, M2, Ecole Nationale Supérieure de Cognitique / Bordeaux INP, France.
- Master: David Daney , Mathématiques pour la robotique, 30h eqTD, M2, Enseirb/Ensc, Bordeaux INP, France.
- Master: Vincent Padois , Literature review - What, Why and How?, 20h éqTD, M2, Enseirb/Ensc, Bordeaux INP, France.
- Master: Vincent Padois , Maintenance du futur - Cours introductif, 2h éqTD, M1, ENSPIMA, Bordeaux INP, France.
- Master: Vincent Padois , Maintenance du futur - Inrtroduction à la Robotique, 10h éqTD, M2, ENSPIMA, Bordeaux INP, France.
- Master: Vincent Padois , Introduction à la Robotique, 2h éqTD, M1, Université de Bordeaux, France.
- Master: Antun Skuric, Mathématiques et Informatique (Intermédiaire), 30h éq TD, M1, ENSAM Bordeaux, Arts et Métiers ParisTech, France.
- Master: Antun Skuric, Mathématiques et Informatique (Avancé), 30h éq TD, M2, ENSAM Bordeaux, Arts et Métiers ParisTech, France.
- Master: Sébastien Dignoire, Projets de Robotique Mobile et introduction ROS, 8h éq TD, M2, Polytech Sorbonne Université, Paris, France.
- Master: Guillaume de Mathelin, IA & Robotics: Planning,10h, M2 , Enseirb/ENSC, Bordeaux INP, France.
- Licence: Gautier Laisné, Biomécanique, 31.5h éqTD, L1 STAPS, Université de Poitiers.
- Master: Gautier Laisné, Projet Informatique Individuel, 8h éqTD, M1, École Nationale Supérieure de Cognitique.
- Master: Gautier Laisné, Bases de l'intelligence artificielle, 15h éqTD, M1, École Nationale Supérieure de Cognitique.
- Master: Gautier Laisné, Interactions humaines-robots et architectures cognitives, 12h éqTD, M2, École Nationale Supérieure de Cognitique.
- Master: Gautier Laisné, Projet de fin d'études, 9h éqTD, M2, École Nationale Supérieure de Cognitique.
- Master: Benjamin Camblor, Fonction cognitives en situation et Handicap, 22,5h éqTD, M1 (S7), Master de Sciences cognitives et Ergonomie, Université de Bordeaux, France.
- Master: Benjamin Camblor, Supervision of projects, 4h éqTD, ENSC 2A, Bordeaux INP, France.
- Licence : Benjamin Camblor, TER (supervision of projects), 5h éqTD, L3 (S6), Licence MIASHS, Université de Bordeaux, France.
- Licence : Benjamin Camblor, Facteurs Humains et Ergonomie, 21,75h éqTD, L2 (S3), Licence MIASHS, Université de Bordeaux, France.
- Licence: Jessica Colombel, Algorithmie, 22h éq TD, L1, Institut des sciences du Digital, Management Cognition, Nancy, France
- Licence: Jessica Colombel, Anatomie et fonctionnalité du corps humain, 30 éq TD, L2, Institut des sciences du Digital, Management Cognition, Nancy, France
- Licence: Jessica Colombel, Algorithmie Avancée, 32h éq TD, L2, Institut des sciences du Digital, Management Cognition, Nancy, France
- PhD in progress: Nassim Benhabib (Inria / Région NA – Woobot project), “Méthodologie de conception et de commande d’un système robotique collaboratif pour assister et sécuriser les gestes d’un opérateur”, November 2018 – , David Daney and Vincent Padois .
- PhD in progress: Pierre Laguillaumie (Université de Poitiers), “Pierre Laguillaumie (2018 - to date) “Methodology for the implementation of a new generation of collaborative robots taking into account the safety and the biomechanical comfort of the operator in a work situation”, November 2018 – , Jean-Pierre Gazeau and Vincent Padois .
- PhD in progress: Benjamin Camblor, Situation Awareness in Collaborative Robotics, funding: ANR Pacbot, January 2020 - Jean-Marc Salotti .
- PhD in progress: Nicolas Torres (Thèse ED SPI, Bordeaux / CIFRE PSA), “Synthesis and dynamic analysis of the shared workspace for safety in collaborative robotics”, April 2020 – Vincent Padois and David Daney .
- PhD in progress: Antun Skuric (Thèse ED SPI, Bordeaux / Financement projet Lichie Airbus), “A coupled view of the physical abilities of human-robot dyad for the online quantitative evaluation of assistance needs”, July 2020 – Vincent Padois and David Daney .
- PhD in progress: Raphaël Bousigues (Thèse ED SPI, Bordeaux / INRS-Inria), “Motor variability in human-robot collaboration”, December 2020 – Pauline Maurice, Vincent Padois , David Daney and Jonathan Savin.
- PhD in progress: Erwann Landais (MIELS project, Région NA), “Approche robotique pour la formulation de nouveaux solvants”,Sept 2021 –, Vincent Padois and Nasser Rezzoug .
- PhD in progress: Gautier Laisné (Inria), "Musculoskeletal models and data driven learning for personalized human force capacities evaluation", October 2021 -, Nasser Rezzoug and Jean-Marc Salotti .
- PhD in progress: Alexis Boulay (Farm3, Cifre), "Assister l’humain par un transfert de compétences au robot en agriculture verticale téléopérée", June 2022 -, David Daney and Margot Vulliez .
- PhD in progress: Virgile Batto (CNRS), "Intelligence artificielle pour la co-conception de nouveaux robots dynamiques à pattes : une approche de conception multidisciplinaire et générique", October 2022 -, Nicolas Mansard (LAAS-CNRS), Thomas Flayols (LAAS-CNRS) and Margot Vulliez .
- PhD in progress: Elio Jabbour (Inria), "Shared-autonomy control for improving Human-Robot collaborarion in haptic teleoperation", funding: ANR ASAP-HRC, October 2022-, Margot Vulliez , Jean-Pierre Gazeau (Pprime Institute CNRS) and Vincent Padois .
- PhD cancelled: Julien Passama (AKKA Cifre), "Étude du robot comme vecteur de l’expression de l’expertise dans une tâche collaborative à distance", January 2021 - September 2022, Jean-Marc Salotti and Vincent Padois .
- PhD passed: Jessica Colombel (Inria), “Analyse du mouvement humain pour l’assistance à la personne”, February 2019 – Dec 2022, François Charpillet and David Daney (Inria Nancy Grand-Est).
- MS Thesis: Gianmarco Panzetta, "Human upper-limb workspace computation by interval analysis for ergonomics", Sept 2021 - Feb 2022, David Daney and Nasser Rezzoug .
- MS Thesis: Clemente Donoso, "Identification of musculoskeletal models for an efficient human/robot collaboration", May 2022 - Aug 2022, Gautier Laisné and Nasser Rezzoug .
- Internship: Simon Gervaise "Utilisation de bibliothèque de vision et de perception pour une utilisation dans le domaine de la cobotique", May 2022 - Aug 2022, Benjamin Camblor and David Daney .
- Stanley Mugisha, Reviewer, "Motion strategies for a haptic interface with intermittent contacts to ensure safe human-robot interaction", École Centrale de Nantes, Damien Chablat, Christine Chevallereau, 06/09/2022.
- Abderahmane Bedouhene, Reviewer,"Interval Constraint Programming for Differential Dynamical Systems", École des Ponts ParisTech, Bertrand Neveu, Gilles Trombettoni, 13/12/2022.
- Jean-Baptiste Riccoboni, Reviewer,"Modélisation Dynamique d’Arborescences – application à la robotique et à l’étude du mouvement humain", Université de Poitiers, Antoine Eon, Tony Monnet,16/12/2022.
- Noëlie Ramuzat, Reviewer, "Robotics Force/Torque Control for Manufacturing Operations", Université de Toulouse, Olivier Stasse, 18/02/2022
- Médéric Fourmy, Reviewer, "State estimation and localization of legged robots: a tightly-coupled approach based on a-posteriori maximization", Université de Toulouse, Nicolas Mansard, Joan Solà, 21/03/2022
- Jimmy Da Silva, Examiner, "Automation of pedicle screw placement by coupling distal bone bio-impedance measurements and robotics", Sorbonne Université, Guillaume Morel, 09/09/2022
- Mohamed Djeha, Examiner, "Unified control/observers of complex multi-robot systems using multi-objectives quadratic programming with constraints, Université de Montpellier, Abderrahmane Kheddar, 09/09/2022
- Zheng Pu, Reviewer, "Towards safe robot arm motion close to humans", Université Grenoble Alpes, Olivier Aycard, Pierre-Brice Wieber, 12/12/2022
- Vincent Gibeaux, Reviewer, "Simulateur de mouvement pour l’accessibilité", Laboratoire de Biomécanique et Mécanique des Chocs, Université Claude Bernard Lyon 1, Raphael Dumas, Nicolas Pronost, 14/12/2023.
- PhD: Lissette Valdès "Methods and elements of graph theory and fuzzy logic for communication network management", University of Malaga, Spain, April 2022.
- PhD: Pol Hamon, Examinator, "Conception et contrôle d'un préhenseur sous-actionné pour la saisie d'objets complexes", École Centrale de Nantes, Damien Chablat, Franck Plestan, 2022/12/15.
- Jean-Marc Salotti participated in the "Jury de repyramidage" 27ème section, that has been held at Université de Tours in November 2022.
- Nasser Rezzoug participated in recruitment jury MCF-4463,Université Sorbonne Paris Nord, 11/05/2022
- Margot Vulliez participated in the jury CNRS Research Engineer BAP C n°21 - ISIR Paris, Oct. 2022.
11.3.1 Articles and contents
- Guillaume de Mathelin de Papigny was interviewed by a mathematical professor within the framework of a program aiming at promoting mathematics in high schools. The format of the interview was a short movie and the interview was made in the AUCTUS laboratory with our robots.
- Vincent Padois was interviwed by the online magazine Pixees regarding his education nad career path.
- Benjamin Camblor - "Conscience de situation et robotique collaborative". Cogtalk with Ascoergo, February, Bordeaux, 2022.
- Benjamin Camblor - "Table ronde forum des sciences cognitives de Bordeaux". Ascoergo-Fresco, March, Bordeaux, 2022.
- Jessica Colombel – Debator at the round table “Féru des Sciences”, Nancy
- Vincent Padois – "Classes Tranplantées Robotique", 2x 1h seminar of general sensibilization to Robotics and its complexity with two primary school classes in Merignac, in collaboration with Cap Sciences, June 2022
- Vincent Padois – "Un scientifique une classe – Chiche !", 1h seminar of general sensibilization to Robotics research with a middle school from La Réole, May 2022
- Vincent Padois – "Le bureau des Enquêtes", 2h intervention on Robotics within the framework of "Les Mardis des sciences" organized by Cap Sciences, Floirac, December 2022
12 Scientific production
12.1 Major publications
- 1 inproceedingsSecuring Industrial Operators with Collaborative Robots: Simulation and Experimental Validation for a Carpentry task.ICRA 2020 - IEEE International Conference on Robotics and AutomationParis, FranceMay 2020
- 2 miscTask-Consistent Signaling Motions for Improved Understanding in Human-Robot Interaction and Workspace Sharing.January 2022
- 3 articleHuman-oriented design of collaborative robots.International Journal of Industrial Ergonomics572017, 88--102
- 4 articleEvidence of movement variability patterns during a repetitive pointing task until exhaustion.Applied Ergonomics962021, 103464
- 5 articleOn-line feasible wrench polytope evaluation based on human musculoskeletal models: an iterative convex hull method.IEEE Robotics and Automation Letters2022
12.2 Publications of the year
International peer-reviewed conferences
Edition (books, proceedings, special issue of a journal)
Doctoral dissertations and habilitation theses
Reports & preprints
Other scientific publications
12.3 Cited publications
- 27 bookAgir sur la prévention durable des TMS.Anact2011
- 28 bookAgir sur Prévenir les risques psychosociaux.Anact2013
- 29 inproceedingsOn the role of robot configuration in Cartesian stiffness control.2015 IEEE International Conference on Robotics and Automation (ICRA)2015 IEEE International Conference on Robotics and Automation (ICRA)ISSN: 1050-4729May 2015, 1010--1016
- 30 articleJoint action with a virtual robotic vs. human agent.Cognitive Systems Research522018, 816--827
- 31 articleThe central nervous system stabilizes unstable dynamics by learning optimal impedance.4146862November 2001, 446--449URL: http://www.nature.com/articles/35106566
- 32 inproceedingsThe Pinocchio C++ library: A fast and flexible implementation of rigid body dynamics algorithms and their analytical derivatives.2019 IEEE/SICE International Symposium on System Integration (SII)IEEE2019, 614--619
- 33 articleMeasurement of situation awareness in dynamic systems.Human factors3711995, 65--84
- 34 articleHumans integrate visual and haptic information in a statistically optimal fashion.Nature41568702002, 429--433
- 35 incollectionPhysical Human-Robot Interaction.Handbook of RoboticsSpringer Verlag2016, 1835--1874
- 36 articleRapid Entire Body Assessment (REBA).Applied Ergonomics312April 2000, 201--205URL: http://www.sciencedirect.com/science/article/pii/S0003687099000393
- 37 articleBiomechanical analysis of upper limb during the use of touch screen: motion strategies identification.Ergonomics603March 2017, 358--365URL: http://dx.doi.org/10.1080/00140139.2016.1175671
- 38 inproceedingsTowards X-ray medical imaging with robots in the open: safety without compromising performances.Proceedings of the IEEE International Conference on Robotics and AutomationBrisbane, AustraliaMay 2018, 6604--6610URL: https://hal.archives-ouvertes.fr/hal-01614508/en
- 39 articleCorrecting working postures in industry: A practical method for analysis.Applied Ergonomics84December 1977, 199--201URL: http://www.sciencedirect.com/science/article/pii/0003687077901648
- 40 articleLUBA: an assessment technique for postural loading on the upper body based on joint motion discomfort and maximum holding time.Applied Ergonomics324August 2001, 357--366URL: http://www.sciencedirect.com/science/article/pii/S0003687001000060
- 41 phdthesisConception d'une Interface de Pilotage d'un Cobot.Université Pierre et Marie Curie - Paris VIMarch 2011
- 42 articleGeneralized hierarchical control.Autonomous Robots4012016, 17--31
- 43 articleDiversity and variation in biomechanical exposure: What is it, and why would we like to know?Applied Ergonomics374Special Issue: Meeting Diversity in Ergonomics2006, 419-427URL: http://www.sciencedirect.com/science/article/pii/S0003687006000482
- 44 articleRULA: a survey method for the investigation of work-related upper limb disorders.Applied ergonomics2421993, 91--99
- 45 incollectionEnergy-based control for safe Human-robot physical interactions.Springer Proceedings in Advanced Robotics - The 2016 International Symposium on Experimental RoboticsSpringer International Publishing AG2017, URL: http://hal.archives-ouvertes.fr/hal-01398790/en
- 46 inbookAppropriate Design of Parallel Manipulators.Smart Devices and Machines for Advanced ManufacturingL.Lihui WangJ.Jeff XiLondonSpringer London2008, 1--25URL: https://doi.org/10.1007/978-1-84800-147-3_1
- 47 articleControl sharing in human-robot team interaction.Annual Reviews in Control442017, 342--354
- 48 inproceedingsCoordination mechanisms in human-robot collaboration.Proceedings of the Workshop on Collaborative Manipulation, 8th ACM/IEEE International Conference on Human-Robot InteractionCiteseer2013, 1--6
- 49 articleDesign Strategy of Serial Manipulators With Certified Constraint Satisfaction.IEEE Transactions on Robotics251Feb 2009, 1-11URL: http://dx.doi.org/10.1109/TRO.2008.2006867
- 50 bookInverse optimal control for redundant systems of biological motion.OrlÃ©ansDecember 2015, URL: http://www.theses.fr/2015ORLE2050
- 51 articleTeaching Robots to Cooperate with Humans in Dynamic Manipulation Tasks Based on Multi-modal Human-in-the-loop Approach.Autonomous Robots361-2January 2014, 123--136URL: http://dx.doi.org/10.1007/s10514-013-9361-0
- 52 articleRobot adaptation to human physical fatigue in human–robot co-manipulation.Autonomous Robots425June 2018, 1011--1021URL: http://dx.doi.org/10.1007/s10514-017-9678-1
- 53 articleSituation awareness in human-robot interaction: Challenges and user interface requirements.Human-Robot Interactions in Future Military Operations2010, 171--192
- 54 articleEvidence of movement variability patterns during a repetitive pointing task until exhaustion.Applied Ergonomics962021, 103464
- 55 inproceedingsMovement Variability and Digital Human Models: Development of a Demonstrator Taking the Effects of Muscular Fatigue into Account.Advances in Applied Digital Human Modeling and SimulationChamSpringer International Publishing2017, 169--179
- 56 inproceedingsCommon wrench capability evaluation of a human-robot collaborative system.46th Congrès de la Société de BiomécaniqueSaint Etienne, FranceOctober 2021
- 57 articleDevelopment and evaluation of an office ergonomic risk checklist: ROSA - Rapid Office Strain Assessment.Applied Ergonomics431January 2012, 98--108URL: http://www.sciencedirect.com/science/article/pii/S0003687011000433
- 58 articleMotor variability in occupational health and performance.Clinical Biomechanics27102012, 979-993URL: http://www.sciencedirect.com/science/article/pii/S0268003312001817
- 59 articlePerformance analysis and design of parallel kinematic machines using interval analysis.Mechanism and Machine Theory1152017, 218-236URL: http://www.sciencedirect.com/science/article/pii/S0094114X17305700
- 60 bookEvaluation of Human Work, Fourth Edition.Google-Books-ID: uXB3CAAAQBAJCRC PressApril 2015
- 61 articleAn Intuitive Formulation of the Human Arm Active Endpoint Stiffness.2018January 2020, 5357URL: https://www.mdpi.com/1424-8220/20/18/5357