7.1.1 rlberry

• Keywords:
  Reinforcement learning, Simulation, Artificial intelligence
• Scientific Description:
  RL-berry is a software library for reinforcement learning. One of its major features is to give access to state-of-the-art RL algorithm implementations in a very simple way. One of the goal is that RL-berry would be used for teaching. We also provide original features that are missing from other RL libraries. One concerns the methodology to compare the experimental performance of various algorithms.
• Functional Description:
  rlberry is a reinforcement learning (RL) library in Python for research and education. The library provides implementations of several RL agents for you to use as a starting point or as baselines, provides a set of benchmark environments, very useful to debug and challenge your algorithms, handles all random seeds for you, ensuring reproducibility of your results, and is fully compatible with several commonly used RL libraries like OpenAI gym and Stable Baselines.
• URL:
  https://­github.­com/­rlberry-py/­rlberry
• Contact:
  Timothee Mathieu

7.1.2 Weight Trajectory Predictor : algorithm

• Name:
  Weight Trajectory Predictor : algorithm
• Keywords:
  Medical applications, Machine learning
• Scientific Description:
  We performed a retrospective study of clinical data collected prospectively on patients with up to five years postoperative follow-up (ABOS cohort, CHU Lille) and trained a supervised model to predict the relative total weight loss (“%TWL”) of a patient 1, 3, 12, 24 and 60 months after surgery. This model consists in a decision tree, written in python, taking as input a selected subset of preoperative attributes (weight, height, type of intervention, age, presence or absence of type 2 diabetes or impaired glucose tolerance, diabetes duration, smoking habits) and returns an estimation of %TWL as well as a prediction interval based on the interquartile range of %TWL observed on similar patients. The predictions of this tool have been validated both internally and externally (on French and Dutch cohorts).
• Functional Description:

  The “Weight Tracjectory Predictor” algorithm is part of a larger project, whose goal is to leverage artificial intelligence techniques to improve patient care. This code is the product of a collaboration between Inria SCOOL and the UMR 1190-EGID team of the CHU Lille. It aims to predict the weight loss trajectory of a patient following bariatric surgery (treatment of severe obesity) from a set of preoperative characteristics.

  We performed a retrospective study of clinical data collected prospectively on patients with up to five years postoperative follow-up (ABOS cohort, CHU Lille) and trained a supervised model to predict the relative total weight loss (“%TWL”) of a patient 1, 3, 12, 24 and 60 months after surgery. This model consists in a decision tree, written in python, taking as input a selected subset of preoperative attributes (weight, height, type of intervention, age, presence or absence of type 2 diabetes or impaired glucose tolerance, diabetes duration, smoking habits) and returns an estimation of %TWL as well as a prediction interval based on the interquartile range of %TWL observed on similar patients. The predictions of this tool have been validated both internally and externally (on French and Dutch cohorts).

  The goal of this software is to improve patient follow-up after bariatric surgery: - during preoperative visits, by providing clinicians with a quantitative tool to inform the patient regarding potential weight loss outcome. - during postoperative control visits, by comparing the predicted and realized weight trajectories, which may facilitate early detection of complications.

  This software component will be embedded in a web app for ease of use.

• Release Contributions:
  Initial version
• URL:
  https://­bariatric-weight-trajectory-prediction.­univ-lille.­fr/
• Contact:
  Julien Teigny
• Participants:
  Pierre Bauvin, Francois Pattou, Philippe Preux, Violeta Raverdy, Patrick Saux, Tomy Soumphonphakdy, Julien Teigny, Hélène Verkindt
• Partner:
  CHU de Lille

7.1.3 Adastop

• Keywords:
  Hypothesis testing, Reinforcement learning, Reproducibility
• Functional Description:
  This package contains the AdaStop algorithm. AdaStop implements a statistical test to adaptively choose the number of runs of stochastic algorithms necessary to compare these algorithms and be able to rank them with a theoretically controlled family-wise error rate. One particular application for which AdaStop was created is to compare Reinforcement Learning algorithms. Please note, that what we call here an algorithm is really a certain implementation of an algorithm.
• URL:
  https://­github.­com/­TimotheeMathieu/­adastop
• Contact:
  Timothee Mathieu

7.1.4 average-reward-reinforcement-learning

• Keywords:
  Mutli-armed bandits, Reinforcement learning, Python
• Functional Description:
  Library of RL and Bandit algorithms.
• URL:
  https://­gitlab.­inria.­fr/­omaillar/­average-reward-reinforcement-learning
• Contact:
  Odalric-Ambrym Maillard
• Participant:
  Odalric-Ambrym Maillard

7.1.5 FarmGym

• Name:
  Farming Environment Gym factory for Reinforcement Learning
• Keywords:
  Reinforcement learning, Simulator, Agroecology
• Functional Description:
  Farming Environment Gym factory for Reinforcement Learning
• Release Contributions:
  This version is an entire rewriting by Odlaric-Ambrym Maillard of the prototype V1 created by Thomas Carta. Version V2 features modular creation of farms, specifications of various entities and monitoring facilities. Recent additions include slight adjustment of the entities dynamics. It also features unit tests and an automatic generation of base policies done by Brahim Driss.
• News of the Year:

  Nov 2022: Déploiement de FarmGym sur gitlab, auparavant en développement interne par Odalric-Ambrym Maillard (2021-2022). Nov-Dec 2022: Organisation de compétition interne par Timothée Mathieu.

  Printemps 2023: Arrivée de Brahim Driss sur le projet, pour mettre en place tests unitaires/fonctionnels et assister Odalric-Ambrym Maillard dans le développement de fonctionnalité.

• URL:
  https://­github.­com/­farm-gym/
• Publication:
  hal-03960683
• Contact:
  Odalric-Ambrym Maillard
• Participants:
  Odalric-Ambrym Maillard, Brahim Driss, Timothee Mathieu
• Partner:
  Inria

7.1.6 Dyjest algorithms

• Name:
  Dyjest Algorithms Toolbox
• Keywords:
  NLP, Statistic analysis
• Scientific Description:
  Smart Tracker uses a combination of Named Entity Recognition (NER) and Language Model Recognition (LLM) methods to structure user input. This data is then transformed into embeddings and processed by semantic search algorithms to find the closest matches in a database of symptoms, foods and activities. Finally, a report generator that presents the results of statistical methods for identifying potential symptom triggers.
• Functional Description:
  This code repository includes work carried out on the following aspects: - Streamlining the tracking of meals, activities, and symptoms through the entry of unstructured textual data. - Using embeddings with fine-tuning to improve the matching of inputs. - Integrating a hierarchical structure (tree) to enhance the robustness of semantic search. - Generating personalized reports. - Exploring symptom prediction (inconclusive) and applying multi-armed bandits to food recommendations.
• Contact:
  Medhi Douch
• Participants:
  Hatim Mrabet, Naafi Dasana Ibrahim, Emilie Kaufmann, Medhi Douch

8.2.1 Linear constraints

Pure Exploration in Bandits with Linear Constraints, 21

We address the problem of identifying the optimal policy with a fixed confidence level in a multi-armed bandit setup, when the arms are subject to linear constraints. Unlike the standard best-arm identification problem which is well studied, the optimal policy in this case may not be deterministic and could mix between several arms. This changes the geometry of the problem which we characterize via an information-theoretic lower bound. We introduce two asymptotically optimal algorithms for this setting, one based on the Track-and-Stop method and the other based on a game-theoretic approach. Both these algorithms try to track an optimal allocation based on the lower bound and computed by a weighted projection onto the boundary of a normal cone. Finally, we provide empirical results that validate our bounds and visualize how constraints change the hardness of the problem.

Learning to Explore with Lagrangians for Bandits under Unknown Linear Constraints, 30

Pure exploration in bandits models multiple real-world problems, such as tuning hyper-parameters or conducting user studies, where different safety, resource, and fairness constraints on the decision space naturally appear. We study these problems as pure exploration in multi-armed bandits with unknown linear constraints, where the aim is to identify an $r$$-\mathrm{𝑔𝑜𝑜𝑑𝑓𝑒𝑎𝑠𝑖𝑏𝑙𝑒𝑝𝑜𝑙𝑖𝑐𝑦}$. First, we propose a Lagrangian relaxation of the sample complexity lower bound for pure exploration under constraints. We show how this lower bound evolves with the sequential estimation of constraints. Second, we leverage the Lagrangian lower bound and the properties of convex optimisation to propose two computationally efficient extensions of Track-and-Stop and Gamified Explorer, namely LATS and LAGEX. To this end, we propose a constraint-adaptive stopping rule, and while tracking the lower bound, use pessimistic estimate of the feasible set at each step. We show that these algorithms achieve asymptotically optimal sample complexity upper bounds up to constraint-dependent constants. Finally, we conduct numerical experiments with different reward distributions and constraints that validate efficient performance of LAGEX and LATS with respect to baselines.

8.2.2 Robustness

CRIMED: Lower and Upper Bounds on Regret for Bandits with Unbounded Stochastic Corruption, 18

We investigate the regret-minimisation problem in a multi-armed bandit setting with arbitrary corruptions. Similar to the classical setup, the agent receives rewards generated independently from the distribution of the arm chosen at each time. However, these rewards are not directly observed. Instead, with a fixed $\epsilon \in (0,\frac{1}{2})$, the agent observes a sample from the chosen arm's distribution with probability $1-\epsilon$, or from an arbitrary corruption distribution with probability $\epsilon$. Importantly, we impose no assumptions on these corruption distributions, which can be unbounded. In this setting, accommodating potentially unbounded corruptions, we establish a problem-dependent lower bound on regret for a given family of arm distributions. We introduce CRIMED, an asymptotically-optimal algorithm that achieves the exact lower bound on regret for bandits with Gaussian distributions with known variance. Additionally, we provide a finite-sample analysis of CRIMED's regret performance. Notably, CRIMED can effectively handle corruptions with $\epsilon$ values as high as $\frac{1}{2}$. Furthermore, we develop a tight concentration result for medians in the presence of arbitrary corruptions, even with $\epsilon$ values up to $\frac{1}{2}$, which may be of independent interest. We also discuss an extension of the algorithm for handling misspecification in Gaussian model.

Bandits Corrupted by Nature: Lower Bounds on Regret and Robust Optimistic Algorithms, 14

We study the Bandits with Stochastic Corruption problem, i.e. a stochastic multi-armed bandit problem with $k$ unknown reward distributions, which are heavy-tailed and corrupted by a history-independent stochastic adversary or Nature. To be specific, the reward obtained by playing an arm comes from corresponding heavy-tailed reward distribution with probability $1-\epsilon \in (0.5,1]$ and an arbitrary corruption distribution of unbounded support with probability $\epsilon \in [0,0.5)$. First, we provide a problem-dependent lower bound on the regret of any corrupted bandit algorithm. The lower bounds indicate that the Bandits with Stochastic Corruption problem is harder than the classical stochastic bandit problem with sub-Gaussian or heavy-tail rewards. Following that, we propose a novel UCB-type algorithm for Bandits with Stochastic Corruption, namely HuberUCB, that builds on Huber's estimator for robust mean estimation. Leveraging a novel concentration inequality of Huber's estimator, we prove that HuberUCB achieves a near-optimal regret upper bound. Since computing Huber's estimator has quadratic complexity, we further introduce a sequential version of Huber's estimator that exhibits linear complexity. We leverage this sequential estimator to design SequentialHuberUCB that enjoys similar regret guarantees while reducing the computational burden. Finally, we experimentally illustrate the efficiency of HuberUCB and SequentialHuberUCB in solving Bandits with Stochastic Corruption for different reward distributions and different levels of corruptions.

8.2.3 Privacy

Concentrated Differential Privacy for Bandits, 19

Bandits serve as the theoretical foundation of sequential learning and an algorithmic foundation of modern recommender systems. However, recommender systems often rely on user-sensitive data, making privacy a critical concern. This paper contributes to the understanding of Differential Privacy (DP) in bandits with a trusted centralised decision-maker, and especially the implications of ensuring zero Concentrated Differential Privacy (zCDP). First, we formalise and compare different adaptations of DP to bandits, depending on the considered input and the interaction protocol. Then, we propose three private algorithms, namely AdaC-UCB, AdaC-GOPE and AdaC-OFUL, for three bandit settings, namely finite-armed bandits, linear bandits, and linear contextual bandits. The three algorithms share a generic algorithmic blueprint, i.e. the Gaussian mechanism and adaptive episodes, to ensure a good privacy-utility trade-off. We analyse and upper bound the regret of these three algorithms. Our analysis shows that in all of these settings, the prices of imposing zCDP are (asymptotically) negligible in comparison with the regrets incurred oblivious to privacy. Next, we complement our regret upper bounds with the first minimax lower bounds on the regret of bandits with zCDP. To prove the lower bounds, we elaborate a new proof technique based on couplings and optimal transport. We conclude by experimentally validating our theoretical results for the three different settings of bandits.

FLIPHAT: Joint Differential Privacy for High Dimensional Sparse Linear Bandits, 38

High dimensional sparse linear bandits serve as an efficient model for sequential decision-making problems (e.g. personalized medicine), where high dimensional features (e.g. genomic data) on the users are available, but only a small subset of them are relevant. Motivated by data privacy concerns in these applications, we study the joint differentially private high dimensional sparse linear bandits, where both rewards and contexts are considered as private data. First, to quantify the cost of privacy, we derive a lower bound on the regret achievable in this setting. To further address the problem, we design a computationally efficient bandit algorithm, ForgetfuLIterative Private HArd Thresholding (FLIPHAT). Along with doubling of episodes and episodic forgetting, FLIPHAT deploys a variant of Noisy Iterative Hard Thresholding (N-IHT) algorithm as a sparse linear regression oracle to ensure both privacy and regret-optimality. We show that FLIPHAT achieves optimal regret up to logarithmic factors. We analyze the regret by providing a novel refined analysis of the estimation error of N-IHT, which is of parallel interest.

Differentially Private Best-Arm Identification, 35

Best Arm Identification (BAI) problems are progressively used for data-sensitive applications, such as designing adaptive clinical trials, tuning hyper-parameters, and conducting user studies. Motivated by the data privacy concerns invoked by these applications, we study the problem of BAI with fixed confidence in both the local and central models, i.e. $ϵ$-local and $ϵ$-global Differential Privacy (DP). First, to quantify the cost of privacy, we derive lower bounds on the sample complexity of any $\delta$-correct BAI algorithm satisfying $ϵ$-global DP or $ϵ$-local DP. Our lower bounds suggest the existence of two privacy regimes. In the high-privacy regime, the hardness depends on a coupled effect of privacy and novel information-theoretic quantities involving the Total Variation. In the low-privacy regime, the lower bounds reduce to the non-private lower bounds. We propose $ϵ$-local DP and $ϵ$-global DP variants of a Top Two algorithm, namely CTB-TT and AdaP-TT*, respectively. For $ϵ$-local DP, CTB-TT is asymptotically optimal by plugging in a private estimator of the means based on Randomised Response. For $ϵ$-global DP, our private estimator of the mean runs in arm-dependent adaptive episodes and adds Laplace noise to ensure a good privacy-utility trade-off. By adapting the transportation costs, the expected sample complexity of AdaP-TT* reaches the asymptotic lower bound up to multiplicative constants.

Open Problem: What is the Complexity of Joint Differential Privacy in Linear Contextual Bandits?, 20

Contextual bandits serve as a theoretical framework to design recommender systems, which often rely on user-sensitive data, making privacy a critical concern. However, a significant gap remains between the known upper and lower bounds on the regret achievable in linear contextual bandits under Joint Differential Privacy (JDP), which is a popular privacy definition used in this setting. In particular, the best regret upper bound is known to be $O\left(d\sqrt{T}log\left(T\right)+blue{d}^{3/4}\sqrt{Tlog(1/\delta )}/\sqrt{ϵ}\right)$, while the lower bound is $\Omega \left(\sqrt{dTlog\left(K\right)}+blued/(ϵ+\delta )\right)$. We discuss the recent progress on this problem, both from the algorithm design and lower bound techniques, and posit the open questions.

8.2.4 Multi-fidelity feedback

Optimal Multi-Fidelity Best-Arm Identification, 25

In bandit best-arm identification, an algorithm is tasked with finding the arm with highest mean reward with a specified accuracy as fast as possible. We study multi-fidelity best-arm identification, in which the algorithm can choose to sample an arm at a lower fidelity (less accurate mean estimate) for a lower cost. Several methods have been proposed for tackling this problem, but their optimality remain elusive, notably due to loose lower bounds on the total cost needed to identify the best arm. Our first contribution is a tight, instance-dependent lower bound on the cost complexity. The study of the optimization problem featured in the lower bound provides new insights to devise computationally efficient algorithms, and leads us to propose a gradient-based approach with asymptotically optimal cost complexity. We demonstrate the benefits of the new algorithm compared to existing methods in experiments. Our theoretical and empirical findings also shed light on an intriguing concept of optimal fidelity for each arm.

8.4.1 Statistical reproductibility in practice

Statistical comparison in empirical computer science with minimal computation usage, 32

The replicability of computational experiments remains a fundamental question. Very often, computational experiments are meant to compare the performance of different algorithms on a given task or on a given set of tasks. For example, the machine learning community has recently become aware of the poor replicability of many experimental ML studies. Among other reasons, the large computational cost of some of these experiments, notably in reinforcement learning and in large language models, makes most of these experiments non replicable. This cost prompts research towards comparison methods that require as few computations as possible to obtain a replicable conclusion. Aside the computational cost, the conclusion of the comparison should also be replicable which calls for appropriate statistical tests. AdaStop is a recently introduced statistical test based on multiple group sequential tests. AdaStop adapts the number of executions of each experiment to stop as early as possible while ensuring that enough information is available to distinguish algorithms that perform better than the others in a statistically significant way. AdaStop has been initially exemplified on reinforcement learning tasks. In this short paper, we give new use cases of AdaStop beyond reinforcement learning to provide the computer science community with a tool that can be used to compare algorithms in any domain requiring computational studies.

8.4.2 Algorithmic auditing

Active Fourier Auditor for Estimating Distributional Properties of ML Models, 29

With the pervasive deployment of Machine Learning (ML) models in real-world applications, verifying and auditing properties of ML models have become a central concern. In this work, we focus on three properties: robustness, individual fairness, and group fairness. We discuss two approaches for auditing ML model properties: estimation with and without reconstruction of the target model under audit. Though the first approach is studied in the literature, the second approach remains unexplored. For this purpose, we develop a new framework that quantifies different properties in terms of the Fourier coefficients of the ML model under audit but does not parametrically reconstruct it. We propose the Active Fourier Auditor (AFA), which queries sample points according to the Fourier coefficients of the ML model, and further estimates the properties. We derive high probability error bounds on AFA's estimates, along with the worst-case lower bounds on the sample complexity to audit them. Numerically we demonstrate on multiple datasets and models that AFA is more accurate and sample-efficient to estimate the properties of interest than the baselines.

How Much Does Each Datapoint Leak Your Privacy? Quantifying the Per-datum Membership Leakage, 34

We study the per-datum Membership Inference Attacks (MIAs), where an attacker aims to infer whether a fixed target datum has been included in the input dataset of an algorithm and thus, violates privacy. First, we define the membership leakage of a datum as the advantage of the optimal adversary targeting to identify it. Then, we quantify the per-datum membership leakage for the empirical mean, and show that it depends on the Mahalanobis distance between the target datum and the data-generating distribution. We further assess the effect of two privacy defences, i.e. adding Gaussian noise and sub-sampling. We quantify exactly how both of them decrease the per-datum membership leakage. Our analysis builds on a novel proof technique that combines an Edgeworth expansion of the likelihood ratio test and a Lindeberg-Feller central limit theorem. Our analysis connects the existing likelihood ratio and scalar product attacks, and also justifies different canary selection strategies used in the privacy auditing literature. Finally, our experiments demonstrate the impacts of the leakage score, the sub-sampling ratio and the noise scale on the per-datum membership leakage as indicated by the theory.

Testing Credibility of Public and Private Surveys through the Lens of Regression, 37

Testing whether a sample survey is a credible representation of the population is an important question to ensure the validity of any downstream research. While this problem, in general, does not have an efficient solution, one might take a task-based approach and aim to understand whether a certain data analysis tool, like linear regression, would yield similar answers both on the population and the sample survey. In this paper, we design an algorithm to test the credibility of a sample survey in terms of linear regression. In other words, we design an algorithm that can certify if a sample survey is good enough to guarantee the correctness of data analysis done using linear regression tools. Nowadays, one is naturally concerned about data privacy in surveys. Thus, we further test the credibility of surveys published in a differentially private manner. Specifically, we focus on Local Differential Privacy (LDP), which is a standard technique to ensure privacy in surveys where the survey participants might not trust the aggregator. We extend our algorithm to work even when the data analysis has been done using surveys with LDP. In the process, we also propose an algorithm that learns with high probability the guarantees a linear regression model on a survey published with LDP. Our algorithm also serves as a mechanism to learn linear regression models from data corrupted with noise coming from any subexponential distribution. We prove that it achieves the optimal estimation error bound for $\ell \_1$ linear regression, which might be of broader interest. We prove the theoretical correctness of our algorithms while trying to reduce the sample complexity for both public and private surveys. We also numerically demonstrate the performance of our algorithms on real and synthetic datasets.

8.4.3 ML for Science and Optimization

Bistability in the sunspot cycle, 17

A direct dynamical test of the sunspot cycle is carried out to indicate that a stochastically forced nonlinear oscillator characterizes its dynamics. The sunspot series is then decomposed into its eigen time-delay coordinates. The relevant analysis reveals that the sunspot series exhibits bistability, with the possibility of modeling the solar cycle as a stochastically and periodically forced bistable oscillator, accounting for poloidal and toroidal modes of the solar magnetic field. Such a representation enables us to conjecture stochastic resonance as the key mechanism in amplifying the planetary influence on the Sun, and that extreme events, due to turbulent convection noise inside the Sun, dictate crucial phases of the sunspot cycle, such as the Maunder minimum.

The Steepest Slope toward a Quantum Few-body Solution, 16

Quantum few-body systems are deceptively simple. Indeed, with the notable exception of a few special cases, their associated Schrodinger equation cannot be solved analytically for more than two particles. One has to resort to approximation methods to tackle quantum few-body problems. In particular, variational methods have been proposed to ease numerical calculations and obtain precise solutions. One such method is the Stochastic Variational Method, which employs a stochastic search to determine the number and parameters of correlated Gaussian basis functions used to construct an ansatz of the wave function. Stochastic methods, however, face numerical and optimization challenges as the number of particles increases. We introduce a family of gradient variational methods that replace stochastic search with gradient optimization. We comparatively and empirically evaluate the performance of the baseline Stochastic Variational Method, several instances of the gradient variational method family, and some hybrid methods for selected few-body problems. We show that gradient and hybrid methods can be more efficient and effective than the Stochastic Variational Method. We discuss the role of singularities, oscillations, and gradient optimization strategies in the performance of the respective methods.

IDEQ: an improved diffusion model for the TSP, 36

We investigate diffusion models to solve the Traveling Salesman Problem. Building on the recent DIFUSCO and T2TCO approaches, we propose IDEQ (constrained Inverse Diffusion and EQuivalence class-based retraining of diffusion models for combinatorial optimization). IDEQ improves the quality of the solutions by leveraging the constrained structure of the state space of the TSP. Another key component of IDEQ consists in replacing the last stages of DIFUSCO curriculum learning by considering a uniform distribution over the Hamiltonian tours whose orbits by the 2-opt operator converge to the optimal solution as the training objective. Our experiments show that IDEQ improves the state of the art for such neural network based techniques on synthetic instances. More importantly, our experiments show that IDEQ performs very well on the instances of the TSPlib, a reference benchmark in the TSP community: it matches the performance of the best heuristics, LKH3, being even able to obtain better solutions than LKH3 on 2 instances of the TSPlib defined on 1577 and 3795 cities. IDEQ obtains 0.3% optimality gap on TSP instances made of 500 cities, and 0.5% on TSP instances with 1000 cities. This sets a new SOTA for neural based methods solving the TSP. Moreover, IDEQ exhibits a lower variance and better scales-up with the number of cities with regards to DIFUSCO and T2TCO.

8.4.4 Robustness in ML

When Witnesses Defend: A Witness Graph Topological Layer for Adversarial Graph Learning, 33

Capitalizing on the intuitive premise that shape characteristics are more robust to perturbations, we bridge adversarial graph learning with the emerging tools from computational topology, namely, persistent homology representations of graphs. We introduce the concept of witness complex to adversarial analysis on graphs, which allows us to focus only on the salient shape characteristics of graphs, yielded by the subset of the most essential nodes (i.e., landmarks), with minimal loss of topological information on the whole graph. The remaining nodes are then used as witnesses, governing which higher-order graph substructures are incorporated into the learning process. Armed with the witness mechanism, we design Witness Graph Topological Layer (WGTL), which systematically integrates both local and global topological graph feature representations, the impact of which is, in turn, automatically controlled by the robust regularized topological loss. Given the attacker's budget, we derive the important stability guarantees of both local and global topology encodings and the associated robust topological loss. We illustrate the versatility and efficiency of WGTL by its integration with five GNNs and three existing non-topological defense mechanisms. Our extensive experiments across six datasets demonstrate that WGTL boosts the robustness of GNNs across a range of perturbations and against a range of adversarial attacks, leading to relative gains of up to 18%.

8.4.5 Federated learning

Don't Forget What I did?: Assessing Client Contributions in Federated Learning, 39

Federated Learning (FL) is a collaborative machine learning (ML) approach, where multiple clients participate in training an ML model without exposing the private data. Fair and accurate assessment of client contributions is an important problem in FL to facilitate incentive allocation and encouraging diverse clients to participate in a unified model training. Existing methods for assessing client contribution adopts co-operative game-theoretic concepts, such as Shapley values, but under simplified assumptions. In this paper, we propose a history-aware game-theoretic framework, called FLContrib, to assess client contributions when a subset of (potentially non-i.i.d.) clients participate in each epoch of FL training. By exploiting the FL training process and linearity of Shapley value, we develop FLContrib that yields a historical timeline of client contributions as FL training progresses over epochs. Additionally, to assess client contribution under limited computational budget, we propose a scheduling procedure that considers a two-sided fairness criteria to perform expensive Shapley value computation only in a subset of training epochs. In experiments, we demonstrate a controlled trade-off between the correctness and efficiency of client contributions assessed via FLContrib. To demonstrate the benefits of history-aware client contributions, we apply FLContrib to detect dishonest clients conducting data poisoning in FL training.

10.1.1 Inria associate team not involved in an IIL or an international program

10.2.1 Visits of international scientists

10.2.2 Visits to international teams

10.3.1 Other european programs/initiatives

• Title:
  CausalXRL
• Partner Institutions:

   

  • University of Sheffield, Department of Computer Science
  • University of Vienna, Faculty of Computer Science
  • Inria, Scool, Lille, France
• Date/Duration:
  48 months
• Additionnal info/keywords:
  Causality, Reinforcement Learning, Explanations.

10.4.1 ANR projects

Scool is involved in 4 ANR projects:

• ANR JCJC FATE, PI: R. Degenne, 2023–2027
• ANR JCJC REPUBLIC, PI: D. Basu, 2023–2026
• ANR BIP-UP, partnership: Scool/Inserm (CHU de Lille), PI: Ph. Preux, 2023–2026.
• ANR JCJC NeuRL, PI: R. Akrour, 2024–2028

10.4.2 PEPR projects

Scool is involved in 2 PEPR:

• PEPR AI: project FOUNDRY, local head: E. Kaufmann (description below);
• PEPR « Agroécologie et numérique », Pl@ntAgroEco, local head: O.-A. Maillard.

• Title:
  FOUNDRY
• Duration:
  July 2024 $\to$ June 2028
• Coordinator:
  Panayotis Mertikopoulous, Polaris, Univ. Grenoble Alpes
• Partners:
  • POLARIS: a joint research team between the CNRS, Inria, and Univ. Grenoble Alpes.
  • ENS Lyon: faculty from the pure and applied mathematics department of ENS Lyon.
  • Inria FAIRPLAY: a joint team between Criteo, IP Paris (ENSAE and Ecole Polytechnique), and Inria.
  • LTCI: the informations and communications laboratory of Télécom Paris.
  • MILES: the machine intellligence and learning systems of the LAMSADE lab at Paris Dauphine.
  • Inria Scool
• Inria contact:
  Emilie Kaufmann
• Summary:
  From automated hospital admission systems powered by machine learning (ML), to flexible chatbots capable of fluent conversations and self-driving cars, the wildfire spread of artificial intelligence (AI) has brought to the forefront a crucial question with far-reaching ramifications for the society at large: Can ML systems and models be relied upon to provide trustworthy output in high-stakes, mission- critical environments? These questions invariably revolve around the notion of robustness, an operational desideratum that has eluded the field since its nascent stages. One of the main reasons for this is the fact that ML models and systems are typically data-hungry and highly sensitive to their training input, so they tend to be brittle, narrow-scoped, and unable to adapt to situations that go beyond their training envelope. On that account, the core vision of the proposed research is that robustness cannot be achieved by blindly throwing more data and computing power to larger and larger models with exponentially growing energy requirements (and a commensurate carbon footprint to boot). Instead, our proposal intends to rethink and develop the core theoretical and methodological FOUNDations of Robustness and reliabilitY (FOUNDRY) that are needed to build and instill trust in ML-powered technologies and systems from the ground up.

• Title:
  Pl@ntAgroEco
• Duration:
  July 2024 $\to$ June 2028
• Coordinator:
  Alexis Joly, Inria Zenith, and Pierre Bonnet CIRAD, AMAP.
• Partners:
  • INRAE
  • INRIA
  • IRD
  • CIRAD
  • Tela Botanica
  • Université de Montpellier
  • Université Paris-Saclay
• Inria contact:
  Odalric-Ambrym Maillard
• Summary:

  Agroecology necessarily involves crop diversification, but also the early detection of diseases, deficiencies and stresses (hydric, etc.), as well as better management of biodiversity. The main stumbling block is that this paradigm shift in agricultural practices requires expert skills in botany, plant pathology and ecology that are not generally available to those working in the field, such as farmers or agri-food technicians. Digital technologies, and artificial intelligence in particular, can play a crucial role in overcoming this barrier to access to knowledge.

  The aim of the Pl@ntAgroEco project will be to design, experiment with and develop new high-impact agro-ecology services within the Pl@ntNet platform. This includes :

  • research in AI and plant sciences ;
  • agile development of new components within the platform;
  • organization of participatory science programs and animation of the Pl@ntNet user community.

  Ce programme de travail a pour but de produire une amélioration de la détection et reconnaissance des maladies végétales, de l'identification des niveaux infraspécifiques. Il permettra le développement d'outils d'estimation de la sévérité des symptômes, carences, stades de déclin et stress hydrique ou de caractérisation des associations d'espèces à partir d'images multi-spécimens. Il améliorera la connaissance des espèces.

  Le projet Pl@ntAgroEco rassemble des forces complémentaires en matière de recherche, de développement et d'animation. S'ajouteront à l'équipe pluridisciplinaire chargée de la plateforme Pl@ntNet de nouvelles forces de recherche ayant une expertise reconnue dans les sciences participatives. Le consortium rassemblera 10 partenaires incluant des organismes de recherche, des universités, des acteurs de la société civile et des partenaires internationaux.

10.4.3 Other projects in France

Scool is involved in the Regalia pilot-project.

Other collaborations:

• L. Richert, R. Thiébaut, Inria SISTM, Bordeaux, bandits for vaccine clinical trials.
• W. M. Koolen, CWI Amsterdan & University of Twente, concentration of information divergences.

11.1.1 Scientific events: organisation

• H. Kohler co-organized the Workshop on Interpretable Policies in Reinforcement Learning (InterPol) at the Reinforcement Learning Conference, Amherst, USA, August 24.

11.1.2 Scientific events: selection

11.1.3 Journal

11.1.4 Invited talks

• R. Akrour: talk on preference-based reinforcement learning at Journées Synthèse de Programmes in Bordeaux.
• T. Mathieu: talk on robust statistics in Séminaire de Statistiques de Montpellier, talk on robust statistics in Séminaire parisien de statistique SemStat.
• O.-A. Maillard: Invited speaker at Inria-Brasil workshop on Digital Agriculture, September 10-11 2024, Montpellier (France), at W'Happy Digital Day "IA et Agroécologie", May 16 2024, Tournai (Belgium), at Table Ronde DigitAgora "Le numérique en agriculture", April 24 2024, Montpellier (France).
• D. Basu: Invited talks: (i) When Privacy Meets Partial Information: Privacy-Utility Trade-offs in Reinforcement Learning, IndoML, India (keynote). (ii) Auditing Bias of ML Systems: An Algorithmic Montage, Digital regulation: the contribution of science, Bruxelles, EU (keynote). (iii) Challenges of Data-centrism, Workshop on Data-Driven Futures in the Age of AI: Leveraging Insights and Addressing Challenges, ADRI, India (keynote). (iv) Panel Discussion: A Multidisciplinary View of Responsible Data-Centrism, ADRI, India (Public Talk). (v) The fair game: auditing and debiasing AI algorithms over time, JUST-AI Summer Colloquium on Individual Safeguards in the Era of AI by EU & University of Liége. (vi) Sequential Decision Making under Partial Information: RL, Uncertainty Quantification, & Responsible AI, Séminaire doctoral – ED SHS Université de Lille. (vii) The Privacy Game: Attacks with and Defenses for Online ML Al- gorithms, Workshop Inria- University of Waterloo–Université de Bordeaux, Bordeaux, France. (viii) AI for Agricultural Sciences, CAFT training program on “Harnessing Disruptive Technology in ICT for Agricultural Extension and Research”, Sabour, India (Public talk).

11.1.5 Leadership within the scientific community

• D. Basu: selected as a fellow of European Laboratory of Learning and Intelligent Systems (ELLIS) Society.

11.1.6 Scientific expertise

• Ph. Preux: evaluation of an ANR project.
• D. Basu: evaluation of an ANR grant application.

11.1.7 Research administration

• Ph. Preux:
  • scientific coordinator of CPER CornelIA
  • member of the CSS5 (data science and models) at IRD
  • member of the scientific committee of the MathNum department at Inrae
  • member of the scientific committee of PEPR agroécologie et numérique
  • member of the scientific and ethical committee of INCLUDE (data warehouse of CHU Lille)
  • member of the DAS1 health at Région Hauts-de-France
  • member of the BCEP at Inria Lille.

11.2.1 Teaching

• J. Achddou: « Algorithmique Numérique pour l'Optimisation », M2 in Computer Science and Statistics, Polytech Lille.
• J. Achddou: « Algorithmique Numérique pour l'Optimisation », M1 in Computer Science and Statistics, Polytech Lille.
• R. Akrour: Sequential Decision Making, M2 in Data Science, Centrale Lille and Université de Lille
• R. Akrour: « Option découverte: Machine Learning », L3 in Computer Science, Université de Lille
• R. Akrour: « Perception et motricité 2 », L2 MIASHS, Université de Lille
• R. Degenne: « Sequential learning », M2 MVA, ENS Paris-Saclay
• R. Degenne: « Sequential learning », Centrale Lille
• D. Basu: « Sequential Decision Making », M2 in Data Science, Centrale Lille and Université de Lille
• D. Basu: « Research Reading Group », M2 in Data Science, Centrale Lille and Université de Lille
• D. Basu: « Advanced Machine Learning and Decision Making », Centrale Lille
• E. Kaufmann: « Statistics 2 », M1 Data Science, Ecole Centrale Lille.
• Ph. Preux: « Prise de décision séquentielle dans l'incertain », M2 in Computer Science, Université de Lille.
• Ph. Preux: « Apprentissage par renforcement », M2 in Computer Science, Université de Lille.
• Ph. Preux: « Science des données II », L3 MIASHS, Université de Lille.
• Ph. Preux: « Réseaux de neurones », L1 Maths-Informatique, Université de Lille.
• Ph. Preux: « IA et apprentissage automatique », DU IA & Santé, Université de Lille.
• O-A. Maillard: « Reinforcement Learning Research Challenges », Executive Master Ecole Polytechnique.

11.2.2 Supervision

• Ph. Preux:
  • C. Blondelle, L3 MIASHS
  • Q. Uguen, Centrale-Lille
  • Ph.D. students: M. Centa, M. Basson, P. Saux,
• Ph. Preux and R. Akrour: Ph.D. students: H. Kohler, Y. Berthelot
• Ph. Preux and O-A. Maillard: Ph.D. student: P. Saux
• Ph. Preux and D. Basu: Ph.D. students: A. Achraf, A. Ajarra
• E. Kaufmann and R. Degenne: PhD students: M. Jourdan, A. Tuynman
• E. Kaufmann and D. Basu: PhD student: T. Michel
• E. Kaufmann: PhD student: C. Kone (with Laura Richert, Université de Bordeaux, Inria SISTM)
• R. Akrour: PhD student: Brahim Driss.
• R. Akrour: Research engineer: A. Davey.
• R. Akrour: Masters student: M. Kabouri (with Joni Pajarinen, Aalto University, Finland).
• O-A. Maillard: Ph.D. students: S. Vashishtha, A. Kobanda.
• O-A. Maillard and D. Basu: Ph.D. student: U. Das.
• O-A. Maillard: Research engineer: W. Radji (until Sep 2024), then as Ph.D student.
• O-A. Maillard: Research engineer: H. Carvajal
• O-A. Maillard and T. Mathieu: Master student: A. Prevost (until Sep 2024), then as Ph.D. student.
• O-A. Maillard: Postdoc: T. Dan (until Sep 2024).
• O-A. Maillard: Postdoc: T. Lefort.
• D. Basu: Masters student: A. Olivier (with Bits2beat, Bordeaux).
• D. Basu: PhD student: E. Jorge (with C. Dimitrakakis, Chalmers Univ. of Technology, Sweden).
• D. Basu: Research engineer: G. Pourcel.
• R. Degenne: Master student: L. Luccioli.

11.2.3 Juries

• Ph. Preux:
  • CR and DR IRD CSS5
  • Ph.D. defenses: P. Saux (Lille, co-supervisor), É. Arnaud (Amiens, reviewer), J. Bujalance Martin (PSL/Mines, reviewer), A. Azize, (Lille, co-supervisor), I. Harith (IMT, chair), M. Zadem (ENSTA, reviewer)
• E. Kaufmann:
  • Ph.D. defenses: A. Pacaud (Telecom Paris), P. Wang (KTH, Stockholm), A. Chaouki (Ecole Polytechnique), M. Jourdan, (Lille, co-supervisor), I. Harith (IMT, chair), J. Whitehouse (CMU, Pittsburgh)
• O-A. Maillard:
  • Ph.D. defenses: A. Barrier (ENS Lyon, reviewer), D. Brellmann (Telecom Paris, reviewer), T. Lefort (Univ. Montpellier), D. Delande, (Univ. Toulouse).
  • HdR defense: A. Cleynen (Univ. Montpellier, reviewer).
  • CSI: F. Fabre (Univ. Réunion), T. Delliaux (Univ. Toulouse).
• D. Basu:
  • Ph.D. defense: A. Azize (Lille, co-supervisor)
• R. Degenne:
  • Ph.D. defense: M. Jourdan (Lille, co-supervisor)

11.3.1 Others science outreach relevant activities

• Ph. Preux:
  • 1/2/2024 : « l'intelligence artificielle, qu'en est-il ? », université populaire de Lille, Palais des Beaux-Arts, Lille.
  • 29/11/2024 : « L'intelligence artificielle. De quoi parle-t-on ? », public library Saint-Omer.
• J. Teigny:
  • « Introduction à l'alignement des IA »: at Inria Lille, journée du réseau métier Min2rien, CRIStAL, Science Po Lille.
  • « Introduction a l'intelligence artificielle », association Tourcoing Loisir Sortir
  • 4 sessions of co-animation of a game related to AI for high school students.

International journals

• 14 articleT.Timothée Mathieu, D.Debabrota Basu and O.-A.Odalric-Ambrym Maillard. Bandits with Stochastic Corruption: Lower Bounds on Regret and Robust Optimistic Algorithms.Transactions on Machine Learning Research JournalJanuary 2024HALback to text
• 15 articleT.Timothée Mathieu, R.Riccardo Della Vecchia, A.Alena Shilova, M.Matheus Medeiros Centa, H.Hector Kohler, O.-A.Odalric-Ambrym Maillard and P.Philippe Preux. AdaStop: adaptive statistical testing for sound comparisons of Deep RL agents.Transactions on Machine Learning Research Journal2024HALback to text
• 16 articleP.Paolo Recchia, D.Debabrota Basu, M.Mario Gattobigio, C.Christian Miniatura and S.Stéphane Bressan. The Steepest Slope toward a Quantum Few-body Solution: Gradient Variational Methods for the Quantum Few-body Problem.Few-Body Systems654November 2024, 102HALDOIback to text
• 17 articleS.Sumit Vashishtha and K.Katepalli Sreenivasan. Bistability in the sunspot cycle.EPL - Europhysics Letters1482October 2024, 23001In press. HALDOIback to text

International peer-reviewed conferences

• 18 inproceedingsS.Shubhada Agrawal, T.Timothée Mathieu, D.Debabrota Basu and O.-A.Odalric-Ambrym Maillard. CRIMED: Lower and Upper Bounds on Regret for Bandits with Unbounded Stochastic Corruption.Proceedings of Machine Learning ResearchInternational Conference on Algorithmic Learning Theory (ALT)237San Diego (CA), United StatesMarch 2024, 74-124HALback to text
• 19 inproceedingsA.Achraf Azize and D.Debabrota Basu. Concentrated Differential Privacy for Bandits.2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)Toronto, CanadaIEEEApril 2024, 78-109HALDOIback to text
• 20 inproceedings A.Achraf Azize and D.Debabrota Basu. Open Problem: What is the Complexity of Joint Differential Privacy in Linear Contextual Bandits? Proceedings of Thirty Seventh Conference on Learning Theory PMLR Edmonton (Alberta), Canada July 2024 HAL back to text
• 21 inproceedingsE.Emil Carlsson, D.Debabrota Basu, F. D.Fredrik D. Johansson and D.Devdatt Dubhashi. Pure Exploration in Bandits with Linear Constraints.Proceedings of Machine Learning Research (PMLR)International Conference on Artificial Intelligence and Statistics238Proceedings of Machine Learning Research (PMLR)Valencia (Espagne), SpainMay 2024, 334-342HALback to text
• 22 inproceedingsT. Q.Tuan Quang Tuan Dam, O.-A.Odalric-Ambrym Maillard and E.Emilie Kaufmann. Power Mean Estimation in Stochastic Monte-Carlo Tree Search.40th conference on Uncertainty in Artificial IntelligenceUncertainty in Artificial IntelligenceBarcelona, SpainJuly 2024HALback to text
• 23 inproceedingsM.Mahdi Kallel, D.Debabrota Basu, R.Riad Akrour and C.Carlo d'Eramo. Augmented Bayesian Policy Search.The Twelfth International Conference on Learning Representations (ICLR)Vienna, AustriaMay 2024HALback to text
• 24 inproceedingsH.Hector Kohler, Q.Quentin Delfosse, R.Riad Akrour, K.Kristian Kersting and P.Philippe Preux. Interpretable and Editable Programmatic Tree Policies for Reinforcement Learning.European Workshop on Reinforcement Learning17Toulouse, France2024HALback to text
• 25 inproceedingsR.Riccardo Poiani, R.Rémy Degenne, E.Emilie Kaufmann, A. M.Alberto Maria Metelli and M.Marcello Restelli. Optimal Multi-Fidelity Best-Arm Identification.Advances in Neural Information Processing Systems (NeurIPS)Vancouver (BC), CanadaDecember 2024HALback to text
• 26 inproceedingsH.Hassan Saber and O.-A.Odalric-Ambrym Maillard. Bandits with Multimodal Structure.Reinforcement Learning Conference (Reinforcement Learning Journal)RLC 2024 - Reinforcement Learning Conference15Amherst Massachusetts, United States2024, 39HALback to text
• 27 inproceedingsA.Apurv Shukla and D.Debabrota Basu. Preference-based Pure Exploration.Advances in Neural Information Processing Systems (NeurIPS)Vancouver (CA), CanadaDecember 2024HALback to text
• 28 inproceedingsA.Adrienne Tuynman, R.Rémy Degenne and E.Emilie Kaufmann. Finding good policies in average-reward Markov Decision Processes without prior knowledge.NeurIPSVancouver (Canada), CanadaDecember 2024HALback to text

Conferences without proceedings

• 29 inproceedingsA.Ayoub Ajarra, B.Bishwamittra Ghosh and D.Debabrota Basu. Active Fourier Auditor for Estimating Distributional Properties of ML Models.NeurIPS 2024 Workshop on Regulatable MLVancouver (BC), CanadaDecember 2024HALback to text
• 30 inproceedingsU.Udvas Das and D.Debabrota Basu. Learning to Explore with Lagrangians for Bandits under Unknown Linear Constraints.Seventeenth European Workshop on Reinforcement Learning (EWRL)Toulouse, FranceOctober 2024HALback to text
• 31 inproceedingsH.Hannes Eriksson, T.Tommy Tram, D.Debabrota Basu, M.Mina Alibeigi and C.Christos Dimitrakakis. Reinforcement Learning in the Wild with Maximum Likelihood-based Model Transfer.23rd International Conference on Autonomous Agents and Multiagent Systems (AAMAS)Auckland, New ZealandACMMay 2024, 516-524HALDOIback to text

Edition (books, proceedings, special issue of a journal)

• 32 proceedingsStatistical comparison in empirical computer science with minimal computation usage.ACM REP '24: ACM Conference on Reproducibility and Replicability35ACMJune 2024, 20-24HALDOIback to text

Reports & preprints

• 33 miscN. A.Naheed Anjum Arafat, D.Debabrota Basu, Y.Yulia Gel and Y.Yuzhou Chen. When Witnesses Defend: A Witness Graph Topological Layer for Adversarial Graph Learning.September 2024HALback to text
• 34 miscA.Achraf Azize and D.Debabrota Basu. How Much Does Each Datapoint Leak Your Privacy? Quantifying the Per-datum Membership Leakage.February 2024HALback to text
• 35 miscA.Achraf Azize, M.Marc Jourdan, A. A.Aymen Al Marjani and D.Debabrota Basu. Differentially Private Best-Arm Identification.June 2024HALback to text
• 36 reportM.Mickael Basson and P.Philippe Preux. IDEQ: an improved diffusion model for the TSP.RR-9558INRIA Lille - Nord EuropeJuly 2024HALback to text
• 37 miscD.Debabrota Basu, S.Sourav Chakraborty, D.Debarshi Chanda, B. D.Buddha Dev Das, A.Arijit Ghosh and A.Arnab Ray. Testing Credibility of Public and Private Surveys through the Lens of Regression.October 2024HALback to text
• 38 miscS.Sunrit Chakraborty, S.Saptarshi Roy and D.Debabrota Basu. FLIPHAT: Joint Differential Privacy for High Dimensional Sparse Linear Bandits.May 2024HALback to text
• 39 miscB.Bishwamittra Ghosh, D.Debabrota Basu, F.Fu Huazhu, W.Wang Yuan, R.Renuga Kanagavelu, J. J.Jiang Jin Peng, L.Liu Yong, G. S.Goh Siow Mong Rick and W.Wei Qingsong. Don't Forget What I did?: Assessing Client Contributions in Federated Learning.March 2024HALback to text
• 40 miscC.Chao Han, D.Debabrota Basu, M.Michael Mangan, E.Eleni Vasilaki and A.Aditya Gilra. Dynamical-VAE-based Hindsight to Learn the Causal Dynamics of Factored-POMDPs.November 2024HALback to text
• 41 miscR. M.Reabetswe M. Nkhumise, D.Debabrota Basu, T. J.Tony J. Prescott and A.Aditya Gilra. Measuring Exploration in Reinforcement Learning via Optimal Transport in Policy Space.February 2024HALback to text
• 42 reportA.Alena Shilova, T.Thomas Delliaux, P.Philippe Preux and B.Bruno Raffin. Learning HJB Viscosity Solutions with PINNs for Continuous-Time Reinforcement Learning.RR-9541Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189; Univ. Lille, CNRS, Centrale Lille, Inria UMR 9189 - CRIStAL,INRIA Lille Nord Europe, Villeneuve d’Ascq, France; Univ. Grenoble Alps, CNRS, Inria, Grenoble INP, LIG, 38000 Grenoble, FranceFebruary 2024, 1-30HALback to text