2024Activity reportProject-TeamFAIRPLAY
RNSR: 202224251U- Research center Inria Saclay Centre at Institut Polytechnique de Paris
- In partnership with:Institut Polytechnique de Paris, Criteo
- Team name: Coopetitive AI: Fairness, Privacy, Incentives
- In collaboration with:Centre de Recherche en Economie et Stastistique
- Domain:Applied Mathematics, Computation and Simulation
- Theme:Optimization, machine learning and statistical methods
Keywords
Computer Science and Digital Science
- A4.8. Privacy-enhancing technologies
- A8.11. Game Theory
- A9.2. Machine learning
- A9.9. Distributed AI, Multi-agent
Other Research Topics and Application Domains
- B9.9. Ethics
- B9.10. Privacy
1 Team members, visitors, external collaborators
Research Scientists
- Patrick Loiseau [Team leader, INRIA, Researcher, until Sep 2024, HDR]
- Patrick Loiseau [Team leader, INRIA, Senior Researcher, from Oct 2024, HDR]
- Marc Abeille [Criteo, Industrial member]
- Benjamin Heymann [Criteo, Industrial member]
- Simon Mauras [INRIA, Researcher, from Feb 2024]
- Hugo Richard [Criteo, Industrial member]
- Maxime Vono [Criteo, Industrial member]
Faculty Members
- Vianney Perchet [Team leader, Criteo & ENSAE, Professor, HDR]
- Cristina Butucea [ENSAE, Professor, HDR]
- Matthieu Lerasle [ENSAE, Professor, HDR]
Post-Doctoral Fellows
- Dorian Baudry [ENSAE, Post-Doctoral Fellow, until Jun 2024]
- Lorenzo Croissant [ENSAE]
- Simon Finster [INRIA, Post-Doctoral Fellow]
- Solenne Gaucher [ENSAE, Post-Doctoral Fellow]
- Nadav Merlis [ENSAE]
- Denis Sokolov [INRIA, Post-Doctoral Fellow]
- Bartholome Vieille [INRIA, Post-Doctoral Fellow, from Oct 2024]
PhD Students
- Ziyad Benomar [ENSAE]
- Maria Cherifa [Criteo, CIFRE]
- Hafedh El Ferichichi [ENSEA]
- Côme Fiegel [ENSAE]
- Reda Jalal [INRIA, from Feb 2024]
- Mike Liu [ENSAE]
- Mathieu Molina [INRIA]
- Giovanni Montanari [INRIA]
- Marius Potfer [EDF, CIFRE]
- Melissa Tamine [Criteo, CIFRE]
- Matilde Tullii [ENSAE]
- Rémi Verroneau [Footvision, CIFRE]
- Axel Xerri [INRIA, from Dec 2024]
- Minrui Xu [ENSAE, from Sep 2024]
Technical Staff
- Felipe Garrido Lucero [INRIA, Engineer, until Aug 2024]
Interns and Apprentices
- Yeniffer Munoz Fuentez [INRIA, Intern, until Mar 2024]
Administrative Assistant
- Melanie Da Silva [INRIA]
External Collaborators
- Clément Calauzènes [Criteo]
- Julien Combe [Ecole Polytechnique, HDR]
- Azadeh Khaleghi [ENSAE]
- Laurent Menard [UNIV PARIS NANTERRE, until Jun 2024, HDR]
- Jaouad Mourtada [ENSAE]
2 Overall objectives
2.1 Broad context
One of the principal objectives of Machine Learning (ML) is to automatically discover using past data some underlying structure behind a data generating process in order either to explain past observations or, perhaps more importantly, to make predictions and/or to optimize decisions made on future instances. The area of ML has exploded over the past decade and has had a tremendous impact in many application domains such as computer vision or bioinformatics.
Most of the current ML literature focuses on the case of a single agent (an algorithm) trying to complete some learning task based on gathered data that follows an exogenous distribution independent of the algorithm. One of the key assumptions is that this data has sufficient “regularity” for classical techniques to work. This classical paradigm of “a single agent learning on nice data”, however, is no longer adequate for many practical and crucial tasks that imply users (who own the gathered data) and/or other (learning) agents that are also trying to optimize their own objectives simultaneously, in a competitive or conflicting way. This is the case, for instance, in most learning tasks related to Internet applications (content recommendation/ranking, ad auctions, fraud detection, etc.). Moreover, as such learning tasks rely on users' personal data and as their outcome affect users in return, it is no longer sufficient to focus on optimizing prediction performance metrics—it becomes crucial to consider societal and ethical aspects such as fairness or privacy.
The field of single agent ML builds on techniques from domains such as statistics, optimization, or functional analysis. When different agents are involved, a strategic aspect inherent in game theory enters the picture. Indeed, interactions—either positive or negative—between rational entities (firms, single user at home, algorithms, etc.) foster individual strategic behavior such as hiding information, misleading other agents, free-riding, etc. Unfortunately, this selfishness degrades the quality of the data or of the predictions, prevents efficient learning and overall may diminish the social welfare. These strategic aspects, together with the decentralized nature of decision making in a multi-agent environment, also make it harder to build algorithms that meet fairness and privacy constraints.
The overarching objective of FAIRPLAY is to create algorithms that learn for and with users—and techniques to analyze them—, that is to create procedures able to perform classical learning tasks (prediction, decision, explanation) when the data is generated or provided by strategic agents, possibly in the presence of other competing learning agents, while respecting the fairness and privacy of the involved users. To that end, we will naturally rely on multi-agent models where the different agents may be either agents generating or providing data, or agents learning in a way that interacts with other agents; and we will put a special focus on societal and ethical aspects, in particular fairness and privacy. Note that in FAIRPLAY, we focus on the technical challenges inherent to formalizing mathematically and respecting ethical properties such as non-discrimination or privacy, often seen as constraints in the learning procedure. Nevertheless, throughout the team's life, we will reflect on these mathematical definitions for the particular applications studied, in particular their philosophical roots and legal interpretation, through interactions with HSS researchers and with legal specialists (from Criteo).
2.1.1 Multi-agent systems
Any company developing and implementing ML algorithms is in fact one agent within a large network of users and other firms. Assuming that the data is i.i.d. and can be treated irrespectively of the environment response—as is done in the classical ML paradigm—might be a good first approximation, but should be overcome. Users, clients, suppliers, and competitors are adaptive and change their behavior depending on each other's interactions. The future of many ML companies—such as Criteo—will consist in creating platforms matching the demand (created by their users) to the offer (proposed by their clients), under the system constraints (imposed by suppliers and competitors). Each of these agents have different, conflicting interests that should be taken into account in the model, which naturally becomes a multi-agent model.
Each agent in a multi-agent system may be modeled as having their own utility function
The natural language to model and study multi-agent systems is game theory—see below for a list of tools and techniques on which FAIRPLAY relies, game theory being the first of them. Multi-agent systems have been studied in the past; but not with a focus on learning systems where agents are either learning or providing data, which is our focus in FAIRPLAY and leads to a blend of game theory and learning techniques. We note here again that, wherever appropriate, we shall reflect (in part together with colleagues from HSS) on the soundness of the utility framework for the considered applications.
2.1.2 Societal aspects and ethics
There are several important ethical aspects that must be investigated in multi-agent systems involving users either as data providers or as individuals affected by the ML agent decision (or both).
Fairness and Discrimination
When ML decisions directly affect humans, it is important to ensure that they do not violate fairness principles, be they based on ethical or legal grounds. As ML made its way in many areas of decision making, it was unfortunately repeatedly observed that it can lead to discrimination (regardless of whether or not it is intentional) based on gender, race, age, or other sensitive attributes. This was observed in online targeted advertisement 99, 125, 39, 83, 99, 41, but also in many other applications such as hiring 70, data-driven healthcare 77, or justice 100. Biases also have the unfortunate tendency to reinforce. An operating multi-agent learning system should be able in the long run to get rid by itself of inherent population biases, that is, be fair amongst users irrespective of the improperly constructed dataset.
The mathematical formulation of fairness has been debated in recent works. Although a few initial works proposed a notion of individual fairness, which mandates that “similar individuals” receive “similar outcomes” 72, this notion was quickly found unpractical because it relies on a metric to define closeness that makes the definition somewhat arbitrary. Most of the works then focused on notions of group fairness, which mandate equality of outcome “on average” across different groups defined by sensitive attributes (e.g., race, gender, religious belief, etc.).
Most of the works on group fairness focus on the classification problem (e.g., classifying whether a job applicant is good or not for the job) where each data example
The fair classification literature proposed, for each of these fairness notions, ways to train fair classifiers based on three main ideas: pre-processing 133, in-processing 131, 132, 128, and post-processing 84. All of these works, however, focus on idealized situations where a single decision-maker has access to ground truth data with the sensitive features and labels in order to train classifiers that respect fairness constraints. We use similar group fairness definitions and extend them (in particular through causality), but our goal is to go further in terms of algorithms by modeling practical scenarios with multiple decision-makers and incomplete information (in particular lack of ground truth on the labels).
Privacy vs. Incentives
ML algorithms, in particular in Internet applications, often rely on users' personal information (whether it is directly their personal data or indirectly some hidden “type” – gender, ethnicity, behaviors, etc.). Nevertheless, users may be willing to provide their personal information if it increases their utility. This brings a number of key questions. First, how can we learn while protecting users' privacy (and how should privacy even be defined)? Second, finding the right balance between those two a-priori incompatible concepts is challenging; how much (and even simply how) should an agent be compensated for providing useful and accurate data?
Differential privacy is the most widely used private learning framework 71, 73, 120 and ensures that the output of an algorithm does not significantly depend on a single element of the whole dataset. These privacy constraints are often too strong for economic applications (as illustrated before, it is sometimes optimal to disclose some private information).
This privacy concept can be refined up to a single user level, into the so-called local differential privacy. Informally speaking, the algorithm output can also depend on a single user data that still must be kept private. Estimation are actually sometimes more challenging under this constraint, i.e., estimation rates degrade 121, 58, 59 but is sometimes more adapted to handle user-generated data 79.
Interestingly, we note that the notions of privacy and fairness are somewhat incompatible. This will motivate Theme 2 developed in our research program.
2.2 A large variety of tools and techniques
Analyzing multi-agent learning systems with ethical constraints will require us to use, develop, and merge several different theoretical tools and techniques. We describe the main ones here. Note that although FAIRPLAY is motivated by practical use-cases and applications, part of the team's objectives is to improve those tools as necessary to tackle the problems studied.
Game theory and economics
Game theory 78 is the natural mathematical tool to model multiple interacting decision-makers (called players). A game is defined by a set of players, a set of possible actions for each player, and a payoff function for each player that can depend on the actions of all the players (that is the distinguishing feature of a game compared to an optimization problem). The most standard solution concept is the so-called Nash equilibrium, which is defined as a strategy profile (i.e., a collection of possibly randomized action for each player) such that each player is at best response (i.e., has the maximum payoff given the others' strategies). It is a “static” (one-shot) solution concept, but there also exist dynamic solution concepts for repeated games 62, 109.
Online and reinforcement learning 55
In online learning (a.k.a. multi-armed bandit 56, 116), data is gathered and treated on the fly. For instance, consider an online binary classification problem. Some unlabelled data
These techniques will be central to our approach as we aim to model problems where ground truth data is not available upfront and problems involving sequential decision making. There have been some successful first results in that direction. For instance, there are applications (e.g., cognitive radio) where several agents (users) aim at finding a matching with resources (the different bandwidth). They can do that by “probing” the resources, estimating their preferences and trying to find some stable matchings 53, 101.
Online algorithms 51 and theoretical computer science
Online algorithms are closely related to online learning with a major twist. In online learning, the agent has “0-look ahead”; for instance, in the online binary classification example, the loss at stage
Optimal transport 126
Optimal transport is a quite old problem introduced by Monge where an agent aims at moving a pile of sand to fill a hole at the smallest possible price. Formally speaking, given two probability measures
Recently, optimal transport gained a lot of interest in the ML community 115 thanks to its application to images and to new techniques to compute approximate matchings in a tractable way 118. Even more unexpected applications of optimal transport have been discovered: to protect privacy 54, fairness 47, etc. Those connections are promising, but only primitive for the moment. For instance, consider the problem of matching students to schools. The unfairness level of a school can be measured as the Wasserstein distance between the distribution of the students within that school compared to the overall distribution of students. Then the matching algorithms could have a constraint of minimizing the sum of (or its maximum) unfairness levels; alternatively, we could aim at designing mechanisms giving incentives to schools to be fair in their allocation (or at least in their list preferences), typically by paying a higher fee if the unfairness level is high.
2.3 General objectives
The overarching objective of FAIRPLAY of to create algorithms to learn for and with users—and techniques to analyze them—, through the study of multi-agent learning systems where the agents can be cooperatively or competitively learning agents, or agents providing or generating data, while guaranteeing that fairness and privacy constraints are satisfied for the involved users. We detail this global objective into a number of more specific ones.
-
Objective 1: Developing fair and private mechanisms
Our first objective is to incorporate ethical aspects of fairness and privacy in mechanisms used in typical problems occurring in Internet applications, in particular auctions, matching, and recommendation. We will focus on social welfare and consider realistic cases with multiple agents and sequential learning that occur in practice due to sequential decision making. Our objective is both to construct models to analyze the problem, to devise algorithms that respect the constraints at stake, and to evaluate the different trade-offs in standard notions of utility introduced by ethical constraints.
-
Objective 2: Developing multi-agent statistics and learning
Data is now acquired, treated and/or generated by a whole network of agents interacting with the environment. There are also often multiple agents learning either collaboratively or competitively. Our second objective is to build a new set of tools to perform statistics and learning tasks in such environments. To this end, we aim at modeling these situations as multi-agent systems and at studying the dynamics and equilibrium of these complex game-theoretic situations between multiple learning algorithms and data providers.
-
Objective 3: Improving the theoretical state of the art
Research must rely on theoretical, proven guarantees. We develop new results for the techniques introduced before, such as prophet inequalities, (online) matchings, bandits and RL, etc.
-
Objective 4: Proposing practical solutions and enhancing transfer from research to industry
Our last scientific objective is to apply and implement theoretical works and results to practical cases. This will be a crucial component of the project as we focus on transfer within Criteo.
-
Objective 5: Scientific Publications
We aim at publishing our results in top-tier machine learning conferences (NeurIPS, ICML, COLT, ICLR, etc.) and in top-tier game theory journals (Games and Economic Behavior, Mathematics of OR, etc.). We will also target conferences at the junction of those fields (EC, WINE, WebConf, etc.) as well as conferences specifically on security and privacy (IEEE S&P, NDSS, CSS, PETS, etc.) and on fairness (FAccT, AIES).
All the five objectives are interlaced. For instance, fairness and privacy constraints are important in Objective 2 whereas the multi-agent aspect is also important in Objective 1. Objectives 4 and 5 are transversal and present in all the first three objectives.
3 Research program
To reach the objectives laid out above, we organize the research in three themes. The first one focuses on developing fair mechanisms. The second one considers private mechanisms, and in particular considers the challenge of reconciling fairness and privacy—which are often conflicting notions. The last theme, somewhat transverse to the first two, consists in leveraging/incorporating structure in all those problems in order to speed up learning. Of course, all themes share common points on both the problems/applications considered and the methods and tools used to tackle them; hence there are cross-fertilization between the different themes.
3.1 Theme 1: Developing fair mechanisms for auctions and matching problems
3.1.1 Fairness in auction-based systems
Online ads platforms are nowadays used to advertise not just products, but also opportunities such as jobs, houses, or financial services. This makes it crucial for such platforms to respect fairness criteria (be it only for legal reasons), as an unfair ad system would deprive a part of the population of some potentially interesting opportunities. Despite this pressing need, there is currently no technical solution in place to provably prevent discriminations. One of the main challenge is that ad impression decisions are the outcome of an auction mechanism that involves bidding decisions of multiple self-interested agents controlling only a small part of the process, while group fairness notions are defined on the outcome of a large number of impressions. We propose to investigate two mechanisms to guarantee fairness in such a complex auction-based system (note that we focus on online ad auctions but the work has broader applicability).
-
Advertiser-centric (or bidder-centric) fairness
We first focus on advertiser-centric fairness, i.e., the advertiser of a third-party needs to make sure that the reached audience is fair independently of the ad auction platform. A key difficulty is that the advertiser does not control the final decision for each ad impression, which depends on the bids of other advertisers competing in the same auction and on the platform's mechanism. Hence, it is necessary that the advertiser keeps track of the auctions won for each of the groups and dynamically adjusts its bids in order to maintain the required balance.
A first difficulty is to model the behavior of other advertisers. We can first use a mean-field games approach similar to 87 that approximates the other bidders by an (unknown) distribution and checks equilibrium consistency; this makes sense if there are many bidders. We can also leverage refined mean-field approximations 81 to provide better approximations for smaller numbers of advertisers. Then a second difficulty is to find an optimal bidding policy that enforces the fairness constraint. We can investigate two approaches. One is based on an MDP (Markov Decision Process) that encodes the current fairness level and imposes a hard constraint. The second is based on modeling the problem as a contextual bandit problem. We note that in addition to fairness constraints, privacy constraints may complicate the optimal solution finding.
-
Platform-centric (or auction-centric) fairness
We also consider the problem from the platform's perspective, i.e., we assume that it is the platform's responsibility to enforce the fairness constraint. We also focus here on demographic parity. To make the solution practical, we do not consider modification of the auction mechanism, instead we consider a given mechanism and let the platform adapt dynamically the bids of each advertiser to achieve the fairness guarantee. This approach would be similar to the pacing multipliers used by some platforms 67, 66, but using different multipliers for the different groups (i.e., different values of the sensitive attribute).
Following recent theoretical work on auction fairness 60, 86, 64 (which assumes that the targeted population of all ads is known in advance along with all their characteristics), we can formulate fairness as a constraint in an optimization problem for each advertiser. We study fairness in this static auction problem in which the auction mechanism is fixed (e.g., to second price). We then move to the online setting in which users (but also advertisers) are dynamic and in which decisions must be taken online, which we approach through dynamic adjustment of pacing multipliers.
3.1.2 Fairness in matching and selection problems
In this second part, we study fairness in selection and matching problems such as hiring or college admission. The selection problem corresponds to any situation in which one needs to select (i.e., assign a binary label to) data examples or individuals but with a constraint on the number of maximum number of positive labels. There are many applications of selection problems such as police checks, loan approvals, or medical screening. The matching problem can be seen as the more general variant with multiple selectors. Again, a particular focus is put here on cases involving repeated selection/matching problems and multiple decision makers.
-
Fair repeated multistage selection
In our work 75, we identified that a key source of discrimination in (static) selection problems is differential variance, i.e., the fact that one has quality estimates that have different variances for different groups. In practice, however, the selection problem is often ran repeatedly (e.g., at each hiring campaign) and with partial (and increasing) information to exploit for making decisions.
Here, we consider the repeated multistage selection problem, where at each round a multistage selection problem is solved. A key aspect is that, at the end of a round, one learns extra information about the candidates that were selected—hence one can refine (i.e., decrease the variance of) the quality estimate for the groups in which more candidates were selected. We will first rethink fairness constraints in this type of repeated decision making problems. Then we will both study the discrimination that come out of natural (e.g., greedy) procedure as well as design (near) optimal ones for the constraints at stake. We also investigate how the constraints affect the selection utility.
-
Multiple decision-makers
Next, we investigate cases with multiple decision-makers. We propose two cases in particular. The first one is the simple two-stage selection problem but where the decision-maker doing the first-stage selection is different from the decision-maker doing the second-stage selection. This is a typical case for instance for recruiting agencies that propose sublists of candidates to different firms that wish to hire. The second case is when multiple-decision makers are trying to make a selection simultaneously—a typical example of this being the college admission problem (or faculty recruitment). We intend to model it as a game between the different colleges and to study both static solutions as well as dynamic solutions with sequential learning, again modeling it as a bandit problem and looking for regret-minimizing algorithms under fairness constraints. A number of important questions arise here: if each college makes its selection independently and strategically (but based on quality estimates with variances that differ amongst groups), how does it affect the “global fairness” metrics (meaning in aggregate across the different colleges) and the “local fairness” metrics (meaning for an individual college)? What changes if there is a central admission system (such as Parcoursup)? And in this later case, how to handle fairness on the side of colleges (i.e., treat each college fairly in some sense)?
-
Fair matching with incentives in two-sided platforms
We will study specifically the case of a platform matching demand on one side to offer on the other side, with fairness constraints on each side. This is the case for instance in online job markets (or crowdsourcing). This is similar to the previous case but, in addition, here there is an extra incentives problem: companies need to give the right incentives to job applicants to accept the jobs; while the platform doing the match needs to ensure fairness on both sides (job applicants and companies). This gives rise to a complicated interplay between learning and incentives that we will tackle again in the repeated setting.
We finally mention that, in many of these matching problems, there is an important time component: each agent needs to be matched “as soon as possible”, yielding a trade-off between the delay to be matched and the quality of the match. There is also a problem of participation incentives; that is how the matching algorithm used affect the behavior of the participants in the matching “market” (participation or not, information revelation, etc.). In the long-term, we will incorporate these aspects in the above models.
Throughout the work in this theme, we will also consider a question transverse and present in all the models above: how can we handle multidimensional fairness, that is, where there are multiple sensitive attributes and consequently an exponential number of sub-groups defined by all intersections; this combinatorial is challenging and, for the moment, still exploratory.
3.2 Theme 2: Reconciling, and enforcing privacy with fairness
In the previous theme, we implicitly assumed that we know the users' group, i.e., their sensitive attributes such as gender, age, or ethnicity. In practice, one of the key question when implementing fairness mechanisms is how to measure/control fairness metrics without having access to these protected attributes. This question relates to the link between privacy and fairness and the trade-off between them (as fairness requires data and privacy tends to protect it) 119, 69.
A first option to solve this problem would be (when it is possible) to build proxies 82, 122 for protected attributes using available information (e.g., websites visited or products bought) and to measure or control for fairness using those in place of the protected attributes. As the accuracy of these proxies cannot be assessed, however, they cannot be used for any type of “public certification”—that is, for a company to show reasonable fairness guarantees to clients (e.g., as a commercial argument), or (even less) to regulators. Moreover, in many cases, the entity responsible for fairness should not be accessing sensitive information, even through proxies, for privacy reasons.
In FAIRPLAY, we investigate a different means of certifying fairness of decisions without having access to sensitive attributes, by partnering with a trusted third-party that collects protected attributes (that could for instance be a regulator or a public entity, such as Nielsen, say). We distinguish two cases:
- If the third-party and the company share a common identifier of users, then computing the fairness metric without leaking information to each other will boil down to a problem of secure multi-party computation (SMC). In such a case, there could be a need to be able to learn, which opens the topic of learning and privacy under SMC. This scenario, however, is likely not the most realistic one as having a common identifier requires a deep technical integration.
- If the third-party and the company do not share a common identifier of users, but there are common features that they both observe 90, then it is possible only to partially identify the joint distribution. With additional structural assumptions on the distribution, however, it could be identified accurately enough to estimate biases and fairness metrics. This becomes a distribution identification problem and brings a number of questions such as: how to do the distribution identification? how to optimally query data from the third party to train fair algorithms with high utility? etc. An important point to keep in mind in such a study is that it is likely that the third party user-base is different from that of the company. It will therefore be key to handle the covariate shift from one distribution to the other while estimating biases.
This distribution identification problem will be important in the context of privacy, even independently of fairness issues. Indeed, in the near future, most learning will happen in a privacy-preserving setting (for instance, because of the Chrome privacy sandbox). This will require new learning schemes (different from e.g., Empirical Risk Minimization) as samples from the usual joint distribution
In relation to the above problems, a key question is to determine what is the most appropriate definition of fairness to consider. Recall that it is well-known that usual fairness metrics are not compatible 94. Moreover, in online advertising, fairness can be measured at multiple different levels: at the level of bids, at the level of audience reached, at the level of clicking users, etc. Fairness at the level of bids does not imply fairness of the audience reached (see Theme 1); yet external auditors would measure which ad is displayed—as was done for some ad platforms 123—hence in terms of public image, that would be the appropriate level to define fairness. Intimately, the above problem relates to the question of measuring what is the relevant audience of an ad, which would define the label if one were to use the EO fairness notion. This label is typically not available. We can explore three ways to overcome this issue. The first is to find a sequential way to learn the label through users clicking on ads. The second and third options are to focus in a first step on individual fairness, or on counterfactual fairness 97, which has many possible different level of assumptions and was popularized in 2020 98. The notion of counterfactual is key in causality 117. A model is said counterfactually fair if its prediction does not change (too much) by intervening on the sensitive attribute. Several works already propose ways of designing models that are counterfactually fair 93, 130, 129. This seems to be quite an interesting, but challenging direction to follow.
Finally, an alternative direction would be to purse modeling the trade-off between privacy and fairness. For instance, in some game theoretic models, users can choose the quantity of data that they reveal 80, 54, so that the objective functions integrate different levels of fairness/privacy. Then those models model should be studied both in terms of equilibrium and in the online setup, with the objective of identifying how the strategic privacy considerations affect the fairness-utility tradeoff.
3.3 Theme 3: Exploiting structure in online algorithms and learning problems
Our last research direction is somewhat transverse, with possible application to improving algorithms in the first three themes. We explore how the underlying structure can be exploited, in the online and learning problems considered before, to improve performance. Note that, in all these problems, we will incorporate the fairness and privacy aspects even if they are somewhat transverse to the structure considered.1 The following sections are illustrating examples on how hidden structure can be leveraged in specific examples.
3.3.1 Leveraging structure in online matching
Finding large matchings in graphs is a longstanding problem with a rich history and many practical and theoretical applications 108, 85, 45, 44. Recall that given a graph
A natural measure for the quality of an online matching algorithm is the “competitive ratio” (CR): the ratio between the size of the created matching to the size of the optimal one 108. The seminal work 92 introduced an optimal algorithm for the adversarial case 48, that guarantees a CR of
A more recent approach restricts the distribution
-
Introducing correlation and inhomogeneity
We will start by deriving and studying optimal online matching strategies on widely studied classes of graphs that present simple inhomongeneity or correlation structures (which are often present in applications). The stochastic block model 40 is often used to generate graphs with an underlying community structure. It presents a simple correlation structure: two vertices in the same community are more likely to have a common neighbors than two vertices in different communities. Another focus point will be a generalized version of the Erdös-Renyi model, where the inplace vertices
are divided into sets , where generates an edge with probability . These two settings should give us a better understanding of how heterogeneity and correlation affect the matching performance.Deriving the competitive ratio implies to study the asymptotic size of maximum matchings in random graphs. Two methods are usually used. The first and constructive one is the study of the Karp-Sipser algorithm on the graph 49. The second one involves the rich theory of graph weak local convergence 50. A straightforward application of the methods, however, requires the graph to have independence properties; adapting them to graphs with a correlation structure will require new ideas.
-
Configuration models and random geometric graphs
A configuration model is described as follows (in the bi-partite case). Each vertex
has a number of half-edges drawn for the same distribution and each vertex has a number of half-edges drawn from (with the assumption that the expected total numbers of half edges from and are the same). Then a vertex that arrives in the sequential fashion has its half-edges “completed” by a (still free) half-edge of . This is a standard way of creating random graphs with (almost) fixed distribution of degrees. Here the question would simply be the competitive ratio of some greedy algorithm, whether the distributions and are known beforehand or learned on the fly. An interesting variant of this problem would be to assume the existence of a (hidden or not) geometric graph. Each is drawn i.i.d in (say a Gaussian centered at 0) and similarly for . Then there is an edge between and with a probability depending on the distance between them. Here again, interesting variants can be explored depending on whether the distribution is known or not, and whether the locations of and/or are observed or not. -
Learning while matching
In practical applications, the full stochastic structure of the graphs may not be known beforehand. This begs the question: what will happen to the performance of the algorithms if the graph parameters are learned while matching? In the generalized Erdös-Renyi graph, this will correspond to learning the probability of generating edges. For the stochastic block model, the matching algorithm will have to perform online community detection.
3.3.2 Exploiting side-information in sequential learning
We end with an open direction that may be relevant to many of the problems considered above: how to use side-information to speed-up learning. In many sequential learning problems where one receives feedback for each action taken, it is actually possible to deduce, for free, extra information from the known structure of the problem. However, how to incorporate that information in the learning process is often unclear. We describe it through two examples.
-
One-sided feedback in auctions
In online ad auctions, the advertisers' strategy is to bid in a compact set of possible bids. After placing a bid, the advertiser learns whether they won the auction or not; but even if they do not observe the bids of other advertisers, they can deduce for free some extra information: if they win they learn that they would have won with any higher bid and if they loose they learn that they would have lost with any lower bid 127, 61. We will investigate how to incorporate this extra information in RL procedures devised in Theme 1. One option is by leveraging the Kaplan-Meier estimator.
-
Side-information in dynamic resource allocation problems and matching
Generalizing the idea above, one can observe side-information in many other problems 42. Typically, in resource allocation problems (e.g., how to allocate a budget of ad impressions), one can leverage a monotony property: one would not have gained more by allocating less. Similarly, in matching with unknown weights, it is often possible upon doing a particular match to learn the weight of other potential pairs.
4 Application domains
4.1 Typical problems and use-cases
In FAIRPLAY, we focus mainly on problems involving learning that relate to Internet applications. We will tackle generic problems (in particular auctions and matching) with different applications in mind, in particular some applications in the context of Criteo's business but also others. A crucial property of those applications is the aforementioned ethical concerns, namely fairness and/or privacy. The team was shaped and motivated by several such use-cases, from more practical (with possible short or middle term applications in particular in Criteo products) to more theoretical and exploratory ones. We describe first here the main types of generic problems and use-cases considered in this context.
Auctions 96
There are many different types of auctions that an agent can use to sell one or several items in her possession to
Matching 95, 110
A matching is nothing more than a bi-partite graph between some agents (patients, doctors, students) and some resources (respectively, organs, hospital, schools). The objective is to figure out what is the “optimal” matching for a given criterion. Interestingly, there are two different—and mostly unrelated yet—concepts of “good matching”. The first one is called “stable” in the sense that each agent expresses preferences over resources (and vice-versa) and be such that no couple (agent-resource) that are un-paired would prefer to be paired together than with their current paired resource/agent. In the other concept of matching, agents and resources are described by some features (say vectors in
Matching is used is many different applications such as university admission (e.g., in Parcoursup). Notice that strategic interactions arise in matching if agents or resources can disclose their preferences/features to each other. Learning is also present as soon as not everything is known, e.g., the preferences or costs. Many applications of matching (again, such as college admission) are typical examples where fairness and privacy are of utmost importance. Finally, matching is also at the basis of several Internet applications and Criteo products, for instance to solve the problem of matching a given ad budget to fixed ad slots.
Ethical notions in those use-cases
In both problems, individual users are involved and there is a clear need to consider fairness and privacy. However, the precise motivation and instantiation of these notions depends on the specific use-case. In fact, it is often part of the research question to decide which are the most relevant fairness and privacy notions, as mentioned in Section 2.1. We will throughout the team's life put an important focus on this question, as well as on the question of the impact of the chosen notion on performance.
4.2 Application areas
In FAIRPLAY, we consider both applications to Criteo use-cases (online advertisement) and other applications (with other appropriate partners).
4.2.1 Online advertisement
Online advertising offers an application area for all of the research themes of FAIRPLAY; which we investigate primarily with Criteo.
First, online advertising is a typical application of online auctions and we consider applications of the work on auctions to Criteo use-cases, in particular the work on advertiser-centric fairness where the advertiser is Criteo. From a practical point of view, privacy will have to be enforced in such applications. For instance, when information is provided to advertisers to define audiences or to visualize the performance of their campaigns (insights) there is a possibility of leaking sensitive information on users. In particular, excellent proxies on protected attributes should probably not be leaked to advertisers, or transformed before (e.g., with the differential privacy techniques). This is therefore also an application of the fairness-vs-privacy research thread.
Note that, even before considering those questions, the first very important theoretical question is to determine what is the more appropriate definition of fairness (as there are, as mentioned above, many different variations) in those applications. We recall that it is well-known that usual fairness metrics are not compatible 94. Moreover, in online advertising, fairness can be measured in term of bidding and recommendation or in term of what ads are actually displayed. Being fair on bidding does not lead to fairness in ads displaying 111, mainly because of the other advertising actors. While fairness in bidding and/or recommendation seem the most important because they only rely on our models, external auditors can easily get data on which ads we display.
We will also investigate applications of fair matching techniques to online advertsing and to Criteo matching products—namely retargeting (personalized ads displayed on a website) and retail media (sponsored products on a merchant website). Indeed, one of Criteo major products, retail media can be cast as an online matching problem. On a given e-commerce website (say, target), several advertisers—currently brands—are running campaigns so that their products are “sponsored” or “boosted”, i.e., they appear higher on the list of results of a given query. The budgets (from a Criteo perspective) must be cleared (daily, monthly or annually). This constraint is easy thanks to the high traffic, but the main issue is that, without control/pacing/matching in times, the budget is depleted after only a few hours on a relatively low quality traffic (i.e., users that generate few conversions hence a small ROI for the advertisers). The question is therefore whether an individual user should be matched or not to boosted/sponsored products at a given time so that the ROI of the advertisers is maximized, the campaign budget is depleted and the retailer does not suffer too much from this active corruption of its organic results. Those are three different and concurrent objectives (for respectively the advertisers, Criteo and the retailers) that must be somehow conciliated. This problem (and more generally this type of problems) offers a rich application area to the FAIRPLAY research program. Indeed, it is crucial to ensure that fairness and privacy are respected. On the other hand, users, clicks, conversion arrival are not “worst case”. They rather follow some complicated—but certainly learnable—process; which allows applying our results on exploiting structure.
4.2.2 “Physical matching”
We investigate a number of other applications of matching: assignment of daycare slots to kids, mutation of professors to different academies, assignment of kidneys to patients, assignment of job applicants to jobs. In all these applications, there are crucial constraints of fairness that complicate the matching. We leverage existing partnership with the CNAF, the French Ministry of Education and the Agence de la biomédecine in Paris for the first three applications; for the last we will consolidate a nascent partnership with Pole Emploi and LinkedIn.
5 Highlights of the year
Simon Mauras joined the team in Feb 2024 as a CR.
The team's kickoff meeting took place on Oct 3rd in the Turing building.
The team had a record number of 11 papers accepted at NeurIPS'24.
5.1 Awards
Dr. Solenne Gaucher, a postdoc in the team, received the “L’Oréal-UNESCO Young Talents Award”.
6 New results
6.1 Auctions and mechanism design
Participants: Benjamin Heymann, Patrick Loiseau, Simon Mauras, Vianney Perchet, Hugo Richard.
6.1.1 Auctions
Most of the display advertising inventory is sold through real-time auctions. The participants of these auctions are typically bidders (Google, Criteo, RTB House, and Trade Desk for instance) that participate on behalf of advertisers. In order to estimate the value of each display opportunity, they usually train advanced machine learning algorithms using historical data. In the labeled training set, the inputs are vectors of features representing each display opportunity, and the labels are the generated rewards. In practice, the rewards are given by the advertiser and are tied to whether a particular user converts. Consequently, the rewards are aggregated at the user level and never observed at the display level. A fundamental task that has, to the best of our knowledge, been overlooked is to account for this mismatch and split, or attribute, the rewards at the right granularity level before training a learning algorithm. We call this the label attribution problem. In 4, we develop an approach to the label attribution problem, which is both theoretically justified and practical. In particular, we develop a fixed point algorithm that allows for large-scale implementation and showcase our solution using a large-scale publicly available data set from Criteo, a large demand-side platform. We dub our approach the fixed point label attribution algorithm. Managerial implications: There is often a hidden leap of faith when transforming the advertiser’s signal into display labeling. Demand Side Platforms providers should be careful when building their machine learning pipeline and carefully solve the label attribution step.
In 8, we use a fixed point gradient flow algorithm to compute the equilibria of first-price procurement auctions in the presence of losses and Bayesian priors. We use this efficient algorithm to compare optimal, first-price and VCG auctions. This allows us to numerically estimate the social cost of sub-optimality of the nodal pricing mechanism in wholesale electricity markets. We also derive a closed form expression of the optimal mechanism procurement cost when the types are uniformly distributed. In 9, we discuss a procurement problem with transportation losses and piecewise linear production costs. We first provide an algorithm based on Knaster-Tarski’s fixed point theorem to solve the allocation problem in the quadratic losses case. We then identify a monotony condition on the types distribution under which the Bayesian cost minimizing mechanism takes a simple form.
In 11, motivated by online display advertising, we consider repeated second-price auctions, where agents sample their value from an unknown distribution with cumulative distribution function F . In each auction t, a decision-maker bound by limited observations selects n t agents from a coalition of N to compete for a prize with p other agents, aiming to maximize the cumulative reward of the coalition across all auctions. The problem is framed as an N -armed structured bandit, each number of player sent being an arm n, with expected reward r(n) fully characterized by F and p + n. We present two algorithms, Local-Greedy (LG) and Greedy-Grid (GG), both achieving constant problem-dependent regret. This relies on three key ingredients: 1. an estimator of r(n) from feedback collected from any arm k, 2. concentration bounds of these estimates for k within an estimation neighborhood of n and 3. the unimodality property of r under standard assumptions on F . Additionally, GG exhibits problem-independent guarantees on top of best problem-dependent guarantees. However, by avoiding to rely on confidence intervals, LG practically outperforms GG, as well as standard unimodal bandit algorithms such as OSUB or multi-armed bandit algorithms.
In 25, motivated by the strategic participation of electricity producers in electricity day-ahead market, we study the problem of online learning in repeated multi-unit uniform price auctions focusing on the adversarial opposing bid setting. The main contribution of this paper is the introduction of a new modeling of the bid space. Indeed, we prove that a learning algorithm leveraging the structure of this problem achieves a regret of
6.1.2 Dynamic pricing
In 26, we proposed improved algorithms for contextual dynamic pricing.
In contextual dynamic pricing, a seller sequentially prices goods based on contextual information. Buyers will purchase products only if the prices are below their valuations.
The goal of the seller is to design a pricing strategy that collects as much revenue as possible. We focus on two different valuation models. The first assumes that valuations linearly depend on the context and are further distorted by noise. Under minor regularity assumptions, our algorithm achieves an optimal regret bound of
6.1.3 Mechanism design
In 17, we study auction design within the widely acclaimed model of interdependent values, introduced by Milgrom and Weber [1982]. In this model, every bidder
In 19, we study best-of-both-worlds guarantees for the fair division of indivisible items among agents with subadditive valuations. Our main result
establishes the existence of a random allocation that is simultaneously ex-ante
In 6, we propose and theoretically analyze a measure to encourage greater voluntary contributions to public goods. Our measure is a simple intervention that restricts individuals’ strategy sets by imposing a minimum individual contribution level while still allowing for full free riding for those who do not want to contribute. We show that for a well-chosen value of the minimum individual contribution level, this measure does not incentivize any additional free riding while strictly increasing the total contributions relative to the situation without the minimum contribution level. Our measure is appealing because it is nonintrusive and in line with the principle of “freedom of choice.” It is easily implementable for many different public goods settings where more intrusive measures are less accepted. This approach has been implemented in practice in some applications, such as charities.
6.2 Matching
Participants: Matthieu Lerasle, Patrick Loiseau, Vianney Perchet.
In 7, we tackle the pair-matching problem. The pair-matching problem appears in many applications where one wants to discover good matches between pairs of entities or individuals. Formally, the set of individuals is represented by the nodes of a graph where the edges, unobserved at first, represent the good matches. The algorithm queries pairs of nodes and observes the presence/absence of edges. Its goal is to discover as many edges as possible with a fixed budget of queries. Pair-matching is a particular instance of multi-armed bandit problem in which the arms are pairs of individuals and the rewards are edges linking these pairs. This bandit problem is non-standard though, as each arm can only be played once. Given this last constraint, sub-linear regret can be expected only if the graph presents some underlying structure. This paper shows that sub-linear regret is achievable in the case where the graph is generated according to a Stochastic Block Model (SBM) with two communities. Optimal regret bounds are computed for this pair-matching problem. They exhibit a phase transition related to the Kesten-Stigum threshold for community detection in SBM. The pair-matching problem is considered in the case where each node is constrained to be sampled less than a given amount of times. We show how optimal regret rates depend on this constraint. The paper is concluded by a conjecture regarding the optimal regret when the number of communities is larger than 2. Contrary to the two communities case, we argue that a statistical-computational gap would appear in this problem.
In 18, we address the challenge of actively ranking a set of items/players with varying values/strengths. The comparison outcomes are random, with a greater noise the closer the values. A crucial requirement is that, at each iteration of the algorithm, all items must be compared once, i.e., an iteration is a perfect matching. Furthermore, we presume that comparing two players with closely matched strengths incurs no cost and, in contrast, a unit cost is associated with comparing players whose strength difference is more substantial. Our secondary objective is to determine an optimal matching between players based on this cost function: we propose and analyze an algorithm that draws on concepts from both AKS sorting networks and bandit theory. Our algorithm achieves both objectives with high probability, and the total cost is optimal (up to logarithmic terms).
6.3 Learning
Participants: Cristina Butucea, Matthieu Lerasle, Patrick Loiseau, Vianney Perchet, Maxime Vono, Hugo Richard.
6.3.1 Data valuation
In 20, we consider the dataset valuation problem, that is the problem of quantifying the incremental gain, to some relevant pre-defined utility of a machine learning task, of aggregating an individual dataset to others.The Shapley value is a natural tool to perform dataset valuation due to its formal axiomatic justification, which can be combined with Monte Carlo integration to overcome the computational tractability challenges. Such generic approximation methods, however, remain expensive in some cases. In this paper, we exploit the knowledge about the structure of the dataset valuation problem to devise more efficient Shapley value estimators. We propose a novel approximation, referred to as discrete uniform Shapley, which is expressed as an expectation under a discrete uniform distribution with support of reasonable size. We justify the relevancy of the proposed framework via asymptotic and non-asymptotic theoretical guarantees and illustrate its benefits via an extensive set of numerical experiments.
6.3.2 Bandits and reinforcement learning
In 5, we provide a survey of recent results in multi-player bandits.
In 12, we consider the classical multi-armed bandit problem, but with strategic arms. In this context, each arm is characterized by a bounded support reward distribution and strategically aims to maximize its own utility by potentially retaining a portion of its reward, and disclosing only a fraction of it to the learning agent. This scenario unfolds as a game over
In 23, we look at The Value of Reward Lookahead in Reinforcement Learning. In reinforcement learning (RL), agents sequentially interact with changing environments while aiming to maximize the obtained rewards. Usually, rewards are observed only after acting, and so the goal is to maximize the expected cumulative reward. Yet, in many practical settings, reward information is observed in advance – prices are observed before performing transactions; nearby traffic information is partially known; and goals are oftentimes given to agents prior to the interaction. In this work, we aim to quantifiably analyze the value of such future reward information through the lens of competitive analysis. In particular, we measure the ratio between the value of standard RL agents and that of agents with partial future-reward lookahead. We characterize the worst-case reward distribution and derive exact ratios for the worst-case reward expectations. Surprisingly, the resulting ratios relate to known quantities in offline RL and reward-free exploration. We further provide tight bounds for the ratio given the worst-case dynamics. Our results cover the full spectrum between observing the immediate rewards before acting to observing all the rewards before the interaction starts. In 24, we study reinforcement learning (RL) problems in which agents observe the reward or transition realizations at their current state before deciding which action to take. Such observations are available in many applications, including transactions, navigation and more. When the environment is known, previous work shows that this lookahead information can drastically increase the collected reward. However, outside of specific applications, existing approaches for interacting with unknown environments are not well-adapted to these observations. In this work, we close this gap and design provably-efficient learning algorithms able to incorporate lookahead information. To achieve this, we perform planning using the empirical distribution of the reward and transition observations, in contrast to vanilla approaches that only rely on estimated expectations. We prove that our algorithms achieve tight regret versus a baseline that also has access to lookahead information - linearly increasing the amount of collected reward compared to agents that cannot handle lookahead information.
6.4 Online (and offline) algorithms
Participants: Patrick Loiseau, Simon Mauras, Vianney Perchet.
6.4.1 Prophet inequalities
In 13, we study Lookback Prophet Inequalities. Prophet inequalities are fundamental optimal stopping problems, where a decision-maker observes sequentially items with values sampled independently from known distributions, and must decide at each new observation to either stop and gain the current value or reject it irrevocably and move to the next step. This model is often too pessimistic and does not adequately represent real-world online selection processes. Potentially, rejected items can be revisited and a fraction of their value can be recovered. To analyze this problem, we consider general decay functions
In 14, we consider a hiring process with candidates coming from different universities. It is easy to order candidates with the same background, yet it can be challenging to compare them otherwise. The latter case requires additional costly assessments, leading to a potentially high total cost for the hiring organization. Given an assigned budget, what would be an optimal strategy to select the most qualified candidate? We model the above problem as a multicolor secretary problem, allowing comparisons between candidates from distinct groups at a fixed cost. Our study explores how the allocated budget enhances the success probability in such settings.
In 22, we study online selection problems in both the prophet and secretary settings, when arriving agents have interdependent values. In the interdependent values model, introduced in the seminal work of Milgrom and Weber [1982], each agent has a private signal and the value of an agent is a function of the signals held by all agents. Results in online selection crucially rely on some degree of independence of values, which is conceptually at odds with the interdependent values model. For prophet and secretary models under the standard independent values assumption, prior works provide constant factor approximations to the welfare. On the other hand, when agents have interdependent values, prior works in Economics and Computer Science provide truthful mechanisms that obtain optimal and approximately optimal welfare under certain assumptions on the valuation functions. We bring together these two important lines of work and provide the first constant factor approximations for prophet and secretary problems with interdependent values. We consider both the algorithmic setting, where agents are non-strategic (but have interdependent values), and the mechanism design setting with strategic agents. All our results are constructive and use simple stopping rules.
6.4.2 Algorithms with prediction and scheduling
In 15, we address the problem of learning-augmented priority queueus. Priority queues are one of the most fundamental and widely used data structures in computer science. Their primary objective is to efficiently support the insertion of new elements with assigned priorities and the extraction of the highest priority element. In this study, we investigate the design of priority queues within the learning-augmented framework, where algorithms use potentially inaccurate predictions to enhance their worst-case performance. We examine three prediction models spanning different use cases, and we show how the predictions can be leveraged to enhance the performance of priority queue operations. Moreover, we demonstrate the optimality of our solution and discuss some possible applications.
In 16, we consider the non-clairvoyant scheduling problem. The non-clairvoyant scheduling problem has gained new interest within learning-augmented algorithms, where the decision-maker is equipped with predictions without any quality guarantees. In practical settings, access to predictions may be reduced to specific instances, due to cost or data limitations. Our investigation focuses on scenarios where predictions for only
In 21, we consider the question: How should an expert send forecasts to maximize her utility subject to passing a calibration test? We consider a dynamic game where an expert sends probabilistic forecasts to a decision maker. The decision maker uses a calibration test based on past outcomes to verify the expert's forecasts. We characterize the optimal forecasting strategy by reducing the dynamic game to a static persuasion problem. A distribution of forecasts is implementable by a calibrated strategy if and only if it is a mean-preserving contraction of the distribution of conditionals (honest forecasts). We characterize the value of information by comparing what an informed and uninformed expert can attain. Moreover, we consider a decision maker who uses regret minimization, instead of the calibration test, to take actions. We show that the expert can achieve the same payoff against a regret minimizer as under the calibration test, and in some instances, she can achieve strictly more.
In 10, we consider algorithms for scheduling deadline-sensitive malleable tasks. Due to the ubiquity of batch data processing, the related problems of scheduling malleable batch tasks have received significant attention. We consider a fundamental model where a set of tasks is to be processed on multiple identical machines and each task is specified by a value, a workload, a deadline and a parallelism bound. Within the parallelism bound, the number of machines assigned to a task can vary over time without affecting its workload. In this paper, we identify a boundary condition and prove by construction that a set of malleable tasks with deadlines can be finished by their deadlines if and only if it satisfies the boundary condition. This core result plays a key role in the design and analysis of scheduling algorithms: (i) when several typical objectives are considered, such as social welfare maximization, machine minimization, and minimizing the maximum weighted completion time, and, (ii) when the algorithmic design techniques such as greedy and dynamic programming are applied to the social welfare maximization problem. As a result, we give four new or improved algorithms for the above problems.
7 Partnerships and cooperations
7.1 International research visitors
7.1.1 Visits of international scientists
Other international visits to the team
H. Stein
-
Status:
PhD
-
Institution of origin:
Heidelberg University
-
Country:
Germany
-
Dates:
May (1week), 2024
-
Context of the visit:
Research stay
J. Correa
-
Status:
Professor
-
Institution of origin:
Universidad de Chile
-
Country:
Chile
-
Dates:
November (1week), 2024
-
Context of the visit:
Research stay
7.1.2 Visits to international teams
Research stays abroad
Cristina Butucea
-
Visited institution:
Rostock University
-
Country:
Germany
-
Dates:
September 1-5, 2024
-
Context of the visit:
Research, discussions
-
Visited institution:
Freiburg University
-
Country:
Germany
-
Dates:
March (1 week), 2024
-
Context of the visit:
Research, discussions
7.2 National initiatives
Foundry (PEPR IA)
Participants: Patrick Loiseau.
-
Title:
Foundry: Foundation of robustness and reliability in AI
-
Partner Institution(s):
- Inria
- CNRS
- Université Paris Dauphine
- Institut Mines Telecom
- ENS de Lyon
-
Date/Duration:
2023-2027 (4 years)
-
Additionnal info/keywords:
PEPR IA projet cible, 245k euros. Fairness, matching, auctions.
FairPlay (ANR JCJC)
Participants: Patrick Loiseau.
-
Title:
FairPlay: Fair algorithms via game theory and sequential learning
-
Partner Institution(s):
- Inria
-
Date/Duration:
2021-2025 (4 years)
-
Additionnal info/keywords:
ANR JCJC project, 245k euros. Fairness, matching, auctions.
BOLD (ANR)
Participants: Vianney Perchet.
-
Title:
BOLD: Beyond Online Learning for Better Decisions
-
Partner Institution(s):
- Crest, Genes
-
Date/Duration:
2019-2024 (4.5 years)
-
Additionnal info/keywords:
ANR project, 270k euros. online learning, optimization, bandits.
DOOM (ANR)
Participants: Vianney Perchet.
-
Title:
DOOM: Design of Optimal Online Matching Markets
-
Partner Institution(s):
- Crest, Genes
-
Date/Duration:
2014-2028 (4 years)
-
Additionnal info/keywords:
ANR project, 456k euros. online markets, learning, matching.
8 Dissemination
8.1 Promoting scientific activities
8.1.1 Scientific events: organisation
General chair, scientific chair
Participants: Cristina Butucea.
-
Title:
Asymptotically equivalent representations of non-stationary processes.
-
Partner Institution(s):
- Crest, Genes
-
Date/Duration:
March 4-15
-
Location:
CIRM, Marseille
Member of the organizing committees
Participants: Vianney Perchet.
-
Title:
Member of the scientific advisory committee of the Hi! Paris summer school
-
Date/Duration:
July (1 week)
-
Location:
HEC campus
8.1.2 Scientific events: selection
Chair of conference program committees
Participants: Patrick Loiseau.
-
Title:
Program chair of the Hi! Paris summer school
-
Date/Duration:
July (1 week)
-
Location:
HEC campus
Member of the conference program committees
-
Patrick Loiseau:
NeurIPS, ECML-PKDD (Area Chair)
-
Vianney Perchet:
NeurIPS, ICLR, ICML, COLT, ALT
-
Simon Mauras:
WINE, EC.
Reviewer
-
Hugo Richard:
NeurIPS, ICML, AISTATS
-
Clément Calauzènes:
ICML, NeurIPS
-
Benjamin Heymann
AISTATS
-
Maxime Vono
NeurIPS
-
Simon Mauras
ESA, SODA, STOC
8.1.3 Journal
Member of the editorial boards
-
Vianney Perchet:
Foundations and Trends in Machine Learning, Operation Research, Operation Research Letters, Journal of Machine Learning Research, Journal of Dynamics and Games,
-
Cristina Butucea:
Annals of Statistics, Bernoulli
Reviewer - reviewing activities
-
Patrick Loiseau:
Dynamic games and applications, Operation Research letters
-
Vianney Perchet:
Annals of Statistics, Mathematics of Operation Research, Journal of the ACM
-
Cristina Butucea
Annals of Statistics, Bernoulli, IEEE Transactions on Information Theory
-
Matthieu Lerasle
Annals of statistics, Journal of the European Mathematical Society, Probability and Related Fields, Journal of Machine Learning Research, Journal of the American Statistical Association.
-
Marc Abeille:
Journal of Machine Learning Research
-
Maxime Vono
Journal of Computational and Graphical Statistics
-
Benjamin Heymann:
SIAM Control and Optimization, Mathematical Reviews
-
Simon Mauras:
Annals of Mathematics and Artificial Intelligence, Mathematics of Operations Research, Operations Research Letters, Games and Economic Behavior
8.1.4 Invited talks
-
Cristina Butucea
Anniversary Conference V. Koltchinskii (Bertinoro, Italy), ISNP (Braga, Portugal), MFOberwolfach Germany, Journées de la SMAI (Poitiers), University of Nanjing (China)
-
Hugo Richard
PGMO Days (France)
-
Simon Mauras
Junior Workshop on Mathematical Game Theory (Rome), Algorithms, Learning, and Games (Sicily), Alpine Game Theory Symposium (Grenoble), Game theory seminar (Paris), Optimization seminar (Paris)
8.1.5 Scientific expertise
-
Vianney Perchet:
Expert for the evaluation of the LABEX MME:DII
-
Patrick Loiseau:
External expert for the ANR and the Royal Society
-
Cristina Butucea:
ANR evaluation committee, hiring committees France and Germany,
8.2 Teaching - Supervision - Juries
8.2.1 Supervision
-
Patrick Loiseau:
PhD students: Rémi Castera, Mathieu Molina, Reda Jalal, Minrui Xu, Marie Generali, Melissa Tamine; postdocs: Felipe Garrido Lucero, Simon Finster, Denis Sokolov
-
Vianney Perchet:
PhD students: Come Fiegel, Maria Cherifa, Mathieu Molina, Ziyad Benomar, Mike Liu, Hafedh El Ferchichi, Matilde Tullii, Giovanni Montanari. postdocs: Felipe Garrido Lucero, Nadav Merlis, Solenne Gaucher, Dorian Baudry
-
Matthieu Lerasle
PhD Students: Clara Carlier, Hugo Chardon, Hafedh El Ferchichi.
-
Cristina Butucea
PhD students: Nayel Bettache, Henning Stein, Antoine Schoonaert
-
Marc Abeille:
Ahmed Ben Yahmed
-
Clément Calauzènes:
Morgane Goibert, Maria Cherifa
-
Benjamin Heymann:
Mélissa Tamine
-
Maxime Vono:
Mélissa Tamine
-
Simon Mauras:
PhD student: Minrui Xu.
8.3 Teaching
-
ENSAE:
-
Advanced OptimizationThird year, lectures
-
Theoretical Foundations of Machine LearningSecond year, lectures
-
Stopping time and online algorithmsThird year, lectures
-
Statistics(ML) 1st and second year
-
Nonparametric Statistics3rd year, M2
-
Mathematical Foundations of Probabilities1st year
-
Advanced Optimization
-
Ecole Polytechnique:
-
INF421: design and analysis of algorithms(Patrick Loiseau and Simon Mauras). Second-year level, PCs.
-
INF581: Advanced Machine Learning and Autonomous Agents(Patrick Loiseau). Third-year/M1 level, lectures and labs.
-
MAP433: Statistics(ML). First-year cycle polytechnicien, PCs.
-
MAP576: Learning Theory(ML). Second-year cycle polytechnicien, Lecture.
-
INF471S: Algorithms and Advanced Programming(Simon Mauras) Bachelor and cycle polytechnicien, lectures and tutorials.
-
INF421: design and analysis of algorithms
-
Université Paris-Saclay:
-
High Dimensional Probability
(ML). Master 2
-
Stopping Time and Random Algorithm
(ML). Master 2
-
High Dimensional Probability
-
PSL:
-
Introduction to machine learning
(Hugo Richard). L3 level, Lectures and labs.
-
Introduction to machine learning
-
Master IASD:
-
Recommender Systems
(Clément Calauzènes). Master 2, Lectures.
-
Recommender Systems
8.4 Popularization
8.4.1 Specific official responsibilities in science outreach structures
-
Simon Mauras:
- Future co-responsable médiation at INRIA Saclay.
- Volunteer and treasurer of association France-IOI: training and selecting the french team for the International Olympiads in Informatics; organizing summer camps to teach algorithmic to groups of
20 highschool students, creating online contests and educational content about informatics (Concours castor, concours Alkindi, Concours Algoréa, online content for SNT option in highschool)
9 Scientific production
9.1 Major publications
- 1 inproceedingsLocal and adaptive mirror descents in extensive-form games.2023 International Conference on Machine LearningNew Orleans, United StatesSeptember 2023HAL
- 2 inproceedingsTrading-off price for data quality to achieve fair online allocation.NeurIPS 2023 - 37th Conference on Neural Information Processing SystemsNew orleans, USA, United States2023, 1-43HAL
- 3 articleOptimal Change-Point Detection and Localization.Annals of Statistics5142023, 1586-1610HALDOI
9.2 Publications of the year
International journals
- 4 articleFixed Point Label Attribution for Real-Time Bidding.Manufacturing and Service Operations Management263May 2024, 1043-1061HALDOIback to text
- 5 articleA survey on multi-player bandits.Journal of Machine Learning ResearchJanuary 2024HALback to text
- 6 articleEnhancing voluntary contributions in a public goods economy via a minimum individual contribution level.Public Choice2011-2April 2024, 237-261HALDOIback to text
- 7 articlePair-Matching: Link Prediction with Adaptive Queries.Mathematical Statistics and LearningMarch 2024HALback to text
- 8 articleProcurement auctions with losses.Computational Management Science212July 2024, 38HALDOIback to text
- 9 articleReverse auctions with transportation and convex costs.Computational Management Science211February 2024, 20HALDOIback to text
- 10 articleAlgorithms for Scheduling Deadline-Sensitive Malleable Tasks.SN Operations Research Forum52April 2024, 30HALDOIback to text
International peer-reviewed conferences
- 11 inproceedingsOptimizing the coalition gain in Online Auctions with Greedy Structured Bandits.NeurIPS 2024NeurIPS 2024 - 38th Conference on Neural Information Processing SystemsVancouver (BC), CanadaDecember 2024HALback to text
- 12 inproceedingsStrategic Multi-Armed Bandit Problems Under Debt-Free Reporting.NeurIPS 2024 - 38th Annual Conference on Neural Information Processing SystemsVancouver, CanadaDecember 2024HALback to text
- 13 inproceedingsLookback Prophet Inequalities.NeurIPS 2024 - 38th Conference on Neural Information Processing SystemsVancouver (CA), CanadaDecember 2024HALback to text
- 14 inproceedingsAddressing Bias in Online Selection with Limited Budget of Comparisons.PMLRNeurIPS 2024 - 38th Conference on Neural Information Processing SystemsVancouver (CA), CanadaDecember 2024HALback to text
- 15 inproceedingsLearning-Augmented Priority Queues.NeurIPS 2024 - 38th Conference on Neural Information Processing SystemsVancouver (CA), CanadaDecember 2024HALback to text
- 16 inproceedingsNon-clairvoyant Scheduling with Partial Predictions.Proceedings of the 41 st International Conference on MachineLearning, Vienna, Austria. PMLR 235, 2024ICML 2024 - The 41st International Conference on Machine LearningVienna, AustriaJuly 2024HALback to text
- 17 inproceedingsPrivate Interdependent Valuations: New Bounds for Single-Item Auctions and Matroids.EC '24: Proceedings of the 25th ACM Conference on Economics and ComputationEC 2024 - 25th ACM Conference on Economics and ComputationNew Haven, United StatesACMJuly 2024, 448-464HALDOIback to text
- 18 inproceedingsActive Ranking and Matchmaking, with Perfect Matchings.Proceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024ICML 2024 - The Forty-First International Conference on Machine LearningVienne, AustriaMay 2024HALback to text
- 19 inproceedingsBreaking the Envy Cycle: Best-of-Both-Worlds Guarantees for Subadditive Valuations.EC '24: Proceedings of the 25th ACM Conference on Economics and ComputationEC 2024 - 25th ACM Conference on Economics and ComputationNew Haven, United StatesACMJuly 2024, 1236 - 1266HALDOIback to text
- 20 inproceedingsDU-Shapley: A Shapley Value Proxy for Efficient Dataset Valuation.Advances in Neural Information Processing Systems 37 (NeurIPS 2024)Advances in Neural Information Processing Systems 37 (NeurIPS 2024)Vancouver, Canada2024HALback to text
- 21 inproceedingsCalibrated Forecasting and Persuasion.EC 2024 - The Twenty-Fifth ACM Conference on Economics and ComputationNew haven, United StatesJune 2024HALback to text
- 22 inproceedingsOptimal Stopping with Interdependent Values.EC '24: Proceedings of the 25th ACM Conference on Economics and ComputationEC 2024 - 25th ACM Conference on Economics and ComputationNew Haven, United StatesACMJuly 2024, 246-265HALDOIback to text
- 23 inproceedingsThe Value of Reward Lookahead in Reinforcement Learning.NeurIPS 2024 - 38th Conference on Neural Information Processing SystemsVancouver, CanadaDecember 2024HALback to text
- 24 inproceedingsReinforcement Learning with Lookahead Information.NeurIPS 2024 - 38th Conference on Neural Information Processing SystemsVancouver, CanadaDecember 2024HALback to text
- 25 inproceedingsImproved learning rates in multi-unit uniform price auctions.NeurIPS 2024NeurIPS 2024 - 38th Conference on Neural Information Processing SystemsVancouver (BC), CanadaDecember 2024HALback to text
- 26 inproceedingsImproved Algorithms for Contextual Dynamic Pricing.NeurIPS 2024 - 38th Conference on Neural Information Processing SystemsNeurIPS 2024 - The 38th Annual Conference on Neural Information Processing SystemsVancouver (BC), CanadaarXiv2024HALback to text
Doctoral dissertations and habilitation theses
- 27 thesisEfficiency and Fairness in Matching Problems.UGA (Université Grenoble Alpes)October 2024HAL
Reports & preprints
- 28 miscMode Estimation with Partial Feedback.2024HALDOI
- 29 miscMulti-Armed Bandits with Guaranteed Revenue per Arm.January 2024HAL
- 30 miscTwo-sided Matrix Regression.January 2024HAL
- 31 miscOff-the-grid prediction and testing for linear combination of translated features.July 2024HAL
- 32 miscSimultaneous off-the-grid learning of mixtures issued from a continuous dictionary.February 2024HAL
- 33 miscCorrelation of Rankings in Matching Markets.February 2024HAL
- 34 miscDemographic parity in regression and classification within the unawareness framework.September 2024HAL
- 35 miscSupervised Contamination Detection, with Flow Cytometry Application.April 2024HAL
- 36 miscA pragmatic policy learning approach to account for users' fatigue in repeated auctions.July 2024HAL
- 37 miscLearning in Games with progressive hiding.September 2024HAL
- 38 miscPosition Paper: Open Research Challenges for Private Advertising Systems under Local Differential Privacy.February 2024HAL
9.3 Cited publications
- 39 unpublished24 CFR § 100.75 - Discriminatory advertisements, statements and notices.., https://www.law.cornell.edu/cfr/text/24/100.75back to text
- 40 articleCommunity detection and stochastic block models: recent developments.The Journal of Machine Learning Research1812017, 6446--6531back to text
- 41 inproceedingsDiscrimination through optimization: How Facebook's ad delivery can lead to skewed outcomes.CSCW2019back to text
- 42 inproceedingsOnline learning with feedback graphs: Beyond bandits.Annual Conference on Learning Theory40Microtome Publishing2015back to text
- 43 articleOn matching and thickness in heterogeneous dynamic markets.Operations Research6742019, 927--949back to text
- 44 miscKidney Exchange in Dynamic Sparse Heterogenous Pools.2013back to text
- 45 inproceedingsOnline stochastic optimization in the large: Application to kidney exchange.Twenty-First International Joint Conference on Artificial Intelligence2009back to text
- 46 inproceedingsImproved bounds for online stochastic matching.European Symposium on AlgorithmsSpringer2010, 170--181back to text
- 47 inproceedingsObtaining fairness using optimal transport theory. arXiv:1806.031952018, 1--25back to text
- 48 articleOn-line bipartite matching made simple.Acm Sigact News3912008, 80--87back to text
- 49 articleThe width of random graph orders.The Mathematical Scientist2001 1995back to text
- 51 bookOnline Computation and Competitive Analysis.Cambridge University Presss1998back to text
- 52 miscAn Experimental Study of Algorithms for Online Bipartite Matching.2018back to text
- 53 inproceedingsSIC-MMAB: Synchronisation Involves Communication in Multiplayer Multi-Armed Bandits.arXiv:1809.081512018, 1--31back to text
- 54 inproceedingsUtility/Privacy Trade-off through the lens of Optimal Transport.Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics108Proceedings of Machine Learning ResearchOnlinePMLRAugust 2020, 591--601back to textback to text
- 55 articleRegret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems.Machine Learning512012, 1--122back to text
- 56 articleBounded regret in stochastic multi-armed bandits.Journal of Machine Learning Research: Workshop and Conference Proceedings (COLT)302013, 122--134back to text
- 57 inproceedingsOnline Primal-Dual Algorithms for Maximizing Ad-Auctions Revenue.Berlin, Heidelberg2007, 253--264back to textback to text
- 58 articleLocal differential privacy: Elbow effect in optimal density estimation and adaptation over Besov ellipsoids.Bernoulli2632020, 1727--1764back to text
- 59 articleInteractive versus non-interactive locally, differentially private estimation: Two elbows for the quadratic functional.Annals of Statsto appear2023back to text
- 60 inproceedingsToward Controlling Discrimination in Online Ad Auctions.ICML2019back to textback to text
- 61 inproceedings Regret minimization for reserve prices in second-price auctions.Proceedings of SODA 20132013back to text
- 62 bookPrediction, Learning, and Games.Cambridge University Press2006back to text
- 63 inproceedingsCapacity bounded differential privacy.Advances in Neural Information Processing Systems2019, 3469--3478back to text
- 64 miscFairness in ad auctions through inverse proportionality.arXiv:2003.139662020back to textback to text
- 65 inproceedingsRandomized online matching in regular graphs.Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete AlgorithmsSIAM2018, 960--979back to text
- 66 inproceedingsPacing equilibrium in first-price auction markets.EC2019back to text
- 67 inproceedingsMultiplicative pacing equilibria in auction markets.WINE2018back to text
- 68 inproceedingsProphet inequalities for iid random variables from an unknown distribution.Proceedings of the 2019 ACM Conference on Economics and Computation2019, 3--17back to text
- 69 inproceedingsOn the Compatibility of Privacy and Fairness.FairUMAP2019back to text
- 70 miscAmazon scraps secret AI recruiting tool that showed bias against women.Reuters, https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G2018back to text
- 71 articleDifferential privacy.Encyclopedia of Cryptography and Security2011, 338--340back to text
- 72 inproceedingsFairness through awareness.ITCS2012back to text
- 73 inproceedingsCalibrating noise to sensitivity in private data analysis.Theory of cryptography conferenceSpringer2006, 265--284back to text
- 74 techreportOptimal Privacy-Constrained Mechanisms.C.E.P.R. Discussion Papers2019back to text
- 75 inproceedingsOn the Effect of Positive Discrimination on Multistage Selection Problems in the Presence of Implicit Variance.EC2020back to text
- 76 miscOnline Stochastic Matching: Beating 1-1/e.2009back to text
- 77 techreportFairness in Precision Medicine.Data & Society2018back to text
- 78 bookGame Theory.MIT press1991back to text
- 79 articleLocal Differentially Private Regret Minimization in Reinforcement Learning.arXiv preprint arXiv:2010.077782020back to text
- 80 articleLinear Regression from Strategic Data Sources.ACM Transactions on Economics and Computation82May 2020, 10:1--10:24back to text
- 81 inproceedingsA Refined Mean Field Approximation.SIGMETRICS2017back to text
- 82 articleProxy fairness.arXiv preprint arXiv:1806.112122018back to text
- 83 inproceedingsBias in online freelance marketplaces: Evidence from taskrabbit and fiverr.CSCW2017back to text
- 84 inproceedingsEquality of Opportunity in Supervised Learning.NIPS2016back to textback to text
- 85 inproceedingsOnline task assignment in crowdsourcing markets.Twenty-sixth AAAI conference on artificial intelligence2012back to textback to text
- 86 inproceedingsMulti-Category Fairness in Sponsored Search Auctions.FAT*2020back to textback to text
- 87 articleMean field equilibria of dynamic auctions with learning.Management Science60122014, 2949--2970back to text
- 88 articleOnline stochastic matching: New algorithms with better bounds.Mathematics of Operations Research3932014, 624--646back to text
- 89 articleOn the consistency of supervised learning with missing values.arXiv preprint arXiv:1902.069312019back to text
- 90 articleAssessing algorithmic fairness with unobserved protected class using data combination.arXiv preprint arXiv:1906.002852019back to text
- 91 inproceedingsOnline bipartite matching with unknown distributions.Proceedings of the forty-third annual ACM symposium on Theory of computing2011, 587--596back to text
- 92 inproceedingsAn optimal algorithm for on-line bipartite matching.Proceedings of the twenty-second annual ACM symposium on Theory of computing1990, 352--358back to text
- 93 inproceedingsAvoiding discrimination through causal reasoning.Advances in Neural Information Processing Systems2017, 656--666back to text
- 94 articleModel-Agnostic Characterization of Fairness Trade-offs.arXiv preprint arXiv:2004.034242020back to textback to text
- 95 bookStable Marriage and Its Relation to Other Combinatorial Problems: An Introduction to the Mathematical Analysis of Algorithms.English translation, (CRM Proceedings and Lecture Notes), American Mathematical Society1996back to text
- 96 bookAuction Theory.Elsevier2009back to text
- 97 inproceedingsCounterfactual fairness.Advances in neural information processing systems2017, 4066--4076back to text
- 98 miscThe long road to fairer algorithms.2020back to text
- 99 articleAlgorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads.Management Science2019back to textback to text
- 100 miscHow We Analyzed the COMPAS Recidivism Algorithm.ProPublica, https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm2016back to text
- 101 inproceedingsCompeting Bandits in Matching Markets.arXiv:1906.053632019, 1--15back to text
- 102 articleBusiness cycle dynamics under rational inattention.The Review of Economic Studies8242015, 1502--1532back to text
- 103 inproceedingsOnline bipartite matching with random arrivals: an approach based on strongly factor-revealing lps.Proceedings of the forty-third annual ACM symposium on Theory of computing2011, 597--606back to text
- 104 articleOnline stochastic matching: Online actions based on offline statistics.Mathematics of Operations Research3742012, 559--573back to textback to text
- 105 bookMicroeconomic Theory.Oxford University Press1995back to text
- 106 articleGreedy online bipartite matching on random graphs.arXiv preprint arXiv:1307.25362013back to text
- 107 articleRational inattention to discrete choices: A new foundation for the multinomial logit model.American Economic Review10512015, 272--98back to text
- 108 articleOnline Matching and Ad Allocation.Found. Trends Theor. Comput. Sci.84October 2013, 265–368URL: https://doi.org/10.1561/0400000057DOIback to textback to textback to text
- 109 bookRepeated Games.Econometric Society MonographsCambridge University Press2015back to text
- 110 articleAlgorithms for the Assignment and Transportation Problems.Journal of the Society for Industrial and Applied Mathematics511957, 32--38back to text
- 111 inproceedingsBidding Strategies with Gender Nondiscrimination Constraints for Online Ad Auctions.FAT*2020back to textback to text
- 112 inproceedingsThe bidder’s standpoint : a simple way to improve bidding strategies in revenue-maximizing auctions.Workshop on Learning in the Presence of Strategic Behavior (EC 2019)2019back to text
- 113 inproceedingsLearning to bid in revenue-maximizing auctions.Proceedings of the 36th International Conference on Machine Learning (ICML 2019)2019, 4781--4789back to text
- 114 bookAlgorithmic Game Theory.New York, NY, USACambridge University Press2007back to text
- 115 articleA differential game on Wasserstein space. Application to weak approachability with partial monitoring.Journal of Dynamics and Games62019, 65--85back to text
- 116 articleThe multi-armed bandit problem with covariates.Annals of Statistics412013, 693--721back to text
- 117 bookElements of causal inference.The MIT Press2017back to text
- 118 bookComputational Optimal Transport.ArXiv:1803.005672018back to text
- 119 inproceedingsFair Inputs and Fair Outputs: The Incompatibility of Fairness in Privacy and Accuracy.FairUMAP2020back to text
- 120 inproceedingsDistance makes the types grow stronger: a calculus for differential privacy.ACM Sigplan Notices452010, 157--168back to text
- 121 articleGeometrizing rates of convergence under local differential privacy constraints.Annals of Statistics4852020, 2646--2670back to text
- 122 articleWhat's in a Name? Reducing Bias in Bios without Access to Protected Attributes.arXiv preprint arXiv:1904.052332019back to text
- 123 articleAlgorithms that" Don't See Color": Comparing Biases in Lookalike and Special Ad Audiences.arXiv preprint arXiv:1912.075792019back to text
- 124 articleImplications of rational inattention.Journal of monetary Economics5032003, 665--690back to text
- 125 inproceedingsOn the Potential for Discrimination in Online Targeted Advertising.ACM FAT*2018back to text
- 126 bookTopics in optimal transportation.58Graduate studies in Mathematics, AMS2003back to text
- 127 inproceedingsOnline learning in repeated auctions.Proceedings of COLT 20162016back to textback to text
- 128 inproceedingsLearning Non-Discriminatory Predictors.COLT2017back to text
- 129 inproceedingsCounterfactual Fairness: Unidentification, Bound and Algorithm..IJCAI2019, 1438--1444back to text
- 130 inproceedingsPc-fairness: A unified framework for measuring causality-based fairness.Advances in Neural Information Processing Systems2019, 3404--3414back to text
- 131 inproceedingsFairness Beyond Disparate Treatment & Disparate Impact: Learning Classification Without Disparate Mistreatment.WWW2017back to text
- 132 inproceedingsFrom Parity to Preference-based Notions of Fairness in Classification.NIPS2017back to text
- 133 inproceedingsLearning Fair Representations.ICML2013back to text