2.1 Context

In several respects, modern society has strengthened the need for statistical analysis both from applied and theoretical point of view. The genesis comes from the easier availability of data thanks to technological breakthroughs (storage, transfer, computing), and are now so widespread that they are no longer limited to large human organizations. The more or less conscious goal of such data availability is the expectation of improving the quality of “since the dawn of time” statistical stories which are namely discovering new knowledge or doing better predictions. These both central tasks can be referred respectively as unsupervised learning or supervised learning, even if it is not limited to them or other names exist depending on communities. Somewhere, it pursues the following hope: “more data for better quality and more numerous results”.

However, today's data are increasingly complex. They gather mixed type features (for instance continuous data mixed with categorical data), missing or partially missing items (like intervals) and numerous variables (high dimensional situation. As a consequence, the target “better quality and more numerous results” of the previous adage (both words are important: “better quality” and also “more numerous”) could not be reached through a somewhat “manual” way, but should inevitably rely on some theoretical formalization and guarantee. Indeed, data can be so numerous and so complex (data can live in quite abstract spaces) that the “empirical” statistician is quickly outdated. However, data being subject by nature to randomness, the probabilistic framework is a very sensible theoretical environment to serve as a general guide for modern statistical analysis.

2.2 Goals

Modal is a project-team working on today's complex data sets (mixed data, missing data, high-dimensional data), for classical statistical targets (unsupervised learning, supervised learning, regression etc.) with approaches relying on the probabilistic framework. This latter can be tackled through both model-based methods (as mixture models for a generic tool) and model-free methods (as probabilistic bounds on empirical quantities). Furthermore, Modal is connected to the real world by applications, typically with biological ones (some members have this skill) but many other ones are also considered since the application coverage of the Modal methodology is very large. It is also important to note that, in return, applications are often real opportunities for initiating academic questioning for the statistician (case of some projects treated by bilille platform and some bilateral contracts of the team).

From the academic communities point of view, Modal can be seen as belonging simultaneously to both the statistical learning and machine learning ones, as attested by its publications. Somewhere it is the opportunity to make a bridge between these two stochastic communities around a common but large probabilistic framework.

3.1 Research axis 1: Unsupervised learning

Scientific locks related to unsupervised learning are numerous, concerning the clustering outcome validity, the ability to manage different kinds of data, the missing data questioning, the dimensionality of the data set etc. Many of them are addressed by the team, leading to publication achievements, often with a specific package delivery (sometimes upgraded as a software or even as a platform grouping several software). Because of the variety of the scope, it involves nearly all the permanent team members, often with PhD students and some engineers. The related works are always embedded inside a probabilistic framework, typically model-based approaches but also model-free ones like PAC-Bayes (PAC stands for Probably Approximately Correct), because such a mathematical environment offers both a well-posed problem and a rigorous answer.

3.2 Research axis 2: Performance assessment

One main concern of the Modal team is to provide theoretical justifications on the procedures which are designed. Such guarantees are important to avoid misleading conclusions resulting from any unsuitable use. For example, one ingredient in proving these guarantees is the use of the PAC framework, leading to finite-sample concentration inequalities. More precisely, contributions to PAC learning rely on the classical empirical process theory and the PAC-Bayesian theory. The Modal team exploits such non-asymptotic tools to analyze the performance of iterative algorithms (such as gradient descent), cross-validation estimators, online change-point detection procedures, ranking algorithms, matrix factorization techniques and clustering methods, for instance. The team also develops some expertise on the formal dynamic study of algorithms related to mixture models (important models used in the previous unsupervised setting), like degeneracy for EM algorithm or also label switching for Gibbs algorithm.

3.3 Research axis 3: Functional data

Mainly due to technological advances, functional data are more and more widespread in many application domains. Functional data analysis (FDA) is concerned with the modeling of data, such as curves, shapes, images or a more complex mathematical object, though as smooth realizations of a stochastic process (an infinite dimensional data object valued in a space of eventually infinite dimension; space of squared integrable functions etc.). Time series are an emblematic example even if it should not be limited to them (spectral data, spatial data etc.). Basically, FDA considers that data correspond to realizations of stochastic processes, usually assumed to be in a metric, semi-metric, Hilbert or Banach space. One may consider, functional independent or dependent (in time or space) data objects of different types (qualitative, quantitative, ordinal, multivariate, time-dependent, spatial-dependent etc.). The last decade saw a dynamic literature on parametric or non-parametric FDA approaches for different types of data and applications to various domains, such as principal component analysis, clustering, regression and prediction.

3.4 Research axis 4: Applications motivating research

The fourth axis consists in translating real application issues into statistical problems raising new (academic) challenges for models developed in Modal team. Cifre PhDs in industry and interdisciplinary projects with research teams in Health and Biology are at the core of this objective. The main originality of this objective lies in the use of statistics with complex data, including in particular ultra-high dimension problems. We focus on real applications which cannot be solved by classical data analysis.

4.1 Economic world

The Modal team applies its research to the economic world through CIFRE PhD supervision such as CACF (credit scoring), A-Volute (expert in 3D sound), Meilleur Taux (insurance comparator), Worldline. It also has several contracts with companies such as COLAS, Nokia-Apsys/Airbus, Safety Line (through the PERF-AI consortium), Agence d'Urbanisme Métropole Européenne de Lille.

4.2 Biology and health

The second main application domain of the team is biology and health. Members of the team are involved in the supervision and scientific animation of bilille, the bioinformatics platform of Lille, and of OncoLille Institute. Members of the team also co-supervise PhD students of Inserm teams.

Action Exploratoire (AEx)

Participants: Sophie Dabo, Christophe Biernacki, Cristian Preda, Vincent Vandewalle, Guillemmette Marot.

Acronyme et nom du projet :

PATH (PAtient PaThway in the Hospital environment)

Collaboration

: CHU Lille and Faculty of Medicine (METRICS team)

Researchers involved : Jean-Baptiste Beuscart, Grégoire Ficheur, Emmanuel Chazard, Michaël, Genin, Antoine Lamer, Génia Babykina, Cyrielle Dumont.

European healthcare systems are faced with multiple challenges, including an aging population, an increase in chronic diseases and patients with multiple illnesses, and limited financial and human resources. The response to these challenges relies in particular on the organization of care into care pathways, justified by the scientific literature and supported in France by political orientations. The analysis of care pathways and their adequacy to needs and resources has thus become a major scientific and administrative challenge. Although the numerical data available for this purpose are rapidly increasing, the methods and statistical tools available to researchers and health authorities remain limited and inefficient. PATH proposes to develop statistical methods for the construction/analysis of the patient pathway through two applications dealing with the re-hospitalization of the elderly and post-operative complications.

Inria London.

Benjamin Guedj has led the emerging Inria London Programme since 2019. The partnernship involves Inria and University College London (UCL) as of February 1st, 2021 and the official kickoff.

7.1 New software

7.2 New platforms

8.1 Axis 1: Model-based Co-clustering for Ordinal Data of Different Dimensions

Participants: Christophe Biernacki.

This work has been motivated by a psychological survey on women affected by a breast tumor. Patients replied at different moments of their treatment to questionnaires with answers on ordinal scale. The questions relate to aspects of their life called dimensions. To assist the psychologists in analyzing the results, it is useful to emphasize a structure in the dataset. The clustering method achieves that by creating groups of individuals that are depicted by a representative of the group. From a psychological position, it is also useful to observe how questions may be grouped. This is why a clustering should also be performed on the features, which is called a co-clustering problem. However, gathering questions that are not related to the same dimension does not make sense from a psychologist stance. Therefore, the present work corresponds to perform a constrained co-clustering method aiming to prevent questions from different dimensions from getting assembled in a same column-cluster. In addition, evolution of co-clusters along time has been investigated. The method relies on a constrained Latent Block Model embedding a probability distribution for ordinal data. Parameter estimation relies on a Stochastic EM-algorithm associated to a Gibbs sampler, and the ICL-BIC criterion is used for selecting the numbers of co-clusters. The resulting work has been accepted in an international journal in 2019 and the related R package ordinalClust has been accepted this year in another international journal 26.

This is a joint work with Margot Selosse (PhD student) and Julien Jacques, both from Université de Lyon 2, and Florence Cousson-Gélie from Université Paul Valéry Montpellier 3.

8.2 Axis 1: Gaussian-based Visualization of Gaussian and non-Gaussian Model-based Clustering

Participants: Christophe Biernacki, Vincent Vandewalle.

A generic method is introduced to visualize in a Gaussian-like way, and onto $R^2$, results of Gaussian or non-Gaussian model-based clustering. The key point is to explicitly force a spherical Gaussian mixture visualization to inherit from the within cluster overlap which is present in the initial clustering mixture. The result is a particularly user-friendly draw of the clusters, allowing any practitioner to have a thorough overview of the potentially complex clustering result. An entropic measure allows us to inform of the quality of the drawn overlap, in comparison to the true one in the initial space. The proposed method is illustrated on four real data sets of different types (categorical, mixed, functional and network) and is implemented on the R package ClusVis. This work has been published last year in an international journal 85, and has been presented in a conference 40. This is a joint work with Matthieu Marbac from ENSAI.

8.3 Axis 1: Dealing with Missing Data in Model-based Clustering through a MNAR Model

Participants: Christophe Biernacki.

Since the 90s, model-based clustering is largely used to classify data. Nowadays, with the increase of available data, missing values are more frequent. Traditional ways to deal with them consist in obtaining a filled data set, either by discarding missing values or by imputing them. In the first case, some information is lost; in the second case, the final clustering purpose is not taken into account through the imputation step. Thus, both solutions risk to blur the clustering estimation result. Alternatively, we defend the need to embed the missingness mechanism directly within the clustering modeling step. There exists three types of missing data: missing completely at random (MCAR), missing at random (MAR) and missing not at random (MNAR). In all situations logistic regression is proposed as a natural and flexible candidate model. In particular, its flexibility property allows us to design some meaningful parsimonious variants, as dependency on missing values or dependency on the cluster label. In this unified context, standard model selection criteria can be used to select between such different missing data mechanisms, simultaneously with the number of clusters. Practical interest of our proposal is illustrated on data derived from medical studies suffering from many missing data. Currently, a preprint is being finalized for submission to an international journal and a talk to a conference has been given 37. A more general talk on missing data and its impact on mixtures and clustering has also been given this year in a workshop 38.

It is a joint work with Claire Boyer from Sorbonne Université, Gilles Celeux from Inria Saclay, Julie Josse from Inria Montpellier, Fabien Laporte from Institut Pasteur and Matthieu Marbac from ENSAI.

8.4 Axis 1: Predictive Clustering

Participants: Christophe Biernacki, Vincent Vandewalle.

Many data, for instance in biostatistics, contain some sets of variables which permit evaluating unobserved traits of the subjects (e.g. we ask question about how many pizzas, hamburgers, chips etc. are eaten to know how healthy are the food habits of the subjects). Moreover, we often want to measure the relations between these unobserved traits and some target variables (e.g. obesity). Thus, a two-steps procedure is often used: first, a clustering of the observations is performed on the sets of variables related to the same topic; second, the predictive model is fitted by plugging the estimated partitions as covariates. Generally, the estimated partitions are not exactly equal to the true ones. We investigate the impact of these measurement errors on the estimators of the regression parameters, and we explain when this two-steps procedure is consistent. We also present a specific EM algorithm which simultaneously estimates the parameters of the clustering and predictive models. A paper has now been accepted in an international journal 24 and has been presented in an national conference 34.

It is a joint work with Matthieu Marbac from ENSAI and Mohammed Sedki from Université Paris-Saclay.

8.5 Axis 1: A Binned Technique for Scalable Model-based Clustering on Huge Datasets

Participants: Filippo Antonazzo, Christophe Biernacki.

Clustering is impacted by the regular increase of sample sizes which provides opportunity to reveal information previously out of scope. However, the volume of data leads to some issues related to the need of many computational resources and also to high energy consumption. Resorting to binned data depending on an adaptive grid is expected to give proper answer to such green computing issues while not harming the quality of the related estimation. After a brief review of existing methods, a first application in the context of univariate model-based clustering is provided, with a numerical illustration of its advantages. Finally, an initial formalization of the multivariate extension is done, highlighting both issues and possible strategies. This work has been submited to an international journal 54, a short version has been accepted in a book of short papers associated to an international conference 47 and has to a talk in an international workshop and also to a seminar .

It is a joint work with Christine Keribin from Université Paris-Saclay.

8.6 Axis 1: Regularized spectral methods for clustering signed networks

Participants: Hemant Tyagi.

We study the problem of $k$-way clustering in signed graphs. Considerable attention in recent years has been devoted to analyzing and modeling signed graphs, where the affinity measure between nodes takes either positive or negative values. Recently, Cucuringu et al. [CDGT 2019] proposed a spectral method, namely SPONGE (Signed Positive over Negative Generalized Eigenproblem), which casts the clustering task as a generalized eigenvalue problem optimizing a suitably defined objective function. This approach is motivated by social balance theory, where the clustering task aims to decompose a given network into disjoint groups, such that individuals within the same group are connected by as many positive edges as possible, while individuals from different groups are mainly connected by negative edges. Through extensive numerical simulations, SPONGE was shown to achieve state-of-the-art empirical performance. On the theoretical front, [CDGT 2019] analyzed SPONGE and the popular Signed Laplacian method under the setting of a Signed Stochastic Block Model (SSBM), for $k=2$ equal-sized clusters, in the regime where the graph is moderately dense. In this work, we build on the results in [CDGT 2019] on two fronts for the normalized versions of SPONGE and the Signed Laplacian. Firstly, for both algorithms, we extend the theoretical analysis in [CDGT 2019] to the general setting of $k\ge 2$ unequal-sized clusters in the moderately dense regime. Secondly, we introduce regularized versions of both methods to handle sparse graphs – a regime where standard spectral methods underperform – and provide theoretical guarantees under the same SSBM model. To the best of our knowledge, regularized spectral methods have so far not been considered in the setting of clustering signed graphs. We complement our theoretical results with an extensive set of numerical experiments on synthetic data.

This is joint work with Mihai Cucuringu (University of Oxford, United Kingdom), Apoorv Vikram Singh (NYU), Deborah Sulem (University of Oxford, United Kingdom). It was initiated when Apoorv Vikram Singh visited the MODAL team to work with Hemant Tyagi from Oct 2019-Jan 2020. This has now been accepted for publication in the journal: Journal of Machine Learning Research12. A summary of the results was presented at the GCLR (Graphs and more Complex structures for Learning and Reasoning) workshop at AAAI 2021.

8.7 Axis 1: Dynamic Ranking with the BTL Model: A Nearest Neighbor based Rank Centrality Method

Participants: Eglantine Karlé, Hemant Tyagi.

Many applications such as recommendation systems or sports tournaments involve pairwise comparisons within a collection of n items, the goal being to aggregate the binary outcomes of the comparisons in order to recover the latent strength and/or global ranking of the items. In recent years, this problem has received significant interest from a theoretical perspective with a number of methods being proposed, along with associated statistical guarantees under the assumption of a suitable generative model. While these results typically collect the pairwise comparisons as one comparison graph G, however in many applications-such as the outcomes of soccer matches during a tournamentthe nature of pairwise outcomes can evolve with time. Theoretical results for such a dynamic setting are relatively limited compared to the aforementioned static setting. We study in this paper an extension of the classic BTL (Bradley-Terry-Luce) model for the static setting to our dynamic setup under the assumption that the probabilities of the pairwise outcomes evolve smoothly over the time domain [0, 1]. Given a sequence of comparison graphs ${\left(G_t^{\text{'}}\right)}_{t^{\text{'}}\in T}$ on a regular grid $T\subset [0,1]$, we aim at recovering the latent strengths of the items $w_t^{*}\in R^n$ at any time $t\in [0,1]$. To this end, we adapt the Rank Centrality method-a popular spectral approach for ranking in the static case-by locally averaging the available data on a suitable neighborhood of t. When ${\left(G_t^{\text{'}}\right)}_{t^{\text{'}}\in T}$ is a sequence of Erdös-Renyi graphs, we provide non-asymptotic ${\ell }_2$ and ${\ell }_{\infty }$ error bounds for estimating $w_t^{*}$ which in particular establishes the consistency of this method in terms of n, and the grid size |T |. We also complement our theoretical analysis with experiments on real and synthetic data. This PhD work 65 has been submitted to a journal and is currently under review.

8.8 Axis 1: An extension of the angular synchronization problem to the heterogeneous setting

Participants: Hemant Tyagi.

Given an undirected measurement graph $G=\left(\right[n],E)$, the classical angular synchronization problem consists of recovering unknown angles ${\theta }_1,\cdots ,{\theta }_n$ from a collection of noisy pairwise measurements of the form $({\theta }_i-{\theta }_j)mod2\pi$, for each $\{i,j\}\in E$. This problem arises in a variety of applications, including computer vision, time synchronization of distributed networks, and ranking from preference relationships. In this paper, we consider a generalization to the setting where there exist $k$ unknown groups of angles ${\theta }_{l,1},\cdots ,{\theta }_{l,n}$, for $l=1,\cdots ,k$. For each $\{i,j\}\in E$, we are given noisy pairwise measurements of the form ${\theta }_{\ell ,i}-{\theta }_{\ell ,j}$ for an unknown $\ell \in \{1,2,...,k\}$. This can be thought of as a natural extension of the angular synchronization problem to the heterogeneous setting of multiple groups of angles, where the measurement graph has an unknown edge-disjoint decomposition $G=G_1\cup G_2...\cup G_k$, where the $G_i$'s denote the subgraphs of edges corresponding to each group. We propose a probabilistic generative model for this problem, along with a spectral algorithm for which we provide a detailed theoretical analysis in terms of robustness against both sampling sparsity and noise. The theoretical findings are complemented by a comprehensive set of numerical experiments, showcasing the efficacy of our algorithm under various parameter regimes. Finally, we consider an application of bi-synchronization to the graph realization problem, and provide along the way an iterative graph disentangling procedure that uncovers the subgraphs $G_i$, $i=1,...,k$ which is of independent interest, as it is shown to improve the final recovery accuracy across all the experiments considered.

This is joint work with Mihai Cucuringu (University of Oxford, United Kingdom) and has been accepted for publication in the journal Foundations of Data Science13.

8.9 Axis 1&2: Clustering on Multilayer Graphs with Missing Values

Participants: Christophe Biernacki, Guillaume Braun, Hemant Tyagi.

Multilayer graphs clustering have gained increasing interest this last decade due to numerous applications in various fields. Several clustering methods have been proposed, but they rely all on the assumption that the network is fully observed. We propose a statistical framework to handle nodes that are missing on some layers as well as a method to estimate the model parameters and to impute missing edge values.

This work has been published in an international conference 41.

8.10 Axis 1&2: An iterative clustering algorithm for the Contextual Stochastic Block Model with optimality guarantees

Participants: Christophe Biernacki, Guillaume Braun, Hemant Tyagi.

Real-world networks often come with side information that can help to improve the performance of network analysis tasks such as clustering. Despite a large number of empirical and theoretical studies conducted on network clustering methods during the past decade, the added value of side information and the methods used to incorporate it optimally in clustering algorithms are relatively less understood. We propose a new iterative algorithm to cluster networks with side information for nodes (in the form of covariates) and show that our algorithm is optimal under the Contextual Symmetric Stochastic Block Model. Our algorithm can be applied to general Contextual Stochastic Block Models and avoids hyperparameter tuning in contrast to previously proposed methods. We confirm our theoretical results on synthetic data experiments where our algorithm significantly outperforms other methods, and show that it can also be applied to signed graphs. Finally we demonstrate the practical interest of our method on real data.

This work has been submited to an international conference and is currently available as a preprint 57.

8.11 Axis 2: Denoising modulo samples: $k$-NN regression and tightness of SDP relaxation

Participants: Hemant Tyagi.

Many modern applications involve the acquisition of noisy modulo samples of a function $f$, with the goal being to recover estimates of the original samples of $f$. For a Lipschitz function $f:{[0,1]}^d\to ℝ$, suppose we are given the samples $y_i=(f\left(x_i\right)+{\eta }_i)mod1;i=1,\cdots ,n$ where ${\eta }_i$ denotes noise. Assuming ${\eta }_i$ are zero-mean i.i.d Gaussian's, and $x_i$'s form a uniform grid, we derive a two-stage algorithm that recovers estimates of the samples $f\left(x_i\right)$ with a uniform error rate $O\left({\left(\frac{logn}{n}\right)}^{\frac{1}{d+2}}\right)$ holding with high probability. The first stage involves embedding the points on the unit complex circle, and obtaining denoised estimates of $f\left(x_i\right)mod1$ via a $k$NN (nearest neighbor) estimator.

Recently, Cucuringu and Tyagi proposed an alternative way of denoising modulo 1 data which works with their representation on the unit complex circle. They formulated a smoothness regularized least squares problem on the product manifold of unit circles, where the smoothness is measured with respect to the Laplacian of a proximity graph $G$ involving the $x_i$'s. This is a nonconvex quadratically constrained quadratic program (QCQP) hence they proposed solving its semidefinite program (SDP) based relaxation. We derive sufficient conditions under which the SDP is a tight relaxation of the QCQP. Hence under these conditions, the global solution of QCQP can be obtained in polynomial time.

This is joint work with Michael Fanuel (KU Leuven). It is has been accepted for publication in the journal Information and Inference: A journal of the IMA17.

8.12 Axis 2: Error analysis for denoising smooth modulo signals on a graph

Participants: Hemant Tyagi.

In many applications, we are given access to noisy modulo samples of a smooth function with the goal being to robustly unwrap the samples, i.e. to estimate the original samples of the function. In a recent work, Cucuringu and Tyagi proposed denoising the modulo samples by first representing them on the unit complex circle and then solving a smoothness regularized least squares problem – the smoothness measured w.r.t. the Laplacian of a suitable proximity graph $G$ – on the product manifold of unit circles. This problem is a quadratically constrained quadratic program (QCQP) which is nonconvex, hence they proposed solving its sphere-relaxation leading to a trust region subproblem (TRS). In terms of theoretical guarantees, ${\ell }_2$ error bounds were derived for (TRS). These bounds are however weak in general and do not really demonstrate the denoising performed by (TRS).

In this work, we analyse the (TRS) as well as an unconstrained relaxation of (QCQP). For both these estimators we provide a refined analysis in the setting of Gaussian noise and derive noise regimes where they provably denoise the modulo observations w.r.t. the ${\ell }_2$ norm. The analysis is performed in a general setting where $G$ is any connected graph.

This work has been accepted for publication in the journal Applied and Computational Harmonic Analysis27.

8.13 Axis 2: Recovering Hölder smooth functions from noisy modulo samples

Participants: Hemant Tyagi.

In signal processing, several applications involve the recovery of a function given noisy modulo samples. The setting considered in this paper is that the samples corrupted by an additive Gaussian noise are wrapped due to the modulo operation. Typical examples of this problem arise in phase unwrapping problems or in the context of self-reset analog to digital converters. We consider a fixed design setting where the modulo samples are given on a regular grid. Then, a three stage recovery strategy is proposed to recover the ground truth signal up to a global integer shift. The first stage denoises the modulo samples by using local polynomial estimators. In the second stage, an unwrapping algorithm is applied to the denoised modulo samples on the grid. Finally, a spline based quasi-interpolant operator is used to yield an estimate of the ground truth function up to a global integer shift. For a function in Hölder class, uniform error rates are given for recovery performance with high probability. This extends recent results obtained by Fanuel and Tyagi for Lipschitz smooth functions wherein kNN regression was used in the denoising step.

This is joint work with Michaël Fanuel (CRIStAL, Universite de Lille) and was presented in an invited session on Computational Sampling in an international conference (ASILOMAR 2021). The paper 63 will be appearing in the proceedings of the conference.

8.14 Axis 2: Asymptotic efficiency of some nonparametric tests for location on hyperspheres

Participants: Sophie Dabo-Niang.

In the paper, we show that several classical nonparametric tests for multivariate location in the Euclidean case can be adapted to nonparametric tests for the location problem on hyperspheres. The tests we consider are spatial signs and spatial signed-rank tests for location on hyperspheres. We compute the asymptotic powers of the latter tests in the classical rotationally symmetric case. In particular, we show that the spatial signed-rank based test uniformly dominates the spatial sign test and has performances that are extremely close to the asymptotically optimal test in the well-known von Mises-Fisher case. Monte-Carlo simulations confirm our asymptotic results.

It is a joint work with Baba Thiam (University of Lille, Painleve), Thomas Verdebout (ULB, Belgium). This work has been been submitted for publication 62.

8.15 Axis 2: $k$-nearest neighbors prediction and classification for spatial data

Participants: Sophie Dabo-Niang.

This paper proposes a spatial $k$-nearest neighbor method for nonparametric prediction of real-valued spatial data and supervised classification for categorical spatial data. The proposed method is based on a double nearest neighbor rule which combines two kernels to control the distances between observations and locations. It uses a random bandwidth in order to more appropriately fit the distributions of the covariates. The almost complete convergence with rate of the proposed predictor is established red and the almost sure convergence of the supervised classification rule was deduced. Finite sample properties are given for two applications of the $k$-nearest neighbor prediction and classification rule.

It is a joint work with Mohamed Salem Ahmed (University of Lille, CERIM), Mohamed Attouch (Uviversity Sidi Bel Abbes, Algeria), Mamadou Ndiaye (UCAD, Senegal). This work is under revision. 53.

8.16 Axis 3: Regression models for spatially distributed autoregressive functional data

Participants: Sophie Dabo-Niang.

A functional linear autoregressive spatial model, where the explanatory variable takes values in a function space while the response process is real-valued and spatially autocorrelated, is proposed. The specificity of the model is due to the functional nature of the explanatory variable and the structure of a spatial weight matrix that defines the spatial dependency between neighbors. The estimation procedure consists of reducing the infinite dimension of the functional explanatory variable and maximizing the quasi-maximum likelihood. We establish the consistency and asymptotic normality of the estimator. The ability of the methodology is illustrated via simulations and by application to real data.

It is a joint work with Mohamed Salem Ahmed (University of Lille, CERIM), Zied Gharbi (University of Lille) Laurence Broze (Unievrsity of Lille). This work has been published in the book Geostatistical Functional Data Analysis: Theory and Methods. J. Mateu and R. Giraldo (Eds). John Wiley and Sons, Chichester, UK. ISBN: 978-1-119-38784-8

8.17 Axis 3: Non-parametric statistical analysis of spatially distributed functional data

Participants: Sophie Dabo-Niang.

A nonparametric estimator of the regression function of a scalar spatial variable given a functional spatial variable is proposed. Mean square and almost complete consistencies of the estimator are obtained when the sample considered is an $\alpha$-mixing sequence and composed of non i.i.d observations. Lastly, an application to spatial prediction and numerical results are provided to illustrate the behavior of our estimator.

It is a joint work with Baba Thiam (University of Lille), Camille Ternynck (University of Lille, CERIM), Anne-Françoise Yao (University of Clermont Auvergne). This work has been published in the book Geostatistical Functional Data Analysis: Theory and Methods. J. Mateu and R. Giraldo (Eds). John Wiley and Sons, Chichester, UK. ISBN: 978-1-119-38784-8

8.18 Axis 3: Clustering spatial functional data

Participants: Vincent Vandewalle, Cristian Preda, Sophie Dabo-Niang.

In this work we present two approaches for clustering spatial functional data. The first one is the model-based clustering that uses the concept of density for functional random variables. The second one is the hierarchical clustering based on univariate statistics for functional data such as the functional mode or the functional mean. These two approaches take into account the spatial features of the data: two observations that are spatially close share a common distribution of the associated random variables. The two methodologies are illustrated by an application to air quality data.

This work has been published in the book Geostatistical Functional Data Analysis: Theory and Methods. J. Mateu and R. Giraldo (Eds). John Wiley and Sons, Chichester, UK. ISBN: 978-1-119-38784-8 49.

8.19 Axis 3: Investigating spatial scan statistics for multivariate functional data

Participants: Sophie Dabo-Niang.

This paper introduces the R package HDSpatialSca that allows users to apply easily spatial scan statistics on real-valued multivariate data or both univariate and multivariate functional data. It also permits to plot the detected clusters and to summarize them. In this article the methods are presented and the use of the package is illustrated through examples on environmental data provided in the package.

It is a joint work with Camille Frévent (University of Lille, CERIM), Mohamed-Salem Ahmed (University of Lille, CERIM), Michaël Genin (University of Lille, CERIM). A R package name HDSpatialSca has been developped, a related article is under revision in R journal. 64.

8.20 Axis 3: Categorical functional data analysis

Participants: Cristian Preda, Quentin Grimonprez, Vincent Vandewalle.

We have developed the methodology to visualize, perform dimension reduction and extract features from categorical functional data. For this, the cfda R package has been developed and added to CRAN repository. A paper presenting the features of the cfda R package (7.1.3) with an application to care data (clustering of patient paths) is published in 9. The cfda R package has been presented to the ERCIM 77 and SFDS 78.

8.21 Axis 3: Clustering categorical functional data

Participants: Cristian Preda, Vincent Vandewalle.

The objective of this research direction was: (i) to propose possible modelling approaches of categorical functional data and (ii) to investigate the identifiability problem of such models. A first modelling framework is to consider that an observed functional data path represents a sample path of Markov process and thus $n$ sample paths come from several, say $K$, different processes. Consequently, we have here a mixture of $K$ different Markov processes. A second modelling framework is to consider that the observed sample path come from several semi-Markov processes. The parameter estimation is obtained through techniques based on the EM algorithm, while the selection of the number of classes is based on information criteria. An important problem is to determine the class membership for each sample paths, but our main concern is related to the identifiability problem. The identifiability of this type of models cannot be obtained in general, but only by imposing restrictions on the parameters of the model, cf. 87, 88. Our work in progress is related to find sufficiently general conditions that guarantee this identifiability.

8.22 Axis 4: Statistical analysis of high-throughput proteomic data

Participants: Guillemette Marot, Vincent Vandewalle, Wilfried Heyse.

Since November 2019, Wilfried Heyse has started a PhD thesis granted by INSERM and supervised by Christophe Bauters, Guillemette Marot and Vincent Vandewalle. The aim is to identify earlier after myocardial infarction (MI) patients at high risk of developing left ventricular remodelling (LVR) that is quantified by imaging one year after MI or to identify patients with high risk of death. For that purpose, high throughput proteomic approach is used. This technology allows the measurement of 5000 proteins simultaneously. In parallel to these measures corresponding to the concentration of a protein in a plasma sample collected from one patient at a specific time, echocardiographic and clinical information have been collected on each of the 200 patients. One of the main challenge is to take into account the variations of the biomarkers according to the time (several measurement times), in order to improve the understanding of biological mechanisms involved on LVR or survival of the patient.

By selecting 46 proteins significantly associated to long-term survival in Cox models we have identified 2 groups of patients (one group with high risk and the other with lower risk). Network analysis identified common pathways from the 46 proteins related to cell death and survival and cell-to-cell communication. This work as been submitted for possible publication in an international journal, and has been presented in a national conference 42.

We are now investigating the possibilities to take into account the temporal structure and the high dimension of the data. The aim of this work is to jointly model the temporal structure of all the proteins and the long-term survival of the patients. The main challenges of this work are both the number of proteins and the repeated measurements and the strategy to address these challenges are to introduce the information of known groups of proteins defined by the GO categories (known categories of proteins which are part of a same biological function). This work could lead to the identification of new biomarkers for heart failure.

This is a joint work with Florence Pinet from INSERM.

8.23 Axis 4: Contribution to the nutritional transition

Participants: Wilfried Heyse.

The nutritional transition of a country is characterized by a shift from a traditional, generally plant-rich diet to a meat-rich diet. In the last half century, several countries have experienced significant economic development and have seen their food supply increase. The aim of this work was to identify similarities in food transitions across countries of the world over the past 60 years. The food availability data of 171 countries (public FAOSTATS) gather the informations total food availability, as well as per capita animal and plant products per year, for each country over the period 1961-2018. In order to identify transition patterns, we used unsupervised clustering (hierarchical clustering) analyses which led us to identify 5 distinct clusters with different food transitions patterns. In conclusion, between 1961 and 2018, total food availability increased overall, but with regional disparities. Several transition patterns were identified, characterized by a fairly marked increase in the availability of plant products and to a lesser degree meat. The level of economic development and geographical location are strong indicators of food transition patterns in the world. This work has been presented in a national peer review conference 35.

8.24 Axis 4: Identification of a new master regulator through transcriptomics and epigenomics data analysis

Participants: Guillemette Marot.

8.25 Axis 4: Reject Inference Methods in Credit Scoring

Participants: Christophe Biernacki, Adrien Ehrhardt, Philippe Heinrich, Vincent Vandewalle.

The granting process of all credit institutions rejects applicants having a low credit score. Developing a scorecard, i.e. a correspondence table between a client’s characteristics and his score, requires a learning dataset in which the target variable good/bad borrower is known. Rejected applicants are de facto excluded from the process. This biased learning population might have deep consequences on the scorecard relevance. Some works, mostly empirical ones, try to exploit rejected applicants in the scorecard building process. This work proposes a rational criterion to evaluate the quality of a scoring model for the existing Reject Inference methods and dig out their implicit mathematical hypotheses. It is shown that, up to now, no such Reject Inference method can guarantee a better credit scorecard. These conclusions are illustrated on simulated and real data from the french branch of Crédit Agricole Consumer Finance (CACF). This work is now published in an international journal 16.

This is a joint work with Sébastien Beben of Crédit Agricole Consumer Finance.

8.26 Axis 4: Usability study

Participants: Vincent Vandewalle.

Since 2018, Vincent Vandewalle is working with Alexandre Caron and Benoît Dervaux (ULR 2694 – METRICS) on issues of estimating the number of problems and the value of information in the field of usability. Based on usability study of a medical device the objective is to determine the number of possible problems linked to the use of a medical device (e.g. insulin pump) as well as their respective occurrence probabilities. Estimating this number and the different probabilities is essential to determine whether or not an additional usability study should be conducted, and to determine the number of users to be included in this study to maximize the expected benefits.

The discovery process can be modeled by a binary matrix, a matrix whose number of columns depends on the number of defects discovered by users. In this framework, they have proposed a probabilistic modeling of this matrix. They have included this modeling in a Bayesian framework where the number of problems and the probabilities of discovery are considered as random variables. It shows the interest of the approach compared to the approaches proposed in the state of the art in usability. The approach beyond point estimation also makes it possible to obtain the distribution of the number of problems and their respective probabilities given the discovery matrix.

The proposed model published in last year 89 also allows to implement an approach aiming at measuring the value of additional information in relation to the discovery process. In this framework, they have written a second paper accepted for publication in Value in Health. They are also developing the R package useval available soon.

8.27 Axis 4: Artificial intelligence for aviation

Participants: Florent Dewez, Benjamin Guedj, Arthur Talpaert, Vincent Vandewalle.

Since November 2018, Benjamin Guedj and Vincent Vandewalle have been participating in the European PERF-AI project (European PERF-AI project: Enhance Aircraft Performance and Optimization through the utilization of Artificial Intelligence) in partnership with the company Safety Line. In particular, using data collected during flights involves developing Machine Learning models to optimize the aircraft's trajectory concerning fuel consumption, for example. In this context they have hired Florent Dewez (post-doctoral researcher) and Arthur Talpaert (engeneer).

The article 86 has been published last year. It explains how, using flight recording data, it is possible to implement learning models on variables that have not been directly observed, and in particular to predict the drag and lift coefficients as a function of the angle and speed of the aircraft.

A second article is in revision about the optimization of the aircraft's trajectory based on a consumption model learned from the data.

The originality of the approach consists in decomposing the trajectory on a functional basis, and thus carrying out the optimization on the coefficients of the decomposition on this basis, rather than approaching the problem from the angle of optimal control. Furthermore, to guarantee compliance with aeronautical constraints, we have proposed an approach penalized by a deviation term from reference flights. A generic Python module (PyRotor) to solve such optimization problems in conjunction with the proposed approach has been developed.

8.28 Axis 4: Interpretable Domain Adaptation for Hidden Subdomain Alignment in the Context of Pre-trained Source Models

Participants: Christophe Biernacki, Luxin Zhang.

Domain adaptation aims to leverage source domain knowledge to predict target domain labels. Most domain adaptation methods tackle a single-source, single-target scenario, whereas source and target domain data can often be subdivided into data from different distributions in real-life applications (e.g., when the distribution of the collected data changes with time). However, such subdomains are rarely given and should be discovered automatically. To this end, some recent domain adaptation works seek separations of hidden subdomains, w.r.t. a known or fixed number of subdomains. In contrast, this paper introduces a new subdomain combination method that leverages a variable number of subdomains. Precisely, we propose to use an inter-subdomain divergence maximization criterion to exploit hidden subdomains. Besides, our proposition stands in a target-to-source domain adaptation scenario, where one exploits a pre-trained source model as a black box; thus, the proposed method is model-agnostic. By providing interpretability at two complementary levels (transformation and subdomain levels), our method can also be easily interpreted by practitioners with or without machine learning backgrounds. Experimental results over two fraud detection datasets demonstrate the efficiency of our method.

It is a joint work with Pascal Germain from Université Laval (Canada) and with Yacine Kessaci from Worldline company.

8.29 Axis 4: Interpretable Domain Adaptation Using Unsupervised Feature Selection on Pretrained Source Models

Participants: Christophe Biernacki, Luxin Zhang.

We study a realistic domain adaptation setting where one has access to an already existing “black-box” machine learning model. Indeed, in real-life scenarios, an efficient pre-trained source domain predictive model is often available and required to be preserved. The solution we propose to this problem has the asset to provide an interpretable target to source transformation, by seeking a sparse and ordered coordinate-wise adaptation of the feature space, in addition to elementary mapping functions. To automatically select the subset of features to be adapted, we first introduce a weakly-supervised process relying on scarce labeled target data. Then, we address a more challenging unsupervised version of this domain adaptation scenario. To this end, we propose a new pseudo-label estimator over unlabeled target examples, which is based on the rank-stability in regards to the source model prediction. Such estimated “labels” are further used in a feature selection process to assess whether each feature needs to be transformed to achieve adaptation. We provide theoretical foundations of our method as well as an efficient implementation. Numerical experiments on real datasets show particularly encouraging results since approaching the supervised case, where one has access to labeled target samples. This work has been submited to an international journal 75.

It is a joint work with Pascal Germain from Université Laval (Canada) and with Yacine Kessaci from Worldline company.

8.30 Other: Projection Under Pairwise Control

Participants: Christophe Biernacki.

Visualization of high-dimensional and possibly complex (non-continuous for instance) data onto a low-dimensional space may be difficult. Several projection methods have been already proposed for displaying such high-dimensional structures on a lower-dimensional space, but the information lost is not always easy to use. Here, a new projection paradigm is presented to describe a non-linear projection method that takes into account the projection quality of each projected point in the reduced space, this quality being directly available in the same scale as this reduced space. More specifically, this novel method allows a straightforward visualization data in R2 with a simple reading of the approximation quality, and provides then a novel variant of dimensionality reduction.

It is a joint work with Hiba Alawieh and Nicolas Wicker, both from Université de Lille.

8.31 Other: On the Local and Global Properties of the Gravitational Spheres of Influence

Participants: Christophe Biernacki.

We revisit the concept of sphere of gravitational activity, to which we give both a geometrical and physical meaning. This study aims to refine this concept in a much broader context that could, for instance, be applied to exo-planetary problems (in a Galactic stellar disc-StarPlanets system) to define a first order “border” of a planetary system. The methods used in this paper rely on classical Celestial Mechanics and develop the equations of motion in the framework of the 3-body problem (e.g. Star-Planet-Satellite System. We start with the basic definition of planet’s sphere of activity as the region of space in which it is feasible to assume a planet as the central body and the Sun as the perturbing body when computing perturbations of the satellite’s motion. We then investigate the geometrical properties and physical meaning of the ratios of Solar accelerations (central and perturbing) and planetary accelerations (central and perturbing), and the boundaries they define. We clearly distinguish throughout the paper between the sphere of activity, the Chebotarev sphere (a particular case of the sphere of activity), Laplace sphere, and the Hill sphere. The last two are often wrongfully thought to be one and the same. Furthermore, taking a closer look and comparing the ratio of the star’s accelerations (central/perturbing) to that of the planetary acceleration (central/perturbing) as a function of the planeto-centric distance, we have identified different dynamical regimes which are presented in the semi-analytical analysis.

This a joint work with Damya Souami from Observatoire de Paris and with Jacky Cresson from Université de Pau et des Pays de l’Adour.

8.32 Axis 4: Single cell classification using statistical learning on mechanical properties measured by mems tweezers

Participants: Sophie Dabo-Niang.

Cell population is heterogenous and so presents a wide range of properties as metastatic potential. But using rare cells for clinical applications requires precise classification of individual cells. Here, we propose a multi-parameter analysis of single cells to classify them using statistical learning techniques and to predict the sub-population of each cell, although they may have close characteristics. We used MEMS tweezers to analyze mechanical properties (stiffness, viscosity, and size) of single cells from two different breast cancer cell lines in a controlled environment and run supervised learning methods to predict the population they belong to. This label-free method is a significant step forward to distinguish rare cell sub-populations for clinical applications.

This work has been presented ton an international conference "The 35th International Conference on Micro Electro Mechanical Systems", on January 2022. 58.

It is a joint work with Dominique Collard (LIMMS, CNRS, Universities of Lille and Tokyo), Cagatay Mehmed (LIMMS, CNRS, Universities of Lille and Tokyo) and others colleagues from University of Tokyo.

8.33 Axis 4: Dimensionality Reduction and Bandwidth Selection for Spatial Kernel Discriminant Analysis

Participants: Sophie Dabo-Niang.

Spatial Kernel Discriminant Analysis is a powerful tool for the classification of spatially dependent data. It allows taking into consideration the spatial autocorrelation of data based on a spatial kernel density estimator. The performance of SKDA is highly influenced by the choice of the smoothing parameters, also known as bandwidths. Moreover, computing a kernel density estimate is computationally intensive for high-dimensional datasets. In this paper, we consider the bandwidth selection as an optimization problem, that we resolve using Particle Swarm Optimization algorithm. In addition, we investigate the use of Principle Component Analysis as a feature extraction technique to reduce computational complexity and overcome curse of dimensionality drawback. We examined the performance of our model on Hyperspectral image classification. Experiments have given promising results on a commonly used dataset.

This work has been presented in the 13th International Conference on Agents and Artificial Intelligence ICAART and published in the proceeding.

It is a joint work with Soumia Boumeddane; Leila Hamdad; Hamid Haddadou (ESI, Algeria).

8.34 Axis 4: A kernel discriminant analysis for spatially dependent data

Participants: Sophie Dabo-Niang.

We propose a novel supervised classification algorithm for spatially dependent data, built as an extension of kernel discriminant analysis, that we named Spatial Kernel Discriminant Analysis (SKDA). Our algorithm is based on a kernel estimate of the spatial probability density function, which integrates a second kernel to take into account spatial dependency of data. In fact, classical data mining algorithms assume that data samples are independent and identically distributed. However, this assumption is not verified when dealing with spatial data characterized by spatial autocorrelation phenomenon. To make an accurate analysis, it is necessary to exploit this rich source of information and to capture this property. We have applied our algorithm to a relevant domain, which consist of the classification of remotely sensed hyperspectral images. In order to assess the efficiency of our proposed method, we conducted experiments on two remotely sensed images datasets (Indian Pines and Pavia University) with different characteristics and scenarios. The experimental results show that our method is competitive and achieves higher classification accuracy compared to other contextual classification methods.

This work has been published in Distributed and Parallel Databases. It is a joint work with Soumia Boumeddane; Leila Hamdad; Hamid Haddadou (ESI, Algeria).

8.35 Axis 2: Progress in Self-Certified Neural Networks

Participants: Benjamin Guedj.

A learning method is self-certified if it uses all available data to simultaneously learn a predictor and certify its quality with a tight statistical certificate that is valid with high confidence on any random data point. Self-certified learning promises to bring two major advantages to the machine learning community: First, it avoids the need to hold out data for validation and test purposes, both for certifying the model’s performance as well as for model selection. This could lead to a simplification of the machine learning data pipeline, while additionally, using all the available data for training could also lead to better representations of the underlying data distribution and ultimately lead to more accurate models. Secondly, self-certified learning focuses on delivering performance certificates that are valid with high confidence and are informative of the out-of-sample error, properties that are crucial for appropriately comparing machine learning models as well as setting performance standards for algorithmic governance of these models in the real world. In this paper, we assess how close we are to achieving self-certification in neural networks. In particular, recent work has shown that probabilistic neural networks trained by optimising PAC-Bayes generalisation bounds could bear promise towards achieving self-certified learning, since these can leverage all the available data to learn a posterior and simultaneously certify its risk with tight statistical performance certificates. In this work we empirically compare (on 4 classification datasets) test set generalisation bounds for deterministic predictors and a PAC-Bayes bound for randomised predictors obtained by a self-certified learning strategy (i.e. using all available data for training). We first show that both of these generalisation bounds are not too far from test set errors. We then show that in data small regimes, holding out data for the test set bounds adversely affects generalisation performance, while self-certified strategies based on PAC-Bayes bounds do not suffer from this drawback, showing that they might be a suitable choice for this small data regime. We also find that self-certified probabilistic neural networks learnt by PAC-Bayes inspired objectives lead to certificates that can be surprisingly competitive compared to commonly used test set bounds.

Accepted at the Bayesian Deep Learning workshop at NeurIPS 2021.

8.36 Axis 2: MMD Aggregated Two-Sample Test

Participants: Benjamin Guedj, Antonin Schrab.

We propose a novel nonparametric two-sample test based on the Maximum Mean Discrepancy (MMD), which is constructed by aggregating tests with different kernel bandwidths. This aggregation procedure, called MMDAgg, ensures that test power is maximised over the collection of kernels used, without requiring held-out data for kernel selection (which results in a loss of test power), or arbitrary kernel choices such as the median heuristic. We work in the non-asymptotic framework, and prove that our aggregated test is minimax adaptive over Sobolev balls. Our guarantees are not restricted to a specific kernel, but hold for any product of one-dimensional translation invariant characteristic kernels which are absolutely and square integrable. Moreover, our results apply for popular numerical procedures to determine the test threshold, namely permutations and the wild bootstrap. Through numerical experiments on both synthetic and real-world datasets, we demonstrate that MMDAgg outperforms alternative state-of-the-art approaches to MMD kernel adaptation for two-sample testing.

8.37 Axis 2: Learning PAC-Bayes Priors for Probabilistic Neural Networks

Participants: Benjamin Guedj.

Recent works have investigated deep learning models trained by optimising PAC-Bayes bounds, with priors that are learnt on subsets of the data. This combination has been shown to lead not only to accurate classifiers, but also to remarkably tight risk certificates, bearing promise towards self-certified learning (i.e. use all the data to learn a predictor and certify its quality). In this work, we empirically investigate the role of the prior. We experiment on 6 datasets with different strategies and amounts of data to learn data-dependent PAC-Bayes priors, and we compare them in terms of their effect on test performance of the learnt predictors and tightness of their risk certificate. We ask what is the optimal amount of data which should be allocated for building the prior and show that the optimum may be dataset dependent. We demonstrate that using a small percentage of the prior-building data for validation of the prior leads to promising results. We include a comparison of underparameterised and overparameterised models, along with an empirical study of different training objectives and regularisation strategies to learn the prior distribution.

8.38 Axis 2: On Margins and Derandomisation in PAC-Bayes

Participants: Benjamin Guedj, Felix Biggs.

We give a general recipe for derandomising PAC-Bayesian bounds using margins, with the critical ingredient being that our randomised predictions concentrate around some value. The tools we develop straightforwardly lead to margin bounds for various classifiers, including linear prediction – a class that includes boosting and the support vector machine – single-hidden-layer neural networks with an unusual erf activation function, and deep ReLU networks. Further, we extend to partially-derandomised predictors where only some of the randomness is removed, letting us extend bounds to cases where the concentration properties of our predictors are otherwise poor.

Accepted at AISTATS 2022.

8.39 Axis 2: Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound

Participants: Benjamin Guedj, Valentina Zantedeschi.

We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective. The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives – both with uninformed (data-independent) and informed (data-dependent) priors.

8.40 Axis 4: Covid-19 and AI: Unexpected Challenges and Lessons

Participants: Benjamin Guedj.

On May 21st, 2021, we held the webinar "Covid-19 and AI: unexpected challenges and lessons". This short note presents its highlights.

8.41 Axis 1: Forecasting elections results via the voter model with stubborn nodes

Participants: Benjamin Guedj, Antoine Vendeville.

In this paper we propose a novel method to forecast the result of elections using only official results of previous ones. It is based on the voter model with stubborn nodes and uses theoretical results developed in a previous work of ours. We look at popular vote shares for the Conservative and Labour parties in the UK and the Republican and Democrat parties in the US. We are able to perform time-evolving estimates of the model parameters and use these to forecast the vote shares for each party in any election. We obtain a mean absolute error of 4.74%. As a side product, our parameters estimates provide meaningful insight on the political landscape, informing us on the proportion of voters that are strong supporters of each of the considered parties.

8.42 Axis 2: Differentiable PAC–Bayes Objectives with Partially Aggregated Neural Networks

Participants: Benjamin Guedj, Felix Biggs.

We make two related contributions motivated by the challenge of training stochastic neural networks, particularly in a PAC–Bayesian setting: (1) we show how averaging over an ensemble of stochastic neural networks enables a new class of partially-aggregated estimators, proving that these lead to unbiased lower-variance output and gradient estimators; (2) we reformulate a PAC–Bayesian bound for signed-output networks to derive in combination with the above a directly optimisable, differentiable objective and a generalisation guarantee, without using a surrogate loss or loosening the bound. We show empirically that this leads to competitive generalisation guarantees and compares favourably to other methods for training such networks. Finally, we note that the above leads to a simpler PAC–Bayesian training scheme for sign-activation networks than previous work.

8.43 Axis 2: PAC-Bayes Unleashed: Generalisation Bounds with Unbounded Losses

Participants: Benjamin Guedj, Maxime Haddouche.

We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval [0;1]). In order to relax this classical assumption, we propose to allow the range of the loss to depend on each predictor. This relaxation is captured by our new notion of HYPothesis-dependent rangE (HYPE). Based on this, we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.

8.44 Axis 2: Still No Free Lunches: The Price to Pay for Tighter PAC-Bayes Bounds

Participants: Benjamin Guedj.

“No free lunch” results state the impossibility of obtaining meaningful bounds on the error of a learning algorithm without prior assumptions and modelling, which is more or less realistic for a given problem. Some models are “expensive” (strong assumptions, such as sub-Gaussian tails), others are “cheap” (simply finite variance). As it is well known, the more you pay, the more you get: in other words, the most expensive models yield the more interesting bounds. Recent advances in robust statistics have investigated procedures to obtain tight bounds while keeping the cost of assumptions minimal. The present paper explores and exhibits what the limits are for obtaining tight probably approximately correct (PAC)-Bayes bounds in a robust setting for cheap models.

8.45 Axis 1: Online k-means Clustering

Participants: Benjamin Guedj.

Abstract on AISTATS 2021 proceedings.

8.46 Axis 1: Sequential Learning of Principal Curves: Summarizing Data Streams on the Fly

Participants: Benjamin Guedj.

When confronted with massive data streams, summarizing data with dimension reduction methods such as PCA raises theoretical and algorithmic pitfalls. A principal curve acts as a nonlinear generalization of PCA, and the present paper proposes a novel algorithm to automatically and sequentially learn principal curves from data streams. We show that our procedure is supported by regret bounds with optimal sublinear remainder terms. A greedy local search implementation (called slpc, for sequential learning principal curves) that incorporates both sleeping experts and multi-armed bandit ingredients is presented, along with its regret computation and performance on synthetic and real-life data.

9.1 Bilateral contracts with industry

9.2 Bilateral grants with industry

10.1 European initiatives

10.2 National initiatives

10.3 Regional initiatives

11.1 Promoting scientific activities

11.2 Teaching - Supervision - Juries

11.3 Popularization

12.1 Major publications

• 1 articleP.Pierre Alquier and B.Benjamin Guedj. Simpler PAC-Bayesian Bounds for Hostile Data.Machine Learning2018
  HALDOI
• 2 articleP.Parmeet Bathia, S.Serge Iovleff and G.G. Govaert. An R Package and C++ library for Latent block models: Theory, usage and applications.Journal of Statistical Software2016
  HAL
• 3 articleC.Christophe Biernacki and A.Alexandre Lourme. Unifying Data Units and Models in (Co-)Clustering.Advances in Data Analysis and Classification1241May 2018
  HAL
• 4 articleA.Alain Celisse. Optimal cross-validation in density estimation with the L2-loss.The Annals of Statistics4252014, 1879--1910
  HAL
• 5 articleS.Sophie Dabo-Niang, C.Camille Ternynck and A.-F.Anne-Francoise Yao. Nonparametric prediction in the multivariate spatial context.Journal of Nonparametric Statistics2822016, 428-458
  HALDOI
• 6 articleJ.Julie Dubois, V.Vanessa Dubois, H.Hélène Dehondt, P.Parisa Mazrooei, C.Claire Mazuy, A. A.Aurélien A. Sérandour, C.Céline Gheeraert, P.Penderia Guillaume, E.Eric Baugé, B.Bruno Derudas, N.Nathalie Hennuyer, R.Réjane Paumelle, G.Guillemette Marot, J. S.Jason S. Carroll, M.Mathieu Lupien, B.Bart Staels, P.Philippe Lefebvre and J.Jerome Eeckhoute. The logic of transcriptional regulator recruitment architecture at cis -regulatory modules controlling liver functions.Genome Research276June 2017, 985--996
  HALDOI
• 7 inproceedingsG.Gaël Letarte, P.Pascal Germain, B.Benjamin Guedj and F.François Laviolette. Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks.NeurIPS 2019Vancouver, CanadaDecember 2019
  HAL
• 8 articleM.Matthieu Marbac, C.Christophe Biernacki and V.Vincent Vandewalle. Model-based clustering of Gaussian copulas for mixed data.Communications in Statistics - Theory and MethodsDecember 2016
  HAL
• 9 articleC.Cristian Preda, Q.Quentin Grimonprez and V.Vincent Vandewalle. Categorical Functional Data Analysis. The cfda R Package.Mathematics 923December 2021, 31
  HALDOIback to text
• 10 articleH.Hemant Tyagi and J.Jan Vybiral. Learning general sparse additive models from point queries in high dimensions.Constructive ApproximationJanuary 2019
  HAL

12.2 Publications of the year

12.3 Other

12.4 Cited publications

• 85 articleC.Christophe Biernacki, M.Matthieu Marbac and V.Vincent Vandewalle. Gaussian Based Visualization of Gaussian and Non-Gaussian Based Clustering.Journal of ClassificationJuly 2020
  HALDOIback to text
• 86 articleF.Florent Dewez, B.Benjamin Guedj and V.Vincent Vandewalle. From industry-wide parameters to aircraft-centric on-flight inference: improving aeronautics performance prediction with machine learning.Data-Centric EngineeringOctober 2020
  HALDOIback to text
• 87 articleH.Halina Frydman. Estimation in the Mixture of Markov Chains Moving With Different Speeds.Journal of the American Statistical Association1004712005, 1046-1053
  back to text
• 88 inproceedingsR.Rishi Gupta, R.Ravi Kumar and S.Sergei Vassilvitskii. On mixtures of Markov chains.Proceedings of the 30th International Conference on Neural Information Processing SystemsCiteseer2016, 3449--3457
  back to text
• 89 articleV.Vincent Vandewalle, A.Alexandre Caron, C.Coralie Delettrez, R.Renaud Périchon, S.Sylvia Pelayo, A.Alain Duhamel and B.Benoit Dervaux. Estimating the number of usability problems affecting medical devices: modelling the discovery matrix.BMC Medical Research Methodology201September 2020
  HALDOIback to text