7.2.1 PDiscoFormer

• Name:
  PDiscoFormer
• Keywords:
  Deep learning, Computer vision, Artificial intelligence
• Functional Description:
  Computer vision methods that explicitly detect object parts and reason on them are a step towards inherently interpretable models. Existing approaches that perform part discovery driven by a fine-grained classification task make very restrictive assumptions on the geometric properties of the discovered parts, they should be small and compact. Although this prior is useful in some cases, in this paper we show that pre-trained transformer-based vision models, such as self-supervised DINOv2 ViT, enable the relaxation of these constraints. In particular, we find that a total variation (TV) prior, which allows for multiple connected components of any size, substantially outperforms previous work. We test our approach on three fine-grained classification benchmarks: CUB, PartImageNet and Oxford Flowers, and compare our results to previously published methods as well as a re-implementation of the state-of-the-art method PDiscoNet with a transformer-based backbone. We consistently obtain substantial improvements across the board, both on part discovery metrics and the downstream classification task, showing that the strong inductive biases in self-supervised ViT models require to rethink the geometric priors that can be used for unsupervised part discovery.
• Release Contributions:
  Developed the library from scratch
• URL:
  https://­github.­com/­ananthu-aniraj/­pdiscoformer
• Contact:
  Ananthu Aniraj

7.3.1 MORINGA: an open-source platform for automatic land cover classification from multi-sensor imagery

Participants: Raffaele Gaetano.

Started in 2015 in the framework of the activities of the Land Cover Scientific Expertise Center as part of the French Land Surface Data and Services Hub - THEIA, the developement of the MORINGA 1 processing chain was initially aimed at providing a “turn key” solution, addressed to thematic specialists with relatively low programming skills, for the automatic land cover classification from multi-sensor, multi-resolution and multi-temporal satellite imagery. It particularly targets the needs for accurate land cover mapping in the context of tropical agricultural systems, where several specificities (landscape heterogeneity and fragmentation, small field sizes, high cloud coverage during cropping seasons) call for the combination of different resolutions and acquisition modes to both capture spatial details and temporal profiles. Leveraging an object-based approach and a suitable supervised classification framework based on legacy machine learning techniques, the MORINGA processing chain takes in charge imagery provided by different satellite missions, both at high (Sentinel-1 and -2, Landsat 8/9, etc.) and very high (Pléiades, SPOT6/7) spatial resolution, and automatically manages their pre-processing and ingestion in the object based machine learning framework, with limited user interaction. Reference data are also automatically processed to provide the best possible validation also in cases of data paucity, which are rather common in the targeted application. To date, MORINGA has become a feature-rich, modular platform for remote sensing image analysis, which can also be used as a lower-level API to ease common image processing tasks. Thanks to the support of thematic experts and cartography specialists, it has since been used for the production of high quality land cover maps in many different scientific and dissemination contexts (see 38 for a notable example in 2024). The software package is currently bound to evolve to a larger remote sensing based land cover workbench, including novel deep learning techniques for both image pre-processing/enhancement and multi-sensor classification.

7.3.2 Benchopt: an open optimization benchmarking platform

Participants: Cássio Fraga Dantas.

Participants: Dino Ienco, Diego Marcos, Raffaele Gaetano, Cássio Fraga Dantas, Roberto Interdonato.

A worth-mentioning open-source initiative, in which participates the team member Cassio F. Dantas, is the Benchopt platform 37 2. It consists in a larger-scale collective effort involving researchers from various institutions in a national level, including École Normale Supérieure Paris and Institut Mines-Télécom as well as other Inria teams such as MIND (Paris) and DANTE (Lyon). Benchopt provides a collaborative framework to automate, reproduce and publish benchmarks of optimization algorithms across programming languages and hardware architectures. Among the initiated benchmarks are several inverse problems and machine learning tasks which are often present in the backbones of many application domains, including in remote sensing: from ordinary and non-negative least-squares, to several denoising and regression problems, or even image classification with residual neural networks. But more importantly, the idea is for the research community to contribute by adding new competing algorithms or even new benchmarks on their problems of interest. This may benefit the entire community by providing a reliable, transparent and reproducible comparison between existing approaches, allowing practitioners to easily choose a suitable off-the-shelf approach for their application scenario and alleviating researchers from the burden of reimplementation when proposing new algorithms. Since its official release marked by the NeurIPS publication in 2022 37, the platform is permanently maintained by an expanding community that remains in direct contact via a Discord channel and organizes yearly in-person meetups (Benchopt sprints). The purpose of these events is to harness collective efforts to drive key enhancements to the platform, identify new working groups and kick-start projects to be continued remotely after the event. ———————————–

8.1.1 DIAMANTE: A data-centric semantic segmentation approach to map tree dieback induced by bark beetle infestations via satellite images

Participants: Dino Ienco.

Collaborators: Giuseppina Andresini (University of Bari, Italy), Annalisa Appice (University of Bari, Italy), Vito Recchia (University of Bari, Italy).

Keywords: Forest mapping, Forest Disturbances, deep learning, multi-sensor fusion, semantic segmentation.

Forest tree dieback inventory has a crucial role in improving forest management strategies.

Show image description

Late fusion approach to merge together information from Sentinel-1 and Sentinel-2 remote sensing data for the downstream task of semantic segmentation of forest tree dieback.

This inventory is traditionally performed in forests through laborious and time-consuming human assessment of individual trees. On the other hand, the large amount of Earth satellite data that are publicly available with the Copernicus program and can be processed through advanced deep learning techniques has recently been established as an alternative to field surveys for forest tree dieback tasks. However, to realize its full potential, deep learning requires a fine understanding of satellite data since the data collection and preparation steps are essential as the model development step. Here, we have explored the performance of a data-centric semantic segmentation approach to detect forest tree dieback events due to bark beetle infestation in satellite images. The proposed approach prepares a multisensor data set collected using both the SAR Sentinel-1 sensor and the optical Sentinel-2 sensor and uses this dataset to train a multi-sensor semantic segmentation model (Figure 1). The evaluation shows the effectiveness of the proposed approach in a real inventory case study that regards non-overlapping forest scenes from the Northeast of France acquired in October 2018. The selected scenes host bark beetle infestation hotspots of different sizes, which originate from the mass reproduction of the bark beetle in the 2018 infestation.

This work has been published in the Journal of Intelligent Information Systems (Springer) journal 7.

8.1.2 Generation of country-scale canopy height maps over Gabon using deep learning and TanDEM-X InSAR data

Participants: Dino Ienco.

Collaborators: Daniel Carcereria (German Aerospace Center (DLR), Germany), Paola Rizzoli (German Aerospace Center (DLR), Germany), Luca Dell’Amore (German Aerospace Center (DLR), Germany), José-Luis Bueso-Bello (German Aerospace Center (DLR), Germany), Lorenzo Bruzzone (Università degli Studi di Trento, Italy)

Keywords: Forest height, forest parameter regression, deep learning, bistatic SAR, interferometric coherence, InSAR, TanDEM-X, LVIS.

Operational canopy height mapping at high resolution remains a challenging task at country-level. Most of the existing state-of-the-art inversion methods propose physically-based schemes which are specifically tuned for local scales.

Show image description

Country-scale mosaic of Gabon representing the Canopy Height Map, generated using TanDEM-X acquisitions from the first global covered of the mission (Dec. 2010 - end of 2011).

Only few approaches in the literature have attempted to produce country or global scale estimates, mostly by means of data-driven approaches and multi-spectral data sources. In this work, we propose a robust deep learning approach that exploits single-pass interferometric TanDEM-X data to generate accurate forest height estimates from a single interferometric bistatic acquisition. The model development is driven by considerations on both the final performance and the trustworthiness of the model for large-scale deployment in the context of tropical forests. We train and test our model over the five tropical sites of the AfriSAR 2016 campaign, situated in the West Central state of Gabon, performing spatial cross-validation experiments to test its generalization capability. We define a specific training dataset and input predictors to develop a robust model for country-scale inference, by finding an optimal trade-off between the model performance and the large-scale reliability. The proposed model achieves an overall estimation bias of 0.12 m, a mean absolute error of 3.90 m, a root mean squared error of 5.08 m and a coefficient of determination of 0.77. Finally, we generate a time-tagged country-scale canopy height map of Gabon at 25 m resolution (Figure 2), discussing the potential and challenges of these kinds of products for their application in different scenarios and for the monitoring of forest changes.

This work has been published in the Remote Sensing of Environment (Elsevier) journal 9.

8.1.3 Early Season Forecasting of Corn Yield at Field Level from Multi-Source Satellite Time Series Data

Participants: Dino Ienco.

Collaborators: Johann Desloires (Airbus DS), Antoine Botrel (Syngenta Seeds, France)

Keywords: Yield forecasting, machine learning, thermal time, Sentinel-2, land surface temperature, early season forecasting.

Crop yield forecasting during an ongoing season is crucial to ensure food security and commodity markets. For this reason, here, a scalable approach to forecast corn yields at the field-level using machine learning and satellite imagery from Sentinel-2 and Landsat missions is proposed. The model, evaluated on 1319 corn fields in the U.S. Corn Belt from 2017 to 2022, integrates biophysical parameters from Sentinel-2, Land Surface Temperature (LST) from Landsat, and agroclimatic data from ERA5 reanalysis dataset. Resampling the time series over thermal time significantly enhances predictive performance. The addition of LST to our model further improves in-season yield forecasting, through its capacity to detect early drought, which is not immediately visible to optical sensors such as the Sentinel-2. Here, we propose a new two-stage machine learning strategy (see Figure 3) to mitigate early season partially available data. It consists in extending the current time series on the basis of complete historical data and adapting the model inference according to the crop progress.

Show image description

DNN architecture with LSTM and dense layers. Concatenation of dynamic and static paths. Node dimensions indicated. Each Dense hidden layer is followed by batch normalization and ReLU activation function with a dropout rate of 0.5.

This work has been published in the Remote Sensing (MDPI) journal 11.

8.1.4 Machine Learning-Based Summer Crops Mapping Using Sentinel-1 and Sentinel-2 Images

Participants: Cássio Fraga Dantas, Dino Ienco.

Collaborators: Saeideh Maleki (UMR TETIS, INRAE, France), Nicolas Baghdadi (UMR TETIS, INRAE, France), Hassan Bazzi (UMR TETIS, AgroParisTech, France), Yasser Nasrallah (UMR TETIS, INRAE, France) ,Sami Najem (UMR TETIS, INRAE, France),

Keywords: Rapeseed mapping, Sentinel-1, Sentinel-2, machine learning, neural networks.

Accurate crop type mapping using satellite imagery is crucial for food security, yet accurately distinguishing between crops with similar spectral signatures is challenging. This study assessed the performance of Sentinel-2 (S2) time series (spectral bands and vegetation indices), Sentinel-1 (S1) time series (backscattering coefficients and polarimetric parameters), alongside phenological features derived from both S1 and S2 time series (harmonic coefficients and median features), for classifying sunflower, soybean, and maize. Random Forest (RF), Multi-Layer Perceptron (MLP), and XGBoost classifiers were applied across various dataset configurations and train-test splits over two study sites and years in France. Additionally, the InceptionTime classifier, specifically designed for time series data, was tested exclusively with time series datasets to compare its performance against the three general machine learning algorithms (RF, XGBoost, and MLP). The results showed that XGBoost outperformed RF and MLP in classifying the three crops. The optimal dataset for mapping all three crops combined S1 backscattering coefficients with S2 vegetation indices, with comparable results between phenological features and time series data (mean F1 scores of 89.9% for sunflower, 76.6% for soybean, and 91.1% for maize). However, when using individual satellite sensors, S1 phenological features and time series outperformed S2 for sunflower, while S2 was superior for soybean and maize. Both phenological features and time series data produced close mean F1 scores across spatial, temporal, and spatiotemporal transfer scenarios, though median features dataset was the best choice for spatiotemporal transfer. Polarimetric S1 data did not yield effective results. The InceptionTime classifier further improved classification accuracy over XGBoost for all crops, with the degree of improvement varying by crop and dataset (the highest mean F1 scores of 90.6% for sunflower, 86.0% for soybean, and 93.5% for maize).

This work has been published in the Remote Sensing (MDPI) journal 15.

8.1.5 Determining Effective Temporal Windows for Rapeseed Detection Using Sentinel-1 Time Series and Machine Learning Algorithms

Participants: Cássio Fraga Dantas, Dino Ienco.

Collaborators: Saeideh Maleki (UMR TETIS, INRAE, France), Nicolas Baghdadi (UMR TETIS, INRAE, France), Sami Najem (UMR TETIS, INRAE, France), Hassan Bazzi (Atos France, France)

Keywords: Rapeseed mapping, Sentinel-1, machine learning, neural networks.

This study investigates the potential of Sentinel-1 (S1) multi-temporal data for the early-season mapping of the rapeseed crop. Additionally, we explore the effectiveness of limiting the portion of a considered time series to map rapeseed fields. To this end, we conducted a quantitative analysis to assess several temporal windows (periods) spanning different phases of the rapeseed phenological cycle in the following two scenarios relating to the availability or constraints of providing ground samples for different years: i) involving the same year for both training and the test, assuming the availability of ground samples for each year; and ii) evaluating the temporal transferability of the classifier, considering the constraints of ground sampling. We employed two different classification methods that are renowned for their high performance in land cover mapping: the widely adopted random forest (RF) approach and a deep learning-based convolutional neural network, specifically the InceptionTime method. To assess the classification outcomes, four evaluation metrics (recall, precision, F1 score, and Kappa) were employed. Using S1 time series data covering the entire rapeseed growth cycle, the tested algorithms achieved F1 scores close to 95% on same-year training and testing, and 92.0% when different years were used, both algorithms demonstrated effective performance. Our findings underscore the importance of a concise S1 time series for effective rapeseed mapping, offering advantages in data storage and processing time. Overall, the study establishes the robustness of RF and InceptionTime in rapeseed detection scenarios, providing valuable insights for agricultural applications.

This work has been published in the Remote Sensing (MDPI) journal 16.

8.1.6 Integrating Predictive Process Monitoring Techniques in Smart Agriculture

Participants: Roberto Interdonato, Dino Ienco.

Collaborators: Simona Fioretto (University of Naples Federico II, Italy), Elio Masciari (University of Naples Federico II, Italy)

Keywords: Predictive Process Monitoring, crop rotation, machine learning.

Problems related to the environment are increasingly commonly known and consequently also technology is adapting to find suitable solutions. The ancestral technique of crop rotation was identified as a solution to address the problems related to pollution due to intensive food production (i.e. using fertilizers and pesticides). To ensure that this technique can actually improve food production, it is necessary to understand how modern technologies can support it. In particular, the analysis of crop rotation can support farmers in decision-making process and the optimization of farm management practices. The aim of this paper is to investigate how predictive process monitoring techniques can enhance crop rotation strategies by leveraging Agriculture 4.0 through real-time monitoring, resulting in more accurate and adaptive strategies. It is a position paper that proposes research questions for further study, which may help to develop the research area.

This work has been published at the International Symposium on Methodologies for Intelligent Systems (ISMIS) 2024 25.

8.1.7 Multi-source deep-learning approach for automatic geomorphological mapping: the case of glacial moraines

Participants: Isabelle Rocamora, Dino Ienco.

Collaborators: Matthieu Ferry (Univ. Montpellier, France)

Keywords: Deep learning, data fusion, geomorphology, moraines, mapping.

Show image description

(a) Flowchart of our framework. (b) MorNet’s architecture.

Landform mapping is the initial step of many geomorphological analyses (e.g., assessment of natural hazards and natural resources) and requires vast resources to be applied to wide areas at high-resolution. Among geomorphological objects, we focus on glacial moraine mapping, since it is a task relevant to many fields (e.g., paleoclimate and glacial geomorphology). Here we proposed to exploit the potential of Deep Learning-based approaches to map moraine landforms by exploiting multi-source remote sensing imagery. To this end, we propose the first Deep Learning model to map glacial moraines, namely MorNet (Figure 4). As multi-source remote sensing information, we combine together three different sources: Topographic (Pleiades-derived Digital Surface Model), Multispectral (Sentinel-2), and SAR (Sentinel-1) data. To cope with such heterogeneous information, the proposed model has a dedicated branch for each input source and, a late fusion mechanism is leveraged to combine them with the aim to provide the final mapping. The performance of the MorNet model is evaluated on several glacier valleys in China in the Himalayan range. This area contains minimally eroded moraines, so they are well-defined and of varied morphology. The behavior of the proposed method is compared to models using individual mono-source models in order to highlight the benefit to simultaneously leverage multi-source information. The use of multi-source data allows MorNet to exploit the complementarity of the three input sources and improve its performance from an F1-score of about 41.6 using a single source to 52.8 using three sources. MorNet provides a first-order moraine map through its ability to identify well-defined moraines. Consequently, MorNet can identify areas likely to contain moraines and intends to be used as a tool by experts to facilitate and support large-scale mapping.

This work has been published in the Geo-spatial Information Science (Taylor & Francis) journal 22.

8.1.8 GeoPlant: Spatial Plant Species Prediction Dataset

Participants: Diego Marcos.

Collaborators: Lukas Picek (Inria, Zenith, France), Christophe Botella (Inria, Zenith, France), Maximilien Servajean (LIRMM, Université de Montpellier, France), César Leblanc (Inria, Zenith, France), Rémi Palard (Inria, Zenith, France), Théo Larcher (Inria, Zenith, France), Benjamin Deneu (Inria, Zenith, France), Pierre Bonnet (Amap, Cirad, France), Alexis Joly (Inria, Zenith, France)

Keywords: Deep learning, data fusion, species distribution models.

The difficulty of monitoring biodiversity at fine scales and over large areas limits ecological knowledge and conservation efforts. To fill this gap, Species Distribution Models (SDMs) predict species across space from spatially explicit features. Yet, they face the challenge of integrating the rich but heterogeneous data made available over the past decade, notably millions of opportunistic species observations and standardized surveys, as well as multi-modal remote sensing data. In light of that, we have designed and developed a new European-scale dataset for SDMs at high spatial resolution (10-50 m), including more than 10k species (i.e., most of the European flora). The dataset comprises 5M heterogeneous Presence-Only records and 90k exhaustive Presence-Absence survey records, all accompanied by diverse environmental rasters (e.g., elevation, human footprint, and soil) that are traditionally used in SDMs. In addition, it provides Sentinel-2 RGB and NIR satellite images with 10 m resolution, a 20-year time-series of climatic variables, and satellite time-series from the Landsat program. In addition to the data, we provide an openly accessible SDM benchmark (hosted on Kaggle), which has already attracted an active community and a set of strong baselines for single predictor/modality and multimodal approaches. All resources, e.g., the dataset, pre-trained models, and baseline methods (in the form of notebooks), are available on Kaggle, allowing one to start with our dataset literally with two mouse clicks.

This work has been accepted to NeurIPS 2024 Datasets and Benchmarks track 32, 31.

8.1.9 Mapping the diversity of land uses following deforestation across Africa

Participants: Diego Marcos.

Collaborators: Robert N. Masolele (Wageningen University and Research, The Netherlands), Veronique De Sy (Wageningen University and Research, The Netherlands), Itohan‑Osa Abu (Julius-Maximilians-University, Germany), Jan Verbesselt (Wageningen University and Research, The Netherlands), Johannes Reiche (Wageningen University and Research, The Netherlands), Martin Herold (German GeoResearch Center, Germany)

Keywords: Deforestation drivers, continental scale, land-use following deforestation, high-resolution satellite imagery.

African forests are increasingly in decline as a result of land-use conversion due to human activities. However, a consistent and detailed characterization and mapping of land-use change that results in forest loss are not available at the spatial-temporal resolution and thematic levels that are suitable for decision-making at the local and regional scales; so far they have only been provided on coarser scales and restricted to humid forests. Here we present the first high-resolution (5 m) and continental-scale mapping of land use following deforestation in Africa, which covers an estimated 13.85% of the global forest area, including humid and dry forests. We use reference data for 15 different land-use types from 30 countries and implement an active learning framework to train a deep learning model for predicting land-use following deforestation with an F1-score of 84 $\pm$ 0.7 for the whole of Africa. Our results show that the causes of forest loss vary by region. In general, small-scale cropland is the dominant driver of forest loss in Africa, with hotspots in Madagascar and DRC. In addition, commodity crops such as cacao, oil palm, and rubber are the dominant drivers of forest loss in the humid forests of western and central Africa, forming an “arc of commodity crops” in that region. At the same time, the hotspots for cashew are found to increasingly dominate in the dry forests of both western and south-eastern Africa, while larger hotspots for large-scale croplands were found in Nigeria and Zambia. The increased expansion of cacao, cashew, oil palm, rubber, and large-scale croplands observed in humid and dry forests of western and south-eastern Africa suggests they are vulnerable to future land-use changes by commodity crops, thus creating challenges for achieving the zero deforestation supply chains, supporting REDD+ initiatives, and towards sustainable development goals.

This work has been published as a research article in the journal Scientific Reports 18.

8.2.1 DisCoM-KD: Cross-Modal Knowledge Distillation via Disentanglement Representation and Adversarial Learning

Participants: Cássio Fraga Dantas, Dino Ienco.

Keywords: Multi-modal learning, Cross-Modal Knowledge Distillation, Classification, Disentanglement learning.

Cross-modal knowledge distillation (CMKD) refers to the scenario in which a learning framework must handle training and test data that exhibit a modality mismatch, more precisely, training and test data do not cover the same set of data modalities. Traditional approaches for CMKD are based on a teacher/student paradigm, where a teacher is trained on multi-modal data with the aim to successively distill knowledge from a multi-modal teacher to a single-modal student. Despite the widespread adoption of such paradigm, recent research has highlighted its inherent limitations in the context of cross-modal knowledge transfer. Taking a step beyond the teacher/student paradigm, in this work, we introduce a new framework for cross-modal knowledge distillation, named DisCoM-KD (Disentanglement-learning based Cross-Modal Knowledge Distillation), that explicitly models different types of per-modality information with the aim to transfer knowledge from multi-modal data to a single-modal classifier (Figure 5). To this end, DisCoM-KD effectively combines disentanglement representation learning with adversarial domain adaptation to simultaneously extract, for each modality, domain-invariant, domain-informative and domain-irrelevant features according to a specific downstream task. Unlike the traditional teacher/student paradigm, our framework simultaneously learns all single-modal classifiers, eliminating the need to learn each student model separately as well as the teacher classifier. We evaluated DisCoM-KD on three standard multi-modal benchmarks and compared its behavior with recent state of the art knowledge distillation frameworks. The findings clearly demonstrate the effectiveness of DisCoM-KD over competitors considering mismatch scenarios involving both overlapping and non-overlapping modalities. These results offer insights to reconsider the traditional paradigm for distilling information from multi-modal data to single-modal neural networks.

Show image description

Schematic overview of DisCoM-KD: On the left, there are two per-modality branch extractors for modalities $M1$ and $M2$, along with two per-modality task classifiers to obtain the final prediction. On the right, several auxiliary classifiers, acting on intermediate representations, help disentangling per-modality information and make representations task informative. The training of the two parallel architectures is performed jointly, but at inference time, each model is deployed independently.

This work has been published at the British Machine Vision Conference (BMVC) 2024 26.

8.2.2 Semi Supervised Heterogeneous Domain Adaptation via Disentanglement and Pseudo-Labelling

Participants: Cássio Fraga Dantas, Raffaele Gaetano, Dino Ienco.

Keywords: Domain adaptation, heterogeneous data, Feature disentanglement, Pseudo-labeling, consistency regularization.

Semi-supervised domain adaptation methods leverage information from a source labelled domain with the goal of generalizing over a scarcely labelled target domain. While this setting already poses challenges due to potential distribution shifts between domains, an even more complex scenario arises when source and target data differs in modality representation (e.g. they are acquired by sensors with different characteristics). For instance, in remote sensing, images may be collected via various acquisition modes (e.g. optical or radar), different spectral characteristics (e.g. RGB or multi-spectral) and spatial resolutions. Such a setting is denoted as Semi-Supervised Heterogeneous Domain Adaptation (SSHDA) and it exhibits an even more severe distribution shift due to modality heterogeneity across domains. To cope with the challenging SSHDA setting, here we introduce SHeDD (Semi-supervised Heterogeneous Domain Adaptation via Disentanglement) an end-to-end neural framework tailored to learning a target do-main classifier by leveraging both labelled and unlabelled data from heterogeneous data sources (Figure 6). SHeDD is designed to effectively disentangle domain-invariant representations, relevant for the downstream task, from domain-specific information, that can hinder the cross-modality transfer. Additionally, SHeDD adopts an augmentation-based consistency regularization mechanism that takes advantages of reliable pseudo-labels on the unlabelled target samples to further boost its generalization ability on the target domain. Empirical evaluations on two remote sensing bench-marks, encompassing heterogeneous data in terms of acquisition modes and spectral/spatial resolutions, demonstrate the quality of SHeDD compared to both baseline and state-of-the-art competing approaches

Show image description

Schematic view of the proposed method architecture with a separate encoder for each of the data modalities (source and target). Feature disentanglement enables domain-specific and domain-invariant information to be encoded separately into each half of the generated embedding vectors (depicted in orange and green respectively). The domain-invariant information ($z^{inv}$) is used by the task classifier, while the domain classifier receives the domain-specific portion of the embedding vector ($z^{spe}$). At inference time, only the bottom part of the architecture is used, the top part being instrumental in the training stage to enable the feature disentanglement procedure.

This work has been published at the European conference on Machine Leanring (ECML/PKDD) 2024 24.

8.2.3 Orthrus: multi-scale land cover mapping from satellite image time series via 2D encoding and convolutional neural network

Participants: Dino Ienco.

Collaborators: Ali Ben Abbes (Univ. Manouba, Tunisia), Imed Riadh Farah (Univ. Manouba, Tunisia)

Keywords: Deep Learning, pixel-object classification, convolutional neural networks (CNN), multivariate time-series , classification, 2D encoding representation, land use land cover.

With the advent of modern Earth observation (EO) systems, the opportunity of collecting satellite image time series (SITS) provides valuable insights to monitor spatiotemporal dynamics. Within this context, accurate land use/land cover (LULC) mapping plays a pivotal role in supporting territorial management and facilitating informed decision-making processes. However, traditional pixel-based and object-based classification methods often face challenges to effectively exploit spectral and spatial information. In this study, we propose Orthrus, a novel approach that fuses multi-scale information for enhanced LULC mapping. The proposed approach exploits several 2D encoding techniques to encode times series information into imagery. The resulting image is leveraged as input to a standard convolutional neural network (CNN) image classifier to cope with the downstream classification task. The evaluations on two real-world benchmarks, namely Dordogne and Reunion-Island, demonstrated the quality of Orthrus over state-of-the-art techniques from the field of land cover mapping based on SITS data. More precisely, Orthrus exhibits an enhancement of more than 3.5 accuracy points compared to the best competing approach on the Dordogne benchmark, and surpasses the best competing approach on the Reunion-Island dataset by over 3 accuracy points.

This work has been published in the Neural Computing and Applications (Springer) journal 6.

8.2.4 A constrastive semi-supervised deep learning framework for land cover classification of satellite time series with limited labels

Participants: Roberto Interdonato, Raffaele Gaetano, Dino Ienco.

Keywords: Satellite image time series, land cover mapping, semi-supervised learning, deep learning.

In this work, we present a new semi-supervised learning framework to cope with satellite image time series (SITS) classification in a data paucity scenario, considering extremely low levels of supervision. The proposed methodology, referred to as S${}^3$ITS (Semi-Supervised Satellite Image Time Series classification method), is based on temporal convolutional neural networks and it takes advantage of both labelled and unlabelled information. S ${}^3$ITS enforces the data to be projected in a discriminative manifold via contrastive learning, in order to produce a data representation where samples belonging to the same category are closer than the ones belonging to different ones. Pseudo-labelling is employed on unlabelled samples to take the most out of the available unlabelled information. Experiments on two study sites described by satellite image time series of Sentinel-2 images highlight the quality of the proposed method with respect to common classification methods and recent machine learning approaches especially tailored for the semi-supervised classification of multi-variate time series data. This work has been published in the Neurocomputing (Elsevier) journal 13.

8.2.5 Reuse out-of-year data to enhance land cover mapping via feature disentanglement and contrastive learning

Participants: Cássio Fraga Dantas, Raffaele Gaetano, Dino Ienco.

Collaborators: Claudia Paris (University of Twente, Netherlands)

Keywords: Satellite image time series (SITS), land cover mapping, domain adaptation, contrastive learning, data-centric artificial intelligence.

Given the systematic acquisition of satellite data, it is possible to generate up-to-date land cover (LC) maps, essential for effective agricultural territory management, environmental monitoring, and informed decision-making. Typically, creating a LC map requires collecting high-quality labeled data, a process that is both costly and time-consuming. To mitigate the need to collect large volume of labeled data, we propose a deep learning framework called REFeD (data Reuse with Effective Feature Disentanglement for land cover mapping), which leverages already available out-of-year reference data to enhance the production of up-to-date LC maps (Figure 7). To this end, REFeD integrates remote sensing and reference data from different domains (e.g., historical and recent data) utilizing a disentanglement strategy based on contrastive learning. By separating domain-invariant and domain-specific features, REFeD isolates useful information associated to the downstream LC mapping task and mitigates distribution shifts between domains. Moreover, REFeD incorporates an effective supervision scheme to reinforce feature disentanglement through multiple levels of supervision at different granularities. Experimental evaluation on study areas characterized by diverse landscapes, including Koumbia (West Africa, Burkina Faso) and Centre-Val de Loire (central Europe, France), demonstrates the effectiveness of the proposed approach

Show image description

Architecture of the proposed pseudo-siamese network used in the training stage and composed of two independent branches which disentangle the domain-invariant information (top branch) from domain-specific information (bottom branch). Class (${ℒ}_{cl}$) and domain (${ℒ}_{dom}$) discrimination losses used respectively on the top and bottom branches, while a multi-level contrastive loss (${ℒ}_{con}$) is used to intermediate features at different depths from both branches. At inference time, only the domain-invariant encoder is used for classifying the target domain.

This work is currently available as a preprint 10 and under submission to a remote sensing journal.

8.2.6 Prompt-guided and multimodal landscape scenicness assessments with vision-language models

Participants: Diego Marcos.

Collaborators: Alex Levering (Vrije Universiteit Amsterdam, Netherlands), Nathan Jacobs (Washington University in St. Louis, USA), Devis Tuia (EPFL, Switzerland)

Keywords: Vision-language models, landscape scenicness.

Recent advances in deep learning and Vision-Language Models (VLM) have enabled efficient transfer to downstream tasks even when limited labelled training data is available, as well as for text to be directly compared to image content. These properties of VLMs enable new opportunities for the annotation and analysis of images. We test the potential of VLMs for landscape scenicness prediction, i.e., the aesthetic quality of a landscape, using zero- and few-shot methods. We experiment with few-shot learning by fine-tuning a single linear layer on a pre-trained VLM representation. We find that a model fitted to just a few hundred samples performs favorably compared to a model trained on hundreds of thousands of examples in a fully supervised way. We also explore the zero-shot prediction potential of contrastive prompting using positive and negative landscape aesthetic concepts. Our results show that this method outperforms a linear probe with few-shot learning when using a small number of samples to tune the prompt configuration. We introduce Landscape Prompt Ensembling (LPE), which is an annotation method for acquiring landscape scenicness ratings through rated text descriptions without needing an image dataset during annotation. We demonstrate that LPE can provide landscape scenicness assessments that are concordant with a dataset of image ratings. The success of zero- and few-shot methods combined with their ability to use text-based annotations highlights the potential for VLMs to provide efficient landscape scenicness assessments with greater flexibility.

This work has been published as a research aricle in the journal PloS one 14.

8.3.1 Coarse-to-Fine Concept Bottleneck Models

Participants: Konstantinos Panousis, Dino Ienco, Diego Marcos.

Keywords: Interpretability, explainability, multi-modal, bayesian deep learning, sparsification.

Deep learning algorithms have recently gained significant attention due to their impressive performance.

Show image description

(Left) The Concept Discovery Block (CDB). Given a set of concepts and an image, we compute their similarity via a VLM; we consider a data-driven mechanism for concept discovery, sampling from an amortized Bernoulli posterior. (Right) A schematic of the envisioned CF-CBMs. We consider a set of high level concepts, each described by a number of attributes; this forms the pool of low-level concepts. Our objective is to discover concepts that describe the whole image, while exploiting information residing in, in this case P = 9, patch-specific regions by matching low-level concepts to each patch and aggregate the information to obtain a single representation. Each level comprises CDBs, while the levels are linked together via the binary indicators $Z_H$ and $Z_L$.

However, their high complexity and un-interpretable mode of operation hinders their confident deployment in real-world safety-critical tasks. This work targets ante hoc interpretability, and specifically Concept Bottleneck Models (CBMs). Our goal is to design a framework that admits a highly interpretable decision making process with respect to human understandable concepts, on two levels of granularity. To this end, we propose a novel two-level concept discovery formulation leveraging: i) recent advances in vision-language models, and ii) an innovative formulation for coarse-to-fine concept selection via data-driven and sparsity-inducing Bayesian arguments. Figure 8 depicts the general schema of the proposed approach. Within this framework, concept information does not solely rely on the similarity between the whole image and general unstructured concepts; instead, we introduce the notion of concept hierarchy to uncover and exploit more granular concept information residing in patch-specific regions of the image scene. As we experimentally show, the proposed construction not only outperforms recent CBM approaches, but also yields a principled framework towards interpetability.

This work has been published at the Neural Information Processing Systems (NeurIPS) 2024 conference 30.

8.3.2 PDiscoFormer: Relaxing Part Discovery Constraints with Vision Transformers

Participants: Ananthu Aniraj, Cássio Fraga Dantas, Dino Ienco, Diego Marcos.

Keywords: Part Detection, transformers, interpretability.

Computer vision methods that explicitly detect object parts and reason on them are a step towards inherently interpretable models. Existing approaches that perform part discovery driven by a fine-grained classification task make very restrictive assumptions on the geometric properties of the discovered parts; they should be small and compact. Although this prior is useful in some cases, in this paper we show that pre-trained transformer-based vision models, such as self-supervised DINOv2 ViT, enable the relaxation of these constraints. In particular, we find that a total variation (TV) prior, which allows for multiple connected components of any size, substantially outperforms previous work. We test our approach (the architecture is depicted in Figure 9) on three fine-grained classification benchmarks: CUB, PartImageNet and Oxford Flowers, and compare our results to previously published methods as well as a re-implementation of the state-of-the-art method PDiscoNet with a transformer-based backbone. We consistently obtain substantial improvements across the board, both on part discovery metrics and the downstream classification task, showing that the strong inductive biases in self-supervised ViT models require to rethink the geometric priors that can be used for unsupervised part discovery.

Show image description

The PDiscoFormer architecture for part discovery.

This work has been published at the European Conference on Computer Vision (ECCV) 2024 23.

8.3.3 Explaining the decisions and the functioning of a convolutional spatiotemporal land cover classifier with channel attention and redescription mining

Participants: Dino Ienco.

Collaborators: Enzo Pelous (Université Savoie Mont Blanc, France), Nicolas Méger (Université Savoie Mont Blanc, France), Abdourrahmane Atto (Université Savoie Mont Blanc, France), Hermann Courteille (University of Rennes, IRISA, France), Christophe Lin-Kwong-Chon (Université Savoie Mont Blanc, France)

Keywords: Explainable AI, convolutional neural networks, land cover classification, satellite image time series, attention, redescription mining, grouped frequent sequential patterns.

Convolutional neural networks trained with satellite image time series have demonstrated their potential in land cover classification in recent years. Nevertheless, the rationale leading to their decisions remains obscure by nature. Methods for providing relevant and simplified explanations of their decisions as well as methods for understanding their inner functioning have thus emerged. However, both kinds of methods generally work separately and no explicit connection between their findings is made available. This work presents an innovative method for refining the explanations provided by channel-based attention mechanisms. It consists in identifying correspondence rules between neuronal activation levels and the presence of spatiotemporal patterns in the input data for each channel and target class. These rules provide both class-level and instance-level explanations, as well as an explicit understanding of the network operations. They are extracted using a state-of-the-art redescription mining algorithm. Experiments on the Reunion Island Sentinel-2 dataset show that both correct and incorrect decisions can be explained using convenient spatiotemporal visualizations.

This work has been published in the ISPRS Journal of Photogrammetry and Remote Sensing (Elsevier) journal 20.

8.3.4 Understanding the sentiment associated with cultural ecosystem services using images and text from social media

Participants: Diego Marcos.

Collaborators: Ilan Havinga (Wageningen University, Netherlands), Patrick Bogaart (National Accounts Department, Statistics Netherlands), Devis Tuia (EPFL, Switzerland), Lars Hein (Wageningen University, Netherlands)

Keywords: Ecosystem services, Cultural ecosystem services, Natural language processing, Machine learning, Social media, Big data.

Social media is increasingly being employed to develop Cultural Ecosystem Services (CES) indicators. The image-sharing platform Flickr has been one of the most popular sources of data. Most large-scale studies, however, tend to only use the number of images as a proxy for CES due to the challenges associated with processing large amounts of this data but this does not fully represent the benefit generated by ecosystems in terms of the positive experiences expressed by users in the associated text. To address this gap, we have applied several Computer Vision (CV) and natural language processing (NLP) models to link CES estimates for Great Britain based on the content of images to sentiment measures using the accompanying text, and compare our results to a national, geo-referenced survey of recreational well-being in England. We have found that the aesthetic quality of the landscape and the presence of particular wildlife results in more positive sentiment. However, we have also found that different physical settings correlate with this sentiment and that sentiment is sometimes more strongly related to social activities than many natural factors. Still, we found significant associations between these CES measures, sentiment and survey data. The obtained findings illustrate that integrating sentiment analysis with CES measurement can capture some of the positive benefits associated with CES using social media. The additional detail provided by these novel techniques can help to develop more meaningful CES indicators for recreational land use management

This work has been published in the Ecosystem Services (Elsevier) journal 12.

10.1.1 Visits of international scientists

Juan Li

Status: PhD Student

Institution of origin: Beijing Normal University

Country: China

Dates: From January 2024 until November 2025

Context of the visit: Research stay

Roger Ferrod

Status: PhD Student

Institution of origin: University of Torino

Country: Italy

Dates: From June 2024 until November 2024

Context of the visit: Research stay

Valerie Zermatten

Status: PhD Student

Institution of origin: École Polytechnique Fédérale de Lausanne

Country: Switzerland

Dates: From September 2024 until December 2024

Context of the visit: Research stay

Giuseppe Guarino

Status: PhD Student

Institution of origin: University of Naples

Country: Italy

Dates: From November 2024 until March 2025

Context of the visit: Research stay

Francisco Mena Toro

Status: PhD Student

Institution of origin: DFKI, Kaiserslautern

Country: Germany

Dates: From November 2024 until January 2025

Context of the visit: Research stay

Timothée Stassin

Status: Postdoctoral Researcher

Institution of origin: German Research Centre for Geosciences, Postdam

Country: Germany

Dates: From November 2024 until December 2024

Context of the visit: Research stay

10.3.1 H2020 projects

• Eco2Adapt

  Participants: Dino Ienco.

  Eco2Adapt project on cordis.europa.eu

  Title : Ecosystem-based Adaptation and Changemaking to Shape, Protect and Maintain the Resilience of Tomorrow’s Forests

  Duration: From September 1, 2022 to August 31, 2027

  Summary: Forests can be destroyed through climatic events such as storms or drought, or attacked by pests and pathogens, leaving a devastated landscape and despairing local populations. The EU-funded eco2adapt project will develop the ecosystem-based adaptation framework derived from nature-based solutions and work in Living Labs located in climate hotspots in Europe and China. It will use an advanced Decision Theatre approach to investigate how to integrate disturbance and vulnerability into forest management by developing changemaking scenarios. Furthermore, it will create innovative technical, economic and governance mechanisms at a regional level, apply semantic technology to establish a knowledge base for hosting FAIR data, create a smartphone application (OneForest ToolBox) to enable users to add data on climate-resilient forests, and provide cutting-edge tools to monitor vulnerability and resilience.

  EVERGREEN is involved in Task 3.3 "Defining and estimating forest ecosystem services" that D. Ienco co-leads. The interest for the team is to advance spatio-temporal analysis for the analysis of forest disturbances at large scale with the exploitation of satellite derived products. We have a post-doctoral researcher for 18 months funded by this project.

10.4.1 ANR

• ANR GLOURB

  Participants: Cássio Fraga Dantas, Roberto Interdonato, Dino Ienco.

  Title: Floodplain urbanization at global scale

  Duration: From December 2022 to November, 2026

  Summary: GloUrb addresses the issue of floodplain urbanization at global scale since the 1980s based on an interdisciplinary and integrated approach. Floodplains are amongst the most threatened and vulnerable ecosystems, but vital for human society . GloUrb builds on the need for global references to support understanding of floodplain urbanization processes and their socio-ecosystem consequences (biodiversity, flood risk, urban resilience, environmental justice); explain such trends and distinguish between local and global scale drivers; inform and increase the public awareness; monitor to prevent threats and manage future changes. GloUrb will use existing local and global information, web data mining, remote sensing and innovative signal analysis techniques. We will inform people on urbanization processes to support sustainable, integrated and adaptive management, developing a global online information system with a monitoring interface showing urbanization trends and targeting potential threats.

  EVERGREEN is involved in the project for the analysis of remote sensing time series data with the aim to conceive, design and develop techniques for the downstream task of land use land cover mapping with a focus on how the process can be transferred temporally from one period of time to another period of time.

• ANR GeoReSeT

  Participants: Roberto Interdonato, Diego Marcos, Dino Ienco.

  Title: Generalized Earth Observation with Remote Sensing and Text

  Duration: From September 2023 to August, 2027

  Summary: This research proposal aims to develop a versatile foundation model for geo-spatial data that can be used for any task and with any data modality. By using location on the Earth's surface as the common link between different modalities, the model will be able to incorporate a variety of data sources, including remote sensing imagery, textual descriptions of places, and features in maps. Through self-supervised learning methods such as contrastive learning or multi-modal masked autoencoders, the model will leverage the large amounts of unlabeled geo-spatial data from these different sources to learn a better representation of any geo-spatial location and convey a semantic representation of the information.

  EVERGREEN is co-leading the project with D. Marcos as co-PI (principal investigator) of the project. The project objectives are at the core of the research activities of the team.

11.1.1 Scientific events: organisation

Local

• Cássio Fraga Dantas and Diego Marcos are part of the organization committee of Machine Learning in Montpellier (ML-MTP) seminars.

International

• Cássio Fraga Dantas , Roberto Interdonato , and Dino Ienco have organized the sixth edition of the MACLEAN (Machine Learning for Earth Obervation Data) workshop co-located with the European conference on Machine Learning and Data Mining (ECML/PKDD).
• Roberto Interdonato and Dino Ienco have organized the FRCSS24 (French Regional Conference on Complex Systems) conference in Montpellier.
• Roberto Interdonato organized the ASONAM 2024 (The 16th International Conference on Advances in Social Networks Analysis and Mining) conference at Università della Calabria, Rende (Italy).
• Diego Marcos has co-organized MVEO 2024, the Workshop on Machine Vision for Earth Observation and Environment Monitoring in conjunction with the British Machine Vision Conference (BMVC) 2024, Glasgow (United Kingdom).

11.1.2 Scientific events: selection

Chair of conference program committees

• Roberto Interdonato has been the General Chair of the FRCSS24 (French Regional Conference on Complex Systems) conference
• Roberto Interdonato has been the General Chair of the ASONAM 2024 (The 16th International Conference on Advances in Social Networks Analysis and Mining) conference

Member of the conference program committees

• Cássio Fraga Dantas : The British Machine Vision Conference (BMVC), Workshop on Machine Learning for Earth Observation (in conjunction with ECML/PKDD 2024 conference), Workshop on Machine Vision for Earth Observation and Environment Monitoring (in conjunction with BMVC 2024 conference).
• Dino Ienco : NLDB24 (Natural Language and Database), ISMIS24 (International Symposium on Methodologies for intelligent Systems), DS24 (Discovery Science), ECML/PKDD24 - Journal Track (European Conference on Machine Learning and Principle and Practice of Knowledge Discovery), PAKDD24 (Pacific Asian Conference on Knoweldge Discovery from Data), IJCAI24 (International Joint Conference on Artificial Intelligence), AAAI24 (AAAI Conference on Artificial Intelligence), EGC24 (Extraction et de Gestion des Connaissances), APIA24 (Conférence Nationale sur les Applications Pratiques de l’Intelligence Artificielle), WACV24 (IEEE/CVF Winter Conference on Applications of Computer Vision).
• Roberto Interdonato : ECML/PKDD24 (European Conference on Machine Learning and Principle and Practice of Knowledge Discovery), AAAI24 - Special Track on AI for Social Impact (AAAI Conference on Artificial Intelligence), ISMIS24 (International Symposium on Methodologies for intelligent Systems), DS24 (Discovery Science), WACV24 (IEEE/CVF Winter Conference on Applications of Computer Vision), CCS24 (Conference on Complex Systems 2024), Complex Networks 2024 (International Conference on Complex Networks and their Applications), IC2S2'24 (International Conference for Computational Social Science), CARI'2024 (African Conference on Research in Computer Science).
• Diego Marcos : ICLR24 (International Conference on Learning Representations 2024), NeurIPS24 (Conference on Neural Information Processing Systems 2024), ICML24 (International Conference on Machine Learning 2024), ECCV24 (European Conference on Computer Vision 2024), CVPR24 (Conference on Computer Vision and Pattern Recognition 2024), BMVC24 (British Machine Vision Conference 2024), WACV24 (Winter Conference on Applications of Computer Vision 2024).

11.1.3 Journal

Member of the editorial boards

• Dino Ienco : Associate Editor for the Scientific Report journal (Springer Nature), Associate Editor for the Remote Sensing journal (MDPI), member of the Editorial Board of the ISPRS Journal of Photogrammetry and Remote Sensing, Action Editor for the Machine Learning journal (Springer Nature).
• Roberto Interdonato : Associate Editor for the Applied Network Science journal (Springer Nature), member of the Editorial Board of the Frontiers in Big Data journal.
• Diego Marcos : Guest Editor in the special issue on Computer Vision for Environment Monitoring and Preservation for the ISPRS Open Journal of Photogrammetry and Remote Sensing.

Reviewer - reviewing activities

• Cássio Fraga Dantas : IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of Biomedical and Health Informatics, IEEE Transactions on Geoscience and Remote Sensing, IEEE Transactions on Signal Processing, Data Mining and Knowledge Discovery (Springer) besides reviewing activities for conferences such as the International Conference on Data Mining (ICDM) and the British Machine Vision Conference (BMVC), and workshops like Machine Vision for Earth Observation and Environment Monitoring (MVEO) and Machine Learning for Earth Observation (MACLEAN).
• Dino Ienco : IEEE Geoscience and Remote Sensing Magazine, IEEE Geoscience and Remote Sensing Letters, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Transactions on Geoscience and Remote Sensing, ISPRS Journal of Photogrammetry and Remote Sensing (Elsevier), Data Mining and Knowledge Discovery (Springer), Machine Learning Journal (Springer), Remote Sensing of Environment (Elsevier), Multimedia Tools and Applications (Elsevier), Applied Intelligence (Elsevier), International Journal of Remote Sensing (Taylor and Francis), Computers and Electronics in Agriculture (Elsevier), Neural networks (Elsevier)
• Roberto Interdonato : IEEE Transactions on Geoscience and Remote Sensing, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, PLOS One, Expert Systems with Applications (Elsevier), ISPRS Journal of Photogrammetry and Remote Sensing (Elsevier), International Journal of Applied Earth Observation and Geoinformation (Elsevier), ACM Computing Surveys, Artificial Intelligence (Springer), World Development (Elsevier), Data Mining and Knowledge Discovery (Springer), Neurocomputing (Elsevier).
• Diego Marcos : Scientific Reports (Nature Springer), IEEE Transactions on Image Processing, Transactions on Machine Learning Research, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.
• Diego Marcos and Cássio Fraga Dantas were listed as outstanding reviewers for the British Machine Vision Conference (BMVC) edition 2024, link, and Diego Marcos was listed as top reviewer for the Conference on Neural Information Processing Systems (NeurIPS) 2024, link.

11.1.4 Invited talks

• Dino Ienco : "A Data Centric perspective in Machine Learning: put the focus on valuable and reliable data", PhenoHarmonIS, Montpellier, France, May 2024
• Dino Ienco : Panel for the AI for a Sustainable Planet round table at "Special Environment & AI Day" in conjuction with ECML/PKDD 2024, Vilnius, Lithuania, September 2024.
• Diego Marcos : Computer Vision for Ecology (CV2Ecology) workshop ar the European Conference on Computer Vision (ECCV) 2024.

11.1.5 Scientific expertise

• Dino Ienco : Member of the Evaluation Committee for ANR for the ANR JCJC track, Remote Reviewer for the Belgian Defence Call for Research Proposal, Remote Reviewer for the Belgian National Fund for Scientific Research- F.R.S.-FNRS, Member of the recruitment committee for an INRAE Junior Research - Chargé de Recherche - (INRAE), Member of a COS (Comité de Sélection) for an Associate Professor position at AgroParisTech.
• Roberto Interdonato : Remote Reviewer for the Belgian STEREO IV programme for Earth observation research.
• Diego Marcos : Reviewer for ANR for the ANR JCJC track.

11.1.6 Research Administration

• Dino Ienco is participating to the activities of the Doctoral School I2S (Information Structures Systèmes) as a sub referee of the Computer Science speciality. Among the other tasks related to this activity, this involves the participation to several Doctoral advisory committee per year (around 25).
• Dino Ienco is member of the CEP (Committee of the Project Team) of the Inria Center at Université Côte d'Azur.
• Dino Ienco is member of the BCEP (Bureau of the the Project Team Committee) of the Inria Center at Université Côte d'Azur.

11.2.1 Teaching

• Cássio Fraga Dantas : Introduction to Machine Learning module (12h of practical sessions) for master students from the Energy and Environment major at EPF engineering school, Montpellier.
• Raffaele Gaetano : at MSc level, responsible for the teaching unit “Spatial imagery for the management of environmental resources”, M2 Géomatique, jointly organized by Université de Montpellier, Université Paul Valéry and AgroParistech (11.5h courses, 21.5h practical sessions). For professional training, participation to the teaching unit “Information extraction from remote sensing imagery”, Mastère SILAT, AgroParistech (2h courses, 5h practical session); “Orfeo Toolbox and Applications”, AgroParistech (1h courses, 15h practical sessions); “Copernicus for Agriculture”, AgroParistech (1h courses, 15h practical sessions).
• Diego Marcos : Advanced Data Science (3h of courses and 12h of practical sessions) and Artificial Intelligence at Université de Montpellier.

11.2.2 Supervision

• PhD defended in January 2024: Alex Levering (Wageningen University, the Netherlands), Landscape quality assessments using deep learning, co-advised by Devis Tuia (EPFL, Switzerland) and Diego Marcos .
• PhD defended in May 2024 36: Azza Abidi, Exploring deep learning approaches and image-encoded time series for land cover mapping analysis, advised by Dino Ienco , in collaboration with Imed Riadh Farah and Ali Ben Abbes (University of Manouba, Tunis, Tunisia), funded by the Tunisian Government.
• PhD defended in October 2024: Isabelle Rocamora, Artificial Intelligence approaches for geomorphological analysis based on multi-source and multi-scale satellite imagery: Application to glacial and alluvial environments, advised by Dino Ienco , in collaboration with Matthieu Ferry (Geosciences Montpellier), funded by French Ministry of Education.
• PhD in progress: Ananthu Aniraj, Explainable image classification through supervised and unsupervised part detection, co-advised by Diego Marcos , Cássio Fraga Dantas and Dino Ienco , funded by the CPJ OBTEA, since April 2023.
• PhD in progress: Camille Portes, Increased risk-based epidemio-surveillance for Xylella fastidiosa, advised by Dino Ienco , in collaboration with Edith Gabriel (BioSP - Biostatistique et Processus Spatiaux Unit), funded by École Universitaire de Recherche (EUR) Implanteus, since October 2021.
• PhD in progress: Quentin Yeche, Multi-source satellite image segmentation for the extraction of geometric landscape objects with an application to the extraction of agricultural land parcels, co-adivsed by Raffaele Gaetano and Dino Ienco , funded by ATOS (CIFRE PhD), since October 2023.
• PhD in progress: Bruno Bio Nikki, Deep learning and multi-sensor remote sensing imagery for land use and land cover mapping of agricultural systems in Northern Benin, co-adivsed by Raffaele Gaetano , Roberto Interdonato (co-supervised by Prof. Yvon C. Hountoundji, University of Parakou - Benin), funded by the EU OBSYDYA project, since October 2023.
• PhD in progress: Ron van Bree, Hybrid AI for Food Security, co-advised by Ioanis Athanasiadis (Wageningen University) and Diego Marcos , since April 2024.
• PhD in progress: Valerie Zermatten, co-advised by Devis Tuia (EPFL) and Diego Marcos .
• PhD in progress: Hugo Riffaud–De Turckheim, Enhancing foundational Model in the remote sensing field with textual informaiton, co-advised by Diego Marcos , Roberto Interdonato , funded by the ANR GEORESET, since November 2024.

11.2.3 PhD Juries

• Dino Ienco was a member of the following PhD juries in 2024 (15): Eatidal Amin Darei, Univ. of Valencia, Valencia, Spain (reviewer); Valentine Bellet, University of Toulouse 3, Toulouse, France (reviewer); Etienne Lehembre, Univ. of Caen, Cane, France (president); Luca Colomba, Politecnico of Torino, Torino, Italy (examiner); Edoardo Arnaudo, Politecnico of Torino, Torino, Italy (examiner); Zeeshan Nisar, Univ. of Strasbourg, Strasbourg, France (examiner); Jules Bourcier, Univ. of Grenoble, Grenoble, France (president); Tristan Montagnon, Univ. of Grenoble, Grenoble, France (reviewer); Salim Hafid, Univ. of Montpellier, Montpellier, France (president); Assaad Oussama Zeghina, Univ. of Strasbourg, Strasbourg, France (reviewer); Thomas Bockel, Univ. of Montpellier, Montpellier, France (president); Paul Berg, Univ. of South Brittany, Vannes, France (examiner); Basile Rousse, Univ. Paris Cité, Paris, France (examiner); Olivier Gauriau, University of Rennes, Rennes, France (examiner); Nosheen Abid, Lulea Technology University, Lulea, Sweden (examiner).
• Roberto Interdonato was a member of the following PhD juries in 2024 (2): Alessia Galdeman, Univ. of Milan, Italy (reviewer), Vincent Bridonneau, Univ. of Le Havre, France (reviewer).

11.2.4 Doctoral advisory commitee

• Cássio Fraga Dantas: was a member of 3 mid-term evaluation comittees: Edmond Menya (Univ. Montpellier), Paul Sevellec (Univ. Rennes) and Rémy Decoupes (Univ. Montpellier).
• Raffaele Gaetano was a member of 2 mid-term evaluation committee: Ananthu Aniraj (Univ. Montpellier) and Anas Zakroum (Univ. Montpellier).
• Dino Ienco was a member of 23 mid-term evaluation committees: Aziz Sfar (Univ. Montpellier), Guillaume Picaud (Univ. Montpellier), Matthieu De Castelbajac (Univ. Montpellier), Guillaume Coulaud (Univ. Montpellier), Khelian Larvet (Univ. Montpellier), Eirwan Reinders (Univ. Montpellier), Guillaume Fourret (Univ. Montpellier), Daniel Maksimov (Univ. Montpellier), Tiziano Maisonhaute (Univ. Montpellier), Theo Velletaz (Univ. Montpellier), Ensie Raoufi (Univ. Montpellier), Kawtar Zaher (Univ. Montpellier), Eugenio Dias Riberto Neto (Univ. Montpellier), Anas Zakroum (Univ. Montpellier), Chabel Daoud (Univ. Montpellier), Cesar Leblanc (Univ. Montpellier), Remy Decoupes (Univ. Montpellier), Théodore Michel Picque (Univ. Montpellier), Leonardo Moros (Univ. Montpellier), Imran Meghazi (Univ. Montpellier), Hiba Hamdi (Univ. Rennes), Yan Chastagnier (Univ. Montpellier), Hugo Ruiz (Univ. Montpellier).
• Roberto Interdonato was a member of 4 mid-term evaluation committees: Camille Portes (Univ. Avignon), Christian Wolff (Univ. Bourgogne), Edward Giamphy (Univ. La Rochelle), Théo Morel (Univ. Normandie Le Havre).
• Diego Marcos : was a member of 1 mid-term evaluation committee: Lucrezia Tosato (Univ. Paris Cité).

International journals

• 6 articleA.Azza Abidi, D.Dino Ienco, A.Ali Ben Abbes and I. R.Imed Riadh Farah. Orthrus: multi-scale land cover mapping from satellite image time series via 2D encoding and convolutional neural network.Neural Computing and ApplicationsAugust 2024HALDOIback to text
• 7 articleG.Giuseppina Andresini, A.Annalisa Appice, D.Dino Ienco and V.Vito Recchia. DIAMANTE: A data-centric semantic segmentation approach to map tree dieback induced by bark beetle infestations via satellite images.Journal of Intelligent Information SystemsSeptember 2024HALDOIback to text
• 8 articleJ.Jeremy Bourgoin, R.Roberto Interdonato, Q.Quentin Grislain, M.Matteo Zignani and S.Sabrina Gaito. Mining resources, the inconvenient truth of the “ecological” transition.World Development Perspectives35September 2024, 100615HALDOIback to text
• 9 articleD.Daniel Carcereri, P.Paola Rizzoli, L.Luca Dell’amore, J.-L.José-Luis Bueso-Bello, D.Dino Ienco and L.Lorenzo Bruzzone. Generation of country-scale canopy height maps over Gabon using deep learning and TanDEM-X InSAR data.Remote Sensing of Environment311September 2024, 114270HALDOIback to text
• 10 articleC. F.Cassio F. Dantas, R.Raffaele Gaetano, C.Claudia Paris and D.Dino Ienco. Reuse out-of-year data to enhance land cover mapping via feature disentanglement and contrastive learning.IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2024, 1-15HALDOIback to text
• 11 articleJ.Johann Desloires, D.Dino Ienco and A.Antoine Botrel. Early Season Forecasting of Corn Yield at Field Level from Multi-Source Satellite Time Series Data.Remote Sensing169April 2024, 1573HALDOIback to text
• 12 articleI.Ilan Havinga, D.Diego Marcos, P.Patrick Bogaart, D.Devis Tuia and L.Lars Hein. Understanding the sentiment associated with cultural ecosystem services using images and text from social media.Ecosystem Services65February 2024, 101581HALDOIback to text
• 13 articleD.Dino Ienco, R.Raffaele Gaetano and R.Roberto Interdonato. A constrastive semi-supervised deep learning framework for land cover classification of satellite time series with limited labels.Neurocomputing567January 2024, 127031HALDOIback to text
• 14 articleA.Alex Levering, D.Diego Marcos, N.Nathan Jacobs and D.Devis Tuia. Prompt-guided and multimodal landscape scenicness assessments with vision-language models.PLoS ONE1992024, e0307083HALDOIback to text
• 15 articleS.Saeideh Maleki, N.Nicolas Baghdadi, H.Hassan Bazzi, C. F.Cassio Fraga Dantas, D.Dino Ienco, Y.Yasser Nasrallah and S.Sami Najem. Machine Learning-Based Summer Crops Mapping Using Sentinel-1 and Sentinel-2 Images.Remote Sensing1623December 2024, 4548HALDOIback to text
• 16 articleS.Saeideh Maleki, N.Nicolas Baghdadi, S.Sami Najem, C. F.Cassio F. Dantas, H.Hassan Bazzi and D.Dino Ienco. Determining Effective Temporal Windows for Rapeseed Detection Using Sentinel-1 Time Series and Machine Learning Algorithms.Remote Sensing163January 2024, 549HALDOIback to text
• 17 articleL.Liliana Martirano, D.Dino Ienco, R.Roberto Interdonato and A.Andrea Tagarelli. DyHANE: dynamic heterogeneous attributed network embedding through experience node replay.Applied Network Science91July 2024, 30HALDOI
• 18 articleR. N.Robert N Masolele, D.Diego Marcos, V.Veronique de Sy, I.-O.Itohan-Osa Abu, J.Jan Verbesselt, J.Johannes Reiche and M.Martin Herold. Mapping the diversity of land uses following deforestation across Africa.Scientific Reports1412024, 1681HALDOIback to text
• 19 articleE.Edmond Menya, R.Roberto Interdonato, D. O.Dickson Odhiambo Owuor and M.Mathieu Roche. Explainable epidemiological thematic features for event based disease surveillance.Expert Systems with Applications250September 2024, 123894HALDOI
• 20 articleE.Enzo Pelous, N.Nicolas Méger, A.Alexandre Benoit, A.Abdourrahmane Atto, D.Dino Ienco, H.Hermann Courteille and C.Christophe Lin-Kwong-Chon. Explaining the decisions and the functioning of a convolutional spatiotemporal land cover classifier with channel attention and redescription mining.ISPRS Journal of Photogrammetry and Remote Sensing215September 2024, 256-270HALDOIback to text
• 21 articleC.Camille Portes, D.Dino Ienco, E.Eric Verdin and E.Edith Gabriel. Environmental and bioclimatic data for epidemiological analysis over French Mediterranean areas.Environmental Data Science32025, e31HALDOI
• 22 articleI.Isabelle Rocamora, D.Dino Ienco and M.Matthieu Ferry. Multi-source deep-learning approach for automatic geomorphological mapping: the case of glacial moraines.Geo-spatial Information Science16January 2024, 1-20HALDOIback to text

International peer-reviewed conferences

• 23 inproceedingsA.Ananthu Aniraj, C. F.Cassio F. Dantas, D.Dino Ienco and D.Diego Marcos. PDiscoFormer: Relaxing Part Discovery Constraints with Vision Transformers.Proceedings of the European Conference on Computer Vision (ECCV)ECCV 2024 - 18th European Conference on Computer Vision15143Lecture Notes in Computer ScienceMilano, ItalySpringer Nature SwitzerlandNovember 2025, 256-272HALDOIback to text
• 24 inproceedingsC. F.Cassio F. Dantas, R.Raffaele Gaetano and D.Dino Ienco. Semi Supervised Heterogeneous Domain Adaptation via Disentanglement and Pseudo-Labelling.Machine Learning and Knowledge Discovery in Databases. Research TrackECML PKDD 2024 - European Conference on Machine Learning and Knowledge Discovery in Databases14943Machine Learning and Knowledge Discovery in Databases. Research TrackVilnius, LithuaniaSpringerSeptember 2024, 440-456HALDOIback to text
• 25 inproceedingsS.Simona Fioretto, D.Dino Ienco, R.Roberto Interdonato and E.Elio Masciari. Integrating Predictive Process Monitoring Techniques in Smart Agriculture.Lecture notes in computer science. Lecture Notes in Artificial IntelligenceISMIS 2024 - 27th International symposium on methodologies for intelligent systemsLNCS-LNAI-14670Foundations of Intelligent Systems : 27th International Symposium, ISMIS 2024, Poitiers, France, June 17–19, 2024, ProceedingsPoitiers, FranceSpringer Nature SwitzerlandJune 2024, 306-313HALDOIback to text
• 26 inproceedingsD.Dino Ienco and C. F.Cassio Fraga Dantas. DisCoM-KD: Cross-Modal Knowledge Distillation via Disentanglement Representation and Adversarial Learning.BMVC 2024 - 35th British Machine Vision ConferenceGlasgow, United KingdomNovember 2024HALback to text
• 27 inproceedingsA.Alexis Joly, L.Lukáš Picek, S.Stefan Kahl, H.Hervé Goëau, V.Vincent Espitalier, C.Christophe Botella, B.Benjamin Deneu, D.Diego Marcos, J.Joaquim Estopinan, C.César Leblanc, T.Théo Larcher, M.Milan Šulc, M.Marek Hrúz, M.Maximilien Servajean, J.Jiří Matas, H.Hervé Glotin, R.Robert Planqué, W.-P.Willem-Pier Vellinga, H.Holger Klinck, T.Tom Denton, A.Andrew Durso, I.Ivan Eggel, P.Pierre Bonnet and H.Henning Müller. LifeCLEF 2024 Teaser: Challenges on Species Distribution Prediction and Identification.Lecture notes in computer scienceECIR 2024 - 46th European Conference on Information RetrievalLNCS-14613Advances in Information Retrieval. ECIR 2024 - 46th European Conference on Information Retrieval, ECIR 2024, Glasgow, UK, March 24–28, 2024, Proceedings, Part VIGlasgow, United KingdomSpringer Nature SwitzerlandMarch 2024, 19-27HALDOI
• 28 inproceedingsA.Alexis Joly, L.Lukáš Picek, S.Stefan Kahl, H.Hervé Goëau, V.Vincent Espitalier, C.Christophe Botella, D.Diego Marcos, J.Joaquim Estopinan, C.Cesar Leblanc, T.Théo Larcher, M.Milan Šulc, M.Marek Hrúz, M.Maximilien Servajean, H.Hervé Glotin, R.Robert Planqué, W.-P.Willem-Pier Vellinga, H.Holger Klinck, T.Tom Denton, I.Ivan Eggel, P.Pierre Bonnet and H.Henning Müller. Overview of LifeCLEF 2024: Challenges on Species Distribution Prediction and Identification.Lecture notes in computer scienceCLEF 2024 - 15th International Conference of the Cross-Language Evaluation Forum for European LanguagesLNCS-14959Experimental IR Meets Multilinguality, Multimodality, and Interaction : 15th International Conference of the CLEF Association, CLEF 2024, Grenoble, France, September 9–12, 2024, Proceedings, Part IIGrenoble, FranceSpringer Nature SwitzerlandSeptember 2024, 183-207HALDOI
• 29 inproceedingsI.Ivica Obadic, A.Alex Levering, L.Lars Pennig, D.Dario Oliveira, D.Diego Marcos and X.Xiaoxiang Zhu. Contrastive Pretraining for Visual Concept Explanations of Socioeconomic Outcomes.EarthVision 2024 workshop (in conjuction with CVPR 2024)Seatle, United States2024HAL
• 30 inproceedingsK.Konstantinos Panousis, D.Dino Ienco and D.Diego Marcos. Coarse-to-Fine Concept Bottleneck Models.NeurIPS 2024 - 38th Annual Conference on Neural Information Processing SystemsVancouver (BC), CanadaDecember 2024HALback to text
• 31 inproceedingsL.Lukáš Picek, C.Christophe Botella, M.Maximilien Servajean, C.César Leblanc, R.Rémi Palard, T.Théo Larcher, B.Benjamin Deneu, D.Diego Marcos, P.Pierre Bonnet and A.Alexis Joly. GeoPlant: Spatial Plant Species Prediction Dataset.NeurIPS ProceedingsNeurIPS 2024 - 38th Conference on Neural Information Processing SystemsTrack on Datasets and BenchmarksAdvances in Neural Information Processing Systems 37Vancouver, CanadaDecember 2024HALback to text
• 32 inproceedingsL.Lukáš Picek, C.Christophe Botella, M.Maximilien Servajean, C.César Leblanc, R.Rémi Palard, T.Théo Larcher, B.Benjamin Deneu, D.Diego Marcos, J.Joaquim Estopinan, P.Pierre Bonnet and A.Alexis Joly. Overview of GeoLifeCLEF 2024: Species Composition Prediction with High Spatial Resolution at Continental Scale using Remote Sensing.CLEF 2024 Working Notes - 25th Conference and Labs of the Evaluation Forum3740CEUR workshop proceedings186Grenoble, FranceCEURSeptember 2024, 1966-1977HALback to text
• 33 inproceedingsF.Federico Ulliana, P.Pierre Bisquert, A.Akira Charoensit, R.Renaud Colin, F.Florent Tornil and Q.Quentin Yeche. Collaborative Benchmarking Rule-Reasoners with B-Runner.RuleML+RR 2024 - 8th International Joint Conference on Rules and Reasoning15183Lecture Notes in Computer ScienceBucarest, RomaniaSeptember 2024, 21-31HALDOI

National peer-reviewed Conferences

• 34 inproceedingsF.Federico Ulliana, P.Pierre Bisquert, A.Akira Charoensit, R.Renaud Colin, F.Florent Tornil and Q.Quentin Yeche. Collaborative Benchmarking Rule-Reasoners with B-Runner.BDA 2024 - 40e Conférence sur la Gestion de Données – Principes, Technologies et ApplicationsOrleans, FranceOctober 2024HAL

Reports & preprints

• 35 reportA.Alexandre Defossez, S.Samuel Alleaume, M.Marc Montadert, D.Dino Ienco, S.Sandra Luque and J.Josselin Giffard-Carlet. Cartographie de l’habitat de reproduction du tétras-lyre (Lyrurus tetrix) dans les Alpes françaises.INRAE; OFB, Office Français de la BiodiversitéFebruary 2024HAL