7.1.1 Cifre PhD Thesis Titane - DS

• Name:
  Software for Cifre PhD thesis Titane - Dassault Systemes
• Keyword:
  Geometric computing
• Functional Description:
  This software addresses the problem of simplifying two-dimensional polygonal partitions that exhibit strong regularities. Such partitions are relevant for reconstructing urban scenes in a concise way. Preserving long linear structures spanning several partition cells motivates a point-line projective duality approach in which points represent line intersections and lines possibly carry multiple points. We implement a simplification algorithm that seeks a balance between the fidelity to the input partition, the enforcement of canonical relationships between lines (orthogonality or parallelism) and a low complexity output. Our methodology alternates continuous optimization by Riemannian gradient descent with combinatorial reduction, resulting in a progressive simplification scheme. Our experiments show that preserving canonical relationships helps gracefully degrade partitions of urban scenes, and yields more concise and regularity-preserving meshes than previous mesh-based simplification approaches.
• Contact:
  Pierre Alliez
• Participants:
  Pierre Alliez, Florent Lafarge
• Partner:
  Dassault Systèmes

7.1.2 CGAL - Orthtree

• Name:
  Quadtrees, Octrees, and Orthtrees
• Keyword:
  Octree/Quadtree
• Functional Description:

  Quadtrees are tree data structures in which each node encloses a square section of space, and each internal node has exactly 4 children. Octrees are a similar data structure in 3D in which each node encloses a cubic section of space, and each internal node has exactly 8 children.

  We call the generalization of such data structure "orthtrees", as orthants are generalizations of quadrants and octants. The term "hyperoctree" can also be found in literature to name such data structures in dimensions 4 and higher.

  This package provides a general data structure Orthtree along with aliases for Quadtree and Octree. These trees can be constructed with custom point ranges and split predicates, and iterated on with various traversal methods.

• Release Contributions:
  Initial version
• Contact:
  Pierre Alliez
• Participants:
  Pierre Alliez, Jackson Campolattaro, Simon Giraudot
• Partner:
  GeometryFactory

7.1.3 CGAL - 3D Alpha Wrapping

• Keyword:
  Shrink-wrapping
• Scientific Description:

  Various tasks in geometric modeling and processing require 3D objects represented as valid surface meshes, where "valid" refers to meshes that are watertight, intersection-free, orientable, and combinatorially 2-manifold. Such representations offer well-defined notions of interior/exterior and geodesic neighborhoods.

  3D data are usually acquired through measurements followed by reconstruction, designed by humans, or generated through imperfect automated processes. As a result, they can exhibit a wide variety of defects including gaps, missing data, self-intersections, degeneracies such as zero-volume structures, and non-manifold features.

  Given the large repertoire of possible defects, many methods and data structures have been proposed to repair specific defects, usually with the goal of guaranteeing specific properties in the repaired 3D model. Reliably repairing all types of defects is notoriously difficult and is often an ill-posed problem as many valid solutions exist for a given 3D model with defects. In addition, the input model can be overly complex with unnecessary geometric details, spurious topological structures, nonessential inner components, or excessively fine discretizations. For applications such as collision avoidance, path planning, or simulation, getting an approximation of the input can be more relevant than repairing it. Approximation herein refers to an approach capable of filtering out inner structures, fine details and cavities, as well as wrapping the input within a user-defined offset margin.

  Given an input 3D geometry, we address the problem of computing a conservative approximation, where conservative means guaranteeing a strictly enclosed input. We seek unconditional robustness in the sense that the output mesh should be valid (oriented, combinatorially 2-manifold and without self-intersections), even for raw input with many defects and degeneracies. The default input is a soup of 3D triangles, but the generic interface leaves the door open to other types of finite 3D primitives.

• Functional Description:
  The algorithm proceeds by shrink-wrapping and refining a 3D Delaunay triangulation loosely bounding the input. Two user-defined parameters, alpha and offset, offer control over the maximum size of cavities where the shrink-wrapping process can enter, and the tightness of the final surface mesh to the input, respectively. Once combined, these parameters provide a means to trade fidelity to the input for complexity of the output
• Contact:
  Pierre Alliez
• Participants:
  Pierre Alliez, Cedric Steveen Portaneri, Mael Rouxel-Labbé
• Partner:
  GeometryFactory

7.1.4 CGAL - NURBS meshing

• Name:
  CGAL - Meshing NURBS surfaces via Delaunay refinement
• Keywords:
  Meshing, NURBS
• Scientific Description:
  NURBS is the dominant boundary representation (B-Rep) in CAD systems. The meshing algorithms of NURBS models available for the industrial applications are based on the meshing of individual NURBS surfaces. This process can be hampered by the inevitable trimming defects in NURBS models, leading to non-watertight surface meshes and further failing to generate volumetric meshes. In order to guarantee the generation of valid surface and volumetric meshes of NURBS models even with the presence of trimming defects, NURBS models are meshed via Delaunay refinement, based on the Delaunay oracle implemented in CGAL. In order to achieve the Delaunay refinement, the trimmed regions of a NURBS model are covered with balls. Protection balls are used to cover sharp features. The ball centres are taken as weighted points in the Delaunay refinement, so that sharp features are preserved in the mesh. The ball sizes are determined with local geometric features. Blending balls are used to cover other trimmed regions which do not need to be preserved. Inside blending balls implicit surfaces are generated with Duchon’s interpolating spline with a handful of sampling points. The intersection computation in the Delaunay refinement depends on the region where the intersection is computed. The general line/NURBS surface intersection is computed for intersections away from trimmed regions, and the line/implicit surface intersection is computed for intersections inside blending balls. The resulting mesh is a watertight volumetric mesh, satisfying user-defined size and shape criteria for facets and cells. Sharp features are preserved in the mesh adaptive to local features, and mesh elements cross smooth surface boundaries.
• Functional Description:
  Input: NURBS surface Output: isotropic tetrahedron mesh
• Publication:
  hal-03276860
• Contact:
  Pierre Alliez
• Participants:
  Pierre Alliez, Laurent Busé, Xiao Xiao, Laurent Rineau
• Partner:
  GeometryFactory

8.1.1 Finding Good Configurations of Planar Primitives in Unorganized Point Clouds

Participants: Mulin Yu, Florent Lafarge.

Planar primitives are detected on a surface triangle mesh.

We propose an algorithm for detecting planar primitives from unorganized 3D point clouds. Departing from an initial configuration, the algorithm refines both the continuous plane parameters and the discrete assignment of input points to them by seeking high fidelity, high simplicity and high completeness. Our key contribution relies upon the design of an exploration mechanism guided by a multiobjective energy function. The transitions within the large solution space are handled by five geometric operators that create, remove and modify primitives. We demonstrate the potential of our method on a variety of scenes, from organic shapes to man-made objects, and sensors, from multiview stereo to laser. We show its efficacy with respect to existing primitive fitting approaches and illustrate its applicative interest in compact mesh reconstruction, when combined with a plane assembly method (See Figure 1). This work was presented at the Conference on Computer Vision and Pattern Recognition 19.

8.1.2 Single-Shot End-to-end Road Graph Extraction

Participants: Gaétan Bahl, Florent Lafarge.

In collaboration with Mehdi Bahri (Imperial College London).

Road graph extraction from a satellite image.

Automatic road graph extraction from aerial and satellite images is a long-standing challenge. Existing algorithms are either based on pixel-level segmentation followed by vectorization, or on iterative graph construction using next move prediction. Both of these strategies suffer from severe drawbacks, in particular high computing resources and incomplete outputs. By contrast, we propose a method that directly infers the final road graph in a single pass. The key idea consists in combining a Fully Convolutional Network in charge of locating points of interest such as intersections, dead ends and turns, and a Graph Neural Network which predicts links between these points. Such a strategy is more efficient than iterative methods and allows us to streamline the training process by removing the need for generation of starting locations while keeping the training end-to-end (See Figure 2). We evaluate our method against existing works on the popular RoadTracer dataset and achieve competitive results. We also benchmark the speed of our method and show that it outperforms existing approaches. Our method opens the possibility of in-flight processing on embedded devices for applications such as real-time road network monitoring and alerts for disaster response. This work was presented at the Computer Vision and Pattern Recognition (CVPR) EarthVision Workshop 16.

8.1.3 Scanner Neural Network for On-board Segmentation of Satellite Images

Participants: Gaétan Bahl, Florent Lafarge.

Low power neural network.

Traditional Convolutional Neural Networks (CNN) for semantic segmentation of images use 2D convolution operations. While the spatial inductive bias of 2D convolutions allow CNNs to build hierarchical feature representations, they require that the whole feature maps are kept in memory until the end of the inference. This is not ideal for memory and latency-critical applications such as real-time on-board satellite image segmentation. In this work, we propose a new neural network architecture for semantic segmentation, "Scan-nerNet", based on a Recurrent 1D Convolutional architecture. Our network performs a segmentation of the input image lineby-line, and thus reduces the memory footprint and output latency. These characteristics make it ideal for on-the-fly segmentation of images on-board satellites equipped with push broom sensors such as Landsat 8, or satellites with limited compute capabilities, such as Cubesats. We perform cloud segmentation experiments on embedded hardware and show that our method offers a good compromise between accuracy, memory usage and latency. This work was presented at the International Geoscience and Remote Sensing Symposium 17.

8.1.4 Sharp Feature Consolidation from Raw 3D Point Clouds via Displacement Learning

Participants: Tong Zhao, Mulin Yu, Florent Lafarge, Pierre Alliez.

Consolidation of sharp features.

Detecting sharp features in raw 3D point clouds is an essential step in designing efficient priors in several 3D Vision applications. We present a deep learning-based approach that learns to detect and consolidate sharp feature points on raw 3D point clouds. We devised a multi-task neural network architecture that identifies points near sharp features and predicts displacement vectors toward the local sharp features (see Figure 4). The so-detected points are thus consolidated via relocation. Our approach is robust against noise by utilizing a dynamic labeling oracle during the training phase. The approach is also flexible and can be combined with several popular point-based network architectures. Our experiments demonstrate that our approach outperforms the previous work in terms of detection accuracy measured on the popular ABC dataset. We show the efficacy of the proposed approach by applying it to several 3D Vision tasks. A preprint version of this work is available on arXiv 24, currently under submission.

8.1.5 SCR: Smooth Contour Regression with Geometric Priors

Participants: Gaétan Bahl, Florent Lafarge.

In collaboration with Lionel Daniel (IRT Saint Exupéry).

Regression of smooth contours.

While object detection methods traditionally make use of pixel-level masks or bounding boxes, alternative representations such as polygons or active contours have recently emerged. Among them, methods based on the regression of Fourier or Chebyshev coefficients have shown high potential on freeform objects. By defining object shapes as polar functions, they are however limited to star-shaped domains. We address this issue with SCR: a method that captures resolution-free object contours as complex periodic functions. The method offers a good compromise between accuracy and compactness thanks to the design of efficient geometric shape priors (see Figure 5). We benchmark SCR on the popular COCO 2017 instance segmentation dataset, and show its competitiveness against existing algorithms in the field. In addition, we design a compact version of our network, which we benchmark on embedded hardware with a wide range of power targets, achieving up to real-time performance. A preprint version of this work is available on arXiv 23.

8.1.6 A kinematic-geometric model based on ankles’ depth trajectory in frontal plane for gait analysis using a single RGB-D camera

Participants: Pierre Alliez.

In collaboration with Mehran Hatamzadeh and Laurent Busé (Aromath Inria project team), Frédéric Chorin and Raphael Zory (CHU Nice) and Jean-Dominique Favreau (EKINNOX).

Analysis of human gait.

The emergence of RGB-D cameras and the development of pose estimation algorithms offer opportunities in biomechanics. However, some challenges still remain when using them for gait analysis, including noise which leads to misidentification of gait events and inaccuracy. Therefore, we present a novel kinematic-geometric model for spatio-temporal gait analysis, based on ankles’ trajectory in the frontal plane and distance-to-camera data (depth). Our approach consists of three main steps: identification of the gait pattern and modeling via parameterized curves, development of a fitting algorithm, and computation of locomotive indices. The proposed fitting algorithm applies on both ankles’ depth data simultaneously, by minimizing through numerical optimization some geometric and biomechanical error functions. For validation, 15 subjects were asked to walk inside the walkway of the OptoGait, while the OptoGait and an RGB-D camera (Microsoft Azure Kinect) were both recording. Then, the spatio-temporal parameters of both feet were computed using the OptoGait and the proposed model (see Figure 6). Validation results show that the proposed model yields good to excellent absolute statistical agreement. Our kinematic-geometric model offers several benefits: (1) It relies only on the ankles’ depth trajectory both for gait events extraction and spatio-temporal parameters’ calculation; (2) it is usable with any kind of RGB-D camera or even with 3D marker-based motion analysis systems in absence of toes’ and heels’ markers; and (3) it enables improving the results by denoising and smoothing the ankles’ depth trajectory. Hence, the proposed kinematic-geometric model facilitates the development of portable markerless systems for accurate gait analysis. This work was published in the Journal of Biomechanics 11.

8.1.7 Deep learning architectures for onboard satellite image analysis

Participants: Gaétan Bahl.

This Ph.D. thesis was advised by Florent Lafarge.

The recent advances in high-resolution Earth observation satellites and the reduction in revisit times introduced by the creation of constellations of satellites has led to the daily creation of large amounts of image data hundreds of TeraBytes per day). Simultaneously, the popularization of Deep Learning techniques allowed the development of architectures capable of extracting semantic content from images. While these algorithms usually require the use of powerful hardware, low-power AI inference accelerators have recently been developed and have the potential to be used in the next generations of satellites, thus opening the possibility of onboard analysis of satellite imagery. By extracting the information of interest from satellite images directly onboard, a substantial reduction in bandwidth, storage and memory usage can be achieved. Current and future applications, such as disaster response, precision agriculture and climate monitoring, would benefit from a lower processing latency and even real-time alerts. In this thesis, our goal is two-fold: On the one hand, we design efficient Deep Learning architectures that are able to run on low-power edge devices, such as satellites or drones, while retaining a sufficient accuracy. On the other hand, we design our algorithms while keeping in mind the importance of having a compact output that can be efficiently computed, stored, transmitted to the ground or other satellites within a constellation. First, by using depth-wise separable convolutions and convolutional recurrent neural networks, we design efficient semantic segmentation neural networks with a low number of parameters and a low memory usage. We apply these architectures to cloud and forest segmentation in satellite images. We also specifically design an architecture for cloud segmentation on the FPGA of OPS-SAT, a satellite launched by ESA in 2019, and perform onboard experiments remotely. Second, we develop an instance segmentation architecture for the regression of smooth contours based on the Fourier coefficient representation, which allows detected object shapes to be stored and transmitted efficiently. We evaluate the performance of our method on a variety of low-power computing devices. Finally, we propose a road graph extraction architecture based on a combination of fully convolutional and graph neural networks. We show that our method is significantly faster than competing methods, while retaining a good accuracy 20.

8.2.1 Simplification of 2D Polygonal Partitions via Point‐line Projective Duality, and Application to Urban Reconstruction

Participants: Florent Lafarge, Pierre Alliez.

In collaboration with Julien Vuillamy and André Lieutier (Dassault Systemes).

Simplification of a 2D polygonal partition.

We address the problem of simplifying two-dimensional polygonal partitions that exhibit strong regularities. Such partitions are relevant for reconstructing urban scenes in a concise way. Preserving long linear structures spanning several partition cells motivates a point-line projective duality approach in which points represent line intersections, and lines possibly carry multiple points. We propose a simplification algorithm that seeks a balance between the fidelity to the input partition, the enforcement of canonical relationships between lines (orthogonality or parallelism) and a low complexity output. Our methodology alternates continuous optimization by Riemannian gradient descent with combinatorial reduction, resulting in a progressive simplification scheme. Our experiments show that preserving canonical relationships helps gracefully degrade partitions of urban scenes, and yields more concise and regularity-preserving meshes than common mesh-based simplification approaches (see Figure 7). This work was published in the Computer Graphics Forum journal and presented at the Eurographics Symposium on Geometry Processing 14.

8.2.2 ConSLAM: Periodically Collected Real-World Construction Dataset for SLAM and Progress Monitoring

Participants: Kacper Pluta, Pierre Alliez.

In collaboration with Maciej Trzeciak, Yasmin Fathy and Ioannis Brilakis (University of Cambridge), and Lucio Alcalde, Stanley Chee and Antony Bromley (Laing O'Rourke).

Hand-held scanners are progressively adopted to workflows on construction sites. Yet, they suffer from accuracy problems, preventing them from deployment for demanding use cases. In this work, we present a real-world dataset collected periodically on a construction site to measure the accuracy of SLAM algorithms that mobile scanners utilize. The dataset contains time-synchronised and spatially registered images and LiDAR scans, inertial data and professional ground-truth scans. To the best of our knowledge, this is the first publicly available dataset which reflects the periodic need of scanning construction sites with the aim of accurate progress monitoring using a hand-held scanner. This work was presented at the European Conference on Computer Vision (ECCV) Workshop 18, 25.

8.2.3 Wearable Cooperative SLAM System for Real-time Indoor Localization Under Challenging Conditions

Participants: Pierre Alliez, Fernando Ireta Munoz.

This research was funded by the ANR/DGA MALIN challenge. In collaboration with Viachaslau Kachurka, David Roussel, Fabien Bonardi, Jean-Yves Didier, Hicham Hadj-Abdelkader and Samia Bouchafa (IBSIC lab Evry), and Bastien Rault and Maxime Robin (Innodura TB).

Wearable system for indoor localization.

Real-time globally consistent GPS tracking is critical for an accurate localization and is crucial for applications such as autonomous navigation or multi-robot mapping. However, under challenging environment conditions such as indoor/outdoor transitions, GPS signals are partially available or not consistent over time. In this project, a real-time tracking system for continuously locating emergency response agents in challenging conditions is presented. A cooperative localization method based on Laser-Visual-Inertial (LVI) and GPS sensors is achieved by communicating optimization events between a LiDAR-Inertial-SLAM (LI-SLAM) and Visual-Inertial-SLAM (VI-SLAM) that operate simultaneously. The estimation of the pose assisted by multiple SLAM approaches provides the GPS localization of the agent when a stand-alone GPS fails. The system has been tested under the terms of the MALIN Challenge, which aims to globally localize agents across outdoor and indoor environments under challenging conditions (such as smoked rooms, stairs, indoor/outdoor transitions, repetitive patterns, extreme lighting changes) where it is well known that a stand-alone SLAM will not be enough to maintaining the localization. The system achieved Absolute Trajectory Error of 0.48%, with a pose update rate between 15 and 20 Hz. Furthermore, the system is able to build a global consistent 3D LiDAR Map that is post-processed to create a 3D reconstruction at different level of details. This work was published in the IEEE Sensors journal 12.

8.3.1 Alpha wrapping with an offset

Participants: Cedric Portaneri, Pierre Alliez.

In collaboration with Mael Rouxel-Labbe (Geometry Factory), David Cohen-Steiner (Inria DataShape), and Michael Hemmer (formerly at Google X).

Wrapping a 3d model with an offset.

Given an input 3D geometry such as a triangle soup or a point set, we address the problem of generating a watertight and orientable surface triangle mesh that strictly encloses the input. The output mesh is obtained by greedily refining and carving a 3D Delaunay triangulation on an offset surface of the input, while carving with empty balls of radius alpha. The proposed algorithm is controlled via two user-defined parameters: alpha and offset. Alpha controls the size of cavities or holes that cannot be traversed during carving, while offset controls the distance between the vertices of the output mesh and the input (see Figure 9). Our algorithm is guaranteed to terminate and to yield a valid and strictly enclosing mesh, even for defect-laden inputs. Genericity is achieved using an abstract interface probing the input, enabling any geometry to be used, provided a few basic geometric queries can be answered. We benchmark the algorithm on large public datasets such as Thingi10k, and compare it to state-of-the-art approaches in terms of robustness, approximation, output complexity, speed, and peak memory consumption. This work was presented at the ACM SIGGRAPH Conference 13 and is available as a software component for the open source CGAL library.

8.3.2 Repairing geometric errors in 3D urban models with kinetic data structures

Participants: Mulin Yu, Florent Lafarge.

In collaboration with Sven Oesau and Bruno Hilaire (CSTB).

to do

Urban 3D models created either interactively by human operators or automatically with reconstruction algorithms often contain geometric and semantic errors. Correcting them in an automated manner is an important scientific challenge. Prior work, which traditionally relies on local analysis and heuristic-based geometric operations on mesh data structures, is typically tailored-made for specific 3D formats and urban objects. We propose a more general method to process different types of urban models without tedious parameter tuning. The key idea lies on the construction of a kinetic data structure that decomposes the 3D space into polyhedra by extending the facets of the imperfect input model. Such a data structure allows us to rebuild all the relations between the facets in an efficient and robust manner. Once built, the cells of the polyhedral partition are regrouped by semantic classes to reconstruct the corrected output model. We demonstrate the robustness and efficiency of our algorithm on a variety of real-world defect-laden models and show its competitiveness with respect to traditional mesh repairing techniques from both Building Information Modeling (BIM) and Geographic Information Systems (GIS) data. This work was published in the ISPRS Journal of Photogrammetry and Remote Sensing 15

8.3.3 Geometric simplification for radiative thermal simulation of satellites

Participants: Vincent Vadez.

This Ph.D. thesis was co-advised by Pierre Alliez and François Brunetti.

The life cycle of a satellite includes the launch phase, the positioning on the desired orbit, different maneuvers (deployment of solar panels and safety position), and finally placing the satellite on the junk orbit. The satellite gravitates in a hostile environment,exposed to thermal variations of very large amplitude, alternating sun exposure and eclipse phases. The survival of the satellite depends on the temperature of its components, the variation of which must be monitored within safety intervals. In this context, the thermal simulation of the satellite for its design is crucial to anticipate the reality of its operation. Radiative thermal simulation is essential for anticipating the generation of energy from solar and albedo radiation, and for regulating temperatures of on-board equipment. Ideal operation consists in providing appropriate cooling for components exposed to radiation, and conversely, heating of unexposed components. As an order of magnitude, the external temperature ranges from -150 to +150 degrees Celsius, and the internal electronic equipment has a safe range between -50 and +50, with a safety margin of 10 degrees. In the eclipse phase where the radiation is significantly lower, heating is provided by the energy accumulated during the exposed phase, combined with heat pipes for thermal regulation.In this thesis, the objective is to advance the knowledge on radiative thermal simulation calculation methods for satellites. To this end, two approaches are considered. The first approach consists in establishing a reference calculation of a quantity governing radiative thermal simulation: view factors. Being subject to time constraints, this method is based on a hierarchical data structure enabling progressive computation of view factors, in order to offer a satisfactory tradeoff between time dedicated to computations and desired accuracy. For the sake of accuracy, a prediction step is added to guarantee a better convergence towards the reference value.The second approach, also motivated by time constraints, aims at reducing the geometric model of a mechanical part or a spacecraft while being faithful to the numerical simulation. In order to render the decimation physics-informed, a preprocessing step relying on a sensitivity analysis is carried out. To better preserve the physical simulation, the geometric cost of a simplification operator is coupled to a factor deduced from the simulation deviation between the reference model and the reduced model 21.

8.3.4 Reconstructing and Repairing Urban Models with Kinetic Data Structures

Participants: Mulin Yu.

This Ph.D. thesis was advised by Florent Lafarge.

Compact and accurate digital 3D models of buildings are commonly used by practitioners for the visualization of existing or imaginary environments, the physical simulations or the fabrication of urban objects. Generating such ready-to-use models is however a difficult problem. When created by designers, 3D models usually contain geometric errors whose automatic correction is a scientific challenge. When created from data measurements, typically laser scans or multiview images, the accuracy and complexity of the models produced by existing reconstruction algorithms often do not reach the requirements of the practitioners. In this thesis, we address this problem by proposing two algorithms: one for repairing the geometric errors contained in urban-specific formats of 3D models, and one for reconstructing compact and accurate models from input point clouds generated from laser scanning or multiview stereo imagery. The key component of these algorithms relies upon a space-partitioning data structure able to decompose the space into polyhedral cells in a natural and efficient manner. This data structure is used to both correct geometric errors by reassembling the facets of defect-laden 3D models, and reconstruct concise 3D models from point clouds with a quality that approaches those generated by Computer-Aided-Design interactive tools. The first contribution of this thesis is an algorithm to repair different types of urban models. Prior work, which traditionally relies on local analysis and heuristic-based geometric operations on mesh data structures, is typically tailored-made for specific 3D formats and urban objects. We propose a more general method to process different types of urban models without tedious parameter tuning. The key idea lies on the construction of a kinetic data structure that decomposes the 3D space into polyhedra by extending the facets of the imperfect input model. Such a data structure allows us to re-build all the relations between the facets in an efficient and robust manner. Once built, the cells of the polyhedral partition are regrouped by semantic classes to reconstruct the corrected output model. We demonstrate the robustness and efficiency of the algorithm on a variety of real-world defect-laden models and show its competitiveness with respect to traditional mesh repairing techniques from both Building Information Modeling (BIM) and Geographic Information Systems (GIS) data. The second contribution is a reconstruction algorithm inspired by the Kinetic Shape Reconstruction method, that improves the later in different ways. In particular, we propose a data fitting technique for detecting planar primitives from unorganized 3D point clouds. Departing from an initial configuration, the technique refines both the continuous plane parameters and the discrete assignment of input points to them by seeking high fidelity, high simplicity and high completeness. The solution is found by an exploration mechanism guided by a multi-objective energy function. The transitions within the large solution space are handled by five geometric operators that create, remove and modify primitives. We demonstrate its potential, not on buildings only, but on a variety of scenes, from organic shapes to man-made objects 22.

X Development LLC (formerly Google X)

Participants: Cédric Portaneri, Pierre Alliez.

We explored an adaptive framework to repair and generate levels of detail from defect-laden 3D measurement data. Repair herein implies to convert raw 3D point sets or triangle soups into watertight solids, which are suitable for computing collision detection and situational awareness of robots. In this context, adaptive stands for the capability to select not only different levels of detail, but also the type of representation best suited to the environment of the scene and the task of the robot. This project focused on geometry processing for collision detection of robots with mechanical parts and other robots in motion.

Dorea technology

Participants: Vincent Vadez, Pierre Alliez.

In collaboration with SME Dorea Technology, our objective was to advance the knowledge on the radiative thermal simulation of satellites, via geometric model reduction. The survival of a satellite is related to the temperature of its components, the variation of which must be controlled within safety intervals. In this context, the thermal simulation of the satellite for its design is crucial to anticipate the reality of its operation. This CIFRE project started in August 2018, for a total duration of 3.5 years.

Luxcarta

Participants: Johann Lussange, Florent Lafarge.

The goal of this collaboration is to develop efficient, robust and scalable methods for extracting geometric shapes from the last generation of satellite images (at 0.3m resolution). Besides satellite images, input data will also include classification maps that identify the location of buildings and Digital Surface Models that bring rough pixel-based estimation of the urban object elevation. The geometric shapes will be first restricted to planes that typically describe well the piecewise planar geometry of buildings. The goal will be then to detect and identify each roof section of facade component of a building by a plane in the 3D space. This DGA Rapid project started in September 2020, for a total duration of 2 years.

CNES - Airbus

Participants: Marion Boyer, Florent Lafarge.

The goal of this collaboration is to design an automatic pipeline for modeling in 3D urban scenes under a CityGML LOD2 formalism using the new generation of stereoscopic satellite images, in particular the 30cm resolution images offered by Pléiades Néo satellites. Traditional methods usually start with a semantic classification step followed by a 3D reconstruction of objects composing the urban scene. Too often in traditional approaches, inaccuracies and errors from the classification phase are propagated and amplified throughout the reconstruction process without any possible subsequent correction. In contrast, our main idea consists in extracting semantic information of the urban scene and in reconstructing the geometry of objects simultaneously. This project started in October 2022, for a total duration of 3 years.

CSTB

Participants: Mulin Yu, Florent Lafarge.

This collaboration took the form of a research contract. The project investigated the design of as-automatic-as-possible algorithms for repairing and converting Building Information Modeling (BIM) models of buildings in different urban-specific CAD formats using combinatorial maps. This project started in November 2019, for a total duration of 3 years.

IRT Saint-Exupéry

Participants: Gaetan Bahl, Florent Lafarge.

This project (Ph.D. thesis) investigated low-power deep learning architectures for detecting, localizing and characterizing changes in temporal satellite images. These architectures are designed to be exploited on-board satellites with low computational resources. The project started in March 2019, for a total duration of 3 years.

Samp AI

Participants: Pierre Alliez, Armand Zampieri.

This project (Cifre Ph.D. thesis) investigates algorithms for progressive, random access compression of 3D point clouds acquired on industrial sites.

The context is as follows. Laser scans of industrial facilities contain massive amounts of 3D points which are very reliable visual representations of these facilities. These points represent the positions, color attributes and other metadata about object surfaces. Samp turns these 3D points into intelligent digital twins that can be searched and interactively visualized on heterogeneous clients, via streamed services on the cloud. The 3D point clouds are first structured and enriched with additional attributes such as semantic labels or hierarchical group identifiers. Efficient streaming motivates progressive compression of these 3D point clouds, while local search or visualization queries call for random accessible capabilities. Given an enriched 3D point cloud as input, the objective is to generate as output a compressed version with the following properties: (1) Random accessible compressed (RAC) file format that provides random access (not just sequential access) to the decompressed content, so that each compressed chunk can be decompressed independently, depending on the area in focus on the client side. (2) Progressive compression: the enriched point cloud is converted into a stream of refinements, with an optimized rate-distortion tradeoff allowing for early access to a faithful version of the content during streaming. (3) Structure-preserving: all structural and semantic information are preserved during compression, allowing for structure-aware queries on the client. (4) Lossless at the end: allows the original data to be perfectly reconstructed from the compressed data, when streaming is completed.

Geometry Factory

Participants: Pierre Alliez, Daniel Zint.

In this collaborative project with a postdoctoral fellowship funded by the plan de relance programme, we explored a novel algorithm for computing offsets of surface triangle meshes, with guarantees on topology and geometric error bounds. Reliable offsets are becoming indispensable for automated design exploration approaches that loop between simulation and shape editing. We are currently benchmarking on industrial use cases.

AI Verse

Participants: Pierre Alliez.

We are currently pursuing our collaboration with AI Verse on statistics for improving sampling quality, applied to the parameters of a generative model for 3D scenes. A postdoctoral fellowship is funded for three years (co-advised with Guillaume Charpiat from Inria Saclay and Ai Verse), but recruitment is pending. The full research topic is described next.

The AI Verse technology is devised to create infinitely random and semantically consistent 3D scenes. This creation is fast, consuming less than 4 seconds per labeled image. From these 3D scenes, the system is able to build quality synthetic images that come with rich labels that are unbiased unlike manually annotated labels. As for real data, no metric exists to evaluate the performance of the synthetic datasets to train a neural network. We thus tend to favor the photorealism of the images but such a criterion is far from being the best. The current technology provides a means to control a rich list of additional parameters (quality of lighting, trajectory and intrinsic parameters of the virtual camera, selection and placement of assets, degrees of occlusion of the objects, choice of materials, etc). Since the generation engine can modify all of these parameters at will to generate many samples, we will explore optimization methods for improving the sampling quality. Most likely, a set of samples generated randomly by the generative model does not cover well the whole space of interesting situations, because of unsuited sampling laws or of sampling realization issues in high dimensions. The main question is how to improve the quality of this generated dataset, that one would like to be close somehow to the given target dataset (consisting of examples of images that one would like to generate). For this, statistical analyses of these two datasets and of their differences are required, in order to spot possible issues such as strongly under-represented areas of the target domain. Then, sampling laws can be adjusted accordingly, possibly by optimizing their hyper-parameters, if any.

Naval group

Participants: Pierre Alliez.

In collaboration with the Acentauri project-team. The project has started but we still need to recruit a postdoctoral fellow. The context is that of the factory of the future for Naval Group, for submarines and surface ships. As input, we have a digital model (e.g., a frigate), the equipment assembly schedule and measurement data (images or Lidar). We wish to monitor the assembly site to compare the "as-designed" with the "as-built" model. The main problem is a need for coordination on the construction sites for decision making and planning optimization. We wish to enable following the progress of a real project and to check its conformity with the help of a digital twin. Currently, since we need to see on site to verify, rounds of visits are required to validate the progress as well as the equipment. These rounds are time consuming and not to mention the constraints of the site, such as the temporary absence of electricity or the numerous temporary assembly and safety equipment. The objective of the project is to automate the monitoring, with sensor intelligence to validate the work. Fixed sensor systems (e.g. cameras or Lidar) or mobile (e.g. drones) sensors will be used, with the addition of smartphones/tablets carried by operators on site.

10.1.1 Visits of international scientists

10.2.1 H2020 projects

10.3.1 3IA Côte d'Azur

Pierre Alliez holds a senior chair from 3IA Côte d'Azur (Interdisciplinary Institute for Artificial Intelligence). The topic of his chair is “3D modeling of large-scale environments for the smart territory”. In addition, he is the scientific head of the fourth research axis entitled “AI for smart and secure territories”.

10.3.2 ANR

11.1.1 Scientific events: selection

11.1.2 Journal

11.1.3 Invited talks

• Pierre Alliez gave a keynote at the ISPRS conference in Nice, on geometry processing and learning.
• Pierre Alliez gave an invited talk on geometric modeling for digital twinning, at the International Conference on Discretization in Geometry and Dynamics, in Germany.
• Florent Lafarge gave invited talks at TU Deft and at the Journées Françaises d'Informatique Graphique (JFIG 2022) on efficient Space Partitioning Data Structures for Vision problems.

11.1.4 Leadership within the scientific community

Since January 2022, Pierre Alliez is the editor in chief of the Computer Graphics Forum (CGF), in tandem with Helwig Hauser. CGF is one of the leading journals in Computer Graphics and the official journal of the Eurographics association.

Pierre Alliez is chair of the Ph.D. award committee of the Eurographics Association, since 2020.

Pierre Alliez is a member of the following steering committees: Eurographics, Eurographics Symposium on Geometry Processing and Eurographics Workshop on Graphics and Cultural Heritage.

PIerre Alliez is a member of the editorial board of the CGAL open source project.

11.1.5 Scientific expertise

• Pierre Alliez was a reviewer for the ERC and for the Eureka program. He is a scientific advisory board member for the Bézout Labex in Paris (Models and algorithms: from the discrete to the continuous).
• Florent Lafarge was a reviewer for the MESR (CIR and JEI expertises).

11.1.6 Research administration

• Pierre Alliez was Head of Science (VP for Science) of the Inria Sophia Antipolis center, in tandem with Fabien Gandon, until mid 2022. In this role, he worked with the executive team to advise and support the Institute's scientific strategy, to stimulate new research directions and the creation of project-teams, and to manage the scientific evaluation and renewal of the project-teams.
• Pierre Alliez is scientific head for the Inria-DFKI partnership since September 2022.
• Pierre Alliez is a member of the scientific committee of the 3IA Côte d'Azur.
• Florent Lafarge is a member of the NICE committee. The main actions of the NICE committee are to verify the scientific aspects of the files of postdoctoral students, to give scientific opinions on candidates for national campaigns for postdoctoral stays, delegations, secondments as well as requests for long duration invitations.
• Florent Lafarge is a member of the technical committee of the UCA Academy of Excellence "Space, Environment, Risk and Resilience".

11.2.1 Teaching

• Master: Pierre Alliez (with Xavier Descombes, Marc Antonini and Laure Blanc-Feraud), advanced machine learning, 10h, M2, Univ. Côte d'Azur, France.
• Master: Pierre Alliez, Gaétan Bahl and Guillaume Cordonnier (from Graphdeco), deep learning, 21h, M2, Univ. Côte d'Azur, France.
• Master: Florent Lafarge, Applied AI, 8h, M2, Univ. Côte d'Azur, France.
• Master: Pierre Alliez, Florent Lafarge and Angelos Mantzaflaris (from Aromath), Interpolation numérique, 60h, M1, Univ. Côte d'Azur, France.
• Master: Florent Lafarge, Mathématiques pour la géométrie, 22h, M1, EFREI, France.

11.2.2 Supervision

• PhD defended in June: Gaétan Bahl, Deep learning architectures for onboard satellite image analysis, Florent Lafarge.
• PhD defended in September: Vincent Vadez, Geometric simplification for radiative thermal simulation of satellites, Cifre thesis with Dorea, co-advised by Pierre Alliez and Francois Brunetti (Dorea).
• PhD defended in December: Mulin Yu, Reconstructing and Repairing Urban Models with Kinetic Data Structures, Florent Lafarge.
• PhD withdrawn in March: Alexandre Zoppis, City reconstruction from satellite data, Florent Lafarge.
• PhD in progress: Tong Zhao, Shape reconstruction, since November 2019, co-advised by Pierre Alliez and Laurent Busé (Aromath Inria project-team), funded by 3IA, in collaboration with Jean-Marc Thiery and Tamy Boubekeur (formerly at Telecom ParisTech, now at Adobe research), and David Cohen-Steiner (Datashape Inria project-team). Tong will defend in March 2023.
• PhD in progress: Rao Fu, Piecewise-curved reconstruction from raw 3D point clouds, thesis funded by EU project GRAPES, in collaboration with Geometry Factory, since January 2021, advised by Pierre Alliez.
• PhD in progress: Jacopo Iollo, , thesis funded by Inria challenge ROAD-AI, in collaboration with CEREMA, since January 2022, co-advised by Florence Forbes (Inria Grenoble), Christophe Heinkele (CEREMA) and Pierre Alliez.
• PhD in progress: Marion Boyer, Geometric modeling of urban scenes with LOD2 formalism from satellite images, since October 2022, advised by Florent Lafarge.
• PhD in progress: Nissim Maruani, Physics-informed geometric modeling and processing, funded by 3IA, since November 2022, co-advised by Pierre Alliez and Mathieu Desbrun (Inria Saclay).
• PhD in progress: Armand Zampieri, Compression and visibility of 3D point clouds, Cifre thesis with Samp AI, since December 2022, co-advised by Pierre Alliez and Guillaume Delarue (Samp AI).

11.2.3 Juries

• Pierre Alliez was a reviewer for the PhD of Claudio Mancinelli (Univ. Genova), Raphael Sulzer (IGN) and Theo Deprelle (Ecole des Ponts), and a PhD committee member for Florent Jousse (Epione project-team) and Gaétan Bahl.
• Pierre Alliez was a member of the "Comité de Suivi Doctoral" for the PhD thesis of Di Yang (Stars project-team) and Emilie Yu (Graphdeco project-team).
• Florent Lafarge was a member of the "Comité de Suivi Doctoral" for the PhD thesis of Cherif Abedrazek (GEOAZUR) and Mariem Mezghanni (LIX).

International journals

• 11 articleM.Mehran Hatamzadeh, L.Laurent Busé, F.Frédéric Chorin, P.Pierre Alliez, J.-D.Jean-Dominique Favreau and R.Raphael Zory. A kinematic-geometric model based on ankles’ depth trajectory in frontal plane for gait analysis using a single RGB-D camera.Journal of Biomechanics145December 2022, 111358
  HALDOIback to text
• 12 articleV.Viachaslau Kachurka, B.Bastien Rault, F. I.Fernando Israel Ireta Muñoz, D.David Roussel, F.Fabien Bonardi, J.-Y.Jean-Yves Didier, H.Hicham Hadj-Abdelkader, S.Samia Bouchafa, P.Pierre Alliez and M.Maxime Robin. WeCo-SLAM: Wearable Cooperative SLAM System for Real-time Indoor Localization Under Challenging Conditions.IEEE Sensors Journal226March 2022, 5122--5132
  HALDOIback to text
• 13 articleC.Cédric Portaneri, M.Mael Rouxel-Labbé, M.Michael Hemmer, D.David Cohen-Steiner and P.Pierre Alliez. Alpha Wrapping with an Offset.ACM Transactions on Graphics414June 2022, 1-22
  HALDOIback to textback to text
• 14 articleJ.Julien Vuillamy, A.André Lieutier, F.Florent Lafarge and P.Pierre Alliez. Simplification of 2D Polygonal Partitions via Point‐line Projective Duality, and Application to Urban Reconstruction.Computer Graphics ForumMay 2022
  HALDOIback to text
• 15 articleM.Mulin Yu, F.Florent Lafarge, S.Sven Oesau and B.Bruno Hilaire. Repairing geometric errors in 3D urban models with kinetic data structures.ISPRS Journal of Photogrammetry and Remote Sensing192October 2022
  HALDOIback to text

International peer-reviewed conferences

• 16 inproceedingsG.Gaetan Bahl, M.Mehdi Bahri and F.Florent Lafarge. Single-Shot End-to-end Road Graph Extraction.CVPR 2022 – IEEE Conference on Computer Vision and Pattern Recognition EarthVision WorkshopNew Orleans, United States2022
  HALback to text
• 17 inproceedingsF.Florent Lafarge and G.Gaétan Bahl. Scanner Neural Network for On-board Segmentation of Satellite Images.IGARSS 2022 – IEEE International Geoscience and Remote Sensing SymposiumKuala Lumpur, MalaysiaJuly 2022
  HALback to text
• 18 inproceedingsM.Maciej Trzeciak, K.Kacper Pluta, Y.Yasmin Fathy, L.Lucio Alcalde, S.Stanley Chee, A.Antony Bromley, I.Ioannis Brilakis and P.Pierre Alliez. ConSLAM: Periodically Collected Real-World Construction Dataset for SLAM and Progress Monitoring.European Conference on Computer Vision WorkshopsTel Aviv, IsraelOctober 2022
  HALback to text
• 19 inproceedingsM.Mulin Yu and F.Florent Lafarge. Finding Good Configurations of Planar Primitives in Unorganized Point Clouds.CVPR 2022 - IEEE Conference on Computer Vision and Pattern RecognitionLa Nouvelle-Orléans, United StatesJune 2022
  HALback to text

Doctoral dissertations and habilitation theses

• 20 thesisG.Gaétan Bahl. Deep learning architectures for onboard satellite image analysis.Université Côte d'AzurJune 2022
  HALback to text
• 21 thesisV.Vincent Vadez. Geometric simplification for radiative thermal simulation of satellites.Université Côte d'AzurJune 2022
  HALback to text
• 22 thesisM.Mulin Yu. Reconstructing and Repairing Urban Models with Kinetic Data Structures.Université Côte d'AzurDecember 2022
  HALback to text

Reports & preprints

• 23 miscG.Gaétan Bahl, L.Lionel Daniel and F.Florent Lafarge. SCR: Smooth Contour Regression with Geometric Priors.February 2022
  HALback to text
• 24 miscM.Mulin Yu, T.Tong Zhao, P.Pierre Alliez and F.Florent Lafarge. Sharp Feature Consolidation from Raw 3D Point Clouds via Displacement Learning.August 2022
  HALback to text

Other scientific publications

• 25 inproceedingsM.Maciej Trzeciak, K.Kacper Pluta, Y.Yasmin Fathy, L.Lucio Alcalde, S.Stanley Chee, A.Antony Bromley, I.Ioannis Brilakis and P.Pierre Alliez. ConSLAM: Periodically Collected Real-World Construction Dataset for SLAM.European Conference on Computer VisionTel Aviv, IsraelOctober 2022
  HALback to text