The MANAO project is a joined project between CNRS, Inria, Institut d'Optique Graduate School (IOGS) and, University of Bordeaux. The project is working with two laboratories: LaBRI and LP2N.
The main event of this year is the creation of the team MANAO. This is a big step for defining a new research domain, at the frontier of optical science and computer graphics.
The second hightlight is shared with our partners of the ANR SeARCH project (see Section ). The results of our collaborative work on the Alexandria lighthouse was one of the key event of the exhibition dedicated to lighthouses at the "musée de la marine" in Paris (cf. Figure ). These results were possible thanks to the new visualization and re-assembly tools developed in our team, using data from the new acquisition process developed by our partners Archéovision and CEAlex.
This year was also very successful in terms of publications. We managed to publish 6 papers in major journals and conferences (2 at TOG/SIGGRAPH , , 2 at IEEE TVCG , and finally 2 at Computer Graphics Forum , ). They cover the whole range of our project, from material properties to geometry analysis , , shading analysis , content creation and, augmented reality . These publications have received a lot of attention as proved by the two interviews , and the 3rd best paper award at the national conference on computer graphics .
Computer generated images are ubiquitous in our everyday life. Such images are the result of a process that has seldom changed over the years: the optical phenomena due to the propagation of light in a 3D environment are simulated taking into account how light is scattered , according to shape and material characteristics of objects. The intersection of optics (for the underlying laws of physics) and computer science (for its modeling and computational efficiency aspects) provides a unique opportunity to tighten the links between these domains in order to first improve the image generation process (computer graphics, optics and virtual reality) and next to develop new acquisition and display technologies (optics, mixed reality and machine vision).
Most of the time, light, shape, and matter properties are studied, acquired, and modeled separately, relying on realistic or stylized rendering processes to combine them in order to create final pixel colors. Such modularity, inherited from classical physics, has the practical advantage of permitting to reuse the same models in various contexts. However, independent developments lead to un-optimized pipelines and difficult-to-control solutions since it is often not clear which part of the expected result is caused by which property. Indeed, the most efficient solutions are most often the ones that blur the frontiers between light, shape, and matter to lead to specialized and optimized pipelines, as in real-time applications (like Bidirectional Texture Functions and Light-Field rendering ). Keeping these three properties separated may lead to other problems. For instance:
Measured materials are too detailed to be usable in rendering systems and data reduction techniques have to be developed , , leading to an inefficient transfer between real and digital worlds;
It is currently extremely challenging (if not impossible) to directly control or manipulate the interactions between light, shape, and matter. Accurate lighting processes may create solutions that do not fulfill users' expectations;
Artists can spend hours and days in modeling highly complex surfaces whose details will not be visible due to inappropriate use of certain light sources or reflection properties.
Most traditional applications target human observers. Depending on how deep we take into account the specificity of each user, the requirement of representations, and algorithms may differ.
With the evolution of measurement and display technologies that go beyond conventional images (e.g., as illustrated in Figure , High-Dynamic Range Imaging , stereo displays or new display technologies , and physical fabrication , , ) the frontiers between real and virtual worlds are vanishing . In this context, a sensor combined with computational capabilities may also be considered as another kind of observer. Creating separate models for light, shape, and matter for such an extended range of applications and observers is often inefficient and sometimes provides unexpected results. Pertinent solutions must be able to take into account properties of the observer (human or machine) and application goals.
The main goal of the MANAO project is to study phenomena resulting from the interactions between the three components that describe light propagation and scattering in a 3D environment: light, shape, and matter. Improving knowledge about these phenomena facilitates the adaption of the developed digital, numerical, and analytic models to specific contexts. This leads to the development of new analysis tools, new representations, and new instruments for acquisition, visualization, and display.
To reach this goal, we have to first increase our understanding of the different phenomena resulting from the interactions between light, shape, and matter. For this purpose, we consider how they are captured or perceived by the final observer, taking into account the relative influence of each of the three components. Examples include but are not limited to:
The manipulation of light to reveal reflective or geometric properties , as mastered by professional photographers;
The modification of material characteristics or lighting conditions to better understand shape features, for instance to decipher archaeological artifacts;
The large influence of shape on the captured variation of shading and thus on the perception of material properties .
Based on the acquired knowledge of the influence of each of the components, we aim at developing new models that combine two or three of these components. Examples include the modeling of Bidirectional Texture Functions (BTFs) that encode in a unique representation effects of parallax, multiple light reflections, and also shadows without requiring to store separately the reflective properties and the meso-scale geometric details, or Light-Fields that are used to render 3D scenes by storing only the result of the interactions between light, shape, and matter both in complex real environments and in simulated ones.
One of the strengths of MANAO is that we are inter-connecting computer graphics and optics. On one side, the laws of physics are required to create images but may be bent to either increase performance or user's control: this is one of the key advantage of computer graphics approach. It is worth noticing that what is not possible in the real world may be possible in a digital world. However, on the other side, the introduced approximations may help to better comprehend the physical interactions of light, shape, and matter.
The MANAO project specifically aims at considering information transfer, first from the real world to the virtual world (acquisition and creation), then from computers to observers (visualization and display). For this purpose, we use a larger definition of what an observer is: it may be a human user or a physical sensor equipped with processing capabilities. Sensors and their characteristics must be taken into account in the same way as we take into account the human visual system in computer graphics. Similarly, computational capabilities may be compared to cognitive capabilities of human users. Some characteristics are common to all observers, such as the scale of observed phenomena. Some others are more specifics to a set of observers. For this purpose, we have identified two classes of applications.
Physical systems: Provided our partnership that leads to close relationships with optics, one novelty of our approach is to extend the range of possible observers to physical sensors in order to work on domains such as simulation, mixed reality, and testing. Capturing, processing, and visualizing complex data is now more and more accessible to everyone, leading to the possible convergence of real and virtual worlds through visual signals. This signal is traditionally captured by cameras. It is now possible to augment them by projecting (e.g., the infrared laser of Microsoft Kinect) and capturing (e.g., GPS localization) other signals that are outside the visible range. These supplemental information replace values traditionally extracted from standard images and thus lower down requirements in computational power . Since the captured images are the result of the interactions between light, shape, and matter, the approaches and the improved knowledge from MANAO help in designing interactive acquisition and rendering technologies that are required to merge the real and the virtual world. With the resulting unified systems (optical and digital), transfer of pertinent information is favored and inefficient conversion is likely avoided, leading to new uses in interactive computer graphics applications, like augmented reality , and computational photography .
Interactive visualization: This direction includes domains such as scientific illustration and visualization, artistic or plausible rendering. In all these cases, the observer, a human, takes part in the process, justifying once more our focus on real-time methods. When targeting average users, characteristics as well as limitations of the human visual system should be taken into account: in particular, it is known that some configurations of light, shape, and matter have masking and facilitation effects on visual perception . For specialized applications, the expertise of the final user and the constraints for 3D user interfaces lead to new uses and dedicated solutions for models and algorithms.
The MANAO project aims to study, acquire, model, and render the interactions between the three components that are light, shape, and matter from the viewpoint of an observer. As detailed more lengthily in the next section, such a work will be done using the following approach: first, we will tend to consider that these three components do not have strict frontiers when considering their impacts on the final observers; then, we will not only work in computer graphics, but also at the intersections of computer graphics and optics, exploring the mutual benefits that the two domains may provide. It is thus intrinsically a transdisciplinary project (as illustrated in Figure ) and we expect results in both domains.
Thus, the proposed team-project aims at establishing a close collaboration between computer graphics (e.g., 3D modeling, geometry processing, shading techniques, vector graphics, and GPU programming) and optics (e.g., design of optical instruments, and theories of light propagation). The following examples illustrate the strengths of such a partnership. First, in addition to simpler radiative transfer equations commonly used in computer graphics, research in the later will be based on state-of-the-art understanding of light propagation and scattering in real environments. Furthermore, research will rely on appropriate instrumentation expertise for the measurement , and display of the different phenomena. Reciprocally, optics researches may benefit from the expertise of computer graphics scientists on efficient processing to investigate interactive simulation, visualization, and design. Furthermore, new systems may be developed by unifying optical and digital processing capabilities. Currently, the scientific background of most of the team members is related to computer graphics and computer vision. A large part of their work have been focused on simulating and analyzing optical phenomena as well as in acquiring and visualizing them. Combined with the close collaboration with the optics laboratory (LP2N) and with the students issued from the “Institut d'Optique”, this background ensures that we can expect the following results from the project: the construction of a common vocabulary for tightening the collaboration between the two scientific domains and creating new research topics. By creating this context, we expect to attract (and even train) more trans-disciplinary researchers.
At the boundaries of the MANAO project lie issues in human and machine vision. We have to deal with the former whenever a human observer is taken into account. On one side, computational models of human vision are likely to guide the design of our algorithms. On the other side, the study of interactions between light, shape, and matter may shed some light on the understanding of visual perception. The same kind of connections are expected with machine vision. On the one hand, traditional computational methods for acquisition (such as photogrammetry) are going to be part of our toolbox. On the other hand, new display technologies (such as augmented reality) are likely to benefit from our integrated approach and systems. In the MANAO project we are mostly users of results from human vision. When required, some experimentation might be done in collaboration with experts from this domain, like with the European PRISM project (cf. Section ). For machine vision, provided the tight collaboration between optical and digital systems, research will be carried out inside the MANAO project.
Analysis and modeling rely on tools from applied mathematics such as differential and projective geometry, multi-scale models, frequency analysis or differential analysis , linear and non-linear approximation techniques, stochastic and deterministic integrations, and linear algebra. We not only rely on classical tools, but also investigate and adapt recent techniques (e.g., improvements in approximation techniques), focusing on their ability to run on modern hardware: the development of our own tools (such as Eigen, see Section ) is essential to control their performances and their abilities to be integrated into real-time solutions or into new instruments.
The MANAO project is organized around four research axes that cover the large range of expertise of its members and associated members. We briefly introduce these four axes in this section. More details and their inter-influences that are illustrated in the Figure will be given in the following sections.
Axis 1 is the theoretical foundation of the project. Its main goal is to increase the understanding of light, shape, and matter interactions by combining expertise from different domains: optics and human/machine vision for the analysis and computer graphics for the simulation aspect. The goal of our analyses is to identify the different layers/phenomena that compose the observed signal. In a second step, the development of physical simulations and numerical models of these identified phenomena is a way to validate the pertinence of the proposed decompositions.
In Axis 2, the final observers are mainly physical captors. Our goal is thus the development of new acquisition and display technologies that combine optical and digital processes in order to reach fast transfers between real and digital worlds, in order to increase the convergence of these two worlds.
Axes 3 and 4 focus on two aspects of computer graphics: rendering, visualization and illustration in Axis 3, and editing and modeling (content creation) in Axis 4. In these two axes, the final observers are mainly human users, either generic users or expert ones (e.g., archaeologist , computer graphics artists).
Challenge: Definition and understanding of phenomena resulting from interactions between light, shape, and matter as seen from an observer point of view.
Results: Theoretical tools and numerical models for analyzing and simulating the observed optical phenomena.
To reach the goals of the MANAO project, we need to increase our understanding of how light, shape, and matter act together in synergy and how the resulting signal is finally observed. For this purpose, we need to identify the different phenomena that may be captured by the targeted observers. This is the main objective of this research axis, and it is achieved by using three approaches: the simulation of interactions between light, shape, and matter, their analysis and the development of new numerical models. This resulting improved knowledge is a foundation for the researches done in the three other axes, and the simulation tools together with the numerical models serve the development of the joint optical/digital systems in Axis 2 and their validation.
One of the main and earliest goals in computer graphics is to faithfully reproduce the real world, focusing mainly on light transport. Compared to researchers in physics, researchers in computer graphics rely on a subset of physical laws (mostly radiative transfer and geometric optics), and their main concern is to efficiently use the limited available computational resources while developing as fast as possible algorithms. For this purpose, a large set of tools has been introduced to take a maximum benefit of hardware specificities. These tools are often dedicated to specific phenomena (e.g., direct or indirect lighting, color bleeding, shadows, caustics). An efficiency-driven approach needs such a classification of light paths in order to develop tailored strategies . For instance, starting from simple direct lighting, more complex phenomena have been progressively introduced: first diffuse indirect illumination , , then more generic inter-reflections , and volumetric scattering , . Thanks to this search for efficiency and this classification, researchers in computer graphics have developed a now recognized expertise in fast-simulation of light propagation. Based on finite elements (radiosity techniques) or on unbiased Monte Carlo integration schemes (ray-tracing, particle-tracing, ...), the resulting algorithms and their combination are now sufficiently accurate to be used-back in physical simulations. The MANAO project will continue the search for efficient and accurate simulation techniques, but extending it from computer graphics to optics. Thanks to the close collaboration with scientific researchers from optics, new phenomena beyond radiative transfer and geometric optics will be explored.
Search for algorithmic efficiency and accuracy has to be done in parallel with numerical models.
The goal of visual fidelity (generalized to accuracy from an observer point of view in the project) combined with the goal of efficiency leads to the development of alternative representations.
For instance, common classical finite-element techniques compute only basis coefficients for each discretization element: the required discretization density would be too large and to computationally expensive to obtain detailed spatial variations and thus visual fidelity.
Examples includes texture for decorrelating surface details from surface geometry and high-order wavelets for a multi-scale representation of lighting .
The numerical complexity explodes when considering directional properties of light transport such as radiance intensity (Watt per square meter and per steradian -
Before being able to simulate or to represent the different observed phenomena, we need to define and describe them. To understand the difference between an observed phenomenon and the classical light, shape, and matter decomposition, we can take the example of a highlight. Its observed shape (by a human user or a sensor) is the resulting process of the interaction of these three components, and can be simulated this way. However, this does not provide any intuitive understanding of their relative influence on the final shape: an artist will directly describe the resulting shape, and not each of the three properties. We thus want to decompose the observed signal into models for each scale that can be easily understandable, representable, and manipulable. For this purpose, we will rely on the analysis of the resulting interaction of light, shape, and matter as observed by a human or a physical sensor. We first consider this analysis from an optical point of view, trying to identify the different phenomena and their scale according to their mathematical properties (e.g., differential and frequency analysis ). Such an approach has leaded us to exhibit the influence of surfaces flows (depth and normal gradients) into lighting pattern deformation (see Figure ). For a human observer, this correspond to one recent trend in computer graphics that takes into account the human visual systems both to evaluate the results and to guide the simulations.
Challenge: Convergence of optical and digital systems to blend real and virtual worlds.
Results: Instruments to acquire real world, to display virtual world, and to make both of them interact.
For this axis, we investigate unified acquisition and display systems, that is systems which combine optical instruments with digital processing. From digital to real, we investigate new display approaches , . We consider projecting systems and surfaces , for personal use, virtual reality and augmented reality . From the real world to the digital world, we favor direct measurements of parameters for models and representations, using (new) optical systems unless digitization is required , . These resulting systems have to acquire the different phenomena described in Axis 1 and to display them, in an efficient manner , , , . By efficient, we mean that we want to shorten the path between the real world and the virtual world by increasing the data bandwidth between the real (analog) and the virtual (digital) worlds, and by reducing the latency for real-time interactions (we have to prevent unnecessary conversions, and to reduce processing time). To reach this goal, the systems have to be designed as a whole, not by a simple concatenation of optical systems and digital processes, nor by considering each component independently .
To increase data bandwidth, one solution is to parallelize more and more the physical systems. One possible solution is to multiply the number of simultaneous acquisitions (e.g., simultaneous images from multiple viewpoints , ). Similarly, increasing the number of viewpoints is a way toward the creation of full 3D displays . However, full acquisition or display of 3D real environments theoretically requires a continuous field of viewpoints, leading to huge data size. Despite the current belief that the increase of computational power will fill the missing gap, when it comes to visual or physical realism, if you double the processing power, people may want four times more accuracy, thus increasing data size as well. Furthermore, this leads to solutions that are not energy efficient and thus cannot be embedded into mobile devices. To reach the best performances, a trade-off has to be found between the amount of data required to represent accurately the reality and the amount of required processing. This trade-off may be achieved using compressive sensing. Compressive sensing is a new trend issued from the applied mathematics community that provides tools to accurately reconstruct a signal from a small set of measurements assuming that it is sparse in a transform domain (e.g., , ).
We prefer to achieve this goal by avoiding as much as possible the classical approach where acquisition is followed by a fitting step: this requires in general a large amount of measurements and the fitting itself may consume consequently too much memory and preprocessing time.
By preventing unnecessary conversion through fitting techniques, such an approach increase the speed and reduce the data transfer for acquisition but also for display.
One of the best recent examples is the work of Cossairt et al. .
The whole system is designed around a unique representation of the energy-field issued from (or leaving) a 3D object, either virtual or real: the Light-Field.
A Light-Field encodes the light emitted in any direction from any position on an object.
It is acquired thanks to a lens-array that leads to the capture of, and projection from, multiple simultaneous viewpoints.
A unique representation is used for all the steps of this system.
Lens-arrays, parallax barriers, and coded-aperture are one of the key technologies to develop such acquisition (e.g., Light-Field camera
Those are only some examples of what we investigate. We also consider the following approaches to develop new unified systems. First, similar to (and based on) the analysis goal of Axis 1, we have to take into account as much as possible the characteristics of the measurement setup. For instance, when fitting cannot be avoided, integrating them may improve both the processing efficiency and accuracy . Second, we have to integrate signals from multiple sensors (such as GPS, accelerometer, ...) to prevent some computation (e.g., ). Finally, the experience of the group in surface modeling help the design of optical surfaces for light sources or head-mounted displays.
Challenge: How to offer the most legible signal to the final observer in real-time?
Results: High-level shading primitives, expressive rendering techniques for object depiction, real-time realistic rendering algorithms
The main goal of this axis is to offer to the final observer, in this case mostly a human user, the most legible signal in real-time. Thanks to the analysis and to the decomposition in different phenomena resulting from interactions between light, shape, and matter (Axis 1), and their perception, we can use them to convey essential information in the most pertinent way. Here, the word pertinent can take various forms depending on the application.
In the context of scientific illustration and visualization, we are primarily interested in tools to convey shape or material characteristics of objects in animated 3D scenes. Expressive rendering techniques (see Figure c,d) provide means for users to depict such features with their own style. To introduce our approach, we detail it from a shape-depiction point of view, domain where we have acquired a recognized expertise. Prior work in this area mostly focused on stylization primitives to achieve line-based rendering , or stylized shading , with various levels of abstraction. A clear representation of important 3D object features remains a major challenge for better shape depiction, stylization and abstraction purposes. Most existing representations provide only local properties (e.g., curvature), and thus lack characterization of broader shape features. To overcome this limitation, we are developing higher level descriptions of shape with increased robustness to sparsity, noise, and outliers. This is achieved in close collaboration with Axis 1 by the use of higher-order local fitting methods, multi-scale analysis, and global regularization techniques. In order not to neglect the observer and the material characteristics of the objects, we couple this approach with an analysis of the appearance model. To our knowledge, this is an approach which has not been considered yet. This research direction is at the heart of the MANAO project, and has a strong connection with the analysis we plan to conduct in Axis 1. Material characteristics are always considered at the light ray level, but an understanding of higher-level primitives (like the shape of highlights and their motion) would help us to produce more legible renderings and permit novel stylizations; for instance, there is no method that is today able to create stylized renderings that follow the motion of highlights or shadows. We also believe such tools also play a fundamental role for geometry processing purposes (such as shape matching, reassembly, simplification), as well as for editing purposes as discussed in Axis 4.
In the context of real-time photo-realistic rendering ((see Figure a,b), the challenge is to compute the most plausible images with minimal effort. During the last decade, a lot of work has been devoted to design approximate but real-time rendering algorithms of complex lighting phenomena such as soft-shadows , motion blur , depth of field , reflexions, refractions, and inter-reflexions. For most of these effects it becomes harder to discover fundamentally new and faster methods. On the other hand, we believe that significant speedup can still be achieved through more clever use of massively parallel architectures of the current and upcoming hardware, and/or through more clever tuning of the current algorithms. In particular, regarding the second aspect, we remark that most of the proposed algorithms depend on several parameters which can be used to trade the speed over the quality. Significant speed-up could thus be achieved by identifying effects that would be masked or facilitated and thus devote appropriate computational resources to the rendering , . Indeed, the algorithm parameters controlling the quality vs speed are numerous without a direct mapping between their values and their effect. Moreover, their ideal values vary over space and time, and to be effective such an auto-tuning mechanism has to be extremely fast such that its cost is largely compensated by its gain. We believe that our various work on the analysis of the appearance such as in Axis 1 could be beneficial for such purpose too.
Realistic and real-time rendering is closely related to Axis 2: real-time rendering is a requirement to close the loop between real world and digital world. We have to thus develop algorithms and rendering primitives that allow the integration of the acquired data into real-time techniques. We have also to take care of that these real-time techniques have to work with new display systems. For instance, stereo, and more generally multi-view displays are based on the multiplication of simultaneous images. Brute force solutions consist in independent rendering pipeline for each viewpoint. A more energy-efficient solution would take advantages of the computation parts that may be factorized. Another example is the rendering techniques based on image processing, such as our work on augmented reality . Independent image processing for each viewpoint may disturb the feeling of depth by introducing inconsistent information in each images. Finally, more dedicated displays would require new rendering pipelines.
Challenge: Editing and modeling appearance using drawing- or sculpting-like tools through high level representations.
Results: High-level primitives and hybrid representations for appearance and shape.
During the last decade, the domain of computer graphics has exhibited tremendous improvements in image quality, both for 2D applications and 3D engines. This is mainly due to the availability of an ever increasing amount of shape details, and sophisticated appearance effects including complex lighting environments. Unfortunately, with such a growth in visual richness, even so-called vectorial representations (e.g., subdivision surfaces, Bézier curves, gradient meshes, etc.) become very dense and unmanageable for the end user who has to deal with a huge mass of control points, color labels, and other parameters. This is becoming a major challenge, with a necessity for novel representations. This Axis is thus complementary of Axis 3: the focus is the development of primitives that are easy to use for modeling and editing.
More specifically, we plan to investigate vectorial representations that would be amenable to the production of rich shapes with a minimal set of primitives and/or parameters. To this end we plan to build upon our insights on dynamic local reconstruction techniques and implicit surfaces , . When working in 3D, an interesting approach to produce detailed shapes is by means of procedural geometry generation. For instance, many natural phenomena like waves or clouds may be modeled using a combination of procedural functions. Turning such functions into triangle meshes (main rendering primitives of GPUs) is a tedious process that appears not to be necessary with an adapted vectorial shape representation where one could directly turn procedural functions into implicit geometric primitives. Since we want to prevent unnecessary conversions in the whole pipeline (here, between modeling and rendering steps), we will also consider hybrid representations mixing meshes and implicit representations. Such research has thus to be conducted while considering the associated editing tools as well as performance issues. It is indeed important to keep real-time performance (cf. Axis 2) throughout the interaction loop, from user inputs to display, via editing and rendering operations. Finally, it would be interesting to add semantic information into 2D or 3D geometric representations. Semantic geometry appears to be particularly useful for many applications such as the design of more efficient manipulation and animation tools, for automatic simplification and abstraction, or even for automatic indexing and searching. This constitutes a complementary but longer term research direction.
In the MANAO project, we want to investigate representations beyond the classical light, shape, and matter decomposition. We thus want to directly control the appearance of objects both in 2D and 3D applications (e.g., ): this is a core topic of computer graphics. When working with 2D vector graphics, digital artists must carefully set up color gradients and textures: examples range from the creation of 2D logos to the photo-realistic imitation of object materials. Classic vector primitives quickly become impractical for creating illusions of complex materials and illuminations, and as a result an increasing amount of time and skill is required. This is only for still images. For animations, vector graphics are only used to create legible appearances composed of simple lines and color gradients. There is thus a need for more complex primitives that are able to accommodate complex reflection or texture patterns, while keeping the ease of use of vector graphics. For instance, instead of drawing color gradients directly, it is more advantageous to draw flow lines that represent local surface concavities and convexities. Going through such an intermediate structure then allows to deform simple material gradients and textures in a coherent way (see Figure ), and animate them all at once. The manipulation of 3D object materials also raises important issues. Most existing material models are tailored to faithfully reproduce physical behaviors, not to be easily controllable by artists. Therefore artists learn to tweak model parameters to satisfy the needs of a particular shading appearance, which can quickly become cumbersome as the complexity of a 3D scene increases. We believe that an alternative approach is required, whereby material appearance of an object in a typical lighting environment is directly input (e.g., painted or drawn), and adapted to match a plausible material behavior. This way, artists will be able to create their own appearance (e.g., by using our shading primitives ), and replicate it to novel illumination environments and 3D models. For this purpose, we will rely on the decompositions and tools issued from Axis 1.
Participants: G. Guennebaud, D. Nuentsa
Keywords : Linear algebra
Efficient numerical computation is central to many computer science domains. In particular, in computer graphics, space transformations and local regressions involve dense linear algebra, data interpolation and differential equations require sparse linear algebra, while more advanced problems involve non-linear optimization or spectral analysis. On the one hand, solutions such as MatLab are limited to prototyping. On the other hand, optimized libraries coming from the HPC (high performance computing) world are often tedious to use and more adapted for very large problems running on clusters. Moreover, all these solutions are very slow at handling very small problems which often arise in computer graphics, vision, or robotics. As a result, researchers of these domains used to waste a lot of time at either implementing their own half cooked solution, or dealing with dozens of complex to use libraries.
The objective of Eigen is to fill this gap by proposing an easy to use, efficient, and versatile C++ mathematical template library for linear algebra and related algorithms. In particular it provides fixed and dynamic size matrices and vectors, matrix decompositions (LU, LLT, LDLT, QR, eigenvalues, etc.), sparse matrices and solvers, some basic geometry features (transformations, quaternions, axis-angles, Euler angles, hyperplanes, lines, etc.), some non-linear solvers, automatic differentiations, etc. Thanks to expression templates, Eigen provides a very powerful and easy to use API. Explicit vectorization is performed for the SSE, AltiVec and ARM NEON instruction sets, with graceful fallback to non-vectorized code. Expression templates allow to perform global expression optimizations, and to remove unnecessary temporary objects.
Eigen is already a well established library with about 20000 unique visitors of the website per month. Eigen is co-developed and maintained with a couple of other researchers and occasional contributors spread over the world. Its development started in 2008, and the last release is the 3.1 version in June 2012. Eigen is currently supported by Inria through an ADT started in January 2012.
Facts:
License: LGPL3+
We introduced a novel method for producing convincing pictures of shaded objects based entirely on 2D image operations. This approach, which we call image-based shading design, offers direct artistic control in the picture plane by deforming image primitives so that they appear to conform to specific 3D shapes. Using a differential analysis of reflected radiance, we have identified the two types of surface flows involved in the depiction of shaded objects, which are consistent with recent perceptual studies. We have also introduced two novel deformation operators that closely mimic surface flows while providing direct artistic controls in real-time.
Over the last two decades, much effort has been devoted to accurately measuring Bidirectional Reflectance Distribution Functions (BRDFs) of real-world materials and to use efficiently the resulting data for rendering. Because of their large size, it is difficult to use directly measured BRDFs for real-time applications, and fitting the most sophisticated analytical BRDF models is still a complex task. In this paper, we introduce Rational BRDF , a general-purpose and efficient representation for arbitrary BRDFs, based on Rational Functions (RFs). Using an adapted parametrization, we demonstrate how Rational BRDFs offer 1) a more compact and efficient representation using low-degree RFs, 2) an accurate fitting of measured materials with guaranteed control of the residual error, and 3) efficient importance sampling by applying the same fitting process to determine the inverse of the Cumulative Distribution Function (CDF) generated from the BRDF for use in Monte-Carlo rendering.
In augmented reality, one of the key tasks to achieve a convincing visual appearance consistency between virtual objects and video scenes is to have a coherent illumination along the whole sequence. As outdoor illumination is largely dependent on the weather, the lighting condition may change from frame to frame. We have proposed a full image-based approach for online tracking of outdoor illumination variations from videos captured with moving cameras. Our key idea is to estimate the relative intensities of sunlight and skylight via a sparse set of planar feature-points extracted from each frame. To address the inevitable feature misalignments, a set of constraints are introduced to select the most reliable ones. Exploiting the spatial and temporal coherence of illumination, the relative intensities of sunlight and skylight are finally estimated by using an optimization process. We have validated our technique on a set of real-life videos and show that the results with our estimations are visually coherent along the video sequences (cf. Figure ).
Rendering techniques are often used to convey shape in scientific illustrations. We present an analysis technique that leverages the complexity found in detailed 3D models for illustrative shading purposes. Given a smooth base surface with relief, it locates relief features (concavities, convexities and inflections) around each surface point and at multiple scales, using cubic-polynomial fitting. This object-space, per-vertex information is then used to guide a variety of shading techniques including normal enhancement, feature visualization, accessibility shading and radiance scaling. Thanks to this approach, features at multiple scales are easily combined, filtered and shaded, allowing users to explore surface relief in real-time (cf. Figure ).
The creation of free-form vector drawings as been greatly improved in recent years with techniques based on harmonic or bi-harmonic interpolation. Such methods offer the best trade-off between sparsity (keeping the number of control points small) and expressivity (achieving complex shapes and gradients). Unfortunately, the lack of a robust and versatile method to compute such images still limits their use in real-world applications. We developed a vectorial solver for the computation of free-form vector gradients based on a non-conform Finite Element Methods (FEM). Its key feature is to output a low-level vector representation suitable for very fast GPU accelerated rasterization and close-form evaluation (fig. ). This intermediate representation is hidden from the user: it is dynamically updated using FEM during drawing when control points are edited. We demonstrated novel usages of vector drawings such as instancing, layering, deformation, texture and environment mapping. Finally, we also generalized and extended the set of drawing possibilities, in particular, by showing how to locally control vector gradients. This work has been published at SIGGRAPH Asia and featured by the 3DFV website .
We created a novel approach for the multi-scale analysis of point-sampled manifolds of co-dimension 1. It is based on a variant of Moving Least Squares, whereby the evolution of a geometric descriptor at increasing scales is used to locate pertinent locations in scale-space, hence the name “Growing Least Squares (GLS)”. Compared to existing scale-space analysis methods, our approach is the first to provide a continuous solution in space and scale dimensions, without requiring any parametrization, connectivity or uniform sampling. An important implication is that we identify multiple pertinent scales for any point on a manifold, a property that had not yet been demonstrated in the literature. In practice, our approach exhibits an improved robustness to change of input, and is easily implemented in a parallel fashion on the GPU, and it can be used in a wide variety of applications. For example, the GLS can be used for the detection of similarity, according to a given scale range (see Figure ). This work has been published at the Symposium of Geometry Processing .
U. Zaragoza, U. Girona
Leader: P. Barla (MANAO)
This collaboration between regions on both French and Spanish sides of Pyrénées aims at studying material properties through their connections between physical and image space. Although the purpose of such a study is general in scope, we also target a particular application: the acquisition of material properties from a single image of an object of unknown shape, under unknown illumination.
MAVERICK, REVES
Leader: N. Holzschuch (MAVERICK)
The project ALTA aims at analyzing the light transport equations and at using the resulting representations and algorithms for more efficient computation. We target lighting simulations, either offline, high-quality simulation or interactive simulations.
IRIT
Leader: L. Barthe (IRIT)
This project aims at the definition of simple and robust tools for the modeling of 3D objects. To this end, the proposed approach consists in combining the nice mathematical properties of implicit surfaces with classical meshes.
PFT3D Archéovision (CNRS), CEAlex (USR CNRS 3134), ESTIA
Leader: P. Reuter
Cultural Heritage (CH) artifacts often come as a set of broken fragments leading to difficult 3D puzzles and sometime impossible to solve in a real world. The project's goal is to propose solutions from on-site acquisition, 3D surface reconstruction and semi-automatic virtual reassembly, taking into account the expertise of CH scientists.
IMB (UPR 5251), LABRI (UMR 5800), Inria (CENTRE BORDEAUX SUD-OUEST), I2M (NEW UMR FROM 2011), IMS (UMR 5218), CEA/DAM
Some members of MANAO participate the local initiative CPU. As it includes many thematics, from fluid mechanics computation to structure safety but also management of timetable, safety of networks and protocols, management of energy consumption, etc., numerical technology can impact a whole industrial sector. In order to address problems in the domain of certification or qualification, we want to develop numerical sciences at such a level that it can be used as a certification tool.
partners available at http://
Leader: S. Pescarin (CNR - Italy)
V-MusT.net is a new European Network of Excellence dedicated to Virtual Museums. A Virtual Museum is a personalized, immersive, interactive experience that aims to enhance our understanding of the past in museums or on the Internet. The V-Must.net network enables heritage professionals around the world to connect, collaborate and advance the development and use of virtual museums.
Giessen University, Université Paris-Descartes, Bilkent University, Université de Leuven, Delft University, Birmingham University, Philips and NextLimit
Leader: Roland Fleming (Giessen University)
The goal of this project is to better understand how the human visual system understands images in terms of meaningful components: How is shape perceived consistently in varying illumination conditions and for different materials? To which extent are humans able to guess the main illumination directions in a scene? What visual properties do we make use of to estimate the material an object is made of without touching it? Answering these questions will require inter-disciplinary research and collaborations.
Conferences: Web3D 2012, ACM SIGGRAPH Asia 2012 (posters)
The members of MANAO have also participated to the reviewing process for conferences and journals:
Journals: ACM Transaction on Graphics, Computer and Graphics, Computer Graphics Forum, The Visual Computer, Signal Image and Video Processing.
Conferences: ACM Siggraph 2012, ACM Siggraph Asia 2012, Eurographics 2013, Eurographics Symposium on Rendering 2012, Graphics interface (GI) 2012, CGI 2012, Web3D 2012
In 2012, the members of MANAO have been involved in the following responsibilities:
Inria - Evaluation committee member - Gaël Guennebaud.
Inria Bordeaux - commission for technological development (CDT) - Gaël Guennebaud.
AFIG - Best paper jury - Romain Pacanowski, Gaël Guennebaud.
The members of our team are implied in teaching computer science at University Bordeaux 1 and 2, ENSEIRB Engineering School, and IOGS. General computer science is concerned, as well as the following graphics related topics:
Master : Xavier Granier, Algorithmic and Numerical Algorithmes, 30HETD, M1, IOGS, France
Master : Xavier Granier and Romain Pacanowski, Radiometry and Colorimetry, 15HETD, M1, IOGS, France
Master : Gaël Guennebaud and Simon Boyé, High-performance 3D Graphics, 60HETD, M1, Univ. Bx 1, France.
Master : Gaël Guennebaud and Simon Boyé, Physically based Image Synthesis, 60HETD, M2, Univ. Bx 1, France.
Master : Romain Pacanowski, Simon Boyé, Object-Orientated Programming, 60HETD, M1, IOGS, France
Some members are also in charge of some fields of study:
Master : Xavier Granier, Optics and Computer Science, M1/M2 , IOGS, France.
License : Patrick Reuter, Science and Modeling, L2, Univ. Bx 2, France.
HdR : Ivo Ihrke, Computational Optical Measurement and Display: Case Studies in Plenoptic Imaging and Projection, Univ. Bordeaux 1, 7th of December, Xavier Granier
PhD : Simon Boyé, Représentation hybride pour la modélisation géométrique interactive, Univ. Bordeaux 1, 12th of December, Gaël Guennebaud and Christophe Schlick.
PhD : Jiazhou Chen, Image structures: From augmented reality to image stylization, Université Sciences et Technologies - Bordeaux I, 12th of July, Xavier Granier and Qunsheng Peng and Pascal Barla
PhD : Nicolas Mellado, Analyse des objets 3D a plusieurs échelles: application à l'assemblage de formes, Univ. Bordeaux 1, 6th of December, Patrick Reuter and Christophe Schlick.
PhD : Anthony Pajot, Toward robust and efficient physically-based rendering, Université de Toulouse, 26th of April, Examiner
PhD : Jonathan Claustres, Modèle particulaire 2D et 3D sur GPU pour plasma froid magnétisé : application à un filtre magnètique, Université de Toulouse, 17th of December, Examiner
Our results of the ANR SeARCH project (see Section ) applied to the Alexandria lighthouse were one of the key event of the exhibition dedicated to lighthouses at the "musée de la marine" in Paris.
Our work on 2D images deformations and vector graphics has lead to interviews published on 3DVF.com, the most important french online magazine on image synthesis and numeric content creation in general.
http://
Moreover, the french radio channel "France Inter" published an interview about the leader MANAOof the ANR SeARCH project.
The french daily newspaper "Le monde" and the french bi-mestrial science magazine "Science Magazine" have published rather long reports on the ANR SeARCH project where MANAOis the leader.