## Section: Scientific Foundations

### Rendering

We consider plausible rendering to be a first promising research
direction, both for images and for sound. Recent developments, such
as point rendering, image-based modeling and rendering, and work on
the simulation of aging indicate high potential for the development of
techniques which render *plausible* rather than extremely accurate
images. In particular, such approaches can result in more efficient
renderings of very complex scenes (such as outdoors environments). This is
true both for visual (image) and sound rendering. In the case of images,
such techniques are naturally related to image- or point-based methods.
It is important to note that these models are becoming more and more
important in the context of network or heterogeneous rendering, where
the traditional polygon-based approach is rapidly reaching its limits.
Another research direction of interest is realistic rendering using
simulation methods, both for images and sound. In some cases, research
in these domains has reached a certain level of maturity, for example in
the case of lighting and global illumination. For some of these domains,
we investigate the possibility of technology transfer with appropriate partners.
Nonetheless, certain aspects of these research domains, such as visibility or
high-quality sound still have numerous and interesting remaining research
challenges.

#### Plausible Rendering

##### Alternative representations for complex geometry

The key elements required to obtain visually rich simulations, are
sufficient geometric detail, textures and lighting effects. A variety of
algorithms exist to achieve these goals, for example displacement mapping,
that is the displacement of a surface by a function or a series of
functions, which are often generated stochastically. With such methods,
it is possible to generate convincing representations of terrains or
mountains, or of non-smooth objects such as rocks. Traditional approaches
used to represent such objects require a very large number of polygons,
resulting in slow rendering rates. Much more efficient rendering can
be achieved by using point or image based rendering, where the number
of elements used for display is view- or image resolution-dependent,
resulting in a significant decrease in geometric complexity.
Such approaches have very high potential. For example, if all object
can be rendered by points, it could be possible to achieve much higher
quality local illumination or shading, using more sophisticated and
expensive algorithms, since geometric complexity will be reduced. Such
novel techniques could lead to a complete replacement of polygon-based
rendering for complex scenes. A number of significant technical challenges
remain to achieve such a goal, including sampling techniques which adapt
well to shading and shadowing algorithms, the development of algorithms
and data structures which are both fast and compact, and which can allow
interactive or real-time rendering. The type of rendering platforms used,
varying from the high-performance graphics workstation all the way to the
PDA or mobile phone, is an additional consideration in the development
of these structures and algorithms.
Such approaches are clearly a suitable choice for network rendering, for
games or the modelling of certain natural object or phenomena (such as
vegetation, e.g. Figure 1 , or clouds). Other
representations merit further research, such
as image or video based rendering algorithms, or structures/algorithms
such as the "render cache" [35] , which we have developed in the
past, or even volumetric methods. We will take into account considerations
related to heterogeneous rendering platforms, network rendering, and
the appropriate choices depending on bandwith or application.
Point- or image-based representations can also lead to novel solutions
for capturing and representing real objects. By combining real images,
sampling techniques and borrowing techniques from other domains (e.g.,
computer vision, volumetric imaging, tomography etc.) we hope to develop
representations of complex natural objects which will allow rapid
rendering. Such approaches are closely related to texture synthesis and
image-based modeling. We believe that such methods will not replace 3D
(laser or range-finger) scans, but could be complementary, and represent a
simpler and lower cost alternative for certain applications (architecture,
archeology etc.).
We are also investigating methods for adding "natural appearance" to synthetic
objects. Such approaches include *weathering* or *aging* techniques,
based on physical simulations [25] ,
but also simpler methods such as accessibility maps [32] .
The approaches we intend to investigate will attempt
to both combine and simplify existing techniques, or develop novel approaches
founded on generative models based on observation of the real world.

##### Plausible audio rendering

Similar to image rendering, plausible approaches can be designed for audio rendering. For instance, the complexity of rendering high order reflections of sound waves makes current geometrical approaches inappropriate. However, such high order reflections drive our auditory perception of "reverberation" in a virtual environment and are thus a key aspect of a plausible audio rendering approach. In complex environments, such as cities, with a high geometrical complexity, hundreds or thousands of pedestrians and vehicles, the acoustic field is extremely rich. Here again, current geometrical approaches cannot be used due to the overwhelming number of sound sources to process. We study approaches for statistical modeling of sound scenes to efficiently deal with such complex environments. We also study perceptual approaches to audio rendering which can result in high efficiency rendering algorithms while preserving visual-auditory consistency if required.

#### High Quality Rendering Using Simulation

##### Non-diffuse lighting

A large body of global illumination research has concentrated on finite
element methods for the simulation of the diffuse component and stochastic
methods for the non-diffuse component. Mesh-based finite element approaches
have a number of limitations, in terms of finding appropriate meshing
strategies and form-factor calculations. Error analysis methodologies
for finite element and stochastic methods have been very different in
the past, and a unified approach would clearly be interesting. Efficient
rendering, which is a major advantage of finite element approaches,
remains an overall goal for all general global illumination research.
For certain cases, stochastic methods can be efficient for all types of
light transfers, in particular if we require a view-dependent solution. We
are also interested both in *pure* stochastic methods, which do not
use finite element techniques. Interesting future directions include
filtering for improvement of final image quality as well as beam tracing
type approaches [33] which have been recently developed for
sound research.

##### Visibility and Shadows

Visibility calculations are central to all global illumination simulations, as well as for all rendering algorithms of images and sound. We have investigated various global visibility structures, and developed robust solutions for scenes typically used in computer graphics. Such analytical data structures [29] , [28] , [27] typically have robustness or memory consumption problems which make them difficult to apply to scenes of realistic size. Our solutions to date are based on general and flexible formalisms which describe all visibility event in terms of generators (vertices and edges); this approach has been published in the past [26] . Lazy evaluation, as well as hierarchical solutions, are clearly interesting avenues of research, although are probably quite application dependent.

##### Radiosity

For purely diffuse scenes, the radiosity algorithm remains one of the most well-adapted solutions. This area has reached a certain level of maturity, and many of the remaining problems are more technology-transfer oriented We are interested in interactive or real-time renderings of global illumination simulations for very complex scenes, the "cleanup" of input data, the use of application-dependent semantic information and mixed representations and their management. Hierarchical radiosity can also be applied to sound, and the ideas used in clustering methods for lighting can be applied to sound.

##### High-quality audio rendering

Our research on high quality audio rendering is focused on developing efficient algorithms for simulations of geometrical acoustics. It is necessary to develop techniques that can deal with complex scenes, introducing efficient algorithms and data structures (for instance, beam-trees [30] [33] ), especially to model early reflections or diffractions from the objects in the environment. Validation of the algorithms is also a key aspect that is necessary in order to determine important acoustical phenomena, mandatory in order to obtain a high-quality result. Recent work by Nicolas Tsingos at Bell Labs [31] has shown that geometrical approaches can lead to high quality modeling of sound reflection and diffraction in a virtual environment (Figure 2 ). We will pursue this research further, for instance by dealing with more complex geometry (e.g., concert hall, entire building floors).

Finally, several signal processing issues remain in order to properly and efficiently restitute a 3D soundfield to the ears of the listener over a variety of systems (headphones, speakers). We would like to develop an open and general-purpose API for audio rendering applications. We already completed a preliminary version of a software library: AURELI [34] .