The MIRAGES project research activity is oriented towards the development of model-based approaches with feedback. It is the way that MIRAGES has chosen to develop Computer Vision/Computer Graphics collaboration techniques since 1984 when this approach was chosen by the members of the project. Its activity is concentrated on the manipulation of 3D objects in image sequences with an application domain oriented towards new services (related to those 3D objects) which will appear in future communication networks as well as new products for objects customization. Our research activity is carried out in three directions:

We are interested in the determination of properties such as structure, movement and photometry of 3D objects (especially human beings) in image sequences. Our approach differs from traditional methods as we mainly look for feedback model-based methods to guide the analysis. The main problems we are handling are:

3D object tracking in image sequences when the geometry of the object is known but its movement has to be determined. Both rigid, articulated and deformable objects are considered: particularly human body and face tracking.

Interactive, semi-automatic and automatic camera calibration: Here, we use 3D object models (very often, generic models) as calibration tools, in order to perform calibration. Calibration is necessary for 3D model specification.

Automatic and semi-automatic object model specification: starting from a generic model of an object(for example a human body or a face) and an image sequence, this task consists in constructing the specific model which will be used for tracking in image sequences (see above).

Inverse rendering (Computer Graphics denomination corresponding to photometric analysis in Computer Vision): we dispose of input images of a scene and from a 3D geometric model scene. The aim is to compute the photometry of the part of the scene available on those images in such a way that the digital synthesis of the scene model corresponds faithfully to the input images. The main application of this specific research is the creation of advanced tools for audio-visual productions (advertising, films,... )

Collaboration is carried out with AXIATEC (a french company interested in products customization) on face modelling and animation. The Golf-STREAM contract (RIAM funding) started in june 2002 in collaboration with the french Symah Vision production company. The task was to study complete human body 3D tracking and more specifically professional golfers from image sequences in order to allow them to improve their technique and also enrich journalists comments on television broadcasts of professional golf tournements. This task was not entirely accomplished during the period of the contract but we plan to continue this research with our own funding. We hope to extend this work to other types of sports, but also other applications such as video-surveillance, tele-conferencing, video games, while mixing real and synthetic images.

The second research direction is the creation and deformation of soft objects, particularly non linear and hysteretic ones. Presently, our research activity is limited to the 3D simulation of garments. In this research area, we are mainly studying:

The control of the mechanical properties of soft 2D structures (like textile materials) modelled by mass/spring systems.

The automatic construction of garments from 2D patterns around a human body modelled as an articulated mechanical system.

Potential applications are the realistic synthesis of 3D garments on 3D numerical mannequins applied to future e-commerce of garments for the textile industry. We put a great effort to build a strong consortium in order to submit a proposal to the ANR. We finally obtained an important funding source for this application (SIMULVET contract).

This research direction is new and related to an activity in the TERRA NUMERICA French competitivity network coordinated by Thalès. The aim of this research is to produce a 3D representation from Ile De France from aerial photos (produced by IGN) and from videos and 3D scans obtained from cars and sensing facades from the ground. The activity of Mirages will consist in using an analysis/synthesis approach to replace the raw data obtained from our partners as well as building and vegetation models in order to replace raw data by compact 3D representations suitable for new services which will be offered to the population.

A 3D model-based approach is applied to human body tracking for production and post-production applications.

We are currently developing a 3D tracking system using video cameras able to produce outdoor motion capture without the use of markers. A generic 3D human body is used as input. Then, body adjustments are made using images. After positioning the puppet on the first frame, the texture is learnt and a tracker has to retrieve the position of the puppet automatically in the rest of the image sequence.

The 3D reconstruction of a realistic face model from a small set of images is one of the greatest challenges in the field of audio-visual post-production and special effects generation. Previously manual, tracking is becoming more and more automated. We are working on faces, which are one of the most difficult entities to track because of their complex deformations. In order to perform high quality tracking, we developed a technique to model a 3D mesh of a face from a small set of images which may be taken from a video sequence. This model, first constructed in a neutral expression, is then animated for tracking purposes.

Cities are of multi-dimensions, high functional and visual complexity. They are the concentration of history, culture, economy, and ecology, etc. Therefore, modelling and visualizing 3D cities with visual reality using computers covers various disciplines and is a real challenge to all the researchers in different research domains. The potential applications for creating 3D cities range from research and educational purposes like urban planning and global navigation to entertainment like tourism and 3D computer games. Buildings, as one of the main objects in cities, differ from one another in many aspects like motif, style, dimensions, etc.: facades can to a great extent reflect most aspects of one building. Therefore, our research will mainly focus on 3D facade reconstruction based on multi-source information (including airial images, ground images, architectural knowledge, etc.). Our goal is to create a real time 3D facade reconstruction system with a user friendly interface. Three major challenges will be tackled:

our system will face not only expert users but also non-expert users
**friendly interactivity constraints**: is there a way to let all of our future users express their ideas in 3D? This is a demanding task to provide our various potential users with
easy-to-use interface that require no/not much learning while keeping high quality results.

real-time and realism are two critical factors for 3D object modeling and visualization. Here, architectural knowledge, like floor height and width should be adopted in order to get plausible results, and meanwhile considering the application of our system, memory consuming and time consuming should be reduced.

robustness is another important factor in facade reconstruction from multi-source information. Since the quality and complexity of the input information cannot perfectly meet our requirements (usually there are heavy image noise, measurement errors, small irregular elements, etc.), how to get stable and precise reconstruction result will be a tough nut to crack.

Up to now, there is much work in procedural architectural modeling, and Pascal Müller, et al. did a lot of work in this domain. Currently, we are firstly trying to implement the algorithms proposed by Müller in the SIGGRAPH07 paper "Image-based procedural modeling of facades". In this paper, a 3D façade model with high visual quality and the related shape grammar rules are derived from a single rectified façade image. The basic philosophy of Müller's work is to mimic the procedural modeling pipeline from computer graphics to subdivide a facade texture in a top-down image analysis manner into meaningful elements such as floor, tiles, windows, and doors based on architectural knowledge. The authors introduce mutual information to extract the irreducible facade which consists of all the basic elements existing in the original facade image. From the hierarchical subdivision of the facade image, a set of shape grammar rules, which can be somehow modified to generate different facades from the original one, could be automatically derived.

Usual geometric modeling is not sufficient to simulate cloth: an in-depth knowledge of fabrics behaviour is required. Relations between stress and strain have to be dealt with, that are not defined by ordinary differential equations. Since analytical solutions are not available, numerical models are used; we have chosen mass-spring systems.

These models have to be fed with correct parameters. The Kawabata Evaluation System is the de-facto industry standard for fabrics measurements (limited to warp/weft textile materials): it operates typical deformations upon cloth samples and evaluates the resulting stress, which provides digital data. Our textile models are now able to simulate precisely Kawabata curves of real textile for tension, shear and bending.

Once a numerical model of cloth is stated, we have to produce complete garments. The technique we use directly comes from the textile industry. We use the same 2D patterns as those really manipulated by garment designers. This incorporates sewing information. We have to construct the garment automatically from this data. It appears to be a very complex problem, not solved in the literature (and even almost never tackled !). Our goal is then to compute the evolution of this complete mass/spring system with respect to time; it relies on the integration of the fundamental equation of dynamics. Several families of algorithms exist, adequation to the underlying problem of which vary both in terms of stability and computating time. Checking the validity of our approach is also a very strong challenge that we began to undertake.

Handling contact phenomenon between pieces of cloth, and the characters that wear them is the other big challenge: large amounts of data have to be processed to accurately detect possible collisions, and a realistic model has to be used once an actual collision has been detected.

Emilion is the cloth simulation system that was developed first by J. Denise during his Ph.D. Then, it was extended by D. Reversat to allow draping on animated characters (See ). An automatic pre-positioning technique by T. Le Thanh and A. Gagalowicz (See ) was recently added. Thanks to these new results garment simulation becomes really affordable. In order to improve the realism of the simulation, we are now adding buckling effects which should produce nicer folds.

This software computes the evolution along time of garments that are represented by mass-spring systems. Both explicit and implicit integration schemes are available. Hysteretic behavior is taken into account in order to reach a high degree of realism. Contact and inverse kinematics are introduced through the constraints handling method initiated by Baraff. An attractive feature is the spatial partitioning scheme that makes collision detection a low-cost task, even with large scenes (less than 15 % computation time in typical simulations).

Emilion was written in C++ and makes heavy use of Generic Programming Concepts (which most famous incarnation is STL, the Standard Template Library). Tcl was chosen as its scripting language to ease both its feeding with external data and the simulation of Kawabata experiments.

We are targeting the problem of how to track in 3D a human body captured with several video cameras in order to reduce the cost of post-production by making it
*almost*automatic. The first problem is to produce a realistic 3D model of the person to track. The idea is to propose a reference position, called a stanza position, to be performed off
line by the person to track. We take pictures of the person in that position and then construct the 3D shape of the person in that position from the set of images. Then we incorporate a 3D
skeleton automatically inside this reference shape.

A model-based analysis-by-synthesis framework for human motion tracking was proposed (see Fig. ). Prior to this framework a textured 3D puppet model was built and will be used for tracking the human motion. At this point of time, rigid skin deformation is used. Our method does not require image background segmentation or skeletonisation, which are very sensitive to noise.

Once the 3D puppet is pre-positioned, the computing system will synthesize the necessary motion by manipulating the skeleton of the model puppet with the necessary degrees of freedom. The skeleton in-turn drives the skin deformation. Finally, the deformation synthesizes an image that is compared with a real input image. The differences between the synthesized and the real image are fed back as the process iterates until it finds the best result (minimum error). This human movement tracking framework uses a simulated annealing algorithm. Rigid skin deformation function is used. For testing, we concentrate on tracking the motion of arms since they are highly articulated and appear qualitatively small in the images.

We have tested our framework under different scenarios. Here, we show one of our results in cluttered environment and change background. The "white" and "grey" labels show the superimposed tracking of the upperarm and forearm (See Fig. and ). The top 5 images show the results from the camera viewpoints that we used for tracking. The bottom 5 images show the results on another camera from the same video scene that we used for verification and visualization only.

The implementation of this system uses C++. The texture synthesis is done by rendering using the Wildmagic 3D rendering engine that utilizes the GPU hardware acceleration using OpenGL.

Our approach to 3D face modeling consists in using a deformable generic model which, given a set of images, is adapted to the specific one in two steps: first step consists of the adaptation of certain predefined characteristic points which is done in parallel with the per-view camera calibration; the second one consists of the adaptation of the contours in all the views. Shortly, the process in shown in Figure .

Once the specific 3D model has been constructed, it is possible to change its facial expression by manipulating special Bezier curves ("muscles") which are defined on the mesh. Also a big variety of predefined expressions has been created based on this curves which can be accessed through sliders (see Figure ).

Numerous enchancements were made to the interface part of the application in order to simplify modelisation process for the user and facilitate the development of the expression part of the algorithm. In order to increase accuracy in case of low-quality images or only one photo available, the user can adjust geometry in specific zones by manipulating 3D curves, which are similar to the expression ones. It allows to adapt very precisely the most important regions, such as mouth and eyes shape. The algorithmic model of the muscles itself became much more flexible, allowing to add new chracterisctic points and curves in an interactive manner. Construction of the new curves on the mesh is performed using gradient descent minimization algorithm. Basic algorithm being the same, we've also made several improvements to the modelisation process. That is:

Automatic detection of the central caracteristic points and their special treatement in antiparallel views.

Automatic assignment of priority for some views when reconstructing certain characteristic points.

Minimization of the mesh deformation after calibration/reconstruction process by automatic adjustment of the positions of the cameras.

"Releasing" of the characteristic points after calibration/reconstruction process during silhouette adaptation according to their average error in all the cameras.

Creation of the model backup during modelisation process and possibility to return to the previous steps at any moment.

Special treatement of the "muscles" of the mouth and eyes so that the curves from the upper and the lower parts, being close to each other in 3D space, didn't influence each other during RBF interpolation.

Implementation of the basic texturing algorithm from all the photos available.

In addition to the system of muscles, another animation mechanism based on the MPEG-4 specification was introduced. It proposes a different approach for the creation of expressions allowing to define deformation only within special zones of the face by activating one of the 68 pre-defined facial animation points (see Figure ).

Along with animation we decided to use the basic idea of this technique for the face modeling as well in order to refine shapes in the regions that cannot be reconstructed from images as well as some special features as wrinkles (see Figure ).

At last, approximate error computation between two surfaces was implemented which allowed to estimate the difference in geometry between the original model and the adapted one and locate the zone where additional changes are necessary (if the number of available images is not enough to capture the whole geometry). Example of such an error map is presented in Figure . Here the red color shows the negative difference between the reconstructed model and the original one, and green color - the positive one. The brighter are the colors, the greater are the errors.

Our tracking algorithm is based on the use of the 3D model of the person for detecting the location and the facial expression of this person on a number of successive images. In initialization step the 3D model is textured by the first image of the sequence. To do this the model is manually prepositioned and given the proper facial expression in order to fit the face on the first image. After that the tracking process is launched automatically for every image of the sequence. It is based on the minimization of the error between the projected textured model and the face on the image. For the moment error is being minimized by the simulated annealing method for the expression part of tracking and by gradient descent algorithm for the rigid one. Expression part is now rather slow since RBF deformations have to be performed on every optimization step. So we plan to incorporate the MPEG-4 facial animation standart mentioned above which is based on points and not on curves and provides local deformation.

The proposed procedural pipeline by Müller, et al. consists of four stages: 1) facade structure detection; 2) tile refinement; 3) element recognition; 4) editing and shape grammar rule extraction.

The input is a single rectified image from ground-based or aerial imagery, and the output is 3D reconstructed facade and a set of shape grammar rules which can be reused to create a large variety of facades.

facade structure detection: by using mutual information and architectural floor heights and widths, similar regions in the input image(Fig. ) is firstly detected, and then a data structure named irreducible facade (IF) (Fig. ) is created to encode the symmetrical information about floors and tiles. This step is quite time consuming and sensitive to image noise, and the content in IF is highly dependent on a similarity threshold. Therefore, we are now considering adopting some other image processing method like morphology analysis to detect the feature areas (pattern). In order to detail the facade into windows and doors (even window and door frames), the IF is further analyzed to get the optimal splitting lines between neighboring tiles (Fig. ) and the subdivision on the IF is then symmetrically unpacked onto the original facade (Fig. ).

tile refinement: at this stage, the detected tiles from the first stage are subdivided into smaller regions like windows and doors in a recursively selection way. Considering the repetitions embedded in IF and the image noise, local and global synchronization is adopted. Here, the factors related to the minimum region size and the noise level have great influence on the processing time and the final results. We refine the tile until the window frame by introducing the frame width (Fig. ).

element recognition: at this stage, based on mutual information, the detected architectural elements are matched with the objects in one 3D element library created from Autodesk's Maya MEL scripts. Here, we adopted the similar idea at the second stage to get the interior structure of the windows and tiles (Fig. ). This idea is intuitive and meanwhile effective for reconstructing rectangle-shaped windows and doors. But in Paris there are a lot of buildings with other different and complicated window and door structures other than rectangles, therefore we will consider employing other detection and representation methods like morphology and NURBS to detect and describe the structures of windows and doors of the buildings in Paris.

editing and shape grammar rule extraction: since the previous procedures are hierarchical and semantic, which share the same features with shape grammar rules, the shape-grammar based architectural modeling method is integrated into the procedural pipeline after simple editing operations. The whole procedure is ended up by producing a complete rule set that describes the segmented facade and can be applied to differently dimensioned facades as illustrated in Fig. .

Garment simulation seems to be a completely different research domain which has nothing to do with the other activities of MIRAGES. In fact, they are intimately related. As it has been said previously, human body modeling and tracking are the core of our research activity. Usually, people that we have to track are dressed, so that what we have to track is more a garment than a human body itself. As we use model-based approaches, we need to have a garment model at our disposal. As it is a very complex domain, it became a research domain in itself in MIRAGES. Research on cloth animation and computer generated garments is a field of major interest among both Computer Graphics and Textile research communities. The most common approach is to model cloth objects using mass-spring systems and to resolve Newton's equation governing the system motion using implicit integration methods. Unfortunately, results usually lack of realism. So the main effort carried out was to improve realism which is both necessary for its use in the textile industry as well as for a precise 3D human body tracking.

We want to develop a virtual dressing system which can be used easily by an end user who can try garments using some very simple actions. Our previous work gave a fast 3D reconstruction of 2D garments around 3D mannequin, then an energy minimization was performed to reduce the deformation of the garments, in order to produce the first position of the garments within a very short time.

Many enhancements of the 3D reconstruction of garments were performed.

First of all, we recover the 3D mannequin given by client and determine its coordinate system and its symmetry plane (front and back) by using its inertia matrix. Then we "stick” the two
client "figurines” on one another. These stuck figurines are then placed in the
XOYplane of the mannequin so that their forms coincide exactly with the silhouettes (front and back) of the client. The result of this placement of the figurines in the
XOYplane is displayed on figure
.

Then we sample each figurine according to a grid of horizontal and vertical lines which are similar to the sampling of the 2D pattern (see figure ).

We first reconstruct in 3D only the points whose projections in the
XOYplane are inside the silhouette of the client. We thus rebuild the surface of garments in two halves which fit exactly on the
XOYplane.

To perform the task, for each point of the grids whose projection in the
XOYplane is inside the client silhouette, we reposition it by computing its
z-coordinate so that the point is out of the surface of the 3D mannequin at a distance
d_{z}=
const(see the figure
).

To reconstruct the points whose projection is not inside the client silhouette, we reposition them thanks to the 3D silhouette of client. This silhouette is reconstructed and approximated by a contour line whose projection is always on the 2D silhouette (see figure ).

Then we used an elliptic function to interpolate the surface of garments whose projection is between the client silhouette and the extreme of the figurines.

The surfaces of garment (after smoothing) are shown in figure .

The masses/spring system of 2D patterns is finally positioned on the surfaces of garments. The technique used consists in sampling each "polygon” of each 2D figurine according to a vertical and horizontal grid. We determine then the image of this grid on the real 2D pattern with a method of 2D adapted deformation. The positions of the masses on the surfaces of garments are then given without difficulty by using the correspondence between the two grids.

The behavior of textile has drawn the attention of researchers from both manufacture domain and Computer Graphics domain for long time. A realistic simulation of textile requires sufficient knowledge of textile's response to both compressive force and stretching force. The KES (Kawabata Evaluation System) is widely used as the industry standard for textile measurement. It measures typical deformations on cloth samples and evaluates the resulting forces, which provide us with the proper parameters for our textile simulation system. However, it is only effective for stretching forces. An accurate model for buckling, which is caused by compressive force, is still not available due to its unstable property. Thus an in-depth study of buckling behavior of textile is very necessary. We plan to embed it into Emilion for more realistic simulation results.

For rigid materials, buckling usually means a 'failure'. It first presents strong resistance to compressive forces, but after the force reaches a critical load, an unrestorable deformation will happen. The rigid object is not able to restore its original shape even if the force is released.

For textile, its response to compressive force is very different from rigid materials. When a compressive force is applied, the textile shows weak resistance. It easily passes the unstable post-buckling state, and gets into a new stable state. It does not break or collapse, but forms wrinkles instead. Thus, we can assume that in any time step of the simulation, the textile is not in the unstable post-buckling state.

Choi and Ko have modified the mass-spring model, which has been used for textile simulation for a long time, to simulate buckling behavior without import of a fictitious damping force.
Besides the springs between adjacent masses, they add springs between
P(
i,
j)and
P(
i±2,
j),
P(
i,
j±2),
P(
i±2,
j±2)(see figure
). They call this kind of connections
*Interlaced*connections, which are responsible for buckling created by compressive and bending forces.

For the connections between adjacent particles, they are represented by a simple linear spring model. The energy between adjacent particles
iand
jis

where
x_{ij}=
x_{j}-
x_{i},
Lis the natural length of the spring, and
k_{s}is the spring constant. But this energy function only accounts for tension. It cannot be applied to buckling.

They simplify this kind of connection to a beam structure. Before buckling, the shape is a straight beam of length
L. Compressive loads are added at the pinned ends of the beam as shown in Figure
. The post-buckling equilibrium is approximated to be a circular arc with constant arc length.

Based on the assumption in section
, they use the moment equilibrium equation (under the pinned ends condition)
k_{b}+
Py= 0to predict the post-buckling equilibrium shape, where
k_{b}is the flextural rigidity,
is the curvature,
Pis the compressive load, and
yis the deflection.

According to the property of textile, they consider the length
Las a constant in the buckling process, so the bending deformation energy can be estimated from the shape as

When the compressive force is applied, shear force
Vand bending moment
Mare produced, and we ignore shear force
V. The bending moment
Mhas a linear relationship with the curvature
:
M=
k_{b}. Thus the bending deformation energy can be calculated by

Since the arc length is considered to be a constant, the curvature can be expressed as

So

Then the force vector can be derived as

where .

So the Jacobian matrix of the force vector is

The method described in section achieved realistic results, and allowed to use large time steps in the simulation which enable high performance. However, their model is based on a quad mesh, which severely limits its use. They have modified also their model to extend it to triangular meshes.

The total bending energy assigned to a particle in triangular meshes is shown in Figure . Similar with Figure , the buckling behavior in triangular meshed is defined for every pair of triangles which sharing an edge, as the red lines in Figure .

For a particle
iin the triangular mesh, all the stretching and shear energy function of its surrounding area is

where
T(
x)is the triangles around particle
i, and
is the areal energy of the triangle
.

The definition of curvature
is the same as the case of quad mesh. The bending deformation energy between two particles
iand
jis defined as

where
Mis the bending moment,
A_{i}and
A_{j}are the areas of the triangles associated with particles
iand
j.

In this research report, we present meshless methods for simulating deformable cloth. The dynamics equations of cloth are deduced from Kirchhoff-Love shell model based on continuum mechanics and sampled by meshless local Petrov-Galerkin methods. With solid physical background, the presented meshless method is a brand new approach for cloth simulation and will bring new advantages, such as

The mechanics behavior of cloth is consistent, unified and independent of the resolutions;

Meshless methods can easily provide a smooth higher order interpolation field;

Meshless discretization can provide accurate representation of geometric details;

The data structure of point sampled models can be linked more easily with a CAD database than mesh-based ones, since it is not necessary to generate a mesh;

It supports a very flexible adaptive refinement procedure, such that computational accuracy can be easily controlled;

It is a better computational paradigm for multi-scale computations than mesh-based discretization method.

The program is under construction and debuging now and in the near future we can compare the numerical results between the presented method and existing mass-spring models.

In recent years meshless methods have been introduced into computer graphics and gained increasing attention as alternative computational methods to traditional mesh-based methods as FEM. Point sampled models, which do neither have to store nor to maintain globally consistent topological information, are used in meshless methods. A set of meshless methods, such as SPH, EFG, MLPG etc., have been introduced to simulate the physical behavior of point-based models. The mathematical basis and a comparison of the numerical properties of these methods are given to instruct the application issues.

The combination of meshless simulation methods with point sampled surfaces in so-called point-based animations has become popular today. It has been an ongoing research topic in the field of computer graphics. The application of meshless methods on cloth simulation is an interesting issue. Since the 1990s, we believe that although great achievements have been made, the full potential of the alternative modeling primitives and also the possible areas of application have not yet been fully explored.

Till now, mass-spring method is the most widely used simulation technique in cloth simulation for its low computational cost and easy implementation. But this method has some natural drawbacks due to the non-continuum configuration, for instance, the material cannot be simulated consistently, the results depend on the mesh of springs and the spring parameters are lack of the basis of physics.

For more realistic and reliable behaviour of cloth, methods based on continuum mechanics are required, e.g. finite element methods or meshless methods. By continuum methods, material behavior can be reproduced accurately, independent of discretisation.

The application of meshless methods on cloth simulation is an interesting issue. The special features of cloth as a thin shell bring several problems to traditional meshless methods, such as the cloth has a very different stiffness on the stretching and torsion direction. In most of the cloth simulation approaches, the treatment of bending models is not accurate. This means that realistic material parameter and resolution independence cannot be expected. To resolve this problem, a meshless approach to the analysis of arbitrary Kirchhoff-Love shells by the Meshless Petrov-Galerkin (MLPG) method is presented.

The discretization method is based on moving least squares (MLS) or radial basis functions (RBF) approximant, which is meshless. This means that the sampling is independent of the
geometric subdivision into finite elements. The satisfaction of the
C^{1}continuity requirements are easily met; therefore, it is not necessary to refer to Mindlin-Reissner theory or to discrete Kirchho theory. The requirements of consistency are met by
the use of a polynomial basis of quadratic or higher order functions.

In this report we present an approach to cloth simulation which models the deformation based on continnum mechanics and sampled with meshless methods. By this way, the mechanics behavior of cloth is consistent, unified and independent of the resolutions. We use Kirchhoff-Love (KL) thin shell theory as the basis of the cloth model in order to fulfill the the big difference in scale between the thickness and the cloth surface, which is nearly imposible to solve with ordinary three dimensional models. Compared to finite element methods, meshless methods provide higher continuity in the displacement field, which meets the requirement of KL model.

The shells, considered in the present work, are assumed to be thin so that the Kirchhoff-Love theory can be considered as appropriate. The shell in the 3D space is described in a global cartesian coordinate system . The pair defines the position of an arbitrary point of the shell, gives the position of a point on the shell mid-surface, and is a director unit vector (normal to the shell surface both in the reference and deformed states – the usual Kirchhoff-Love hypothesis). The configuration (generic state) can be put down as

with
^{1},
^{2}and
<
h
^{-},
h
^{+}>.

Here
denotes the parametric space,
<
h
^{-},
h
^{+}>are the distances of the “lower” and “upper” surfaces of the shell from the reference surface.

We define the convective basis vectors by the tangent map

where the upper Latin subscript
Idenotes 1 to 3.

In the parameter surface , we define

where the Greek indices range from 1 to 2. The shell director coincides with the normal to the middle surface of the shell and has the properties:

In the configuration , we have the covariant base vectors simply as

A contravariant (dual) basis can be obtained from the standard relation . The corresponding covariant component of the metric tensor is .

Till now, we have not distinguished the variables in the reference (undeformed) and current (deformed) configuration. The following contents will use upper-script to denote variables in the reference configuration, for instance is a point on the reference surface; and in the current configuration, variables are without upper-scripts.

The Green-Lagrange strain tensor is defined as the difference between the metric tensors on the current and reference configurations of the shell, i.e.

the non-zero components of the tensors
_{IJ}and
_{IJ}are in turn related to the deformation of the shell as follows:

It follows from these relations that, by virtue of the assumed Kirchhoff-Love kinematics, all the strain measures of interest may be deduced from the deformation of the middle surface of the shell.

For simplicity, we restrict the scope of subsequet discussions to linearized kinematics. By

where is the displacement field of the middle surface of the shell, we can write the membrance and bending strains as:

We can conclude from these expressions that the displacement field of the middle surface furnishes a complete description of the deformation of the shell. So we regard as the primary unknown of the analysis.

For a so-called meshless implementation, a meshless interpolation scheme is required, in order to approximate the trial function over the solution domain. Actually, meshless methods may be classified according to the meshless approximation method used.

The moving least squares (MLS) approximation was the key to the development of many meshless weak form methods, because the MLS can provide a continuous approximation for a field function over the entire problem domain. Although the MLS approximation has some drawbacks in dealing with the essential boundary conditions, the major advantages of the MLS is that, the shape functions are constructed from the local points only, with high order continuities.

In our work, the augmented radial basis functions (RBF) approximations are also used to construct shape functions. They have some distinct advantages over the MLS approximation, including the shape functions possess the delta function property and the second derivatives of the shape function are smoother, and are computationally less costly. Its delta function property makes it possible to directly enforce the essential boundary conditions.

Consider a sub-domain
_{s}, the neighbourbood of a point
, which is local in the solution domain. To approximate the distribution of function
uin
_{s}, over a number of scatterd points
,the local augmented interpolation of both RBF and MLS can be expressed as standard form

with being the shape functions. We take RBF interpolation for example:

where
is a set of radial basis functions centered around the
nscattered points, and

is a monomial basis of order
m.
is a matrix composited by
Rand
Pat the scattered points:

The derivatives of can be expressed using the derivatives of the shape function ,

As at least
C^{1}continuity is required for shape functions
in shells, special techniques must be adopted to model surfaces with creases and similar discontinuities in slope, e.g. such surfaces might have to be split with appropriate
boundary conditions imposed at the seam.

In point-based graphics, object as cloth can be rendered with the only use of the coordinates of scatterd points on surface.

The present approach is targeted at cloth modelled by general shells. It means that we have to deal with the issue of surface shape approximation. At eath time step, we can obtain the shape function of all integrated points of the shell. Naturally the method of computing shape function can be applied to fit the approximate surface to a collection of scattered data points. It should be noted that the requirements on the approximate surface are governed by the desired mechanical properties of the surface.

The RBF or MLS technique can be applied immediately to obtain the surface approximation. Let us assume that a set of
m(scattered) points in space is given. These points lie directly on the surface to be approximated at locations
. The approximate surface may then be described by:

where
^{1},
^{2}is the parameterization of the surface. Note, that the unknown coefficients are now 3D position vectors. Note also, if the shape function is using MLS, the constructed
surface does not in general pass through the prescribed points, ie. the technique is not interpolating.

The approach to approximate displacement is very simple. Just express the displacement vector in the global cartesian basis .

Note that to evaluate the derivatives of the displacement vector
one needs only to differentiate the shape function
_{I}(
^{1},
^{2}). This means that the smoothness of the displacement approximation depends on the smoothness of the shape function. To compute the strains, one needs only second order
derivatives with respect to the parameters
^{1},
^{2}.

We can relate a pair of energy conjugated strain and stress
Eand
Swith a material law as

S=
C(
E)

In stable elastic equilibrium situations the total energy must be at a minimum. Mathematically, this can be reformulated by setting the first varitation of energy to zero:

where the three terms account for elastic strain energy, potential energy due to applied force , viscosity force and inertial force.

With the definition of the membrane and bending strains and stress, the elastic strain energy can be rewritten as:

where and are matrices corresponding to the membrane and bending part of the material law.

Owing to the linear relationship between , and and the approximation of displacement by the nodal displacement in equation ( ), we obtain:

where and are matrices relating nodal displacement to membrane and bending strain.

The dynamics equation form of the second order ordinary differential equation in time is

where is the diagonal nodal mass matrix, is the viscosity matrix and is the stiffness matrix, with the nodal displacement vector and forces vector .

The weak form of partial derivative equations, in contrast to the strong form, requires a weaker consistency on the approximate function. This is achieved by introducing an integral operation to the system equation. There are two major categories of principles used for constructing weak forms: variational and weighted residual methods. The Galerkin weak form and the Petrov-Galerkin weak form may be the most widely used approaches for establishing system equations. Minimum total potential energy principle is equal to the weak form of equations of balance. In section , the minimum total potential energy principle has been adopted for deriving discrete system equations.

In the local Petrove-Garlerkin approaches, one may write a weak form over a local sub-domain
_{s}of a point
, which may have an arbitrary shape. A generalized local weak form corresponding to the stiffness matrix and nodal force vector of equation (
) will be

The stiffness matrix is evaluated by numerical integration in the local domain
_{s}. Usually, Gaussian integration will be applied. Due to the arbitrary shape of the local domain, it does not need any mesh or background mesh for the integration of the weak
forms.

The global stiffness matrix and nodal force vector are assembled by local ones:

For numerical solution, this equation is translated into first order ODEs and any time integration scheme can be applied. The arising linear system of equations with nodal velocities as primary unknowns is a large and usually sparce system, which can be solved by direct factorization methods or iteration factorization methods.

The implementation of the numerical part of the present method can be carried out according to the following outline:

Parametrize the given cloth model to the mid-surface .

Choose a finite number of nodes in the parametrized mid-surface
; compute the mass
mof nodes from the given thickness
hand density
.

Determine the local sub-domain
_{s}for each node.

Loop over all time steps, the interval is
t

Compute
Gof each node; compute the shape functions
_{I}in the define domain of each node; and compute the derivatives of node coordinates.

Loop over all nodes located inside the global domain

Determine Gaussian quadrature points
in the subdomain
_{s}.

Loop over quadrature points

determine the nodes located in the domain of definition of the approximation at point ;

compute the shape function and its derivatives;

evaluate numerical integrals.

assemble contributions to the system equation for all nodes in , ;

Assemble , for the dynamics equations.

End node loop.

Solve the arising linear system of time integration and obtain the velocity .

Update the unknow variables and their derivatives at the sample points.

End time step loop.

Buckling is one of the most difficult problems for the cloth simulation research. We present a physically-based buckling model for garment simulation. In this model, cloth will no longer have a compressed state normally. This constraint is now incorporated directly in the equations of dynamics. With this method, the simulator can produce vivid and nature buckling similar to the case of real garment. This report will briefly introduce the buckling model and the experimental results.

When a structure undergoes visibly large displacements that transverse to the load then it is said to buckle. It is one of the major difficulties for the cloth simulation because firstly the deformation has not been well defined in the mathematical equations; secondly, the existing approaches failed to reliably reproduce buckling. From the literature review, most of the existing methods use geometrical methods, for example to imagine the post-buckling shape or define the folding models. Since the cloth simulator is based on a physical model only, the geometrical buckling model may lose some reality .

In our simulator, we designed the buckling model based on two features. One is the perfect control of KES (Kawabata evaluation system) characteristic curves. The other is a physical control of buckling. Researchers in that domain found that when a material undergoes buckling, it is in an unstable equilibrium state. We designed our cloth buckling model according to the following two principles. The main idea is to release the compression and make then the system converge to a stable equilibrium state. In our buckling model, the constraint is directly inserted in the conjugate gradient used to solve the dynamics equations.

Our cloth simulator Emilion is based on the implicit Baraff's integration scheme which is also used in Mayacloth. The collision part is based on the Provot's collision handling method. The Emilion simulator has the following features. Firstly, it uses the KES curves obtained from the analysis of real cloth as input.This allows a realistic control of the mechanics of the cloth. Secondly, it takes advantage of the implicit method for the numerical computation process which allows the simulator to have larger time steps. Thirdly, constraints are directly inserted for the control of collisions and buckling. Based on the considerations above, we can design the buckling model with a modified CG method which incorporates the constraints. The detailed system numerical computation, the constraints implementation and the modified CG method, was mainly presented in Baraff's paper. The basic idea of the collision handling, was presented in Provot's papers.

The basic idea of the buckling model is to release the compressed tension springs and stay in the non-compressed state until the system comes to an equilibrium. Therefore, we designed the detailed buckling implementation algorithm as follows.

Find the most compressed spring with the properties of the springs.

Compute the destination position for the most compressed spring in order to eliminate the compression.

Choose the destination position with the curvature computation.

Add the buckled position to the constraints matrix.

Run the modified CG method in order to update the position of all the masses.

Evaluate the energy and statistic the status of the springs. Update the control variables of the buckling status.

Find the next most compress spring and iterate and iterate until all compressed springs dissapear.

**Destination Position Computation**

We choose two springs in the main compression direction and another one in the other direction of the masss/spring mesh. We eliminate all of compression of the three springs by constraining them to get their original length. We can prove that if the three springs are totally released, the centered mass should be move the intersection point of three spheres, which use the three connected masses as the center of the circle, the springs' original length as the radius. Therefore, We solve three spheres intersection equations to get the destination positions.

**Position Selection and Curvature Computation**

As three spheres intersections may get two results (this corresponds to the random aspect of real buckling), we use the following procedure to select the destination position.

Check possible collision: if one destination position has a collision, choose the other one.

Compare the distances of the destinations and the original position, choose the smaller one.

If the two distances are equal, compute the curvature of the destination position and the original position. Choose the one with bigger curvature.

If the two curvatures are also equal to each other, such as in the saddle surface situation, randomly choose one position.

In the distance comparison in step 2, we can prove that the selected position has the biggest curvature which simplifies the curvature computation.

As for the curvature computation, consider a planar curve with arc-length l from the point
p0to the point
p, and the counter-clockwise angular difference between the tangents at
p0and
pas illustrated. Then the curvature of the curve at the point
p0is defined as:

For the surface, the two principal curvatures
k_{1}and
k_{2}are defined as the minimum and maximum normal curvatures. The mean curvature is defined as
. As for the scan data, we adopt the curvature calculation which is used in the point based graphics. The computation is performed as follows.

where
v_{1}=
p_{2}-
p_{1},
v_{2}=
p_{3}-
p_{2}. We only estimate the curvature in the two main directions.
p_{1},
p_{2}and
p_{3}are the three points used for the computation of the curvature. The two curvatures may be the same, such as in the situation of a planar surface. In that case, we choose the
destination position randomly.

**Data Structure and Constraints**

We use a status vector to control the status of the buckling points. Each status variable changes after the dynamic solving and energy evaluation. The constrains are set according to the status vectors. The position holding duration of the buckling operation points is also controlled by the status vector.

The constraints are applied to the modified conjugate gradient method. In our model, two types of constrain matrices are used: the
Smatrix related to mass and the velocity
zmatrix. In each time step, we add our buckling operations to the constraints. We can apply the constraints to the conjugate gradient method as a filter function. The details of the
theory of the constrain implementation are described in Baraff's paper. We apply the constraints to the buckling model which makes it easy to be combined to the existing simulator.

**Collision Handling**

Collision handling is one of the most important aspects for the cloth simulation. In our simulator the collisions are treated with constraints. We have still to improve this handling in order to make it more suitable for the case of multiple collisions which would allow to reduce the energy more rapidly. We also add the buckling operations to the collision handling parts. For the collision response process, if a mass is in the biggest compression state, we make it come to the original length. Meanwhile, the modification was also made for releasing some of the useless constraints and for feedback.

From the mechanics point of view, the buckling properties of a system are reflected in the shape of the equilibrium path, ie. the load-deflection curve. We can get the analyzis from
the literature of buckling modeling. If a system is in equilibrium then its total potential energy,
U, has a minimum value since an infinitesimal disturbance
d, of the system from its equilibrium position does not change the potential energy,
PE-
which is any characterizing displacement.

The type of equilibrium which exists at any point may be stable, neutral or unstable; it may be deduced by taking the second derivative :

U^{''}>0;PE - minimum stable equilibrium -
U^{''}= 0;PE - zero slope neutral equilibrium -
U^{''}<0;PE - maximum unstable equilibrium

We treated several examples with our simulator to test the model. The experimental results show that the model can make vivid and natural results when compared to real cloth. We chose the example of a cylinder, which reflects both the buckling and the collision handling. The first four figures is the example of a quare cloth, with force added to one border. Without buckling, it will get blocked. The other four figures are the cylinder example, the cloth is displacement with a solid static cylinder inside. We have compared the square cloth example with the real cloth and it produces the same effects which demonstrates the effectiveness of our approach.

We also analyzed the energy of the springs. The following figures are the energy of the body garment example. It is demonstrated that with our buckling model the energy will decrease more than without buckling. From this aspect, it also proves the efficiency of the method.

One of our future works is the improvement of collision handling algorithm. For the cylinder example, when further forces and displacements are added, it needs stable and efficient collision handling algorithm to deal with both the enormous cylinder collisions and the cloth cloth collisions which are still problematic. The other future research direction is speeding up the simulator.

The Golf-Stream project was funded by the French research agency Riam (Research and Innovation in Audiovisual and Multimedia). Its research topic is the capture of a real life character movement. The funding stopped in july 2006. This is a complex problem that we tried to solve by adjoining INRIA's skills with those of the Symah Vision production company. PGA (professional Golfer association) and FFG (French Federation of Golf) were part of that project and served as experts in order to drive the analysis and 3D tracking of the swing of professional golfers that we chose as an application (see section )

MIRAGES develops a tool that makes it possible to capture actual movements of a golfer on site. As an initial stage, static images of the player are shot from different angles and are used to reconstruct his or her skeleton and body envelope. The 3D virtual character is then superimposed over the player in action and it adjusts itself to the player's body with few interactions. The software has to determine body movements (pelvis rotation, arms position, etc.) as well as the position of the player's center of gravity and direction imposed to the ball with great precision. This method does not require the use of sensors placed on the athlete's body. Unfortunately, we could not finish this research during the funding period but we are still pursuing this research with our internal funding. We still have to work on the automatic pre-positioning of the 3D puppet at the swing movement start and implement the complete 3D tracker. To obtain a result on a sequence, we also have to optimize software to be much faster and usable by TV crews. Standard video cameras are able to provide up to fifty images per second. It will thus be possible to broadcast a slow-motion sequence showing the superimposed skeleton of the player with added visualization tools that support live comments on such and such aspect of the shot, barely a few minutes after the shot. This product should be available for TV broadcasts in the future. Golf Stream was coordinated By Simah Vision but as this company disappeared, Golf-Stream continued under the MIRAGES Coordination. Although Golf-Stream funding has stopped, we continue to develop this project with our own budget in order to finalize it.

This contract deals with the "cheap" realistic 3D modeling of human faces from a small set of images (even from only one picture) whatever the position of the face on the images. This research is conducted with the collaboration of AXIATEC, a small company interested in customization products. This research is an extension of the former work done within the framework of the VIP3D project. We have already implemented its extension to the case of one image, have added eyes, teeth, tongue and hair to the faces reconstructed before. Realism of the model is not sufficient to recognize the human person for the moment, so we are going to investigate more refined interaction tools to reach realism.

Since the beginning of 2007, Mirages is a member of a new competitivity network funded by the French Government. The coordinator of this project is Thalès. The aim of this network is to produce a full 3D version of a tool similar to Google-Earth. Mirages is the coordinator of the biggest workpackage of this network. Its role is to use an analysis/synthesis approach in order to produce a compact 3D representation of a part of Ile De France from building and vegetation models and from geo-referenced raw input data (images and 3D scans) obtained from other partners of the consortium.

Nadine Corrado is a Fashion designer; we collaborate on the validation of our 3D simulation of garments software. She is a potential end-user and also brought us a lot of information about garment conception. We are presently testing the 2D pre-positioning software which has to be connected with her design software.

MIRAGES recently got an important funding from the ANR (French Research Funding Agency) through the RNTL programme. The aim of this project is to design, within 3 years, a first prototype of a virtual try-on system which will allow any person to buy garments through Internet. The client will have the possibility to choose the type and style of garment and the kind of the textile material and to see himself/herself in 3D wearing the chosen garment. This raises very difficult and fascinating scientific problems that we will have to solve. This project is coordinated by Telmat (a company building 3D scanners for human persons), La Redoute (the biggest garment distributor in France) will bring its expertise in the contacts and interfaces with the clients, Nadine Corrado will serve as the expert on garment design and ENSITM (university laboratory specialized in the mechanics of textile) will furnish physical data for the garment simulator developed by MIRAGES.

Collaboration with XID Technologies, a company located in Singapore. André Gagalowicz is a scientific advisor for the company.

Collaboration with NTU (Nanyang Technological University) in Singapore has been initiated. A PHD student Quah Chee Kwang is supervised by both A. Gagalowicz from MIRAGES and Seah Hock Soon from NTU. this collaboration expands after the signature of a collaboration framework between the French Ministry of Foreign affairs and Singapore (Merlion contract)

André Gagalowicz was a scientific advisor at the INSA Lyon Scientific Committee.

André Gagalowicz was a scientific advisor at the University of Bordeaux III Scientific Committee.

André Gagalowicz was a member of the scientific advisory board of "Machine, Graphics and Vision" journal

André Gagalowicz was a member of the scientific advisory board of "Computer Graphics and Geometry" journal,

André Gagalowicz was a member of the scientific advisory board of the LCPC journal.

André Gagalowicz was a scientific advisor of XID Technologies, Singapore.

André Gagalowicz has taught Computer Vision in the DESS on images of the University of Bordeaux III.

Tung Le Thanh has defended his thesis at Pari VI University, November 30; title of the thesis : Automatic pre-positioning of garments

André Gagalowicz was the chairman of MIRAGE 2007 international conference specialized inn computer vision/computer graphics collaboration techniques (participation of 38 countries).

André Gagalowicz was a member of the conference program committees of:

GRAPP 2007 Conference, Barcelona, Spain.

GraphiCon 2007 Conference, Moscow, Russia.

VISAPP 2007 conference, Barcelona, Spain.

CAIP 2007 Conference, Vienna, Austria

Computer Graphics and Visualization 2007 conference, Lisbon, Portugal, 5 to 7 July 2007

CVMP 2007 conference, 27-28 November 2007, Savoy Place, London, Great Britain

André Gagalowicz was invited by NTU in Singapore from October 26 to November 5 in the framework of the MERLION collaboration between France and Singapore.

André Gagalowicz was the editor of the book "Computer Vision/Computer Graphics Collaboration techniques" edited by Springer Verlag LNCS 4418, ISBN-10? 36540871456.

André Gagalowicz was a member of the thesis committee of Weiming Dong, april 4 and Tung Le Thanh November 30.

André Gagalowicz has presented an invited talk or paper at:

the ICSXII 2007 conference in Saint Etienne, France, September 3-7 (see );

GRAPP 2007 Conference, Barcelona, Spain, March 8-11 (see ); http://www.grapp.org/grapp2007/index.htm

ACIVS 2007 conference, in Delft, The Netherlands, August 29- 31 (see );

INRIA/City University of Hong Kong Workshop, Hong Kong, China, May 11-12 ;

the tenth Anniversary of the LIAMA laboratory in Beijing,January 16-19 ;

C. K. Quah, A. Gagalowicz and H. SS Seah presented a paper in the Asia-Pacific Congress on Sports Technology in 2007 (see ).

C. K. Quah, A. Gagalowicz and H. SS Seah presented another paper in the Asia-Pacific Congress on Sports Technology in 2007 (see ).

T. Le Thanh and A.Gagalowicz presented a paper for MIRAGES 2007 conference which was published both in the MIRAGES 2007 proceedings as well as in the LNCS series of Springer Verlag (see )

T. Le Thanh and A.Gagalowicz presented a paper for VISAPP 2007 conference (see )

D. Kalinkina and A. Gagalowicz presented a paper for MIRAGES 2007 conference which was published both in the MIRAGES 2007 proceedings as well as in the LNCS series of Springer Verlag (see ).