Section: Overall Objectives

Objectives

The Tyrex team aims at developing a vision of a web where content is enhanced and protected, applications made easier to build, maintain and secure. It seeks to open new horizons for the development of the web, enhancing its potential, effectiveness, and dependability. In particular, we aim at making significant contributions by obtaining fundamental results, by building advanced experimental applications showcasing these results and by contributing to web standards. The challenging part here is that contributing to each of these lines of work requires progress in all of them simultaneously.

The scientific goal that we want to reach is to establish a unifying framework for designing advanced (robust, flexible, rich, novel) web applications. More precisely it means building a new generation of multimedia and augmented reality applications with new design foundations. The challenging part is to propose means to combine in an easy and compositional manner rich content, augmented reality and data gathered dynamically from the environment and collected from various sensors, cameras, etc. We propose to address this objective along two complementary directions:

1. modeling, which consists in capturing various aspects of document processing, data and communication in a unifying model, and whose difficult part involves taking into account the specificities of the web, which require new programming models and supporting theoretical tools that do not exist today.

2. analysis, verification and optimization, which consist in guaranteeing safety and efficiency properties, and whose hard part consists in dealing with problems close to the frontier of decidability, and therefore in finding useful trade-offs between programming ease, expressivity, complexity, succinctness, algorithmic techniques and effective implementations.

From a technology perspective, the generalized use of mobile terminals is having a profound impact on the way users perceive and interact with their environment. The ubiquitous use of search engines capable of producing results in fractions of a second raised the expectations level to a much higher degree. Users do now expect relevant information to be made available to them instantly and directly by context-sensitivity to the environment itself. However, the information that needs to be processed is becoming more and more complex compared to the one found in the traditional web. In order to unlock the potential introduced by this new generation of the web, a radical rethinking of how web information is produced, organized and processed is necessary.

Until now, content rendering on the web has mainly been based on supporting media formats separately. It is still notably the case in HTML5 where, for instance, vector graphics, mathematical content, audio and video are supported only as isolated media types, even though they are more and more present in web pages.

With the increasing use of web content in mobile terminals, we also need to take into account highly dynamic information flowing from sensors (positioning and orientation moves) and cameras. To reach that goal, web development platforms need to ease the manipulation of such content with carefully designed programming interfaces and by developing supporting integrative methods. More precisely, we will focus on the following aspects: (1) the design of rich content models (2) physical environment modeling and integration (3) native streams support and (4) cross-platform programming abstractions.

Our objective is to investigate the concrete design of robust and efficient mobile web applications capable of processing heterogeneous data in a simple manner. Mobile web applications are among the most complex pieces of code, as they involve both: 1) web services able to efficiently gather and process relevant data from the network [5] , [1] and 2) a front-end, tightly integrated with the terminal, which communicates with them.

Multimedia Models and Formats

For a long time, most multimedia web pages have isolated continuous content behind the fences of add-ons or plug-ins, thus preventing real interaction between these contents and the rest of their host page or the whole web. In addition, the many interactive features that are available with discrete content have no equivalent within plug-ins, where users are limited to the same level of control they have with a VCR.

New models are required to represent the many dimensions of such a rich content. Ideally, such models should keep the aspects of traditional documents that have proven useful, and extend them with the specificities of the web environment and continuous contents. The key issue here is to allow all these aspects to be present simultaneously for representing a single document. This would allow document models to be rich and versatile enough to offer many possibilities to a broad range of applications handling multimedia contents.

To be used in real applications, such multimedia document models have to be instantiated in actual formats and languages. As documents have to be part of the web, these formats must interoperate with existing web formats. They could be extensions of existing formats, or new languages that share as many features as possible with the existing ones. The goal is not to create a separate web for multimedia content, but to seamlessly extend the web as we know it.

XML Processing

XML was created for representing documents and data on the web in a secure and rigorous way. XML is now the ground on which web formats are built. If we want to propose new formats for the web, they have to be based on XML, and we need to make sure new applications will be able to take advantage of these formats. It is therefore crucial to better understand how XML structures can be handled, and what are the theoretical tools that may help to develop an effective framework for processing XML structures.

This is of course an ambitious and long-term goal that requires intermediate steps. The first specialized languages for handling XML structures were transformation-oriented (XSLT , XQuery, XDuce, CDuce, etc.). Typically, programs written in these languages read an XML structure and produce another XML structure as their output, after performing some transformations. Query languages can also be considered as behaving that way. So, the transformation paradigm is an interesting intermediate step towards general XML processing. Actually, a number of applications can be built as transformations: document formatting (XSLT was initially developed as part of the XSL formatting language), filtering, merging, conversion, re-purposing, data query, etc.

A major component in an XML transformation language is the part that allows a programmer to select, in the input structure, the data of interest for a given transformation. We thus focused on this part of XML processing languages and we studied, in particular, the XPath language, which is used in a variety of other languages for XML (XSLT, XQuery , XML Schemas). We also studied the CSS Selectors which play a similar role in the CSS language for style sheets. The main goal of this work is to find the theoretical tools and formalisms that are needed for static analysis of XPath expressions, in order to help programmers develop better and more reliable code for XML data and documents.

This work on XML has been recently extended to RDF and its query language SPARQL, in order to extend to the semantic web the results achieved for the web of documents and data.

Multimedia Authoring

Before they can be processed, multimedia XML documents have to be created. A significant part of web documents are generated by programs from other documents and data (see XML processing above), but another part is created by human authors using authoring tools. For multimedia formats to be really used, it is important that authoring tools be available.

Our work in the area of multimedia authoring tools aims at developing editing techniques for creating rich multimedia documents that take advantage of the many new dimensions of multimedia formats. The challenge is to keep these tools simple enough for average web users. Methods used for static, textual documents do not work for dynamic, multimedia web resources. New approaches have to be developed and experimented.

Research in this area is strongly connected with software development projects, with the goal of creating real tools that can be deployed on the web and that real users can use.

Augmented Environments

For the previous three objectives, we have chosen Augmented Reality as an application domain that helps us focus our work in accordance with application requirements.

To recreate or augment our perception of the real world, all modalities may be involved. For visual perception, the media that come to mind are text, graphics, photographs, video (live or recorded). But augmented reality is not restricted to the visual space. The auditory space also contributes to re-creating or extending the user environment. Moreover, the visual and auditory spaces are connected: events happening in one space often have consequences in the other, and all this is synchronized.

The geographical space is important in augmented environments. The location of the users in the real or virtual world plays a key role, as well as the moves they make. This involves mobility and specific kinds of information, such as maps or points of interest (PoIs). A number of information resources required to build augmented environments are available on the web. Applications then not only have to capture a lot of information about the local environment of their user (mainly through various sensors), but also need to access additional information on the web.

All these features of augmented environments are very demanding for the other activities in the team. They require all kinds of multimedia information, that they have to combine. This information has to be processed efficiently and safely, often in real time, and it also, for a significant part, has to be created by human users.