Project WAM aims at making it easier to develop and use rich multimedia contents and applications on the web.
Many web sites are specializing in a single type of content, such as Picasa and Flickr for photographs, YouTube and Dailymotion for videos, iTunes and Deezer for music, etc. Some other sites offer web pages that contain text, pictures, videos and audio simultaneously (newspaper sites, for instance). So, different types of contents coexist on the web, even on the same web page, but this does not really make a multimedia web or multimedia pages. The web has demonstrated how links, relations, connections, interactions between pieces of information can enhance the raw content of each piece. We are not there yet with multimedia content. Integrating and connecting heterogeneous contents on the web still have to be explored.
That is the reason why we pay a particular attention to documents and applications that tightly integrate different types of media objects, be they discrete (text, images, equations) or continuous (video, audio, animations). Continuous contents add a time dimension to documents that mix various sorts of contents. This extra dimension raises new issues. It has to be combined with other, more traditional points of view on documents, such as their layout and style (spatial dimension), their organization often represented as a hierarchical structure (logical dimension), etc.
In the context of the web, multimedia resources are distributed and can be assembled in various ways to make different documents and to be processed by multiple applications, running on all sorts of computers, devices and networks. For this reason, they have to be represented in platform-neutral formats.
This approach to web multimedia content and applications raises a number of issues. We have chosen to address four categories of problems:
For a long time, most multimedia web pages have isolated continuous content behind the fences of add-ons or plug-ins, thus preventing real interaction between these contents and the rest of their host page or the whole web. In addition, the many interactive features that are available with discrete content have no equivalent within plug-ins, where users are limited to the same level of control they have with a VCR.
New models are required to represent the many dimensions of multimedia documents. Ideally, such models should keep the aspects of traditional documents that have proven useful, and extend them with the specificities of the web environment and continuous contents. The key issue here is to allow all these aspects to be present simultaneously for representing a single document. This would allow document models to be rich and versatile enough to offer many possibilities to a broad range of applications handling multimedia contents.
To be used in real applications, such multimedia document models have to be instantiated in actual formats and languages. As documents have to be part of the web, these formats must be compatible with existing web formats. They could be extensions of existing formats, or new languages that share as many features as possible with the existing ones. The goal is not to create a separate web for multimedia content, but to seamlessly extend the web as we know it.
XML was created for representing documents and data on the web in a secure and rigorous way. XML is now the ground on which web formats are built. If we want to propose new formats for the web, they have to be based on XML, and we need to make sure new applications will be able to take advantage of these formats. It is therefore crucial to better understand how XML structures can be handled, and what are the theoretical tools that may help to develop an effective framework for processing XML structures.
This is of course an ambitious and long-term goal that requires intermediate steps. The first specialized languages for handling XML structures were transformation-oriented (XSLT, XDuce, CDuce, etc.). Typically, programs written in these languages read an XML structure and produce another XML structure as their output, after performing some transformations. Query languages can also be considered as behaving that way. So, the transformation paradigm is an interesting intermediate step towards general XML processing. Actually, a number of applications can be built as transformations: document formatting (XSLT was initially developed as part of the XSL formatting language), filtering, merging, conversion, re-purposing, data query, etc.
A major component in an XML transformation language is the part that allows a programmer to select in the input structure the data of interest for a given transformation. We have then focused on this part of XML processing languages, and we have in particular studied the XPath language, which is used in a variety of other languages for XML (XSLT, XQuery, XML Schemas). We have also studied the CSS Selectors which play a similar role in the CSS language for style sheets. The main goal of this work is to find the theoretical tools and formalisms that are needed for static analysis of XPath expressions, in order to help programmers develop better and more reliable code for XML data and documents.
This work on XML has been recently extended to RDF and its query language SPARQL, in order to extend to the semantic web the results achieved for the web of documents and data.
Before they can be processed, multimedia XML documents have to be created. A significant part of web documents are generated by programs from other documents and data (see XML processing above), but another part is created by human authors using authoring tools. For multimedia formats to be really used, it is important that authoring tools be available.
Our work in the area of multimedia authoring tools aims at developing editing techniques for creating rich multimedia documents that take advantage of the many new dimensions of multimedia formats. The challenge is to keep these tools simple enough for average web users. Methods used for static, textual documents do not work for dynamic, multimedia web resources. New approaches have to be developed and experimented.
Research in this area is strongly connected with software development projects, with the goal of creating real tools that can be deployed on the web and that real users can use.
For the previous three objectives we have chosen Augmented Reality as an application domain that helps us focus our work in accordance with application requirements.
To recreate or augment our perception of the real world, all modalities may be involved. For visual perception, the media that come to mind are text, graphics, photographs, video (live or recorded). But augmented reality is not restricted to the visual space. The auditory space also contributes to re-creating or extending the user environment. Moreover, the visual and auditory spaces are connected: events happening in one space often have consequences in the other, and all this is synchronized.
The geographical space is important in augmented environments. The location of the users in the real or virtual world plays a key role, as well as the moves they make. This involves mobility, navigation, and specific kinds of information, such as maps or points of interest (PoIs). A number of information resources required to build augmented environments are available on the web. Applications have then not only to capture a lot of information about the local environment of their user (mainly through various sensors), but they also need to access additional information on the web.
All these features of augmented environments are very demanding for the other activities in the team. They require all kinds of multimedia information, that they have to combine. This information has to be processed efficiently and safely, often in real time, and it has also, for a significant part, to be created by human users.
Extensible Markup Language (XML) has gained considerable interest from industry, and plays now a central role in modern information system infrastructures. In particular, XML is the key technology for describing, storing, and exchanging a wide variety of data on the web. The essence of XML consists in organizing information in tree-tagged structures conforming to some constraints which are expressed using type languages such as DTDs, XML Schemas, and Relax NG.
There still exist important obstacles in XML programming, especially in the areas of performance and reliability. Programmers are given two options: domain-specific languages such as XSLT, or general-purpose languages augmented with XML application programming interfaces such as the Document Object Model (DOM). Neither of these options is a satisfactory answer to performance and reliability issues, nor is there even a trade-off between the two. As a consequence, new paradigms are being proposed which all have the aim of incorporating XML data as first-class constructs in programming languages. The hope is to build a new generation of tools that are capable of taking reliability and performance into account at compile time.
One of the major challenges in this line of research is to develop automated and tractable techniques for ensuring static type safety and optimization of programs. To this end, there is a need to solve some basic reasoning tasks that involve very complex constructions such as XML types (regular tree types) and powerful navigational primitives (XPath expressions or CSS selectors). In particular, every future compiler of XML programs will have to routinely solve problems such as:
XPath query emptiness in the presence of a schema: if one can decide at compile time that a query is not satisfiable, then subsequent bound computations can be avoided
query equivalence, which is important for query reformulation and optimization
path type-checking, for ensuring at compile time that invalid documents can never arise as the output of XML processing code.
All these problems are known to be computationally heavy (when decidable), and the related algorithms are often tricky.
We have developed an XML/XPath static analyzer based on a new logic of finite trees. This analyzer consists of:
compilers that allow XML types, XPath queries, and CSS selectors to be translated into this logic
an optimized logical solver for testing satisfiability of a formula of this logic.
The benefit of these compilers is that they allow one to reduce all the problems listed above, and many others too, to logical satisfiability. This approach has a couple of important practical advantages. First of all, one can use the satisfiability algorithm to solve all of these problems. More importantly, one could easily explore new variants of these problems, generated for example by the presence of different kinds of type or schema information, with no need to devise a new algorithm for each variant.
We have participated in the international endeavor for defining a standard multimedia document format for the web that accommodates the constraints of different types of terminals. SMIL is the main outcome of this work. It focuses on a modular and scalable XML format that combines efficiently the different dimensions of a multimedia web document: synchronization, layout and linking. Our current work on multimedia formats follows the same trend.
With the advent of HTML5 and its support in all popular browsers, HTML is becoming an important multimedia language. Video and audio can now be embedded in HTML pages without worrying about the availability of plugins. However, animation and synchronization of a HTML5 page still require programming skills. To address this issue, we are developing a scheduler that allows HTML documents to be animated and synchronized in a purely declarative way. This work is based on the SMIL Timing and Synchronization module and the SMIL Timesheets specification. The scheduler is implemented in JavaScript, which makes it usable in any browser. Timesheets can also be used with other XML document languages, such as SVG for instance.
Audio is the poor relation in the web format family. Most contents on the web may be represented in a structured way, such as text in HTML or XML, graphics in SVG, or mathematics in MathML, but sound was left aside with low-level representations that basically only encode the audio signal. Our work on audio formats aims at allowing sound to be on a par with other contents, in such a way it could be easily combined with them in rich multimedia documents that can then be processed safely in advanced applications. More specifically, we have participated in IAsig (Interactive Audio special interest group), an international initiative for creating a new format for interactive audio called iXMF (Interactive eXtensible Music Format). We are now developing A2ML, an XML format for embedded interactive audio, deriving from well-established formats such as iXMF and SMIL. We use it in augmented environments (see section ), where virtual, interactive, 3D sounds are combined with the real sonic environment.
Regarding discrete media objects in multimedia documents, popular document languages such as HTML can represent a very broad range of documents, because they contain very general elements that can be used in many different situations. This advantage comes at the price of a low level of semantics attached to the structure. The concepts of microformats and semantic HTML were proposed to tackle this weakness. More recently, RDFa and microdata were introduced with the same goal. These formats add semantics to web pages while taking advantage of the existing HTML infrastructure. With this approach new applications can be deployed smoothly on the web, but authors of web pages have very little help for creating and encoding this kind of semantic markup. A language that addresses these issues is developed and implemented in WAM. Called XTiger, its role is to specify semantically rich XML languages in terms of other, less expressive XML languages, such as HTML. Recent extensions to the language make it now usable also to edit pure XML documents and to define their structure model (see section ).
Multimedia documents are considered through several kinds of structures: logical organization, layout, time, linking, animations. We are working on techniques that allow authors of such documents to manipulate all these structures in homogeneous environments. The main objective is to support new advances in document formats without making the authoring task more complex. The key idea is to present simultaneously several views of the document, each view putting the emphasis on a particular structure, and to allow authors to manipulate each view directly and efficiently. As the various structures of a document are not independent from each other, views are “synchronized” to reflect in each of them the consequences of every change made in a particular view. The XML markup, although it can be accessed at any time, is handled by the tools, and authors do not have to worry about syntactical issues.
We have more recently experimented another way to edit highly structured XML documents without the usual complexity of the most common XML editors. The novelty of the approach is to use templates instead of XML schemas or DTDs, and to run the editor as a web application, within the browser. This way, it is much easier to create new document types and to provide an editing environment for these document types, that any web user can instantly use. This lightweight approach to XML editing complements the previous approach by covering new categories of XML applications.
The term Augmented Environments refers collectively to ubiquitous computing, context-aware computing, and intelligent environments. The goal of our research on these environments is to introduce personal Augmented Reality (AR) devices, taking advantage of their embedded sensors. We believe that personal AR devices such as mobile phones or tablets will play a central role in augmented environments. These environments offer the possibility of using ubiquitous computation, communication, and sensing to present context-sensitive information and services to the user.
AR applications often rely on 3D content and employ specialized hardware and computer vision techniques for both tracking and scene reconstruction. Our approach tries to seek a balance between these traditional AR contexts and what has come to be known as mobile AR browsing. It first acknowledges that mobile augmented environment browsing does not require that 3D content be the primary means of authoring. It provides instead a method for HTML5 and audio content to be authored, positioned in the surrounding environments and manipulated as freely as in modern web browsers.
Many service providers of augmented environments desire to create innovative services. Accessibility of buildings is one example we are involved in. However, service providers often have to strongly rely on experience, intuition, and tacit knowledge due to lack of tools on which to base a scientific approach. Augmented environments offer the required rigorous approach that enables Evidence-Based Services (EBS) if adequate tools for AR technologies are designed. Service cooperation through exchange of normalized real-time data or data logs is one of these tools, together with sensor data streams fusion inside an AR mobile browser. EBS can improve the performance of real-world sensing, and conversely EBS models authoring and service operation can be facilitated by real-world sensing.
The applications we use to elaborate and validate our concepts are pedestrian navigation for visually impaired people and applications for cultural heritage visits. On the authoring side, we are interested in interactive indoor modeling, audio mobile mixing, and formats for Points of Interest. Augmented environment services we consider are, among others, behavior analysis for accessibility, location services, and indoor geographical information services.
Broadly speaking, the main application domain of our research is the web and its numerous applications. This includes the recent evolutions of the web, with a special attention paid to the mobile web, the multimedia web, and the web as a platform for applications. The goal of our research is to enable new multimedia and mobile applications that can be deployed easily on the web, taking advantage of the existing infrastructure and the latest advances in web technology.
More specifically, our work this year has focused on two main application domains: web development and pedestrian navigation.
Current content representation practices and programming methods for the web remain severely limited. Designing web applications is becoming increasingly complex as it relies more and more on a jungle of programming languages, tools and data formats, each targeted toward the different application layers (presentation, application and storage). This often yields complex and opaque applications organized in silos, which are costly, inefficient, hard to maintain and evolve, and vulnerable to errors. In addition, the communication aspects are often handled independently via remote service invocations and represent another source of complexity and vulnerability.
Most research activities in WAM address these issues and try to cope with the fundamentals aspects of web applications (advanced content, data and communication) by studying rich document formats, data models and communication patterns, to offer correction guarantees and flexibility in the application architecture. For instance, applications need to be checked, optimized and managed as a whole while leveraging on the consistency of their individual components and data fragments.
A number of factors are changing our thinking of an accessible town, namely the open data movement exemplified by OpenStreetMap, MEMS sensors embedded in mobile phones (accelerometers, magnetometers, gyroscopes), web and Augmented Reality technologies, increase in processing power of mobile phones. All these changes are allowing us to build energy efficient urban pedestrian navigation systems. These systems can now be based on Inertial Measurement Unit (IMU) and will run on mobile phones with customized embedded geographical data, a routing engine, and real time queries to urban information systems.
Richer and more precise AR mobile applications in such fields as cultural heritage visits, outdoor games, or guidance of people with disabilities will be enabled by MMG navigation, i.e. the join use of micro, macro and global navigation.
Micro-navigation builds upon embedded software ability to create a greater awareness of the immediate environment, using texture-based tracking or vision algorithms and relating this information to map and IMU data. Micro-navigation includes avoiding obstacles, locating a clear path in the proximate surroundings or at a complex crossing, finding objects and providing absolute positioning using known landmarks or beacons. Micro-navigation works at a precision level of a few centimeters by using predefined landmarks.
Macro-navigation refers to the actions required to find a route in a larger, not immediately perceptible environment, and builds upon carefully designed pedestrian ways incorporating speech instructions, audio guidance, environmental queries and IMU instructions among other things. Macro-navigation works at a precision level of one step using carefully designed routes with map-matching instructions.
Global navigation is based on an absolute global localization system like the GPS. Its precision is that of a few meters if used in a adequate geographical environment where data from external sensors is accessible. It can be used to bootstrap macro-navigation.
There is a duality relation between micro-navigation and macro-navigation. Micro-navigation is based on a localization system giving an absolute position which makes it possible to compute a relative position with respect to the planned route. Macro-navigation is based on a localization system giving a relative position which allows to compute an absolute position on the route through a process called map-matching. As a consequence, this two kinds of navigation complement and enhance each other.
The XML Reasoning Solver is a tool for the static analysis of XPath queries and XML schemas based on the latest theoretical advances. It allows automated verification of properties that are expressed as logical formulas over trees. A logical formula may for instance express structural constraints or navigation properties (like e.g. path existence and node selection) in finite trees.
The tool can solve many fundamental XML problems such as satisfiability of XPath expressions in the presence of XML schemas, containment and equivalence of XPath expressions, and many other problems that can be formulated with XPath expressions and schemas (DTDs, XML Schemas, Relax-NG).
The system is implemented in Java and uses symbolic techniques (binary decision diagrams) in order to enhance its performance. It is capable of comparing path expressions in the presence of real-world DTDs (such as the W3C SMIL and XHTML language recommendations, for instance). The cost ranges from several milliseconds for comparison of XPath queries without tree types, to several seconds for queries under very large, heavily recursive, type constraints, such as the XHTML DTD. These measurements shed light for the first time on the cost of solving static analysis problems in practice. Furthermore, the analyzer generates XML counter-examples that allow program defects to be reproduced independently from the analyzer.
We have introduced the first system capable of statically verifying properties of a given cascading style sheet (CSS) over the whole set of documents to which this stylesheet applies . The system is composed of a set of parsers for reading the CSS and schema files (XML Schema, Relax NG, or DTD) together with a text file corresponding to problem description as a logical formula. We have developed a compiler that translates CSS files into their logical representations. Then, the solver takes the overall problem formulation and checks it for satisfiability.
We have started the development of an XQuery IDE with a web interface. This prototype integrates static analyses performed by the solver inside a development environment suited for XQuery programmers.
The goal of the Timesheets library is to synchronize HTML5 content using declarative synchronization languages defined by W3C standards (namely SMIL Timing and Synchronization and SMIL Timesheets).
With the raise of HTML5 which natively supports continuous content (audio, video) there is a dramatic need for handling synchronization, animation and user interaction in an efficient and homogeneous way. As web browsers do not support SMIL, except for SVG Animation (which is based on the SMIL BasicAnimation module), multimedia web authoring remains difficult and relies on code-based, non-standard solutions.
Therefore we are developing a generic, cross-browser JavaScript implementation for scheduling the dynamic behavior of HTML5 content that can be described with declarative SMIL markup. Using a declarative language makes sense for the most common tasks, which currently require JavaScript programming:
it is much easier for web authors and for web authoring tool developers;
it is a much better way to achieve good accessibility and indexability;
it is easier to maintain, since no specific JavaScript code is used.
This open source library is now deployed and used by external users. As far as we know, ENS Lyon was the first user: its site html5.ens-lyon contains several dozens of scientific conferences where the video capture of each conference is synchronized with the slides, a structured timeline and a table of contents. This web site was demonstrated in May at the WWW 2012 conference. University of Evry makes also a important use of the Timesheets library as a tool for teaching multimedia concepts at master level.
The MAUDL library (Mobile AUDio Language) is an evolution of the ARIA library whose primary target was games on mobile devices.
Augmented Reality Audio applications use sound objects to create a soundscape. A sound object is a time structure of audio chunks whose duration is on the time scale of 100 ms to several seconds. These sound objects have heterogeneous and time-varying properties. In order to describe Interactive Audio (IA) contents, we created MAUDL, an XML language inspired by iXMF that is well adapted to the design of dynamic soundtracks for navigation systems.
MAUDL prevents audio information overwhelming through categorization at the declarative level and the use of priority queues at the execution level. This allows to take account of speed when walking, and rapid hand gestures when interrogating the environment for example. MAUDL can be used as an authoring time interchange file format for interactive mobile applications or as a runtime file format that is actually loaded through the web and played directly in the mobile. MAUDL is a cue-oriented interactive audio system, audio services being requested using named events and the systems response to each event being determined by the audio artist. The current version of the API supports iOS and further support for other mobile platforms (Android) is planned.
A virtual 3D audio pointer provides an intuitive guide to the user of a mobile navigation application, reducing the need for cognitive work when compared to vocal instructions. We have built such a pointer using the MAUDL language. It gives the user the azimuth using HRTF spatialized audio cues, with additional hints taking the form of variations in the sound used. It allows to superpose other kinds of audio contents, such as voice while the pointer is active, to indicate distance for example. This audio object is suitable for different sorts of navigation systems, such as POIs browsers, self-guided audio tours, or applications for following predefined routes.
The concept of Mixed Reality comes from the fact that the real/virtual dichotomy is not sharp, but interpolatively smooth over a virtuality continuum. Idealized notions of reality and virtuality can be thought of as endpoints on a continuum, an instance of the former approach corresponding for example to a see-through display with natural sounds, an instance of the latter to texture-mapped image-based rendering (panoramas) with synthetic sound objects.
Augmented Reality (AR) mode refers to all cases in which the auditory or visual display of an otherwise real environment is augmented by means of virtual sound or graphic objects. The converse case on the virtuality continuum is Augmented Virtuality (AV), where a virtual world, one that is generated primarily by computer, like with synthetic 3D graphic or synthetic panoramic, is being augmented with the audio-visual content of points of interest (POIs).
The Mixed Reality Browser (MRB) is a geolocalized web browser running on mobile devices. It uses standard and open XML formats for content authoring (HTML5, OSM and MAUDL) to allow anyone to create an augmented or virtual reality city tour that can be used with this application.
The introduction of mobile augmented reality browsers has forced a rethink on what kind of reality should be offered. Mobility induces a need for telepresence and simulation to free the user or the developer of the necessity to go every time in the real world. Mobility is the main reason behind the concept of the Mixed Reality Browser. By its intrinsic characteristics, MRB supports advance MR applications like mobile remote maintenance and assisted navigation.
GPS navigation systems when used in an urban environment are limited in precision and can only give instructions at the level of the street and not of the sidewalk. GPS is limited to outdoor navigation and requires some delicate transitioning system when switching to another positioning system to perform indoor navigation.
IXE is an open source urban pedestrian navigation system based on Inertial Measurement Unit (IMU) and running on mobile phones with onboard geographic data and a routing engine. With IXE, the distinction between indoor and outdoor is blurred as an IMU-based location engine can run indoor and outdoor. IXE allows augmented reality queries on customized embedded geographical data. Queries on route nodes or POIs, on ways and relations are predefined for efficiency and quality of information.
Following the web paradigm, IXE is a browser for XML documents describing navigation networks: by using the micro-format concept, one can define inside OpenStreetMap a complex format for pedestrian navigation networks allowing navigation at the level of sidewalks or corridors. The big advantage of doing this instead of defining new XML languages is that we can use the standard OpenStreetMap editor JOSM to create navigation networks in a short amount of time.
The purpose of the IXE browser is to read these OSM documents and to generate from them visible or audible navigation information. IXE works on any mobile phone running under iOS or Android. Its heart is composed of three engines, one for dead-reckoning navigation, one for interactive audio and the last one for Augmented Reality visual information, allowing quick reconfiguration for extremely varied applications.
IXE can be used for accessible navigation allowing independent living for people with disabilities.
In the context of the CLAIRE project (see section ), a new model for educational documents has been defined. The objectives of this model are:
to seamlessly handle conventional and richmedia content in the context of a unique pedagogical web platform.
to be able to store and recover any multimedia document including its spatial and time structure, consistent with HTML5 and Timesheets specifications.
to have a data model which is format agnostic to cope with existing and future rendering systems.
to cope with the authoring needs of all users.
We have more specifically worked on the multimedia modelling part for defining spatial and temporal fragment types. These types are used to express the synchronization beween different elements within the document.
We are now using this model in the definition and implementation of a web-based authoring user interface.
In the area of XML processing, we have obtained new results in several directions:
We have introduced the first system capable of statically verifying properties of a given cascading style sheet (CSS) over the whole set of documents to which this style sheet applies . Properties include coverage of styling information and absence of erroneous rendering.
In a joint work with the EXMO team, we have introduced a novel approach for deciding the SPARQL query containment problem in the presence of schemas, that paves the way for future extensions .
We have revisited the problem of XML Query-Update Independence Analysis, and showed the relevance of an approach that has been neglected in the literature so far . In particular, we have compared an SMT-modulo with a tree logic approach to Independence Analysis.
We have made progress on the characterization of the impacts of schema changes on XQuery programs .
We have formally proved a result about the factorization power of the Lean: a construction that we use to speed up the XML Reasoning Solver. We have characterized which kind of duplicate subformulas this construction eliminates, and how .
We have proposed a novel technique and a tool for the static type-checking of XQuery programs, using backward type inference .
We have defined a type system for integrating session types for objects in object-oriented languages such as Java, with full structural subtyping, without altering the language semantics . Session types are protocol specifications which describe which sequences of method calls are allowed or disallowed on a given object.
We briefly review these results below.
Developing and maintaining cascading style sheets (CSS) is an important issue to web developers as they suffer from the lack of rigorous methods. Most existing means rely on validators that check syntactic rules, and on runtime debuggers that check the behavior of a CSS style sheet on a particular document instance. However, the aim of most style sheets is to be applied to an entire set of documents, usually defined by some schema. To this end, a CSS style sheet is usually written w.r.t. a given schema. While usual debugging tools help reducing the number of bugs, they do not ultimately allow to prove properties over the whole set of documents to which the style sheet is intended to be applied. We have developed a novel approach to fill this lack . The main ideas are borrowed from the fields of logic and compile-time verification and applied to the analysis of CSS style sheets. We have implemented an original tool (see section ) based on recent advances in tree logics. The tool is capable of statically detecting a wide range of errors (such as empty CSS selectors and semantically equivalent selectors), as well as proving properties related to sets of documents (such as coverage of styling information), in the presence or absence of schema information. This new tool can be used in addition to existing runtime debuggers to ensure a higher level of quality of CSS style sheets.
The problem of SPARQL query containment is defined as determining if the result of one query is included in the result of another for any RDF graph. Query containment is important in many areas, including information integration, query optimization, and reasoning about Entity-Relationship diagrams .
We encode this problem into an expressive logic called μ-calculus: where RDF graphs become transition systems, queries and schema axioms become formulas . Thus, the containment problem is reduced to formula satisfiability test. Beyond the logic’s expressive power, satisfiability solvers are available for it. Hence, this study allows to exploit these advantages.
In addition, in order to experimentally assess implementation limitations, we have designed a benchmark suite offering different experimental settings depending on the type of queries, projection and reasoning (RDFS) . We have applied this benchmark to three available systems using different techniques highlighting the strengths and weaknesses of such systems.
XML transformations can be resource-costly in particular when applied to very large XML documents and document sets. Those transformations usually involve lots of XPath queries and may not need to be entirely re-executed following an update of the input document. In this context, a given query is said to be independent of a given update if, for any XML document, the results of the query are not affected by the update. We have revisited Benedikt and Cheney's framework for query-update independence analysis and we have shown that performance can be drastically enhanced, contradicting their initial claims . The essence of our approach and results resides in the use of an appropriate logic, to which queries and updates are both succinctly translated. Compared to previous approaches, ours is more expressive from a theoretical point of view, equally accurate, and more efficient in practice. We have illustrated this through practical experiments and comparative figures.
Updating XQuery programs in accordance with a change of the input XML schema is known to be a time-consuming and error-prone task. We have designed an automatic method aimed at helping developers realign the XQuery program with the new schema . First, we have devised a taxonomy of possible problems induced by a schema change. This allows to differentiate problems according to their severity levels, e.g. errors that require code revision, and semantic changes that should be brought to the developer's attention. Second, we have provided the necessary algorithms to detect such problems using our solver (see section ) to check satisfiability of XPath expressions.
We have developed a functional approach to design rich type systems based on an elegant logical representation of types . The representation is not only clean but it also avoids exponential increases in combined complexity due to subformula duplication. This opens the way to solving a wide range of problems such as subtyping in exponential-time even though their direct translation into the underlying logic results in an exponential blowup of the formula size, yielding an incorrectly presumed two-exponential time complexity.
We have designed a novel technique and a tool for static type-checking of XQuery programs . The tool looks for errors in the program by jointly analyzing the source code of the program, input and output schemas that respectively describe the sets of documents admissible as input and as output of the program. The crux and the novelty of our results reside in the joint use of backward type inference and a two-way logic to represent inferred tree type portions. This allowed us to design and implement a type-checker for XQuery which is more precise and supports a larger fragment of XQuery compared to the approaches previously proposed in the literature; in particular compared to the only few actually implemented static type-checkers such as the one in Galax. The whole system uses compilers and a satisfiability solver for deciding containment for two-way regular tree expressions. Our tool takes an XQuery program and two schemas
Session types allow communication protocols to be specified type-theoretically so that protocol implementations can be verified by static type checking. In , we extend previous work on session types for distributed object-oriented languages in three ways. (1) We attach a session type to a class definition, to specify the possible sequences of method calls. (2) We allow a session type (protocol) implementation to be modularized, i.e. partitioned into separately-callable methods. (3) We treat session-typed communication channels as objects, integrating their session types with the session types of classes. The result is an elegant unification of communication channels and their session types, distributed object-oriented programming, and a form of typestate supporting non-uniform objects, i.e. objects that dynamically change the set of available methods. We define syntax, operational semantics, a sound type system, and a sound and complete type checking algorithm for a small distributed class-based object-oriented language with structural subtyping. Static typing guarantees that both sequences of messages on channels, and sequences of method calls on objects, conform to type-theoretic specifications, thus ensuring type-safety. The language includes expected features of session types, such as delegation, and expected features of object-oriented programming, such as encapsulation of local state. The main ideas have been implemented as a prototype, extending Java 1.4.
In cooperation with EPFL (Lausanne) we pursue our research on template-driven editing for XML multimedia contents (see section ). Experiments with very different types of contents have been done with the AXEL library. AXEL is developed by EPFL, based on our joint work on template languages. It is an innovative multi-purpose client-side authoring framework intended for web users with limited skills.
We have addressed the issue of authoring XML multimedia content on the web, focusing on methods that apply to such contents as structured documents, factual data, and multimedia objects . We have shown that a template-based approach enhances the ability for multiple applications to use the produced content.
Most results in the area of augmented environments were presented through various software products and prototypes, including:
IXE, Interactive eXtensible Engine (see section for details). In particular, IXE allowed us to show that a precision of one step is attainable, guidance being done through a mix of spatialized vocal instructions and 3D audio.
GIF Demonstrator: This application was used to showcase our technologies at the Grenoble Innovation Fair (GIF). Augmented reality was used to find the various booths and products, while our indoor navigation system was guiding visitors to any booth.
Interactive Audio Panorama: A fun interactive experience with virtual audio. It immerses the user in a complete 360° audio panorama and allows her/him to discover a futuristic house. It demonstrates the authoring possibilities offered by the MAUDL interactive audio language.
PDRTrack: An indoor localization utility demonstrating the various correction parameters of our IMU-based localization system. The user can record data sets and simulate using various parameters to find out the effect of different map matching settings and their result on localization accuracy. The user can also simply walk in real-time with tracking enabled on a given OpenStreetMap network.
Sugimotocho Stn: A model of this railway station has been built with the help of the GISLab (Osaka City University). An electronic kick-scooter was used to measure distances and a navigation network was designed to help people to move around in the station.
These products and prototypes were presented in various fora in 2012, in particular at:
CLAIRE
Title: Community Learning through Adaptive and Interactive multichannel Resources for Education
Call: Technologies for e-education
Duration: March 2012 - February 2014
Coordinator: SimpleIT
Others partners: LIRIS
See also: http://www.projet-claire.fr/
Abstract: Project CLAIRE aims at developing an open-source tool for collaborative authoring in an e-learning environment (Learning Content Management System), targeting teachers and students in high-school and universities. Its innovative features include:
a platform for collaborative structured editing of rich media and semantic content, e.g.: tools for generating interactive evaluation tests
processes for continuous enhancement of content, e.g.: social annotation, behavior analysis, accessible multi-support publishing (web, PDF, ODT, LaTeX, smartphones, tablets).
Codex
Title: Efficiency, Dynamicity and Composition for XML: Models, Algorithms, and Systems
Call: Emerging Domains program (DEFIS)
Duration: March 2009 - June 2012
Coordinator: Inria Saclay-Île-de-France
Others partners: Inria Lille-Nord-Europe (MOSTRARE), University Paris Sud, Sorbonne - University Paris 7 (PPS), Centre universitaire de Blois (LI - Université F. Rabelais Tours), Innovimax SARL.
See also: http://codex.saclay.inria.fr/
Abstract: Codex seeks to push the frontier of XML technology innovation in three interconnected directions.
Languages and algorithms: prototypes are developed for efficient and expressive XML processing, in particular advancing towards massively distributed XML repositories.
Codex considers models for describing, controlling, and reacting to the dynamic behavior of XML corpora and XML schemas with time.
The project proposes theories, models and prototypes for composing XML programs for richer interactions, and XML schemas into rich, expressive, yet formally grounded type descriptions.
Typex
Title: Typeful certified XML: integrating language, logic, and data-oriented best practices
Call: Programme Blanc
Duration: January 2012 - December 2014
Coordinator: PPS (CNRS - Paris 7 Diderot)
Others partners: LRI (Orsay)
See also: http://typex.lri.fr
Abstract: The highly ambitious and final goal of this project is to produce a new generation of XML programming languages stemming from the synergy of integrating three approaches into a unique framework:
a logical approach based on solvers
a programming language approach
a data-oriented approach
Autonomy
Title: High-tech to serve people with disabilities
Call: Global competitiveness cluster Minalogic, 6th call for R&D projects
Duration: March 2010 - June 2012
Coordinator: ST Microelectronics
Others partners: ST-Ericsson, Raisonance, Grenoble University, IVèS
See also: http://autonomie.minalogic.net/
Abstract: The goal of the project is to develop high-tech tools to improve autonomy for people with disabilities. These tools are integrated in mobile devices such as cell phones or special-purpose devices, to improve the quality of life of people with disabilities. These devices access remote dedicated services to help geolocation and guiding. They take advantage of the latest advances in embedded systems: cameras, audio, video, compass, accelerometer, gyroscope. Two major application areas are addressed: software tools on cell phones for sight disabled people, and guiding and information tools for moving around in a city.
Title: immersiVe ENhancemenT of User-woRld Interactions
Type: Cooperation (ICT)
Call: FP7-ICT-20111.5 Networked Media and Search Systems
Instrument: Specific Targeted Research Project (STREP)
Duration: October 2011 - September 2014
Coordinator: Fondazione Bruno Kessler (Italy)
Others partners: Fraunhofer Heinrich Hertz Institute (Germany), ST Microelectronics (Italy), ST-Ericsson (France), Metaio (Germany), e-Diam Interactive (Spain), Sony-Ericsson (Sweden)
See also: https://venturi.fbk.eu/
Abstract: Venturi aims to create a pervasive Augmented Reality paradigm, where available information will be presented in a user- rather than device-specific way. The goal is to create an experience that is always present whilst never obstructing. Venturi will exploit, optimize and extend current and next generation mobile platforms; verifying platform and QoE performance through life-enriching use cases and applications to ensure device-to-user continuity.
EPFL, MEDIA group (Switzerland)
We have been working jointly for years on XML editing, more specifically on the template-driven approach. This collaboration was recently extended to XML processing .
Cécile Roisin is a member of the steering committee of the ACM Symposium on Document Engineering.
N. Layaïda is a member of Inria Grenoble-Rhône-Alpes Committee for Employment of Scientists: Inria hiring of Post-docs, Delegates and Secondments.
N. Layaïda, is program committee member of the 2nd International Conference on Model & Data Engineering (MEDI’2012). 3-5 October 2012. Poitiers, Futuroscope - France.
P. Genevès and N. Layaïda are program committee members of ACM Symposium on Document Engineering. 4-7 September 2012. Telecom ParisTech. Paris.
P. Genevès is a permanent member of Inria Grenoble-Rhône-Alpes Committee for Technological Development: Inria hiring of Research Engineers.
In 2012, P. Genevès has been a referee for the following international journals: ACM Transactions on Database Systems, Data & Knowledge Engineering, and Theoretical Computer Science.
In 2012, C. Roisin has been a referee for the following international conferences: ACM Multimedia 2012, ACM Document Engineering, and WWW2012 Demo Track.
Licence: Computer Networks, 190h/year, L2, University of Grenoble – C. Roisin
Licence: Logic, 45h/year, L3, University of Grenoble – N. Gesbert
Licence: C Programming project, 12h/year, L3, University of Grenoble – N. Gesbert
Master: Logic, 22h30/year, M1, University of Grenoble – N. Gesbert
Master: Web development, 22h30/year, M1, University of Grenoble – N. Gesbert
Master: Software analysis, conception and validation, 45h/year, M1, University of Grenoble – N. Gesbert
Master: Foundations for XML: logics and automata, 18h/year, M2 (Mosig), University of Grenoble – P. Genevès
Master: SIGAL Multimedia systems and web languages, 9h/year, M2 (Mosig), University of Grenoble – N. Layaïda
PhD: M. Chekol, Static analysis of semantic web queries, Université de Grenoble, 19 December 2012, N. Layaïda and J. Euzenat
PhD in progress: N. Guido, Satisfiability for Expressive
PhD in progress: M. Junedi, Query-Update Independence Analysis under Constraints , since October 2012, C. Roisin and N. Layaïda
PhD: B. Baccot, 23 November 2012, University of Toulouse (C. Roisin as reviewer).
PhD: T.N. Luong, 12 December 2012, University of Pau (C. Roisin as reviewer).
J. Lemordant and N. Layaïda have contributed to Inriality by a short video entitled Voyager dans le temps. Paris. November 2012.
J. Lemordant, N. Layaïda and V. Quint have contributed to the Forum 4i (Innovation, Industrie, Investissement, International), Grenoble, 2012. We showcased indoor navigation with IXE.
J. Lemordant has participated in several events to popularize recent results in the areas of pedestrian navigation and mixed reality, namely the 6th European eAccessibility Forum and the State Of The Map 2012 conference.