Section: Research Program

Research Program

Our research programs is structured into three complementary research axis : models, languages and systems, allowing us to develop our multi-disciplinary approach while validating each progress in the related specific fields of computer science ranging among computer music, multi-modal system design, reactive and real-time programing, typed functional programming, formal languages, graph representation theory, applied algebra, logic in computer science, etc.

Models

Inverse semigroup theory has recently been shown [13], [7], [12] [20] to unify most string-based, tree-based or even graph-based modeling approaches. It thus provides a consistent and robust mathematical framework to model the sequential, parallel and reactive aspects of temporal media. Developing the mathematical foundations of our proposal amounts to:

• studying the combinatorial and algorithmic properties of the emerging algebra-based model of structured temporal media,

• developing formal techniques and tools for expressing and verifying properties of temporal media programs especially with a view towards capturing temporal media programing by constraint satisfaction approaches,

• deriving from the known generators of these models adequate sets of application-oriented modeling functions.

Languages

Functional programming is the key link between well-defined mathematical structures and their computerized realizations. Based on functional programming frameworks such as Haskell (See  [31] for an historical presentation of the Haskell programming language.), we are prototyping a Domain Specific Language (DSL) [10] [15] dedicated to the programming of interactive temporal media programming. In this research axis, we aim more specifically at

• designing a robust and modular software architecture that allows to reuse existing pieces of software as well as simply combining them together with new ones,

• defining and implementing a DSL for programming interactive multimedia systems via a simple algebra-based high-level and multi-scale control and combination layer,

• finding the right balance between generic views of temporal media when seen as abstract temporal frames and their specializations when representing concrete gestures, sound, audio, videos, animations, etc.

Systems

Multi-modal interactive systems gather various techniques to capture and analyze gestures, and to combine, transform and produce temporal media. Through regular experiments in collaboration with artists, we also aim at assessing, refining and extending the applicability of our proposal by:

• developing a robust and mathematically well-founded representation of systems and of their behaviors, both programmatic and visual,

• developing and evaluating the adequacy of the GUI induced by this representation when used by artists,

• relating the new models with more classical models of music formalisms and, beyond, other temporal media such as animations, videos, etc.