In Situ is a collaborative project between INRIA (unité Futurs) and the Laboratoire de Recherche en Informatique of the University Paris-Sud within the framework of the PCRI (Pôle Commun de Recherche en Informatique).
In Situ is a collaborative project between INRIA (unité Futurs) and the Laboratoire de Recherche en Informatique of the University Paris-Sud within the framework of the PCRI (Pôle Commun de Recherche en Informatique).
The In Situ projet develops novel interaction techniques as well as new tools to develop these techniques, and new methods to control the design process of interactive systems. The goal is to develop situated interfaces, i.e. interfaces that are adapted (or adaptable) to their contexts of use by taking advantage of the complementary aspects of humans and computers. In the long run, the project seeks to create a new generation of interactive environments as an alternative to the current generation of desktop environments.
Our research is organized in three main research topics:
The project addresses novel interaction techniques such as multi-scale (or zoomable) interfaces, interactive information visualization, bimanual interaction, and the use of video and non-speech audio, as well as the integration of these techniques within a consistent environment. The project also addresses augmented reality, i.e. the integration of computation and interaction within physical objects and environments.
Participatory design involves users at all stages of the design process. It turns users into innovators and helps understand the situated aspect of the users' activity. The project develops participatory design methods and techniques that make the role of context explicit in the design process.
Novel interaction techniques and interaction paradigms require the development of specific tools to facilitate their integration and adoption. The project studies component-based architectures where components that implement, e.g. interaction techniques, may be added, removed or substituted dynamically.
Researchers have competing views of what constitutes research. Much of computer science relies on formal theory, which, like mathematics, is evaluated with respect to its internal consistency. The social sciences are based more on descriptive theory, attempting to explain observed behavior, without necessarily being able to predict it. The natural sciences seek predictive theory, using quantitative laws and models to not only explain, but also to anticipate and control naturally-occuring phenomena. We are exploring a fourth approach, which we call generative theory, which builds upon existing knowledge in order to create new categories of artifacts and explore their characterstics.
Human Computer Interaction is a multi-disciplinary field that borrows from multiple disciplines, using social science methods to study and understand human behavior with respect to interactive systems, engineering methods to construct the technical foundations and working prototyes, and design methods, to create and explore novel ideas.
Our goal is to produce prototypes, research methods and software tools that facilitate the design, development and evaluation of interactive systems.
There are several potential application domains for In Situ. To validate our goals, we have selected domains with the following features:
multiple validation criteria: increasing productivity, security or comfort;
diverse users: professionals, specialists or non-professionals;
diverse environments: classical desktops, non-standard (e.g. airplane cockpits) or mobile.
We have selected the following application domains:
in cooperation with the French Research Center of Air-Traffic Control (CENA) and EuroControl;
in cooperation with Institut Pasteur;
in cooperation with the University of Maryland, USA, the CID-KTH in Sweden (see the Interliving project), the Royal College of Art in London, England and France Telecom R&D, Thomson Multimedia and Philips.
VideoSpace uses of video, rather than the technologies
it requires. In this perspective, it is not focused on performance
or reliability issues, but rather on the ability to support rapid
prototyping and incremental development of video
applications. This approach contrasts with many of the research
themes usually associated to video in the Multimedia or Network
communities such as compression, transport or
synchronization. VideoSpace is not aimed at these topics. It is
rather intended to help HCI and CSCW researchers who want to
explore new uses of the images.
VideoSpace is designed after Alan Kay's famous saying: ``simple things should be simple, complex things should be possible''. It provides users and developers with a set of basic tools and a C++ class library that make it easy to integrate image streams within existing or new documents and applications. The tools, for example, allow users to display image streams in HTML documents in place of ordinary static images (see Fig. ) or to embed these streams into existing X Window applications. Creating a video link with the library requires only a few lines of code; managing multiple sources and including video processing is not much more complicated. Since the image streams managed by videoSpace often involve live video of people, the toolkit also provides a flexible mechanism that allows users to monitor and control access to their own image.
Source code for videoSpace compiles on Linux and Mac OS X and is
freely available under the GNU Lesser General Public License
(LGPL). For more information, see
Ametistapseudo-windows that are randomly-colored rectangles;
placeholders that display a fixed image or a video stream
and live windows of X Window applications, through a redirection
mechanism. Pseudo-windows can be used for low-fidelity prototyping
in the early stages of the exploration of a new window management
technique. Placeholders help getting a better idea of the
envisioned technique by displaying snapshots or movies of real
applications. Finally, live X windows can be used for
high-fidelity prototyping and evaluation of the technique.
Ametista uses OpenGL to display windows. This library offers a
rich graphics model well adapted to the exploration of new window
management techniques. Alpha blending, for example, makes it easy
to create translucent objects and shadows. Scaling, rotation and
translation can also be used with a perspective projection to
position windows in 2D
Source code for Ametista compiles on Linux and Mac OS X and is
freely available under the GNU Lesser General Public License
(LGPL). For more information, see
SVGL is a toolkit designed to display W3C SVG documents using
accelerated graphics (see Fig. ). It relies on the
OpenGL API and translates high-level SVG constructs into hardware
accelerated OpenGL primitives. Rendering time is improved by
factors of 50 or more compared to other SVG implementations such
as Batik
SVGL is used by the renderer of the Indigo project (see section ). It can also be used as a C++ library for interactive graphics for more classical applications. It provides a simpler API than the standard W3C DOM for SVG.
SVGL runs under Linux, MacOS X and Windows and is distributed with
an open source license at
The InfoVis Toolkit is a Interactive Graphics Toolkit written in Java to ease the development of Information Visualization applications and components.
The main characteristics of the InfoVis Toolkit are:
The base data structure is a table of columns. Columns contain objects of homogeneous types, such as integers or strings. Trees and Graphs are derived from Tables.
Using homogeneous columns instead of compound types dramatically improves the memory required to store large tables, trees or graphs, and in general the time to manage them.
Interactive filtering (a.k.a. dynamic queries) can be performed with the same control objects and components regardless of the data structure, simplifying the reuse of existing components and the design of generic ones.
the InfoVis Toolkit can use accelerated graphics
provided by
Agile2D
the InfoVis Toolkit is meant to incorporate new information visualization techniques and is distributed with the full source and a very liberal license. It can be used for student projects, research projects or commercial products.
The InfoVis Toolkit, as of version 0.5, implements seven (7) types of visualization (Fig. ): Time Series and Scatter Plots for tables, Node-Link diagrams, Icicle trees and Treemaps for trees, Adjacency Matrices and Node-Link diagrams for graphs.
The toolkit is used for teaching Information Visualization (DEA I3 and DESS SCHM and II at University of Paris-Sud, Infovis course at Virginia Tech). It is also used in several contracts of the group such as OADymPPaC and Micromegas (see section ).
More information can be found at
The rfid library is a set of C functions to dialog with RFID devices. A RFID transponder is a wireless memory area made of a chip and an antenna. A RFID reader creates a magnetic field that is used as a power source by RFID transponders and as a carrier for communication. A program can use the rfid library to be notified when a RFID transponder enters or leaves the range of a RFID reader. When a RFID transponder is in range, data can be read from its memory or written into it.
The library supports all ISO-15693-3 conformant RFID transponders,
is available under the GPL license at the following location:
The interLiving (see section ) project has
generated several new design methods Interactive Thread, technology probes
We call this type of interactive communication system a
communication appliance, to capture the idea of a
single-function, very easy to use method of staying in touch over
time and space. Although other researchers have prototyped
communication appliances, none have ever left the laboratory. Yet
it is clear from our work with families, that such systems are
highly desirable.
The key finding of the interLiving project has been the
identification of the critical missing element: currently, we do
not have an easy way to create and manage the corresponding
networks that underlie communication appliances. So we developed
FamilyNet
We are currently in the process of patenting the hardware interface to FamilyNet (based on cards and RFID tags) and plan to release the software under a free-software license, to maximize potential use. Both Philips and Intel have expressed interest in the concept and we plan to explore further collaborations with industry in 2004.
Graphical user interfaces haven't quite changed since the creation of the Desktop metaphor. User interaction hasn't improved as nearly as quickly as the increase in computing power. New interactive applications don't take advantage of new interaction paradigms or interaction techniques, the latter remaining too often at the prototype stage. We are exploring new programming environment and systems able to quickly integrate the new techniques or interaction paradigms as they are discovered.
Our activity focuses on creating a new software architecture that will enable new interactive applications to profit from richer graphics and novel interaction paradigms without changing the non-interactive part of the application. We are coordinating the RNTL Indigo project with ILOG, the W3C and the CENA in Toulouse to create new mechanisms based on technologies standardized by the W3C for the infrastructure and from our experience in toolkits for the separation of application and their interaction and presentation.
Graphical user interfaces (GUIs) are based on a small set of interaction techniques, which rely heavily on two elementary actions: pointing a target on the screen, e.g. an icon or button, and navigating to a non-visible part of the information space, e.g. by scrolling or zooming.
We are working on improving pointing and navigation performance in
GUIs. Indeed, the performance of pointing on a screen is similar
to that of pointing in the physical world, and it should be
possible to take advantage of the computing power to get a
significant advantage when pointing in an information world. The
major theoretical tool to study pointing performance is Fitts'
law
We have explored a technique called target
expansion
We have developed two techniques that look for more promising. The
first one is semantic pointing
A target with a small visual size and a large motor size will display little information but will be easy to select, which makes it appropriate for, e.g., buttons or links in a Web Page. A target with a large visual size and a small motor size will display more information but will be hard to select, which is appropriate for non-interactive informative labels, for example. A controlled experiment has confirmed that the index of difficulty that best predicts pointing time is the one that uses the motor size rather than the visual size. A prototype application shows how this technique applies to standard GUIs, e.g. scrollbars, dialog boxes, menus and web pages.
The second technique is called vector
pointing
We have also persued our work on multiscale navigation, i.e.
navigation of an information world that can be zoomed in or out at
any scale (also called Zoomable User Interface or ZUI). Following
up on our work that showed that Fitts' law applies to tasks with
an extremely high level of difficulty (30 or more, i.e. a ratio of
230 between target size and target distance) we have developed a
theoretical model to explain this result
The complexity of an interaction technique for a given task,
i.e. a given interaction sequence, measures the cost of
the actions relative to the size of the task when using this
interaction technique. The work on pointing and navigation
studies the limit performance of human subjects in such tasks.
However it is often difficult to observe such performance when the
technique is used in the context of a real application. In order
to better understand how interaction techniques behave in context,
we are developing a model and associated tool to describe
interaction techniques and predict their comparative performance
for multiple tasks representative of different interaction
contexts. The model, called CIS
Most computer software for managing on-line documents assumes that
all user interaction takes place via a keyboard, mouse and screen
and that documents exist only in their on-line form. Yet users
often write by hand and annotate printed documents, forcing them
to juggle on-line and off-line versions. We are exploring the
concept of ``interactive paper''
We have been working for several years with research biologists at
the Pasteur Institute, observing their use of laboratory
notebooks
This year, we began working with a new technology, the Anoto
pen
In the past two months, we began two new projects in a related area, which involves working with historical manuscripts. For historians, the physical characteristics of these manuscripts are often as important as the information they contain. They need to interact with the actual physical object (often without touching it) while simultaneously entering information into a computer, for subsequent analysis. We are interested in exploring new techniques for linking physical and on-line documents, as well as providing novel methods of visualizing their contents. We are also working with archivists, who are interested in identifying and visualizing the evolution of related documents over time. We ran an initial workshop with historians and archivists in November, and will continue collaborating closely with them next year.
Creating new information visualization techniques using traditional GUI toolkits is long and difficult. We have designed a new toolkit that allows us to experiment with new techniques in a much simpler a general way than before: the Infovis Toolkit (see section ).
The InfoVis Toolkit
We are currently exploring new techniques for the visualization of large graphs (for constraint-based programs, large social networks, software engineering), time-based data, exploration and management of familiar datasets. We are also exploring new interaction techniques to filter large datasets since existing techniques do not scale well.
See
One of the advantages of video over audio for mediated communication is the ability to transmit non-verbal information. Physical proximity between people is a language for non-verbal communication that we all employ everyday, although we are barely aware of it. Yet, existing systems for video-mediated communication fail to fully take into account these proxemics aspects of communication.
The MirrorSpace
project
MirrorSpace units (see Fig. ) combine a digital camera, a flat screen and a proximity sensor. As we aim to support intimate forms of communication, it felt important to us that people could actually look into each other's eyes and possibly merge their portraits into one, so the camera was placed right in the middle of the screen. This setup allows participants to come very close to the camera while still being able to see the remote people and interact with them.
The proximity sensor that measures the distance to the closest object or person in front of it. This distance is used by MirrorSpace software to alter the remote images displayed, and possibly the local one. A blur filter is applied on the images to visually express a distance computed from the local and remote sensor values. Blurring distant objects and people in MirrorSpace allows one to perceive their movement or passing with a minimum involvement. It also offers a simple way of initiating or avoiding a change to a more engaged form of communication by simply moving closer or further away.
This work started as part of the interLiving project (see above).
MirrorSpace was first exhibited in a public setting in February
2003, at Jeune Création, a contemporary art exhibition in
Paris. It was then exhibited at Mains d'Oeuvres
(Saint-Ouen) (May 2003), Pas vu, pas pris (July 2003) and
at the Interactive Design exhibit in the Pompidou Center (Paris
December 2003 - February 2004). See
The InterLiving project is a multi-disciplinary research project
that uses social science, computer science and design methods to
develop communication technologies for families. The project has
multi-year relationships with six multi-household,
multi-generational families in two EU countries who are active
partners in testing novel research methods and prototyping ideas.
The project investigates the communication patterns and needs of
distributed families, especially peripheral communication;
produces novel research methods for working with families, with an
emphasis on collaborative design; and creates interactive ``shared
surfaces'' to help family members stay in touch and facilitate
everyday interaction
The OADymPPaC project aims at providing tools for the dynamic analysis and debugging of constraint-based programs. It is funded by the RNTL French network. Partners include INRIA (from the Rocquencourt Contraintes project and Futurs IN-SITU project), Ecole des Mines de Nantes, University of Orleans, IRISA in Rennes, the ILOG company and the Cosytec company. The project started in 2000 and will end in 2004
Our role in the project is to provide tools for visualizing
dynamically the execution of constraint-based programs. We have
designed several tools to read a generic trace produced by
constraint-based programs and translate it into generic and
specific visualization components. Our current work is focused on
visualizing large dynamic graphs that are built and maintained by
constraint solvers. Two graphs are important: the
constraint/variable graph available on all solvers and the graph
of ``explanations'' produced by recent solvers such as
PaLM
Instead of focusing on node-link diagrams for representing graphs,
we are exploring the use of adjacency matrices to achieve higher
graph density and real-time performances. We are currently
investigating three directions: readability of adjacency matrices
compared to node-link diagrams, clustering techniques to help
aggregating large graphs and interaction techniques to explore the
evolution of large
graphs
The goal of the INDIGO project is to design, develop and validate a distributed software architecture for the development of a new generation of interactive systems characterized by the following requirements:
visualize and interact with more and more complex and dynamic information;
adapt to a more and more divers set of platforms (mobile phone, PDA, PC, immersive VR, etc.) and input devices;
support cooperative work, in particular real-time sharing and editing of information across multiple sites.
The INDIGO software architecture is based on a high-level communication protocol between Conceptual Objects (CO) servers and Rendering and Interaction (RI) servers. This architecture is similar to that of the popular X Window System, with the important difference that the RI servers will implement higher-level models for displaying data and interacting with it than the X Server, and CO servers will therefore manage interaction and visualization at a higher level of abstraction than X clients. This has several advantages, including the following: a higher-level protocol requires less bandwidth, novel interaction techniques can be added to the IR servers transparently, CO servers can use multiple RI servers simultaneously to support collaborative work, and CO servers are independent from the end-user platform.
The partners of the project are In Situ (coordinator, RI server, protocol), the French company ILOG (CO server, protocol), the W3C (protocol) and CENA (requirements and sample applications), the French research center for air traffic control. In order to foster the adoption of the INDIGO architecture, the protocol will be publicly available and submitted to the W3C, and reference implementations of the servers will be available under an open source license.
Over the last 12 months, we have designed the first version of the
protocol and sample implementations of the CO and RI servers. In
particular, rendering is based on an implementation of SVG (Scalable
Vector Graphics) with the graphics API OpenGL and interaction is
based on the instrumental interaction model
Over the twenty years that elapsed since the Xerox Star, the first personal computer with a graphical user interface ever commercialized, the amount of information stored on our computers has been subject to a thousand-fold increase. The mass of electronic data we have nowadays at our disposal in both our professional and personal lives is such that the risk of being overwhelmed with information – even with the information we have stored ourselves – has become a serious concern.
Micromegas involves four teams that share their experience in the fields of human movement and cognition, human-computer interaction, information visualization, and multi-modal interaction: LMP in Marseille, In Situ and MerLIn at INRIA, and Institut Pasteur in Paris. One essential feature of our approach is an emphasis on multi-scale interaction. Complexity, we believe, cannot be mastered by the human unless it can be tackled hierarchically: the information contained in a huge set of files or an electronic world atlas cannot be retrieved and utilized unless one can easily manipulate the level of granularity at which one wishes to interact with the data, from the most global level (a view of the subsuming folders, a general view of the planet) to the most local (the contents of a file, a detailed city map). The cognitive capabilities of humans, however, are too limited to encompass such a scope, and hence the challenge is to understand how they spontaneously vary the scale factor and, in the context of computerized information, to help them do so.
Micromegas deliberately focuses on the case of familiar
data – both professional and personal – that have been stored by
the users themselves, who not only save their own production but
also collect external data. Thus, we are more concerned with
personal hard disks than the Web. Still, we are facing huge data
bases (on the order of several tens of gigabytes) whose size,
which keeps on growing exponentially, makes the multi-scale
approach compulsory.
The project is organized in three sub-projects, designed to foster collaboration between the participants and structure the research effort.
Sub-project 1 addresses the fundamental aspects of multiscale navigation. Through an experimental approach, it applies the principles of the ecological approach to visual perception from psychologist J.J. Gibson to design and evaluate novel navigation techniques for multiscale information worlds.
Sub-project 2 addresses visualization techniques. Many visualization techniques have been developed over the years, however few address the actual presentation of large data sets. In many cases, data is aggregated before being presented to the user. Such aggregation essentially supports a hierarchical view of the data, while we are interested in richer representations that support multiscale navigation, transformation between views, and efficient use of the display surface.
Sub-project 3 consists of two case studies. The first one covers management of personal file systems, a task facing almost every computer user and not well supported by current desktop interfaces. The second study covers the management of experimental data by biologists at the Institut Pasteur. Rather than focusing on the data used and produced by an experiment, it addresses the wider picture of sense-making that is part of the scientific process of designing, running and analyzing series of experiments.
The expected results of the project include fundamental results on multiscale visualization and navigation, practical tools to create multiscale interfaces, guidelines and recommendations to design multiscale applications, prototype systems for file management and laboratory notebooks, and, in general, a deeper understanding of how humans can take advantage of and interact with multiscale information worlds.
Manuscripts are special kinds of documents not well supported in the digital world. When considered as images, they cannot be used for full-text searching or indexing. When transcribed and used as textual documents, they loose all their graphic features. These features can be ever more important than the textual content. For example, laboratory notebooks contain more than text: formulas (mathematical, biological, chemical), references to experimental objects, photographs or results of printouts of various machines. Historical manuscripts can contain ornamented letters, diacritical marks, hard-to-read text portions or schemas. Literary manuscripts can be very complex graphically and convey an intimate relationship with the author lost in the textual form.
We are working on two contracts aimed at digitizing and supporting manuscripts for literary and historical purposes: ``Collaborative annotation for online manuscripts'' (IDA) and ``Publishing ancient Ethiopian manuscripts'' (Ethiopia). The IDA project is led by INRIA with two partners: the French National Library (BnF) and the Institute of Modern Textes (ITEM). The Ethiopia project is led by the French National Archives with INRIA , the Ethiopian ministry of culture and the British National Library as partners.
For both projects, we will provide our expertise on augmented
documents
Michel Beaudouin-Lafon is co-chair of the CNRS Thematic Network of Human-Computer Interaction (RTP16) representing around 200 researchers
CNRS STIC: Action Spécifique ``Plasticit2 Xes Interfaces'': Michel Beaudouin-Lafon
UID-Net: Catherine Letondal
Convivio: Wendy Mackay
6th Framework Multi-modal program reviewer: Wendy Mackay
International Journal of Human-Computer Study (formerly International Journal of Man-Machine Study, founded in 1968): Wendy Mackay (co-editor in Chief)
ACM Transactions on Computer-Human Interaction (TOCHI): Wendy Mackay (associate Chair)
Revue de l'Interaction Homme-Machine (RIHM): Michel Beaudouin-Lafon, Wendy Mackay
Revue I3: Michel Beaudouin-Lafon (member of the board)
CSCW Journal: Michel Beaudouin-Lafon (member of the advisory board)
HCI Journal: Nicolas Roussel
Software Practice and Experience (SPE): Jean-Daniel Fekete
IEEE Transactions on Graphics and Visualization: Jean-Daniel Fekete
Document Numérique, Hermès, France: Jean-Daniel Fekete (member of the board of the special issue on preservation and cultural heritage)
Pervasive Computing: Wendy Mackay
ACM/Transactions on Computer-Human Interaction: Wendy Mackay
IEEE Software: Wendy Mackay
Unversity of British Columbia Distringuished Lecture Series, Canada: Wendy Mackay
Oregon Research Institute Distringuished Lecture Series, USA: Wendy Mackay
Philips Human-Computer Interaction Conference, Netherlands: Wendy Mackay
Interaction Homme-Machine Conference, France: Wendy Mackay
Seminar on Safety-Critical systems, Denmark: Wendy Mackay
ACM UIST 2004: Michel Beaudouin-Lafon
ACM AVI'04: Wendy Mackay (Associate Chair)
ACM CHI 2003-2004: Michel Beaudouin-Lafon (Associate Chair for 2004)
IHM 2004: Michel Beaudouin-Lafon (vice-chair), Jean-Daniel Fekete (proceedings chair)
Graphics Interface 2004: Jean-Daniel Fekete
ECSCW 2003: Michel Beaudouin-Lafon
CADUI 2003: Jean-Daniel Fekete
CIDE 06 (French conference on electronic documents): Jean-Daniel Fekete
IHM 2003: Jean-Daniel Fekete
IEEE Symposium on Information Visualization 2003: Jean-Daniel Fekete is co-chair of the Infovis 2003 Contest and will be co-chair of the Infovis 2004 contest. He launched this new submission category in 2003.
ACM CHI 2004: Nicolas Roussel
ACM IUI 2004: Jean-Daniel Fekete
ACM DIS'04: Wendy Mackay
AVI'04: Wendy Mackay
ACM DUX 2003: Wendy Mackay
ACM UIST 2003: Michel Beaudouin-Lafon, Wendy Mackay
UBICOMP 2003: Michel Beaudouin-Lafon
Tales of the Disappearing Computer 2003: Wendy Mackay
ACM CHI 2003: Jean-Daniel Fekete, Wendy Mackay, Nicolas Roussel, Michel Beaudouin-Lafon
IHM 2003: Jean-Daniel Fekete, Wendy Mackay
Infovis 2003: Wendy Mackay, Jean-Daniel Fekete
IFIP Interact 2003: Nicolas Roussel
BR-CHI CLIHC 2003: Nicolas Roussel
GDR I3: Jean-Daniel Fekete (Head of WG 2.2)
AFIHM (French speaking HCI asssociation): Michel Beaudouin-Lafon, Jean-Daniel Fekete, Nicolas Roussel, Executive Committee members
ACM SIGCHI: Wendy Mackay Vice-Chair for Publications and Chair or SIGCHI Publications Board
ACM: Michel Beaudouin-Lafon member at large of ACM Council and member of ACM Publications Board
Michaël Baron, PhD Thesis ``Vers une approche sûre du développement des Interfaces Homme-Machine'', Jean-Daniel Fekete jury member
Judith Aston, Ph.D. Thesis ``Interactive Multimedia: an Investigation into its Potential for Communicating Ideas and Arguments'', Wendy Mackay, jury member