Section: New Results

Accessing Information

Ontological modeling of human reading experience

Participants : Guillaume Gravier, Pascale Sébillot.

Done as part of the JPI CH READ-IT projects, in collaboration with Open University (UK) and Université Le Mans (FR)

Diaries, correspondence and authors' libraries provide important evidence into the evolution of ideas and society. Studying these phenomena is connected to understanding changes of perspective and values. Within the framework of the READ-IT project, we developed an ontological data approach modelling changes in the contents of diaries, correspondence and authors' libraries related to reading. By considering these three types of sources, we designed a conceptual data model to permit the study and increase the usability of sources containing evidence of reading experiences, highlighting common challenges and patterns related to changes to readers and to the medium of reading when confronting historical events [36], [8].

Integration of Exploration and Search: A Case Study of the $M^3$ Model

Participants : Snorri Gíslason [IT Univ. Copenhagen] , Björn Þór Jónsson [IT Univ. Copenhagen] , Laurent Amsaleg.

Effective support for multimedia analytics applications requires exploration and search to be integrated seamlessly into a single interaction model. Media metadata can be seen as defining a multidimensional media space, casting multimedia analytics tasks as exploration, manipulation and augmentation of that space. We present an initial case study of integrating exploration and search within this multidimensional media space [11]. We extend the $M^3$ model, initially proposed as a pure exploration tool, and show that it can be elegantly extended to allow searching within an exploration context and exploring within a search context. We then evaluate the suitability of relational database management systems, as representatives of today’s data management technologies, for implementing the extended $M^3$ model. Based on our results, we finally propose some research directions for scalability of multimedia analytics.

Exquisitor: Breaking the Interaction Barrier for Exploration of 100 Million Images

Participants : Hanna Ragnarsdóttir [Reykjavik University] , Þórhildur Þorleiksdóttir [Reykjavik University] , Omar Shahbaz Khan [IT Univ. Copenhagen] , Björn Þór Jónsson [IT Univ. Copenhagen] , Gylfi Þór Gudmundsson [School of Computer Science, Reykjavik] , Jan Zahálka [bohem.ai] , Stevan Rudinac [University of Amsterdam] , Laurent Amsaleg, Marcel Worring [University of Amsterdam] .

We present Exquisitor, a media explorer capable of learning user preferences in real-time during interactions with the 99.2 million images of YFCC100M. Exquisitor owes its efficiency to innovations in data representation, compression, and indexing. Exquisitor can complete each interaction round, including learning preferences and presenting the most relevant results, in less than 30 ms using only a single CPU core and modest RAM. In short, Exquisitor can bring large-scale interactive learning to standard desktops and laptops, and even high-end mobile devices [16].