Section: New Results

Narrowing the gap between QoS metrics and Web QoE using Above-the-fold metrics

Participants: Diego da Hora (Telecom Paris Tech), Alemnew Sheferaw Asrese (Aalto University), Vassilis Christophides, Renata Teixeira, Dario Rossi (Telecom Paris Tech)

Page load time (PLT) is still the most common application Quality of Service (QoS) metric to estimate the Quality of Experience (QoE) of Web users. Yet, recent literature abounds with proposals for alternative metrics (e.g., Above The Fold, SpeedIndex and variants) that aim at better estimating user QoE. The main purpose of this work is thus to thoroughly investigate a mapping between established and recently proposed objective metrics and user QoE. We obtain ground truth QoE via user experiments where we collect QoS metrics over 3,000 Web accesses annotated with explicit user ratings in a scale of 1 to 5, which we make available to the community. In particular, we contrast domain expert models (such as ITU-T and IQX) fed with a single QoS metric, to models trained using our ground-truth dataset over multiple QoS metrics as features. Results of our experiments show that, albeit very simple, ex- pert models have a comparable accuracy to machine learning approaches. Furthermore, the model accuracy improves considerably when building per-page QoE models, which may raise scalability concerns as we discuss.