Section: New Results

ControllAR: Appropriation of visual Feedback on Control Surfaces

Florent Berthaut, Alex Jones

Despite the development of touchscreens, many expert systems for working with digital multimedia content, such as in music composition and performance, video editing or visual performance, still rely on control surfaces. This can be due to the accuracy and appropriateness of their sensors, the haptic feedback that they offer, and most importantly the way they can be adapted to the specific subset of gestures and tasks that users need to perform. On the other hand, visual feedback on controllers remains limited and/or fixed, preventing similar personalizing. In this paper, we propose ControllAR, a novel system that facilitates the appropriation of rich visual feedback on control surfaces through remixing of graphical user interfaces and augmented reality display. We then use our system to study current and potential appropriation of visual feedback in the case of digital musical instruments and derive guidelines for designers and developers.

Figure 1. ControllAR: (left) ControllAR is used to augment a control surface with the remixed graphical user interface of music software, (right) Visual feedback designed by electronic musicians during our study belong to three categories: mappings feedback, processes feedback and content feedback.