This WP concerns research into measurement of behavior coordination and active support of behavior coordination in social and entertainment (human-human & human-computer) interaction.

In natural (face to face) interaction people coordinate their behavior, not just the content but also the timing (movement synchrony). Adaptation of the timing of nonverbal behavior to each other is strongly correlated with positive evaluation of the interaction and interaction partner. In human-machine interaction this mutual coordination of behavior is still largely absent. Giving interactive systems capabilities for such temporal coordination will enhance the quality of human-machine interaction. Temporal dynamics of natural human- human interaction have only recently started to be addressed in human-machine interaction research.

Research question

There are still many research questions to be answered before these mechanisms can be applied productively in interactive systems. (Automatically) measuring the degree of synchrony in interaction may yield valuable insights into many qualitative aspects of the interaction, such as the quality of the interaction, the degree of cooperation displayed by the people involved, and the effectiveness of the interaction. Interfaces that implement mechanisms of synchrony in (multimodal) interaction can improve the quality of human-computer interaction. Engaging a user in mutually coordinated multimodal interaction should make systems more robust, pleasant, and efficient to use.

The focus of activities is on:

1. Developing (and evaluating) algorithms for measuring coordination, synchrony and interaction flow in human-human and human-machine interaction.
2. Implementation of mechanisms of movement synchrony and other temporal coordination in interactive (entertainment) installations.
3. Evaluation of user perception of (the quality of) interaction with such systems The WP focuses on use cases in entertainment applications with co-located multiparty interaction, like interactive playgrounds, museum installations, or interactive installations in public spaces.

Research Questions:

1. What are suitable approaches to measuring the degree of synchrony and coordination in human-human and human-machine interaction?
2. How can one build interactive systems that intentionally implement mechanisms of synchrony and coordination?
3. What is the possible role of such mechanisms of coordinated interaction in interactive entertainment installations?