Hardware

International CLASS Workshop on

Natural, Intelligent and Effective Interaction in Multimodal Dialogue Systems
Proceedings

Edited by Jan van Kuppevelt, Laila Dybkjær and Niels Ole Bernsen

Copenhagen, Denmark 28-29 June 2002

© 2002 Printed at University of Southern Denmark

ii

PREFACE
We are happy to present the proceedings of the International CLASS Workshop on Natural,Intelligent and Effective Interaction in Multimodal Dialogue Systems, that was held in Copenhagen, Denmark, 28-29 June 2002. The workshop was sponsored by the European CLASS project (http://www.classtech.org). CLASS was initiated on the request of the European Commission with the purpose of supporting and stimulating collaboration within and among Human Language Technology (HLT) projects, as well asbetween HLT projects and relevant projects outside Europe. The workshop was given a special format with the main purpose of bringing into focus both theoretically and practically oriented research that has given rise to innovative and challenging approaches on natural, intelligent and effective interaction in multimodal dialogue systems. In order to reach this goal we planned the workshop to containa relatively high number of invited contributions in addition to papers solicited via an open Call for Papers. We invited a group of 9 internationally leading researchers with a balanced composition of expertise on the topics of the workshop. We were especially interested in the following topics: • Multimodal Signal Processing Models for multimodal signal recognition and synthesis, includingcombinations of speech (emotional speech and meaningful intonation for speech), text, graphics, music, gesture, face and facial expression, and (embodied) animated or anthropomorphic conversational agents. • Multimodal Communication Management Dialogue management models for mixed initiative conversational and user-adaptive natural and multimodal interaction, including models for collaboration andmulti-party conversation. • Multimodal Miscommunication Management Multimodal strategies for handling or preventing miscommunication, in particular multimodal repair and correction strategies, clarification strategies for ambiguous or conflicting multimodal information, and multimodal grounding and feedback strategies. • Multimodal Interpretation and Response Planning Interpretation and responseplanning on the basis of multimodal dialogue context, including (context-semantic) models for the common representation of multimodal content, as well as innovative concepts/technologies on the relation between multimodal interpretation and generation. • Reasoning in Intelligent Multimodal Dialogue Systems Non-monotonic reasoning techniques required for intelligent interaction in various types ofmultimodal dialogue systems, including techniques needed for multimodal input interpretation, for reasoning about the user(s), and for the coordination and integration of multimodal input and output. • Choice and Coordination of Media and Modalities Diagnostic tools and technologies for choosing the appropriate media and input and output modalities for the application and task under consideration, aswell as theories and technologies for natural and effective multimodal response presentation. • Multimodal Corpora, Tools and Schemes Training corpora, test-suites and benchmarks for multimodal dialogue systems, including corpus tools and schemes for multilevel and multimodal coding and annotation. • Architectures for Multimodal Dialogue Systems New architectures for multimodal interpretation andresponse planning, including issues of reusability and portability, as well as architectures for the next generation of multi-party conversational interfaces to distributed information. • Evaluation of Multimodal Dialogue Systems Current practice and problematic issues in the standardisation of subjective and objective multimodal evaluation metrics, including evaluation models allowing for...