Andrea Corradini, University of Potsdam
In this intensive introduction course I will present an approach to natural multimodal interaction for immersive environments and embodied conversational characters. In the first case, I will outline a multimodal architecture and provide examples from a preliminary 3D multimodal testbed, which was developed to explore a way to disambiguate the user's intent by fusing symbolic and statistical information from a set of 3D gesture and speech agents. In the embodied conversational scenario, I will be focusing on architectural requirements and on the explanation of existing prototype architectures.
An introductory overview of the following topics will be provided:
- gesture recognition
- natural language processing
- multimodal systems
- spoken dialogue systems
- multimodality fusion
- embodied conversational agents
- augmented and reality reality
The focus of the classes is fairly broad, covering theory, empirical evaluation, design, and applications. At the end of the course, the student
should have a good appreciation of the key research topics in this area, methodology used by researchers in this field, research results to date, and open research problems.