Interactive Multimedia Lab

Research Activities

Our research in Interactive & Digital Media is mainly in the areas of vision-based human-computer interfaces, human-centric image analysis, multimedia processing and perception modeling. Some of the questions we are investigating are:

How can user interfaces be made more intuitive through the visual analysis of user movements, gestures and intentions?
How can better understanding and interpretation of the user and his actions enhance entertainment, education and security applications?
How can objects from the real world be seamlessly combined with related visual information and presented to the user in the form of an augmented reality?
How well is this augmented/mixed reality perceived by the user, and which aspects is he/she most sensitive to?
How can multimedia information be presented, stored, and managed in the optimal user-centric way?

You can find more information on specific research projects below.

Mixed-Reality Interfaces

Input functionality via keyboard and mouse is rather inefficient for many applications, for example managing multimedia information. One of our goals is to develop systems for more natural management of files and in particular photos or videos through a mixed-reality desktop. The aim is to facilitate the handling of such data, and to interface naturally with real-world objects such as the printer or a photo album using vision-based gesture analysis and tracking.

Camera Motion-Based User Interfaces

The user interface of mobile devices is severely limited due to the small form factor of these devices. The goal of this research is to design novel, intuitive UI controls based on the detection of 3D motion of the device using the integrated camera. These movements can then be used to control applications on the device. Application areas of particular interest multi-user mobile gaming and map navigation.

Multi-View Scene Analysis

A basic problem in computer vision is to reconstruct a real-world scene given several distinct views. Multiple cameras are placed in a room to take pictures of a scene, and their images are then combined into a 3D reconstruction of the objects in the scene. The purpose of this research is to develop algorithms for multi-view camera setup and scene analysis. Applications include 3D scene modeling, object tracking and motion capture, which can be of great use for immersive environments.

Perception and Attention

One of the challenges of perception is to understand where people focus their attention in an image or video. In still images of natural scenes, viewers tend to fixate on locations with high local image contrast. However, little is known about the features that attract attention when people watch video. We are studying the correlation of eye movements with both spatial and temporal features in order to understand, capture and predict visual attention. Applications of this work are found in human-computer interaction, assistive display technologies, video coding, video retrieval as well as surveillance.