What You Write Is What You Get: A Novel Mixed Reality Interface

Project Members:
Syed Omer Gilani, Steven Zhou

Idea:
Children develop their understanding of the world by means of physical and social interaction. This cognitive development process can be further enriched through the use of interactive multimedia technology. We propose a novel interactive system which seamlessly combines the flexibility of the virtual world with real world handwriting to realize a tangible mixed reality learning scenario. User’s handwriting is translated in to multimedia contents (like audio, text, 3D graphics and animation) and is registered with the mixed reality environment in real-time. In addition the system can be deployed on a desktop personal computer (PC) with a normal webcam and without the need for special equipment.