FYP2009 MXRToolkit for Mobile Phones

Project Member: Benjamin Ang

Idea:

This project involves the porting of MXRToolkit from the desktop platform to the mobile platform. Targeted platform is Windows Mobile 5 due to the similarities in the APIs used. Project direction is to fully port all features of MXRToolkit to the mobile platform and develop a sample application to showcase the capabilities of the ported toolki

FYP2009 Musical Computing on Multi-touch Table-top

Project Member: Ong JunJie

Idea:

The introduction of the multi-touch music tabletop has created another dimension in musical computing where users are able to compose and play music by interacting with the table interface through sense of touch, or through manipulation of tangible objects such as fiducial markers on the tabletop. However, playing and composing music without any music background is tedious.

This project aims to explore any creative and interactive alternatives to compose or play music in an intuitive manner on the multi-touch table, which is suitable for musicians and also beginners with no music background. It also tries to express music and motion in a graphical representation rather than the traditional Common Music Notation which is commonly used in writing music.

FYP2008 TangiCube: Introducing Bi-Directional Interactivity with an Autonomous Agent in a Tangible Mixed Reality Environment

Project Members:Derek Tan, Zhou ZhiYing and Yuta Nakayama

Idea:
Abstract:

TangiCube introduces bi-directional interactivity with a virtual autonomous agent in a tangible mixed reality environment. While the virtual agent interacts with the user, the user is able to react to the agent with the help of physical output modality. Likewise, the user can interact with the virtual agent and the agent behaves and reacts as if it has a physical embodiment. This two-way interaction process is mediated by a tangible hardware cube. A user study conducted showed how the introduction of bi-directional interaction, especially with the haptics from the cube and physical embodiment of the virtual agent, improves the overall experience within the augmented space, as well as narrows the reality-virtuality gap in the user’s perception of the augmentation. This opens up new form of experiences and applications in the tangible mixed reality domain, allowing it to utilize more of its affordances to support and enhance interactions in its own unique ways.

For more details, please download the paper (paper currently being vet by Dr Zhou)

FYP2008 Realistic Virtual Lighting and Shadow are crucial to the realism of the Mixed Reality scene

Project Members:
Yeoh ZhenTing Ryan Christopher

Idea:
In Mixed Reality (MR) applications, realistic virtual lighting and shadow are crucial to the realism of the Mixed Reality scene. Current methods of recovering a scene’s illuminant distribution and virtually lighting and shadowing the MR environment require extensive calibration techniques and special equipment, such as multiple cameras, fish-eye lenses and high dynamic range (HDR) cameras. Other methods might have less stringent equipment requirements but are unable to achieve real-time frame rates. This project aims to achieve photorealistic virtual lighting and shadow for virtual objects by heuristically recovering the real world’s illuminant distribution, aligning the virtual light sources with real-world light sources in 3D space, and finally lighting virtual elements in the scene with these virtual light sources. The method proposed is to place a known object , such as a small needle, in the scene at a known location (referenced by a marker) and, via vision-based tracking and observation of its shadow(s), recover the positions of any real-world light source(s) illuminating the scene. It is hoped that this will allow subsequent applications to achieve realistic interactive lighting and shadow in MR and allow virtual objects to respond to interactive changes in real-world lighting, without complex equipment.