FYP2008 Realistic Virtual Lighting and Shadow are crucial to the realism of the Mixed Reality scene

Project Members:
Yeoh ZhenTing Ryan Christopher

Idea:
In Mixed Reality (MR) applications, realistic virtual lighting and shadow are crucial to the realism of the Mixed Reality scene. Current methods of recovering a scene’s illuminant distribution and virtually lighting and shadowing the MR environment require extensive calibration techniques and special equipment, such as multiple cameras, fish-eye lenses and high dynamic range (HDR) cameras. Other methods might have less stringent equipment requirements but are unable to achieve real-time frame rates. This project aims to achieve photorealistic virtual lighting and shadow for virtual objects by heuristically recovering the real world’s illuminant distribution, aligning the virtual light sources with real-world light sources in 3D space, and finally lighting virtual elements in the scene with these virtual light sources. The method proposed is to place a known object , such as a small needle, in the scene at a known location (referenced by a marker) and, via vision-based tracking and observation of its shadow(s), recover the positions of any real-world light source(s) illuminating the scene. It is hoped that this will allow subsequent applications to achieve realistic interactive lighting and shadow in MR and allow virtual objects to respond to interactive changes in real-world lighting, without complex equipment.