Projects & Awards

Shared Design Space - Version 2

Project Members:
Daniel Leithinger, Steven Zhou

External Collaborator:
Michael Haller, Media Interaction Lab, Upper Austria University of Applied Sciences, http://www.mi-lab.org

Link to Shared Design Space - Version 1,http://mi-lab.org/projects/office-of-tomorrow/#sds

Idea:
SDS stands for Shared Design Space 1. The SDS system is designed to support creative group discussions and the simple exchange of data. In SDS Version 2, we emphasized on the usage of mobile devices as an extension of personal workspace, together with the public workspace shared on the digital table. Users can send files from their mobile phones to a digital tabletop display. On the tabletop, content can be moved, scaled, rotated and annotated using digital pens. Multiple users can work on the tabletop simultaneously. As group members stand around the tabletop display during the meeting, team discussion and interaction is encouraged (see below). Devices like digital pens enable a natural and easy input and demand little attention during group discussion. Mobile devices allow the input of the content files from local designers and even remote mobile users.

On a the tabletop, photos, videos and 3D content can be arranged and annotated.
	Assets on the tabletop display can be moved, scaled and rotated using the digital pens.

Back to top

Architectural Fly-Thru™

Idea:
Engineers, designers and architects now have a tool in which they can see and navigate through a building before the construction has even begun. The Architectural Fly-Thru takes blue-prints and renders them into full scale 3D models on the computer.

Once the rendering is complete, the user can then use a 'virtual-camera' to navigate through the building in a manner than a conventional mouse would find cumbersome. By tilting the camera like you would in real life in all directions, the 'virtual-camera' will guide the user through the building like he or she is actually walking though it.

Back to top

Mobile AR

Project Members:
Karthik Rangaswamy, Stefan Winkler, Steven Zhou

Idea:
Our research is an effort to develop new alternatives to controlling the Mobile phone with the non-ergonomic keys on the device. Using the phone’s in-built camera and state-of-the-art computer vision algorithms, we estimate its pose in 3D space (position and orientation). Our research path has taken us from utilizing reference-based tracking (markers, face etc.) to developing reference-less visual tracking for pose estimation without scene constraints.
The applications designed to test our new motion-based interfaces are explained below:

Car Racing: To steer a car in the game, the user turns the phone like a steering wheel. Moving it back and forth controls the speed.

Map Navigation: The phone acts as a magnifying glass over a virtual map. Moving the phone forwards and backwards zooms through different resolutions of the map, and moving the phone along the map plane scrolls it in any direction.

Back to top

What You Write Is What You Get: A Novel Mixed Reality Interface

Project Members:
Syed Omer Gilani, Steven Zhou

Idea:
Children develop their understanding of the world by means of physical and social interaction. This cognitive development process can be further enriched through the use of interactive multimedia technology. We propose a novel interactive system which seamlessly combines the flexibility of the virtual world with real world handwriting to realize a tangible mixed reality learning scenario. User’s handwriting is translated in to multimedia contents (like audio, text, 3D graphics and animation) and is registered with the mixed reality environment in real-time. In addition the system can be deployed on a desktop personal computer (PC) with a normal webcam and without the need for special equipment.

Back to top

 

Field Analysis of Open Source OCR Engine Using Mobile Devices

Project Members:
Syed Omer Gilani, Steven Zhou

Idea:
Mobile phones have evolved from passive one-to-one communication device to powerful handheld computing device. Today most new mobile phones are capable of capturing images, recording video, and browsing internet and do much more. Exciting new social applications are emerging on mobile landscape, like, business card readers, sing detectors and translators. These applications help people quickly gather the information in digital format and interpret them without the need of carrying laptops or tablet PCs. However with all these advancements we find very few open source software available for mobile phones. For instance currently there are many open source OCR engines for desktop platform but, to our knowledge, none are available on mobile platform. Keeping this in perspective we propose a complete text detection and recognition system with speech synthesis ability, using existing desktop technology. In this work we developed a complete OCR framework with subsystems from open source desktop community. This includes a popular open source OCR engine named Tesseract for text detection & recognition and Flite speech synthesis module, for adding text-to-speech ability.

Back to top

 

Multiplayer First Person Shooting Game with Tangible and Physical Interaction

Project Members:
Jefry Tedjokusumo, Steven Zhou

Idea:
First Person Shooting game is a popular computer game genre. The game required high level accuracy of aiming, which currently is provided by mouse. This traditional method lacks physical and social interaction.

We present a new application of multiplayer FPS game. In our system, the players wear head tracking, Head Mounted Display (HMD), and wand tracking (see Figure below). With these tracking devices we change the way multiplayer FPS
games are played. The player’s view orientation and position tracked by the head tracker. His hand holds a wand tracking device. This wand’s orientation corresponds to the gun aiming orientation in the game. In our system it is possible for the player to shoot backward, even though his head facing forward. To make this feasible, we give a small screen showing the
gun point of view in the player’s eye view.

Our system gives new perspective of how multiplayer FPS games are played. It provides tangible and physical interactions among users both collaboratively and competitively.

In-game screenshots and corresponding player poses. We separate the weapon and head view; hence the player can aim using their hand. The small window on the right corner is the weapon point of view. This separation gives ability for the player to shoot enemies outside his/her view range (even at the backside).

Back to top

 

Tangible Game Interface using Projector-Camera System

Project Members:
Jefry Tedjokusumo, Steven Zhou

Idea:
Projector and camera are mounted on tripods. The cardboard is held in the field of view of the camera and the projector. Since the board is light and can be manipulated easily by the user, this tangible interface provides direct viewing and interactive capability. Users move the board for interaction and watch as the projected content changes accordingly.

As shown in Figure below, the camera is overlooking the projected content on the cardboard. The geometric distortion is captured by the camera and compensated in real-time such that the game interface shown on the cardboard is always undistorted. The geometric compensation is achieved by homography transformation. Fiducial markers are placed on the paper board for tracking its 3D orientation.

Our system only requires a cheap commodity projector with an off-the-shelf webcam. Users need not wear or hold any special devices other than a piece of paper to play games; hence it provides direct, natural and flexible interactions between the player and the computer.

Back to top