The future of virtual gaming

[dlevere]

By Jd8531

During CES, Microsoft showed off IllumiRoom, a new technology that turns your environment into part of games or movies; a step into the future of virtual gaming.

A lot of incredibly advanced and stunning technology has come out of CES 2013 including some updates and previews on other virtual gaming gear like the Oculus Rift but something just announced by Microsoft really stole the spotlight this year.

During Samsung’s keynote at CES 2013, Microsoft demonstrated IllumiRoom, a new technology that I talked about in one of my articles awhile back about the future of gaming and virtual gaming.

The new technology that Microsoft had patented has finally come before consumer eyes, the tech turns the entire room you are playing in part of the game or movie. IllumiRoom uses Kinect and a projector to blur on screen content and the environment around you to combine virtual and physical worlds.

“Blending our virtual and physical worlds with projected visualizations.”

Microsoft actually showed a demonstration of the technology at work during the keynote, which you can watch below in its entirety.

Microsoft said that the video demo shown during the CES keynote was captured live and is not the result of any special effects added in post production, which is extraordinary with some of the environment effects.

Microsoft explained that IllumiRoom “augments the area surrounding a television screen with projected visualizations to enhance the traditional living room entertainment experience” and “can change the appearance of the room, induce apparent motion, extend the field of view, and enable entirely new game experiences.”

The Kinect captures the layout and arrangement of any room and adapts projected visuals in real time without any need to custom pre-process the graphics, which makes this no ordinary projector; breaking the borders of a TV. Crazy, right?

The video shows off some seriously mind blowing effects, such as an entire room pulsing, snow falling and stars surrounding the room around the tv. IllumiRoom really looks to create a more immersible gaming or movie experience. The video looks great, and hopefully operates just as good as it looks. Microsoft hasn’t announced if IllumiRoom will be included with the next XBox and Kinect 2.0, however.

In the hands of developers a ton more can be done than what's shown in this small video demo, which has a lot of people excited, including myself. If virtual gaming continues to look this good, sign me up.

[PEOPLECANFLY]

Ummm... Cool?

[dlevere]

The Project Holodeck Virtual Reality Game World

Project Holodeck Hands-on Demo and Interview with Director Nathan Burba

[dlevere]

Transform any room into a giant XBox game

By Tom Warren

When Microsoft first demonstrated its IllumiRoom research project at CES last year it generated a huge amount of attention ahead of a next-generation Xbox unveiling.

A video showed off a projection system that was linked to the Xbox to extend games from a TV to nearby walls, and appeared to be more than just a concept demo. It turned out that IllumiRoom was "just research" after all, but Microsoft is back this year with IllumiRoom 2.0: RoomAlive.

RoomAlive builds on the familiar concepts of IllumiRoom, but pushes things a lot further by extending an Xbox gaming environment to an entire living room. It’s a proof-of-concept demo, just like IllumiRoom, and it combines Kinect and projectors to create an augmented reality experience that is interactive inside a room. You can reach out and hit objects from a game, or interact with games through any surfaces of a room. RoomAlive tracks the position of a gamers head across all six Kinect sensors, to render content appropriately.

RoomAlive’s video projectors and Kinect combination is far too costly and large for living rooms right now, but Microsoft is imagining a future where this technology will be smaller and low-cost. Sony has created similar virtual reality experiments in the past, but Microsoft’s system can automatically detect the floors and walls of the room to let the system work anywhere. "There’s still lots to explore with RoomAlive as a gaming platform," explain’s a Microsoft Research spokesperson. "We envision a future where games can use physical objects as part of the game."

[dlevere]

Our potential, holodeck-style gaming future

By Kyle Orland

Kinect and projectors combine for room-filling games on the walls and floors.


This image from Microsoft Research's "Mano a Mano" paper shows how three projector/Kinect pairs can create believable 3D virtual objects from two different perspectives. MSR/ACM

Microsoft may be taking an official wait and see approach before following companies like Oculus and Sony down the virtual reality headset path. That isn't stopping the company's research arm from looking into interesting ways to use Kinect and projector technology to create holodeck-style augmented reality experiences in the living room, though. Microsoft Research has prepared a number of interesting demos and papers on these lines for the Association for Computing Machinery's User Interface Software and Technology Symposium, showing off just how far those efforts have come and how they could lead to interesting new forms of gaming in the future.

The first project, RoomAlive, promises to "transform any room into an immersive augmented virtual gaming experience," as the researchers put it. The system uses six paired projector/Kinect units, mounted to the ceiling so they have somewhat overlapping fields of view. These units can auto-calibrate themselves with a series of projected light patterns, transforming their individual Kinect depth maps into a unified 3D point-cloud model of the room.

From there, RoomAlive translates the point data into a series of vertical and horizontal surfaces representing the walls and furniture, then translates that into a 3D environment in the Unity game engine. Using that virtual representation of the room, the system then figures out how to project a unified image on those walls and surfaces, warping the projection so it appears correct on each surface. The effect is akin to transforming the entire room into a computer screen or monitor, complete with player-tracking through the array of Kinect cameras.

In addition to some non-interactive demos, MSR showed off a few gaming concepts that use the system. In one "whack-a-mole" game, users, tracked by Kinect, can touch or shoot at critters that appear on the wall. In another, a gun-toting character controlled with a handheld controller runs across the wall, down on to a table, and then onto the floor while being chased by robots. The final demo puts virtual spike traps on the wall for users to dodge and bathes the room in red when and if they are hit.

Dual perspectives and finger detection

In a similar ACM demo, called Mano-a-Mano, a team of two MSR researchers uses a trio of projector/Kinect combos to create an augmented reality effect that provides correct three-dimensional perspectives for two different users. Each projector displays virtual objects against the walls, floors, and fixtures in a room in such a way that they appear to float in the middle of the room. The apparent perspective and size of those virtual objects changes as the user's position and head angle are detected by Kinect to give the illusion of real depth and position in the middle of the room.

That's a decent faux 3D solution for a single user, but how can such a system account for two people looking at a virtual object from different angles? That's where the multiple projector setup comes in, giving each user their own view of the virtual scene. By "assuming that each user is unaware of graphics projected on the wall behind them or their own bodies," as the researchers explain, the system can show two different perspectives of the same scene that look correct to each user. In the demo, the system is shown for a simple game of catch and for a "combat style game" where a user can summon fireballs in their hand and fling them at the user on the other side of the room.

https://www.youtube.com/watch?featur...&v=A-xXrMpOHyc

The last of MSR's ACM demos that might be of interest to gamers and game makers is Handpose, a system that adds a degree of detail and articulation to Kinect-based hand and finger tracking. With a new tracking algorithm, researchers appear to be able to distinguish individual fingers and hand gestures with much more detail than was previously possible with a standard Kinect v2 sensor.

Users are shown throwing complex finger positions at many different angles while the tracking system quickly and accurately tracks those positions in a 3D model of the hand. This tracking is "robust to tracking failure," works up to "several meters" away from the sensor, and works regardless of where the camera is positioned, even if the tracking camera is moving, the researchers say. In a video demo, users are shown using the system to easily grasp and move virtual objects simply by moving their fingers together and apart.

Coming to stores... never?

https://www.youtube.com/watch?featur...&v=sJ4hWa6y710

These kinds of augmented reality experiments aren't exactly new for Microsoft and Microsoft Research. MSR's latest demos pivot off of IllumiRoom, an impressive demonstration from last year that showed projectors being used to extend game action past the bounds of a TV screen.

And let's not forget that Microsoft's 2012 "Project Fortaleza" leak and subsequent patents both point to an interest in heads-up augmented reality displays.

Of course, no actual products, or even real hints of actual consumer products, have actually come from those revelations as of yet. Microsoft Research's efforts are only loosely connected to the consumer-focused divisions of the company, and these proof-of-concept demonstrations shouldn't be seen as indications that the Xbox division will be heading in this direction any time soon. Even if they did, technology would have to get a lot smaller and cheaper before the average consumer was willing to mount three to six projector/camera combos on the ceiling.

Still, it's nice to see at least one Microsoft division pushing the bounds of gaming past the flat screens and controls we're used to. If we do one day get to a world where projector-and-tracker-based gaming is a feasible consumer reality, it'll be this kind of basic research that provided the seed.