The Current World of Virtual Reality

For the past few months I've spent some time with the Oculus Rift, and have gotten to learn the basic ins and outs of how to make virtual reality applications, where the market for it is going, and some of the challenges that come along with it.  It's been quite a fun experience, and while I am no means an expert now, I feel pretty comfortable with my ability to use these technologies and develop applications for them should I ever need to.  While I plan to slowly add-on new features to Project Virtua as needed, there is not much more to do on it as of now, and so I've decided to start looking into other realms of technology to mess with.  This post is a quick summary of some of the things I've learned, as well as some of my views on where this technology should go.  While I won't be super active on these technologies, I will still post some things about them, so don't count me out completely.  Specifically, I have a few games I would like to code for it that I plan to start sometime over the summer if I can muster up the free time!

Stereoscopic 3D

Stereoscopic displays are an interesting technology.  While they've been around for quite a number of years, they are still relatively primitive and simplistic in how they work, and many people prefer not to use them in things such as 3D TVs, or game systems.  The Oculus Rift is a key component to virtual reality right now though, and makes heavy use of this technology.  The Oculus Rift however, displays 3D images differently from how most current stereoscopic displays do so, such as the Nintendo 3DS.  As a result, it has effectively created a second way to display 3D images to users.

Notice how the left eye can see more of the soccer ball
in the top left-hand corner than the right eye.

The first method that is often used to display 3D images was through two viewing points and showing images to each eye through a single display.  What I mean by this, is that each eye is projected a completely different image, at a incredibly fast rate to which you do not notice.  This is why when you buy a 3D TV, you'll want to make sure that it has a very high refresh rate, because if it is too low, there will be a lot of lag with what you see to each eye.  The Nintendo 3DS also uses this technology, and simply projects these images almost directly into your eyes.  This is obviously pretty simple to render with as a programmer, as all you have to do is render to two separate buffers, one for the left eye and one for the right eye.

The second method of rendering 3D images is how the Oculus Rift does it, in which your eyes see half of a single screen.  This is usually coupled with 3D lenses, in order to further separate each eye into it's own viewing "world".  However, this causes complications from the programming side, as you have to warp each viewing area to match these lenses.

Both of these methods pose a variety of other challenges too.  In both, you'll have to render your entire scene twice, bringing a game like Call of Duty down to half of its normal frame-rate, and at different viewing points for each eye.  You also have to make sure that you find proper ways to combat motion sickness, as this is very prevalent with both of these technologies.  Separate graphics cards could help to speed things up, but due to costs, this could also still be tough for end-users.  At this point in time, I believe that these are still very early iterations, and that improvements can be made on them to make their experiences even smoother in the future.

Controls

This is a huge issue that I have seen, and have yet to find a proper solution to.  Many games rely on controllers, simply due to how big the worlds you move around in are.  While the Microsoft Kinect can provide a more immersive experience, it limits how much you can move around, and can often limit the game-play elements as a result.  A normal game controller or keyboard/mouse combo also provides challenges though, as users cannot see their devices with a headset on.  Keyboards are especially bad in this aspect, making controllers a must-have, even for developers to test with.

The Microsoft Kinect for Windows can really help
VR better

I think that this issue is largely unfocused on, and should really have more attention to it.  As of now, much of virtual reality is focused on the displays, which while important, is only a piece of the puzzle.  Without a better method of controlling the applications that use it, this will become yet another 3D TV fad, simply because it does not add enough value with just a new display.

One of the ways I experimented with this was mainly with the Microsoft Kinect for Windows V1.  While there is a bit of lag with the system still, it will obviously get better as the technology gets better.  While things like the Hydra controllers are neat, they still do not provide the right level of immersion in my personal opinion, which the Kinect can provide.  The Kinect also seemed to reduce motion sickness, probably due to the fact that the user is now moving around with their character, instead of staying in place.

Edit: I've had a comment or two about the Virtuix Omni and how it solves this issue of controls, and so I thought I'd clarify a few things about it.  I personally am not a fan of the Omni, but that aside, I do think it solves the partial issue of walking around in a world.  While it doesn't allow for crawling, you probably won't want to do that with a headset like the Oculus Rift on, and so it's fine that you can't.

That being said, it's a $500 device, that still has the problem of controls, because it only tracks walking.  You still need your hands, feet, etc. to be able to do actions, and your feet specifically will not be usable with the Omni.  Unfortunately, kicking, holding your foot forward, etc. are not possibly with it, and it does not include a way to track anything other than your steps.  I feel like this is not worth $500 at all, which is part of why I did not address it originally.

While it is a neat idea, I do not feel like the Omni fulfills the complexity needed for a proper virtual reality control system.  Combined with other technologies, you can make it better, but the costs will add up too quickly for the Omni to be a proper solution, especially since it cannot do so on its own.

Content

This partially ties in with controls, but I still think it deserves its own category.  Many of the games that are using the Oculus Rift and Kinect are currently just 3D adaptions of existing games that do not require 3D.  Yes, I know, there are a few games that may be exceptions, but aside from those, most games can be played without this new technology.  That needs to change, as again, why would people use this new technology if they do not need it?  Wearing a Rift is going to be strenuous due to it's weight, and the Microsoft Kinect limits what you can do with many of these games.  This means users would lose more than if they just didn't use the technology.

New experiences need to be built on these technologies, not vice-versa, and I think the best way to do so is through entirely new content.  This means less first-person shooters, which unfortunately is a huge part of the gaming market currently.  Even RPGs need to be reworked majorly due to how they currently work, because again, who is honestly going to spend $350 for a fancy monitor, that tires you as you play your favorite games?  I know I won't be.

That being said, I think there is hope.  I've looked around at some potential games, and while I haven't made any, this is more due to the limitations of my artistic capabilities.  One game style I'd love to see is point-and-click adventures through virtual reality.  I feel like this technology has the potential to help those games make a huge comeback, as it would really provide a new way to experience them.  Another game genre that could be cool are puzzle games, specifically those involving heavy physics.  The level of immersion possible with virtual reality makes it loads of fun to explore different types of physics interactions, especially those not possible in the real world (like random floating orbs and such).

Final Thoughts

I've had loads of fun messing around with this tech, and seeing what exactly you can do with it all.  While there are many challenges ahead, I think it will be interesting to see what happens and who can fund further development on it.  While I am skeptical of the Facebook buyout for Oculus VR, I think that ultimately this may be a good thing for both parties.  I plan to do small improvements to Project Virtua for those who want a C++ library with no extra hand-holding, but development will be a bit slower unless interest suddenly jumps up.  I'm also planning on making some VR games, and exploring some other technologies too.  As for what exactly I'm planning to work on next, I guess you'll have to wait and see ;)

Feel free to leave your thoughts in the comments below!  If you have something else for me to look into with VR, feel free to let me know!