The Oculus Rift combines a head mounted display with a sensor that tracks your head's absolute rotation. Starting with v3.0, Torque 3D now has preliminary support for this device. This page describes how to set up and use the Oculus Rift with Torque 3D. You may also want to check out the following related pages:
Table of Contents
Setting Up Your Project for the Oculus Rift
Before you can use the Oculus Rift with Torque 3D you will need to download the OculusVR SDK and then recompile your game to include the required source code.
Get the OculusVR SDK
You can download the OculusVR SDK from their web site and extract it to a convenient directory. To date, version 0.1.5 of the OculusVR SDK has been tested with Torque 3D.
Manually Configuring Your project.conf
There are two ways to set up your project to load in the required source code. The first is by manually editing your project's project.conf file as outlined here. The second way is to use the Project Manager to set everything up using its GUI. Please see the next section if you're interested in using the Project Manager.
With the OculusVR SDK in place, you will need to modify your project's buildFiles/config/project.conf file in a text editor. This will allows us to recompile our game with the required source code. The first step is to let Torque 3D's Project Generator know where the OculusVR SDK is located. The easiest way to do this is to add the $OCULUSVR_SDK_PATH global variable to the top of your project.conf file. For example:
$OCULUSVR_SDK_PATH = "C:\Users\dwyand_2\Downloads\OculusRift\ovr_sdk_0.1.5_src\OculusSDK";
You may also create a Windows environmental variable named TORQUE_OCULUSVR_PATH that also provides an absolute path to the OculusVR SDK. Going this route doesn't store the path within the project.conf file, which may be required if multiple developers are working on the same project.
The second addition to your project.conf file is a reference to the Torque 3D's Project Generator module for the Oculus Rift. Add the following between the Torque3D::beginConfig() and Torque3D::endConfig() lines:
// Include Oculus Rift module includeModule( 'oculusVR' );
The last step is optional but if you want to use Torque 3D's standard Player and Camera classes with the Rift then you should probably make this change. In order to pass the Rift's head tracking information between the client and server (which exist internally even if running a single player game) you need to enable the ExtendedMove class. This is done by setting the $TORQUE_EXTENDED_MOVE global variable to true:
// Set this to true to enable the ExtendedMove class. This // allows the passing of absolute position and rotation input // device information from the client to the server. $TORQUE_EXTENDED_MOVE = true;
With these changes in place you may run your project's generateProjects.bat file to rebuild the various Visual Studio files, and then recompile your game.
Here is a complete example of a project.conf file with all of the required changes:
// Set this true to enable hifi networking instead of standard. // In general ... hifi is designed to better deal with fast // moving players in close proximity to each other, such as // a racing game. $TORQUE_HIFI_NET = false; // Set this to true to enable the ExtendedMove class. This // allows the passing of absolute position and rotation input // device information from the client to the server. $TORQUE_EXTENDED_MOVE = true; // Our path to the Oculus VR SDK: $OCULUSVR_SDK_PATH = "C:\Users\dwyand_2\Downloads\OculusRift\ovr_sdk_0.1.5_src\OculusSDK"; // Configure Torque 3D Torque3D::beginConfig( "win32", "RiftValley" ); // Include Oculus Rift module includeModule( 'oculusVR' ); // Enable for optional minidump debugging support // addProjectDefine( 'TORQUE_MINIDUMP' ); Torque3D::endConfig();
Using the Project Manager to Configure Your Project
The previous section outlined how to manually modify your project's project.conf file to include the required source code for using the Oculus Rift. In this section we will step you through using the Project Manager to accomplish the same task.
Start by launching the Project Manager (at least version 2.0) and selecting your project from the list. Then click on the Modules button to open the Project Modules window.
To add Oculus Rift support click on the Oculus VR Devices checkbox. You also need to tell the system where your OculusVR SDK is located. Either click on the button beside the $OCULUSVR_SDK_PATH text field, or manually enter in the path to your SDK's location. It should look something like this:
The last step is optional but if you want to use Torque 3D's standard Player and Camera classes with the Rift then you should probably make this change. In order to pass the Rift's head tracking information between the client and server (which exist internally even if running a single player game) you need to enable the ExtendedMove class. This is done by clicking on the Move Class and Networking dropdown and selecting the ExtendedMove choice.
With all of these steps complete you click on the Project Modules window's Regenerate button to rebuild your Visual Studio solution files. If you're using an older version of Visual Studio, be sure to not have your project loaded in Visual Studio when you click on this button. When the regenerate step is complete you can load your project back into Visual Studio and recompile it.
Using the Oculus Rift Head Mounted Display
The Oculus Rift has a number of special requirements in order to render your application on its display. The first is that it requires side-by-side stereo images (one for each eye) which effectively means you are rendering each frame twice. The second requirement is that for each eye's view you need to render from a slightly different world position (the distance between the eyes) in order to have a true stereo view.
The third requirement is that the perspective projection needs to take place off center. With a normal render this perspective projection is set to the center of the view. However, as a user's eyes do not line up with the center of the Rift's half-panel this offset needs to be taken into account.
Finally, the Oculus Rift's optics require that a barrel distortion be perform on the rendered frame prior to it being displayed. This distortion needs to occur separately for each eye.
Fortunately, all of this is taken care of by Torque 3D with a little set up.
Setting Up Your Application
There are a number of support functions in core/scripts/client/oculusVR.cs that help you get your application ready for displaying on the Oculus Rift. We will be referencing these functions throughout this section.
Step 1: Preparing the Canvas
The Oculus Rift is treated as a separate monitor by your computer. Often users will have this set up as an extension to their desktop, but it may also be set to mirror their desktop. In either case, when you have your application go full screen you will want it appear on the Rift's display. The way we do this is by informing Torque 3D that we would like to favour the graphics adapter that is attached to the Rift.
Fortunately, we can just use the pointCanvasToOculusVRDisplay() function and it will take care of this set up. We need to call this function just before the Canvas (the main application window) is created. We do this by adding the function to the createCanvas() function in main.cs:
Step 2: Prepare for Rendering to the Rift
Before we can render to the Rift we need to tell Torque 3D about the correct field of view to use, what the eye offsets are, etc. We also need to enable side-by-side stereo rendering as well as the barrel distortion shader to account for the Rift's lenses.
Fortunately we can just call the enableOculusVRDisplay() helper function and it performs this work for us. A good place to do this is in GameConnection::initialControlSet() in scripts/client/serverConnection.cs just before you switch to the PlayGui GUI control.
Step 3: Setup a Control Scheme
A control scheme is something new for Torque 3D and is something you set on the GameConnection at either the client or server. This tells any game objects that the GameConnection controls what to expect for input events and how to make use of them. For the Rift we need to turn on an absolute rotation control scheme (which also required using the ExtendedMove class).
Often we want to also add the mouse or gamepad relative rotation to that obtained from the Rift's head tracker. This allows for a user to rotate all the way around in-game without having to turn all the way around in the chair or while standing.
To modify the control scheme for a GameConnection we need to call its setControlSchemeParameters() method. Fortunately, we can also use the setStandardOculusVRControlScheme() helper function which sets up for Rift head tracking and heading input from the mouse or gamepad. You can call this at any time but a convienent place is during GameConnection::initialControlSet() right after enableOculusVRDisplay():
Step 4: Setup Up The Action Map
In order to pass along the Rift's head tracking to Torque 3D's input system we need to set up an action map. When using the control scheme talked about above, we need to reference the ExtendedMove class's global input variables to pass along the data. We also want to set the Rift's input events to generate Euler angles rather than an angled axis. All of this may be done in scripts/client/default.bind.cs:
Step 5: Player and Camera Banking
You'll want to have the Player and Camera instances take the Rift's bank rotation into account for the best experience. To do so you'll want to add to your PlayerData and CameraData datablock instances the cameraCanBank property and set it to true.
Step 6: Remove Unused HUD Elements
There are a number of HUD elements that are part of the PlayGui GUI control. These should all be removed when using the Rift. This includes the chat display, and the reticle.
Using the Oculus Rift Sensor Input Events
The new Torque 3D Oculus Rift input device provides a number of head tracking sensor input events that may be used with Torque 3D's action map system. There are a few different ways to use the Oculus Rift input device, which are set up using global TorqueScript variables.
Action Map Device Type
When binding a TorqueScript function to an Oculus Rift sensor input event using Torque 3D's action map system, the device name to use is oculusvr. As all connected Oculus Rift sensors and any stand alone sensors are handled through the same input device there is no need to append a device instance number to the end of the device name. The following is an example of setting up a proper action map binding:
Sensor Absolute Rotation Events
The OculusVR sensor is able to detect an absolute rotation in space. This means that the sensor that is included within the Oculus Rift head mounted display is able to report the head's absolute rotation. While in theory the OculusVR SDK supports more than one sensor plugged into your computer, individual sensors are not currently available for testing. Because of this Torque 3D currently only supports a single OculusVR sensor — the one that is within the Oculus Rift. This sensor is reported as index . It is straight forward to expand the number of supported sensors when and if Oculus starts offering individual ones for sale.
Sensor rotation events may be reported in two different way: using an angled axis or using Euler angles. One or both of these forms may be chosen and it depends on your own application. To activate the angled axis form of rotation reporting you may set the TorqueScript global variable $OculusVR::GenerateAngleAxisRotationEvents to true. To activate the Euler angle form, set the TorqueScript global variable $OculusVR::GenerateEulerRotationEvents to true.
The following action map input events are available depending on how these two global variables are set:
- ovr_sensorrot0 - Absolute rotation in angled axis form (vector plus angle)
- ovr_sensorrotang0 - Absolute rotation in Euler angle form (about x, y, and z axis)
Normally when working with rotation events you would want to work with the angled axis form. This prevents issues such as *gimbal lock* and Torque 3D's internal structures make it easy to work with (you could convert it to a forward vector with a couple of TorqueScript functions, for example). Both the Leap Motion and Razer Hydra input devices operate this way.
However, for the Oculus Rift there is a special use case that often has you turn to using Euler angles. As the sensor's rotation represents the user's head rotation you often want to supplement this with input from a mouse or gamepad (joystick). By adding in the relative input from one of these devices you may effectively turn the player's head in-game without requiring the actual user to spin in their chair 360 degrees (or more). The Player class does this internally when combined with the ExtendedMove class and the appropriate GameConnection control scheme is set, for example. Under this use case you will want to use the ovr_sensorrotang0 form of the rotation input event.
Using Rotation Input Events
You bind OculusVR sensor rotation events to an action map just like any other input event. Here is an example of making use of the rotation (the Euler angle version) events generated by the Oculus Rift sensor (place it in scripts/client/default.bind.cs):
Sensor as Thumb Stick Input Events
Torque 3D allows an OculusVR sensor to be used like a gamepad thumb stick. Imagine that a thumb stick is coming out of the top of the sensor when in a neutral rotation and you move it by tilting the sensor.
To activate these thumb stick input events we set the $OculusVR::GenerateRotationAsAxisEvents global TorqueScript variable to true. With that variable set the following action map input events are available:
- ovr_sensorrotaxisx0 - thumb stick like x-axis motion in the range of -1.0 to 1.0
- ovr_sensorrotaxisy0 - thumb stick like y-axis motion in the range of -1.0 to 1.0
Internally, these x and y axis values are normalized to ensure the length of their vector is never more than 1, just like a real thumb stick.
In order to calculate the -1.0 to 1.0 range, the tilt of the sensor with respect to a vector pointing straight up (technically this vector is normal to the plane of the sensor's neutral rotation) is used. When this sensor to up vector angle reaches the $OculusVR::MaximumAxisAngle global script variable value (the default is 25 degrees) then the virtual thumb stick is considered all the way over. Adjusting $OculusVR::MaximumAxisAngle for your application determines how far over the user must tilt their sensor for a 100% value.