Learn how to work with Legacy XR and Oculus VR-based applications built in Unity using GameDriver
Configuration
The 2023.04 release includes support for working with Unity’s Legacy XR implementation. For working with the Input System package, refer to this article.
To enable support for Legacy XR inputs, simply enable XR Hooks through the GDIOAgent configuration:
Or using the API:
api.EnableHooks(HookingObject.XRLEGACY);
Note: In mixed input environments (i.e. LegacyXR and Input System), you may need to use the "Force Old Input System" option in the agent. To trigger this via the client API, simply use a SetObjectFieldValue command after the initial connection, such as:
api.SetObjectFieldValue("//*[@name='GDIOAgent']/fn:component('gdio.unity_agent.GDIOAgent')", "m_ForceOldInputSystem", true);
Where "GDIOAgent" is the name of the object holding the GDIOAgent component.
Input Usage
Input calls use the same paths as with the Input System, i.e. using ButtonPress and InputEvent actions, but using paths set by the agent:
- GDIOHMD for the Headset
- GDIOLeftHand for the Left Controller
- GDIORightHand for the Right Controller
Example for setting the location of the HMD:
api.Vector3InputEvent("GDIOHMD/centerEyePosition", new Vector3(0.0f, 1.8f, 0.0f), 0);
Some additional device paths can be used for setting the position and rotation of the simulated devices:
GDIOHMD/centerEyePosition
GDIOHMD/centerEyeRotation
GDIOLeftHand/devicePosition
GDIOLeftHand/deviceRotation
GDIORightHand/devicePosition
GDIORightHand/deviceRotation
Legacy XR is also supported by the GameDriver Recorder and can be used to determine the inputs to be used for testing. If the project contains both old and new input systems, checking the “Force Old Input System“ may be required.
An example test demonstrating the use of LegacyXR inputs is attached below and includes a recording from the Escape Room tutorial project from Unity. You may need to adjust the initial positioning of the XR
Oculus VR (OVR) Support
As of the 2023.10 release, GameDriver's LegacyXR support is extended to include the Oculus VR (OVR) package. This support utilizes the same input methods and general usage as outlined above, with the addition of support for hand gestures.
Hand gesture support is provided via multiple rotations representing each finger, or around 14 rotation calls per gesture. To help with this, we provide a helper class named HandPosePrinter.cs that can print out to a file all the rotations that form the hand gesture.
To use the script which can be found below, run the project without the GDIOAgent enabled, and with the headset on, make the desired gesture with your hand and press the [P] key on the keyboard. Two files named OVRPoseRightHand and OVRPoseLeftHand will be created on the desktop folder containing the test code necessary. The test code used to support these gestures will include the use of the SkeletonInputEvent method, such as:
api.SkeletonInputEvent("OculusLeftHand/skeleton", new float[]{0.0f,0.0f,0.0f,0.0f,0.0f}, 2); // Set hand skeleton to fully open hand.
You can organize these into methods in your test code to execute the specific captured pose. An example of this approach is shown below.
public void PinchPose()
{
//Hand_Thumb0
api.SkeletonInputEvent("GDIORightHand/skeleton", 2, api.EulerToQuat(39.00648f, -297.9758f, -323.5771f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 3, api.EulerToQuat(32.08998f, -341.2277f, -336.2333f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 4, api.EulerToQuat(351.5882f, -7.960947f, -342.8726f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 5, api.EulerToQuat(7.265752f, -353.5541f, -332.6898f), 0);
//Hand_Index1
api.SkeletonInputEvent("GDIORightHand/skeleton", 6, api.EulerToQuat(4.145337f, -349.4534f, -307.5352f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 7, api.EulerToQuat(357.1325f, -0.7786186f, -306.1896f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 8, api.EulerToQuat(358.6137f, -2.876725f, -335.6526f), 0);
//Hand_Middle1
api.SkeletonInputEvent("GDIORightHand/skeleton", 9, api.EulerToQuat(359.0811f, -1.158072f, -349.7017f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 10, api.EulerToQuat(358.6111f, -0.5548517f, -302.3149f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 11, api.EulerToQuat(352.5197f, -0.6495829f, -333.5562f), 0);
//Hand_Ring1
api.SkeletonInputEvent("GDIORightHand/skeleton", 12, api.EulerToQuat(354.2881f, -10.8357f, -10.93568f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 13, api.EulerToQuat(355.0336f, -1.024279f, -297.8333f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 14, api.EulerToQuat(357.1022f, -357.3359f, -327.2591f), 0);
//Hand_Pinky0
api.SkeletonInputEvent("GDIORightHand/skeleton", 15, api.EulerToQuat(336.6904f, -17.70523f, -354.1525f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 16, api.EulerToQuat(11.74857f, -13.97097f, -29.30103f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 17, api.EulerToQuat(353.5829f, -5.809196f, -309.7157f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 18, api.EulerToQuat(358.9135f, -354.639f, -333.5012f), 0);
}
public void PinchPoseLeft()
{
//Hand_Thumb0
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 2, api.EulerToQuat(38.82063f, -297.5854f, -327.9914f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 3, api.EulerToQuat(29.59883f, -338.8843f, -335.5561f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 4, api.EulerToQuat(351.7505f, -7.906457f, -340.3772f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 5, api.EulerToQuat(6.546093f, -354.0137f, -322.5562f), 0);
//Hand_Index1
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 6, api.EulerToQuat(4.216051f, -351.8521f, -311.5842f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 7, api.EulerToQuat(357.14f, -0.7670153f, -299.3587f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 8, api.EulerToQuat(358.7017f, -2.832536f, -330.9977f), 0);
//Hand_Middle1
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 9, api.EulerToQuat(359.0885f, -2.093114f, -355.8686f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 10, api.EulerToQuat(358.6343f, -0.5280398f, -319.4061f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 11, api.EulerToQuat(353.742f, -0.207874f, -346.9665f), 0);
//Hand_Ring1
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 12, api.EulerToQuat(354.183f, -9.713001f, -15.66434f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 13, api.EulerToQuat(355.2855f, -0.6915488f, -316.6288f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 14, api.EulerToQuat(358.1482f, -356.8446f, -342.4605f), 0);
//Hand_Pinky0
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 15, api.EulerToQuat(336.6904f, -17.70522f, -354.1525f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 16, api.EulerToQuat(12.01481f, -9.105035f, -32.43835f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 17, api.EulerToQuat(354.1174f, -5.321207f, -325.8288f), 0);
api.SkeletonInputEvent("GDIOLeftHand/skeleton", 18, api.EulerToQuat(359.5704f, -354.4142f, -348.7165f), 0);
}
public void OpenHandPose()
{
api.SkeletonInputEvent("GDIORightHand/skeleton", 2, api.EulerToQuat(38.69f, -297.28f, -335.78f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 3, api.EulerToQuat(24.92f, -333.71f, -334.27f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 4, api.EulerToQuat(351.11f, -8.08f, -350.09f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 5, api.EulerToQuat(8.26f, -353.17f, -345.33f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 6, api.EulerToQuat(3.04f, -350.32f, -350.40f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 7, api.EulerToQuat(357.09f, -0.81f, -331.50f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 8, api.EulerToQuat(358.26f, -2.98f, -352.86f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 9, api.EulerToQuat(359.10f, -2.30f, -355.11f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 10, api.EulerToQuat(358.65f, -0.52f, -329.55f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 11, api.EulerToQuat(354.60f, -0.08f, -355.84f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 12, api.EulerToQuat(354.39f, -10.51f, -357.40f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 13, api.EulerToQuat(355.55f, -0.50f, -331.40f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 14, api.EulerToQuat(358.53f, -356.74f, -347.70f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 15, api.EulerToQuat(336.69f, -17.71f, -354.15f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 16, api.EulerToQuat(10.75f, -4.33f, -2.96f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 17, api.EulerToQuat(354.32f, -5.19f, -331.19f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 18, api.EulerToQuat(359.56f, -354.42f, -348.54f), 0);
}
public void PointPose()
{
//Hand_Thumb0
api.SkeletonInputEvent("GDIORightHand/skeleton", 2, api.EulerToQuat(38.89811f, -297.7541f, -325.8609f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 3, api.EulerToQuat(35.59741f, -7.550374f, -351.1602f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 4, api.EulerToQuat(352.6481f, -7.44523f, -325.6596f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 5, api.EulerToQuat(5.831383f, -354.6906f, -310.8864f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 6, api.EulerToQuat(3.839882f, -355.4543f, 0f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 7, api.EulerToQuat(357.0401f, -0.8260307f, 0f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 8, api.EulerToQuat(357.9413f, -2.972085f, 0f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 9, api.EulerToQuat(359.4679f, -4.349733f, -289.7907f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 10, api.EulerToQuat(358.5927f, -0.6143429f, -272.0609f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 11, api.EulerToQuat(350.0163f, -3.198704f, -296.062f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 12, api.EulerToQuat(354.7221f, -8.257318f, -289.0827f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 13, api.EulerToQuat(354.8939f, -1.762349f, -263.6889f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 14, api.EulerToQuat(355.621f, -358.9403f, -298.2752f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 15, api.EulerToQuat(336.6904f, -17.70524f, -354.1525f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 16, api.EulerToQuat(8.31627f, -353.5389f, -308.8625f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 17, api.EulerToQuat(353.0914f, -8.18574f, -253.8036f), 0);
api.SkeletonInputEvent("GDIORightHand/skeleton", 18, api.EulerToQuat(357.8244f, -355.6555f, -300.4726f), 0);
}
By organizing these poses into separate methods, we can call upon them in a test case much more readily. For example, the following test will first position the simulated devices into specific locations in the scene, then call the required methods captured above.
[Test]
public void OVRHandsTest()
{
// LegacyXR Hooking must be enabled for OVR Inputs
api.DisableHooks(HookingObject.MOUSE);
api.DisableHooks(HookingObject.KEYBOARD);
api.EnableHooks(HookingObject.XRLEGACY);
// In this example, we're forcing Legacy Inputs in the agent due to a mixed input method project
api.SetObjectFieldValue("//*[@name='GDIOAgent']/fn:component('gdio.unity_agent.GDIOAgent')", "m_ForceOldInputSystem", true);
// We're now loading a device description from a JSON file. Refer to Input System usage for more details.
//string desc = api.LoadDeviceDescription("GDIORightHand.json");
//api.CreateInputDeviceFromDescription(desc, "GDIORightHand", new string[] { "GDIOInputProvider" });
api.Wait(2000);
api.ButtonPress("GDIOHMD/UserPresence", 0, 1f);
api.ButtonPress("GDIOHMD/IsTracked", 0, 1f);
api.IntegerInputEvent("GDIOHMD/TrackingState", 63, 0);
api.IntegerInputEvent("GDIORightHand/TrackingState", 63, 0);
api.IntegerInputEvent("GDIOLeftHand/TrackingState", 63, 0);
//api.Vector3InputEvent("GDIOHMD/CenterEyePosition", new Vector3(0.0f, 0.0f, 0.0f), 0);
api.QuaternionInputEvent("GDIOHMD/CenterEyeRotation", api.EulerToQuat(35f, 0f, 0f), 0);
api.Vector3InputEvent("GDIOLeftHand/DevicePosition", new Vector3(-0.3f, -1.0f, 0.6f), 0);
api.Vector3InputEvent("GDIORightHand/DevicePosition", new Vector3(0.3f, -1.0f, 0.6f), 0);
api.Wait(2000);
// Position hand on buttons
PinchPose();
PinchPoseLeft();
// These are test-specific methods used to position the simulated XR devices at the right place in the scene
Vector3 position = api.GetObjectPosition("//*[@name='StartStopButtonHousing']");
Vector3 up = api.GetObjectFieldValue<Vector3>("//*[@name='StartStopButtonHousing']/fn:component('UnityEngine.Transform')", "up");
position.x += 0.02f;
position.y += up.y * 0.22f;
position.z += up.z * 0.22f;
api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
api.Wait(1000);
Quaternion rotation = RotateTo("RightHandAnchor", "StartStopButtonHousing", new Vector3(0.02f, 0.0f, 0.0f));
api.QuaternionInputEvent("GDIORightHand/DeviceRotation", rotation, 0);
api.Wait(1500);
// Point pose and press
PointPose();
position.y -= up.y * 0.04f;
position.z -= up.z * 0.04f;
api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
api.Wait(1000);
// Reposition
position.y += up.y * 0.04f;
position.z += up.z * 0.04f;
api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
api.Wait(1000);
position.x = 0f;
position.y += 0.1f;
position.z += 0.04f;
api.Vector3InputEvent("GDIORightHand/DevicePosition", position, 0);
api.QuaternionInputEvent("GDIORightHand/DeviceRotation", api.EulerToQuat(-20.0f, 0.0f, 180.0f), 0);
api.Wait(2000);
position.x += 1.8f;
position.z -= 2.0f;
api.Wait(1000);
OpenHandPose();
api.Wait(1000);
api.Vector3InputEvent("GDIORightHand/DevicePosition", new Vector3(position.x, -1.0f, 0.43f), 0);
api.QuaternionInputEvent("GDIORightHand/DeviceRotation", api.EulerToQuat(0.0f, 0.0f, 0.0f), 0);
api.Wait(1000);
PinchPose();
api.Wait(3000);
OpenHandPose();
api.Wait(1000);
api.QuaternionInputEvent("GDIORightHand/DeviceRotation", api.EulerToQuat(-90.0f, 0.0f, 180.0f), 0);
api.Wait(1000);
PinchPose();
api.Wait(2000);
OpenHandPose();
api.Wait(3000);
}
This example code also makes use of some useful helper functions, such as RotateTo, which can be found in this article.