Simulating Unreal XR inputs using GameDriver

Learn how to test XR applications built in Unreal Engine using GameDriver

Introduction

GameDriver provides a new way to simulate XR inputs to projects that use both the legacy Input system (Unreal 4.27) and the Enhanced Input system (Unreal 5.1 and above).

The GameDriver tests can be executed as if a physical device were present, allowing for automated test execution independent of the physical device. Simulated devices are created by the GameDriver agent, which processes input commands passed into the project using the GameDriver API client.

Simulated Device Creation

The GameDriver Agent allows you to add simulated devices for testing interactions and inputs without the need to plug in a physical device during automated testing. CreateInputDevice call is used to create a generic device.

// Legacy Input
api.CreateInputDevice("OculusTouch", ""); 

// Enhanced Input (using mapping contexts)
api.CreateInputDevice("OculusTouch", "IMC_Menu,IMC_Weapon_Right,IMC_Hands");

Supported Commands

The GameDriver API supports the following commands for use with the Unreal Engine:

KeyPress - Controller button presses.
FloatInputEvent - Controller thumbstick axis input (x/y axes separately) in Legacy Input.
Vector2InputEvent - Controller thumbstick axis input in Enhanced Input.
Vector3InputEvent - HMD/Controller position.
QuaternionInputEvent - HMD/Controller rotation.

Details on the use of these commands can be found in the API reference.

Testing the Virtual Reality template project

Open Unreal Editor and select the Virtual Reality template project.

Add the GameDriver plugin to the project and create an NUnit Test project as outlined in the documentation here.

Now let us write a simple test to grab the gun and shoot a cube.

[Test]
public void GrabGunAndShootSmallCube()
{
    Vector3 playerStartPos, gunOriginalPos, pos;
    string smallCube1HPath = "//GrabActor_StaticMesh_Physics6";
    playerStartPos = new Vector3(20, 0, 110);

    // Save the Original gun position to check later
    gunOriginalPos = api.GetObjectPosition("//Pistol_00");
    // Move the player to the starting position.
    MoveObjectToPosition("//VRPawn_C_0", playerStartPos);
    api.Wait(500);
    
    // Move the right controller close to the Gun.
    pos = MoveObjectCloseToObjectOnAxes("//VRPawn_C_0/fn:component('MotionControllerRight')", "//Pistol_00", new Vector3(-1, 1, 2));
    
    // Hold down the grip button of the controller to grab the gun.
    //api.KeyPress(new KeyCode[] { KeyCode.OculusTouch_Right_Grip_Click }, 0); //key down
    api.KeyPress(new KeyCode[] { KeyCode.OculusTouch_Right_Grip_Axis }, 0); //key down
    
    api.Wait(3000);
    //Move the controller to the right of the small cube
    pos = MoveObjectCloseToObjectOnAxes("//VRPawn_C_0/fn:component('MotionControllerRight')", smallCube1HPath, new Vector3(0, 50, 15));
   
    api.Wait(3000);
    // look at cube
    Vector3 newRot = api.FindLookAtRotation("//VRPawn_C_0/fn:component('MotionControllerRight')", smallCube1HPath);
    
    //Controller forward is gun up, so need to add 90 to yaw (pitch,yaw,roll)
    newRot = new Vector3(newRot.x, newRot.y + 90, newRot.z);
    api.SetObjectRotation("//VRPawn_C_0/fn:component('MotionControllerRight')", newRot);
    api.Wait(3000);
    //Pull gun trigger 
    //api.KeyPress(new KeyCode[] { KeyCode.OculusTouch_Right_Trigger_Click}, 25);
    api.KeyPress(new KeyCode[] { KeyCode.OculusTouch_Right_Trigger_Axis }, 25);
    api.Wait(5000);
    //Release gun
    //api.KeyPress(new KeyCode[] { KeyCode.OculusTouch_Right_Grip_Click }, 0);//key up
    api.KeyPress(new KeyCode[] { KeyCode.OculusTouch_Right_Grip_Axis }, 0);//key up
    api.Wait(5000);
}


#region HelperFunctions
Vector3 MoveObjectToPosition(string hPath, Vector3 position)
{
    if (hPath.Contains("fn:component("))
        api.CallMethod(hPath, "K2_SetWorldLocation", new object[] { position, false, true });
    else
        api.CallMethod<bool>(hPath, "K2_SetActorLocation", new object[] { position });

    return api.GetObjectPosition(hPath);
}

Vector3 MoveObjectCloseToObjectOnAxes(string hPath, string toObjHpath,Vector3 distFromObjectOnAxes)
{
    Vector3 toObjPos = api.GetObjectPosition(toObjHpath);

    return MoveObjectToPosition(hPath, new Vector3(toObjPos.x + distFromObjectOnAxes.x, toObjPos.y + distFromObjectOnAxes.y, toObjPos.z + distFromObjectOnAxes.z));
}

#endregion

Note: Only Oculus devices are fully certified as of this release. Other devices may work but are untested.