Testing Xbox Games built in Unity

Learn how to test Xbox games built in Unity using GameDriver

GameDriver supports testing against the Xbox console, as long as the console is in Developer mode and the application is built using the Unified Windows Platform build target in Unity. After you Install and select the Unified Windows Platform build target, switch your platform to the following settings.

Note that you may need to install additional Visual Studio components to proceed. More information can be found on the Microsoft Learn portal.

https://learn.microsoft.com/en-us/windows/uwp/xbox-apps/development-environment-setup

Build Prerequisites

Use Web Socket for Agent communication

Testing games built for Xbox is similar to those using WebGL in that both platforms make use of the reversed WebSockets communications. This is due to the console not allowing inbound network connections. Once you have defined your test and can run it in the editor, enable the "Use Winsockets" setting in the GDIO Agent configuration dialog before building.

For the Web Sockets URL, you will need to supply the IP address of the host that is executing the tests. This could be your local machine or any machine or container that is used to run the test, such as a Jenkins host or Docker container.

Disable Legacy Input Hooking

You will also need to disable Input Hooking in the agent configuration. This is only used for legacy Input Manager projects, which are not supported by GameDriver on the Xbox; only Input System projects. If you are working with a legacy Input Manager project and would still like to perform some amount of automated testing using GameDriver, you can still use non-input related commands such as CallMethod, GetObjectFieldValue, SetObjectFieldValue, and more.

To disable legacy Input Hooking, add the following to the agent configuration under Assets > GDIO > Resources > config > gdio.unity_agent.config.txt:

  <hooking>
    <initialize enabled="false" />
  </hooking>

Ensure the websocketstubs support for UWP is enabled

Navigate to the GDIO > Plugins > Standalone > websocketstubs.cpp file in the Unity editor, and make sure the WSAPlayer platform is enabled in the Inspector view, as shown below.

Building the Game

Building for Xbox using the Universal Windows Platform target is a 2-step process. First, build the project in Unity using the settings shown above. Next, open the project in Visual Studio using the solution file saved to the directory where the build was processed.

Once the project is open, configure the debugger as outlined in the Microsoft guide. Open the Debug Properties dialog, and add the Xbox IP using the dialog shown. You may need to select the dropdown in the "Machine Name" property and locate the device manually.

Once configured, you can deploy the game to your Xbox directly by starting the remote debugger.

However, you will need to carefully configure the test to run this game to run tests consistently.

Configuring the Test

When testing using Web Sockets, communications are reversed between the ApiClient (test-side) and GDIOAgent (game-side). This means the test needs to start in listening mode before the game and the agent are initialized. This method also requires the use of the `UseWebSockets` API command, rather than the standard `Connect` or `ApiClient.Launch` commands.

If you consider the order of operations, it is sometimes necessary to use this command as a Task rather than a blocking call. For example, if the test flow is as follows:

Start Test
Launch Game
Connect Test to Game
Run Tests...
Disconnect

In this scenario, we need to switch steps 2 and 3, which will make the test ready to receive the communication request from the game once it has launched. You could add any commands used to launch the game before the UseWebSockets command, but it may require some timing adjustments to avoid starting the Client too early or timing out too soon.

One approach is to use the Task class in .NET:

// Needs to be a Task when using Xbox_LaunchGame
var task = new Task(() => api.UseWebSockets("192.168.68.100", 7072, true, 300));
task.Start();
api.Wait(300000); // 300s. May need to be adjusted depending on your deployment and launch time

Note: The hostname argument shown in the UseWebSockets command above is the IP or hostname of the test execution machine, not the target Xbox. This is required to open the listening server on the machine running the test, and will not work using the localhost or 127.0.0.1 values, which are not accessible to outside machines.

This code above will start the task to listen using UseWebSockets and wait for 300s for the incoming connection. After this, you will need some way to launch the game. This can be done manually using the Remote Machine debugger mentioned above. However, this approach is not scalable.

Another method to launch the game is to use the Xbox Device Portal interface. This web page can be used to upload apps directly to the Xbox if they are built as an appx package. It also exposes endpoints for launching the app as part of the test script. We can leverage these endpoints in a helper function, such as the following:

        static bool Xbox_LaunchGame(string hostnameOrIp, int port, string AUMID)
        {
            System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };

            HttpClientHandler handler = new HttpClientHandler
            {
                AllowAutoRedirect = true,
                UseCookies = true,
                CookieContainer = new System.Net.CookieContainer()
            };

            using (var client = new HttpClient(handler))
            {
                client.BaseAddress = new Uri($"https://{hostnameOrIp}:{port}/");

                HttpResponseMessage response = client.GetAsync("").GetAwaiter().GetResult();
                System.Net.CookieCollection cookies = handler.CookieContainer.GetCookies(client.BaseAddress);
                string token = string.Empty;
                foreach (System.Net.Cookie cookie in cookies)
                {
                    if (cookie.Name.Equals("CSRF-Token"))
                        token = cookie.Value;
                }
                client.DefaultRequestHeaders.Add("x-csrf-token", token);

                byte[] plainTextBytes = System.Text.Encoding.ASCII.GetBytes(AUMID);
                string encodedname = System.Convert.ToBase64String(plainTextBytes);
                
                response = client.PostAsync($"api/taskmanager/app?appid={encodedname}", null).GetAwaiter().GetResult();
                if (response.IsSuccessStatusCode)
                {
                    return true;
                }
            }

            return false;
        }

Calling this method requires the hostname or IP and port strings referring to the Xbox, as well as the app package as it appears in the Device Portal interface. This can then be called using the method, where "WebGLDemo_nxkjkw2pvc1ja!App" is the name of the app package.

Xbox_LaunchGame("XBOX", 11443, "WebGLDemo_nxkjkw2pvc1ja!App");

We can follow a similar approach for terminating the game at the end of the test, such as:

static bool Xbox_TerminateGame(string hostnameOrIp, int port, string packageFullName)
        {
            System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };

            HttpClientHandler handler = new HttpClientHandler
            {
                AllowAutoRedirect = true,
                UseCookies = true,
                CookieContainer = new System.Net.CookieContainer()
            };

            using (var client = new HttpClient(handler))
            {
                client.BaseAddress = new Uri($"https://{hostnameOrIp}:{port}/");

                HttpResponseMessage response = client.GetAsync("").GetAwaiter().GetResult();
                System.Net.CookieCollection cookies = handler.CookieContainer.GetCookies(client.BaseAddress);
                string token = string.Empty;
                foreach (System.Net.Cookie cookie in cookies)
                {
                    if (cookie.Name.Equals("CSRF-Token"))
                        token = cookie.Value;
                }
                client.DefaultRequestHeaders.Add("x-csrf-token", token);

                byte[] plainTextBytes = System.Text.Encoding.ASCII.GetBytes(packageFullName);
                string encodedname = System.Convert.ToBase64String(plainTextBytes);
                response = client.DeleteAsync($"api/taskmanager/app?package={encodedname}").GetAwaiter().GetResult();
                if (response.IsSuccessStatusCode)
                {
                    return true;
                }
            }

            return false;
        }

This can called using the following:

Xbox_TerminateGame("XBOX", 11443, "WebGLDemo_nxkjkw2pvc1ja!App");

Putting this all together looks like this in the [OneTimeSetup] of our NUnit test:

// Needs to be a Task when using Xbox_LaunchGame
var task = new Task(() => api.UseWebSockets("192.168.68.100", 7072, true, 300));
task.Start();
api.Wait(300000); // 300s. May need to be adjusted depending on your deployment and launch time

//Need the name of the app from the Web UI
Xbox_TerminateGame("XBOX", 11443, "WebGLDemo_nxkjkw2pvc1ja!App");