GameDriver Knowledge Base

Unity

General

GameDriver SmartAgents

Learn how to create complex interactions with your Unity project using the SmartAgent feature in GameDriver

The 2023.10 release introduces SmartAgents, which is an entirely new way to test your application or game using GameDriver. It is based on Moonsharp, an open-source Lua interpreter written in C#. The GameDriver Agent allows you to schedule and execute these Lua scripts as a part of your tests by utilizing various API calls as well as additional Lua functions to create dynamic actions or behaviors, and to more easily access specific functionality and objects in your game.

An example use case might be where you have a pop-up in your game that can be triggered at any time. Instead of coding your test to check for this popup every 10s, and containing the entire test in a loop, you would simply schedule a Lua script to be run by the agent every 10s and send a signal back to the calling test should the pop-up occur. This allows you to build more complex test cases that can handle various dynamic conditions without having to write increasingly complex test-side code. This example is outlined at the bottom of the article.

Executing Scripts

In Moonsharp, the entirety of a Lua script is represented as a single continuous string, which can then be passed as a parameter to the various GameDriver Moonsharp API calls.

string msScript = @"
    print('Hello World')

    local num = 1 + 1
    print(num)";

api.ExecuteScript(msScript);

The above code first creates a string representing a simple Moonsharp script that prints the message "Hello World" (to the Unity console, which is the output location of all Moonsharp script print statements) and then creates a local variable consisting of the sum of two numbers and then prints the result. This script is then executed immediately by passing the string to the 'ExecuteScript' call.

It is also possible to collect data returned by the script via a return statement.

string msScript = @"
    local a = 3
    local b = 7
    return a * b";

int output = api.ExecuteScript<int>(msScript);

Using a typed variation of ExecuteScript will allow the call to return a value equivalent to the return value of the Moonsharp script that is executed.

Scheduling Scripts

While the 'ExecuteScript' call executes the script immediately and once, we can alternatively use the 'ScheduleScript' call to schedule a script for repeated execution.

string msScript = "print('Scheduled script executed.')";

string scriptID = api.ScheduleScript(msScript, ScriptExecutionMode.EveryFrame);

The above code will schedule a script to be executed every frame. However, we could instead have it only execute the script every set amount of frames by changing the script execution mode and specifying an interval of frames, such as in the following which will execute the script every ten frames.

string msScript = "print('Scheduled script executed.')";

string scriptID = api.ScheduleScript(msScript, ScriptExecutionMode.EveryNthFrames, 10);

The string returned by the 'ScheduleScript' call represents a unique identifier of the script, which can be passed to the 'UnscheduleScript' call to stop any further executions of that script.

GameDriver Lua Functions

GameDriver implements several Lua functions that allow you to interact with the game more easily. The subset of GameDriver commands supported by the SmartAgents feature include Wait, ResolveObject, KeyPress, AxisPress, ButtonPress, and Notify.

For example, Wait can be used to have the script sleep for a number of milliseconds passed to the function, similar to the GameDriver API call of the same name.

string msScript = @" -- This script sleeps for three seconds
    print('Starting Sleep')
    Wait(3000)

    print('Sleep Ended')";

By using a HierarchyPath query, an object can be resolved using the ResolveObject function, which returns a LiteGameObject that represents the successfully queried object. If multiple objects are expected you can use ResolveObjects, which similarly takes a HierarchyPath as its parameter, but instead returns an IEnumerable containing all objects that match the query.

string msScript = @"
    -- LiteGameObject ResolveObject(string path)
    local childObject = ResolveObject('//Child')
    if childObject ~= nil then
        return childObject
end

    -- IEnumerable<LiteGameObject> ResolveObjects(string path)
    local objects = ResolveObjects('//*')
    for object in objects do
        print(object.name)
end";

The above script would search for an object in the scene with the tag "Child" and return it if one is found. Otherwise, it will print the name of all objects in the scene to the editor output. Note that both ResolveObject and ResolveObjects will return 'nil' if no objects are found matching the given query.

GameDriver's implementation of Moonsharp also allows for the use of numerous input-based functions, such as KeyPress, AxisPress, and ButtonPress, all of which are modeled after the like-named GameDriver API calls.

string msScript = @"
    -- KeyPress(Keycode key, ulong frames)
    -- Presses the space key for ten frames
    KeyPress(KeyCode.Space, 10)

    -- AxisPress(string name, float value, ulong frames)
    -- Presses the Horizontal axis fully forward for thirty frames
    AxisPress("Horizontal", 1.0, 30)

    -- ButtonPress(string name, ulong frames)
    -- Presses the input button registered under the name "Jump" for five frames
    ButtonPress("Jump", 5)";

Script Signals

It is possible to have a script send information/signals to C# methods by making use of the 'Notify' function, which allows any supported Lua object type to be passed as a parameter which is then sent to the 'ScriptSignal' event handler implemented by GameDriver.

string msScript = @"
    -- LiteGameObject ResolveObject(string path)
    local object = ResolveObject('//GrandChild')

    -- Notify(DynValue o)
    -- Send the queried object to the event handler
    Notify(object)";

api.ScheduleScript(msScript, ScriptExecutionMode.EveryNthFrames, 10);

The above code schedules a script to run every ten frames that queries an object with the "GrandChild" tag and sends it to the 'ScriptSignal' event handler.

To process any data that is being sent to the 'ScriptSignal' event handler, we must first register an appropriately constructed method to the event, such as in the following:

void HandleNotify(object sender, ScriptSignalEventArgs args) {
  LiteGameObject object = (LiteGameObject) args.obj;
  Console.WriteLine($"A script has resolved the object: {object.name}");
}

api.ScriptSignal += HandleNotify;

Working Example

In this example, we want to check whether a popup occurs in the sample game found here and click on the button to close the popup when it appears. The script below is scheduled to run every ~5 seconds, and the test will run through a loop to close the popup window 10 times. The LUA script makes use of the built-in ResolveObject command, which is looking for a clone of the Popup object in the scene.

// Popup listener
string popupListener = api.ScheduleScript(@"local obj = ResolveObject(""//*[@name='Popup(Clone)']"");
    if obj != nil then
        Notify(true)
    end
    ", ScriptExecutionMode.EveryNthFrames, (int)api.GetLastFPS() * 5);

You can also use ResolveObject on a field or property, or it's value, such as:

// Find the Object by the component/property value
string popupListener = api.ScheduleScript(@"local obj = ResolveObject(""(//*[@name='Text (TMP)'])[1]/fn:component('TMPro.TextMeshProUGUI')/@text"");
                    if obj == 'Close' then
                        Notify(true)
                    end
", ScriptExecutionMode.EveryNthFrames, (int)api.GetLastFPS() * 5);

In this example, we are separating the callback from the main test for illustration purposes. However, in an actual game, this callback could contain the logic to handle the popups if they occur.

// Trigger callback when a popup is detected
api.ScriptSignal += (sender, args) => {
    Console.WriteLine("Found a popup!");
    isPopup = true;
};

We're then going to handle 10 such popups as they occur by checking whether isPopup is true, and then calling a helper function to handle the popup. This helper function contains the logic that could be part of the callback handler if this were to be used in a larger test scenario.

while (popupCount < 10)
{
    Console.WriteLine("Searching for popups...");
    api.Wait(1000);

    if (isPopup)
    {
        HandlePopup();
     }
}

Helper function:

public void HandlePopup()
{
    api.Wait(300);

    // Click close button by specific path
    //api.ClickObject(MouseButtons.LEFT, "//*[@name='Close Button']", 10);

    // Finds any object with a Button that is interactable and click it
    api.ClickObject(MouseButtons.LEFT, "//*[contains(.,fn:component('UnityEngine.UI.Button')/@interactable)]", 10);

    // Reset popup flag
    isPopup = false;
    popupCount++;
}

Note that the helper function contains the logic to find the button by name, or by locating any object with an interactable button. If there were multiple buttons here we could also look for one of the property values of that interactable property to ensure it is unique.

Below is the test in action.

Debugging

SmartAgents supports debugging via the Debug Server protocol, which can be used with Visual Studio Code and similar environments. This allows you to see how scripts are evaluated at runtime and troubleshoot any issues. To enable debugging of your SmartAgent scripts, follow the steps below.

Enabling the Debug Server

By default, the debug server is disabled. It can be enabled through the gdio.unity_agent.config.txt file, by adding the lines below:

<?xml version="1.0" encoding="utf-8"?>
<config version="0.1">
<!-- Other config options omitted... -->
<luaDebugging enabled="true" port="41912"/>
</config>

When enabled and in play mode, a log message should indicate that the server is listening on the configured port:

Lua debug server listening on 0.0.0.0:41912

Attaching the Debugger

Open your test project in Visual Studio Code and create the file .vscode/launch.json. Inside this file should be the following:

{
    "version": "0.1.0",
    "configurations": [
    {
        "name": "SmartAgent Attach",
        "type": "node",
        "debugServer": 41912,
        "request": "attach"
    }
    ],
}

With this configuration, you can connect to a running game’s debug server by clicking Start Debugging or by pressing F5.

Debugging Scripts

When running a script via ExecuteScript or ScheduleScript, the debugger should pause immediately upon invocation. The script will come into focus in VSCode and from there the user can step through the program.

Remote Debugging

Currently, remote debugging only works with port-forwarded local devices (eg. An Android device plugged into the local machine and forwarded via ADB). When running GameDriver on an Android device, it is required to forward the GDIOAgent communication port for the test client to connect:

$ adb forward tcp:19734 tcp:19734

When debugging a Lua script running on an Android device, you'll additionally need to forward the configured debug server port:

$ adb forward tcp:41912 tcp:41912

Summary

GameDriver SmartAgents allows you to create simple or even complex agent-side scripts to execute independently during your test execution, enabling you to build more complex automated tests for your application or game. For feedback or additional information on how to use SmartAgents in your tests, please contact us by email or on Slack.