How-to
      Back to home
      1. GameDriver Knowledge Base
      2. Unity
      3. How-to

      2D Game Kit Automation Tutorial

      Learn how to automate a 2D project developed in Unity by using GameDriver

      2D Game Kit Automation Tutorial

      Continued from this blog post ...

      Note: For this demonstration, we are using the LTS (long-term support) version of Unity 2019.4, which can be downloaded from the Unity Hub. Newer versions may not work exactly as described.

      Step 1 - Create the 2D Game Kit project
      This is a fairly straightforward process. Simply start a new 2D Project in Unity and go to the Asset Store (Window Menu > Asset Store) and download the "2D Game Kit " from Unity Technologies. From there, simply Import the assets into your project.

      Follow the prompts, and be sure to select all assets in the package to import. In newer versions of Unity (2020+), you will need to add the asset to your library, then import it via the Package Manager in the Unity Editor under Window > Package Manager, then select "My Assets" within the Package Manager window.

      Step 2 - Setting up the game to play
      Once the assets have been fully imported, which can take several minutes, simply open the Scenes folder in the Project window and open the "Start" scene.

      Once the scene is open, you can play the game by pressing the "Play" button. Give the game a try! You can control Ellen, the character, using the WSAD or arrow buttons to move, Spacebar to jump, and O and K keys to attack. The objective of the game is to find 3 keys hidden (poorly) throughout the 4 levels, which unlocks the door to the boss.

      We will automate the collection of one key in this tutorial, and give you enough tools to be able to automate tests for the rest if you wish.

      Download the GameDriver software here. Once you have the software and license, follow the installation instructions  (we'll wait here for you).

      Step 3 - Creating your first automated test
      You're back? Great. With the GameDriver agent installed, you can start building your automation in Visual Studio. If you followed the installation instructions to the end, you should have a base NUnit test to work from. Just be sure you have added the following declarations.

      Next, you will add references to the GameDriver libraries in the project references, specifically to the 3 highlighted below. These can be found in the <Unity Project>/Assets/GDIO folder from the installation instructions.

      We're going to start by creating an instance of the ApiClient class. This needs to happen outside of the [Test] and [OneTimeSetup] attributes to maintain scope across the different test methods.

      ApiClient api;

      We then create a [OneTimeSetup] method, which will be run any time a test method within this Class is executed. This allows us to run tests in isolation, without requiring every test to pass in sequence. Within the method, we will connect to the Unity IDE, which is handled with the command:

      api.Connect("localhost", 19734, true);

      This command will put the Unity editor into Play mode using the "true" argument, and connect to the GameDriver Agent on port 19734, which is the default. Other ports can be configured in the Unity editor.

      The next few lines will wait 3 seconds for the game to load, then enable the GameDriver agent hooks for keyboard and mouse control.

      api.Wait(3000);
      api.EnableHooks(HookingObject.MOUSE);
      api.EnableHooks(HookingObject.KEYBOARD);

      To ensure the game is loaded and running, we will wait for the StartButton object to be enabled, then click it to start the game and wait a few more seconds for the next scene to load.

      api.WaitForObject("//*[@name='StartButton']");
      api.ClickObject(MouseButtons.LEFT, "//*[@name='StartButton']", 30);
      api.Wait(3000);

      When the game plays, inputs are calculated by the number of frames a key is pressed, which can fluctuate depending on the performance of the machine running the game. So rather than hard code the duration of key presses, a best practice is to calculate the number of frames to input using the frames per second (FPS) captured from the game engine. First, we will capture the FPS using the GameDriver API.

      var fps = api.GetLastFPS();

      Next, we need a target object for our player character to move towards. In level 1, there is a path down to the next scene that is also the location of an InfoPost object, but not the first one in the scene. So we will find the target InfoPost, and store its x coordinate for future reference using the api.GetObjectPosition() command. We then log the value for troubleshooting purposes.

      var infoPost = api.GetObjectPosition(infoPostNum);
      Console.WriteLine("InfoPost x:" + infoPost);


      Now that we have it, we will loop over some actions so long as we're in the Zone1 scene, and until Ellen has reached the post we need to proceed.

      while (api.GetSceneName() == "Zone1")

      Zone 1

      Last thing's first. If we're near the target post, as in +/- 1 on the x-coordinate, jump down to exit the zone. We do this by testing whether Ellen's position is greater than AND less than 1, meaning we're within that range. Next, we log something to indicate the InfoPost was reached, for debugging purposes, then press the down and jump keys at almost the same time. We do this by pressing 'S' for one second using (ulong)api.GetLastFPS() and use that as an argument for the KeyPress command. Then we wait half a second using api.Wait(500), before "jumping" for half a second by dividing the current FPS by 2 as an argument. This is a long explanation for a very short IF statement:

      //If we're near the post, down-jump
      if (api.GetObjectPosition("/Player[@name='Ellen']").x >= infoPost.x - 1 && api.GetObjectPosition("/Player[@name='Ellen']").x <= infoPost.x + 1)
      {
          Console.WriteLine("InfoPost found!");
          api.Wait(3000);
          api.KeyPress(new KeyCode[] { KeyCode.S }, (ulong)api.GetLastFPS());
          api.Wait(500);
          api.KeyPress(new KeyCode[] { KeyCode.Space }, (ulong)api.GetLastFPS() / 2);
      }

      Note: For this tutorial, we have tagged the Ellen object prefab with the "Player" tag, in order to demonstrate the usage of this feature. You will need to tag this object yourself to reuse the code above or remove the references to "Player" in the code. For more information on how to add tags to an object, see the Unity manual --> here.

      But since we haven't reached the InfoPost yet, we will need to find a way to it. In this next section, we're going to move to the right while jumping, to try and avoid a rather large hole between the spawn point and the target InfoPost. This time, we test whether Ellen's x-coordinate is less than 10, and y-coordinate is less than -1 which is roughly where the left-side of the gap starts. So as long as we're to the left of that gap, keep trying.

      //If we're on the left of the gap, find a way over
      while (api.GetObjectPosition("/Player[@name='Ellen']").x < 10 && api.GetObjectPosition("/Player[@name='Ellen']").y < -1)
      {
          //Jump the chasm. This is tricky, so it might take a few tries
          Console.WriteLine("We're going to jump!");
          api.KeyPress(new KeyCode[] { KeyCode.D }, (ulong)api.GetLastFPS() * 2); //Move right for ~2 seconds
          api.Wait(500);
          api.KeyPress(new KeyCode[] { KeyCode.Space }, (ulong)api.GetLastFPS() / 2); //Jump for a 1/2 second
          api.Wait(500);
      }


      If there's a sudden drop in FPS, there's a chance Ellen will fall into the hole. So after each iteration of the previous while loop, we will test whether Ellen fell in the hole, and if so to jump left out of it. This time we check that the x-coordinate is greater than 10 (where the gap begins), and the y-coordinate is less than -3. Note that these jumps are 1/2 - 1/4 of a second, as we don't want to jump too high.

      //If we fall in the hole, need to back up and try again
      while (api.GetObjectPosition("/Player[@name='Ellen']").x >= 10 && api.GetObjectPosition("/Player[@name='Ellen']").y <= -3)
      {
          Console.WriteLine("We're stuck in the hole, please stand by!");
          api.KeyPress(new KeyCode[] { KeyCode.A }, (ulong)api.GetLastFPS()); //Move left for 1 second
          api.Wait(500);
          api.KeyPress(new KeyCode[] { KeyCode.Space }, (ulong)api.GetLastFPS() / 4); //Jump for a 1/4 second
          api.Wait(250);
      }

      Once we have successfully crossed the chasm, we simply move right until we reach the desired position. Since these movements rely on the FPS for input, there's a chance we will go too far. So after each iteration, we will check again and move back to the left if we've gone too far. The movements here are shorter, only 1/3 of a second with 333ms (1/3 of a second) wait in between. You might wonder why we do this after each movement, and that's because inputs are asynchronous. Not all tests will require a wait after each movement, but in this game, it prevents inputs from overlapping and being missed.

      //Once we have crosses the gap, move right
      while (api.GetObjectPosition("/Player[@name='Ellen']").x > 13 && api.GetObjectPosition("/Player[@name='Ellen']").x < infoPost.x && api.GetObjectPosition("/Player[@name='Ellen']").y > -2)
          {
              Console.WriteLine("x:" + api.GetObjectPosition("/Player[@name='Ellen']").x + " < " + infoPost + ". Moving Right.");
              api.KeyPress(new KeyCode[] { KeyCode.D }, (ulong)api.GetLastFPS() / 3);
              api.Wait(333);
          }

      This is where we move left if we've gone too far right.

      //If Ellen is to the right of the InfoPost, move left
      while (api.GetObjectPosition("/Player[@name='Ellen']").x > infoPost.x)
      {
          Console.WriteLine("x:" + api.GetObjectPosition("/Player[@name='Ellen']").x + " > " + infoPost + ". Moving Left.");
          api.KeyPress(new KeyCode[] { KeyCode.A }, (ulong)api.GetLastFPS() / 3);
          api.Wait(333);
      }

      Note: Several of the sections above include a Console.WriteLine method which includes Ellen's current position relative to the target. This can be useful for troubleshooting if something doesn't work.

      Finally, we test whether we have successfully left Zone1. This is done using an Assert.AreEqual method from the UnitTesting framework. So be sure to add using NUnit.Framework; to the beginning of your test. If this assertion fails, it will also fail your test.

      Assert.AreEqual(api.GetSceneName(), "Zone2", "Zone 2 not loaded!");

      Zone 2

      Once we're in Zone 2, we need to touch the key in order to obtain it. This is done first by locating the key, and then simply moving toward it using the same approach as we did in Zone 1. Only the key is on a pedestal and there is no chasm to jump over. Only, the only way to know whether we have the key is to check the color value of the key in UI. Initially, these are grayed out but become illuminated once we have found the item.

      First, we wait for the Key object to load. For demonstration purposes, we're using a combination of the object's tag (Objective), name (Key), and a check for whether the object is active in the current scene, which is the little checkbox in the Unity editor marked "Active".

      api.WaitForObject("/Objective[@name='Key' and ./fn:component('UnityEngine.Behaviour')/@isActiveAndEnabled = 'True']");

      Note: Like the Player tag above, for this tutorial, we have tagged the Key object prefab with the "Objective" tag, in order to demonstrate the usage of this feature. You will need to tag this object yourself to reuse the code above or remove the references to "Objective" in the code.

      The main loop for this section tests whether the color of the key on the screen has changed. For this, we will use the GetObjectFieldValue<T> command, which can search for any active object in the game and return a field value. In this example, the path to the Object is /KeyCanvas/KeyIcon(Clone)/Key which has an Image component.

      There are also 3 keys in the UI, so we use //*[@name='KeyIcon(Clone)'][0]/*[@name='Key'] to indicate the 1st instance (or [0]) of the object. When the key is obtained, the RGBA color value of that key will change from 1, 1, 1, 0 to 1, 1, 1, 1. The last number refers to the Alpha blending or transparency.

      while (api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 0.000)"))

      Note that we're returning the native <Color> type of the object here, and need to use the type's ToString method to perform a comparison. Otherwise, we would need to first create a Color object for that comparison.

      While the above test is true, we will loop over a set of movements in the same way we did in Zone 1, this time checking for the x and y coordinate values relative to Ellen's position. The first action following that test will exit the loop if Ellen gets the key, so as not to waste any more cycles. We're also capturing a screenshot using CaptureScreenshot, and performing a final Assert that the key color changed as expected.

      //Check if we're on either side of the key
      while ((api.GetObjectPosition("/Player[@name='Ellen']").x != keyVector.x) & (api.GetObjectPosition("/Player[@name='Ellen']").y != keyVector.y))
      {
          //If we've hit the key, break
          if (api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 1.000)"))
          {
              api.CaptureScreenshot("Zone2.jpg",false, true);
              Console.WriteLine("Key Get!");
              api.Wait(1000);
              Assert.IsTrue(api.GetObjectFieldValue<Color>("/*[@name='KeyCanvas']/*[@name='KeyIcon(Clone)'][0]/*[@name='Key']/fn:component('UnityEngine.UI.Image')/@color").ToString().Equals("RGBA(1.000, 1.000, 1.000, 1.000)"));
              break;
          }

      Next, if we are to the left of the key, move right. Within that loop, if we are lower than the key, jump.

      else
      {
          //If we're to the left, move right
          while (api.GetObjectPosition("/Player[@name='Ellen']").x < keyVector.x)
          {
              api.KeyPress(new KeyCode[] { KeyCode.D }, (ulong)api.GetLastFPS());
              api.Wait(500);

              // While moving right, if we're lower than the key, jump
              if (api.GetObjectPosition("/Player[@name='Ellen']").y < keyVector.y)
              {
                  api.KeyPress(new KeyCode[] { KeyCode.Space }, (ulong)api.GetLastFPS() / 3);
              }
      }

      If we're to the right of the key, move left. Within that loop, if we are lower than the key, jump.

      //If we're to the right, move left
      while (api.GetObjectPosition("/Player[@name='Ellen']").x > keyVector.x)
      {
          api.KeyPress(new KeyCode[] { KeyCode.A }, (ulong)api.GetLastFPS());
          api.Wait(500);

          // While moving left, if we're lower than the key, jump
          if (api.GetObjectPosition("/Player[@name='Ellen']").y < keyVector.y)
          {
              api.KeyPress(new KeyCode[] { KeyCode.Space }, (ulong)api.GetLastFPS() / 3);
          }
      }

      After all of the above, move to the right until we leave the zone. This assumes we have the key and are to the right of it.

      //Move to the next level entrance
      while (api.GetSceneName() != "Zone3")
      {
          api.KeyPress(new KeyCode[] { KeyCode.D }, (ulong)api.GetLastFPS());
          api.Wait(500);
          api.KeyPress(new KeyCode[] { KeyCode.Space }, (ulong)api.GetLastFPS() / 2);
      }

      If the above was successful, we have the key and are now in Zone 3.

      Assert.AreEqual(api.GetSceneName(), "Zone3");

      And that's it! It took 10 times as long to write this tutorial than it did to write the test above, which is attached here for your reference. You will need to copy this code into a new NUnit project, as outlined in the installation instructions linked above.

      If your test doesn't behave quite the way you expect, try different movements and wait durations between them. It can have a significant impact on the test.

      Some games such as puzzles will be much more deterministic in their inputs, meaning there is less variation in the replay and much simpler steps to produce the desired results. Other games, such as FPS or other 3D environments require a little more effort, but we can use a similar approach to what is shown here.

      Note: The object identification method used throughout this guide is a core capability of GameDriver, named HierarchyPath. It's a powerful way to identify objects and values similar to XPath. More information on how to use HierarchyPath is available in this support article.