A/B testing#

A/B testing is a powerful tool to test different variables on various user segments in parallel to see which variable has the most impact on your game. A/B tests can be very useful for fine-tuning retention, monetization, or balancing mechanics. This sample simulates an A/B test to evaluate how much experience it should take to gain a level.

Prerequisites#

To use this sample use case, you must download and install the UGS Use Cases project in your Unity project.

Overview#

To see this use case in action, open the samples menu and navigate to AB Test Level Difficulty. To open this scene directly and interact with the use case:

1. In the Unity Editor Project window, select Assets > Use Case Samples > AB Test Level Difficulty, then double-click ABTestLevelDifficultySample.unity to open the sample scene.
2. Press Play to enter Play mode.

When a player signs in, the screen displays their level and an XP meter that tracks their current XP and the total amount of XP required to level up. In this case, the amount of total XP to level up is either 100, 80, 60, 50, or 30, depending on which test group the player is randomly assigned. For diagnostic purposes, the player's test group is also displayed (A, B, C, D, or E).

Initialization#

The ABTestLevelDifficultySceneManager.cs script performs the following initialization tasks in its Start function:

1. Initializes Unity Services.
2. Sends a SceneOpened custom event to the Analytics service. This, along with the SceneSessionLength custom event (sent when you click the scene's back button), tracks the player's session length as an example of how you can implement analytics to evaluate the results of your test.
3. Signs in the player anonymously using the Authentication service. If you’ve previously initialized any of the other sample scenes, Authentication will use your cached Player ID instead of creating a new one. If not, your initial level (1) and XP (0) are saved in Cloud Save instead of loading existing data.
4. Retrieves and updates the player's currency balances from the Economy service.
5. Queries the Remote Config service to fetch the appropriate test group name and corresponding XP threshold for leveling up.

All of this updated data is displayed in the scene, and the Gain XP button becomes interactive.

Functionality#

Gain XP#

When you click the Gain XP button, the simulated player gains 10 XP. When the XP bar is full (according to the test group threshold), the simulated player levels up and receives currency. The following occurs:

1. The button’s OnClick method sends an ActionButtonPressed custom event to the Analytics service and makes a call to the Cloud Code GainXPAndLevelIfReady.js script.

2. This server-authoritative call fetches the player's information from the Cloud Save and Remote Config services, increases the XP by the amount specified by Remote Config (in this case, 10), then tests whether the new player XP total equals or exceeds the level-up threshold.

   • If the player did not level up, their new XP is saved in Cloud Save and returned to the client, which updates the values and shows text that indicates how much XP increased.
   • If the player did level up, it calls the Economy service to distribute the level-up rewards (in this case, 100 Coins), and calls Cloud Save to save the increased player level and new player XP before it returns that information to the client code.

3. The client code opens a level-up dialog and updates the relevant data in the scene UI. Note that this step provides an opportunity to show players different art based on their segmentation, and a convenient way to retrieve additional data for a specific currency at runtime.

Note: In this example, a cross-reference dictionary located in Remote Config and initialized at start-up converts the rewarded currency ID (in this case, COIN) to an Addressables address, which you can use to display the sprite (in this case, Sprites/Currency/Coin). A simpler approach to displaying different art according to a player’s segmentation is to attach additional information to the custom data associated with Currencies in the Economy configuration data. However, here the data was added to the Campaigns to demonstrate the flexibility of the Remote Config service.

Sign In As New Player#

Click Sign In As New Player to simulate a new player with a new randomly assigned test group that starts with 0 XP. The following occurs:

1. The button’s OnClick method deletes the current anonymous player ID, resets all cached values to their empty or default states, and then initiates a new sign-in with the Authentication service that follows the same flow as when the scene first loads.
2. As with the Gain XP button, it sends an ActionButtonPressed custom event to the Analytics service.
3. Each time you click the Sign In As New Player button, the Authentication service creates a new anonymous player ID, to which the Remote Config service assigns a new A/B test group.

Back button#

If you press the back button (the arrow in the top-left corner of the scene) to return to the "Start Here" or samples menu scene, it triggers a SceneSessionLength custom Analytics event, which captures the amount of time spent in this scene.

Setup#

Requirements#

To replicate this use case, you'll need the following Unity packages in your project:

Note: Although it is listed as a package and requires separate dashboard configuration, Game Overrides doesn't actually have an SDK to install from Package Manager. It is a server-side offering that affects values returned from other services.

To use these services in your game, activate each service for your Organization and project in the Unity Cloud Dashboard.

Unity Cloud services configuration#

To replicate this sample scene's setup in your own Unity project, configure the following items:

• Cloud Code scripts
• Economy items
• Remote Config values
• Remote Config Game Overrides
• Custom events and parameters for the Analytics service

To configure these items you can use the Deployment package, or manually enter them using the Unity Cloud Dashboard. The recommended best practice is to use the Deployment package as it greatly accelerates this process.

Using the Deployment package#

To deploy configurations using the Deployment package:

1. Open the Deployment window.
2. Check in Common and AB Test Level Difficulty.
3. Click Deploy Selection.

This deploys the following items:

• Cloud Code scripts
• Economy items
• Remote Config values

The Deployment package doesn't support the following items:

• Remote Config Game Overrides
• Custom events and parameters for the Analytics service

To configure them, refer to Using the Unity Cloud Dashboard.

Using the Unity Cloud Dashboard#

You can use the Unity Cloud Dashboard to manually configure your services by project and environment. Refer to the following sections to configure this sample.

Analytics#

Important: This sample demonstrates the code that is needed to trigger Analytics events. However, additional code might be necessary to meet legal requirements such as GDPR, CCPA, and PIPL. For more information, see the documentation on managing data privacy.

Configure and enable the following custom Analytics events:

Configure the following custom parameters to support your custom events:

Note: This extended list of potential parameters allows for a more flexible analysis of different parameter groupings in the Data Explorer in the Unity Cloud Dashboard. Alternatively, you can send only the ungrouped parameters (for example, buttonName or sceneName) and perform any kind of grouped analysis you want by using the Data Export feature within the Data Explorer.

Cloud Code#

Publish the following script in the Unity Cloud Dashboard:

Note: The Cloud Code scripts included in the Cloud Code folder are local copies because you cannot view the sample project's dashboard. Changes to these scripts do not affect the behavior of this sample because they are not automatically uploaded to the Cloud Code service.

Economy#

Configure the following resource in the Unity Cloud Dashboard:

Remote Config#

Set up the following config values in the Unity Cloud Dashboard:

json
{
	"currencyData": [{
    	"currencyId": "COIN",
        "currencySpec": {
        	"spriteAddress": "Sprites/Currency/Coin"
        }
	},{
    	"currencyId": "GEM",
        "currencySpec": {
        	"spriteAddress": "Sprites/Currency/Gem"
		}
	},{
		"currencyId": "PEARL",
        "currencySpec": {
			"spriteAddress": "Sprites/Currency/Pearl"
    	}
	},{
    	"currencyId": "STAR",
        "currencySpec": {
        	"spriteAddress": "Sprites/Currency/Star"
		}
	}]
}

Game Overrides#

Configure the following Overrides in the Unity Cloud Dashboard: