Matchmaker A/B testing

Test and optimize matchmaking configurations by comparing different rule variants and their performance metrics.

Read time 3 minutes

Last updated 22 days ago

Finding the right matchmaking rule that provides the best game experience is important for player engagement. A/B testing allows you to test, evaluate, and optimize matchmaking rules without tampering with game operations or revenue. You can experiment with different matchmaking rule configurations and compare match outcomes.

The A/B testing dashboard displays real-time metrics about how the different matchmaking rules are performing, which can help you evaluate and decide which ones align with your goals. The dashboard displays retention, engagement, and monetization data to give a full picture of each of the matchmaking rule variants.

Requirements

To use A/B testing, you need to use the latest SDK package for each of the following services:

Multiplayer Services
Analytics

The Multiplayer Services SDK contains the Unity Matchmaker Service API. If the Analytics SDK isn't installed, tickets are silently sent to your pool's primary variant.

To install the latest packages, do the following:

In the Unity Editor's Package Manager, select Unity Registry.
Search for the following packages, then Install them:
- Multiplayer Services:
```
com.unity.services.multiplayer
```
- Analytics:
```
com.unity.services.analytics
```

Set up A/B testing

To set up A/B testing in your project, do the following:

In the Unity Dashboard, go to Matchmaker > Queues > A/B Testing. Select a pool and then A/B testing.
Navigate to the pool variant you want to do the A/B test on and select A/B Test with Game Overrides.
Select Create Override and then enter an A/B test name.
Set the target audience that you want to aim your A/B test to:
- Audiences: Build custom stateful audiences using Unity Analytics tracking.
- JEXL is not supported at the moment.
Choose your content type and create different matchmaking rule variations.
Set when the A/B test will run and how long it’ll run for.

Troubleshooting

Why aren't my tickets showing up in the variant pool?

First, check the A/B test result using the following code:


var ticketResponse = await MatchmakerService.Instance.CreateTicketAsync(players, options);object abTestingSerialize = (object)ticketResponse.AbTestingResult;string abTestingJsonOutput = JsonConvert.SerializeObject(abTestingSerialize, Formatting.Indented);Debug.Log(abTestingJsonOutput);

The A/B test result should return an object similar to the following example:


{  "isAbTesting": true,  "variantId": "aaaaaaaa-cdf2-4752-bae1-52ecc90d38c6:Variant 1",  "poolId": "bbbbbbbb-722f-40f2-87f4-28ec307b7d0c",  "basePoolId": "cccccccc-39a0-42ca-a15e-77544d7ffe41"}

The A/B test result might return

null

for the following reasons:

Analytics SDK is not installed.
Pool variant has been deleted.
Game Override is disabled, expired, or otherwise misconfigured.

In any of these cases, tickets will be sent to your pool's primary variant.

To fix, review the initial steps on this page to validate your configuration.

Limitations

Matchmaker A/B Testing doesn't support the following Game Overrides features:

Contextual data attributes (JEXL) for player targeting.
On the Unity Dashboard, in Game Overrides, the Reporting tab doesn't show results from Matchmaker A/B tests. However, users can view results from the graphs in the Reporting tab.
In the Reporting tab, Total participants refers to the total number of participants involved in the A/B tests, but isn't equal to the total number of confirmed players who have been served by the Matchmaker configurations.

Documentation

Development

User Acquisition

Monetization

Industry