Matchmaker A/B testing

Test and optimize matchmaking configurations by comparing different rule variants and their performance metrics.

Read time 2 minutes

Last updated 2 months ago

Important

Unity Matchmaker will support Multiplay Hosting until the deprecation date of March 31st, 2026. To aid your migration away from Multiplay Hosting, migration examples for alternative hosting providers are available. Matchmaker will continue to work with Relay and Distributed Authority after the Multiplay Hosting deprecation.

Finding the right matchmaking rule that provides the best game experience is important for player engagement. A/B testing is a feature that allows you to test, evaluate and optimize matchmaking rules without tampering with game operations or revenue. You can experiment with different matchmaking rule configurations and compare match outcomes. The A/B testing dashboard displays real-time metrics about how the different matchmaking rules are performing. This data helps you evaluate and decide which ones align with your goals. The dashboard displays retention, engagement and monetization data to give a full picture of each of the matchmaking rule variants. You can also use the dashboard real-time data to tweak the matchmaking rules to best fit your needs.

Requirements

Note

If the Analytics SDK isn't installed, tickets are silently sent to your pool's primary variant.

To use A/B testing, install the latest SDK package for the following services:

In the Unity Editor's Package Manager, select Unity Registry.
Search for the following packages, then Install them:
- Matchmaker:
  - For Unity 2022 LTS and later:
```
com.unity.services.multiplayer
```
  - For earlier versions of Unity:
```
com.unity.services.matchmaker
```
- Analytics:
```
com.unity.services.analytics
```

To set up A/B testing

In the Unity Dashboard, go to Matchmaker > Queues > A/B Testing. Select a pool and then A/B testing.
Navigate to the pool variant you want to do the A/B test on and select A/B Test with Game Overrides.
Select Create Override and then enter an A/B test name.
Set the target audience that you want to aim your A/B test to:
- Audiences: Build custom stateful audiences using Unity Analytics tracking.
- JEXL is not supported at the moment.
Choose your content type and create different matchmaking rule variations.
Set when the A/B test will run and how long it’ll run for. Adjust the start and end dates for your A/B test.

Troubleshooting

Why aren't my tickets showing up in the variant pool?

First, check the A/B test result using the following code:


var ticketResponse = await MatchmakerService.Instance.CreateTicketAsync(players, options);object abTestingSerialize = (object)ticketResponse.AbTestingResult;string abTestingJsonOutput = JsonConvert.SerializeObject(abTestingSerialize, Formatting.Indented);Debug.Log(abTestingJsonOutput);

The A/B test result should return an object similar to the following example:


{  "isAbTesting": true,  "variantId": "aaaaaaaa-cdf2-4752-bae1-52ecc90d38c6:Variant 1",  "poolId": "bbbbbbbb-722f-40f2-87f4-28ec307b7d0c",  "basePoolId": "cccccccc-39a0-42ca-a15e-77544d7ffe41"}

The A/B test result might return

null

, if the:

Analytics SDK is not installed.
Pool variant has been deleted.
Game Override is disabled, expired, or otherwise misconfigured.

In any of these cases, tickets will be sent to your pool's primary variant. To fix, review the initial steps on this page to validate your configuration.

Limitations

The current version of Matchmaker A/B Testing does not currently support the following Game Overrides features:

Contextual data attributes (JEXL) for player targeting are not supported at the moment.
On the Unity Dashboard, in Game Overrides, the Reporting tab does not show results from Matchmaker A/B tests. However, users can view results from the graphs in the Reporting tab.
In the Reporting tab, Total participants refers to the total number of participants involved in the A/B tests, but it is not equal to the total number of confirmed players who have been served by the Matchmaker configurations.

Documentation

Development

User Acquisition

Monetization

Industry