Matchmaker A/B testing
Test and optimize matchmaking configurations by comparing different rule variants and their performance metrics.
阅读时间3 分钟最后更新于 16 天前
Finding the right matchmaking rule that provides the best game experience is important for player engagement. A/B testing allows you to test, evaluate, and optimize matchmaking rules without tampering with game operations or revenue. You can experiment with different matchmaking rule configurations and compare match outcomes.
The A/B testing dashboard displays real-time metrics about how the different matchmaking rules are performing, which can help you evaluate and decide which ones align with your goals. The dashboard displays retention, engagement, and monetization data to give a full picture of each of the matchmaking rule variants.
Requirements
To use A/B testing, you need to use the latest SDK package for each of the following services:
- Multiplayer Services
- Analytics
The Multiplayer Services SDK contains the Unity Matchmaker Service API. If the Analytics SDK isn't installed, tickets are silently sent to your pool's primary variant.
To install the latest packages, do the following:
- In the Unity Editor's Package Manager, select Unity Registry.
- Search for the following packages, then Install them:
- Multiplayer Services:
com.unity.services.multiplayer - Analytics:
com.unity.services.analytics
- Multiplayer Services:
Set up A/B testing
To set up A/B testing in your project, do the following:
- In the Unity Dashboard, go to Matchmaker > Queues > A/B Testing. Select a pool and then A/B testing.
- Navigate to the pool variant you want to do the A/B test on and select A/B Test with Game Overrides.
- Select Create Override and then enter an A/B test name.
- Set the target audience that you want to aim your A/B test to:
- Audiences: Build custom stateful audiences using Unity Analytics tracking.
- JEXL is not supported at the moment.
- Choose your content type and create different matchmaking rule variations.
- Set when the A/B test will run and how long it’ll run for.
Troubleshooting
Why aren't my tickets showing up in the variant pool?
First, check the A/B test result using the following code:
var ticketResponse = await MatchmakerService.Instance.CreateTicketAsync(players, options);object abTestingSerialize = (object)ticketResponse.AbTestingResult;string abTestingJsonOutput = JsonConvert.SerializeObject(abTestingSerialize, Formatting.Indented);Debug.Log(abTestingJsonOutput);
The A/B test result should return an object similar to the following example:
{ "isAbTesting": true, "variantId": "aaaaaaaa-cdf2-4752-bae1-52ecc90d38c6:Variant 1", "poolId": "bbbbbbbb-722f-40f2-87f4-28ec307b7d0c", "basePoolId": "cccccccc-39a0-42ca-a15e-77544d7ffe41"}
The A/B test result might return for the following reasons:
null- Analytics SDK is not installed.
- Pool variant has been deleted.
- Game Override is disabled, expired, or otherwise misconfigured.
In any of these cases, tickets will be sent to your pool's primary variant.
To fix, review the initial steps on this page to validate your configuration.
Limitations
Matchmaker A/B Testing doesn't support the following Game Overrides features:
- Contextual data attributes (JEXL) for player targeting.
- On the Unity Dashboard, in Game Overrides, the Reporting tab doesn't show results from Matchmaker A/B tests. However, users can view results from the graphs in the Reporting tab.
- In the Reporting tab, Total participants refers to the total number of participants involved in the A/B tests, but isn't equal to the total number of confirmed players who have been served by the Matchmaker configurations.