Creative Testing campaign best practices

Review the recommended best practices for Creative Testing campaigns with Unity Ads User Acquisition.

Read time 3 minutes

Last updated 2 months ago

Before setting up your test campaign, take some time to consider what you want to discover about your creatives. The following information about test parameters and best practices can help you set up the best test campaign for your needs.

Creative testing parameters

This section details the parameters to consider before starting your test campaign. For step-by-step instructions on how to set up your campaign, refer to the Start a Creative Testing campaign documentation.

Creative pack quantity

There is no limit to how many creative packs you can include in a test campaign. However, adding more packs will reduce the impression volume each pack receives. This happens because the campaign distributes impressions across all packs in the test.

Creatives combinations within packs

To test how different combinations of end cards, videos, and playables perform for a particular market, you can create packs with different combinations of the same creatives. A Creative Testing campaign provides a fair opportunity for impression delivery, which can help to identify which combinations are statistically most successful. For more information about creative pack configurations, refer to the documentation on Creative packs.

Country targeting

When conducting a test campaign, it's recommended best practice to test creative packs in the same market you intend to use the creative packs in real implementation. Within this scope, you can select multiple countries for one test campaign, or you can create multiple test campaigns that each target a different country.

Recommendations for best results

The following are the recommended best practices for conducting a Creative Testing campaign:

Test with a control creative pack

To get the most exact results, the recommended best practice is to use a control creative pack. A control pack is a creative pack you have already used within Unity's ad networks. Using a creative pack that already has a performance history will yield results much faster than using only new, untested creative packs in your experiment.

Note

Although a control pack isn't a prerequisite for creative testing, running a test campaign with only new creative packs will result in a much slower testing process.

Calculate sample size

Before acting on the results of your Creative Testing campaign, it's recommended best practice to calculate the number of impressions needed to be confident in the results. A reputable test calculator such as the Sample Size Calculator from Evan Miller will calculate how big your impression sample size needs to be to deliver statistically significant results. Sample size calculators will help you identify whether the differences in your creative packs' performance are significant enough to make changes to your user acquisition strategy.

Test with a hypothesis

As with any experiment, Creative Testing campaigns are most effective when used to test a theory or hypothesis. This means it's best practice to have an idea of something specific you expect to learn from the campaign. For example: If you want to test how a video creative performs when paired with differing end cards, you can build a Creative Testing campaign that delivers creative packs made up of different combinations of the same creatives. If you want to test which playable creatives get the most clicks, you can build a creative testing campaign and use the simplified overview to compare clicks across creatives.

Keep creative packs consistent

Keep the creative packs in your test as consistent as possible. Significant differences between packs can affect how impressions are distributed, which can influence the results of your experiment. Differences often occur when packs contain sensitive content or have different start times. Sensitive content includes themes like gambling, violence, or other material that might receive a higher age rating during human moderation. For more information about Unity's moderation process, refer to the Creative pack moderation documentation. Start-time differences can occur when you add or update creative packs after the campaign has already started. For more information about how to adjust start times, refer to the Change or add creative packs section of this documentation. If these differences cause a noticeable imbalance in impression delivery, the dashboard displays an Uneven Distribution warning. For more information on this warning, refer to the Campaign management section of this documentation.

Maintain consistent testing parameters

The recommended best practice for Creative Testing campaigns is to avoid adding or removing creative packs during an active test campaign. Although you can pause a Creative Testing campaign to add additional creative packs, doing so can lead to inaccurate and misleading results. To learn more about the effects of adding creative packs to a live Creative Testing campaign, continue to the Campaign management section of this documentation.

Documentation

Development

User Acquisition

Monetization

Industry