Pick a campaign and hit “Edit”, go to the Creatives section and add desired creative variations (variation A, Variation B, etc.). Once a new creative variation is added Kayzen Creative Optimization will divert sufficient traffic to explore the merits of new variation(s). Your A/B test will need to run between 7 and 14 days. This is required so that Kayzen can get the sample size necessary for you to determine a winner.

Reports

You can create a performance report in the Reports tab and immediately download it from the system. You can also save reports to be viewed later and they can be seen in the Overview tab. 

Click on ”Create A New Report” and choose the report type from the dropdown menu. Choose the metrics you would like to analyze in the reports. Select the “Filter By” and “Group By” options, based on the type of insights you'd like to derive from the reports. Make sure you have “Creative name” checked in the “Group by” filter.

Dashboard 

Kayzen Dashboard provides a quick overview of creatives’ metrics, allowing you to monitor and compare performance without the need to dive deep into reports. “Performance by Creative” table lists all the creative variations per filtered campaign(s). You can easily edit/remove your desired performance metrics from the widget editor.

In the context of A/B testing experiments, statistical significance is how likely it is that the difference in performance between your creative variations isn’t due to error or random chance. For instance, if you run a test with a 95% significance level, you can be 95% confident that the differences are real. When testing different creative variations to find out which one is more appealing to your audience, make sure the difference in their results is statistically significant before deciding which one to keep.

You can use the Statistical Significance Calculator to calculate the statistical significance of your results. Plug your desired metrics and run tests with 95% and 90% significance level. For instance, if you wish to test conversion rate between two creative variations, plug click and install values of Variation A and Variation B; the statistical significance calculator will tell you if the difference in performance is real or random chance.