Transparency or lack of it to be precise, is a big issue in ad-tech industry. Since inception, Kayzen has always provided full transparency around every aspect of programmatic ad-buying – apps where ads are shown, users who are being targeted, intricate campaign details , even media cost per individual impression and this transparency is something that we are proud of.
However, there is another key component of “ad-tech” which is always a black-box for advertisers and that is the “Machine Learning” model of the platform that the advertiser is using to buy media. Today, we are excited to announce that Kayzen has gone a step further and have added more transparency around how our ML models are working and impacting your campaigns and all this information is available in your Kayzen dashboard.
There is no reason why ML models need to be shrouded with so much mystery. Open and transparent communication about model intricacies help our advertisers understand the ML models in a much better way which enables them to share relevant feedback with us thereby fueling further model enhancements- a win-win situation for all
- Kayzen UI now displays 3 new CPI model specific attributes- CPI Algo State, CPI Algo Factor, Auto BL. For reference, here’s a high level documentation on how our models work. For campaigns running on CPM or CPC models, these metrics will show NA values
- CPI Algo State- The CPI model can either be in an Explore or Optimization phase. In the explore phase, the model tries to explore as much inventory as possible in an intelligent way to figure out patterns and pockets of inventory that works for your campaign. In this phase, the model's primary goal is not to achieve the target CPI; rather the focus is on learning. In the Optimization phase, the model uses its learning and focuses on achieving the target KPI.
The CPI Algo State always shows the current state of the model for each campaign. This means changing date range filter will have no impact on this attribute’s value. If you want to see the historical values of CPI Algo State, please refer to changelogs (explained below)
3. CPI Algo Factor- CPI Algo factor is a lever that our system uses to achieve maximum scale given the campaign constraints. In the “Explore” mode, with relatively relaxed budget constraints, the factor is usually >1 since the system tries to spend Explore budget in an optimized way to garner learnings. When the campaign moves from “Explore” to “Optimize” mode, usually you will notice a drop in factor value which will then stabilize around the new value. If the achieved CPI (aCPI) is < target CPI (tCPI), factor will be > 1 and if aCPI > tCPI, the factor will be < 1.
The CPI Algo Factor always show the current value of the factor for the campaign. This means changing date range filter will have no impact on this attribute’s value.
4. Auto BL- Auto BL stands for Auto Blacklist. This is another model that looks at performance of app placements and calculates the probability that whether an app will perform well for your campaign or not. If the Auto BL model predicts that the app is not a good fit for your campaign, it will automatically blacklist the app for the specific campaign. Once an app is blacklisted, it will remain blacklisted for some days but it can be given a chance again after few days to test and see if the performance of the app has improved or not. For instance, in the below ss, for campaign ID 144353, the model has blacklisted 5 publisher apps
The Auto BL always show the current value of the blacklist status for the campaign. This means changing date range filter will have no impact on this attribute’s value. If you want to see the history of which apps were added or removed to blacklist , please refer to changelogs (explained below)
Other Dashboard changes
- The campaign details page now shows two sections-
Running Apps- The apps on which the campaign is serving
Blacklisted Apps- Apps which are blacklisted. There are 3 reasons why an app might get blacklisted
a. Manual BL- An app which is manually blacklisted by the user. Please note that if an app is manually blacklisted by a user, it will take some time before the app moves from Running to Blacklist section in the UI
b. Auto Exclude- An app which is auto excluded because the user has set a manual rule to blacklist an app which doesn’t satisfy certain performance criteria. Eg Blacklist an app if the achieved CPI > $ 3 after spending more than $100. This is a static rule applied by a human and has no ML involved
c. Auto Blacklist- An app which is blacklisted by the Auto Blacklist algorithm.
2. Changelogs- You can also see historical values of CPI Algo State and which apps got Auto BL or were removed from the Auto BL by the algorithm
Should you have more questions or feedback about this release, do reach out to your Programmatic Success Specialist