A/B Testing
A/B Testing allows you to test different segmented cancel flows against each other to determine the most effective setup. Because the primary cancel flow targets all audiences that do not fall under segments, A/B tests cannot be run on it. Example A/B testing usage could include testing offers, phrasing, or whether a longer or shorter pause decreases churn.
Example Usage
Suppose you wanted to know if a longer pause of three months or a shorter pause of one month would decrease churn at your company. Your current cancel flow for a segment of customers allows the longer pause. In order to test your theory that a shorter pause would decrease churn, you could set up an A/B Test on your current cancel flow and change the subscription pause allowance to one month. After publishing the test, and after enough time has passed to see change, your A/B Test dashboard will show the running results of the test.
Best Practices
After activating your A/B Test, please allow time in order to see the effects of your test. The more time that passes, the more accurate the results. We recommend allowing at least two months before reviewing A/B test results.
Make sure to use the same customer segment for both cancel flows in your test. This ensures that there is no bias on your results.
Setup
Step One
Create your non-primary Cancel Flow.
Step Two
Select Create A/B Test from Edit Dropdown in Cancel Flows > Flows.
Step Three
Give your A/B Test a descriptive name.
Step Four
Update the copied Cancel Flow to have the new factor that you want to test against.
Then,
All you have to do is activate and stay tuned for the results! A/B tests need at least two months worth to have enough data for accurate results.
From cancel flows view, you can view A/B test results, or finish or pause your test.
From the A/B Tests tab, you can view the ongoing results of your test.