A/B Testing

A/B Testing

A/B Testing Guide

icon
Make sure that your cancel flows are retaining the most subscriptions as possible.

A/B Testing allows you to test different segmented cancel flows against each other to determine the most effective setup. Because the primary cancel flow targets all audiences that do not fall under segments, A/B tests cannot be run on it. Example A/B testing usage could include testing offers, phrasing, or whether a longer or shorter pause decreases churn.

Example Usage

Suppose you wanted to know if a longer pause of three months or a shorter pause of one month would decrease churn at your company. Your current cancel flow for a segment of customers allows the longer pause. In order to test your theory that a shorter pause would decrease churn, you could set up an A/B Test on your current cancel flow and change the subscription pause allowance to one month. After publishing the test, and after enough time has passed to see change, your A/B Test dashboard will show the running results of the test.

Best Practices

After activating your A/B Test, please allow time in order to see the effects of your test. The more time that passes, the more accurate the results. We recommend allowing at least two months before reviewing A/B test results.

Make sure to use the same customer segment for both cancel flows in your test. This ensures that there is no bias on your results.

Setup

Step One

Create your non-primary Cancel Flow.

(See Quick Start Guide ->)

Step Two

Select Create A/B Test from Edit Dropdown in Cancel Flows > Flows.

image

Step Three

Give your A/B Test a descriptive name.

image

Step Four

Update the copied Cancel Flow to have the new factor that you want to test against.

Then,

All you have to do is activate and stay tuned for the results! A/B tests need at least two months worth to have enough data for accurate results.

From cancel flows view, you can view A/B test results, or finish or pause your test.
From cancel flows view, you can view A/B test results, or finish or pause your test.
From the A/B Tests tab, you can view the ongoing results of your test.
From the A/B Tests tab, you can view the ongoing results of your test.
💠
h

Log In Sign up

© Churnkey, Inc. 2024