A few months ago, the salesman at the nursery I buy plants from sold me a new, advanced fertiliser, with the promise that it would cure all my concerns about the roses not blooming enough.
I used the fertiliser in a few of the pots – not all – to test its effectiveness. After a couple of weeks, I observed that the roses did bloom…in all the pots! Apparently, it was the weather that needed blaming. There was no magic in the fertiliser I had purchased!
The result: Some money wasted, but many lessons learnt, the hard way!
Do you see where I’m going with this?
As marketers, you spend precious time and effort sending optimised and personalised campaigns with the intention to generate more revenue. But are you sure that it is your campaigns that are driving the business numbers? How much do your marketing campaigns actually affect customer behaviour?
How can you figure out if it was your Diwali blockbuster campaign that shot up apparel sales or was it the festive season that you should be attributing the sales numbers to?
It may not be the campaigns always that you can attribute your rising (or stagnant) revenues to.
Well, it isn’t too hard to find out.
This is where Control Group Testing comes into play, and Smartech allows you to leverage this feature to drive up the effectiveness of every marketing campaign that you run.
What are Control Groups?
A control group is a section of the customer segment you are targeting with a campaign to whom you DO NOT send the marketing campaign. The rest of the customers to whom you send the campaign constitute the test group.
You then track and measure specific activities (that you intended to promote via your campaign) of both the control group and the test group. The difference between the performances relevant to both groups gives you eye-opening insights into how effective and profitable your campaign is.
Also Read: Analytics-based Smart Segments for Targeted Marketing.
Why are Control Groups Important?
To exclude a segment of customers from your marketing campaigns is always a hard pill to swallow. On the surface, it equates to a loss in potential revenue or customer engagement getting impacted, but trust us; control groups are critical to the long-term success of your overall marketing strategy.
Here are the two major reasons why control groups are significant:
A. Identify and eliminate the unnecessary marketing spends
Imagine a customer who wants to buy a Bluetooth speaker from an e-commerce app. He almost completes the purchase but the transaction does not go through. As a result, the customer drops off.
The next time he launches the app, he receives an in-app message offering a whopping 20% off on that very brand of speakers and he goes ahead to complete the transaction.
In such a scenario, did the in-app marketing campaign specifically result in a conversion or was the customer in question going to purchase the item regardless?
For the marketer like you, it may appear that the promotional campaign was the winner. Actually, the ROI is lower, because you did not have to necessarily spend the offer to gain this conversion.
With control group testing you can now identify that there are customers out there who are ready to buy even without the offered discount.
Related Download: 4 Tips to improve conversions from App Push Notifications
B. Recalibrate the Marketing Strategy
If the control group behaves the same way as your test group, it clearly states that the campaign was pointless. The last thing you want to do is to invest in a campaign you don’t need to. Control group testing is one of the easiest ways to gauge what’s working and what’s not.
4 Tips to Help You Conduct Effective Control Groups Testing
While control groups are usually a random subset of customers culled from the overall set, there are certain points you must keep in mind to ensure that meaningful insights are unearthed:
1. Consider a significant group size: If your control group comprises only a handful of customers (say 50-100), and 100% of them pass your behavioural test, you cannot trust the results. The group should represent a good mix of the audience that you are trying to gain insights about. A standard number for the size of the control group is 10 percent of the size of the campaign or test group
2. Ensure you aren’t comparing apples to oranges: If the control group comprises all male millennials living in Mumbai, and the test group constitutes female users above 40 years of age and living in Chennai, and you are testing a campaign promoting IPL tickets for a match in Mumbai, the results of control group testing would be utterly meaningless. So, compare two groups that are actually comparable.
3. Limit the number of variables you want to measure: Too many variables may prevent you from getting actionable and reliable insights. Measure just the number of conversions from a promotional sale campaign, and find crisp results
4. Allow the test to take the time it requires: Occasions, days of the week, holidays, seasons, etc. play a role in the performance of a campaign. Also, the longer a test runs, the more likely that you’ll be able to produce statistically significant results
When Should Control Groups Not be Used?
Although we recommend using control groups for all your campaigns, there can be situations when you need to send the message to 100% of your audience. So, control group testing is not a thumb rule.
If your campaign is regarding an update in your app, or about a revision in your subscription policy, or a breaking news, you want your entire audience to receive the message. For such essential and urgent notifications, control group testing can be avoided.
Also, if the size of the recipient audience is quite small, then control groups would not give very significant results. There is no perfect size, but as a general rule, control groups are passable for customer bases smaller than 1000 people.
For a data-driven marketer like you, it is risky to assume that sending a message can only improve the business objectives, or whenever a user receives a message and then does something valuable, the message is what drove the valuable action.
What testing methods do you use to measure ROI?
Empower yourself with yet another sure-shot way to gain crystal-clear clarity regarding your marketing efforts. To know more about Control Groups, get started, today!
