The Power of Performance Testing in Email Marketing: A Conversation with Jeanne Jennings
Written by
Netcore Cloud

Subscribe for updates

The Power of Performance Testing in Email Marketing: A Conversation with Jeanne Jennings

Published : August 11, 2023

Welcome to the Twelfth blog in our ‘Ask the Expert’ interview series, where we bring you industry experts to talk about outstanding practices in email marketing.rela

Jeanne Jennings_General Manager at Only Influencers

Jeanne Jennings is the General Manager at Only Influencers, the highly popular community of email industry professionals. She is an expert in email marketing, and a consultant who helps medium- to enterprise-sized organizations make their email marketing more effective and profitable. Jeanne is also the Founder and CEO of Email Optimization Shop, a boutique consultancy for optimizing bottom-line email marketing performance with strategic testing.

In this Q&A blog, Jeanne Jennings talks about the importance of performance testing and the relevant KPIs to measure to ensure the success of email marketing campaigns.

1. Could you tell us about your professional journey? What led you to become an expert in email marketing, and what kind of projects and experiences have you had throughout your career?

Thanks for asking!

I started my career in 1989, working for CompuServe, helping organizations (clients included Marriott and The American College of Physicians) move their communications online (these networks were the precursors to websites and intranets, and we also did email).

In the mid-1990s, when the World Wide Web brought the Internet into daily life, I jumped into the publishing industry, helping Congressional Quarterly, Reed Business Information US (formerly Cahners), and other organizations create websites and email newsletters.

Cahners gave me my first email-only job as head of email marketing and product development for their 100+ publications, including Variety. This is where I fell in love with email marketing, and I’ve never looked back.

In 2001 I went out on my own as a consultant with my boutique consultancy, Email Optimization Shop. I’ve been blessed to work with many great organizations since then, including Capital One (B2C), the Dartmouth College Fund (B2C), Hasbro (B2C), the New York Times (B2C), Verizon (B2B and B2C), Vocus (B2B, now part of Cision), and WeightWatchers (B2C).

My consulting work is focused on email marketing strategy, creative direction, and performance optimization. I do project-based consulting and customized workshop training, as well as fractional VP/Director of Digital Marketing engagements. I help companies make their email marketing programs more effective and more profitable.

2. In your experience, what role does testing play in improving the performance of email marketing campaigns?

Testing is critical for two reasons.

i. If you make changes to your program without testing them, you could be inadvertently depressing future performance. Things like changing the color of your call-to-action buttons, adding a hero image to your template, or rewriting your body copy can impact how recipients engage with your email marketing messages. You want to make sure that these edits increase rather than decrease engagement – and the only way to do this is to test them against your current version.

ii. Testing is the only way to continually improve or optimize your performance. The scientific method is the best way to test – it starts with looking at what you’re doing and coming up with hypotheses about what might improve performance. This type of testing allows you to fine-tune your email marketing efforts based on what’s resonating with your audience – increasing relevance and value – and improving your bottom line.

3. Could you discuss some critical email tests marketers should conduct? And what are the different elements of an email that marketers should consider while testing?

Everyone pretty much tests subject lines – but these are also often the least useful types of tests. This is a ‘top of the funnel’ test – it’s the start of the email interaction. A 10% lift here is less valuable than a 10% lift on a  ‘bottom of the funnel’ test – something like a call-to-action test – which is the last action they need to take before they hit the landing page to convert.

I wrote an article about 15 elements you can test a while back; some of my favorite things to test are wireframe layout, body copy, and call-to-action location(s).  The key is to brainstorm things you think will improve your performance and test those.

4. When measuring the success of email marketing campaigns, how do you determine which KPIs are relevant? Can you walk us through your process for selecting and analyzing them? And once you’ve identified the metrics that matter, how do you use them to refine and optimize your campaigns?

Your key performance indicator (KPI) is determined by your business goal, not by what element you’re testing. The conventional wisdom about using open rate as a KPI for subject line tests and that click-through rate is a good proxy for conversions or sales are fatally flawed. It’s causing many people to hurt, rather than help, their email programs when they use these as KPIs for their testing.

If you’re looking for a direct sale from your email, then your KPI should be revenue-per-email-sent (revenue generated divided by quantity sent, not including bounces). This is true whether you’re doing a subject line test, a body copy test, or a call-to-action test.

If you are looking to drive leads, then your KPI should be conversion-rate-from-sent (the number of conversions divided by quantity sent, not including bounces), where conversion is defined as a lead. Again, this is true no matter what element you’re testing.

Once you have defined a KPI for your program (or for a campaign in your program), you’ll use it on an ongoing basis to optimize performance.

5. How can email marketers maintain an ongoing testing program that stays efficient? How do you suggest marketers prioritize their testing efforts when they have limited resources or time?

It’s important to have a plan for testing that looks at least three months into the future – and to execute it. Using a scientific method based on hypotheses rather than a random approach will improve your effectiveness.

How much you test depends on your average number of sends and the resources at your disposal.

If you are a high-frequency sender, with eight or more sends a month, you should be able to do one big ‘swing for the fences’ test a month, along with three smaller tests geared toward incremental lifts in performance. Ideally, if you are sending more than one email a week, you will be doing at least one test a week.

If you are not a high-frequency sender, then apply this plan on a quarterly basis. One big test plus three smaller ones a quarter.

With a little planning, neither of these test frequencies should be onerous to implement.

6. How important is copywriting in email marketing performance? Do you recommend any specific strategies or techniques for creating persuasive and engaging copy?

For most emails, copy is king. For some things, like clothing sales, images rule. But usually, it’s the copy that will drive the sale or conversion. So, make sure it’s great.

You need to have a direct response approach – the copy has to make a case and be written in a persuasive manner. This is very different from journalism or writing press releases.

Effective email marketing copy incorporates features, benefits, and advantages of your product or service, in addition to overcoming common objections. It needs to be written in an inverted pyramid style, with the most important and most engaging information first.

7. What is the role of Email Service Providers (ESPs) in helping marketers successfully implement email performance tests and grow their email ROI?

Email marketing platforms need to support scientific methods and effective KPIs.

Many platforms offer industry benchmarks to help customers identify where their program is lagging or leading competitors. That’s helpful.

In an effort to make testing easier, many platforms have created testing structures that limit what can be tested and, more importantly, what KPIs will be used. As discussed above, your business goal, not what element you’re testing, should determine your KPI. So automatically using open rate as your KPI when it’s a subject line does a disservice to your users.

8. AMP emails/Interactive emails are picking up as a trend in email marketing. What are your views on the performance testing of AMP emails?

You should performance test AMP just as you would any other element of your email program. Your initial tests should focus on determining if there is a lift in performance from using AMP and, if so, quantifying it.

AI is the ‘bright shiny thing’ in email (and elsewhere) right now. And that’s not a bad thing. But AI alone isn’t enough.

AI can give you a list of things you could test in email – not unlike the article I wrote a few years ago: ‘#TBT: 15 Types Of Email Marketing Tests You Should Be Doing’. Eventually, AI may be able to analyze your recent emails and suggest things to test.

But AI alone isn’t enough. You need to collaborate with AI to get the best results. It’s not a silver bullet. And it’s not always better than what you can do on your own.

There are a lot of trends and new developments in the email world, as well as some things we’ve been talking about for years that companies haven’t fully embraced. Things like AI, AMP, automated journeys, BIMI, MPP, segmentation and targeting, the list goes on.

The key is to separate the ‘bright shiny things’ from the things that will actually move the needle on your bottom-line performance.

The Email Innovations Summit, which I program, is a great way to stay up-to-date with new things while gaining practical guidance on leveraging tried-and-true tactics to the fullest.

Only Influencers (OI), where I am the general manager, is another good resource for email marketers who want to stay on top of emerging trends. We were one of the first organizations offering detailed information on Apple’s MPP and its implications for email marketers when it was announced in late 2021.

Today, our OI blog is one of the best sources in the industry for information on using AI for email marketing. With more than 25 email experts as contributors, we are usually at the forefront of issues impacting email marketers, positive and negative, as well as giving practical tips for improving your email marketing program.

The blog is free for all to read; to really up your value, become a paying member ($200/year) and join us on our weekly members-only Zoom discussions, where attendees converse on recent blog posts and other issues in the email industry, with authors and experts in the industry.

Considering the dynamic nature of marketing campaigns, testing the performance and tweaking the parameters consistently is the key to success.

By rigorously evaluating the efficiency and effectiveness of campaign elements, businesses can optimize their strategies to drive engagement and conversions. Key Performance Indicators (KPIs) provide valuable insights into the campaign’s overall health and impact, and act as a guide for course corrections.

Netcore are leaders in deploying comprehensive performance testing and monitoring critical metrics to enable businesses to deliver impactful messages, foster customer relationships, and achieve remarkable ROI in their email marketing. Connect with us to understand how you can benefit from our expertise and experience. We send over 20 billion emails a month on behalf of 6500+ businesses across 40 countries.

Unlock unmatched customer experiences,
get started now
Let us show you what's possible with Netcore.
Written By: Netcore Cloud
Netcore Cloud
Admin @ NetcoreCloud