Sender and subject line often decide whether a newsletter is opened at all. In this guide, you set up an A/B test in Mailchimp that compares two sender names against each other. You will also learn how to prepare subject lines, variants, and content for the test.
The video is only loaded from YouTube when you play it – data (including your IP address) is then transmitted to YouTube/Google. More in the privacy policy
A note in advance: The Mailchimp interface is updated regularly. The labels and the order of the buttons may therefore differ slightly. However, the logic of the process remains the same.
Getting started with the A/B test
First, log in to your Mailchimp account. On the left side, you will find the area for creating new content. Click on "Create" there and then select "E-Mail". You take this path every time you create a new campaign.
Then scroll down the page until you find the "A/B-Test" option. This type allows you to directly compare two or more variants of a campaign without having to set up two separate sends.
Next, assign a unique name to your campaign, for example "Fifth Newsletter". This way you keep track of things when you later run several tests in parallel. Then confirm the step with "Beginn".
Defining the target group
In the next step, you select the target group you want to test with. In the example, the audience "Gesunde Ernährung" is used. Make sure your target group is large enough. With very small lists, a test delivers only little meaningful results, because the differences can be due to chance.
Click on the appropriate audience and then select "Next". This takes you to the step where you define what exactly is to be tested.
Selecting the test factor "From Name"
Mailchimp offers various factors for A/B tests. In the last video, it was about the subject line. This time, you test the sender name, that is, the "From Name" entry. For the subject line, Mailchimp compares the open rates. For the sender name, you select here the option that you actually want to check.
So click on "From Name". You can now test two names against each other. Think beforehand about which hypothesis you want to test. A personal name with an addition, such as a job title, often seems more trustworthy than a plain company name. Whether this applies to your target group is only shown by the test.
Defining the criterion for the winner
Mailchimp needs to know by which criterion it determines the better variant. This decision is important, because it determines which variant is automatically considered the winner. In the example, the click rate is used. It shows how many recipients clicked on a link after opening.
The time period is set to two days here. After this time, Mailchimp evaluates the results and sends the winning variant to the remaining recipients. Choose the time period to suit your newsletter. For time-critical content, short windows make sense; for offers with a longer duration, longer ones are more appropriate.
Confirm with "Next" to proceed to the next step.
Entering campaign information and subject lines
Now you fill in the campaign information. Enter the campaign name, such as "Fifth Newsletter", and select the inbox management for your sender. Via the "Personal Field" field, you can additionally insert the recipients' first name. In the example, the "First Name" field is used for this. This way, a personal salutation appears in the subject line.
Scroll further down and enter the subject lines. In the example, they read "5 tips to stay healthy in winter". Below that follows the preheader, that is, the short preview text that many email programs display next to the subject line. Here you can tease the first tip, for example with a sentence that arouses curiosity.
Create variants with different senders
Now comes the core of the test. Scroll to the "Variations" section. There you create a separate sender for each variant. In the first variant, enter your name and add the matching email address below it. Make sure the address comes from your domain and has already been verified in Mailchimp, otherwise sending will be rejected.
Then switch to the second variant and enter a different name. In the example, the name is supplemented with a job title, such as "Entrepreneur and Lecturer". Here too, you select the matching email address. Make sure that the two variants really differ only in the sender name. Only then can you clearly attribute the difference to the name.
Next, scroll on to the tracking information. Mailchimp automatically records opens and clicks here. You only need to make sure that the tracking options are active so that the test can be evaluated.
Once you have entered both variants, you check which option performs better. This evaluation takes place automatically after sending. Then complete the step by clicking the "Next" button.
Design the newsletter content
In the next step, you create the actual content. For this example, the "Basic One Column" template is used. It is simple and well suited for focusing on the text. Select the template, and it opens in the editor.
In the editor, you can edit each block individually. You can remove the logo by clicking the block and deleting it with "Delete" via the trash can icon. Afterwards, you can add a new image block and drag and drop it to the desired position. There you can upload your own photo, use an image from the Mailchimp library, or insert a GIF.
For the text, a click into the field is enough. You can make words bold or italic, underline them, and add links. To do so, highlight the text and select the link icon. With a file link, you can also provide PDF documents that recipients can download directly. This is handy if you want to send a checklist or a guide, for example.
Via the top menu items, further design options are available to you. You can create bulleted and numbered lists, change fonts, adjust the text size, and modify the line height. In the settings, you can also specify whether the content should be displayed in one or two columns. In the example, it stays with one column because it usually looks best on a smartphone.
Review and send
When the content is finished, click "Confirm before sending" at the bottom. You will then arrive at the overview. There you can see once more the two senders you want to test against each other.
Check all links, images, and subject lines before sending. A typo in a subject line cannot be corrected after sending. If everything is correct, you save the campaign or send it directly via the "Send" button.
After sending, Mailchimp evaluates the test over the specified period. The winning variant is then sent to the remaining recipients. In this way, you learn over time which sender name works best for your target audience.
Summary – Setting up a Mailchimp A/B test for sender names correctly
With an A/B test in Mailchimp, you can check which sender name achieves the higher click rate with your audience. You create the campaign, choose the audience, set the test variable "From Name" and the winning criterion, design the content, and send the newsletter. Make sure your audience is large enough and test only one variable at a time, so your results remain meaningful.
FAQ
Why should I change only one variable per test?If you change the sender name and the subject line at the same time, the success cannot be clearly attributed to either of the two changes.
What click rate is considered a good result?That depends heavily on the industry and the audience. Therefore, compare your figures primarily with your previous newsletters, not with general averages.
Can I choose the test duration freely?Yes, you can adjust the time period to suit your newsletter. For time-sensitive content, a shorter window is a good choice; for offers with a longer duration, a longer one.
Why do I have to verify my sender address?Mailchimp only sends from addresses whose affiliation with your domain has been confirmed. This protects against misuse and improves the deliverability of your emails.
