Recently, one of our customers tried to run an A/B test as part of a blast email he was sending out. He ran into multiple issues while trying to keep his old email blast program structure intact, and contacted us for help. The A/B tests he wanted to set up were:
- The email’s content (‘Whole Email’)
- The email’s subject line (‘Subject Line’)
- The email’s date and time (‘Date/Time’)
As part of the A/B test, our customer wanted to use the ‘declare winner - automatic’ option. It was most important to him to keep all processes intact since the batch campaigns program templates worked very well for him until now. He just wanted to add the A/B test functionality to the flow. As you may know, Marketo’s built-in A/B testing is only meant to be used in an Email Program. Using an email program easily allows you to create a test, to determine the date and time of the test and the date and time of the main blast. The A/B testing dashboard is most helpful. It has a step-by-step structure, ending at the last step of approving the program.
Our customer's program template had nine smart campaigns. The first of these was designed for sending the email, and the other eight were used for ‘program status change.’ This means the actions that the lead performs after receiving the email. Some of these actions were connected directly to the email statistics – such as ‘open’ and ‘click’, while others were related to activities in Salesforce.
All of the triggers and filters used in these eight smart campaigns (see below for more details on the send email smart campaign) used the original email name.
As some of you may know, when using an email within an email program for A/B testing, Marketo will NOT recognize it anywhere else. This can be a major hindrance for some users, and for our customers, it definitely was one. The old structure of the program became irrelevant, and it was certainly possible that the tracking system of the program may have become irrelevant as well.
The next step is to fill in all of the fields in all triggers and filters that require the email name with the following syntax:
[parent program name].[email program name].[A/B test name] test
For example:
Parent program name: Black Friday
Send email smart campaign:
Using an email program makes this process impossible since it was built by Marketo as a smart campaign with one flow step – send an email.
Our solution was to create a "preparation" smart campaign. Its smart list included the population for the program, and the flow step separated it into a control group and a target group (via change program status).
Change Program Status
The last step in the process is to set up the smart list of the email program to include only the recipients marked as "Target Group.”