Estimated reading time: 7 minutes
Regardless of your industry, you’ll never be able to escape trends that change the way you’re expected to do business. Like trends in clothing, some are worthwhile staples that will never fully go out of fashion like a dark, tailored blazer. And some are mustard-colored corduroy bell-bottoms; fly-by-night fads that seem like a good idea at the time, but really don’t last. A/B testing has been around for years, but is newly popular for eCommerce sites. But is it a tailored blazer or a pair of unfortunate bell-bottoms?
What is A/B Testing?
A/B testing is a method of gauging user response to changes on your site. It’s an experiment meant to find the best option between two choices: Option A or Option B. You can find this method of testing across all areas of running on online business.
Social Media and Ads
YouTubers will use A/B testing to find out which video title or which thumbnail image draws in more clicks from the site users. Facebook and Google advertisers will use it to run two different ad designs simultaneously, to see which one garners the most attention.
Newsletters and Subscriber Emails
When sending out your monthly newsletter, you can often choose to test between two different subject lines. It’s a good way to measure which style of subject line invites more people to open the email.
Page Designs and Features
If you aren’t sure about the best way to proceed with a page design, or adding specific features to your website, you can use this method to test the waters. Want to offer sales to your shoppers? You could test option A (a site-wide banner with sale details) and option B (a popup-style notification) to see which user responds to best. Note the number of times the coupon code is used in checkout compared to which test option the users were presented with.
When you use A/B testing, you implement two options, and they are evenly distributed randomly to your audience. If you’re testing on social media or pay-per-click ads (like Google), the audience is anyone within your target demographic. If you’re testing in your newsletter emails, the audience is your subscriber list. And when you test on your site, the audience is anyone that visits.
The Pros and Cons of A/B Testing
As with any other trend, and any other method of development and design, it isn’t one-size-fits-all. And mustard corduroy bell-bottoms definitely don’t work for everybody. There are things about this method that will work for some, but not be realistic or valuable to others. Let’s look at this in regards to testing on an eCommerce site.
Pros
- Data-based results. The numbers you get from testing aren’t guesstimated or predicted. They are real, fact and data-based responses from real people using your site. There’s no guesswork involved.
- Explore new ideas before committing. You have the ability to try out changes to your website before fully committing to them. You can gauge how design changes will be received by your users.
- Address specific design questions. You can explore and address specific design options to help you decide which is better. Should I use a banner or a popup? Should my buttons be red or blue?
- Clear and quantitative. The results you get from A/B testing are numerical and statistical. There’s no interpretation, like with survey responses.
Cons
- Expensive and time-consuming. A/B testing can be a drain on resources – both time and money. If you don’t get several thousand of visitors to your site per month, your testing will likely have to go on for ages before you get enough of a response to be worthwhile. By the time you get your results, the design options may already be out of date.
- Goals must be narrow and specific. This can help you if you have a specific question to ask, but most store owners need more information than that. Issues on websites tend to be more nuanced than one specific question.
- Results may be skewed without enough users testing. As mentioned above, smaller sites (even if you are still an Enterprise level business) simply don’t get enough traffic to provide effective results. When your sample size is too small, you run into serious credibility issues with your results.
- Can’t layer your test results. Because of the specific and narrow nature of the testing, you are limited in the ways you can use the results. The results you get only apply to that one decision. In a very simplistic example, you may test and find that blue buttons do better than red buttons on your site. Then during another test you find that round buttons do better than square. However, there’s a chance that square red buttons do better than round blue ones, but you won’t know until you test that, too. You can end up in a cycle of constantly testing different combinations of features and designs.
- Draw focus to the wrong conclusions. During all your testing, it’s easy to become fixated on one design issue while a bigger usability issue escapes your notice. You can quickly get caught up focusing on the color and design of your buttons, without ever realizing that the button leads to the wrong page, or has a terrible call to action. With such a narrow scope, you risk missing more serious UI/UX problems.
Is A/B Testing Right For You?
When it comes to your site, probably not.
Playing around with email subject line testing is low-risk. It takes no time to implement with a service like Sendlane or Constant Contact. You can quickly gauge the preferences of your subscribers with little to no effort, or cost. It can be a fun way to experiment with finding your brand’s voice!
Using A/B testing with ads and social media can give you a broad sample size. Users aren’t required to actually do anything (like go to your site) in order to be counted. They are passively interacting with your testing just by scrolling.
But when it comes to implementing the testing on your site, you will likely find that the information you do manage to gain really isn’t worth the cost or the effort. Instead, look to the following resources for advice on how to improve your site:
Best Practice Recommendations
There are a ton of blogs and resources on the internet that can help guide you through site design. Make sure the changes you’re considering focus on functionality, usability, and accessibility. Being up-to-date on what is considered a best practice for online shopping, in general, should help guide your site updates.
Competitors In Your Industry
This is one of the best ways to A/B test without actually having to test. Consider your site as A, and your competitor as B. Check out their reviews online. Ask your staff, friends, and family to use the site and let you know what they think. Then look beyond what they’ve done.
If you’ve been thinking about implementing a change that all your competitors already have, use that as a springboard for where to start. Take that feature and find a way to make it even better. Or solve the problem that it doesn’t address.
UI/UX Developers
When in doubt, talk to the professionals. A consultation with a UI/UX expert can give you a ton of insight into things on your site that may need help. And they can provide potential solutions for you that will help you achieve your new goal.
ECommerce developers have the expertise to give you the answers you might get from A/B testing. They understand design, and they potentially already have other clients that have tried the same thing. Use their knowledge and experience to your advantage.
Are You Ready To Make Some Changes?
Before you jump into A/B testing, we encourage you to reach out to our team of experts, many of whom have been store owners themselves. We will be able to help guide you to a solution that works for your shoppers, without the added expense of third party testing solutions.
Save yourself the time and cost of unnecessary testing by getting feedback from a panel of developers and eCommerce professionals first.