11 simple A/B tests you can set up

In one of my earlier posts, I talked about why you need to be doing conversion rate optimisation. One of the primary aspects of a CRO project is A/B testing, which can be quite daunting if you’ve not had any previous experience. One of the first things you’ll probably sit there thinking about is ‘what can I even A/B test’. To help you out, I’ve provided a list below of some of the most basic aspects of your site that you can run A/B tests on. Hopefully this should help to get you started!

  1. Simple text changes on your CTA buttons can have a massive impact. It’s definitely worth creating a number of variations to test.
  2. Try changing the location of your CTA’s to see which gives you the most conversions.
  3. There’s a lot of psychology around colours and which you should use for CTA’s, so it’s worth reading into this, but you’ll never know which works best for you until you test!
  4. It’s also a good idea to play around with the shape and size of your CTA’s to see if one works better than another.
  5. Another aspect of your site you should consider testing is your content. There are many different variations you can create including amount of text, copy positioning, tone of voice and the type of copy you display (i.e. does ‘About Us’ information on a product page enhance or hinder your conversions?).
  6. When it comes to certain aspects of your copy, there are also many variations you can test. For example; length and position of headlines, paragraphed copy vs. bullet pointed copy and general length of copy.
  7. Test different types of images on your landing pages. In some cases, users prefer to see a photographic image of a real person, rather than a generic illustration.
  8. Once you’ve tested which type of image works better, you should then test different variations of this image. For example, if you’ve found that a photo of a real life person works better, play around with the person you use i.e. try an image of someone older or try an image of multiple people to see if this has an effect.
  9. Another aspect of your site that you could like to A/B test is your site navigation. Now by this I mean, test the order of items in your main navigation bar, place the navigation bar in different locations and test out the title of the items in your navigation bar.
  10. Now you CTA might not always be as simple as a button. Your users may have to fill out a form instead. There are multiple aspects of a form that you can test including the length, the type of fields you include and where to place the form.
  11. As a side note, don’t forget to also test different aspects of your mobile site – don’t just focus on desktop. When it comes to your mobile site, it’s important to test aspects such as the length of the pages and the navigation options to see which users better interact with.

So, now you’ve gathered a list of which A/B tests you’d like to start out with, remember to start simple and build your confidence, there are a few ‘do’s & don’ts’ that you need to be aware of if you’re going to successfully run your first and any future A/B tests.

Do’s

  • Ensure you make you’re A/B tests consistent throughout your website. For example, if you’re changing the colour of a call to action button that appears throughout your site, make sure you don’t just change the colour of the button on one page. Users should see the same variation everywhere it appears, otherwise this could affect the accuracy of your results.
  • Do as many A/B tests as necessary. There are usually 3 A/B testing outcomes; no result, a negative result and a positive result. It is important to continue testing as many A/B variations as it takes to achieve positive results that will have a notable increase to your conversion rates.
  • It is important to take into consideration how long to run each A/B test for. For example, if you’re running an A/B test and end it too early, this will affect the accuracy of your results. You will get much more meaningful results if you wait the appropriate amount of time.
  • Start simple. You’ll often find that the simplest of A/B tests can cause major results. There are definitely more advanced tests you can do, but especially if you’re a newbie to A/B testing, it won’t hurt to test simple aspects of your site such as colours and sizing of call to action buttons. This will help you to test and improve your skills so you’re confidant in setting up the more advanced tests.
  • Establish statistical significance by ensuring you are getting enough people testing your variations in order to validate the results.
  • Finally, if you’re going to take anything away from this list of ‘Do’s’, it should be to act on your A/B testing results. If your test provided accurate, positive and significant results, you should permanently implement your changes. If you didn’t see a positive result, keep testing a range of different variables.

Don’ts

  • Always split traffic between your control and your variations. If you test one at a time this could massively skew your results. You might get a positive increase from one of the variations, but this could be purely down to the fact you had better sales (for example) at this time.
  • Don’t let your personal opinion overrule the hard data. Yes you might not agree with a green button on bright yellow background, but if the results tell you this will give you a 450% increase in your conversion rate, then you’ll have to learn to live with it.
  • Once you’ve grasped the simple A/B tests, don’t be afraid to branch out into more complex and advanced tests.
  • Although you should test many different variables, these shouldn’t all be done at once. It’s important to be able to isolate each variable to understand why it did better than another.
  • Don’t give up testing if your results aren’t significant. In-fact this gives you something to act upon. If your results aren’t significant, it shows your variable didn’t have a big enough impact on your click through rate, so you need to think of other ways that will.
  • Don’t only focus on the ‘bigger’ changes. As I mentioned earlier, you’ll often find that the smaller changes have the biggest impact.
  • Be disheartened if your test fails. At the end of the day, not every test will be a success; it’s all about continuous testing until you find a variation that does work.

If I had to sum up this blog post in one sentence, which I guess a conclusion does anyway, it would be to remember to start out with simple A/B tests and that not every test you run will be successful – but that’s the point. Continuous testing, leads to continuous improvement! Never give up.

  • Great article, thanks Grace

    • Grace Hartnett

      Thanks Warren – I’m glad it was an interesting read!