The terms of A / B testing, testing and multivariate testing are very often confused with each other and often misunderstood. Basically, A / B tests and splitting tests are the same concepts, but testing on many varieties is different. Therefore, in all practical concepts, testing A / B = split testing should be considered.

In the world of design optimization methods, A / B testing attracts the most attention. Multivariate testing is a rarer and less understandable alternative, often considered too time-consuming to get to it. Although this method has its limitations, they are offset by its advantages, which can not easily be achieved with the A / B tests alone.

We will try to show you the advantages and disadvantages of both solutions and show in which cases to use them.

What are A / B tests?

When you perform the A / B test, you create two different versions of the website and distribute traffic equally between each page. You can also do the A / B / C test, which tests three different versions of websites, the A / B / C / D test, which tests four different versions of websites and you get a full picture of the page. In the A / B test, you can literally change any variable from page to page, and it is actually the best test practice to create two (or three, four or any) radically different pages for your test.

What are Multivariate Tests?

When performing a multivariate test, you do not test another version of the website, such as, for example, in the A / B test. You do a much more subtle test of elements within one page. The purpose of the multivariate test is to show which elements on the website play the greatest role in achieving the goal of this website. Multivariate tests are more complex and best suited to more advanced marketing testers because they test many variables. The way they communicate with each other, giving much more possible combinations for visitors to the site.

Which type of tests should I use better?

A / B testing is a great test method if you need quick results. Since page-to-page changes are so serious, it will be easier to determine which page is the most effective. It’s also the right way to choose if you do not have a lot of traffic to your website.

To run a successful test on many varieties, you need to have enough traffic on your site. It’s a good idea to use this method of testing when you want to subtly change the page and understand how certain elements work together to gradually improve your existing project. You can also use multivariate tests to perform a test that will give you results that you can extrapolate and apply to a larger redesign of your site.

Just remember that for multivariate and A / B tests to give significant results. It’s not enough to get general traffic on your site, because the pages you’re checking also need to receive significant traffic. Make sure you choose sites that people can find and regularly visit, so your test actually provides data for analysis.

When should I use A / B tests or multivariate tests?

The answer depends on the specific situation and the number of conversions generated by the site during the month. If your website, landing page or campaign receives less than 200 conversions per month, you should first increase the number of visitors to your website using various channels (SEO, PPC, community, etc.). If you want to start testing, you should use A / B tests with micro conversions.

When a website, landing page or campaign gets more than 200 conversions, but less than 500 conversions per month, you can start A / B testing by making two to three changes from the original.

Another case is when your website, landing page or campaign gets more than 500 conversions, but less than 1000 conversions per month, you can start A / B testing by entering five to eight changes from the original.

You have the choice between extended A / B tests or limited MVT tests if your website, landing page or campaign receives more than 1000 conversions per month,

However, if you have not tested on your website before and your data indicates that you can start testing, start with the A / B tests.

How long should I take my tests?

Do not interrupt the test before the original project, and the winner will collect 100 conversions. In our practice, we require at least 500 conversions per variation. We also recommend allowing testing for at least a week. If the test is completed for more than four weeks, you should consider the lack of a winner. You should aim for a test confidence level of over 95%. We recommend a minimum 90% confidence level. There are, however, several cases in which 85% are acceptable.

How to perform any test correctly?

Remember that the process of collecting relevant data and information is the basis for optimizing conversions. A / B and MVT tests are great tools for this process if they are run correctly:

• Planning: knowing what to expect from testing is critical and has a huge impact on results. Any form of testing that is just a mass collaboration of tests, without first setting goals and the type of results you plan to collect at the end, is probably a waste of time, money and, most importantly, effort.
• Movement: one of the biggest challenges for a multidimensional test is the amount of traffic needed to achieve significant results. Tests of many varieties are greedy and require the participation of many visitors in the test. So, for starters, you should make sure that you have enough traffic to run this kind of test. When it comes to A / B testing, traffic is not a problem if you can attract enough users to your sample.
• Length of time: The right time to terminate tests is significant and affects the results. Tests should not exceed one month. Always leave enough time to achieve statistical significance. For both A / B and MVT tests, do not hurry to stop the test when the conversion rate is drastically high to avoid false alarms. During the MVT test you will be tempted more than in the A / B test to stop.

Testing conversion factor optimization

A / B testing is the default and most-used procedure in CRO programs. In some cases, running a multi-variant test can add significant value. Other times you have space for both tests interchangeably.

After proper implementation, testing eliminates guessing from the optimization of conversions and we go to the stage where each action taken is the result of a conscious decision.

In the previous chapters of our guide, we introduced the directors of conversion schemes and a lot of practical applications that can be used on your website. You must always remember that each site is unique and that what works on one page may or may not work for you.

Increasing conversion rates starts with understanding the target market, creating people representing that market, and then applying the elements of the conversion structure in the context of these people.

These best practices are great in theory, but their actual use is not to go to the park. The more time you devote to understanding them, the more likely you will get accurate and approximate results.

Before you start your A / B and multivariate tests, keep in mind the following tips.

In some cases, you can look for a pattern that increases the average value of an order, or a pattern that keeps visitors more involved, or even one that generates more social shares.

Team members usually disagree about the best website or landing page project, the best user flows or the best sales. Stakeholders usually have different opinions on the changes that need to be made on their website. Attempting to resolve differences is a challenge. Your goal is to use better designs, copying and flow of visitors that convince more users to convert.

Can you determine which projects, copying and flow of users increase the number of conversions? Test any modifications made to your site compared to the old one and compare the impact on conversions. Split testing software allows you to make comparisons. By testing two or more page variations against each other, you can see which projects result in higher conversions. For example, split testing software allows you to determine which of the two main home page designs is better for conversion.

Conclusions

The steps in the tests can be summarized in the following points:

Finding the right page for optimization. If you have a large website, it will probably take some time to find these pages. The best page to optimize is the one that leaks the most visitors. Then check the number of visitors on this page who will actually participate in the tests. Remember, never run tests for more than four weeks.

Next, define your goal, which does not necessarily increase the macro conversion. Each test on a smaller scale can still have a huge impact on the conversion rate of macros.

Be selective in terms of the elements you are going to test, select only those that have the greatest impact.