You’ve probably heard that split testing elements of your website can improve your conversion rates. And that’s true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.
There are many differences among the split testing software options on the market today. Let me give you a brief taxonomy of the subject matter.
One big distinction is between single variable and multiple variable (multivariate) testing.
Single variable split testing is very simple. You just try stuff and measure the results. And all you need is a tool that rotates and keeps score.
There are a few simple split testing tools available. Some are free and some cost money. Google offers a free tool, though you do have to have an Adwords, and an Analytics account to use it.
Single variable testing can be slow. How would you like to test 10 times as much with the same amount of traffic? Well, you can. You use a method called multi-variable, or multivariate testing. This allows you to test several elemens on your page simultaneously.
There are further distinctions among multivariate split testing tools. There are hosted solutions, and solutions you install on your server. Typically a split testing program can run a little more quickly if it’s on the same server as your webpage. Installed programs tend to be less expensive, too. You also will have complete control of your data. On the other hand, with a hosted solution, you don’t have to do an installation.
The Split Test Accelerator is a good choice if you want to install the software on your server. Kaizentrack is another optoin. If you want a hosted solution, you can look at Sitespect and Vertster.
Another 3-way distinction among multivariate split testing tools is between Taguchi-based systems, Full factorial systems, and Random independent rotation of factors. With a Taguchi system, you run traffic to only a small, carefully-selected subset of the possible combinations of your factor levels. This allows you to cycle through every combination every 12, 18, or 36 pages, whichever the case may be for the test design you’re using. This allows you to find the best combination very quickly. Some care does have to be taken to avoid interaction effects among the factors. Taguchi-based tools include the Split Test Accelerator and Vertster.
Full factorial tools allow you to cycle through all the combinations for your factors. So if you have 3 headlines, 3 offers, 3 P.S. statements, and 3 images you’re testing, you will cycle through 81 pages. One advantage of full factorial testing is that you don’t have to worry as much about interaction effects. Good and bad interactions should show up in the data. However, if you want to receive this advantage, you have to run many times the traffic through your test as you would with a Taguchi test. You also can’t run as many combinations. Google’s website optimizer is a full factorial tool.
The Random independent rotators are a third option. These treat factors independently. So you can just plug in a new factor with as many options as you want at any time, and ignore the fact that other factors are being tested at the same time. This is the most flexible form of testing, but it is also plagued with the most interaction effects.
Now that’s just the overview. There are many other features to compare and questions to ask. Does a tester allow you to exclude bots and spiders? Does it allow real time access to data? What kind of reports does it show? What kind of information does it track for each visitor, and is the visitor-level data available for viewing. Etc.
Dominic DeLong has used taguchi testing software for years to improve landing pages.
Leave a Reply
