Website Optimizer – Are you having trouble getting experiments to complete – AMA Q&A

Q: How did "petwellbeing.com" generate enough traffic to get complete Web Optimizer (GWO) results. We have used GWO and have had problems generating enough traffic to make GWO timely.

A: This is a very common problem. You should always mine your analytics data prior to running a GWO test. You need to pick pages that get some traffic, it doesn't have to be as much traffic as you think and here’s why. Most people make the conversion page or goal page their “Thank you” page. This is a common mistake; you should make the conversion page the next page after the test page (the next page in the funnel process). So if your running a GWO test on a product page, the conversion page should be the shopping page, if your running a test on the testimonial landing page, the conversion page should be the product page.

If you have your experiments set up properly and your still not getting enough traffic – well you can always buy some from Google Adwords – that’s what they are hoping for.

Q: Would you recommend removing some of the test pages after a few weeks of data based on poor performance?

A: Let your website optimizer tests run through to completion, there is no mechanism to stop showing certain versions unless you stop the current test and create a new test.

Q: Are the variations tested at the same time or over a period of X months?

A: The variations are tested randomly and concurrently.

Q: The importance of the "day effects" is well noted. Are there periods of time during the calendar year which are better/worse for testing (for example the holiday season)?

A: Many clients like to do their tests from January to late summer, because the go into lock down mode during the peak Christmas shopping.

Q: Why is there a step 10 in the process – doing another A/B test on the results you get from the multi variant testing? Why would confidence levels be low enough in order to feel a need to do this?

A: You always A/B test the winner against the original after all the multivariate testing is finished, so that you can definitely see how much lift the process created.