Thursday Testing: A/B is Plentiful, at Least Initially!

If you are in Silicon Valley, there is almost always a worthwhile event to attend and this past week was no exception. We had the opportunity to socialize, chat with, and listen to Avinash about testing (you know, the A/B and multivariate testing stuff 🙂 ). The event I am referring to was “Thursday Testing” coordinated by Lily Chiu.  Thanks a bunch Lily, and thanks to the Omniture folks for sponsoring the event!

It’s always a pleasure to speak with Avinash. You’ll hear about analytics, online marketing, and also about what is going on in some of the world’s top brands.

I was able to take some notes that I wanted to share. OK, the notes are brief as I was typing on my iPhone and I think Avinash thought I was playing Sudoku too 🙂 . Here you go:

Why aren’t more people testing and why is testing a rarity. Avinash mentioned a couple of things that made a lot of sense:

• Vendors and consultants are sometimes at fault. In their effort to highlight the capabilities of their products and services, they inadvertently set a high barrier to entry. All they talk about is multivariate testing and the gazillion variations you can have.  While impressive, it might be overwhelming or intimidating to marketing managers and site owners. Simplifying what to test, starting with a couple of ideas, and setting realistic expectations might get the decision makers to grant you the go-ahead and get your testing effort off the ground.
• The second point is that while it is easy to simplify things, it’s hard to come up with two good ideas to test (not just “let’s change the button color”). It does take creativity, knowledge of your audience, among other factors to come up with two very good ideas to test.

These were the two points that Avinash emphasized and I find very helpful. Then there was an interesting discussion about some experiments with 19 PhD’s (I’ll pass on this for now), and another discussion about embarrassing the highest level manager you can (this way they listen to you), but this one needs another post altogether, and Avinash talks about it on his blog, so you can read his comments on the topic and apply it at your own risk! 🙂

I would also add that looking for some easy wins, taking some risks, and finding a sponsor that will support your effort, will pay off big time in getting more out of your site, and also in setting/starting a testing program within your organization.

So to summarize:

• If you/your organization is new to testing, start with A/B Testing
• Be creative, take some risk, look for low hanging fruits and get some early results to get your testing program supported within your organization

For more information on testing, check out Google Website Optimizer, make sure you get a copy of Always Be Testing, and feel free to contact one of E-Nor’s consultants.