Every change on your e-Commerce site interacts with everything else. The intensity and the outcomes of those interactions are highly unpredictable and often negative to your revenue performance.
- The human eye is wired for a flight response
- The order in which information is absorbed determines visitor reaction
- Interaction between element changes are highly unpredictable
A typical e-Commerce site is subject to frequent content, layout, and functional changes. From time to time, an e-Commerce team will run A/B or MVT tests and discover that certain changes in the layout or content are providing a revenue lift. For example, the marketing team might decide that they should implement a social widget that showed signs of success for a competitor; or perhaps the UX team will decide to enhance the brand appearance of the site following some newly-published UX best practices. Periodically, all these good ideas will be scheduled to be implemented all at once.
These changes are all viewed as independent changes and the expectation is that they will produce overall positive results. However, the bigger and often neglected problem is that these changes interact with each other and the outcomes are actually far from certain.
These interactions can be positive, wherein two or more changes create a synergistic effect that results in an overall positive lift, or negative, which happens when two or more changes undercut each other and cancel out some of their individual effects.
Let me try to explain what is happening here and how you can take advantage of it.
The Nature of the Human Eye
Recently we stumbled upon the “100 Visual Phenomena & Optical Illusions” site. I encourage you to visit and entertain yourself with a few of the nicely presented visual examples.
What is the connection with visual phenomena and e-Commerce?
Any visual element on your e-Commerce site has the potential to trigger the same kind of neuronal activity as an optical illusion. To illustrate this point, imagine you’re at a magician’s show. The likelihood for his tricks to fool you, depends on his ability to distract your attention to one area while he performs an action in another. By doing this, he can swap or change things around without you taking notice.
Similarly, by placing new content or changing a website’s layout or functionality, you focus the visitor’s attention on one area of your web page and this could potentially render the other area less important.
A major brand ran two independent A/B tests on their product detail page to evaluate the impact of both a free shipping offer and different colored Add to Cart buttons.
‣ Free Shipping Offer: 4.83% lift ‣ Red Add to Cart Button: 3.12% lift
After combining the two winning outcomes onto a single page, they were surprised to learn that the new product detail page underperformed the baseline in a follow-up experiment:
Although the results from the follow-up experiment were surprising at first, they can be explained rationally: when presented one at a time, each winning element was able to attract the visitor’s attention and produce the expected results. However, once placed together, these elements began to compete and created a fragmented visual experience.
This visitor reaction is not driven by actual eye optics but rather with the way in which the brain processes visual information.
The human flight response causes us to notice highly-visible things first: for example, certain colors like red draw attention to things like calls to action or banner ads. Also, moving objects, like content rotators, sliders, or Flash, can cause a high level of distraction. We’re naturally wired to be drawn to these elements. This means that negotiating a website is very different than reading a book – we don’t read websites from left to right or top to bottom; rather, we process visual data in according to a hierarchy of distracting objects.
The way in which visual data is collected and stored in visual working memory greatly affects how purchase decisions are made – understanding the basic principles of attention and visual memory can help you draw a clear line between page elements and purchase patterns.
Multi Variate Changes
When you’re making multiple simultaneous changes to your e-Commerce site (e.g., multivariate optimization, or MVO), you should be consider that even small changes to on-page elements have the potential for tremendous impact on visitor behavior. Even simple and clean product pages, like the one shown below, can contain numerous elements within one combination:
If your goal is to improve the revenue performance of pages like this, the only truly viable approach is to explore multivariate optimization. Running a sequence of A/B tests, or several A/B initiatives in parallel, ignores element interactions (or intercorrelations) and creates a false impression that you are improving the results, when in fact you might be going in the opposite direction.
To illustrate this point, let’s assume that the baseline user experience is represented by a picture below:
- Upon loading the page, a visitor’s eye might be drawn to see a big product picture
- Next, the visitor will notice social widgets
- Then the visitor will focus on different product views
- If the visitor liked the product, they will notice free shipping
- Examine the price
- Order the product
- The visitor will also notice breadcrumbs placed above the product picture
As result of this progression, 9.23% of all visitors will select and buy a product.
Now, let’s assume that your goal is to improve the performance of this page and that you wanted to experiment with multiple page elements:
- Breadcrumbs: move them to the bottom of the page
- Free Shipping: make it more visible by placing orange color behind free shipping label
- Social Links: move below product views
- Add to Cart: change button color to maroon
Now, after running a multivariate experiment, you would find the winning page combination looked like the one below:
The visual impact of the yellow free shipping banner and maroon Add to Cart button could have been that that it made other page elements less visible such that new visitor experiences looked like the picture below:
- Visitor noticed free shipping first
- He liked the product
- The price was right
- He made a purchase
The final outcome of this new experience was that more visitors purchased the product and revenue grew by 25%.
Note: No one can actually visualize the average user experience and the example above was our attempt to illustrate the importance of the multivariate approach and the magnitude of element interaction; we routinely see 10% – 20% revenue lift created by changes that most perceive to be “small” or “invisible”.
We frequently meet companies that already had licensed an expensive MVT testing solution but never had the time or “stomach” to actually implement the multivariate experiment. Instead, they are staying in the “A/B comfort zone”, thinking that MVT is just a “nice to have” but logistically difficult feature.
If you have multiple groups influencing changes on your e-Commerce site, running multiple testing or personalization campaigns in parallel, or if you’re running A/B tests sequentially, you may be ignoring the implications of interactions on your site – this is a risk that no e-Commerce program owner should take lightly.