This year’s Final Four marks the beginning of the end of an exciting NCAA Men’s Basketball tournament. As we wrap up March Madness, many of us who have a stake in the office pool are checking our brackets to see what the next games will mean for their overall scores.
Have you ever thought about the probability of predicting a perfect NCAA bracket? Accurately predicting the outcome of each of the 63 games is like guessing every single number in the lottery – the odds are definitely against you. Now imagine accurately predicting the outcome of each of your A/B tests: scary, right?
- Just a single e-Commerce sales funnel can contain hundreds of variable options.
- If you’re running multiple back-to-back or simultaneous A/B tests, your chances for success are exponentially small.
- Predictive multivariate optimization is the best way to tackle the dynamic nature of e-Commerce websites.
The odds are against you
Let’s face it: whether you’re a basketball guru or a statistician, filling out a tournament bracket is (at best) an educated guess based on historical data or personal intuition. Your picks, then, might depend on such variables as team colors, the name or location of the school, conferences, your loyalty, or even your feelings about a specific player.
According to DePaul mathematics professor Jeff Bergen, the probability of predicting a perfect NCAA bracket for the general population is 1:9,223,372,036,854,775,808. If you’re a basketball guru – that is, someone who consistently follows the NCAA tournament’s historical outcomes, your chances improve to a “more favorable” 1:128 billion. To put this in perspective, imagine that everyone in the U.S. was a basketball guru and everybody filled out a bracket. The odds of at least one person correctly guessing the outcomes of each of the 63 games is 1:400.
And while it may be nearly impossible to get yourself a perfect bracket, being fairly close is not so outrageous. There are many variables that moderate whether a team wins or loses, such as the star player’s performance, whether the team is home or away, injuries like that of Kevin Ware, and more.
How March Madness relates to eCommerce testing
You can draw a parallel between many of these moving pieces and e-Commerce websites. Hopefully, your checkout flow doesn’t resemble a tournament bracket on its side, but what if you weren’t too far off? Think about each one of these pages individually – how many elements can you manipulate in order to glean more out of each visit? Typically, brands and retailers experiment with CTAs, copy, images, and layout – but there’s also functionality, user experience, and much more to consider. All this can make the number of variables on your site exponentially large, really complicating any efforts to improve performance. This is one of the many reasons why more brands don’t try to tackle e-Commerce site navigation.
Now think about how each one of these pages flows into the next, and how different visitors might react differently at various times in the year, month, or even on a day-to-day basis. All of this complexity lends to a really difficult task to accomplish with the use of tactical testing tools like A/B or traditional multivariate testing (MVT) – your probability for success becomes geometrically smaller with each new variable.
Adaptive multivariate optimization solves this problem with bi-layer capabilities: first, variables are presented and evaluated based on relative performance across the funnel. Then, the relative importance of each variable is calculated and a predictive model becomes updated. By presenting more of what’s statistically probable to work and less of what doesn’t, you don’t have to display millions of variables until your outcomes are found statistically-significant. Better still, you glean improved performance from both the prioritization of positive (high-performance) variables, and the removal of negative (poorly performing) elements.
It’s easy to overlook how a small number (63, 100, or even 500) can quickly and geometrically lend to impossible probabilities. And unless you treat your bracket like your job depends on it, you might be better served (from a time perspective) to simply make some fun and educated guesses. In e-Commerce, however, your website’s potential revenue lays on the line. By subscribing to an adaptive method rather than brute-force testing, you can turn those probabilities more in your favor and glean more revenue from your optimization campaign.