In our previous blog , we provided you with a few last minute ideas for the checkout process during the holiday season. We want to focus on one of those ideas now – why you need to stop A/B testing but continue optimizing your website during the holidays.



The ongoing digital transformation has caused some disruption in the e-Commerce play field with all marketing activities being geared towards customer experience. Given this paradigm shift and the fluctuating nature of consumer behavior, e-Commerce professionals are revisiting how they prepare for the crucial holiday season.

For instance, the belief was that testing during the holiday season is a bad idea given the influx of traffic causing spikes and skewing results. Also, what most digital commerce companies are realizing now is that you can’t rely on results from traditional testing practices like A/B testing unless viewed from the scope of “real time”, continuous monitoring.

However, the consensus most e-Commerce brands have arrived upon is that: Optimizing your website is an ongoing process rather than a run-once-and-play-the-winner trick. They are beginning to see the value in the continuous learning and the significant competitive advantage one derives from the optimum digital experience to every visitor, at all steps of the journey, at all times.

This learning is applicable (even more so) during the holidays because:

Consumer Behavior is different during the holidays

Behavior of online visitors is always changing and in particular, the behavior during the holiday season is completely different from the behavior before the season. When users find something they like during the holiday season, they want to react quickly. They shop around, compare prices but at the same time are looking for a quick and easy checkout process. Therefore, it becomes imperative to provide them with optimum experience at all times. The ability to test multiple variables as opposed to being limited by one or two, on your checkout page during the holidays, for instance, may reveal many opportunities that could impact conversion rate or revenue per visit.

You should optimize during code freeze

Most businesses have a challenge trying to test aspects of their website during this time. A big reason for this is CODE FREEZE. Similar to when everything freezes over during winter; websites go into a freeze period where little to no change is allowed by development teams. Dev teams go into hibernation and work on bug fixes and improvements to their current sites rather than new ideas or changes. The challenge here is that you want to be testing during this time. This is prime time to learn about who your users are. Understand the different personas and analyze your segments. Are users shopping using their mobile devices, tablet or desktop devices? With the influx of holiday traffic, more data = more results = more understanding about your business and website. You have to be able to leverage the learning and deliver tailor-made experiences to the various visitor-types. You may come to realize a third of your holiday sales is coming from mobile. You could leverage this insight by optimizing your mobile site and targeting specific promotions for mobile traffic but you will be limited if you were using an ordinary A/B test

Your value proposition in real-time

Now that we have established how visitor behavior is different during the holiday season and how marketers need to cater to their different visitor types, it is also crucial to realize that the pre-defined business goals or value propositions can yield drastic or unexpected results. For instance, a company may decide upon ‘Free shipping ’ as a business strategy during the holidays. But in the midst of holidays realize that they need to modify their value proposition to focus on “Free gift wrapping available’ instead as this has more appeal with the holiday shoppers.

Ensure smooth sailing through the entire buying journey

A huge gamble with traditional testing is that most efforts are focused on the bottom line while ignoring micro conversions. Especially during the holiday season, when visitors demand a smooth sailing checkout process, it is crucial to optimize the checkout funnel from end-to-end. Gently nudging your visitor along the different stages of the buying journey (from category page to PDP to Add to cart to actual purchase) helps speed up the check out process. For example, in your product display page, stating free shipping or percentage off may not work but showing a low stock meter or greying out the sizes no longer in stock can encourage the customer to act immediately.


US Online Holiday Retail Sales. Source: Forrester Research, Inc.

These insights and learning are most effective when you are able to take timely action based on them. This is one of the main reasons why you shouldn’t stop testing during the holiday season. Forrester projects that online sales this holiday season will reach $89billion, they say it represents a deceleration in growth owing to a shorter holiday selling season -28 days between Thanksgiving and Christmas. The onus falls on digital marketers to ensure optimum experience to each visitor at all times in this short but most profitable window.