Ahh, Thanksgiving. – a wonderful time to come together and remember what we are all thankful for.
Family often comes to mind first. I know that personally, my family is phenomenal. I am lucky to have fantastic relationships with each of them on an individual basis. However, as many of you may be able to relate to, when my family all gets together… it’s not necessarily the same wholesome experience.
In a weird way, a family gathering can be compared to a multivariate test. You have your individual family members, whom we can compare to the individual A/B tests, and then you have the interactions between them, similar to the ensembles (or combinations) of treatments playing at the same time.
We can illustrate this concept with a fictional family (that may or may not be loosely based on my actual family), and all of the wonderful elements that make them… unique.
You’ll see how, similarly to multivariate tests, switching two people out at the dinner table can be the thin line between scintillating conversation and yelling to the point where flecks of turkey fly out of an angry relative’s mouth (This is not based on personal experience at all – nope, definitely not).
On that note, let’s meet the family!
This is Salma and Jon. One of their dogs recently won first prize at the Big Boy Breeders’ cup last month. Their winnings are why the dessert is fancy this year – thanks Cocoa! However, they’re constantly shedding dog fur, which is tough for both those with allergies and those who don’t enjoy dog hair in their food.
Then we have Dev and his parents, Aditya and Kala. They’re all hilarious – the absolute lives of the party. But Aditya won’t stop grilling Dev about his life goals in front of everyone, which stresses Kala out to the point where she takes it out on the football players on TV. The yelling doesn’t help the existing cacophony of Salma and Jon constantly trying to convince everyone to change the channel over to the National Dog Show.
When you throw perfumed Patricia, vegetarian Victor, imbibing Ida, and overcooked onions Owen into the mix, we’re looking at a whole lot of ways to arrange 9 people at 3 different tables.
I can’t imagine any reasonable situation where someone would be sitting at a table by themselves (unless Patricia really overdid it with the perfume this time), so we can assume that all three tables have three people sitting at each. If we were to run a multivariate test with all of these different ensembles of family members, we’d be at 280 potential situations!
In an ideal world, there would be a HiConversion-esque testing application that could handle all 280 ensembles and sift out the good from the bad to make creating your Thanksgiving seating charts a breeze, but this is the real world and we can’t simply throw our family members into a multivariate test (if only). Luckily, we know a little bit about how the interactions might go, so we can whittle down the number of ensembles a bit.
To illustrate, first we will want to separate Dev from his dad. If Kala is at the table when the interrogation starts, she’ll lose her voice yelling at the TV, and if anyone else takes her seat, they’ll feel the discomfort of the awkward conversation.
We also want to keep Salma and Jon together to minimize the area in which dog hair is shedding, but away from Victor, who can’t handle people that idolize domestic pets but freely eat other animals.
If Ida over-indulges (“if”), she might say something hurtful about Patricia’s heavy perfume or Owen’s poorly cooked onions. Let’s skip those ensembles as well.
Over time, we can continue winnowing down to determine who can’t sit with whom to unearth an optimal table set-up:
Amazing. We did it! Now all that’s left is the cooking – at this point, that’s the easy part. Don’t get me started on combinations of ingredients!
There aren’t enough Thanksgivings in the world to receive statistically significant results on a multivariate test determining how your family should sit together (which is something we can all be thankful for). Without testing, unfortunately, we’ll never truly know what the best combination of family members is. However, we can at least appreciate the similarities between a family seating chart and a multivariate optimization test.
As great as any individual treatments (or individual family members) may be, they don’t always interact well. This is why multivariate tests are necessary to determine the best performing ensembles of A/B tests on an e-commerce website. Just because Ida, Dev, and Patricia are all incredibly great people with hearts of gold, doesn’t mean they should ever be sitting together.
At the end of the day, the lesson is simple: if you want to avoid virtual turkey flecks landing on your cheek, you should play it safe rather than sorry and run a multivariate test the next time you aim to optimize your e-commerce site.
Happy Thanksgiving, and as always, happy testing!
Note: all names and characters are fictional