Travel firms should embrace experimentation and not fear failure because they learn more from failing, the Contentsquare Future:Travel 2019 conference was told last week.
Thom Shurrock, head of product at specialist cruise and ski agency Iglu.com, said failure actually increases your chance of a success in the future.
“An experiment never really truly fails,” he said. “It will not always give you conversion uplift, but it will give you insight into what’s happening.
“I have learned more from failing experiments than I have from winning experiments. The last thing you want is for an experiment to be absolutely neutral because you learn nothing.
“If you understand what does not work for your customers you can understand better what does work for your customers.
“It’s always worth digging in to why something has failed – it will really help you win more in the future.”
However, Shurrock said there comes a point when failure indicates a change of direction. When you are iterating your experiments it’s always important to know when to walk away,” he said.
He advised delegates to categorise experiments using hashtags so firms can keep track of the areas in which experiments have the most chance of bearing fruit.
“That allows us to increase our win rate move forward,” he said.
Shurrock said all experiments should start with a hypothesis to either prove or disprove and that experiments should determine a firms’ development roadmap, not vice versa.
And he said firms should look to carry out low-resource tests first to validate plans before going ahead and building something new.
Examples of tests firms can do to validate hypotheses include Takeaway Tests – removing a feature for a short time to assess what people and do and whether they value it when it’s not there.
Another is assessing demand for a new feature with a face door test when a feature or button is added for a short time but it goes to a “coming soon” page.
“What’s the simplest thing you can design to validate your hypothesis? You are always having to think about how to prove yourself wrong rather than right. Confirmation bias is always a problem.
“Confirmation bias can kill. We are all inherently inclined to look at things that prove us right. You always have to predict what happens first.”
Shurrock set out a number of instances where the “Power of Paradox” can come into play when firms are experimenting.
The first was the power of novelty in which new features see huge uplift but when they are rolled out they fall flat, the second was the default effect where users are biased to the first item in a list.
There is also the ‘Ikea effect’, Shurrock said, when consumers place more value on something they build themselves, and reciprocation; people are more likely to do a favour in return for a favour.
Finally, he said there is a paradox of choice where adding more options can actually depress conversion.
Shurrock said he saw this when working at Open Table when it added more dining times to search results thinking it would be helpful.
“With more choice consumers have to feel inherently more confident that the choice they are picking is the correct one. If you provide more it has an adverse effect.”