The $10 Million Sentence
How an A/B test led to a predicted $10 million in revenue.
The Ask
Test and optimize the fare introduction copy in the booking flow.
Background
After a user selects the details for their flight, they are presented with different cabins and fare types. Within the Economy Cabin, there are three different fare types available.
How it started
The benefit of testing content is it is often less effort than changing the design. We can make a small change with the potential for a big impact. With this test, we wanted to understand if the copy had an impact. We wanted to know if listing multiple benefits of this fare type was more effective or focusing on one or two key benefits. This information would also inform future content decisions by helping us better understand what is most important to users purchasing this type of fare.
Process
The Advanced Analytics & Automation, Responsive Booking Flow, and UX Writing teams meet every two weeks. The group collaborates and identifies areas in the booking flow where we can test and optimize the experience. The fare introduction copy was identified in 2023 as a content block that could be tested to determine how this sentence impacts the fare type a user selects.
UX Writing created a few possible variables for the test. These options were reviewed with the Booking Flow Product Owner, and we determined experience B. This option reduced the number of attributes and focused on price and flexibility.
Experience A (Control) and Experience B were served to a 50/50 user split. The test reached confidence and revealed that Experience B was the better choice. In addition to a predicted $10 million in flight revenue, Experience B increased booking conversion by four basis points and decreased abandonment rate by 21 points. Multiple teams within the organization validated the results.
What happened next
In 2023, UX Writing and Advanced Analytics & Automation presented this case study at WestJet’s Digital Day.