The 2012 A/BAMA Campaign
Written by Dylan Langei
According to Optimizely, A/B testing, in essence, is a method to validate that any new design or alteration to the content on a webpage, email, newsletter, and/or person-to-person script is improving your conversion rate before making the actual content change. A/B testing provides you with a measurement tool and real data so that business decisions can move from "we think" to "we know." An A/B test involves testing two versions - version A (the control) and version B (the variation) - utilizing live traffic and measuring the effect each version has on your conversion rate.
Our soon to be guest, Amelia Showalter, who was the director of digital analytics for the 2012 Barack Obama campaign, utilized A/B testing with her team to market the reelection of President Obama. Considering Obama was reelected in 2012, we have much to learn from Amelia and her team's tactics. The articles from Businessweek & TechPresident provide details on the campaigns approach of leveraging A/B testing to listen, experiment, and analyze. The campaign mainly utilized A/B testing for emails to see if people were more receptive to one method over another. The interesting part was that they would create around 18 variations with differing subject lines, amount of money asked for, formatting, and even the message themselves - yet even the best writers, analysts and managers could not identify which of the variations would perform the best and worst. This relates to the concept of HiPPO (Highest Paid Person's Opinion) and why it is so ineffective. Many businesses can learn from this, as HiPPO is often a method still practiced. Many businesses figure, well he/she is a top-level executive who makes three times as much as the normal employee, so they have the best decision-making skills. Well this is incorrect, it is impossible to predict what A/B testing measurements provide - accurate and concrete data so businesses can know what content will drive conversion rates. However, Amelia's team was provided with many resources, which allowed them to scale more easily than a traditional business might. Therefore, it would be difficult for a traditional business to send out as many emails as Amelia's team did, or engage in as many person-to-person interactions.
There are thousands if not millions of businesses using optimization to gather actionable data, track conversion rates, and discover winning combinations across a range of ideas. LifeProof is one of these businesses. LifeProof manufactures and markets SmartPhone and Tablet PC cases for all-environmental protection, functionality and interactivity. Lifeproof's goal was to implement a testing culture and launch a series of tests on the site's product pages. LifeProof tested various landing pages with original CTAs like "Buy Now" and "Learn More" to less direction options like "Purchase" and "Explore," and more explanatory options like "Choose Your Case" and "Find Out More." LifeProof also learned through testing failures. In an effort to increase the purchase of the Total Water Protection (TWP) feature, the team wanted to automatically pre-check that upsell on the product page. However, the team learned that pre-checking TWP was causing more harm than benefit. Pre-checking this item equated to an 11% decrease in monthly revenue projections. In the end, LifeProof experienced a 19% life in revenue, scaleable testing program, and strengthened branding.
