Before I started my own business, I thought of marketing as this wishy-washy thing people in suits do. They wave their hands in the air, use a lot of buzzwords, and present a few slides with pictures and tag lines–anyone can do it. How can we tell if they are good at their jobs anyway? Based on our opinions of their pretty pictures?
For us developers on the other hand, it is easy to tell whether we’re doing a good job. Does our code run fast? Is it buggy? Is it unit-tested? …etc.
I was wrong; marketing is not subjective. We are just not used to measuring it objectively.
Opinions are like belly buttons
…everyone has one.
As I started running my own business, I had to learn how to market. I read best practices and SEO books. I started designing and became more opinionated. I did okay. I even started enjoying it.
But, I still did not like the subjectivity of it. I would iterate on a design not knowing whether I made it better or worse. I could ask 3 or 4 people around me what they think, but their opinions are just that their opinions.
Removing Subjectivity from Design
- Which landing page is better?
- Which color scheme is nicer?
- Which advertising campaign is better?
- How do I write a killer email subject line?
While better, nicer and killer are subjective terms, as a business you probably have clear objectives that you are hoping to achieve via your better, nicer and killer designs. You want people who come across these designs to do something; it could be purchase your product, sign up for a free trial, book an appointment, subscribe to your blog …etc. Fortunately, those things can be measured!
A/B Testing
…made me feel right at home.
A/B Testing is testing two versions of a design against a defined metric to determine which version is better. Typically version A is your existing design, version B is your new design. You split the traffic between those two designs and measure which of them performs better. The one that performs better is the one you go with.
Steps
- Know what you want to measure and measure it correctly. Whether it is conversion rate, sign up rate or subscriber rate; make sure your numbers are correct.
- Test both versions simultaneously. Some weeks perform better than others for various reasons. To obtain unbiased results you must run both versions during the same time and for the same time length.
- Give it time to run. Your results need to be statistically significant.
- Always serve the same page for the same visitor How else would you know which page led to the conversion? You need to track based on the IP address and User ID.
- Don’t hesitate to throw away your new design. Sometimes the old design is better, that is why you ran the experiment!
- Test only one thing at a time. So to test both the images and button color, you will need to run 2 experiments.
- Start with the concept and then refine the details. Wondering whether a red or green button performs better? You’ll get to it. Start with the overall concept first, because the details might not apply when you change the overall concept afterwards.