A lot of content on conversion rate optimization (CRO) is published every day. Most of it is spot-on, but some articles are downright cringeworthy.

A lot of the advice being shared gives people false hope that if they conduct CRO correctly, they’ll see the millions roll in. It’s not that easy.

The process is vigorous and requires a lot of time and effort — much more than the advice being shared will lead you to believe.

Whenever you hear a marketing practice referred to as “easy,” it’s usually not. Let’s look at some common CRO misconceptions and their uncommon realities.

Misconception 1: Anyone can do it

Not hardly! To do well in CRO, you need good people on your team. A conversion rate optimization team usually includes:

• Two or three conversion optimization specialists.
• A UX designer.
• A front-end developer.
• A customer research specialist (can be part-time).
• An analytics specialist (can be part-time).
• A data analyst (can be part-time).
• A product or program manager, depending on your business.

Note: In practically every project I have ever worked with clients, the bottleneck is always at development stage. No matter if it was a two-person company or (literally) Amazon. Keep that in mind when starting a CRO process.

Misconception 2: There are CRO best practices

CaffeineInformer and Bookings.com both tested the same navigational menus and found the most commonly recommended menu worked for one but not the other.

For CaffeineInformer a descriptive button name resulted in 13% more clicks. For booking.com the hamburger menu and the descriptive button performed practically the same. There was no difference in clicks.

So, although your competitors may inspire you, most of the time you’ll find what they introduce on their site may not work on yours. In the case above, it’s a small change, but sometimes companies make a bet on a change that costs hundreds of thousands of dollars and produces a negative impact on their site.

Know what is out there and get inspiration from other sites, but validate through research, prototyping and usability testing before rolling out a change on your site (especially if it’s major).

Note: I only partially agree with the point above. Sometimes there really ARE the best practices. They seem to work in the majority of the cases, across many industries. There always are exceptions but the best practices provide a good start.

Misconception 3: More testing yields positive results

Statistically speaking, more variations = greater possibilities of false positive and inaccurate results.

But, at the same time, statistically, each variant brings a chance of a false positive. So of course, more variants = more chance of false positives.

So, don’t run more than four split tests at a time. Unless you have massive traffic. If you have low conversions (even in the presence of a high volume of traffic), you definitely shouldn’t test beyond one variation.

Misconception 4: CRO is A/B testing

A/B testing is a part of the conversion rate optimization process, but they are not one in the same.

Over 70 percent of the time spent doing CRO is the scrutinize (research, planning) phase of the process. An unplanned test that is not backed by data does not usually do well.

Note: While testing is the crucial step, it’s almost at the end of the process. The vast majority of the time is spent in researching, numbers, analysis, conversations, and insight compilation.

Misconception 5: A simple change will impact your bottom line

Sometimes a simple change can have an impact. but let’s be real: that’s the exception, not the rule.

Expecting a color change on your site will increase conversion by 40 to 50 percent is really a stretch.

I love talking about how we can increase conversion rate and how we can optimize it, because most sites, especially ecommerce merchants, get this wrong. They think it’s all about A/B testing and trying different button colours, etc. In reality, for 90% of small to medium-sized businesses, the #1 change you can make to your site to increase conversion rate is your MESSAGING.

– Allen Burt, BlueStout.com

Misconception 6: A/B test everything

Some things just need to be fixed on a site. It doesn’t take an A/B test to figure out a usability issue or to understand that conversions increase when common problems are fixed. A simple investigation may be all that is required to determine whether or not an A/B test should be done.

Note: For this, you need to have experience with usability, and to know what works and what definitely does not. Best practices help here as well 🙂

Misconception 7: Statistical significance is the most important metric

We hear it all the time: The test reached 95 percent statistical confidence, so we should stop it. However, when you look back at the test, between the control and the variation, only 50 conversions were collected (about 25 for each), and the test ran for only two days.

That is not enough data.

Note: Simple rule of thumb to start with: 200 conversions per variation, minimum 2 weeks of testing, increment by full weeks to account for the inconsistencies between workdays and weekends.

Misconception 8: Business owners understand their customer base and visitors

It’s easy to get stuck in the “A/B testing world,” looking at data and numbers, etc. But one of the best sources of learning is still having real conversations with your customers and ideal contacts. It also increases the conversion rate.

Note: Despite business owners running the business sometimes for decades, and knowing their customers base intimately, there NEVER was a case when a comprehensive round of research and user testing failed to uncover things that surprised them.

Misconception 9: Only change one thing at a time

It isn’t template tweaking (e.g., just changing locations and design of elements); it’s testing against an entire hypothesis which is backed by data resulting in data-driven changes that visitors can see and feel.

So don’t be afraid and test holistic changes, in addition to simpler tests.

Misconception 10: Make multiple changes each time

Sometimes a hypothesis becomes muddled because other changes are included within a single test. That makes it difficult to decipher the authenticity of the results and what element impacted the test.

If you run a test where you change a homepage and checkout, you’re not sure which change caused the effect. Isolate different areas in separate tests.

Misconception 11: Unpopular elements should be avoided

Elements that have been deemed unpopular (like image carousels) aren’t always unpopular with your customer base or your specific needs. If the research shows an element can provide a solution for you, test it before you completely discount it.

Misconception 12: Your site is too small for CRO

Conversion rate optimization is not only about testing. CRO is about understanding your visitors and giving them a more engaging experience.

If you have the traffic to justify your theories, test! Otherwise, continuously update your site and measure your changes through observation of key metrics through your analytics or through usability testing.

Conclusion

Research the hell out of it. Then test. One idea at a time. Unless you have loads of traffic. Then listen to feedback. Repeat.

12 pieces of conversion optimization advice you should ignore

PS. I hate it when I explain what I do, only for people to ask if it’s SEO, because that’s about the only acronym they know in e-commerce. That’s another misconception which I can’t even start to explain here 🙂