Quick Introduction to A/B Testing Tools and Best Practices

November 09, 2018 

Quick Introduction to A/B Testing Tools and Best Practices

What is A/B testing?

A/B testing is mainly a comparison of two version of web pages to check which version is performing better. For example, make a comparison of two web pages by showing the two variants(let's call them as A and B ) to similar visitors at the same time. The version that gives a better conversion rate will be preferred. This type of testing is mainly an experiment where two or more variants of a page shown to users at random and Statistical Analysis used to determine which variation of page performs better for a conversion goal. It takes the guesswork out of website optimization and helps to take data-informed decisions that make business conversions from " think" to "know." It helps to know where to work for the growth of website or application.

All websites available on the web have a goal - a reason for them to exist. The reason explained as follows -

  • E-Commerce websites want visitors to buy their products.
  • SaaS Web applications want visitors to make Sign-up for a trial and then convert to paid visitors.
  • News and media websites always want from their readers to click on Ads or Sign up for paid subscriptions.

What to test while an A/B testing

The components of A/B test explained as follows -

  • Subheadline
  • Paragraph Text
  • Testimonials
  • Call to Action text
  • Call to Action Button
  • Links
  • Images
  • Content near the fold
  • Social proof
  • Media mentions
  • Awards and badges

Benefits of A/B Testing

It is testing that allows individuals, teams, and companies to make changes to user experiences by creating a new version for the same page while a process of collecting data on results of A/B testing.

While making changes, it helps to learn that why certain elements are impacted while collecting data for user behavior.

A/B testing is not only a one-off question or settling a disagreement; A/B testing is more than that.

It is a type of testing that is used to continually improve a given experience(which can be determined by collected stats).

It also helps to improve a single goal like improving a conversion rate over time.

For Example -

B2B technology company wants to improve sales, lead quality and also a volume from their campaign landing pages. To achieve above stated goal, a team of a company would try to do A/B testing changes for elements that mentioned under "what to test while A/B test section."

This type of testing acts as a pinpoint for changes that had an effect on visitor behavior and which changes did not effect on behavior.

With time, the effect of multiple changes from experiments combined into one that helps to demonstrate the measurable improvement of a new experience over an old experience of website or web application. For example, while making a comparison of two different version of Ads. What can be checked under this? In this, check what version of Ads gathers more clicks.


How A/B Testing Works?

While working on an A/B test, web page or application screen chosen and modified to create a second version of the same web page or screen.

It is made on the second version of the page as simple as a single headline or button or can be a complete redesign of the page.

Then after the above step, half of the traffic rendered to an original version of a page and half of the traffic rendered to a modified version of a page.

Then visitors of the same web page can either be rendered to a control or a Variation.

Engagement of users is measured for each either for control or for variation.

Is it possible to determine that after making changes what type of effect it is producing? It is producing a positive effect, adverse effect or not making any impact on visitor behavior.


Why A/B Testing Matters?

It is used by product developers as well as designers to check the impact of new features or changes for user experience.

The product onboarding, user engagement, modal, in-product experience optimized by using A/B testing; this process continues till goals not fulfilled.

A/B Testing process

The following A/B testing framework runs A/B tests -

Collect Data - Data collected while analytics will provide insight from where optimization starts. It begins with a high traffic area of a site or application to gather data faster. Using this process, pages with low conversion rate and high-drop off-rates obtained and improved.

Identify Goals - Conversion goals are mainly metrics used to determine whether or not a variation is more successful as compared to an original version. Goals can be small or can be considerable. Goals can be anything from clicking a link or button to buying a product.

Generate Hypothesis - Once getting a clarity that why this version should be better than previous. Once having a list of ideas or goals that why attempting to get new ideas. After making a list of ideas, prioritize them regarding expected impact and difficulty of implementation.

Create Variations - Make a new variation of the page including working on changing the color of a button, swapping the order for elements present on the page, hiding navigation items current on a page or customization. Many leading A/B testing tools like Visual Editor make changes easy. But it requires experience from a QA that new variations created using tools are correct.

Run Experiment - Make a new variation of the page along with this wait for visitors to participate or wait for visitors to render a web page. In this process, visitors on a site or application randomly rendered to either to control or variation of the web page. Interaction of user with each experience measured, countered and compared to determine how each version executing.

Analyze Results - Once experience is completely rendering users randomly either to control or variation, analyze the results. Mainly A/B testing of the web application or website present data from two different experiments. Apart from making a comparison, it also shows the difference between the two versions of pages( static difference or not).


How to Adopt A/B Testing?

Four step process for A/B test by Visual Editor explained as follows -

Add a Visual Website Optimizer code snippet in websites code - Adding or including code snippet allows test cases to run on a website. To make things easier for further, plugins for Wordpress, Drupal, and Joomla available to create a whole process of A/B tests hassle-free.

Creation of variations using a WYSIWYG Visual Editor - Load a website in a Visual Editor and create changes using a simple point and click interface ( include clicking on the link, clicking on a button). More advanced users make changes to CSS and JS Code.

Select goals - All the A/B tests for a web page of the website have goals. Here goal means to render a web page whose conversion rate required to increase(goals can be links, button, etc.). All goals defined straightforward(such as clicks on links and number of users visited a page), or some code level changes made to execute A/B tests for web pages.

Get started and track A/B tests - When defined A/B test is ready to go live. Then after that Real-Time reports achieved. Data obtained on reports as soon as visitors start arriving on a Real-Time test.


Best Practises of A/B Testing

Google permits that A/B test does not possess not any risk for the website and not affects search rank of the website. Following Best Practices considered while execution of A/B Test cases -

No Cloaking

To show search engine different content than a typical user would see. It results in the removal of content from search results. To prevent cloaking, do not adapt visitor segmentation to display different-different content for different users.

Use rel="canonical"

While execution of a split test with multiple URLs, rel="canonical" attribute used for variations to back to the original version of a page. Preferring this, prevents Googlebot getting confused by multiple versions of the same page.

Use 302 Redirects Instead Of 301s

While execution of test that redirects to original URL to a variation URL, such as uses 302 (temporary) redirect via a 301(permanent) redirect. This tells a search engine such as Google that the redirection is going on a temporary basis and Original URL indexed than a testing URL.

Run experiments as long as they are necessary

To run tests for a longer time than an essential, especially in a case while serving one variation of the page to a more significant percentage of users then this condition seen as an attempt to deceive (fraud)search engines. Google recommends updating site and removing all test variations from the site as soon as testing purpose fulfilled and to avoid unnecessary test runs from a site.

For more information about the implementation of A/B tests. This thing can be cleared more from the following example -

A technology company requires to increase the number of high-quality leads for their sales team, enhance the number of free trial users and to attract a specific type of buyers. Then a different variation of the same page required tested for various components and these components are defined as follows -

  • Lead from components.
  • Free trial Sign up the flow.
  • Homepage messaging and making a call to action (such as clicking on button and links generates a different event).

A/B Testing Tools

A different number of tools used to do tests on a web page of the website. But measurably of tests improved using Google analytics.

Following tools can be used to do A/B tests -

  • Visual Editor Analytics
  • Google Analytics

Concluding A/B testing

In conclusion, A/B testing is the best way to learn about audiences preferences and to make changes to a website in such a way that it helps to fulfill all the essential goals or helps to meet requirements expected from a website.

Such as a technology company's website wants to get more and more bids from clients. For such a site there is a client, there is a goal to get a lot and lot number of bids from a client.

Once A/B test case is written or basic framework is set, test cases can be executed and improvements can be made to obtain more customers.