A practical guide to A/B testing

Visitors are split at random between the original and one or more alternative versions. Because the split is random and the versions run at the same time, differences in results can be linked to the change itself rather than to the season, an ad campaign or a lucky week.

• The page gets steady traffic — ideally hundreds of visitors per version over a few weeks.
• There is one clear action you want more of, such as a form submission or booking.
• You are unsure which option is better. If the answer is obvious, just make the change.

With very little traffic, a test may run for months without a clear answer. In that case, focus on larger changes with bigger expected differences, or test one change at a time.

A hypothesis names the change, the expected effect and the reason. An illustrative example:

"Changing the button from Submit to Get my free plan will increase form submissions, because it tells visitors what they receive."

The reason matters: if the test confirms it, you have learned something you can apply to other pages.

If you change the headline, image and button at once and the result improves, you won't know which change helped. Test one variable when you want to learn, or test a deliberate bundle (for example a rewritten hero section) when you only need to know whether the new section beats the old one.

Pick the action that represents real progress, and decide on it before launch — switching goals afterwards makes it easy to find a "win" by accident. Good goals include:

• A visit to the thank-you or confirmation page.
• A completed form submission or booking.
• A custom event sent from your own code or tag manager.

A click on a button is a weaker goal: it shows interest, not a completed signup or sale.

An even split gives the fastest answer. Each visitor should keep seeing the same version on return visits, otherwise their behaviour gets mixed across versions. Watch for a sample ratio mismatch: if a 50/50 test ends up 60/40, something is wrong with the setup and the results shouldn't be trusted.

Conversion rate is conversions divided by visitors who actually saw that version. As an illustration, 40 conversions from 1,000 visitors is a 4% rate. Count only visitors who were genuinely shown the version — someone who left before it appeared shouldn't count against it.

Early numbers swing wildly. With 50 visitors per version, a difference of two or three conversions can look like a huge improvement and disappear a week later. Decide a minimum number of visitors and days in advance, and run for at least one or two full weeks so weekday and weekend visitors are both included.

Checking results every day and stopping as soon as one version leads is the most common way to declare a false winner. Stop only when your planned sample and duration are reached and the statistical check agrees. "No clear difference" is a valid result — it tells you the change didn't matter much.

• Write down the hypothesis, dates, goal, visitors per version and the outcome.
• Apply the winning version in your page builder, then remove the test.
• Use the reasoning behind wins and losses to choose your next test.

Flowvert runs these tests on the pages you already have: install one script, edit text visually, pick and check a goal, and launch. Results stay marked as preliminary until the minimum sample, duration and traffic-split checks pass. See how Flowvert works or read the landing page testing guide.