A/B test two versions of a quiz

An A/B test shows your live quiz (A) to some visitors and a changed version (B) to the others, at random, so you can compare them on the same days and the same traffic. Open it from the quiz page or the editor: A/B test. It's on every plan, at no extra charge.

This release is descriptive only; no winner yet: the test page shows how the test is running, not which version is better. Quizproof doesn't pick a winner, estimate revenue or say whether a difference is significant yet.

Set up B#

B is your draft. Make your changes in the editor as usual; the A/B test page shows what B changes compared with A.

• Edit B in the builder opens the editor on your draft.

• Start B from a copy of live replaces your draft with the live version first, so B starts from A. If your draft has changes that aren't published, you're asked first: they are lost. Cancel keeps your draft as it is.

A has to be live: publish the quiz first, or resume it if it's paused.

Start the test#

1. Pick the Split: each new visitor's chance of getting B, from 10% to 90% in steps of 5 (50% by default).

2. Add a Note if you like (what you're testing). Only staff see it, in the test's history.

3. Press Start test.

B goes through the same checks as a publish. If they find warnings you're asked to confirm them; errors are listed with a link to fix them. Start uses the draft exactly as you saw it: if someone saved the draft (or published) since the page loaded, nothing starts and you're asked to refresh.

Quizproof then uploads B to your store without making it live, checks it, and starts the test by itself. Shoppers keep getting A until then. Your theme blocks, buttons, links and the hosted page need no change: every placement takes part.

There's no estimate of how long a test needs yet ("Time estimate unavailable").

While it runs#

• Each visitor keeps their version for 7 days from their first visit (the follow-up), across your quiz's placements and visits.

• Enrollment closes after six weeks (42 days) at the latest. Pausing doesn't extend that.

• B is frozen when the test starts. Editing the draft doesn't change B; the editor says so. To test a different B, end this test (see below) and start a new one.

• Visitors whose cookie choice doesn't allow analytics still take part, for that page visit only.

• Completions count toward your plan as usual, in both versions. If your plan's limit stops new quizzes, the test stops taking new visitors for both versions alike.

Pause, end, keep A or make B live#

• Pause test: no new visitors join. Visitors already in it keep their version. Resume test continues with the same versions and split. Pausing the quiz itself pauses the test too: resume the quiz first, then the test.

• End test keeps A: no new visitors join, and visitors already in it keep their version until their 7 days are up; then the test ends. New visitors get A. Tick Also stop the follow-up now to end at once (the visitors still in their 7 days stop counting). During the follow-up, Stop follow-up now does the same.

• Make B live now ends the test at once (the follow-up is cut short) and makes B live for everyone through the normal publish steps. If that publish fails, A stays live, the page says "B isn't live yet" and Try again repeats it. This isn't a statistical result: it doesn't mean B is better.

• Cancel test stops a test that hasn't started yet. No visitor was enrolled.

You can start a new test once the current one has ended.

Publishing, rolling back or archiving the quiz while a test runs ends the test. Quizproof asks you first; the test ends at once, and its history is kept.

Preview each version#

Get A link and Get B link open that version on your store. They're for you and your team: visits through them aren't part of the test, don't count toward your plan's completions and aren't attributed to orders. A link that has expired or was changed shows the live quiz.

How the test is running#

The test page counts, for each version: the visitors who joined, split into visitors with analytics consent and page visits without it; how many saw the first question; how many couldn't load; and how many left the test later (for example by withdrawing consent). Visitor coverage mixes the two kinds of visit, so it isn't a share of people. These counts show that the split and the storefront work; they aren't results.

History#

Every test is listed with its versions, split, dates, note and how it ended, and Show what happened lists each step with who did it. In Versions, B shows as "In an A/B test" while the test needs it.

Related#

• Preview, test, publish and roll back

• Plans and limits

• Privacy and consent

• What Quizproof doesn't do yet