How to Run Statistically Significant A/B Tests
Decide the goal and the change before you start, split visitors randomly between the two versions, run the test to a length you fixed in advance, and read the result only at the end. Small sites should test bigger changes, because small ones need more visitors than they have.
Free plan, no card needed to open it. Affiliate link: we may earn a commission at no cost to you. How we make money.
Publisher

Fix it step by step
7 clear steps, a few hours to set up; two to four weeks to run for a typical site.. Jump to the steps.
Free to read
No sign-up to read this guide or use the free resource at the end.
A no-cost place to start
Our pick is Systeme.io: a free plan with email, funnels and automation. No card needed to open it. See why.
The problem
Stopping a test the moment one version leads is the most common mistake. Early differences are often chance, and the result flips if you wait.
A trustworthy test needs enough visitors and a rule for when to stop. Many small sites do not have the traffic for small changes, and that is useful to know.
This guide is for you if
- You can show two versions of a page to different visitors
- You have a steady flow of visitors and a clear goal
- You are willing to wait for the test to finish
It is not for you if
- You get very few visitors
- You change the page during the test
- You want a quick answer from a handful of visits
The steps
A few hours to set up; two to four weeks to run for a typical site.
- 1
Choose one goal
One number such as form submissions per visitor. Decide it before you look at any data.
- 2
Choose one change
A headline, a button, a form length. If you change several things you will not know which one worked.
- 3
Estimate the visitors you need
Use a sample size calculator with your current conversion rate and the smallest improvement that would matter. If the number is more than you can get in a month, test a bigger change.
- 4
Split traffic randomly
Half see the current version and half see the new one at the same time. Use your testing tool's split, not alternate days.
- 5
Fix the end date
Choose it from the sample size and run whole weeks. Do not stop early because one version looks ahead.
- 6
Read the result at the end
Most tools show a significance level; 95% is a common convention. If the result is not clear, treat it as no difference.
- 7
Record and act
Keep the winner, note what you learned, and test the next idea. A test that shows no difference still teaches you that the change does not matter.
Finished the steps? Put them to work.
Free plan, no card needed to open it.
Common mistakes
- Stopping when one version leads.
- Testing several changes at once.
- Running tests with too few visitors.
- Changing the page mid-test.
Our recommendation
Systeme.io: our pick for solo creators and small businesses starting out
Why
- Free plan with funnels, email, a course and a blog; no credit card needed.
- Paid plans start at $17/month when you outgrow 2,000 contacts.
- Funnels, email automation and checkout live in one account.
Not right for you if
- You need two-way SMS, call tracking or a shared team inbox.
- You manage several client or location accounts.
- You need advanced CRM pipelines and reporting.
Price: Free plan (2,000 contacts); Startup $17/month, Webinar $47/month, Unlimited $97/month. Source, checked 27 Oct 2026.
Alternative: If you need SMS, calls, booking and reviews for a local service business or agency, HighLevel fits better (14-day trial).
Disclosure: Unircretraites is reader-supported. If you sign up through our links, we may earn a commission at no extra cost to you. It never changes our recommendations, and we say when a tool is not right for you. How we make money
A free resource for this
Landing page test plan. Free to use, no sign-up.