You’re paying for Google Ads. People are clicking. A few estimate requests come in, but not as many as you hoped.
So you change things. You rewrite the ad, shorten the form, update the landing page and raise the daily budget, all in the same week.
Two weeks later, leads are up 30%.
Success?
Not yet. You have no idea which change did it. You also can’t tell whether the jump came from your changes or from something else, like the first cold snap of the season. You can’t repeat what you can’t identify, so your next budget decision is still a guess.
A good A/B test fixes that. You change one thing, decide what counts as a win before you start, cap the spend, and let the numbers answer.
What an A/B test is (and what it isn’t)
An A/B test splits your visitors at random between two versions of one thing, at the same time. Version A is what you have now. Version B has the one change you want to try. Then you compare the results.
The “at the same time” part matters more than most people think.
| A/B test | Before-and-after change | |
|---|---|---|
| How it works | Visitors are split at random between two versions during the same weeks | You run the old version, then switch to the new one |
| What else can sway the result | Very little. Both versions see the same weather, season and competitors. | The season, a cold snap, a competitor’s sale, a holiday, a new ad |
| What it tells you | Whether the change most likely caused the difference | Whether the numbers moved, not why |
| Use it for | Decisions with real money behind them | Quick fixes you’d make anyway |
Both have their place. Just don’t call a before-and-after change a proven test.
Step 1: Find where you’re losing customers
Before you test a headline or a button, ask a bigger question: where are people getting stuck?
Testing the wrong part of your marketing wastes the whole test budget. A shorter form won’t fix phone calls nobody answers.
| Where it leaks | What you’ll notice | Start here |
|---|---|---|
| Traffic | Few clicks, or clicks from people you can’t serve | Search-Term Review |
| The page | Clicks, but few calls or form fills | Ad-to-Page Match Checker and Website Next-Step Check |
| Lead quality | Plenty of form fills, but many are junk or out of your area | Lead Cost Calculator |
| Follow-up | Good leads that never become customers | Lead Follow-Up Planner and our guide to getting leads but no customers |
Step 2: Pick one change and write down why
Meet Example Heating, a made-up HVAC company in Cherry Hill. (Everything about Example Heating is fictional: the company, its traffic and its results. It’s here to show the method, not to promise a result.)
It’s fall. Example Heating runs a furnace tune-up ad, and its estimate form asks for 11 pieces of information. The owner suspects the long form is costing her requests.
She doesn’t change the ad, the offer, the headline or the budget. She changes one thing: she cuts the form to five fields (name, phone, address, system type and best time to call).
Then she writes down why she expects it to work:
Step 3: Decide what counts as a usable lead
If you measure “form submissions,” a shorter form will almost always win. That doesn’t mean it brought you more business. It might just bring more junk.
So Example Heating defines a usable request before the test starts:
- A real phone number that reaches a person
- An address inside her service area
- Someone who wants a tune-up or repair she actually offers
She also picks guardrails: numbers that must not get worse, even if usable requests go up.
| Guardrail | Why it matters | Her line in the sand |
|---|---|---|
| Junk requests | More spam means more wasted office time | Stays under 5% of requests |
| Requests missing details | Missing info means extra callbacks | Watch it, and plan for it if it rises |
| Booked jobs per usable request | The number that actually pays the bills | Must not fall |
Step 4: Work out how many visitors you need (the test that almost fooled her)
This is the step most small businesses skip. It’s also where most test budgets are wasted.
Example Heating’s first plan was simple: split visitors 50/50 between the two forms for two weeks and see what happens. Here’s what she got:
| First two weeks | Visitors | Usable requests | Rate |
|---|---|---|---|
| Original 11-field form | 200 | 10 | 5.0% |
| New 5-field form | 200 | 15 | 7.5% |
Half again as many requests. She was ready to switch forms and call it a 50% win.
Then she checked whether the test was big enough to tell her anything. It wasn’t.
With only 200 visitors per version, a gap like 10 versus 15 shows up about 3 times in 10 even when the change does nothing at all. That’s not evidence. That’s a coin toss with better odds.
The fix is to work backward from the smallest improvement worth acting on:
| The jump she wants to be able to catch | Her rate would go from 5% to | Visitors needed per version | Total visitors |
|---|---|---|---|
| Half again (+50%) | 7.5% | about 1,470 | about 2,940 |
| Double (+100%) | 10% | about 435 | about 870 |
| What 200 per version could catch | about 13% (nearly triple) | 200 | 400 |
Calculated at the settings most testing tools use by default: 95% confidence and 80% power.
At her traffic, catching a 50% jump would take about 15 weeks, and the heating season would change underneath the test. So she asks a business question instead of a statistics question: what improvement would actually change my decision?
Her answer: a doubling. Going from about 44 to about 87 usable requests a month would mean roughly 9 more booked jobs. Anything smaller isn’t worth changing the form for. A doubling needs about 435 visitors per version, so she plans for 600 per version over six weeks to leave room.
Step 5: Set the budget and lock it in the ad platform
Your test budget isn’t a number you pick. It comes from the visitors you need:
Test budget = visitors needed × your cost per click
| Example Heating’s test | |
|---|---|
| Visitors needed | 1,200 (600 per version) |
| Average cost per click | $3 |
| Test budget | $3,600 over six weeks |
| Spending cap | $600 per version every two weeks, set inside Google Ads |
| Changes allowed mid-test | None |
Set the cap inside the ad platform, not in a spreadsheet. Know how your platform handles it. Google Ads, for example, says a campaign can spend up to twice its average daily budget on a given day but no more than 30.4 times that daily average over the month.
If you’re running the test as a Google Ads experiment, Google’s own guidance is worth following:
| Google Ads experiment guidance | What it means for you |
|---|---|
| Run for at least 4 to 6 weeks, longer if leads take a while to come in | Example Heating’s six weeks fits |
| The first 7 days are left out of results to allow for ramp-up | Don’t judge anything in week one |
| Running several experiments at once isn’t recommended | One test at a time |
| A 50/50 split doesn’t guarantee equal spend | Compare the results, not the spend |
Step 6: Let it run, and don’t peek
Here’s the most expensive habit in testing: checking every day and stopping the moment the new version looks ahead.
Early leads swing back and forth. If you stop at the first good-looking day, you’ll “find” winners that aren’t real.
- Do check that both forms work and that requests reach the office.
- Don’t read the results to pick a winner before your review date.
- Don’t change the budget, the page, the offer or the targeting mid-test. Any of those resets the comparison.
Step 7: Read the result and decide
Six weeks and 1,200 visitors later, here’s Example Heating’s fictional result:
| After six weeks | Visitors | Usable requests | Rate | Booked jobs |
|---|---|---|---|---|
| Original 11-field form | 600 | 30 | 5.0% | 6 |
| New 5-field form | 600 | 60 | 10.0% | 12 |
This time, a gap this size would show up by chance only about 1 time in 1,000 if the form made no difference. Now she checks her guardrails before celebrating:
| Guardrail | Original | New form | Verdict |
|---|---|---|---|
| Junk requests | 1 of 30 (3.3%) | 2 of 60 (3.3%) | Under her 5% line |
| Requests missing details | 2 of 30 (6.7%) | 7 of 60 (11.7%) | Higher. Planned for. |
| Booked per usable request | 20% | 20% | Didn’t fall |
Every rule she wrote down before launch is met. She keeps the short form.
It isn’t free, and she writes that down too. More requests now arrive with something missing. Her office adds two questions to the callback, which takes less than a minute.
Did the $3,600 pay for itself?
| Before | After | |
|---|---|---|
| Usable requests per month (about 870 visits) | about 44 | about 87 |
| Booked jobs per month (20% book rate) | about 9 | about 17 |
| Extra revenue per month at a $400 average job | about $3,500 |
A $3,600 test that pays back in about a month, and keeps paying every month after.
Every test ends in one of three places. Decide in advance what you’ll do with each:
| Result | What it means | What to do |
|---|---|---|
| Keep | The change hit your target and the guardrails held | Make it permanent and write down what you learned |
| Drop | It didn’t hit the target, or a guardrail broke | Go back to the original. You just saved yourself from a bad change. |
| Tie | No clear difference after the full test | That’s an answer too. Keep whichever version is simpler and test a bigger idea next. |
What if you don’t have enough traffic?
Many local businesses don’t, and that’s OK. If the math says you’d need months to get an answer, you have three good options:
- Test a bigger change. Big changes need fewer visitors to spot.
- Test where the traffic is. Pick your busiest page or campaign.
- Make the change without a test. If it’s clearly better (a working phone link, a form that actually reaches your inbox), just fix it and watch the numbers. Just don’t call it a proven win.
What we check before anyone runs a test
Here’s our approach at ThinkDMG. Before we test anything for a client, we check three things:
- Does the form actually reach the office? A test can’t fix a lead that never arrives.
- Does the page match the ad? Mismatched promises cost more than form length ever will.
- Does someone call back quickly? The best page in the world can’t save a lead nobody follows up on.
Fixing a broken path almost always beats testing a better one.
Five mistakes that waste test budget
| Mistake | What it costs you | The fix |
|---|---|---|
| Changing several things at once | You can’t tell which change worked, so you can’t repeat it | One change per test |
| Stopping as soon as it looks good | You “find” winners that aren’t real | Set the review date before launch and wait for it |
| Counting clicks or form fills instead of customers | You reward junk leads | Define a usable lead and track booked jobs |
| Trusting broken tracking | Duplicate or missing conversions send you the wrong way | Test the form yourself and confirm requests arrive |
| Spending more before fixing obvious problems | You pay to send more people to a broken page | Fix the basics first, then test |
Your blank test plan
Copy this table, or fill it in with the free Campaign Test Planner.
| Plan item | Your answer |
|---|---|
| Question (one sentence, one change) | |
| Original (exactly what you have now) | |
| Revised (exactly what changes) | |
| What stays the same (ad, offer, targeting, follow-up) | |
| Hypothesis (what you expect and why) | |
| Usable lead (your written definition) | |
| Guardrails (what must not get worse) | |
| Smallest improvement worth acting on | |
| Visitors needed per version | |
| Test budget (visitors × cost per click) | |
| Spending cap (set inside the ad platform) | |
| Review date | |
| Keep / Drop / Tie rules |
When should you ask a marketing agency for help?
Planning a test is one thing. Setting it up, checking the tracking and deciding where to invest next is another. You’ll likely get value from a second set of eyes if:
- You spend on Google Ads every month but can’t say where your leads come from
- Your reports show conversions that don’t match the calls and forms you actually get
- You can’t tell whether your landing page or your targeting is the problem
- You want better leads without simply spending more
- You have a promising result but aren’t confident enough to make a bigger budget call
Digital Marketing Group works with home services and local businesses across South Jersey and Greater Philadelphia on paid advertising, conversion tracking and landing pages that turn clicks into calls.
Frequently asked questions
How long should I run an A/B test?
Long enough to reach the number of visitors you worked out before launch, and at least two full weeks so every day of the week is covered. If you use Google Ads experiments, Google recommends at least 4 to 6 weeks and leaves the first 7 days out of the results. Set the end date before you start.
How much should I budget for an A/B test?
Multiply the visitors you need by your cost per click. In our fictional example, 1,200 visitors at $3 a click comes to $3,600. Don’t pick a budget first and hope it buys enough traffic. If the budget can’t buy the visitors, test a bigger change or a busier page.
Can I A/B test with a small budget?
Yes, but you can only spot big improvements. Test changes that could make a large difference, like form length, the offer or a clearer call button, rather than small tweaks like button color. If your traffic can’t answer the question in a reasonable time, make the obvious fixes instead.
Can I stop a test early if the new version is winning?
No. Early results swing back and forth, and stopping at the first good-looking day leads to false winners. Wait for your planned visitor count and review date. Check during the test only to make sure nothing is broken.
What if my A/B test is a tie?
That’s a real answer. It means the change doesn’t matter much at your traffic level. Keep whichever version is simpler or cheaper to run, and test a bigger idea next. If you stopped before reaching your visitor target, it isn’t a tie. The test was too small to answer the question.
Should I test my ad or my landing page first?
Start where people drop off. If people click but don’t contact you, test the page. If the page converts well but few people click, test the ad. Either way, make sure the ad and the page promise the same thing before you test anything.
Sources
- Google Ads Help: Experiments FAQs. Recommended experiment length of at least 4 to 6 weeks, the first 7 days left out for ramp-up, running one experiment at a time, and 50/50 splits not guaranteeing equal spend.
- Google Ads Help: About average daily budgets. Daily spend can reach up to twice the average daily budget; monthly charges are capped at 30.4 times the average daily budget.
About the numbers: Example Heating, its traffic and its results are fictional. The statistics in the example are real calculations on those fictional counts: a two-proportion z-test (p of about 0.30 for 10 vs. 15 out of 200; about 0.001 for 30 vs. 60 out of 600) and the standard two-proportion sample size formula at 95% confidence and 80% power.
