Incrementality Testing: How to Know Whether Your Google Ads Are Really Driving Sales

You can have a Google Ads account with excellent reported ROAS, clean conversion tracking and a sensible structure, and still not know whether your ads are actually creating extra revenue.

That is because every attribution model, including Google’s data-driven one, answers the question “which ads got credit for the conversions?”. None of them answers “would those conversions have happened without the ads?”. They are different questions, and only the second tells you whether advertising creates value.

Incrementality testing is how you answer it.

Why Reported ROAS Is Not Incremental ROAS

When someone clicks your Shopping ad and buys, the sale is credited to your campaign. Now imagine pausing that campaign for a week:

• Some people who would have clicked the ad find you through organic search instead
• Some go straight to your URL from memory
• Some click a competitor’s ad and buy elsewhere
• A small number would never have found you at all without the ad

Revenue from the first two groups is not incremental, because it would have happened anyway. Only the last two groups represent revenue that advertising is genuinely responsible for.

An advertiser with strong brand awareness, high organic rankings and lots of direct traffic may find that pausing Shopping costs far less revenue than reported ROAS suggests. One with weak awareness and little organic visibility may find its ads are almost entirely incremental, with nearly every paid conversion impossible without the click.

Most accounts sit somewhere in between, and without testing you cannot know where.

Method 1: Geo Holdout Test (the Gold Standard)

A geo holdout is the most rigorous incrementality method available to most eCommerce advertisers. Split your market into two groups of regions, run ads in one, pause or heavily reduce them in the other, and measure the revenue difference.

Setting one up:

Step 1: choose comparable regions, similar in size, demographics and historical buying behaviour. For a UK store that might mean comparing regions such as the Midlands and the North West. For a US store, paired DMAs with similar income profiles and population.

Step 2: assign regions to treatment (ads running normally) and holdout (ads paused or sharply reduced). Randomise where you can, and do not put all your best markets in the treatment group.

Step 3: run the test for at least four weeks to cover weekly patterns and reach statistical significance. Shorter tests are usually underpowered.

Step 4: measure revenue in each group using your Shopify backend or order system, segmenting orders by delivery destination rather than by attributed channel. That gives you order counts that do not depend on ad attribution.

Step 5: calculate the lift: (treatment revenue per head minus holdout revenue per head) divided by holdout revenue per head. That is the incremental revenue lift from advertising.

Step 6: compare that lift with ad spend to get your true incremental ROAS.

Limitations: you need enough orders per region to reach significance. A store with 30 orders a month nationally cannot run a meaningful geo test, because there is too little signal above the noise. A store with 500+ orders a month spread across regions can run a credible one.

Method 2: Google Ads Campaign Experiments

Google Ads has a built-in experiments framework that splits traffic between a control (your current setup) and a treatment (the change you want to test). It is the easiest controlled test to run inside the interface.

For Search, Shopping and PMax, you can run an experiment alongside the existing campaign on a 50/50 split to test, for example:

• A different bid strategy
• A different ROAS target
• A different asset group setup
• Adding a campaign type, such as measuring the uplift from adding PMax alongside existing Shopping campaigns

Experiments are designed to compare two live versions of a campaign rather than to switch a campaign off entirely. For a full “ads on versus ads off” measurement, use a geo holdout (Method 1) or a Conversion Lift study (Method 3).

Their big advantage over geo tests is that the split is randomised by Google at the auction or user level, which avoids the self-selection bias that can creep into regional comparisons.

The limitation is volume: you need enough conversions in the experiment to reach significance. Google Ads shows a significance indicator in the results. Do not draw conclusions until confidence is above 95%.

Method 3: Conversion Lift Studies

Google offers Conversion Lift studies to accounts with enough scale (typically in the region of $30K or more in monthly spend). Google runs the test for you, using a holdout group that is deliberately not shown your ads and measuring the difference in conversion rates.

Conversion Lift is more rigorous than a do-it-yourself geo test because Google controls the randomisation of the holdout. You set it up with a Google Ads representative.

If your account qualifies, it is worth asking for. You get a measured incremental conversion rate and an incremental ROAS that is methodologically stronger than any self-service test.

Method 4: Causal Impact Analysis

When a controlled test is not practical, a time-series causal impact analysis can estimate incrementality from historical data. It is particularly useful if you run several channels (Google Ads, Meta Ads, email) and want an early read before investing in a full experiment.

The approach uses Google’s open-source CausalImpact package (available for R and Python) to model what revenue would have been without advertising, using control variables such as organic traffic, email revenue and external demand. The gap between the modelled baseline and actual revenue during the advertising period is your incrementality estimate.

It is more technical and less precise than a controlled experiment, but it needs no live test and is useful for establishing a starting point before you run one.

What to Do With the Results

If testing shows high incrementality (80% or more of attributed conversions are truly incremental), your reported ROAS is a fairly reliable guide, and you can trust Smart Bidding because its inputs reflect real ad-driven value.

If testing shows low incrementality (40% or less), you have a problem. Your reported ROAS is heavily inflated by credit for organic, direct and brand sales that would have happened anyway. The answer is not to panic. It is to restructure where you spend and where you do not.

Common low incrementality findings and responses:

Brand campaigns show low incrementality. Your organic brand ranking is strong, so most people searching for you would find you without the paid ad. Reduce brand spend, or move the brand campaign to Maximise Clicks to keep CPCs low rather than running it on tROAS.

Retargeting shows low incrementality. Basket abandoners are coming back to buy whether or not they see retargeting ads. This is common for stores with strong email marketing: the emails do the work and the retargeting ad simply collects the credit. Reduce retargeting spend and watch whether basket recovery rates change.

Non-branded Shopping shows high incrementality. This is where your ad spend genuinely creates new revenue, and where to invest more.

The Honest Baseline

Most eCommerce stores have never run an incrementality test, so all their advertising decisions rest on attribution data that may significantly overstate ad-driven revenue.

That does not mean Google Ads is not working. For most stores it is. But how well it works, and where in the account the real value sits, can only be known by testing.

Running one geo holdout or controlled test a year on your biggest campaign, during a representative trading period, gives you a calibration point. You might find your reported 400% ROAS is really 280% incremental ROAS. That is still profitable, and now you have a realistic benchmark to manage against.

Or you might find your non-branded Shopping is almost fully incremental at 350% ROAS, while retargeting reporting 900% ROAS is barely incremental at all. That reallocation insight alone justifies the test.

The aim is not to prove advertising does not work. It is to know which parts work, and by how much, so budget follows real value rather than attributed credit.