Measurement
Part of How to run a retail media incrementality test
How to run a retail media incrementality test
How to run a retail media incrementality test in the UK using Nectar and Clubcard loyalty data, with holdout design, uplift maths and UK GDPR rules.
What to take away
- Incrementality testing answers a different question from attribution: it estimates the sales that would not have happened without the campaign.
- UK tests usually rest on a holdout group, matched store pairs or a planned switch-off window, chosen before the campaign starts.
- Loyalty data from Nectar and Clubcard can carry much of the measurement, once the lawful basis and the purpose are settled.
- Report uplift as a range, name the source of the baseline, and record what the test could not cover.
Why attribution and incrementality give different answers
Attribution divides credit for sales that happened. Incrementality asks how many of those sales would have happened anyway. A sponsored placement shown to a shopper already searching for the brand can post a strong return on ad spend and sell almost nothing extra.
The two numbers rarely agree, and that is expected. A platform dashboard reports what it observed. A test reports what changed because of the campaign.
Deciding what to hold back
The design question is what you hold back from advertising, and for how long. Three options cover most UK campaigns.
| Design | How it works | Best fit | Main limit |
|---|---|---|---|
| Holdout group | Withhold the campaign from a share of shoppers or stores | Always-on sponsored placements | Needs enough volume per cell |
| Matched store pairs | Pair similar stores and advertise in one of each pair | In-store screens and off-site | Pairs drift over several weeks |
| Switch-off window | Pause the campaign for a set period and compare | Concentrated brand spend | Seasonality confounds short windows |
| Geo split | Compare matched regions | Regional stock differences | Spillover between neighbouring areas |
Whichever unit you choose, fix the split before the campaign goes live. Redrawing the holdout after launch destroys the comparison.
Using Nectar and Clubcard data lawfully
Loyalty schemes supply a persistent identifier across baskets. That is what an incrementality test needs in order to link exposure to later purchases. Nectar, sold through Nectar360, and Tesco Clubcard both sit behind measurement products that brands can buy.
The limit is purpose, not usefulness. Data collected for a loyalty scheme is not automatically available for advertising measurement, so check the notice given to shoppers and the lawful basis recorded at the time. The ICO's guide to the UK GDPR sets out how that choice is made and documented.
Where a test combines loyalty identifiers with campaign exposure at scale, a data protection impact assessment is normally expected. The UK GDPR text is the reference for what counts as personal data and what that assessment has to cover. Also agree in writing who controls the joined dataset: the retailer, the media network or the brand.
A five-step sequence for the test
Definitions come first. Decide whether the test measures sales, units, new customers or penetration, and align the terms with the IAB UK retail media standards so that a retailer's report and your own figures describe the same event.
- Write the hypothesis and the decision it informs, including the smallest uplift that would change your spend.
- Pick the unit of the test: shopper, store, region or session.
- Set the baseline from the ONS retail industry series and your own prior-year sales for the same weeks.
- Split the audience in advance and keep the holdout closed for the whole read window, including any late reporting period.
- Report uplift as test sales minus control sales, divided by control sales.
Working out the uplift in pounds
A worked example shows the arithmetic. Forty advertised stores take £240,000 over four weeks; forty matched control stores take £200,000. The difference is £40,000, or 20 per cent uplift. If media and fees cost £25,000, the campaign returned £1.60 in incremental sales for each £1 spent.
Then check what the uplift contains. Some of it may come from shoppers who switched between the brand's own lines, or from stock that would have sold in a later week anyway.
Record the limits as carefully as the result. A retail media quality checklist gives you a place to log the channels the retailer cannot see, the stores excluded from matching and the weeks lost to stock shortages.
Common questions
What sample size does a retail media incrementality test need?
It depends on the expected effect and the variance of sales in the unit you choose. A test looking for a 5 per cent uplift needs far more stores or shoppers than one looking for a 25 per cent change.
Can a brand test incrementality without loyalty data?
Yes, using store-level till data, matched store pairs or regional splits. The trade-off is coarser resolution, because you can see a sales change but not which shoppers caused it.
How long should a holdout stay closed?
Long enough to cover the whole purchase cycle and any reporting lag, often four to eight weeks for grocery. Shorter windows invite seasonality to explain the result.
Does a platform's reported uplift count as incrementality?
Rarely on its own. A platform measures the sales it can observe and attribute, and that usually includes sales that would have happened without the ad.