Your dashboard can tell you a channel converted. It cannot tell you whether the sale would have happened anyway.

Key points

• Attribution tells you which ad got the credit. Incrementality tells you whether the ad caused the sale.
• Media mix modelling looks at all channels over a year or more of data. It suits large, multi-channel budgets.
• An incrementality test changes one thing for part of the business and compares. It works at almost any budget, in weeks.
• In Qatar, audience holdouts and time-based tests are usually more practical than splitting the country by area.

Every platform report answers the same question: which touchpoint gets the credit. None of them answer the harder one — would this customer have bought anyway, without the ad? That second question is what incrementality testing and media mix modelling exist to answer, and they answer it in almost opposite ways.

Both start from the same complaint: platform-reported ROAS is not economic reality. It is a story the platform tells about its own contribution, and the platform has an incentive to tell that story generously. Both methods exist to check the story against something the platform cannot influence — an actual change in the real world, or a long enough run of real-world data to separate marketing’s effect from everything else moving revenue at the same time.

Two methods, two very different shapes

Media mix modelling is a statistical model built across every channel — paid, organic, offline, even weather and seasonality where it is available — regressed against revenue over a long historical window. It is comprehensive and it can weigh in on channels no pixel ever touches, like a billboard or a sponsorship. It needs months, usually a year or more, of clean historical data before it says anything trustworthy, and it is slow to pick up a channel that only started three months ago.

Incrementality testing is narrower and faster: a live, in-market experiment on one specific change — a channel, a geography, a budget level — measured against a control that did not receive the change. It answers one question well rather than the whole mix loosely, and it can be run in weeks rather than quarters. The trade-off is that it only tells you about the thing you tested, not the system as a whole.

Both have three advantages over platform-reported numbers. Neither depends on a pixel or a cookie, so a browser update that removes tracking does not break them. Both rely on aggregate results rather than tracking individuals, so they hold up better as privacy rules tighten. And both give you a verdict the platform did not produce about itself.

How an incrementality test is actually run

The mechanics are simple to state and easy to get wrong in practice. You run a control period where nothing changes, then a test period where you scale or launch the thing you want to measure, then compare the two.

The simplest version: turn off a channel entirely, for one geography or one audience, for a defined period, and measure what actually happened to revenue there against a comparable area where nothing changed. The gap between the two is your real, incremental ROAS for that channel — not the number the platform was reporting for it.

Any channel that supports geographic targeting can run this way — paid search and Meta both do. The audience-side version works the same way through prospecting cohorts on Snapchat, or a CRM-contact holdout on email and EDM: some contacts see the campaign, a comparable group does not, and the difference is the lift.

The design decision that actually matters is how much of the business sits inside the test. Cell A stays business-as-usual and carries the majority of revenue, so the test cannot damage the quarter. Cell B is deliberately the smaller, lower-stakes share of the business, and it is where you make a change large enough to produce a signal you can actually read — a full holdout, or a meaningful multiple of the existing budget. A change too small to notice is also too small to measure; the protection against risk comes from limiting how much of the business is exposed to the test, not from limiting the size of the change within it.

The condition that is easiest to break

An incrementality test only means something if nothing else moved during it. No pricing change, no new offer, no website redesign, no promotion running in the background — because if anything else changed, you cannot tell whether the lift came from the media or from the other change.

The second mistake is subtler and more common: pausing every other channel to isolate the one you are testing. It feels rigorous, but it usually is not. Channels interact — paid social often primes a search click that search then gets credited for — and switching several off at once changes the whole system’s behaviour, not just the one variable you meant to isolate. Once everything comes back on, the account frequently does not return to where it was, and the clean read you thought you got was measuring a temporarily different business.

The steadier approach is to leave everything else running exactly as it was and make one deliberate, sizeable change to the channel under test — scale it up, or scale it to zero in one cell — while the rest of the account continues undisturbed. It is a less dramatic experiment and a more trustworthy one.

A lighter-weight alternative: coupon-code splits

A simpler version of the same logic, and one that does not require pausing anything: offer a unique coupon code in some ad campaigns and withhold it from others running in parallel — split by geography, by device, or by audience segment. Redemption rates between the two groups give you a working estimate of incremental lift without switching a channel off at all, which matters if you cannot afford to stop advertising anywhere while you test.

At the more sophisticated end of this — ghost ads and clean-room measurement through a demand-side platform — the same causal logic runs as a permanent background test rather than a one-off project, comparing an exposed group against a matched group that would have seen the ad but did not. It is genuinely elegant, and it typically needs a platform partnership and a media budget that puts it out of reach for most Qatar-sized accounts. Worth knowing it exists; not usually worth chasing before the basics are in place.

The lookback window is not a setting. It is a finding.

Every attribution model needs a lookback window — the gap it assumes between someone seeing an ad and eventually buying. Most advertisers inherit the platform default and never revisit it. An incrementality test is one of the few reliable ways to actually find the right number for your business, because it shows you how long the lift from a change takes to show up and how long it takes to fade, rather than assuming a figure that was set for a different category entirely.

What this means at a Qatar-sized budget

Full media mix modelling wants a large multi-channel budget and a long, clean data history — both offline and online — before it earns its cost. For most of the businesses we work with, that bar is not worth clearing yet, and we would say so rather than sell a model against data that cannot support one.

Qatar adds one practical wrinkle. Most of the population lives in and around Doha, so splitting the country into test and control areas is hard: the areas are too close and too few. Two designs work better here. An audience holdout, where a share of your CRM contacts or prospecting audience is kept out of a campaign. Or a time-based test, where you change one channel for a set period and compare it with a matched period before.

A simple incrementality test is a different story. A geo or audience holdout on one channel is achievable at almost any budget, takes weeks rather than quarters, and answers a question worth answering long before the account is large enough for MMM: is this channel’s reported ROAS the real number, or the number the platform would like you to believe.

There are third-party platforms built specifically to run this kind of testing at scale — Measured is one well-known example. At the budgets we typically manage, a plainer, in-house version of the same test design gets most of the same answer without the platform fee, which is the version we would actually run for you.

This sits next to a point we make elsewhere: incremental return and last-click return are different numbers answering different questions, and only one of them tells you whether the spend caused the revenue. Incrementality testing and media mix modelling are simply the two disciplined ways of getting to that number, at two different scales.

Common questions