A growth lead we work with put two numbers side by side. Meta reported purchases on a handful of landing pages. Their own analytics counted orders from the same pages. Meta’s count was several times higher on every page. They asked two questions. Does the algorithm spend our money on sales it claims but did not cause? And which number should we trust when we decide the budget?
Every media buyer knows Meta over-reports. That knowledge does not help you decide anything. You can only act on the size of the gap. That size changes from one account to the next.
The gap has no fixed ratio
The best evidence comes from randomized trials. Gordon and colleagues compared Facebook’s observational measurement with 15 randomized Facebook trials: 500 million user-experiment observations and 1.6 billion impressions. They threw away each control group and re-estimated the lift with six observational methods. In half the studies, every method missed the true lift in purchases by a factor of three. In one study, a naive exposed-versus-unexposed comparison found a 316% lift where the trial found 73%. In another, the trial found 2.4%. The closest observational estimate found 1,306%. Only 6.6% of the target audience saw the ads in that study.
We read the research this way. Platform overstatement is structural, with no fixed ratio. It ranges from about 1x to more than 50x. It gets worse when few people see the ads, and when the outcome is a rare purchase.
Industry figures point the same way. Haus, an incrementality testing firm, has a benchmark that three guests repeated on the Marketing Operators podcast. In 82% of experiments, platform reporting and experimental results differ by more than 25%. In 60%, they differ by more than 50%. Polar Analytics reports that in 99% of the cases it observed, Google’s real CAC was two to three times the platform CAC, checked through post-purchase surveys. Each figure comes from specific accounts and periods. Do not apply any of them to your account as a constant.
Why the algorithm takes credit for sales it did not cause
The same research suggests an answer to the growth lead’s first question. Gordon’s team found that most estimates overstated lift because ad delivery favoured people already likely to buy. The algorithm optimizes for purchases. People who were going to buy anyway are the easiest purchases to find.
eBay showed the same mechanism on search. eBay switched off brand-keyword ads nationally. Organic search recovered 99.5% of the lost clicks. Its ads moved new and infrequent buyers. Frequent buyers supplied most attributed sales and would have bought anyway.
So the danger is real. Meta’s own report rewards the campaigns that reach your warmest people. Those campaigns can be the least incremental.
Turn the gap into a multiplier
PayFit, interviewed on a French growth podcast, shows how to make the gap usable. Meta reported a CPA of 100. PayFit’s internal attribution reported 500. A 50/50 test, with half the audience exposed and half held out, measured 300. The true CPA was three times Meta’s figure. PayFit kept the ratio from the experiment as a multiplier for later budget decisions. In practice: take Meta’s number, apply the multiplier, decide.
A multiplier beats a debate about which dashboard is right. It also has limits. Gordon’s results show that a multiplier measured at one exposure level, for one outcome, does not transfer to another. Re-measure it when spend, targeting, offer or market change.
What a small brand can run
Clean experiments get expensive. The DTC measurement stack described on Marketing Operators puts geo-holdouts and media mix modelling at $50,000 a month in spend for statistical significance. They become most meaningful around $250,000 a month. Haus places its own geo-test threshold around $1M a month in marketing spend. Lewis and Rao studied 25 field experiments covering $2.8M of spend. Only three could tell a 50% ROI from zero. At €10,000 to €20,000 of monthly spend, a one-month holdout cannot separate a 25% ROI from 5%. A small brand’s test answers a coarse yes-or-no question, over 10 to 16 weeks or more.
Faris, another podcast guest, goes further. He says most brands under $25M do not need a single incrementality test. In-platform ROAS plus blended MER is enough for him. The gap stays. The instrument changes. Three cheap instruments work at small scale.
Cut and observe. Switch a channel or campaign off and watch the whole business. Tshirt Corner, a brand interviewed on a French e-commerce podcast, switches ads off every Saturday, its weakest day. It reports unchanged revenue with better margin. Polar Analytics recommends a three-week shutoff so the effect shows in blended results. Seasonality, promotions and competitors add noise, so read the result as rough.
Ask the customer. A post-purchase survey asks new customers how they heard about you. Garcia, an agency operator, converts answers into a channel CPA. Divide the channel’s responses by the survey response rate to project its customers. Divide the channel’s spend by that number to get a declared CPA. Boku reached a 70% completion rate at €250,000 of monthly spend. Customers misremember. Question wording shifts answers. Treat surveys as directional.
Read the blended business. MER compares total revenue with total media spend. Lilly Skin saw Meta report a ROAS of 3. The team raised spend. MER did not rise the next day. Blended numbers cost little. Every platform has to share credit in them.
Trust follows from this order: contribution margin first, then MER, then platform ROAS, then CPA. Each step down is cheaper to measure and further from the truth.
Start measuring your multiplier this week
- Turn on a one-question post-purchase survey: “How did you first hear about us?” Offer one answer per channel, with Facebook and Instagram listed separately.
- Pick a window of at least four weeks. Count new customers in Shopify, survey answers per channel and your response rate.
- Project Meta’s customers: Meta answers divided by response rate. Compare with the purchases Meta reported for the same window, new customers only if your setup splits them. Meta’s figure divided by your projection is your first survey multiplier.
- Plan one shutoff. Choose a retargeting campaign or your weakest weekday. Record blended MER and new customers for three weeks before the cut and three weeks during it.
- Divide every Meta ROAS by your multiplier before you compare it with your break-even ROAS. Re-run the survey count each quarter.
The survey multiplier is rough. It still gives you a number of your own instead of a vendor average.
Judge your pages with the right number
The same multiplier tells you whether a landing page works. A page judged on Meta’s count can look like a winner while it adds nothing. On a binding-lever call, we look at your live angles, the pages they send people to and the numbers you use to judge them. We tell you which page to build first, or tell you to wait. Book the call.