Why your MMM's range matters more than its number
A mix model that says TV returns 2.6 is also saying it might return 1.5 or 3.8. Budget decisions made on the single number ignore most of what the model knows.
By Nathan Barling Product Marketing Principal 5 min read
A mix model readout gives each channel one ROAS number, and the range around it rarely makes the slide. So when a refresh moves TV's ROAS from 2.6 to 1.9 and the CFO asks which one is right, the honest answer may be both.
If the model's range for TV runs from about 1.5 to 3.8, a move from 2.6 to 1.9 is the model wobbling inside what it already said it didn't know. The question that matters is different: does TV's range clear the point where it pays for itself?
One dataset, many good answers
David Chan and Mike Perry of Google laid out why in a 2017 paper on media mix modeling. The data is thin for what it's asked to do. Three years of national weekly data is 156 data points, and the model is often expected to cover 20 or more channels, each needing three to four parameters for lagged effects and diminishing returns.
Data points in three years of weekly national data
Chan and Perry, Google, 2017
Ad channels a typical model is asked to separate
Chan and Perry, Google, 2017
How far five equally well-fitting models disagreed on sales
Chan and Perry, Google, 2017
They then showed what that means in practice. On one real advertiser's data, they fit five plausible models that differed in small, defensible ways: a different seasonality proxy, two years of data instead of all of it, branded search left out, a time trend added. Every model scored an R² of 0.98 or 0.99. Their sales predictions under different budgets differed by up to 50%.
Their conclusion is blunt: a focus on predictive accuracy "will not allow a modeler to eliminate models that produce inaccurate ROAS estimates."
Even experiments are noisy
Randomized tests are the usual cure, and they are the cleanest causal evidence marketing has. They're also noisier than most people expect. Randall Lewis and Justin Rao studied 25 large field experiments with major US retailers and brokerages, covering $2.8 million in digital ad spend. The median confidence interval on return on investment was over 100 percentage points wide.
The reason is that individual sales swing far more than ad exposure does. Lewis and Rao found a coefficient of variation of 10 is common, so an informative experiment can easily need more than 10 million person-weeks. A test is still worth running. It just doesn't hand back a single exact number either.
Shortcuts tend to flatter
Methods that skip randomization usually look more certain than they are. In 15 advertising experiments at Facebook, Brett Gordon and colleagues compared randomized results with common observational methods on the same campaigns. The observational methods often failed to produce the same effects, even with extensive demographic and behavioral controls.
The direction matters. Florian Zettelmeyer, one of the authors, said the more common methods overestimate ad effectiveness relative to the randomized tests, though in some cases they significantly underestimate it. A precise number from a biased method is still wrong. It's just wrong with confidence.
Four ways to decide with a range
1. Ask for the range with every number
Every channel's ROAS or ROI should come with an interval, such as an 80% or 90% credible interval from a Bayesian MMM. If a model can't give one, it hasn't told you how much it knows. Report the range in the deck, next to the number, every time.
2. Compare the range with breakeven
The decision rarely turns on whether TV returns 2.4 or 2.7. It turns on whether TV pays for itself. Put each channel's range next to its breakeven ROAS, which is one divided by gross margin.
| Channel | Point estimate | Range | What to do |
|---|---|---|---|
| Paid search | 3.4 | 3.0 to 3.9 | FundThe whole range clears breakeven |
| Online video | 2.6 | 1.4 to 4.1 | TestThe range straddles breakeven |
| Display | 1.1 | 0.7 to 1.6 | CutThe whole range falls short |
| TV | 2.9 | 2.6 to 3.3 | FundNarrow and above breakeven |
The ranges in this table are illustrative. Notice that online video's point estimate is higher than breakeven, and a team reading only the number would fund it. Its range says the model doesn't know. That makes it the channel to test before anyone scales it.
3. Test where the range is wide and the money is big
An experiment is most valuable where uncertainty and spend are both high. That's also how a model gets better. Google's Meridian documentation describes calibration as the process of using experiment results and other domain knowledge to set channel-specific priors. A holdout or geo-lift test narrows that channel's range in the next refresh.
Meridian's documentation also notes that experiments reflect their own conditions, such as the time window, regions, and campaign settings, and that different experiments can give different results. Calibrate with several tests over time and treat any single one as one piece of evidence.
4. Plan inside the data you have
Chan and Perry show four response curves that fit a narrow spend range equally well and then split apart as spend rises. A model fit to budgets between $2 million and $3 million can't say much about $6 million. Move in steps: shift budget within or just past the range the model has seen, measure, refit, and move again. The diminishing-returns curve at the edge of your data is the least certain part of it.
Where Alli fits
Alli Scenario Planner is built for planning under uncertainty. It gives modeled estimates with the assumptions in view, so teams can compare options and defend the choice to finance before money moves.
Every forecast blends a media response model built on historical sales with a brand equity model built on current signals from Alli Brand Equity. Two-stage estimation is designed to reduce the instability common in traditional models, so forecasts stay steadier as data changes. For every channel, minimum and maximum spend come with predicted revenue and ROAS, showing where each one sits on its diminishing-return curve, and teams can lock the channels they want to hold while testing the rest.
When the point estimate is enough
Sometimes the range doesn't change the decision. If every plausible value for a channel clears breakeven, or none does, the point estimate is fine for the call in front of you. Ranges also matter less for small budgets, where the cost of a wrong call is smaller than the cost of a test. Use the range when a decision is big and the range crosses the line that decides it.
Open your latest mix model readout. Find the channel with the most spend and ask what its range is. If nobody knows, that's your next question for the modeler.
Frequently asked questions
Why do marketing mix models give different answers?
Mix models have little data relative to what they estimate, ad spend across channels tends to move together, and modelers make dozens of reasonable choices. Google researchers found five equally well-fitting models whose sales predictions differed by up to 50%.
What is a credible interval in a mix model?
It's the range of values a Bayesian mix model considers likely for a channel's effect, such as an 80% interval for ROAS. A wide interval means the data hasn't pinned the effect down.
How do you make budget decisions with uncertain MMM results?
Compare each channel's range with its breakeven ROAS. Fund channels whose whole range clears it, cut those whose whole range falls short, and test the channels whose range straddles it.
How does calibration improve a mix model?
Calibration uses experiment results, such as holdout or geo-lift tests, to set the model's expectations for a channel. That narrows the channel's range in later model runs and anchors it to causal evidence.
Are randomized ad experiments precise?
Less than people expect. Across 25 large field experiments, Lewis and Rao found the median confidence interval on ad ROI was over 100 percentage points wide, because individual sales vary so much.
Sources
- Google, Challenges and Opportunities in Media Mix Modeling, David Chan and Mike Perry, 2017. Google sells advertising.
- The Quarterly Journal of Economics, The Unfavorable Economics of Measuring the Returns to Advertising, Randall Lewis and Justin M. Rao, 2015.
- Marketing Science, A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook, Brett Gordon, Florian Zettelmeyer, Neha Bhargava, and Dan Chapsky, 2019.
- INFORMS, How measurable is online advertising?, March 20, 2019.
- Google Meridian, ROI priors and calibration. Undated documentation. Google sells advertising.
Last updated October 29, 2026. Figures are quoted from the sources above and were checked on that date.