PyMC Bayesian (Shapley) — RECOMMENDED
Trust: 8.5/10
Strengths: Principled Bayesian inference with full uncertainty quantification. Shapley attribution handles channel interactions correctly. Geometric adstock + Hill saturation are well-calibrated. R-sq of 0.852 with 12.3% MAPE is strong. 26% base suggests meaningful media attribution without over-claiming.
Weaknesses: Shapley amplifies small-spend channels (ga_nonbrand 14.5x, other 15.5x). May over-attribute to channels that only correlate with high-revenue weeks. Two-year time window limits what posteriors can learn.
Best for: Overall budget allocation decisions, channel-level ROAS estimates, understanding diminishing returns via response curves.
Robyn Ridge (2_674_3) Optimized — STRONG SECONDARY MODEL
Trust: 7.5/10
Strengths: Massively improved after optimization: NRMSE train 0.079 (R-sq ~0.994), test 0.151 (R-sq ~0.977). All 12 channels now active (fb_intent was previously zeroed). Weibull PDF adstock captures delayed-peak effects for awareness channels. Penalty factor regularization handles collinearity. 35K iterations with tighter hyperparameter ranges produced stable, converged results. DECOMP.RSSD of 0.093 shows good spend/effect balance.
Weaknesses: Still point estimates only (no uncertainty quantification). Nevergrad optimization is black-box. Conservative ROAS estimates (0.76x-3.23x range) may understate some channels. 66% base is higher than PyMC's 26%.
Best for: Strong cross-validation of PyMC results. Conservative ROAS lower bounds. Awareness-channel timing via Weibull adstock. Budget allocation as a secondary reference.
Google Meridian (Bayesian) — STRONG TERTIARY MODEL
Trust: 6.5/10
Strengths: Google’s official open-source MMM framework with principled Bayesian inference (TFP MCMC). Full posterior uncertainty quantification with credible intervals per channel. Geometric adstock + Hill saturation properly model carryover and diminishing returns. All 12 channels positive. ROI priors encode domain knowledge. R²=0.784 with 15.2% MAPE is reasonable.
Weaknesses: Low base % (9.1%) suggests over-attribution to media channels. Very wide credible intervals (e.g., ga_brand 0.38x–35.2x) due to only 68 weeks of data. fb_adv gets highest revenue ($18M, 28%) which may be inflated. Only 2 chains × 300 samples — more MCMC may tighten estimates. Median ROI often much lower than mean, indicating right-skewed posteriors.
Best for: Cross-validating PyMC channel rankings with an independent Bayesian framework. Response curve analysis. Posterior uncertainty exploration. Budget allocation as a secondary reference alongside PyMC.
Orbit DLT (Bayesian STS) — STRONG TERTIARY MODEL
Trust: 7.0/10
Strengths: Principled Bayesian structural time series with damped local trend. R-sq of 0.862 with ~12% MAPE. All 12 channels positive with 100% confidence. 29.7% base is reasonable and close to PyMC’s 26.3%. Full posterior uncertainty quantification. ROAS values (0.70x–12.0x) align well with PyMC and Robyn estimates. Pre-computed geometric adstock captures carryover effects.
Weaknesses: No saturation (Hill) modeling—assumes linear response which can overstate high-spend channels. Coefficients have relatively high standard deviations. Linear model cannot capture diminishing returns. Cannot be used for marginal ROAS or response curve analysis.
Best for: Cross-validating PyMC and Robyn channel rankings. Confirming ROAS direction with full Bayesian uncertainty. Adstock sensitivity analysis. Strong third opinion for budget allocation.
Practical Recommendations
High-Confidence Findings (All 4 Models Agree)
1. fb_adv is a strong performer — PyMC (1.94x), Robyn (1.04x), Orbit (3.05x), Meridian (7.31x) all positive. Meridian’s higher estimate reflects its lower base allocation.
2. ga_shoppmax is a top performer — Positive ROAS in PyMC (4.80x), Robyn (1.56x), Orbit (3.10x), Meridian (4.33x). Strong 4-model consensus.
3. fb_awareness has low marginal returns — Despite being the largest FB spend ($4.3M), all models show lowest FB-channel ROAS: PyMC (0.76x), Robyn (1.26x), Orbit (0.70x), Meridian (1.74x).
4. ga_nonbrand has high ROI at current spend — PyMC (14.5x), Robyn (3.23x), Orbit (10.14x). Meridian (1.94x) is lower but still positive. Small spend ($353K) with outsized returns.
5. All 12 channels positive across all 4 models — Unlike the previous XGBoost model, Meridian agrees with all other models on positive ROAS for every channel.
Areas of Uncertainty (Magnitude Spread)
1. Meridian base % is low (9.1%) — Compared to PyMC (26%), Orbit (30%), and Robyn (66%), Meridian attributes more revenue to media. This may inflate individual channel ROAS. Use PyMC/Orbit base estimates as more reliable.
2. Absolute ROAS magnitudes vary — Meridian (7.31x fb_adv) is 3.8x higher than PyMC (1.94x). Wide credible intervals (0.68x–14.5x) overlap. Use Robyn's conservative estimates as a floor.
3. TikTok ROAS spread — PyMC (4.52x), Meridian (3.74x), Orbit (3.39x), Robyn (0.76x). Direction is clearly positive, but magnitude ranges from 0.76x to 4.52x.
4. Influencer cost uncertain — PyMC (2.82x), Meridian (2.75x), Orbit (2.34x), Robyn (0.84x). Direction positive, but Robyn suggests near-breakeven. Consider testing spend levels.
Bottom Line
Use PyMC Shapley as the primary decision model for budget allocation, with Robyn as a conservative secondary reference, Orbit DLT as a strong Bayesian cross-check, and Meridian as a response-curve reference. All four models agree on all 12 channels being active with positive ROAS, producing directionally consistent rankings — a significant improvement over the previous XGBoost model which had negative ROAS on 2 channels. PyMC provides Shapley attribution with uncertainty; Robyn provides conservative lower-bound estimates; Orbit confirms via structural time series (R²=0.862, 29.7% base); Meridian adds Google’s Bayesian framework with Hill saturation curves (R²=0.784, though base % of 9.1% warrants caution). For channels where Robyn is substantially lower than other models (tiktok, influencer), consider geo-lift experiments to establish ground truth.