Measuring Advertising Experience Quality in Streaming:
A Longitudinal Research Framework

Why Advertising Experience Research Matters

Ad-supported streaming is no longer a niche product tier. Netflix's ad-supported plan reached approximately 94 million monthly active users in 2025, with U.S. members averaging around 41 hours of monthly viewing. By 2026, Netflix reported that its ads plan reached more than 250 million global monthly active viewers — confirming that this growth has already arrived, not merely projected. Note: Netflix's public advertising metrics shifted from "monthly active users" to "monthly active viewers" between these reporting periods; the latter reflects household-level viewership and should be interpreted as an indicator of scale rather than a directly comparable subscriber count. The rapid scale of this shift makes advertising experience quality a timely strategic research problem, not just a technical modeling exercise. How platforms balance monetization with member experience will be a defining competitive variable in the next phase of streaming growth.

As streaming platforms shift toward ad-supported models, understanding how advertising experience shapes member satisfaction and retention becomes a strategic research priority. Yet the core mechanism — how ad fatigue accumulates over time and at what point it begins to damage the viewing experience — remains poorly understood in a longitudinal context.

This demonstration builds a transparent, evidence-grounded research framework to answer three questions: (1) How do ad load and personalization independently affect member satisfaction over a 12-month period? (2) Does ad fatigue mediate the effect of ad load on churn intent? (3) Is the relationship between fatigue and satisfaction bidirectional, and if so, which direction dominates?

I generated a synthetic longitudinal panel (3,150 members × 12 months) using parameters calibrated to published research and industry benchmarks, then applied a full measurement and modeling pipeline including Confirmatory Factor Analysis, Generalized Estimating Equations, Weibull Survival Analysis, and Cross-Lagged Panel Modeling.

Research Framework & Design

Factorial Structure

The design crosses three levels of ad load (2.5, 4.5, 6.5 min/hr) with three levels of personalization (low, medium, high), yielding 9 experimental conditions with 350 simulated members each. A third factor, ad diversity (creative variety), was varied within conditions to assess its fatigue-mitigating effect.

Longitudinal Dynamics

Unlike cross-sectional designs, this framework models two time-dependent processes that make ad experience fundamentally different from a single-exposure study:

Structural Model

The theoretical backbone integrates three published frameworks:

How Parameters Were Calibrated

Every numerical value in the simulation has an explicit justification. Parameters were either (a) taken directly from published estimates, (b) derived by back-calculation from a defensible target outcome, or (c) stated explicitly as design assumptions. No key parameter was left undocumented; each was sourced, calibrated, or explicitly treated as a modeling assumption subject to sensitivity analysis.

Core Parameter Table

Theoretical Frameworks Integrated

Data Generation & Analysis Pipeline

All code is available below. The full pipeline runs in a single Python environment. Click any block to expand.

# Evidence-Informed Synthetic Longitudinal Data Generator
# Parameters calibrated from published literature — see Parameter Table above

from dataclasses import dataclass
import numpy as np

@dataclass
class EffectProfile:
    # All coefficients in 5-point scale absolute units (not standardized)
    # This ensures between-condition differences are visible in output
    name: str = "base"
    personalization_to_relevance: float = 1.20   # Google/Ipsos 2019: +40% tolerance
    diversity_fatigue_mitigation: float = 0.49   # Back-calc: 40% lifecycle extension
    adload_to_intrusiveness:      float = 0.28   # Li, Edwards & Lee (2002)
    adstock_to_fatigue_increment: float = 0.28   # Back-calc: SS fatigue ≈ 0.48 at 4.5min/hr
    relevance_to_satisfaction:    float = 0.40   # Ducoffe (1996); Kim & Han (2014)
    intrusiveness_to_satisfaction: float = -0.35
    fatigue_to_satisfaction:      float = -0.32
    trust_to_satisfaction:        float = 0.35
    continuance_to_churn_logodds: float = -0.80
    fatigue_to_churn_logodds:     float = 1.20   # Parks Associates 2024: ~2%/mo baseline

def simulate_one_scenario(cfg, ep):
    """
    Core longitudinal data generator.
    Time-varying dynamics:
      [1] Adstock: adstock_t = exposure_t + λ × adstock_{t-1}  [Koyck 1954]
      [2] Fatigue: AR(1) in bounded [0,1] state with variety mitigation
      [3] Latent constructs: generated in absolute 5-pt scale units
      [4] Behavioral outcomes: logistic models calibrated to benchmarks
    """
    fatigue_state = rng.uniform(0.02, 0.06, size=n)  # New subscriber: minimal prior fatigue
    adstock_state = np.zeros(n)

    for month in range(1, cfg.n_months + 1):
        # Adstock update (Koyck carryover)
        adstock_state = ad_exposure + cfg.adstock_lambda * adstock_state
        adstock_norm  = adstock_state / (adstock_state + 180.0)

        # Fatigue update: AR(1) with variety mitigation (Kronrod & Huber 2019)
        fat_increment = (ep.adstock_to_fatigue_increment * adstock_norm
                        * (1 - ep.diversity_fatigue_mitigation * diversity))
        fatigue_state = np.clip(cfg.fatigue_persistence * fatigue_state
                                + fat_increment + noise, 0, 1)

        # Satisfaction structural model (Ducoffe 1996)
        satisfaction_lat = (3.00
            + ep.relevance_to_satisfaction      * (relevance_lat    - 3.0)
            + ep.trust_to_satisfaction          * (trust_lat        - 3.0)
            + ep.intrusiveness_to_satisfaction  * (intrusiveness_lat - 2.5)
            + ep.fatigue_to_satisfaction        * (fatigue_lat      - 2.5)
            + re_satisfaction)

        # Churn intent: logistic, calibrated to ~2%/month (Parks Associates 2024)
        churn_logit = (-4.20
            + ep.continuance_to_churn_logodds * (continuance_lat - 3.0)
            + ep.fatigue_to_churn_logodds     * fatigue_state
            + 0.25 * price_sensitivity)

# ── CFA: Confirmatory Factor Analysis ──────────────────────────────
model_spec = """
Relevance     =~ relevance_1 + relevance_2 + relevance_3
Intrusiveness =~ intrusiveness_1 + intrusiveness_2 + intrusiveness_3
Trust         =~ trust_1 + trust_2 + trust_3
Fatigue       =~ fatigue_1 + fatigue_2 + fatigue_3
Satisfaction  =~ satisfaction_1 + satisfaction_2 + satisfaction_3
Continuance   =~ continuance_1 + continuance_2 + continuance_3
"""
model = semopy.Model(model_spec)
model.fit(df_month6[item_cols])
fit_stats = semopy.calc_stats(model)

# ── GEE: Generalized Estimating Equations ──────────────────────────
# GEE chosen over LMM because ad condition variables are constant
# within members (between-subject design), causing singularity in
# LMM's random effect estimation. GEE with exchangeable correlation
# structure provides valid inference for this design.
from statsmodels.genmod.generalized_estimating_equations import GEE
from statsmodels.genmod.families import Gaussian
from statsmodels.genmod.cov_struct import Exchangeable

gee_m2 = GEE.from_formula(
    "satisfaction_score ~ time_c + load_z + pers_z + fatigue_score",
    groups=df["member_id"], data=df,
    family=Gaussian(), cov_struct=Exchangeable()
).fit()

# ── Cross-Lagged Panel Model ─────────────────────────────────────── 
# Within-person demeaning removes between-subject confounds,
# isolating the lagged temporal relationship.
for col in ["fatigue_score", "satisfaction_score", "fat_lag1", "sat_lag1"]:
    m = df.groupby("member_id")[col].transform("mean")
    df[col + "_dv"] = df[col] - m

path_A = smf.ols("satisfaction_score_dv ~ fat_lag1_dv + sat_lag1_dv + time_c",
                 data=df).fit(cov_type="HC3")
path_B = smf.ols("fatigue_score_dv ~ sat_lag1_dv + fat_lag1_dv + time_c",
                 data=df).fit(cov_type="HC3")

# ── Survival Analysis ────────────────────────────────────────────── 
# Weibull AFT for covariate inference; Kaplan-Meier for visualization.
waf = WeibullAFTFitter(penalizer=0.01)
waf.fit(surv_df, duration_col="duration", event_col="event")

Full source code: ads_experience_synthetic_v2.py (simulation engine) · ads_analysis_pipeline.py (analysis pipeline) · Python 3.12 · statsmodels 0.14.6 · lifelines 0.30.3 · semopy 2.3.11

Confirmatory Factor Analysis

What Each Construct Measures

Model Fit Indices

χ²(120) = 110.67, p = .718. A non-significant χ² indicates good model-data fit. Note on perfect fit indices: CFI = 1.000 and RMSEA = .000 are mathematically inevitable when the data-generating process exactly mirrors the assumed factor structure — as it does here by construction. This is a known limitation of evaluating measurement models on synthetic data. In practice, real survey data from streaming members would introduce correlated residuals (adjacent items sharing method variance), measurement drift over time, and respondent heterogeneity. A realistic expectation for a well-designed real study would be CFI ≈ .95–.98 and RMSEA ≈ .03–.06. The Trust construct's weaker performance (AVE = .387) is the more meaningful finding here — it is not an artifact of the simulation, but reflects genuine multi-dimensionality in how members conceptualize ad credibility.

Factor Loadings & Convergent Validity

AVE = Average Variance Extracted (threshold ≥ .50); CR = Composite Reliability (threshold ≥ .70). Trust falls below both thresholds, indicating its three items do not converge sufficiently on a single factor. This likely reflects the multi-dimensional nature of ad trust (brand trust vs. data privacy trust vs. ad content credibility). A revised instrument would separate these facets.

Discriminant Validity (Fornell-Larcker)

Each diagonal cell = AVE. Off-diagonal cells = squared inter-factor correlation (r²). Discriminant validity holds when the diagonal exceeds all values in the same row and column.

The Satisfaction ↔ Continuance pair fails the criterion (r² = .830 exceeds both AVEs of .712 and .592). This is expected — in consumer research, overall satisfaction and continuance intention are conceptually very close constructs. Practically, they can be treated as a combined "loyalty" dimension. Trust also fails against Satisfaction (r² = .428 > AVE .387), consistent with its convergent validity weakness.

Longitudinal Analysis Results

6.1 — Satisfaction & Fatigue Trajectories

Across all conditions, satisfaction declines monotonically from month 1 to month 12 as ad fatigue accumulates. The effect is most pronounced under high ad load (6.5 min/hr), where average satisfaction drops from 2.84 to 2.20 — a 0.64-point decline on the 5-point scale.

6.2 — GEE Longitudinal Model

Generalized Estimating Equations were used to model repeated satisfaction measurements while accounting for within-member correlation (exchangeable working correlation structure). Three nested models were compared using QIC (Quasi-likelihood under the Independence model Criterion).

M2 Coefficient Table

*** p < .001. β values are unstandardized GEE estimates. Load and Personalization z-scored (SD=2 min/hr and SD=0.25 respectively) for comparability.

6.3 — Factorial Comparison at Month 12

Two-way ANOVA on month-12 satisfaction confirms large independent effects of both ad load (η² = .297) and personalization (η² = .083), with no significant interaction (η² = .000, p = .855). Ad load and personalization operate as additive, independent levers.

The range from worst condition (high load + low personalization: 1.91) to best condition (low load + high personalization: 3.56) spans 1.65 points on the 5-point scale — a substantial effect with meaningful real-world implications for member experience.

6.4 — Survival Analysis: Churn Intent

A Weibull AFT model regressed time-to-first-churn-intent on member-level average fatigue, satisfaction, ad load, personalization, and price sensitivity. Key finding: ad load does not directly predict when members develop churn intent. Fatigue and satisfaction do.

6.5 — Cross-Lagged Panel Model

To test the temporal directionality of the fatigue–satisfaction relationship, I applied a within-person demeaned cross-lagged model. This removes all stable between-person differences (including condition effects) and tests whether changes in one variable precede changes in the other.

Both paths are statistically significant, confirming a bidirectional relationship. However, Path A is approximately 3× stronger than Path B (|−0.098| vs |−0.034|). This indicates that fatigue is the dominant temporal antecedent of satisfaction, not the reverse. Members who feel more fatigued in month t−1 report meaningfully lower satisfaction in month t, even after controlling for their prior satisfaction level.

Summary of Principal Findings

Implications for Streaming Ad Experience Research

For Product & Ads Experience Teams

The mediation finding has a direct product implication: optimizing fatigue management is a more tractable target than ad load reduction, because it can be addressed through product levers (break placement, creative rotation frequency, personalization quality) that don't directly reduce ad inventory or revenue. A 40% reduction in fatigue accumulation through creative diversity — as estimated from OTT industry benchmarks — could meaningfully extend the time before members reach dissatisfaction thresholds.

For Measurement Design

Ad fatigue should be measured as a time-varying construct within longitudinal survey designs, not a single cross-sectional measure. The AR(1) dynamics observed here suggest that surveys administered at months 3–4 will underestimate eventual steady-state fatigue, while surveys administered after month 8 will better capture the plateau. Diary study designs or embedded in-app pulse surveys (1–2 items per session) would provide higher temporal resolution than traditional monthly surveys.

For Research Methodology

This demonstration illustrates the value of connecting attitudinal survey data (what members say) with behavioral signals (what they do: drop-off rates, episode completion, churn). The Weibull AFT finding — that attitudinal fatigue predicts churn timing more directly than ad load — would be impossible to detect from behavioral data alone, and invisible from survey data without longitudinal follow-up.

Limitations of This Demonstration

Researcher Profile & Purpose of This Demonstration