DIAG-9 · Diagnosis · Invariant: holds by construction, no expiry

Definition changes break comparability

Melvin Salas

The law

Conversion counts produced under different definitions (events counted, attribution, dedupe) are different measurands; any comparison spanning a definition change confounds measurement with demand unless the change is modelled, bridged, or shown null.

Scope

Discrete changes (action added/removed) and gradual ones (modeled-conversion rollouts) alike. A bridged series, both definitions measured over an overlap window, restores comparability; that is the repair, not a counterexample.

What this licenses you to do

Enumerate every definition, attribution, and dedupe change inside any analysis window; insert a changepoint, mask, or bridge at each.

Riibon's own analysis is held to these laws

The laws on this site are a public subset of the rulebook Riibon's analysis works to. Before a metric movement is called a market event, the order of checks they set has to rule out the measurement first.

Ask the Riibon app why a number in your account moved, and see which checks it ran first. Access starts with a short call: we look at your ads together, then run them for you or set the app up with you.

Share this pageLinkedIn