Skip to main content

Confidence Scores

Overview ​

We evaluate our confidence in a panel dataset across four dimensions​

Absolute Volume​

  • Measures how much volume of an entity’s spend or engagement is captured by the panel as compared to a ground truth data source. For example, how much of a brand’s reported financial revenue is represented by its total receipt spend in our transaction panel? Does this volume amount change across brands, time or location?

Relative Trend - Correlation​

  • Measures the correlation of an entity’s change in spend or engagement across space and time, as compared to a ground truth data source. A high score indicates strong correlation with ground truth and suggests higher confidence in the panel data, regardless of its volume coverage.

Relative Trend - Predictability​

  • Measures how well the panel metric value can predict the reference metric value, using historical data. It is estimated by regressing the reference volume change on panel volume change over time, and compute the mean absolute percent error (MAPE).

Relative Trend - Directional Correctness​

  • Measures how much of second derivative of the change in volume (growth rate) of the panel metric value time series is moving in the same direction of the reference data’s second derivative.

  • Panel Variance: Assesses the stability of the dataset over time by analyzing the variance in volume percentages. In other words, how much is the panel’s coverage of an entity and/or location varying across time? Low variance suggests more stability and increased confidence. High variance suggests significant fluctuations and indicates poor fit.