How do you generate probabilistic attribution scores, and how do you ensure accuracy?
We start with raw event data: the impressions (ads we’ve served) and the conversions (such as events passed to us via pixels on the marketer’s website or events passed to us via a marketer’s conversion API). If there is no deterministic link between the two events, we feed these events plus any related data into our model, which tells us the probability that the impression was for the same person who converted.
We train our models using probabilistic events combined with deterministic data (which more accurately determine which events are for the same user). We can train our models on this subset of data and then score them on the larger complete set of events. In this way, our models expand our knowledge of the small set of deterministic events into probabilistic scores on the larger set of events.
To ensure accuracy, we measure our performance in various ways, particularly using holdout data from the training set and comparing our results against third-party attribution solutions. Our customers have run many cookieless campaigns with us, and those with their own measurement solutions report that our measurement tracks with their own.
Do you have thresholds to ensure you only attribute conversions above a certain confidence level?
From running many tests, we have seen that using all the conversion scores in aggregate gives the most accurate results (as opposed to eliminating the information in some lower-scoring conversions). We retain all conversion scores to inform our attributions. While some individual conversions may have lower likelihood scores, aggregated totals are highly predictive; evaluating the total sum of your conversion probabilities across cookieless conversions allows us to achieve a high level of confidence overall. Keep in mind that a low-scoring conversion will only contribute a small amount to the total, i.e., an attributed conversion with 50% of being correctly attributed will contribute only 0.5 of a conversion to the total.
In many cases, our attribution gives a conversion likelihood score of 0 (a frequent example is for impressions and conversions in different countries). We will not count any attribution signal with a score of 0.
How do you validate that attribution results are correct/accurate?
When we train our cookieless identity models, we test them against a known data set. We can validate this using deterministic IDs, such as those based on emails. In addition, while third-party cookies exist, we can also test our measurement against what third-party cookies report on some media.
Can you provide a match rate between users visiting a client website or digital destination and being visible to Quantcast across the open web?
This is simply the ratio of cookieless/everything for the internet at large.
Over half the internet is now cookieless. Even though Chrome is rolling out very slowly, we already see plenty of Chrome traffic without third-party cookies due to users’ individual choices. For a particular client, the opportunity will vary a bit depending on the type of client: for example, a client selling Apple software will have a much higher percentage of cookieless users than someone targeting Windows users.
Do you report on individual conversions?
We do not report on individual conversions; we report the aggregate counts in Report Builder.
We take into account the probability/score for each conversion when reporting. For example, if we say that we think there were five possible conversions with probabilities: 0.3, 0.5, 0.6, 0.8, and 0.9, then we would report three conversions (3 ~= 3.1 = 0.3+0.5+0.6+0.8+0.9)
Why does Quantcast not use thresholds ( i.e., only counting attribution above a certain percentage of confidence?)
We believe a system that does not rely on thresholds is more accurate. Each time you apply a threshold (e.g., 75% likelihood), you remove data that informs modeling. The more you remove, the more data accumulates that you cannot use. When you model and use thresholds, your results are not as good as when you model and use all of the resulting data.
An example best explains this. If you are targeting females, and models result in three outcomes—a score of 74% likelihood that your target is female, a score of 76%, and a score of 95%—thresholding above 75% would mean that the score of 74% would get a no-bid, and you would bid the same for 76% likelihood and 95%. Without thresholding, you would bid highest for 95%, lower for 76%, and still lower for 74%.