16 December 2015

Better Attribution: Using Clickstream Data and Shapley Analysis to Get More Accurate CPA & ROAS



Attribution

Attribution is the process of identifying a set of actions/events/factors that contribute to a desired outcome (a.k.a. a conversion).

The purpose of all advertising and marketing is to drive consumers towards performing some desired business or organizational goal (sign ups, views of a certain page, donations, purchases, etc.).

Your goal as a business is to maximize the return you get from your marketing spend.

Last Click Attribution is a Problem

The problem with most (or all) ad placement and bidding platforms is that they use simple attribution, generally attributing the "win" to the last source in the user acquisition path in the measure window.  This distorts the true value of the channels and can lead to bad budget allocation decisions.

For example, consider users who convert via a PPC then Display then Affiliate path (shown below). Generally, the last source is considered the one that led to the conversion and is given credit for the "win".  PPC and Display contributed, however, and the traditional last-click attribution that  your tools and marketing partners are performing does NOT assign the correct value to these channels.  How should the contributions of these channels be valued?

The analysis you are getting from your tools and from your marketing partners assigns the following last-click attribution values.

Wins%CostCPARevenueROAS
PPC79528.96%$19,875$25.00$7,950,000$399.00
Display65723.93%$36,135$55.00$6,570,000$180.82
Affiliate1,29347.10%$116,370$90.00$12,930,000$110.11
Total2,745100.00%$172,380$62.80$27,450,000$158.24

Unfortunately, the Cost Per Action (CPA) and Return on Advertising Spend (ROAS) values do not account for the contributions of the channels before the last click -- this is a BIG problem. Based on CPA and ROAS, one would be tempted to dump Affiliate marketing.  This is NOT the correct call.

Advanced Attribution

If you analyze the clickstream data for the above results, you see the following chains (note: for this article, we're showing simple chains where users do not consume media in more complicated ways, e.g. PPC > PPC > Display > Organic > Remarketing > Win):

Session 1Session 2Session 3Channel 1Channel 2Channel 3WinsLast Click
PPC386386PPC
PPCDisplay910333333Display
PPCDisplayAffiliate973388Affiliate
PPCAffiliate887723723Affiliate
PPCAffiliateDisplay8101598383Display
Display156156Display
DisplayPPC4966PPC
DisplayPPCAffiliate385373134134Affiliate
DisplayAffiliate222Affiliate
DisplayAffiliatePPC145135124124PPC
Affiliate426426Affiliate
AffiliatePPC414273273PPC
AffiliatePPCDisplay4433258484Display
AffiliateDisplay411Display
AffiliateDisplayPPC976866PPC

With clickstream data, you are able to build a table like the following where the wins after each stage are identified.

Session 1Session 2Session 3Wins @ 1Wins @ 2Wins @ 3
PPCDisplayAffiliate386719727
PPCAffiliateDisplay38611091192
DisplayPPCAffiliate156162296
DisplayAffiliatePPC156158282
AffiliatePPCDisplay426699783
AffiliateDisplayPPC426427433

Shapley Value

With this in hand, you can quickly figure out the marginal contribution (MC) for each component of the acquisition path using the approach suggested by Shapley Game Theory.  (The value in each MC cell is the value of the cell to the left in which the channel contributed to the wins minus the value in the cell before it (e.g. row 3 PPC, 162-156=6).

MC PPCMC DisplayMC Affiliate
PPCDisplayAffiliate3867197273863338
PPCAffiliateDisplay3861109119238683723
DisplayPPCAffiliate1561622966156134
DisplayAffiliatePPC1561582821241562
AffiliatePPCDisplay42669978327384426
AffiliateDisplayPPC42642743361426

To obtain the Shapley Value for each of the three channels, simply sum each channel column and divide by the number of paths (in this example, there are 6 paths).

You then figure out what percentage of the total Shapley value each of the three components represents and multiply the total wins by this percentage to come up with the effective wins for each channel.

The Results

As you will notice below, the value of PPC in this advertising mix increases and the value of the other two channels decreases.

PPCDisplayAffiliate
Path 13863338
Path 238683723
Path 36156134
Path 41241562
Path 527384426
Path 661426
Shapley Value196.83135.50286.50
Relative Value31.81%21.90%46.30%
Adjusted Wins8736011,271
± Wins+78-56-22
Adjusted CPA$22.76$60.12$91.57
± CPA-$2.24+$5.12+$1.57
ROAS$438.30$165.33$108.21
± ROAS+$39.30-$15.48-$1.90

Guidance

With these results in hand, it would be wise to increase the funds allocated to PPC. Although Affiliate marketing performs the worst with respect to CPA and ROAS, it plays a role in 46.3% of wins. Display contributes the least and is the candidate for reduction if you need to maintain the same total marketing spend.

Collecting Clickstream Data

If you are a current client, Stand Sure is already collecting your clickstream data for free as part of our engagement. If you are not currently a client and need help collecting clickstream data, please let us know. Clickstream data collection costs scale with website activity and retention but are reasonable ($100s/mo in most cases).

Getting Help with the Crunching & Analysis


Although the Shapley analysis is straightforward once you have the table of values (i.e. you don't need us), it is often a time-consuming and error-prone process for many to crunch the clickstream data to these results. If you need our help, please let us know.

No comments: