Cascade Team
+3 Contributors
Analysis
Last Update: Today

Product features that drive delight and detractors

We combine NPS surveys and product engagement data to determine which features in our product deliver happy customers, and which features deliver not-so-happy ones. Pulls in product engagement data from Amplitude, Mixpanel or others, and relies on major feature flags to bucket engagement into features. Then takes NPS survey data from Delighted or other sources to compare against engagement data to detect patterns among promoters and detractors.

Assumptions

For this model, we'll just be looking at dates between 8/1/19 and the end of that year, 12/31/19. We'll be pulling in data from Mixpanel and Redshift for this.

We also make some basic assumptions about the amount of credit we give to each campaign that helped convert a user:

Product
1

If a campaign is the first to see any given customer, that campaign get most of the credit for that customer

60%

Product
2

all the campaigns that are neither the first or the last to see a customer before they convert are given pt. That number is divided across the campaigns in that group.

10%

Product
3

the campaign that is the last to touch a customer gets the remainder.

20%

Results

For this date range, we acquired 17,829 customers via 4 campaigns. Each customer had an average of 3 touches before they converted. If we weight each customer by the assumptions above, we can compare the efficacy of each campaign by comparing the amount spent on each campaign and the weighted LTV of each customer converted by that campaign:

Revenue by Sentiment Quartile

Quartile
1
$98,399
$398,898
4.1x
Revenue increase
Quartile
2
$121,457
$965,452
8.0x
Revenue increase
Quartile
3
$151,424
$254,212
1.7x
Revenue increase
Quartile
4
$172,272
$754,214
4.4x
Revenue increase

Details

1. Multi-touch attribution reveals very different results than last-touch-only

Based on Google Analytics UTM data, we can see where in the conversion path each campaign saw each user. Weighting each touch using our assumptions, we see that a multi-touch conversion system reveals very different results than a last touch-only system:

Weighted Number of Customers Converted

Yahoo Campaign - Jan 2019
4,284
9,521
Retargeter - Jan 2019
6,189
2,856
Google - Competitive Keywords
3,808
5,237
Facebook Campaign - Jan 2019
3,332
2,380

Original - Last Touch Only

Multi-touch weighted

2. For a fuller picture, we incorporate campaign spend to get weighted cost per customer

A comparison of customers converted is great, but what we really want to know is our cost per customer, based on how much we spent on each campaign. To get there, we need to import our campaign spend data from Quickbooks.

We then easily get a comparison of cost per customer for each campaign:

Weighted Cost Per Customer

Yahoo Campaign - Jan 2019
$23
$10
Retargeter - Jan 2019
$20
$43
Google - Competitive Keywords
$40
$29
Facebook Campaign - Jan 2019
$52
$72

CAC - Last Touch Only

CAC - Multi-touch weighted

A comparison of CAC quickly reveals that while Retargeter seems to perform well on a multi-touch basis, the efficacy of spend is much lower.

3. Finally, we incorporate projected customer LTV for a full comparison of the efficacy of each campaign

For each customer, we have a projected LTV that is calculated based on demographics and prior behavior. Weighting that LTV by our multi-touch assumptions, we get to a weighted LTV of customers acquired from each campaign.

LTV by Marketing Campaign
4
Records
3
Columns
2
Data Types
Campaign
Channel
LTV of Customers Acquired
1
Display
Yahoo Campaign - Jan 2019
$398,898
2
Retargeting
Retargeter
$965,452
3
SEM
Google - Competitive Keywords
$254,212
4
Social
Facebook Campaign - Jan 2019
$754,214

That allows us to compare fully-weighted CAC to the LTC of customers acquired from that campaign, as shown above.

And that's it! Feel free to change the dates or weights of the workbook to view updated results.

Welcome to the Cascade Designer

Present Mode

01

of

03

Cascade's Present Mode is where the story of your analysis gets told. Live inputs and outputs can be dynamically inserted into your narrative, and data can be visualized in any way you want. Rerun the workbook to update assumptions or pull the latest data.