TV analysis
TV attribution is one of the hardest problems to get right in all web analytics. To be clear, attribution is how marketers make sense out of their spend, so they know where to invest more or less in the future. Since attribution, for most companies, is composed of various channels, there are many competing variables in the attribution model. Let's limit today's discussion to TV only.
TV attribution comes with a host of complexities, including:
- Advertising on multiple channels adds a lot of noise, bias and correlated variables to your data
- Seasonality (daily, weekly, ...) of your user base will impact your analysis
- Your user base will naturally be growing (hopefully!) so you need to adjust for that somehow too
Many companies opt for running a survey because the intuition is that people will just tell you how they heard of you. Running a survey, however, has its own host of issues, like users:
- Not remembering
- Mis-remembering
- Not filling out your survey!
In addition to those issues, there's the whole TV-networks-are completely-ass-backwards-when-it-comes-to-data-transparency-and-customer-segmentation thing. This thing may only exist in my head but I suspect other digital analysts agree with me. However, let's not digress..
The biggest complexity, from an analytics perspective, is this: when a commercial airs, people generally aren't clamoring for their phones or computers to log on to your site. So other than the small, immediate lift you'll notice from a spot (if you're lucky), how can you tell what the impact is? Furthermore, how do you know it was TV specifically that caused them to go to your site?
The short answer is: you can't.
As data professionals, we're left guessing. Much of what we do is make inferences based on the data provided and try to come up with causal relationships. This is very difficult to do accurately, but that's the majority of the work done in data analysis. Don't let anyone tell you otherwise.
However, there are some easy wins in this game, so let's see what we can do. Let's quantify TV's impact at a high level.
Since your business is web-based, we can proxy "impact" with total landings on your site.
Can you guess when TV was turned on? Of course you can - it's not data rocket science.
Next, let's project forward what our landings would have been, had we not advertised on TV. Now of course this part comes with tons of assumptions, but that's part of the deal.
I'm going to project forward the landing activity prior to TV using an ARIMA model with a seasonality and trend component. The seasonality, in this case, is by hour of day and day of week. For example, every Monday at 9 pm is assumed to repeat itself. You can tell this from visually inspecting the chart. And it makes intuitive sense for subscription based B2C businesses. So the seasonality component is 24 hours * 7 days = 168 hours. And there is a slightly upward trend component, which is based on the growth of the underlying business. Call it 0.8% per week. There are more complex things I could have done, but let's keep it simple for now.
Here is the projection (expected) and the actual landings.
Projected
Now comes the intuitive part: how would you find the extra landings gained from TV, given the data you've just seen? If you said you'd use an obscure methodology called subtraction, you'd be right.
Projected Difference
The difference is now clearly obvious.
Finally, to get your high level estimate of the impact of extra landings gained from advertising on TV, you employ a neat mathematical technique.
It's called addition.
Sometimes, it's best not to overthink it.
Enjoy!
