Peek at the ever-increasing cost of a Super Bowl television ad and you’d likely be unconvinced that many companies’ marketing budgets are shrinking. Nevertheless, it’s happening and although many consumer brands continue the predictable migration of funds into this annual, grid-iron advertising ritual, many are also investing in something much more esoteric (and, possibly, more effective): Marketing Mix Modeling.

An often-ambiguous mix of science and intuition, MMM is an analytical process businesses used to quantitatively identify the drivers of sales and the relative impact of each. At its heart, MMM is a model explaining how various marketing tactics impact business results. Their goal is to answer one, simple question: How can marketing increase sales? Some models consider revenue, profitability, or market share, but the best model the effect of marketing on sales. The models generally take the form of sales response functions or sales curves, and are modeled assuming diminishing marginal returns, as the picture below shows:

This concept reflects the idea that each dollar spent in marketing returns marginally less than the dollar before it. Occasionally models assume constant returns to scale, depicted in the next diagram. This is a dangerous assumption because it implies that each dollar spent returns the same as the dollar before it and could lead a marketer to unwisely over-invest in a program no longer generating returns.

One particularly useful feature of Marketing Mix models is isolation of baseline from incremental growth. Marketers can distinguish and estimate the inherent inertia of mature products. The realization of their usefulness has resulted in smarter, better-informed marketing investments in many organizations, but the opposite has also occurred.

There are myriad ways in which MMM can be misused or mis-informed. Among them, three are particularly problematic: Garbage In/Garbage Out (GIGO), Collinearity & Omitted Variable Bias.

Garbage In, Garbage Out

Like all pursuits, MMM’s usefulness is only as good as its input. Put bad data in and bad data will come out. Three examples of GIGO are particularly egregious and are the most common culprits of invalid modeling:

• Time Dimension Mismatch – A marketing mix model which accurately gauges the relationship between sales and marketing inputs requires sales and marketing data collected at similar frequencies. While the granularity and frequency of data collection has increased, and will continue to in the future, it’s still common to track marketing inputs less frequently than sales. This mismatch may result in omission of important elements or bad assumptions about correlation.
• Inaccurate Data – Sometimes data is simply bad. Organizations frequently depend on 3rd parties or agency relationships for data. Often, data is inaccurately reported and, in some of the worst cases, faked. Without reliable and accurate data, MMM fails.
• Poor Unit Consistency – MMM requires complicated unit conversions and or comparisons. If the tracked units are incompatible, the effectiveness of the MMM can be compromised. In broadcast media, for example, a marketing investment in dollars will be counted or tracked as a Gross Rating Point (GRP). These are not equivalent to Impressions, which appear in MMM for other channel types, particularly digital.

Collinearity

It’s not uncommon for marketers to simultaneously adjust the timing and amount of their marketing spend in multiple channels. A marketer, for instance, may typically decide to adjust TV and radio spending at the same time and by the same amount. This collinearity makes it difficult for MMM to determine whether TV or radio spending was more impactful on sales.

To deal with collinearity, marketers change the model specification – they reduce the number of variables. In the example above, the marketer may choose either TV or radio spending as the driver in the model or they may combine them as a “broadcast media” variable. The key is to ensure that collinearity is properly measured and reported, and that the model specification accounts for it.

Omitted Variable Bias

While having too many, highly-correlated variables can result in collinearity, omitting variables can lead to other, invalid conclusions. When key sales drivers are omitted from the model, their influence may be incorrectly attributed. A price change may increase or decrease sales substantially. Left from the model, however, its impact might be wrongly attributed to a simultaneous change that occurred in a company’s in digital spending, leading to invalid conclusions about digital ROI and, potentially, misguided future investments. Unfortunately, choosing the right sales drivers to include in the model requires more business knowledge and intuition than technical or analytical prowess. Managers should carefully list all the possible impacts to sales, objectively quantify those impacts in a set of drivers and test their validity in the marketing mix model.

Marketing mix is a process of finding the simplest, most likely explanation for a set of results, but finding it is not a guarantee of repeatability. False conclusions occur, and so business and marketing intuition are critical in the success of marketing mix projects. The keys are awareness of the shortcomings of the data, and mindfulness that MMM is best used as framework for aligning organizations around marketing planning and performance.

— Amanda Hand & Lloyd Lay