To download this article as a print-friendly pdf, click here and submit your email address.

A brief introduction to econometric modelling

Part 1: Econometrics in theory
Part 2: Econometrics in practice
Part 3: Typical outputs from econometrics
Part 4: Conclusion – is econometrics for us?

Part 1: Econometrics in theory

What is econometric modelling?

A mathematical technique for quantifying the relationships within a set of variables, some of which are thought to ‘cause’ the others. For anyone interested, the mathematical technique used for the vast majority of econometric modelling is known (for technical reasons) as ‘Ordinary Least Squares’, or ‘OLS’. This is combined with inferential statistics to draw conclusions about the relationships we measure.

Where did econometrics come from?

It was developed in recent years for the social sciences, in particular economics, where it has had its widest application. In the early days of its use (before desktop computers) it was used to measure very broad macroeconomic relationships, for example between national income and national consumption levels. As theory and computers evolved, academic applications have become ever more complex, subtle and detailed. By and large, OLS is sufficient for most commercial applications, where the usefulness of more sophisticated techniques tends to be undermined by data availability and the operating environment.

How is econometrics related to marketing mix modelling?

Market mix modelling (MMM) is econometric modelling applied to the task of quantifying the drivers of sales. The various elements of the marketing mix (price, promotions, advertising and so on) are the causal variables, which are believed to determine sales. By correlating changes in sales with changes in each of the causal variables over time, MMM establishes the exact contribution of each element of the marketing mix to sales.

What’s the USP of econometric, or market mix, modelling?

Econometrics is the only technique capable of disentangling the effects of simultaneous drivers of sales. Unfortunately we do not operate in a laboratory, where we can conduct controlled experiments, operating one lever (e.g. price, promotions) at a time. What we do have is a series of historical data points of sales, and a numerical record of the marketing activity that took place over the same period.

Why do marketing mix modelling?

You spend a lot of money on your sales and marketing activities, and it has serious implications for sales and ultimately financial performance. The ability to accurately and robustly assign a value to each activity in your marketing mix is immensely valuable. Put crudely, we can do more of the activities which offer the highest return on investment (ROI), and less of the activities with low ROIs. This process is known as optimising the marketing mix and is a way to generate significant extra sales and profit from the same marketing spend.

What’s all this Bayesian business?

It’s rather hard to explain without unpacking the mathematics, but Bayesian statistics is another school of thought which has methodological differences from the more traditional, ‘frequentist’ approach. Bayesian statistics has gained a following in academic circles in recent years, and it is beginning to spill over into commercial applications. For reasons we go into elsewhere, we are sceptical about the practical benefits of Bayesian statistics when used in a MMM context. Watch out for our forthcoming discussion paper on Bayesian statistics for more details (click here to be notified when it’s published).

Part 2: Econometrics in practice

What data do you need to do econometric (market mix) modelling?

Ideally, 3 years of weekly data for all variables – sales, promotions, advertising, distribution and so on. If that sounds daunting, in practice most of this data is contained within the standard EPOS data deliveries supplied by data providers. Advertising data is the main exception, and media agencies have become versed in providing this to their clients usually free of charge, on demand. Whilst 3 years of data is ideal, we have built meaningful models with much less data…2 years’ worth, 1 year, or even less. At the end of the day, the point at which you have ‘enough’ data is determined by the questions you want to answer and the complexity of your circumstances.

What do econometricians do?

There’s an art as well as a science to model-building. The statistics can take you so far towards a model, but the best – accurate, robust, reliable – models are built by humans who make decisions, guided by the statistics, on what to include and what to exclude from their models. Clearly the more experienced, the more skilled, the more talented those humans are, the better your models will be.

What is the model-building process?

Generally there are 3 phases to any model-building process.

i. Data collection and analysis

This is the foundation of any econometric model. There is a famous dictum in modelling, “Garbage in, garbage out,” and the econometrician building the model should be involved in gathering the data, to ensure they have a thorough acquaintance with, and understanding of, it. The better an econometrician is, the more time they will spend ‘staring’ at your data before starting to build the model.

ii. Model building

Model building commences when an econometrician, happy with the data he or she is using, starts the process of ‘testing’ variables in the model. This is an iterative process involving many twists and turns as variables are tried and either accepted, rejected or refined. All the time this is going on, the econometrician is developing an understanding in their heads of ‘how your brand works’, which drivers influence sales, which are irrelevant, whether there are period of sales activity that are not explained by any of the causal variables they have. This understanding is crucial to giving you confidence in the insights and recommendations they develop.

iii. Interpretation and recommendation

Once a final model has been arrived at, it remains to distil the findings from the model into insights, and to turn insights into actionable recommendations. Many econometricians, more comfortable with numbers than with business, falter at this point, and rush towards the end of the project. But this phase is as important as either of the preceding ones. Only when a client understands what has happened (metrics), why (insight) and what to do about it (recommendations) is the process completed.

Is there a quicker way to build econometric models?

Yes, but it’s not very good. There are plenty of automated modelling systems out there which use pre-determined statistical rules to decide the variables that find their way into your models. Some newer variants use Bayesian approaches, and genetic algorithms, but all have in common replacement of human decisions with a set of rules. Clients are sometimes tempted towards these solutions because of the apparent cost-savings they offer, but there are some major pitfalls they encounter sooner or later. Watch out for our forthcoming briefing paper on automated modelling. Click here to be notified.

Part 3: Typical outputs from econometrics

The best econometrics projects tend to be bespoke affairs, where an experienced consultant uses model-generated insights to tell a story about your brand and leaves you with a clear and pressing sense of what needs to be done. That said there are a few standard charts that you are likely to encounter at some point during the course of an econometrics project. Here’s a quick run-through.

Part 4: Conclusion – is econometrics for us?

What does econometrics do well?

Econometric modelling is excellent for identifying and quantifying the following:

• Large short term effects, such as promotions, seasonality, distributional changes or out-of-stock situations, new product launches and SKU interactions.
• The effect of RSP changes, both in absolute and relative terms.
• Short to medium term media effects, specifically the effects around 1 to 3 months after deployment.

What doesn’t it do well?

Measure things you haven’t done

Fundamentally econometric modelling is backward-looking. In general that’s not a problem as the immediate future tends to work in similar ways to the immediate past. However, it does require a degree of nuancing and sophistication to use the outputs from an econometric model to make valid inferences in fluid situations, such as deploying a new medium, or brand new marketing activity.

Insight and interpretation

Essentially the outputs of econometrics are numbers. This is where the quality of the consultant plays a large part, in taking these numbers and stitching them together to tell a story, create insight and generate recommendations for your business.

Long-term, brand-building effects of media

The timescale over which brands are built – years, if not decades – means that even the heaviest deployments of the most impactful media will cease to have attributable effects on sales. We can look back at successful brands and instinctively know that the excellent media they consistently deployed must have played a part in their success, but econometrics is going to struggle to prove and quantify that link.

I’m thinking of doing an econometrics project. How should I go about it?

First and foremost, identify exactly the questions that you need help answering. Secondly, select a specialist modelling consultancy, such as Piquant, who can help you work through your questions and shape an appropriate solution for you. Thirdly, finalise the brief and agree scope, timings and costs.

Is Piquant any different from other econometrics providers?

Yes. There’s a lot of handle-turning in our industry, meaning it’s relatively easy to pick up something purporting to be an econometric model, and for less than you might think. But ultimately these will disappoint you with their lack of insight, and, in the worst cases, drawing inferences that are misleading and incorrect. Who wants the wrong advice, even at half the price? Piquant are different because we focus on your business issues, and tailor our approach accordingly. If your situation demands a model, we’ll provide one, and the best one you could buy. If not, we will find other innovative analytical ways of helping you find the solutions you’re looking for.

Uniquely among suppliers, we also place an equal emphasis on the people and process part of the equation, so that you have the ability to absorb and use the insights we create. Fundamentally we’re experienced business consultants who know numbers, rather than analysts and modellers who struggle to apply insights meaningfully in a business context.