A revolution is coming, and it’s called quantitative marketing

In case you haven’t noticed, a revolution is underway for us marketers.

It is the same quant revolution that, like a perfect storm, forever changed many other industries such as professional sports and investment.

In the investment business, data has always been very important. Especially for value investors, it is critical to access as much data as possible for an investment target and then try to determine whether it is over- or under-valued or just right. Value investing as a strategy involves a fundamental judgement: is a stock trading at a significant discount against its intrinsic value?

The quant investing revolution started in the late 1970s and early 1980s, but major breakthroughs were not seen until the1990s. In contrast to value investing, quant investment does not bother with fundamental judgement on intrinsic value, but instead tries to exploit market inefficiencies or to act on predictions on short-term trading movement.

Perhaps not surprisingly, marketing has been following a rather similar trajectory, yet is still some twenty years behind quant investing. How has the quant revolution altered our industry? What are the advantages and limitations this revolution has brought? Where is the revolution headed towards?

To answer the above questions, it is necessary to divide the quantitative revolution in marketing into three stages, each corresponding to the past, the present and the future (see Table).

Stage 1: Pre-revolution—the era of market research

The invention of market research in the 1910s allowed companies for the first time to make informed marketing decisions and to a limited extend, improve consumer centricity. Instead of selling what one produces, brands sell what consumers want. This simple common sense was a major breakthrough in thinking in the age of ‘any customer can have a Ford in any colour they want, as long as it’s black’.

While informed decisions were on the rise, the role of data was to serve as either the knowledge base or validation for human decisions. Human intelligence, in all of its glory, also comes with many flaws. Often, market research data was cherry-picked or entirely brushed aside when it contradicts with a decision maker’s thinking or reduced to a rubber stamp to ‘validate’ a desired decision.

Furthermore, two major limitations in market research data cast long shadows on how much impact data can exert: data scope and granularity. Most surveys nowadays last some 20 minutes and on hundreds of samples. The limited amount of data that can be collected means narrow and shallow data input; and the fact that findings from small samples cannot be reliably extrapolated out to the broader population confines any conclusions to PPT slides while attempts to execute them tend to be ‘lost in translation’.

Hence there has never been a shortage of jokes on market research.

Stage 2: The dawn of quant revolution—efficiency exploitation

Clearly, given the limitations of market research data, it takes a new type of data in order to summon the dawn of a new era. Yes, it had to be big data.

And in China, anyone would agree that Alibaba’s launch of the Uni-Marketing framework in 2018 marked the beginning of this stage. With DataBank and StrategyCenter, marketers for the first time got their hands on a wide array of data on real consumer ecommerce behavior. Suddenly, insight extractions such as consumer profiling and segmentation can be done with a wide range of real behavioral data that yield fresh insight. And better yet, the resulted profiling or segmentation can now become more than just a theory on a PPT slide: target audiences can be tagged with unprecedented accuracy in the real world (albeit in a database). And with insights and accurate tagging, precise messaging and media plans can be tested in the real world against all kinds of KPIs including purchase conversion to identify the more effective combinations. Suddenly, marketers were thrilled by the magic of increasing ROI by many folds, waste and fraud reduced, the existence of GMV, and so on.

For the first time in history, marketers were able to extract insight with huge amount of data both in width and depth, and directly translate strategy into testable and optimisable executions with fewer ‘loss in translation’ moments. Plus, for the first time, marketers were able to link specific strategy and campaigns to short-term GMV and therefore ROI becomes a plausible and measurable KPI.

While big data allowed for powerful solutions, the limitations in big data and data platforms got in the way.

First of all, most of the big data available is behavioral data. While the data provides more granularity and allowed for tagging on real consumers, it had one major drawback: behavioral data can only tell how consumers behave, not how they make decisions. In other words, we know ‘what’ but not ‘why’. As a result, identifying target groups with higher ROI based on correlations or A/B testing is the norm. On rare occasions, some analysts would dig out ‘why’ using traditional market research and then use the insight to guide big data analysis, but that’s the exception rather than the norm. Many probably will ask, didn’t some big data guru state that correlation is everything? Well, if you are just predicting stock movement in the next 72 hours, yes, correlation is sufficient. But if you are trying to figure out the next blockbuster product or where the consumer taste is heading to or if there is a killer selling message to be had, correlation is utterly insufficient.

Another major limitation has to do with the level of access existing marketing data platforms offer. Almost invariably, the platforms do not allow apps to be embedded which can execute the analysis automatically based on existing algorithms; nor can the A/B testing, tagging, media placement be carried out seamlessly without significant human intervention. This means that humans are still the thinker and worker, the one who makes the decision and pushes the buttons. Marketing automation in this sense is simply beyond reach.

Stage 3: Paradigm shift—changing the game beyond human recognition

If there was ever a sci-fi writer who wanted to write about the future of marketing, it will most likely look like this:

• Many routine marketing decisions, especially the executional level ones such as media spot plans, target selections, or incremental improvement in communications, will be carried out and optimised automatically based on self-improving algorithms; some of which are already a reality today
• Most advertising and social media contents will be largely generated by AI engines. Already, AI can compose a piece of Bach-like music to pass as the real thing on trained ears. Similarly, machine learning can be deployed to roam through existing creative content and the resulting black-box algorithms can be used to generate new content with little human intervention. There are already some companies striving in this area. It is just a matter of time until some of them churn out AI content that can pass as the fruits of a decent copywriter.
• Similarly, machine learning will comb through consumer journey data to devise individualised consumer journey intervention plan to pinpoint the best timing or occasions to reach out to certain consumers with the most effective messages/activities in order to serve the marketing objectives, be it seeding, brand building, purchase conversion, retention, or fission.
• Aside from routine deployment of AI and algorithms, Stage 3 will demonstrate a major departure compared to Stage 2. Stage 2, for the most part, is about incremental efficiency gain working within the existing consumer behavioral and decision patterns, while Stage 3 will proactively reshape these patterns by re-engineering the consumer journey and restructuring consumer decision-making. Here, machine learning coupled with historical data alone won’t be sufficient. Experimental design guided by behavioral sciences will shed light on the new frontiers in marketing and reveal new insight and solutions to nudge consumer decisions.

To fully realise the potential that Stage 3 has to offer, it requires data that can depict the entire consumer journey as completely as possible. Today’s data silos (eg TMall’s databank) are very powerful but each of them alone can only cover a small corner of the vast digital universe. Given the transforming value a full data set possesses, it is just a matter of time before someone figures out a viable offering.

With such a magical dataset available, the profile of marketing professionals will increasingly look like that of a quant fund: less related to business students but more towards mathematicians, programmers and behaviour scientists.

Marketing in this stage will be full of excitement, wonder, and magic. In a sense, marketing armed with data and algorithm will be more powerful than ever in shaping consumer behaviour. But will consumers turn against marketing for pushing unwanted products like they have with Facebook? Will governments become weary of the industry for manipulating the mass like the they have been with Cambridge Analytica and its incarnations?

Clearly marketers at the forefront of the quant revolution need to adhere to two principles:

1. Utmost respect for consumer privacy

2. Nudging consumer behaviour only when it adds value to the consumers

With the stage set for the great quant revolution, what are the implications for today’s marketers? All great shifts in an industry inevitably bring unprecedented opportunities. I believe that these opportunities will be made available in this revolution:

• Data asset: data is the new oil. Any unique data sets that can complement or, better yet, replace existing data silos will be of great value.
• Algorithms and applications: Algorithms—simple or complex—that can make decisions on behalf of marketers will be embedded in automation apps. In the very near future, these apps will be making daily targeting and messaging decisions. Given more time, they would make strategic marketing decisions better than their human counterparts.
• Inter-system automation systems: Integration systems that can link and facilitate collaboration among various data silos, algorithms, digital media outlets, ecommerce platforms, digitalised offline touch points, and other stakeholders.
• Behavioral sciences and social/cultural forces: Unlike quant investing, arbitraging market inefficiency will not make brand owners a lot of money. As such, pure statistical approach is not sufficient. We need profound insight on how consumers make decisions in order to learn how to nudge their decisions. Behavioral sciences, especially behavioral economics, coupled with social and cultural forces that sway how consumers think, will serve as a beacon when data scientists race to build killer algorithms and apps.

Certainly, the jury is still out on how the third stage will turn out. However, I would kill to hear thoughtful opinions on what Stage 4 might look like. 

---

Paul Zhou is CEO of The Illuminera Group.