Email article to a friend

Print this article

Judging marketing effectiveness by Tim Riches

by Media on May 19, 2011
Tim Riches, FuturebrandTim Riches, chief growth officer for Asia-Pacific at Futurebrand, reflects on his experience as a judge at the Asian Marketing Effectiveness Festival in Shanghai last ...

Blogger profile: Media

Media magazine -

Tim Riches, Futurebrand

Tim Riches, chief growth officer for Asia-Pacific at Futurebrand, reflects on his experience as a judge at the Asian Marketing Effectiveness Festival in Shanghai last week.

Over the last few weeks I have had the pleasure of judging submissions for the Asian Marketing Effectiveness Awards – an annual conference and awards ceremony in Shanghai. As my first major award judging experience it has been a fascinating process, diluted only by the fact that I was given 108 entries to read, view and score on 4 dimensions: strategy, creative/innovation, execution and results. Quite an onerous task!

I reviewed entries in the categories of Best Sustained Success, Design/Packaging and Best Idea. As you can imagine, assessing effectiveness is quite a different challenge across these 3 – relatively confined for packaging, very broad in idea and harder to prove in sustained.

Some entrants took the sawn-off shotgun approach and entered multiple categories with the same entry, often not tailored to the category, which highlighted one of the key issues of this process – the fact that because effectiveness is a diverse and multi-faceted concept, the categories themselves have a role to play in becoming sub-sets of the overall idea. More on that later…

A tangled web of lies, damned lies and effectiveness metrics

I remember the first thing I read when doing a marketing course at uni was the classic John Wanamaker quote: “Half the money I spend on advertising is wasted; the trouble is, I don't know which half”.

So it goes without saying that knowing whether we’re really making a difference as marketers or agencies has been the perpetual challenge. The challenge for marketers in terms of their influence within their companies; the challenge for agencies in terms of substantiating their value proposition (or at least the size of their invoice). That’s why these kinds of awards are so important.

Overall, I thought that the quality of submissions I reviewed was higher than I actually expected, with nearly all of them able to field some semi decent data on the alleged effect of the marketing investment. Few attempted an actual “ROI” calculation though, preferring to rely on campaign KPIs rather than business investment.

But what was quite palpable was the amount of selectivity and even “spin” in the data shared in the entries – should I have been surprised? Many of them read more like selling documents than submissions that would receive critical examination, showing a lack of self-awareness that’s often displayed by agencies who struggle to achieve business credibility with senior clients.

Despite the importance of the effectiveness challenge, and the effort displayed by agencies to demonstrate it, reviewing the 108 entries brought it home to me how complex effectiveness is in the marketing concept – as soon as you move outside very specific, self-contained initiatives.

However, there were 2 factors lacking in most entries that really did get in the way of building the effectiveness case:

Isolating the marketing effect – the majority of entries I reviewed made insufficient attempt to isolate the effect of the marketing activity from key external factors that may drive change. Most entries of course examined market share and category growth – as they were required to – but few took the trouble to address the wider context. And we should bear in mind that for business leaders, those that marketers and agencies often criticize for “not understanding” marketing, the broader considerations of the opportunities and threats of the market environment are precisely the focus – not the artificially constrained world of traditional marketing metrics.

Although I won’t mention names, and without any comment on how these ended up being scored, two examples of neglecting key external factors from “marketing results” that I saw were:

  • A defence force recruitment campaign, that while impressive in its detail and very well argued, took little account of what is surely an important factor shaping perceptions of what serving in the armed forces might be like: the deployment of that armed force in action in Afghanistan, a conflict which appears in media as diverse as the headline news and Iron Man movies! Surely such factors impact how people feel about “signing up” – one way or the other.

  • A national tourism communications initiative that claimed itself to the primary driver of increased inbound arrivals from key regional markets, even though the period in question coincided with the launch of three major international tourism attractions targeting exactly those same markets.

Now, regardless of the factual importance of these factors (although one must surely think them significant) they should be given consideration in terms of the credibility of the entry – because they go right to the heart of the big challenge, knowing whether what the marketing activity made the difference.

The “golden thread”

Considering the point above, it’s probably close to impossible in the world of marketing to be totally watertight on cause and effect.

But in light of this, when considering the question of marketing effectiveness, this only places more emphasis on establishing the “golden thread” of a logical link connecting business objective, consumer insight, marketing strategy, creativity, execution and results. For marketers, the integrity of the entire chain is crucial as it’s the core of a marketing business case. For agencies, the close nexus between insight, strategy, creativity and execution is surely the focus.

It was this golden thread that I found notable by its absence. The submissions typically included these elements, but often did not credibly connect them together, especially addressing the hard questions of how a consumer insight was really at the heart of the approach to creative and execution, how effective was the message “take-out”. In fact, there was often a sense of disconnect between insight, strategy and execution – despite the fact that the process of entry-writing must surely be the process of weaving together these elements which may in fact have been delivered by different departments, if not entirely different agencies.

Conversations I had during the event with senior clients confirmed this problem – between the “unbundling” of the traditional ad agency, media fragmentation and the emergence of more specialist agencies, there are many agencies fighting for the steering wheel. It’s very hard to achieve actual synergy of effort. Plus of course you have the client in this mix too with responsibility for selecting agencies, defining their relative responsibilities and translating business objectives into marketing activities.

In this environment, maintaining the integrity of the “golden thread” is a challenge indeed, notwithstanding the ever-increasing appetite for robust marketing investment cases.

What about brand effectiveness?

Coming at all this with a brand consultant’s perspective is interesting too. Because we tend to view brand in a broader sense as a business asset, rather than an aid (or impediment) to a sales initiative, I guess our natural orientation is to take a long term view of the evolution of the brand over time: what is the heritage of the brand, how has its story unfolded over time, what’s driving it in future, and how does a branding initiative seek to align that changing story with drivers of business value.

Very few submissions I saw, even those in the “sustained success” category really examined this point – how a brand improved its relevance and value by evolving its meaning over time, although it was a topic of some discussion during the judging process.

Of course the obvious metric in branding is brand valuation, and no entry I saw incorporated changes in brand value as an effectiveness metric either.

What does all this mean?

Effectiveness is complicated in marketing, and the bigger the marketing challenge, the harder it is to show what you’re doing, isolated from its context, is making a real difference.  I think this is why most entries addressed effectiveness in terms of KPIs not ROI. It’s a significant difference: KPIs are typically based on benchmarks and goals expressed as marketing measures – really from the point of view of a marketing director. ROI is a business measure. As a business,  I spent $x and achieved a financial outcome of $y – it’s a CEO perspective, weighed up against all the other things a business can do to drive income. Clearly it’s also a perspective that both marketers and their agencies don’t always find it easy to engage with.

It seems to me that the complexity of the effectiveness challenge highlights the opportunity within the awards process to use entry categories to more closely frame effectiveness: tailoring effectiveness definitions by category would help in this regard.

One of the best submissions I saw, from a rival brand consultancy, demonstrated the effect of insight-driven packaging change in terms of brand appeal – with impressive results. But outside the nice, neat world of packaging changes, branding seems very hard to submit to the effectiveness microscope. It seems to me that branding, with its longer perspective and deliberate engagement with business change, category dynamics, the innovation agenda, talent attraction and motivation is even more difficult to isolate and assess as an investment.

Cultural nuances

One of the most interesting aspects of reviewing entries from around a region as diverse as this is the insight that the entries provide into some of the cultures in which. Some lighthearted observations on three that stood out to me:

India – Is Hyperbole as city in India? For stories of epic human drama, unprecedented achievements, intellectual breakthroughs of Einsteinian significance, David and Goliath challenges of unimaginably heroic valour, look no further than marketing in India – only for the brave, it seems… Also the clear leader in the fine art of charts and graphs, many of which show how many ways there seem to be to justify using a Bollywood star for “women’s” brands or a cricket player for “men’s”. In the unlikely event that no celebrity is appropriate – or perhaps affordable – then some kind of positive social outcome is preferred, maybe literacy or environment.

Malaysia – earthy, cheeky humour and connecting with the “man on the street” seems to be the order of the day – evidenced by the hitherto untapped marketing potential of circumcision in the telco sector. Certainly an eye-opener for me! There’s also a healthy a dash of the Indian appetite for dialing up the drama of the challenge and the genius of the solution, must be that “Truly Asia” mix of influences…

Singapore – hmmm, a bit boring I’m sorry to say, although certainly the leader in effective social conditioning – how to act towards your parents, which uni course to choose, how to behave in public, the list goes on... If your agency isn’t doing its bit to shape the future of the nation, what is it doing? How can you have a business? And at least there’s living proof of effectiveness, ROI – return on inculcation.

And finally the judging panel. Despite some recent umbrage taken by Sir Martin in a media interview to a question on the apparent dearth on local talent in senior agency roles, there were very few Asian judges in the room, and those that were, were all client-side, not agency. And of the “expats” in the room, there were certainly some pretty colonial views in evidence. Surely this is not an accurate reflection on the state of the agency world in Asia…?

Data & Research

Cookie consumers: Brand traits help offset guilt

May 23, 2016

This research into the consumption of sweet biscuits found that friendliness was the personality trait ASEAN consumers identified with the most in the biscuits they buy and eat most frequently. This was followed by brands that were regarded as ‘cheerful and fun’, and ‘trendy’.