Helen Roxburgh
Nov 20, 2017

The role for face-recognition in customer experience

China has the lead on facial-recognition technology as marketers look ahead to some tempting applications.

Finger-lickin’ recognition: A customer orders KFC using facial recognition technology at a restaurant in Hangzhou city in September.
Finger-lickin’ recognition: A customer orders KFC using facial recognition technology at a restaurant in Hangzhou city in September.

When the new iPhone X was unveiled this autumn, the phone’s capacity for face recognition instantly became its key talking point. Say goodbye to tapping in passwords—just a quick scan of your face, and the iPhone software would know who it is answering to.

But in China, this software isn’t really new to brands and marketers. “Facial recognition (FR) has been in the China market for a while,” says Carter Chow, CEO for JWT China. “Its application and usage have seen gradual pick up, but with the more recent launch of the upcoming iPhone X, the topic has resurfaced as being very hot on social media. Chinese consumers are generally very open to new tech concepts and have generally felt less resistance to sensitive topics like privacy and data security.”

In July, the Beijing government committed to making AI a “new and important” part of the economy by 2020, and China’s technology firms both old and new are rushing to apply the commercial use of FR technology, often working with the government’s image database of 700 million citizens.

In September Ant Financial, the financial services spinoff of Alibaba, rolled out a new service at KFC in Hangzhou allowing customers to pay for orders using only their faces. Although buyers had to input phone numbers as an extra layer of verification, the technology still works even if the purchaser’s phone is turned off. 

Caught in traffic: A Chinese traffic cop explains the facial recognition screen to a girl at a road crossing in Shanghai.

“We’ve had positive feedback so far from our customers, who think the idea is amazing, convenient and fun,” says Steven Li, senior vice president of Yum China. “We find our China customers are very excited by the new technology and want to try it and use it.” Li adds KFC will introduce it to more stores in November.

Although this is the first time Alibaba has rolled out the technology in commerce, it was first unveiled by Jack Ma two years ago at a technology conference in Germany, dubbing it “smile to pay.”

In the main, Chinese consumers seem to be embracing the new technology. A recent HSBC survey found nearly half of people interviewed in China believe FR to be useful, 16% higher than the global average.

Car-hailing service Didi Chuxing is using the technology to verify drivers’ identities, while ecommerce giant Suning has launched a new unmanned store that allows buyers to pay just by showing their face to the scanner after registering on Suning’s app.

See the complete China Innovation 2017 report

It is also being experimented with fairly widely in banking. HSBC has introduced facial recognition capabilities in its mobile banking application for customers in China, meaning customers can open their HSBC mobile app, blink at their camera, and transfer up to $7,500 a day. China Construction Bank allows customers to pay with their faces at some vending machines, while China Merchants Bank does the same at about 1,000 ATMs. A mobile affiliate of Ping An Bank also uses facial recognition to authenticate a borrower or investor’s identity over the internet.

Airlines are experimenting too, and earlier this year China Southern Airlines used facial recognition in place of boarding passes for the first time.

Chinese citizens used to facing up

The technology is also proving useful to state-run organisations. In Shenzhen, Shanghai and Beijing, facial recognition is used to identify jaywalkers, with names and pictures displayed on a scene above the road, while in Beijing officials have started using the technology to catch thieves of toilet paper in public toilets.

In September, police arrested 19 people at the Qingdao beer festival who tested positive for drug use by using entrance cameras to record the faces of more than two million attendees. Using facial recognition, they identified individuals with past histories of drug abuse and tested them on the spot.

IHS Markit has estimated China has 176 million surveillance cameras in public and private hands, and it forecasts about 450 million new ones installed by 2020. The US, by comparison, only has about 50 million.

In the near future, tech firms and marketers will combine to personalise this technology to their user’s needs. Facial recognition is already being used by KFC to predict their customers’ orders, for example, depending on previous history and their demographic.

To catch a thief: These toilet paper dispensers at the Temple of Heaven in Beijing use facial recognition, aimed at ending a years-long crime spree by toilet paper thieves.

“The use of FR technology can be very wide,” says JWT’s Chow. “We are seeing more and more different applications every day. However, the true test of any innovation is if the related applications can actually bring practical benefits to the users - for example, the use of QR codes. QR codes proliferated without bringing users enough actual benefits, but this all changed when social media apps like WeChat used QR codes as an easy and quick way to add new social contacts, followed by mobile payment platforms like Alipay and WeChat Wallet.

“So, marketers and agencies need ensure that they are not using facial recognition only as a cool or trendy tool but rather, really think how the new technology can deliver meaningful benefits to their consumers.”

Aram Sinnreich, author of The Privacy Crusade, has predicted a future where advertisements will soon be able to gauge the likelihood of an individual making a purchase, and adjust their messaging in real time. For instance, a customer might see an advert for 50% off a meal deal on their phone just as they pass by a fast-food restaurant. If the phone registers disinterest when it scans their face, it could increase the discount level, to see if it persuades the customer to buy.

“If you walk into a supermarket, and examine a coffee maker but do not buy it, in a few days you could be shown an internet ad of the same item with a discount or receive a personal message on social media with relevant information about such products,” says Mikhail Ivanov, CEO of technology group Ntechlab. “Visitors’ photos may function as cookies referring to the identification and storage of user settings. In other words, loyalty cards will become obsolete. As soon as you walk into a store, the staff will already know what you bought last time, thanks to the camera’s footage and our technology.”

In 2016, Apple bought a tech startup called Emotient, which can reportedly detect up to nine different emotions by looking at a person’s face. Since every Apple device now has a user-facing camera installed in it, Sinnreich suggests that in the future marketers could in theory collaborate with Apple to record how consumers feel about their adverts, and then adjust them to capture interest in real time.

Walmart is reportedly working on a FR system to improve customer service, including cameras that detect dissatisfaction on shoppers’ faces and dispatch help.

(Story continues below)


Smile for the screen: Pay with a face scan


Alibaba’s “smile to pay” project is powered by Face ++ technology, developed by Beijing start-up Megvii. Founded in 2011 by three graduates of Tsinghua University, it is one of a series of Chinese start-ups that are competing globally in developing AI technology. Megvii also runs the technology behind popular apps like Chinese beauty fad Meitu.

One rising star in China’s AI start-up scene is SenseTime, currently valued at $1.5bn. It uses facial ID to replace office key cards. Its software also turns ordinary traffic cameras into “smart” cameras, able to analyse the types of vehicle and human traffic flowing through them.

Search giant Baidu showcased its facial-recognition technology at the company’s first AI developer conference in Beijing this August. It will use this technology at the city’s airport, initially for recognising ground crew and staff at entrance gates, but gradually expanding to verify passenger identification. Baidu has already tested this technology at tourist destination Wuzhen water town, allowing visitors to gain entry using their faces, rather than tickets.

“Retailers are starting to use facial recognition to improve the customer experience for their consumers,” adds Ivanov. “Firstly, they can monitor the facial expression of their workers and customers to ensure that the employees act according to their guidelines and provide a great experience. Secondly, they can recognise their recurring customers and provide personalised offers and discounts without even asking a card or a phone number.”

Beware of abuses

South African coffee company Douwe Egberts set up a coffee machine with FR at an airport that offered free coffee to travellers that yawned as they walked past. Yet once travellers figured out the scheme, it proved costly as word spread throughout the airport.

This shows brands and marketers need to tread carefully in this area, bearing in mind the potential for abuse by customers  or even damage to their brand.

Google, for example, has explicitly turned its back on matching faces to identities, for fear of its misuse by authoritarian regimes. Chairman Eric Schmidt has called face recognition technology ‘creepy’, and said Google will not build a database of users’ faces.

Similarly Tan Jianfeng, founder of encryption tech firm PeopleNet and president of the Shanghai Information Security Trade Association, has been outspoken against using FR technology for gimmicks.

“Once biometric libraries are breached and biometric data are stolen they can never be restored. This is the real weak spot for biological certification,” he warned.

However, the march of FR technology in China seems inevitable, with an enthusiastic government, interested consumers, and growing numbers of technology firms investing heavily. The challenge for brands and marketers will be to use it in a way that excites and interests Chinese consumers, rather than monitors them. 

Campaign Asia

Related Articles

Just Published

11 hours ago

Income Insurance uses feline flexibility for a ...

Life is unpredictable, but investments should be simple and adaptable. This new campaign uses the internet's favourite furball to convey just that.

12 hours ago

Can Asia up its voice at Cannes?

Ian Whittaker unpacks the underwhelming Asian presence at Cannes this year, and why greater representation might be the key to a better learning experience for all.

15 hours ago

Can marketing reconcile its growth and sustainabilit...

Many CMOs are stuck in the paradigm of growth at all costs. But the concepts of 'green growth' and 'degrowth' to counter traditional profit-making structures are slowly making waves in an attempt to mould climate-responsible corporations.