Lamplight rethinks effect of Asian vernaculars on social analytics

"People think of social media sentiment as purely negative or positive, but it may not be extreme black or white in a number of cases," said Sam Olsen, CEO at Lamplight Analytics. "It is in a grey spectrum depending on context".

Instagram user Mandy Chi's post in traditional Chinese began with: "Halfway on the CX845 (JFK-HKG) flight, my eye capillaries burst. Fortunately I prepared eye drops, but they stung my eyes with every drop. I suffered through the long-haul flight to Hong Kong that arrived in the middle of the night." 

Context-aware language capability is lacking in the Asian region, Olsen said, even among the handful of competitors already working in the social media monitoring space here—Salesforce, Synthesio, Brandwatch, Meltwater.

However, given limited capability to analyse (and not just monitor) Asian languages, turning listening into actionable social intelligence is challenging; even well-established companies have been using unscientific approaches to make commercial decisions.

A case in point from Olsen’s experience was with an international event agency that had no idea which Western music band to bring to Asia for a show. The organisers had "no information with certainty around fan engagement", said Olsen. "So they just asked people around them, and in the end, nobody bought the tickets." 

Lamplight Analytics was set up in the first half of 2015, based in Hong Kong, to provide social data analytics in local vernaculars: English, Cantonese, Mandarin, Japanese, Korean, Malaysian Bahasa and Indonesian Bahasa.

"We construct searches that extract the most relevant content from each region. We are definitely listening with a local ear," said Olsen. What clients want is an Asia-first platform that focuses on Asian languages and Asian social networking services but that is also globally competitive.

The most common refrain Fergus Clarke, co-founder and COO at Lamplight Analytics, hears from marketers is, “what I have, in terms of social insights, do not work in Asia".

Existing social analytics providers have not been too helpful either. "One Western client told us that they were given the impression that all Mandarin-language tweets appearing on Twitter are from Chinese users in the mainland," added Clarke. That is only partly true, but it is worrying when social analytics providers have not gotten the basic knowledge of China's social landscape.

Lamplight tries to simplify the task of gleaning contextually-aware intelligence by on-going investment in Natural Language Processing (NLP), machine learning techniques and related statistical modelling. It has also licence agreements for metadata from over 20 million different social media sources, including Facebook, Weibo, local forums, Twitter, blogs and frequent flyer forums. 

Another data source that Lamplight uses to enhance its analytics tool, is language libraries from academic institutions that contain virtually all of the universal linguistic laws that filter through Asian languages.

Even in the ad-hoc development of slang, there is a tipping point and a learning curve for social analytics providers. "Slang words don't become lexicon until they are in the undergrowth of a language for a long time. What we won't be able to do is track the first utterance of a word; but we will be able to track the context around it, combine probabilistic logic to adapt to machine learning, and apply the usage to marketing," explained Olsen.

"There is a misunderstanding that data is being used to instruct,” Olsen said,” but it is to guide".

"We want to shed light on 4 billion [Asian] people by removing the language lens off them".