Opening up new worlds with WebAR: What is it and how are brands using it?

Augmented reality is one of the fastest growing technology trends, tipped to reach meteoric heights over the next few years.

A technology that was catalysed by the 2016 release of Pokémon Go, it has almost limitless applications in various business sectors, but is especially valuable in video games, transforming physical sectors like retail, and adding an interactive layer to advertising.

Vibrant Media research in 2017 found that nearly seven out of 10 media planners want to incorporate more AR experiences into their advertising efforts to boost customer engagement, while Deloitte’s 2018 report on technology trends for mid-market companies (those with annual revenues between $100 million and $1 billion) found that a majority of them are experimenting with AR in various forms to increase business growth.

There are wildly different estimates of the current and future value of the AR market. It is expected to grow to anywhere between US$31 billion and $75 billion by 2023 (according to Greenlight Insights and Digi-Capital) or to US$198 billion by 2025 (according to Statista). Orbis Research believes the AR gaming market alone could reach $285 billion by 2023. All predictions point to AR overtaking virtual reality in revenue and usage, if it hasn’t already, even though the VR sector is more mature.

The driving factor behind this is ease of use. AR is accessible via the majority of smartphone devices on the market, while VR requires a headset. But most AR experiences require users to download an app to activate an experience, which can limit its reach. This has given rise to a new phenomenon: WebAR.

WebAR allows consumers to access an AR experience within a desktop and mobile browser, and it is therefore able to reach twice as many smartphones as native AR apps, according to a recent report by ARtillery Intelligence. Sounds simple enough, but the technology required to compress huge AR files into a browser is significant.

“It is challenging technology to build that requires not only state-of-the-art computer vision, but also incredible code optimisation to allow this to run within the constraints of standard web scripting,” explains Erik Murphy-Chutorian, the founder and CEO of AR startup 8th Wall, in an interview with Campaign Asia-Pacific.

Heineken recently ran the first web AR-based campaign in Singapore with an interactive prize draw to promote its sponsorship of Formula 1. The ‘Heineken AR Cheers’ campaign allowed consumers to race an F1 car within a real-world environment in order to win tickets to the brand’s pre-race party and to the Singapore Grand Prix. 

The campaign was built in just eight weeks by Just After Midnight (JAM), the technical lead for the F1 campaign. JAM coordinated with two other agencies to build the AR experience, including Singapore development agency Megapixel, which was responsible for the website development and prize engine system, and US-based AR agency MUN, which designed the graphics.

JAM Asia chief executive Freddie Heygate explains that it was WebAR’s flexibility that attracted the Heineken Global team to the technology, versus the more common app-based AR.

“Native apps often suffer from reduced engagement through the download barrier, high development costs and lengthy approval times from individual app stores," he explains. "WebAR not only overcomes these user-experience hurdles—allowing consumers to interact with brands in real-time—but 8th Wall’s open application framework also allowed us to integrate other technologies into the experience, including image recognition and a prize draw element.”

Heygate selected 8th Wall to power the web AR experience since it offered “the most advanced and flexible technology on the market”. While Facebook is one of the biggest and most well-known AR companies (it has so far acquired around 12 AR/VR companies) its branded options are limited, Heygate explains. Facebook’s ‘Spark AR Studio’ doesn’t allow alcohol brands to build AR experiences, and campaigns are limited to its suite of apps. While Apple and Google both offer AR development tools, they are also limited to apps in their respective operating systems (iOS and Android).

Launched in 2016, 8th Wall claims to be one of the first companies to bring WebAR to market. Founded by a former Google and Facebook engineer, the Palo Alto-based company uses computer vision to enable six degrees of freedom tracking (so it stays in place as a user moves their phone), light estimation (so shadows will be aligned using GPS data, for example), and surface estimation (so the objects can be placed on surfaces).

“It was believed to be a technological impossibility on today's devices to build this type of computer vision into a web platform,” explains Murphy-Chutorian. “It required a very significant effort on the engineering team to re-architect everything to use the highest performance set of web technologies, and then to combine it into an easy-to-use set of libraries that people can integrate into today’s web pages and activations.”

Its customers are primarily advertising agencies and brands (although it offers a ‘lite’ version of its developer platform called AR Camera for “general purpose use”) and it has so far powered 80 WebAR activations for brands across industry verticals including retail, food and beverage, travel and tourism, automotive, fashion, sports and entertainment.

“AR allows customers to experience brands like never before, turning the physical world into a digital space. The technology not only gives shoppers the ability to better visualise products, but it brings immersive entertainment, activities, events and experiences to users in real-time,” explains Murphy-Chutorian. “It also ties into existing traffic streams, so if you have an established website or physical presence, you can have this new type of media on your website or platform that you own and control, rather than relegating to a social network.”

To extend the capabilities of the tech, 8th Wall recently struck a partnership with Microsoft to allow developers to integrate video "holograms" shot in Microsoft's Mixed Reality Capture studios into an AR experience on a browser, opening up a “whole new class of activations that involve real people”, Murphy-Chutorian explains. This will allow brands to integrate celebrities or influencers into their AR campaigns.

It is also working on making its technology "device agnostic" so it can support AR (or mixed reality) experiences within smart glasses/headsets. It can currently run on VR headsets in a "windowed" 2D mode, by calling up the 8th Wall tech inside the headset's browser.

But new technology often comes with growing pains. Heygate explains that one of the biggest technical challenges the team experienced while creating the 'Heineken AR Cheers' campaign was compressing the graphics in a way that would reduce the load time without affecting the quality of the visuals.

"We found that one application was unable to compress the file to our desired size as it struggled with texture output. After discussion we decided to rebuild the visual in a different piece of software to maintain a low page load speed," he says.

Apple's privacy settings were also "notoriously hard to navigate", he adds, due to a setting rolled out in Safari in iOS 12.2 that required users to opt-in to provide websites with motion sensor access (8th Wall subsequently worked with Apple to switch the motion access setting back to be on by default in iOS 13).

In order to ensure the experience worked in low-light situations (such as when consumers are out at a bar), the team trained an image recognition model with over 200,000 variants of Heineken bottles, which Heygate describes as a "resource-heavy task".

All things considered, Heygate says he would "absolutely" encourage brands to consider WebAR to provide an engaging experience for consumers, he cautions it is a "challenging technology to build" while it is still in its early phases.

“Brands need to think hard about the audience they’re targeting and how the technology will perform on their smartphones," he says. "Performance planning from day one is key—taking into account how the experience will work on different devices with different levels of connectivity—and code optimisation can make or break a campaign. You can have the best WebAR campaign in the world but if it takes 20 seconds to load, it will be abandoned by your users."