Snapchat goes bigger into image recognition and AR maps amid spate of launches

More voice commands for lenses and more image recognition are coming to Snapchat, the company announced as part of its annual ‘up fronts’ today.

The company’s second annual Partner Summit announced a range of new features and updates that open up the platform more to third-party developers and publishers.

Snapchat’s Voice Scan offers Snapchatters Lens results based on voice commands. Powered by a partnership with SoundHound, users can now press and hold on the camera screen to tell Snapchat what kind of Lens to display.

The platform is also making bigger moves into image recognition through its Scan feature, which is activated when you press and hold the camera button. Today Snapchat announced it is working with new third-party companies to help it enable users to scan and identify more images, such as Plantsnap, which it said can identify 90% of all known plants and trees, and Dog Scanners, which can recognise almost 400 breeds. An integration with Yuka later this year powers Nutrition Scanner, which provides a rating on the quality of ingredients in many packaged foods by scanning an item’s label.

It is also pushing Scan as a brand opportunity, and said users will be able to scan the Louis Vuitton monogram to be "transported into the inspiration behind their latest collection".

Key among the many announcements today is the launch of SnapML, a feature that is part of Lens Studio 3.0, which enables any developer to bring their own machine-learning models directly into Lens Studio. Snap has partnered with Wannaby, Prisma, CV2020, and several Official Lens Creators on their first SnapML creations.

Lens Studio is also now offering Face Landmarks and Face Expressions for improved facial tracking, new Hand Gesture templates, and an updated user interface to simplify navigation within the tool.

Many of the new tools provide opportunities for fashion brands, a sector which has taken to Snapchat more than most. Lens Studio is also releasing a foot tracking template powered by an ML model from Wannaby that lets developers easily create Lenses that interact with feet, which presents opportunities to launch ‘trying on’ activations for shoe brands.

It is also previewing Local Lenses, which is an upgrade on the current landmarkers feature that enables creators to build a augmented reality virtual world built right on top of real-world locations.

"Soon, Snapchatters and their friends can step into this virtual space together to decorate nearby buildings with colorful paint and experience a new dimension of AR," a spokesperson said.

The company said it is also providing free ad credits for local businesses as they reopen following the coronavirus lockdown. 

Other announcements that advertisers should know include:

Snap Map: It is launching "Places" on the Snap Map, the personal map that shows what your friends are up to. Now you can more easily find the spots that are popular with our community, no matter where you are. Snapchat said it is adding "millions of listings" for businesses around the world. The Place Profile includes Snaps from the location, the address, hours of operation, and reviews from TripAdvisor and Foursquare. In the US, you will soon be able to order food through Postmates, DoorDash, and Uber Eats.

Topics: Users can now share interests by using a Topic sticker or submit your Snap to a Community Topic, like "Life Hacks" or "Oddly Satisfying" and browse Topic Pages to explore Snaps based on what you care about most.

Story Replies: Snapchat is filtering out bullying and hate speech from replies to "Snap Stars" so that they may only ‘feel the love’ from fan messages. Snap Stars can turn Story Replies into stickers and add them to their Stories or feed into Q&A Sessions.

It is also expanding its content range, mostly through publishing deals with publishers and and broadcasters. It is introducing a dedicated news product called Happening Now, aimed at being the fastest way for Snapchatters to consume breaking news by turning updates into a new, easy-to-consume format customized for mobile. Partners include The Washington Post, Bloomberg, Reuters, NBC News, ESPN, NowThis, E! News and BuzzFeed News.

The company also announced a new slate of Snap Originals, featuring talent such as comedian Kevin Hart, writer and director Catherine Hardwicke and actor and producer Anthony Anderson.

There are also new Brand Health tools, such as Friends Check Up, a new safety notification that prompts Snapchatters to review their list of friends and make sure they are only friends with people they know and trust in real life.

It is also launching Here For You, which provides Snapchatters with expert resources when they search for topics related to their emotional and physical wellbeing, including stress related to Covid-19, anxiety, depression, domestic violence, bullying and more.