Introducing Unity Face Tracking Plugin to Create Snapchat-like Filters in Unity

Our Face AR extension for Unity brings Banuba’s 3D face tracking technology cross-platform and allows developers to create realistic and complex face tracking filters, augmented reality effects, and 3D masks in Unity. This post provides an overview of our Unity face tracking plugin and explains how you can use it in your apps.

February 18, 2020

You can generate Snapchat-like lens and AR masks in Unity and add them to your apps with our face mesh Unity package. You can purchase the limited version of Face AR plugin at Unity Asset Store or contact us via a website form to request the full version of the plugin which includes 30 Unity face masks.

Unity Face Tracking Plugin: Features & Overview

Face filter plugin for Unity is a native library compiled for Windows, MacOS, iOS, Android and Web. It includes the face tracking package that detects the face in real-time with vertices of face mesh and landmarks to display AR masks.

The plugin is a perfect out-the-of-the-box engine to bring face filters, 3D masks, effects and avatars in Unity projects like augmented reality apps, games or photo booths. It supports a variety of graphical technologies, e.g. the AR model in a mask can support gravity or mirror the light with the head tilts.

Features:

Unity 3D face tracking technology with face mesh and landmarks/tracking points to map face filters in real-time
Supports glasses and partial face occlusion
Multiple faces support to build and apply filters in Unity to several faces simultaneously
Triggers to implement small AR scenarios in face filters, e.g. call effect with mouth opening.
Animation support to implement dynamically changing and transforming AR models.

System Requirements

Any mobile platform: ARMv8.
Android: supported from Android 5.1. Minimal Android SDK version - 21.
iOS: supported from iOS 10.0. Minimal supported device - iPhone 5s.
Web: WebAssembly support required.

How developers can use Face AR Plugin for Unity

Unity is a platform for creating rich cross-platform 2D, 3D, VR, and AR games and apps. Its strong graphics engine and editor allows developers to implement the most creative game concepts, and launch them on any media or device.

The ability to target multiple platforms including mobile, web and desktop has been among the key drivers why many developers choose Unity for game development. And if you want to add 3D content, Unity is no doubt a good choice for implementing augmented reality apps featuring immersive effects and scenarios.

However, the Face AR experiences like Snapchat-like face tracking filters, AR effects, facial animation and avatars have been not an easy task for Unity.

To help developers streamline their workflow with face filters, we integrated our AR face tracking with Unity and designed the plugin with Unity face masks. You can use our ready masks or design and upload your own face filters to Unity Face AR plugin.

Is Face AR plugin only for Unity games?

No. Unity has primarily been considered as a game engine, however, it’s not the only type of apps you can build with Unity Face AR plugin. Unity has out of the box AR and VR support, so if your non-game app is 3D type, AR, VR type, the plugin will be a perfect tool to implement real-time facial animation based on 3D face tracking.

What apps you can build with Unity face AR plugin:

Cross-platform AR apps with face filters and Unity masks
Realistic virtual-try on solutions for glasses, hats, accessories, jewelry
Real-time video chats with avatars and face filters
Entertainment Unity apps with Snapchat-like lens and AR effects
Utility or business apps with Face AR features

What Unity face masks are included in Face AR plugin

Within 3 years we’ve designed over 1,000 face filters and AR effects to give people the most entertaining Face AR experience on any device and platform, be it iPhone X or low-level Android device. We transferred some of our best face filters in Unity, so you can modify and use these templates to build your face filter app. The plugin includes the following examples:

Beard_Flowers. The filter features a fun moustache and beard and can be used in entertainment apps and modified for realistic virtual try on.
Tarantula: The filter shows AR effect animation and is top-favoured in entertainment apps.
Butterflies_Crown: The filter includes animation, accessories, and headwear. You can use it as a beauty effect or modify for realistic virtual try on filter.
Oktoberfest: The filter includes a virtual beard and hat. Category: entertainment and fun.
Football_Fan: A fun face filter with a hat and face mask.
Groot: A filter with action units to implement avatars and 3D emoji in Unity.

Animate AR models, design realistic try on filters, modify user appearance with bears and moustaches — engage your users with the best entertainment filters implemented with our Unity Face AR plugin.