Developing the best face tracking software for Android requires time, efforts and non-stop optimization. That’s why it’s important to keep an eye on what tech giants like Google do in face tracking and align yourself with them.
Explore Banuba technology and try it on your projects
Face tracking cycle explained
This post explains how each performance parameter impacts the user experience on the example of face filters and approaches the choice of real-time face tracking software from the business perspective.
We compared our 3D face tracking software for Android with Google’s ARCore examining different performance criteria.
Face Tracking Cycle: SSD and FRX Dependency
The face tracking cycle involves two stages: Face Detection stage (SSD) and Face Tracking stage (FRX).
Face Detection or SSD (single-shot detection) shows how fast the algorithms detect a face in a single shot. Once the face is detected, the algorithms step into Face Tracking (FRX) stage to track the facial feature points located in the screen.
In case the face is lost, the algorithms switch back to SSD, and the process goes round in circles.
Having two stages in the face tracking cycle makes sense — in case of the face loss, the application switches to power-saving face detection mode.
Why SSD metrics are important
The SSD stage is the starting point of face tracking software performance. Faster and more stable SSD provides a better and smoother user experience.
In practice, the device will heat lesser, the FPS will be more stable and the functionality of the app built on top of this face tracking software will perform better from the end-user experience perspective.
If SSD is tuned to detect the face quickly, at different angles, lighting, occlusion and on different types of people, the starting point of the camera experience runs smoothly. The poorly optimized SSD results in AR masks lagging and frequent losses.
Why FRX metrics are important
The FRX stage aka face tracking enables the software performance in the context of real-time user experience.
It’s natural for users to move, turn and tilt when using the camera app. They may wear glasses, have poor lighting when using the app not to mention the limited device capabilities. All these factors influence the performance of face filters or any other AR camera experience built on top of face tracking.
It's important to consider different FRX metrics when choosing the face tracking software. With FRX, optimized for different conditions such as low lighting, camera shakes and rotates, the user experience feels better, making AR camera apps more enjoyable.
3D face tracking software comparison: SSD
To compare the SSD criteria we considered 5 key criteria including its speed, angles, distance support, multi-face performance and technical features.
Device used: Pixel 2
Comparison summary:
-
Both Google's ARCore and Banuba face tracking software are pretty fast with face detection when looking straight into the camera.
-
Both Google and our face tracking detect a face with 80-90° angles and different camera positions (under and above the face). ARCore performs more accurately.
-
Banuba supports longer distance being able to detect a face from 1.8-2.0 meters while Google supports 1.6-1.7 meters.
-
Banuba’s face tracking software has shown better performance with multi-face detection at different angles compared to ARCore.
How speed impacts user experience
The speed of face detection shows how fast the face is detected when the user opens the camera or after the face loss.
Fast SSD: Face filter appears on a face within milliseconds once the user opens the camera. The mask sticks well from the very first moment.
Slow SSD: The user may need a couple of seconds to have a face filter located on his face. Face filter may “blink” for some time when adjusting to the face.
How angles impacts user experience
The angles metric shows how fast and accurate the face is detected at different angles or when the user is not looking straight into the camera.
Wide angles (80-90°): The user may hold the device at any angle without looking straight to the camera. Face filter sits fast and well or with insignificant inclinations.
Narrow angles (15-45°): The user needs to look straight in the camera to have a filter placed on his face. The filter may disappear if the user inclines or turns the head.
How distance impacts user experience
The distance metric shows how fast the face is detected at a long distance. Usually applied for the back camera.
The longer distance support makes it possible to apply filters beyond the selfie mode using the back camera.
How multi-face impacts user experience
The multi-face criterium shows how fast and stable the face detection is with several faces in a shot.
Fast multi-face: Face filters can be used with several people simultaneously with equally good performance.
Slow multi-face: Face filter doesn’t fit properly with the second face in the camera, blinks and falls apart.
3D face tracking software comparison: FRX
To compare the FRX, we replaced ARCore’s Augmented Faces mask with the 3D face mesh which provides a better visual representation of face tracking performance.
We tested a variety of use cases e.g. camera jitter, different head position, low lighting and others and paid attention to mask stability and performance. The table below explains the performance differences.
Device used: Galaxy S9
NOTE: The 3D face mesh doesn’t fully show how Google’s face tracking performs on the mouth. It’s important for filters with mouth triggers, virtual lips recoloring or 3D avatar performance.
FRX comparison on Galaxy S9: Banuba vs ARCore
FRX technical features
FRX technical features cover the device outreach that face tracking software supports. Broader device coverage makes face detection and tracking solutions accessible to a wider user audience.
When comparing our face tracking software with ARCore, we found ARCore is supported only on relatively new devices, with minimal Android version 7.0 (in practice, 8.0 and later), and an API version of at least 24. The price range of such devices is $ 250 -300 on average, which makes ARCore face tracking inaccessible to users who stick to low-end phones.
Banuba: Min device price — $100
ARCore: Min device price — $250-350
View the full list of the devices ARCore supports. Many on the devices require Android 8.0 or later to enable face tracking.
Besides, in the content of AR camera apps, users will need to install an additional Android package Google Play Services for AR to unlocks augmented reality experiences.
Open source 3D face tracking vs proprietary software
Google provides its tracking as an open-source solution meaning you, as a developer, can use it for free. The proprietary face tracking software comes on paid terms meaning you need to buy the licence.
So, why to pay?
In the context of apps, 3D face tracking usually stands as the core technology to enable specific functionality.
With face tracking, you can implement face AR experiences, camera effects or face analytics in your app. But you don’t need an app that only detects the square of the face, right? Moreover, you want your app to be accessible for as many users as possible.
From a business perspective, implementing such experiences involves significant development and design efforts.
Open source technology fails to fulfill them while the proprietary software provides them out-of-the-box.
5 advantages of Banuba’s 3D face tracking software over ARCore
1. Cross-platform coverage
Even though Android devices are currently the most popular out there, with an astonishing 81.7% share of the global market, cutting off iOS devices narrows down the potential audience reach.
ARCore face tracking works only on Android while Banuba covers both Android an iOS also giving developers the maximum user coverage without any coding hassles.
2. Broader device reach
ARCore face tracking is supported only on mid and top-level devices with an average price of $350. Banuba’s face tracking supports low-end $100-devices being the only face tracking software able to run even on Android Go. It significantly increases the audience outreach.
3. Power and capacity saving
To augment a user, you need to develop a separate renderer component which is a timely and costly endeavor. Banuba’s 3D renderer includes the most powerful and feature-rich user modification capabilities and it’s shipped with ready-to-deploy packages and examples.
To enable the ARCore performance, an additional package needed which takes additional space on a user device. Our face tracking is lightweight and integrates seamlessly as a mobile SDK or API.
The open-source software usually requires significant customization and optimization efforts not to mention the design costs for AR content creation. Banuba has all that’s needed to enable a quick MVP launch including the ready to use AR filter packs.
4. 3D Rendering
From a technical perspective, Google’s face tracking is great. It’s super fast being able to detect faces instantly. With different angles and camera rotation, the mask sticks well allowing for the seamless end-user experience.
5. Time to market
However, from a business perspective, ARCore has a couple of serious drawbacks i.e. no iOS support and narrow device coverage.
We were quite satisfied to see our face tracking software for Android being on par with Google’s ARCore. Banuba's 3D face tracking has shown better performance with occlusion, low lightning and camera rotates.
From the business side, our face tracking software has a range of advantages. Brands and developers benefit of broader audience reach, quality performance on low-end devices and ready-to-deploy features that enable the quick face AR app launch.
