Face AR SDK v0.28: Face Tracking Optimized For Older iPhones
Our Face AR SDK release 0.28.0 brings new, optimized AR face tracking for older iPhone devices that don’t have a TrueDepth camera. The update improves AR Face Tracking iOS performance allowing to achieve better distance support, no jitter, more stable angles, accuracy with lighting and glasses on iPhone 6s and higher. Read on to learn how it impacts user experience in face tracking apps for iOS and check our test video to see the difference.
AR Face Tracking for iOS [Backstory]
Not just long ago Apple released the 3rd generation of iPhones with the TrueDepth camera. You can learn more about the True Depth camera from the Apple’s video on Face Tracking with ARKit.
The new camera contains a dot infrared projector which detects and tracks your face by mapping 30,000 invisible dots. Above biometrics, initially seen as a prime purpose of this update, the TrueDepth camera and face tracking with ARkit unleash way more powerful and interesting use cases of Face AR experiences. It recognizes even tiny facial movements and emotional expressions, allowing developers to convert them to live avatars and emojis or overlay augmented reality face filters with stunning accuracy.
However, face tracking with ARKit impacts only a new generation of iPhones with the TrueDepth camera including iPhone X, XS, XR, or iPad Pro. As for older iPhones which include 6 plus, 6s, 7, 8 and older, face tracking imposed some limits to Face AR experiences and required more “efforts” from users in terms of face position and lighting conditions.
Still, many iOS users don’t have the TrueDepth camera and therefore developers could not realize all the potential of AR face tracking apps to the fullest while users can’t experience all its benefits. It especially matters for sophisticated face AR experiences like face-based AR games, realistic virtual try ons or complex AR face filters which millions of people love and use daily.
To help solve that, we’ve optimized our iOS Face Tracking SDK for better distance support and more stable performance with different lighting conditions. See the video above to estimate the difference or contact us if you’d like to try our new iOS face tracking for developing your Face AR apps.
iOS Face Tracking SDK Performance Improvements
The data below provides our rough test estimates of AR face tracking iOS performance and can be used for your reference only but not as final data for guaranteed results when used in your apps. We provide the trial period to all our technology for you to assess face tracking performance deliverables within your iOS apps.
Distance. AR face tracking supports 2.5-3 meters average.
Process photo time. Avg 0,5 seconds for most technologies for the front camera.
Angles. Wider face angles with 90-degree support and camera rotate.
Occlusion. Faster face detection and more accurate face tracking with sunglasses and glasses.
Lighting. More stable AR face tracking with bright sun and street light.
Performance. Min 30 FPS on iPhone 6s and higher.
Other Face AR SDK Improvements
Updated switcher for full body segmentation neural networks (iOS)
Banuba Viewer UI changes
Switch Action Units to use new face recognition algorithm (iOS)
Switch triggers to use Action Units (iOS)
Physics behavior in effects on devices with ARKit (iOS)