A new release of Face AR augmented reality SDK expands the possibilities of our Beauty AR SDK bringing in more portrait retouching possibilities, like eye bag removal and acne removal features. We’ve also improved the Face AR SDK performance on iOS devices adding ARKit face tracking. Here are the updates we’ve made.
Eye bag removal feature
Banuba eye bag removal feature (before/after)
We trained the neural network to detect and remove the bags under the eyes. We analyzed over 100 thousand images to form a dataset which included faces of different skin tones, genders and ages. The algorithm recognizes the skin color of the face and removes the darkness under the eyes in photos according to human visual perception.
Using the active learning followed by the manual correction of the beautification designer, we aimed to implement the technology that removes the bags under the eyes the way a regular user would do it. The eye bag removal feature can be implemented in face retouching software, i.e. photo editors and beauty apps.
Neural network based acne removal
Banuba acne removal feature demo (before/after)
Our acne removal technology is based on the same approach of human visual perception. The neural network recognizes the presence of skin defects and retouches them in the context of the user's face. Similar to the eye bag removal feature, the blemish removal can be applied for photos as a post-processing effect.
The feature is a great add-on to photo editors giving users a possibility to remove significant imperfections from their skin.
ARKit face tracking integration
The TrueDepth camera has come as one of the most exciting Apple features within the past year. It captures accurate face data by projecting and analyzing over 30,000 invisible dots to create a depth map of your face and also captures an infrared image of your face. Introduced as part of FaceID, the TrueDepth camera empowered with ARKit face tracking opens many opportunities for Face AR apps and experiences as well.
By integrating ARKit face tracking into Banuba Face AR SDK, we significantly improved the Face AR experiences on iOS devices. Faster face detection and more stable face tracking results in a better face filter experience with AR masks performing well at different lighting conditions, head movements and face angles and reduced jitter.
The improvement is available for iOS devices with front-facing TrueDepth camera:
-
iPhone X, iPhone XS, iPhone XS Max, or iPhone XR.
-
iPad Pro (11-inch) or iPad Pro (12.9-inch, 3rd generation).
-
iOS 11.0 or later.
-
Xcode 10.0 or later.
Other changes and improvements
-
Eyes state trigger and ruler features in recognizer API -
-
API to change sound volume from Java Script
-
Option to add effect from external folder in sample application (Android)
-
Improve API (SDK for browsers)
Changes
-
Don't reload effect if there was an error in Java Script
Fix
- Decrease memory pressure while creating multiple `BanubaSdkManager` instances (Android)
- Crash on effects with 3 and more faces
- Improved camera FPS on selected low-end Android devices
Banuba Face AR SDK includes several components, i.e. Face Filters SDK, Beauty AR SDK, and Avatars . You can use each component separately to build a standalone augmented reality app or integrate it into your existing app.
Explore how our Face AR SDK can help you to empower your app with some amazing features that your users will love.
