How Face AR SDK Evolved Within 2019: Key Features & Milestones

We thank each member of the Banuba team for their awesome job. Together we make great things happen! We thank each of our clients for choosing us as a Face AR technology provider. Our best joy to see you succeed with amazing augmented reality apps empowered with Face AR SDK. We thank our users for the warm feedback, your 5-star ratings and amazing photos and videos you share with the Banuba apps. When you create, smile and feel inspired - our mission comes true.

Beauty AR SDK

We added more features to our Beauty AR SDK which can be implemented in a variety of beautification apps, selfie editors, video streaming services and real-time chats as well as in virtual makeup try on apps for realistic cosmetics representation. The features include:

• Eye bag removal. We trained the neural network to detect and remove the bags under the eyes the way the human would do it.

  

• Acne removal. The AR beauty neural network recognizes the presence of skin defects and retouches them in the context of the user's face.

  

• Hair recoloring. The technology allows users to change the hair colors in real time and see it live like in the mirror.

• Lips recoloring. We can enable realistic lipstick try on as a separate item or with full face makeup looks.

  

• Eye/Iris recoloring. The technology changes the color of the eyes from realistic color lenses or magic eyes, e.g. of a beast.

  

• Bokeh effect. The technology blurs the background in a sophisticated way giving your users a true selfie-focused experience.

  

Face Filters SDK

We evolved our Face Filter SDK for our clients could engage with their users through amazing face filter apps like Snapchat, creative AR ad campaigns and augmented reality experiences on mobile and web.

• The best face filter catalog in the world. Within the past years we’ve designed over 1,000 face filters with all things you can imagine. If you’d like to purchase our filters or have a custom design service for a quick app launch, contact us via a website form.

  

• A collection of post-processing effects (LUT). We designed color processing photo filters like Instagram has that your users can apply to previously taken videos.

  

• Action Units. With the support of Action Units and Blend Shapes rendering, you can create live emojis and animated avatars.
  
  

• Touch events. You can create gamified face filters that react to use taps, e.g. users can catch AR objects by tapping or swiping on the screen.

  

• Glasses detection. You can automatically turn off the virtual glasses in face filters when the user wears real glasses keeping the rest part of the animation.

  

  

Unity face tracking plugin for Face Filter SDK

We added the Unity support to Face Filter SDK that brings our 3D face tracking technology cross-platform and allows developers to create realistic and complex face filters, augmented reality effects, and 3D masks in Unity.

The plugin is a perfect out-the-of-the-box engine to bring face filters, 3D masks, effects and avatars in Unity projects like augmented reality apps, games or photo booths

Technology improvement

We tuned face detection and tracking algorithms, resolved a dozen of performance issues and optimized Face AR SDK components making it even more lightweight and speedy.

• ARKit face tracking integration. By integrating ARKit face tracking into Banuba Face AR SDK, we significantly improved the Face AR experiences on iOS devices. Faster face detection and more stable face tracking results in a quality performance at different lighting conditions, head movements, face angles and reduced jitter.

• Neural network-based face detection. We did some important updates to improve the speed of face detection and achieve wider angles support. As a result, the quality of the face filter experience significantly improved.

• Neural networks for Windows. We’ve extended the face augmenting technology availability adding some cool neural networks for Windows. Now it includes neural networks for Background, Hair, Skin, Occlusion, Eyes, Lips allowing to detect, track modify and recolor different parts of the human face with stable 30 FPS performance on most iOS, Android devices and Windows OS.

Check the full Face AR SDK technical specification for more details. There are still challenges we look to meet in 2020. So, stay tuned.

Documentation

And finally, we rolled out the first version of Face AR documentation aimed to help you get started with our products and make the best use of our platform. We appreciate your feedback, and we’re working on improvement adding more programming guides and tutorials about the AR effect construction.

See http://docs.banuba.com

Thank you, 2019!

It was a great year, and we look ahead bringing you more Face AR features in future. We thank 2019 for the challenges we were excited to work on and for the happiness we shared seeing the results.

We wish you all a Merry Christmas and Happy New Year. Be happy, wealthy, and creative in 2020! 🎄