Blog
Face AR SDK Release

Face AR SDK v1.4: Better Face Tracking, Eyebrow Makeup, and Developer-Friendly Improvements

Business users would appreciate an even more precise face tracking, and the ability to add makeup to eyebrows. The developers would find many additions that speed up the integration: the SDK is now released as CocoaPods and Maven packages, there is support for M1 simulators and glTF, a new demo scene in Unity, etc. Moreover, there has been a series of improvements to WebAR, so open this article and read the details.

Face tracking banuba

Improved face tracking

We have implemented a new approach to face tracking. Not only does this improve the precision, but it also significantly boosts performance. The most notables change on iOS was RAM consumption - it decreased by 5-7% depending on the device. On Android, this resulted in a massive improvement in FPS - up to 30% (tracking only), depending on the device. 

banuba face tracking New face tracking in action

The new face tracking works across all platforms that Banuba supports: iOS, Android, Web, and Desktop. But it only works in online mode - for now.

Unity: demo scene with the effects carousel

Now using Banuba Face AR SDK in Unity will be easier. Thanks to the demo scene (a basic part of an app - a 3D environment with some assets) with the effects carousel, your dev team will be able to see how everything should work and quickly modify it to suit your needs.

face ar for unityProperly configured scene result

Eyebrow makeup

The toolbox of our Beauty AR SDK has been expanded to include eyebrow makeup. Now your users will be able to paint them with a realistic replica of a real product or a wild color.

AR eyebrow makeupAR eyebrow makeup

This feature works on all platforms: iOS, Android, Web, and Desktop

SDK as CocoaPods and Maven Packages

The SDK versions for Android and iOS will now be distributed as CocoaPods and Maven packages. This will make them significantly easier to integrate.

How the integration works without such packaging:

  1. Download the SDK
  2. Unpack the archive
  3. Copy the framework
  4. Specify dependencies
  5. Build the app
  6. Repeat all the steps above with every new release

How it works now:

  1. Add two lines of code to specify dependencies
  2. Build the app
  3. Repeat step 2 with every new release

Take a look at the new documentation:

The SDK now also supports M1 simulators, glTF, and external textures on Android. Testing the work became easier. 

WebAR updates

There are three main additions to the WebAR version of the SDK.

  • The ability to load Effect via Request
  • The ability to play gif as background texture
  • Effect.preload progress listener. You can help your users see the progress as an effect is being downloaded.

We have also expanded the tutorials section of our WebAR documentation with best practices, examples, and insights, so check it out.

Optimization

  • iOS: Upgraded lips segmentation neural network
  • B&W lips coloring support without additional parameters. Now using black or white colors on lips is simple and straightforward.
  • Makeup API: reduced amount of uniforms. Simply put, this will make makeup effects take up less space.
  • WebAR: Improved WebAR archive UX. A more convenient, trimmed-down version.
  • Rendering Engine: Reduce LUT memory size. LUTs will take use less RAM.
  • OEP: extend supported image formats

Bug fixes

  • Black screen flash on effects loading
  • Android: Memory leak in the OEP
  • Android: Pixel 3a crash after applying certain face filters
  • Android: deadlock during video_frame draw
  • Android: Xiaomi Mi 8 Pro crash
  • iOS: iPhone 13 recognized as middle-end hardware
  • Fixed invisible entities for some face filters
  • Effects API: Api.drawingAreaWidth() & Api.drawingAreaHeight() return 0
  • Windows: Impossible to apply effect with path containing extended Unicode
  • Windows, MacOS: Banuba Viewer saving video without sound
  • MSAA issues in face filters
  • WebAR: Fixed processing of RGB images
  • WebAR: Fixed MediaStreamCapture crash on Safari 14
  • Makeup API: Fixed serialization of empty textures
  • Hand AR: hand skeleton false detections
  • OEP: crash if imput buffer format changes
  • OEP: green frame if texture is not ready
  • OEP: fixed rapid camera switch
Top