Blog
Face AR

Getting Started With Creating Face-based AR Experiences

From entertainment AR masks to realistic virtual try-ons allowing to preview cosmetics and accessories in real-time, AR face filters find its applications in many mobile and web apps. With our Face AR platform, you can make the most versatile face-based AR experiences letting users transform themselves into anyone, turning selfie-taking into a storytelling adventure. This post will introduce you to the Face AR platform and types of face-based AR experiences you can create for mobile and web.

creating face based ar experience banuba

What Is Face AR?

Face AR Platform is a set of tools and libraries to enable face-based AR experiences and facial animation in face tracking apps. AR face recognition technology detects faces in a camera and overlays virtual content or modifies the face in real-time e.g. change hair color or apply cosmetics. It can be used to:

  • Animate day-to-day communication and expression with personalized avatars and emojis, AR masks and face filters.
  • Let consumers virtually try on makeup, glasses, jewelry at home to see how products look
  • Develop creative advertising and marketing campaigns enabling consumers to create and share branded content.
  • Edit, smarten up and express users’ identity in selfies and stories.
  • Build cross-platform AR mobile apps and Unity games with Snapchat-like face filters.
  • Empower social networking apps with AR video editing features helping users to generate content easily.

Face AR turns a camera into a lifestyle companion bringing warmth to the digital landscape and making the impersonal personal – through the human face.

Explore

How Face AR platform empowers developers

Face AR platform provides developers with three core components for building face-based AR experiences:

  • Augmented Reality SDK
  • Effect Constructor & Viewer
  • Face Filter Catalogue

Augmented reality SDK includes technologies and libraries enabling a certain type of face augmentation experience - AR Face Filters, Beautification or Avatars. Each of them is a cross-platform SDK with AR face recognition, rendering engine and neural networks at its core to allow adding face-based AR experiences to any app (for Android or iOS) and website or build new apps from scratch.

Effect Constructor and Viewer tools allow developers and designers to configure and preview AR face filters before uploading to production.

Face Filter Catalogue facilitates and speeds up the process of creating AR content. Developers can buy ready filters or customize them for their audience. For 4 years we’ve created over 1,000 AR face filters with more than a quarter of a million monthly active users. We’ve included some of them into our face filter catalogue for developers to view, create, personalize and purchase them to delight their users.

Face-based AR experiences immerse users more deeply into developers’ apps. They increase the engagement, freedom and enjoyment of the camera experience allowing users to create interesting, fun content in a breeze.

How Developers Can Create AR Filters For Web and Mobile

With the release of Facebook Spark AR and Snapchat lens Studio, AR face filter creation has become accessible to the masses, and it has even fueled the emergence of a new profession - AR creators. Still, developers can’t use their masks outside Snapchat or Facebook and have little control over their content.

These tools are a perfect choice for individual developers, designers, or social media influencers. Face AR platform is designed for development teams and brands who want to launch their own face tracking solution.

Unlike Snapchat and SparkAR, with Face AR platform developers can integrate AR face recognition and face filters in their apps, games or websites. To get started creating your face-based AR experiences with Face AR platform you need to:

1. Install Face AR components

Get in touch with us to get links to downloading SDK, Effect Constructor and Viewer components. Before purchasing the license you have a free trial period to validate our SDK performance. 

2. Explore effect samples

Our Face AR SDK is delivered with examples featuring SDK applications for each platform and a variety of use cases giving developers a comprehensive overview of how to run and use it. View the full list of SDK features and try the functionality in demo apps for iOS and Android

3. Define your filter concept

Think of your USP (unique selling points) and app concept. Why will people go to your face filter app over the others? Think about how your filters will differ and bring value to users. If you nail down your killer filter concept, your app will go viral, and you’ll succeed.

4. Prepare materials and textures

You may use any 3D graphical editor like Maya to design the visual concept of your effect and import them in .fbx format.


Also Read: How To Create Your Own Face Filters Like Snapchat In Banuba Studio


 

5. Upload source files into Effect Constructor

Our Effect Constructor converts files into the supported formats so that you further be able to configure and run your effect.

6. Preview your effect with a Viewer

The Viewer component allows running the constructed effects in real-time on desktop before uploading them to the app. To preview effects, you just need to drag'n'drop the constructed effect folder into the Viewer window.

7. Test and optimize

Once you’re done with effect configuration and viewing, make sure to test your face filter performance on multiple devices and platforms before uploading them to the live app.

What Face-based AR Experiences You Can Create

As the purpose and the audience of your app vary, so our Face AR platform allows for creating different types of face-based AR experiences. These possibilities make our Face AR a great fit for a broad range of apps and use cases, from live chats and video streaming services, social networking and dating apps to professional makeup and skincare brands.


3D Animation

animated textures

AR face filters with 3D animation is one of the most popular types of face-based AR experience in entertainment apps. You can animate 3D geometry making objects move on a given trajectory. The animation can play continuously in a loop, according to a specific scenario or enabled with triggers.


Background change


background change face filters banuba

You can separate a user on the foreground from the background in a video to remove the background, replace it with another graphical image or add a video file. Above entertainment, the background subtraction technology that stands behind these filters find its usage in video chats and video conferencing solutions. It increases privacy and allows for safe and comfortable communication.


Face Touch Up

beauty-1

You can create beautification or face touch up effects with skin smoothing, eyes and teeth whitening, "expressive eyes" effect, color correction and skin tone adjustment (darker or lighter). You can use it as a standalone filter in live streaming, social networking, dating apps or video chats to enhance the user camera experience or as part of face filters adding animation, virtual accessories or entertainment masks.


Also Read: How To Create Face Beautification Filter In Effect Constructor


Virtual Makeover

lips gloss facear 26 release

Makeup try on filters allow users to test cosmetics products virtually via a mobile camera. Unlike face beauty filters, virtual makeup try ons feature realistic representation of AR content (lipstick, eye shadows, hair color, etc.) without modifying the use face for beauty purposes but rather helping people find best-fit makeup products. 


Realistic Try On: Glasses, Jewelry, Hats

physics banuba

You can design AR models that behave like the real objects in the flow of real-world light, e.g. support gravity or mirror the light with the camera rotates and user head tilts. Physics property allows for realistic 3D model behaviour which is essential for virtual try on apps. 


Multiple Faces

Multi

With multiple face detection, you can apply face filters to several users where one effect is applied to each face recognized in the camera, or different effects are applied to several faces simultaneously. AR filters for multiple faces are extremely popular in magic-mirror photobooth solutions and kids apps to add fun and engagement of enjoying the AR experience together with friends and family.


Triggers

banuba spider effect gif (1)

You can enable AR scenarios in face filters with user facial expressions. The user can interact with effects or call them opening mouth, smiling, raising eyebrows or frowning.


Game Face Filters

face filter game banuba gif

AR filters with triggers are best-fit for quick Face AR games or interactive face filters to increase engagement. Users can control AR objects with facial expressions, head movements or touches.


Morphing

Morphing

Morphing allows for modifying the size of the face and its facial features by moving the vertices of the 3D model of the face. You can slim down or enlarge the face, change the shape of the nose, eyes, lips, chin, or produce any other deformations.


Instagram-like LUT Effects

Lut

You can create Instagram-like filters enabling real-time or offline color correction of recorded images.


Live Emojis and Avatars

Avatar

You can animate 3D emojis, avatars or full-face AR masks that support and copy user facial expressions in real-time.

What It Takes To Create Quality Face-Based AR Experience

Good-looking and nice performing face-based AR experiences are the result of multiple efforts from the team and technology implementation. So, what does it take to create professional face filters that keep the audience engaged and inspire social shares?

  1.  Robust technology. Fast and stable AR face recognition optimized for multiple platforms and broad device support is crucial. No matter how creative your AR filter, the users won’t be able to experience it to the fullest unless it functions well on their devices.
  2.  Powerful effect renderer. Gone are the days of primitive 2D face stickers. Now, the AR industry requires creative Face AR experiences similar or even bigger than Snapchat, and our proprietary Graphical Renderer engine allows implementing AR face filters just like that.
  3. 3D art team. Unique and interesting visual concepts and their graphical realization are the result of experienced Art Team who have been creating face AR experiences for 4 years and are constantly testing and discovering what different audiences need.
  4.  Development. Complex effects may require coding part, therefore, knowledge of JS (or any other programming language at the Jr-Mid level) to write a script to control the effect are nice-to-have.
  5.  Optimization and testing. A high-quality AR face filter needs to look visually appealing, run smoothly on a wide range of devices and take as little memory space as possible. AR mask optimization and testing steps allow achieving better performance and higher user satisfaction from your app.
  6.  Regular updates. Dynamic and regularly updated AR content in your app provides better user retention. You always need to experiment bringing new AR experiences to your users.

Summing up

Businesses and developers can capitalize on Face AR experiences to engage with their audience with AR face filters, advertise products, build brand awareness and drive consumers and prospects organically. Creating quality face-based AR experiences can be a science but we're always ready to share our knowledge and experience helping brands to realize their creative thinking and implement it with our Face AR platform. Get in touch if you want to discuss your idea!

Top
# Tags: