Blog
Unity

Real-time Unity Background Subtraction: Full Guide

Unity is the world's most popular cross-platform game engine.

However, developers still face a range of limitations when it comes to building face tracking games and apps in Unity.

One of them is the ability to separate a person from the background in real-time.

In this post, you'll learn how to create a background Unity subtraction featuring:

- Why is Unity background segmentation beneficial

- 4 use cases for background subtraction

- How to subtract background in Unity with Banuba.

unity background subtraction banuba

Why Background Subtraction in Unity

Being the most popular cross-platform game engine, Unity still provides limitations.

One of them is the real-time video background subtraction which is crucial for building face tracking games and apps in Unity. 

Our background remover is designed specifically for a selfie-focused experience. It works accurately with all skin tones and hides whatever is behind the person in real-time.

Our Face AR SDK for Unity now supports background segmentation along with face detection and tracking, allowing users to replace backgrounds in real-time.

Power Your App with AR Face Filters SDK  Start Free Trial

What's more, the background changer is compatible with face filters and supports multi-face tracking.

You can build cross-platform games with face tracking, video chats, and augmented reality software in Unity or even use this feature for mixed reality projects.

You can change the user background into a green screen, and replace it with background images, animated wallpapers, or videos per your choice.

unity background subtraction banubaBackground Subtraction in Unity by Banuba

The existing Unity chroma key tools select the color to be removed, e.g. green used in physical green screens. Developers need to avoid colors that match the skin tone so it won’t blend into the background.

Our Unity background segmentation is based on the neural network trained to recognize whether a pixel belongs to the class 'person' or 'background', enabling automatic real-time background separation.

Unity Background Segmentation: Technical Features

We tested how background subtraction works with other Face AR features in Unity both on early and latest devices (iOS, Android) and the results are pretty good.

Unity for iOS, FPS

  iPhone 5s iPhone Xr
Background 21 30
Background + Face Filter 18-20 30
Background 2 faces 16-19 29-30

Unity for Android, FPS

  Galaxy A 50 Galaxy S 10+
Background 24-30 29-31
Background + Face Filter 17-19 29-31
Background 2 faces 25-29 29-30

4 Background Subtraction Use Cases 

The demand for virtual backgrounds has grown fast in recent months, becoming a number one feature for any face tracking and video chatting app.

Here are 4 common use cases when Unity background subtraction becomes a must-have feature for your app.


Games with face tracking

You can build face-to-face games in Unity where players can see each other or themselves while playing. Players may have fun changing backgrounds, applying face filters, or even full-face avatars if they don’t want to show their faces. 

Hyperspeed AR video chat examplesHyperspeed | Face Tracking Case Study


Video chat apps 

Video chat apps are gaining more popularity. By adding Face AR features, you can make the user experience more immersive and fun, allowing users to change boring backgrounds into scenery per their choice.


Live streaming

Virtual backgrounds for streaming apps bring digital privacy and comfort, letting users not bother about their real surroundings. Users can easily create high-quality streams teleporting themselves into the center of the video featuring their favorite game, movie, or just abstract animation.


Unity Chroma Key (Green screen) 

Our background subtraction technology can be perfectly used as an alternative to Unity Chroma Key tools to create mixed reality and augmented reality videos and experiences.

Developers can record the user without the background and then overlay video animation to get high-quality video clips.

Power Your App with AR Face Filters SDK  Start Free Trial

FAQ

What is the purpose of background subtraction?

The purpose of background subtraction is to detect and separate moving objects in video or image processing. The most common use cases are AR-powered face tracking apps, video chat apps, and live streaming. 

What is background subtraction algorithm in computer vision?

The background subtraction algorithm in computer vision means detecting moving objects in video content. It separates objects from the background utilizing so-called classes which identify the background and input images.

Top