Blog
Virtual Background

Video Background Subtraction In a Nutshell

While they might be convenient, online meetings bring their own set of problems. For example, the room where the person is might be untidy, contain something embarrassing, or give out clues about the person’s location, making it easier to doxx them. Whatever the case, background subtraction is in more demand than ever before. In this article, we will explain how this technology works, where it is used, and how you can integrate it into your app.

How background subtraction works

This is how this is done in Banuba products.

The background subtraction software is based on neural networks. They are taught using a dataset of over 200K photos that include pictures of men and women of all skin colors, in good and awful lighting conditions (dark or too bright), and with both low-end and high-end cameras. As a result, the networks can reliably detect people and automatically separate them from the background, even if the photos or videos are dim or flared.  And the dataset is getting regular expansions to further improve the technology’s effectiveness.

The neural networks were built specifically for each platform: iOS, Android, Windows, Mac, and Web. This is done to achieve the best performance possible on every compatible device. The general user experience is essentially the same on both the simplest and most advanced devices: even on iPhone 7, the system is able to keep up 30 fps for at least an hour of non-stop work without lags or overheating. On the latest devices, this can reach a whopping 300 fps, as the technology won’t load the hardware by more than 10% of its capabilities. The better the hardware, the better the system works with corner cases (e.g. in low light or high occlusion conditions). 

Another important feature is the patented anti-jitter algorithms that keep the image stable even if the user’s hands are shaking. 

background subtraction 1Banuba Virtual Background

Background subtraction use cases

These are the applications that include the background eraser feature most often. 


Videoconferencing

As proven by thousands of people, from office workers to priests, online meetings can often be embarrassing. Family members and pets intruding onto the video feed used to make news, now it has become a common occurrence. 

Video background subtraction can be a simple way to address that. The camera will perceive only the human in the foreground, with everything else in its field of view replaced by a picture, a 3D environment, or just blurred enough to be unrecognizable. 


Dating apps

Video dating has skyrocketed during the pandemic and has become a new norm. Bumble, for example, reported an 85% increase in the use of its video chat feature, and 65% of Hinge users who tried this, intend to use this functionality further.

However, privacy is still an issue there, perhaps even more so than in other apps due to stalking. Replacing the background will limit the number of clues available to any potential stalker and keep the users safe. Not only this improves the user experience, but it can also lead to higher retention rates


Social networks

Background subtraction is not just a way to enhance security. It can also be a tool for self-expression. This is most evident in social media based on videos, like TikTok or Chingari. The more video editing options are available, the better the content that users can create.


Video editing

Mobile video editors today can give users studio-like power on regular smartphones, and video background subtraction is an important part of it. It can replace chroma-key (green screen) and allow custom backgrounds to be added, or let people transfer themselves to exotic places, and even to other planets. The possibilities are immense.

background subtraction 2Banuba Virtual Background

How you can implement background subtraction in your app

There are two main ways you can approach this problem.

The first one is building your own background subtraction system from scratch. It is entirely possible if you have a solid R&D team with experience in computer vision and neural networks. This approach will give you complete creative and technical control over the way your software will work. Moreover, it could increase the valuation of your company as the technology could be a valuable intellectual property asset. 

However, it is also a lengthy and expensive process. Expect to invest hundreds of thousands of dollars and at least six months in development - and that’s just the minimum viable product version.

The second option is getting a premade SDK with background subtraction and replacement. This is a premade module that could be easily built into your app and perform the necessary functions. The integration process can be done within a day. 

On the one hand, by using a third-party system you are giving up some degree of control over how certain functions work. On the other hand, it is much quicker and cheaper. 

Selecting a video background eraser SDK is not an easy task. This is what you need to pay attention to:

  • Platform (Android, iOS, Win, Mac, Web…) and device compatibility; 
  • Overall feature set (many SDKs contain other functionalities besides the background replacement);
  • Performance;
  • Real-time background subtraction capabilities;
  • Types of replacement background (blurred, static picture, GIF, video, 3D environment);
  • Customer service quality;
  • Trial period conditions (e.g. Banuba offers 14 days of free trial);
  • Price and pricing options.

background subtraction 3Banuba Virtual Background

How background subtraction benefits your app


Apps with video calls

  • Improves user experience 
  • Increases user retention
  • Helps you distinguish yourself from competitors

Short video apps

  • Adds an easy way to improve clips
  • Attracts content creators
  • Helps retain users longer
  • Adds new monetization options

Conclusion

Background subtraction and replacement is a useful technology that can improve user experience, add a layer of privacy, or be a tool for self-expression. Based on specialized neural networks, it can work with great efficiency on many devices. And with a relevant SDK it can be quickly added to a mobile, web, or desktop application - try ours for free and see for yourself!

Start Free Trial

Top