While they might be convenient, online meetings bring their own set of problems. For example, the room where the person is might be untidy, contain something embarrassing, or give out clues about the person’s location, making it easier to doxx them. Whatever the case, background subtraction is in more demand than ever before. In this article, we will explain how this technology works, where it is used, and how you can integrate it into your app.
How background subtraction works
This is how this is done in Banuba products.
The background subtraction software is based on neural networks. They are taught using a dataset of over 200K photos that include pictures of men and women of all skin colors, in good and awful lighting conditions (dark or too bright), and with both low-end and high-end cameras. As a result, the networks can reliably detect people and automatically separate them from the background, even if the photos or videos are dim or flared. And the dataset is getting regular expansions to further improve the technology’s effectiveness.
The neural networks were built specifically for each platform: iOS, Android, Windows, Mac, and Web. This is done to achieve the best performance possible on every compatible device. The general user experience is essentially the same on both the simplest and most advanced devices: even on iPhone 7, the system is able to keep up 30 fps for at least an hour of non-stop work without lags or overheating. On the latest devices, this can reach a whopping 300 fps, as the technology won’t load the hardware by more than 10% of its capabilities. The better the hardware, the better the system works with corner cases (e.g. in low light or high occlusion conditions).
Another important feature is the patented anti-jitter algorithms that keep the image stable even if the user’s hands are shaking.
Banuba Virtual Background
Background subtraction use cases
These are the applications that include the background eraser feature most often.
Videoconferencing
As proven by thousands of people, from office workers to priests, online meetings can often be embarrassing. Family members and pets intruding onto the video feed used to make news, now it has become a common occurrence.
Video background subtraction can be a simple way to address that. The camera will perceive only the human in the foreground, with everything else in its field of view replaced by a picture, a 3D environment, or just blurred enough to be unrecognizable.
Dating apps
Video dating has skyrocketed during the pandemic and has become a new norm. Bumble, for example, reported an 85% increase in the use of its video chat feature, and 65% of Hinge users who tried this, intend to use this functionality further.
However, privacy is still an issue there, perhaps even more so than in other apps due to stalking. Replacing the background will limit the number of clues available to any potential stalker and keep the users safe. Not only this improves the user experience, but it can also lead to higher retention rates.
Social networks
Background subtraction is not just a way to enhance security. It can also be a tool for self-expression. This is most evident in social media based on videos, like TikTok or Chingari. The more video editing options are available, the better the content that users can create.
Video editing
Mobile video editors today can give users studio-like power on regular smartphones, and video background subtraction is an important part of it. It can replace chroma key (green screen) and allow custom backgrounds to be added, or let people transfer themselves to exotic places, and even to other planets. The possibilities are immense.
Banuba Virtual Background
How you can implement background subtraction in your app
There are two main ways you can approach this problem.
The first one is building your own background subtraction system from scratch. It is entirely possible if you have a solid R&D team with experience in computer vision and neural networks. This approach will give you complete creative and technical control over the way your software will work. Moreover, it could increase the valuation of your company as the technology could be a valuable intellectual property asset.
However, it is also a lengthy and expensive process. Expect to invest hundreds of thousands of dollars and at least six months in development - and that’s just the minimum viable product version.
The second option is getting a premade SDK with background subtraction and replacement. This is a premade module that could be easily built into your app and perform the necessary functions. The integration process can be done within a day.
On the one hand, by using a third-party system you are giving up some degree of control over how certain functions work. On the other hand, it is much quicker and cheaper.
Selecting a video background eraser SDK is not an easy task. This is what you need to pay attention to:
- Platform (Android, iOS, Win, Mac, Web…) and device compatibility;
- Overall feature set (many SDKs contain other functionalities besides the background replacement);
- Performance;
- Real-time background subtraction capabilities;
- Types of replacement background (blurred, static picture, GIF, video, 3D environment);
- Customer service quality;
- Trial period conditions (e.g. Banuba offers 14 days of free trial);
- Price and pricing options.
Banuba Virtual Background
How background subtraction benefits your app
Apps with video calls
- Improves user experience
- Increases user retention
- Helps you distinguish yourself from competitors
Short video apps
- Adds an easy way to improve clips
- Attracts content creators
- Helps retain users longer
- Adds new monetization options
Conclusion
Background subtraction and replacement is a useful technology that can improve user experience, add a layer of privacy, or be a tool for self-expression. Based on specialized neural networks, it can work with great efficiency on many devices. And with a relevant SDK it can be quickly added to a mobile, web, or desktop application - try ours for free and see for yourself!
