Blog
Web AR

Introducing Background Removal And Replacement For Webcam

With millions of people switched to video meetings, background removers have escalated in demand. To make virtual backgrounds available on the web, we introduce the background removal library for a webcam. You can integrate it into your web conferencing apps to let users remove, change, blur and animate backgrounds in real-time without a physical green screen. Discover how webcam background removal works, its use cases and key features.

Video Background Subtraction hero image

Real-time background removal software for webcam

Our webcam background removal comes as a feature of Web AR SDK which you can test now right in your browser. You can use it to allow for Zoom-like video background replacement in your web app. The library provides real-time background subtraction letting your users remove, change and animate backgrounds in interactive web-based applications, online meetings and video chats. 

Real-time background subtraction using web cameraReal-time background subtraction using web camera

The background subtraction library builds on a convolutional neural network and machine learning algorithms trained to segment the humans from the background in videos captured by a web camera. 

Users don't have to download an app to use background replacement. With our SDK you can build a cross-platform background remover experience accessible via any browser. It works automatically and does not require a physical green screen. Webcam background replacement is a perfect match for video communication, broadcasting, video conferencing and live streaming on the web interface.

Compatibility with Face AR

The background removal feature is fully compatible with Face AR, i.e. facial effects, filters and beautification. Our Web AR face tracking technology which recognizes the presence of the human face in real-time. With this, users can add realistic or fun webcam face filters to enrich their video communication.

Background removal and face filterBackground removal and face filter

Virtual backgrounds solve privacy concerns letting users hide everything behind them. To make them feel comfortable about their appearance, you can add face touch up filters to virtual backgrounds.

Background removal and face beautificationBackground removal and face beautification

For video processing and effect rendering, we use OpenGL shaders where the challenge is to render 3d graphics realistically without glitches. It’s also important to keep it lightweight for fast inference and small download size. 

To solve that, we provide a pack of AR filters optimized for the web. We also work on bringing video textures to bring the most immersive face AR filters and animated backgrounds.

Technical Features

To deliver real-time background subtraction on the web, we optimized machine learning algorithms for low power consumption and extensive device coverage. 


Real-time performance

Apart from Chrome and Firefox, it supports Safari and WebGL 1.0 starting from SDK v1.0 allowing for a cross-platform virtual background experience. It can be integrated in any web AR application or platform. 

  • Firefox: 22-30 FPS (camera), 40-45 FPS (background model)
  • Chrome: 30 FPS (camera), 50-60 FPS (background model)
  • Safari: Webcam 20 FPS (camera), 40 FPS (background model)

Size

The webcam background remover is supplied with SIMD (single instruction multiple data) that processes multiple data with a single instruction which dramatically improves performance. If the browser supports SIMD, it'll be used automatically to enhance performance. If the browser goes without SIMD support, a regular, lighter version of the background subtraction library is used. 

If you want face filters or beautification applied together with backgrounds in your app, you'll need face tracking, which implies additional weight.

  • Background Subtraction: 8.4 mb (SIMD), 7.3 mb (regular)
  • Background Subtraction + Face Tracking Neural Network: 12.8 mb (SIMD), 11.6 mb (regular)

Distance

It supports up to 1,8 m which fits most of the video communication scenarios.

  • 180 cm on MacBook Pro (13", 2017)

Application fields


Web conferencing

Video background removal perfectly fits video conferencing apps and can be integrated together with video calling SDK providers such as Agora. Virtual backgrounds embedded in video chat apps enhance the communication experience. Combined with face filters and beautification, they can add a lot of fun too. 


Live streaming

Users can change backgrounds while broadcasting from their web camera. By integrating the background remover feature, developers can help content creators to produce professional and high-quality live stream videos.


E-learning

Virtual backgrounds make the educational process more immersive. It can recreate a classroom environment to eliminate distraction or help teachers present material more effectively.

How to add background replacement to your web app 

The trend for custom backgrounds is unlikely to wear off, so developers look to add this feature to their apps. Here’s how you can integrate the background replacement into your web app. 

  1.  Request Web AR SDK by filling our website form.
  2.  Get the trial token.
  3.  Download the face effect example. The background removal feature comes as part of the effect along with beautification and AR makeup.
  4.  To call the background subtraction, use the API methods as described.

Want to build immersive web AR apps with face filters, beauty and virtual backgrounds? Test our SDK possibilities!

Start Free Trial

Top