Blog
Web AR

Introducing the Video Background Subtraction API for Web

With millions of people switched to video meetings, background removers have escalated in demand. To make virtual backgrounds available on the web, we introduce the Video Background Subtraction API. You can integrate it into your web apps to let users remove, change, blur and animate backgrounds in real-time. Discover how webcam background removal works, its use cases and advantages.

Video Background Subtraction API hero image

Real-time Background Subtraction and Removal In Web Apps

The API is an in-browser solution for video background replacement. It provides real-time background subtraction letting users remove, change and animate backgrounds in interactive Web AR applications. 

Real-time background subtraction using web cameraReal-time background subtraction using web camera

The background subtraction library builds on a convolutional neural network and machine learning algorithms trained to segment the humans from the background in videos captured by a camera. 

Users don't have to download an app to use the feature. The solution allows implementing cross-platform background remover experience accessible via any browser. It works automatically and does not require a physical green screen. The API is a perfect match for video communication, broadcasting, video conferencing and live streaming on the web interface.

Compatibility with Face AR

Among the Background Removal API advantages, it’s fully compatible with Face AR features, i.e. facial effects, filters and beautification. We use a face tracking neural network which recognizes the presence of the human face in real-time. With this, users can add realistic or fun AR masks to enrich their video communication.

Background removal and face filterBackground removal and face filter

Video background removers solve privacy concerns letting hide everything behind the users. To make them feel comfortable about their appearance, you can add face touch up filters to virtual backgrounds.

Background removal and face beautificationBackground removal and face beautification

For video processing and effect rendering, we use OpenGL shaders where the challenge is to render 3d graphics realistically without artifacts. It’s also important to keep it lightweight for fast inference and small download size. 

To solve that, we provide a pack of AR filters optimized for the web. We also work on bringing video textures to bring the most immersive face AR filters and animated backgrounds.

Background Remover API: Technical Features

To deliver real-time background subtraction on the web, we optimized machine learning algorithms for low power consumption and extensive device coverage. 


Real-time performance

Apart from Chrome and Firefox, it supports Safari and WebGL 1.0 starting from SDK v1.0 allowing for a cross-platform virtual background experience. It can be integrated in any web AR application or platform. 

  • Firefox: 22-30 FPS (camera), 40-45 FPS (background model)
  • Chrome: 30 FPS (camera), 50-60 FPS (background model)
  • Safari: Webcam 20 FPS (camera), 40 FPS (background model)

Size

The background remover API is supplied with SIMD (single instruction multiple data) that processes multiple data with a single instruction which dramatically improves performance. If the browser supports SIMD, it'll be used automatically to enhance performance. If the browser goes without SIMD support, a regular, lighter version of the background subtraction library is used. 

If you want face filters or beautification applied together with backgrounds in your app, you'll need face tracking, which implies additional weight.

  • Background Subtraction: 8.4 mb (SIMD), 7.3 mb (regular)
  • Background Subtraction + Face Tracking Neural Network: 12.8 mb (SIMD), 11.6 mb (regular)

Distance

The webcam background remover supports up to 1,8 m which fits most of the video communication scenarios.

  • 180 cm on MacBook Pro (13", 2017)

Application fields


Web conferencing

Video background removal perfectly fits video conferencing apps and can be integrated together with video calling SDK providers such as Agora. Virtual backgrounds embedded in video chat apps enhance the communication experience. Combined with face filters and beautification, they can add a lot of fun too. 


Live streaming

Users can change backgrounds while broadcasting from their web camera. By integrating the background remover feature, developers can help content creators to produce professional and high-quality live stream videos.


E-learning

Virtual backgrounds make the educational process more immersive. It can recreate a classroom environment to eliminate distraction or help teachers present material more effectively.

How to Integrate the Webcam Video Background Removal Into Your App 

The trend for custom backgrounds is unlikely to wear off, so developers look to add this feature to their apps. Here’s how you can integrate the background remover API into your web app. 

  1.  Request Web AR SDK by filling our website form.
  2.  Get the trial token.
  3.  Download the face effect example. The background removal feature comes as part of the effect along with beautification and AR makeup.
  4.  To call the background subtraction, use the API methods as described.

Want to build immersive web AR apps with face filters, beauty and virtual backgrounds? Test our SDK possibilities!

Start Free Trial

Top