Introducing the Video Background Subtraction API for Web
With millions of people switched to video meetings, background removers have escalated in demand. To make virtual backgrounds available on the web, we introduce the Video Background Subtraction API. You can integrate it into your web apps to let users remove, change, blur and animate backgrounds in real-time. Discover how webcam background removal works, its use cases and advantages.
Real-time Background Subtraction and Removal In Web Apps
The API is an in-browser solution for video background replacement. It provides real-time background subtraction letting users remove, change and animate backgrounds in interactive Web AR applications.
Real-time background subtraction using web camera
The background subtraction library builds on a convolutional neural network and machine learning algorithms trained to segment the humans from the background in videos captured by a camera.
Users don't have to download an app to use the feature. The solution allows implementing cross-platform background remover experience accessible via any browser. It works automatically and does not require a physical green screen. The API is a perfect match for video communication, broadcasting, video conferencing and live streaming on the web interface.
Compatibility with Face AR
Among the Background Removal API advantages, it’s fully compatible with Face AR features, i.e. facial effects, filters and beautification. We use a face tracking neural network which recognizes the presence of the human face in real-time. With this, users can add realistic or fun AR masks to enrich their video communication.
Background removal and face filter
Video background removers solve privacy concerns letting hide everything behind the users. To make them feel comfortable about their appearance, you can add face touch up filters to virtual backgrounds.
Background removal and face beautification
For video processing and effect rendering, we use OpenGL shaders where the challenge is to render 3d graphics realistically without artifacts. It’s also important to keep it lightweight for fast inference and small download size.
To solve that, we provide a pack of AR filters optimized for the web. We also work on bringing video textures to bring the most immersive face AR filters and animated backgrounds.
Background Remover API: Technical Features
To deliver real-time background subtraction on the web, we optimized machine learning algorithms for low power consumption and extensive device coverage.
Apart from Chrome and Firefox, it supports Safari and WebGL 1.0 starting from SDK v1.0 allowing for a cross-platform virtual background experience. It can be integrated in any web AR application or platform.
The background remover API is supplied with SIMD (single instruction multiple data) that processes multiple data with a single instruction which dramatically improves performance. If the browser supports SIMD, it'll be used automatically to enhance performance. If the browser goes without SIMD support, a regular, lighter version of the background subtraction library is used.
If you want face filters or beautification applied together with backgrounds in your app, you'll need face tracking, which implies additional weight.
The webcam background remover supports up to 1,8 m which fits most of the video communication scenarios.
180 cm on MacBook Pro (13", 2017)
Video background removal perfectly fits video conferencing apps and can be integrated together with video calling SDK providers such as Agora. Virtual backgrounds embedded in video chat apps enhance the communication experience. Combined with face filters and beautification, they can add a lot of fun too.
Users can change backgrounds while broadcasting from their web camera. By integrating the background remover feature, developers can help content creators to produce professional and high-quality live stream videos.