Blog
Blog arrow right Face Tracking arrow right From MediaPipe Prototype to a Production Face AR App

From MediaPipe Prototype to a Production Face AR App

MediaPipe is a good place to start a face feature, and it is rarely where the feature ships. Banuba Face AR SDK is a real-time, on-device face tracking and AR effects SDK that runs at 30 FPS on mid-range mobile hardware with a -80° to +80° head-angle tracking range. That is the layer a prototype is missing: Google's free, Apache-2.0 licensed library puts 478 landmarks on a live camera feed in an afternoon, while production asks for beauty, backgrounds, effect authoring, eight platforms and a support contract. In this article, we will describe what a MediaPipe prototype actually gives you, what production adds on top, and the steps to move the tracking layer across.

MediaPipe Prototype to Production: Face AR Guide
Stay tuned Keep up with product updates, market news and new blog releases
Thank You!

[navigation]

TL;DR:

  • MediaPipe Face Landmarker gives you 478 3D landmarks, 52 blendshape scores and facial transformation matrices, which is a tracking layer, not an effects engine;
  • Google labels MediaPipe Solutions an early release, publishes no pricing page and no SLA, and directs support to Slack, GitHub issues and Stack Overflow;
  • Banuba Face AR SDK tracks 68 facial anchor points, builds a 3D mesh of up to 3,308 vertices and tracks 37 mesh morphs, with makeup, hair color, beautification, segmentation and triggers already written against that mesh;
  • MediaPipe publishes Face Landmarker implementation guides for Android, Python and Web, while Banuba builds and maintains iOS, Android, Web, Unity, Flutter, React Native, Windows and macOS itself;
  • the migration is a swap of the tracking and rendering layer inside your existing camera pipeline, not a rewrite of the app;
  • budget for three production items MediaPipe does not cover: effect authoring, license tokens, and a named person to escalate to.

Methodology

What was compared: the published capabilities of MediaPipe Face Landmarker against Banuba Face AR SDK, for the job of shipping a face AR feature in a product. MediaPipe facts come from Google's own MediaPipe Solutions documentation and the google-ai-edge/mediapipe repository. Banuba facts come from the Banuba Face AR SDK documentation and Banuba product pages. Both sets were checked on 16 November 2026. What was not tested: side-by-side frame rates on the same handset, landmark accuracy, and skin-tone fairness. Google publishes no FPS benchmark and no fairness evaluation for Face Landmarker, so no head-to-head performance number appears below. Hands-on testing remains essential.

What the prototype gives you

MediaPipe Face Landmarker is a bundle of three models: a face detection model, a face mesh model, and a blendshape prediction model. The face mesh model outputs an estimate of 478 three-dimensional face landmarks. The blendshape model predicts 52 blendshape scores, which are coefficients representing facial expressions. The task also returns facial transformation matrices for effects rendering.

That is a lot for free.

It runs in three modes: IMAGE for a single image, VIDEO for decoded video frames, and LIVE_STREAM for a camera feed, where results come back asynchronously through a result listener. Processing happens on the device, and MediaPipe does not send that input data to Google servers. The face detector is the BlazeFace short-range model, described by Google as lightweight and optimized for mobile GPU inference.

So the prototype answers one question well: where is the face, and what is it doing. It does not answer the next one.

What production adds

Firstly, effects. Landmarks and matrices are coordinates. Someone still has to write the lipstick shader, the hair strand painting, the virtual background, and the morph rig, and then keep them stable while the user turns away from the camera. Banuba Face AR SDK ships those as prefabs against its own mesh: makeup (foundation, concealer, contour, highlighter, blush, lipstick, lip liner, eyeshadow, eyelashes, eyeliner, eyebrows, lip gloss, glitter), background separation, skin, hair, eye, lips and brow segmentation, hair strands painting, face morphing, LUTs and avatars. Effects are scripted in JavaScript and authored in Banuba Studio.

img_masks-min-Oct-07-2021-07-46-35-36-AMBanuba's face filters example

Secondly, platform coverage. MediaPipe publishes Face Landmarker implementation guides and code examples for Android, Python and Web, and its Solutions availability table carries four platform columns: Android, Web, Python and iOS. Banuba Face AR SDK ships to eight platforms from one integration: iOS, Android, Web, Unity, Flutter, React Native, Windows and macOS. Banuba builds and maintains all eight, including the Flutter and React Native plugins, so there is no community wrapper in the chain.

Thirdly, the commercial layer. Google publishes no pricing page for MediaPipe and no vendor SLA; the repository directs users to a Slack community, GitHub issues, and Stack Overflow. The Solutions documentation still carries the label "MediaPipe Solutions Preview is an early release". Google also ended support for the MediaPipe Legacy Solutions on 1 March 2023, folding Face Mesh and Iris into Face landmark detection, which is the kind of event a roadmap has to survive.

None of this makes MediaPipe a bad choice. It makes it a different product.

  MediaPipe Face Landmarker Banuba Face AR SDK
Platforms with published guides Android, Python, Web (Solutions table also lists iOS) iOS, Android, Web, Unity, Flutter, React Native, Windows, macOS
Languages Tasks APIs, plus C++, Android and iOS Framework examples Swift, Objective-C, Kotlin, Java, C++, C#, JavaScript (TypeScript types provided)
Licence Apache License 2.0 (repository LICENSE) Commercial license, activated by an SDK token
Deployment On device; input data is not sent to Google servers On device; every frame processed on the user's device
Pricing No pricing page published by Google Custom quote, scales with monthly active users, no public price list
Face model 478 3D landmarks, 52 blendshapes 68 facial anchor points, 3D mesh up to 3,308 vertices, 37 mesh morphs
Multi-face num_faces defaults to 1; landmark smoothing is applied only when num_faces is 1 Up to 5 faces tracked with acceptable quality on most mobile devices
Ready-made effects Not published as part of the task Makeup, hair color, beautification, segmentation, morphing, triggers, LUTs, avatars
Support Slack community, GitHub issues, Stack Overflow Support and consultancy included with the license
Published performance Not publicly documented on the Face Landmarker task page 25 to 30 FPS single-face tracking on Android, 30 FPS on iOS (lab conditions)

The shortest route from a mediapipe prototype to production face AR is to keep the camera pipeline you already have and swap the tracking and rendering layer underneath it.

Get Face Filters Order Now

How to move it across

Get a token. An SDK token activates the licensed functionality. A demo token is valid for 14 days, Banuba's standard trial period, and activates all SDK features so you can assess them inside your own project. A commercial token is valid throughout the prepaid period. Banuba recommends storing the token on the server, because that speeds up renewal; a token stored in the app means a new App Store or Play Market upload every time it is renewed. Do not ship a demo token in a live app. Full detail sits in the token management guide.

Install the package. iOS takes the BanubaSdk SPM or CocoaPods packages, Android takes the custom Maven repository plus the SDK dependency in build.gradle.kts, Web takes the @banuba/webar NPM package, and Flutter takes flutter pub add banuba_sdk. The per-platform steps are in the getting started guide.

Replace the tracker, keep the frames. On iOS, the pattern is: set banubaClientToken, initialize BanubaSdkManager, create a Player, and load an effect. On Web, it is: set the client token, import Webcam, Player, Effect, and Dom from @banuba/webar, then initialize the player and apply the effect. Your camera capture, your UI, and your upload path do not have to change.

Keep the landmark coordinates if your feature needs them. If the prototype used raw landmarks for measurement rather than for rendering, Banuba exposes the coordinates too. On iOS, you add a BNBFrameDataListener to the player and read getLandmarks off BNBFrameData, which returns an array sized 2 * (landmarks number) in camera coordinates that you transform to screen coordinates. The face landmarks guide has iOS, Android, and Web versions.

Check the requirements against your install base. Banuba Face AR SDK needs OpenGL ES 3.0 and higher, Android 8.0 (API level 26) and up, and iOS 15.0 and up. Desktop needs OpenGL 4.3 and up (4.1 for macOS), Windows 8.1 and up, or macOS 12.0 and up. On the web, it runs in any browser with WebGL 2.0 and higher. The current list lives on the system requirements page.

Trim the bundle before you ship. The SDK adds around 15 Mb to your app, depending on the feature set enabled. On the web, the heavy files are BanubaSDK.wasm (12 Mb original, 3.5 Mb gzipped, 2.5 Mb with Brotli) and BanubaSDK.simd.wasm (13 Mb, 3.8 Mb, 2.7 Mb). The WebAR SDK is tree-shakable, so use named imports, and it loads the SIMD build automatically where the browser supports it. The optimization guide covers both.

That's it. Five steps, and the prototype logic you already wrote stays where it is.

Explore Banuba's Face AR SDK now  Learn more

What to expect on device

Banuba publishes its own lab numbers, measured under fixed conditions, which means they are a reference point and not a promise for your handset. Single-face tracking runs at 25 FPS on a low-end Android device and 30 FPS on a high-end one, at 30 FPS on both mid and high-end iPhones, across an 80-degree angle range. Multi-face tracking handles up to 5 faces, at 22 to 27 FPS on Android and 30 FPS on iOS. Effects that only need face tracking, which covers face filters, avatars with action units, beautification, and makeup without lipstick, run at 25 to 29 FPS on Android and 30 FPS on iOS. Lips coloring on the web runs at 30 FPS in Chrome and 26 FPS in Safari.

The tracker is built for real conditions rather than for a lit studio: it holds across the -80° to +80° head-angle range and works with up to 70% facial occlusion.

Many factors move these numbers, including the state of the device and other apps running in the background, so each feature should be validated on your target devices.

Limitations worth knowing

Banuba has its own rough edges, and it is better to read them now than to find them in QA.

Safari pauses the web stream. From Safari 15.3, MediaStreams obtained from canvas.captureStream(), which WebAR's MediaStreamCapture uses internally, are paused when the browser tab is not visible. A participant who minimizes or hides the tab freezes for everyone else on the call, and there is currently no known workaround. The known issues page tracks it.

Web memory needs housekeeping. A dangling Player instance will not be collected by the browser garbage collector. Call Player.destroy() when the player is no longer needed.

Cross-platform video calls need a native plugin. For Flutter and React Native, every video-call solution requires a native plugin or module. Banuba developed one for Agora, and the Agora Extension covers the common case, but a different video platform means native work.

Tokens expire loudly. Once a token expires, the Banuba watermark and screen blur appear automatically. For one month after expiry, the SDK still works with a watermark; after that, the functionality stops.

MediaPipe's documented limits are different in kind: landmark smoothing only applies when num_faces is 1, output_face_blendshapes and output_facial_transformation_matrixes both default to False and must be enabled explicitly, the three confidence thresholds all default to 0.5, and the Tasks APIs send metrics about API performance and utilization in your app to Google, with the integrator responsible for obtaining informed user consent where the law requires it.

When MediaPipe is the right answer

Keep MediaPipe if the feature is the landmarks. A liveness check, a head-pose gate, a research pipeline in Python, an attention estimator, a hobby project: none of those need an effects engine, and paying for one would be a waste. Keep it too while you are still proving that users want the feature at all. The Apache-2.0 license and the on-device processing make it a reasonable long-term home for that class of work.

Move when the roadmap says effects, when the second platform appears, or when someone asks who to call at 2 a.m.

What the move looks like in production

b.stage, the SaaS fandom platform run by bemyfriends, uses Banuba's beauty effects and face filters in its live streaming and video chat. In two years, it surpassed 1 million monthly active users from 224 countries. The live-video side of that integration runs through the documented Agora Extension, which is the same pattern described above: existing pipeline, swapped effects layer.

If you want the reverse view of the same decision, our Banuba Face AR SDK vs MediaPipe comparison lines the two up feature by feature, Best Face Landmarks SDKs Compared widens the field to the rest of the market, and How to Implement Face Landmarks Detection Using an SDK stays on the coordinates themselves. For the integration seen from the other end, our engineers wrote up the six steps from token to first effect on Android and iOS.

Conclusion

MediaPipe and a commercial face AR SDK are complementary rather than competing: one proves the feature, the other ships it. The verdict for a team with a working prototype and a product deadline is to keep the camera pipeline, swap the tracking and rendering layer, and spend the saved months on the feature instead of on the mesh. And you will get 14 days on a demo token to check that claim inside your own app before anything is signed.

Power your app with Face AR SDK  Get free trial

Reference list

  • MediaPipe Solutions guide, Google: https://developers.google.com/edge/mediapipe/solutions/guide
  • MediaPipe Face Landmarker task, Google: https://developers.google.com/edge/mediapipe/solutions/vision/face_landmarker
  • google-ai-edge/mediapipe repository: https://github.com/google-ai-edge/mediapipe
  • MediaPipe license: https://github.com/google-ai-edge/mediapipe/blob/master/LICENSE
  • Banuba Face AR SDK technical specification: https://docs.banuba.com/far-sdk/tutorials/capabilities/technical_specification/
  • Banuba minimum Face AR SDK size FAQ: https://www.banuba.com/faq/minimum-face-ar-sdk-size
FAQ
  • Yes. Banuba Face AR SDK ships to iOS and Android from one integration, and Banuba builds and maintains both, along with Web, Unity, Flutter, React Native, Windows, and macOS. The platform minimums are OpenGL ES 3.0 and higher, Android 8.0 (API level 26) and up, and iOS 15.0 and up. Working per-platform code lives in the Banuba Android samples repository.

  • For Flutter and React Native, the question to ask is who maintains the plugin. Banuba builds and maintains its own Flutter and React Native plugins rather than relying on a community wrapper, so there is no third party in the dependency chain. The Flutter plugin installs with flutter pub add banuba_sdk and is published as the banuba_sdk package on pub.dev. One caveat: for video calling on either framework, a native plugin or module is required per video platform, and Banuba supplies one for Agora.

  • Both MediaPipe and Banuba run in the browser. Banuba's browser build works wherever WebGL 2.0 and higher is available. It installs from the @banuba/webar NPM package. You can try it online in your own browser before writing any code, and the starter project is on GitHub as quickstart-web. Note the Safari limitation above if the browser build has to feed a video call.

  • MediaPipe is distributed under the Apache License 2.0, and Google publishes no pricing page for it. Banuba does not publish a price list either: a Banuba Face AR SDK license is quoted for the project and scales with monthly active users, and the figure is fixed inside the contract year. The reasoning behind that model is set out in the Banuba Face AR SDK pricing guide, and a demo token gives 14 days of full-feature access before any commitment. Teams that want the scoping conversation first can request a demo instead.

  • They can, and for a staged migration that is often the sensible path: keep the MediaPipe pipeline serving the analytical feature it already handles, and bring Banuba Face AR SDK in for the rendered AR surface. The two process frames on device and do not depend on each other. Running both does add two model bundles to the download, so most teams consolidate once the AR surface is the primary feature. The feature matrix for deciding what moves is on the Banuba face filters SDK page.

  Face AR SDK Face tracking, virtual backgrounds, beauty, effects & more Start  free trial
Top