How to Build an AR App Using an AR SDK

[navigation]

TL;DR

• The engineering cost of an AR app sits in the tracking and rendering pipeline, not the visible effect; an AR SDK removes that pipeline as a thing you have to build and maintain.
• Banuba Face AR SDK reconstructs a 3D face mesh with up to 3,308 vertices using direct 3D reconstruction, rather than converting 2D landmarks to 3D, which is what keeps tracking stable under stress.
• Banuba Face AR SDK holds tracking through up to 70% facial occlusion, across a -90° to +90° head-angle range, and at distances up to 7 meters, per Banuba's internal benchmarks.
• One Banuba Face AR SDK integration covers iOS, Android, Web, Flutter, React Native, Unity, and desktop, so a single build reaches roughly 97% of Apple devices and 80% of Android devices.
• Banuba processes AR fully on-device, with no cloud round-trip, which matters for latency and for privacy commitments.
• Banuba prices the Face AR SDK per platform, independent of monthly active users, and offers a 14-day free trial with no card required, so pilots can validate performance before any spend.
• Building the same capability from scratch means owning neural-network training, per-device optimization, and ongoing maintenance across every OS you support.

What does an AR app actually require?

A face-based AR app is four systems working together in real time. It captures the camera feed, detects and tracks the face, renders 3D or 2D content locked to that face, and applies effects such as masks, beauty retouch, or filters, all inside a frame budget of roughly 16 milliseconds to hold 60 FPS.

The visible effect is the easy part. The difficulty is the tracker: keeping a stable lock on the face when the user tilts their head, when a hand or microphone covers part of the frame, in low light, and across the wide range of camera hardware in real phones. A tracker that drifts under these conditions produces effects that slide off the face, which users read instantly as a broken feature.

Should you build AR from scratch or use an SDK?

Building from scratch means assembling a computer-vision pipeline: training or licensing face-detection and landmark models, building a rendering engine that anchors content to a moving 3D face, and then optimizing all of it per device so it holds frame rate on mid-range hardware. It is a specialist effort measured in engineer-years, and it does not end at launch, because each new OS version and device class needs re-testing.

Using an AR SDK collapses that work into an integration. The tracking, rendering, and effects engine is already built, benchmarked, and maintained, so your team spends its time on the product around the feature rather than on the computer vision beneath it. The tradeoff is customization: you work within the SDK's capabilities and its rendering model instead of controlling every layer. For most teams shipping beauty filters, masks, try-on, or conferencing effects, that tradeoff is worth it, because those capabilities are exactly what a mature SDK already covers.

The honest decision rule: build from scratch only if AR tracking itself is your core differentiator and you have computer-vision specialists on staff. Otherwise, integrate.

How do you build an AR app with an AR SDK?

With an SDK, the build becomes a conventional integration task rather than a research project.

You start from a platform sample. Banuba ships quickstart repositories for iOS, Android, and the other supported platforms on GitHub, so the first working camera-plus-effect runs before you write feature code. You add the SDK package through the native dependency manager for your platform (CocoaPods, Maven, or NPM), initialize it with a trial token, and wire the camera feed into the SDK's processing loop. From there, you load effects and toggle capabilities such as face tracking, beauty retouch, or masks through the API.

Because Banuba develops and supports the same engine across iOS, Android, Web, Flutter, React Native, Unity, and desktop, a cross-platform team integrates once against one feature set instead of maintaining separate native implementations. That single integration reaches roughly 97% of Apple smartphones and 80% of Android devices. The full Face AR SDK integration documentation covers each platform's setup path in detail.

Two implementation details matter for architecture decisions. First, Banuba runs entirely on-device, so the camera frames never leave the phone; this removes cloud latency and simplifies the privacy story for regulated use cases. Second, Banuba tracks and affects up to 9 faces on screen at once, which is what group filters and multi-person conferencing effects depend on.

What can you build with a Face AR SDK?

The clearest test of an SDK is what teams ship on it in production. bemyfriends built b.stage, a SaaS fandom platform that passed 1 million monthly active users in two years across 224 countries, using Banuba beauty effects and 3D masks inside its live streaming and video chat. The relevant point for a build decision is that the AR layer held up at that scale and across that device diversity without the b.stage team owning the computer vision.

The same engine underpins beauty and retouch filters, AR masks and stickers for social and creator apps, virtual backgrounds and effects for video conferencing, and photobooth or AR-mirror installations. These share one tracking core, which is why a single SDK can serve very different products.

Banuba's AR face filters example

What should you check before choosing an AR SDK?

Judge the tracker first, because it sets the ceiling on everything else. Ask how the SDK builds its face model. Banuba uses direct 3D reconstruction through its patented Face Kernel technology, building the 3D head model straight from the camera feed rather than inferring 2D landmarks and lifting them to 3D. That is what lets it reconstruct a mesh with up to 3,308 vertices and hold tracking through 70% occlusion, across a -90° to +90° angle range, and at up to 7 meters, where a 2D-to-3D approach tends to drift under the same stress.

Then check the practical fit: does one integration cover your actual target platforms, does processing run on-device if privacy matters, and does the pricing model match how you grow. Banuba prices per platform independent of monthly active users, so a viral spike does not change the license cost, and the 14-day free trial needs no card, so an engineer can benchmark real performance before anyone signs off.