Blog
Blog arrow right Web AR arrow right How to Build an AR App Using an AR SDK

How to Build an AR App Using an AR SDK

To build an AR app, you either engineer the full computer-vision stack in-house or integrate an AR SDK that provides face tracking, rendering, and effects out of the box. Banuba Face AR SDK is a real-time, on-device face tracking and AR effects SDK that runs at min 30 FPS on mid-range mobile hardware with an -80° to +80° head-angle tracking range. That choice usually decides whether an AR feature ships in weeks or turns into a multi-quarter research project, because the hard part of AR is not the effect on screen; it is the tracking and rendering pipeline underneath it. The build-versus-buy line therefore turns on a single question: is real-time face tracking your product's core differentiator, or a capability you need shipped, benchmarked, and maintained without owning the computer vision behind it?

How to Build an AR App Using an AR SDK
Stay tuned Keep up with product updates, market news and new blog releases
Thank You!

[navigation]

TL;DR

  • The engineering cost of an AR app sits in the tracking and rendering pipeline, not the visible effect; an AR SDK removes that pipeline as a thing you have to build and maintain.
  • Banuba Face AR SDK reconstructs a 3D face mesh with up to 3,308 vertices using direct 3D reconstruction, rather than converting 2D landmarks to 3D, which is what keeps tracking stable under stress.
  • Banuba Face AR SDK holds tracking through up to 70% facial occlusion, across a -80° to +80° head-angle range, and through 360 degrees of device rotation.
  • One Banuba Face AR SDK integration covers iOS, Android, Web, Flutter, React Native, Unity, and desktop, so a single build reaches every platform the product ships on.
  • Banuba processes AR fully on-device, with no cloud round-trip, which matters for latency and for privacy commitments.
  • Banuba prices the Face AR SDK with a custom quote rather than a public price list, and the figure is fixed inside the contract year and re-quoted at renewal. A 14-day free trial covers every feature on every supported platform, so pilots can validate performance before any spend.
  • Building the same capability from scratch means owning neural-network training, per-device optimization, and ongoing maintenance across every OS you support.

What does an AR app actually require?

A face-based AR app is four systems working together in real time. It captures the camera feed, detects and tracks the face, renders 3D or 2D content locked to that face, and applies effects such as masks, beauty retouch, or filters, all inside a frame budget of roughly 33 milliseconds to hold 30 FPS.

The visible effect is the easy part. The difficulty is the tracker: keeping a stable lock on the face when the user tilts their head, when a hand or microphone covers part of the frame, in low light, and across the wide range of camera hardware in real phones. A tracker that drifts under these conditions produces effects that slide off the face, which users read instantly as a broken feature.

Should you build AR from scratch or use an SDK?

Building from scratch means assembling a computer-vision pipeline: training or licensing face-detection and landmark models, building a rendering engine that anchors content to a moving 3D face, and then optimizing all of it per device so it holds frame rate on mid-range hardware. It is a specialist effort measured in engineer-years, and it does not end at launch, because each new OS version and device class needs re-testing.

Using an AR SDK collapses that work into an integration. The tracking, rendering, and effects engine is already built, benchmarked, and maintained, so your team spends its time on the product around the feature rather than on the computer vision beneath it. The tradeoff is customization: you work within the SDK's capabilities and its rendering model instead of controlling every layer. For most teams shipping beauty filters, masks, try-on, or conferencing effects, that tradeoff is worth it, because those capabilities are exactly what a mature SDK already covers.

The honest decision rule: build from scratch only if AR tracking itself is your core differentiator and you have computer-vision specialists on staff. Otherwise, integrate.

Explore Banuba's Face AR SDK now  Learn more

How do you build an AR app with an AR SDK?

With an SDK, the build becomes a conventional integration task rather than a research project.

You start from a platform sample. Banuba ships quickstart repositories for iOS, Android, and the other supported platforms on GitHub, so the first working camera-plus-effect runs before you write feature code. You add the SDK package through the native dependency manager for your platform (CocoaPods, Maven, or NPM), initialize it with a trial token, and wire the camera feed into the SDK's processing loop. From there, you load effects and toggle capabilities such as face tracking, beauty retouch, or masks through the API.

Because Banuba develops and supports the same engine across iOS, Android, Web, Flutter, React Native, Unity, and desktop, a cross-platform team integrates once against one feature set instead of maintaining separate native implementations. The full Face AR SDK integration documentation covers each platform's setup path in detail.

Two implementation details matter for architecture decisions. First, Banuba runs entirely on-device, so the camera frames never leave the phone; this removes cloud latency and simplifies the privacy story for regulated use cases. With no camera data leaving the device, there is also less personal data to account for under laws like GDPR. Second, Banuba tracks and affects up to 9 faces on screen at once, which is what group filters and multi-person conferencing effects depend on.

Watch how many SDKs your feature set needs, too. Pairing a face tracking SDK from one vendor with a video editor from another can lead to fights over the camera pipeline and two ML runtimes in one app. Banuba's Face AR SDK plugs into its own Video Editor SDK, so effects carry through to recording and editing on one camera stack.

Where does the AR content come from?

The tracker finds the face. Someone still has to make what sits on it: the masks, filters, and makeup looks users actually see. That work needs 3D artists, shader knowledge, and a review loop for every effect, and teams often notice the gap only after integration, when the effects shelf is empty.

Banuba covers it two ways. The Asset Store holds 1,000+ ready-made effects you can license and load straight into the app. For custom effects, Banuba Studio lets designers build and preview them for the Face AR SDK without writing rendering code.

What can you build with a Face AR SDK?

The clearest test of an SDK is what teams ship on it in production. bemyfriends built b.stage, a SaaS fandom platform that passed 1 million monthly active users in two years across 224 countries, using Banuba beauty effects and 3D masks inside its live streaming and video chat. The relevant point for a build decision is that the AR layer held up at that scale and across that device diversity without the b.stage team owning the computer vision.

The same engine shows up in very different products. Bermuda, a random video chat app, uses Banuba face filters, beauty touch-up, background replacement, and LUTs. The Bermuda app has passed 20 million installs, and its users trigger those AR features more than 15 million times a month. In e-commerce, the same face tracking core drives virtual try-on for makeup, hair color, glasses, and jewelry. Brands that want try-on without an engineering project can use TINT, Banuba's ready-made try-on platform, as a plugin, a no-code setup, or a custom integration. Social and creator apps run masks and stickers on it. Photobooths and AR mirrors use it on event and retail floors.

Banuba's AR face filters example

What should you check before choosing an AR SDK?

Judge the tracker first, because it sets the ceiling on everything else. Ask how the SDK builds its face model. Banuba uses direct 3D reconstruction, building the 3D head model straight from the camera feed rather than inferring 2D landmarks and lifting them to 3D. That is what lets it reconstruct a mesh with up to 3,308 vertices from 68 facial anchor points and hold tracking through 70% occlusion and across a -80° to +80° angle range, where a 2D-to-3D approach tends to drift under the same stress.

Ask what the tracker was trained on as well. A model built on a narrow dataset looks fine in the demo, then slips on darker skin tones, side lighting, or budget front cameras. Test those exact conditions during the trial instead of trusting the spec sheet.

Then check the practical fit: does one integration cover your actual target platforms, does processing run on-device if privacy matters, and does the pricing model match how you grow. Banuba quotes the license per project rather than publishing a price list, holds the figure fixed inside the contract year, and the 14-day free trial covers every feature, so an engineer can benchmark real performance before anyone signs off.

FAQ
  • For nearly all teams, an SDK is faster by a wide margin, because it removes the computer-vision pipeline (tracking, rendering, per-device optimization) that dominates a from-scratch build. Banuba's Face AR SDK ships that pipeline pre-built and benchmarked, so integration replaces a multi-quarter research effort with a standard dependency setup. Building from scratch only makes sense when AR tracking is your core differentiator.

  • It depends on the vendor, and it is worth confirming in writing. Banuba's Augmented Reality SDK runs entirely on-device: camera frames are processed locally with no cloud round-trip, which lowers latency and keeps user video off external servers, an important point for regulated or privacy-sensitive apps.

  • A single integration covers iOS 13+, Android 8.0+, Web, Flutter, React Native, Unity, and desktop (Windows and macOS). Because Banuba maintains one engine across all of them, cross-platform teams build against a single feature set; the per-platform setup is documented in the Banuba Face AR SDK docs.

  • Pricing models vary, so match them to how you scale. Banuba Face AR SDK pricing is a custom quote rather than a public price list, billed yearly, with the figure fixed inside the contract year and re-quoted at renewal. A 14-day free trial covers every feature on every supported platform, so teams can validate performance before committing.

  • The Banuba Face AR SDK adds around 15 MB, depending on the feature set enabled, since a build that uses only face tracking ships lighter than one bundling beauty, masks, and background effects. The current figure and the per-feature breakdown are in the Face AR SDK size FAQ.

  • There is no single benchmark. Cost depends on the feature set, target platforms, and whether the AR layer is built in-house or licensed. In-house, the largest line items are computer-vision engineers and ongoing per-device maintenance. With an SDK, the AR layer becomes a license fee plus integration time. You can explore Banuba’s Face AR SDK pricing options here.

  Face AR SDK Face tracking, virtual backgrounds, beauty, effects & more Start  free trial
Top