How to Build Face Detection Features Using a Face Detection API

[navigation]

TL;DR:

• A face detection API returns face location and geometry. Face recognition, which establishes whose face it is, is a separate problem built on top of that output.
• Banuba Face API tracks 68 facial anchor points and builds a 3D mesh of up to 3,308 vertices, which is what makes downstream effects, measurements, and morphing stable.
• Banuba's face tracking holds up to 70% facial occlusion and across the full -80° to +80° angle range, so detection survives masks, hands, and off-axis heads.
• On-device detection removes per-request cost and keeps frames on the phone. Cloud face APIs such as Amazon Rekognition and Azure AI Face are the better fit for batch processing and large-scale identity search.
• Banuba Face API runs on Android, iOS, Web, Windows, macOS, Flutter, React Native and Unity, so one license covers a cross-platform product.
• Building a comparable detector in-house means data collection, model training, per-device optimization, and an ongoing accuracy budget, which is why the build-versus-license decision usually turns on whether face detection is your product or a feature inside it.
• Banuba Face API supplies the inputs for a liveness challenge, including head pose, eye openness, gaze, blink, and pulse. The decision logic and any compliance layer stay in your application.
• A 1M-MAU SaaS fandom platform scaled to 1 million monthly active users in two years on Banuba Face AR, which is the kind of load a licensed face layer is expected to carry.

What does a face detection API actually give you?

Detection is the first stage of a pipeline, not the whole pipeline. A face detection API locates one or more faces in a frame and returns their position and geometry. Tracking keeps that geometry locked to the same face across frames. Recognition, a separate step, compares the face to a reference and decides whose it is.

Banuba Face API covers the first two stages. Its face tracking detects 68 facial anchor points and builds a 3D mesh of up to 3,308 vertices, delivering real-time structural data for the nose, eyes, lips, and chin. That mesh is the difference between a bounding box you can draw on and a surface you can attach effects, measurements, or morphs to.

The same layer reads face attributes in real time, including face shape, skin tone, hair and facial-hair style, eyewear detection, and pupillary distance. It also estimates heart rate from facial color variation and camera distance from face-area size, all on the device.

Multi-face handling matters as soon as your camera points at more than one person. Banuba's face detection locates and tracks several faces in the same frame, and on mobile Banuba recommends up to three at once for a smooth experience. The ceiling there is device compute, not the algorithm.

Should face detection run on the device or in the cloud?

This is the architectural decision that shapes everything else, and it is worth settling before you compare vendors.

Cloud face APIs upload a frame, run inference on a server, and return a result. They are strong at batch processing, at large-scale identity search against a database, and at workloads where latency is not user-visible. They also bill per request, so cost scales with usage rather than with revenue.

On-device detection processes the frame locally. Nothing is uploaded, latency is bounded by the handset rather than the network, and the feature keeps working on a bad connection. Banuba Face API runs on-device by default: detection, tracking, and segmentation all execute locally, which is why there is no per-request charge as usage grows. On-device processing is also the single most repeated request in Banuba's own inbound inquiries, usually from teams that need to put a written privacy commitment in front of a customer.

The browser is its own case. The same decision reappears there with different constraints, because the runtime is WebAssembly inside a tab rather than a native SDK, and the frame still never has to leave the machine.

Banuba's face detection and tracking in action 

What does it take to build face detection from scratch?

The from-scratch path is viable, and it is worth pricing honestly before you take it.

You need a training set that covers skin tones, lighting, angles, and occlusion. You need a model architecture and the training budget behind it. You then need per-device optimization, because a detector that hits real-time on a flagship will drop frames on a mid-range Android, and your users are on both. After launch, you own an accuracy budget forever: every new device class, camera, and failure report comes back to your team.

A licensed face detection API moves that work behind a versioned interface. Banuba has spent more than ten years on face detection and tracking technology, with patented tracking algorithms, and ships the results as a library you link against.

The decision rule is simpler than the comparison tables suggest. If face detection accuracy is the product you sell, build it. If it is a feature inside a product that sells on something else, such as a social app, a conferencing tool, a beauty commerce flow, or a verification step, license it and spend the engineering time on the thing customers actually pay for.

How do you integrate a face detection API step by step?

The integration path for Banuba Face API is short because the heavy pieces ship precompiled.

1. Start a trial and get a client token. Banuba offers a 14-day free trial of the Face API.
2. Add the SDK to your project for the platforms you ship on, and initialize it with the token.
3. Feed it the camera stream and read back the detection, tracking, and segmentation output your feature needs.
4. Enable only the modules you use. Licensing is feature-based, which keeps both the binary and the license scoped to what you shipped.

The Face AR SDK documentation carries the platform-by-platform setup guides and the API reference. Working integrations are published as sample projects rather than described in prose: see the iOS samples repository and the Android samples repository, which are the fastest way to see the initialization and frame-handling patterns in context.

Which platforms and devices does it support?

Banuba offers face detection and tracking for Web, Windows, macOS, Android, iOS, Flutter, React Native and Unity, which matters for teams shipping one product across several runtimes on one license.

Device requirements are modest: Android 8.0 or later, or iOS 13 or later, with a 1280x720 camera recommended. Detection and tracking hold min 30 FPS on mid-range mobile hardware, and the SDK selects quality settings for the device automatically, so a broad-install-base app does not need a separate low-end code path.

Robustness is the specification that decides whether a feature survives contact with real users. Banuba's face tracking holds steady under up to 70% facial occlusion and across the full -80° to +80° angle range, so a hand across the chin, a medical mask, or a head turned away from the camera does not drop the track.

Banuba's face detection and tracking API for creating face filters in games

How do you go from detection to face matching and liveness?

Plenty of teams arrive at a face detection API because the roadmap says "verify the user", so it is worth being precise about where the boundary sits.

Banuba Face API supplies the foundation for both 1:1 face match against a reference image and a 1:N database search, and it exposes the signals a liveness check needs: head pose, eye openness, mouth movement, gaze, blink and head-turn triggers, and pulse. Your application issues the challenge and confirms the response. The decision logic, and any compliance layer around it, is yours to build. Banuba's liveness detection guide walks through how active and passive checks combine.

Read that as a scope statement rather than a limitation. Banuba supplies the real-time face layer; identity products, audit trails, and regulatory attestation are a different category of vendor.

What are teams actually building with it?

The common pattern is a face layer that has to hold up at consumer scale rather than an isolated detection demo.

A SaaS fandom platform passed 1 million monthly active users in two years with Banuba Face AR powering beauty effects and face filters inside its live streaming and video chat, with users in 224 countries. Detection there is invisible: it is the thing that keeps a touch-up or a 3D mask attached to the right face while a broadcast runs.

Beyond that, detection output feeds photo auto-tagging, AR effects in social and creator apps, background separation and framing in video conferencing, touchless interfaces in kiosks and photobooths, and the front end of a verification flow.

Conclusion

Face detection is easy to demo and expensive to make reliable. The cost sits in occlusion, angle, lighting, multi-face frames, and the long tail of mid-range devices, which is exactly the work a licensed face detection API absorbs.

If the processing has to happen on the user's device, across several platforms, with predictable licensing and no per-request bill, that is the case Banuba Face API was built for. If you need large-scale identity search over a server-side database, a cloud provider is the better tool, and it is worth saying so before you start the integration.