How to Build AR Makeup Features Using an SDK

[navigation]

TL;DR

• AR makeup rendering needs three subsystems working together in real time: face tracking, color/material rendering tuned to skin tone, and a product catalog layer. Each block is a multi-month build on its own.
• The hardest problem isn't tracking the face; it's making a virtual lipstick or foundation look correct on every skin tone under uneven lighting. Getting this wrong is the #1 buyer complaint about virtual try-on ("product color not displayed correctly").
• Banuba TINT applies 16 makeup product types with skin-tone-aware application, which directly targets that color-accuracy complaint.
• TINT supports 16+ product categories (makeup, eyewear, hair color, contacts, jewelry, accessories), so a single integration can extend beyond makeup as the catalog grows.
• A web-based SDK ships as a CDN-loaded widget and a custom HTML element, so a first integration can go live in under two weeks rather than the months an in-house pipeline takes.
• An in-house build gives full control over the rendering stack but commits the team to maintaining ML models, browser compatibility, and a digitization pipeline indefinitely.
• Océane, a Brazilian cosmetics brand, integrated TINT and lifted its add-to-cart rate from an industry-average 3% to 32%, a 1000%+ increase.

What does an AR makeup feature actually require?

Before choosing a build path, it helps to be precise about the components. A production AR makeup try-on is not one feature. It's three systems running together at interactive frame rates.

The first is face tracking: locating the face, the lips, eyes, brows, and cheeks frame by frame, and holding that lock as the user turns, talks, or moves into bad lighting.

The second is rendering: drawing a virtual product onto those regions so it reads as real cosmetics rather than a flat color overlay. Lipstick has a finish (matte, gloss, satin); foundation interacts with the underlying skin; eyeshadow blends across a curved lid.

The third is the catalog layer: mapping each real SKU to render parameters so "Shade 04 Warm Nude" looks like that exact product, not a generic tint.

Each of these is independently hard. Together, they're the reason teams underestimate AR makeup by an order of magnitude.

Why is color accuracy the hard part of an AR makeup mirror?

Face tracking is largely a solved problem with mature tooling. Color accuracy is not. The most common failure mode reported by virtual try-on buyers is that the on-screen shade doesn't match the physical product. The experience looks like a fun filter rather than a buying tool, so it doesn't convert.

The difficulty is that the same lipstick has to render believably across a wide range of skin tones and lighting conditions. A naive color overlay washes out on deep skin and looks artificial on light skin. This is why an AR makeup mirror that customers trust requires skin-tone-aware rendering rather than a fixed color blend. Banuba TINT addresses this directly: its 16 makeup product types apply with skin-tone-aware rendering, which is the specific lever that turns a novelty filter into a conversion driver.

How do you build AR makeup from scratch?

The in-house path means owning all three subsystems. A realistic scope looks like this:

• Build or license a face-tracking model, then tune it for the makeup regions (lips, lash line, brow, lid, cheeks) at interactive frame rates across devices and browsers.
• Develop a rendering layer for each finish type and validate it against real product photography on a representative range of skin tones.
• Build a digitization pipeline so the merchandising team can turn a physical SKU into render parameters without an engineer in the loop.
• Maintain browser and device compatibility as Safari, Chrome, and mobile OS versions move underneath you.

This path makes sense when AR makeup is core to the product and the team wants full control of the rendering stack. The honest trade-off is that it's a multi-month initial build plus indefinite maintenance of ML models and a digitization workflow. And this work doesn't stop after launch.

What does the AR makeup SDK path look like?

An AR makeup SDK collapses those three subsystems into an integration task. With a web-based engine like TINT, the work is front-end integration rather than computer vision research:

• The widget loads from a CDN bundle and renders as a custom HTML element on the product page, so it drops into an existing storefront without a separate app.
• Isolated-SKU mode focuses the experience on a single product for product-detail pages, with a graceful fallback to a model photo when a shopper declines webcam access.
• White-labeling lets the widget match the brand's site, so it doesn't read as a third-party bolt-on.
• A merchant ID controls which categories and features are enabled, so the same integration can expand from makeup to other supported categories over time.

Because TINT supports 16+ product categories (makeup, eyewear, hair color, contacts, jewelry, and accessories), a team that integrates once for lipstick and foundation can later switch on glasses or jewelry without a second integration. The practical result is a first launch measured in weeks rather than months.

The trade-off runs the other way from the in-house build: less control over the internals of the rendering stack, in exchange for not having to build or maintain it.

Banuba's TINT makeup virtual try-on in action

How fast can an AR makeup try-on go live?

For a web integration, the gating work is loading the widget bundle, placing the element on the relevant pages, choosing when it opens (on load or on a button click), and connecting the product catalog through the merchant configuration. A focused team can have a first version of AR makeup running in under two weeks, which is why most beauty brands evaluating build-vs-buy land on the SDK path for the initial release.

Océane, a Brazilian cosmetics manufacturer, started with exactly this kind of limited rollout: foundation and concealer only, and saw its add-to-cart rate rise from an industry-average 3% to 32%, a 1000%+ increase, before expanding to more categories.

Integration references

For implementation, the relevant starting points are the Banuba SDK documentation and the open integration samples on GitHub for Android, iOS, and web. These cover setup and minimal working examples for the beauty/makeup use case.

Conclusion

Building an AR makeup well means solving three hard problems at once: face tracking, skin-tone-accurate rendering, and catalog mapping. The rendering problem, in particular, is what separates a try-on that converts from a novelty filter. Building it from scratch is viable for teams that want to own the stack and can fund the maintenance; for everyone else, an AR makeup SDK turns a multi-month research project into a front-end integration that ships in weeks. If you're weighing the two paths, the fastest way to judge fit is to try the engine against your own product range.