How to Build AR Beauty Features Using an SDK

[navigation]

AR beauty features rely on real-time face tracking, neural networks for skin and lip segmentation, GPU-accelerated rendering, and color-accurate shaders that simulate how cosmetics interact with light and skin. Building all of that in-house can take six to twelve months of specialist engineering work. A beauty AR SDK like Banuba packages tracking, segmentation, makeup APIs, and rendering into a single integration, which lets product teams ship working features in weeks rather than years.

TL;DR

• AR beauty drives real commerce results: the virtual try-on technology market is expected to grow from $12.09 billion in 2025 to $15.29 billion in 2026 at a 26.5% CAGR, with beauty as one of the fastest-moving verticals.
• Building from scratch means owning face tracking, segmentation models, GPU shaders, and a multi-platform rendering pipeline. That is rarely the right use of a product team's runway.
• A beauty AR SDK like Banuba unites those layers into a single integration, with tested support for iOS, Android, Web, Unity, Flutter, and React Native.
• SDKs make sense when speed to market matters, when AR is a feature within a larger product, or when the team lacks a dedicated computer vision group.
• Banuba's Beauty AR SDK ships makeup, beautification, skin retouch, and color analysis features that have been deployed by Gucci, Schwarzkopf, Looké, and Boca Rosa.

Why Beauty AR Features Succeed in Apps

Three Banuba-powered products show how the same underlying capability translates into very different commercial outcomes. Boca Rosa, an influencer-led Brazilian cosmetics brand, used Banuba's TINT virtual try-on at a pre-launch event and recorded 1.7 million try-on sessions and roughly $900,000 in sales in 4 hours. Looké, an Indonesian niche brand, built a full virtual makeup app and crossed 55,000 installs with mostly 5-star reviews. FaceYoga by Mental Growth used the same SDK to power a "before and after" wellness feature and reached 20,000+ downloads.

None of those teams invented the technology. They all relied on the same primitives. What separates a strong AR beauty feature from a forgettable one is consistent across all three:

• Real-time response. Anything above roughly 100 ms of latency between head movement and overlay update breaks the illusion.
• Lifelike color. Lipstick has to read as matte or glossy. Foundation has to blend with skin tone, not paint over it. Highlighter has to react to light.
• Stable tracking under real conditions. People move, tilt their heads, hold their phones at strange angles, and use bad lighting.
• A clear next action. The AR layer is rarely the goal. The goal is buying, sharing, calling, or streaming. The AR has to get out of its own way.

What AR Beauty Features Have to Do

Most "core capabilities" lists for AR beauty are flat checklists. That hides where the real work lives. A more useful way to think about it is to follow what happens between the moment a user opens the camera and the moment a virtual lipstick lands on their lip. Each step in that chain has its own engineering cost, and each step is something a team will either build or inherit from an SDK.

Step 1. The camera frame arrives. The first job is to identify the face within the frame and determine whether it is one face, several faces, or no face at all. This is the detection layer. It has to run on every frame on devices ranging from a recent iPhone to a $150 Android, without dropping below interactive frame rates.

Step 2. The face is tracked, not just detected. Detection alone is not enough. The system has to know how the face is oriented in 3D space, where the lips and eyes are, how the head is rotating, and whether the user is partially occluded by hands, glasses, or hair.

Step 3. Skin, lips, and hair are separated from everything else. A lipstick has to apply to lips, not to the area around them. A hair color has to apply to hair, not to a shirt collar. This is segmentation, and it is where neural networks earn their keep. Each surface (skin, lips, hair, eyes, teeth, neck) needs its own trained model.

Step 4. The cosmetic is rendered with realistic light behavior. This is the part most in-house builds underestimate. A matte lipstick should look matte. A satin lipstick should pick up a soft highlight. A foundation should respect the underlying skin tone instead of painting over it. A highlighter should react when the user turns toward a light source. This requires shader work that goes well beyond a color overlay, and it must be tuned for each finish type.

Step 5. Beautification effects run alongside, not instead. Skin smoothing, acne removal, eye bag reduction, teeth whitening, and face morphing are usually applied at the same time as makeup.

Step 6. Output goes to wherever the product needs it. Sometimes, that is a live preview on the user's screen. Sometimes it is a recorded video for a social post. Sometimes it is a photo for an e-commerce review. Sometimes it is a video call frame routed through Agora, Zoom, or a custom WebRTC stack. The integration layer has to plug into all of those without rewriting the rendering pipeline each time.

Step 7. The whole chain runs in real time on the user's device. This is the constraint that ties everything together. AR beauty cannot send a video to a server and wait for a response. It has to run locally, fast enough that head movement does not lag, and on a thermal and battery budget the user will tolerate.

A team that builds AR beauty in-house signs up for every step in that chain. A team that adopts an SDK signs up for steps 1, 2, 3, 4, and 5 to be solved already, and spends its engineering effort on steps 6 and 7 in the context of its specific product.

In-house Build vs. Integrating a Beauty AR SDK

Building In-house

Building AR beauty internally is technically possible. Several teams do it, usually because they have specific IP requirements or a dedicated computer vision group. The realistic shape of that project looks like this.

Tech stack and components

• Native frameworks (Swift/Metal on iOS, Kotlin/OpenGL ES or Vulkan on Android)
• Cross-platform layers (Flutter, React Native, or Unity if multi-platform support is required)
• A face tracking model trained on a balanced dataset across skin tones, ages, and lighting conditions
• Segmentation models for skin, lips, and hair
• A custom GPU shader library for cosmetics and skin retouch
• A LUT and color management system
• An asset pipeline for digitizing cosmetic products
• A device compatibility matrix and a thermal/battery testing harness

Risks

• Computer vision research time. Tracking on mobile under poor lighting and occlusion is a hard problem, and getting it stable usually takes multiple iterations.
• Performance regressions on mid-range and low-end Android devices. AR beauty has to work on phones that cost $150, not just flagships.
• Color accuracy on diverse skin tones. Models trained on narrow datasets fail in production. Building a balanced dataset is expensive.
• Maintenance cost. Every iOS or Android release can break camera and GPU behavior. New device form factors arrive every quarter.
• Time to market. Six to twelve months is a typical floor before an in-house beauty AR feature is production-ready, and that is before any commercial use cases are added on top.

Integrating a Beauty AR SDK

The SDK option trades a recurring license cost for a much shorter critical path. The team integrates a tested binary with a documented API, plugs it into the existing product, and spends engineering time on the parts that are actually unique to the business: UX, commerce flow, asset library, brand customization.

What a beauty AR SDK gives a team

• Pre-built face tracking that has already been validated across thousands of devices
• Pre-trained neural networks for skin, lip, and hair segmentation
• Pre-built shaders and effect libraries for makeup, hair color, and skin retouch
• Cross-platform parity, so iOS, Android, and Web behave the same
• A documented integration flow with code samples and platform plugins
• A vendor team that maintains the underlying tech as devices and OSes change

Tradeoffs

• License cost replaces internal headcount cost. For most teams the math favors the SDK, but it should still be modeled.
• Customization happens through the SDK's effect editor and API, not by editing model weights.
• Some highly specific use cases may need vendor R&D collaboration.

The honest version of the tradeoff is that an SDK gives a team about 90% of the AR beauty capability for roughly 10% of the engineering effort. The remaining customization is done through the SDK's tools.

In-house vs. Beauty AR SDK Compared

SDK-focused Implementation: Banuba Beauty AR SDK

Banuba's Beauty AR SDK packages real-time face beautification and virtual makeup try-on into a single integration. It runs on iOS, Android, Web, macOS, Windows, Unity, Flutter, and React Native, and replaces the parts of an AR beauty stack that most teams should not be building themselves: face tracking, segmentation models, makeup rendering, and beautification effects.

The capability set covers both sides of AR beauty:

Makeup

• Eyeshadow with adjustable color and transparency
• Eyeliner for upper or lower eyelid
• Eyelashes of any length and color
• Blusher on upper cheeks
• Highlighter on nose, forehead, and chin
• Contour on lower cheeks and upper forehead
• Lipstick across matte, satin, glossy, and shimmer finishes

Beautification

• Skin smoothing and wrinkle softening
• Skin tone enhancement
• Face morphing for cheeks and nose
• Teeth whitening
• Expressive eyes effect
• Acne removal driven by a feature recognition neural network
• Eye bag and neck smoothing through dedicated neural networks
• Color correction via LUTs

The SDK is built on Banuba's Face Tracking Software, which uses a 3D face model rather than a flat landmark map. The tracker tracks 68 facial points on the visible face mesh and runs alongside Banuba's patented anti-jitter mechanism for smooth, stable overlays. The face tracker supports 360 degrees of device rotation, works in low-light environments, handles up to 70% facial occlusion, and detects faces from up to 7 meters away.

Recent releases push the SDK further into AI-driven personalization. Banuba's automated seasonal color analysis feature categorizes individuals into seasonal color palettes based on the tone of their skin, hair, and eyes, helping users identify which shades enhance their natural features. In December 2025, Banuba announced significant upgrades to its Face AR SDK, introducing improvements to the virtual background feature and launching a new face shape detection capability, both of which feed directly into the personalization logic that beauty apps rely on.

The commercial track record is concrete. Companies leveraging Banuba's Beauty AR have reported higher add-to-cart rates, higher sales, and lower return rates. For example, Oceane’s add-to-cart rate rose from 3% to 32% after integrating Banuba’s solution.

Integration overview

At a conceptual level, integrating the Beauty AR SDK into a product looks like this:

1. Get a license and client token. A 14-day free trial covers all SDK features.
2. Add the SDK to the build. iOS via CocoaPods or SPM, Android via Maven, Web via the @banuba/webar npm package, Flutter via the banuba_sdk pub package, React Native via the @banuba/react-native npm package, or Unity via the Unity plugin.
3. Initialize the SDK with the client token and connect it to the camera input or video stream.
4. Call the relevant API.
5. Customize. Banuba Studio is a web-based editor for building effects compatible with the SDK. Existing 3D assets can be imported and adapted.
6. Ship. The SDK ensures cross-platform parity, so the same effects render consistently on iOS, Android, and the Web.

Full implementation guides, platform-specific requirements, and code samples live in the official documentation:

• Integration documentation
• LLM-friendly docs (for AI-assisted development):
• GitHub repositories with working samples

For teams who want to read more around the topic, related Banuba pieces worth a look include the AR in the beauty industry overview.

A practical decision framework

When teams ask whether to build or buy AR beauty, the answer usually falls into three categories.

1. Is AR beauty the product, or a feature inside the product? If it is a feature, an SDK is almost always the right call. The product team's time is better spent on the surrounding experience.
2. Does the team have an in-house computer vision and GPU shader group? If not, the SDK route avoids the multi-quarter ramp required to hire and ship that capability.
3. Does the use case need IP-differentiated tracking or rendering? If yes (and this is rare outside of vendors selling AR tech itself), a custom build may be justified. If no, an SDK is faster and lower-risk.

For beauty brands, e-commerce platforms, video and streaming apps, social and dating products, and wellness apps, the answer is consistently the same: use an SDK and spend the engineering budget on the parts of the product that customers actually pay for.

Conclusion

AR beauty has matured from a marketing experiment into a measurable layer for commerce and engagement. The technical requirements (real-time tracking, multi-class segmentation, accurate shading across skin tones, cross-platform parity) are the same regardless of whether the use case is virtual makeup try-on, video call beautification, or a wellness "before and after" experience. Most product teams should not be building all of that themselves.

A beauty AR SDK collapses six to twelve months of specialist engineering into a few weeks of integration work, and shifts maintenance onto a vendor whose full-time job is keeping the underlying technology current.

If AR beauty is on the roadmap, the practical next step is to start a 14-day free trial, integrate the SDK into a prototype, and measure against the team's existing baseline.

References

Banuba. (2025, March 20). Banuba enhances AI makeup recommendations based on a proprietary seasonal color analysis in TINT virtual try-on platform. Business Wire. https://www.businesswire.com/news/home/20250320071573/en/Banuba-Enhances-AI-Makeup-Recommendations-Based-On-a-Proprietary-Seasonal-Color-Analysis-in-TINT-Virtual-Try-On-Platform

Banuba. (2025, April 23). Banuba launches Shopify AR try-on plugin. Business Wire. https://www.businesswire.com/news/home/20250423465644/en/Banuba-Launches-Shopify-AR-Try-On-Plugin

Banuba. (2025, December 22). Banuba enhances Face AR SDK with superior virtual backgrounds and face shape detection. Business Wire. https://www.businesswire.com/news/home/20251222329858/en/Banuba-Enhances-Face-AR-SDK-with-Superior-Virtual-Backgrounds-and-Face-Shape-Detection

Banuba. (n.d.). AI Beauty AR API SDK | Makeup beautification retouch filters. https://www.banuba.com/facear-sdk/beauty-ar

Banuba. (n.d.). Face Tracking Software. https://www.banuba.com/technology/face-tracking-software

Banuba. (n.d.). Virtual try-on by Banuba helps beauty brand earn $900,000 in 4 hours. https://www.banuba.com/blog/virtual-try-on-helps-beauty-brand-earn-900.000-in-4-hours

The Business Research Company. (2026). Virtual try-on technology global market report 2026. https://www.thebusinessresearchcompany.com/report/virtual-try-on-technology-global-market-report

Intel Market Research. (2026, January). Virtual makeup try-on market outlook 2026–2032. https://www.intelmarketresearch.com/virtual-makeup-try-on-market-22056

Mordor Intelligence. (2025, September 30). Virtual try-on market size, share & 2030 trends report. https://www.mordorintelligence.com/industry-reports/virtual-try-on-market