Blog
Virtual Try-on

Creating AR Nails Try On With Nail Recognition Neural Network

Our recent AR SDK release brings cutting-edge nail recognition technology allowing developers to create virtual manicure apps. Learn how virtual nail painting works and how to build your own AR nail color simulator.

ar nails try on

Engaging customers with AR nails try on 

The nail market has started to recover after the pandemic, and augmented reality is here to help. Offline businesses shift to mobile and attract clients via virtual nail salon apps. Brands, following their successful experience with AR makeup, now aim to replicate one with nail color try on. In both cases, app creators struggle to engage users through quality AR nails experience while speeding up time to market. 

Our augmented reality SDK solves their challenges by bringing realistic and robust nail color simulation to mobile devices. It lets users color their nails into any shade imaginable just by pointing the mobile camera at their hand.

AR nails color simulatorAR Nails Color Demo

The best part, nails color try-on feature is provided as an API meaning developers can easily integrate it with their apps. Now, it supports iOS only, and Android and web are coming soon. You can test it for virtual manicure app development by filling in our contact form.

Start Free Trial

How virtual nail painting works 

Under the hood, virtual nail painting combines several innovative computer vision technologies.


Nails segmentation

For accurate nail recoloring, we use a separately trained nails segmentation neural network. It detects user nails at the pixel-perfect level and helps to eliminate finger coloring. 


Achieving realistic nail polish simulation in AR

Accurate nail detection is just one part of the challenge when building a nail color try on experience. Another one is a realistic rendering of the nail polish in AR. To solve it, our technology uses several algorithms to add the gloss effects. It mimics real nail polish shine under the light. 

AR nails color try onAR Nails Demo: Different Color Try On

As a result, users can see how a nail varnish color will look on their hand. They can try on real brand shades or recolor nails virtually into vibrant shades. Whatever the experience, it feels and looks real.

Technology

What makes our AR nails technology unique is that it works effortlessly for a user. Most existing manicure try on apps require users to place their hand in the frame at a certain angle. It’s a tiresome process that creates friction, but our technology eliminates one. Here how it works for the user.

  • Automatic nail recognition in real-time
  • No frames needed
  • Right and left hands recognized, user can color nails on any
  • Accurate coloring at any angle, even with part of the nails overlapped
  • Different skin tones and nails of varied length apply
  • Robust nail segmentation with any background

To implement a robust mobile nail color try on, we developed custom convolutional neural networks.To train the neural network for accurate nail segmentation, we used the input image datasets and further marked the results to improve the accuracy.

The neural networks run in real-time on NPU (for example, iPhoneXR+) and GPU-based devices. Currently, virtual nail painting runs on the iOS platform. We're working to bring the technology of nail color try-on to Android and the web.

Post-processing and nail colouring works as follows.

Having an initially combined segmentation nail mask, we divide it into separate finger masks. It allows us to apply different coloring techniques to different fingers. Then we replace the color tone for all pixels with the desired one. We shift the average brightness and saturation to match the parameters of the required color. The saturation spread determines the perception of the nail as matte (small spread) or glossy (large spread).

Application fields

You can integrate our AR nails technology into try on apps, virtual nail salons, manicure apps or in-store mirrors. Beauty brands can add the AR nails try-on feature into their apps to showcase real shades. Consumers can try on nail polish virtually before purchasing.

AR nails try on demoNail Color Try On Showcase

Nail salons looking to adapt to new post-pandemic realities can connect and retain customers on mobile. Virtual salon apps like Modiface can fully replace physical catalogs and showcase nail service right on the customer's hands with the possibility to order one. 

Added to AR mirrors in-store, virtual nails try-on will engage visitors who can pick up the color that goes with their makeup or outfit.

Finally, you can embed our Nails API into machines to enable robotic manicure. Nail segmentation technology allows the creation of software like in Clockwork’s nail painting robot.

With nail polish simulation, the challenges are accuracy, realism and performance. Our technology addresses them all delivering realistic nail coloring on mobile. Integrate it into your iOS app via API, easily pick colors and engage users with virtual nails try-on experience.

Build your nail color simulator or showcase nail polish products in AR. Our technology makes it easy and fast.

Start Free Trial

 

Top