Banuba in 2023: TINT, Photo Editor SDK and Face AR Updates

TL;DR:

• Banuba Photo Editor SDK launched for Android and iOS with AI retouching, facial feature adjustment, virtual makeup, background replacement, colour filters and overlays.
• Banuba TINT added skincare try-on, which predicts and shows the effect of products that leave no visible colour on the skin.
• Banuba TINT widened its catalogue beyond makeup into eyewear, jewelry, headwear, hair colour and nail polish, the category range eCommerce buyers still ask about most.
• Banuba Face AR SDK added acne removal on iOS in v1.7.1 (June 2023) and on Android in v1.8.0 (August 2023).
• Banuba Face AR SDK v1.9.0 (October 2023) introduced earlobe detection and a new Effect Player API for Web, iOS and Android.
• Banuba Face AR SDK v1.6.0 (January 2023) added per-finger ring rendering, which is what turns ring try-on from a demo into something a jewelry shopper can use.
• Banuba Face AR SDK v1.10.0 (December 2023) added teeth segmentation and new face morphs in the Makeup API.

TINT virtual try-on platform

TINT gained traction on the market during 2023 and received a steady stream of updates from the development team.

• Skin care support. TINT added support for skin care products. They are invisible on the skin, so the platform uses AI to predict the effects of prolonged use and shows them on the user's face. That work is still the basis of Banuba's skincare virtual try-on today.
• Expanded try-on options. TINT started out as a makeup try-on application, and by the end of 2023 it also worked with headwear, glasses and sunglasses, jewelry, hair colour, nail polish and eye lenses. Category coverage is the thing eCommerce teams weigh first, and there is a useful walk-through of how to compare eyewear try-on tools if glasses are the category you are starting with.
• Improvements in the recommendations algorithm. The system began giving better product suggestions thanks to updated neural networks.
• Product digitization at scale. The conversion pipeline turned client catalogues into virtual items in bulk, so a merchant could launch with a full product range rather than a handful of samples.

Photo Editor SDK release

Banuba’s Photo Editor SDK for Android and iOS is designed to turn regular photos into viral masterpieces. It has a powerful suite of retouching features while still being intuitive and user-friendly.

This is what it can do:

• AI retouching. The built-in AI analyzes the face of the person in the photo and applies effects to enhance their natural appearance and remove skin imperfections (wrinkles, problem spots, acne, etc.). A user can then adjust these effects as needed. 
• Facial feature adjustment. Modifying the size and shape of eyes, nose, cheekbones, eyebrows, and lips; whitening teeth.
• Virtual makeup. AR cosmetics for eyes, lips, and hair, as well as an eye color changer. Dozens of options available for each category.
• Virtual backgrounds. Changing picture backdrop to one of the many scenes.
• Color filters. Changing the color scheme of the image.
• Overlays. Captions, stickers, pictures, and more. 

Social media, dating apps, eCommerce, dedicated photo editors, these are just a few of the apps that could benefit from Banuba Photo Editor SDK.

Face AR SDK updates

Our flagship product Face AR SDK kept getting new features and technical improvements through eleven releases in 2023, from v1.6.0 in January to v1.10.0 in December.

• • Acne removal. Manual acne removal arrived on iOS in v1.7.1 (June 2023), Android support followed in v1.8.0 (August 2023), and the same release made the correction size adjustable from the effect settings.
  • Earlobe detection. Added in v1.9.0 (October 2023). Detecting that specific part of the ear is what lets virtual earrings sit where they should.
  • New Effect Player API. v1.9.0 also shipped a new Effect Player API for Web, iOS and Android, a more streamlined interface with a documented migration path off the old one.
  • Rendering quality for wearables. The same release improved rendering quality for head wearable and arm wearable products, and render quality for caps and rings specifically.
  • Per-finger ring try-on. v1.6.0 (January 2023) added rings for each finger, and v1.6.1 and v1.7.0 followed with a proper ring fitting interface. AR Rings technology itself had shipped earlier, in v1.3.0 (January 2022).
  • Face morphing improvements. Face morphing is a long-standing capability rather than a 2023 arrival, and the year focused on refining it: new makeup morphings in v1.7.0, eye morphings in v1.7.1, nose morphing and morphing on by default in v1.8.0, and new face morphs in the Makeup API in v1.10.0.
  • Skin smoothing. Reworked in v1.8.0 and improved again for Unity in v1.9.0.
  • Segmentation. v1.7.1 brought portrait background segmentation to desktop and web, v1.8.0 shipped new lip segmentation, v1.9.2 (November 2023) added face skin segmentation, and v1.10.0 added teeth segmentation.
  • Hand detector performance. v1.7.0 (May 2023) reduced the hand detector's CPU consumption. Hand tracking and the hand skeleton model themselves date back to 2021 and 2022.

  Other technical upgrades that year included GLTF serialization changes in v1.8.0, video watermarking and Full HD video recording on iOS in v1.9.2, and Swift Package Manager downloads for iOS packages in v1.10.0.

Video Editor SDK updates

Video Editor SDK also received a host of updates. It gained new functionality and improved some of the existing features.

• New UI. The editor interface was redesigned around current design and user experience conventions.
• Soundstripe integration. Video Editor SDK gained a native integration with a second royalty-free music provider alongside Banuba Music.
• Improved API. The Video Editor API was modified to make integrations easier and more time-efficient.
• Picture-in-Picture updates. The audio recording and synchronization options were expanded and improved.

Virtual Store platform

Banuba virtual store platform is an interactive environment where users can browse the products, see important information (e.g., components or allergy warnings), virtually try on items, and easily make purchases. It is available online, making it accessible to anyone with a smartphone and an internet connection. 

• Integration with Banuba TINT and its recommendations AI
• Detailed, realistic 3D models of items
• One-click access to product characteristics
• Simple purchasing process

See the virtual store demo to learn more.

Where the 2023 releases stand now

Most of what shipped in 2023 is still in the products, in a more developed form.

• The Face AR SDK line continued past v1.10.0 and is now on the v1.17 and v1.18 series, so the 2023 Effect Player API is the interface current integrations are built on.
• TINT became a self-serve product with its own Shopify app, and the category list has kept moving: hats, glasses, scarves, jewelry, eye lenses and eyelashes are the categories the widget documentation lists today, alongside makeup, hair dye and nail polish on the platform.
• Skincare try-on grew from a 2023 module into a standalone offer with its own page and demo.
• Photo Editor SDK and the virtual store platform are both still live products.

One correction worth flagging for anyone using this post as a changelog: teeth segmentation belongs to December 2023 (v1.10.0), not to a later year.