RTE2021: Cutting Edge Interactions in Video Communication

The point of RTE2021

Human interactions were becoming more and more digitalized. However, when the lockdowns hit, the importance of video communication skyrocketed. Companies scrambled to adapt to the new conditions with almost everyone understanding the value of talking to colleagues and customers online. According to the research by Twilio, 87% of businessmen thought that digital communications were critical or very important for their survival during the pandemic.

Now that the people are free to move around, they still want video communication. Most office workers want to continue working from home, and companies are moving towards more digital touchpoints. As a result, the need for online communications continues to rise. 

So do the problems associated with it. 

These include voice and image quality, protecting privacy, distractions during work, and exhaustion after many meetings. The vendors of real-time interaction solutions have their own issues to deal with, like releasing the product ahead of the competitors or getting new functionality quickly. 

All these topics and more were discussed at RTE 2021.

Better Together: How Extensions Enhance Real-Time Interaction

Today’s video communication is more than it seems. It is becoming a popular feature in all kinds of apps, from dedicated videoconferencing solutions to dating and social media. Correspondingly, the demand for them increased. But building such functionality is complicated and takes a lot of time. Even more so if you want the video communication experience you provide to be enhanced with additional features.

One of the quickest ways to do so is to use SDKs - pre-made modules that can be integrated into an app and perform certain functions (video calls, voice or facial modification, analytics, background replacement, speech-to-text, etc.). Using them lets you accomplish in days what would otherwise take months. However, this presents another challenge: what if you want to use several SDKs from different companies? 

RTE2021, Better Together: Speakers from Agora, Hive, Bose, Voicemod, Banuba

We know from our clients that making several SDKs work well in one product is a daunting task. There inevitably are issues with compatibility and performance, as you try to marry separate products in one system. So once we heard that Agora is launching an extension marketplace, we were among the first to jump on board and start working on our plugin. Native compatibility between our products would’ve helped our clients immensely.

Now that the marketplace is up, the integration problem will be a thing of the past. The companies will be able to use multiple plugins and be sure that everything will work as intended. This frees a lot of time and brainpower and allows businesses to focus on improving their own offering.

Moreover, the marketplace will further shorten your own time-to-market. There will be no need to negotiate with each vendor separately, you can just click on the pricing plan that fits you and start working.

The session called Better Together: How Extensions Enhance Real-Time Interaction was dedicated to the Agora extension marketplace that has been officially launched on September 1. The participants covered the use cases for their plugins and gave advice for those who want to join the marketplace.

The participants of a panel discussion were: 

• Brad Mortensen, Partnership Development Manager, Agora
• Kevin Guo, CEO, Hive
• Vadim Nekhai, CRO, Banuba
• Charlie Taylor, Corporate Innovation Manager, Bose
• Michael Karliner, Chief Architect, Voicemod

Now onto the extensions themselves.

Hive specializes in AI and has many computer vision-related use cases. Their Agora extension is focused on automatic content moderation (e.g. for live streaming). It can automatically detect unsafe content like self-harm, nudity, or drugs, thus doing the work of human moderators. For people, this can be tedious and even traumatic. For businesses, this can be expensive, take a lot of time, and present litigation risk. So having a plugin handle moderation is a reasonable move.

Banuba specializes in Face AR and so does our extension. It includes the following feature set: 

• Face Filters & AR Lens
• Beautification
• Makeup Try-On
• Background replacement

These functionalities are at the core of the custom video-calling experience. Of course, they can be used for fun, letting people put on funny masks and weird makeup. The popularity of short video apps like TikTok or Likee attests to that. 

However, there are also serious applications. For example, this can be used to let people virtually try various facewear items on live stream and sell them to the audience. Beautification and makeup try-on can make business meetings more comfortable. During a videoconference, people are unconsciously checking out how they look, which creates an additional cognitive load. Without it, the meetings become much less exhausting, and “Zoom fatigue” decreases. Finally, the filters can be used as icebreakers for teachers who work with small children. 

You can check out and license our extension at the Agora Marketplace.

Voicemod offers voice modification functionality, letting people sound like robots or movie characters. Their extension has the most traction with voice chats for gamers or for live streamers and similar content creators. However, the company hopes to transition to making “audio-avatars”, helping people sound the way they want to sound. This also has a “serious” application: helping deal with toxicity in multiplayer video games. Children, for example, might want to change their actual voice to not appear vulnerable.

Bose is known for its high-quality sound equipment. Their plugin focuses on removing all the unnecessary noise: lawnmowers, barking dogs, playing children, etc. This is useful for professional meetings as well as regular calls, as it helps avoid distractions and embarrassment, as well as get a higher quality of the call. What’s important is that one doesn’t have to have Bose hardware to use this plugin. Any microphone and headphones/speakers will do, though the better the input, the better the output.

Agora is open to other developers pitching their own extensions. The panel participants had a few pieces of advice for those who want to release their plugin:

1.  Think about business model compatibility. Develop an extension with a business model that is aligned with that of Agora.
2.  Think about performance and resources. For a good user experience, make sure your extension doesn’t outrageously consume battery and computing power. Agora optimizes their SDKs, and so should you.
3.  Be ready to work with Agora. The company is willing to cooperate with you to help make the finished product better, as it is in their best interest as well.

Conclusion

Thanks to the flexibility afforded by the extension marketplace, it is now much easier to enhance your Agora-made video communication with effects from Banuba. So if you want such functionality in your product, go ahead and check it out.