Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/infobip/infobip-rtc-extensions-js

Quick Start Guide docs for our RTC JS Extensions
https://github.com/infobip/infobip-rtc-extensions-js

Last synced: 4 days ago
JSON representation

Quick Start Guide docs for our RTC JS Extensions

Awesome Lists containing this project

README 

        
## Introduction



Infobip RTC extensions is a JavaScript library which provides extended functionality to Infobip RTC SDK.



Currently available functionalities are:



- audio filter implementations

- video filter implementations



Here you will find an overview, and a quick guide on how to include and use these extensions in your application.

There is also in-depth reference documentation available.



## Prerequisites



Infobip RTC Extensions requires ES6.



## Getting the library



There are a few ways in which you can get our library. We publish it as an NPM package and as a standalone JS file

hosted on a CDN.



If you want to add it as an NPM dependency, run the following:



```bash

npm install infobip-rtc-extensions --save

```



After which you would use it in your project like this:



```javascript

let infobipRtcExtensions = require('infobip-rtc-extensions');

```



or as ES6 import:



```javascript

import {RTCVideoFilter} from "infobip-rtc-extensions"

```



You can include our distribution file in your JavaScript from our CDN:



```html



```



The latest tag is also available:



```html



```



## Audio filters



The Infobip RTC supports user-defined audio filters which manipulate outgoing audio streams during a call. This library

implements several audio filters which are easy to configure and use.



Filters which are currently available are:



- [`BackgroundMusicAudioFilter`](#background-music-audio-filter)

- [`NoiseSuppressionFilter`](#noise-suppression-filter)






### BackgroundMusicAudioFilter



`BackgroundMusicAudioFilter` allows the user to specify audio to be played alongside their outgoing stream. This audio

is heard by other participants of the call, but not by the user.



To use this filter, an instance of

[`BackgroundMusicAudioFilter`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/BackgroundMusicAudioFilter)

needs to be created. The only parameter of the class constructor is a string containing the URL of the audio file which

is to be combined with the user's outgoing stream. The audio file loops until the filter is turned off or the call is

terminated.



```javascript

const musicURL = "path/to/desired/audio.mp3";

const backgroundMusicFilter = new BackgroundMusicAudioFilter(musicURL);

```






### NoiseSuppressionFilter



The `NoiseSuppressionFilter` enhances speech by removing several types of background noise. This filter works in

real-time. Currently, it is focused on removing background noises commonly encountered in call centers, such as babble,

noise produced by different devices (e.g. air conditioner) and keyboard typing sounds. However, it performs well on a

wider range of noise types. It is also independent of the language spoken.



To use the noise suppression filter, an instance of the class

[`NoiseSuppressionFilter`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/NoiseSuppressionFilter) needs to be

created. This can be done using the [`NoiseSuppressionFilter.create()`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/NoiseSuppressionFilter#create)

function. 



Real-time noise suppression is a hardware-intensive process, so not all hardware will be able to sustain it. In case of

insufficient performance, the filter is automatically disabled in order to not interfere with the call.



```javascript

const noiseSupressionFilter = await NoiseSuppressionFilter.create();

```



During initialisation, the created audio filter instance allocates resources which should be manually released once

you are certain you won't need this filter instance anymore.

```javascript

// load the filter

const noiseSupressionFilter = await NoiseSuppressionFilter.create();



// start using it

activeCall.setAudioFilter(noiseSupressionFilter);



// at the end of the call

await noiseSupressionFilter.release();

```



## Video filters



The Infobip RTC supports user-defined video filters capable of manipulating outgoing video streams during calls. The

library provides an extensive implementation of commonly used video filters, making configuration easier and enabling

seamless integration.



Currently available implementations are:



- [`RTCVideoFilter`](#rtc-video-filer)






### RTCVideoFilter



This filter allows users to modify their background during video calls.



Supported video filter modes include:



- Virtual background

  ([`RTCVideoFilterMode.VIRTUAL_BACKGROUND`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/RTCVideoFilterMode#virtual-background)) -

  Users can set a custom image to be displayed as their background

- Background blur

  ([`RTCVideoFilterMode.BACKGROUND_BLUR`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/RTCVideoFilterMode#background-blur)) -

  Users can blur their background.

- Face track

  ([`RTCVideoFilterMode.FACE_TRACK`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/RTCVideoFilterMode#face-track)) -

  Automatically adjusts the video to keep the user's face centered and properly framed within the view.

- None ([`RTCVideoFilterMode.NONE`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/RTCVideoFilterMode#none)) -

  No video filtering is applied; video frames are passed through unchanged. This option is recommended over repeatedly

  reallocating video filter resources to avoid visible disruptions.



To utilize this feature, begin by creating an instance of

the [`RTCVideoFilter`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/RTCVideoFilter)

object. The constructor accepts optional

[`RTCVideoFilterOptions`](https://github.com/infobip/infobip-rtc-extensions-js/wiki/RTCVideoFilterOptions)

for customization.



```javascript

const options = {

    mode: RTCVideoFilterMode.VIRTUAL_BACKGROUND,

    image: sourceImage // can be an instance of ImageBitmap, ImageData, HTMLImageElement, …

};

const videoFilter = new RTCVideoFilter(options);

```



For optimal performance, it's recommended to avoid reallocating video filter instances solely for mode changes. Instead,

pass the new options directly to the existing video filter instance. This approach minimizes resource overhead and

enhances overall efficiency.



```javascript

const options = {

    mode: RTCVideoFilterMode.NONE

};

await videoFilter.setOptions(options);

``` 



### Applying the video filter



Once you've created the video filter, you can utilize it during calls.



You can set it beforehand when initiating a

new [`ApplicationCall`](https://github.com/infobip/infobip-rtc-js/wiki/ApplicationCall)

using [`VideoOptions`](https://github.com/infobip/infobip-rtc-js/wiki/VideoOptions) object within

the [`ApplicationCallOptions`](https://github.com/infobip/infobip-rtc-js/wiki/ApplicationCallOptions) object:



```javascript

let token = obtainToken();

let infobipRTC = new InfobipRTC(token);

infobipRTC.connect();



let videoOptions = VideoOptions.builder().setVideoFilter(videoFilter).build();

let applicationCallOptions = ApplicationCallOptions.builder().setVideo(true).setVideoOptions(videoOptions).build();

let applicationCall = infobipRTC.callApplication('45g2gql9ay4a2blu55uk1628', applicationCallOptions);

```



Alternatively, you can apply the filter to the

existing [`ApplicationCall`](https://github.com/infobip/infobip-rtc-js/wiki/ApplicationCall) using the

[`setVideoFilter`](https://github.com/infobip/infobip-rtc-js/wiki/ApplicationCall#set-video-filter) method:



```javascript

await applicationCall.setVideoFilter(videoFilter);

```