Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/t-mullen/video-stream-merger

Merge multiple video MediaStreams into one composite.
https://github.com/t-mullen/video-stream-merger

multiple-mediastreams stream video webrtc

Last synced: 5 days ago
JSON representation

Merge multiple video MediaStreams into one composite.

Lists

README 

        
# video-stream-merger



[![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com)



Merges the video of multiple MediaStreams. Also merges the audio via the WebAudio API.  



- Send multiple videos over a single WebRTC MediaConnection.

- Hotswap streams without worrying about renegotation or delays.

- Crop, scale, and rotate live video.

- Add creative effects through the canvas API.



[Demo](https://t-mullen.github.io/video-stream-merger/)



[P2P Demo](https://t-mullen.github.io/video-stream-merger/demo/p2p-fixed.html)



[Existing Files Demo](https://t-mullen.github.io/video-stream-merger/demo/files.html)



## install



```

npm install video-stream-merger

```



or



```html



```



## usage



Let's first get two media streams. One from the webcam, and another a screen capture.

```javascript

const getusermedia = require('getusermedia')

const screenRecord = require('screen-record')



getusermedia({video: true, audio:true}, (err, webcamStream) => {

  screenRecord(window, (err, sourceId, constraints) => {

    getusermedia(constraints, (err, screenStream) => {

      // We now have 2 streams: webcamStream, screenStream

    })

  })

})

```



We want to overlay the webcam stream in the corner of the screen stream.  

```javascript

var VideoStreamMerger = require('video-stream-merger')



var merger = new VideoStreamMerger()



// Add the screen capture. Position it to fill the whole stream (the default)

merger.addStream(screenStream, {

  x: 0, // position of the topleft corner

  y: 0,

  width: merger.width,

  height: merger.height,

  mute: true // we don't want sound from the screen (if there is any)

})



// Add the webcam stream. Position it on the bottom left and resize it to 100x100.

merger.addStream(webcamStream, {

  x: 0,

  y: merger.height - 100,

  width: 100,

  height: 100,

  mute: false

})



// Start the merging. Calling this makes the result available to us

merger.start()



// We now have a merged MediaStream!

merger.result

```



## API



### `merger = new VideoStreamMerger([opts])`



Create a new video merger.



Optional `opts` defaults to the below:



```

{

  width: 400,   // Width of the output video

  height: 300,  // Height of the output video

  fps: 25,       // Video capture frames per second

  clearRect: true, // Clear the canvas every frame

  audioContext: null, // Supply an external AudioContext (for audio effects)

}

```



### `merger.addStream(mediaStream|id, [opts])`



Add a MediaStream to be merged. Use an `id` string if you only want to provide an effect.



The order that streams are added matters. Streams placed earlier will be behind later streams (use the `index` option to change this behaviour.)



Optional `opts` defaults to the below:

```

{

  x: 0, // position of the top-left corner

  y: 0,

  width: ,     // size to draw the stream

  height: ,

  index: 0, // Layer on which to draw the stream (0 is bottom, 1 is above that, and so on)

  mute: false,  // if true, any audio tracks will not be merged

  draw: null,    // A custom drawing function (see below)

  audioEffect: null // A custom WebAudio effect (see below)

}

```



### `merger.removeStream(mediaStream|id)`



Remove a MediaStream from the merging. You may also use the ID of the stream.



If you have added the same MediaStream multiple times, all instances will be removed.



### `merger.addMediaElement(id, mediaElement, opts)`



A convenience function to merge a HTML5 MediaElement instead of a MediaStream.



`id` is a string used to remove or update the index of the stream later.



`mediaElement` is a **playing** HTML5 Audio or Video element.



`opts` are identical to the opts for `addStream`.



Streams from MediaElements can be removed via `merger.removeStream(id)`.



### `merger.updateIndex(mediaStream|id, newIndex)`



Update the z-index (draw order) of an already added stream or data object. Identical to the `index` option.



If you have added the same MediaStream multiple times, all instances will be updated.



### `merger.setOutputSize(width, height)`



Change the size of the canvas and the output video track.



Automatically updates `merger.width` and `merger.height`.



### `merger.start()`



Start the merging and create `merger.result`.



You can call this any time, but you only need to call it once.  



You will still be able to add/remove streams and the result stream will automatically update.



### `merger.result`



The resulting merged MediaStream. Only available after calling `merger.start()`



Never has more than one Audio and one Video track.



### `merger.destroy()`



Clean up everything and destroy the result stream.  



### `merger.getAudioContext()`



Get the WebAudio AudioContext being used by the merger.



### `merger.getAudioDestination()`



Get the MediaStreamDestination node that is used by the merger.



## Hot-Swapping Streams (P2P Streaming)



This library makes it easy to change streams in a WebRTC connection without needing to renegotiate.



The result MediaStream will appear to be constant and stable, no matter what streams you add/remove!



[P2P Streaming Demo](https://t-mullen.github.io/video-stream-merger/demo/p2p-dynamic.html)



```javascript

getusermedia({video: true, audio:true}, (err, webcamStream) => {

  const merger = new VideoStreamMerger()

  merger.start()

  players[0].srcObject = merger.result

  players[0].play()

  

  const peer1 = new SimplePeer({initiator: true, stream:merger.result})

  const peer2 = new SimplePeer()



  peer1.on('signal', (data) => {

    peer2.signal(data)

  })

  peer2.on('signal', (data) => {

    peer1.signal(data)

  })



  peer2.on('stream', (stream) => {

    players[1].srcObject = stream

  })

  

  const clones = []

  

  shareWebCamStream.addEventListener('click', () => {

      clones.push(webcamStream.clone())

      merger.addStream(clones[clones.length-1])

  })

  removeWebCamStream.addEventListener('click', () => {

      merger.removeStream(clones.pop())

  })

})

```



## Custom Draw Function



If sizing and positioning aren't enough, you can directly draw the video frames by passing a function to the `draw` option.



```javascript

merger.addStream(mediaStream, {

  draw: (ctx, frame, done) => {

    // You can do whatever you want with this canvas context

    ctx.drawImage(frame, 0, 0, merger.width, merger.height)

    done()

  })

})

```



See the bottom example of the [Live Demo](https://t-mullen.github.io/video-stream-merger/) to see this in action.  



## Custom WebAudio Effects



You can also take direct control over how audio streams are merged, and apply effects.



```javascript

merger.addStream(mediaStream, {

  audioEffect: (sourceNode, destinationNode) => {

    // sourceNode is the input streams audio (microphone, etc)

    // destinationNode is the output streams audio

    

    sourceNode.connect(destinationNode) // The default effect, simply merges audio

  })

})

```



Both the `draw` and `audioEffect` options can be used without any MediaStream at all. Just pass a string instead.



## Sponsors

Support this project by becoming a sponsor. Your logo will appear here with a link to your website. [[Become a sponsor](https://github.com/sponsors/t-mullen)]