{"id":22253029,"url":"https://github.com/axiscommunications/media-stream-library-js","last_synced_at":"2025-05-14T20:08:06.186Z","repository":{"id":30247981,"uuid":"124252480","full_name":"AxisCommunications/media-stream-library-js","owner":"AxisCommunications","description":"JavaScript library to handle media streams on the command line (Node.js) and in the browser.","archived":false,"fork":false,"pushed_at":"2025-05-09T11:33:41.000Z","size":913919,"stargazers_count":310,"open_issues_count":11,"forks_count":104,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-05-09T12:34:00.065Z","etag":null,"topics":["aac","axis","camera","h264","ip-camera","javascript","jpeg","media-pipelines","mp4","rtp","rtp-client","rtp-player","rtp-streaming","rtsp","rtsp-client","stream-processing","typescript","video-streaming"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AxisCommunications.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-03-07T15:03:55.000Z","updated_at":"2025-05-09T11:32:29.000Z","dependencies_parsed_at":"2023-10-11T02:00:27.380Z","dependency_job_id":"58a01a7b-f006-46e8-a4c3-78d00af4de35","html_url":"https://github.com/AxisCommunications/media-stream-library-js","commit_stats":{"total_commits":970,"total_committers":39,"mean_commits":"24.871794871794872","dds":0.7010309278350515,"last_synced_commit":"998d5e2b3c22711d91c566f949d52ad2ca503fcc"},"previous_names":[],"tags_count":92,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxisCommunications%2Fmedia-stream-library-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxisCommunications%2Fmedia-stream-library-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxisCommunications%2Fmedia-stream-library-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxisCommunications%2Fmedia-stream-library-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AxisCommunications","download_url":"https://codeload.github.com/AxisCommunications/media-stream-library-js/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254219373,"owners_count":22034397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aac","axis","camera","h264","ip-camera","javascript","jpeg","media-pipelines","mp4","rtp","rtp-client","rtp-player","rtp-streaming","rtsp","rtsp-client","stream-processing","typescript","video-streaming"],"created_at":"2024-12-03T07:15:50.149Z","updated_at":"2025-05-14T20:08:06.178Z","avatar_url":"https://github.com/AxisCommunications.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Media Stream Library JS\n\n[![CI][ci-image]][ci-url]\n[![NPM][npm-image]][npm-url]\n\n[ci-image]: https://github.com/AxisCommunications/media-stream-library-js/actions/workflows/verify.yml/badge.svg?branch=main\n[ci-url]: https://github.com/AxisCommunications/media-stream-library-js/actions/workflows/verify.yml\n[npm-image]: https://img.shields.io/npm/v/media-stream-library.svg\n[npm-url]: https://www.npmjs.com/package/media-stream-library\n\n## Installation\n\nMake sure you have Node installed on your machine.\nThen, to install the library:\n\n```sh\nnpm install media-stream-library\n```\n\nor\n\n```sh\nyarn add media-stream-library\n```\n\n## Streams\n\nProvides a way to play RTP streams (H.264/AAC or JPEG) in a browser by converting\nthem to ISO-BMFF and feeding the stream to a SourceBuffer using the [Media Source\nExtensions](https://www.w3.org/TR/media-source/) standard. The RTSP server should\nprovide two-way communication over WebSocket.\nAdditionally, streaming MP4 over HTTP to a SourceBuffer is also provided as\na way to lower latency compared to using a URL on a video tag directly.\n\nThis library is not a full media player: the framework provides no video\ncontrols, progress bar, or other features typically associated with a media\nplayer. For a simple React-based player we refer to the [player](##player).\n\nHowever, getting video to play in the browser is quite easy (check the browser\nexample). There are currently no codecs included either, we rely on browser\nsupport for that.\n\n### Importing\n\n**script tag** You can directly include the `msl-streams.min.js` file (available\nas GitHub release asset) in your browser (check the browser example) as\nan ES module. For this to work, your own script needs to be a module (use\n`type=\"module\" in the script tag`). Make sure the `src` in your script tag\nmatches the path of the file on the server, e.g. if it's at the top level:\n\n```htmls\n\u003cscript type=\"module\" scr=\"/index.js\"\u003e\u003c/script\u003e\n```\n\nand in `index.js` you would use:\n\n```js\nimport {...} from '/msl-streams.min.js';\n```\n\n**bundler** Alternatively, you use import statements in your JS/TS code using\nthe package name if you are going to bundle it yourself:\n\n```js\nimport {...} from 'media-stream-library';\n```\n\n### Components and pipelines\n\nThe library contains a collection of components that can be connected together\nto form media pipelines. The components are a low-level abstraction on top of\nthe Web Streams API to allow two-way communication, while media pipelines are\nsets of components where the streams are connected. The provided pipelines are\na minimal set that provide WebSocket+RTSP =\u003e H.264/AAC or JPEG, and HTTP+MP4,\nwith some extra functionality such as authentication, retry, capture. For more\nadvanced usage, you can construct your own pipelines using the provided ones as\na template.\n\nCheck the `examples` section to see how these can be used in your own code. To\nrun the examples yourself, you'll need to clone this repository loccally and\nfollow the developer instructions.\n\n## Player\n\nA video player based on React intended primarily for Axis cameras. The main\nidea is to define the video state entirely within specialized React components\nfor each of the different supported formats (currently MP4 over HTTP, RTP over\nWebSocket, and still images). The main video player will only handle the\nintended video state (attached to handlers) and format. The player is built on\ntop of [streams](##streams) which provides basic pipeline functionality\nfor the different formats.\n\nYou can either import the `Player` or `BasicPlayer` and use them directly (see\nthe example applications). If you want to build your own customized player, you\ncan look at the latter component and build your own player, using the\n`Container`, `Layer`, and `PlaybackArea` components.\n\n### Basic requirements\n\nThe player specifically targets [AXIS IP cameras](https://www.axis.com/products/network-cameras) because\nwe make underlying API-calls to AXIS specfic APIs to get the video streams.\n\n**Firmware requirements**\n\n- For WebSocket+RTSP to work you need at least firmware 6.50 (LTS)\n- For HTTP+MP4 to work you need at least firmware 9.80 (LTS)\n\n### Importing\n\nIf you don't use the player as part of a React app, the easiest\nway to use it is to download the `msl-player.min.js` file from the\n[releases](https://github.com/AxisCommunications/media-stream-library-js/release\ns/latest) page and include it as an ES module. Make sure your own script has\n`type=\"module\"` and then import directly from the file, e.g.:\n\n```js\nimport {...} from '/msl-player.min.js';\n```\n\nThen, you can use the `\u003cmedia-stream-player/\u003e` tag, similar to how you would use\n`\u003cvideo/\u003e` to include a video element, and provide the camera IP as hostname:\n\n```html\n\u003cmedia-stream-player hostname=\"192.168.0.90\" /\u003e\n```\n\nYou can find an example of this under `example-player-webcomponent`.\n\nSupported properties right now are:\n\n| Property              | Comment                                                                            |\n| --------------------- | ---------------------------------------------------------------------------------- |\n| `variant`             | Supported choices are `basic` or `advanced`. Refers to `BasicPlayer` and `Player`. |\n| `hostname`            | The ip address to your device                                                      |\n| `autoplay`            | If the property exists, we try and autoplay your video                             |\n| `autoretry`           | If the property exists, we try to auto retry your video on errors and if ended     |\n| `secure`              | If the property exists, we will connect with https instead of http                 |\n| `format`              | Accepted values are `JPEG`, `RTP_JPEG`, `RTP_H264`, or `MP4_H264`                  |\n| `compression`         | Accepted values are `0..100`, with 10 between each step                            |\n| `resolution`          | Written as WidthXHeight, eg `1920x1080`                                            |\n| `rotation`            | Accepted values are `0`, `90`, `180` and `270`                                     |\n| `camera`              | Accepted values are `0...n` or `quad` depending on your device                     |\n|                       | **RTP_H264 / RTP_JPEG / MP4_H264 specific properties**                             |\n| `fps`                 | Accepted values are `0...n`                                                        |\n| `audio`               | Accepted values are `0` (off) and `1` (on)                                         |\n| `clock`               | Accepted values are `0` (hide) and `1` (show)                                      |\n| `date`                | Accepted values are `0` (hide) and `1` (show)                                      |\n| `text`                | Accepted values are `0` (hide text overlay) and `1` (show text overlay)            |\n| `textstring`          | A percent-encoded string for the text overlay                                      |\n| `textcolor`           | Accepted values are `black` and `white`                                            |\n| `textbackgroundcolor` | Accepted values are `black`, `white`, `transparent` and `semitransparent`          |\n| `textpos`             | Accepted values are `0` (top) and `1` (bottom)                                     |\n\nExample:\n\n```html\n\u003cmedia-stream-player hostname=\"192.168.0.90\" format=\"RTP_H264\" autoplay /\u003e\n```\n\nYou may need to start a localhost server to get H.264 or Motion JPEG video to\nrun properly. It doesn't work with the `file:///` protocol. The easiest way to\ndo that is to run:\n\nFirst run\n\n```sh\njust run example-player-webcomponent\n```\n\nNote that using anything other than the actual hostname you're hosting from\nwill result in CORS errors for some video formats. You'll need to proxy the\ncamera or load a page from the camera (in which case you can set\n`window.location.host` as the hostname).\n\n### As part of your React application\n\nIf you want to import the player as a React component into your own code, or use\nparts of the player, you'll need to install the package as a dependency.\nYou will also need to install a number of peer dependencies\nsuch as [luxon](https://github.com/moment/luxon), which we use for date and time purposes,\n`react`/`react-dom`.\nYou can find an example of this under `example-player-react`, e.g.:\n\n```js\nimport { BasicPlayer } from 'media-stream-library/player'\n```\n\nTo run our example react app, you can start a vite dev server with:\n\n```sh\nexport MSP_CAMERA=http://192.168.0.90\ncd player\nnode vite.mjs\n```\n\nwhere you specify the IP of the camera you want to proxy as the `MSP_CAMERA`\nenvironment variable (default is `192.168.0.90`). The vite dev server will\nproxy requests to the camera, so that you'll have no CORS issues.\n\n## Overlay\n\nA small React library to make it easier to draw SVG elements with a custom\nuser-defined coordinate system, especially when there is also a transformation\nmapping the user coordinates onto the visible area.\n\nA typical example of this is drawing overlays on top of a transformed image,\nwhen the overlay coordinates are relative to the non-transformed image.\nIn that case, the coordinates are often relative to the image size, and not\nthe actual SVG drawing size.\n\nIn addition, a set of helper components and hooks are provided to make it easier\nto control drawing (e.g. by offering clamping functions), or make it simpler to\nmanipulate objects (e.g. dragging).\n\n### Importing\n\nCheck the `example-overlay-react/` directory for an example on how to use this\nlibrary with your application.\n\n### Coordinate conversion\n\nThe main component is called `Foundation`, and provides you with the functions\nthat transform between user and SVG coordinate systems. This is basically all\nyou ever need, and what this library is about.\n\nTo show how this works, let's say you want to draw a rectangle on top of an\nimage of your cat (1920x1080), around the cat's face, and you now the\ncoordinates of the face in the image. The size of the drawing area in the\nbrowser is 800x450 pixels (the viewbox of the SVG element overlaying the\nimage).\n\nThe first example shows a situation where you have the image's resolution as\ncoordinates (pixel coordinates):\n\n```\n      User coordinate system =                  SVG coordinate system\n        Image coordinates\n\n                 x                                          x\n           +----------\u003e                               +----------\u003e\n   (0,0)                                      (0,0)\n     +----------------------+                   +----------------------+\n     |                      |                   |                      |\n  +  |               /\\_/\\  |                +  |              XXXXXXX |\n  |  |              ( o.o ) |                |  |              X o.o X |\n  |  |               \u003e ^ \u003c  |                |  |              XXXXXXX |\ny |  |                      |    +------\u003e  y |  |                      |\n  |  |                      |                |  |                      |\n  |  |                      |                |  |                      |\n  v  |                      |                v  |                      |\n     |                      |                   |                      |\n     +----------------------+                   +----------------------+\n                       (1920,1080)                                 (800,450)\n```\n\nin this case it would be trivial to overlay an SVG and convert the sun's\ncoordinates to SVG coordinates, and use them for the `\u003ccircle\u003e` `cx` and `cy`,\nyou just scale 1920 to 800 and 1080 to 450.\n\nHowever, you might only have the coordinates of the face relative to the\npicture boundaries:\n\n```\n      User coordinate system =                  SVG coordinate system\n    Relative image coordinates\n\n               x                                            x\n        +--------------\u003e                              +----------\u003e\n                          (1,1)               (0,0)\n     +----------------------+                   +----------------------+\n     |                      |                   |                      |\n ^   |               /\\_/\\  |                +  |              XXXXXXX |\n |   |              ( o.o ) |                |  |              X o.o X |\n |   |               \u003e ^ \u003c  |                |  |              XXXXXXX |\n | y |                      |      +----\u003e  y |  |                      |\n |   |                      |                |  |                      |\n |   |                      |                |  |                      |\n +   |                      |                v  |                      |\n     |                      |                   |                      |\n     +----------------------+                   +----------------------+\n   (0,0)                                                           (800,450)\n```\n\nwhere now you would have to take into account the reversal of the y coordinates\nas well, so the face which is approximately at a y coordinate of 0.66 would turn\nout to have SVG y coordinate of around 150.\n\nAs a third example, you still have the realtive coordinates of the face to the\nwhole picture, but only part of the picture is shown:\n\n```\n             User coordinate system\n                                        (1,1)\n     +------------------------------------+\n     |                                    |\n     |               Visible area         |    SVG coordinate system\n     |                                    |\n     |         (0.4,0.8)        (0.9,0.8) |  (0,0)\n     |             +----------------+     |    +----------------+\n     |             |        /\\_/\\   |     |    |       XXXXXXX  |\n     |             |       ( o.o )  |     |    |       X o.o X  |\n     |             |        \u003e ^ \u003c   | +------\u003e |       XXXXXXX  |\n     |             |                |     |    |                |\n     |             +----------------+     |    +----------------+\n     |         (0.4,0.5)                  |                 (800,450)\n     |                                    |\n     |                                    |\n     |                                    |\n     |                                    |\n     |                                    |\n     |                                    |\n     |                                    |\n     |                                    |\n     +------------------------------------+\n   (0,0)\n```\n\nin which case you'll need a transformation to take into account how the visible\narea maps onto the complete area, before you can determine the final SVG coordinates.\n\nThis library aims to take care of all these conversions for you, as long\nas you can defined your user coordinate system (with the location of the \"real\"\nobjects), and an optional transformation matrix (describing the visible area\nthe SVG overlay applies to). The fact that this mainly comes in handy when\nmatching what you draw to some underlying image and coordinate system is\nthe reason the name of this library is the way it is.\n\n### Utilities\n\nOther than coordinate conversion, there are also a couple of utilities aimed to\nmake it easier to interact with the SVG components.\n\nConvenience functions for clamping are provided by the `Liner` component, which\nlets you specify an area to which to limit your components. There is also a\n`useDraggable` hook to simplify having to deal with moving around stuff.\n\n### Components\n\nWith the React SVG elements and utilities as building blocks, you can then make\nyour own SVG components that will be used inside the `Foundation` component.\nThe best way to get started is to have a look at the example section, which\nshows how you can build your components to make use of this library. The\nexample can be run with `just run overlay`. Instead of defining a whole array of new\nSVG components that wrap the browser elements, the idea is that you can easily\ndo this already with React, and therefore we focused on providing the basics to\naid with building your components, instead of creating a component library.\n\n## Contributing\n\nFor development, you'll need a local installation of [Node.js](https://nodejs.org/),\nand [yarn](https://v3.yarnpkg.com/) to install dependencies.\nTo run commands, you need [just](https://just.systems/), which can be installed using\n[prebuilt binaries](https://just.systems/man/en/chapter_5.html#pre-built-binaries) or\n[yarn](https://just.systems/man/en/chapter_8.html#nodejs-installation), e.g.\n`yarn global add just-install`.\n\nPlease read our [contributing guidelines](CONTRIBUTING.md) before making pull\nrequests.\n\n## FAQ\n\n**Will it work with this library if it works with VLC?**\n\nNot necessarily. We only support a particular subset of the protocol useful for\nbasic streaming from IP cameras. With RTSP that is H.264+AAC or JPEG, and only\nsome simple profiles/levels are supported. For MP4, it depends entirely on your\nbrowser if the media can be played.\n\n**Do I need to use RTSP for live (or low-latency) video?**\n\nSince this library only supports RTSP through some form of TCP connection, it's\ngoing to have similar latency as streaming MP4 over HTTP. For true low-latency\nreal-time usage, you should either use a stand-alone player that can handle RTP over UDP,\nor use WebRTC in the browser.\n\nYou should expect in-browser latency of several frames. When using Firefox, you\nmight need to set the duration of the MediaSource to `0` to force live behaviour\nwith lower latency (see one of the browser examples).\nThe exact latency is controlled by the browser itself, and the data inside the\nmedia stream can affect this (e.g. if audio is present or not).\n\n**Does this library support audio?**\n\nYes, yes it does. With a few caveats though.\n\n- Make sure your AXIS camera actually supports audio\n- Make sure the audio is enabled on the camera.\n- It only works with H.264 and only after user interaction with the volume slider\n\n**How do I autoplay video?**\n\nBrowsers will only allow to autoplay a video if it's muted. If the video is\nnot muted, then it will only play if the `play` method was called from inside\na handler of a user-initiated event. Note that different browsers can have\ndifferent behaviours. Read https://developer.chrome.com/blog/autoplay for more\ndetails.\n\n## Acknowledgements\n\nThe icons used are from https://github.com/google/material-design-icons/, which\nare available under the Apache 2.0 license, more information can be found on:\nhttp://google.github.io/material-design-icons\n\nThe spinner is from https://github.com/SamHerbert/SVG-Loaders, available under\nthe MIT license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxiscommunications%2Fmedia-stream-library-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faxiscommunications%2Fmedia-stream-library-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxiscommunications%2Fmedia-stream-library-js/lists"}