{"id":13906670,"url":"https://github.com/AlexKordic/HLSProxy","last_synced_at":"2025-07-18T04:32:31.395Z","repository":{"id":50617150,"uuid":"203599652","full_name":"AlexKordic/HLSProxy","owner":"AlexKordic","description":"HTTP proxy server. Used for P2P streaming - HLS protocol.","archived":false,"fork":false,"pushed_at":"2019-08-21T17:03:00.000Z","size":2310,"stargazers_count":3,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-08-07T23:48:49.520Z","etag":null,"topics":["cpp","hls-live-streaming","proxy-server"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlexKordic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-08-21T14:19:21.000Z","updated_at":"2023-01-11T18:59:44.000Z","dependencies_parsed_at":"2022-09-24T16:00:56.946Z","dependency_job_id":null,"html_url":"https://github.com/AlexKordic/HLSProxy","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexKordic%2FHLSProxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexKordic%2FHLSProxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexKordic%2FHLSProxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexKordic%2FHLSProxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlexKordic","download_url":"https://codeload.github.com/AlexKordic/HLSProxy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226353390,"owners_count":17611694,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","hls-live-streaming","proxy-server"],"created_at":"2024-08-06T23:01:40.297Z","updated_at":"2024-11-25T15:30:29.626Z","avatar_url":"https://github.com/AlexKordic.png","language":"Python","funding_links":[],"categories":["HarmonyOS"],"sub_categories":["Windows Manager"],"readme":"# HLSProxy\n\nHLSProxy is fast, low-footprint, HTTP proxy server implemented in C++. Integration test suite done in Python.\n\nIts needed in peer-to-peer streaming system as player's first point of contact where Apple's HTTP live streaming protocol is used. \n\nThe proxy function is first step. In the future it may be upgraded to take media segments from peer-to-peer networking layer. For more info about the concept see https://en.wikipedia.org/wiki/Popcorn_Time.\n\n![P2P streaming purpose](http://dzeri.com/streamlabs/proxy_usage.png)\n\nIt's not used in production and so far 4 days of work old.\n\nBasic requirements:\n\n- This local server will serve as a relay between VLC and the CDN\n- Supporting only HLS media format\n- Solution should be as robust as possible\n- HLSProxy shall make request to CDN and serve the result to VLC\n- Support both relative and absolute URLs in manifest\n- The launch procedure:\n  1) Launch HLSProxy server on localhost\n  2) Then start a network video from VLC pointing to your local server \n- Required events in the log:\n  - When a request is intercepted from VLC\n  - When the CDN answers and we send back the answer to VLC\n  - Make distinction between a manifest and segment for each new incoming request and served response\n  - When the server detects the player switched track \n\n### Demo:\n\n[VLC GUI and integration test suite screen-cast](http://dzeri.com/streamlabs/HLSProxy-operation-demo.mp4)\n\n### The design of the solution:\n\nData pipeline: `VLC` \u003e\u003e `HLSProxy` \u003e\u003e `CDN` \u003e\u003e `HLSProxy` \u003e\u003e `VLC`\n\n![Pipeline overview](http://dzeri.com/streamlabs/pipeline-diagram.png)\n\nThe processing happens in `URL transformation` stage where URI's from CDN's response gets transformed to hit `HLSProxy` instead of CDN.\n\nThe integration tests shall use libvlc to play each stream in specified file.\n\nInput file containing test streams shall have single URL per line. Blank lines are allowed.\n\nlibVLC test will first try to play stream from CDN directly and then test if playback works through `HLSProxy`.\n\nTests that fail to play direct CDN URL will yield \"INVALID_URL\".\n\nPlayback time before integration test concludes successful run shall be different for CDN and `HLSProxy`.\n\n### URL transformation rules:\n\nWe can call transformed url `encoded-url`.\n\nCDN url is transformed to connect to `HLSProxy` from\n\tplain-url:   http://`cdn-host:`cdn-port`/`path`?`query`#`fragment`\nto\n\tencoded-url: http://`hlsproxy-host`:`hlsproxy-port`/0~`cdn-host`~`cdn-port`/`path`?`query`#`fragment`\n\nAnd in case of HTTPS:\n\tplain-url:   https://`cdn-host:`cdn-port`/`path`?`query`#`fragment`\nto\n\tencoded-url: http://`hlsproxy-host`:`hlsproxy-port`/1~`cdn-host`~`cdn-port`/`path`?`query`#`fragment`\n\nVLC always use HTTP to connect to `HLSProxy` and `HLSProxy` then uses HTTPS to connect to CDN.\n\nTransformation steps:\n1) VLC uses `encoded-url` to create request to `HLSProxy`.\n2) `HLSProxy` decodes `encoded-url` and get information `cdn-host`, `cdn-port` and whether to use HTTPS or not\n3) `HLSProxy` uses `plain-url` to connect to CDN\n4) `HLSProxy` streams response from CDN to VLC transforming URLs from playlist from `plain-url` to `encoded-url`\n5) VLC completes HTTP transfer and because URLs are transformed next request will also go to `HLSProxy` server\n\nExample:\n\nplain-url:   `https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8`\n\nencoded-url: `http://127.0.0.1:8080/1~443~bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8`\n\n### Internals:\n\n`class HLSProxyServer` accepts new clients and keeps books about connected VLC players. It is possible to direct multiple VLC players on same `HLSProxy` server.\n\n`class HLSProxyServer` implements threading server. Each new connection is processed in new thread.\n\nCookies are used to distinguish different players among many requests thay may make. VLC player handles cookies, some other player may ignore cookes.\n\n`class HLSClient` take care of VLC request parsing, contacting CDN and parsing CDN's response. \n\n`HLSClient::run_player_request_parsing` method is where VLC's request processing takes place. Cookie is read here and if not found new place is created in `PlayerActionTracker HLSProxyServer::_player_action_tracker`.\n\n`class PlayerActionTracker` is used for keeping global state about players. This allows track-change to be detected.\n\n`HLSClient::run_cdn_response_parsing` method is where reponse from CDN is streamed to VLC. \n\nDetecting if content will be HLS playlist can be done in request, in response and in start of the content. Official spec states:\n```\n    Each Playlist file MUST be identifiable either by the path component\n    of its URI or by HTTP Content - Type.In the first case, the path MUST\n    end with either.m3u8 or .m3u.In the second, the HTTP Content - Type\n    MUST be \"application/vnd.apple.mpegurl\" or \"audio/mpegurl\".Clients\n    SHOULD refuse to parse Playlists that are not so identified.\n```\n`class HLSClient` does this by checking HTTP requestline to end in \".m3u8\" or \".m3u\" and later in `HLSClient::run_cdn_response_parsing` method by checking Content-Type of the response. In paractice not everyone honors latest spec so several other Content-Type variations are checked. See `HLSClient::media_context_type_from_request_url` and `HLSClient::media_context_type_from_response_header`.\n\nWhen status of response is 404 then response is not marked as MANIFEST type. It is normal to detect request as MANIFEST and response UNKNOWN.\n\nIf response is not MANIFEST then response is sent unmodified to VLC.\n\nIf response is MANIFEST then we need to modify URLs in content. When `Content-Length` header is defined by CDN and we modify URLs (we need to add a path segment) the `Content-Length` value is no longer correct. Because we decided to stream response to VLC keeping minimal latency we can't know correct `Content-Length` value when HTTP headers are sent to VLC. Because of this chunked encoding is used in sending response. Method to do this operation is `HLSClient::send_to_player_nossl_chunked`.\n\nAt the end of `HLSClient::run_cdn_response_parsing` method time measurement is taken. Time is measured from start of VLC's request to end of CDN response streamed to player.\n\n`class CDNConnection` is used to make request to CDN and read response from socket. `class CDNConnectionSSL` is the same for SSL connection. mbedtls-2.16.2 library is used for SSL in `class CDNConnectionSSL`. Version used during dev is https://tls.mbed.org/download/mbedtls-2.16.2-apache.tgz. Just unpack it in root of the project, compiler and linker options should be already set in VC solution file.\n\nHTTP redirects are handled by rewriting `Location` header so VLC is left to handle recursive redirect situations.\n\nHTTP request and response parsing is done using `http-parser` library from node.js https://github.com/nodejs/http-parser . \n\n\n### Building:\n\nDepends on https://tls.mbed.org/download/mbedtls-2.16.2-apache.tgz. Unpack in project dir .\n\nDepends on ftp://sourceware.org/pub/pthreads-win32/prebuilt-dll-2-9-1-release/. Copy in project dir under pthreads-win32/ .\n\nDepends on https://github.com/nodejs/http-parser.\n\nBuild using Visual Studio 2017 .\n\n### Server usage:\n\n```\nHLSProxy.exe \u003chost-to-listen-on\u003e \u003cport\u003e\n```\n\nExaple:\n```\nHLSProxy.exe 127.0.0.1 8080\n```\n\nNote that `0.0.0.0` is not valid parameter for `host` as this is later used to rewrite URLs.\n\nIf using VLC UI please note that you must enter `encoded-url` so stream goes through `HLSProxy` server.\n\n### Running integration tests:\n\nScripts for integration test are located in `test_integration_py/` dir.\n\nCompose list of test HLS sources and place them in a file. There is example file located `test_integration_py/hls_sources.txt`.\n\nFirst start HLSProxy server. For example:\n```\nHLSProxy.exe 127.0.0.1 8080\n```\n\nThen start test suire with:\n```\n\\test_integration_py\u003epython test.py hls_sources.txt 127.0.0.1 8080\n```\n\nEach HLS source will be tested. libVLC will decode media for some time and report results.\n\n```\nTest run complete:\n[TestOutcome PASSING http://dzeri.com/tmp/lsv2-1920x1080.mp4]\n[TestOutcome PASSING https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8]\n[TestOutcome PASSING http://dzeri.com/streamlabs/absolute.m3u8]\n[TestOutcome PASSING http://dzeri.com/streamlabs/redirected_absolute.m3u8]\n[TestOutcome INVALID_URL http://dzeri.com/s/o/m/e/w/h/e/r/e/notexisting_url.m3u8]\n[TestOutcome INVALID_URL http://184.72.239.149/vod/smil:BigBuckBunny.smil/playlist.m3u8]\n[TestOutcome INVALID_URL http://www.streambox.fr/playlists/test_001/stream.m3u8]\n[TestOutcome FAILED https://mnmedias.api.telequebec.tv/m3u8/29880.m3u8]\n```\n\n### Status\n\n```\nDebuggable     #########\nMaintainable   ##########\nUnderstandable ########\nDocumented     #######\nPortability    ########\n```\n\n### Limitations:\n\nOther HTTP media streams may work (like MP4) but are not tested.\n\nIPv6 not supported. Changes to socket logic is needed.\n\nCommandline parsing is not robust.\n\nTCP tuning not done.\n\nCookie handling is basic. Proper HTTP Cookie parser should be used in the future.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlexKordic%2FHLSProxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAlexKordic%2FHLSProxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlexKordic%2FHLSProxy/lists"}