{"id":15058187,"url":"https://github.com/w3c/mediacapture-region","last_synced_at":"2025-10-20T00:31:02.032Z","repository":{"id":37932494,"uuid":"413939523","full_name":"w3c/mediacapture-region","owner":"w3c","description":" This document introduces an API for cropping a video track derived from display-capture of the current tab. ","archived":false,"fork":false,"pushed_at":"2024-10-25T20:45:48.000Z","size":182,"stargazers_count":42,"open_issues_count":21,"forks_count":10,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-01-30T09:41:27.690Z","etag":null,"topics":["mediacapture","screen-capture","specification"],"latest_commit_sha":null,"homepage":"http://w3c.github.io/mediacapture-region/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/w3c.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-10-05T18:42:36.000Z","updated_at":"2024-12-22T06:03:03.000Z","dependencies_parsed_at":"2023-12-12T09:41:52.955Z","dependency_job_id":"60386182-d0d7-4ba5-8e52-541e8744abc5","html_url":"https://github.com/w3c/mediacapture-region","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w3c%2Fmediacapture-region","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w3c%2Fmediacapture-region/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w3c%2Fmediacapture-region/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/w3c%2Fmediacapture-region/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/w3c","download_url":"https://codeload.github.com/w3c/mediacapture-region/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237236940,"owners_count":19277081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mediacapture","screen-capture","specification"],"created_at":"2024-09-24T22:18:53.970Z","updated_at":"2025-10-20T00:30:56.703Z","avatar_url":"https://github.com/w3c.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Summary\n## Problem Overview\nRecall that applications may currently obtain a capture of the tab in which they run using [getDisplayMedia](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia), either with or without [preferCurrentTab](http://go/get-current-browsing-context-media). Moreover, soon another API will allow similar functionality - [getViewportMedia](https://github.com/w3c/mediacapture-screen-share/issues/155). In either case, the application may then also wish to crop the resulting video track so as to remove some content from it (typically before sharing it remotely). We introduce a performant and robust API for cropping a self-capture video track.\n\n## Core Challenges\nLayout can change asynchronously when the user scrolls, zooms or resizes the window. The application [cannot](https://docs.google.com/document/d/1dULARMnMZggfWqa_Ti_GrINRNYXGIli3XK9brzAKEV8/edit#heading=h.vtnc1viphmzf) robustly react to such changes without risking mis-cropping the video track on occasion. The browser therefore needs to step in and help.\n\n## Sample Use Case\nConsider a combo-application consisting of two major parts - a video-conferencing application and a productivity-suite application co-existing in a single tab. Assume the video-conferencing uses existing/upcoming APIs such as [getDisplayMedia](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia) and/or [getViewportMedia](https://github.com/w3c/mediacapture-screen-share/issues/155) and captures the entire tab. Now it needs to crop away everything other than a particular section of the productivity-suite. It needs to crop away its own video-conferencing content, any speaker notes and other private and/or irrelevant content in the productivity-suite, before transmitting the resulting cropped video remotely.\n\n\u003cimg width=\"1344\" alt=\"DocsSidebar\" src=\"https://user-images.githubusercontent.com/22117736/136094995-48beda21-232b-48c6-b4fc-b8d2a9286e69.png\"\u003e\n\nMoreover, consider that it is likely that the two collaborating applications are cross-origin from each other. They can post messages, but all communication is asynchronous, and it's easier and more performant if information is transmitted sparingly between them. That precludes solutions involving posting of entire frames, as well as solutions which are too slow to react to changes in layout (e.g. scrolling, zooming and window-size changes).\n\n## Goals and Non-Goals\n### Goals\n* The new API we introduce allows an application which is already in possession of a self-capture video track, to crop that track to the contours of its desired element.\n* The API allows this to be done performantly, consistently and robustly.\n### Non-Goals\n* This API does not introduce new ways to obtain a self-capture video track.\n* This API does not introduce mechanisms by which a captured document may control what the capturing document can see.\n\n# Solution\n## Solution Overview\nA two-pronged solution is presented:\n* **CropTarget production:** A mechanism for tagging an HTMLElement as a potential target for the cropping mechanism.\n* **Cropping mechanism**: A mechanism for instructing the user agent to start cropping a video track to the contours of a previously tagged HTMLElement, or to stop such cropping and revert a track to its uncropped state.\n\n## CropTarget production\nWe introduce `navigator.mediaDevices.produceCropTarget()`.\n```webidl\npartial interface MediaDevices {\n  Promise\u003cCropTarget\u003e\n  produceCropTarget(HTMLElement target);\n};\n```\nGiven an HTMLElement, `produceCropTarget()` produces an opaque class that uniquely identifies that element to our second mechanism - the cropping mechanism.\n(The `Promise` returned by `produceCropTarget()` is only resolved when the CropTarget is ready for use, allowing the browser time to set up prerequisites and propagate state cross-process.)\n\n## Cropping mechanism\nWe introduce a `cropTo()` method, which we expose on all video tracks derived of tab-capture.\n```webidl\n[Exposed = Window]\ninterface BrowserCaptureMediaStreamTrack : FocusableMediaStreamTrack {\n  Promise\u003cundefined\u003e cropTo(CropTarget cropTarget);\n};\n```\nGiven a CropTarget, `cropTo()` starts cropping the video track to the contours of the referenced HTMLElement.\nGiven `undefined`, `cropTo()` reverts a video track to its uncropped state.\n\"On-the-fly\" changing of crop-targets is possible.\n\n# Code Samples\n```js\n/////////////////////////////////\n// Code in the capture-target: //\n/////////////////////////////////\n\nconst mainContentArea = navigator.getElementById('mainContentArea');\nconst cropTarget = await navigator.mediaDevices.produceCropTarget(mainContentArea);\nsendCropTarget(cropTarget);\n\nfunction sendCropTarget(cropTarget) {\n  // Can send the crop-target to another document in this browsing context\n  // using postMessage() or using any other means.\n  // Possibly there is no other document, and this is just consumed locally.\n}\n\n/////////////////////////////////////\n// Code in the capturing-document: //\n/////////////////////////////////////\n\nasync function startCroppedCapture(cropTarget) {\n  const stream = await navigator.mediaDevices.getDisplayMedia();\n  const [track] = stream.getVideoTracks();\n  if (!!track.cropTo) {\n    handleError(stream);\n    return;\n  }\n  await track.cropTo(cropTarget);\n  transmitVideoRemotely(track);\n}\n```\n\n# Changelog\nSee [dedicated file](https://github.com/eladalon1983/region-capture/blob/main/CHANGELOG.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fw3c%2Fmediacapture-region","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fw3c%2Fmediacapture-region","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fw3c%2Fmediacapture-region/lists"}