{"id":30858944,"url":"https://github.com/oaubert/mediafragment-prototype","last_synced_at":"2025-09-07T14:55:39.313Z","repository":{"id":33092791,"uuid":"36730288","full_name":"oaubert/mediafragment-prototype","owner":"oaubert","description":"mediafragment-prototype","archived":false,"fork":false,"pushed_at":"2015-10-26T17:27:20.000Z","size":5136,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2023-03-12T18:55:27.581Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"ShawnLin013/NumberPicker","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oaubert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-06-02T12:04:26.000Z","updated_at":"2015-06-19T14:39:25.000Z","dependencies_parsed_at":"2022-07-24T17:47:15.269Z","dependency_job_id":null,"html_url":"https://github.com/oaubert/mediafragment-prototype","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/oaubert/mediafragment-prototype","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oaubert%2Fmediafragment-prototype","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oaubert%2Fmediafragment-prototype/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oaubert%2Fmediafragment-prototype/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oaubert%2Fmediafragment-prototype/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oaubert","download_url":"https://codeload.github.com/oaubert/mediafragment-prototype/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oaubert%2Fmediafragment-prototype/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274051961,"owners_count":25214030,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-07T14:55:29.073Z","updated_at":"2025-09-07T14:55:39.277Z","avatar_url":"https://github.com/oaubert.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dynamic Media Fragment\n\nThis is a proposal for dynamic spatial media fragments, as well as\narbitrary shaped spatial media fragments, for highlighting moving\nareas in videos. \n\nIt adopts the principle of not reinventing the wheel, so since we are\ntalking about graphical shapes and trajectories, SVG is a natural\ntarget.\n\nThe [live demo](http://olivieraubert.net/dynamic-media-fragments/)\nexhibits different uses of the implementation.\n\n## Shape syntax\n\nThe shape syntax is \"simply\" a [SVG \u0026lt;path\u003e\ndefinition](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths),\nwhich brings interesting features :\n\n- it is standardized\n- it is compact and expressive\n- it make simple things relatively simple (freehand shapes), while allowing more complex things (bezier curves, etc)\n- there is software/libraries (such as d3.js) for editing it and generating it\n- there is software for reading it, and most especially web browsers\n- it can be styled through standard CSS syntax\n\nThe path definition is passed as the `shape=` parameter. The\ncurrent implement considers a viewbox with the same dimensions as the\nvideo widget, through this should be further specified and refined.\n\nFor example, the URL ```video.mp4#t=10,50\u0026shape=M0 0l0 50 50 50 50 0z```\ndefines a 50pixel-wide square, in the top left corner of the video.\n\n### Styling\n\nThe current implementation defines the shape id according to the video\nid. Given a video with the id `v1`, the defined shape will have the id\n`v1_overlay_shape`. This allows to style the element through standard\nCSS, by specifying fill and stroke colors, patterns, etc (see\n[main.css]). If no video id is specified, it will be automatically\ngenerated.\n\nThis could be defined through a CSS pseudo-element such as\n`v1::overlay_shape`, though it actually is an element, so it\nprobably does not match the specification.\n\n## Dynamic syntax\n\nThe dynamic syntax also uses SVG paths for its definition, and makes\nthe defined shape follow the given path during the duration of the\nfragment. The current implementation uses the browser internal SVG\nsupport for parsing and interpolating paths.\n\nFor example, the URL ```video.mp4#t=10,50\u0026shape=M0 0l0 50 50 50 50 0z\u0026trajectory=M0 0L200 200```\nwill move the aforementioned square in a diagonal line.\n\nIn order to facilitate debugging, you can add the ``debug=1``\ndevelopement parameter, which will then display the trajectory in\naddition to the moving shape.\n\n## Possible extensions\n\n### Common shapes\n\nIf it is deemed desirable to have more common shapes available in SVG\nsuch as circles and rectangles, the following extension is proposed\n(and implemented): the shape element can optionnaly start with a shape\ntype specifier (one of `path`, `rect`, `circle`, `ellipse`) followed\nby a colon and by a type-specific list of data points.\n\nFor path (which is the default if no shape type is specified), a [SVG\npath](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths)\ndefinition.\n\nFor rect, 4 numbers `x`, `y`, `width`, `height`\n\nFor circle, 3 numbers `cx` (center y), `cy` (center y) and `r` (radius)\n\nFor ellipse, 4 numbers `cx` (center y), `cy` (center y), `rx`\n(horizontal radius) and `ry` (vertical radius)\n\n## Shortcomings/evolutions\n\nThis proposal makes some simplifying assumptions, for which an\nappropriate trade-off has to be found.\n\n1. A mediafragment does not define more than one shape. If multiple\nshapes must be defined, then multiple fragments should be used. Or you\ncan define multiple shapes in a single SVG path (see [demo page](http://olivieraubert.net/dynamic-media-fragments/#v7)).\nAnother idea could be to define the notion of MediaFragmentGroup, which\nwould group multiplefragments, so that we can address discontinuous\ntemporal or spatial ranges.\n\n2. The shape itself does not change over its trajectory. This will not\nwork for instance in case of a face tracking algorithm that follows a\nface in a shot, which could vary in scale and orientation. A possible\nway to solve this would be to extend the trajectory path syntax by\nadding a new \"Scale\" command, which would specify an object scale\nchange, e.g. `trajectory=M9,323C147,8 E0.5 C270,78 E1.0 C270,78`\n(using `E` as command abbreviation, since `S` is already used). The\nSVG standard defines an additional \u0026lt;animateTransform\u003e element for\nthis, but in XML syntax.\n\n3. Similarly, the temporal specification is not flexible (it is a\nlinear interpolation of the trajectory of the fragment duration in the\ncurrent implementation). Temporal keypoints could be specified again\nby extending the path syntax, e.g.  `t=0-10\u0026trajectory=M9,323C147,8 K4\nC270,78 K8 C270,78` (using `K` as `Keypoint` abbreviation, since SVG\ndefines the notion of Keypoints but with no associated compact syntax)\nwould specify durations associated to the current trajectory command.\n\n4. Due to the flexibility of the SVG path definition, the\nmediafragments will be more complex to interpret and query in\napplications such as the [SPARQL-MM proposal](http://2014.eswc-conferences.org/sites/default/files/eswc2014pd_submission_65.pdf).\n\n5. For complex SVG paths, that are too cumbersome to be put in a\nsingle URL, a simple syntax extension could allow to reference\nexternal SVG files, e.g. `shape=src:filename.svg` or\n`shapesrc=filename.svg`. Note that this would make the dereferencing\nmore complex and less robust (since the referenced file could be\nunavailable).\n\n## Implementation details\n\nThe dynamic path support has been implemented using the SVG API method\n`getPointAtLength`, using a linear interpolation between the temporal\nposition and the position on the trajectory path. This provides a\ncompact representation, but does not allow to specify specific keypoints.\n\nI have made some tests with standard SVG animation features\n(\u0026lt;animate\u003e and \u0026lt;animateMotion\u003e) but it proved impractical for 2 reasons :\n\n- browser support is variable, even though\n  [CanIUse](http://caniuse.com/#feat=svg-smil) states that there is\n  good support\n\n- video synchronisation should be achieved by SMIL timing events\n  (begin/end), but I did not find any way to make it handle seeks\n  inside the fragment (i.e. setting the position to something other\n  than the begin).\n\nIf anyone has experience with combining SVG/SMIL animation events with\nHTML5 video, feel free to comment.\n\n## TODO\n\nSome more ideas that could be ironed out, time permitting:\n\n- provide an editor for both shapes and trajectories, using one of the\n  many existing javascript SVG editors.\n\n- using the shape as a clipping mask. It would involve defining it as\n  a \u0026lt;clipPath\u003e or \u0026lt;mask\u003e element, and specifying the\n  clip-path/mask CSS property for the video.  Silvia Pfeiffer's [HTML5\n  Video\n  guide](http://html5videoguide.net/DefinitiveGuide/chapter5.html) is\n  a good starting point for this.\n\n- find a way to specify timing information. SVG animation uses\n  keypoints, but in XML syntax. We could extend the trajectory path\n  definition with timing information, but we would then to split the\n  path into multiple segments before interpolating them.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foaubert%2Fmediafragment-prototype","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foaubert%2Fmediafragment-prototype","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foaubert%2Fmediafragment-prototype/lists"}